Your "hard work" seems NOT tennis related, but overall fitness related.
You now swim farther than ever before, and you jog quite well, compared to if you never did those two things.
Now play tennis more. Play doubles to start, maybe 3-5 sets a day.
Then add singles to the equation, subtracting 2 doubles for every ONE added singles.