Let's try to break this down. What makes an elite athlete? Some examples:
Endurance: Lance Armstrong types, soccer players, triathletes, decathletes, marathon runners, etc. Many I could consider with better endurance that tennis players.
Variety surfaces or conditions: Many athletes compete in a variety of conditions and multiple surfaces. NFL players can play in 80 degree temps, in\outdoors, snow. Soccer players, Tour de France cyclist, etc. Again tennis players don't seem to be at the top of this list either.
Speed: Usain Bolt, Wide receivers, etc. Not too many tennis players running a sub 4.4 40.
Quickness:Boxing, Race car drivers(hand speed is incredible. They can go opposite lock and catch a car at triple digit speeds insanely fast). MLB hitters, golfers, tennis. All have to accelerate something super quickly be it bat or body parts.
Hand\eye coordination:MLB hitters, NHL guys, tennis players, goalies.
Anticipation\reaction times
rag racers, mlb hitters, goalies, tennis players.
Mental toughness\concentration:Race car drivers hands down. Try having to concentrate at 100% with no breaks(F1 drivers can't relax even during a pit stop) for that amount of time. How many nutcases have been ranked top10 in tennis?
Violence(I mean being physical with another person):NFL, NHL, hockey, boxing, mma, etc.
Killer instinct: Jordan
I just threw at some quick examples. Not the difinitive list. Please add examples or categories. Let's see what it takes to be an elite athlete and then where tennis players fit in each category. They don't seem to be the best in any one category, but are towards the top in a few.