Fortunately having speed is not "more" important that any other skill. The reason I love tennis is that there are so many ways to be good. It's also probably one of the reasons I gravitated towards tennis. Aside from the strategy and mental game, if I just looked at the "physical" aspects of tennis...
If I was the fastest person, I might have run track. Presuming I didn't have any speed, if I had the best arm speed, I might have tried to be a pitcher in baseball. Presuming I had no speed, if I had the best eye hand coordination I might have played baseball (eg. designated hitter). If I had great endurance, I might have run xcountry/cycling/etc... You get the picture...
Obviously getting to the ball is an important criteria, but there are so many other areas that you can excel (or combination of areas), that might help you dominate someone who excels in retreiving (eg. very fast person). While I'm not the fastest person on the court, nor the hardest hitting, nor the most accurate, etc..., I do have enough "+'s" in each category to be a decent player, and beat folks that might be faster, harder hitting or more accurate (or might be "-er" in any category)
my $0.02