I think strength, specifically upper body strength, is a very important factor between the men and the women. All things being equal, good form and all, a guy can swing a tennis racquet faster than a woman. For that matter a big guy and swing faster than a smaller guy.
If you're just hitting through the ball then there's an upper limit on rhs that going to make a difference. After a certain point you're just going to hit the ball long. For most tennis shots anyone, man or woman, can get to that point. After that if you want more pace, or more margin, you have to have more rhs that will go into generating topspin.
The men have that extra strength to maintain, even increase the pace and still add loads of spin. Watch these guys hitting their shots. Does it look like they're holding back much? The women are going to have a harder time doing this. It seems like the woman generally are making the trade-off to go with more pace and less spin because they have to have the pace to hang in the rallies with the top players.
Note that these are all generalizations. I do think there are technique differences that make a difference, even among top pros. I think a guy like Federer is probably getting more pace and spin that someone else his size could because his form more efficiently focuses the energy his body creates right on to the contact point with the ball.
I wonder if anyone's made measurements of racquet head velocity (magnitude and direction) throughout the swing. I would be interesting to compare a profile of racquet head velocity between different players.