Yeesh...Rafter? I'm fairly confident in saying Rafter's serve was not in the Top 50 all-time. He might be Top 10 all-time at the net, though.
From this poll, I say Karlovic. Without his serve, he's a club player.
I think Wayne Arthurs is in the discussion as well.
As much as I'd love to say Ivanisevic had the greatest serve in history, the reality is, those two hit more aces per match and had far less all-around game.