First of all, he has beaten Nadal, so it's possible.
Second, despite how awesome Ferrer looks, he still has trouble against Federer or Djokovic. The very best players have something that bothers Ferrer which is typically speed and power.
Federer, for example, serves really well and earns a lot of free points. He also takes chances on his groundies and often makes it, but the free points is helpful.
Nadal is as quick as Ferrer, but hits the ball harder, and being a lefty, he nullifies Ferrer's usual patterns which is attacking a righty's backhand.
Djokovic is also similarly quick, and returns really well, so he can make it tough by being in many of Ferrer's serve. Ferrer's serve really helped in the Berdych match.
Ferrer looks good against Berdych and del Potro because he can make them run. As fast as they are for men of their size, they still aren't as fast as a guy like Ferrer (or Djokovic or Murray or Nadal).
Still, you have to say that Ferrer looked on fire against a tired Berdych.