I think Andre Agassi put it best during his 2007 US Open commentary (Federer vs Roddick). Agassi said that Federer was one of the best returners the game has ever seen.
"I'd be surprised at how often in a match Federer does not out-ace his opponents. And that's not because he's throwing down 140 mph serves" [but because of his ability to get his racket on everything, and neutralize the serves of most of his opponents]. When Andre Agassi is calling someone else one of the best returners, I'd put stock into that.
Like I said, it's because he doesn't have a "flashy" return that hits a lot of flat out winners that people don't realize how good it is. But the ability to put difficult serves back and play, and reset the point to zero is a skill that most tennis players would love to have. Federer's strategy and instinct is to get balls back in play, not to hit winners off the return, because he's going to win better than 50% of points on his opponent's serve if he can get into a rally.