Federer was several things back in 2004 that he isn't today. Back then, he was peak-erer, speed-erer, crush-erer, bomb-serve-erer, consistent-erer, and forehand-erer. Fed occasionally got his serves into the 130's back then. His forehand had much more pace on it. One thing about Fed's forehand is that it had the ability to blast peak Hewitt off of the court. Nobody displayed the power to do that; not even Sampras. This is something that people don't realize. Hewitt was blazing-fast back then.
Contrary to popular belief, I think that Federer was at his very best in 2004, not 2006. Now granted, Fed was better on clay in 2006 and he was more consistent against the scrub players. But in 2004, I think that Fed peaked higher, especially against the top players. I think that Federer also relaxed too much against scrubs that year too. So although Fed had a better year in 2006, I think that he displayed his very best form in 2004.
Picking Fed's best year is like picking Shakespeare's best work. There's a lot to pick from. But here's the case for 2004:
Federer was 18-0 vs the top 10. Nobody in the Open Era has gone undefeated against the top 10. Of Fed's 18 top-10 wins, 11 were in straight sets. That is remarkable. Also, Fed didn't drop a set to Safin. Fed went 4-0 vs Hewitt. Hewitt did manage to win 1 set during those 4 matches. However, Fed dished out 3 bagels and a bread stick to Lleyton. Fed also went 3-0 vs Roddick. And he also dished out a bagel to Roddick. That is insane in itself. Roddick was hitting serves up to 150, yet Fed bageled him. Unreal.
4 of Fed's 6 losses were to players ranked 30th or worse. Fed didn't get up for these matches like he did in 2006. Otherwise, Fed against a top player was going to end up in a Fed blowout for the most part, except for when he played Agassi in a hurricane at the USO.