Those years were transition years of a sort for just about all, but Federer. I'd blame it on technology (Poly strings) making baseline tennis the new game while players (even Baghdatas) who developed games more with gut strings and touch and feel in mind were up against a new coming reality. For clay courters Poly came much earlier, but Fed in 2002 really changed the rest of the tour with his success and adaption to Poly becoming apparent by 2003. Robredo was still also a bit of a throwback like Baghdatis. Nadal was still relatively new in 2006, but already dominant on clay. Better examples would be Berdych in Paris (late 2005), Davydenko 2006 Paris, and Djokovic early 2007 (Miami). All of these featured an overall weak pathway to the title. For 2006 Davydenko was 4th seed and the top 3 seeds withdrew for example. One could say that for Nadal on hard courts he was more of a new player from 2005-2007 breaking through with mainly the slam focused Federer as the big obstacle on hard courts.
Zverev in Rome had a perfect pathway to the final and really only had to beat one strong clay player in the final (about like Nole getting by Nadal in 2007 Miami). Thiem had an opportunity in Madrid, but had Nadal on clay. With Federer out on clay and Murray off his game this has really opened things up as lesser players don't have to play three of the big 4 to win an event.
The door may shut on hard courts again later this year as Federer and the rest of the Poly veterans still have excellent games on hard courts versus the younger players who lack the upper body strength to handle the heavy topspin, hard, high bouncing poly game. Towers like Kyrgios and Zverev still will have a chance as bounce height and their efficient serve games (short matches) will give them more apparent stamina for deep runs on hard courts. Dimitrov is another talent entering his best years at age 25 and he may get some success as he is fully developed and has proven stamina at Auz Open. Those hard court events are just not easy these days with most of the draw playing a heavy baseline game. There is no technological change affecting the game now just a slow evolution.
At this moment with Nadal and Djokovic's loss to NextGen in Rome, Murray's poor play, and Federer's convalescing off clay, we may continue to have unexpected openings in the big events sooner rather than later. On clay and grass it will be more likely this year than on hard as age is no advantage on clay and grass while the younger players are less impacted by the Poly advantages for the older players on hard courts.
Its very different from 2006 in that technology is no aid to the young. The veterans have the better coaches, teams, and everything else. We really got to see Thiem step up versus Nadal in the last few weeks for primo generational matches, but Zverev versus Djokovic was much more about Djokovic's level being low this year. Hopefully we have more great matches and some break through wins by players and perhaps another title or two, but it won't be as easy after Wimbledon.
Amazingly through it all stands Federer blocking the way in both years, adapting his game for later success against an increasingly more and more powerful baseline tennis. Simply a wonderful player.