Because he made so many semis and finals that he lost to the greatest clay courter ever? It's a simple prospect, really.
Since Nadal is the best clay courter to ever grace the court, he's a very tough measuring stick to use. Put Kuerten up against Nadal in his finals, never wins a single one. Same thing with Wilander and Lendl.
What we can measure is what they could have won in absence of Nadal. If Djokovic wins another RG, in absence of Nadal he wins at least 3 more, but up to 5 more, with several extra finals to his name. That's better than Kuerten, Lendl, and Wilander could say. Now, obviously we can't just gift him those victories because they didn't happen, but Djokovic has an 82.9% win rate at RG, even losing 6 times to Nadal (he's only lost 8 times to people not named Nadal). Take away the Nadal matches and he has an 89% win rate. Even if another clay courter just as good as Djokovic showed up in Nadal's place and won 50% of the time against him (a Nadal-lite, if you will) and played him 10 times, he'd still have 3+ RGs and an 85% win rate.
The only thing that stopped Djokovic from being a clay ATG was Nadal - same with Federer. And if Djokovic wins 2 RG, he's definitely top 5.