First, if we pretend that Djokovic's two older ATG opponents are as typical as all before them, and that they haven't been more competitive and played better in the latter part of their career (compared to previous ATGs),
Then, if we also pretend that in 2010s those two wouldn't have won several extra Slams easily if it wasn't for Djokovic, basically if we pretend that similar outcomes to the 2017 season wouldn't have happened two or three more times,
After that, if we pretend that the age difference between Nadal and Djokovic is humongous, and more importantly that they haven't met at Roland Garros far more than at any other Major,
If we pretend that Federer is an average older ATG when he loses to Djokovic (for the sake of devaluing Djokovic's achievement) while he isn't average when he does something extraordinary,
If we pretend that these two haven't had a head start against someone who was a late bloomer (but still went deep in Majors quite a few times) just like he used the occasional slump they had later,
Also, if we pretend that Djokovic peaking later is irrelevant to the wins Fed and Nadal scored in the beginning, while impressive longevity of his rivals still makes his wins against them unimpressive or purely vultured,
Or to sum it up easily, if we pretend that the situation in the moment suited Djokovic in all of those NINETY-NINE meetings he had against them just because he is younger while they declined centuries ago,
Then and only then could we be so bold to hold Djokovic's narrow head to head lead against Federer and Nadal against him.
But none of those are the case, so it's not an issue.
Not that I think Djokovic has a claim to be the GOAT in the first place. Hardly anyone thinks that anyway, and no amount of complaining about the hype around him from a rare poster here and there when he does well (sinful behavior LOL) is going to change that fact. What holds him back is losing a couple of big matches to Murray and Wawrinka, especially being a combination of chicken and unlucky against the latter player. He's done a tremendous job against Federer and Nadal since he became a dominant force, which resulted in staggering 13 Majors among other achievements in this decade, and I don't even want to imagine the rioting here if he beat them even more often.