He's not even close to being the fittest athlete in his era (though of course he is very fit), you're confusing longevity with fitness/stamina when reality is that many of the fittest players in the game throughout history burned out quickly due to injuries and relying on outlasting opponents. The only time Fed was at or close to the top of the game fitness wise was 2006, that's about it. Fed's longevity is due to his smooth Edberg-like movement on court and that terrific serve, not some legendary fitness.
He doesn't sweat much, so? Nadal makes a puddle of sweet 5 minutes into a match, the guy who utterly dominated the surface where fitness rules supreme (sorry it's "point construction" LOL) for nearly a decade. He doesn't grab his thigh/back/ass, make pained facial expressions, breathe heavily etc. those are largely meaningless theatrics, if you allow those to affect your opinion to such a degree then maybe it's time to switch to WWE or atleast reduce the time spent watching soap operas and cheesy Holywood sport movies.
You want to see if a player's tired? Watch how he moves his feet, that's all there is to it. Ever since 2009 Fed's footwork has had a tendency to become sloppy and labored nearly every time he got pushed the distance against opponents who make him work for it physically (Djokodal and big hitters, not Murray), it's been his Achilles heel against younger fitter opponents on today's slower surfaces much more so than his supposed mental weakness. His current success is largely built on ultra efficient playing style that won't work on clay where fitness and defense/movement get exposed which is why he's skipping an entire clay season bar FO (and even there I'm doubtful if he'll play).