IIRC 2005 Agassi had played about as many matches as 2012 Federer up to that point. Not saying that completely balances out Agassi being 4 years older in 2005 vs 2012 but it's not insignificant that Agassi had several off years in his 20's and his most consistent results came in 1999-2003 and he'd been playing a reduced schedule for a couple of years at least. Not quite as simple as 6 years versus 11, of course Agassi also had back troubles which you didn't mention either.
I would also point out that Agassi had an extremely tough QF opponent in Blake who played at an incredibly high level - instant classic. I'm not sure when Federer last beat an opponent playing at that level, he certainly didn't do it in 2015.
I'm mostly concerned with how people play on the day, the reasons for why they're good or bad come later. IMO Agassi played a better match than Federer despite the bigger age gap. Does that mean younger Agassi would have a field day with Federer, no - because older Agassi in that match played at a level closer to his best than Federer did in his.
If you really are an objective observer let's not generalise and instead look at each match on it's own merits without assumptions