Why don’t we look at the H2H stats themselves and how they fit in to each players' seasons? Skip to the bottom if you're short on time.
Here's how I rank seasons in order of worst to best in case some find my metric unclear:
Poor (It should be obvious)
OK (Good slam showing, not many big titles; usually slamless unless the player wins one slam but no other big titles)
Good (One slam; a few masters or the WTF)
Great (Two slams, a few masters and the WTF)
Very Good (Again, it should be obvious)
Matches in 2006:
1 (H2H 1-0) -- Federer season: Very good -- Djokovic season: Poor
Matches in 2007:
4 (H2H 3-1) -- Federer season: Very good -- Djokovic season: Good
Matches in 2008:
3 (H2H 2-1) -- Federer season: Good -- Djokovic season: Good (A rare tie in level of play!!!

)
Matches in 2009:
5 (H2H 2-3) -- Federer season: Great -- Djokovic season: OK
Matches in 2010:
5 (H2H 4-1) -- Federer season: Good -- Djokovic season: Poor
Matches in 2011:
5 (H2H 1-4) -- Federer season: OK -- Djokovic season: Very good
Matches in 2012:
5 (H2H 2-3) -- Federer season: Good -- Djokovic season Great
Matches in 2013:
2 (H2H 0-2) -- Federer season: Poor -- Djokovic season: Great
Matches in 2014:
5 (H2H 3-2) -- Federer season: OK -- Djokovic season: Good (H2H does not include walkover in the WTF)
Matches in 2015:
8
(H2H 3-5) -- Federer season: Good -- Djokovic season: Very good
Matches in 2016:
1 (H2H 0-1) -- Federer season: Poor -- Djokovic season: Very good, then Poor after RG; Overall Great
Matches in 2017:
0 (H2H 0-0) -- Federer season: Great -- Djokovic season: Poor
Matches in 2018:
2 (H2H 0-2) -- Federer season: OK -- Djokovic season: Poor, then Great; Overall Good
If we count 2010 as the final year for Djokovic playing "poorly" and 2011 as the year when Federer began to "age", that leaves us with:
Pre-2011:
18 matches played
2011-present:
28 matches played
I'm not a huge fan of the age argument as it doesn't factor in level of play (Ex. Fed had a great 2017 season despite being 35 yrs old). Let's use level of play itself:
Matches played in seasons when Federer was clearly the better player:
15
Matches played in seasons when Djokovic was clearly the better player:
23
Matches played in seasons when Federer and Djokovic were playing at roughly a similar level:
8 (funnily enough, these matches are equally split between the players)
For the third statistic, I picked the 2008 and 2012 seasons; you're free to add or remove seasons.
I realize that it's impossible to truly compare seasons with H2H as sometimes, a player can have an excellent season and lose a lot to one player because of matchups and have a weaker season but totally own that player due to reversing the matchup.
What would end the argument IMO is if three matches were played in 2017 to make up for 2016 and 2018; this would tie the H2H and leave it out of the debate when regarding actual achievements.
My answer to the OP: Fed started to really become old in the year he turned 32: 2013. After this he began to make many changes in his game to accommodate for his declining power and movement (The bigger racket, the SABR, and an increased reliance on volleys to end points quicker). This does not mean that Djokovic's wins against Fed post-2014 are useless. I was just answering the question. Feel free to make up your own mind.