I said I removed two of the top 5 servers of all time, Roddick and Karlovic, to give a clearer picture and to make it more fair which still favored 2012 overall, but I'm not about to pump up Soderling who had a 17-14 record on grass before Wimbledon that year and an opponent in 2008 that Federer won 39% of return points against compared to 22% in 2009. It's not like Soderling just decided in 2009 to serve big since in 2008 he averaged 126 mph on the 1st and 107 on the 2nd. In 2009, he averaged 124 mph on the 1st and 107 on the second, and in both years served about 55% 1st serves. He served more aces in 2009 so let's give him the benefit of the doubt and say he served better in 2009. That still doesn't explain why Federer had a -17% differential from one year to the next, even though he won both matches in straight sets, on the same opponent's serve without conceding that he returned worse in 2009.
Federer had two great returning performances, against Djokovic and Murray, so not sure how you think he didn't return well against Djokovic. Also, Haas didn't serve better than Djokovic did in 2012. Djokovic averaged 120 mph on the 1st and 94 mph on the 2nd and Haas averaged 119 mph on the 1st and 102 on the 2nd. Djokovic had 9 aces, 2 double faults and Haas had 11 aces, 5 double faults and they both served in the mid 60% range. Federer actually won more return points against the Djokovic 1st serve which was slightly bigger than Haas', 29% on Djokovic's compared to 26% for Haas, but Haas got his 2nd serve hammered by Federer even though his 2nd serve was bigger. Haas only won 45% of those points where Djokovic won 57% of his meaning Djokovic protected his 2nd serve a lot better.