Remember that match-up is not about the absolute level but about how a player Y strength and weakness interact with player X strength and weakness.
Djokovic is the best player right now, so, even in the case of a bad match-up he can still win. I think Djokovic is more vulnerable to Fed than Nadal or Murray on HC, even if Federer might be a lesser player than them now.
Murray gave Fed a lot of trouble since day 2007, which prove the match-up issue for Fed. The slams finals are specials. It's the match-up in murray's head who made them so one sided.
Simon and Benneteau are by far the less talented player on this list, but somehow they give Fed a lot of trouble. I think they should be ranked higher than the big hitters, except Berdych.