Djokovic, mainly. He's also expressed irritation at playing big servers, since the match is largely out of his control and he has to just hang in there.
Altho' he's not played Nadal since 2011, Murray had been playing fairly well against Rafa in their last 7 meetings (but has only won 2 of those). Outside of clay, Federer's h2h against Nadal is even. However, in their past 7 meetings on hard courts, Nadal has won 4.
Fed only bothered Nadal indoors and nowhere else. Across all surfaces? Only Djokovic seems to play him well. That said, outside 2011 his record vs. Nadal is similar to Fed's.
Djokovic, Federer when he's on form, Murray when he's on form.
Big hitters always seem to give him trouble, like Kohlschreiber or Soderling. But then I'd include Berdych in that list and he's beaten him like 14 times in a row or something ridiculous so you know.
He gave Nadal more trouble than pretty much everybody else (except the guys I already mentioned) in outdoor HC. I mean, AO 2009 wasn't exactly an easy match. If they had met more often before 2009, who knows.
Kohlschreiber is a big hitter? No more than Fed, certainly.