It's been exaggerated because people can't get off the earlier Federer struggles with Nadal in this part of the game.
But in recent years, Federer's losing to Nadal has had much less to do with the forehand-to-backhand matchup than other parts of the matchup. Nadal's defense bothers him. His passing shots bother him. And Federer struggles on the backhand return of serve against Nadal's lefty slice.
Plus, Federer's inside-out forehand plays right into Nadal's down the line forehand, and the same for his cross court backhand. He has more trouble hitting down the line backhand winners against Nadal because Nadal camps to his backhand corner, like most players.
In essence, Nadal just moves too well for Federer and can do too much off his forehand side in general, plus Federer doesn't like returning Nadal's serve.