It wasn't the words he said, they were overall pretty innocuous. It was the excitement with which he said them.
I believe it was Musberger, announcing a Florida St. game, who seemed to be drooling over a fan in the stands years ago. Later, she turned out to be the one Brett Favre had his issues with. I think that association made it worse.
Still, being the non-PC person that I am, I don't think the network should have apologized. Musberger should have spoken to the family and offered his apology to them personally and in private if they were offended. It really isn't anyone else's issue.