I think the field was fine tbh (what a surprise I think that lol). The likes of Federer, Roddick, Hewitt, Grosjean, Henman and Ancic were all good in at least a few of those 03-06 years.The sad thing about Roddick is he actually had the weapons to achieve more but not the belief or mindset. His level in the 2004 Wimbledon final was mindblowing and he was just ripping the match from Federer's hands and then the rain delay happened, and he came back out without the intensity he had. That was the story of Roddick's career. I do agree though that the grass field in particular did not have many challengers early on and there was a void there until Djokovic and Murray matured.
I think 2004/early 2005 broke Roddick's spirit, never had the same belief and his game went in the wrong direction as a result. I don't think he got the most out of his game.