Personally, I don't hold back on serve unless I'm winning quite comfortably. As far as I've seen, the best players in this county are pretty much the same.
Still, it seems like common sense that, if you're facing a team that is good enough to beat you unless you hit your biggest serves, then they're good enough to face your biggest serves without getting hurt. That said, I go for angles rather than trying to jam people, so I'm not likely to hit too many.
(On a side note, I did get hit by a guy's biggest serve one day. I was playing on some courts foolishly built facing east and west, and I was blinded by the setting sun. I completely missed the ball and got hit on the stomach, but all I got was a bruise.)
You only have yourself to blame.