I haven't seen your serve, but...
If it's only your wife who thinks your serve is fast, I'd not place too much importance on it. If your tennis BUDS start to complain your serve is too fast, then I'd think your serve is doing well.
Almost EVERY female, regardless of level, complains I hit with too much spin on spin serves, and mix in too much ball speed when I hit the corners, or into their bodies.
Very few 4.0 GUYS complain, they just hunker down, concentrate harder, step back 2 steps, and return my serves.
Then again, on another note.... You can rotate to get your service power, or you can use a stomach crunch, like Roddick does, to get your shoulder's moving forewards. BOTH work, so it's not confined to "your way, or the highway".
"Easy Power" is great to have, but more important, does it bother your peers when they are trying to return your serves?