You don't have to. There are many ways to serve, but when you do the real abbreviated serve like you are doing, the racquet stops moving. I like to have the racquet in motion at all times, as I find that less goes wrong.
I use an abbreviated motion, but I really believe in the rhythm of the serve and find that a unified motion of both hands can really help with that.
Please help TT Member BruceD get a wheelchair so he can play tennis again - http://tt.tennis-warehouse.com/showthread.php?t=496820