Upon further review, and you might already know this.
Your body follows the serve into the court just fine.
But your prep to hit, your body is moving left to right, then changes direction as you hit to follow thru correctly. That change of direction will cause problems later, when you're a 5.5+. The problem is added complication and slight lost of ball speed.