I think it is also important to point out, since OP specifically asked about kick serve, that the type of serve is not relevant to the timing of the tossing arm coming down.
Many rec players do not get tossing arm close to vertical in the first place (do not get into proper trophy pose and are at 45 degrees instead of ~90 degrees). Even if they do that, they still have the problem of dropping the left arm way too soon.
I think TTPS had some threads on this issue that has since been banned/deleted. Not easy to fix this problem.
One tip I've tried to follow is not to let the tossing arm drop until I see the ball drop at least a few inches (I make contact 1.5 feet from toss apex). I think that helps.
Keep Your Tossing Arm Up as Long as Possible
The tip is this. When tossing the ball after you have released the ball, try to keep your tossing arm up as long as possible. If you get out and practice some serves and really think about this, I think you will find that most of us drop our tossing arm a little too quickly. When this happens, dropping our arm means that we are also dropping our shoulder and we are pulling off to our left side if we are a rightie. If you are a leftie you are going to be pulling towards your right. I am a rightie so I am going to talk in those terms.
Dropping Your Arm Too Quickly Shifts Your Body
If you drop your left arm too quickly after tossing, your whole body is pulled off to your left side. The result of this can be two things, two negative things. First of all it shifts your body towards your left. So you are not in your best possible body position to hit your serve. You are more off to your left. That is going to move everything including your right arm and your racket.
Dropping Your Arm Too Quickly Reduced Your Momentum
The second negative thing that can happen is that it reduces your momentum. This is because now your weight has shifted over and instead of your body going up and into the ball and hopefully forward you are sort of going off to your left and you are pulling that weight, that direction away from the serve.
Using the Trophy Position
So what you want to do is try to keep that arm up there as long as possible. I am not talking about an awkward long pause because obviously you need to hit your serve but I am talking about trying to work in a little extra hesitation up there. The best way to adopt this is to practice it, to really force yourself to hold that so-called trophy position, the classic serve position where you have your left arm up, the tossing arm, and your right arm back with your racket, almost ready to hit that serve. Try holding that position for just a split second extra, see how that feels and then try to work that into your service motion. If you can keep that arm up a little bit longer you will have a much better body position, a more forward position when you are hitting your serve and you will get that extra boost in momentum going up into serve as a opposed to being thrown off to the side.