It can be tricky to strike the right balance if your tennis schedule includes both playing and practice sessions. That feeling of being recharged after a day or two off is rather telling I think. Give your serving muscles at least that day or two for recovery between practice sessions and you probably won't be overdoing it.
Generally I recommend serving one bucket of balls a week (maybe 65-70) to keep the serve well tuned to some degree. Less than that and I think that it can get rusty, even if we're getting a couple of matches a week under our belts. For getting more dialed in, I think that two or three of those buckets a week is okay, but that practice needs to be good and deliberate.
Remember to also practice pre-serve rituals that you use on match day, along with planning the placement of a first or second serve. Hit a bucket or two day after day and it's easy to switch into auto-pilot where we're just unconsciously going through the motions. That's not productive.
If a little rest keeps that shoulder more fresh and ready for prime time, I say (cue caveman speak)... rest good!