Without question, move up to the next level. I've played a 4.5 over the last year who was kind enough to play me when I could barely hit the ball. We're talking 1-6, 2-6, 1-6 at best. (Came back to tennis 18 months ago after 24 year layoff).
Got my butt kicked plenty, but now play at the same level with him, and have even managed to edge him the last few times. It sounds like you have a lot of natural talent, so you'll probably catch the tougher players fairly quick. It's hard on the ego at first, but it will pay off big time down the road...
(3) Dunlop Aerogel 100, (3) AG100 4D. Crossfire 18 @ 40/50. All-court magic wands.