It isn't dumb to despise these kinds of players. They are frustrating and most of us PAY MONEY and end up having to play these people. Some months, I have to play four pushers to get to a normal player. So frustrating.
Yes, it is pretty dumb. You're a 3.5, they're 3.5s. You all play at roughly the same level. They are playing by the same rules as you. So what is there to despise?
As far as paying money goes-- if you were paying that money directly to your opponents so they would show you a good time on the courts, you'd have all the right in the world to complain. I presume you're not doing that. Instead you're paying for the opportunity to compete with players at your level-- and so are they.
If you want to play better players, improve your level of play. If you don't want to spend money to play "pushers," play social tennis.
Or you could just try to have fun no matter what kind of player you're against. It might sound corny, but I honestly can't remember ever playing tennis and not having fun.