I use to have the same problem as the OP, no rhythm, game fell apart but lately I am finding there is a rhythm in playing such players. It is just a different way of playing and constructing the point. I mean I have beaten 3 pushers/junkballers in a row in league matches and I didn't do it by pushing back.
The first important thing is to not expect to win every point, you need to respect the pusher/junker. If you try to play aggressive against them, you're going to miss and mistake. That is OK, shake it off and reset. Do not fall into the trap of trying push back. That doesn't mean trying absurd shots when you're off balance but only go defensive when you have to.
The junker pusher balls tend to move in the vertical plane, their moonballs stop on the court and so do allot of their slice shots. This takes a bit of getting use to, if you have learnt against more conventional players. I have found that the best plan is to keep as many shots as possible on my forehand and aim for spots, a bit like serving. So if I get one of these balls I will lock my eyes on it, ignore what the pusher does and just aim my spots. Normally the backhand corner.
Of course it often comes back, at which point you have a choice. Take the net, the best option or if you suck at volleying like me, move to hit another forehand, once again aiming at a spot. The good news is, you can generally run down the soft stuff the pusher/junker throws at you fairly easily. So it is then a case of holding your nerve till their defence breaks or they leave a gap. It will happen eventually believe me.
The next problem is the short ball, pushers and especially junkers love to drop it just over the net, especially on the backhand side. You need a way of dealing with this. If you aren't confident enough to volley it, you have to develop a groundstroke to deal with it. Normally I try to take it on the forehand and hit it down the line. Do not blast, put some spin on it and place it. I understand the temptation to overhit because you are worried about being passed. You have to accept that, if they lob or pass you, too good. Move on.
You also need to have a reliable serve. It doesn't matter how you avoid double faulting, be it with a slice, topspin or even a dolly serve. You cannot give them free points on your serve. On their serve I recommend taking their second serve high up the court, come forward from the baseline. Generally a pushers second serve is the easiest ball of theirs to attack. Either aim for the short angle or down the line. It is possible to one shot them from that position.
My last bit of advice, is don't expect it to be easy. I have won 3 in a row this round but I also dropped two sets in those matches. Pushers and junkers are going to make you work for this win, that is what makes them both frustrating but also satisfying to play.