The players he's lost 4th rounders to are Djokovic (3 times), Federer (twice), Nadal, Berdych, Dimitrov, and Raonic. There's no total scrubs there, and every one of those guys has a bigger weapon or two with which to overwhelm him.
The guy is a super steady hand, good off the forehand side, quick, but he's so underwhelming when it comes to the business end of things. I have no idea why Novak (relatively) struggles with him the way that he does, because Federer, Nadal and Murray's major meetings with him have all been boringly one sided as his game is to watch
He's like if you were creating a tennis player on a video game, but didn't change the appearance, and left all the attributes on the default level. The fact that he is so solidly a top 15 guy is testament to the crappy state of tennis the last few years. Not quite slow and steady wins the race, but quick and steady wins the race. Not much better. It's like sitting and staring at a metronome.