Honestly, I am quite sure the answer is Borg. He was a more excellent physical specimen just generally, including attributes such as unreal stamina. Djokovic is more flexible than Borg was, though.
"If you're gonna be 5'8'', you better not be bald, fat and drunk whilst asking to be put in the 'all-time great sportsman' conversation." - Zoid