There are 3 things Djokovic excels at that, in combination, give Federer no end of trouble:
1. Return of serve. He's either 1st or 2nd all time in this category (most tennis pros would give him the nod). And Djokovic is by far the best returner Federer has ever faced in his career.
2. Depth of groundstrokes. Djokovic consistently puts in more depth close to BL than anyone, this makes it harder for his opponents on their own groundstrokes, and to attack the net.
3. Change in ball direction. Nobody does it as often or successfully, especially the BHDL that he puts right in the corner.
Federer has never had a successful response to this 1-2-3 punch of Djokovic.