I mean, obviously it takes hindsight to assess the end of one's prime. Who saw Federer lose in back to back slam QFs to Soderling and Berdych in early 2010?
It probably wasn't obvious even then. I think it can become obvious only some time afterwards when the loss of form turns out to be long term. I do think that some feared for Federer in the spring and summer of 2010, but by contrast I think that Djokovic's performance at US Open 2016 didn't seem that bad (obviously Wimbledon was a bad loss). The most obvious characteristic at the time was his luckiness in getting so many withdrawals. Also, Nadal actually seemed to be playing relatively well in the first week of Wimbledon 2014, and to be honest even in the defeat against Kyrgios - who hadn't been worked out yet and so looked very dangerous.
There can probably be a grey time in which a player might seem to have begun a permanent decline but might also just be a little off-form.