It is ok to notice the toss and its location but you should probably not dwell on it too much. If I recall correctly, gaze control studies show that elite players focus primarily on the upward swing (path) of the server's racquet. It was found that eyes of lesser skilled players tended to dart around much more than elite players. Elite players kept their eyes quieter -- they did not necessarily follow the toss up and own as one might expect. Less is more. Try focusing on the upward swing more and see if that doesn't help.
If u have one foot stuck in yesterday & one foot stuck in tomorrow, u are peeing on today.