My first serve was a fault. As I prepared to start my second, a ball from the next court stopped behind me about 4 feet back and one yard to my left. I walked over to it (about 3-4 steps) and scooted it to the back of the next court. I viewed this as an interruption between my first and second serve and so asked for a first serve. It was declined. Could/should I have called a let?