1. I agree that you (the OP) didn't do anything wrong. You weren't sure enough to make a prompt out call so you gave your opponents the benefit of the doubt. It is then their responsibility to play the point.
2. At the same time, if I think a serve is out but don't make the call and my opponent stops and says "that was out wasn't it?" I will usually agree and tell them to take a 2nd serve.