Yeah if there are people actually willing to watch a recreational match involving me, that is cool by me, they can pretty much say or do whatever they want, within certain limits of course.
Once in a mixed doubles match my partner's guest cheered for her loudly after a 1st serve fault, on break point at 3-4 in the 3rd set. That distracted her and she double-faulted, and we lost the next game to lose the match. We all had a laugh afterward about how this person cost us the match.
But if you let the crowd or outside events distract you then it is your own fault. Stuff happens in a match, you have to keep your focus.