I had this same situation come up in a practice match earlier this week.
My partner was returning serve and I was up at the service box watching the incoming serve and saw that the ball was out about 2 inches. My partner continued to play the ball since it was coming at a decent speed.
If you're the returner, it is very difficult to see balls that land OUT within 2-4 inches if the ball is coming at a fast speed and at a flat trajectory.
When I returned serve there were a couple times when my partner called the ball out even though I thought the ball looked in on the line. These were all line calls.
When you're focusing on returning serve, it is difficult to see line calls that are within inches of being in or out. Usually as a returner I give the benefit of the doubt to the server. When I'm at the service box looking at the line, I'll call it out even when it's 1 inch out because it is what it is. I have the best view of the ball and I'm not moving around like the others.