Given the cards as dealt, the issue lies squarely with your partner, not you. You correctly didn't make the call, as you did not feel comfortable doing so, given your poor look at the ball. Your partner either felt the ball was in or also didn't get a good look at it (perhaps because it is usully your call and this time you were asleep at the wheel). Therefore the ball is in because noone on your team accurately saw it out. The fact that the return was a sitter and that the netman missed it is just confusion. The call stands.
Now your guilty conscience is not that you made the incorrect call, but that you didn't do your job (calling the line) accurately.