...I... hit it solid in between net guy and the T... The server ran it down and hit a running forehand right down the middle of our court for an out-right winner...
How hard was the serve, and where exactly did your return bounce - around the service line, deep close to the baseline, or somewhere in between?
Let me explain. If the serve didn't have much pace on, and it was mainly placement close to the middle of the service line (basically, a jammer at the body), then you as returner have a lot of choices to play that put the opponent at the net in trouble - a whipped forehand topspin dipper to his trams, and hard flat drive straight at them, a topspin lob over them, or the forehand down the T you played. In that situation, the netman simply stood his ground and did nothing - so at least, he didn't sell himself and opened a gap for you - and simply bet on good reactions if the ball went close to him.
Now, your return. If it was deep on the baseline, then the server effectively played the equivalent of a singles' running pass down the line when pulled wide. One heck of a good shot. However, the shorter your ball was, the greater the chance you gave him to actually run into the court and take the ball earlier (and possibly, higher on the bounce) so he can both play a more forcing shot, and also take time away from you (ie, don't give you and your partner enough time to close the gap in the middle).
You mention you could have picked the ball on a half-volley. To me, that suggests he managed to put both quite a bit of pace and topspin on his ball, to get it to bounce in front of you - it's the equivalent of a return to the shoelaces to a S&V server, so if the serve is good, the return has to be even better (taken early, with power or using the server's power). My guess therefore is that your ball was a bit too short, so he was able to run forward to take it early and hit it down at you. He may well have intended to follow his serve into the net, and your return (which might have troubled him had you gone straight back to him) instead fell into his forehand wheelhouse once he adjusted his position to move right to cover his partner.
How do you defend against it? Well, don't go for that kind of shot again! If you can have a go at the netman because the serve falls nicely in your wheelhouse, then have a proper go - but make sure that once you get the ball past him it's a winner (so, in your case, if you go on his inside down the middle, then put both pace and depth on in). Or if you don't have a go at the netman, then play a return to the server that will neutralise them (bootlaces if he's a volleyer, deep to where he served from, or short acute angle, if he stays back). The point about playing a neutralising return is that it brings your partner into the game (ie, he can move into a net-attacking position, with you backing him up on a stagger, so the entire net is covered) for a possible put-away if the server's next ball is weak.