It matters. But so does your gpa.
If you have an opportunity to get into a better school, I say take it. But once you get there, you'll have to work even harder to make it pay off for your future.
I attended a top-tier school. I don't think it helped me get into grad schools because my undergrad gpa was crummy. But once I had my PhD (from a large research school), having a Stanford undergraduate degree on my resume has definitely given me more credibility. People seem to trust your competence more when you've gone through the filtering process of being admitted to an elite school.