Do college rankings matter?
They matter to the extent that other people believe them, which is not nothing. But rankings measure institutional characteristics and reputation surveys, not what a specific student will experience or learn. They are a starting point for discovering schools and a poor basis for ordering them.
What rankings actually measure
Depending on the publisher: graduation and retention rates, faculty resources, spending per student, reputation surveys of administrators, and in some cases outcomes data. Notice what is missing. Teaching quality, whether students are happy, and what happens to a particular kind of student are all absent, because they are hard to measure and harder to compare.
The methodology moves
Rankings change their formulas, and schools move sharply as a result without anything changing on campus. Several institutions have declined to participate in various rankings in recent years, and a number have been caught submitting inaccurate data. Treat a rank as one publisher's opinion in one year, because that is what it is.
How to use them well
As a discovery tool. Ranked lists surface schools you had not heard of, which is genuinely useful for a family whose knowledge stops at twenty famous names. Use them to find candidates, then evaluate the candidates on the four parts of fit.
The honest counterpoint
Reputation is real in some fields. Certain industries and graduate programs recruit narrowly, and pretending otherwise does not help a student. The right response is to ask where graduates of a specific program actually go, which is a more precise question than where the university sits on a list.