- College rankings don’t measure aspects of universities that are the most beneficial for students.
- They’re mainly based on factors that are determined by wealth, access, privilege, and selectivity.
- The pandemic showed how truly broken the system is, yielding worse experiences for students.
Early September is a blisteringly busy time for colleges and universities, with the start of the new semester and a whole new class of students kicking off the year of studies. But while everyone else is celebrating the start of something new, administrators are wracking their nerves over the results of something old, something they devote a considerable amount of prep to — their own big exam, the one that makes up an enormous amount of their grade: Will this be the year they take a big hit on the US News and World Report rankings?
Though circulation was down to 1.1 million in 2010 from 2 million in 2005, the once-powerful newsweekly commands an outsized presence in our popular understanding of universities.
“US News, of course, was the first one to really monetize rankings. They made huge big business out of rankings,” said Susan Paterno, author of the new book, “Game On: Why College Admission Is Rigged and How to Beat the System“.
When US News develops a rating, they select what is valuable in higher education. When they decide what’s valuable, colleges react, and try to get the numbers up. And when getting the numbers are this important, when they’re translated into a zero-sum ranking against their peers, colleges are incentivized to do things that make the college experience worse and more expensive for students, applicants, and faculty in order to appease that algorithm.
The birth of rankings
In the 1980s, two businesses emerged that would change higher education permanently: the test prep business, which was largely forged by Stanley Kaplan, who founded the company bearing his last name, and the beginning of the US News and World Report rankings.
As US News began to assign scores to universities using the test scores of admitted students as a proxy for quality, colleges began to desire more and more applicants who tested high.
The universities offered merit-based scholarships and financial incentives to those students, and as a result affluent parents of college applicants would pay for test prep for their kids to get those incentives.
“Even though that metric had nothing to do with the quality of the education that the particular college or university was providing — it was really just a number that describes kids in high school — that metric became the foundation that US News used to measure quality,” Paterno said. “But it was a bogus measure, really.”
Only 20% of the score on its Best Colleges rankings comes from academic instruction and faculty resources.
Optimizing this 20% may still lead to negative outcomes for students. Paying faculty more can raise the sticker price of colleges. Optimizing class size to appease US News can lead to things like capping classes and thus limiting the number of large, efficient lectures, or constraining the ability of students to get into entry-level courses.
It appears US News has more influence on class size than many professors. In 2014, Northeastern University’s former president Richard Freeland told Boston Magazine in 2014, “You get credit for the number of classes you have under 20 [students], so we lowered our caps on a lot of our classes to 19 just to make sure.”
How the pandemic and college ratings caused a perfect storm
When the pandemic set in, the college experience imploded overnight. All the academic and interpersonal hallmarks of a college education — landscaped verdant campuses, faculty-student mentoring, research opportunities, everything from sports to labs to parties to dorms — was all flattened to the width of an LCD screen.
Everyone was paying full sticker price to go to the same college, Zoom University.
During the pandemic, students needed academic flexibility. Thanks to the rigors of rankings, they did not get it.
When graduation and retention is 35% of their score, colleges simply could not afford to have students transfer out, or defer their graduation, or pause their enrollment, because that could have a disastrous impact on their completion rate.
One-fifth of the score comes from a survey sent out to admissions offices asking for assessments of peer universities, and another fifth is an implicit reckoning of institutional wealth — per-student spending, alumni giving, admitted student selectivity. Improving per-student spending also has the knock-on effect of pushing colleges to spend more, which has to be paid for by tuition.
“If you look at how US News ranks, they had no definition of academic excellence, so they created one,” Paterno said. “And how did they create that? They didn’t measure actual learning at all. Instead, they measured students’ selectivity, test scores and grades, institutional resources.”
“And that’s: how wealthy are you?”
Robert Morse, chief data strategist at US News, disputed the incentives Paterno described.
“It’s an incorrect premise to say that our methodology only incentivizes colleges to enroll affluent students. Schools will do better in the U.S. News Best Colleges ranking if they enroll and graduate high proportions of Pell Grant students vs those that only enroll very small percentages of Pell Grant students,” Morse said in an email.
Five percent of the US News score is related to Pell Grant student performance.
“It is also an incorrect assertion to imply that graduation rates only represent institutional wealth,” Morse said. “Graduation rates are highly important to students and their parents. When a student enrolls in a school their intent is to graduate with a diploma. If a school is only graduating half their students, that is an important indicator of poor outcomes and is independent of the type of students and their economic background.”
Bad for students, worse for applicants
The rankings make life even worse for applicants. The incentives are obvious, Paterno says: If you have two applicants of equal academic merit, and one comes from difficult circumstances and the other is affluent, who is more likely to graduate in four years? With 35% of the university’s score at stake, this isn’t hypothetical.
“Graduation rates are a very controversial metric,” Paterno said, “because it it gives colleges incentives to reject students they think won’t complete.”
During a pandemic — and especially during a year in which the number of college applications are through the roof — which applicants stood to gain and which stood to lose?
Another 5% is a newer figure that discerns how much money graduates owe. Making this a factor is an excellent way to push universities into accepting more wealthy or affluent people who don’t need to take loans.
“They don’t really want to accept students with hardships that might cause them to drop out,” Paterno said.
The pandemic turned the college application process from a merely Kafkaesque nightmare to an abattoir of despair.
Having dropped their testing requirements, universities received an onslaught of applications. One survey of applicants from consulting firm Art & Science Group found 20% were on some kind of wait list. While just 18% of middle- or lower-income kids were on a wait list, 32% of applicants from higher-income households were.
Some wait lists are still active today: the Applying to College subreddit inventories this anxiety, with applicants venting about “extended waitlists” in the University of California system, at Johns Hopkins, Duke, and more.
The US News applicant performance rating, worth 7%, doesn’t measure the quality of the instruction at the college but rather the standardized testing and class ranking of the people who are admitted. Even that, Paterno said, is a proxy for finances, namely the budget to recruit and advertise and offer scholarships to desirable high performers.
The incentives for colleges and universities are clear: with 35% of the score derived from the fraction who stick around, make it incredibly financially painful for students to take a break during the pandemic and don’t gamble on students who are financially precarious. With 20% derived from money, don’t cut tuition or offer breaks. With 5% from graduate indebtedness, only take the richest applicants. Remember, you’ve got a reputation to maintain: that’s 20% of your score, after all.
Is there a good rating system out there?
The current rating system that Paterno argues is fundamentally broken is, even still, the result of years of improvements.
This includes the removal of the acceptance rate, which was responsible for lots of applicant misery and a heap of third-party mailers going out to applicants the universities knew would likely fail to make the grade. In order to get their acceptance rate down, colleges and universities would mount colossal ad campaigns and direct mail to drive the applicant counts up.
Developing a new and improved rating system is fraught, as the Obama administration learned during a 2013 rollout of a federal college rating system that was eventually scrapped.
Even the best-intentioned arbiters face an uphill battle in trying to assign an arbitrary rating and ranking to a college, probably because the idea of reducing a colossal institution to a fragmentary rank completely misunderstands what a university actually is, and further sets up incentives that undermine their abilities to accomplish their actual mission.
There’s no indication that the market for ratings is slowing down. Following the outstanding commercial success of their college rankings, US News has since expanded to medical schools, law schools, and even high schools.
If anything, the market for ratings is only growing — their database now includes preschools and elementary schools, too.