November 21, 2020 by Philip Lewenstein

Keep College Rankings in Perspective

Americans are fascinated with rankings, whether they are for football teams, consumer products, or colleges and schools. Each fall, I eagerly look forward to the release of various rankings.

On September 14, U.S. News and World Report announced its 36^th college rankings, a source of interest for many parents and students working on their college searches. Meanwhile, the rankings continue to elicit criticism from educators and policymakers who question their purpose and methodology.

The U.S. News rankings are considered a powerful and controversial force shaping American higher education, fueling admission by students to prestige schools. Critics say these rankings contribute to an elitist culture, putting prestige and exclusivity ahead of inclusivity and student success.

Two years ago, for example, six Democratic U.S. senators wrote a letter to U.S. News asking that more weight be given to institutions that open their doors to students from underrepresented backgrounds. They asked U.S. News to better align rankings with three longstanding goals of federal financial aid: improving college access, supporting student success, and providing every talented student a pathway to economic stability and meaningful participation in the nation’s economic, social, and civic life. The senators were Tammy Baldwin, Cory Booker, Christopher Coons, Kamala Harris, Chris Murphy, and Brian Schatz.

“We fear U.S. News continues to create a perverse incentive for schools to adopt or maintain policies that perpetuate social and economic inequalities,” they wrote. The senators also called on U.S. News to better recognize community colleges and minority-serving institutions that “serve as our country’s engines of social mobility and incent others to do so.”

Debate on Value of Rankings Focuses on Fit Versus Prestige

When new rankings are published, the debate between fit and prestige in college choices is amplified. Ideally, prospective students would select potential colleges that will best meet their educational interests, needs, and goals rather than be consumed with colleges’ prestige as reflected in rankings.

In fact, some educators recommend that students entirely ignore rankings in their college planning and focus on engagement in colleges of interest—that is, how well can students participate fully in the academic, civic, and social life of the college.

The Varsity Blues scandal illustrates how some parents are willing to cheat to see their children admitted to the most-selective, highly-ranked colleges, institutions that often tout their lofty rankings ( see “Scandal Highlights Troubling Admissions Issues, Need for Reform,” www.philsfocus.com).

Melissa Korn, who covered the admission scandal for the Wall Street Journal came up with ten ways to fix college admissions based on interviews with college-admissions officers, high-school and private counselors, parents, students, and others (“How to Fix College Admissions,” November 30-December 1, 2019).

The first suggestion is to eliminate rankings, which “fuel a vanity race for admission to the No. 1 school,” she said.

Concerned with the overwhelming focus on U.S. News rankings, other organizations have developed alternative rankings based on different criteria:  the Washington Monthly, Wall Street Journal, and Money.

U.S. News asserts that it provides objective information and can be one of several factors in choosing the right fit for students. Further, the information helps families understand the differences between colleges and can be a starting point to inform decision-making.

Authors Provide Perspectives on College-ranking Juggernaut

In The Merit Myth (New Press, 2020), authors Anthony P. Carnevale, Peter Schmidt, and Jeff Strohl offer an insightful perspective on “the college-ranking juggernaut.”

The volatility and competitiveness of the higher-education market intensified with the emergence of the U.S. News rankings, the authors say. Before the rankings debuted in 1983, college-goers based their judgments of best colleges mostly on impressions rather than numerical data such as SAT scores.

“U.S. News changed the game by purporting to rank America’s top colleges objectively, immediately spurring demand for seats at those institutions among students seeking to get into the best college possible,” they write. “The popularity of U.S. News rankings exploded, and the colleges that had already been difficult to get into soon became almost impenetrable for anyone but legacy applicants, athletes, and wealthy students who attended top prep schools.

“Most profoundly, the rankings helped to give rise to the now common belief that colleges can be judged on inputs—the students they admit, the faculty members they hire, the financial support they receive—rather than on output that is their raison d’etre, the learning and growth they produce in their students”

Carnevale, Schmidt, and Strohl note that the “the power U.S. News asserted over colleges was immense.” Those that dropped in rankings could experience steep declines in applications, find it harder to recruit faculty or solicit donations, and even suffer downgrading of their bond rating that forced them to pay higher interest rates.

Not all colleges have competed in the rankings honestly, the authors point out. Many colleges have submitted false data to the magazine, while others have sought to skew their numbers upward by withholding data on certain parts of the student population such as legacies, recruited athletes, or those who arrived on campus early via summer-remediation programs or through mid-year admissions in January. “Many colleges have embraced strategies intended to game the metrics,” the authors conclude.

Clearly, the most prominent rankings are those of U.S. News and World Report. Since the rankings were first released in 1983, they have continued to evolve with more schools and data points.

The academic yardsticks U.S. News favored didn’t just provide fodder to produce the rankings but also signaled to students and parents what to emphasize when looking for a college—mostly the quality of incoming students rather than what undergraduates learned or what they did after graduation, according to Jeffrey Selingo, author of Who Gets In and Why: A Year Inside College Admissions (Scribner, 2020).

“Over decades, the rankings narrowed the view of seniors to focus on just a handful of schools,” Selingo says. “And, in turn, the standards also sent a message to college leaders about what they should prioritize. By the late 1990s, college presidents and trustees were paying close attention to the rankings.”

Selingo describes how Richard Freeland, who became president of Northeastern in Boston in 1996, set a target in his university’s strategic plan to become a top 100 university (it was ranked 162) during his presidency. Freeland was unapologetic about setting a goal using a magazine ranking that most of his counterparts criticized publicly, though many secretly wanted to move up, too, Selingo says.

By 2004, Northeastern had risen 42 spots in the U.S. News rankings to 120. As Freeland retired in 2006, Northeastern was 98th, and the university reached 44th in 2019. However, rising in the rankings required tradeoffs, Selingo notes.

“While the number of students from families making more than $200,000 at Northeastern grew from the single to double digits in percentage terms under Freeland’s watch, he was almost embarrassed to tell me that the proportion of Pell Grant recipients—the poorest students at the university—dropped to 15 percent,” Selingo writes. “That’s at a school that once prided itself on educating the children of blue-collar Bostonians.”

Another university that focused on increasing its U.S. News ranking is the University of Florida. Francie Diep describes how the University of Florida’s Board of Trustees in 2003 sought a president who would commit to moving the institution up in the U.S. News rankings (“The Rules of the Game: How the U.S. News rankings helped reshape one state’s public colleges,” The Chronicle of Higher Education, October 2, 2020).

In 2003, the University of Florida ranked 52 overall and 17 among public universities. The trustees wanted to see the Gators among the top ten public institutions. Board chair Manny Fernandez said that, “We needed to change the external perception of the University of Florida. U.S. News was one very effective way to do that.”

Bernie Machen, who became president in January 2004, shared the trustees’ goal. While many people inside and outside higher education today argue that a high ranking doesn’t necessarily translate to high quality, Machen told Diep that he believed in many of U.S. News’s metrics. “I feel many of them are relevant and worth shooting for, and improving on those variables made us a better institution,” he said.

When Machen retired in 2014, the University of Florida was 14th among public universities. The Gators broke into the top ten in 2017, and this year ranked sixth.

Despite criticizing the rankings, many colleges are quick to tout their successes. For example, on September 14, the date U.S. News released its 2021 rankings, Northwestern University published a release: “Northwestern ranked in top 10 by two major ranking organizations.”  Northwestern maintained its number 9 spot in U.S. News for the second straight year, “cementing its position among the country’s top research universities for three straight years.” Also, Northwestern is ranked 10^th in the Wall Street Journal/Times Higher Education Rankings 2021, up one position from last year.

Even as the rankings evolve, they remain crucial to both the future of U.S. News and the elite schools in the top tier seeking to maintain their exclusive brand, Selingo says.

More college freshmen report each year that the rankings are an important, but not top, factor in their final decision, especially among those who go to a selective school, according to Selingo. When a top school drops in the rankings, it is forced to admit more students in subsequent years and is more generous with financial aid to make up for falling yield rates, according to the National Bureau of Economic Research.

“But the rankings, in large part, also give prospective students a narrow lens on the college search,” Selingo says. “They don’t look up and out beyond the top 10 or 20 or whatever.

“In many ways, the rankings have become the tail that wags the dog in higher education. The race for prestige has produced winners among colleges, their presidents, and trustees, as well as their alumni who feel their degree is worth more now than the day they graduated. But many of those victories have come at a cost to applicants and society as a whole.”

Selingo notes that prospective high-school students might like that their college lists include only top-ranked colleges, yet they typically don’t realize they are paying much of the bill to build or maintain that prestige. Colleges seeking to rise in the rankings need to substantially increase their spending per student, which can result in large tuition increases, higher enrollments of students from wealthy families, and decreases in the poorest students.

U.S. News Revises Metrics over Time

The 2021 edition of U.S. News provides data on 1,800 colleges and universities, with rankings for more than 1,400 institutions. Schools provided most of the data used to compile the rankings to U.S. News in an annual survey and were instructed to confirm the accuracy of their data.  U.S. News points out that the rankings data pertain to student and faculty cohorts that predate the coronavirus pandemic, and the impact of COVID-19 on higher education is not reflected in schools’ overall performances.

U.S. News places colleges in several categories. For example, national universities are often research-oriented and offer bachelor’s, master’s, and doctoral degrees. National liberal-arts colleges emphasize undergraduate education and award half or more of their degrees across liberal-arts fields. Regional colleges and universities are split into eight categories dependent on geographic location and whether master’s degrees are offered.

U.S. News uses 17 metrics to assess academic quality, with the greatest emphasis on outcomes, particularly a college’s ability to retain and graduate students. Other factors include class size, undergraduate academic reputation, and spending on instruction and other educational investments.

U.S. News revises its methodology over time. Two years ago, for example, it tweaked its rankings to give greater weight to the graduation rates of students who receive Pell grants. U.S. News also removed from the formula a college’s acceptance rate and reduced the importance of standardized-test scores and high-school class standing, two measures often associated with students from wealthier families.

In 2019, U.S. News added more consideration for first-generation students, giving more emphasis and credit to schools doing more to help those students.

Major changes in 2020 include two measures of student debt: graduate indebtedness takes into account the average amount of federal loan debt among graduating classes in 2019 at the bachelor’s level and, second, the proportion of graduates who took out federal loans. More weight was placed on “outcome measures” from 35 % of weighting to 40 % with a corresponding reduction in the weights given to SAT and ACT scores, high-school class standing, and alumni giving.

Alternatives to U.S. News Broaden Indicators for Rankings

An increasingly popular alternative to the U.S. News rankings is published annually by the Washington Monthly, a bimonthly, nonprofit magazine of U.S. politics and government based in Washington D.C.

The Monthly’s 2020 College Guide and Rankings describes itself as “the socially conscious alternative” to U.S. News and World Report. “While U.S. News rewards colleges for their wealth, exclusivity, and prestige, the Monthly ranks them on how well they serve the country as a whole—by recruiting and graduating non-wealthy students, encouraging student activism, and producing research and technologies that create high-paying jobs and address threats like climate change,” the magazine says.

While 19 of the U.S. News top national universities are elite private institutions, more than half of the Monthly’s top 20 are public institutions, the editors explain. For example, Texas A & M ranked 70^th by U.S. News but is 12^th on the Washington Monthly list because it enrolls and graduates large numbers of first generation and Pell grant students as well as science and engineering Ph.Ds.

The rankings consist of three equally-weighted portions: social mobility, research, and community and national service.

For social mobility, the magazine looks at whether an institution does a good job at both enrolling and graduating students from lower-income families or who are the first in their family to go to college. The Monthly looks at whether students who can repay their loans and how affordable a college is for a student after all grants and scholarships are provided.

Primary research measures are the number of students who earn doctorates and the amount of research spending on campuses. Service includes the share of students involved in Peace Corps and ROTC branches, and whether colleges use federal work-study funds to support community service.

“The colleges that score well on the Monthly’s rankings are swimming against the tide of a higher education system that forces non-affluent students to pay ever-higher tuition, take on growing amounts of debt, and mortgage their futures,” the magazine says. “The same system has put many admirable small schools under severe stress—made worse by the pandemic.”

Another alternative to U.S. News is the Wall Street Journal/Times Higher Education College Rankings (September 18, 2020). The rankings are based on 15 key indicators that assess colleges in four areas: outcomes, resources, engagement, and environment.

Outcomes account for 40 % of the weighting and measure items like the salary graduates earn and the debt burden they take on. Resources, with a 30 % weighting, is mainly a proxy for the spending schools put into instruction and student services. Engagement, drawn mostly from a student survey and with a 20 % weighting, examines views on factors like teaching and interactions with faculty and other students. Environment, at 10 %, assesses the diversity of the university community.

The Journal points out that the relative ranking of any university should not be the final word on whether it’s right for every applicant, and the rankings are intended to serve as a starting point for families considering their options. Further, the Journal notes, all the universities ranked, whether at the top or bottom of the list, can produce exceptional students; college success is a matter of what students make of the opportunities.

Money’s best colleges for 2020 ranked by value uses a methodology based on 37 factors in three categories: quality of education (30 % of weighting), affordability (40 %), and outcomes (30 %).

Money analyzes data for more than 700 four-year colleges such as estimated price without aid, estimated price with average percentage of students who get a grant, graduation rate, average student debt, and early career earnings.

Four Key Rankings Have Much in Common but Differences Based on Criteria

Michael T. Nietzel, president emeritus of Missouri State University who writes on higher education for Forbes, provides observations comparing rankings by U.S. News, Washington Monthly, the Wall Street Journal, and Money (“Comparing the Major College Ranking Systems,” September 9, 2020).

First, considerable consensus exists, not surprising because all the ranking systems give substantial weight to factors like graduation rates, faculty quality, student debt, and graduate earnings, he says. Among national universities, six institutions were ranked in the top 20 by all four systems: Princeton, Harvard, Yale, MIT, Stanford, and Duke. Five more made three of the four top-20 lists: Columbia, University of Pennsylvania, Vanderbilt, Rice, and UCLA.

Second, Nietzel notes, private universities dominated the U.S. News and WSJ/THE top-20 rankings; only one public university received a top-20 placement—UCLA—which slipped in at number 20 on the U.S. News list.

Third, public research universities fared better in the Money and Washington Monthly rankings with 11 universities making the top 20 in both, Nietzel says.

Fourth, the University of California has a dominant position among top-ranked public institutions, he says. University of California campuses make up five of the eleven public schools in both Money’s and Washington Monthly’s top-20 lists.

Fifth, the public-private discrepancy is one of the most significant consequences of the many methodological differences.

“U.S. News and the WSJ/THE give considerable weight to institutions’ financial resources and reputation, two factors that favor wealthier universities and are either minimized or eliminated by Money and Washington Monthly,” Nietzel says. “Conversely, within their systems, the latter two sources put greater premiums on institutions’ affordability, their impact on the social mobility of graduates, and their promotion of various forms of public service.”

I checked the rankings for four colleges of interest to me (since my family members attended them), and I noticed variation depending on criteria. The University of Minnesota tied for 66th in the U.S. News national universities group, 67th in the Washington Monthly, 58th in Money, and 91 in the Wall Street Journal.

Northwestern University tied for 9^th in U.S. News, 30th in the Washington Monthly, 53 in Money, and 10^th in the Wall Street Journal. The University of Southern California tied for 24th in U.S. News, 53 in the Washington Monthly, 156th in Money, and 19th in the Wall Street Journal. Wellesley tied for fourth in the U.S. News national liberal-arts group, 13^th in the Washington Monthly, 88^th in Money, and 29th in the Wall Street Journal.

Critics Advise Ignoring Rankings to Provide Best Fit for Students

Despite fascination with rankings, the first step to choosing the right college should be to ignore the rankings, according to Stanford researchers. A study by scholars at the Stanford Graduate School of Education concluded that rankings are problematic (“A Fit over Rankings: Why College Engagement Matters More than Selectivity,” October 2018).

The study concludes that rankings are problematic; college selectivity is not a reliable predictor of student learning, job satisfaction, or well-being; and engagement in college is more important than where one attends.

The researchers found that the metrics used in rankings are weighted arbitrarily and were inaccurate indicators of a college’s quality or positive outcomes for students.

“A good fit is a college where a student will be engaged—in class and out—by what the college has to offer,” the study says. “With over 4,500 colleges in the United States, there are many schools from which to choose. We encourage students and families to look beyond rankings in the college search process, and instead seek a school where students can participate fully in academic, civic, and social life in order to thrive both during the college years and beyond.”

Peter Van Buskirk, a nationally acclaimed author and leading voice for student-centered college admissions, urges students to keep rankings in perspective in their college planning. “Resist the temptation to obsess on a set of numbers,” he says. “Instead, focus on developing a list of colleges based on who you are, why you want to go to college, and what you want to accomplish during your undergraduate years. And don’t lose sight of how you like to learn. Stay student-centered and you will discover the colleges that are truly best for you”

Van Buskirk points out that rankings are not science because the data-collection process relies on self-reported information from colleges and universities. Further, rankings are highly subjective, he notes.

“While many institutions might look alike on the surface, they are very different with regard to programs, instructional styles, cultures, values, and aspirations—another reason why trying to rank them is a daunting, if not impossible task,” Van Buskirk says.

Van Buskirk urges students to look for evidence that rankings will make a difference in their college-planning outcomes. “Rankings don’t get kids into college nor do they necessarily point you in the direction that is best for you,” he says.

 Over the past 30 years, the college-going process has been turned upside down by ranking guides. Whereas the focus should be on the kids—and what is best for them—college-ranking guides focus on destinations that are presumed to be most desirable, Van Buskirk says. In reality, college rankings are artificial metrics for quality in education that detract from sensible, student-centered decision-making.

“Where the student might be headed becomes more important than what is to be accomplished or why that goal might be important or how the institution might best serve the students,” he says. “When distracted by the binding power and prestige that rankings bestow upon a few institutions, it is easy to lose sight of one’s values and priorities as well as the full range of opportunities that exist.”

Keep College Rankings in Perspective

College rankings benefit the media that publish them, they influence strategic planning by some colleges, they provide useful information to students and families, and they satiate the public’s desire for ratings.

Some of the purveyors of rankings publish guidebooks informing families about colleges. Best Colleges published by U.S. News, for example, annually presents not only the rankings but informative articles about getting in and finding money as well as a directory of colleges and universities.

Nevertheless, the rankings, especially those of U.S. News, have generated critics, the harshest of whom call for eliminating rankings. These critics attack the methodology, and argue that the rankings don’t measure quality and incorrectly focus on inputs rather than outcomes.

Critics assert that the rankings lead to prospective students focusing more on the prestige of colleges rather than the best fit for the student.

College rankings are unlikely to go away. Therefore, students and parents need to keep them in perspective and not base decisions on their college selection on the rankings. Prospective students should not limit their college lists to top-ranked colleges but should consider the wide range of choices available and determine which schools best meet their educational interests, needs, and goals.