Chronicle of Higher Education, Academics, Too, Can Be Led Astray by College Rankings, Study Finds, by Peter Schmidt:

Prospective students are hardly the only ones who can be overly fixated on college rankings and make poor judgments of the institutions as a result. A new study suggests that academics, too, can have their opinion of a university seriously skewed by how some publication ranks it—even when judging the quality of the institution's offerings in their own academic field.

The fault appears to lie with a psychological phenomenon known as the "anchoring effect." When people are asked to make judgments in ambiguous circumstances—by, for example, putting a dollar value on a home or a used car—most will start with whatever information is made available to them and work from there. …

Nicholas A. Bowman, a postdoctoral research associate at the Center for Social Concerns at the University of Notre Dame, and Michael N. Bastedo, an associate professor of education at the University of Michigan's Center for the Study of Higher and Postsecondary Education, found a similar dynamic at work in a new study of university rankings and reputations.

In a paper they plan to present at the annual meeting this weekend of the American Educational Research Association, the two researchers say The Times Higher Education Supplement created a "natural experiment" for measuring the anchoring effect's influence when it introduced its World University Rankings, in 2004.

In each of the first three years it published its rankings, the magazine, now known as Times Higher Education, based 40 percent of the score it assigned each university on a reputational survey that asked college faculty members and administrators around the world to list up to 30 universities that they considered the leaders in their general area of study. (The five areas were defined as science, technology, medicine, social science, and arts and the humanities.) In the first year the academics were surveyed, they answered the reputational survey in the absence of any world rankings published by the magazine. It was only in subsequent years that rankings potentially could influence their opinion.

In their analysis, Mr. Bastedo and Mr. Bowman found that results of the reputational survey conducted in the second year much more closely matched the overall published rankings than did the results of the first year's survey. In other words, institutions that ranked high in the first year fared significantly better on the reputational survey in the second, suggesting that the widely publicized rankings had helped a consensus form around the perceived superiority of certain institutions. …

Although their paper is their first to examine the anchoring effect's impact on college reputations, Mr. Bastedo and Mr. Bowman have scrutinized rankings systems before. In a paper published in February in the American Journal of Education, they examined the U.S. News & World Report rankings and similarly concluded that colleges' reputations are influenced by rankings.