Following up on my previous post, The Most-Cited Law Faculties: Sisk-Leiter (Westlaw) v. Sag (HeinOnline): Matthew Sag (Emory; Google Scholar), Forward Looking Academic Impact Rankings for U.S. Law Schools, 51 Fla. St. U. L. Rev. ___ (2023):
Although the very concept of law school rankings is currently under fire, rankings abolitionism is misplaced. Given the number, diversity, and geographic dispersion of the more than 190 law schools fully accredited by the American Bar Association, rankings are essential to enable various stakeholders to make comparisons between schools. However, the current rankings landscape is dire. The U.S News law school rankings rely on poorly designed, highly subjective surveys to gauge “reputational strength,” rather than looking to easily available, objective citation data that is more valid and reliable. Would-be usurpers of U.S. News use better data, but make other arbitrary choices that limit and distort their rankings. One flaw common to U.S. News and those who would displace it is the fetishization of minor differences in placement that do not reflect actual differences in substance. This information is worse than trivial: it is actively misleading. This Article proposes a new set of law school rankings free from all of these defects.
The Forward-Looking Academic Impact Rankings (“FLAIR rankings”) introduced in this Article are based on data that shows how many times law review articles by each of 5,139 individual faculty members at 191 American law schools have been cited by other law review articles in the last five years. The FLAIR rankings can be used as an objective guide to the relative academic impact of law schools, or as a component in broader objective rankings. The FLAIR rankings are based on publicly available, reliable, and objective data obtained from law school websites and the research platform, HeinOnline. The FLAIR rankings include all fully ABA accredited law schools, unlike alternative rankings of academic influence that are selective, often arbitrarily so. Moreover, the FLAIR rankings are designed to impart meaningful information by clustering schools into tiers based on their distance from the mean of all schools and deemphasizing ordinal rankings. Thus, the FLAIR rankings enable readers to make rational comparisons between law schools, rather than simply creating a hierarchy for hierarchy’s sake.
Matthew Sag (Emory; Google Scholar), Law School Academic Impact Rankings, With FLAIR:
I am pleased to announce the release of the Forward-Looking Academic Impact Rankings (FLAIR) for US law schools for 2023. I began this project two years ago because of my intense frustration that my law faculty (Loyola Chicago, at the time) had yet again been left out of the Sisk Rankings. The project has evolved and matured since then, and the design of the FLAIR rankings owes a great deal to debates that I have had with Prof. Gregory Sisk, partly in public, but mostly in private. …
How do the FLAIR rankings compare to other law school rankings?
Among their many flaws, the U.S News law school rankings rely on poorly designed, highly subjective surveys to gauge “reputational strength,” rather than looking to easily available, objective citation data that is more valid and reliable. Would-be usurpers of U.S. News use better data but make other arbitrary choices that limit and distort their rankings. One flaw common to U.S. News and those who would displace it is the fetishization of minor differences in placement that do not reflect actual differences in substance. In my view, this information is worse than trivial: it is actively misleading.
The FLAIR rankings use objective citation data that is more valid and reliable than the U.S. News surveys, and unlike the Sisk rankings, FLAIR gives every ABA accredited law school a chance have the work of its faculty considered. Obviously, it is much fairer to assess every school rather than arbitrarily excluding some based on an intuition (a demonstrably faulty intuition at that) that particular schools have no chance to ranking the top X%. Well, it’s obvious to me at least. But perhaps more importantly, looking out all the data gives us a valid context to assess individual data points. The FLAIR rankings are designed to convey relevant distinctions without placing undue emphasis on minor differences in rank that are substantively unimportant. This goes against the horserace mentality that drives so much interest in U.S. News, but I’m not here to sell anything.
What are the relevant distinctions?
The FLAIR rankings assign law faculties to four separate tiers based on how their mean and median five-year citation counts compared to the standard deviation of the means and mediums of all faculties. Tier 1 is made up of those faculties that are more than one standard deviation above the mean, Tier 2 is between zero and one standard deviations above the mean, Tier 3 ranges from the mean to half a standard deviation below, and Tier 4 includes all of the schools more than half a standard deviation below the mean. In other words, Tier 1 schools are exceptional, Tier 2 schools are above average, Tier 3 are below average, and Tier 4 are well-below average.
