Following up on Saturday’s post, New York Times, Rankings Schadenfreude: Elite Law Schools Boycotted U.S. News But Now May Be Paying A Price: Derek Muller (Iowa; Google Scholar), Some Law Schools Fundamentally Misunderstand the USNWR Formula, in Part Because of USNWR’s Opaque Methodology:
Earlier this week, USNWR announced it was indefinitely postponing release of its law school rankings, after delaying their release one week. It isn’t the first data fiasco that’s hit USNWR in law rankings. In 2021, it had four independent problems, two disputed methodology and two disputed data, that forced retraction and recalculation.
There are likely obvious problems with the data that USNWR collected. For instance, Paul Caron earlier noted the discrepancies in bar passage data as released by the ABA. I noticed similar problems back in January, but (1) I remedied some of them and (2) left the rest as is, assuming, for my purposes, close was good enough. (It was.) The ABA has a spreadsheet of data that it does not update, and individual PDFs for each law school that it does update—that means any discrepancies that are corrected must later be manually supplemented to the spreadsheet. It is a terrible system. It is exacerbated by the confusing columns that ABA uses to disclose data. But it only affected a small handful of schools. It is possible USNWR has observed this issue and is correcting it. And it is possible this affects a small number of schools.
A greater mistake advocated by law school deans, however, relates to employment data. Administrators and deans at Yale, Harvard, and Berkeley, at the very least, have complained very publicly to Reuters and the New York Times that their employment figures are not accurate.
They are incorrect. It reflects a basic misunderstanding of the USNWR data, but it is admittedly exacerbated by how opaque USNWR is when disclosing its metrics.
In 2014, I highlighted how USNWR publicly shares certain data with prospective law students, but then conceals other data that it actually uses in reaching its overall ranking. This is a curious choice: it shares data it does not deem relevant to the rankings, while concealing other data that is relevant to the rankings. …
USNWR’s opaque approach to “employment rank” certainly contributes to law schools misunderstanding the formula. But law schools—particularly elite ones who initiated the boycott and insisted they do not care about the rankings, only now to care very much about them—should spend more effort understanding the methodology before perpetuating these erroneous claims.
Prior TaxProf Blog coverage:
- U.S. News Releases Preview Of 2024 Law School Rankings: Top 14 And New Methodology (Apr. 11, 2023)
- Will Law Schools With Strong Tax Programs Get A Boost In The 2024 U.S. News Rankings? (Apr. 11, 2023)
- Will U.S. News Refuse To Count Votes From Deans, Faculty, Lawyers, And Judges From Boycotting Law Schools In Reputation Metric In 2024 Rankings? (Apr. 12, 2023)
- U.S. News Delays Release Of Law School Rankings By One Week Due To ‘Unprecedented Number Of Inquiries From Schools.’ Is Bar Data To Blame? (Apr. 14, 2023)
- The U.S. News Law School Rankings: Say Hello To The New T14 (Apr. 15, 2023)
- What Is Behind The Delay In Releasing The U.S. News Law School Rankings? (Apr. 18, 2023)
- U.S. News Delays Release Of Law School Rankings Again Due To ‘Unprecedented Number Of Inquiries,’ Including From Schools That Are Ostensibly Boycotting The Rankings (Apr. 19, 2023)
- U.S. News Indefinitely Postpones Law And Medical School Rankings Amid Backlash (Apr. 21, 2023)
- NY Times: Rankings Schadenfreude — Elite Law Schools Boycotted U.S. News But Now May Be Paying A Price (Apr. 22, 2023)
