March 30, 2021

Derek Muller (Iowa) on the US News rankings disaster this year

Professor Muller's analysis and his recommendations are worth taking seriously.   I'm far from being against rankings, as readers know; but the US News rankings use mostly garbage data, aggregate it through a meaningless and indefensible formula, and create massive incentives for dishonesty and strategic behavior, that distort educational goals and values.  As Professor Muller suggests, we need massive non-cooperation to put Bob Morse & co. out of business.

ADDENDUM:  Some examples of perverse strategic behavior to game the rankings, from law professor Jeffrey Harrison (Florida).


March 30, 2021 in Rankings | Permalink

March 29, 2021

US News seems intent on driving home the fact that the ranking formula is arbitrary and meaningless

Yesterday, they issued the third correction to the embargoed rankings (which will be released tomorrow) in the last week!   Here's the explanation:

For the overall ranking, U.S. News removed the metric for ratio of credit-bearing hours of instruction provided by law librarians to full-time equivalent law students [.25%, reducing the library weighting to 1.75%] and increased the weighting for the bar passage rate indicator [by .25%, for a new total of 2.25%]. As a result, we recalculated the rankings.

Why these one-quarter of one percent adjustments?  Who knows?  Certainly not the US News.com editors.  But here's the real kicker:  this tiny change altered the rank of 35 law schools, including 9 in the top 30!  Imagine what might have happened had they decided the bar passage indicator should be 3.5%!

Is it possible to overstate the sheer stupidity of all this?

 


March 29, 2021 in Rankings | Permalink

Your annual reminder that movements in the US News.com rankings are almost all meaningless

With the new nonsense numbers about to appear, it's worth reminding everyone (and especially journalists) that:  95% of movement in the US News.com "overall" rank is attributable to (1) schools puffing, fudging or lying about the self-reported data more than their peers (or the reverse, for those schools that drop); or (2) simply being more aggressive at  manipulating the metrics they can control than their peers (or the reverse, for schools that drop in the overall ranking). 

Remember that US News.com audits none of the self-reported data on job placement, expenditures, student credentials, faculty-student ratios etc..   Schools can also inflate their rank by shrinking the size of their 1L class (thus improving median credentails), and taking more transfers or LLM students, among other "tricks of the trade."

Any journalist that reports a change in US News rank as "news" without further investigation of the underlying "data" is perpetuating a fraud on the public.

ADDENDUM:  As law professor Derek Muller (Iowa) reminds me, some movement this year will be due to the new criterion US News.com added:  5% of the total score will factor in a mix of average debt (for those students with debt) and the percentage of students with debt.  To make room for this, they arbitrarily reduced the weights of some of the other factors by small amounts (e.g., expenditures, student credentials) in their arbitrarily weighted stew of factors.  This change will, of course, lead to new forms of "gaming" the rankings, which I'll write about soon.


March 29, 2021 in Rankings | Permalink

February 25, 2021

Blast from the past: on the uselessness of ranking law reviews by Google Scholar metrics

Back in 2016.


February 25, 2021 in Rankings | Permalink

February 02, 2021

Rostron & Levit update their invaluable guide to submitting to law reviews

They write:

 

We  just updated our charts about law journal submissions, expedites, and rankings from different sources for the Spring 2021 submission season covering the 199 main journals of each law school.   

We have created hyperlinks for each law review to take you directly to the law review’s submissions page. Again the chart includes as much information as possible about what law reviews are not accepting submissions right now and what months they say they’ll resume accepting submissions.

Washington and Lee has changed its methodology on law review statistics.  Now Washington and Lee only ranks the top 400 law review (many of which are specialty journals, online supplements, etc.), so not all flagship journals are now ranked by them.  But we put in the data for those that are ranked.  [BL comment:  the W&L data is junk, ignore it]

Continue reading


February 2, 2021 in Advice for Academic Job Seekers, Of Academic Interest, Rankings | Permalink

January 11, 2021

Ratio of median debt of law school graduates to their median income

Another interesting study from Professor Derek Muller (Iowa).


January 11, 2021 in Legal Profession, Rankings | Permalink

December 23, 2020

Non-JD enrollment in law schools continues to grow

Derek Muller (Iowa) documents the trend.   Non-JD students (mostly LLM students, but also some undergradutes [especially at Arizona], as well as non-degree candidates) have the virtue of generating revenue while being invisible in the U.S. News rankings.


December 23, 2020 in Legal Profession, Of Academic Interest, Rankings | Permalink

October 27, 2020

How NYU recruited faculty from top law schools in the 1990s: a case study

Richard Stewart, a leading scholar of administrative and environmental law at Harvard, moved to NYU in the early 1990s.  He lived in a 4,000 square foot town house in Greenwich Village, a few blocks from NYU's law school.  This was rented to him by the University at an undisclosed price, but no doubt well below market rates which even a law professor could not ordinarily afford in that area.  As part of the deal, he had the option to sell the property, with some of the gain going to NYU, and some to Professor Stewart.  He exercised that option a few years ago, netting over eight million dollars.  Eight million divided by the 25 years he taught at NYU up to that time works out to an additional $320,000 per year of service, on top of his salary.  (The article notes that another NYU law faculty member, recruited from Chicago in the early 1990s, lives in a similar townhouse, but without the option to profit from a sale.)


October 27, 2020 in Faculty News, Of Academic Interest, Rankings | Permalink

October 23, 2020

The era of Condorcet polls is over

The results of the last poll are here, and basically regurgitate US News (or the "halo effect" of school name) with a couple of exceptions.  But there was also more mischief this time.   As one reader reported (by examining the detailed breakdown of votes):

I’m reaching out because I was looking at the raw balloting data on the website and I noticed something curious.  There are 66 voters in your survey who rank UC Davis as one of the top 10 law schools and UC Hastings as one of the bottom 8 law schools.  Interestingly, only 5 voters rank UCLA as one of the top 10 law schools and UC Hastings as one of the bottom 8 law schools and only 2 voters rank UC Irvine as one of the top 10 law schools and UC Hastings as one of the bottom 8 law schools.  Whoever these pro-Davis, anti-Hastings voters are, they appear to be a large percentage of the respondents and to have a material impact on Hastings’ rank.  It is possible my read of the data is incorrect, but this is what jumps out at me once I load your spreadsheet into Stata.

Conversely, no voters rank Hastings in the top 10 and Davis in the bottom 8.

There were other, shall we say, peculiar patterns in the voting.   If someone wants to undertake a serious and informed survey about law faculty quality, get in touch, and I'll offer guidance about how to do it.  I don't have the time myself, but am happy to be an advisor and to publicize the results.


October 23, 2020 in Rankings | Permalink

October 20, 2020

Rank the top 40 law schools in terms of the scholarly strength of the faculty

It's time for our annual Condorcet poll of the best scholarly faculties in U.S. law schools.   Please note the instructions:  "Rank order the law schools below in terms of the scholarly strength of the faculty (consider only scholarly strength in your best judgment, not current U.S. News rank!)."  If you don't have informed opinions about the scholarly strength of different law faculties, then you should not participate. 

I listed 58 schools that might have some claim to being in the top 40 for scholarly accomplishment.   Have fun!  Note that the more schools you rank, the more impact your vote will have on the results.

(Any faculty found mobilizing votes on social media will have their school eliminated from the results!)


October 20, 2020 in Rankings | Permalink