Monday, March 16, 2015

USnews.com rankings came out last week

Last week, a website, USNews.com, released its annual rankings of law and other professional schools.  (Since I was off-line last week, my comments had to wait.)  Commentary, predictably, focused on the overall rank assigned by the website--what is known to "insiders" as the "nonsense number," since it is the upshot of an inexplicable weighting of 12 different factors, many self-reported and so of dubious accuracy anyway, but the amalgamation basically stipulative.  Fortunately for USNews.com, many superficial journalists report the nonsense number, and changes in the nonsense number, as though they meant something. 

So, for example, much ink was spilled on the "fact" that the University of Michigan Law School had a nonsense number of 11th, just outside "the top ten" where it usually resides.  I did not see any journalist note, however, that just one raw score point (83 vs. 84) separated Michigan from Duke, Virginia, and Berkeley, all with a nonsense number of 8th.  In other words, even by its own terms, the USNews.com demotion of Michigan to 11th was meaningless.  (For those paying attention, Yale, because of its off-the-charts per capita expenditures, got a  raw score of 100, Harvard and Stanford got 96, Columbia and Chicago 93, NYU 89, Penn 88, and then Duke et al. with 84.) 

The most interest was in the nonsense number for UC Irvine, ranked for the first time this year.  UCI came in at #30, the highest nonsense number debut I've ever seen in USNews.com.  In USNews.com land, this puts UCI third in the UC system--behind Berkeley and UCLA, and ahead of UC Davis (31st) and UC Hastings (59th).  (Hastings has probably been the most dramatic victim, over many years, of the small, private school bias in the USNews.com rankings.)  Interestingly, UCI got this result despite weaker reputational scores:  29th in reputation among lawyers/judges (a rather good result, though, for a new school), and only 42nd among academics.  Almost every school with a nonsense number around UCI had a higher academic reputation score, and my guess is UCI's will now improve accordingly.  (The evidence for the echo chamber effect of the "overall rank" on the reputation scores in subsequent yeras is even greater now than in the past.)  My guess is all those annoyed by the UC system starting a new law school penalized UCI in the reputational survey--if USNews.com published the median and mode, we'd have some idea, but I wouldn't be surprised if the distribution was skewed in that way.  Counting against UCI is that it is still very small, and will presumably have to grow, which may affect other metrics.

USNews.com reported some curious data in various categories.  I note two examples.  Columbia, for the first time, reported the best student-faculty ratio in the nation:  6.3 to 1.  Yale was 7.6/1, Stanford 7.3/1.  Virginia reported 97.3% employed at graduation, but only 97% nine months out.  This might be an artifact of the fact that USNews.com for the first time did not give schools full credit for graduates in law school funded positions--though many of these positions are quite legitimate.

The USNews.com reign of terror has now been going on for 25 years.  It has been a disaster for legal education, though a boon for students with the favored numbers.  In the 1990s, I used to try to reason with the USNews folks, and they in fact corrected some of their worst mistakes--for example, using starting salary data without taking into account regional differences; doing reputation surveys based on "quartiles" (meaning the dumbest evaluator--the one who forgot to put Harvard in the top quartile--determined the score); and failing to adjust expenditures for cost-of-living differences.  But with regard to the basic problems--namely, that the weightings of the inputs are arbitrary, and that a lot of the data relied upon is bogus--they've done nothing.  The only remedy for the USNews.com reign of terror will be competing systems, though hopefully not ones that simply replicate the USNews.com mistakes, though that is mostly what we have had so far.

http://leiterlawschool.typepad.com/leiter/2015/03/usnewscom-rankings-came-out-last-week.html

Rankings | Permalink