April 14, 2022

"Do the U.S. News rankings rely on dubious data?"

CHE actually posted this as a question, and not a rhetorical one!

April 14, 2022 in Rankings | Permalink

April 13, 2022

More on expenditures and the USNews.com ranking stew

Derek Muller (Iowa) comments.

April 13, 2022 in Rankings | Permalink

April 05, 2022

Yale Law melodrama continues: Professor Stith disagrees with Dean, says student protest violated the Law School's Free Speech Policy

Story here (prior coverage--and earlier examples of the clearly dysfunctional institutional culture).  Despite the continued bad press, and even as its USNews.com reputation score falls to 3rd, its #1 scholarly impact positions depends increasingly on an ageing faculty, and its younger faculty increasingly live in New York City (or decamp to NYC schools), the per capita expenditures metric will still keep Yale at #1 in USNews.com.

April 5, 2022 in Of Academic Interest, Rankings | Permalink

March 28, 2022

Law Schools Unfairly Ranked by U.S. News


I've occasionally commented in the past about particular schools that clearly had artificially low overall ranks in U.S. News, and readers e-mail me periodically asking about various schools in this regard.   Since the overall rank in U.S. News is a meaningless nonsense number, permit me to make one very general comment:   it seems to me that all the law schools dumped into what U.S. News calls the unranked lower tiers--indeed, all the law schools ranked ordinally beyond the top 25 or 30  based on irrelevant and trivial differences-- are unfairly ranked and represented.  This isn't because all these schools have as good faculties or as successful graduates as schools ranked higher--though many of them, in fact, do--but because the metric which puts them into these lower ranks is a self-reinforcing one, and one that assumes, falsely and perniciously, that the mission of all law schools is the same.  Some missions, to be sure, are the same at some generic level:  e.g., pretty much all law schools look to train lawyers and produce legal scholarship.  U.S. News has no meaningful measure of the latter, so that part of the shared mission isn't even part of the exercise.  The only "measures" of the former are the fictional employment statistics that schools self-report and bar exam results.  The latter may be only slightly more probative, except that the way U.S. News incorporates them into the ranking penalizes schools in states with relatively easy bar exams.  So with respect to the way in which the missions of law schools are the same, U.S. News employs no pertinent measures. 

But schools differ quite a bit in how they discharge the two generic missions, namely, producing scholarship and training lawyers.  Some schools focus much of their scholasrhip on the needs of the local or state bar.  Some schools produce lots of DAs, and not many "big firm" lawyers.    Some schools emphasize skills training and state law.  Some schools emphasize theory and national and transnational legal issues.   Some schools value only interdisciplinary scholarship.  And so on.  U.S. News conveys no information at all about how well or poorly different schools discharge these functions.  But by ordinally ranking some 150 schools based on incompetently done surveys, irrelevant differences and fictional data, and dumping the remainder into a "second tier", U.S. News conveys no actual information, it simply rewards fraud in data reporting and gratuitously insults hard-working legal educators and scholars and their students and graduates.

March 28, 2022 in Rankings | Permalink

March 23, 2022

Blast from the past: Per capita expenditures is the tail that wags the US News ranking dog

March 21, 2022

USNews.com makes more arbitrary changes to the ranking formula...

...and one can rest assured that U.S. legal education-- which is run by a defunct news magazine with a website--will adjust accordingly.  One change will be a boon for law libraries:  1% of the score (why 1%?  no one knows, not even Bob Morse) now consists in the ratio of full-time law librarians to students at a school.  That metric will also exacerbate the general bias in many aspects of the USNews.com formula that favors smaller schools over larger ones, due to economies of scale.  Bar passage rates will also account for 3% rather than 2.25% (why not 10% or 15%?  no one knows, and certainly not Bob Morse), but will take into account how a school's graduates perform across jurisdictions.  If one is going to count bar passage rates, then that at least makes some sense, even if the relative weighting is inexplicable.

Unknown at the time of this writing is which other factors in the ranking stew had their weight reduced.

March 21, 2022 in Rankings | Permalink

March 14, 2022

Top 15 Law Faculties in scholarly impact (2016-2020) based on their faculty under age 60, 2021

This is a complement to the original Sisk study of scholarly impact for the period 2016-2020.  As many readers have noted, the scholarly impact of many schools depends on their older faculty.  Using the Sisk data, we looked at the ten most-cited faculty under age 60 in 2021, calculating the weighted score the same way as in the original study:  the mean score times two, plus the median.  Some schools--Chicago, Harvard, NYU, Berkeley, Michigan--performed comparably (or slightly better, or slightly worse) in this study as in the original study of all tenured faculty, regardless of age.  But other schools under-performed--for example, Yale and Stanford--suggesting that their "scholarly impact" performance depends heavily on an "old guard," as it were.  Some other schools outperformed their overall rank noticeably--UCLA and Virginia, for example--suggesting they are going to be likely recruitment targets for other law schools.

Time permitting, we'll try to expand this beyond the top 15.

Full results below the fold:

Continue reading

March 14, 2022 in Faculty News, Rankings | Permalink

March 07, 2022

"Web of Science" may record "impact" in some disciplines, but not necessarily *inter*disciplinary impact

Kevin Gerson, the Director of the law library at UCLA, writes:

You’ve noted several issues with the Vanderbilt scholarly impact study. I’d like to add to the list another issue with interdisciplinary scholarly impact studies.

A non-law article written by someone currently on a law faculty may not, by those facts alone, be widely regarded as “interdisciplinary” with law. Take, for example, this article: The 1000 Genomes Project Consortium, A map of human genome variation from population-scale sequencing, Nature 467, 1061–1073 (2010), https://doi.org/10.1038/nature09534. This article is purely computational genomics. It has had no impact on legal scholarship, yet one of its scientist/authors currently sits on a law school faculty. The article in Web of Science shows over 5300 citations, which would make that particular author one of the most cited legal scholars of all time if the article is included in measuring legal scholarly impact. The irony is that the other 400+ authors of that article would also be considered among the most impactful legal scholars of all time by virtue of that single article as long as they could get a joint appointment on a law faculty. That seems like an undesirable outcome.

March 7, 2022 in Faculty News, Rankings | Permalink

March 03, 2022

The Vanderbilt study of "scholarly impact" based on the Web of Science database

One thing this study is not is a study of "interdisciplinary impact," for three reasons:  (1) the Web of Science has better coverage of some disciplines than others; (2) citation practices vary dramatically across disciplines; and (3) the study only looked at articles published in Web of Science journals that were published and cited during 2012-2018 (books, the major form of scholarly impact in most humanities disciplines, counted for nought here).   The weird skew should be obvious from the fact that the two most-cited scholars (Lawrence Gostin at Georgetown and Susan M. Wolf at Minnesota) both work in health law and bioethics:  that's either because medical and medicine-related journals are wildly overrepresented in the database (which looks to be true), or because much more of the scholarly literature is based on articles that were cited by other articles during that period.  (Four of the top ten faculty on their list work in or around health law--indeed, health law and bioethics faculty are wildly over-represented in the top fifty.)

Professor Gostin, helpfully, has a Google Scholar page, and he does indeed have a lot of citations (more than 36,000!).  My colleague, the philosopher Martha Nussbaum, who publishes many books as well as articles, largely in philosophy and classics journals (which are not well-represented in Web of Science), does not have a Google Scholar page, but you can get a sense of her Google Scholar citations here.  Her first ten most cited works have over 60,000 citations:  that's just the first ten.   I haven't tallied all her citations, but I looked at several pages of her Google Scholar results, and I can say with some confidence that she has at least four times as many citations as Professor Gostin.  Yet Nussbaum is not even in the top 50 in the Vanderbilt study!

All citation studies have limitations due to their database.  But Web of Science is wholly inadequate for measuring interdisciplinary impact, as the stunning Nussbuam example reveals.   I would describe this as more a measure of impact for those who work in fields adjacent to the medical sciences, including health law, bioethics, psychology, etc.

March 3, 2022 in Faculty News, Rankings | Permalink

February 17, 2022

Citation counts vary by field, 2016-2020 edition

Here's an updated version of a post from last summer, now using the Sisk data for the period 2016-2020.

An important fact about interpreting citation data is that citation rates vary quite a bit by field.  One can see all the subject-specific citation lists for the latest Sisk study (2016-2020) here

Of the ten most-cited faculty in the U.S. in the last Sisk study, eight worked at least partly in constitutional law.  Indeed, constitutional law is the most high-citation field, although corporate, law & economics, criminal law & procedure, law & technology, and intellectual property also get cited a lot.  By contrast, tax, evidence, and health law, among others, are low-citation fields.   300 cites in a five-year period will get you into the top five in tax, but not anywhere close to the top 20 in constitutional law (maybe the top 50?).

Here's the fields ranked from highest to lowest citations based on the sum of the cites for the scholar ranked first, fifth and tenth in each area (those totals follow in parentheses).

1.  Constitutional Law (4,880 citations)

2.  Law & Economics (3,350 citations)

3.  Intellectual Property (2,900 citations)

4.  Corporate Law & Securities Regulation (2,440 citations)

5.  Criminal Law & Procedure (2,290 citations)

6.  Law & Technology (2,040 citations)

7.  Law & Social Science (1,980 citations)

8.  Administrative and/or Environmental Law (1,910 citations)

9.  International Law & Security (1,900 citations)

10. Critical Theories of Law (1,870 citations)

Continue reading

February 17, 2022 in Rankings | Permalink