August 28, 2018

In a USNews.com world, don't confuse citation counts with quality

I assume this is obvious, but just in case let me say it:   citation counts have a very imperfect correlation with quality.  But in a world where law faculties are ranked by Bob Morse, an ignorant non-academic looking to make a living, we need alternative metrics that reflect what we in the legal academy actually do.  There are many first-rate scholars who are as good as any of those on the various lists I have been and will be posting but who didn't happen to make them; off the top of my head:  in law & philosophy, Mark Greenberg (UCLA) and Stephen Perry (Penn); in law & economics, Eric Talley (Columbia) and Abraham Wickelgren (Texas); in legal history, Risa Goluboff (Virginia) and Sally Gordon (Penn); in empirical legal studies, Anup Malani (Chicago) and Ed Morrison (Columbia); in administrative law, Anne O'Connell (Stanford) and Ed Rubin (Vanderbilt); and many others.

 


August 28, 2018 in Faculty News, Rankings | Permalink

August 24, 2018

10 Most-Cited Legal History Scholars in the U.S. for the period 2013-2017

Based on the latest Sisk data, here are the ten most-cited legal historians in the U.S. (teaching in law schools) for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then).  Numbers are rounded to the nearest five.    Faculty for whom 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area."    

Rank

Name

School

Citations

Age in 2018

1

Lawrence Friedman

Stanford University

  990

88

2

Michael Klarman

Harvard University

  800

59

3

G. Edward White

University of Virginia

  535

77

4

James Whitman

Yale University

  425

61

5

Phillip Hamburger

Columbia University

  395

61

6

Stuart Banner

University of California, Los Angeles

  345

55

7

William Nelson

New York University

  315

78

8

John Witt

Yale University

  275

46

9

Edward A. Purcell, Jr.

New York Law School

  245

77

10

William E. Forbath

University of Texas, Austin

  240

66

 

Christopher Tomlins

University of California, Berkeley

  240

65

 

Other highly-cited scholars who work partly in this area

     
 

Reva Siegel

Yale University

1340

62

 

Herbert Hovenkamp

University of Pennsylvania

  985

70

 

Robert W. Gordon

Stanford University

  365

77

 

David Bernstein

George Mason University

  420

51

 


August 24, 2018 in Faculty News, Rankings | Permalink

August 23, 2018

10 Most-Cited Law & Philosophy Scholars in the U.S. for the period 2013-2017

Based on the latest Sisk data, here are the ten most-cited law & philosophy faculty in the U.S. for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then).  Numbers are rounded to the nearest five.    Faculty for whom 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area." 

Rank

Name

School

Citations

Age in 2018

1

Jeremy Waldron

New York University

1120

65

2

Martha Nussbaum

University of Chicago

  930

71

3

Joseph Raz

Columbia University (part-time)

  705

79

4

Michael S. Moore

University of Illinois

  470

75

5

Brian Leiter

University of Chicago

  460*

55

6

John Finnis

University of Notre Dame

  350

78

7

Scott Shapiro

Yale University

  300

52

8

Seana Shiffrin

University of California, Los Angeles

  280

49

9

Brian Bix

University of Minnesota

  210

56

10

Andrei Marmor

Cornell University

  205

59

   

Scholars who work partly in this area

   
 

Frederick Schauer

University of Virginia

1530

72

 

Lawrence Solum

Georgetown University

  845

64

 

Larry Alexander

University of San Diego

  680

75

 

David Luban

Georgetown University

  640

69

 

Kent Greenawalt

Columbia University

  580

82

* Raw count adjusted downward (based on a sample of 100 hits) by 17% (to arrive at 460) to reflect percentage of citations to my blogs unrelated to my law & philosophy work (a small number were related, most were about legal education).


August 23, 2018 in Faculty News, Rankings | Permalink

Westlaw searches: misspellings, multi-author articles and other problems

Greg Sisk gave me permission to share his response to an inquiry (on which I was cc'd) about some deficiencies of the Westlaw database and the searches Prof. Sisk and colleagues perform.  One difficulty that has come up is common misspellings of names; another is that in multi-author articles or books, sometimes (not all the time) only the first author is listed.   Here is what Prof. Sisk had to say, which seems to me sensible:

Your email raises an issue that we’ve seen and thought about every time we do this.  And it affects me as well, as I’ve conducted empirical research that has left me at the end of a list of authors on a piece.

Any methodology has limitations, which we've always forthrightly acknowledged.  The strength of Westlaw is also sometimes a weakness – that it is quite literal.  Being a literal search engine means that if a name is missing in a citation, then a Westlaw search simply will not uncover it.

When it comes to methodology, we have to consider what is practical and possible in a large-scale study involving thousands of tenured faculty members at a hundred law schools and how to implement that in a manner consistent across-the-board.  When we are looking a literally hundreds of thousands of citations for thousands of law professors, we have to rely mostly on a mechanical counting method.

Your email illustrates why we’re unable to integrate a resolution to the et al. issue into our methodology.  To do so consistently across the board, we’d first have to know each individual professor that is affected by this (which of course wouldn’t be apparent in the Westlaw database but would require outside information), and then run not merely one alternative search but potentially multiple alternatives for each type of “et al.” citation.  Within each of those searches, we’d then have to eyeball each of the citations to determine whether it is a correct hit and is duplicative of results from another alternative, and then calculate the right formula for coming up with a final count.  And we’d have to replicate the process across 3,378 law professors.

Importantly, our primary objective is comparison of law faculties, and this issue is not isolated to a particular law school’s faculty.  I've run some test searches in the past -- admittedly on an ad hoc basis and not thoroughly empirical in nature -- and it appears that this problem is vanishingly small when looking at the collective impact of a law school's faculty, which is the central feature of the Leiter Scholarly Impact ranking.  In other words, given that this phenomenon exists at any school with productive faculty, it washes out across the comparison of one faculty to another.  Indeed, as a rough calculation, citation counts for the typical school would have to be under-stated by a few hundred before it likely would affect a school’s overall ranking.

By contrast, for a dean conducting an annual evaluation, it would be quite right for these individual re-calculations to be made to come up with a better count.  Indeed, as we’ve noted, for individual evaluation, one might consult other databases, such as Google Scholar which allows for setting up a profile that, on a case-by-case basis, pulls these citations into one measure.  While that’s not practical for a large-scale study like ours, it may be indispensable for an individual evaluation.

And, because this affects me as well, it is one more reason that I have a policy of insisting with law reviews that citations in my articles include the names of all authors, at least out to three (and sometimes I’ve been able to insist that it go out to four).

Now this is probably far more than you wanted to know.  But I hope it helps explain things and at least shows that we really do take methodology matters seriously and try to think them through.

Of course, in the lists of high-impact scholars in particular areas, this may matter more, though whether it would have significant effects on the results (as opposed to just affecting one or two ordinal placements, which are meaningless anyway) is not clear.

UPDATE:  Ted Sichelman (San Diego) sent me an example, and its effect was to move the author from 4th to 3rd in the law & social science category.  That strikes me as quite minor, but I did make the change.


August 23, 2018 in Faculty News, Rankings | Permalink

August 22, 2018

15 Most-Cited Law & Economics (incl. behavioral L&E) faculty in the U.S. for the period 2013-2017

Based on the latest Sisk data, here are the fifteen most-cited law & economics scholars (including behavioral law & economics) in the U.S. for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then). Typically these scholars deploy economic analysis across multiple legal domains.  Numbers are rounded to the nearest five.    Faculty for whom 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area."   

Rank

Name

School

Citations

Age in 2018

1

Eric Posner

University of Chicago

2340

53

2

Steven Shavell

Harvard University

1245

72

3

Ian Ayres

Yale University

1170

59

4

Louis Kaplow

Harvard University

1080

62

5

Robert Cooter

University of California, Berkeley

  840

73

6

Einer Elhauge

Harvard University

  645

57

7

Russell Korobkin

University of California, Los Angeles

  610

50

8

Christine Jolls

Yale University

  560

51

9

A. Mitchell Polinsky

Stanford University

  485

71

10

George Priest

Yale University

  480

65

11

Michael Abramowicz

George Washington University

  470

46

12

Saul Levmore

University of Chicago

  415

65

13

W. Kip Viscusi

Vanderbilt University

  410

68

14

Lewis Kornhauser

New York University

  405

68

15

Douglas Ginsburg

George Mason University

  400

72

   

Other highly-cited scholars who work partly in this area

   
 

Cass Sunstein

Harvard University

4955

64

 

Mark Lemley

Stanford University

2200

52

 

Richard Epstein

New York University, University of Chicago

2165

75

 

Lucian Bebchuk

Harvard University

  985

63

 

Robert Scott

Columbia University

  890

74

 


August 22, 2018 in Faculty News, Rankings | Permalink

Westlaw JLR vs. Google Scholar

According to the Sisk data using the Westlaw JLR ("Journals and Law Reviews") database, 553 articles mentioned me between 2013 and 2017; according to Google Scholar, I had over 2,000 citations during this same time period (and over 500 in 2017 alone).   This isn't atypical for many law professors, and certainly is quite typical for those doing interdisciplinary scholarship of various kinds.  Why the differences?  There are two main reasons:

First, Google Scholar tracks references to individual works, so an article that mentions, say, five different articles by a scholar produces 5 citations in Google Scholar, but just 1 via the Westlaw method, which tracks names, not individual works.

Second, Google Scholar tracks not simply law journals, but journals in all fields, as well as books, dissertations, and draft papers that are on-line.  That is the most important difference.

Should we prefer Google Scholar as an "impact" measure in law?  Put aside the logistical problem that most law faculty do not have Google Scholar pages, though that is a serious obstacle.  Google Scholar may give a better picture of general scholarly impact based on citations, but it does less well in terms of impact on law and legal scholarship precisely because it is so broad in its reach, capturing citations in, e.g., philosophy journals, books and dissertations.   (Do the 600+ citations to my 2002 book on Nietzsche's moral philosophy make me a more impactful legal philosopher?  Not for most legal philosophers, I suspect (but they don't know what they're missing)!)  Westlaw is probably a better snapshot of impact on other legal academics.

Still, if more law professors would set up Google Scholar pages, some enterprising person could examine the differences more carefully with Westlaw's database of law journals and reviews.


August 22, 2018 in Rankings | Permalink

10 Most-Cited Election Law Scholars in the U.S. for the period 2013-2017

 Based on the latest Sisk data, here are the ten most-cited election law scholars in the U.S. for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then). Numbers are rounded to the nearest five.    Faculty for whom roughly 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area."   (The 75% rule was a bit harder to apply here, since some of these scholars write influentially in other fields; but it's fair to say that each of these scholars may be best-known for the election law work.)

Rank

Name

School

Citations

Age in 2018

1               

Samuel Issacharoff

New York University

1000

64

2

Richard Pildes

New York University

895

61

3

Richard Hasen

University of California, Irvine

740

54

4

Pamela Karlan

Stanford University

670

59

5

Heather Gerken

Yale University

650

49

6

Richard Briffault

Columbia University

570

64

7

Nathaniel Persily

Stanford University

390

48

8

James Gardner

University at Buffalo, State University of New York

240

59

9

Michael Kang

Northwestern University

235

45

10

Daniel Tokaji

Ohio State University

225

51

 

Other highly cited scholars who work partly in this area

     
 

Adam Cox

New York University

425

44

 


August 22, 2018 in Faculty News, Rankings | Permalink

August 21, 2018

Specialty rankings coming up

Coming up over the next couple of weeks:  election law; international law; intellectual property & cyberlaw; tax; law & economics; law & philosophy; law & social science; legal history; civil procedure; antitrust; torts; property; critical theories of law (including feminist jurisprudence and critical race theory); and perhaps others.  Thanks for feedback and corrections on the rankings posted so far.


August 21, 2018 in Rankings | Permalink

20 Most-Cited Constitutional Law Scholars in the U.S. for the period 2013-2017 (CORRECTED AND UPDATED 8/21)

Based on the latest Sisk data, here are the twenty most-cited constitutional law professors in the U.S. for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then).  This time around I've defined the "constitutional law" category a bit more carefully and also checked more carefully the percentages of citations that appear to be in the field for some highly-cited scholars; I will also be expanding the other public law coverage to reflect contributions that belong more clearly to, e.g., administrative law, legislation etc. (I will also be adding a separate ranking for election law).  Numbers are rounded to the nearest five.    Faculty for whom 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area."    

Rank

Name

School

Citations

Age in 2018

1

Erwin Chemerinsky

University of California, Berkeley

2590

65

2

Akhil Amar

Yale University

1600

60

3

Mark Tushnet

Harvard University

1590

73

4

Jack Balkin

Yale University

1580

61

5

Laurence Tribe

Harvard University

1480

77

6

Bruce Ackerman

Yale University

1445

75

7

Richard Fallon

Harvard University

1365

66

8

Reva Siegel

Yale University

1340

62

9

Robet Post

Yale University

1280

71

10

Eugene Volokh

University of California, Los Angeles

1205*

50

11

Michael McConnell

Stanford University

1170

63

12

Randy Barnett

Georgetown University

1055

66

13

Michael Dorf

Cornell University

1045

54

14

Martin Redish

Northwestern University

  975

73

15

Sanford Levinson

University of Texas, Austin

  935

77

16

Barry Friedman

New York University

  845

60

 

Lawrence Solum

Georgetown University

  845

64

18

David A. Strauss

University of Chicago

  820

66

19

Steven Calabresi

Northwestern University

  810

60

20

Douglas Laycock

University of Virginia

  750

70

   

Other highly-cited scholars who work partly in constitutional and public law

   
 

Cass Sunstein

Harvard University

4955

64

 

Richard Epstein

New York University, University of Chicago

2165

75

 

William Eskridge, Jr.

Yale University

2160

67

 

Frederick Schauer

University of Virginia

1530

72

 

Adrian Vermeule

Harvard University

1385

50

 

Daniel Farber

University of California, Berkeley

1365

68

*Adjusted downwards by 5% based on a sample (to arrive at 1205) to reflect cites to blog posts unrelated to his scholarship (many blog posts were in fact related, those were not excluded).


August 21, 2018 in Faculty News, Rankings | Permalink

25 Most-Cited Public Law Scholars (excluding Constitutional) in the U.S. for the period 2013-2017 (UPDATED 8/31)

 Based on the latest Sisk data, here are the twenty-five most-cited public law professors in the U.S. for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then). There isn't a fine line bettwen public law and constitutional law, obviously, but for purposes here "public law" includes administrative law, environmental law, legislation and legislative process, statutory interpretation, and regulatory law more generally (e.g., food and drug regulation, or telecommunications).  (Election law will be ranked separately.)  Given the large number of fields, I've listed 25 highly-cited scholars; several work in more than one of these sub-fields.  Numbers are rounded to the nearest five.    Faculty for whom 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area."     

Rank

Name

School

Citations

Age in 2018

1

John F. Manning

Harvard University

945

57

2

Jody Freeman

Harvard University

800

54

3

Richard J. Pierce, Jr.

George Washington University

785

75

4

Richard Stewart

New York University

770

78

 5

Richard Revesz

New York University

740

60

6

Gary Lawson

Boston University

680

60

7

J.B. Ruhl

Vanderbilt University

625

60

8

Jonathan Adler

Case Western Reserve University

620

49

9

Lisa Bressman

Vanderbilt University

615

52

 

Gillian Metzger

Columbia University

615

53

11

Abbe Gluck

Yale University

580

43

12

Matthew Stephenson

Harvard University

575

44

13

Richard Lazarus

Harvard University

560

64

14

Thomas O. McGarity

University of Texas, Austin

500

69

15

Jim Rossi

Vanderbilt University

480

53

16

Robin Kundis Craig

University of Utah

450

54

17

Robert Glicksman

George Washington University

440

66

 

Sidney Shapiro

Wake Forest University

440

71

19

Cary Coglianese

University of Pennsylvania

430

54

20

Orly Lobel

University of San Diego

420

45

21

James Salzman

University of California, Los Angeles (part-time)

395

54

22

Douglas Kysar

Yale University

390

45

 

Mark Seidenfeld

Florida State University

390

64

 

Christopher Yoo

University of Pennsylvania

390

54

25

Jacob Gersen

Harvard University

375

45

   

Other highly-cited scholars who work partly in these public law areas

   
 

Cass Sunstein

Harvard University

4955

64

 

William Eskridge

Yale University

2160

67

 

Thomas Merrill

Columbia University

1595

69

 

Adrian Vermeule

Harvard University

1385

50

 

Daniel Farber

University of California, Berkeley

1365

64

 


August 21, 2018 in Faculty News, Rankings | Permalink