August 29, 2018

Correcting for the problem of multi-author articles cited as "John Smith et al." in citation studies

Ted Sichelman (San Diego) contacted me with a proposed solution to the problem of undercounting multi-author articles in citation studies, a problem that washes out at the school level, but not necessarily at the level of individual authors, as noted before.  Prof. Sichelman here explains what he did:

Using the HeinOnline citation data, which does not suffer from multi-author limitations, I examined every 3+-author article with a large number (70+) of all-time citations in HeinOnline (as of late 2016), because articles with fewer citations are very unlikely to have any sizable effect on the most-cited lists (which I confirmed for a medium-sized sample set). For each second and later-listed author of these highly cited articles, I estimated a high number of "missing" citation counts in Westlaw's "law review & journals" database for the period 2013-17. Then, using this high-estimate missing count, Brian examined the raw data from Sisk et al. to determine if any author in the Hein-generated list might make a most-cited list (who didn't originally) or might substantially move up a list in ranking. Next, for each of the authors Brian flagged, I searched Westlaw to determine the missing (Westlaw) citation count (using the Sisk et al. methodology) for the period 2013-17 in two phases (providing information to Brian in each phase), generating accurate counts of missing citations for flagged authors. For completeness, these final counts included missing Westlaw citations not only for highly cited articles, but all articles in HeinOnline with more than 10 citations (as of late 2016) published since 1995 (other than for Mark Lemley and Cass Sunstein, because the additional citations for their less highly cited articles would not materially increase their cite counts). If you have any questions on the methodology, please feel free to email me (tsichelman@sandiego.edu).

I am grateful to Prof. Sichelman for undertaking this and sharing the results with me.  I will be updating some earlier rankings (no dramatic differences, but some slight ones) and incorporating this data into the specialty area citation rankings to come.


August 29, 2018 in Faculty News, Rankings | Permalink

August 28, 2018

Which law school's alumni are in the first FAR?

This sometimes includes LLM and SJDs, as well as laterals:

1.  Harvard University (28)

2.  New York University (25)

3.  Yale University (20)

4.  Columbia University (15)

5.  Stanford University (12)

5.  University of Michigan (12)

7.  University of California, Berkeley (11)

8.  University of Chicago (9)

9.  University of California, Los Angeles (8)

9.  University of Pennsylvania (8)

11. University of Texas, Austin (7)

12. University of Virginia (6)

13.  Northwestern University (5)

14. Cornell University (3)

14.  Duke University (3)

14. University of Minnesota (3)

Apart from NYU, which has far too many candidates on the market, these numbers look about right given past placement performance.


August 28, 2018 in Advice for Academic Job Seekers, Faculty News, Rankings | Permalink

In a USNews.com world, don't confuse citation counts with quality

I assume this is obvious, but just in case let me say it:   citation counts have a very imperfect correlation with quality.  But in a world where law faculties are ranked by Bob Morse, an ignorant non-academic looking to make a living, we need alternative metrics that reflect what we in the legal academy actually do.  There are many first-rate scholars who are as good as any of those on the various lists I have been and will be posting but who didn't happen to make them; off the top of my head:  in law & philosophy, Mark Greenberg (UCLA) and Stephen Perry (Penn); in law & economics, Eric Talley (Columbia) and Abraham Wickelgren (Texas); in legal history, Risa Goluboff (Virginia) and Sally Gordon (Penn); in empirical legal studies, Anup Malani (Chicago) and Ed Morrison (Columbia); in administrative law, Anne O'Connell (Stanford) and Ed Rubin (Vanderbilt); and many others.

 


August 28, 2018 in Faculty News, Rankings | Permalink

August 24, 2018

10 Most-Cited Legal History Scholars in the U.S. for the period 2013-2017

Based on the latest Sisk data, here are the ten most-cited legal historians in the U.S. (teaching in law schools) for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then).  Numbers are rounded to the nearest five.    Faculty for whom 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area."    

Rank

Name

School

Citations

Age in 2018

1

Lawrence Friedman

Stanford University

  990

88

2

Michael Klarman

Harvard University

  800

59

3

G. Edward White

University of Virginia

  535

77

4

James Whitman

Yale University

  425

61

5

Phillip Hamburger

Columbia University

  395

61

6

Stuart Banner

University of California, Los Angeles

  345

55

7

William Nelson

New York University

  315

78

8

John Witt

Yale University

  275

46

9

Edward A. Purcell, Jr.

New York Law School

  245

77

10

William E. Forbath

University of Texas, Austin

  240

66

 

Christopher Tomlins

University of California, Berkeley

  240

65

 

Other highly-cited scholars who work partly in this area

     
 

Reva Siegel

Yale University

1340

62

 

Herbert Hovenkamp

University of Pennsylvania

  985

70

 

Robert W. Gordon

Stanford University

  365

77

 

David Bernstein

George Mason University

  420

51

 


August 24, 2018 in Faculty News, Rankings | Permalink

August 23, 2018

10 Most-Cited Law & Philosophy Scholars in the U.S. for the period 2013-2017

Based on the latest Sisk data, here are the ten most-cited law & philosophy faculty in the U.S. for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then).  Numbers are rounded to the nearest five.    Faculty for whom 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area." 

Rank

Name

School

Citations

Age in 2018

1

Jeremy Waldron

New York University

1120

65

2

Martha Nussbaum

University of Chicago

  930

71

3

Joseph Raz

Columbia University (part-time)

  705

79

4

Michael S. Moore

University of Illinois

  470

75

5

Brian Leiter

University of Chicago

  460*

55

6

John Finnis

University of Notre Dame

  350

78

7

Scott Shapiro

Yale University

  300

52

8

Seana Shiffrin

University of California, Los Angeles

  280

49

9

Brian Bix

University of Minnesota

  210

56

10

Andrei Marmor

Cornell University

  205

59

   

Scholars who work partly in this area

   
 

Frederick Schauer

University of Virginia

1530

72

 

Lawrence Solum

Georgetown University

  845

64

 

Larry Alexander

University of San Diego

  680

75

 

David Luban

Georgetown University

  640

69

 

Kent Greenawalt

Columbia University

  580

82

* Raw count adjusted downward (based on a sample of 100 hits) by 17% (to arrive at 460) to reflect percentage of citations to my blogs unrelated to my law & philosophy work (a small number were related, most were about legal education).


August 23, 2018 in Faculty News, Rankings | Permalink

Westlaw searches: misspellings, multi-author articles and other problems

Greg Sisk gave me permission to share his response to an inquiry (on which I was cc'd) about some deficiencies of the Westlaw database and the searches Prof. Sisk and colleagues perform.  One difficulty that has come up is common misspellings of names; another is that in multi-author articles or books, sometimes (not all the time) only the first author is listed.   Here is what Prof. Sisk had to say, which seems to me sensible:

Your email raises an issue that we’ve seen and thought about every time we do this.  And it affects me as well, as I’ve conducted empirical research that has left me at the end of a list of authors on a piece.

Any methodology has limitations, which we've always forthrightly acknowledged.  The strength of Westlaw is also sometimes a weakness – that it is quite literal.  Being a literal search engine means that if a name is missing in a citation, then a Westlaw search simply will not uncover it.

When it comes to methodology, we have to consider what is practical and possible in a large-scale study involving thousands of tenured faculty members at a hundred law schools and how to implement that in a manner consistent across-the-board.  When we are looking a literally hundreds of thousands of citations for thousands of law professors, we have to rely mostly on a mechanical counting method.

Your email illustrates why we’re unable to integrate a resolution to the et al. issue into our methodology.  To do so consistently across the board, we’d first have to know each individual professor that is affected by this (which of course wouldn’t be apparent in the Westlaw database but would require outside information), and then run not merely one alternative search but potentially multiple alternatives for each type of “et al.” citation.  Within each of those searches, we’d then have to eyeball each of the citations to determine whether it is a correct hit and is duplicative of results from another alternative, and then calculate the right formula for coming up with a final count.  And we’d have to replicate the process across 3,378 law professors.

Importantly, our primary objective is comparison of law faculties, and this issue is not isolated to a particular law school’s faculty.  I've run some test searches in the past -- admittedly on an ad hoc basis and not thoroughly empirical in nature -- and it appears that this problem is vanishingly small when looking at the collective impact of a law school's faculty, which is the central feature of the Leiter Scholarly Impact ranking.  In other words, given that this phenomenon exists at any school with productive faculty, it washes out across the comparison of one faculty to another.  Indeed, as a rough calculation, citation counts for the typical school would have to be under-stated by a few hundred before it likely would affect a school’s overall ranking.

By contrast, for a dean conducting an annual evaluation, it would be quite right for these individual re-calculations to be made to come up with a better count.  Indeed, as we’ve noted, for individual evaluation, one might consult other databases, such as Google Scholar which allows for setting up a profile that, on a case-by-case basis, pulls these citations into one measure.  While that’s not practical for a large-scale study like ours, it may be indispensable for an individual evaluation.

And, because this affects me as well, it is one more reason that I have a policy of insisting with law reviews that citations in my articles include the names of all authors, at least out to three (and sometimes I’ve been able to insist that it go out to four).

Now this is probably far more than you wanted to know.  But I hope it helps explain things and at least shows that we really do take methodology matters seriously and try to think them through.

Of course, in the lists of high-impact scholars in particular areas, this may matter more, though whether it would have significant effects on the results (as opposed to just affecting one or two ordinal placements, which are meaningless anyway) is not clear.

UPDATE:  Ted Sichelman (San Diego) sent me an example, and its effect was to move the author from 4th to 3rd in the law & social science category.  That strikes me as quite minor, but I did make the change.


August 23, 2018 in Faculty News, Rankings | Permalink

August 22, 2018

15 Most-Cited Law & Economics (incl. behavioral L&E) faculty in the U.S. for the period 2013-2017

Based on the latest Sisk data, here are the fifteen most-cited law & economics scholars (including behavioral law & economics) in the U.S. for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then). Typically these scholars deploy economic analysis across multiple legal domains.  Numbers are rounded to the nearest five.    Faculty for whom 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area."   

Rank

Name

School

Citations

Age in 2018

1

Eric Posner

University of Chicago

2340

53

2

Steven Shavell

Harvard University

1245

72

3

Ian Ayres

Yale University

1170

59

4

Louis Kaplow

Harvard University

1080

62

5

Robert Cooter

University of California, Berkeley

  840

73

6

Einer Elhauge

Harvard University

  645

57

7

Russell Korobkin

University of California, Los Angeles

  610

50

8

Christine Jolls

Yale University

  560

51

9

A. Mitchell Polinsky

Stanford University

  485

71

10

George Priest

Yale University

  480

65

11

Michael Abramowicz

George Washington University

  470

46

12

Saul Levmore

University of Chicago

  415

65

13

W. Kip Viscusi

Vanderbilt University

  410

68

14

Lewis Kornhauser

New York University

  405

68

15

Douglas Ginsburg

George Mason University

  400

72

   

Other highly-cited scholars who work partly in this area

   
 

Cass Sunstein

Harvard University

4955

64

 

Mark Lemley

Stanford University

2200

52

 

Richard Epstein

New York University, University of Chicago

2165

75

 

Lucian Bebchuk

Harvard University

  985

63

 

Robert Scott

Columbia University

  890

74

 


August 22, 2018 in Faculty News, Rankings | Permalink

10 Most-Cited Election Law Scholars in the U.S. for the period 2013-2017

 Based on the latest Sisk data, here are the ten most-cited election law scholars in the U.S. for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then). Numbers are rounded to the nearest five.    Faculty for whom roughly 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area."   (The 75% rule was a bit harder to apply here, since some of these scholars write influentially in other fields; but it's fair to say that each of these scholars may be best-known for the election law work.)

Rank

Name

School

Citations

Age in 2018

1               

Samuel Issacharoff

New York University

1000

64

2

Richard Pildes

New York University

895

61

3

Richard Hasen

University of California, Irvine

740

54

4

Pamela Karlan

Stanford University

670

59

5

Heather Gerken

Yale University

650

49

6

Richard Briffault

Columbia University

570

64

7

Nathaniel Persily

Stanford University

390

48

8

James Gardner

University at Buffalo, State University of New York

240

59

9

Michael Kang

Northwestern University

235

45

10

Daniel Tokaji

Ohio State University

225

51

 

Other highly cited scholars who work partly in this area

     
 

Adam Cox

New York University

425

44

 


August 22, 2018 in Faculty News, Rankings | Permalink

August 21, 2018

20 Most-Cited Constitutional Law Scholars in the U.S. for the period 2013-2017 (CORRECTED AND UPDATED 8/21)

Based on the latest Sisk data, here are the twenty most-cited constitutional law professors in the U.S. for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then).  This time around I've defined the "constitutional law" category a bit more carefully and also checked more carefully the percentages of citations that appear to be in the field for some highly-cited scholars; I will also be expanding the other public law coverage to reflect contributions that belong more clearly to, e.g., administrative law, legislation etc. (I will also be adding a separate ranking for election law).  Numbers are rounded to the nearest five.    Faculty for whom 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area."    

Rank

Name

School

Citations

Age in 2018

1

Erwin Chemerinsky

University of California, Berkeley

2590

65

2

Akhil Amar

Yale University

1600

60

3

Mark Tushnet

Harvard University

1590

73

4

Jack Balkin

Yale University

1580

61

5

Laurence Tribe

Harvard University

1480

77

6

Bruce Ackerman

Yale University

1445

75

7

Richard Fallon

Harvard University

1365

66

8

Reva Siegel

Yale University

1340

62

9

Robet Post

Yale University

1280

71

10

Eugene Volokh

University of California, Los Angeles

1205*

50

11

Michael McConnell

Stanford University

1170

63

12

Randy Barnett

Georgetown University

1055

66

13

Michael Dorf

Cornell University

1045

54

14

Martin Redish

Northwestern University

  975

73

15

Sanford Levinson

University of Texas, Austin

  935

77

16

Barry Friedman

New York University

  845

60

 

Lawrence Solum

Georgetown University

  845

64

18

David A. Strauss

University of Chicago

  820

66

19

Steven Calabresi

Northwestern University

  810

60

20

Douglas Laycock

University of Virginia

  750

70

   

Other highly-cited scholars who work partly in constitutional and public law

   
 

Cass Sunstein

Harvard University

4955

64

 

Richard Epstein

New York University, University of Chicago

2165

75

 

William Eskridge, Jr.

Yale University

2160

67

 

Frederick Schauer

University of Virginia

1530

72

 

Adrian Vermeule

Harvard University

1385

50

 

Daniel Farber

University of California, Berkeley

1365

68

*Adjusted downwards by 5% based on a sample (to arrive at 1205) to reflect cites to blog posts unrelated to his scholarship (many blog posts were in fact related, those were not excluded).


August 21, 2018 in Faculty News, Rankings | Permalink

25 Most-Cited Public Law Scholars (excluding Constitutional) in the U.S. for the period 2013-2017 (UPDATED 8/31)

 Based on the latest Sisk data, here are the twenty-five most-cited public law professors in the U.S. for the period 2013-2017 (inclusive) (remember that the data was collected in late May of 2018, and that the pre-2018 database did expand a bit since then). There isn't a fine line bettwen public law and constitutional law, obviously, but for purposes here "public law" includes administrative law, environmental law, legislation and legislative process, statutory interpretation, and regulatory law more generally (e.g., food and drug regulation, or telecommunications).  (Election law will be ranked separately.)  Given the large number of fields, I've listed 25 highly-cited scholars; several work in more than one of these sub-fields.  Numbers are rounded to the nearest five.    Faculty for whom 75% or more of their citations (based on a sample) are in this area are listed; others with less than 75% of their citations in this field (but still a plurality) are listed in the category of "other highly cited scholars who work partly in this area."     

Rank

Name

School

Citations

Age in 2018

1

John F. Manning

Harvard University

945

57

2

Jody Freeman

Harvard University

800

54

3

Richard J. Pierce, Jr.

George Washington University

785

75

4

Richard Stewart

New York University

770

78

 5

Richard Revesz

New York University

740

60

6

Gary Lawson

Boston University

680

60

7

J.B. Ruhl

Vanderbilt University

625

60

8

Jonathan Adler

Case Western Reserve University

620

49

9

Lisa Bressman

Vanderbilt University

615

52

 

Gillian Metzger

Columbia University

615

53

11

Abbe Gluck

Yale University

580

43

12

Matthew Stephenson

Harvard University

575

44

13

Richard Lazarus

Harvard University

560

64

14

Thomas O. McGarity

University of Texas, Austin

500

69

15

Jim Rossi

Vanderbilt University

480

53

16

Robin Kundis Craig

University of Utah

450

54

17

Robert Glicksman

George Washington University

440

66

 

Sidney Shapiro

Wake Forest University

440

71

19

Cary Coglianese

University of Pennsylvania

430

54

20

Orly Lobel

University of San Diego

420

45

21

James Salzman

University of California, Los Angeles (part-time)

395

54

22

Douglas Kysar

Yale University

390

45

 

Mark Seidenfeld

Florida State University

390

64

 

Christopher Yoo

University of Pennsylvania

390

54

25

Jacob Gersen

Harvard University

375

45

   

Other highly-cited scholars who work partly in these public law areas

   
 

Cass Sunstein

Harvard University

4955

64

 

William Eskridge

Yale University

2160

67

 

Thomas Merrill

Columbia University

1595

69

 

Adrian Vermeule

Harvard University

1385

50

 

Daniel Farber

University of California, Berkeley

1365

64

 


August 21, 2018 in Faculty News, Rankings | Permalink