Brian Leiter's Law School Reports

Brian Leiter
University of Chicago Law School

A Member of the Law Professor Blogs Network

Wednesday, February 20, 2013

Update on the lawsuits against law schools

They've mostly been failing, except in California, where they're still going forward.

February 20, 2013 in Legal Profession, Of Academic Interest | Permalink

Paul Campos admits he doesn't "even [know] what it means" to think like a lawyer

This probably explains a lot.  Fortunately, Fred Schauer has recently written a book that could help him with his questions, like, "What does it mean to teach people to think like lawyers?  How is thinking like a lawyer different from ordinary thinking?" 

(Thanks to Nick Smith for the pointer.)

UPDATE:  A senior legal academic, who has been involved extensively with legal education reform, writes:  "Keep up the Campos bashing.  I think that some of the law school critics have done a good service.  Even when I don't agree with everything, it was necessary for legal educators to give up a bit of complacency.  I've never met Campos, but he is disgraceful."  It's hard to disagree with any of that, but I don't really plan to keep up the "bashing," since, as we saw a few weeks back, by Campos's own admission, there really isn't much content to his routine. 


February 20, 2013 in Jurisprudence, Law Professors Saying Dumb Things, Legal Humor, Of Academic Interest, Professional Advice | Permalink

Tuesday, February 19, 2013

Racial, ethnic and other "identity" diversity in article selection?

A reader alerted me a couple of weeks ago to the fact that Scholastica collects demographic data on authors, which apparently some journals are using in the selection process.  Before I got around to it, the issue has exploded on various blogs; Blog Emperor Caron has links to the various discussions.  Given that the law reviews had been moving to more peer (i.e., faculty) review of scholarship, this does seem a step backwards, but much will turn on what law reviews are really doing with this information.

February 19, 2013 in Of Academic Interest, Professional Advice | Permalink

Monday, February 18, 2013

Dworkin-Lite and Constitutional Theory

Cass Sunstein (Harvard) gives a nice re-statement of those aspects of Dworkin's views that made him such an appealing figure in modern constitutional theory.  Sunstein writes:

Consider a question about which people fiercely disagree: Does the U.S. Constitution require states to recognize same-sex marriages? In answering that question, judges have to deal with many precedents. For example, the Supreme Court has ruled that states can’t criminalize sexual acts between people of the same sex. The court has also forbidden states from banning racial intermarriages. At the same time, the court allows states to forbid polygamous marriages.

In resolving the same-sex marriage dispute, how can judges deal with such precedents? Here Dworkin introduced an arresting metaphor. Suppose that you are a participant in writing a chain novel. Others have written earlier chapters. Now it’s your turn. How shall you proceed?

Dworkin’s answer is that you have to engage in an act of interpretation. You can’t disregard what has come before. If your predecessors have started to write a romance, you can’t suddenly turn it into a work of science fiction without doing violence to what they have done. You owe a duty of fidelity to their work.

But your task is not mechanical. You have to fit the existing materials, and you have to justify them, by writing a new chapter that makes the emerging novel, taken as a whole, the best it can be.

Dworkin thinks that judging is a lot like that. Precedents are like the existing chapters, and a new case is an opportunity to produce a fresh one. Judges can’t just make the law up. But at least in hard cases, they can’t merely “follow the law,”because there isn’t anything to “follow.” What they have to do is produce a principle that both fits and justifies the existing legal materials. This is Dworkin’s conception of law as integrity.

This does, indeed, seem like one sensible recommendation about how judges should proceed when the law leaves a question unsettled (such that there is no law to follow), and nothing in the theory of law known as legal positivism actually takes issue with such a recommendation.  But this isn't quite Dworkin's conception of law as integrity, and it elides the quite radical character of his views about the nature of law.  

According to Dworkin, what the law is (not simply what a court should do in a hard case, but what the law in any given jurisdiction really is) is whatever follows from the best constructive interpretation (in roughly the fit-justification sense glossed by Sunstein) of the institutional history of the legal system, i.e., the prior legislative enactments, court decisions, and so on.  That means, among other thing, that some prior legislative enactments and prior court decisions might not really express the law, since they might not follow from the best constructive interpretation of the system.  Indeed, it means that no one may actually know what the real law is in the United States, or in any other jurisdiction, since no one may have yet figured out the best justification for the institutional history of the legal system.  A theory of law that entails that no one might actually know what the law is faces a serious, shall we say, reductio ad absurdum problem. 

The trademark Dworkinian move in his decades-long battle with legal positivism was always to run together questions about what the law is (on which he and positivists had opposing views) with the question how should courts decide cases (where positivists could often agree with Dworkin).  Here's how I put it in reviewing his Justice in Robes a few years ago:

Dworkin organizes his reply [to Hart's criticisms of Dworkin in the "Postscript" to The Concept of Law]around a hypothetical case involving "Mrs. Sorenson", who has been injured by a defective drug whose precise manufacturer she cannot identify because many companies produced the same product. Common-law courts responded to this kind of problem by inventing a new doctrine, "market-share liability," requiring manufacturers to pay damages for harm based on their "market share" of the dangerous product without requiring individualized proof by the plaintiff that defendant’s version of the product caused the injuries in question. Dworkin thinks Mrs. Sorenson was legally entitled to this remedy even before clever judges crafted the new rule, and—more to the point here—he thinks Hart was committed to denying Mrs. Sorenson was entitled to such a remedy, since the existing legal authorities did not explicitly establish it. "Hart and I hold opposite opinions about the same issue", says Dworkin.

But this is false, if the issue is, as it appears to be, the remedy that ought to be awarded Mrs. Sorenson. For Dworkin runs together the question "What is the law in this jurisdiction?" with the question "How ought a particular case be decided?" Positivists have always been clear that a judge’s legal duty to apply valid law can be overridden by moral or equitable considerations in any particular case, and Hart’s general theory ("the sources thesis") that "laws" are distinguished by their source—by their being enacted, for example, by a legislative body or figuring in the holding of a court—is simply silent on how the wronged Mrs. Sorenson should be treated. On Dworkin’s [theory], however, it seems every moral wrong must have a preordained legal remedy—even though no one knew the law required it!— so that forward-looking jurists who craft new legal rules in response to real-world problems are really only "discovering" a legal remedy that already existed in Dworkinian Heaven.


Sunstein's Dworkin--who is, I think, the Dworkin that has been influential in constitutional theory--is Dworkin-lite, though I intend nothing pejorative by that (think "coca cola lite"--still quite good, but not the "real thing").  It's not quite the real Dworkin, since it obscures from view the counter-intuitive commitments of his conception of the nature of law.  But it is the bit of Dworkin that seems most appealing, and helps make sense of one way in which constitutional lawyers and theorists argue.

ADDENDUM:  This obituary from The Telegraph is indicative of how Dworkin's own misreadings have influenced the reception of legal positivism; the obituary states:  "Hart held that the law is a set of rules based on customs and institutions and that legal deliberation need not take account of considerations of morality or fairness – the province of the legislature."  Hart held something like the first bit, but never wrote anything that even suggests "that legal deliberation need not take account of considerations of morality or fairness," let alone that those were "the province of the legislature."   It's really quite extraordinary that such basic features of Hart's views should be so consistently misrepresented, and I fear some of the blame for that does go to Dworkin.

ANOTHER:  Is Dworkin "the most important" legal philosopher of the last century, as Professor Sunstein suggests he might be?   I think the comments of John Gardner, who succeeded Dworkin as the Professor of Jurisprudence at Oxford, are closer to the mark:  "There are several contenders for the title of greatest philosopher of law of the late twentieth century. But nobody rivals Ronald Dworkin for the titles of most innovative and most provocative."  I assume Professor Gardner, in referring to the "late twentieth century," is also thinking of Joseph Raz and John Finnis.  If we take the long view of the whole last century, and in a global rather than American perspective, I think it would be hard to find any legal philosophers who rival H.L.A. Hart or Hans Kelsen for the depth and breadth of their jurisprudential corpus and influence.  


February 18, 2013 in Jurisprudence | Permalink

Friday, February 15, 2013

National Jurist Now Back-Pedalling on its Thomas Cooleyesque Law School Rankings

Since it's obvious--and should have been obvious prior to anyone pointing it out--that using Rate My Professors as part of a ranking of academic institutions is preposterous on its face, National Jurist is now doing a partial mea culpa, though so far, it hardly goes far enough.  Mr. Crittenden, the editor, actually had the audacity to do a video interview 48 hours ago in which he defended the use of Rate My Professors, stating, falsely, that studies had found it to be a valid measure.  There are no studies that validate the use of Rate My Professors the way NJ has used it.  None.  Zero.

A study at the University of Maine, involving more than 400 ratings by undergraduates, found that their scores on Rate My Professors generally matched internal Maine evaluations in two categories:  overall course evaluation and easiness/difficulty of the course.  But NJ used the results for the teaching quality categories, not these other categories.  The Maine study found no correlation with teaching quality, and they also found that the correlations that did exist in the other categories broke down outside the highest rated courses. 

Since law students barely use Rate My Professors, and since in some cases identified, the Rate My Professors score was based on non-law faculty and just a a few dozen responses (not 400!), it's clear that the data has no validity.  None.  It's worse than garbage. 

Mr. Crittenden even told an assistant dean here that if they dropped the fraudulent Rate My Professors data, Chicago would have gone from something like 50 to the top 5.   Since the data they used for Chicago included less than two dozen evaluations for actual law faculty, and twice as many evaluations for non-law faculty (including the lowest scores), this is a remarkable admission.   But this is almost surely just the tip of the iceberg!  As I noted earlier this week, Rate My Professors is driving all the variance among schools with very similar employment and other results.  (I learn from Mr. Crittenden's partial mea culpa that Princeton Review gives a very different portrait, but I know nothing about the methods by which PR compiles its data.)

Although National Jurist is now actually reviewing the "accuracy" of the Rate My Professors data, the question is why didn't they do that before publishing the ranking?  NJ says "we believe that the voice of students is essential."  But that's not in dispute.  The point is that Rate My Professors is not "the voice of students," and a magazine with any integrity, as opposed to an interest in generating hype, never would have utilized such an absurd source.  (I'm not even sure what it means to check the accuracy of Rate My Professors data:  anyone with Internet access on the planet earth can fill out a Rate My Professors survey, how could that be meaningful?)

As I indicated in an earllier post, I spoke with Mr. Crittenden just a few weeks ago about what would be required to do a sensible alternative to U.S. News.  He never let on that he and his staff had already concocted this nonsense ranking, and that he was going to use my quotes in an article accompanying this ranking, whose existence he had not disclosed to me.  Mr. Crittenden is clearly not an honest man or journalist.  He can only redeem his reputation by repudiating this whole fiasco.

NJ was always a bit of a joke, a magazine so unneeded that it is given away free in piles at every law school in the country (most copies end up in the trash).  But now they've done something perniciously stupid, which doesn't help students, and groundlessly defames dozens of law faculties.  I hope it marks the end of a pointless magazine.

ADDENDUM:  Professor Scott Bauries (Kentucky) writes:

I am enjoying your skewering of the obviously ridiculous effort of the National Jurist to create a new ranking system.  One further flaw with the use of Rate My Professors occurs to me that, to my knowledge, has not yet been reported (perhaps because it is obvious).  Other than by logging the IP address of the rater’s computer, Rate My Professors does not appear to ask its users for any personally identifying information or proof that a rater ever took a class from the professor rated.  This means that literally anyone who knows a professor’s name and institution can “rate” that professor. 

Therefore, if a school were of a mind to game this set of rankings, then it would be as easy as (1) pulling up the site; (2) searching for the school’s own professors; and (3) giving each one as many fraudulently high ratings, from as many unique IP addresses, as possible.  The site does ask for the course number and textbook, but these would be easy to communicate to false raters.  From the perspective of the unethical school, this sort of fraud would be perfect because it would be completely undetectable by outside observers (at least without a discovery request).  Given all of the efforts (both yours and those of the Law School Transparency folks and others) to eliminate opportunities to game the US News system, this flaw in the National Journal’s system should be particularly troubling. 

February 15, 2013 in Rankings, Student Advice | Permalink

"U.S. News and the Price of Legal Education"

More sensible observations from Dean Yellen of Loyola/Chicago.

February 15, 2013 in Legal Profession, Of Academic Interest, Rankings | Permalink

Thursday, February 14, 2013

In Memoriam: Ronald Dworkin (1931-2013)

More here.

February 14, 2013 in Memorial Notices | Permalink

Wednesday, February 13, 2013

In Memoriam: Daniel Meador

Dan Meador, the James Monroe Professor of Law Emeritus at the University of Virginia, passed away on Saturday.  He was 86.  In addition to teaching at UVa, Meador - who received his JD from the University of Alabama - served as Alabama's dean from 1966-70.

February 13, 2013 in Faculty News, Memorial Notices | Permalink

Tuesday, February 12, 2013

National Jurist in Competition to Displace Thomas Cooley Rankings as Biggest Joke in Legal Academia

Years ago, when Texas had the misfortune to be #1 in the Cooley rankings, the law school was asked by the public affairs department whether we wanted to produce a press release; the immediate answer was, "No, don't mention it, it's an embarrassment to be #1 in the Cooley rankings."  National Jurist has now replicated the Cooley feat, with a somewhat more baroque methodology that can only make Bob Morse and the U.S. News editors smile, since it makes their approach look like rocket science.  Like U.S. News, the National Jurist has a multitude of different factors, all inexplicably weighted (5% for bar pas rate and diversity, but 12.5% for the number of Super Lawyer alumni!), but some of which are independently interesting, but aggregated make no sense.

But the coup de grace is that 20% of the overall score is based on Rate My Professors, the notorious on-line rating site used mainly by undergraduates, and hardly at all by law students.  (In a remarkable display of editorial good judgment, Jack Crittenden, the editor, decided not to incorporate the "hotness" score, however.)

What kind of sample was this?  In the case of the University of Chicago Law listings on Rate My Professor, it consisted of 54 responses total for ten faculty over a period of six years--but only 23 responses for actual full-time law school faculty!  Indeed, Rate My Professor lists one person, Smigelskis, who has never even taught in the Law School here (he accounts for almost 20% of the responses, and also had the lowest scores).  (Actual law faculty, like me, didn't even appear, because Rate My Professors had me listed in the wrong unit!)  I've gotten e-mails from colleagues elsewhwere reporting similar anomalies.

In short, 20% of the overall score is fraudulent on its face.  And it's that 20% that explains all the variance.  Stanford, Harvard, Virginia, Chicago, Michigan, Yale all get A and A+ scores in all the employment categories (NLJ 200 partners, Super Lawyers, etc.), what differentiates them is the fraudulent Rate My Professors data.  This means the National Jurist has one advantage over the Cooley rankings:  its absurdity isn't quite as obvious.

I've talked to Jack Crittenden on occasion, and I even talked to him in the past about how to improve on what U.S. News tries to do.  He never mentioned that he was thinking of incorporating bogus and inaccurate pseudo-data in order to rank law schools.

I hope Mr. Crittenden will have the good sense to issue a retraction and apology for putting this misinformation into circulation.  It's the second time in recent months that they have put out misleading rankings.  Maybe this signals desperation, I don't know.

If readers catch any law schools publicizing their National Jurist ranking, please let me know.

UPDATE:  Shame on the University of Oklahoma and Texas Tech!  (Thanks to a reader who asked for anonymity.)

ANOTHER:  A colleague elsewhere writes:  "The methodology says that the RateMyProfessors score was only used if there were more than 40 ratings. Wash U is listed as “NA” in that column, but a perusal of that site shows 63 professors in the “law” department with over 200 ratings – far more than most schools. Haven’t checked to see if their number is accurate…."

ANOTHER READER points out that the National Jurist article falsely states that "Rate My Professors" has been found valid in "scientific" studies.  No citations are given, for good reason.   One study, at the Univeristy of Maine, focusing on undergraduate teaching, found a strong correlation only for overall course evaluation and easiness or difficulty of the course (not for teaching quality), and also found that the correlations were strongest for highly rated faculty, but fell apart thereafter.   There is no evidence--as in none--that Rate My Professors has any validity for measuring teaching at law schools, or for measuring the distribution of quality teaching at an institution of higher education. 

SHAME WATCH CONTINUES:  North Carolina joins the hall of shame. (UPDATE:  UNC seems to have removed the announcement--kudos to them for their good sense!)

MORE 'RATE MY PROFESSORS' VALIDITY INDICATORS:  Steven Freedman (Kansas) writes:  "If you search for Mickey Mouse on, you will find out he’s a Chemical Engineering professor at Ohio State.  I googled to make sure there wasn’t an actual Professor Mouse at Ohio State, but there seems to be no record of that."

ANOTHER:  The "Above the Law" blog calls the National Jurist ranking (not unfairly) "pure ridiculousness."

AND MORE ERRORS:  A colleague who examined this with some care writes:

I did some data analysis to try to figure out how they calculated the number they used for RMP. I did the top 5 schools (three of which didn’t have their numbers used) and the lowest score on the whole thing, which happens to be your school. Here’s the takeaway:

There are two possible numbers they could be using – the overall average, and the average of the Helpfulness and Clarity scores (which is what they say they did in the methodology). In all cases I went to the school page, selected the “law” department and included anyone with at least one rating.



NJ's score

Score used?

overall avg

Help+Clar avg































As you can see, in no case does my calculation exactly match up with the NJ number, but at least for 4 of the top five schools, it’s within a few hundredths of a point, and my two numbers are so close as to be statistically identical anyway. So it’s fair to say that I have the method pretty close to correct. But note that the difference for Virginia served to inflate their score from a 3.32 to a 3.47, and it deflates Chicago’s score from 3.67 to 2.83. (This is irrelevant to Virginia, of course, because their RMP score was not used.)

In addition, I looked at the three schools on this list who had their numbers used to see how many of the ratings were for actual law school professors. Stanford fares best – all the ratings are for current or former faculty (and only three are former, representing 18 of the 93 ratings). But at Alabama, only 12 of the 20 ever served on the law school faculty (taking the most generous view of “on the law faculty” possible) and they account for only 98 of the 219 ratings. Without the inclusion of the non-law faculty (most of whom are current or former members of the Economics, Finance, and Legal Studies Department), Alabama’s score goes from 3.96 to 3.60. (Chicago’s would go from a 3.67 to a 4.03 without the completely non-law professor you noted earlier – the others have all at least taught law courses.)

So basically, not only did they use bad data, but they used it badly, and they appeared to have miscalculated it in multiple cases (2 out of the 6 that I looked at – one egregiously).

MORE SHAME:  Houston.  But why tout being ranked 45th?  Schools need to pause and recognize that if they legitimate this nonsense by broadcasting it, it will come back to bite them another time around, given that the National Jurist ranking seems to be based on an even less good sense than U.S. News.  UPDATE:  Houston has also removed it--good for them!

ANOTHER UDPATE (FEB. 15):  The National Jurist editor has admitted to one of our assistant deans that if they dropped the fraudulent Rate My Professors data (which they admit seems problematic), Chicago would be in the top five overall.  One wonders how many other schools were penalized without any reason in this way?  In any case, this simply confirms that the variance is all due to the fraudulent data input, and that the whole thing is "pure ridiculousness."


February 12, 2013 in Rankings | Permalink

Monday, February 11, 2013

"Drastic changes" in how we educate new lawyers?

That's the theme emphasized in this Times report on the ABA task force on legal education meeting in Dallas.  On the table are sensible ideas--making it possible to get a two-year law degree, and creating a legal analogue of "nurse practitioner"--but one hopes that the would-be regulators bear in mind that the most important thing they can do is de-regulate or relax regulations on many fronts, so as to permit more experimentation with models of legal education at differential prices.   Many of the current problems result from a "one size fits all" model of regulation of law schools, as well as a "one size fits all" assessment of law schools by U.S. News.  The ABA isn't well-equipped, and never has been, to police the particulars of legal pedagogy.  Employers "vote" with their employment decisions and on that basis, it is quite clear that the most selective employers still vote for the "Yale" or "Chicago" or "Stanford" model of legal education (whatever exactly those are, but they clearly combine commitment to research along with teaching, extensive clinical opportunities, and a total commitment to interdisciplinary scholarship and teaching).  Certainly those schools have been modifying their own curricula and faculty in response to changes in the legal profession and to feedback from lawyers and judges over the past decade, and that will continue with or without the ABA.  But rather than top-down edicts that apply across the board--which is what the ABA has always done--it would do well to move in the opposite direction, to see what different approaches might also work.

Of course, no changes in legal education can help the tens of thousands of graduates saddled with debt and a dismal job market over the last half-dozen years in particular, and it would be good not to lose sight of the fact that any "drastic" changes in ABA regulation of law schools still leaves unaddressed the debt crisis that afflicts students in all parts of higher education.   But the ABA is not without its own political muscle, and one thing it might also do is use some of that muscle to put debt relief, including bankruptcy reform, on the table at the national level.

ADDENDUM:  The article does include one dangerous innuendo, namely, that somehow the current difficulties in the job market and the current need for more differentiation in models of legal education can somehow be laid at the doorstep of "tenured" faculty.  In fact, the lack of differentiation in models of legal education can be laid at only one doorstep, that of the ABA, and the dismal economy might be blamed on a variety of parties, from the deregulation of banking that began in the Clinton era, to miscreants on Wall Street.   Tenure remains an important feature of all serious academic institutions, and it would be a travesty were the current "crisis" mentality to undermine it.  That being said, and as we have written before, tenure has never meant lifetime employment, and the ABA, in consultation with AAUP lawyers, might think about drafting post-tenure review standards for law schools, to deal with the sometimes egregious abuse of tenure at many law schools.

February 11, 2013 in Legal Profession, Of Academic Interest | Permalink