"How to Rank Law Schools"

Here's a draft (Download how_to_rank_law_schools.doc) of my contribution to the Indiana Law Journal symposium on law school rankings (other papers from that symposium have been referenced here and here).  My contribution is largely a commentary on papers by others.  Since I still have time to correct egregious errors and ridiculous statements, I've opened comments and feedback is welcome.

A provocative excerpt for those who don't want to bother to peruse the document:

All of these criticisms [of U.S. News] presuppose, of course, that a ranking of academic institutions ought to reflect certain relevant attributes which serve as a benchmark for critiquing the U.S. News result.  Here I part company with Professor Korobkin, who reiterates his well-known Marxist view that rankings essentially serve a coordination function—allowing good students to find good employers and vice-versa—such that the criteria by which schools are ranked hardly matters.[1]  On this view, legal education is really about pedigree and certification, not education and training; as I once heard a prospective law student put it:  “I'm going to law school to get my ticket punched. Everyone knows you learn the material on your own anyway.”

There is certainly something to this:  if, sotto voce, the Fordham faculty were swapped for the Yale faculty next year, Yale would still continue to produce hugely successful graduates for the foreseeable future.  But that is surely, in significant part, because the Fordham faculty is rather good, and so the real question should be:   what if we swapped, say, the Baylor faculty for the Yale faculty tomorrow?  While the Yale “name” would continue to carry forward for a short while, surely it would not be long before both students and judges and employers noticed that something significant had changed—and not only that Yale students were being taught by folks who actually knew how to practice law!

But what is it exactly that they would notice?  According to Professor Korobkin, it would be nothing that matters to either the students or the employers, and, therefore, the only reason to prefer a ranking that favors the Yale faculty over the Baylor faculty is that we have made a societal value judgment to encourage the kind of scholarly work Yale faculty do.  Perhaps this is right, though I am skeptical.

I am still attracted to the old-fashioned view that those who are smarter and more learned can provide higher-quality instruction.  (I am not saying that this is true of the Yale faculty, though it may be in some cases.)  This is not to say that the best scholars are the best teachers:  that plainly is not true, since there are a variety of pedagogical skills that are unrelated to intellectual acumen.  But it is to say that no set of pedagogical skills can compensate for lack of intellectual depth in one’s subject-matter, and I’m reasonably confident, based on experience on both sides of the podium, that this is true.  That difference may be lost on many students, to be sure, but it won’t be lost on the better ones.  And whether noticed or not, if the old-fashioned view is correct, then it will affect educational outcomes.  With all that in mind, I think an assessment of academic institutions ought to weigh heavily the intellectual and scholarly caliber of the faculty, not to the exclusion of other factors, but as a way of putting education at the center of any evaluation of institutions in the business of educating.

Let me conclude by suggesting four general guidelines for how law schools can be meaningfully and usefully ranked.

            First, rankings of academic institutions should emphasize and reward academic values:  namely, scholarly excellence, pedagogical skill, and student ability and achievement.  It is odd to have to lay emphasis on this, but in an era in which U.S. News ranks schools based on the inefficiency of their spending and their self-reported, and thus largely fictional, job placement statistics, it is, I fear, necessary to state the obvious.

            Second, it is desirable to evaluate law schools along dimensions where there can be measurable change and constructive competition.  Not all the elements of academic value are equally susceptible to measurement, but some certainly are.  If Professor Stake is correct in his contribution to this symposium[2] (and I am persuaded that he is), then one of the many deficiencies of U.S. News is that its reputational surveys of academics are so poorly conducted that they have simply become echo chambers of the prior year’s U.S. News ranking.  But this doesn’t mean faculty quality can not be measured more reliably by better-designed surveys or by the use of “objective” measures like citations.  So, too, measures of student quality in terms of LSAT scores are hostage both to a similar echo chamber effect, as well as the many other factors identifies by Professors Henderson and Morriss in their contribution.[3]  To the extent more academically sound rankings proliferate, serious students will begin making better-informed choices, and rankings of student quality may tell us more than how U.S. News recently ranked particular schools.

            Third, those elements worth measuring should be measured separately rather than aggregated on the basis of unprincipled and unrationalizable schema.  One can rank schools based on SSRN downloads, student LSAT scores, faculty reputation, scholarly impact as measured by citations, job placement, Supreme Court clerkships, and so on, but there is no way these criteria can be meaningfully amalgamated.

            Fourth, we should encourage and welcome many different kinds of academic rankings from many different sources;[4] that is the only way to counteract the excessive influence of the academically unreliable U.S. News rankings.  If the Association of American Law Schools were not in contention for being recognized as the most useless professional organization in the United States, it would have long ago taken the lead in promoting alternatives, instead of giving students the laughable advice that they should discount prestige and reputation in choosing schools.  There is, in fact, a sizable audience looking for rankings that convey genuine academic information:  contrast my own academically-oriented ranking site,[5] which garners upwards of 10,000 hits per week during the peak admissions season and which has been frequently discussed in this symposium, with the bizarre Thomas M. Cooley law school rankings,[6] which contain no useful information and are uniformly ignored by students, faculty, and in most discussions of rankings.  If Professor Korobkin were right, though, then the Cooley method of simply aggregating A.B.A. data without regard to its meaning or importance would have worked as well for U.S. News as the methods it actually adopted, which at least attempt to identify some factors of relevance to legal education.

            Academic rankings that provide actual information on matters of educational value have a useful role to play for students, quite obviously, but they also have a constructive role to play for faculty.  Professor Korobkin suggests that in ranking schools we want to discourage “status competition.”[7]  I guess my own view is more Nietzschean, and so let me close with a quote I have used before.[8]  This is Nietzsche from his early essay on “Homer’s Contest”:

Jealousy, hatred, and eveny…spur[] men to activity:  not to the activity of fights of annihilation but to the activity of fights which are contests.  The Greek is envrious, and he does not consider this quality a blemish but the gift of a beneficial godhead….  The greater and more sublime a Greek is, the brighter the flame of ambition that flares out of him, consuming everybody who runs on the same course….

     Every talent must unfold itself in fighting:  that is the command of Hellenic popular pedagogy, whereas modern educators dread nothing more than the unleashing of so-called ambition…And just as the youths were educated through contests, their educators were also engaged in contests with each other.

We should produce rankings, and more of them, that unleash academic talent and ambition, not rankings that reward decanal connivance at manipulating ranking schemes cooked up by journalists.  Although many of the scholarly critiques of U.S. News in this symposium are devastating, only alternative ranking schemes, that embody academic values we share, will counteract the pernicious impact of U.S. News on legal education.  And, in the process, the right kinds of academic rankings may also stimulate and strengthen our scholarly community in law.

---

[1] Korobkin, supra n. __ at __.  On the Marxian nature of Professor Korobkin’s analysis, see Brian Leiter, Measuring the Academic Distinction of Law Faculties, 29 J. OF LEGAL STUDIES  451, 454 (2000).

[2] [cite]

[3] [cite]

[4] One must note, however, that even in the world of business schools, where there are five different media outlets ranking schools, faculty still bemoan the effect of rankings.  See http://papers.ssrn.com/sol3/papers.cfm?abstract_id=766404

[5] [cite].

[6] See http://www.cooley.edu/rankings/index.htm

[7] Korobkin, supra n. __ at (p. 10 of typescript).

[8] Leiter, Measuring the Academic Distinction of Law Faculties, supra n. __ at 451.

Comments

[anon]

I know someone who had the lofty ambition of clerking for the supreme court. She was given the misguided advice to simply "choose a school which felt right to her." In the end, she chose a top 20-ish rather than a top 3-ish school (she was accepted both places). While statistics about clerkships are available, they are not thrown in your face like the US News Rankings are. And when the AALS tells people that prestige should be ignored, they go a long way toward distorting the truth. I say this because the misinformation really does affect have a detrimental effect on many young, talented people. It is certainly a scandal that instead of telling people "if you want to clerk for SCOTUS, or if you want to teach law here are the statistics on particular schools," we bombard them with worthless US News rankings and advice like "ignore prestige."

We would be much better off giving, as Prof. Leiter suggests, non-amalgamated ranking information and giving them guidance as to which statistics are important.

[Andrew Perlman]

I enjoyed your excerpt about law school rankings, and I agreed with most of it. I disagreed, however, with a couple of your claims.

First, I don't believe in the "old fashioned view" that "smarter and more learned" faculty provide better instruction. The first step in your argument appears to be that "no set of pedagogical skills can compensate for lack of intellectual depth in one's subject matter."
That claim seems right to me. But the next step in your reasoning is that because intellectual depth is important to educational quality, we should "weigh heavily" the intellectual and scholarly caliber of faculties when assessing the quality of education.

The problem with the second step in your reasoning is that you assume rather than prove that educational quality improves beyond a base level of intellectual depth in one's subject matter. Let's assume for the sake of argument that Yale's professors have more intellectual depth in their areas of expertise than Baylor's. Even if that's true, I'm sure you would agree that Baylor's professors have intellectual depth in their fields; it's just less deep than Yale's. The key empirical issue
is whether pedagogical skills are correlated to intellectual depth beyond some base level of knowledge. It is by no means clear that one's
abilities as a law teacher improve once one has established some basic level of intellectual depth. Although this correlation is critical to your argument, you don't provide support for it.

In fact, in my experience, once one achieves a certain base level of knowledge, teaching abilities may even be adversely affected by
additional "depth." As a student at Harvard, I found that the more "depth" a professor had, the more difficulty she had communicating basic
concepts to students. At the very least, there didn't seem to be a correlation with depth. In any event, I think it is far from clear that a Yale professor's greater intellectual depth in her subject area gives her any more of an advantage in training elite law students than a Baylor professor would have.

A second problem I had with your argument is that you implicitly assume that Yale's professors are "smarter and more learned" just because they have more intellectual depth in their fields. With respect to intelligence, I'm skeptical that one's IQ is strongly related to one's intellectual depth. The Yale faculty may be more skilled at legal scholarship. They may have been stronger students in law school. But
smarter? I'd love to see the evidence. As for being more learned, I guess it depends what you mean by the term. If learned means more
productive as a scholar, I guess you're right that Yale scholars are more learned. But what about involvements in professional associations,
pro bono work, or other law-related activities? A law professor who engages in these activities gains a kind of learnedness about the profession that the pure scholar can never achieve. And again, it's far from clear what sort of learnedness makes for the best law teachers.

Like I said, I enjoyed the excerpt. I'm just a bit skeptical about your assumed correlation between intellectual depth (at least beyond a certain minimum) and teaching abilities.

[C.J.Colucci]

What if you swapped the Fordham or Baylor STUDENTS with the Yale students and didn't tell anybody?

[Alfred Brophy]

One thought: you’re too modest about what good rankings schemes can accomplish. I understand your desire to appeal to greed (9), but I think there’s something more positive at work here. A good ranking system encourages faculty to exhibit good behavior— like hiring young folks who’ll produce thoughtful scholarship. Rankings schemes are part of developing a professional culture. I hope that the schemes will emphasize factors that are important to the profession (like high-quality, original, thoughtful, and engaged scholarship; excellent teaching; assisting students develop as lawyers and citizens). In hiring at my school we frequently discuss how a candidate will make us look in the eyes of peers (like the Leiter Report raters and the US News raters).

Scholarship has recently become substantially more important in the academy (and in hiring), which I think is in part testimony to the US News and to the Leiter Report rankings. The Leiter Report peer survey has caused already a shift in hiring practices and I expect it to continue to have a positive effect. This is because we are developing an increasing sense in the profession of what constitutes good faculty behavior and we’re developing good behavior in hiring committees, which are increasingly recognizing the importance of scholarship.

[Stephen M (Ethesis)]

What if you swapped the Fordham or Baylor STUDENTS with the Yale students and didn't tell anybody?

Excellent point. Posner makes it as well.