What Law School Rankings Should and Shouldn't Do: A Former Law Dean's Perspective

Professor Paul L. McKaskle, former Dean of the law school at the University of San Francisco, sent the following interesting remarks about rankings by e-mail (as a follow-up to an earlier thread), and kindly gave me permission to post them.  I've opened comments, since the issues raised by Professor McKaskle warrant discussion.  He wrote:

I think the idea of a comprehensive national ranking for ALL law schools is absurd.  It is one thing to say some schools are in the top ten or twenty, and generally speaking the rankings for schools in that range are probably not that far off.  But, beyond that, not only is there little reliable information that would allow an accurate comparison but I'm not sure that there is any meaningful definition of what makes one "lesser" school better than another.

For proper evaluation of law schools, there are probably two tiers and each tier should be evaluated with different criteria.  First, there are what I will call the “prestige” law schools–Harvard, Yale and another six to twenty schools (depending on whom one asks).  Many of the
students admitted to these schools have glittering credentials and some are truly gifted.  If a student is interested in a career in academia (especially at a “prestige” school) or is interested in clerking at the Supreme Court or for top Court of Appeals judges, or employment at a top 20 (or even a top 50) law firm (or hot public interest firms such as NAACP LDF) then the prestige of the school is of considerable importance.  A truly gifted student may even benefit from intensive contact with a faculty member who is on the cutting edge of some legal field.  Sponsorship is extremely important too, and top faculty at top ranked schools can do a much more effective job of this than someone in a lesser school. 

However I think the biggest advantage at being at a top ranked school for most students is the chance to interact with other very bright students–who are more plentiful at top rated schools.  John Roberts, no matter how talented he may be, would have been very unlikely to be where he is today (or this Fall) if he had gone to, say Notre Dame, rather than Harvard.  It isn’t that the teaching at a Harvard is “better” (whatever that is) it is the combination of prestige, faculty who can sponsor and other very bright students that make the difference.  (I have a niece who graduated magna cum laude from Harvard LS who has some horror stories about neglectful teaching–but she studied with very bright students.)

For these ‘prestige” schools, the USNews evaluations are probably more or less accurate.  By most measures (USNews measures as well as others) Harvard or Yale are the top ranked law schools–tops in faculty evaluation, tops in practitioner evaluation, tops in writing productivity of the faculty (except maybe for Chicago), tops in law clerk placement at the Supreme Court (and in Courts of Appeal as well, I imagine) and tops in placement at the top five, ten or fifty law firms.  A handful of other schools are not far behind.  One can quibble about exactly where on the list these other law schools would fall.  You have suggested that Boalt is unfairly downgraded a few places (and as a Boalt alum, I agree).  But for these schools it is just a quibble and the ranking probably has very little effect on the quality of students accepted by any of them.  (Until recently, when tuition at Boalt skyrocketed, I know a number of students who chose Boalt rather than Stanford simply because of the cost.)

But the number of students who are “truly gifted,” even at prestigious law schools, is very limited.  Somewhere between 90 to 99 percent of the students in law school today do not fall in that category.  Not even all students at the top five or ten law schools fit that category.  There may be only a handful of “truly gifted” students at lesser schools (Steve Shiffrin [ed.-now a Cornell law professor], who went to Loyola LS in L.A, is an example from the past).  But, that doesn’t mean that many of the “lesser” students won’t be good lawyers–some will be enormously successful.    But, what would be a “good” law school for these students is, probably, something which should be measured differently than for what I have classified as “prestige” law schools.  I don’t mean to suggest that the factors which put a law school at or near the top of the list are irrelevant, but only that they should be given different weight.

So the other tier in my classification includes the other 140 or 150 odd schools which do not qualify as what I refer to as a “prestige” school.  It is these schools for which the USNews evaluations by professors, judges and practitioners  are likely to be inaccurate or at least seriously misleading, because it is probably impossible for any small group of professional “evaluators” (law faculty or practitioner) to evaluate with even a pretense of accuracy.  Many such evaluators may simply be a look at last years evaluation with a quick look at the LSAT range (which for the “bottom” schools is pretty atrocious) in making a decision.  Indeed, as you noted, it is because of the impossibility of evaluating all law schools that you confine
yourself to simply analyzing the top fifty or so law schools–and even fewer for specific characteristics.

What makes one law school in this second tier better than another?  Ithink, first of all, it is more important that the school be a place where the student can learn how to be a competent lawyer.  (A good student at Yale can probably figure most of this out on his or her own.)  I suspect that even in a “lesser” school the quality of classmates is an important factor (something the reported LSAT figures measure somewhat).  Good teaching is important, and I think there is a correlation (though imperfect) between good teaching and a reasonable
amount of scholarly productivity.  Good teaching includes mentoring and other non-classroom interaction, not simply classroom teaching and the latter may be a more plentiful quantity at a school where publication isn’t the foremost requirement of a faculty member.  But all of this is something which is incredibly difficult for an outsider to measure (it is pretty hard to measure inside as well).  I don’t pretend to know how such evaluation should be done, or even what various factors should be considered, but I don’t think USNews comes within a thousand miles of knowing how to measure this for what I define as “non-prestige” schools–except possibly by using the LSAT data....

I agree completely that LSAT data is an imperfect measure.  It can be gamed, at least somewhat (though I doubt it would raise the quartile marks by more than a point.)  LSAT doesn’t say much about other characteristics of the student body which may be equally important--or even more important--in terms of the learning environment.  (I am chair of our Admissions Committee and I have the chance to look at hundreds of files, and among them are people with only so-so LSAT scores but other characteristics that are remarkable, and also some people with high LSAT scores whose other characteristics are mediocre or even alarming.)  Although I suggest that the only (modestly) valid factor reported by USNews is the LSAT data, I don’t think it is an ideal measure by any means.  And, to the degree it is the only somewhat valid widely available factor it puts highly unfortunate pressure on schools to admit based primarily on the LSAT, ignoring other interesting characteristics of an applicant.  (Should I refuse to admit someone who might lower our third LSAT quartile score who is very interesting–a Russian immigrant who learned English at age 17, earned stellar  undergraduate grades while working full time and has done many other interesting things, when, alternatively, I can admit someone who has an LSAT which is two points above our first quartile–thus might raise it--even though he was only a moderately good student in an undemanding major in a middling college who has no discernable interests?)  But since it is the only somewhat valid data which USNews reports, it is the best of a bad bargain!

As usual, non-anonymous comments are preferred, though substantive, on-point anonymous comments will be permitted.

Comments

[Stephen M (Ethesis)]

Nicely said, though I do think there is a two part "bottom" tier -- the first part of it is schools that are surprisingly good, but that have bad placement problems (e.g. Texas Wesleyan -- we've had great experiences with their graduates where I work, but all of them had trouble getting traction) and others, often characterized by Bar passage rates below 50% that just are not very good by any measure.

I know an attorney who has 24 zeros on the year so far. Part of what has made him successful is the number of people from the second part of the bottom tier he has encountered. I really think that such programs would do a great deal of good if they converted themselves into two year programs teaching people the legal skills necessary to be a good bankruptcy paralegal or a good claims adjuster or a contracts administration officer or some similar quasi-legal profession.

I'd divide things into:

(A) the real top tier
(B) the 50 or so schools in the "top 25" -- schools that recruiters, judges, and others draw from.
(C) the solid workmanship schools that train most of America's lawyers.
(D) bottom tier schools that teach, but that have placement problems, often because of rankings systems.
(E) bottom tier schools that don't teach and whose graduates do terrible work, if and when they pass the bar (schools that should be closed).

All in all I found Posner's article on ratings interesting as well.

[Kaimi]

Interesting idea, but I doubt that it would be workable. (It sounds a little like the "two-tiered clerk hiring" proposal that came out several months back).

One obvious problem is the schools on the margin. Is Northwestern a prestige school or a general-consumption school? How about Wisconsin? B.C.? Vandy? And so forth. The division into AAA and AA is crucial, and it's not as if there are any easy break points.

The second problem is that the audience for the U.S. News rankings is not academics (how many of us are there, really?). It's a list for the masses, and the masses would like to see a ranking from 1 to 200.

A third problem is the the rankings are used by prospective students. And many students fall in the range right at the border between Dean McKaskle's break. Say someone has an LSAT of 163 and a 3.5 GPA -- should she be looking on the prestige list or the general-consumption list? It may not be clear.

[Steve Diamond]

One problem I see in the rankings is job placement issues. I teach at Santa Clara after five years of practice, four of them in Silicon Valley. I have not looked at the rankings lately but I am certain there are quite a few schools in the east and south that rank higher than Santa Clara. But the chances of a student from University of Mississippi getting a job in the Valley is quite low, whereas Santa Clara students are bombarded by the culture of the Valley on a daily basis. Of course, there could be some self selection. But at the end of the day what use is it to anyone to compare Mississippi to Santa Clara?

[Anon]

I attended a second tier law school for my first year of law school and then transferred to a "prestige" law school (ordinarily included in the top handful or so), so perhaps I can offer some thoughts from a different perspective.

I should say that my credentials as an undergrad didn't qualify me for admission to a prestige law school. In some ways, they hardly qualified me for admission to any law school. I was a poor student in a relatively demanding major at a middling state school, and had been for more years than I should say. (My interests as an undergrad were certainly discernable from my record.) I also had an LSAT score in the 99th percentile.

I think that McKaskle is right that one of the primary advantages of a prestige school is the chance for interaction with lots of talented students.

But chances for faculty interaction are significantly reduced, in my experience. Faculty are very busy and aren't particularly interested in talking with or advising students. In the end, I traded off the chance to spend time with faculty for the chance to spend time with bright students. It isn't clear to me that students--even the truly gifted students McKaskle refers to--are better off in one scenario than in the other.

Of course, there are other considerations: What if one doesn't do as well in law school as one expects? At least if one does poorly at, say, Harvard, one can fall back on a job at an AmLaw 20 firm, with the good salary and miserable working conditions that go with that. And if one does well at Harvard, then all of the advantages of that degree follow. But those considerations are quite separate from the educational experience itself.

[Don]

To me, any form of numerical 'ranking' misses the point, and gives the false impression that a scool ranked a couple of places higher or lower is, by default, better or worse.

I advocate a system of grouping. That is, put the 'prestige' schools in a group, but don't rank that group. Then put the 'good' schools in another group, again, unranked. Then put the 'bad' schools in another group. Still no ranking within groups.

This would allow applicants and employers to determine the general quality of the schools they are looking at without the ridiculous comparisons between two good schools (e.g. Harvard v. Yale - neither is 'better' than the other). One would know that any school in the 'prestige' group would be great, and one would know that any school in the 'good' group would be as good as any other school in the 'good' group.

That said, my thoughts are that the schools, for all their public dislike of rankings, secretly love them. The school ranked, for example, 16 in the US News rankings loves the fact that a large majority of applicants and employers think that it's better than the school ranked 20, and as such, they get a better pool of applicants and students.

It's all academic though, since US News makes a huge amount of money from its position as the purveyor of rankings. What motivation have they to change?

[santo]

I would add that only 5-10% of all law faculty is truly gifted as well.

[AM]

Like most assessments, this one leaves out those schools that fall between the top 15 and the top 50. What about students coming out of Fordham, GW, Illinois? Are they the rest, or the best? Somewhere in between?

Also, having met students at a variety of schools from all tiers, I have to say--there is little difference. There may be a difference in the opportunities for students coming out of Yale than at Widener. Nevertheless, the overall intelligence of law students differs little.

[John]

"To me, any form of numerical 'ranking' misses the point, and gives the false impression that a scool ranked a couple of places higher or lower is, by default, better or worse.

I advocate a system of grouping. That is, put the 'prestige' schools in a group, but don't rank that group. Then put the 'good' schools in another group, again, unranked. Then put the 'bad' schools in another group. Still no ranking within groups."

This idea is theoretically attractive, and reflects somewhat what the author above was discussing. It even probably contains an element of validity with regard to many schools.

However, there is one major problem with such a system: where exactly do you draw the line? Wherever you draw it, there will probably be some schools, near the top of any given group, that are closer in quality to the bottom schools of the higher group than they are to the bottom schools in their own group.

In other words, such a system would exaggerate small differences (for certain schools) even more than the current system.

It's similar to the concept of providing fewer grades in law school (eliminating plusses and minuses, and only having straight letter grades.) This may sound attractive to the kid who just got a B-. However, it will be much less helpful to the kid who just got a B+, especially if he was very close to getting an A-.

Ultimately, even though small differences may be overemphasized by some readers of USNews, I think it's probably better to have more information than less.

(The current USNews system actually follows many of your suggestions by using a tier structure, and not ranking schools in the 3rd and 4th tiers. However, the latter fact could also be seen as unfair to the best schools in the 3rd and 4th tiers, for the reasons noted above.)

[John]

"Like most assessments, this one leaves out those schools that fall between the top 15 and the top 50. What about students coming out of Fordham, GW, Illinois? Are they the rest, or the best? Somewhere in between?

Also, having met students at a variety of schools from all tiers, I have to say--there is little difference. There may be a difference in the opportunities for students coming out of Yale than at Widener. Nevertheless, the overall intelligence of law students differs little."

First off, I'm not sure if it's helpful to focus on overall intelligence, as opposed to those specific skills and abilities that are important in the study and practice of law.

With regard to the latter, I'm sure that the top students from lower-ranked schools are often in fact superior to the worst students at top-ranked schools. The screening system used by law schools isn't perfect, and there are other qualities involved that aren't always measured well in admissions.

I also think it's quite likely that most law students, generally, are probably "smarter" (at least in the above sense) than the average person.

However, I don't personally believe it would really be accurate to say there is little difference in terms of the overall class at Yale vs. most 4th tier schools. You are, after all, talking about people with 99th percentile LSAT scores, universally strong GPA's, and strong majors, undergrads and intangibles vs. students with LSAT's around the 50th percentile, and generally far more modest academic accomplishments. On average, therefore, for most students, I've got to think there's going to be a significant difference, even if it may not be universal.

[John]

Just in regard to the idea of rankings generally:

One major problem with USNews, as noted by the author, is that few attorneys, judges, and academics really know that much about most schools.

As a result, regional reputation and placement is very poorly measured by the current system, which, on the other hand, probably does a decent job of noting the most national programs.

The best solution to this problem, in my opinion, is to perform a more comprehensive study that breaks the country down into regions, and then surveys lawyers, judges, and academics within each region concerning local schools. The survey results would not only be more informed, but they would more accurately reflect actual placement quality, since local firms would be doing most of the hiring for those programs.

You could still include student numbers, bar passage, etc., but either way, the result would be regional rankings that would be far more meaningful and helpful to most students.

Finally, in conjunction with each regional ranking, you could identify those schools with the best reputations across all regions, and the most national placement, and have a separate "national" ranking for those. This could be restricted to the top 20, 30, or 50, with membership presumably fluctuating.

(This would therefore address the desire for different tiers, while allowing even more regional schools to shine near the top of their respective regional rankings.)

Given that the reputation and placement of most schools (outside the top 15) is fairly regional, I think this system would address most of the problems with the current ranking system, and would again be much more helpful to those students who simply want to know the relative reputation and quality of schools in their desired region. There's little question, for example, that certain well-regarded 3rd-tier schools will usually place better in their home market than higher-ranked 2nd tier (or perhaps even 1st tier) schools that are more distant.

(Actually, given that the need and demand for such rankings is probably pretty strong, I'm surprised someone hasn't come out with it yet. Someone like the American Lawyer could make a ton of money if they produced such a publication, I would imagine.)

[Alejandro]

While the debate over the LSAT and its importance in the admission process as a predictive tool will never subside, I think more attention needs to be made to those special cases that Dean McKaskle himself has identified in his article; the poor LSAT student with fabulous grades, hard work ethic and/or tremendous drive and experience.

Simply said there are a great deal of students for whom, for one reason or another, the LSAT, GMAT, SAT simply fail as a predicitive measure. I was one such student and was blessed to have been admitted into a school ranked 6 at the time despite an LSAT of 151. I was not one of the "gifted" students of my class nor was I at the bottom of the class. What I did have that differentiated me a great deal from other students who had more of a natural ability to perform on standardized exams was a drive and work ethic that helped compensate for other things I lacked competitivly.

I resent the label of "gifted" student versus other student. The fact is that I have met many of these "gifted"students and quite frankly found that they mostly all shared a social awkwardness that was memorable. The typical "looks great on paper" story.

But is that what the educational community should be striving for? The mere mention of a class structure even within the top schools is counter productive and arrogant. The fact is that lower ranked schools, while not the same intellectual challenge, can teach many of these same "gifted' students a great deal about community involvment, social competition and just plain old street smarts.

If anything rankings need to better identify how schools have improved and emphasize those factors rather than keep a running tally of where they stand now. The tops schools will never change, but outside the top 10 the best infomration a potential applicant can recieve is an idea of where the school is going and how it is improving. But how you measure that and whether it is marketable is another issue.

[John]

Just to note, under the current 25th/75th midpoint calculation used by USNews, schools can essentally ignore the LSAT scores of a full 25% of their entering class. That would seem to provide plenty of room for many "special case" admits at most schools.

I also don't see how the being "socially awkward" takes away from someone being "gifted". After all, much of law involves quiet analysis and thought, not schmoozing and rainmaking. Those lawyers with stronger social skills certainly have an advantage down the road, but this is much harder to quantify, and opens to the door to much more bias and subjectivity in the admissions process.

(In other words, why not reward studious geeks for their unique abilities, especially if they are otherwise often handicapped? Only seems fair.)

All schools can teach students about community involvement, social competition and street smarts. The top schools have the benefit of also potentially providing greater intellectual challenge, as noted. It wouldn't make much sense to deprive gifted students of such an environment, especially if they'd struggle more in the other contexts.

[Alejandro]

my point was that to say a student is gifted immediately places an order of value to a syatem thereby ignoring and depreciating other aspects of both the law school experience and "average" students, that should not only be as valuable, but in my opinion are more valuable to what it means to be a gifted lawyer.

After all lawyers are idealized as leaders in their community and this leadership skill is essentially the hallmark feature of the type A personality which pervades law schools. To grant someone status as "gifted" based simply on the fact that they are law review or in a top school, thereby ignoring their practical contribution to the practice of law is a diservice to the profession.

Lastly , medical schools, dental schools and selective other professional programs have a long history of evaluating the social skills of its applicants in the admission process. Its not an impossible factor to evaluate.

But i am not saying "hey lets weed out the weirdos and nerds". Not at all. Just that any evaluation of an admission system using words like "gifted" to selectively substantiate the existence of a ranking system lacks focus.

Lastly , all schools cannot teach about street smarts or community involvement. In fact its not even at school one learns about these things but rather from fellow students or teh real world working environment. Furthermore, when your faculty is made up of harvard BA, harvard JD, princton PHD, supreme court clerk, associate professorship, there is not much street smarts there. In fact its probably a very self centered highly intellectual upbringing that has little to do with the real world most lawyers deal with on a daily basis. Is this a diservice to students? Aboslutely not, but dont tell me that a top 10 school is superior because of it.

[John]

First off, the writer in question wasn't using the term "gifted" to substantiate a specific ranking system. He was simply noting that some students are in fact more gifted than others, and that there are therefore probably consequential differences between different types of schools which serve different types of students.

I don't really understand your 2nd paragraph, but I don't personally agree that "leadership" skills are really that prevalent among most lawyers or law students. Maybe that was also your point, though.

However, I think it's hard to argue that someone on law review isn't generally more intellectually gifted than most of his classmates, or that someone at a top school isn't generallly more intellectually gifted than most other law students.

This doesn't mean we shouldn't look at a person's actual contributions in practice. Thurgood Marshall didn't attend a top school, and I don't know if he was on law review, but he certainly had a positive impact on the legal system, and is recognized for that.

But if we really wish to pursue excellence (and not just uniformity), then I think we need to be willing to recognize distinctions between students, both within and between schools, as well as distinctions in quality between professional practitioners. Not being willing to recognize effort and achievement simply because it creates distinctions is probably not in the long-term interest of any society.

Finally, most schools can teach about street smarts and community involvement, because most offer clinical opportunities to work with legal aid clinics, etc. Also, I think you're making a lot of assumptions about what kind of people attend elite undergrads and law schools today, and end up teaching. There are plenty of people with modest backgrounds and real-world work history who end up at HLS and other top schools (because today, fortunately, we have a essentially meritocratic admissions system).

Finally, most of the faculty at lower-ranked schools are also from top schools, so that would eliminate any advantage or deficiency in that regard.