Tuesday, April 8, 2008
Vault, whose rankings of law firms by prestige attracts some interest, has now produced, via a largely obscure methodology, a ranking of law schools. The "top 25" is not as silly as the Cooley "top 25," but at least with Cooley one knows why the results are ridiculous, because the ranking method is transparent (and transparently absurd). The ordinal listing in Vault is equally absurd, but the rhyme-or-reason of the absurdity is harder to pin down given how little Vault tells us about the "method":
Vault surveyed nearly 400 hiring partners, hiring commitee members, associate interviewers and recruiting professionals across the country on which law schools best prepare their graduates to achieve in the firm environment....
Vault surveyed only those people who directly assess the value of law school graduates in the real world once they enter the workforce- those individuals responsible for evaluating and hiring law school students. The respondents--who represent over 100 law firms-- were advised to consider the following factors in their rankings: research and writing skills; knowledge of legal doctrine; possession of other relevant knowledge (e.g., science for IP lawyers); and ability to manage a calendar and work with an assistant.
What was the distribution of these 400 professionals representing a mere 100+ law firms? Were they permitted to evaluate the schools from which they graduated? What kind of scores could they award schools? How were the scores aggregated? Was more weight given to some of the criteria noted above as against others? What were the differences in raw scores separating schools in the ordinal listing? Vault supplies no answers or information, and that lack of information probably explains why the ordinal listing is so inexplicable, even if the 25 schools listed seem like a reasonable top 25, apart from the ordering.
The results are even weirder when one looks at the representative comments from evaluators that Vault shares on its web site. Regarding #1 Stanford:
"Mixed bag, not always practical lawyers"
"Unprepared and uninterested in private practice; entitled"
Where is the love for #1? (Well, there's a little: "Every graduate I've encountered knows how to take an issue apart; great analytical skills.")
Are they happier with #4 Virginia? Not really:
"Maybe it's the law school's emphasis on softball and beer, but the lack of work ethic shows in recent grads."
Regarding #12 Duke:
"Graduates of this school are not prepared to practice law. There must be no practical training provided at all.
But it's not just Duke, it's even #5 Chicago:
"Law school education is too theoretical - would like to see lawyers with more 'practical' experience and application of the law"
And #10 Yale
"Weakness is lack of exposure to practical experience"
And #17 Georgetown:
"The lawyers I've worked with from this school were sloppy and unprepared."
"We've hired several lawyers from this school and they all have poor people skills and produce low-quality work."
Ouch. But wait, look what Vault reports someone saying about #2 Michigan:
"Overrated. Produces a lot of mediocre lawyers"
Ouch again! And a double put-down for #13 Penn:
"Like Columbia, students sometimes seem a little ungrounded"
And #22 Texas gets whacked too:
"Not impressed. Research skills not up to par."
Sigh. But at least some other lawyers love UT grads (almost as much as they love Harvard): "The Harvard of the West," says one. And another: "We have more success there than at any school other than Harvard."
But what do they think of Harvard students? According to Vault it runs the gamut from,
"Harvard graduates perform much better on average than their peers from other schools."
"Produces smart people, but don't know how to do basic legal research; new lawyers have a sense of entitlement that many of their peers do not"
"Often arrogant and think they walk on water even those lower down in the class"
The purportedly positive comments on the schools, including some of those zapped above, are an equally eclectic mix, ranging from genuinely enthusiastic ("Outstanding, hard to call University of Chicago underrated but it is" and regarding Duke grads, "Hard-working, smart, marketable to clients on paper right out of law school") to many involving comparisons that seem vaguely left-handed. UCLA, we are told, is "The Fordham of the west coast." On Michigan: "For producing quality graduates, UM is on the heels of the top Ivies." On Minnesota: "One of the better state schools." On George Washington: "A law school on the rise, but not there yet." BU: "Solid; high 2nd tier." North Carolina: "Getting better." Vanderbilt: "Very strong at top of class, not quite as strong as very top tier schools below top of class." Wisconsin: "Surprisingly good candidates."
Only Iowa gets nothing but positive comments!
In short, there's no rhyme or reason to any of this. I suspect the entire explanation for the results would be contained in the ridiculously small response pool and its probable lack of pertinent geographic and educational diversity. But until Vault comes clean on its "method," we won't know.