Thursday, March 12, 2009

Which are the highest quality legal journals?

This poll may provide some useful advice for younger scholars wondering where to submit their work.  Do not participate unless you are actually a legal scholar! 

UPDATE:  So the results so far, after about fifty votes, are pretty surprising, and lead me to suspect that students are voting.  So let me open comments and ask:  is there really any legal academic who thinks the quality of articles in, say, the Harvard Law Review is really higher than the quality of articles in Journal of Legal Studies or Oxford Journal  of Legal Studies or almost any of the faculty-edited journals?  I find that quite hard to believe, but I am open to being persuaded otherwise.  Signed comments only; post only once.

Professional Advice, Rankings | Permalink

TrackBack URL for this entry:

Listed below are links to weblogs that reference Which are the highest quality legal journals?:

» Faculty Influence on Article Selection at the Law Reviews from PrawfsBlawg
I wanted to build on Dan's comments about the fascinating thread over at Brian's Leiter's place but then take them in a different direction. What I found most interesting were the comments about faculty influence on the selection process. Here are a fe... [Read More]

Tracked on Mar 17, 2009 9:20:57 AM


I, for one, don't believe it. Interesting that the top and bottom of the list has basically flipped since I initially voted and looked at the results (about a dozen votes in). At that point, the faculty-edited journals were on top.

Posted by: Keith | Mar 12, 2009 12:05:28 PM

I have just voted now. My surprise? Not only to find OJLS doing poorly thus far -- but also Legal Theory. Strange.

Posted by: Thom Brooks | Mar 12, 2009 12:43:55 PM

I agree with Keith. Early on, Journal of Legal Studies was clearly number 1. The question focuses on consistency of quality. Some (but not all) of the faculty-edited journals on the list are much more consistent than any of the student-edited journals.

Posted by: Ben Barros | Mar 12, 2009 12:45:00 PM

I'm surprised at your surprise. Almost all the junior non-quant scholars and most of the senior non-quant scholars in law (ie, not law and ec) I know submit exclusively to the student-edited law reviews. I've not once heard of someone submitting to the Oxford JLS! So anyone who thinks the peer reviewed publications are getting the best of the best must also think that the pipeline for those articles include only the best scholars.

Indeed, I would bet that there is more far more competition to get into the HLR or YLJ than any comparable top level peer reviewed journal. That doesn't say the pool of applicants is as good, but at least the raw competition, I'm guessing, is tougher. But that's just my sense. I could definitely be wrong.

Posted by: Dan Markel | Mar 12, 2009 5:38:42 PM

I think one can grant Dan's point arguendo, that there is far more competition to get into the HLR or the YLJ than, say, the OJLS -- though this isn't obvious to me, insofar as the OJLS gets submissions from the UK and the rest of Europe in addition to the US -- without conceding that the HLS or the YLS is a higher quality journal. The proper measure of a journal's quality is, I would think, its output, not its input. Harvard and Yale students are undeniably bright, but one needs more than bare intelligence to evaluate submissions. So even if they do get many more of the best submissions, they don't necessarily choose the best to publish. Faculty- or otherwise blind/peer-reviewed journals surely do a better job of evaluating the submissions that they get. And I'm pretty sure, for that reason, that a journal like the OJLS publishes better articles than any student edited law review -- even if the OJLS has a more favorable submission-to-publication slot ration. Indeed, I don't doubt that more than one peer-reviewed journal got its start due to faculty dissatisfaction with the quality of articles that the top student-edited journals were publishing.

If I may, let me add that for those of you who are interested, I will be organizing the AALS's Scholarship section panel at the upcoming annual meeting in New Orleans on the movement towards peer-reviewed law journals, and so far I have at least tentative commitments to take part in the panel from Omri Ben-Shahar, an editor of the Journal of Legal Studies, Larry Alexander, an editor of Legal Theory, and Mark Ramseyer, of the Journal of Legal Analysis. Its never too soon to mark your calendars!

Posted by: John Oberdiek | Mar 12, 2009 6:53:10 PM

As a law student, I don't know very much about the peer-reviewed journals. I just have a quick comment on your update. First, without reaching the merits of your criticism, I will say I think it's probably a bad move to have added the update while keeping the poll open. As I'm sure you know, framing the poll in this way (does *anyone* really believe HLR is better than any peer-reviewed journal?) is very likely to skew the results. I assume your goal is to produce an accurate reflection of academic attitudes in the absence of your framing, but I'm not confident that the poll can do so at this point.

Briefly regarding the merits, I would just point out that there are many respected scholars who seem to almost categorically prefer the student-edited reviews over faculty journals for one reason or another, if only based on the fact that they never or seldom publish outside of the student-edited reviews (e.g., Daryl Levinson). Unless you believe that such scholars are simply unable to publish in peer reviewed journals, there must be something else going on. My guess is that peer-reviewed journals are more popular and well-regarded by scholars in some fields (jurisprudence, corporate law, law and econ) than in others.

Posted by: Steve Horowitz | Mar 13, 2009 5:47:35 AM

One need only read a single year's volume of the Yale Law Journal and Oxford Journal of Legal Studies to see which one publishes higher average quality articles. (The difference, I suspect, comes less at the top, than at the bottom: all the leading student-edited law reviews publish a certain amount of sophomoric nonsense that wouldn't even be sent out for review at a faculty-edited journal.) No doubt young scholars prefer to get into HLR or YLJ because of visibility and because, alas, some faculty, who should know better, still think that publishing in one of these journals means something. But the question here was not about demand or desireability of publishing in the forum, but about "quality."

In this regard, Mr. Horowitz's comment may misunderstand the point of the survey. There is a reasonable argument to be had about whether, e.g., Harvard Law Review publishes, on average, better quality articles than Michigan Law Review, but there is no reasonable debate to be had about the average quality as between Harvard Law Review and Journal of Legal Studies. I had hoped the survey might shed some light on how faculty view the hierarchy of faculty-edited journals and how, at the margins, they think some of the faculty-edited journals compare to the student-edited law reviews (e.g., it may well be that better work in constitutional law appears in Harvard Law Review than Constitutional Commentary). But, as with all public on-line polls, one can't control who votes, and so either students are voting *or* faculty who are thinking about this in Dan Markel's way are voting *or* faculty who are voting who misunderstood the question.

Posted by: Brian Leiter | Mar 13, 2009 7:58:08 AM

Brian, I think it is the ambiguity of "quality." From a strictly academic perspective of internal quality, you are undoubtedly right. I can't speak for the philosophy journals, but I suspect they are infected by the flaw that may be found in economic and political science journals. Which address intricate little problems, using the highest level of methodological analysis, but they aren't of great external pragmatic importance. A law review article seems more likely to tackle something of greater external import. If quality is just the internal quality of the analysis, law reviews lose, say 100 to 50. But if you are talking about external value, you have to multiply those numbers by a quantitative measure of the significance of the issue. Once you do so, the law reviews might come out ahead.

Posted by: Frank Cross | Mar 13, 2009 8:18:09 AM

"[B]ut there is no reasonable debate to be had about the average quality as between Harvard Law Review and Journal of Legal Studies."

Sorry, but why is there no reasonable debate to be had? HLR and YLJ get multiple faculty reviews before publishing anything. I'm sure you're right that the voting here might include students or others you did not want to vote. But when you decide before seeing results that a particular set of results are presumptively impossible because of your a priori beliefs, I guess one starts to wonder why you bothered with the survey. You were hoping to get something like JLS -- OJLS -- Harvard -- Yale -- Constitutional Commentary, and then I guess that would be evidence that the voters represented the population group you had hoped would vote?

Posted by: Elisabeth | Mar 13, 2009 11:29:10 AM

There is no reasonable debate to be had because of the a posteriori fact about the average quality of articles in JLS compared to the Harvard Law review.

I "bothered" with the survey because even allowing that the average quality of work in JLS is superior to the average quality in HLR, there are still lots of other interesting comparisons to be made: e.g., as between JLS and JLE and ALER and OJLS and so on, and between some of the student-edited reviews and some of the faculty edited journals (e.g., Constitutional Commentary vs. YLJ). At the time of Keith Whittington's comment above, there had been a complete flip from nothing but faculty-edited journals at the top to nothing but student-edited law reviews at the top. Now things are evening out a bit, and so I'm hopeful some interesting information may emerge, if not strictly about quality, then perhaps also about perceptions of quality.

Posted by: Brian Leiter | Mar 13, 2009 12:59:14 PM

One more thought on Elizabeth's remarks: one thing that is unclear is how much of a role in article selection faculty play at the elite law schools. They clearly play more now than even fifteen years ago, but is it really the case that any article published in Yale or Harvard or Chicago or Columbia is really a 'peer' refereed article at this point? I'd be interested to hear from student editors.

Regarding Frank's comments: Frank, surely you, as well as anyone, knows that most articles publihsed in student-edited are of no practical importance! "The growing disjunction" between the legal academy and the bench and bar has been taking place in the student-edited law reviews, after all!

Posted by: Brian Leiter | Mar 13, 2009 1:56:52 PM

Brian, I can't say that I know that most articles published in student-edited journals are of no practical importance because I don't read that many of them. Which sort of goes to your point about their caliber, I suppose. But my point is that these journals sometimes publish articles on significant, front page, issues. Like the Yoo memoranda and associated Supreme Court decisions, for example.

Posted by: Frank Cross | Mar 13, 2009 2:17:19 PM

An interesting point wrt some law reviews getting faculty reviews. Even if these reviews have a strong effect, there are two impt points I think: 1) as I understand it, these law reviews generally go to their own faculty which might not have an "expert" in the narrow field (e.g., who at Harvard could do a good review of econometric work?); 2) the reviews are just used to accept or decline so you don't have the improvement that comes with revise and resubmits that you get at the journals. Plus if HLR rejects on the basis of a faculty review, the author need not go very far down the list to find a law review that doesn't do these kinds of reviews (e.g., I don't think Columbia LR generally gets reviews).

As for #1, something interesting appears to be happening at Stanford. They sent me an econometric article they were considering (& I'm not in-house). Also, they asked me if I would agree to be on a standing referee list/advisory board for this kind of work. My respect for Stanford went up a lot after this since I see it as a push toward a real refereeing system (though they'll still suffer from #2, and they'll likely have trouble getting lots of referees since they asked for a 2-3 day turn around).

As an aside, I was surprised that JLS is coming out so far ahead of JLE and JLEO. I think almost all economists would put JLS third and, having published in all 3 places a few times (and rejected from all three as well), the reviews are generally more rigorous at JLE and JLEO, though JLS generally has the most interesting articles.

Posted by: L&E Scholar | Mar 13, 2009 3:15:27 PM

Just a quick note that L&E scholar e-mailed me before posting, so I can confirm that the information in the post is legit (as well as interesting!).

On JLS, I think it's big advantage is that it is read by a much wider audience than JLE and the other more 'technical' journals in the L&E area.

Posted by: Brian | Mar 13, 2009 3:25:17 PM

Brian, I won't argue with your a posteriori point, but as to the survey I favor your later suggestion that this could be more about perceptions of quality than anything. I ration my journal reading time to subjects and authors of particular interest; the only journals I read cover to cover are in the true "specialty" journals that you purportedly exclude. (I say "purportedly" because some of the listed journals have real limits to the generality of their coverage -- e.g., it's kind of unlikely that anyone uninterested in UK law or EU law will read the Modern Law Review, that anyone not looking for law and science will read much of Jurimetrics, or that anyone not interested in law and philosophy will read lots of Legal Theory.) If others are in the same boat, and not answering "no opinion" to a high proportion of the entries, I doubt you are getting good data on quality -- more by the way of generalizations based on occasional reads and secondary indicia.

If you ever take another crack at this, I'd suggest (a) changing the format to separate Condorcet tournaments for peer and student reviews, then a head-to-head b/w the winners, esp. if your interest is predominately with intra-species comparisons; (b) define clearly what you mean by "consistently publish the highest quality legal scholarship" (e.g., whether some journal that reliably publishes one tip-top article that exceeds any other anywhere, but surrounds it with a bunch of crap, is "consistent," or whether you are interested in average quality), (c) really emphasizes the "no opinion" or "insufficient information" options.

BTW, the flip in poll results may be because the poll was taken over, but could also have to do with the population that checks this blog frequently and is most likely to spot this item, follow your link, and complete the questions early. My wild guess is that *early* poll-takers (e.g., the first dozen) are disproportionately likely to be among those who share your interests and perceptions of quality. Just a thought.

Posted by: Edward Swaine | Mar 13, 2009 3:28:04 PM

I agree with many of the previous commentators that JLS seems to have deliberately sacrificed precision for audience share: it is now so highly specialized that it isn't useful to most legal academics, who lack the training or inclination to read the articles it publishes. (And, contra Markel, I think that it isn't particularly quanty, but rather mostly devoted to formal modeling. Worse!)

In terms of peer referees in student journals, in this cycle I've heard of articles at each of H-Y-and S sent out for referee *after* the student board had said yes, and in each case, the faculty review resulted in turning the article down. So, I think they are playing some role. Not monitoring, but maybe gatekeeping. And in at least one case, I've definite knowledge that the review was outside of the law school's own faculty.

Posted by: dave hoffman | Mar 13, 2009 5:22:48 PM

As the comments suggest, I think we're seeing some disagreement about the relative "quality" (across several competing dimensions) of these various (quite different) journals. But I think we're also seeing some sampling bias in the poll response. Brian is worried about students bumping up the student-edited law journals -- maybe so. But the current ordering of the social science type journals certainly doesn't match the common understanding of their reputational rank among political scientists and the usual submission order for those with appropriate manuscripts. I suspect things would look rather different if we had a couple of hundred votes flood in from members of the Law and Courts section of APSA.

Posted by: Keith | Mar 13, 2009 6:55:47 PM

I can confirm what is going on at Stanford. As of a few months ago, Stanford Law Review's new policy is to have all articles peer reviewed before they are accepted. (We had to make one exception because another school gave the author a one hour exploding offer.) Since this policy was put in place, most articles we selected have been reviewed by more than one professor, and we've made a special effort to reach out to experts at other schools. Faculty from across the country have been very gracious in giving us thoughtful, incredibly insightful commentary.

Posted by: Jeffrey Kessler | Mar 14, 2009 12:58:00 PM

I think it's good what Stanford and other schools are doing, but its not peer review. I was one of those "faculty from across the country" who commented to Stanford's law review on a submission. It wasn't at all the same as a peer review, in part because of time constraints. Of course, that's a clear benefit of law reviews, peer review often involves extraordinary delay in publication, perhaps why they can't cover important timely issues. And the lesser faculty screening for law reviews is probably good enough to identify any clear mistakes.

Posted by: Frank Cross | Mar 15, 2009 11:02:20 AM

It is common wisdom (which may or may not be true) that student-run edited journals rely heavily on name recognition and author credentials in their decision whether to accept, reject, or at least closely read a paper. Threrefore, it seems to me that the peer-review that HLR, YLJ, and SLR have implemented addresses only one type of errors: it only prevents a low quality paper (written by a well known author) from being published, but it doesn't prevent the problem of high quality papers not being seriously considered because their authors are less well known.

Posted by: Ariel Katz | Mar 16, 2009 11:57:39 AM

It's difficult to compare journals like HLR and YLJ with journals like JLS and OJLS. A brief look at the most recent issues of JLS and OJLS suggests that more than half of the articles in these volumes are not of the right "scope" for the top-ranked student journals. The law reviews' preference for broad-ranging, self-consciously "foundational" work may reflect a lack of specialized knowledge on the part of student editors, as well as an occasional failure to appreciate what's already out there. But it probably also reflects a sense (accurate, in my opinion) that publishing such works has come to be their role in legal scholarship. In any case, if one is going to compare the quality of the articles in these rather different types of journals, it may make sense to consider how well they serve their respective functions.

Posted by: Anthony | Mar 17, 2009 11:28:15 AM

Post a comment