August 7, 2013
The ABA Task Force Working Paper (WP) on "the Future of Legal Education," Part II
In an earlier post, I noted my basic agreement with the majority of the thematic recommendations of the Task Force. Here I will make a few more critical comments on aspects of the WP, though it may be that, at the end of the day, they will not matter much.
1. It is important that the Task Force's recommendations be based on real evidence, not cyber-chatter and cyber-hysteria. That means, in particular, that the final Task Force report should take account of two important developments since the Task Force began its work.
2. Although these two important developments do not undermine the relevance of the Task Force's most important recommendations, they do have some bearing on some (but not all) claims in the WP that are of doubtful accuracy--claims that should either be cut or substantiated:
First, the Simkovic & McIntyre paper provides substantial evidence that the JD degree confers substantial economic benefits on its holders as compared to those who only have a BA. Despite several weeks of cyber-ranting, it's pretty clear that their results are sound. Even so, the Task Force's objective of making legal education available at lower cost remains obviously desirable: first, because more individuals would reap more financial benefits from the JD (the "private good" of legal education as the WP calls it), and, second, more lawyers would likely be available to those in need of legal representation. The "public good" and "private good" aspects of legal training would both benefit. But the Task Force should not lose sight of the reality that as a "private good" legal education has done very well. (This should be incorporated into the WP's discussion, e.g., of "return on investment" [p. 10].)
Second, the federal government's "Income-Based Repayment" program for student loans means that the burden of student loans for education, including legal education, is not what it was just a few years ago. That fact--and it is now a fact--does not mean that it would not be good for society as a whole, and for individuals, it it were possible to get a legal education at a lower cost: IBR is not loan-forgiveness, and so, once again, both the private and public good aspects of legal education would be well-served by lowering the costs of legal education, even in an IBR world.
(a) The WP suggests that law schools used to think of "education as a means to personal growth and development" but now the "consumer outlook" has become dominant, namely, "education as a means to a job or career" (p. 9). This is mystifying: who has defended law school, as a postgraduate professional degree, as being primarily about "personal growth and development"? The defense of including economics, philosophy, psychology, and history into aspects of the legal curriculum has always been that it makes better lawyers, not that it contributes to "personal growth and development" (it may do the latter too, of course).
(b) The WP says that J.D. students "who do not receive substantial scholarships pay for their education through loans" (p. 11). But surely some number of JD students have their legal education funded in whole or in part by their own savings and earnings, or by those of spouses, partners, or parents. What are the facts? In any case, this is minor: the main point is that the section of the WP on "Financing of Legal Education" needs to take into account the advent of IBR.
(c) At the bottom of page 12 of the WP appears the most dramatic factual claim in the whole report, namely, that changes in the "economy of law-related services and the related employment market...over the past five years" are "structural": "The developments are likely to continue, with contuing impact on lawyer employment. It seems probable that this change in employment for lawyers is not just a passing phenomenon caused by the Great Recession." This may be right, but what is the evidence? None is cited in the WP. The Simkovic & McIntyre study adduced strong evidence that the wage premium for a JD is cyclical, and that even since 2008 we are still within past cyclical norms. This may be wrong: the future may not be like the past. But to the extent Task Force recommendations turn on this assumption, it is important to adduce compelling evidence that the assumption is correct. (I suspect some of the Task Force recommendations are independent of this assumption, however.)
(d) The section on "The Business of Legal Education" (pp. 14-15) should probably be cut in its entirety, or substantially revised. As far as I can see, it is just flat out false that, "Law schools have long escaped pressure to adapt programs or practices to customer demands or to the pressures of business competition" (p. 14). The enormous drop in applications to law schools--which is leading schools to cut faculty, leave faculty positions unfilled, and so on--is a clear case of market pressure. The extensive curricular innovations, both in clinical offerings and in other ways (such as Chicago's Business Leadership Program), have arisen precisely in response to student interests and the needs and interests of legal employers. (As noted before, the WP doesn't really come to terms with the fact that some developments it seems not to like--e.g., lower faculty teaching loads--have also arisen precisely from market competition.) If what the WP is driving at is that it thinks law schools are simply "businesses," and not academic institutions, then let it come out and say so and defend that view. I agree that a law school could simply be a business venture, as the rise of for-profit law schools well illustrates (notice, of course, that these schools are widely criticized and derided). Let the ABA deregulate, and we will see which model the "market" actually favors. (And let's also be clear that markets are good at only one thing, namely, figuring out what people want; but we have known since Socrates [if not before!], that not everything people want is what they need. And some things individuals want can only be provided if society as a whole restrains the satisfaction of wants [as the WP recognizes in its discussion of law as a public good]. That is why every successful human civilization has some institutions that are insulated, in some measure, from the market [e.g., churches, universities, charities, public libraries, etc.] And here's what happens, of course, when the 'market' goes awry.)
(e) Although the WP correctly repudiates the absurd and irrational criticisms of law schools that have proliferated in cyberspace, it still evinces a curious obsession with what it calls "law school culture." For example, the WP claims, bizarrely, that, "Law schools' culture is at the root of an enormous number of current conditions...." (p. 3). But Law Schools did not cause the Great Recession or the structural changes in the legal marketplace that the WP believes are permanent and necessitate changes to legal education. The WP gives a quite sensible characterization of the faculty culture of law schools at p. 16, without even pretending to explain how any of these features explain the problems of, inter alia, unemployment of newly minted JDs . This seems a case where cyber-hysteria has exercised a pernicious influence on the WP. That becomes especially clear in the astonishing charge that the legal academy "consists of people who have sought out their positions because of a desire to avoid a market- and change-driven environment" (p. 16). This is astonishing for reasons that are, by now, clear: first, law schools are not immune to the forces of competition (just the opposite); and second, people enter law teaching for all kinds of reasons, but mostly because they like teaching and writing (law schools are always highly suspicious of candidates whose motive appears to be to "escape" practice). It is a bit scandalous for an ABA Task Force to make a fact-free pronouncement about the motives of people who go into law teaching. What were Task Force members thinking?
Since the theme of both the WP and my commentary on it has been "deregulation in the service of experimentation," let me conclude with a suggestion about how ABA regulation could make a constructive contribution: the ABA should prohibit all member law schools from participating in "evaluation" exercises by profit-making organizations, such as U.S. News. U.S. News comes in for some appropriate criticism in the WP, though perhaps not enough. The misleading and fraudulent employment data reported by law schools that I've written about for more than a decade was all brought about by U.S. News (which created an incentive to massage the numbres); U.S. News also creates the idiotic incentive to spend as much as possible, without regard to efficiency. If the ABA were to bring about an end to the U.S. News "reign of terror" it would, immediately, eliminate the worst incentive in legal education--one that trumps "law school culture" and "market insulation." By getting the idiotic U.S. News rankings out of the picture, the ABA would also make it easier for new and innovative models of legal education to flourish.
August 6, 2013
Hiring committees, 2013-14At Prawfs.
On why I post comments from colleagues "anonymously" sometimes
I sometimes post comments from colleagues elsewhere, but without attribution (I always ask them what they would prefer). I imagine the reason is obvious to most readers, but perhaps I should make it explicit, especially for the benefit of those who don't pay much attention to cyberspace.
The basic fact is that anyone these days who speaks in favor of legal scholarship and law schools, or against "scam" blogs and charlatans like Campos, is immediately subjected to a torrent of cyber-abuse, defamation, and harassment. (Anyone skeptical can simply do some searches to see the cyber-response to the sober and scholarly analysis of the economic value of a law degree co-authored by Seton Hall professor Michael Simkovic.) Cyber-vilifcation is meaningless in the end--it is the ranting of the powerless and the deranged--but it is, undestandably, shocking to most adults. Many of my correspondents would prefer not to become targets of this abuse. I have been in cyberspace too long for it to matter anymore, and because I have been out front on many of the issues that generate the most cyber-harassment, the abuse and attempted defamation is both predictable and not very credible. But this also puts me in a position to provide a forum for other academics to comment, a forum in which I can vouch for who the commenters are but at the same time insulate them from the disgusting abuse of cyber-miscreants.
August 5, 2013
Legal scholars talk about their scholarship cited by the U.S. Supreme CourtJack Chin (UC Davis) has been running a nice series of interviews with scholars whose work was cited by the Court during the most recent term--informative and interesting on many levels, including about what kind of scholarship catches the Court's attention.
Sample size, standard errors, and confidence intervals
At law school café (reposted on Tax Prof) Deborah Merritt asks several questions about The Economic Value of a Law Degree related to sample size and uncertainty. We thank Professor Merritt for her comments and hope they helped clarify the annual results for those who were having trouble interpreting Figures 5 and 6. In the paper we are careful to display the large confidence intervals for Figure 6, which looks at young law graduates over time, and we avoid drawing any strong conclusions from them. Also, as we'll discuss below, one can readily reject that Figure 5's ups and downs are just noise.
This post includes brief discussions of some of the interesting points raised.
The estimates in the paper don't depend on cyclical law school premia
We want to be clear that our underlying results do not rely on cyclicality. SIPP annual estimates do not show a recent post-recession decline in the overall law graduate earnings premium that needs to be explained. The recent decline in earnings for law graduates in our sample is matched by a decline in earnings for bachelor’s degree holders, and the law graduates retained their relative advantage. But as one can see in Figure 6, the small sample for young lawyers makes it hard to be sure about the recent outcomes for that group in isolation. Whether the premium cycles up and down or stays flat, over a lifetime every law grad will see many such transitions over their life, averaging out over time.
Is our overall sample size big enough?
Yes, our sample size is more than sufficient to support our conclusions on lifetime earnings. The standard errors in Tables 1 to 4 reflect the degree of uncertainty about our estimates, which pool data over many years to increase precision. The standard errors are very small relative to our law degree earnings premium coefficient estimates, and our results are statistically significant well beyond conventional levels of statistical significance. Deborah Merritt's discussion is focused specifically on what we can say about how the premium has changed over time (Figures 5 and 6). As one can see in Figure 5, any changes in that premium have been fairly small relative to its size.
How strong is the specific evidence from SIPP for cyclicality of earnings premiums?
Consistent with cyclicality, there is evidence of fluctuations of the earnings premium (measured on a percentage basis) in the 1996-2011 period. Prompted by Deborah Merritt's concerns, we went ahead and added the joint test statistics to the figures in question. We can reject the hypothesis that the law degree earnings premium was the same in all years from 1996-2011 (p<0.001). In other words, fluctuations in the point estimate in Figure 5 are not all simply random noise. Further, we don’t see evidence of a notable long term upward or downward trend. Indeed, despite the occasional fluctuations we think the most noticeable feature of the law school premium recently is its stability.
Several previous studies have found evidence of fluctuations in law degree holder earnings premiums and starting salaries. We cite many of these studies in the paper. It would be a bad idea to extrapolate gloom or boom from a downward or upward trend in earnings using the last few years of data. Trends, even when present, can stop or reverse themselves through dynamic labor market responses or exogenous shocks. A sustained 85 percent decline in the lifetime earnings premium would be required for our main result--that a law degree is a value-creating investment for most law graduates--to no longer hold true. Such a steep decline seems unlikely.
Though not crucial to our inqiury into lifetime earnings, it would be interesting to know if the premium rises and falls with the business cycle. Prompted by the interest in this question, we did some exploratory analysis of data from the much larger, but less precise, American Community Survey which also seems to be consistent with fairly stable earnings premiums for recent cohorts of law graduates, but more research on the question will be useful, especially as passing time provides us more data.
How should we understand confidence intervals and point estimates?
Professor Merritt’s description of confidence intervals may seem to suggest that the true population parameter is equally likely to fall close to the point estimate as it is at the outer edges at the top or bottom of the confidence interval.
This interpretation would be incorrect. The probability density is highest at the center of the confidence interval, near the point estimate, and lowest at the outer edges of the confidence interval. The point estimate is the best estimate of the population parameter.
Professor Merritt’s description also doesn't discuss the relationship between different point estimates, looking instead only at the confidence interval for each point estimate individually. In a nutshell, two estimates may have overlapping confidence intervals and still be statistically separable.
How strong is the evidence for a bi-modal distribution of earnings?
We don’t think the evidence for a bimodal distribution of lifetime earnings for law graduates is very compelling. Recent full time starting salaries from NALP are not the same thing as lifetime earnings because:
- Full time salary excludes those who are working less than full time
- Salaries exclude bonuses, which may be more variable than earnings
- Starting salaries tend to be fairly lockstep compared to later earnings
- After the JD II suggests faster growth of earnings (on a percentage basis) for graduates of lower ranked schools who have lower average initial earnings, which suggests convergence of earnings over time
Because earnings across people are close to log normally distributed it is typical to see a few people making a lot more than most people.
Would bimodality cast doubt on the results of our analysis?
Bimodality does not really call for change to our approach, even if present. As the sample gets larger the sampling distribution is asymptotically normal, so standard errors on our key results should be consistent. Regression techniques are consistent regardless of the underlying distribution, but for those concerned about a thick right tail, we'd suggest they concentrate on the results in Tables 1 and 2 that use a log transformation—reducing such concerns. Bi-modality in the earnings distribution would also not change how we did our quantile regressions. Quantile regressions estimate the earnings premium at different points in the distribution independent of the shape of the overall distribution.
August 4, 2013
The ABA Task Force Working Paper (WP) on "the Future of Legal Education": the Key Recommendations
Sections VII and VIII contain the key recommendations of the Task Force. Section VII is billed as "themes addressed to all parties," of which there are eight key ones (excluding the final recommendation that the Task Force's work be "institutionalized" within the ABA). Three strike me as excellent and overdue, namely:
*"There should be greater heterogeneity in law schools" (p. 23-24). That's certainly a theme I've mentioned in the past. There's heterogeneity not just in colleges and universities (of which there are many more), but also even in medical schools (a fact captured even by U.S. News, which ranks "research" schools separately from "teaching" schools [though the latter also do research]).
*"There should be greater heterogeneity in programs that deliver law-related education" (p. 25). This is part of a general and sensible theme in the WP, namely, that there need to be systems of certification for certain kinds of legal professionals "who are qualified to provide limited law-related services without the oversight of a lawyer" (p. 25).
*"The regulation and licensing of law-related services should support mobility and diversity of legal services" (p. 28). Again, there's no point in heterogeneity of law schools and law-related programs, if there isn't a change in regulation and licensing of those providing different kinds of legal services.
One other recommendation strikes me as sensible, but already widely recognized, namely that, "there should be clear recognition that law schools exist to teach people to provide law-related services" (p. 26). No evidence is adduced of who exactly doesn't recognize this. The Task Force should not visit the sins of, say, Yale Law School on the academy as the whole!
Two of the recommendations strikes me as sensible but largely empty as formulated:
*First, it would be great for the "financing of law-related education" to be "re-engineered," but the Task Force, by its own admission, does not really have any concrete proposals. It is true, for example, that the "current system of lending distances law schools from market considerations" and also facilitates "unfettered pursuit of status" (p. 23). But it doesn't completely eliminate market forces: schools still compete for students, for faculty, and for jobs for their graduates. Does the Task Force want to recommend no student lending for legal education? That would seem inconsistent with the idea that legal training is a public good (a theme I'll come back to in another post, but which the WP emphasizes). Here's a more tangible proposal the Task Force might adopt. Recall that it was Bush Senior's Justice Department that ended "collusion" among Ivy League schools on the nature and size of financial aid awards, which used to be overwhelmingly need-based. Perhaps it is time for the ABA to push to reverse that, to permit a bit of "collusion." There may still be perverse competition in "merit" awards by schools on the cusp of moving "up" to another peer cluster, but there would be far less of it within clusters of schools of similar stature, and thus fewer cases of the students with the weakest credentials (relative to others at a particular school) paying the most (a phenomenon Brian Tamanaha correctly diagnosed a number of years ago, and which the WP also notes).
*Second, it would surely be terrific for there to be "greater innovation in law schools and in programs that deliver law-related education" (p. 27), but that will only take place through some ABA deregulation that creates room for those innovative experiments. So this just seems like pointless exhortation, and otiose with respect to the real issue: the loosening of regulatory strictures on both the educational and licensing side of things.
Finally, two recommendations strike me as unclear or misguided.
*The suggestion that law schools need to emphasize "delivery of value to students" (pp. 25-26) is just mumbo-jumbo "management-speak," and nothing else. Law schools are well aware that they are "in the business of delivering legal education services," but they are also academic institutions, not the corner grocery. Regulatory loosening will create "competition" (the supposed panacea of all market enthusiasts), and then we will see what happens. But in an otherwise thoughtful WP, this section stands out as slightly ridiculous.
*It's hard to argue with the abstract proposition that "there should be constructive change in faculty culture and faculty work" (p. 28) But what the Task Force really seems to have in mind is that faculty should teach more and write less. The WP makes the dubious assertion that it is "entrenched [faculty] culture and structure that has led...to declining classroom teaching loads and a high level of focus on publishing and research" (p. 28). In fact, it is market competiton for faculty that has produced these trends, and if some of the changes recommended elsewhere in the report occur, perhaps this competition will abate. The WP seems largely enamored of "markets," and they should be clear that it is market forces that have reduced teaching loads. In that regard, the WP is right to recognize that "some, perhaps many, law schools will continue to operate under the current model" with respect to what the WP calls "faculty culture" (p. 28).
The "specific recommendations" of Section VIII pertain to implementation of the Section VII themes. Most seem pretty sensible (thoughsome fall into the category of mere 'exhortation'), and largely contribute to the strongest themes from Section VII. I'll just mention one where the Task Force should be cautious:
*If the Task Force would like to empower Deans who want to implement the WP's goals, then it will be a serious mistake to remove the requirement that Deans be tenured faculty. A Dean without tenure will be a Dean who can get very little done.
I do commend the Task Force for specifically noting (p. 34) the perverse effect that the use of "expenditures" data in the U.S. News rankings has had--the ABA should issue a strong statement condemning that practice. And a bold move would be for the ABA to produce its own metrics of law school quality, that would help loosen the grip that U.S. News has on many law school applicants.
In one or two follow-up posts, I'll have some more specific, critical comments about portions of the WP. But, as I said originally, this is a serious and thoughtful document, for which the Task Force deserves thanks.
UPDATE: Part II of my commentary.
August 3, 2013
Curious and misleading "defense" of Tamanaha by one of his colleagues at Wash U
I find it hard to believe that the author of this "defense" expects to be taken seriously given that he just makes so many things up. Prof. Rosenzweig writes, regarding Failing Law Schools [FLS], that, "Tamanaha has made an invaluable contribution to the academic literature and to the betterment of the world. The posting of the Simkovic & McIntyre paper should provide the opportunity to make this clear. That it has led to the exact opposite by some in the legal community has proven distressing." Prof. Rosenzweig is, remarkably, completely silent on how Tamanaha's own hostile and careless response to the Simkovic & McIntyre paper triggered the need for a systematic response by the authors to address Tamanaha's misrepresentations and mistakes. I assume Tamanaha responded as he did because he recognized that the Simkovic & McIntyre paper undermined his posture in FLS.
Even more strangely, Prof. Rosenzweig writes:
Let us recall what the state of the debate about the future of legal education looked like prior to the publication of FLS. Law “scam” blogs accusing law schools and law professors of exploiting students, a “cesspool” of threats and slurs, anonymous posts making scandalous and vicious personal attacks on individual law school faculty members, and public statements by law schools, faculty, and the ABA making it appear as if the entire legal community was oblivious to the crisis facing students graduating law school during that period....
Look at the state of the debate after the publication of FLS. Almost all public statements on the issue are now clearly attributed to their authors. Academics publicly publish data under their own names. I am assuming, since it is cited in the paper, that FLS in part led Simkovic and McIntyre to pursue their project in the first place. In other words, FLS has done precisely what the highest and best scholarship can and should do – it increased the amount of knowledge in the world at the time, led to a better and more informed debate, and began the process of replacing emotion and opinion with facts and analysis.
I must say this is pure fiction from top to bottom. It omits, for example, the active role that Tamanaha played in legitimating a number of deranged "scam" blogs that were, and are, still "'cesspools' of threats and slurs" with "anonymous posts making scandalous and vicious personal attacks on individual law school faculty members." (If anything, they've gotten worse since Tamanaha's book, and are even more visible.) He did this by referencing them favorably in his book and, more remarkably, by sometimes posting encouraging comments on some of them. For this alone, he would deserve condemnation by his professional colleagues, even before we get to the damage done to the debate through the carelessness of significant parts of Failing Law Schools (some of that has come out in the recent debate, but when the detailed review of the book, and its reckless allegations, by Simkovic & McIntyre goes on SSRN, this will be clear to all).
But I do agree with Rosenzweig, as I said previously, that FLS collects good anecdotes, has an interesting (and unflattering) history of the regulation of law schools, and sensibly recommends a lighter regulatory hand to permit more experimentation with models of legal education. Its anecdotal approach to systematic issues, however, has seriously distorted the discussion of those issues, as the Simkovic & McIntyre paper makes clear.
I note, finally, that Prof. Rosenzweig, like so many, doesn't know the meaning of ad hominem. Prof. Simkovic was the victim of many ad hominem smears after his paper came out; Prof. Tamanaha has been spared them entirely.
UPDATE: Paul Horwitz (Alabama) has found a nice example of the "state of the debate" after Tamanaha's intervention--this from a "scam" blog on which Tamanaha has posted encouraging comments, I should add.
August 2, 2013
Working Paper by the ABA Task Force on Future of Legal EducationIt's now out and available here. I'll have more detailed comments over the next few days, though having read it through once, I think it's clearly a serious and thoughtful piece of work, though some of it is, for reasons I will explain, misguided. Many of the concrete proposals are sensible ones, but again, I'll go into details over the next week or so.
More on the Law School Transparency "shakedown" of law schools
A Dean at another law school writes:
It is fair to say this was not a very good move on LST's part, and shatters whatever remaining credibility they had. (They've been in a bit of a downward spiral.) Now that the ABA (finally) mandates detailed reporting of employment outcomes, it's not really clear that LST serves a useful purpose any longer, and I will be astonished if any school actually ponies up for this nonsense "certification." (By the way, it is perhaps telling of what's happened to LST that the law professor who sent me the information about the certification shakedown did not want to write about it himself/herself for fear that LST would then go after that professor's school!)
Re the McEntee post: Yes, in fact, Kyle has sent out the following note to a number of deans:
...We're introducing a certification program for law schools.
The program has two purposes. First, we want to increase the quality and consistency of consumer information. Second, we want to help the schools that are transparent (as defined by us of course) signal that transparency. Law schools across the board are facing declining trust, even the good actors. I'm of the opinion that this is bad for the profession in the long run, and that a program like this can help instill a sense of trust where trust is deserved.
The short of what we'll do: Schools that meet our two reviews become "LST Certified," which entitles them to use a certification mark to signal their commitment to transparency. The mark provides assurance to prospective students, students, and the public that your school does things the right way. The fee for the first year is $1925. There will be audits for compliance throughout the year -- sometimes at a defined time, sometimes randomly.
One review is for Standard 509 compliance. The other review is for LST Best Practices. The latter requires that schools produce various consumer information (employment data, financial aid, etc) on their websites. Sometimes the Best Practices require a certain form; usually we just check that certain data or statements are present. Importantly, we require schools to centralize all consumer information for easy access and require that schools indicate info about definitions and methodologies used.
---end of quote from letter--
This is, of course, absolutely outrageous. One dean colleague said it smacks to him as a violation of the Hobbs Act (which prohibits extortion and such). This is, any way you slice it, an extraordinary effort at shaking down law schools by promising some version of “certification” at a price – this from a group which decries, among other things, the high costs of law schools. It is one thing for the deans to call Kyle out. I’d like to see where the scambloggers are in all this.
August 1, 2013
The Economic Value of a Law Degree: Correcting Misconceptions
- Ability sorting and selection
- Occupation and the versatile law degree
- Long term versus short term
- The broader labor market
- Present value and opportunity costs
In The Economic Value of a Law Degree, Frank McIntyre and I estimate the increase in annual and lifetime earnings that is attributable to a law degree. To do so, we compare those with law degrees to similar individuals with less education.
Because those who matriculate at law schools may be different from the average bachelor’s degree holder, we compare law degree holders to a group of similar bachelor’s degree holders.
There is a misperception—apparently started by Brian Tamanaha (here and here) and repeated by others—that we simply compare law degree holders to all bachelor’s degree holders, or that we compare the 25th percentile of law degree holders to the 25th percentile of all bachelor’s degree holders. This is not true.
At a high level, what we essentially did was to create two subgroups of bachelor’s degree holders—all bachelor’s degree holders, and a subset of bachelor’s degree holders who look like the law degree holders with respect to many observable characteristics that predict earnings—demographics, academic achievement, parental socio-economic status, measures of motivation and values. It is this second group of bachelor’s degree holders that we compare to the law degree holders.
To check for ability sorting and selection, we use statistical techniques including:
- Ordinary Least Squares (OLS) regression (at the mean)
- Quantile Regression at the:
- 25th percentile
- 50th percentile
- 75th percentile
- Propensity score matching (for our lifetime earnings premium estimates)
- Heckman Selection (in an appendix)
The observable characteristics (pretreatment covariates) that we focus on as controls in the Survey of Income and Program Participation include:
- Number of years of high school coursework in
- Foreign Language
- Type of High School
- Private vs. Public
- College preparatory classes in high school
- College major (divided into five categories
based on the International
Standard Classification of Education)
These controls bring down our earnings premium estimates by around 10 percent at the mean and around 8 percent at the 25th percentile.
In other words, the data and statistical techniques that we use suggest that the kinds of people who go to law school would probably earn about 10 percent more than the average bachelor’s degree holder even if they hadn’t gone to law school. But the law school earnings premium is much greater than that, and the earnings premiums we report are after controls for ability sorting.
We do an additional check for ability sorting using another data set called the National Education Longitudinal Study (NELS). NELS follows a cohort from 8th grade through their late 20s, and includes additional pretreatment control variables that are not available in SIPP.
Controls that are available in NELS include:
- college quality
- standardized test scores
- college GPA and major
- motivation and interest in careers
- subjective expectations about future income
- Parent SES
The results of the analysis using NELS are very similar to the results of the analysis in SIPP. The bachelor’s degree holders who go on to law school would probably earn about 10 percent more than the average bachelor’s degree holder, even if they had not gone to law school.
Because this level of ability sorting was already taken into account in our SIPP analysis, we do not believe that any further adjustment to our SIPP results would be justified based on the analysis in NELS. Because different measures of ability that predict earnings are often correlated with each other, adding more and more control variables that measure essentially the same thing often won’t substantially change the estimate of the earnings premium.
Thus we found very little to suggest that law graduates’ above average undergraduate academic performance translates into higher earnings other than what we had already accounted for. This may be surprising to people for two reasons. First, law degree holder undergraduate academic performance is better but not fantastically better than the typical BA. Second, that above average performance does not actually translate into much of a boost to earnings. It turns out higher undergraduate grades, for example, do not show a strong correlation with later earnings. We find that this is especially true, by the way, in the majors preferred by law students in the humanities and social sciences.
Eric Rasmusen has an interesting blog post qualitatively describing the "typical" law student.
There are several other issues related to selection on unobservables and offsetting biases that are worth mentioning.
Annual vs. Lifetime and regression to the median:
Annual earnings tend to be much more varied than longer-term lifetime earnings. For one example, job losses or transitions can cause a sharp drop in one year, but tend to be resolved by the next year. People going through such temporary rough spots show up low in the earnings distribution. So the 25th percentile of one year earnings is much lower than the 25th percentile over average lifetime earnings.
When reporting earnings, people tend to not report periods of unemployment and such. The SIPP returns to interview people every four months, so this is not as much of a problem as it could be, but it means that low income people tend to over-report their income relative to those higher up. This typically will bias down estimates of how much more one group earns than another.
People tend to pick the career they will succeed at. Thus those who are bad at some jobs but good at jobs available to law degree holders will gravitate towards law. But, in fact, had they not gone in to law they might end up doing very badly. This has several effects – it means that we will tend to underestimate the value of law school to those who choose law because that is their particular advantage but at the same time we may be overestimating it for those who are not choosing law. It is hard to know for sure if this is a large effect or not. It is very difficult to nail down statistically.
The 25th Percentile:
When we look at the 25th percentile earnings lawyer we use quantile regression to make these ability adjustments to the data before comparing them to the 25th percentile earnings BA, thus we’re correcting for ability as much as possible. Though not reported in the paper we find the ability gap (that we adjust for in our lifetime value estimates) between BA and law grads is about eight percentage points at the 25th percentile. This is completely in line with what we found at the mean both in the SIPP and in our more refined estimates from the NELS survey. It is possible that the gap is larger (or smaller) at the bottom than our data show, so that would be a great place for future research, but we think this is the best currently available estimate, especially given issues (1) and (2) biasing the premium down.
Occupation and the versatile law degree
A very large fraction of law degree holders do not end up practicing law. For some, this is a disappointment and for others it is a preferred outcome. We include all these people in our estimates of the value of a law degree. That is because the question we are interested in answering is the value of the law degree, not the earnings of the subset of individuals who practice law. Controlling for occupation would have been methodologically improper because occupation is an outcome variable, not a pretreatment covariate.
As MIT labor economist Joshua Angrist and LSE labor economist Jörn-Steffen Pischke explain in Mostly Harmless Econometrics:
Some variables are bad controls and should not be included in a regression model even when their inclusion might be expected to change the short regression coefficients. Bad controls are variables that are themselves outcome variables . . . That is, bad controls might just as well be dependent variables too. The essence of the bad control problem is a version of selection bias . . .
To illustrate, suppose we are interested in the effects of a college degree on earnings and that people can work in one of two occupations, white collar and blue collar. A college degree clearly opens the door to higher-paying white collar jobs. Should occupation therefore be seen as an omitted variable in a regression of wages on schooling? After all, occupation is highly correlated with both education and pay. Perhaps it’s best to look at the effect of college on wages for those within an occupation, say white collar only.
The problem with this argument is that once we acknowledge the fact that college affects occupation, comparisons of wages by college degree status within an occupation are no longer apples-to-apples, even if college degree completion is randomly assigned . . . [because of selection bias].
We would do better to control only for variables that are not themselves caused by education.
In a recent article, David Neumark and co-authors also include a helpful explanation of the problems with controlling for occupation and “underemployment”, or relying on BLS occupational earnings projections when trying to measure education earnings premiums:
“For nearly every occupational grouping, wage returns are higher for more highly-educated workers even if the BLS says such high levels of education are not necessary. For example . . . for management occupations, the estimated coefficients for Master’s, professional, and doctoral degrees are all above the estimated coefficient for a Bachelor’s degree, which is the BLS required level. . . ..
If the BLS numbers are correct, we might expect to see higher unemployment and greater underemployment of more highly-educated workers in the United States. As noted earlier, we do not find evidence of this kind of underemployment based on earnings data. Similarly, labor force participation rates are higher and unemployment rates are lower for more highly educated workers.”
Even economists at the BLS emphasize that educational earnings premiums, and not BLS employment projections, are the key measure of the value of education:
The general problem with addressing the question whether the U.S. labor market will have a shortage of workers in specific occupations over the next 10 years is the difficulty of projecting, for each detailed occupation, the dynamic labor market responses to shortage conditions. . . .
Since the late 1970s, average premiums paid by the labor markets to those with higher levels of education have increased.
It is the growing distance, on average, between those with more education, compared with those with less, that speaks to a general preference on the part of employers to hire those with skills associated with higher levels of education.
Long term versus short term
We value a law degree based on the present value of a lifetime of increased earnings. The valuation literature is unambiguous about the correct time period to value the cash flows generated by an asset: the entire life of the asset. The delay and higher risks of cash flows in the distant future are already taken into account through the application of a discount rate and the present value formula.
Our approach, using the typical span of a working life and discounting back to present value, is the correct one for the majority of potential law students who obtain their degrees relatively early, in their 20s or 30s. A much shorter time period would only be appropriate for individuals who complete their law degrees later in life, closer to retirement, or who anticipated working only a few years during their lifetimes.
In a recent post post, Brian Tamanaha suggests that the difference between his approach and ours is that he focused on the short-term value of a law degree while we focused on the long-term value of a law degree.
Michael Froomkin wonders if law degree holders will experience a cash crunch early in their careers when their incomes are lower and debt levels are higher.
It is unlikely that a debt financed law degree would create a cash crunch. Young bachelor’s degree holders also have lower incomes early in their careers. The earnings premium associated with the law degree will typically exceed required debt service payments on law school debt, particularly in light of the availability of extended repayment, deferment, forbearance, and income based repayment plans. Graduate degrees can readily be financed entirely with federal student loans.
The costs of delayed repayment (i.e., higher interest) are already taken into account in our present value calculation, because we discount back at the weighted average interest rate on law school debt. We’re pretty conservative in this respect: we ignore the (likely) possibility that students will prepay their highest interest rate debts first. Indeed, After the JD II found evidence of rapid pre-payment of law school debt.
Our results suggest that most young law degree holders most of the time likely have more positive cash flow—even after debt service payments—than they would likely have had with only a bachelor’s degree.
Because the economic value of a given level of education can generally be maximized by completing that level of education early—and thereby maximizing the number of years of subsequent work with the benefit of higher wages from the education earnings premium—delaying graduate school to try to time the market is a high-cost strategy. And timing the market three or four years in advance is difficult.
We recommend long-term historical data on lifetime earnings premiums as a guide rather than short-term fluctuations in starting salaries. Indeed, starting salaries tell us very little—earnings premiums are what matters, and there is no evidence that premiums have compressed, even for the young.
In a supplemental exploratory analysis using ACS data, we find some evidence that post 2008 cohorts of individuals who are probably young law degree holders (professional degree holders excluding those in medical practice) continue to have the same earnings advantage over bachelor’s as they had prior to 2008.
Ben Barros has done some interesting work comparing outcomes 9 months after graduation to subsequent outcomes for recent graduates of Widener Law School.
The broader labor market
Tamanaha argues that law continues to be depressed while the rest of the labor market has recovered. The data does not support this view. As can be seen from the chart below, the broader employment population ratio remains below 2007 levels across levels of education, and the more educated continue to be more likely to work than those with less education.
Present value and opportunity costs
Many of our critics have made mistakes relating to net present value, opportunity costs, and direct costs of a law degree. Some general guidelines are provided below.
- Everything has to be discounted back to the start of law school
- Costs can't be something that is already taken into account through opportunity cost of lower in school earnings
- Costs have to be something that the law student would only incur for law school and not matched by any other comparable expense if the student were a working BA; the cost has to be something that is a necessary expense to attend law school
- The cost can't provide consumption benefits that justify the greater expense
- The cost has to be what the student actually spends, and not hypothetically what a student might have spent if the student had paid full price
For example, since living expenses would be paid out of higher earnings if law students were working, we have already taken cost of living into account.
Since many students receive scholarships and grants, full-sticker tuition should not be used as a base-case.
Our estimates of in-school earnings are based on data from the SIPP and other Census Bureau Surveys. As we note in footnote 101:
Footnote 101: We assume that law students earn $5,000 in their first year, $7,000 in their second year and $12,000 in their third year with part time and summer work, for a total of $24,000 during law school. SIPP data suggests typical three-year in-school earnings between $21,800 (median) and $48,000 (mean) for fulltime graduate and professional school students. Census data suggests substantial work hours among fulltime graduate and professional students See Jessica Davis, U.S. CENSUS BUREAU, SCHOOL ENROLLMENT AND WORK STATUS: 2011 (Oct. 2012).”
Thanks and Goodbye
It’s been a fun couple of weeks. We’d like to thank Brian Leiter, Brian Tamanaha, and others for the wonderful opportunity they’ve given us to explain our research to a wider audience. And I’d like to thank Frank McIntyre for his contributions to this post and previous posts. This will hopefully be our last post about The Economic Value of a Law Degree, at least for a little while.