June 18, 2016
New York Times reporter Noam Scheiber was kind enough to respond to my open letter and ask if I could point to anything specifically factually wrong with his story. My response is below.
Thanks so much for responding. Yes, there are at least 6 factual errors in the article, and several misleading statements.
I’ll start with my interview with Acosta from earlier today, and then we can discuss empirics. Here’s what Acosta said:
"There’s no way I could pay back my student loans under a 10-year standard payment plan. With my current income, I can support myself and my family, but I need to keep my loan payments low for now. I’ve been practicing law since May, and I’m on track to make $40,000 this year. I think my income will go up over time, but I don’t know if it will be enough for me to pay back my loans without debt forgiveness after 20 years. What happens is up in the air. I’m optimistic that I can make this work and pay my student loans. I view the glass now as half full.
Valparaiso did not mislead me about employment prospects. I had done my research. I knew the job market was competitive going in. I knew what debt I was walking into. I think very few Americans don’t have debt, but for me it was an investment. I saw the debt as an investment in my career, my future, and my family.
Valparaiso gave a guy like me, a non-traditional student a shot at becoming a lawyer. Most law schools say they take a holistic approach, but they don’t really do it. I had to work hard to overcome adversity, and they gave me a shot to go to law school and to succeed. They gave me a shot at something that I wanted to do where most law schools wouldn’t.
My situation might be different from other law students who start law school right out of college. I was older and I have a family to support."
On to empirics.
The story states that:
“While demand for other white-collar jobs has rebounded since the recession, law firms and corporations are finding that they can make do with far fewer full-time lawyers than before.”
This is incorrect.
First, the number of jobs for lawyers has increased beyond pre-recession levels (2007 or earlier), both in absolute terms and relative to growth in overall employment. (error #1)
Focusing only on lawyers working full-time in law firms or for businesses (I’m not sure why you exclude those working in government), there are more full-time corporate and law firm lawyers in 2014 according to the U.S. Census Bureau’s Current Population Survey (CPS)—870,000—than in 2007—786,000. There have been more full-time corporate and law firm lawyers in every year from 2009 on than there were in 2007 and earlier.
You were looking at NALP or ABA data, which is measured at a single point in time—9 or 10 months after graduation—and is therefore much less representative of outcomes for law graduates—even recent law graduates—than Census data. Indeed, many law graduates who will eventually gain admission to a state bar will not have done so as of the date when NALP collects data. NALP and the ABA also use different definitions from the Census, so you cannot readily use their data to compare law graduates to others.
The trend of growth in lawyer jobs holds true for other cuts of the data (all lawyers; all full time lawyers) using other data sources—U.S. Census or Department of Labor (BLS OES) data.[i]
This is in spite of large declines in law school enrollments, which would be expected to reduce the number of working lawyers.
Second, employment has not rebounded to pre-recession (2007 or earlier) levels outside of law. (error #2)
June 17, 2016
An Open Letter to New York Times Journalist Noam Scheiber: Journalists Should Consult Peer-Reviewed Research, Not Bloggers (Michael Simkovic)
Dear Mr. Scheiber:
Have you seen this line of peer-reviewed research, which estimates the boost to earning from a law degree including the substantial proportion of law graduates who do not practice law?
- Michael Simkovic & Frank McIntyre, The Economic Value of a Law Degree, 43 J. Legal Stud. 249 (2014)
- Michael Simkovic & Frank McIntyre, The Economic Value of a Law Degree (2013)
- The Economic Value of a Law Degree PowerPoint Presentation
High quality nationally representative data from the U.S. Census Bureau, analyzed using standard and widely accepted econometric techniques, shows that even toward the bottom of the distribution, the value of a law degree (relative to a terminal bachelor’s degree) is much greater than the costs.
All of the data suggests that this has not changed since the financial crisis. The economy is worse and young people are facing more challenges in the job market, but law graduates continue to have the same relative advantage over bachelor’s degree holders as they have had in the past:
These findings have been covered in the New York Times before:
- Michael Simkovic, Overall Stagnation in Legal Jobs Hides Underlying Shifts, The New York Times Dealbook, April 1, 2016
- Steven Davidoff Solomon, Law Schools and Industry Show Signs of Life, Despite Forecasts of Doom, The New York Times Dealbook, March 31, 2015
- Steven Davidoff Solomon, Debating, Yet Again, the Worth of Law School, The New York Times Dealbook, June 18, 2013
Data from the U.S. Census and the Department of Labor Bureau of Labor Statistics shows that the number of lawyers has grown since the financial crisis, both in absolute terms and relative to overall employment.
Data from the Department of Education shows that law school graduates, even from very low-ranked law schools, have exceptionally low student loan default rates.
I have a number of concerns about factual inaccuracies in your recent story, “An Expensive Law Degree, and No Place to Use It” and your reliance on “experts” such as Paul Campos who lack any technical expertise or even basic financial or statistical literacy.
Your readers would receive more reliable information if you concentrated less on sources like Paul Campos and internet “scamblogs” and focused instead on peer-reviewed research by professional economists using high quality data and well-established methods of statistical analysis.
June 18, 2016: Noam Scheiber replies and I respond by re-interviewing Acosta and pointing out specific factual errors in Scheiber's story.
June 20, 2016: I explain different data sources that are useful for counting lawyers.
June 21, 2016: Steven Davidoff Solomon weighs in at N.Y. Times Dealbook, citing my research and supporting my points.
June 21, 2016, 10:05pm EST: Noam Scheiber sent a lengthy response by email and posted his response to his facebook page. Scheiber informs me that his response was reviewed by his editors at the New York Times.
June 24: I responded to Scheiber and explain Why The New York Times Should Correct The Remaining Factual Errors in Its Law School Coverage. In response, the New York Times posted a correction to the most minor of the 5 remaining errors.
June 09, 2016
Journalism researcher: To correct misinformation, essential to monitor and respond immediately (Michael Simkovic)
Scholars Strategy Network's No Jargon: 13: The Misinformation Age
Professor Brian Southwell explains why people tend to believe false information and discusses strategies for correcting the public perception of misinformation. Southwell is a professor of Mass Communication at University of North Carolina at Chapel Hill.
May 05, 2016
Lawyers traditionally bill a specified hourly rate for the time they spend working on a case. This ideally incentivizes lawyers to work hard and improve outcomes for their clients, and it provides clients transparency with respect to lawyer effort.
However, an hourly rate can reduce the predictability of costs for clients. Some clients worry that hourly rates might encourage inefficient over-work. As a result, some have shifted toward fixed-fee arrangements for their legal services, in which lawyers are paid a flat fee for completion of a task, regardless of how much time it takes to complete.
Preliminary results from empirical research that will be presented at this years’ American Law & Economics Association Conference suggest that a fixed-fee approach to compensating lawyers reduces lawyers’ efforts to assist clients and leads to worse outcomes for clients.
Two separate studies by two groups of researchers using similar research designs with different data sets both come to substantially the same conclusions. (Benjamin Schwall, High-Powered Attorney Incentives: A Look at the New Indigent Defense System in South Carolina and Amanda Y. Agan, Matthew Freedman & Emily Owens, Counsel Quality and Client Match Effects).
One potential obstacle in assessing the effects of different billing practices is reverse causation. Better lawyers may normally be able to bill by the hour because they are better and have more power to negotiate, not because billing by the hour makes them better.
The studies control for differences in lawyer quality by looking at the same lawyers (lawyer fixed effects) sometimes as court–appointed attorneys paid a flat fee and sometimes as attorneys billing by the hour. Schwall’s paper exploits changes in how South Carolina compensates its public defenders, while Agan, Freedman & Owen focus on random assignment of criminal defense counsel in Texas. The studies also attempt to control for differences in the type of case and defendant characteristics. The research designs for causal inference appear to be rigorous, and the results seem intuitive and plausible.
While the context of these studies is the criminal justice system, it would be surprising if the conclusions did not also hold true in civil litigation or in transactional practice. A lawyer on a fixed-fee is likely to be more willing to concede important points to bring a case or transaction to a speedy conclusion than one who can bill by the hour and be compensated for his or her extra efforts. Sophisticated clients may be better able to monitor their attorneys than indigent defendants and criminal courts, but clients probably cannot eliminate agency costs (If they could, an hourly rate would make at least as much sense as a fixed-fee).
Assuming the preliminary results of these studies hold, the incentive problems created by fixed fee arrangements may be an opportunity for shrewd business people or plaintiffs lawyers to target counterparties or defendants. If the businessperson pays his own lawyers by the hour to negotiate opposite lawyers on a fixed-fee, the reward could be contracts with lopsided terms in his favor. Plaintiffs’ lawyers may similarly expect civil defense lawyers on fixed-fee arrangements to advocate a swift settlement on terms relatively favorable to plaintiffs.
Lawyers are likely to know which clients use fixed fee arrangements because such clients often have an RFP process in which law firms bid for their work.
April 30, 2016
New research from Dan Schwarcz and Dion Farganis at Minnesota argues that providing students with practice problems and exercises that are similar to final exams and giving individual feedback prior to the final examination can help improve grades for first year law students.
Schwarcz and Farganis tracked the performance of first year students who were randomly assigned to sections, and as a result took courses with professors who either provided exercises and individual feedback prior to the final examination, or who did not provide feedback.
When the students who studied under feedback professors and the students who studied under no-feedback professors took a separate required class together, the feedback students received higher grades after controlling for several factors that predict grades, such as LSAT scores, undergraduate GPA, gender, race, and country of birth. The increase in grades appears to be larger for students toward the bottom half of the distribution. The paper also attempts to control for variation in instructor ability using student evaluations of teacher clarity.
It’s an interesting paper, and part of a welcome trend toward assessing proposed pedagogical reform through quasi-experimental methods.
The interpretation of these results raises a number of questions which I hope the authors will address more thoroughly as they revise the paper and in future research.
For example, are the differences due to instructor effects rather than feedback effects? Students are randomly assigned to instructors who happen to voluntarily give pre-final exam feedback. These might be instructors who are more conscientious, dedicated, or skilled and who also happen to give pre-exam feedback. Requiring other instructors to give pre-exam feedback—or having the same instructors provide no pre-exam feedback—might not affect student performance.
Controlling for instructor ability based on teaching evaluations is not entirely convincing, even if students are ostensibly evaluating teacher clarity. There is not very strong evidence that teaching evaluations reflect how much students learn. An easier instructor who covers less substance might receive higher teaching evaluations across the board than a rigorous instructor who does more to prepare students for practice. Teaching evaluations might reflect friendliness or liveliness or attractiveness or factors that do not actually affect student learning outcomes but that have consumption value for students. Indeed, high feedback professors might receive lower teaching evaluations for the same quality of teaching because they might make students work harder and because they might provide negative feedback to some students, leading students to retaliate on teaching evaluations.
These issues could be addressed in future research by asking the same instructor to teach two sections of the same class in different ways and measuring both long term student outcomes and teaching evaluations.
Another question is: are students simply learning how to take law school exams? Or are they actually learning the material better in a way that will provide long-term benefits, either in bar passage rates or in job performance? At the moment, the data is not sufficient to know one way or the other.
A final question is how much providing individualized feedback will cost in faculty time, and whether the putative benefits justify the costs.
It’s a great start, and I look forward to more work from these authors, and from others, using quasi-experimental designs to investigate pedagogical variations.
April 22, 2016
Professor Paula Franzese of Seton Hall law school is something of a patron saint of law students. Widely known for her upbeat energy, kindness, and tendency to break into song for the sake of helping students remember a particularly challenging point of law, Paula has literally helped hundreds of thousands of lawyers pass the bar exam through her video taped Property lectures for BarBri.
Paula is such a gifted teacher that she won teacher of the year almost ever year until Seton Hall implemented a rule to give others a chance: no professor can win teacher of the year more than two years in a row. Since the rule was implemented, Paula wins every other year. She’s also incredibly generous, leading seminars and workshops to help her colleagues improve their teaching.
Paula recently wrote a book encouraging law students to have a productive, upbeat happy, and grateful outlook on life (A short & happy guide to being a law school student).
Paula’s well-intentioned book has rather bizarrely been attacked by scambloggers as “dehumanizing”, “vain”, “untrustworthy” and “insidious.” The scambloggers are not happy people, and reacted as if burned by Paula’s sunshine. They worry that Paula’s thesis implies that “their failure must be due to their unwillingness to think happy and thankful thoughts.”
Happiness and success tend to go together. Some people assume that success leads to happiness. But an increasing number of psychological studies suggest that happiness causes success. (here and here) Happiness often precedes and predicts success, and happiness appears to be strongly influenced by genetic factors.
Leaving aside the question of how much people can change their baseline level of happiness, being happier—or at least outwardly appearing to be happier—probably does contribute to success, and being unhappy probably is a professional and personal liability.
People like working with happy people. They don’t like working with people who are unhappy or unpleasant. This does not mean that people who are unhappy are to blame for their unhappiness, any more than people who are born with disabilities are to blame for being deaf or blind.
But it does raise serious questions about whether studies of law graduates’ levels of happiness are measuring causation or selection. We would not assume that differences between the height of law graduates and the rest of the population were caused by law school attendance, and we probably should not assume that law school affects happiness very much either.
April 01, 2016
March 16, 2016
Statistician and data visualization expert Hans Rosling recently took the media to task for misleading readers and viewers using unrepresentative anecdotes and ignoring contradictory data.
Rosling says "You can't trust the news outlets if you want to understand the world. You have to be educated and then research basic facts."
While journalists often depict the developing world as full of "wars, conflicts, chaos" Rosling says "That is wrong. [The press] is completely wrong.. . . You can chose to only show my shoe, which is very ugly, but that is only a small part of me. . . . News outlets only care about a small part but [they] call it the world."
Rosling complains that the slow but steady march of progress is not considered news.
Rosling is famous for his data visualizations, especially this video briefly illustrating 200 years of global progress toward health and prosperity. It's optimism for the data-driven set (and is a big hit in my business law classes).
March 05, 2016
That’s the question Frank McIntyre and I try to answer in Value of a law degree by College Major. Economics seems to be the “best” major for aspiring law students, with both high base earnings with a bachelor’s degree and a large boost to earning with a law degree. History and philosophy/religion get a similarly large boost from a law degree but start at a lower undergraduate base and, among those with law degrees, typically end up earning substantially less than economics majors.
The abstract and a figure are below:
We estimate the increase in earnings from a law degree relative to a bachelor’s degree for graduates who majored in different fields in college. Students with humanities and social sciences majors comprise approximately 47 percent of law degree holders compared to 23 percent of terminal bachelor’s. Law degree earnings premiums are highest for humanities and social sciences majors and lowest for STEM majors. On the other hand, among those with law degrees, overall earnings are highest for STEM and Business Majors. This effect is fairly small at the low end of the earnings distribution, but quite large at the top end. The median annual law degree earnings premium ranges from approximately $29,000 for STEM majors to $45,000 for humanities majors.
These results raise an intriguing question: should law schools offer larger scholarships to those whose majors suggest they will likely benefit less from their law degrees? Conversely, should law schools charge more to those who will likely benefit the most?
Figure 3: ACS Mean Earnings for Professional Degree Holders (Narrow) by Selected Field of Study* (2014 USD Thousands)
- Includes degree fields with more than 700 professional degree holders in sample.
COMMENT FROM BRIAN LEITER: The lumping of philosophy majors together with religion invariably pulls down the performance of philosophy majors!
February 22, 2016
The New York Times and American Lawyer have both reported on the decision of the distinguished New York law firm Milbank Tweed to pull funding for speakers invited by student groups at Harvard Law School after one group invited a speaker of which, apparently, the firm disapproves. This sets a very unfortunate precedent, and I would encourage other law professors to join me as a signatory to this open letter. Law schools depend on generous support from alumni, whether as individuals or as partnerships or otherwise, but that support should be offered in a way consistent with the academic and pedagogical mission of a law school and be viewpoint-neutral.