Brian Leiter's Law School Reports

Grade point averages and law school rankings (Michael Simkovic)

One area of disagreement among law schools and between U.S. News and some of the law schools boycotting its rankings turns on whether incoming students’ undergraduate grade point averages (GPAs) should be included in the rankings, and the extent to which they should be weighed.

The case for including grades initially seems similar to the case for including standardized test scores:  Grades are useful (but incomplete) predictors of future academic and job performance.  They are therefore useful to employers, who are important consumers of the rankings.  Employers cannot recruit in person at every law school, so they prioritize a “target rich environment.”  Moreover, students may benefit from “peer effects” if their classmates are strong academic performers. 

But there are serious problems that are unique to grades: Grades are not standardized.   Grades vary by academic institution, by field or major, by individual courses, by class year, and by instructor.  Moreover, because the other students in the class are at varying levels of ability, and students are graded against each other, the relationship between grades and ability depends on the capabilities of the other students in the class.  Even when average grades are the same, some instructors prefer to give a narrower range of grades, while others prefer to give both high and low grades.

One very serious problem with this lack of standardization is that fields of study that tend to have a relatively low value in the labor market—i.e., that tend to produce graduates with lower earning potential and with lower rates of employment—generally attract students by offering higher average grades for fewer hours of homework to students with lower standardized test scores.  For example, at a given institution, a student who would only earn a “C” in Engineering might earn a “B” in Economics and an “A” in Dramatic Arts, but the reverse is generally not true. 

This difference in grading across fields solves problems for colleges and universities.  It is expensive for colleges to recruit qualified faculty to teach courses in areas that are in high demand in the labor market.  This is because potential faculty members are more likely to have attractive outside offers.  Even when faculty members in high-demand fields are hired, their teaching time may still be scarce because of grant-funded research, clinical time, and other non-teaching obligations. 

Colleges and universities do not need to fully match outside compensation because of differences in job quality in academe versus the private sector.  But they do need to pay faculty in certain fields more than faculty in others.  Thus, professors of Engineering and Medicine earn far more than professors of English and Social Work.  Certain fields of study can also be more expensive for universities because of needs for equipment or facilities. 

Universities with tenured faculty may have a specialized workforce that cannot rapidly adjust to changes in employer needs and parallel changes in student demand.  For example, even if the demand for workers who are knowledgeable about theology declines while the demand for workers who are knowledgeable about technology increases, colleges cannot easily retrain their professors of religion to teach electrical engineering. 

Rather than limit access to costlier majors by directly charging higher tuition, universities instead subtly channel students away from costlier fields, and towards those with excess capacity, by tolerating differences in grades across fields.  This results in many students who initially report that they plan to major in a STEM field getting low grades in their STEM classes and then switching to less challenging majors (i.e., those that offer higher grades for less work). 

However, these differences in grading create problems for students.  Students mistakenly assume that they are “bad” at STEM and “good” in the humanities because they get bad grades in the former and good grades in the latter.  Students dramatically underestimate the differences in earnings potential and employment prospects by major, especially students from poorer and less well-educated families.  Fully informed students would be less likely to switch majors.

A law school ranking system that uses raw undergraduate GPA—without adjusting for differences in grades by major or institution—encourages law schools to also use raw GPA in admissions.  This further encourages undergraduates considering going to law school to hunt for institutions and courses that will give them the best chance of earning the highest grades, and to avoid academically challenging themselves or taking risks.

This might be beneficial to students and employers if high-GPA-majors lined up well with those that best prepared students for law school and subsequent employment.  But this is unlikely.

Continue reading

Standardized test scores and law school rankings (Michael Simkovic)

One area of disagreement among law schools and between U.S. News and some of the law schools boycotting its rankings turns on whether students’ standardized test scores should be included in the rankings, and the extent to which they should be weighted.

The case for including standardized test scores, and weighting them heavily, is as follows:  Standardized test scores are useful (but incomplete) predictors of future academic and job performance.  They are therefore useful to employers, who are important consumers of the rankings.  Employers cannot recruit in person at every law school, so they prioritize a “target rich environment” full of high-scoring law students who are likely to accept an offer to work for them. 

Standardized tests were first developed in the West by the British military to improve the quality of Naval officers commanding ships.  Ships are expensive and are prone to being lost or destroyed when commanded by less competent officers.

Prior to the development of standardized test scores, Naval officers were selected through an interview process and soft assessment of goodness of fit, or offices could be purchased.  This led to officers being judged by their physical appearance, race, sex, social class, and manners, none of which were actually useful predictors of performance as a Naval officer.

Once standardized tests were developed and used for selection of officers, the quality and performance of British Naval officers improved dramatically.  Today, the United States military continues to use a standardized test, the AFQT, to weed out individuals who would likely be poor performers.  It also uses a broader standardized test, the ASVAB, to find talented individuals who could serve in specialist roles.  The Navy and Coast Guard, like the British Navy before them, have higher test score requirements than other branches of the military.

Continue reading

“Unemployment” and law school rankings (Michael Simkovic)

One area of disagreement among law schools and between U.S. News and some of the law schools boycotting its rankings turns on whether “unemployment” should be defined according to the standard definition used by the U.S. government, the Department of Labor, the Census Bureau, and the Department of Education.  The government defines unemployment narrowly to apply to those who are both not employed and actively seeking employment.  Those who are not employed and not seeking employment are considered to be “not participating in the labor force” rather than “unemployed.” However, U.S. News defines unemployment more broadly to include anyone who is not currently employed, even if they are not working by choice. 

I’ve explained before why I favor the standard definition of unemployment.  Under the standard approach, law school statistics would be comparable to statistics for other educational programs.  Moreover, law schools would not be penalized for admitting students who wish to pursue dual degrees (i.e., an MPP, MD, MBA or PhD) or advanced degrees like a Tax LLM.  Nor would law schools be penalized for admitted wealthy students who do not wish or need to work or people who would prefer to stay home to care for children while their spouse works.

The only real argument against using the standard definition is that law schools will “game” the system by misrepresenting the intent of people who are not working or avoiding contacting those who they suspect are not working.  But a better solution to this problem of trust is to use a third party audit or third party data collection service rather than law school self-report. 

An audit would increase costs, but it might be worthwhile for law schools to invite the ABA to require a random audit that would include direct contact with some graduates who are not working to verify their status (unemployed vs. not participating in the labor force) in exchange for collecting data using the standard definition of unemployment. Alternatively, the ABA could require law schools to outsource the graduate employment data collection function to a neutral third party.  Schools could save money by reducing the number of internal FTE employees handling this data collection function and just paying a fee to the trusted third party.

If these solutions are not worthwhile to law schools or to ranking organizations, it may be that the total amount of fraud is small, notwithstanding a few very salient examples.  Corporate law scholars are familiar with the empirical finding that enhanced auditing under SOX (in response to Enron, Worldcom, etc.) actually reduced shareholder value, presumably because the aggregate amount of fraud was relatively low under the old, less expensive regime.

 

*. A related, but more technical problem, is how non-response to the survey is handled.  U.S. News assumes that 100% of graduates who do not respond to the employment survey (or whose employment cannot be verified through another source like an employer website) are unemployed.  By contrast, U.S. statistical agencies use the known characteristics of respondents and non-respondents (race, sex, age, location, etc.) to create weights that apply to each respondent.  These weights enable respondents to stand in for the non-respondents who have similar characteristics.  Thus, for example, if non-respondents characteristics suggest higher rates of unemployment than respondents but less than 100% unemployment, then the weights will lead to an unemployment estimate that is higher than using an unweighted average of respondents, but lower than assuming that 100% of non-respondents are unemployed.  In other words, U.S. News's approach to non-response likely leads to higher unemployment rates for law graduates compared to the rates that would be calculated using the standard weighting methods used by U.S. statistical agencies.

School-funded jobs, long-term outcomes, and law school rankings (Michael Simkovic)

One area of disagreement among law schools and between U.S. News and some of the law schools boycotting its rankings turns on whether and to what extent school-funded jobs should count as post-graduate employment.   

The argument in favor of inclusion is as follows:  School funded jobs pay students real money.  From the perspective of students, some money is better than no money.  These jobs are of varying quality and some may enable students to gain experience or develop skills or build a professional network that will increase their chances of landing higher paid, more permanent positions.  If so, then employers as well as students would prefer that law schools subsidize these jobs.

The argument against inclusion is as follows: these jobs are menial and low paid and have little to no value to students or outside employers.  They only exist to pad schools’ employment statistics.  Students are being hired to file books in the library for minimum wage, with no training and no networking opportunities.  Moreover, offering these jobs may reduce students’ motivation to seek higher value employment.

Without careful studies by labor economists of the long-term effects of these programs on incomes and employment, it’s difficult to assess their value beyond providing a salary in the short term.

But what we can say is that, all else being equal, students would probably prefer to have the safety net of school-funded jobs and the salaries they provide.

Continue reading

Who are law school rankings meant to help? (Michael Simkovic)

Approximately 12 law schools, including Harvard, Yale, and Stanford, have refused to compile data to facilitate the U.S. News rankings. 

Statements by the deans of the protesting law schools suggest that they hope to pressure U.S. News to modify its ranking system.  This raises a fundamental question: What is the purpose of ranking law schools?

I would argue that there are three purposes of law school rankings:

• To help students choose which law schools to apply to, and which law school to matriculate to

• To help employers decide where to recruit employees and to help them compare different candidates for hire

• To place competitive pressure on law schools to use their resources to serve particular policy goals embedded in the rankings

These purposes can help guide discussion of what should ideally be included in law school rankings.  If law schools hope to displace U.S. News, they will need to agree on an alternative ranking system or several rankings systems.  This will be difficult given the collective action problems facing institutions that view each other as competitors.

The first purpose of rankings—informing students—presents an obvious problem.  Students are heterogeneous in their life goals, their idiosyncratic preferences, and their reasons for attending law school.  No one ranking system can capture these differences and serve all students well.  A school that invests heavily in preparing students to pass the bar exam may be ideal for students who would otherwise fail, but wasteful for those who are likely to pass with a few weeks of studying.  A school that pours massive resources into preparing students for the rigors of private practice may be a poor fit for students who intend to be stay-at-home parents or only work part-time after graduation.  A school that emphasizes constitutional and criminal law may be a poor fit for students who view law school as their family’s ticket out of poverty and financial insecurity and are therefore only interested in lucrative areas of law with abundant employment opportunities.  A law school that teaches only practical skills and doctrinal law may be a poor fit for those who wish to become law professors or who see law school as an opportunity to explore interesting theoretical questions. 

In other words, there should not be a single ranking of law schools, but rather multiple rankings of law schools tailored to identifiable subgroups of students, as well as tools that help match students with the right law school for them.

Continue reading

Bankruptcy Scholar Troy McKenzie to Lead NYU (Michael Simkovic)

Troy McKenzie, a highly regarded bankruptcy and civil procedure scholar, was recently appointed as the new Dean of NYU School of Law.  Dean McKenzie will be the second bankruptcy scholar to lead a top-5 law school (Douglas Baird at Chicago was the first).  McKenzie is unusual among law school deans because of his undergraduate background in chemical engineering. 

McKenzie has written about aggregate litigation in bankruptcy, the independence of bankruptcy judges, bankruptcy jurisdiction, mass torts in bankruptcy, forum shopping, and the governance of complex litigation.

McKenzie, who was born in Jamaica, will be the second Black dean of a top law school, after Chris Edley at Berkeley.

The full announcement from NYU is available here.

McCarthyist accusations deployed against law professor in Confirmation hearing (Michael Simkovic)

Cornell Law School professor (and former Davis Polk financial institutions group lawyer) Saule Omarova, was nominated by President Biden to head the Office of Comptroller of the Currency, which regulates nationally chartered banks.  Omarova is a progressive scholar of financial regulation who has written about the possibility of increased public provision of financial services, including a National Investment Authority to help finance infrastructure projects.

Such public financial institutions might sometimes compete with private financial institutions, much as public universities and k-12 schools take some business away from private universities and private schools, and as public transit takes some business away from car companies. Omarova's advocacy for public options, and some of Omarova's critiques of private financial institutions, have made her a target.

During Omarova's Senate confirmation hearing, her critics resorted to name calling and nationalist dog-whistles, with Republican Senators accusing her of being a "radical" and a "comrade" (i.e.,  a communist) out to "end banking as we know it" in part because she emigrated from a central asian country that was once part of the Soviet Union.

In what seemed like an anachronism from the worst days of the 1950s, Omarova felt compelled to defend herself by saying: “I am not a communist. I do not subscribe to that ideology. I could not choose where I was born . . . My family suffered under the communist regime.”

The Financial Times has the full story.

Senators are of course free to take exception with Omarova's proposals for an NIA or publicly backed electronic payments systems, or to disagree with her advocacy for robust financial regulation, but accusing her of being a communist is beyond the pale. 

Nobel prize in economics awarded for innovation in causal inference from observational data (Michael Simkovic)

Three renowned labor economists, David Card (Berkeley), Joshua Angrist (MIT) and Guido Imbens (Stanford Business School) shared the Nobel prize in economics for their pioneering work using observational (i.e., non-experimental) data for causal inference.  This work facilitated empirical analysis of the effects of various legal and public policy changes, which are enacted in the real world and not under laboratory conditions.  Many scholars in law & economics and empirical legal studies built on their work and relied on the techniques the prize-winners developed.  

Card is famous for a series of difference-in-differences analyses across state borders that showed that moderate increases in minimum wage often don't lead to unemployment, as had been previously believed based on economic theory and simplifying assumptions.  Card's work was met with substantial skepticism, and conflicting claims from other empiricists, but he eventually changed the conventional wisdom among economists--a triumph of empiricism over theory and of science over ideology.  Card is a co-editor of the Handbook of Labor Economics.

Angrist and Imbens developed new ways to identify Local Average Treatment Effects, such as the use of instrumental variables. Angrist is also a co-author of Mostly Harmless Econometrics, a text that is widely used to train economists, law professors with an empirical bent, and other researchers.  Imbens' methodological work is taught heavily in an empirical studies workshop run by Bernard Black at Northwestern and the late Matt McCubbins at Duke.  Imbens is also the co-author of a popular book on empirical methods, Causal Inference for Statistics, Social, and Biomedical Sciences.  

Black & McCubbin's workshop--which I highly recommend--is intended to help law professors and other researchers learn to engage in more sophisticated empirical analysis.

In widely cited work, Angrist found strong evidence that military service--specifically in Vietnam--adversely affected subsequent earnings.  Imbens and Angrist have also found strong evidence that education substantially increases subsequent earnings, using changes and variation in compulsory schooling laws. The causal relationship between education and earnings is now widely accepted among labor economists and other empiricists.

 

More highly educated populations are more likely to be vaccinated against COVID (Michael Simkovic)

Economists, sociologists, and public health researchers have long observed that more highly educated groups tend to be healthier and live longer than those that are less educated.  Debates emerged about whether increasing levels of education caused improvements in health.  Some economists argue that those who know themselves to be healthy at a young age will be more likely to pursue additional education because they expect to benefit from it more, over a longer career because their greater health enables them to work longer and harder.  Many others argued that education inculcates healthier habits--diet, exercise, sleep, medical checkups, prophylactic use, skepticism about "alternative" (non-evidence-based) medicine--and provides individuals with the literacy, numeracy and critical thinking skills to make better health-related decisions going forward.

A new study by my colleagues at USC's  Center for Economic and Social Research (CESR) finds that more highly educated populations are more likely to be vaccinated, more likely to choose to be vaccinated, believe that vaccines are more effective, and believe that the risks from vaccination are lower, compared to their less educated counterparts.  Across education levels, Asians are the most pro-vaccination group, while blacks are the least.

Continue reading

Wall Street Journal law school analysis overlooks increased diversity and lower interest rates (Michael Simkovic)

In my last post, I discussed problems with WSJ coverage of law schools. In particular the WSJ has effectively faulted law schools for broad, national declines in employment that are most likely due to macroeconomic conditions such as the financial crisis of 2007-2009 and its aftermath, to COVID, and to other broad secular trends in labor markets, not to declines specific to law graduates or the legal profession.  In challenging economic climates, although law graduates suffer along with others, they continue to do better than most, primarily because they are more highly educated.

There are additional attribution problems with the WSJ's recent coverage.  

The WSJ claims that the real value of a law degree has declined over an extended period of time and cites slow growth in lawyer starting salaries.  This is problematic for several reasons. As the analysis below explains, it is almost certainly the case that over the last 20 years, the lifetime present value of legal education has dramatically increased.  This is particularly obvious when one controls for the changing demographics of law graduates, as one should to assess the value added by law schools.  However, the increase in lifetime value of legal educaiton is largely because of lower interest rates.  Smaller contributions have been made by improvements in legal education.  Those improvements appear to have done more to broaden access for more diverse students than to increase value for students fitting demographics that were more prevalent a few decades ago (i.e., white male students from upper middle class backgrounds with high-quality K-12 and undergraduate educations).

Overview:

• How should we calculate the "economic value of a law degree"?
• Earnings premiums should control for changing demographics
• Falling Interest rates have dramatically increased the lifetime value of a law degree
• Even small percentage increases in lifetime value can offset large percentage increases in tuition
• Student loan default rates are declining nationally, and are much lower for law graduates than overall

Continue reading