Brian Leiter's Law School Reports

Probably not much new here until the New Year

I wish everyone a pleasant holiday break!

In Memoriam: Herbert Morris (1928-2022)

A longtime UCLA faculty member, there are more details here.

Grade point averages and law school rankings (Michael Simkovic)

One area of disagreement among law schools and between U.S. News and some of the law schools boycotting its rankings turns on whether incoming students’ undergraduate grade point averages (GPAs) should be included in the rankings, and the extent to which they should be weighed.

The case for including grades initially seems similar to the case for including standardized test scores:  Grades are useful (but incomplete) predictors of future academic and job performance.  They are therefore useful to employers, who are important consumers of the rankings.  Employers cannot recruit in person at every law school, so they prioritize a “target rich environment.”  Moreover, students may benefit from “peer effects” if their classmates are strong academic performers. 

But there are serious problems that are unique to grades: Grades are not standardized.   Grades vary by academic institution, by field or major, by individual courses, by class year, and by instructor.  Moreover, because the other students in the class are at varying levels of ability, and students are graded against each other, the relationship between grades and ability depends on the capabilities of the other students in the class.  Even when average grades are the same, some instructors prefer to give a narrower range of grades, while others prefer to give both high and low grades.

One very serious problem with this lack of standardization is that fields of study that tend to have a relatively low value in the labor market—i.e., that tend to produce graduates with lower earning potential and with lower rates of employment—generally attract students by offering higher average grades for fewer hours of homework to students with lower standardized test scores.  For example, at a given institution, a student who would only earn a “C” in Engineering might earn a “B” in Economics and an “A” in Dramatic Arts, but the reverse is generally not true. 

This difference in grading across fields solves problems for colleges and universities.  It is expensive for colleges to recruit qualified faculty to teach courses in areas that are in high demand in the labor market.  This is because potential faculty members are more likely to have attractive outside offers.  Even when faculty members in high-demand fields are hired, their teaching time may still be scarce because of grant-funded research, clinical time, and other non-teaching obligations. 

Colleges and universities do not need to fully match outside compensation because of differences in job quality in academe versus the private sector.  But they do need to pay faculty in certain fields more than faculty in others.  Thus, professors of Engineering and Medicine earn far more than professors of English and Social Work.  Certain fields of study can also be more expensive for universities because of needs for equipment or facilities. 

Universities with tenured faculty may have a specialized workforce that cannot rapidly adjust to changes in employer needs and parallel changes in student demand.  For example, even if the demand for workers who are knowledgeable about theology declines while the demand for workers who are knowledgeable about technology increases, colleges cannot easily retrain their professors of religion to teach electrical engineering. 

Rather than limit access to costlier majors by directly charging higher tuition, universities instead subtly channel students away from costlier fields, and towards those with excess capacity, by tolerating differences in grades across fields.  This results in many students who initially report that they plan to major in a STEM field getting low grades in their STEM classes and then switching to less challenging majors (i.e., those that offer higher grades for less work). 

However, these differences in grading create problems for students.  Students mistakenly assume that they are “bad” at STEM and “good” in the humanities because they get bad grades in the former and good grades in the latter.  Students dramatically underestimate the differences in earnings potential and employment prospects by major, especially students from poorer and less well-educated families.  Fully informed students would be less likely to switch majors.

A law school ranking system that uses raw undergraduate GPA—without adjusting for differences in grades by major or institution—encourages law schools to also use raw GPA in admissions.  This further encourages undergraduates considering going to law school to hunt for institutions and courses that will give them the best chance of earning the highest grades, and to avoid academically challenging themselves or taking risks.

This might be beneficial to students and employers if high-GPA-majors lined up well with those that best prepared students for law school and subsequent employment.  But this is unlikely.

JDs can help students from low-earning, high-GPA majors make up some lost ground.  However, JD total earnings are still typically higher among law graduates with undergraduate majors that lead to higher earnings directly out of college.  This is in spite of the fact that JD admissions at elite law schools are biased against students completing low-GPA majors.

When law school admissions channel applicants away from low-GPA majors—typically those that place a greater emphasis on quantitative skills—law schools can end up with many incoming students with low levels of numeracy and high levels of anxiety about taking law school classes that require math skills or the ability to learn a complex, rules-based system.  This can feed a perception, albeit exaggerated, that many lawyers are incompetent at math and science.  This perception undermines respect for the legal profession and the judiciary.

Raw GPA can also give an unfair advantage to students from wealthier backgrounds.  First, these students may be more able to sacrifice earning potential out of college when selecting a major.  Second, wealthier students may be able to purchase a higher GPA by attending a more expensive private institution that inflates grades to attract students and tuition revenue, rather than a state institution that is less dependent on tuition revenue and less responsive to market pressures.

Though more speculative, discriminating against quantitative fields in admissions could have other negative consequences for both law graduates and society.  It is possible, for example, that STEM- and finance-oriented skills might be more useful for corporate transactional lawyers, and less helpful for litigators. Objectively, according to government data, billable rates for corporate transactional work have grown much faster than billable rates for litigation over the last several decades.  Anecdotally, many corporate clients believe that transactional work facilitates productive collaboration, while litigation is too often coercive and destructive.  It would therefore arguably help law graduates financially, and also help society, to place quantitative undergraduate majors on an even playing field in admissions.

To summarize, grades should probably be used in law school rankings.  But those grades should ideally be adjusted by major and institution, for example based on differences in the distribution of grades across majors and institutions.  This would reduce well-documented biases in grading that cause a disconnect between grades and student ability levels, and that distort students’ choice of undergraduate major.

New Hampshire joins USNews.com rankings boycott

Dean Carpenter's statement is here.  Unfortunately for the boycott effort, there's still less than 20 schools boycotting.

Some questions for law schools joining the boycott of USNews.com rankings

Dean Rutledge at the University of Georgia (a school not joining the boycott) kindly gave permission to share some questions he has both for and about law schools that are joining the boycott:

    For those schools who claim to be “boycotting/not cooperating with/posturing with respect to” US News

1. Are they completing the reputational survey?
2. Are they affirmatively asking not to be scored in the reputational survey?

To the extent schools claim US News "compelled" them to making "undesirable" choices [e.g., merit vs. need-based aid, overweighting numerical credentials], is there any indication that they're actually adjusting their admissions and related decisions now that they've "freed" themselves from the burden?  Let's watch the ABA 509's in December 2023.

 

I've heard from a number of Deans that they continue to be deluged with emails promoting schools' achievements, including from some law schools that have announced they have joined the boycott.

Penn State-University Park law faculty are, quite understandably, opposed to any "merger" that would relocate all operations to Carlisle

They are quite correct that it will destroy any possibility of Penn State's law school maintaining a national and international presence, and will negatively affect the recruitment of faculty and students.  The flagship campus of a major state research university should have its law school on that campus.

Campbell Law joins boycott of USNews.com rankings

Statement here.   More law schools that are regularly and unfairly denigrated by USNews.com will need to join the boycott if this is to really affect the operations of USNews.com.

USLaw.com ranking boycott update #2

UVA has joined the boycott.  No word yet from Texas, Vanderbilt, or Southern California, as well as most law schools outside the "top 20."

Standardized test scores and law school rankings (Michael Simkovic)

One area of disagreement among law schools and between U.S. News and some of the law schools boycotting its rankings turns on whether students’ standardized test scores should be included in the rankings, and the extent to which they should be weighted.

The case for including standardized test scores, and weighting them heavily, is as follows:  Standardized test scores are useful (but incomplete) predictors of future academic and job performance.  They are therefore useful to employers, who are important consumers of the rankings.  Employers cannot recruit in person at every law school, so they prioritize a “target rich environment” full of high-scoring law students who are likely to accept an offer to work for them. 

Standardized tests were first developed in the West by the British military to improve the quality of Naval officers commanding ships.  Ships are expensive and are prone to being lost or destroyed when commanded by less competent officers.

Prior to the development of standardized test scores, Naval officers were selected through an interview process and soft assessment of goodness of fit, or offices could be purchased.  This led to officers being judged by their physical appearance, race, sex, social class, and manners, none of which were actually useful predictors of performance as a Naval officer.

Once standardized tests were developed and used for selection of officers, the quality and performance of British Naval officers improved dramatically.  Today, the United States military continues to use a standardized test, the AFQT, to weed out individuals who would likely be poor performers.  It also uses a broader standardized test, the ASVAB, to find talented individuals who could serve in specialist roles.  The Navy and Coast Guard, like the British Navy before them, have higher test score requirements than other branches of the military.

Some employers, like McKinsey and Google, have used their own standardized test to evaluate job applicants for the simple reason that standardized test scores are useful predictors of performance.  Meritocratic education and employment systems have historically enabled countries and companies that adopted them to punch above their weight in terms of growth compared to countries and companies that emphasized family ties, caste, class, sex, race, or ethnicity.  Compare the trajectories of Malaysia--which enshrined ethnic favoritism in its educational and political system--with neighboring Singapore, which adopted unabashed race-neutral meritocracy.

In addition, there is evidence of “peer effects” in education.  Peer effects lead students with lower standardized test scores to benefit more from education when they are placed in contact with higher scoring students.  The higher scoring students, by contrast, do not benefit from being placed in an environment with lower scoring students.  Thus, high scoring students provide benefits to low-scoring peers, but the reverse is not true.  Indeed, students with problematic behaviors, such as drinking or criminal activity, can be a negative influence on their peers.

The argument against including standardized test scores is that standardized test scores are nothing but a proxy for family income and therefore discriminate against under-privileged students.  But this argument is wrong. 

Standardizes test scores are not close proxies for household income.  Within the United States, the correlation between math SAT scores and household income is only 0.22. This means that parental income explains less than 5 percent of the variation in standardized test scores (0.22^2= 0.0484).  It is not surprising that, on the PISA, students from developing economies with limited resources, like China, Estonia, Finland, and Poland routinely outscore wealthy Americans, Norwegians, and residents of the UAE. 

Standardized test scores can be improved on the margin with practice and preparation, but only to a limited extent, no matter how much money parents throw at preparation.  And the amount of money required to max-out the benefits of test preparation is not very much.  Preparation books can be obtained for less than $50.  As of this writing, Princeton review offers LSAT preparation packages ranging from $600 to $3000. 

It is far more resource-intensive to attempt to max out other criteria that law schools could use for admissions, such as donations, essays, extracurricular activities and unpaid internships, private “feeder” colleges, strategic college major selection to maximize GPA, and promises to hire law graduates into the family-owned law firm.

If standardized test scores were removed from the rankings, or their weight reduced, law schools probably would spend less money on merit scholarships.  But there is no guarantee that this money would be spent to help students from poor families.   Indeed, affirmative action, as practiced by elite law schools, generally involves the admission of students from high income families who benefited from high quality K-12 and college educations, and who can afford to pay full tuition.  Many students who are diverse because they are Hispanic are also white and no less wealthy than their non-Hispanic white peers.  

Affirmative action, as practiced by many universities, does not screen for individual circumstances, family history of suffering or moral culpability for the suffering of others, or anything closely related to moral desert.  These programs were initially ostensibly intended as remediation for enslavement of African Americans and forceful expropriation from Native Americans.  But in practice, the benefits and costs to individuals turn only on people's willingness to check boxes on a self-report form or write a well-crafted "diversity statement", knowing that there is substantial upside and little risk to claiming to be 'diverse' in the particular way that will gain favor from admissions officers.  Economists have found that many people change their self-reported racial and ethnic identities in response to incentives created by affirmative action.  Numerous studies find that the rich exhibit more entitled and less honest behaviors than the general population, so this self-report diversity regime likely benefits the rich, entitled, and dishonest at the expense of both the truly disadvantaged and meritocratic efficiency and desert.

There is no mechanism to prevent diversity programs from benefiting the wealthy direct heirs to the fortunes of Spanish Conquistadors, slave traders, plantation owners, Inquisitors, and war criminals.  Nor is there any mechanism to exempt from official discrimination those who families suffered from serfdom, slavery, genocide, or famine overseas or subsequent discrimination in the United States: serfs, concentration camp survivors, Jews, Mormons, Huguenots, Armenians, Poles, Ukrainians, and other Eastern Europeans, Copts, Irish, Boers, Kosovars, Bosnians, Italians, Middle Easterners, among others.

Rather than screen for moral desert based on sources of family wealth or documented persecution of direct progenitors, universities have allowed heirs to shady fortunes to donate their way into our classes and onto our boards.  At the same time, we preach a self-righteous, self-serving ideology, and treat those who challenge it with hostility.  This ideology maintains that moral responsibility for slavery turns not on inheritance of wealth misappropriated from slave labor, but rather on racial and ethnic identity, regardless of individual economic circumstances. Universities with large endowments benefit at the expense of innocent applicants who we scapegoat for the sins of our aristocratic donors.*   

My bet is that if the rankings emphasized standardized test scores less, most law schools would shift even more toward serving students from wealthier backgrounds, and would also increase their net prices and the shadow price of expected donations from students' families.  In other words, law schools—like the unreformed, less effective, and corrupt British Navy of old—would sell seats to the highest bidder. 

Idiosyncratically defined diversity will likely continue to serve as a rationalization for profit-maximizing practices.  Educators, public officials, and media organizations have been arguing for decades that diversity justifies departures from identity-neutral meritocratic standards.  Nevertheless, most of the population still considers universities' race, ethnicity, and legacy admissions policies unethical.  They prefer test scores, grades and community service.

Standardized test scores are not a perfect or complete measure of future job performance.  There’s a case for including other validated predictors of performance, such as tests of health or stamina or social intelligence.  And there’s a case for developing a value-added ranking that would emphasize the boost to student earnings rather than incoming student credentials.  If peer effects aren't very large or important, then a value-added ranking might be more useful to students than a system based on student credentials.

But if rankings are meant to serve employers, then standardized test scores are useful and important to include.

 

----

 

* Many of those who universities turn away in the name of diversity inherited exclusively from post-abolition immigrant laborers.  Their ancestors' origins in the poor, southern and eastern regions of Europe, in the Middle East, or in Asia, made them so despised that by the 1920s the United States changed its immigration laws to exclude their countrymen.  This cut off an escape route during the horrifically violent early half of the 20th century.  Within the U.S., immigrants from these countries were subjected to discrimination in employment and university admissions (see also here).  Some continue to be targeted for hate crimes about as often as African Americans and more often than Hispanics. Thus, diversity policies can perpetuate discrimination against those who have historically faced xenophobic discrimination, while also deflecting attention from the culpability of those who directly profited from slavery by inheriting fortunes extracted from slave labor.

“Unemployment” and law school rankings (Michael Simkovic)

One area of disagreement among law schools and between U.S. News and some of the law schools boycotting its rankings turns on whether “unemployment” should be defined according to the standard definition used by the U.S. government, the Department of Labor, the Census Bureau, and the Department of Education.  The government defines unemployment narrowly to apply to those who are both not employed and actively seeking employment.  Those who are not employed and not seeking employment are considered to be “not participating in the labor force” rather than “unemployed.” However, U.S. News defines unemployment more broadly to include anyone who is not currently employed, even if they are not working by choice. 

I’ve explained before why I favor the standard definition of unemployment.  Under the standard approach, law school statistics would be comparable to statistics for other educational programs.  Moreover, law schools would not be penalized for admitting students who wish to pursue dual degrees (i.e., an MPP, MD, MBA or PhD) or advanced degrees like a Tax LLM.  Nor would law schools be penalized for admitted wealthy students who do not wish or need to work or people who would prefer to stay home to care for children while their spouse works.

The only real argument against using the standard definition is that law schools will “game” the system by misrepresenting the intent of people who are not working or avoiding contacting those who they suspect are not working.  But a better solution to this problem of trust is to use a third party audit or third party data collection service rather than law school self-report. 

An audit would increase costs, but it might be worthwhile for law schools to invite the ABA to require a random audit that would include direct contact with some graduates who are not working to verify their status (unemployed vs. not participating in the labor force) in exchange for collecting data using the standard definition of unemployment. Alternatively, the ABA could require law schools to outsource the graduate employment data collection function to a neutral third party.  Schools could save money by reducing the number of internal FTE employees handling this data collection function and just paying a fee to the trusted third party.

If these solutions are not worthwhile to law schools or to ranking organizations, it may be that the total amount of fraud is small, notwithstanding a few very salient examples.  Corporate law scholars are familiar with the empirical finding that enhanced auditing under SOX (in response to Enron, Worldcom, etc.) actually reduced shareholder value, presumably because the aggregate amount of fraud was relatively low under the old, less expensive regime.

 

*. A related, but more technical problem, is how non-response to the survey is handled.  U.S. News assumes that 100% of graduates who do not respond to the employment survey (or whose employment cannot be verified through another source like an employer website) are unemployed.  By contrast, U.S. statistical agencies use the known characteristics of respondents and non-respondents (race, sex, age, location, etc.) to create weights that apply to each respondent.  These weights enable respondents to stand in for the non-respondents who have similar characteristics.  Thus, for example, if non-respondents characteristics suggest higher rates of unemployment than respondents but less than 100% unemployment, then the weights will lead to an unemployment estimate that is higher than using an unweighted average of respondents, but lower than assuming that 100% of non-respondents are unemployed.  In other words, U.S. News's approach to non-response likely leads to higher unemployment rates for law graduates compared to the rates that would be calculated using the standard weighting methods used by U.S. statistical agencies.