Standardized test scores and law school rankings (Michael Simkovic)

One area of disagreement among law schools and between U.S. News and some of the law schools boycotting its rankings turns on whether students’ standardized test scores should be included in the rankings, and the extent to which they should be weighted.

The case for including standardized test scores, and weighting them heavily, is as follows:  Standardized test scores are useful (but incomplete) predictors of future academic and job performance.  They are therefore useful to employers, who are important consumers of the rankings.  Employers cannot recruit in person at every law school, so they prioritize a “target rich environment” full of high-scoring law students who are likely to accept an offer to work for them. 

Standardized tests were first developed in the West by the British military to improve the quality of Naval officers commanding ships.  Ships are expensive and are prone to being lost or destroyed when commanded by less competent officers.

Prior to the development of standardized test scores, Naval officers were selected through an interview process and soft assessment of goodness of fit, or offices could be purchased.  This led to officers being judged by their physical appearance, race, sex, social class, and manners, none of which were actually useful predictors of performance as a Naval officer.

Once standardized tests were developed and used for selection of officers, the quality and performance of British Naval officers improved dramatically.  Today, the United States military continues to use a standardized test, the AFQT, to weed out individuals who would likely be poor performers.  It also uses a broader standardized test, the ASVAB, to find talented individuals who could serve in specialist roles.  The Navy and Coast Guard, like the British Navy before them, have higher test score requirements than other branches of the military.

Some employers, like McKinsey and Google, have used their own standardized test to evaluate job applicants for the simple reason that standardized test scores are useful predictors of performance.  Meritocratic education and employment systems have historically enabled countries and companies that adopted them to punch above their weight in terms of growth compared to countries and companies that emphasized family ties, caste, class, sex, race, or ethnicity.  Compare the trajectories of Malaysia--which enshrined ethnic favoritism in its educational and political system--with neighboring Singapore, which adopted unabashed race-neutral meritocracy.

In addition, there is evidence of “peer effects” in education.  Peer effects lead students with lower standardized test scores to benefit more from education when they are placed in contact with higher scoring students.  The higher scoring students, by contrast, do not benefit from being placed in an environment with lower scoring students.  Thus, high scoring students provide benefits to low-scoring peers, but the reverse is not true.  Indeed, students with problematic behaviors, such as drinking or criminal activity, can be a negative influence on their peers.

The argument against including standardized test scores is that standardized test scores are nothing but a proxy for family income and therefore discriminate against under-privileged students.  But this argument is wrong. 

Standardizes test scores are not close proxies for household income.  Within the United States, the correlation between math SAT scores and household income is only 0.22. This means that parental income explains less than 5 percent of the variation in standardized test scores (0.22^2= 0.0484).  It is not surprising that, on the PISA, students from developing economies with limited resources, like China, Estonia, Finland, and Poland routinely outscore wealthy Americans, Norwegians, and residents of the UAE. 

Standardized test scores can be improved on the margin with practice and preparation, but only to a limited extent, no matter how much money parents throw at preparation.  And the amount of money required to max-out the benefits of test preparation is not very much.  Preparation books can be obtained for less than $50.  As of this writing, Princeton review offers LSAT preparation packages ranging from $600 to $3000. 

It is far more resource-intensive to attempt to max out other criteria that law schools could use for admissions, such as donations, essays, extracurricular activities and unpaid internships, private “feeder” colleges, strategic college major selection to maximize GPA, and promises to hire law graduates into the family-owned law firm.

If standardized test scores were removed from the rankings, or their weight reduced, law schools probably would spend less money on merit scholarships.  But there is no guarantee that this money would be spent to help students from poor families.   Indeed, affirmative action, as practiced by elite law schools, generally involves the admission of students from high income families who benefited from high quality K-12 and college educations, and who can afford to pay full tuition.  Many students who are diverse because they are Hispanic are also white and no less wealthy than their non-Hispanic white peers.  

Affirmative action, as practiced by many universities, does not screen for individual circumstances, family history of suffering or moral culpability for the suffering of others, or anything closely related to moral desert.  These programs were initially ostensibly intended as remediation for enslavement of African Americans and forceful expropriation from Native Americans.  But in practice, the benefits and costs to individuals turn only on people's willingness to check boxes on a self-report form or write a well-crafted "diversity statement", knowing that there is substantial upside and little risk to claiming to be 'diverse' in the particular way that will gain favor from admissions officers.  Economists have found that many people change their self-reported racial and ethnic identities in response to incentives created by affirmative action.  Numerous studies find that the rich exhibit more entitled and less honest behaviors than the general population, so this self-report diversity regime likely benefits the rich, entitled, and dishonest at the expense of both the truly disadvantaged and meritocratic efficiency and desert.

There is no mechanism to prevent diversity programs from benefiting the wealthy direct heirs to the fortunes of Spanish Conquistadors, slave traders, plantation owners, Inquisitors, and war criminals.  Nor is there any mechanism to exempt from official discrimination those who families suffered from serfdom, slavery, genocide, or famine overseas or subsequent discrimination in the United States: serfs, concentration camp survivors, Jews, Mormons, Huguenots, Armenians, Poles, Ukrainians, and other Eastern Europeans, Copts, Irish, Boers, Kosovars, Bosnians, Italians, Middle Easterners, among others.

Rather than screen for moral desert based on sources of family wealth or documented persecution of direct progenitors, universities have allowed heirs to shady fortunes to donate their way into our classes and onto our boards.  At the same time, we preach a self-righteous, self-serving ideology, and treat those who challenge it with hostility.  This ideology maintains that moral responsibility for slavery turns not on inheritance of wealth misappropriated from slave labor, but rather on racial and ethnic identity, regardless of individual economic circumstances. Universities with large endowments benefit at the expense of innocent applicants who we scapegoat for the sins of our aristocratic donors.*   

My bet is that if the rankings emphasized standardized test scores less, most law schools would shift even more toward serving students from wealthier backgrounds, and would also increase their net prices and the shadow price of expected donations from students' families.  In other words, law schools—like the unreformed, less effective, and corrupt British Navy of old—would sell seats to the highest bidder. 

Idiosyncratically defined diversity will likely continue to serve as a rationalization for profit-maximizing practices.  Educators, public officials, and media organizations have been arguing for decades that diversity justifies departures from identity-neutral meritocratic standards.  Nevertheless, most of the population still considers universities' race, ethnicity, and legacy admissions policies unethical.  They prefer test scores, grades and community service.

Standardized test scores are not a perfect or complete measure of future job performance.  There’s a case for including other validated predictors of performance, such as tests of health or stamina or social intelligence.  And there’s a case for developing a value-added ranking that would emphasize the boost to student earnings rather than incoming student credentials.  If peer effects aren't very large or important, then a value-added ranking might be more useful to students than a system based on student credentials.

But if rankings are meant to serve employers, then standardized test scores are useful and important to include.

 

----

 

* Many of those who universities turn away in the name of diversity inherited exclusively from post-abolition immigrant laborers.  Their ancestors' origins in the poor, southern and eastern regions of Europe, in the Middle East, or in Asia, made them so despised that by the 1920s the United States changed its immigration laws to exclude their countrymen.  This cut off an escape route during the horrifically violent early half of the 20th century.  Within the U.S., immigrants from these countries were subjected to discrimination in employment and university admissions (see also here).  Some continue to be targeted for hate crimes about as often as African Americans and more often than Hispanics. Thus, diversity policies can perpetuate discrimination against those who have historically faced xenophobic discrimination, while also deflecting attention from the culpability of those who directly profited from slavery by inheriting fortunes extracted from slave labor.