Failed the Bar Exam? Try Again (Michael Simkovic)

Noah Feldman recently argued that law schools are not helping students with low standardized test scores by denying those students the opportunity to attend law school simply because those students might find it challenging to pass the bar exam.*  According to Feldman, denying people an opportunity to try to improve their situation in life is “paternalism that verges on infantilization.” Moreover, “A standardized test score, taken alone, shouldn't determine your future.”

Feldman’s perspective is bolstered by an important feature of bar exams:  People who fail an exam can study harder and then retake it.

Standardized test scores might predict the likelihood of passing the bar exam on a single try.  But a more important question is arguably whether an applicant can eventually pass a bar exam.  Delays in bar passage can have short-term financial costs, entail logistical challenges and stress, and in rare instances where employers require recent hires to practice law without supervision, even result in the loss of a job.  But most law graduates won’t lose their jobs for retaking the bar exam, and an individual who passes after multiple attempts will have the same license for the rest of his life as one who passes on the first try.** 

Many law schools maintain exclusive admissions, not necessarily for the benefit of the students to whom they deny admission, but rather for the benefit of prospective employers and clients—in other words, to maintain a brand that suggests certain qualities above and beyond the ability to pass a bar exam.  Different law schools have different brands. 

The probability of eventually passing the bar exam is a function of how many times an applicant is willing to try.***  A bar exam taker with only a 50 percent chance of passing on any one try has an 88 percent chance of passing the bar exam if he is willing to retake the bar exam up to two times, and a 94 percent chance of passing if he is willing to retake it up to 3 times.

Probability of Eventually Passing the Bar Exam as a Function of Number of Tries
	Probability of Passing on Each Try
	90%	70%		50%		30%
Number of Tries	Cumulative Probability of Passing
1	90%	70%		50%		30%
2	99%	91%		75%		51%
3	100%	97%		88%		66%
4	100%	99%		94%		76%
5	100%	100%		97%		83%

These model-driven calculations seem to map well to the real world.  For example, Florida Coastal—which admits many students with very low LSAT scores—reports that although its first time bar passage rate is much lower, 93 percent of its graduates eventually pass the bar exam. 

How motivated is a particular law school applicant?  This is something an applicant is likely to know better than a law school admissions officer.  For those with grit and determination, failure is often temporary.  And anecdotally, law graduates have failed bar exams and gone on to have successful careers.

It would be strange if newspapers claimed that those who fail a road test on the first try are doomed to never obtain a drivers license, will never be able to hold down a job, and should never have enrolled in high school in the first place.  But in the world of legal education, members of the press too often make comparably misinformed claims about law students and the bar exam. 

*Feldman also argues that the benefits of law school are much broader than preparation for the bar exam.  This is consistent with studies showing substantial financial benefits to law school attendance including the large proportion of graduates who do not practice law, although Feldman also means something beyond financial benefits. 

**They likely will not have the same earnings because of selection effects (i.e., failing the bar exam on the first try is probably correlated with lower earnings ability), or if failing the bar on the first try causally alters lifetime earnings.  But those who struggle with standardized tests probably also have relatively limited and unattractive alternatives to law school, given the widespread prevalence of licensing exams across professions.  The critical question is not absolute outcomes, but how much value-added they received from law school and whether this exceeds the cost.

***As long as the chance of passing on any given attempt is greater than zero, the chance of eventually passing is a function of the number of times an applicant is willing to try before giving up. 

The math is

[Probability of Eventual Passage] = 1 – [(Probability of failing on each try)^(number of tries)]. 

This simple model assumes that the probability of passing on each try for a given applicant is equal across tries.  Because of selection effects (the most capable students being more likely to pass on the first try and therefore being absent from the group retaking the bar exam), bar pass-rates for second time takes are lower than for first time takers.  However, for any given student, the probability of passing likely increases with the number of tries because of practice effects.