Sunday, May 10, 2015

Competitive Scholarships, Mandatory Courses, and the Costs and Benefits of Disclosure (Michael Simkovic)

There is a wide range of views about the benefits, costs, and appropriate use of conditional merit scholarships—scholarships that under their terms, will only be retained after the first year of law school if students maintain a minimum GPA or minimum class rank (if there is a mandatory grading curve, a minimum GPA effectively is a class rank requirement).  These questions implicate both broad value judgments and also very specific empirical questions to which we many not have clear answers.  

1)   Is competition for grades a help or a hindrance to learning?

2)   Is competition, with greater rewards for winners than for losers, inherently moral or immoral?

  1. Does the answer depend on whether the outcome of the competition is driven by luck, skill, or effort?
  2. Does the answer depend on how large the differences in rewards are between winners and losers?

3)   Does disclosure alter student decision-making?  

  1. If so, how? 
  2. Is this a good thing or a bad thing?
  3. If it is a good thing do the benefits of disclosure outweigh the costs of providing disclosure?
  4. Are some ways of providing disclosure clearer and more meaningful than others?  Could too much disclosure be overwhelming?

Disclosures are sometimes very effective at improving market efficiency.  Sometimes disclosures appear to have no effect.  Sometimes they have the opposite of the intended or expected effect.  For example, disclosure of compensation of high level corporate executives of publicly traded companies may have contributed to an increase in executive pay (see also here.) 

In the case of conditional merit scholarships, the direct administrative costs of providing disclosure appear minimal.  The effects of such disclosure, if any, remain unknown.  I support access to greater information about conditional scholarship retention rates, not only for law schools but also for all educational institutions. 

Scholarship retention rates at many undergraduate institutions under government-backed programs appear to be lower than scholarship retention rates at most law schools.  Around half of Georgia Hope Scholarship recipients lost their scholarship after the first year.  Around 25 to 30 percent of Georgia Hope Scholarship recipients retained their scholarships for all four years of college.  Nevertheless, conditional merit scholarships can have positive effects on undergraduate enrollment  and academic performance.  A fascinating randomized experiment by Angrist, Lang and Oreopolous found that financial incentives improved grades for women but not for men.  A recent experiment also found evidence that merit scholarships tied to grades can increase student effort and academic performance at community colleges.

Unfortunately, there is some evidence that the use of merit scholarships tied to GPA by undergraduate institutions—where grade distributions and course workload vary widely by major—can reduce the likelihood that students complete their studies in science technology engineering and math (STEM) fields.  Students who major in STEM fields have a higher chance of losing their scholarships 

In other words, if students can shop for “easy As” rather than study harder to improve their performance, they can reduce their own future earning prospects.  The approach law schools take—merit scholarships tied to mandatory grading curves and a required curriculum—may be better for students in the long run.  Indeed, law students might benefit financially if additional courses, such as instruction in financial literacy, were mandatory.*

Greater disclosure of grading distributions may exacerbate grade shopping and grade inflation, which can undermine student effort and learning.  Some models suggest that grade inflation is contagious across institutions (see also here).  (It should be possible to disclose scholarship retention rates without disclosing grade distributions).

In some contexts, such as securities regulation or pharmaceuticals, disclosure requirements tend to be high.  In other areas, such as employment contracts, disclosure tends to be more limited.  We may not always get the balance right.  These questions have lead to a rich research literature in law, economics, and psychology (see Bainbridge, Lang, Mathios, Coffee, Kaplow, Easterbrook and Fischel, Romano, and Schwartz).  In all cases, whether and how disclosures alter behavior is an empirical question.  How the benefits compare to the costs are empirical questions mixed with subjective value judgments.

Given the current limited state of knowledge, and good faith and understandable disagreements about subjective value differences, strident views on one side or another, and moral condemnations of those entertaining different viewpoints, are not appropriate.  

Law professors have an obligation to teach students to think like lawyers, weigh evidence, and consider different arguments and different perspectives.  We should not shut down discussion with swaggering declarations of the moral superiority of our own views or ad-hominem attacks against those with whom we disagree.

A recent post (in the comments) by Brian Tamanaha (or someone posting under his name and with a similar rhetorical style**) highlights the unfortunate tendency by some toward moral posturing.  Tamanaha writes:

“[Those who condemn conditional scholarships are] speaking up for the integrity of legal academia. It is embarrassing that law professors would now rise up to defend employment reporting standards … criticized by outsiders (see New York Times "Bait and Switch" piece), practices which have since been repudiated and reformed by new ABA standards. I do not understand why Simkovic is re-raising these resolved issues, but it does not help us regain our collective credibility.

After reading these posts, I have begun to wonder whether a sense of professional responsibility is what separates the two sides in this discussion. It is not a coincidence that John Steele, [Bernard Burk], and others who strongly condemn these practices have taught legal ethics.” 

In other words, if you question Brian Tamanaha’s reasoning and conclusions—as I have—then you have no integrity and dubious ethics, are irresponsible and unprofessional, and are an embarrassment to the legal academy.

Bernard Burk, though declaring his disdain for ad-hominem attacks, accuses those with whom he disagrees of being “partisan.” He compares competition for grades and scholarships to physically beating students.  Burk compares law schools to gangsters and evil witches.  He claims that the positive effects of conditional scholarships on student motivation and learning “smells of post-hoc rationalization.”  (Most of the labor economics studies demonstrating positive effects of financial incentives on student performance were available before The New York Times and the law school critics targeted law school conditional scholarships; the critics overlooked the peer-reviewed literature).

Deborah Merritt, though generally providing an intelligent discussion of conditional scholarship issues, compares conditional scholarships in which adults who lose the competition for grades receive a free year of law school to the fictional “Hunger Games” in which children who lose a physical struggle are murdered.  (Paul Caron repeats this unfortunate comparison when summarizing the debate; so does Bernard Burk).

Paul Campos compares those who disagree with him about data disclosure standards to “Holocaust deniers.”

Law school critics have not persisted through the force of argument or evidence, but rather through their ability to make an honest discussion of the issues so unpleasant that very few who disagree with them wish to engage.  We should thank Professor Telman for his courage and for elevating the conversation from polemics to evidence-based inquiry.  As more professors and journalists raise substantive questions about law school critics’ narrative, it will become increasingly difficult for the critics to foreclose factual and ethical inquiry through ad-hominem attacks and hyperbole.


* A recent survey by John Coates, Jessie Fried, and Kathryn Spier at Harvard suggests that large law firm employers believe instruction in certain technically challenging business electives, especially accounting, corporate finance, and corporations, is particularly valuable on the job.  Data does not exist to evaluate whether enrollment in such courses actually boosts earnings or employment, or is even correlated with greater earnings or employment.  However, one working hypothesis is that such courses might be the law school equivalent of undergraduate STEM or economics majors.  A study of high school financial literacy mandates suggests positive long-term effects on enrollees’ financial well-being.  

** The first and only time I met Brian Tamanaha in person was at the 2013 Law & Society meeting in Boston where he spoke on a panel.  Professor Tamanaha shut down questions from the audience about whether his presentation of law school data was misleading by insisting that in our hearts surely we all knew he was right and that any question about whether he was wrong on the facts, and any attempt to rely on data rather than emotionally charged anecdotes, was a sign of flawed moral character.

Guest Blogger: Michael Simkovic, Law in Cyberspace, Legal Profession, Ludicrous Hyperbole Watch, Of Academic Interest, Professional Advice, Science, Student Advice, Web/Tech, Weblogs | Permalink