Tuesday, May 12, 2015
Sunday, May 10, 2015
Competitive Scholarships, Mandatory Courses, and the Costs and Benefits of Disclosure (Michael Simkovic)
There is a wide range of views about the benefits, costs, and appropriate use of conditional merit scholarships—scholarships that under their terms, will only be retained after the first year of law school if students maintain a minimum GPA or minimum class rank (if there is a mandatory grading curve, a minimum GPA effectively is a class rank requirement). These questions implicate both broad value judgments and also very specific empirical questions to which we many not have clear answers.
1) Is competition for grades a help or a hindrance to learning?
2) Is competition, with greater rewards for winners than for losers, inherently moral or immoral?
- Does the answer depend on whether the outcome of the competition is driven by luck, skill, or effort?
- Does the answer depend on how large the differences in rewards are between winners and losers?
3) Does disclosure alter student decision-making?
- If so, how?
- Is this a good thing or a bad thing?
- If it is a good thing do the benefits of disclosure outweigh the costs of providing disclosure?
- Are some ways of providing disclosure clearer and more meaningful than others? Could too much disclosure be overwhelming?
Disclosures are sometimes very effective at improving market efficiency. Sometimes disclosures appear to have no effect. Sometimes they have the opposite of the intended or expected effect. For example, disclosure of compensation of high level corporate executives of publicly traded companies may have contributed to an increase in executive pay (see also here.)
In the case of conditional merit scholarships, the direct administrative costs of providing disclosure appear minimal. The effects of such disclosure, if any, remain unknown. I support access to greater information about conditional scholarship retention rates, not only for law schools but also for all educational institutions.
Scholarship retention rates at many undergraduate institutions under government-backed programs appear to be lower than scholarship retention rates at most law schools. Around half of Georgia Hope Scholarship recipients lost their scholarship after the first year. Around 25 to 30 percent of Georgia Hope Scholarship recipients retained their scholarships for all four years of college. Nevertheless, conditional merit scholarships can have positive effects on undergraduate enrollment and academic performance. A fascinating randomized experiment by Angrist, Lang and Oreopolous found that financial incentives improved grades for women but not for men. A recent experiment also found evidence that merit scholarships tied to grades can increase student effort and academic performance at community colleges.
Unfortunately, there is some evidence that the use of merit scholarships tied to GPA by undergraduate institutions—where grade distributions and course workload vary widely by major—can reduce the likelihood that students complete their studies in science technology engineering and math (STEM) fields. Students who major in STEM fields have a higher chance of losing their scholarships
In other words, if students can shop for “easy As” rather than study harder to improve their performance, they can reduce their own future earning prospects. The approach law schools take—merit scholarships tied to mandatory grading curves and a required curriculum—may be better for students in the long run. Indeed, law students might benefit financially if additional courses, such as instruction in financial literacy, were mandatory.*
Greater disclosure of grading distributions may exacerbate grade shopping and grade inflation, which can undermine student effort and learning. Some models suggest that grade inflation is contagious across institutions (see also here). (It should be possible to disclose scholarship retention rates without disclosing grade distributions).
In some contexts, such as securities regulation or pharmaceuticals, disclosure requirements tend to be high. In other areas, such as employment contracts, disclosure tends to be more limited. We may not always get the balance right. These questions have lead to a rich research literature in law, economics, and psychology (see Bainbridge, Lang, Mathios, Coffee, Kaplow, Easterbrook and Fischel, Romano, and Schwartz). In all cases, whether and how disclosures alter behavior is an empirical question. How the benefits compare to the costs are empirical questions mixed with subjective value judgments.
Given the current limited state of knowledge, and good faith and understandable disagreements about subjective value differences, strident views on one side or another, and moral condemnations of those entertaining different viewpoints, are not appropriate.
Law professors have an obligation to teach students to think like lawyers, weigh evidence, and consider different arguments and different perspectives. We should not shut down discussion with swaggering declarations of the moral superiority of our own views or ad-hominem attacks against those with whom we disagree.
A recent post (in the comments) by Brian Tamanaha (or someone posting under his name and with a similar rhetorical style**) highlights the unfortunate tendency by some toward moral posturing. Tamanaha writes:
“[Those who condemn conditional scholarships are] speaking up for the integrity of legal academia. It is embarrassing that law professors would now rise up to defend employment reporting standards … criticized by outsiders (see New York Times "Bait and Switch" piece), practices which have since been repudiated and reformed by new ABA standards. I do not understand why Simkovic is re-raising these resolved issues, but it does not help us regain our collective credibility.
After reading these posts, I have begun to wonder whether a sense of professional responsibility is what separates the two sides in this discussion. It is not a coincidence that John Steele, [Bernard Burk], and others who strongly condemn these practices have taught legal ethics.”
In other words, if you question Brian Tamanaha’s reasoning and conclusions—as I have—then you have no integrity and dubious ethics, are irresponsible and unprofessional, and are an embarrassment to the legal academy.
Bernard Burk, though declaring his disdain for ad-hominem attacks, accuses those with whom he disagrees of being “partisan.” He compares competition for grades and scholarships to physically beating students. Burk compares law schools to gangsters and evil witches. He claims that the positive effects of conditional scholarships on student motivation and learning “smells of post-hoc rationalization.” (Most of the labor economics studies demonstrating positive effects of financial incentives on student performance were available before The New York Times and the law school critics targeted law school conditional scholarships; the critics overlooked the peer-reviewed literature).
Deborah Merritt, though generally providing an intelligent discussion of conditional scholarship issues, compares conditional scholarships in which adults who lose the competition for grades receive a free year of law school to the fictional “Hunger Games” in which children who lose a physical struggle are murdered. (Paul Caron repeats this unfortunate comparison when summarizing the debate; so does Bernard Burk).
Paul Campos compares those who disagree with him about data disclosure standards to “Holocaust deniers.”
Law school critics have not persisted through the force of argument or evidence, but rather through their ability to make an honest discussion of the issues so unpleasant that very few who disagree with them wish to engage. We should thank Professor Telman for his courage and for elevating the conversation from polemics to evidence-based inquiry. As more professors and journalists raise substantive questions about law school critics’ narrative, it will become increasingly difficult for the critics to foreclose factual and ethical inquiry through ad-hominem attacks and hyperbole.
* A recent survey by John Coates, Jessie Fried, and Kathryn Spier at Harvard suggests that large law firm employers believe instruction in certain technically challenging business electives, especially accounting, corporate finance, and corporations, is particularly valuable on the job. Data does not exist to evaluate whether enrollment in such courses actually boosts earnings or employment, or is even correlated with greater earnings or employment. However, one working hypothesis is that such courses might be the law school equivalent of undergraduate STEM or economics majors. A study of high school financial literacy mandates suggests positive long-term effects on enrollees’ financial well-being.
** The first and only time I met Brian Tamanaha in person was at the 2013 Law & Society meeting in Boston where he spoke on a panel. Professor Tamanaha shut down questions from the audience about whether his presentation of law school data was misleading by insisting that in our hearts surely we all knew he was right and that any question about whether he was wrong on the facts, and any attempt to rely on data rather than emotionally charged anecdotes, was a sign of flawed moral character.
May 10, 2015 in Guest Blogger: Michael Simkovic, Law in Cyberspace, Legal Profession, Ludicrous Hyperbole Watch, Of Academic Interest, Professional Advice, Science, Student Advice, Web/Tech, Weblogs | Permalink
Thursday, May 7, 2015
One question in the labor economics literature is why education increases earnings. The dominant view is Human Capital Theory—education increases earnings by increasing productivity of educated workers. A minority view, signaling theory, holds that education increases earnings by revealing information about potential employees to employers and facilitating the employer-employee matching process, similar to marketing expenditures matching products with customers. In other words, education indicates to employers which employees were the best all along.
On some level, the distinction is irrelevant—from the perspective of potential students it does not really matter how education increases earnings, only that it does. Whether education makes workers more efficient individually or makes the economy in aggregate more efficient by moving workers to where they can do the most good does not change the fact that employers, employees, and the economy as a whole are better off with additional education as long as the marginal benefits of additional education exceed marginal costs.
Critics of higher education sometimes claim that there is over-investment in signaling and credentialing, a kind of arms race producing negative externalities in which the social returns to education are lower than the private returns. This view, though popular with political movements seeking to reduce public support for education, does not hold much sway in the modern peer-reviewed labor economics literature.
A recent literature review by Professors Fabian Lange at Yale and Robert Topel at the University of Chicago explains the problems with the view that employers would expend valuable resources paying a premium for employees who had over-invested in education as a signal of ability: “Employer learning about productivity occurs fairly quickly after labor market entry, implying that the signaling effects of schooling are small.”
In other words, employers can quickly and efficiently sort employees on their own by observing their productivity on the job, retaining and promoting strong performers, and terminating weaker employees. Employers do not need educational institutions to perform this task for them, nor are employers willing to pay premium wages for information they can more cheaply obtain themselves.
Suppose, for example, that the only value of a Harvard Law degree were as signal of ability to employers. Harvard admits students based almost exclusively on standardized test scores and undergraduate grades, which are also observable to employers. Almost everyone who is admitted to Harvard graduates.
A clever employer realizes it can save money by employing bachelor’s degree holders with admissions letters from Harvard, but who have not yet attended. Harvard gets wise and refuses to confirm admissions. Employers now look directly at LSATs and GPAs, and perhaps hire former admissions officers to consider softer factors. The employer and employee can effectively split the cost savings of not getting the Harvard degree and also the value of the time that would be spent in schooling instead of working.* If additional information is required, personality testing, assessments of physical and mental health, assessments of writing samples, background checks and the like can all be performed for a fraction of the cost and in a fraction of the time as a Harvard degree.
In spite of potentially massive efficiency gains and financial reward for employers and employees, this is not what actually happens in the real world (with the exception of a stacked "experiment" by Peter Thiel). Why not? Because employers believe that the schooling itself is valuable and makes employees more productive.
Lange and Topel also explain how “sheepskin effects” (disproportionately higher earnings premiums for college completers than for college dropouts with a few years of college) have been misinterpreted as support for the signaling hypothesis, when they actually reflect selection effects and dynamic decision making about educational investments:
"Diploma effects are often presented as evidence for screening theories of schooling. We disagree. Instead we view diploma effects as evidence that individuals face uncertainty about their individual returns to schooling and that this uncertainty is revealed as individuals acquire schooling. Those least capable to profit from schooling drop out before the completion of degree years. Those graduating exhibit larger returns than those who dropped out at lower levels of schooling. This reasoning was informally developed by Chiswick (1973). Since then, a number of authors (Altonji (1993), Heckman, Lochner and Todd (2003), Keane and Wolpin (1997) and Eckstein and Wolpin (1999)) examined different aspects of sequential schooling choice under uncertainty. We build a simple model in this spirit with the intention to show how individual’s schooling decisions can generate (large) diploma effects if individuals learn about their returns while in school."
There are additional problems with signaling theory, which I explained in Risk Based Student Loans:
“Signaling Theory implies that labor market outcomes should not depend on what students study, but only on how well they perform academically relative to other students with similar standardized test scores, or perhaps whether they demonstrate a strong work ethic by choosing a challenging major. Differences in earnings by field of study appear to reflect the value of field-specific skill development rather than differences in ability levels. Even within engineering, there are large starting wage differences by specialty.
Human Capital Theory also helps explain higher average per-capita productivity and wages in states and nations with higher levels of educational attainment. If education only sorted workers according to ability, it would presumably only increase the variance of wages (i.e., income inequality), while leaving the mean unaltered.
Further, Human Capital Theory helps explain the willingness of many employers to pay for professional degree programs for successful employees. Employers’ willingness to educate workers whom employers already know to be of high quality suggests that employers believe that professional education has skill-development value rather than mere sorting value.”
In The Knowledge Tax I expand on this, explaining the evidence linking investment in education to more rapid economic growth rather than merely redistributing income from the less educated to the more educated.
* Note that in many states, including New York and California which collectively comprise 28 percent of the law market, a law degree is not legally required to sit for the bar exam. Instead, “law office study”, or an apprenticeship under the supervision of an attorney, possibly following one year of law school, will suffice.
Tuesday, May 5, 2015
A better grading system
Professor Merritt argues that mandatory grading curves can be unfair when one class has stronger students than another. I agree.
Statistician Valen Johnson—whom I cite in my last post as an authority on grade inflation— has developed a clever solution to this problem which involves adjusting grading curves within each class based on the ability levels of the students. A Johnson-inspired proposal was nearly adopted at Duke University in the late 1990s, but was blocked by departments that offered higher grades and attracted weaker students.
Most law schools try to balance their sections in term of student ability levels and overall quality of faculty. Nevertheless, anomalies like a “smart section” (as Professor Merritt calls it) may occasionally occur. Johnson’s proposal would be an excellent solution to this problem.
Professor Merritt asserts that there is some sort of problem with the market for lawyers and law graduates that makes competition and inequality uniquely bad in the context of law. These assertions are implausible given the low barriers to entry for both law schools and lawyers, aggressive competition between law schools for students and between lawyers for clients, and widespread inequality outside of law school and legal practice. Some form of regulation is the norm in many areas of employment and in many industries, and a licensing regime for lawyers and an accreditation system for law schools do not in any way make these occupations and institutions unique or unusual. According to a recent study, nearly a third of U.S. workers are licensed, licensing is more common as education and skill levels increase, and licensing does not affect inequality among the licensed.
As a general matter, deregulated market competition and greater inequality are a package deal. Inequality can be reduced through regulation, taxation, and politicization of compensation through unionization or growth of public sector employment.
Professor Merritt’s critiques follow the standard playbook of law school critics—take something about law schools that is widespread and common out of context, claim that it is somehow unique to law schools when it is neither unique nor unusual, and then demonize it.
Jeremy Telman responds.
In her latest post, Deborah Merritt maintains that scholarships conditioned on maintaining a minimum GPA or class ranking are troubling when used by law schools, even though such conditions are routinely used by other educational institutions and state government programs.
According to Professor Merritt, the problem is that the mandatory curve in law school is such that not all students can keep their conditional scholarships. But Professor Merritt presents no evidence that conditional scholarships retention rates are any higher for undergraduate or government programs than for law schools. She infers nefarious motives on the part of law schools based only on the fact that law schools require students to compete for scholarship funds that are in limited supply.
Perhaps Professor Merritt believes that competition for scarce and valuable resources is inherently immoral. She does not explain why this is so or whether these views apply outside the context of law school scholarships. If only one out of ten associates hired at a law firm will make partner and earn $1 million per year, is it inherently immoral to ask associates to work hard and compete for the opportunity? If only one actor will be selected for a part, is it immoral to ask more than one actor to try out? Is any competition for promotions, clients, or recognition immoral? If so, we are living in a wicked, wicked world.
Perhaps Professor Merritt believes it is inherently immoral to limit “A” grades to students whose academic performance is superior to most of their peers, since an “A” is simply a data point and can be replicated and distributed to everyone at zero marginal cost. But liberally handing out “A” grades is costly for students and employers. Labor economics studies suggest that grade inflation is associated with reduced effort by students and reduced learning. Educators are not doing students or employers favors if they allow high grades to become a birthright rather than a marker of distinction that must be earned through hard work and exceptional performance.
Statistician Valen Johnson and others have argued that many perverse incentives in undergraduate education could be ameliorated if mandatory grading curves were imposed across majors and grade inflation and grade shopping were stamped out. If certain undergraduate majors have succumbed to the pressure to inflate grades in order to keep student-customers happy, that is quite troubling. Employers will likely distrust grades from such programs, question how much students have learned, and harbor suspicions about the work ethic of students who would opt into programs known for awarding easy “A’s” for minimal effort. Programs that have resisted the pressure to inflate grades and maintained more rigorous academic standards are more likely to retain the confidence of employers and to teach students knowledge and skills that are valued in the labor market. Indeed, grades are notoriously lower in STEM fields than in the humanities, even though STEM majors spend more time studying and have higher standardized test scores.
Professor Merritt suggests that law students do not require any incentives to work harder, since they are all already studying at full capacity. Some students presumably are, but there are many law students who can and should focus more on their studies. A roll call in most classes will reveal students whose attendance is well below 100 percent—so much so that the ABA now requires law schools to enforce minimum class attendance policies. When students do attend class, a visit to the back of a classroom and a glance at computer screens will reveal some students who are not giving their undivided attention. Cold calling will reveal students who have not done the required reading—although they do appear to be well informed about the latest sports and celebrity news. Some students have family or employment obligations that understandably limit the amount of time they can devote to their studies. But in the evening, while some student who are less constrained are studying, a stroll past the local bar will reveal others who are spending their time on less academic pursuits.
Shortly after graduation, some students who did not show up for class enough, did not pay attention enough, did not prepare for class enough, did not review after class enough, and did not seek out their professors when they were confused will find that they have not passed the bar exam and will not be permitted to practice law until they learn how to work hard and study. Others will find, rather less dramatically, that what they did not work hard enough to learn in law school could have made them more valuable to their employers.
Law schools can observe LSAT scores and undergraduate GPAs, but they cannot directly observe students’ work ethic and drive to succeed. Just as law students want to attend the best law schools, law schools want to educate the best students who have the motivation to become leaders in law, business, and government.
Given the goal of attracting and retaining the best students, rewarding motivation and ability seems like a reasonable policy. Anecdotes notwithstanding, the evidence suggests that most college and law students understand the terms of conditional scholarships well.
Monday, May 4, 2015
Many critics have attacked law schools for offering merit scholarships that can only be retained if students meet minimum GPA requirements. Jeremy Telman has a fascinating new post analyzing these scholarships in light of common practices in higher education and the peer-reviewed social science literature. It’s a powerful counterpoint to a previously unanswered critique of law school ethics.
Professor Telman notes that similar conditional scholarships are widely used by undergraduate institutions, and even some state government programs. Undergraduates behave as if they understand how conditional scholarships work, which suggests that most law students, who are older, wiser, and more sophisticated, probably understand the terms of these agreements as well.
Moreover, minimum GPA requirements can motivate students to study harder, pay closer attention, and learn more. This seems particularly likely in the context of the first year of law school where mandatory grading curves and required curriculums remove the opportunity to shop for “easy A’s”. (Professor Telman does, however, express concern about inadequate performance feedback to law students until the final exams at the end of their first semester).
Professor Telman notes that law schools may struggle to predict at the time of admission which students will be the most successful. Conditional scholarships help institutions gather additional information about students’ abilities and work ethic and ensure that limited merit scholarship resources go to the students who are most deserving. Students who are deemed undeserving and lose their scholarships retain the option of transferring to another institution for their remaining years of law school.
Professor Telman doesn't object to additional disclosure about the percent of students retaining their scholarships, but he doubts it would have made much of a difference in prospective law students' matriculation decisions.
Zachary Clopton who will join the faculty at Cornell University. He is currently the Public Law Fellow at the Law School. He graduated magna cum laude in 2007 from Harvard Law School, and also earned a Masters in International Relations from Cambridge. He clerked for Judge Wood on the U.S. Court of Appeals for the Seventh Circuit, and was an Associate at WilmerHale in the National Security group for two years, before serving as an Assistant U.S. Attorney in the Civil Division of the Northern District of Illinois for three years. His teaching and research interests include civil procedure, international business transactions, federal courts, conflicts, torts, and national security law.
Genevieve Lakier who will join the faculty at the University of Chicago. She is presently a Bigelow Fellow at the Law School. She graduated cum laude from New York University School of Law in 2011, where she was a Furman Fellow and Editor-in-Chief of the NYU Review of Law and Social Change. She also earned a PhD in Anthropology from the University of Chicago in 2014. She clerked for both Judge Sand in the Southern District of New York and then Judge Daughtrey on the U.S. Court of Appeals for the 6th Circuit, before coming to Chicago. Her teaching and research interests include constitutional law, criminal law and the criminal justice system, comparative law, administrative law, and law and society.
John Rappaport who will join the faculty at the University of Chicago. He is presently a Bigelow Fellow at the Law School. He graduated magna cum laude from Harvard Law School in 2006, clerked for Judge Reinhardt on the U.S. Court of Appeals for the 9th Circuit, then worked for two years as a Deputy Federal Public Defender in Los Angeles, before clerking for Justice Ginsburg on the U.S. Supreme Court. He also spent two years as a litigation associate at Munger, Tolles & Olson in Los Angeles, and clerked for six months for Judge Watford on the U.S. Court of Appeals for the Ninth Circuit before coming to Chicago. His teaching and research interests include all aspects of criminal procedure and the criminal justice system, as well as federal courts, constitutional law, evidence, and civil procedure.
You can see a list of past Bigelows and where they now teach here.
Sunday, May 3, 2015
In the bad 'ole days when law schools were not very forthcoming about employment outcomes, LST was a useful counterweight. But since Senators Boxer and Coburn prevailed on the ABA to mandate more detailed reporting a few years ago, LST has persisted in its misleading practice of treating JD/PhD students as "underemployed" and treating JD/MBAs or pure JDs in consulting and other renumerative professions as not part of the employment count for schools; so, too, the JD/Masters in Public Policy types who go into think tanks also don't count. Notice that, by the odd LST methodology, Yale only has about 73% of its class employed, no doubt because there are many JD/PhD students as well as students in these other categories. LST really ought to change with the times, instead of massaging the data in ways that are misleading.