January 23, 2017
January 19, 2017
Established datasets, proxies, and customized data collection: The case of international LLMs (Michael Simkovic)
How should researchers make tradeoffs between the costs of data collection, the speed of the analysis, the precision of the measurements, reproducibility by other researchers, and broader context about the meaning of the data: how we might compare one group or one course of action to another, how we might understand historical trends, and the like?
Must we always measure the precise group of interest, with zero tolerance for over-inclusion or under-inclusion? Or might one or a series of proxy groups be sufficient, or even preferable for some purposes? What if the proxies have substantial overlap with the groups of interest and biases introduced by use of proxy groups are reasonably well understood? How close must the proxy group be to the group of interest?
These are important questions raised by a group of legal profession researchers which includes several of the principal investigators of the widely used After the JD dataset.
Professors Carole Silver, Ethan Michelson, Robert Nelson, Nancy Reichman, Rebecca Sandefur, and Joyce Sterling (hereinafter, Silver et al.) recently wrote a three-part response (Parts 1, 2, and 3) to my two-part blog post from December about International LLM students who remain in the United States (Part 1) and International LLM students who return to their home countries (Part 2). The bulk of Silver et al.’s critique appears in Part 2 of their post, and focuses mainly on Part 1 of my LLM post.
My post, which I described as “a very preliminarily, quick analysis intended primarily to satisfy my own curiosity” used U.S. Census data from the American Community Survey and two proxy groups for international LLM (“Masters of Law”) graduates to make inferences about the financial benefits of LLM degrees to international students who remain in the U.S. Silver et al. agree with several of the limitations of this analysis that I noted in paragraphs 5 through 8 of Part 1 of my post. They also note that historically, many LLMs have returned to their home countries and argue that the benefits of LLM programs to returning students may be greater than the benefits to those who remain in the United States. (While I am skeptical of this last claim—especially if we focus exclusively on pecuniary benefits—it seems likely that both groups benefit).
Silver et al. have also helpfully made several additional points about limitations in my proxy approach and ways in which proxies could over-count or under-count foreign LLMs. The most important of these limitations can be addressed with a few modifications to the LLM proxy group approach. Those interested in the technical details are encouraged to read footnote 1 below.
Returning to broader questions about the use of proxy groups, my view is that proxy groups can be helpful and potentially necessary for certain kinds of analysis.
Suppose that we wish to know the temperature in New York’s Central Park before we take a stroll, but we only have temperature readings for LaGuardia and Newark airport. While neither of those proxies will tell us the precise temperature in Central Park, they will usually be sufficiently close that we can ascertain with a reasonable degree of certainty whether we should bring our winter coats, wear sweaters, or proceed with short sleeves. Indeed, readings from Boston or Philadelphia will probably suffice, particularly if we’re aware of the direction and magnitude of typical temperature differences relative to Central Park.
Should we refuse to venture out until we can obtain a temperature reading from Central Park itself?
January 17, 2017
January 16, 2017
January 12, 2017
Law professors weigh in on Trump's "conflicts of interest" given his non-divestment from his businesses
January 10, 2017
U of Oregon President Mike Schill (former Dean of U of Chicago and UCLA Law Schools) responds to commentary on Shurtz case
January 04, 2017
President-elect Trump intends to nominate Sullivan & Cromwell partner Jay Clayton to head the Securities and Exchange Commission according to reports by the Financial Times and other newspapers. Clayton has extensive expertise in M&A, capital markets, and financial regulation.
Clayton is a graduate of the University of Pennsylvannia Law School, where he taught a class on "M&A Through the Business Cycle" from 2009 to 2015.