How to become a better empiricist, or at least start using empirical methods (Michael Simkovic)

I recently wrote about the evolution of economics--and law & economics--from fields that focused on assumptions and priors to fields that emphasizes data, causal inference, and scientific objectivity.  Many law professors and aspiring academics share my enthusiasm for Albert Einstein's vision of universities as “Temples of Science”, but are unsure of how to acquire or sharpen the technical skills that will make them effective empiricists.

Bernard Black at Northwestern runs extremely helpful and practical summer workshops that I highly recommend. The quality of Professor Black's workshops easily justifies the cost.  (There are free law & economics workshops--and some that will even pay you a stipend to attend--but from what I have seen, these  tend to present non-empirical methods and political view points).

Details about Professor Black's workshop are available below the break.

2018 Northwestern-Duke Main and Advanced Causal Inference Workshops 

 

[please recirculate to others who might be interested]

Northwestern University and Duke University are holding our “main” week-long workshop on Research Design for Causal Inference – our ninth annual workshop -- at Northwestern Law School in downtown Chicago.  We invite you to attend.  Our apologies for the length of this message.

 

Main Workshop:  Monday – Friday, June 18-22, 2018

 

We will also be holding an “Advanced” Workshop the following week:

 

Advanced Workshop:  Monday – Wednesday, June 25-27, 2018

 

Both workshops will be taught by world-class causal inference researchers.  See below for details.  Registration is limited to around 100 participants.  In the past we have filled the main workshop quickly.  So please register soon.

For information and to register: www.law.northwestern.edu/research-faculty/conferences/causalinference/   

 

Workshop Organizers

Bernard Black (Northwestern University)

Bernie Black is Nicholas J. Chabraja Professor at Northwestern University, with positions in the Pritzker School of Law, the Institute for Policy Research, and the Kellogg School of Management, Finance Department.  Principal research interests: health law and policy; empirical legal studies, law and finance, international corporate governance.  Web page with link to CV: www.law.northwestern.edu/faculty/profiles/BernardBlack/. Papers on SSRN: http://ssrn.com/author=16042.

 

Mathew McCubbins (Duke University) 

Professor of Political Science and Law at Duke University, with positions in the Political Science Department and the Law School, and director of the Center for Law and Democracy.  Principal research interests: democratic institutions, legislative organization; behavioral experiments, communication, learning and decisionmaking; statutory interpretation, administrative procedure, research design; network economics.  Web page with link to CV:  www.mccubbins.us.  Papers on SSRN:  http://ssrn.com/author=17402.

 

Main Workshop Overview:  Research design for causal inference is at the heart of a “credibility revolution” in empirical research.  We will cover the design of true randomized experiments and contrast them to natural or quasi experiments and to pure observational studies, where part of the sample is treated in some way, the remainder is a control group, but the researcher controls neither the assignment of cases to treatment and control groups nor administration of the treatment.  We will assess the causal inferences one can draw from a research design, threats to valid inference, and research designs that can mitigate those threats.

 

Most empirical methods courses survey a variety of methods.  We will begin instead with the goal of causal inference, and emphasize how to design research to come closer to that goal.  The methods are often adapted to a particular study.  Some of the methods are covered in PhD programs, but rarely in depth, and rarely with a focus on credible causal inference and which methods to use with messy, real-world datasets and limited sample sizes.  Several workshop days will include a Stata “workshop” to illustrate selected methods with real data and Stata code.

 

Advanced Workshop Overview:  The advanced workshop provides in-depth discussion of selected topics that are beyond what we can cover in the main workshop.  Principal topics for 2018 include:  Day 1 (Mon.):  Principal stratification (generalization of causal-IV concepts and applications, including sample censoring through death or attrition.   Day 2 (Tues.):  Direct and indirect causal effects.  Synthetic controls and other advanced “matching” approaches with emphasis on panel data sets.  Day 3 (Wed.):  Application of machine learning methods to causal inference.

 

Target audience for main workshop:  Quantitative empirical researchers (faculty and graduate students) in social science, including law, political science, economics, many business-school areas (finance, accounting, management, marketing, etc), medicine, sociology, education, psychology, etc. –anywhere that causal inference is important.

 

We will assume knowledge, at the level of an upper-level college econometrics or similar course, of multivariate regression, including OLS, logit, and probit; basic probability and statistics including conditional and compound probabilities, confidence intervals, t-statistics, and standard errors; and some understanding of instrumental variables.  Despite its modest prerequisites, this course should be suitable for most researchers with PhD level training and for empirical legal scholars with reasonable but more limited training.  Even for recent PhD’s, there will be much that you don’t know, or don’t know as well as you should.

 

Target Audience for Advanced Workshop: Empirical researchers who are reasonably familiar with the basics of causal inference (from our main workshop or otherwise), and want to extend their knowledge.  We will assume familiarity with potential outcomes notation, difference-in-differences, regression discontinuity, panel data, and instrumental variable designs, but will not assume expertise in any of these areas.

 

Main Workshop faculty (in order of appearance)

Donald B. Rubin (Harvard University, Department of Statistics)

Donald Rubin is John L. Loeb Professor of Statistics, Harvard University.  His work on the “Rubin Causal Model” is central to modern understanding of when one can and cannot infer causation from regression.  Principal research interests:  statistical methods for causal inference; Bayesian statistics; analysis of incomplete data.  Web page, with link to CV: https://statistics.fas.harvard.edu/people/donald-b-rubin; Wikipedia: http://en.wikipedia.org/wiki/Donald_Rubin 

 

Justin McCrary (University of California, Berkeley, Law School)

Justin McCrary is Professor of Law, University of California, Berkeley.  Principal research interests: crime and urban problems, law and economics, corporations, employment discrimination, and empirical legal studies.  Web page with link to CV: http://www.econ.berkeley.edu/~jmccrary/.

 

Jens Hainmueller (Stanford University, Department of Political Science)

Jens Hainmueller is Professor in the Stanford Political Science Department, and co-Director of the Stanford Immigration Policy Lab.  He also holds a courtesy appointment in the Stanford Graduate School of Business.  His research interests include statistical methods, political economy, and political behavior.  Web page with link to CV:  http://www.stanford.edu/~jhain//.  Papers on SSRN: https://ssrn.com/author=739013.

 

Advanced Workshop Faculty (in order of appearance)

Donald Rubin (see above)

 

Fabrizia Mealli (University of Florence, Department of Statistics and Computer Science)

Fabrizia Mealli is Professor of Statistics at the University of Florence and external research associate at the Institute for Social and Economic Research (ISER) at the University of Essex.  Her research focuses on causal inference and simulation methods, program evaluation, missing data, and Bayesian inference.  She is a fellow of the American Statistical Association, and associate editor of Journal of the American Statistical Association (JASA), Biometrics, and Annals of Applied Statistics. Web page with link to CV:  http://local.disia.unifi.it/mealli/ 

 

Yiqing Xu (University of California San Diego, Department of Political Science)

Yiqing Xu is Assistant Professor of Political Science at University of California, San Diego. His main methods research involves causal inference with panel data.  Website: http://yiqingxu.org/.

 

Justin Grimmer (University of Chicago, Department of Political Science)

Justin Grimmer is Associate Professor of Political Science at the University of Chicago.  His primary research interests include political representation, Congressional institutions, and text as data methods.  Website:https://www.justingrimmer.org/

 

Main Workshop Outline

Monday June 18 (Donald Rubin): Introduction to Modern Methods for Causal Inference

Overview of causal inference and the Rubin “potential outcomes” causal model.  The “gold standard” of a randomized experiment.  Treatment and control groups, and the core role of the assignment (to treatment) mechanism.  Causal inference as a missing data problem, and imputation of missing potential outcomes.  Rerandomization.  One-sided and two-sided noncompliance.  

 

Tuesday June 19 (Justin McCrary): Matching and Reweighting Designs for “Pure” Observational Studies

The core, untestable requirement of selection [only] on observables.  Ensuring covariate balance and common support.  Subclassification, matching, reweighting, and regression estimators of average treatment effects.  Propensity score methods.  Methods that aim directly at covariate balance.

 

Wednesday June 20 (Justin McCrary): Instrumental variable methods

Causal inference with instrumental variables (IV), including (i) the core, untestable need to satisfy the “only through” exclusion restriction; (ii) heterogeneous treatment effects; and (iii) intent-to-treat designs for randomized trials (or quasi-experiments) with noncompliance.  

 

Thursday June 21 (Jens Hainmueller): Panel Data and Difference-in-Differences

Panel data methods:  pooled OLS, random effects, correlated random effects, and fixed effects.  Simple two-period DiD.  The core “parallel changes” assumption.  Testing this assumption.  Leads and lags and distributed lag models.  When does a design with unit fixed effects become DiD?  Accommodating covariates.  Triple differences.  Robust and clustered standard errors.  Introduction to synthetic controls.

 

Friday morning June 22 (Jens Hainmueller): Regression Discontinuity

(Regression) discontinuity (RD) research designs: sharp and fuzzy designs; bandwidth choice; testing for covariate balance and manipulation of the threshold; discontinuities as substitutes for true randomization and sources of convincing instruments.

 

Friday afternoon:  Feedback on your own research

Attendees will present their own research design questions from current work in breakout sessions and receive feedback on research design.  Session leaders:  Bernie Black, Mat McCubbins, Jens Hainmueller.  Additional parallel sessions if needed to meet demand.

 

Stata and R sessions

On Tuesday, Wednesday, and Thursday, we will either run parallel Stata and R sessions to illustrate actual code to implement the designs discussed in the lectures, or build Stata code into the lecture slides.

 

Advanced Workshop Outline

Monday June 25 (Donald Rubin and Fabrizia Mealli): Principal Stratification and Censoring

Generalizing the causal-IV strata of compliers-always takers-never takers-defiers.  Which treatment effects can be estimated for which strata?  Handling missing data and censoring through “death” or attrition.

 

Tuesday June 26 morning (Donald Rubin and Fabrizia Mealli): Direct and indirect causal effects.  

“Mediation” analysis:  Direct and indirect causal effects versus principal associative and dissociative effects.

 

Tuesday June 26 afternoon (Yiqing Xu): Advanced matching

Advanced matching and reweighting methods, with an emphasis on panel data applications.  Generalized synthetic controls.  Relative strengths and weaknesses of different matching and reweighting approaches.

 

Wednesday June 27 (Justin Grimmer): Machine learning (predictive inference) meets causal inference

Introduction to machine learning approaches.  When and how can machine learning methods be applied to causal inference questions.

 

Registration and Workshop Cost

Main Workshop: tuition is $900 ($600 for graduate students (PhD, SJD, or law) and post-docs.  The workshop fee include all materials, temporary Stata 15 license, breakfast, lunch, snacks, and an evening reception on the first workshop day.

 

Advanced Workshop:  tuition is $600 ($400 for graduate students (PhD, SJD, or law) and post-docs.  There is a $100 discount for persons attending both workshops.

 

You can cancel from either workshop five weeks in advance (May 14 for main workshop, May 21 for advanced workshop) for a 75% refund and by three weeks in advance 50% refund (in each case, less credit card processing fee), but there are no refunds after that.  

 

We know the workshop is not cheap.  We use the funds to pay our speakers and for meals and other expenses; we don’t pay ourselves.

 

Workshop Schedule

You should plan on full days, roughly 9:00-5:00.  Breakfast will be available at 8:30.

Questions about the workshops

Please email Bernie Black ([email protected]) or Mat McCubbins ([email protected]) for substantive questions or fee waiver requests, and Laura Dimitrijevic ([email protected]) for logistics and registration.