Assignments & Evaluation

Overview

As a 3-credit graduate course, my expectation is that you are here to learn, which involves following along with the course through doing the reading and attending class meetings. I am less interested in grades and more interested in providing a venue to hold you accountable for learning the material. I will provide many opportunities for feedback and workshopping, so completing your assignments and making progress on your project will benefit your learning (and hopefully contribute to your progress on your thesis/dissertation). As such, attendance and participation is 25%, leading of paper discussion/presentation is 25%, and the final report and presentation are worth the remaining 50%.

Main assignment

The main assignment will be either to complete and write up a data analysis or to do a literature review and deep dive on a particular subtopic (ideas listed below). The project or literature review will culminate in a presentation the last week of classes and a final written report.

Many students in the past have used this course as an opportunity to advance their thesis research or pursue and publish a paper based on their analyses or literature review!

Final Project Option 1: Data analysis project

For the final project that takes the form of a data analysis, each student will complete a project according to the following process: identifying a research question, creating a DAG, choosing and applying a data analysis, and writing up a methods and results section that 1) articulates the causal relationship of interest, 2) describes the causal identification strategy and choice of study design/methods, 3) describes the implicit assumptions on which this strategy rests, and 4) interprets the results and their potential biases. I encourage students to base this analysis on their thesis research to get the most out of the class.

Final Project Option 2: Literature review

Alternatively, students can complete a literature review that provides a deeper dive on a topic we do not cover in detail in this course (*some ideas are listed below). This direction may be more appropriate for students who are not at a stage of performing analyses for their thesis, who do not have a relevant project that is tackling an empirical causal question, or who are interested in learning more about a topic not covered under our guidance.

Final project presentation

Time guidelines for all presentations: 7 minute presentation + 2 minutes for questions

Specific guidelines for data analysis project presentations:

  • Briefly introduce the question you address and the motivation for your project (as this was covered in the first presentations)
  • Describe the causal inference approach you took and why you chose that approach; as part of this part, you could briefly present your DAG and describe the data you used (as DAGs and data were covered in Presentation 1)
  • Share your results, explain how you checked their robustness, AND contextualize them with the important limitations and assumptions of the method(s) you used, includinbg their assumptions and why or why not they are met
  • Outline possible next steps you could take to make your analysis more robust in the future

Specific guidelines for literature review presentations:

  • Introduce the topic and describe the challenge for causal inference
  • Describe and critique the current approaches people are taking
  • Present your ideas for how the field could better incorporate causal inference methods

Final project write up

Length

  • For data analysis projects: ~ 5 pages (not including tables and figures, but don’t stress over the page count)
  • For literature review projects: ~ 10 pages, but don’t stress over the page count – focus on what you need to include to write a compelling paper

Content for Project Option 1

Final reports should include:

  • A brief introduction (1-2 paragraphs) that establishes your causal question, hypotheses, and a brief motivation
  • DAG and explanation of variables (could also incldue a table explaning variables in your DAG and their data sources)
  • A Methods section, as you would for a publication. Include the identification strategy (which design you use and why), and how you checked the robustness of your results (or sensitivity of results if you preform a sensivity analysis)
  • Results Section: Present your estimates, confidence intervals and inference, and figures or tables of output
  • Brief Discussion of the analysis – the intention here is not to be a discussion as in a publication, but instead a discussion of your the assumptions made by your study design(s), and an assessment of their validity, or assumptions – and what you will do next if you plan to continue your study (e.g. what additional analyses would you like to do and why? what additional data might you need to collect? What other considerations are lurking?)

Content for Project Option 2 

Final reports should include:

  • Motivation for your review: What is the issue, topic, or gap you are interested in and why?
  • Motivation for use of causal inference (or discovery) approaches. Why could they help here, or why is it needed for the topic or question?
  • Review of methods or approaches that could be helpful, which could cover:
  1. How do these approaches/methods compare to what is currently being done on this topic?;
  2. What do you propose to do and which methods or study design would you use – and why?;
  3. discussion of implications and potential uses or changes – or areas of advances that are needed;
    and
  4. brief conclusions.

Additional assignments

In addition to the project/literature review, each student will also lead at least one paper discussion during the semester, which will be either a discussion of a paper describing or applying a study design or a critique of a paper aiming to address a causal question. When leading the paper discussions, we recommend starting off with drawing a diagram of the causal relationships in question in the paper.

Why is this important? Part of learning this material well is learning it to the point where you can explain it to other people. This is true in general in graduate or post-graduate school as you move towards becoming experts and in this class when presenting technical things. The presentations offer an opportunity to practice these skills in a friendly setting.

Presentation Sign Ups

Readings

The course reading list will include journal articles and book chapters that we will make available to you on our course website. Throughout the semester, the class will compile a running glossary of “jargon” and terms and concepts in causal inference to provide notes as reference. We (the instructors) will provide a bibliography of additional readings and coding resources on various topics for students as reference.

The readings differ in their depth and use of math and notation. You do not need to understand all the equations and notation to gain intuition about the material and concepts. Try to learn it but try not to be too stressed by it.

Course glossary

As noted above, as a class we will compile a running glossary of “jargon” and terms and concepts in causal inference to provide notes as reference.

Ideas for literature review topics (Final project option 2)

You are not limited to these topics, but here are some ideas:

  • Challenges and solutions for experimental designs and interpretation (choose one):
    • Non-compliance and attrition (this is a rich area!)
    • Interference between experimental units and spillovers (G&G Ch8)
    • Partial identification (excludability violations, including those common to ecological field experiments)
    • Mechanisms and mediation analysis (G&G Ch 10)
    • Limits to what experiments tell us about heterogenous treatment effects
    • Integration of experimental and quasi-experimental findings (e.g., meta-analyses and extrapolation)
  • New issues for difference-in-difference and two-way fixed effects estimators (e.g., staggered treatments and heterogeneity
  • Synthetic control methods (i.e. see https://mixtape.scunning.com/10-synthetic_control)
  • Machine learning and causal inference
  • Sensitivity analyses and/or placebo designs
  • Mechanism (Mediation) Analysis; Sequential G estimation
  • Remote sensing and causal inference (e.g. what to do with systematic measurement error !)
  • Causal identification in time series
  • Inferences in observational designs and networks
  • Meta-analyses using observational data

Deadlines

Assignment Deadline
Pre-course survey Tuesday 1/14
Identify project focus (option 1 or 2). Identify research questions + create and workshop DAGs for it. Due 2/7 (by email)
Revised draft of Project Question and revised DAG OR literature review proposal (2 page max) 2/19 (by email)
One on One Consultations ~March 17th with Laura
Short presentation on research questions + DAG OR literature review topic, its significance, + proposed methods (prelim results) Wednesday 4/3 (in class)
Presentation of final project April 23rd & 28th (in class)
Final project write-up due by email Sunday 5/4