"Battling Attrition in Field Experiments"

Attrition in field experiments refers to participants dropping out, posing challenges to the validity and reliability of research findings. Mitigation strategies include clear communication, incentives, ease of participation, and statistical adjustments for missing data.

Term Description Attrition in Field Experiments Attrition refers to the phenomenon where participants drop out or fail to complete a field experiment after it has begun. It is a common challenge in research, as the loss of participants can impact the validity and reliability of the experiment's results. Attrition is particularly concerning in longitudinal studies or experiments that require participants to engage over an extended period. In the context of field experiments, attrition can occur for various reasons, including lack of interest, unforeseen personal circumstances, or dissatisfaction with the experiment. Researchers need to carefully plan for and mitigate attrition to ensure that their findings remain robust and generalizable. High attrition rates can lead to biased results, especially if the participants who drop out are systematically different from those who remain in the study. Strategies to reduce attrition in field experiments include maintaining clear communication with participants, offering incentives, ensuring ease of participation, and addressing potential barriers early in the experiment design phase. Additionally, researchers often use statistical techniques to adjust for the effects of attrition and account for missing data.

THE EXPERIMENTER'S ACHILLES' HEEL **Attrition**, the loss of participants over time, is the most persistent threat to the validity of field experiments. It can break the balance created by randomization, reintroducing the very selection bias an experiment is designed to eliminate. Diagnosing the Bias: A Tale of Two Groups Bias arises from **differential attrition**, where the rate or type of dropout differs between groups. This means the people who remain in the treatment group are no longer comparable to those in the control group. 1. Attrition Rates Diverge A higher dropout rate in one group is the first warning sign. Here, the treatment group's rate is more than double the control's. 2. Baseline Balance is Broken Because of differential attrition, the remaining groups are no longer balanced on key pre-treatment characteristics like income. Bounding the Uncertainty When attrition occurs, we can't be certain of the true treatment effect. Bounding calculates a range of plausible effects under different assumptions about the missing participants. Manski vs. Lee Bounds: The Assumption/Precision Trade-off Manski bounds are assumption-free but wide. Lee bounds are tighter but require the untestable "monotonicity" assumption. The Analyst's Toolbox: Correcting for Bias When willing to make stronger assumptions, researchers can use statistical models to generate a single point estimate of the treatment effect. ⚖️ Inverse Probability Weighting (IPW) Gives more weight to observed individuals who are similar to those who dropped out, creating a re-weighted "pseudo-population". 🔧 Heckman Selection Models Explicitly models the attrition process itself, attempting to correct for bias caused by unobserved factors. ✨ Multiple Imputation (MI) "Fills in" the missing data multiple times based on observed patterns, creating several complete datasets for analysis. Prevention is the Best Cure: Proactive Design The most effective strategy is to minimize attrition from the start with thoughtful fieldwork and design. Build Trust & Rapport Be transparent, use empathetic survey staff, and express appreciation to make participants feel like valued partners. Design Smart Incentives Use cash-equivalent incentives and a phased or completion-bonus structure to motivate long-term participation. Collect Rich Tracking Data At baseline, get multiple phone numbers, emails, and contact details for at least two secondary informants. Minimize Participant Burden Keep surveys concise, pilot them extensively, and respect participants' time by offering flexible scheduling.

2-causal-inference    3-hypothesis-testing    4-covariates    5-one-sided-compliance    6-two-sided-compliance    7-attrition    8-mediation    9-spillover    Field-experiment-protection-o