Reposted by Bruno FermanJulian Reif @julianreif.bsky.social · 01/05/2025Good overview of what to do when there are only a few treated units #EconSky 142
Bruno Ferman @brunoferman.bsky.social · 29/04/2025That's a quick overview! For more details, check out the full survey 📚👇 Link: arxiv.org/abs/2504.19841 Hope you find it helpful! Feedback welcome. 🧠✍️arxiv.orgInference with few treated unitsIn many causal inference applications, only one or a few units (or clusters of units) are treated. An important challenge in such settings is that standard inference methods that rely on asymptotic th... 000
Bruno Ferman @brunoferman.bsky.social · 29/04/202515/ Applied folks: we hope this serves as a warning that standard inference may fail with few treated units + guidance on choosing alternatives. Econometricians: we wanted to provide a state-of-the-art overview — and a call for new methods based on alternative assumptions! 100
Bruno Ferman @brunoferman.bsky.social · 29/04/202514/ And show some equivalences: e.g., wild-bootstrap (with null imposed) asymptotically equivalent to sign-changes when N₁ is fixed and N₀ → ∞ ⇒ theoretical justification for wild-bootstrap in these settings 100
Bruno Ferman @brunoferman.bsky.social · 29/04/202513/ We also provide finite-N₀ improvements for some methods, such as Conley-Taber and sign-changes. Free lunch: gains with finite N₀ & asymptotic equivalent when N₀ → ∞ (with N₁ fixed) 100
Bruno Ferman @brunoferman.bsky.social · 29/04/202512/ What if we have >1 treated (but still few)? More info on treated ⇒ alternatives: sign-changes, Behrens-Fisher solutions, etc Relax some assumptions relative to previous methods (but need new ones!) ⚡Power may be an issue when N₁ is very small Many relevant trade-offs! 100
Bruno Ferman @brunoferman.bsky.social · 29/04/202511/ In these extreme cases: need to impose strong restrictions on treatment effect heterogeneity! If interested, see discussion in Section 4.1.3 on inference on sharp nulls, inference on realized treatment effects, prediction intervals, and sensitivity analysis. 100
Bruno Ferman @brunoferman.bsky.social · 29/04/202510/ 📌Extrapolate from time series Learn about treated error using pre-treatment residuals ⚡Flip assumptions Need time series restrictions (stationarity) but relax assumptions on cross-section Challenges arise when counterfactuals are estimated via high-dimensional approaches 100
Bruno Ferman @brunoferman.bsky.social · 29/04/20259/ Ferman and Pinto (2019): allow for heteroskedasticity that can be estimated based on observables. Example: when units have different variances due to variation in population sizes. See this old Twitter thread: x.com/bruno_ferman...x.com 120
Bruno Ferman @brunoferman.bsky.social · 29/04/20258/ 📌Extrapolate (learn) from control units Learn the distribution of the treated error using controls' residuals (à la Conley and Taber) ⚡Key assumption: Errors of treated and control units must have the same distribution (homoskedasticity) No restriction on time series! 100
Bruno Ferman @brunoferman.bsky.social · 29/04/20257/ Survey is organized based on data availability. 📌Limit case: One treated unit & one treated period. Enough info from the treated to construct an estimator — but no info from the treated to learn its distribution! ⚡Solution: We need to *extrapolate* ⇒ stronger assumptions! 100
Bruno Ferman @brunoferman.bsky.social · 29/04/20256/ We focus on model-based approaches, more common in metrics 📚 Nice citation from Haavelmo to justify this framework + marvel movies to help make the point 🕷️: ) We also discuss design-based approaches at the end 120
Bruno Ferman @brunoferman.bsky.social · 29/04/20255/ Important: 📌Problems arise when the *number* of treated units is small ✅Standard methods are usually fine with 40 or 50 treated units, even when the *share* of treated is small. Feel free to cite our survey to justify sticking to standard methods when that's your case!😉 130
Bruno Ferman @brunoferman.bsky.social · 29/04/20254/ Extreme case: you have only 1 treated and N₀ controls. The true variance is σ₁² + σ₀²/N₀. But with only one treated, you just don’t have enough info to estimate σ₁² using only the treated! Robust SEs simply set σ̂₁² = 0! 😵💫 σ₁²: var of treated σ₀²: var of control 110
Bruno Ferman @brunoferman.bsky.social · 29/04/20253/ Example to illustrate problem: comparison of means Robust SEs estimate the variance of treated (controls) using only treated (controls) data ✅ Great with many treated/many controls! ↪️ Allow for ≠ distributions of treated/control errors ❗ Go bad with few treated units... 100
Bruno Ferman @brunoferman.bsky.social · 29/04/20252/ 🗣️Main message Few treated ⇒ need to rely on stronger assumptions Many alternatives: varying in data requirements, assumptions, etc Choice is highly context-specific. We’ll help you navigate that! Cover cross-section and panel data (Regression, Matching, DiD, SC, etc) 120
Bruno Ferman @brunoferman.bsky.social · 29/04/20251/ Link to paper: arxiv.org/abs/2504.19841 🚨Problem Few treated ⇒ standard methods (e.g., robust/clustered SEs) can go wrong. Even if total N is large! 📌Example DiD with 1 treated cluster, clustered SEs underestimate true var by a factor of N. Expect over-rejections >60%!arxiv.orgInference with few treated unitsIn many causal inference applications, only one or a few units (or clusters of units) are treated. An important challenge in such settings is that standard inference methods that rely on asymptotic th... 100
Bruno Ferman @brunoferman.bsky.social · 29/04/2025🧵New survey paper: "Inference with Few Treated Units" Luis Alvarez, Bruno Ferman and Kaspar Wüthrich Tired of referees saying your standard errors are wrong? This survey will help you understand if you really have a problem — and, if so, how to fix it! 15714
Reposted by Bruno FermanNico Ajzenman @nicolasajz.bsky.social · 27/01/2025Our "Discrimination in the Formation of Academic Networks: A Field Experiment on #EconTwitter" with Pedro Sant'Anna and @brunoferman.bsky.social is now forthcoming in the 😀😀 American Economic Review Insights (@AEAjournals ) 😀😀 papers.ssrn.com/sol3/papers....papers.ssrn.comDiscrimination in the Formation of Academic Networks: A Field Experiment on #EconTwitterThis paper experimentally documents discrimination in the formation of professional networks among academic economists. We created fictitious human-like bot acc 36622