Deep DiD audit — collaborative follow-up
Under development — collaborative audit follow-up, 13 October 2026. These are specification candidates and implementation questions, not a new collection of successfully replicated studies.
Back to Under development · 245-paper reconciliation: CSV · Source tab: Deep DiD audit — 245
Contribution and scope
Pedro H. C. Sant’Anna supplied an independent agent audit covering all 245 archive papers. Its original tab is preserved, credited and snapshotted without rewriting the contributor’s assessments. This follow-up joins all 245 rows by DOI, including the five papers shared with Peter Hull’s original collection. The additional 17 legacy papers were not part of the new tab.
The contributed audit reports 122 observational binary temporal contrasts, including 87 potential absorbing-treatment cases. These are broader than the currently implemented adoption-panel comparisons: they include secondary outcomes, graphical contrasts, first differences, nonlinear models, reversible treatments and specialized distributional targets. They must not be reported as 122 or 87 ready-to-run replications.
Reconciliation with the earlier screen
| Contributed candidates | Previously included | Previously pending | Previously excluded | Total |
|---|---|---|---|---|
| Binary observational temporal contrasts | 49 | 47 | 26 | 122 |
| Absorbing-pattern candidates (subset) | 41 | 31 | 15 | 87 |
There are 73 binary observational candidates not previously admitted, of which 46 carry the contributor’s absorbing-candidate flag. This is a useful implementation queue, not evidence that 73 new fits are available. All currently admitted archive papers have a binary-observational target in the contributed audit, but correspondence to our exact saved specification still requires checking.
Independent checks and changes
All 26 disagreements with exclusions received targeted original-code inspection. In 24, the cited author code implements the proposed binary regression, event study, distributional comparison or graph. In two, the corresponding code is absent from the inspected deposit; the reported target was checked independently in the published PDF instead. The other 219 rows are reconciled only, not newly independently certified by this follow-up. No exhaustive second reading of every paper is claimed.
The 26 assessments are reopened as named candidates with implementation requirements. The administrative register now has 64 included, 78 pending and 120 provisionally excluded. Neither 120 nor the earlier totals establishes that those papers contain no usable DiD. No candidate is automatically admitted to calibration plots, and no paper is ruled out merely because another section uses IV, an experiment or continuous exposure. Experimental assignment, IV second stages and continuous-dose targets remain distinct estimands; a separate observational binary analysis is reviewed on its own terms.
Original collection and numerical results: all 22 original papers remain intact. No estimation was performed, and all existing comparison records and plot samples are unchanged. Historical run receipts remain historical; the linked current register supersedes old design-screen prose.
Specific improvements to the audit
- Graphs versus coefficients: the China WTO markup and textbook-funding examples provide authored binary group profiles, not a supplied scalar DiD ATT. Choosing a new pooled effect would be an extension, not reproduction of an original coefficient.
- Functional form and generated outcomes: the bracero target uses Poisson; the unemployment-insurance target applies DiD to bunching statistics. Neither should be silently converted to an ordinary linear outcome ATT.
- Treatment paths: the SNAP row says ‘absorbing candidate’ while its pattern explicitly says absorption is unverified. Keep it a candidate until the actual policy columns and reversals are checked, separately for each policy.
- Design-matrix fidelity: the virtual-classroom appendix codes existing campuses into the opening-term category (100). Its normalization must be reconciled before assigning a conventional never-treated indicator. Its ZIP supplies code/readme, not the microdata.
- Missing target code is not missing design evidence: the ski-resort appendix and Perth gasoline paper report explicit binary temporal comparisons even though the corresponding code is missing from the inspected deposit.
- Bibliography: the SNAP paper’s published-title correction is retained alongside the original supplied title, rather than changing its identity or DOI.
Reviewed disagreements
School Inputs, Household Substitution, and Test Scores (aeraej-210)
Proposed authored target: Zambia Table 7 cols 2/5: binary discretionary funding effect on one-year test-score gains
Follow-up: Binary discretionary-funding receipt is used in test-score gain regressions (lines 99 and 105), separate from the randomized India sample. Preserve continuous rule funding as a control; construct baseline/endline and resolve receipt selection. This is not permanent policy adoption.
Implementation queue: First-difference / receipt target. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: Replication-Files/stata/ZambiaStataDoFiles/AEJ_Zambia_Funding.do, lines 72–76. SHA-256: 838cf01f4728f1a676ffb18c8e908faabfe1fa2edbc197b68834989c551b6f4b. The anchor records source correspondence, not a successful execution.
Trade Liberalization and Markup Dispersion: Evidence from China’s WTO Accession (aeraej-29)
Proposed authored target: Figure 4 high-versus-low tariff industry changes; main Tables 2–5 continuous tariff×post
Follow-up: The authors split industries at the 2001 tariff median and estimate separate group time profiles. This authenticates the binary graphical comparison, not a reported joint DiD ATT or its variance. Any new pooled scalar model must be labeled an extension, not a reproduction of an authored coefficient.
Implementation queue: Graphical contrast; scalar target not supplied. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: AEJApp-2014-0350/PROGRAM/AEJ_Figures_.do, lines 33–48. SHA-256: 5a95bab84eb8e1f8dbc9e679d4e02f081715d5740950b151a3c4315bbe2277bc. The anchor records source correspondence, not a successful execution.
Understanding the Changing Structure of Scientific Inquiry (aeraej-28)
Proposed authored target: Table 2: field collaboration after Soviet opening
Follow-up: The authors define top-10 and top-5 Soviet-field indicators and interact them with afterdummy. Explicit year/field dummies are present. xtreg lacks the fe option and defaults to random effects despite explicit dummies; do not silently relabel its estimator. Poisson is a separate specification. Check omitted columns and authors coefficient before mapping.
Implementation queue: Linear and nonlinear targets; map sample. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: Data/SovietCollaboration.do, lines 8–25. SHA-256: 2ea1ab91a627e5d4fcf9bc6f6ac78e8b912533097f68c09cfd7d9aabce41e0ae. The anchor records source correspondence, not a successful execution.
Unintended Effects of Anonymous Résumés (aeraej-111)
Proposed authored target: Table 6 John Henry monitoring effect robustness
Follow-up: The control-firm sample compares minority versus other candidates during versus after experimental monitoring, with BEFORE_D, BEFORE and ZouI, weights and vacancy clustering. It is a secondary observational interaction, not randomized policy assignment or an absorbing adoption panel.
Implementation queue: Repeated cross-section / monitoring contrast. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: 20140185_programs/20140185_program_tab6.do, lines 76–88. SHA-256: 5e343b32405fe2777af19da719d3239fdb595e5575102028691ea911bd955a90. The anchor records source correspondence, not a successful execution.
Wintertime for Deceptive Advertising? (aeraej-628)
Proposed authored target: Online Appendix Table A8: iPhone monitoring launch and snow-report exaggeration
Follow-up: Publisher Appendix A8 independently confirms coverage x post-launch x weekend with lower-order effects and resort/day clustering. The inspected deposit contains the main weekend regressions, not A8 implementation. Coverage is defined by at least ten post-launch reports; preserve that definition and flag its identification implications. Do not substitute the recurring weekend indicator for policy adoption.
Implementation queue: Published DDD; corresponding code missing. Status: pending; no new fit.
Contributor row · Original author deposit
Independent PDF check: Online Appendix A8; one-based PDF page 8. SHA-256: 0b3ecbc58bc78e00f8875a5c72ffdcb37401ec77f4e90676566840c1aef8c4ea.
Author-code anchor: wintertime/wintertime_final.do, lines 148–156. SHA-256: 464c92c2021e1fd595537e0aa16876fc584104b6470ebd6468b6b92ab3284686. This anchor shows the available code, not the missing target implementation.
Buy the Book? Evidence on the Effect of Textbook Funding on School-Level Achievement (aeraej-107)
Proposed authored target: Figure 3B: district textbook spending comparison
Follow-up: The code collapses spending by year and affected-district group and plots group means around 2004. It does not estimate a scalar DiD in this file. Recover pre-collapse district data; do not call a newly chosen scalar effect an original replication. Displayed bands are not a DiD standard error.
Implementation queue: Graphical contrast; scalar target not supplied. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: 2015_0112_data/results_district_spending.do, lines 21–38. SHA-256: 81e63c585b024cba74647546c6b2dc8fb93b3412ce53157e3a8581a9760900be. The anchor records source correspondence, not a successful execution.
Does Early Life Exposure to Cigarette Smoke Permanently Harm Childhood Welfare? Evidence from Cigarette Tax Hikes (aeraej-162)
Proposed authored target: Figure 2 large cigarette-tax-hike event-time coefficients; main tables continuous in-utero tax dose
Follow-up: Model 5 uses event indicators, whereas model 6 uses continuous tax09. The setup file drops hikes below 25 cents and limits overlapping hikes and window coverage. The threshold is authored, not invented here; retain variable hike sizes, original windows and microdata-access requirements.
Implementation queue: Author-defined event window; dose / access checks. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: dataanddofiles/figuresnEV/Fig2_ev_25cent.do, lines 29–41. SHA-256: cec7cccd92d9ea6ccd3e4f0812ae78ccd8c345790480e2fc46b47aaa5334407d. The anchor records source correspondence, not a successful execution.
The Local Political Economy Effects of School Construction in Indonesia (aeraej-390)
Proposed authored target: Table 3 column 1: first post-1992 election and public goods
Follow-up: Table 3 column 1 estimates binary post92 with village and year effects. Column 3 separately interacts post92 with binary INPRES exposure. These are not the same target as continuous construction intensity. Verify election timing, sample and exact column before attaching any saved fit.
Implementation queue: Linear binary target; map selected column. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: AEJApp-2015-0447_Dataset/VHeduc_Do.do, lines 118–120. SHA-256: 35ee9e000b5e524dd4778d823099937f4e888b8419d538ac8a76cf2f5df0debb. The anchor records source correspondence, not a successful execution.
Labor Supply and Directed Technical Change: Evidence from the Termination of the Bracero Program in 1964 (aeraej-47)
Proposed authored target: Online Appendix TableA5 binary crop-exposure robustness
Follow-up: Table A5 estimates treatment_binary in ppmlhdfe with crop/year effects; gen_patents_crops.do lines 69 and 86–92 define above-median 1964 exposure x post-1965. Genuine authored binary DiD, but the selected target is Poisson, not a linear ATT. Retain link function and specify the effect scale.
Implementation queue: Poisson adapter required. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: process/stata/tables_patents_crops.do, lines 381–394. SHA-256: 39857c40ab52adb82a028005d1786862ddaa34b1779e99f79388c2d538744826. The anchor records source correspondence, not a successful execution.
Intergenerational Mobility in India: New Measures and Estimates (aeraej-494)
Proposed authored target: Equation 6.1: Scheduled Caste status and education ranks
Follow-up: Code defines late-status x post-1970 cohort and fits it with jati-region, region-cohort and jati-cohort effects (lines 83–94), weighted and multiway-clustered. It is a binary observational comparison with richer cohort structure, not a simple calendar-year adoption panel.
Implementation queue: Cohort / DDD adapter and inference. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: code/a/graph_mech_aa.do, lines 48–56. SHA-256: 7d5de77f31ec5b9be00473f4e12d7916010d9cdbd3ba03291e41d1ae748b3e45. The anchor records source correspondence, not a successful execution.
Taxes Today, Benefits Tomorrow (aeraej-127)
Proposed authored target: Appendix E Figures E1–E3: 1983 earnings-disregard reform
Follow-up: D and T are explicitly defined and used to form treated/control before/after bunching cells (lines 228 onward). This is genuine DiD structure for distributional statistics; raw earnings are not an interchangeable outcome. Preserve first-stage bunching construction and uncertainty.
Implementation queue: Generated bunching outcome; custom inference. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: replication package AEJ Partial UI/do/6.DiD_appendixE.do, lines 42–51. SHA-256: 0d6c4951312494ae53c27d5e371c7cba4fb457d70b6bf54f0d07bd6158ed61dd. The anchor records source correspondence, not a successful execution.
Green Infrastructure: The Effects of Urban Rail Transit on Air Quality (aeraej-84)
Proposed authored target: Table 8 / original reg5.do: Taipei versus comparison-city opening effect
Follow-up: Table 8 directly estimates opening x Taipei against Kaohsiung, with authored 30-day windows, trends/weather in controlled columns and week clustering. This is separate from the main time-discontinuity specifications. Only two cities: generic many-unit clustered inference is not a faithful replacement.
Implementation queue: Two-city target; time-series inference. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: data_programs/programs/regressions/reg5.do, lines 49–57. SHA-256: 6bc390e1b5fde45c5eba65ceaa4adf002f94818d73b272f3059e531de372d5b6. The anchor records source correspondence, not a successful execution.
The Effect of a Universal Child Benefit on Conceptions, Abortions, and Early Maternal Labor Supply (aeraej-83)
Proposed authored target: Tables4–6 seasonal DiD robustness, especially maternal employment Table 5
Follow-up: Table 5 includes distinct local-discontinuity models and an all-sample post indicator with birth-time polynomials, birth-month and interview-quarter effects (line 56; employed outcome line 65). Do not conflate them. Reconstruct cohort/seasonal comparison and preserve polynomial controls before claiming an adoption ATT.
Implementation queue: Seasonal / difference-in-discontinuities mapping. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: Data_20110196/dofile_lfs_20110196.do, lines 50–56. SHA-256: adc283a391bad850adbf2847f3a167c11c23ef4ebf1b918e476c4b9c742aab4b. The anchor records source correspondence, not a successful execution.
Illegal Immigration, State Law, and Deterrence (aeraej-106)
Proposed authored target: Table 5 Arizona immigration law effect on day-labor demand
Follow-up: Table 5 includes Arizona passage and subsequent injunction interactions, with different city or date effects by column. Preserve both phases and the exact sample; collapsing to first adoption would change the estimand.
Implementation queue: Multiple phases / reversals. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: Data_upd/Code-Tables.do, lines 76–85. SHA-256: 3b267ea73185beaa411374ebb8e962c6a5af81df65e38300e7e7b3ba0d25331e. The anchor records source correspondence, not a successful execution.
The Decline, Rebound, and Further Rise in SNAP Enrollment: Disentangling Business Cycle Fluctuations and Policy Changes (DOI matches; supplied title preserved). (aeraej-543)
Proposed authored target: Online Appendix TableA4 individual SNAP-policy adoption effects
Follow-up: The code estimates eight individual policies jointly and separately (lines 42–50), not only their average index. The source flags this as an absorbing candidate but its own pattern says absorbing status unverified. Code alone does not establish 0/1 support or no reversals: stateTakeup.do recodes selected fields, constructs veh by addition and noShortCert with a missing-value rule. Restore external inputs, count each actual policy and check transitions; choose one authored target and retain other-policy controls.
Implementation queue: Multiple-policy binary candidates; paths unverified. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: data/indiv_state_policy.do, lines 21–29. SHA-256: bde66bc951f9ec8ded4df3287494a6bad58ab3b62d92bbd967d2aa6d1f7cc4cd. The anchor records source correspondence, not a successful execution.
The Role of Electoral Incentives for Policy Innovation: Evidence from the US Welfare Reform (aeraej-252)
Proposed authored target: Eq.5 lame-duck effect on welfare experimentation and its electoral-support interaction
Follow-up: The source fits binary lameduck and its interaction with continuous electoral margin, with state/year effects. A standalone lameduck coefficient is conditional on the interaction normalization, not a generic ATT. Preserve reversible spells and margin controls; no first-adoption recoding.
Implementation queue: Reversible status and continuous interaction. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: experiment_replication.do, lines 47–53. SHA-256: b948d607bbb142975e1accea1f3e9afaf0b5cad1d3f7e5805af9b2a5466e5192. The anchor records source correspondence, not a successful execution.
Women’s Suffrage and Children’s Education (aeraej-44)
Proposed authored target: Figure 5 city education expenditures/enrollment; appendix voting models
Follow-up: Figure 5 uses city-level outcomes, event-time indicators and baseline-characteristic interactions, population weights and state-clustered inference. Binary suffrage events differ from the paper main childhood-exposure measure. Preserve event bins and evaluate the authored heterogeneous contrast, not an arbitrary pooled average.
Implementation queue: Event-study heterogeneity; target mapping. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: dofiles/main_results.do, lines 611–625. SHA-256: c9481661ae597f8b34b6d5293049c2de883d9b95e25a333391ce54b6a1588cb9. The anchor records source correspondence, not a successful execution.
Adaptation to Environmental Change: Agriculture and the Unexpected Incidence of the Acid Rain Program (aeraej-149)
Proposed authored target: Figure 1B: sulfur-control installation effects on boiler SO2 emissions
Follow-up: The binary ever-installed group is interacted with installation-relative time, with boiler and year effects. Earlier lines 241–278 define actual installation and never-treated timing. Outcome preparation carries forward sparse early emissions and fills missing emissions with zero; preserve and disclose these authored conventions rather than silently using raw annual outcomes.
Implementation queue: Installation event study; outcome preparation. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: acid_crops_AEJ_EP_data_build_R1.do, lines 305–325. SHA-256: 3d3bb2286ddd11cbaa86ab8b4a9b1ea395de1040e71d98586cd3eab1927ab731. The anchor records source correspondence, not a successful execution.
School Spending and Student Outcomes: Evidence from Revenue Limit Elections in Wisconsin (aeraej-94)
Proposed authored target: Effects of narrowly passing operational and capital referenda on Wisconsin school outcomes
Follow-up: The code uses repeated operational/capital referendum wins, their leads/lags, vote-share polynomials and district/year effects; reported effects are linear combinations (lines 82 onward). This is a binary temporal contrast, but not a single absorbing policy. Preserve the RD controls, repeated treatments and authored aggregation.
Implementation queue: Repeated RD-DiD target. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: Replication/Do Files/onestep_tables.do, lines 23–33. SHA-256: bc8235262c4273ea773bbdee9137a58df2be7da4c8176983863a34ea91c71bdb. The anchor records source correspondence, not a successful execution.
Consumer Spending and the Economic Stimulus Payments of 2008 (aeraej-293)
Proposed authored target: Stimulus payment receipt effects on household consumption
Follow-up: The authors regress consumption changes on binary iesp and calendar-period effects separately from amount regressions and amount-IV models. Timing-based administrative variation is not an investigator-assigned RCT label. Preserve first differences, receipt window and timing-only samples; one-time receipt is not permanent treatment.
Implementation queue: Receipt-window first differences. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: ParkerSoulelesJohnsonmMcClellandReplication/ESPReplication.do, lines 41–48. SHA-256: c2d4771bee5b64cbd104618aa74b5d137540f26f0a5b2f259ad6d206aa170a73. The anchor records source correspondence, not a successful execution.
Virtual Classrooms: How Online College Courses Affect Student Success (aeraej-555)
Proposed authored target: Appendix Figure 3: campus openings and enrollment
Follow-up: Appendix Figure 3 has authored opening dates and campus/term effects, separate from course-taking IV. Existing campuses are coded open_term=100, the same numeric event category as treated opening term; a conventional never-treated sentinel would not reproduce the exact design matrix. Resolve that normalization and pre-opening support. The supplied ZIP has code/readme, no analysis microdata.
Implementation queue: Secondary opening event study; restricted data / normalization. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: AER20151193_analysis.do, lines 329–352. SHA-256: 42b650957ccecef81d1b9b71c98dddf2694d831b39182247c54cb42a7fc9ddca. The anchor records source correspondence, not a successful execution.
Learning to Coordinate: A Study in Retail Gasoline (aeraej-138)
Proposed authored target: Reported Perth margin increase after March 2010 relative to other Australian cities
Follow-up: Published page 597 independently reports Perth-versus-other-cities DiD 3.49 cpl (SE 0.39). The cited code only computes Perth pre/post means; it does not reproduce that multicity estimate. Obtain the exact comparison-city series and specification. Endogenous coordination timing and source-data access remain separate questions.
Implementation queue: Published DiD; corresponding code missing. Status: pending; no new fit.
Contributor row · Original author deposit
Independent PDF check: AER 109(2), p. 597; one-based PDF page 7. SHA-256: c46cc8292b85a5732ec685f80d8bf9a02a4833a464ad9d20bc4ce24c436416df.
Author-code anchor: data/Section3_Facts.do, lines 52–57. SHA-256: 41e7142b88873146e28a1922a9c51fff55eda3bcfffca0cfbafa37cd4dae4672. This anchor shows the available code, not the missing target implementation.
Product Innovation, Product Diversification, and Firm Growth: Evidence from Japan’s Early Industrialization (aeraej-217)
Proposed authored target: Table 3 columns4/6: additional upgrade trials when high-end machine installation coincides with mandated industry output cuts
Follow-up: Code has binary expansion vert_exp x common binary output-cut sotan in Poisson with half-year effects, not firm FE. vert_exp denotes current expansion (not fixed ownership); both group composition and output cuts change. This supports the source specialized comparison, not an absorbing-adoption ATT. Do not strip the link function or add firm effects and call it replication.
Implementation queue: Nonlinear repeated-shock interaction. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: Final-data-and-programs/main_text_replication_code.do, lines 373–388. SHA-256: e3d66130229dbfce0348e78606b89b24167860bf4f2ea32a850070c26c931fcb. The anchor records source correspondence, not a successful execution.
Propagation and Insurance in Village Networks (aeraej-15)
Proposed authored target: Table 1 household health-shock direct effects; event profiles
Follow-up: The source uses Post x Treatment with household, calendar month and event-time effects. The stack construction retains controls shocked more than 24 months later; absorbing status is within the chosen health-shock window, not a lifetime state. Preserve stack IDs, reused controls and household inference.
Implementation queue: Stacked shock-window adapter. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: Replication_pkg/code/03_analysis/03_02_Table1.do, lines 27–43. SHA-256: 7a4fb590daea141f2cfbc14ff2a09cebfa55179fe3344b2757bd178660651eec. The anchor records source correspondence, not a successful execution.
Creating Moves to Opportunity: Experimental Evidence on Barriers to Neighborhood Choice (aeraej-130)
Proposed authored target: AppendixE KCHA2016 and SHA2018 payment-standard reforms
Follow-up: The authors separately estimate KCHA 2016 and SHA 2018 policy DiDs (lines 54–64 and 130–140); the latter explicitly drops experimental treatment families (lines 87–91). This is not the main randomized mobility intervention. The historical administrative data require access; no successful run or ready adapter is implied.
Implementation queue: Observational policy DiD; restricted inputs. Status: pending; no new fit.
Contributor row · Original author deposit
Author-code anchor: CMTO Code Submission/tabs/tab_DD_KCHA_SHA_reforms_oct2022_for_submission.do, lines 21–32. SHA-256: 672be988c528502e8346b652473b56e195a5f772d1f2db4ec97a003e452c30b7. The anchor records source correspondence, not a successful execution.