PDF audit of provisional exclusions

Under development — targeted PDF audit, 12 October 2026. The previous total of 161 exclusions was a provisional code/data/library-note screen, not a verified count of papers without a usable DiD analysis. It should not be cited as such.

Back to Under development · Full 161-row audit register: CSV

Historical snapshot: this page records the October 12 review. The October 13 Deep DiD follow-up supersedes its administrative screen counts and reopens further named specifications. The evidence below is retained as dated review history, not a current decision register.

What the old 161 counted

Previous primary label Papers Share of exclusions
Continuous / multivalued 61 37.9%
RCT / mixed experimental 40 24.8%
IV / mixed IV, including RD-based IV 34 21.1%
Other, including RD, time discontinuity and non-DiD 26 16.1%

These were mutually exclusive primary labels, not verified counts of pure designs. Mixed papers can contain IV, continuous exposure and a separate binary-policy DiD. Continuous treatment accounted for 61/161 = 37.9%, not a majority. The earlier detailed RD subgroups are also provisional; this audit does not validate every RD subtype.

Coverage and consequences

Identification sections and relevant tables in 36 paper PDFs/versions were examined, targeting suspect mixed-design and nonbinary exclusions plus contrasting genuine restrictions. This is a nonrandom targeted review, not a representative sample or an exhaustive reading of every specification. 125 exclusions have not received a PDF review in this audit. Publisher abstracts acquired for navigation do not count as a PDF review.

15 cases are reopened as pending, not automatically admitted. In the other 21 reviewed cases, the inspected specification’s limitation is corroborated; that does not prove the entire paper has no usable alternative. No corrected count of all genuinely out-of-scope papers is yet established. The administrative screen now has 64 included, 146 provisionally excluded and 52 pending; 146 is not a verified paper-level exclusion total either.

The 64 existing design candidates (54 with comparison results, 10 blocked) remain unchanged. Original collection membership, all fitted estimates and both sections’ calibration data are unchanged. Working-paper evidence is version-labeled below; it cannot by itself establish equivalence to a published specification.

Reopened assessments

A DiD reduced form is not the same estimand as an IV second stage. A binary policy can become fractional after temporal/geographic aggregation. A DiD interpretation can still require a cohort, repeated-cross-section, DDD, difference-in-discontinuities, nonabsorbing-treatment or inference adapter. Data availability and numerical failure must be recorded separately from design.

Religion, Division of Labor, and Conflict: Anti-Semitism in Germany over 600 Years

Main city-by-century specification interacts Protestant location with post-Reformation periods, with city/time effects. Separate mechanism IV does not eliminate this observational DiD. Review the chosen analysis, not the entire paper as one IV estimand.

Evidence: Accepted manuscript, 2018, PDF pages 22; 23. Previous label: instrumental_variables.

Investment Banks as Corporate Monitors in the Early Twentieth Century United States

Table 6 Panels B/C explicitly estimate author-defined high-underwriting versus other railroads before/after the Clayton Act; Figure 3 is also binary. Existing template selected that author-defined group. A continuous-exposure or IV headline label is not grounds to claim no binary DiD analysis.

Evidence: Author manuscript, PDF pages 27; 28; 49; 58. Previous label: instrumental_variables.

Expansions in Maternity Leave Coverage and Children’s Long-Term Outcomes

Compares birth dates around maternity-leave reforms to the same dates in non-reform years: DiD/difference-in-discontinuities. Restricted inputs and birth-date/cohort mapping are execution issues, not proof of a non-DiD paper.

Evidence: Published article, AEJ Applied 4(3), 190–224, PDF pages 1; 7; 8. Previous label: not_did.

The Effect of High-Tech Clusters on the Productivity of Top Inventors

Table 2 Panel B follows the same inventors in 1996 and 2007, comparing baseline Rochester residents with other cities. This is a separate binary-shock DiD from the main continuous-cluster IV analysis. Selected Table 2 data remain unavailable in the recorded package.

Evidence: Published article, AER 111(10), PDF pages 16; 17. Previous label: instrumental_variables.

New Evidence on Information Disclosure through Restaurant Hygiene Grading

Table 3 has author-labeled binary policy timing, but its notes retain fractional boundary-ZIP exposure and its target is a triple interaction. Do NOT equate the binary label with a verified 0/1 analysis column. Review county-level alternatives and a faithful DDD mapping; selected continuous-share failure remains valid.

Evidence: Published article, AEJ Policy 11(4), PDF pages 13; 14. Previous label: nonbinary_treatment.

Familiarity Does Not Breed Contempt: Generosity, Discrimination, and Diversity in Delhi Schools

School-by-cohort natural-experiment DiD uses a poor-classmate exposure indicator. Lab/field experiments measure outcomes, rather than randomly assigning the integration policy. Separate peer-adjacency IV is another analysis. Cohort timing and sample/inference mapping still need checking.

Evidence: Accepted manuscript, May 2018, PDF pages 9; 10. Previous label: randomized_trial.

Increasing Organ Donor Registration as a Means to Increase Transplantation: An Experiment with Actual Organ Donor Registrations

Table 2 compares California with other states before/after the 2011 registration-question change. A separate laboratory experiment does not make that policy rollout randomized. Verify final-version/table alignment before admitting a new target; historical source and template results already exist.

Evidence: NBER working paper 20378, 2014; earlier version, PDF pages 9; 10; 11. Previous label: randomized_trial.

E-governance, Accountability, and Leakage in Public Programs: Experimental Evidence from a Financial Management Reform in India

Separate nationwide staggered eFMS rollout is analyzed by DiD; Table 8 uses district/year effects and policy-on indicators. This is distinct from the randomized Bihar intervention. Existing source target must be matched to the observational rollout, not described as an RCT effect.

Evidence: Author manuscript, June 2019, PDF pages 5; 25; 42. Previous label: randomized_trial.

Does Vertical Integration Decrease Prices? Evidence from the Paramount Antitrust Case of 1948

Paper defines theater integration as binary. Library R preparation averages vi to theater-year (R/112_gil_replicate.R), creating fractional annual exposure; headline vit is a post-divestiture trend interaction. Reconstruct appropriate timing/target before fitting; do not threshold annual means or call the policy intrinsically continuous.

Evidence: Author manuscript, March 2014, PDF pages 14; 16; 33. Previous label: nonbinary_treatment.

Are Credit Markets Still Local? Evidence from Bank Branch Closings

Exposed/control tracts around merger-driven branch consolidation supply a reduced-form DiD. The IV effect of branch closures is a different estimand. A binary exposure comparison may be a candidate, but cannot be relabeled the IV effect; recorded numerical checks still apply.

Evidence: Published article, AEJ Applied 11(1), PDF pages 11. Previous label: instrumental_variables.

Tax Me, but Spend Wisely? Sources of Public Finance and Government Accountability

Program participation indicator PMAT identifies program-on reduced forms with municipality/time effects, alongside tax-revenue IV estimates. A PMAT policy DiD and the effect of instrumented revenue are different targets. Preserve that distinction and verify the selected specification.

Evidence: Warwick working paper 1131, 2016, PDF pages 15; 16. Previous label: instrumental_variables.

Do Consumers Exploit Commitment Opportunities? Evidence from Natural Experiments Involving Liquor Consumption

Primary estimand uses permitted sales hours, but authors also estimate binary sales-ban specifications in Table 5 column 4. They explicitly caution that these discard relevant variation and are low-powered. Consider only that author-defined alternative, not an invented threshold or equivalent headline estimand.

Evidence: Author manuscript, PDF pages 32; 33. Previous label: nonbinary_treatment.

Trade and Investment under Policy Uncertainty: Theory and Firm Evidence

Table 1 has an EC10/Spain-by-post-1986 policy DiD, separate from continuous tariff/uncertainty and structural analyses. The existing template chose an author binary accession specification. Recheck that target on its own merits.

Evidence: Author manuscript, December 2014, PDF pages 11; 30; 39. Previous label: nonbinary_treatment.

The Consequences of Radical Reform: The French Revolution

Earlier paper explicitly defines binary revolutionary/Napoleonic occupation measures and reports outcome regressions alongside duration and reform IV. Published-package target is duration of French presence. Binary alternative is a lead requiring published-version/data alignment, not a confirmed replacement replication.

Evidence: NBER working paper 14831, April 2009; earlier version, PDF pages 19; 20; 27; 29. Previous label: nonbinary_treatment.

Corroborated selected-target restrictions

These findings concern the inspected targets, not every analysis in each paper.

Paper Finding Source / PDF pages
Gender Quotas and the Crisis of the Mediocre Man: Theory and Evidence from Sweden Main DiD interacts the continuous change in female representation (quota bite) with election periods. The binary variable elsewhere describes competence, an outcome—not binary treatment. Selected dose-response target cannot be converted by counting outcome categories. PDF; 21; 22
College Enrollment and Mandatory FAFSA Applications: Evidence from Louisiana Statewide mandate analysis uses baseline FAFSA noncompletion intensity interacted with post-policy. Author says state-wide common effects are absorbed by year effects. Binary policy existence does not make this school-level exposure-gradient estimand binary. PDF; 15; 16; 19
Women’s Suffrage and Children’s Education Selected summary regressor is share of ages 0–15 exposed to suffrage, not just law adoption. Paper also has age-at-exposure event studies. Nonbinary limitation applies to selected dose-summary; those event-study alternatives need their own mapping. PDF; 12; 13
Early Retirement Incentives and Student Achievement Treatment intensity is the pre-reform number of experienced teachers interacted with post-ERI; teacher-count controls are retained. Selected coefficient is an intensity response, not a binary adoption ATT. PDF; 17
The Debt-Inflation Channel of the German (Hyper)Inflation Regression interacts continuous initial firm leverage with year indicators. Above/below-median groups illustrate a figure, while regression coefficients retain leverage intensity. A binary display does not establish a binary regression replication. PDF; 22; 25
Expected Returns to Crime and Crime Location Binary South Asian neighborhood designation interacts with the continuously varying gold price; coefficient is a differential price elasticity. Neither fixed group membership nor a nonabsorbing monthly price is a binary adoption intervention. PDF; 10; 11
Media Bias in China Reform_2003 is pre-reform number of county newspapers times post-2003, e.g. 0 to 3. It is count-exposure DiD; a common reform date does not turn the selected target into 0/1 treatment. PDF; 22
Immigration Restrictions as Active Labor Market Policy: Evidence from the Mexican Bracero Exclusion Bracero exclusion coefficient uses pre-exclusion bracero share as continuous treatment intensity. Grouped descriptive contrasts do not replace that dose target. PDF; 8; 9
Are State Governments Roadblocks to Federal Stimulus? Evidence on the Flypaper Effect of Highway Grants in the 2009 Recovery Act Explicit IV-DiD in dollar amounts of highway grants; formula-based instruments address grant endogeneity. Has a DiD interpretation but not the current binary-treatment ATT estimand. IV and nonbinary features overlap. PDF; 5; 16
Taxation and the International Mobility of Inventors Tax-retention-rate elasticities and location-choice models use continuously varying taxes. Quality-group dummies are heterogeneity categories, not binary policy treatment. Review only establishes selected-target limitation. PDF; 17; 18; 21
A Machine Learning Approach to Analyze and Support Anticorruption Policy Includes randomized government-audit timing and another transfer-policy IV application. Fixed effects/ML prediction do not make these observational binary-adoption estimates; preserve mixed-design label rather than counting a pure IV-only paper. PDF; 3; 7
Demand and Supply of Infrequent Payments as a Commitment Device: Evidence from Kenya Payment experiment explicitly randomly assigns farmers to two treatments and control. A numerical DiD fit can run, but observational-DiD scope excludes this randomized intervention; not a failed source replication. PDF; 29
The Negligible Effect of Free Contraception on Fertility: Experimental Evidence from Burkina Faso Randomized full versus 10-percent contraception subsidy, with cross-randomized additional interventions. Binary arm indicators do not remove randomized assignment. PDF; 2; 3
Capital Cities, Conflict, and Misgovernance Conflict-onset by distance-to-capital associations, with fixed effects; authors explicitly refrain from causal interpretation. A binary outcome/onset indicator is not evidence for an adoption DiD target. PDF; 24; 25
Subsidies and the African Green Revolution: Direct Effects and Social Network Spillovers of Randomized Input Subsidies in Mozambique Mozambique input-subsidy intervention is explicitly a randomized controlled trial, with spillover estimands. Its randomization—not treatment cardinality—is the scope issue. PDF; 2; 5
Does Agriculture Generate Local Economic Spillovers? Short-Run and Long-Run Evidence from the Ogallala Aquifer County exposure is fraction of land over the aquifer, interacted with time. A coefficient interpreted as entirely-over versus entirely-not-over is still estimated from continuous coverage. PDF; 10
When the Levee Breaks: Black Migration and Economic Development in the American South Year-specific flood effects use fraction of county land flooded, with county/time effects and baseline controls. Selected target is continuous exposure, not a verified binary county treatment. PDF; 13
Estimating Social Preferences and Gift Exchange at Work Gift/control treatments are randomized between workers, with additional within-worker randomized variation. Repeated outcomes do not turn the experiment into observational policy DiD. PDF; 13; 17
Labor Drops: Experimental Evidence on the Return to Additional Labor in Microenterprises Wage subsidy and supplemental training/savings arms are randomly assigned within strata. The control-versus-subsidy contrast is an experiment, despite repeated outcome observations. PDF; 11; 12
Report Cards: The Impact of Providing School and Child Test Scores on Educational Markets Half of sampled villages are randomly selected to receive test-score report cards. School/child panel structure does not change the assignment mechanism. PDF; 2
Ready for Boarding? The Effects of a Boarding School for Disadvantaged Students Oversubscribed boarding-school offers allocated randomly among applicants; tracks treatment/control after lottery. An offer/attendance distinction may require IV, but not observational adoption DiD. PDF; 2

Reproducibility and scope

The CSV records the prior reason, review coverage, PDF version, page numbers, short evidence excerpt and SHA-256 for every reviewed PDF. The remaining rows explicitly say not PDF-reviewed. Source PDFs and microdata are not redistributed. Historical run receipts are preserved; reopening a design assessment is not a successful new replication. Mixed-paper admission remains a community scope question, separate from whether a particular DiD analysis exists.