Credit Markets: entropy versus augsynth multisynth

Matched sample, selected tuning and country-clustered uncertainty

Both methods use the same 723 stock–country units × 53 years, including 529 treated units and 194 never-treated donors. Each ATT averages all 17,811 treated unit-years equally, in outcome-SD units. Cohort-averaged and individual refer to how donor weights are fitted, not different ATT weights.

Estimates

Weights	Method	ATT	95% lower	95% upper
Cohort-averaged	Augsynth	-0.4733	-0.7055	-0.2514
Cohort-averaged	Entropy	-0.4575	-0.6906	-0.2318
Individual	Augsynth	-0.3574	-0.6508	-0.0838
Individual	Entropy	-0.4749	-0.7373	-0.2536

Bootstrap intervals

All four rows use the same 9,999 Mammen wild-bootstrap draws across 47 countries as the existing multisynth example. Each country receives a common multiplier across its units and years. The saved augsynth draws are reproduced exactly; those multipliers are then applied to the entropy fit’s own treated and donor residual contributions. Both methods remove unit pre-period means and common calendar-time effects before applying the multipliers, matching augsynth’s force=3 residualization.

The corrected cohort bootstrap SEs are 0.1155 (augsynth) and 0.1171 (entropy); individual SEs are 0.1477 and 0.1276. Entropy therefore has nearly the same cohort uncertainty and lower individual uncertainty here.

Correction: an earlier version omitted calendar-time residualization from the entropy bootstrap, inflating its intervals. Common time effects cancel in the original simplex-weighted estimate, but not when treated and donor units receive different country multipliers. All 9,999 corrected scalar draws and all event-study interval endpoints agree with augsynth’s bootstrap after substituting the entropy weights (maximum discrepancy below 10⁻¹³). The fitted weights, point estimates and pre-treatment RMSEs are unchanged.

These are fixed-fit intervals: donor weights, unit and calendar-time baselines, pooling and tuning stay fixed. There is no jackknife or refitting. The ATT uses treated-unit-year weights in every draw. Event-study bands below are pointwise 95% intervals, not simultaneous bands.

Event studies

Common event-time support and pointwise 95% bands. All available post-treatment years enter the ATT; the plot shows the last ten pre-treatment years.

Event-time support changes with the horizon: 529 units contribute at adoption, 24 at year 36 and 11 at year 45. The change at year 36 coincides with a large cohort leaving the observed window. Full paths and support counts are in the event-study download.

Pre-treatment fit

Weights	Method	Pre RMSE	Pooled pre RMSE
Cohort-averaged	Augsynth	0.0222	0.0042
Cohort-averaged	Entropy	0.0255	0.0051
Individual	Augsynth	0.1752	0.0080
Individual	Entropy	0.2307	0.0198

The comparisons use the same 10,226 pre-treatment observations. Cohort RMSE squares cohort-mean gaps, weighted by cohort size; individual RMSE squares unit-level gaps. Pooled RMSE first averages gaps by event time. Augsynth fits better on both disaggregated measures; the difference is larger for individual weights.

Last ten pre-treatment years, with pointwise 95% bands.

Pooling and tuning

Weights	Method	ν	λ
Cohort-averaged	Augsynth	0.60511	0.00000
Cohort-averaged	Entropy	0.57510	5.47723
Individual	Augsynth	0.15164	0.00000
Individual	Entropy	0.14119	0.54772

Entropy uses balnet’s decreasing-λ paths. Separate and fully pooled fits are computed first. The separate fit supplies ν and the two loss normalizers; the final partially pooled path holds these fixed and chooses its smallest penalty passing the numerical checks. Both fully pooled paths reach λ=10⁻⁶. Augsynth uses its separate-fit pooling heuristic with λ=0. Neither procedure uses post-treatment outcomes for tuning.

Tuning details and full path diagnostics include failed attempts, donor concentration and fitting times.

Missing outcomes

The entropy implementation accepts NA outcomes or an explicit observed mask, for both cohort and individual fitting and the tuning paths.

Missing pre-treatment outcomes contribute no balance loss. Group means and unit baselines use observed outcomes only.
In each pre-period, observed donors must retain their prior total weight. This keeps the fitting problem convex and prevents improving apparent balance by shifting weight onto missing donors. It can restrict the feasible weights.
Predictions renormalize weights over donors observed in that period. An observed treated outcome without donor support raises an error.
The ATT averages observed treated outcomes. Treatment adoption dates must be known; missing outcomes are never filled with zeros.

Credit’s panel is complete. Missing-panel behavior is checked separately by independent optimization and masked-value invariance tests; the executable missing-panel example shows it directly. Outcome masking does not by itself correct selection from informative missingness.

Data and downloads

The outcome is IHS-transformed fishing mortality relative to MSY, divided by the source outcome SD. Both fits use all available pre-treatment history, unit-intercept adjustment and no additional covariates. The 21 initially treated units are excluded. All estimates and fitted weights are reused from the validated runs; only the bootstrap summaries and presentation are new.

Estimate table · Event studies and intervals · Aggregate bootstrap draws · Results and checks · Forest plot PDF · Event-study PDF · Inference code

--- title: "Credit Markets: entropy versus augsynth multisynth" subtitle: "Matched sample, selected tuning and country-clustered uncertainty" --- ```{r} #| include: false library(jsonlite) j <- fromJSON("results/credit-tuned.json") s <- j$summary modes <- c("cohort", "individual") methods <- c("augsynth", "tuned_entropy") mode_labels <- c(cohort = "Cohort-averaged", individual = "Individual") method_labels <- c(augsynth = "Augsynth", tuned_entropy = "Entropy") colors <- c(augsynth = "#23678a", tuned_entropy = "#b74e25") path_for <- function(mode, method) { j$inference$results[[paste(mode, method, sep = "_")]]$event_study } ``` Both methods use the same **723 stock–country units × 53 years**, including 529 treated units and 194 never-treated donors. Each ATT averages all **17,811 treated unit-years** equally, in outcome-SD units. Cohort-averaged and individual refer to how donor weights are fitted, not different ATT weights. ## Estimates ```{r} #| echo: false tab <- data.frame( Weights = mode_labels[s$mode], Method = method_labels[s$method], ATT = s$att, Lower = s$lower, Upper = s$upper ) knitr::kable(tab, digits = 4, row.names = FALSE, col.names = c("Weights", "Method", "ATT", "95% lower", "95% upper")) ``` ```{r} #| include: false draw_forest <- function() { par(mar = c(4.5, 10, 1, 1), las = 1) y <- rev(seq_len(nrow(s))) lab <- paste(c(cohort = "Cohort", individual = "Individual")[s$mode], method_labels[s$method], sep = " / ") plot(s$att, y, xlim = range(s$lower, s$upper, 0), ylim = c(.5, 4.5), yaxt = "n", ylab = "", xlab = "ATT (outcome SDs)", pch = 19, col = colors[s$method], cex = 1.2) abline(v = 0, col = "gray65", lty = 3) segments(s$lower, y, s$upper, y, col = colors[s$method], lwd = 2) points(s$att, y, pch = 19, col = colors[s$method], cex = 1.2) axis(2, at = y, labels = lab, tick = FALSE, cex.axis = .9) } ``` ```{r} #| echo: false #| fig-width: 9 #| fig-height: 4 #| fig-cap: "Point estimates and 95% country-clustered fixed-fit bootstrap intervals." draw_forest() ``` ### Bootstrap intervals All four rows use the **same 9,999 Mammen wild-bootstrap draws across 47 countries** as the existing multisynth example. Each country receives a common multiplier across its units and years. The saved augsynth draws are reproduced exactly; those multipliers are then applied to the entropy fit's own treated and donor residual contributions. Both methods remove **unit pre-period means and common calendar-time effects** before applying the multipliers, matching augsynth's `force=3` residualization. The corrected cohort bootstrap SEs are **0.1155 (augsynth) and 0.1171 (entropy)**; individual SEs are **0.1477 and 0.1276**. Entropy therefore has nearly the same cohort uncertainty and lower individual uncertainty here. **Correction:** an earlier version omitted calendar-time residualization from the entropy bootstrap, inflating its intervals. Common time effects cancel in the original simplex-weighted estimate, but not when treated and donor units receive different country multipliers. All 9,999 corrected scalar draws and all event-study interval endpoints agree with augsynth's bootstrap after substituting the entropy weights (maximum discrepancy below 10⁻¹³). The fitted weights, point estimates and pre-treatment RMSEs are unchanged. These are **fixed-fit intervals**: donor weights, unit and calendar-time baselines, pooling and tuning stay fixed. There is no jackknife or refitting. The ATT uses treated-unit-year weights in every draw. Event-study bands below are pointwise 95% intervals, not simultaneous bands. ## Event studies ```{r} #| include: false draw_events <- function(pre_only = FALSE) { par(mfrow = c(2, 1), mar = c(4, 6, 2, 1), las = 1) for (mode in modes) { paths <- lapply(methods, function(method) { p <- path_for(mode, method) p <- p[p$event_time >= -10, ] if (pre_only) p <- p[p$event_time < 0, ] p }) yrange <- range(unlist(lapply(paths, function(p) c(p$lower, p$upper))), 0) plot(range(paths[[1]]$event_time), yrange, type = "n", xlab = if (pre_only) "Years before adoption" else "Years since adoption", ylab = "", main = mode_labels[mode]) mtext("Mean gap (outcome SDs)", side = 2, line = 4, las = 0) abline(h = 0, col = "gray65", lty = 3) if (!pre_only) abline(v = -.5, col = "gray65", lty = 3) for (k in seq_along(methods)) { p <- paths[[k]] color <- colors[methods[k]] polygon(c(p$event_time, rev(p$event_time)), c(p$lower, rev(p$upper)), col = adjustcolor(color, alpha.f = .14), border = NA) } for (k in seq_along(methods)) { p <- paths[[k]] lines(p$event_time, p$estimate, type = "o", pch = c(16, 17)[k], cex = .55, col = colors[methods[k]], lwd = 2) } legend("topleft", method_labels[methods], col = colors[methods], lwd = 2, pch = c(16, 17), bty = "n", cex = .85) } } ``` ```{r} #| echo: false #| fig-width: 9 #| fig-height: 8 #| fig-cap: "Common event-time support and pointwise 95% bands. All available post-treatment years enter the ATT; the plot shows the last ten pre-treatment years." draw_events() ``` Event-time support changes with the horizon: 529 units contribute at adoption, 24 at year 36 and 11 at year 45. The change at year 36 coincides with a large cohort leaving the observed window. Full paths and support counts are in the [event-study download](results/credit-event-intervals.csv). ## Pre-treatment fit ```{r} #| echo: false knitr::kable(data.frame(Weights = mode_labels[s$mode], Method = method_labels[s$method], RMSE = s$rmse, Pooled = s$pooled_rmse), digits = 4, row.names = FALSE, col.names = c("Weights", "Method", "Pre RMSE", "Pooled pre RMSE")) ``` The comparisons use the same **10,226 pre-treatment observations**. Cohort RMSE squares cohort-mean gaps, weighted by cohort size; individual RMSE squares unit-level gaps. Pooled RMSE first averages gaps by event time. Augsynth fits better on both disaggregated measures; the difference is larger for individual weights. ```{r} #| echo: false #| fig-width: 9 #| fig-height: 7 #| fig-cap: "Last ten pre-treatment years, with pointwise 95% bands." draw_events(pre_only = TRUE) ``` ## Pooling and tuning ```{r} #| echo: false knitr::kable(data.frame(Weights = mode_labels[s$mode], Method = method_labels[s$method], nu = s$nu, lambda = s$lambda), digits = 5, row.names = FALSE, col.names = c("Weights", "Method", "ν", "λ")) ``` Entropy uses balnet's decreasing-λ paths. Separate and fully pooled fits are computed first. The separate fit supplies ν and the two loss normalizers; the final partially pooled path holds these fixed and chooses its smallest penalty passing the numerical checks. Both fully pooled paths reach λ=10⁻⁶. Augsynth uses its separate-fit pooling heuristic with λ=0. Neither procedure uses post-treatment outcomes for tuning. [Tuning details and full path diagnostics](tuning.html) include failed attempts, donor concentration and fitting times. ## Missing outcomes The entropy implementation accepts **`NA` outcomes or an explicit `observed` mask**, for both cohort and individual fitting and the tuning paths. - Missing pre-treatment outcomes contribute no balance loss. Group means and unit baselines use observed outcomes only. - In each pre-period, observed donors must retain their prior total weight. This keeps the fitting problem convex and prevents improving apparent balance by shifting weight onto missing donors. It can restrict the feasible weights. - Predictions renormalize weights over donors observed in that period. An observed treated outcome without donor support raises an error. - The ATT averages **observed treated outcomes**. Treatment adoption dates must be known; missing outcomes are never filled with zeros. Credit's panel is complete. Missing-panel behavior is checked separately by independent optimization and masked-value invariance tests; the [executable missing-panel example](index.html#executable-staggered-panel-example) shows it directly. Outcome masking does not by itself correct selection from informative missingness. ## Data and downloads The outcome is IHS-transformed fishing mortality relative to MSY, divided by the source outcome SD. Both fits use all available pre-treatment history, unit-intercept adjustment and no additional covariates. The 21 initially treated units are excluded. All estimates and fitted weights are reused from the validated runs; only the bootstrap summaries and presentation are new. ```{r} #| include: false export_figure <- function(draw, name, width, height) { pdf(paste0("results/", name, ".pdf"), width = width, height = height) draw(); invisible(dev.off()) png(paste0("results/", name, ".png"), width = width * 200, height = height * 200, res = 200) draw(); invisible(dev.off()) } export_figure(draw_forest, "credit-att-intervals", 9, 4) export_figure(draw_events, "credit-event-intervals", 9, 8) export_figure(function() draw_events(TRUE), "credit-pre-intervals", 9, 7) ``` [Estimate table](results/credit-tuned-summary.csv) · [Event studies and intervals](results/credit-event-intervals.csv) · [Aggregate bootstrap draws](results/credit-bootstrap-draws.csv) · [Results and checks](results/credit-tuned.json) · [Forest plot PDF](results/credit-att-intervals.pdf) · [Event-study PDF](results/credit-event-intervals.pdf) · [Inference code](https://github.com/apoorvalal/balmultisynth/blob/feat/entropy-multisynth/scripts/credit-inference.R)