A complete design diagnosis

The declaration below defines a finite-population randomized experiment from its data-generating model through its estimator. The treatment effect varies with a baseline covariate, but the inquiry is the finite-population average treatment effect.

import altair as alt
import numpy as np
import polars as pl
import declaredesign as dd

n = 300
design = dd.declare_design(
    dd.declare_population(
        n,
        X=lambda data, rng: rng.normal(size=data.height),
        U=lambda data, rng: rng.normal(size=data.height),
    ),
    dd.declare_potential_outcomes(
        outcome="Y",
        assignment="Z",
        conditions=[0, 1],
        model=lambda data, z: (
            data["U"].to_numpy()
            + 0.5 * data["X"].to_numpy()
            + z * (0.4 + 0.3 * data["X"].to_numpy())
        ),
    ),
    dd.declare_inquiry(
        "ATE", lambda data: (data["Y_Z_1"] - data["Y_Z_0"]).mean()
    ),
    dd.declare_assignment(name="Z", m=n // 2),
    dd.declare_measurement(outcome="Y", assignment="Z"),
    dd.declare_estimator("Y ~ Z", inquiry="ATE", label="Difference in means"),
    dd.declare_estimator(
        "Y ~ Z",
        estimator=lambda formula, data: dd.lm_lin(formula, data, covariates=["X"]),
        inquiry="ATE",
        label="Lin adjustment",
    ),
)
design

Design(
  1. population: population
  2. potential_outcomes: potential outcomes
  3. inquiry: ATE
  4. assignment: assignment
  5. measurement: measurement
  6. estimator: Difference in means
  7. estimator: Lin adjustment
)

diagnosis = dd.diagnose_design(design, sims=300, seed=2026)
diagnosis.summary

shape: (2, 11)

inquiry	estimator	term	mean_estimand	mean_estimate	bias	sd_estimate	rmse	power	coverage	n_sims
str	str	str	f64	f64	f64	f64	f64	f64	f64	i64
"ATE"	"Difference in means"	"Z"	0.400216	0.404576	0.00436	0.134075	0.134157	0.82	0.953333	300
"ATE"	"Lin adjustment"	"Z"	0.400216	0.400597	0.00038	0.110729	0.110808	0.94	0.966667	300

Sampling distributions of the two declared estimators across complete reruns of the research design.

The output joins each estimate to the inquiry generated in the same simulation. The default diagnosis reports the mean estimand and estimate, bias, Monte Carlo standard deviation, root mean squared error, power, coverage, and simulation count. Diagnosis.plot() returns an Altair chart for a selected diagnosand.