Imputation

Regression-based imputation for the Social Indicators Survey. See Chapter 17 in Regression and Other Stories.
dplyr + ggplot version.

Source: Imputation/imputation_gg.Rmd.

The Python version keeps data handling explicit and uses lapylace for Stan-backed generalized linear models, so the statistical model can be read from a formula rather than from handwritten Stan.

from pathlib import Path
import sys
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
import lapylace as lp

sys.path.append(str(Path.cwd().parent / 'python'))
from data import ros_path

Data

sis = pd.read_csv(ros_path('Imputation/data', 'SIS.csv'))
sis.head()
earnings retirement interest assistance other male over65 white immig educ_r workmos workhrs_top any_ssi any_welfare any_charity
0 84.0 0.7 0.20 0.0 0.0 1 0 1 1 4.0 12 40 0 0 0
1 NaN 0.0 0.00 0.0 0.0 0 0 0 1 4.0 12 40 0 0 0
2 27.5 0.0 0.16 0.0 0.0 1 0 0 1 2.0 10 40 0 0 0
3 85.0 12.0 5.00 0.0 NaN 1 1 1 0 4.0 11 8 0 0 0
4 135.0 0.0 0.10 0.0 0.1 1 0 0 0 4.0 12 40 0 0 0

Models

The formulas below are the Python counterparts of the model formulas in the source example. Use lapylace for the Stan-backed Bayesian fit with the same formula interface.

# fit = lp.stan_glm('(earnings>0) ~ male + over65 + white + immig +', data=sis, family=lp.bernoulli(), chains=4, iter_sampling=1000)

# fit = lp.stan_glm('earnings ~ interest_imp + male + over65 + white +', data=sis, family=lp.gaussian(), chains=4, iter_sampling=1000)

# fit = lp.stan_glm('interest ~ earnings_imp + male + over65 + white +', data=sis, family=lp.gaussian(), chains=4, iter_sampling=1000)

Notes

  • Source computation blocks represented: 25.
  • Data paths are expressed through the shared ros_path() helper.
  • Formula-based Bayesian regressions are routed through lapylace.stan_glm().
  • Plotting and simulation work uses NumPy, pandas, matplotlib idioms.