Train models for CPS-only variables on PUF clone half

[MaxGhenis]

Problem

When building the extended CPS, the PUF clone half receives CPS-only variables (like retirement contributions) either by:

1. Direct duplication from the CPS donor record (else branch in puf_clone_dataset)
2. PUF override via OVERRIDDEN_IMPUTED_VARIABLES (e.g. pre_tax_contributions)

Neither approach preserves the relationship between these variables and income. A PUF clone with $0 wages can end up with $50k in 401(k) contributions, because there's no model linking contributions to the income variables that are common between CPS and PUF.

This creates implausible records and makes calibration harder — you can't calibrate away a structural data quality issue.

Proposed solution

For variables that exist in CPS but not PUF, train predictive models using the CPS half to predict these variables from features common to both CPS and PUF:

Common predictors (available in both datasets):

• Wages/salary income (employment_income)
• Self-employment income
• Interest/dividend income
• Age
• Filing status
• Number of dependents
• Social Security income (sub-components)
• Pension/retirement income

Variables to model (examples):

• pre_tax_contributions (retirement contributions — see Add calibration targets for retirement contributions #553)
• traditional_401k_contributions
• traditional_ira_contributions
• roth_401k_contributions
• self_employed_pension_contributions
• Other CPS-only variables currently in OVERRIDDEN_IMPUTED_VARIABLES that should respect income relationships

Approach

1. On the CPS half (which has both income variables and CPS-only variables), train lightweight models (e.g. quantile regression, gradient boosting) predicting each CPS-only variable from the common features
2. Apply these models to the PUF clone half, using the PUF-derived income values as inputs
3. This ensures that a PUF clone with high wages gets plausible retirement contributions, and one with $0 wages gets $0 contributions

Related

• Add calibration targets for retirement contributions #553 — Retirement contribution calibration targets (symptoms of this structural issue)
• Calibrate retirement contributions: targets, SS reconciliation, and QRF imputation #554 — Proportional retirement contribution allocation (fixes CPS-side allocation)
• Reconcile SS sub-components after PUF imputation #552 — SS sub-component reconciliation and formula variable cleanup

[MaxGhenis]

Reopening — the retirement contribution fix in #554 addressed one piece, but the broader issue remains:

1. Bug: weekly_hours_worked and hours_worked_last_week are incorrectly deflated by WKSWORK/52 (lines 504-505 of cps.py). These CPS variables (HRSWK, A_HRS1) are already weekly — no conversion needed. This makes part-year workers appear to work fewer hours/week than they do, breaking FLSA overtime, SNAP work requirements, and other hours-based eligibility checks.

2. Structural gap: Hours variables aren't QRF-imputed for the PUF clone half — they're copied from the CPS donor, creating inconsistency with QRF-imputed employment_income.

Filing a PR for (1) now.

[MaxGhenis]

Full scope of variables naively duplicated to PUF clone half

After auditing cps.py and extended_cps.py, here's the complete inventory of CPS-only variables that are simply np.concatenate([values, values]) to the PUF clone half instead of being QRF-imputed. Grouped by priority:

High priority (income-correlated, ~20 vars)

Retirement contributions (5): traditional_401k_contributions, roth_401k_contributions, traditional_ira_contributions, roth_ira_contributions, self_employed_pension_contributions

Retirement distributions (10+): taxable_401k_distributions, tax_exempt_401k_distributions, taxable_403b_distributions, tax_exempt_403b_distributions, roth_ira_distributions, regular_ira_distributions, keogh_distributions, taxable_sep_distributions, tax_exempt_sep_distributions, other_type_retirement_account_distributions, taxable_private_pension_income, tax_exempt_private_pension_income

Social Security sub-components (4): social_security_retirement, social_security_disability, social_security_dependents, social_security_survivors — aggregate social_security is QRF-imputed but sub-components aren't, so they're inconsistent with the imputed total.

Medium-high priority (means-tested/income-related, ~25 vars)

Transfer income (9): unemployment_compensation, tanf_reported, ssi_reported, child_support_received, veterans_benefits, workers_compensation, disability_benefits, strike_benefits, receives_wic

SPM variables (15): spm_unit_total_income_reported, snap_reported, spm_unit_capped_housing_subsidy_reported, free_school_meals_reported, spm_unit_energy_subsidy_reported, spm_unit_wic_reported, spm_unit_broadband_subsidy_reported, spm_unit_payroll_tax_reported, spm_unit_federal_tax_reported, spm_unit_state_tax_reported, spm_unit_capped_work_childcare_expenses, spm_unit_spm_threshold, spm_unit_net_income_reported, spm_unit_pre_subsidy_childcare_expenses

Medical expenses (5): health_insurance_premiums_without_medicare_part_b, over_the_counter_health_expenses, other_medical_expenses, medicare_part_b_premiums, child_support_expense

Medium priority (~5 vars)

Hours/employment: weekly_hours_worked, hours_worked_last_week (deflation bug fixed in #588), tip_income

Previous year income (3): employment_income_last_year, self_employment_income_last_year, previous_year_income_available

Low priority (OK to duplicate)

Demographics, occupation flags, random seeds, state/county FIPS — these don't depend on income and are reasonable to copy from the CPS donor.

---

Approach

Add these variables to the QRF imputation pipeline in extended_cps.py, using existing income/demographic predictors. Can be done in stages but the high-priority group alone would fix the worst inconsistencies.