Skip to content

Add a pickle-size regression test for fitted learners - #1064

Open
techi101 wants to merge 1 commit into
uber:masterfrom
techi101:test/pickle-size-guard
Open

techi101 wants to merge 1 commit into
uber:masterfrom
techi101:test/pickle-size-guard

Conversation

@techi101

@techi101 techi101 commented Nov 7, 2026

Copy link
Copy Markdown

Proposed changes

Closes #1063

This adds tests/test_pickle_size.py, the pickle-size guard from M1 in #938. Each learner (S, T, X, R, DR meta-learners for regression and classification, plus CausalTree, CausalRandomForest, UpliftTree and UpliftRandomForest) is fit on the existing conftest data, pickled, and checked against a budget in PICKLE_SIZE_BUDGETS.

The budgets are about 2x the sizes I measured, which are listed next to each one. The meta-learner sizes did not change when I went from 1000 to 4000 training rows, so nothing data-sized is stored today, and a change that starts keeping training data would fail right away. The comment above the dict explains how to re-measure and update a budget when growth is expected. A small check also keeps the budget dict and the tested learners in sync.

Types of changes

  • New feature (non-breaking change which adds functionality)

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works

Further comments

All 14 tests pass locally in about 10 seconds and the file passes black --check. No CI config changes are needed because the file sits in tests/. The two R-learners take a few seconds each because cross-validation starts parallel workers (#1060 adds cv_n_jobs, which could speed this up later).

I used Claude to help write this, and reviewed and tested it myself.

Fit each meta-learner (S/T/X/R/DR, regressor and classifier variants)
and the causal and uplift tree/forest learners on the small conftest
datasets, pickle them, and assert the size stays under a per-learner
budget set at about 2x the current size. A change that starts keeping
training data or other data-sized state on a fitted learner now fails
the test instead of silently growing every saved model.

Part of v1.0 roadmap M1 (discussion uber#938).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Add a pickle-size regression test for fitted learners (v1.0 M1)

1 participant