Skip to content

#1059: Respect cv_n_jobs in BaseRClassifier and XGBRRegressor cross-validation - #1060

Open
ayuxy027 wants to merge 1 commit into
uber:masterfrom
ayuxy027:fix/r-learner-cv-n-jobs
Open

ayuxy027 wants to merge 1 commit into
uber:masterfrom
ayuxy027:fix/r-learner-cv-n-jobs

Conversation

@ayuxy027

Copy link
Copy Markdown

Proposed changes

Fixes #1059.

BaseRClassifier.fit() called cross_val_predict(..., n_jobs=-1) unconditionally, ignoring the cv_n_jobs attribute it inherits from BaseRLearner. Setting cv_n_jobs therefore didn't limit cross-validation parallelism, which matters when fits are themselves dispatched in parallel and each outer worker starts its own joblib pool.

XGBRRegressor.fit() had the same hard-coded n_jobs=-1. Those are the only two direct cross_val_predict calls in rlearner.py. Both now pass n_jobs=self.cv_n_jobs, matching the BaseRLearner path, which already forwards cv_n_jobs to compute_r_residuals() (#714). The default stays -1, so behavior is unchanged unless cv_n_jobs is set.

Note: the R-learner subclasses don't take cv_n_jobs as a constructor argument, so it can only be set as an attribute after construction (as in the issue's repro). I've left that as is to keep this fix focused; happy to add it in a follow-up if you'd like.

Types of changes

  • Bugfix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation Update (if none of the other choices apply)

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • Lint and unit tests pass locally with my changes (see Further comments for the Graphviz-only failures)
  • I have added tests that prove my fix is effective or that my feature works
  • I have added necessary documentation (if appropriate)
  • Any dependent changes have been merged and published in downstream modules

Further comments

Tests: test_r_learner_cross_val_predict_respects_cv_n_jobs in tests/test_meta_learners.py is parametrized over BaseRClassifier and XGBRRegressor. It sets cv_n_jobs = 1, wraps cross_val_predict, and checks the value reaches it. On master both cases fail with assert -1 == 1; with this change they pass. Full suite locally (pytest tests/ --cov causalml/): 568 passed, 18 skipped, 4 failed. The 4 failures are the uplift tree visualization tests, which need Graphviz's dot binary that isn't installed on my machine; they fail the same way on master. black reports no changes.

…ss-validation

BaseRClassifier.fit() and XGBRRegressor.fit() called cross_val_predict()
with n_jobs=-1, ignoring the inherited cv_n_jobs attribute, so setting it
did not limit cross-validation parallelism. Pass self.cv_n_jobs, as the
BaseRLearner path already does through compute_r_residuals().

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

BaseRClassifier ignores cv_n_jobs during outcome cross-validation

1 participant