Skip to content

Fix gini() for tied predictions - #1061

Open
techi101 wants to merge 1 commit into
uber:masterfrom
techi101:fix/gini-ties
Open

techi101 wants to merge 1 commit into
uber:masterfrom
techi101:fix/gini-ties

Conversation

@techi101

@techi101 techi101 commented Nov 7, 2026

Copy link
Copy Markdown

Proposed changes

gini(y, p) in causalml/metrics/regression.py sorts rows by p with argsort and does not handle ties, so when predictions tie the score depends on the input row order:

y = [0, 0, 1, 1], p = [0.5, 0.5, 0.5, 0.5]   ->  gini = 1.0
same rows reversed                            ->  gini = -1.0
y = [0, 1, 0, 1], p = [0, 0, 1, 1]            ->  gini = 0.5 (p carries no information)

A constant prediction gets a perfect score. Ties are common with binary or rounded predictions, and regression_metrics reports Gini by default.

The fix sorts with a stable sort and gives every row in a group of tied keys the group's mean y, so ties no longer depend on row order. With no ties the result is unchanged (same value to full precision on a 500-row random check).

Types of changes

  • Bugfix (non-breaking change which fixes an issue)

Checklist

  • I have read the CONTRIBUTING doc
  • I have signed the CLA
  • Lint and unit tests pass locally with my changes
  • I have added tests that prove my fix is effective or that my feature works

Further comments

Tests in tests/test_metrics.py: a constant prediction gives 0 in both row orders (fails on master), a perfect prediction still gives 1, and untied data gives the same value as before. black --check passes.

I used Claude to help find and fix this, and reviewed and tested the change myself.

gini() sorted rows with argsort and did not handle ties, so the score
depended on input row order: a constant prediction could score 1.0 or
-1.0. Give rows in a tie the group's mean target. Untied input gives the
same result as before. Add tests.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant