Skip to content

Fix consecutive cast elimination for inexact FLOAT intermediates - #3205

Open
itsrayeedscodespace wants to merge 1 commit into
microsoft:mainfrom
itsrayeedscodespace:fix/cast-cast-source-precision
Open

itsrayeedscodespace wants to merge 1 commit into
microsoft:mainfrom
itsrayeedscodespace:fix/cast-cast-source-precision

Conversation

@itsrayeedscodespace

Copy link
Copy Markdown

The consecutive-cast rewrite considers the FLOAT -> FLOAT16/BFLOAT16 pair but ignores the source type. For DOUBLE input 1.00048828125 + 2**-30, the original DOUBLE -> FLOAT -> FLOAT16 graph produces 1.0, while the rewritten direct cast produces 1.0009765625 in the ONNX reference evaluator. The intermediate FLOAT rounding lands on a FLOAT16 midpoint.

Require the source to be exactly representable in FLOAT before eliminating that intermediate: supported floating-point types of at most 32 bits, integers of at most 16 bits, and BOOL. Preserve the cast chain when the source type is missing or its conversion is not proven exact. This retains the intended FLOAT16/BFLOAT16 mixed-precision rewrites and FP8 paths.

Add 40 regression cases covering the numerical counterexample through both the rule and the public optimizer, unknown/wider source types, safe floating/integer conversions, and every finite encoding of the four common FP8 formats.

Fixes #3204

Validation on macOS arm64, Python 3.12.16:

  • New cast tests against unchanged upstream source: 16 failed, 25 passed.
  • python -m pytest onnxscript/rewriter/rules/common/_basic_rules_test.py onnxscript/optimizer -q: 179 passed, 1 existing skip.
  • Common rewrite rules plus optimizer: 489 passed, 1 skipped, 48 subtests passed, and 2 failures in test_fuse_pad_into_conv_6/7. Both failures also reproduce with the original source and original tests (449 passed, the same 2 failures); they are exact-equality convolution comparisons on this local environment.
  • lintrunner -a and git diff --check pass.
  • CPU rewrite-only microbenchmark, 1,000 independent safe cast chains, eight measured runs after warmup: median 113.4 ms upstream versus 120.1 ms patched; samples overlap (107–130 ms versus 111–138 ms). The guard adds a constant-time source-type check.

Numerical validation uses the ONNX reference evaluator and an independent NumPy conversion check. The local ONNX Runtime CPU path returned the lower value for both graphs, so it does not independently demonstrate this rounding difference. No GPU validation or GPU performance measurement was performed. The repository's existing CPU CI matrix provides cross-platform validation.

AI assistance was used for source investigation, implementation, and the local validation runs.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

CastCast rewrite changes rounding for DOUBLE inputs by removing the FLOAT intermediate

1 participant