Skip to content

fix(dsv4/fp8): import suppress so FP8 indexer prefill cleanup stops masking the real error - #1412

Open
Anai-Guo wants to merge 1 commit into
alibaba:mainfrom
Anai-Guo:fix/fp8-indexer-import-suppress
Open

Anai-Guo wants to merge 1 commit into
alibaba:mainfrom
Anai-Guo:fix/fp8-indexer-import-suppress

Conversation

@Anai-Guo

@Anai-Guo Anai-Guo commented Sep 9, 2026

Copy link
Copy Markdown

What

rtp_llm/models_py/modules/dsv4/fp8/indexer.py uses suppress(Exception) in two
places but never imports it:

line context
515 inside except Exception: in _gather_prefill_k_cache, right before raise
543 the whole body of _discard_prefill_k_cache_gather
from __future__ import annotations

import os
from typing import Any, Callable, Dict, NamedTuple, Optional

contextlib never appears in the file. pyflakes on current main:

rtp_llm/models_py/modules/dsv4/fp8/indexer.py:515:26: undefined name 'suppress'
rtp_llm/models_py/modules/dsv4/fp8/indexer.py:543:14: undefined name 'suppress'

Why it matters

Both uses sit on error/cleanup paths, so the missing import turns a real failure
into a misleading NameError and skips the cleanup it was guarding.

_discard_prefill_k_cache_gather is called from two finally: blocks
(indexer.py:1190 and indexer.py:1398). An exception raised inside finally:
replaces the exception being propagated, so whenever a prefill fails while a CP
all-gather handle is still live, the caller never sees the actual error — and the
assembler handle is leaked because discard_assemble_indexer_k_async is never
reached.

The except Exception: handler at 511-517 has the same shape: the
suppress-guarded discard is supposed to run before the raise re-raises the
original prepare_assemble_indexer_k_async failure; instead the handler itself
dies with NameError.

Verification

The two methods were lifted verbatim out of main with ast.get_source_segment
and driven with a stub assembler (no GPU needed), before and after the import:

### UNPATCHED (upstream main)
[1] direct call  Indexer._discard_prefill_k_cache_gather(pending)
    -> NameError: name 'suppress' is not defined
[2] the real shape: forward()'s `finally:` (L1190 / L1398)
    -> caller sees NameError: name 'suppress' is not defined
    cleanup actually reached the assembler: False

### PATCHED  (from contextlib import suppress)
[1] direct call  Indexer._discard_prefill_k_cache_gather(pending)
    -> returned normally; assembler cleanup attempted: 1
[2] the real shape: forward()'s `finally:` (L1190 / L1398)
    -> caller sees RuntimeError: fp8_mqa_indexer_score: DeepGEMM kernel launch failed
    cleanup actually reached the assembler: True

Fix

One import line, matching how the sibling module in the same package already does
it (rtp_llm/models_py/modules/dsv4/fp8/attention.py:19:
from contextlib import contextmanager, suppress), placed in isort order between
import os and the typing import.


🤖 Generated with Claude Code

suppress(Exception) is used at indexer.py:515 (inside an except handler,
just before the re-raise) and at indexer.py:543 (the whole body of
_discard_prefill_k_cache_gather), but the module never imports it.

_discard_prefill_k_cache_gather is called from two finally: blocks
(indexer.py:1190 and indexer.py:1398). An exception raised inside finally:
replaces the exception being propagated, so a prefill failure with a live CP
all-gather handle surfaces as NameError: name 'suppress' is not defined and
the assembler handle is leaked instead of discarded.

The sibling module in the same package already imports it the same way
(dsv4/fp8/attention.py:19).

Signed-off-by: Tai An <antai12232931@outlook.com>
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

from __future__ import annotations

import os
from contextlib import suppress

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P3] import 顺序不符合 isort 规范

补丁中新增行 +from contextlib import suppress 位于上下文行 import os 之后。按 isort 默认排序(按模块名字母序,不区分 import/from-import),contextlib < os < typing,因此 from contextlib import suppress 应排在 import os 之前。若仓库启用了 ruff isort 检查可能触发 lint 失败。

建议:将 from contextlib import suppress 移到 import os 之前,保持 stdlib import 按字母序排列。

评审版本:cdf9a9779fb6

@rtp-llm-review-bot

rtp-llm-review-bot commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

PR #1412 评审:未发现阻塞项,等待人工审核

  • 标题:fix(dsv4/fp8): import suppress so FP8 indexer prefill cleanup stops masking the real error(@Anai-Guo,open)
  • 评审版本:cdf9a9779fb6

无阻塞项

非阻塞发现


自动代码评审 · head cdf9a9779fb6 · 未发现阻塞项,等待人工审核

@LLLLKKKK LLLLKKKK left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review - PR #1412

Status: LGTM

Summary: P0/0 · P1/0 · P2/1 · P3/0

Reviewed: commit cdf9a9779fb6 · 2026-09-09 21:37 UTC+8

lgtm ready to ci

Non-blocking Suggestions

P2

  • 缺少异常清理路径的回归测试 @ rtp_llm/models_py/modules/dsv4/fp8/indexer.py:25
    • 建议:增加 mock 单测:让 prepare 与 discard 分别抛出不同异常,断言 discard 被调用且最终传播 prepare 的原始异常;同时验证 _discard_prefill_k_cache_gather 会吞掉清理异常。

Checklist Findings (1 fail / 75 total)

General Principles Checklist

  • [6.1] Tests — 新逻辑有聚焦单测 + 相关集成/smoke 测试 → issue 缺少异常清理路径的回归测试
    本次修复针对异常清理时 suppress 未定义而掩盖原始错误的问题,但现有测试未覆盖 prepare_assemble_indexer_k_async 失败后的回收分支,也未验证 discard 同时失败时仍传播原始异常,因此该故障可能再次引入而未被自动化测试发现。

Strengths

  • 修改范围准确且最小,正常推理路径与资源生命周期不变。
  • 清理失败不再覆盖主异常,同时保留尽力回收异步资源的语义。
  • 模块级导入不会增加热路径开销。

from __future__ import annotations

import os
from contextlib import suppress

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P2] 缺少异常清理路径的回归测试

本次修复针对异常清理时 suppress 未定义而掩盖原始错误的问题,但现有测试未覆盖 prepare_assemble_indexer_k_async 失败后的回收分支,也未验证 discard 同时失败时仍传播原始异常,因此该故障可能再次引入而未被自动化测试发现。

建议: 增加 mock 单测:让 prepare 与 discard 分别抛出不同异常,断言 discard 被调用且最终传播 prepare 的原始异常;同时验证 _discard_prefill_k_cache_gather 会吞掉清理异常。

Checklist: [6.1] 新逻辑有聚焦单测 + 相关集成/smoke 测试

@rtp-llm-review-bot rtp-llm-review-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm ready to ci

评审版本:cdf9a9779fb62e76425ff8432c49d48cfc3f9a31

@rtp-llm-review-bot
rtp-llm-review-bot dismissed their stale review September 23, 2026 04:57

自动评审结论已更新,此前的通过结论不再适用。

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants