Skip to content

feat(token-capture): add Megatron ledger capture backend - #2823

Open
lauradang wants to merge 4 commits into
NVIDIA-NeMo:pthombre/tq-tokidcap-capturefrom
lauradang:laurad/megatron-adapter
Open

feat(token-capture): add Megatron ledger capture backend#2823
lauradang wants to merge 4 commits into
NVIDIA-NeMo:pthombre/tq-tokidcap-capturefrom
lauradang:laurad/megatron-adapter

Conversation

@lauradang

@lauradang lauradang commented Aug 27, 2026

Copy link
Copy Markdown

Summary

  • add deferred Megatron-Inference ledger capture and token-free pending call records
  • extract vLLM worker and Megatron ledger capture into typed backend handlers with a shared lifecycle
  • preserve served response IDs and token-chain custody through deferred MInf staging
  • stage Megatron records at rollout completion while preserving existing vLLM worker staging behavior
  • add ledger, adapter, handler, and model-server integration coverage

Testing

  • pre-commit run --files on all changed working-tree files
  • Gym token-capture and focused model-server suites — 268 passed

Baseline note

The complete responses_api_models/vllm_model/tests/test_app.py run has 15 response-ID expectation failures (chtcmpl-123 versus resp_123). A representative failure reproduces unchanged on pthombre/tq-tokidcap-capture, so it is not introduced by this branch.

Dependencies

Signed-off-by: Laura Dang <laurad@nvidia.com>
Signed-off-by: Laura Dang <laurad@nvidia.com>
@copy-pr-bot

copy-pr-bot Bot commented Aug 27, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Signed-off-by: Laura Dang <laurad@nvidia.com>
Signed-off-by: Laura Dang <laurad@nvidia.com>
@lauradang
lauradang marked this pull request as ready for review August 28, 2026 19:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant