feat(rollout): integrate MInf ledger capture with checkpointing

[lauradang]

Summary

• configure NeMo Gym to use deferred Megatron-Inference ledger capture for Megatron generation
• fetch completed token records from the per-rank MInf metadata ledger and durably stage them in TQ before sealing an agent rollout
• convert pending Gym receipts into committed TQ manifests while preserving response IDs, chain hashes, cumulative hashes, and policy weight versions
• discard MInf metadata-ledger rows only after TQ staging succeeds
• support the existing colocated and non-colocated SingleController MInf paths

Dependencies

• Base stack: feat(rollout): add partial recovery benchmark telemetry #3585 (amahishi/partial-rollout-telemetry)
• Gym capture contract and handlers: feat(token-capture): add Megatron ledger capture backend Gym#2823
• MInf metadata ledger: tdene:Megatron-LM:tde/ledger_capture

Testing

• Ruff check and format verification pass for all newly touched RL files
• Gym scoped pre-commit passes
• Gym token-capture and focused model-server suites: 268 passed

Local test limitation

The RL lockfile supports Linux x86_64/aarch64 only, so the focused RL pytest files cannot run natively on this macOS host. An ephemeral dependency run progressed through PyTorch, Transformers, and pyzmq before reaching the NVIDIA-only pynvml runtime dependency. The PR remains draft pending Linux CI.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[lauradang]

/ok to test 0ac1707

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: 0ac1707 (PR #3869 from laurad/minf-rollout-checkpointing)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of amahishi/partial-rollout-telemetry branch (fast-forward)

All submodule changes look good! ✨

[lauradang]

Review of the lauradang commits only (08faaee37, 0c5e033ec, 0ac170793). Verified against the pinned Gym submodule (5cd5c0d), the pinned Megatron-Bridge → Megatron-LM (0c565c9a0 → d12f6c8c9), and the tdene:Megatron-LM:tde/ledger_capture branch. 4 inline comments below, plus one Gym-side issue on the submodule bump that I can't anchor inline:

Also verified OK, for the record: the exactly-one-rank expectation in flush_token_capture matches the MInf branch (ledger is populated only on the MP-coordinator rank per DP engine; fan-out-and-union is the documented pattern), and set_generation_epoch from global rank 0 propagates to all engines via the DP coordinator.

[lauradang]

[DEP/blocker] The pinned Megatron-Bridge (0c565c9a0 → Megatron-LM d12f6c8c9) has no local_metadata_ledger_* / fetch_from_metadata_ledger API (verified by git-grep at the pinned commits; only set_generation_epoch exists there). So with this PR as-is, enabling megatron token capture always hits this raise at setup. Failing loudly here is right, and the PR body declares the tdene:Megatron-LM:tde/ledger_capture dependency — but the PR doesn't bump the Megatron-Bridge pin, so this needs the MInf ledger to land and the submodule bump to be part of (or precede) the merge. Flagging so the merge order is explicit.

[lauradang]

Pending merge: https://github.com/tdene/Megatron-LM/tree/tde/ledger_capture

[github-actions]

✅ Submodule Fast-Forward Check Results

Check based on commit: cead0a4 (PR #3869 from laurad/minf-rollout-checkpointing)

✅ Submodules that are properly updated:

Gym: ✅ PR branch is ahead of amahishi/partial-rollout-telemetry branch (fast-forward)

All submodule changes look good! ✨