fix(sc): gate reference logprobs on KL penalty

[jinglinglingling]

Summary

This PR forward-ports the Zero-KL reference-logprob gating introduced in #2443 to NeMo RL v2 (single-controller async).

• gate v2 SingleController reference-logprob requests on both a positive reference_policy_kl_penalty and the explicit skip flag
• fetch only the policy/reference logprob data-plane columns produced for each train step
• reject non-finite/negative KL coefficients and reward-KL configurations that would omit required policy logprobs
• cover initialization, controller calls, train-field propagation, and invalid configuration combinations

This forward-ports the Zero-KL reference-logprob gating behavior from #2443 to the current v2 single-controller async path.

Test plan

• affected SingleController and setup suites: 56 passed
• broader affected CPU suites (controller, setup, checkpointing, split API): 102 passed
• targeted invalid KL coefficient and data-plane field tests passed
• pre-commit run --all-files

GPU/runtime qualification was not run locally because this environment does not expose a GPU.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[jinglinglingling]

Self-review

No action needed — all high-confidence findings from the self-review were resolved and re-reviewed.

Scope

Reviewed the complete v2 Zero-KL reference-logprob gating change with parallel RL-correctness, bug, test, design, and comment reviewers, followed by an independent devil's-advocate pass.

Findings resolved

• Fixed the zero-KL producer/consumer mismatch: the trainer now fetches only logprob columns produced for the current step.
• Added direct train-pump coverage for all four policy/reference logprob gate combinations.
• Constrained reference_policy_kl_penalty to finite non-negative values.
• Added setup-time rejection for reward-KL configurations that would skip required policy logprobs.

All fixes landed in e06a1b7342855ebeb7d36139853121477bd27533 and the final adversarial pass returned LGTM with no remaining findings at or above the 80% confidence threshold.

Validation

• 56 final SingleController/setup tests passed.
• 102 broader affected CPU tests passed across controller, setup, checkpointing, and split-policy APIs.
• pre-commit run --all-files passed, including Ruff, formatting, and Pyrefly.
• PR is conflict-free.

Limitation

GPU/runtime qualification was not run locally because this environment does not expose a GPU. Full NVIDIA-runner validation and reviewer approval remain pending.

Generated by Cursor Agent using the upstream /review-pr-team workflow.

[yfw]

/ok to test ff0df3f