Add producer-owned training captures for harness sessions

[adithya-s-k]

Summary

Add TrainableSession.fetch_training_trace() with validated TrainingTrace and TrainingTurn objects. OpenEnv selects agent calls and supplies required token masks, so trainers can consume captures without turn-selection hooks or token reconstruction. This provides the producer side of the contract discussed in TRL #6947.

• Require engine prompt/completion IDs, aligned behavior logprobs, a full-sequence loss mask, and unique call IDs. Preserve partial masks and zero-masked agent calls; export shared graph nodes once.
• Implement the API in Harbor and native OpenCode. Keep the existing raw trace and legacy export APIs available.
• Integrate with the merged Harbor explorer UI (feat(harbor): rework the serve web UI and make it safe to deploy publicly #1273): add a typed Training trace download behind the existing download grant, and retain the old contract.json as Capture audit.
• Keep failed native upstream requests in the audit without losing valid agent captures; retain sampled choice.token_ids in unary and streamed responses.
• Validate native OpenCode token/logprob identity with Harbor's normalizer. Reject mismatched agent captures without letting auxiliary calls invalidate valid training turns. Recover logprob IDs when choice token IDs are empty, including a final empty stream chunk.
• Apply native OpenCode's training sampling policy at the proxy and retain streamed prompt IDs and sampled token IDs.
• Treat a single-call rollout as a warning rather than invalid capture. The verifier determines task correctness.

verify() remains the reward source. Trainers still own advantages, packing and loss weighting. Existing capture classification heuristics remain in place; this does not add rollout-normalized weighting or tree attention.

Type of Change

• Bug fix
• New feature
• Documentation

Alignment Checklist

• Read .claude/docs/PRINCIPLES.md
• Checked .claude/docs/INVARIANTS.md
• Full repository lint hook is clean: existing import-order issues in two unrelated tests and formatting issues in 24 unchanged environment files remain. Changed files pass import sorting, formatting and lint; full src/ and tests/ lint passes.

RFC Status

• Not required: additive capture types and session API; existing session and wire APIs remain available.

Test Plan

Merged current main, including #1273 and the existing deprecation notices from #1276. Earlier broad validation: 778 OpenEnv tests passed across Harbor, UI access controls, CLI, capture, native OpenCode and public imports. After the pairing and empty-ID fixes: 111 targeted OpenEnv tests and 76 TRL compatibility tests passed. The two empty-ID regression cases failed before the fix and pass after it. Changed-file lint, import sorting, formatting, and generated environment docs checks pass.

Tests cover typed UI downloads matching HarborSession exactly, legacy audit compatibility, partial and zero masks, eval-only captures, download grants, failed upstream requests, streamed choice IDs and token/logprob mismatches. Dependencies for the new Hub APIs and OAuth tests were isolated from the shared training environment.

No live GPU training run was performed for this revision.

Claude Code Review

N/A. Implementation and local checks performed with Codex.

---

Note

Medium Risk
Changes training data contracts, capture validation severity, and proxy request shaping—areas that affect RL training correctness—but keeps legacy exports and adds strict validation at producer boundaries.

Overview
Introduces a producer-owned training capture API: loop-owning sessions implement fetch_training_trace() and return validated TrainingTrace / TrainingTurn objects (engine token IDs, behavior logprobs, explicit full-sequence loss_mask, unique node_ids). Harbor and native OpenCode wire this through shared to_training_trace() export; raw fetch_proxy_trace() and the legacy Harbor audit export stay available.

Harbor UI splits trainable downloads into Training trace (typed object, matches HarborSession.fetch_training_trace()) and Capture audit (former contract.json, including excluded turns). Legacy captures without explicit masks can still audit but cannot emit the strict trace.

OpenCode transparent proxy gains optional trainer sampling (requires proxy mode), forwards return_tokens_as_token_ids / prompt and completion token IDs through streaming, and builds training traces from proxy JSONL.

Capture validation downgrades a single-model-call rollout from FATAL to WARN so verifier scores and valid tokens are not blocked; trainers are steered toward fetch_training_trace() instead of turn-selection hooks in TRL.

^{Reviewed by Cursor Bugbot for commit b1f5ebd. Bugbot is set up for automated code reviews on this repo. Configure here.}

[bot-ci-comment]

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

[cursor]

Stale Bugbot comment from a previous run.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit b1f5ebd. Configure here.}

[cursor]

Post-merge release note: #1280 shipped in v0.7.0. Exact-head probes at 906c9ae2 (CI and 179 focused tests green) still reproduce these, proposed for 0.7.1:

1. TrainingTrace accepts turns=[] and globally all-zero loss masks, so both the generic and Harbor exporters can return a "successful" trace with zero supervised tokens.
2. Native OpenCode never certifies vLLM --logprobs-mode processed_logprobs; raw pre-temperature logprobs are exported as trainable.
3. Native OpenCode drops a failed tool-bearing main call; if a tool-less title call succeeds, structural fallback labels it as agent work and it gets one supervised token.
4. The Harbor typed exporter ignores HarborRolloutResult.ok (accepts ok=False empty and supervised traces); TRL #6947 ignores the session exit code, so failed results are exported.
5. Policy: new public core APIs (TrainingTurn, TrainingTrace, TrainableSession) and the RFC 006 FATAL-to-WARN change are not yet reflected in RFC 006/012.

View PR

  

_{Sent by Cursor Automation: Release}