feat(data): batch 1: request models inherit their TaskData schema (51 servers)

[adil-a]

What this does

Each server's run/verify request model now inherits its task_data.py schema. Task fields are declared once, in the schema. The wire model and the schema can no longer drift apart, because they are the same class.

Before:

class StringMatchRunRequest(BaseRunRequest):
    expected_answer: str
    extraction_mode: Literal["boxed", "final_answer", "last_line", "full_response"] = "final_answer"
    case_sensitive: bool = False
    metadata: Optional[dict[str, Any]] = None

After:

class StringMatchRunRequest(TaskData, BaseRunRequest):
    pass

This is batch 1 of the inherit refactor: the 51 flat-row servers whose models import in the core venv. Later batches cover the servers that nest fields in verifier_metadata (xstest, longmemeval, deepswe, and the vm family), the 13 servers that keep an explicit extra="ignore", the heavy-dependency servers that need their own venvs, and the base-model servers whose verify ignores task fields.

Behavior changes, all intentional

• Handling of undeclared fields does not change. Models that dropped unknown fields before keep an explicit extra="ignore" with a comment, so a stray column in user data behaves exactly as it does today. Only fields the schema declares are kept and typed. Widening any server to extra="allow" is a separate, per-server decision for a later PR.
• 69 fields become typed. Row fields the schema declares but the old wire did not (for example bird_sql.sql_context, litmus_agent.smiles) used to ride as untyped extras. They are now declared fields with types.
• 14 documented contract deltas where the reviewed schema is deliberately stricter than the old wire. Examples: arc_agi.train loses its [] wire default (the default masked missing data and produced a degenerate comparison), and math_advanced_calculations.ground_truth drops the list[float] branch (that branch crashed verify() with json.loads on a list). The full list with justifications is in the verification report comment on this PR.

Two servers needed a code fix, not just the model swap. litmus_agent.verify() read legacy property_type out of model_extra; the typed field no longer lands there, so the resolver now receives it explicitly. indirect_prompt_injection's tests construct verify requests by hand and now supply the wire-required environment row field. Three servers keep duplicated field classes out of the schema on purpose: single_step_tool_use_with_argument_comparison and format_verification redeclare one field each because their verify logic isinstance-checks or dict-reads those exact types, and verifif/indirect_prompt_injection re-point their item classes to the schema module so there is a single definition.

How this was verified

• Contract gate. Every *RunRequest/*VerifyRequest model on this branch was dumped (fields, types, required-ness, defaults, extra config) and diffed against the same dump from the parent branch. Result: 0 unexplained differences and 0 extra-config changes. The only changes are the typed field additions and the 14 documented deltas.
• Row gate. Every committed data row of all 51 servers was validated by both the old and the new models: 910 (model, row) pairs. Accept/reject agrees on every pair. Overlapping dump values are equal on every pair. The only dump difference is 133 schema-declared fields the old wire silently dropped and the new one keeps as typed fields. Undeclared fields are dropped exactly as before.
• Server tests. All 51 servers' own test suites pass. Three suites (asr_with_pc, conversational_tool_use_simulation, math_proof_judgement) fail identically on the parent branch because the shared core venv lacks their extra dependencies; gym env test covers those with per-server venvs.
• Core suite and drift gate. The full tests/unit_tests/ suite passes, including the repo-wide row-vs-schema drift test from the parent PR.
• Golden routing gate. Parent-branch capture vs this branch, strict compare, 0 differences.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[adil-a]

/ok to test 447ccba

[adil-a]

/claude review

[claude]

SHIP — LGTM, no reliability concerns.

Verified this as a contract-preserving refactor: each server's per-task request fields move from an inline model into a task_data.py TaskData mixin. I checked field requiredness, defaults, types, and extra behavior against the removed inline definitions across a representative sample (aalcr, circle_click/count, string_match, mrcr, math_advanced_calculations, workplace_assistant, terminal_bench_2_1, litmus_agent, and the simpleqa judge-QA family heirs abstention/omniscience/frontierscience_judge). In every case the wire contract is preserved; the extra="allow" additions on previously-strict models (mrcr, arc_agi, ruler, …) are a strict loosening, not a break.

The two genuine logic changes both check out:

• litmus_agent/app.py verify() — property_type becomes a typed field, so it no longer lands in model_extra. The patch copies model_extra and re-injects property_type via setdefault, so the legacy resolution path (_resolve_verification_policy, _compute_abs_error, both reading extra.get("property_type")) still resolves correctly. Correct.
• indirect_prompt_injection — IPIVerifyRequest now requires environment (previously an accepted extra). This is the one wire-contract tightening in the PR. Verified the shipped data/example.jsonl rows carry environment top-level and the test suite was updated to pass it (environment=SAMPLE_ENV), so the benchmark works. NOTE (author's call): any external IPI dataset that omits top-level environment will now 422 at verify time instead of being silently ignored — worth a heads-up if custom IPI datasets exist downstream.

The deliberate redeclare-in-subclass cases (format_verification verifier, single_step..._argument_comparison expected_action, graphwalks expected_answer/problem_type) are correct and documented — they keep the concrete wire classes that verify() isinstance/dict-checks against.

No async, dependency, or public-API concerns; task_data.py modules are plain Pydantic with no new imports.

[adil-a]

/ok to test 91974de

[adil-a]

/ok to test 4658119

[adil-a]

/ok to test 744e36e

[adil-a]

/ok to test c01c354

[adil-a]

/ok to test abd8cda

[adil-a]

/ok to test f183a5b

[adil-a]

/ok to test e8c4f86

[adil-a]

/ok to test 76b0b9b