Skip to content

Upstream o1/o3 mid-name segment matching into tinyagents context_window_for_model_id and drop the host fallback #4566

Description

@senamakel

Summary

model_context.rs (merged in #4539) delegates generic context-window resolution to the crate's tinyagents::harness::model::context_window_for_model_id, but the crate does not match an o1/o3 token embedded mid-name (e.g. ollama/mistral-for-o1-benchmark). OpenHuman therefore keeps a host-side o1_o3_segment_context fallback. Upstream the segment matching into TinyAgents so the host fallback can be deleted.

Problem / Context

The crate resolves canonical o1/o3 ids (o1, o1-mini, o3-mini, openai/o1-preview) but returns None for a bounded -o1-/-o3- segment inside a longer id. The pre-port host table (PR #2100 regression guard) matched those. To preserve behavior, #4539 restored a minimal host heuristic (o1_o3_segment_context in src/openhuman/inference/model_context.rs) that runs only when the crate returns None: an o1/o3 token delimited by non-alphanumeric boundaries maps to the 200K window, while substrings (solo1-7b, proto3-chat, octo3thing) do not over-match. This is a small duplication that should live in the crate.

Scope

  • In scope: add bounded o1/o3 (o-series) segment matching to context_window_for_model_id upstream (tinyhumansai/tinyagents), release, bump the host pin, then delete o1_o3_segment_context and route through the crate.
  • Not in scope: OpenHuman tier aliases / cost-catalog arms (stay host-side, checked before the crate fallback).

Acceptance criteria

  • Upstreamcontext_window_for_model_id matches bounded mid-name o1/o3 segments (with the non-alphanumeric-boundary guard so solo1-7b/proto3-chat/octo3thing stay unmatched), with crate tests
  • Host cutoversrc/openhuman/inference/model_context.rs drops o1_o3_segment_context; o1_o3_segment_match_does_not_overmatch passes purely via the crate
  • Diff coverage ≥ 80% — cutover PR meets the changed-lines gate

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    Status
    Todo

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions