docs: keep a real tuned roster, and the script that sizes it

[CryptoJones]

Two additions, both about the same problem: a working config is mostly numbers
somebody had to measure, and none of that survived anywhere durable.

examples/full-roster.yaml

A real thirteen-lane roster. The annotated template stays exactly as it is — it
documents options this roster doesn't use (rpm, strip_prefixes, scrub_env,
synthesizers, price_per_mtok, budget_usd) — but a template can't show what
a config looks like after the measurements are in, and those comments are the
point of the file:

• every cli lane pins --model, because without it they silently run the CLI
  default and two lanes reported different models while running the same one
• the three :free lanes are deliberately not provider-pinned; pinning a
  :free variant returns 429 where the identical unpinned call succeeds
• max_tokens on the reasoning lanes is sized from measured distributions

Sanitised: a local server address and a filesystem path are placeholders.
key_entry values are kept — they name a pass entry rather than holding
one, and the naming is a convention, not a secret. Verified no internal
addresses, home paths, or secret-shaped strings survive, and it loads and lists
13 lanes.

scripts/measure-reasoning.py

How those numbers were arrived at. On an OpenAI-compatible endpoint max_tokens
caps reasoning and answer together, so a model whose thinking overruns it
returns 200 with empty content and finish_reason: "length" — a lane that
delivered nothing, for a reason the response never states.

Worse, the cost is a distribution:

Cerebex      reasoning    20 - 21606     (three orders of magnitude, one brief)
SELMA        reasoning 10153 - 17620
Neuromancer  reasoning  3253 -  6293

Sizing off a single observation is exactly how 4000 was first picked, and it
was wrong. The script takes several samples, reports the range, and says whether
the configured max_tokens clears the worst one — the difference between a
number and a guess, and reproducible now rather than living in a scratchpad that
gets deleted.

Proudly Made in Nebraska. Go Big Red! 🌽 https://xkcd.com/2347/

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: abb012cc-9c66-44aa-a3e7-80d3233ff9db

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}