Document which chat template the released RL Zero checkpoints used - #1929
JOhnsonKC201 wants to merge 2 commits into
Conversation
The RL Zero Code and IF scripts on main select the simple rlzero templates added in allenai#1216, but the released checkpoints were trained with olmo_thinker at d928a7c and ship that template on the Hub. Say so in the olmo3 README and at the top of the two scripts.
Codex · codex://threads/01a0d61a-a285-7931-ab3a-39b0134c4a25 Replace invalid checkout-and-launch advice with historical code and script sources. Distinguish recorded runs, shipped templates and unverified checkpoint/evaluation provenance; identify the original Math template and later 3.1 recipe. Add General and Math script warnings.
abhishekraok
left a comment
There was a problem hiding this comment.
Codex · codex://threads/01a0d61a-a285-7931-ab3a-39b0134c4a25
Reviewed corrected head 0da772f9e3e906f3ee774e1641d66c33154a8016, including an independent Claude review with no remaining findings.
The update removes the invalid checkout-and-launch example, separates recorded Code/IF/General configurations from Mix's shipped-template evidence, identifies the original Math olmo_thinker_dapo template, and links historical script snapshots while preserving the remaining reproduction limits. It also warns that General's template definition and the current Math recipe have changed.
Validation: 54 CPU tests pass; style, quality and type checks pass; the four edited shell scripts parse; documentation builds; the five public release templates match the historical definitions stated in the note. Fresh GitHub CI is still pending on this head. No training run was launched, and the exact paper-evaluation provenance remains open in template-mismatch (#1899).
Refs #1899.
Following up on @abhishekraok's reply in the issue, this documents which chat template the released RL Zero checkpoints were trained with, since the scripts on
mainnow select a different one.What I checked before writing it down:
olmo_thinkerindataset_transformation.pyatd928a7c(the commit the README table lists for the RL Zero runs) is byte-identical to thechat_template.jinjaon the Hub forOlmo-3-7B-RL-Zero-Code,-IF,-Generaland-Mix. All four Hub files are the same 1652 bytes.olmo_thinker_rlzeroandolmo_thinker_code_rlzerodon't exist atd928a7c. They came in with rlzero template and fixes to rlzero scripts #1216 about six weeks later.olmo_thinkeronmainis no longer the same string as atd928a7c, so switching the flag back onmainwould not give the released template either.Olmo-3-7B-RL-Zero-Mathis different: its Hub template is a simple prompt, the same asolmo_thinker_rlzeroexcept it says "math problem".The claim that the Code and IF wandb runs record
chat_template_name=olmo_thinkeris taken from the issue thread, I can't see those runs.Changes:
scripts/train/olmo3/README.md: new "RL Zero chat templates" section with the above.7b_rlzero_code.shand7b_rlzero_instruction_following.sh: a three line comment at the top pointing to that section.I left the
--chat_template_nameflags alone. Whether the scripts should go back to the release recipe or stay on the simple templates seems like your call, and the README now says which is which either way. Happy to change the flags or add the same comment to the math and mix scripts if you'd prefer.No changes under
open_instruct/, so no changelog entry.