feat: super v3.5 posttraining with flexible length penalties - #3852

Merged

yfw merged 27 commits into

super-v3.5-posttrainingfrom

adithyare/super_v3.5_posttraining_lenpen

Aug 27, 2026

Commits on Aug 26, 2026

Port GDPO length adjustment features to GitHub main
arendu
committed
7f013f8
fix: require explicit think_close token id for think-count feature

arendu
and
claude
committed
2407f73
chore: add copyright header to build_profile_band_dataset.py

arendu
and
claude
committed
a55eb22
fix: propagate length_adjustment_config to async NeMo-Gym rollouts

arendu
and
claude
committed
b8ca5c3
feat: profile_band multiplier clamps at f past b instead of decaying to 0

arendu
and
claude
committed
25df5bf
feat: global profile_band defaults for datasets without per-prompt bands

arendu
and
claude
committed
4d73c5b
test: unit tests for profile_band and group relative-length scaling

arendu
and
claude
committed
c0f605f
docs: document profile-gate defaults, fail-closed behavior, and global-band threshold source

arendu
and
claude
committed
f8724c5
chore: drop example/recipe configs from the lenpen PR

arendu
and
claude
committed
50244bc
fix: profiled_length_penalty requires min_samples PASSING profiled rollouts

arendu
and
claude
committed
16e4d84

Commits on Aug 27, 2026

revert: restore should_log_nemo_gym_responses default in grpo_nanov3.yaml

arendu
and
claude
committed
f02100f
refactor: strip GDPO feature mode; length adjustments are reward-mutation only

arendu
and
claude
committed
541be20
fix: keep validation metrics on raw env reward

arendu
and
claude
committed
87bea4f
fix: band multiplier negative-base guard, config validation, top_percentile default

arendu
and
claude
committed
bbcc4d9
feat: clamp length-adjusted rewards at 0 for correct rollouts

arendu
and
claude
committed
7a4d888
refactor: gate per-row metadata copying on an actual length_bonus config

arendu
and
claude
committed
b4d4b73
feat: restrict length adjustments to binary (0/1) env rewards

arendu
and
claude
committed
ded553a
refactor: rename length_bonus -> length_penalty (config key, module, docs)

arendu
and
claude
committed
aa2a1a8
feat: pass-rate-scaled length penalty (MAI-style)

arendu
and
claude
committed
44de07d
fix: pass-rate penalty normalizes by longest CORRECT rollout

arendu
and
claude
committed
89251d4
style: satisfy pre-commit hooks (rename doc, ruff format/isort, F541)

yfw
and
claude
committed
c89b83f
feat(launcher): 4-GPU-node support, sbatch comment/qos/extra-dependency, overridable sandbox command

arendu
and
claude
committed
3973211
feat(launcher): optional stable RESULTS_DIR layout

arendu
and
claude
committed
d202b07
feat(launcher): optional --segment sbatch flag with divisibility guard

arendu
and
claude
committed
49746de
fix(launcher): drop nemo_gym_log_dir override — key not in this branch's schema

arendu
and
claude
committed
9901ae3
fix(launcher): force-override (++) gym venv keys — absent from pipeline-derived configs

arendu
and
claude
committed
79470b5
fix: enabled defaults True consistently in apply_group_length_penalties

arendu
and
claude
committed
1643600

Navigation Menu

Uh oh!

feat: super v3.5 posttraining with flexible length penalties - #3852

feat: super v3.5 posttraining with flexible length penalties#3852
yfw merged 27 commits into
super-v3.5-posttrainingfrom
adithyare/super_v3.5_posttraining_lenpen

Commits on Aug 26, 2026

Port GDPO length adjustment features to GitHub main

fix: require explicit think_close token id for think-count feature

chore: add copyright header to build_profile_band_dataset.py

fix: propagate length_adjustment_config to async NeMo-Gym rollouts

feat: profile_band multiplier clamps at f past b instead of decaying to 0

feat: global profile_band defaults for datasets without per-prompt bands

test: unit tests for profile_band and group relative-length scaling

docs: document profile-gate defaults, fail-closed behavior, and global-band threshold source

chore: drop example/recipe configs from the lenpen PR

fix: profiled_length_penalty requires min_samples PASSING profiled rollouts

Commits on Aug 27, 2026

revert: restore should_log_nemo_gym_responses default in grpo_nanov3.yaml

refactor: strip GDPO feature mode; length adjustments are reward-mutation only

fix: keep validation metrics on raw env reward

fix: band multiplier negative-base guard, config validation, top_percentile default

feat: clamp length-adjusted rewards at 0 for correct rollouts

refactor: gate per-row metadata copying on an actual length_bonus config

feat: restrict length adjustments to binary (0/1) env rewards

refactor: rename length_bonus -> length_penalty (config key, module, docs)

feat: pass-rate-scaled length penalty (MAI-style)

fix: pass-rate penalty normalizes by longest CORRECT rollout

style: satisfy pre-commit hooks (rename doc, ruff format/isort, F541)

feat(launcher): 4-GPU-node support, sbatch comment/qos/extra-dependency, overridable sandbox command

feat(launcher): optional stable RESULTS_DIR layout

feat(launcher): optional --segment sbatch flag with divisibility guard

fix(launcher): drop nemo_gym_log_dir override — key not in this branch's schema

fix(launcher): force-override (++) gym venv keys — absent from pipeline-derived configs

fix: enabled defaults True consistently in apply_group_length_penalties