Conversation
Add RSL-RL config dataclasses for FlashSAC (RslRlFlashSacActorCfg, RslRlFlashSacCriticCfg, RslRlReplayBufferCfg, RslRlFlashSacAlgorithmCfg) and RslRlOffPolicyRunnerCfg, whose asdict() yields the exact key contract that rsl_rl's FlashSAC.construct_algorithm expects (defaults are the single source). Add MjlabOffPolicyRunner (persists common_step_counter, gates W&B upload, ONNX export via dynamo=False) and a VelocityOffPolicyRunner that exports ONNX on save. Register Mjlab-Velocity-Flat-Unitree-G1-FlashSAC (512 envs). Selectable purely by config; train.py/play.py need no changes. Changelog updated.
FlashSAC's own many-env (IsaacLab) config uses 3-step returns for locomotion; align the G1 velocity FlashSAC task with it. Verified the task trains (reward trending up, critic loss decreasing monotonically) on a live 512-env run.
Revert actor/critic hidden dims to 128/256 (paper's unified GPU-sim config) instead of 512; the paper's proven recipe converges faster than an oversized net under a fixed step budget. Verified the full algorithm matches the paper method (inverted-residual + preact-BN + post-RMSNorm + distributional critic + adaptive reward scaling + weight norm; cross-batch value prediction; noise-repetition; unified entropy target).
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Integrating the RSL-RL wrapper for FlashSAC (rsl-rl pull-request). Tested by running training with the example
Mjlab-Velocity-Flat-Unitree-G1environment and training progresses flawlessly.