Skip to content

[SWE] Make Gym startup metrics optional and atomic - #22

Draft
seonjinn wants to merge 3 commits into
sdevare-nv:sdd/multilingual-fixesfrom
seonjinn:seonjinn/swe-startup-metrics-20260811
Draft

[SWE] Make Gym startup metrics optional and atomic#22
seonjinn wants to merge 3 commits into
sdevare-nv:sdd/multilingual-fixesfrom
seonjinn:seonjinn/swe-startup-metrics-20260811

Conversation

@seonjinn

@seonjinn seonjinn commented Aug 12, 2026

Copy link
Copy Markdown

Problem

We need to understand where SWE rollout startup time is spent: connecting to NeMo Gym, creating the runtime, or initializing the agent.

The existing timers write into a NeMo Gym metrics file. This makes standalone OpenHands runs fragile when the file is not configured.

High-level change

This PR makes the existing startup timers safe and optional.

NeMo Gym run:     measure startup phases -> update the metrics file
Standalone run:   no metrics file        -> continue normally

It keeps the existing phase boundaries and timer names. It only changes how their results are published:

  • metrics are enabled only when NEMO_GYM_METRICS_FPATH is set;
  • standalone OpenHands runs cleanly skip the metrics write;
  • existing metric values are preserved; and
  • each JSON update is written atomically.

Expected impact

This is an observability change, not a rollout speedup by itself. It gives us reliable phase-level data needed to find and validate Init and Framework overhead reductions.

It does not change agent commands, retries, controller behavior, or runtime setup.

Change Type

  • Bug fix
  • New feature
  • Breaking change
  • Refactor
  • Other

Release Notes

  • Include this change in the Release Notes.

Signed-off-by: seonjinn <sna@nvidia.com>
Signed-off-by: seonjinn <sna@nvidia.com>
Signed-off-by: seonjinn <sna@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant