Skip to content

fix: throttle worker heartbeat noise and align sidecar state#11

Open
kbentonferguson wants to merge 1 commit into
professorpalmer:mainfrom
kbentonferguson:bferguson/debug-pm-heartbeat-sidecar
Open

fix: throttle worker heartbeat noise and align sidecar state#11
kbentonferguson wants to merge 1 commit into
professorpalmer:mainfrom
kbentonferguson:bferguson/debug-pm-heartbeat-sidecar

Conversation

@kbentonferguson

Copy link
Copy Markdown
Contributor

Summary

  • throttle worker heartbeat/lease renewal away from the hot poll_seconds=0.1 loop
  • add explicit --heartbeat-seconds support while preserving non-default --poll-seconds as the compatibility fallback
  • keep heartbeat cadence safely below lease expiry for short leases
  • scope PUPPETMASTER_STATE_DIR around inline workers so adapter sidecars land under the owning store root
  • add regression coverage for heartbeat cadence, short-lease margin, and inline state-dir restoration

Why

Puppetmaster workers were renewing leases and emitting heartbeat/save events every 0.1s, producing thousands of events for ordinary runs and making logs/status debugging noisy. Slowing the default heartbeat path cuts that event churn while preserving lease safety and existing callers that tuned --poll-seconds.

Inline worker mode also did not export the resolved store root through PUPPETMASTER_STATE_DIR, so adapter sidecar logs could be written under the wrong project state dir when the parent environment pointed elsewhere.

Tests

  • python3 -m pytest tests/test_puppetmaster.py -q -k 'heartbeat or inline_workers_scope_state_dir'
  • env -u CODEX_COMMAND -u CODEX_CI -u CODEX_HOME -u CODEX_PATH -u CODEX_THREAD_ID python3 -m pytest tests/test_puppetmaster.py -q
  • git diff --check
  • Puppetmaster confidence review:
    • tests review job_391b13303b3b: no blocking issue
    • inline state-dir review job_08b8c258fd9e: no blocking issue
    • heartbeat review job_d5fb9427fc66: surfaced compatibility/short-lease findings addressed in this PR

@kbentonferguson kbentonferguson changed the title Introduce heartbeat control, improve task management, and respect state_dir_env fix: throttle worker heartbeat noise and align sidecar state Jun 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant