fix: keep local archive available during GitHub outages

[mariusvniekerk]

Forge is a local-first console, but startup previously required GitHub credential and identity lookups to succeed before it served archived data. A temporary GitHub outage could therefore make the existing local archive unavailable.

This change treats only temporary network, timeout, rate-limit, and GitHub server failures as degraded startup. Forge preserves configured routes and normal scheduled sync, while using conservative host-level personal access token accounting until the next restart. Missing credentials, invalid configuration, and permanent authentication failures still stop startup.

The live daemon was not restarted during verification. The startup decision is covered by focused tests with injected upstream failures.

Validation

• make test-short — 6,559 tests passed; 226 skipped.
• make lint-check — zero issues.

^{generated by a clanker}

[roborev-ci]

roborev: Combined Review (96c0284)

High-severity issue remains: degraded GitHub startup can still block archive availability.

High

• cmd/kenn-forge/main.go:573 — resolveStartupRepos immediately retries GitHub for every configured repository before activating the real server. App credentials are reminted per repository, while PAT requests lack an operation deadline and may stall indefinitely, leaving archive APIs on the startup 503 handler. Propagate degraded GitHub hosts from provider startup and resolve their repositories from SQLite or synthetic fallback; defer network retries to scheduled sync.

---

Reviewers: 2 done | Synthesis: codex, 6s | Total: 3m58s

[wesm]

This overlaps with #928, i will let you reconcile

[roborev-ci]

roborev: Combined Review (91f2504)

Startup resilience is improved, but one medium-severity rate-limit case still blocks degraded startup.

Medium

• cmd/kenn-forge/provider_startup_fallback.go:103 — GitHub App rate limits may return HTTP 403, but githubapp.StatusError treats only 429 as transient. A 403 rate-limit response can therefore still prevent startup, and current tests cover only 429. Preserve response headers in githubapp.StatusError, detect 403 rate limits using X-RateLimit-Remaining or Retry-After, and add behavioral coverage.

---

Reviewers: 2 done | Synthesis: codex, 14s | Total: 4m23s

[wesm]

looking

[roborev-ci]

roborev: Combined Review (bb292f0)

High-severity startup reliability issue remains, along with two medium-severity error-classification gaps.

High

• cmd/kenn-forge/main.go:544 — Degraded startup is not communicated to the caller, so resolveStartupRepos can still make synchronous, serial GitHub requests before the full server replaces the startup handler. A hanging upstream may leave archive APIs returning 503 indefinitely.
  • Fix: Return degraded host state and resolve those repositories exclusively from SQLite/config during startup, deferring network resolution to background sync. Add a test using a blocking repository reader.

Medium

• cmd/kenn-forge/provider_startup_fallback.go:105 — A GitHub App secondary-rate-limit response can be HTTP 403 without Retry-After or an exhausted primary-rate-limit header. The response body identifies the secondary limit, but the code classifies it as permanent and aborts startup.

  • Fix: Parse the structured error body to recognize secondary limits while keeping unrelated 403 responses fatal. Cover the headerless response shape.

• cmd/kenn-forge/provider_startup_fallback.go:42 — The first transient credential failure switches every GitHub route to the synthetic resolver, preventing later routes from exposing permanent errors such as 401. This violates the invariant that permanent authentication failures remain fatal.

  • Fix: Resolve and degrade routes or hosts independently, continuing validation so permanent failures take precedence. Add mixed transient/permanent coverage.

---

Reviewers: 2 done | Synthesis: codex, 7s | Total: 4m48s

[roborev-ci]

roborev: Combined Review (e06048a)

High-risk concurrency issue found, plus a medium-risk retry flaw.

High

• cmd/kenn-forge/provider_startup_fallback.go:50 — BindSourceIdentity allows the fallback resolver to escape startup. Concurrent token revalidation can write to the shared, unsynchronized degradedProviderHosts map, causing a concurrent-map-write crash.
  • Fix: Restrict the fallback resolver to initial discovery and bind live sources to the primary resolver, or synchronize and freeze degraded-host state after startup.

Medium

• cmd/kenn-forge/provider_startup.go:289 — Transient App-token mint failures retain no retry window. Immediate parallel repository syncs can repeatedly retry minting and hammer a rate-limited endpoint.
  • Fix: Single-flight failed App mints and cache a bounded retry deadline based on Retry-After or X-RateLimit-Reset.

---

Reviewers: 2 done | Synthesis: codex, 9s | Total: 6m38s