Refresh API Explorer pages on code and asset changes in serve

[akira28]

docs-builder serve under dotnet watch now shows C#, Razor, JS, and CSS changes on API Explorer pages without a restart. Regenerating the API pages after an edit takes about 12 s instead of 25–31 s.

Affects: API reference

Prompt summary: The author wants to iterate on API Explorer pages in ./dev.sh serve without restarting the container after each C#, HTML, or JS change. They also asked whether spec regeneration can run in parallel.

Why

The dev server generates API pages once, on the first /api/ request, and regenerates them only when a spec or API Markdown file changes. A hot-reloaded code change refreshes the browser, but the browser gets the old pages. A Parcel rebuild does not refresh the browser at all, because live reload only watches .md and .yml. Each regeneration also spends about 13 s deleting the previous output from the in-memory file system before it renders anything.

What

Hot reload marks API pages stale

When dotnet watch applies a code change, the server marks the API pages for regeneration even if no spec changed. The browser refresh that follows gets freshly rendered pages. Flags and timestamps are now recorded before generation, so a change that lands mid-generation is not lost. A failed generation is retried on the next request.

Parcel rebuilds refresh the browser

Under dotnet watch, the server watches src/Elastic.Documentation.Site/_static and refreshes the browser when Parcel writes JS or CSS. /_static responses send Cache-Control: no-cache. Without that header, a refresh can reuse a cached bundle, because generated pages keep the asset hash from when they were rendered.

Versions of one product render in parallel

Products were already generated in parallel. The versions of one product ran one after another, so Elasticsearch (3 versions) and Kibana (4 versions) set the total time. Each version has its own navigation, render context, and output folder, so they now render in parallel.

API pages are overwritten in place

The server no longer deletes the generated API tree before it regenerates it. A recursive delete on the in-memory file system took longer than the render itself. HTML pages are now written with FileMode.Create, so a shorter page no longer keeps trailing bytes from the previous one.

Out of scope: pages within one version still render sequentially, so the largest version (Elasticsearch main, about 8 s) sets the floor. Specs are also downloaded and parsed again on every regeneration (about 3–5 s). A page whose operation is removed from a spec stays reachable until the server restarts.

Verify

dotnet test tests/Elastic.ApiExplorer.Tests/
# GenerateProducts_RegenerateWithShorterPage_TruncatesPreviousOutput
./dev.sh serve
# open http://localhost:3000/api/doc/elasticsearch/
# edit src/Elastic.ApiExplorer/Landing/LandingView.cshtml; the page refreshes with the change

🤖 Generated with Claude Code

[github-actions]

Docs preview (local build)

Handbook preview: https://docs-v3-preview.elastic.dev/elastic/docs-builder/pull/4321/

[github-actions]

[LOW] _forceApiRegeneration can lose a concurrent invalidation

EnsureApiReferencesAsync reads _forceApiRegeneration and then clears it in two separate operations:

var force = _forceApiRegeneration;
_forceApiRegeneration = false;

If InvalidateApiReferences() runs between those two lines, its true write can be overwritten by the clear here. In that interleaving, a hot-reload update may be dropped and the next /api request can return without regenerating.

Using an atomic consume pattern for the flag (for example Interlocked.Exchange on an int-backed field) would avoid the lost-update window.

[akira28]

Good catch, fixed in f085a42. The flag is now an int that InvalidateApiReferences sets with Interlocked.Exchange(ref _forceApiRegeneration, 1), and EnsureApiReferencesAsync consumes it with Interlocked.Exchange(ref _forceApiRegeneration, 0) == 1. An invalidation that lands mid-generation is either consumed by the current run or leaves _apiReferencesStale set for the next request.

[github-actions]

[LOW] Stale API pages can survive spec removals during serve

Switching from recursive delete to in-place overwrites means pages that are no longer generated (for example, after removing an operation/path from a spec) remain on disk and keep being served until restart. That makes removed endpoints still appear valid in local API docs and can mask regressions while iterating.

Could we add a lightweight cleanup strategy during regeneration (for example, tracking generated outputs per run and deleting leftovers) so removed pages stop resolving without bringing back full-tree delete cost?