Start only selected matrix interpreters and repair crashed runners

A plain `run` started every interpreter instance of a matrixed env while
the run itself only executed the selected subset, so unselected children
could fail to start (or be repaired) for no reason. Separately, an ER
that died before publishing its port surfaced as a raw start error even
though a reinstall usually fixes it.

- Thread the run's env/interpreter selection into the payload-schema
  fetch (`runOptions`, honoured only with `startRunners`). The CLI builds
  one dict that feeds both the fetch and the run, so they cannot drift.
  Selection is computed only for projects with a matrixed action and
  selectors are never validated in the fetch, so a selector valid in a
  sibling project cannot fail it.
- Repair a runner that crashed before its port (ServerExitedBeforePort
  in the __cause__ chain) the same way as NO_VENV: install, then
  restart. Only at the run gate and dispatch start, never during
  metadata resolution, and never for timeouts, which are load problems.
- Rename repair_no_venv_env to repair_env and serialize repairs per
  (project, env) so two callers never install into one venv at once.
- Pass the env's configured interpreter to the create_envs step in
  install_env_for_project so a repair rebuilds the venv with the right
  Python.

Author: Aksem

docs/cli.md

@@ -69,7 +69,7 @@ python -m finecode run [options] <action> [<action> ...] [payload] [--config.<ke
 
 In a multi-project workspace, `run` fans out across every project that declares the action; spawned subprocesses are bounded by the machine-wide process budget (default: derived from the machine's CPU budget). Fan-out is throttled, never refused — workspace size does not limit which actions you can run. See [Process budget](guides/wm-server-internals.md#process-budget).
 
-`--env` and `--interpreter` on `run` use the same selector semantics as `prepare-envs` (ADR-0050): they compose by intersection, and a matrix env's config-declared `default_interpreters` policy (see [Preparing Environments — default interpreter subset](guides/preparing-environments.md#default-interpreter-subset)) applies as the default when neither is given — so a plain `run` can execute only a local subset of a matrix (e.g. the newest interpreter) while CI still runs the full axis, mirroring `prepare-envs`.
+`--env` and `--interpreter` on `run` use the same selector semantics as `prepare-envs` (ADR-0050): they compose by intersection, and a matrix env's config-declared `default_interpreters` policy (see [Preparing Environments — default interpreter subset](guides/preparing-environments.md#default-interpreter-subset)) applies as the default when neither is given — so a plain `run` can execute only a local subset of a matrix (e.g. the newest interpreter) while CI still runs the full axis, mirroring `prepare-envs`. The selection also decides which interpreter instances are *started*, not only which run: unselected matrix children are never started or repaired.
 
 WAL environment variable and storage settings are shared with `start-wm-server` — see [`start-wm-server`](#start-wm-server) for details.
 

docs/guides/preparing-environments.md

@@ -224,6 +224,15 @@ The ER signals the problem by returning error code `-32001` (`ENV_REINSTALL_NEED
 
 The WM catches this, runs `CreateEnvsAction` + `InstallEnvsAction` for the affected env, then restarts the ER.
 
+### Other triggers
+
+The same install-then-restart repair also runs when a runner the run needs fails to start:
+
+- `NO_VENV`: the venv is missing (or was just wiped as stale/relocated). Repaired wherever the failure surfaces — the run gate, the dispatch start, and metadata resolution.
+- Crash before the port: the ER process exited before publishing its port (`ServerExitedBeforePort` in the failure's `__cause__` chain). Repaired only for an env the run needs — the run gate and the dispatch start — never during metadata resolution, and never for unselected matrix children, which the gate does not start.
+
+Timeouts (the port wait expiring while the process is still alive) are load problems, not broken venvs, and are never repaired. Every repair runs at most once per start attempt: if the restart still fails, the error names the env and the project. Concurrent repairs of the same env are serialized so two callers never install into one venv at the same time.
+
 ### Runner routing
 
 The runner that executes `CreateEnvsAction` / `InstallEnvsAction` during auto-repair depends on which env is being fixed:

docs/guides/wm-server-internals.md

@@ -545,7 +545,10 @@ Triggered when `runner_manager.update_runner_config` receives error `-32001` fro
 sequence as `prepare-envs` — scoped to the affected env — then restarts the ER via
 `restart_extension_runner`.  The only difference from a manual run is which runner executes
 those actions; see [Automatic env repair](../guides/preparing-environments.md#automatic-env-repair)
-for the routing rules.
+for the routing rules. The same repair also covers two startup failures of an env
+the run needs: a missing venv (`NO_VENV`) and an ER that crashed before publishing
+its port. Timeouts and unselected matrix children are never repaired, and each
+repair runs at most once per start attempt.
 
 ---
 

docs/wm-protocol.md

@@ -432,6 +432,15 @@ probes them again. The CLI sets it so payload values with a path type can be
 converted before dispatch; MCP leaves it off so listing tools never starts
 environments.
 
+`runOptions` (optional, honoured only with `startRunners`): the selection
+inputs the run itself will use — `devEnv`, `envSelectors` and
+`interpreterSelectors` (same shapes as the `actions/runBatch` run options).
+The fetch computes the same per-project interpreter selection as the run, so
+it starts only the interpreter instances the run will select. Selectors are
+never validated here: a selector valid in another in-scope project but not in
+the schema project must not fail the fetch; the run's own validation reports
+bad selectors a moment later.
+
 **Result:**
 
 ```json

finecode_jsonrpc/src/finecode_jsonrpc/__init__.py

@@ -5,6 +5,7 @@
     NoResponse,
     RequestCancelledError,
     ResponseTimeout,
+    ServerExitedBeforePort,
     ServerFailedToStart,
     ServerStoppedError,
     StartupTimeline,
@@ -30,6 +31,7 @@
     "NoResponse",
     "RequestCancelledError",
     "ResponseTimeout",
+    "ServerExitedBeforePort",
     "ServerFailedToStart",
     "ServerStdioTransport",
     "ServerStoppedError",

src/finecode/cli_app/commands/run_cmd.py

@@ -377,6 +377,13 @@ async def _attach_session(*, first_connect: bool) -> None:
             schema_project = _choose_schema_project(
                 project_paths, all_actions, action_sources, workdir_path
             )
+            # The schema fetch and the run must select the same interpreter
+            # instances: one dict feeds both, so they cannot drift.
+            selection_options = {
+                "devEnv": dev_env,
+                "envSelectors": env_selectors or [],
+                "interpreterSelectors": interpreter_selectors or [],
+            }
             action_payload = await _resolve_payload(
                 client=client,
                 action_payload=action_payload,
@@ -385,6 +392,7 @@ async def _attach_session(*, first_connect: bool) -> None:
                 schema_project=schema_project,
                 base_dir=workdir_path,
                 map_payload_fields=map_payload_fields,
+                run_options=selection_options,
             )
 
             # Workspace-scoped actions run once on the root project and stream all
@@ -420,16 +428,14 @@ async def _attach_session(*, first_connect: bool) -> None:
                 "concurrently": concurrently,
                 "resultFormats": result_formats,
                 "trigger": "user",
-                "devEnv": dev_env,
+                **selection_options,
                 # Ask the WM to type-safely merge streamed partials per project/action
                 # and return the merged result, so the returned/saved data is complete
                 # even when one project streams many partials.
                 "mergeResults": True,
                 # PRD-0003 AC8: WM-only selectors restricting a matrixed
                 # action's fan-out to a subset of its declared interpreter axis.
                 # Never forwarded to an ER.
-                "envSelectors": env_selectors or [],
-                "interpreterSelectors": interpreter_selectors or [],
             }
 
             partial_result_token = str(uuid.uuid4())
@@ -619,6 +625,7 @@ async def _resolve_payload(
     schema_project: str,
     base_dir: pathlib.Path,
     map_payload_fields: set[str] | None,
+    run_options: dict[str, typing.Any] | None = None,
 ) -> dict[str, typing.Any]:
     """Build the final payload from the raw CLI strings and the payload schemas.
 
@@ -643,7 +650,7 @@ async def _resolve_payload(
     # action, so a path value can be converted before dispatch.
     try:
         schemas = await client.get_payload_schemas(
-            schema_project, action_sources, start_runners=True
+            schema_project, action_sources, start_runners=True, run_options=run_options
         )
     except ApiError as exc:
         raise RunFailed(

src/finecode/wm_client.py

@@ -16,6 +16,7 @@
 import json
 import pathlib
 import random
+import typing
 
 from loguru import logger
 
@@ -447,6 +448,7 @@ async def get_payload_schemas(
         action_sources: list[str],
         *,
         start_runners: bool = False,
+        run_options: dict[str, typing.Any] | None = None,
     ) -> dict[str, schema_utils.PayloadSchema | None]:
         """Return payload schemas for the given actions in a project.
 
@@ -459,6 +461,11 @@ async def get_payload_schemas(
                 environments before probing so a schema that needs one is
                 available. Defaults to false so passive listing never starts
                 environments.
+            run_options: Selection inputs honoured only with
+                ``start_runners``: ``devEnv``, ``envSelectors`` and
+                ``interpreterSelectors`` — the same values the run itself
+                will use, so the fetch starts only the interpreter
+                instances the run will select.
 
         Returns:
             Mapping of action source → JSON Schema fragment, or ``None``
@@ -467,6 +474,8 @@ async def get_payload_schemas(
         params: dict = {"project": project, "actionSources": action_sources}
         if start_runners:
             params["startRunners"] = True
+            if run_options is not None:
+                params["runOptions"] = run_options
         result = await self.request(
             "actions/getPayloadSchemas",
             params,

src/finecode/wm_server/_api_handlers/_actions.py

@@ -39,19 +39,11 @@ async def _handle_run_action(
         parsed = await _parse_and_validate_run_action_params(raw_params, ws_context)
 
         try:
-            # Start required environments so that canonical_source is populated for all
-            # importable action classes (ADR-0021: canonical_source is the WM's internal
-            # identifier after runner initialization; only find_action_by_source at the
-            # external API boundary may match by config source alias).
-            await run_service.start_required_environments(
-                {parsed.project.dir_path: [parsed.action.name]},
-                ws_context,
-                initialize_all_handlers=True,
-            )
             # PRD-0003 AC8: resolve `--env`/`--interpreter` selectors
             # (+ config default) for this project, to restrict a matrixed
             # action's fan-out. `None` (no selectors, no narrowing default)
-            # runs the full declared axis, unchanged.
+            # runs the full declared axis, unchanged. Resolved before the
+            # gate so the gate starts only the children the dispatch runs.
             run_selection.validate_run_selectors(
                 parsed.options.get("envSelectors", []),
                 parsed.options.get("interpreterSelectors", []),
@@ -65,6 +57,18 @@ async def _handle_run_action(
                 parsed.dev_env.value,
                 ws_context,
             )
+            # Start required environments so that canonical_source is populated for all
+            # importable action classes (ADR-0021: canonical_source is the WM's internal
+            # identifier after runner initialization; only find_action_by_source at the
+            # external API boundary may match by config source alias).
+            await run_service.start_required_environments(
+                {parsed.project.dir_path: [parsed.action.name]},
+                ws_context,
+                initialize_all_handlers=True,
+                selected_interpreters_by_project={
+                    parsed.project.dir_path: selected_interpreters
+                },
+            )
             executor = run_service.ProjectExecutor(ws_context)
             result = await executor.run_action(
                 action_source=parsed.action.canonical_source,
@@ -428,12 +432,36 @@ async def _probe_handler_envs(names: list[str]) -> None:
         # startup tool listing never starts every handler env.
         start_runners = params.get("startRunners", False)
         if start_runners and still_missing:
+            from finecode.wm_server.config import env_selection
+            from finecode.wm_server.config.interpreter_matrix import (
+                InvalidInterpreterError,
+            )
             from finecode.wm_server.services import run_service
+            from finecode.wm_server.services.run_service import run_selection
 
-            await run_service.start_required_environments(
-                {project.dir_path: still_missing}, ws_context
-            )
-            await _probe_handler_envs(still_missing)
+            run_options = params.get("runOptions") or {}
+            try:
+                selection = run_selection.selection_for_matrixed_actions(
+                    {project.dir_path: still_missing},
+                    run_options.get("envSelectors", []),
+                    run_options.get("interpreterSelectors", []),
+                    run_options.get("devEnv", "cli"),
+                    ws_context,
+                )
+            except (env_selection.EnvSelectionError, InvalidInterpreterError) as exc:
+                # Never validate selectors here: the run validates against
+                # every in-scope project and tolerates a selector valid in
+                # only one of them, while this fetch sees a single project.
+                # The run's own validation raises the real error a moment
+                # later; the schemas stay None until then.
+                logger.debug(f"Skipping runner start for schema fetch: {exc}")
+            else:
+                await run_service.start_required_environments(
+                    {project.dir_path: still_missing},
+                    ws_context,
+                    selected_interpreters_by_project=selection,
+                )
+                await _probe_handler_envs(still_missing)
 
     # Re-key schemas by the requested source rather than internal action name.
     result_schemas: dict[str, dict | None] = {}

src/finecode/wm_server/_api_handlers/_streaming.py

@@ -341,7 +341,22 @@ async def _handle_run_batch_with_partial_results(
                 f"{sorted({a for names in actions_by_project.values() for a in names})}"
             )
 
-        await run_service.start_required_environments(actions_by_project, ws_context)
+        try:
+            selection_by_project = run_selection.selection_for_matrixed_actions(
+                actions_by_project,
+                parsed.env_selectors,
+                parsed.interpreter_selectors,
+                parsed.dev_env.value,
+                ws_context,
+            )
+        except env_selection.EnvSelectionError as exc:
+            raise ActionRunFailed(str(exc)) from exc
+
+        await run_service.start_required_environments(
+            actions_by_project,
+            ws_context,
+            selected_interpreters_by_project=selection_by_project,
+        )
 
         payload_overrides = parsed.params_by_project or {}
         # Lock to prevent concurrent writes to the shared writer from project tasks.
@@ -377,18 +392,7 @@ async def _stream_action(
                 else None
             )
             if action_def is not None and matrix_runner.is_matrixed(action_def):
-                try:
-                    selected_interpreters = (
-                        run_selection.selected_interpreters_for_project(
-                            project_path,
-                            parsed.env_selectors,
-                            parsed.interpreter_selectors,
-                            parsed.dev_env.value,
-                            ws_context,
-                        )
-                    )
-                except env_selection.EnvSelectionError as exc:
-                    raise ActionRunFailed(str(exc)) from exc
+                selected_interpreters = selection_by_project.get(project_path)
 
                 async def _on_partial(
                     interpreter_canonical: str, result_by_format: dict

src/finecode/wm_server/context.py

@@ -210,6 +210,15 @@ class WorkspaceContext:
     # from inside that same Phase 2.
     env_install_locks: dict[Path, asyncio.Lock] = field(default_factory=dict)
 
+    # Per-env repair locks.  Guard per-env repair (install + restart) so two
+    # concurrent repairs of the same (project, env) never install into one
+    # venv at the same time.  Keyed per env and separate from
+    # env_install_locks: install_env_for_project takes env_install_locks[project]
+    # inside the repair, so sharing that dict would self-deadlock.
+    env_repair_locks: dict[tuple[Path, str], asyncio.Lock] = field(
+        default_factory=dict
+    )
+
     # Both budgets below are sized from ONE combined machine budget in
     # __post_init__ (ADR-0093): their sum stays at or below
     # machine_subprocess_budget(), so the WM's event loop keeps a free core.