Skip to content

perf(daemon): add constrained ARM edge runtime - #999

Closed
aaf2tbz wants to merge 1 commit into
mainfrom
aaf2tbz/perf-921-edge-runtime
Closed

perf(daemon): add constrained ARM edge runtime#999
aaf2tbz wants to merge 1 commit into
mainfrom
aaf2tbz/perf-921-edge-runtime

Conversation

@aaf2tbz

@aaf2tbz aaf2tbz commented Jul 23, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a documented source/Bun edge runtime profile for constrained 64-bit ARM devices, with Raspberry Pi 3B+ as the minimum target. The profile removes eager idle work, switches the built-in native embedding default to MiniLM/384d, honors configured native model settings, and process-isolates the native runtime so its RSS is reclaimed after 30 seconds idle.

Closes #921.

Changes

  • Add an approved edge-runtime spec, dependency index entry, operator guide, and Linux /proc RSS/query-embedding gate.
  • Default native embeddings to Xenova/all-MiniLM-L6-v2/384d and migrate only the exact legacy native default; custom models/providers remain unchanged.
  • Propagate validated native model/dimension config into the worker and terminate an edge-only child process after the idle window.
  • Keep the edge daemon lazy at idle: fixed-file polling, no startup embedding probe, no pre-spawned synthesis/extraction helper, lazy tokenizer/provider catalogs, and lazy UMAP/Louvain routes.
  • Centralize embedding defaults across core, daemon, setup/configure flows, templates, and docs.

Type

  • feat — new user-facing feature (bumps minor)
  • fix — bug fix
  • refactor — restructure without behavior change
  • chore — build, deps, config, docs
  • perf — performance improvement
  • test — test coverage

Packages affected

  • @signet/core
  • @signet/daemon
  • @signet/cli / dashboard
  • @signet/sdk
  • @signet/connector-*
  • @signet/web
  • predictor
  • Other: docs and edge verification script

Screenshots

N/A — no UI changes.

PR Readiness (MANDATORY)

  • Spec alignment validated (INDEX.md + dependencies.yaml)
  • Agent scoping verified on all new/changed data queries
  • Input/config validation and bounds checks added
  • Error handling and fallback paths tested (no silent swallow)
  • Security checks applied to admin/mutation endpoints
  • Docs updated for API/spec/status changes
  • Regression tests added for each bug fix
  • Lint/typecheck/tests pass locally

Readiness notes:

  • bun scripts/spec-deps-check.ts passes with 87 indexed specs.
  • No new data query or mutation/admin endpoint was added. Existing memory-route scope/auth behavior is unchanged and its isolated scope/auth suites pass.
  • Runtime profile values, embedding dimensions (1–65,536), and model identifier length are bounded. Invalid values fall back with diagnostics.
  • Changed-file validation is green. Repository-wide lint/typecheck/test failures are existing origin/main baseline failures, detailed below.

Migration Notes (if applicable)

  • Migration is idempotent
  • Daemon Rust parity reviewed or explicitly N/A
  • Rollback / compatibility note included in PR description

Config migration v5 is idempotent and changes only native/local configs using the exact legacy Nomic default with absent/768 dimensions. Custom native models/dimensions and external providers are preserved. Source memories are not deleted; the existing tracker re-embeds model/dimension mismatches. Rust parity is N/A because no route surface was added or removed. Rolling back retains valid 384d config/data; operators may explicitly restore the old model/dimensions if desired.

Testing

  • bun test passes
  • bun run typecheck passes
  • bun run lint passes
  • Tested against running daemon
  • N/A

Validation:

  • 157/157 focused tests pass across runtime profile, polling, native embeddings, process selection/idle shutdown, migration, config, tracker, OAuth/router, and tokenizer budgeting.
  • All other adjacent suites pass when run one file per process: daemon auth/CORS/status/refactor, config migration, memory lineage/routes/scoping/curator, skill reconciler, and CLI setup validation/pipeline/plugins/protection/shared.
  • The native event-loop isolation E2E passes in the repository-wide run.
  • Core, daemon, CLI, and self-contained native Bun builds pass.
  • Source and built edge daemons were exercised with the real cached MiniLM model: 384d status succeeds, the daemon remains responsive, the embedding child exits after 30 seconds idle, and no child leaks remain after shutdown.
  • Full bun test: 3,301 pass / 47 skip. The monolithic run has existing cross-suite/global-state and fixture failures. The two isolated adjacent failures reproduce exactly on untouched origin/main: inference-api.test.ts is 14 pass / 12 fail and setup.test.ts is 17 pass / 1 fail.
  • Full daemon typecheck reports exactly the same 421 errors on this branch and untouched origin/main; no new changed-line error is introduced. @signet/core typecheck passes.
  • Changed-file Biome passes with one pre-existing warning. Full lint reports 1,839 errors here versus 1,840 on untouched origin/main.

AI disclosure

  • No AI tools were used in this PR
  • AI tools were used (see Assisted-by tags in commits)

Notes

This is intentionally a draft until the documented release gate is run on physical Raspberry Pi 3B+ hardware. macOS development measurements validate lazy loading and process reclamation but do not substitute for the required Linux ARM result: idle daemon RSS below 100 MiB and a warm query embedding below five seconds.

The constrained path intentionally uses the source/Bun daemon. The self-contained native binary remains the standard portable/offline distribution and continues to embed its release assets.

@aaf2tbz
aaf2tbz marked this pull request as ready for review July 23, 2026 07:07
@aaf2tbz aaf2tbz added enhancement New feature or request priority: P1 High priority bucket: ops-hardening Operational reliability, cost, and safety hardening and removed enhancement New feature or request labels Jul 23, 2026
@aaf2tbz aaf2tbz self-assigned this Jul 24, 2026
Assisted-by: Codex:gpt-5
@aaf2tbz
aaf2tbz force-pushed the aaf2tbz/perf-921-edge-runtime branch from c870976 to abe90db Compare July 28, 2026 03:15
@NicholaiVogel NicholaiVogel added blast: subsystem A subsystem, surface, or single integration component: daemon Daemon API, runtime, pipeline, auth, and workers component: native Rust/native runtime and packaging parity risk: config-compatibility May break existing configuration, migrations, defaults, or upgrades risk: performance May cause event-loop stalls, unbounded work, memory growth, or cost regressions labels Aug 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

blast: subsystem A subsystem, surface, or single integration bucket: ops-hardening Operational reliability, cost, and safety hardening component: daemon Daemon API, runtime, pipeline, auth, and workers component: native Rust/native runtime and packaging parity priority: P1 High priority risk: config-compatibility May break existing configuration, migrations, defaults, or upgrades risk: performance May cause event-loop stalls, unbounded work, memory growth, or cost regressions

Projects

None yet

Development

Successfully merging this pull request may close these issues.

perf(daemon): target Raspberry Pi 3B+ as minimum platform; reduce idle RSS to <100 MB

2 participants