From 4a5af47fda3f36556fa7029149bedf0d0078fa5f Mon Sep 17 00:00:00 2001 From: SheeshDarth Date: Thu, 16 Jul 2026 21:48:39 +0530 Subject: [PATCH 1/2] =?UTF-8?q?feat(dist):=20ship=20an=20installable=20pac?= =?UTF-8?q?kage=20=E2=80=94=20npx=20nirmiqcodesensei=20(MS7)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Turns the repo into distributable software. The old launcher assumed a repo checkout and ran `next dev`; the package was `private: true` and MS6's standalone build was never shipped. Verified end to end by installing the real tarball into an empty directory: HTTP 200, runtime-migrated DB, MCP handshake. Design: ship the prebuilt standalone server, let npm resolve native modules per platform. scripts/pack-standalone.mjs assembles dist/ (standalone + .next/static), strips the build machine's native binaries, and hard-fails if a database, dotenv, git history or .node binary reaches the bundle. Five bugs found only by installing the tarball, each of which would have shipped a broken or unsafe 1.0: - File tracing copies data/ into .next/standalone — the local database *and* the user's imported projects, including their .git. outputFileTracingExcludes does not stop it (tried; zero effect), so the pack script filters at copy time and asserts. Publishing would have leaked third-party source. - Standalone server.js does process.chdir(__dirname), so the DB resolved to node_modules/nirmiqcodesensei/dist/data/ — every `npx …@latest` would have silently wiped the user's learning history. The launcher now pins NCS_DATA_DIR to the invoking cwd (the hook lib/db/client.ts already had). - server.js defaults HOSTNAME to 0.0.0.0, exposing a 127.0.0.1-only product across the LAN. Launcher pins the loopback. - The tracer copies the repo's .gitignore into the bundle; npm honours nested ignore files, so node_modules/ and .next/ were dropped from the tarball — publish succeeds, every install boots a server with nothing behind it. - Turbopack requires serverExternalPackages by *hashed* name (better-sqlite3-90e2652d…), so deleting it 500s every request. Replaced with a shim delegating to the platform-correct copy npm installs. MCP now works from a published install: bundled with esbuild at prepack (CJS — an ESM bundle crashes on typescript-estree's __filename), and its banner moved to stderr so stdout stays pure JSON-RPC. Also: fileURLToPath for ROOT (%20 broke "C:\Program Files"), shell:false for direct node spawns (space-safe args), and lint ignores dist/ (eslint OOM-crashed walking the 12 MB bundle). Version 1.0.0 across package, MCP serverInfo, manifest. Gate: lint + typecheck + build + 24/24 green. Co-Authored-By: Claude Opus 4.8 --- .gitignore | 6 + .npmignore | 21 ++++ bin/nirmiq.mjs | 120 +++++++++++++----- mcp-manifest.json | 2 +- mcp-server/index.ts | 2 +- next.config.ts | 12 ++ package-lock.json | 8 +- package.json | 28 ++++- scripts/pack-standalone.mjs | 236 ++++++++++++++++++++++++++++++++++++ 9 files changed, 397 insertions(+), 38 deletions(-) create mode 100644 .npmignore create mode 100644 scripts/pack-standalone.mjs diff --git a/.gitignore b/.gitignore index 8f3f7bc..d6b8d49 100644 --- a/.gitignore +++ b/.gitignore @@ -55,5 +55,11 @@ next-env.d.ts # build build/ +# distributable bundle — regenerated by `npm run pack:standalone` (prepack). +# Build output, not source: it is shipped in the npm tarball via package.json +# "files", never committed. +dist/ +*.tgz + # vercel .vercel diff --git a/.npmignore b/.npmignore new file mode 100644 index 0000000..1a2a436 --- /dev/null +++ b/.npmignore @@ -0,0 +1,21 @@ +# npm reads THIS file instead of .gitignore when it exists — which is the whole +# point of it existing. +# +# .gitignore lists `node_modules/` and `.next/` (correct for git). npm applies those +# same rules *inside* dist/, which silently strips the entire Next runtime and the +# compiled app out of the tarball and ships a dist/server.js that cannot boot. The +# publish succeeds; every install fails. This file stops that fallback. +# +# The real allowlist is package.json "files" (bin/, dist/, and the docs). Anything +# not listed there never ships regardless of what appears below. +# +# Defence in depth for the things that must never leave this machine: the local +# database and the user's imported source projects. +data/ +*.db +*.db-shm +*.db-wal +*.sqlite +.env +.env.* +*.tgz diff --git a/bin/nirmiq.mjs b/bin/nirmiq.mjs index e9355a9..8894ff9 100644 --- a/bin/nirmiq.mjs +++ b/bin/nirmiq.mjs @@ -3,25 +3,61 @@ * nirmiqcodesensei — NirmiqCodeSensei CLI (aliases: codesensei, nirmiq) * * Usage: - * npx nirmiqcodesensei # start the app (dev mode) - * npx nirmiqcodesensei start # start in production mode + * npx nirmiqcodesensei # start the app + * npx nirmiqcodesensei start # same, explicit * npx nirmiqcodesensei mcp # start the MCP server * npx nirmiqcodesensei open # open the dashboard in the browser * - * The CLI must be run from the NirmiqCodeSensei repo root. + * Two install shapes, detected at runtime: + * - Published package → dist/server.js exists; run the prebuilt standalone server. + * - Repo checkout → no dist/; fall back to `next dev` against the source. + * * All data is stored locally — nothing is sent to any server. */ import { spawn, execSync } from "node:child_process"; import { existsSync, readFileSync, appendFileSync } from "node:fs"; -import { resolve, join } from "node:path"; +import { join } from "node:path"; +import { fileURLToPath } from "node:url"; import { platform } from "node:os"; -const ROOT = new URL("..", import.meta.url).pathname.replace(/^\/([A-Z]:)/, "$1"); +// fileURLToPath, not URL.pathname: the latter leaves paths percent-encoded, so a +// global install under "C:\Program Files\…" would resolve to "…\Program%20Files\…". +const ROOT = fileURLToPath(new URL("..", import.meta.url)); const PKG = JSON.parse(readFileSync(join(ROOT, "package.json"), "utf-8")); -const VERSION = PKG.version ?? "0.1.0"; +const VERSION = PKG.version ?? "0.0.0"; + +// The published server is a prebuilt Next standalone bundle; a repo checkout has none. +const DIST_SERVER = join(ROOT, "dist", "server.js"); +// CJS, not ESM: the bundle pulls in @typescript-eslint/typescript-estree, which +// reads __filename — undefined in an ESM bundle, so an .mjs build crashes on load. +const DIST_MCP = join(ROOT, "dist", "mcp-server.cjs"); +const IS_PACKAGED = existsSync(DIST_SERVER); -const DASHBOARD_URL = "http://127.0.0.1:3000/dashboard"; +const PORT = process.env.PORT ?? "3000"; +const DASHBOARD_URL = `http://127.0.0.1:${PORT}/dashboard`; + +/** + * Env for any spawned server process. + * + * NCS_DATA_DIR is the load-bearing one. Next's standalone server.js does + * `process.chdir(__dirname)` on boot, and lib/db/client.ts resolves the database + * from process.cwd() — so without this, a published install would write the + * user's entire learning history into node_modules/nirmiqcodesensei/dist/data/, + * and the next `npx nirmiqcodesensei@latest` would silently wipe it. Pin the data + * dir to the directory the user actually invoked us from. + * + * HOSTNAME matters too: standalone server.js defaults to 0.0.0.0, which would + * expose the app across the LAN. This product is 127.0.0.1-only by design. + */ +function serverEnv() { + return { + ...process.env, + NCS_DATA_DIR: process.env.NCS_DATA_DIR ?? join(process.cwd(), "data"), + HOSTNAME: "127.0.0.1", + PORT, + }; +} // ── ANSI helpers ─────────────────────────────────────────────────────────────── const C = { @@ -33,13 +69,16 @@ const C = { bold: (s) => `\x1b[1m${s}\x1b[0m`, }; -function banner() { - console.log(""); - console.log(C.cyan(" ╔══════════════════════════════════════╗")); - console.log(C.cyan(" ║") + C.bold(" NirmiqCodeSensei") + C.dim(` v${VERSION}`) + C.cyan(" ║")); - console.log(C.cyan(" ║") + C.dim(" Build with AI, learn like a real engineer") + C.cyan(" ║")); - console.log(C.cyan(" ╚══════════════════════════════════════╝")); - console.log(""); +// `write` is injectable because the MCP transport owns stdout: it speaks +// line-delimited JSON-RPC there, and a banner on the same stream is unparseable +// noise that breaks the handshake. MCP passes console.error. +function banner(write = console.log) { + write(""); + write(C.cyan(" ╔══════════════════════════════════════╗")); + write(C.cyan(" ║") + C.bold(" NirmiqCodeSensei") + C.dim(` v${VERSION}`) + C.cyan(" ║")); + write(C.cyan(" ║") + C.dim(" Build with AI, learn like a real engineer") + C.cyan(" ║")); + write(C.cyan(" ╚══════════════════════════════════════╝")); + write(""); } function openBrowser(url) { @@ -56,11 +95,16 @@ function openBrowser(url) { } } -function run(command, args, cwd = ROOT) { +// shell defaults to true because `npx` on Windows is a .cmd shim and won't spawn +// otherwise. Pass shell:false when invoking node directly — a shell concatenates +// args instead of escaping them, so an install path containing a space +// ("C:\Program Files\…") would be split into two broken arguments. +function run(command, args, cwd = ROOT, env = process.env, shell = true) { const proc = spawn(command, args, { cwd, + env, stdio: "inherit", - shell: true, + shell, }); proc.on("error", (e) => { console.error(C.red(`Failed to start: ${e.message}`)); @@ -92,42 +136,58 @@ switch (cmd) { case "start": { banner(); ensureGitignore(process.cwd()); - const mode = cmd === "start" ? "start" : "dev"; - const modeArgs = - mode === "dev" - ? ["dev", "--turbopack", "--hostname", "127.0.0.1"] - : ["start", "--hostname", "127.0.0.1"]; + const env = serverEnv(); console.log( - C.cyan(` Starting NirmiqCodeSensei`) + - C.dim(` (${mode} mode) …`) + C.cyan(" Starting NirmiqCodeSensei") + + C.dim(IS_PACKAGED ? " …" : " (dev mode — repo checkout) …") ); console.log(C.dim(` Dashboard → ${DASHBOARD_URL}`)); + console.log(C.dim(` Data → ${env.NCS_DATA_DIR}`)); console.log(C.dim(` Privacy → All data stays local. Zero telemetry.`)); console.log(""); // Open browser after a short delay to let the server start setTimeout(() => openBrowser(DASHBOARD_URL), 3000); - run("npx", ["next", ...modeArgs], ROOT); + if (IS_PACKAGED) { + // Prebuilt standalone server — no Next CLI, no devDependencies needed. + run(process.execPath, [DIST_SERVER], ROOT, env, false); + } else { + // Repo checkout: `next dev` compiles from source. `start` would need a + // prior `npm run build`, so dev is the sane default for contributors. + const modeArgs = + cmd === "start" + ? ["start", "--hostname", "127.0.0.1"] + : ["dev", "--turbopack", "--hostname", "127.0.0.1"]; + run("npx", ["next", ...modeArgs], ROOT, env); + } break; } case "mcp": { - banner(); - console.log(C.cyan(" Starting NirmiqCodeSensei MCP server…")); - console.log( + // Everything human-readable goes to stderr: stdout is the JSON-RPC transport. + banner(console.error); + console.error(C.cyan(" Starting NirmiqCodeSensei MCP server…")); + console.error( C.dim( " Connect this to Claude Code / Cursor / Windsurf via their MCP config." ) ); - console.log( + console.error( C.dim( ` Transport: stdio | Config: { "command": "npx", "args": ["nirmiqcodesensei", "mcp"] }` ) ); - console.log(""); - run("npx", ["tsx", join(ROOT, "mcp-server", "index.ts")]); + console.error(""); + // Packaged: a single prebuilt CJS bundle (esbuild, at prepack time) — tsx is a + // devDependency and does not exist in a published install. + // Checkout: run the TypeScript source through tsx. + if (existsSync(DIST_MCP)) { + run(process.execPath, [DIST_MCP], ROOT, serverEnv(), false); + } else { + run("npx", ["tsx", join(ROOT, "mcp-server", "index.ts")], ROOT, serverEnv()); + } break; } diff --git a/mcp-manifest.json b/mcp-manifest.json index 9233b09..7cc2ff3 100644 --- a/mcp-manifest.json +++ b/mcp-manifest.json @@ -2,7 +2,7 @@ "name": "nirmiqcodesensei", "display_name": "NirmiqCodeSensei", "description": "A local-first learning OS that connects to your IDE. Log debug sessions, generate explain-back questions, map DSA concepts, and track daily learning — all inside Claude Code, Cursor, or Windsurf. Your data never leaves your machine.", - "version": "0.1.0", + "version": "1.0.0", "author": "Siddharth Prasad", "license": "PolyForm-Noncommercial-1.0.0", "homepage": "https://github.com/SheeshDarth/NirmiqCodeSensei", diff --git a/mcp-server/index.ts b/mcp-server/index.ts index adbbc94..2dc7b3e 100644 --- a/mcp-server/index.ts +++ b/mcp-server/index.ts @@ -37,7 +37,7 @@ import { createSessionLog } from "../lib/services/session-log.service"; // ── Server setup ─────────────────────────────────────────────────────────────── const server = new Server( - { name: "nirmiqcodesensei", version: "0.1.0" }, + { name: "nirmiqcodesensei", version: "1.0.0" }, { capabilities: { tools: {} }, } diff --git a/next.config.ts b/next.config.ts index 38b926a..0d3879f 100644 --- a/next.config.ts +++ b/next.config.ts @@ -46,6 +46,18 @@ const nextConfig: NextConfig = { "/**": ["./lib/db/migrations/**/*"], }, + // NOTE: tracing sweeps data/ into .next/standalone — the local database *and* + // the user's imported source projects (their code, their .git history). The + // cause is lib/db/client.ts resolving path.join(process.cwd(), "data"): the + // tracer sees a literal directory and pulls the whole thing in. + // + // outputFileTracingExcludes does NOT stop this — it was tried here for + // data/docs/tests/scripts and had zero effect on the standalone copy (Next + // 16.2.7). Do not re-add it expecting protection. The real guard is + // scripts/pack-standalone.mjs, which strips these from dist/ and then hard-fails + // if any database, dotenv or native binary survives into the publishable bundle. + // package.json "files" is the second layer: it allowlists dist/ and bin/ only. + // better-sqlite3 is a native addon — it must be require()'d at runtime, not // bundled. Without this, Turbopack tries to bundle it into its render/ // static-path worker processes, which crashes them (WorkerError) on the diff --git a/package-lock.json b/package-lock.json index 3dc42db..f44b5fc 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "nirmiqcodesensei", - "version": "0.1.0", + "version": "1.0.0", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "nirmiqcodesensei", - "version": "0.1.0", + "version": "1.0.0", "license": "PolyForm-Noncommercial-1.0.0", "dependencies": { "@anthropic-ai/sdk": "^0.101.0", @@ -37,6 +37,7 @@ "@types/react": "^19", "@types/react-dom": "^19", "drizzle-kit": "^0.31.10", + "esbuild": "^0.28.1", "eslint": "^9", "eslint-config-next": "16.2.7", "postcss": "^8.4.31", @@ -46,6 +47,9 @@ }, "engines": { "node": ">=20" + }, + "optionalDependencies": { + "sharp": "^0.34.0" } }, "node_modules/@alloc/quick-lru": { diff --git a/package.json b/package.json index 920a744..c8d8f53 100644 --- a/package.json +++ b/package.json @@ -1,11 +1,24 @@ { "name": "nirmiqcodesensei", - "version": "0.1.0", - "private": true, + "version": "1.0.0", + "description": "Local-first learning OS: import a project, get a learning map, an 8-lens senior review, code-grounded DSA, and explain-back questions. 127.0.0.1 only, no telemetry.", + "keywords": ["learning", "code-analysis", "mcp", "local-first", "education", "static-analysis"], + "homepage": "https://github.com/SheeshDarth/NirmiqCodeSensei#readme", + "repository": { "type": "git", "url": "git+https://github.com/SheeshDarth/NirmiqCodeSensei.git" }, + "bugs": { "url": "https://github.com/SheeshDarth/NirmiqCodeSensei/issues" }, "license": "PolyForm-Noncommercial-1.0.0", "engines": { "node": ">=20" }, + "files": [ + "bin/", + "dist/", + "CHANGELOG.md", + "README.md", + "LICENSE.md", + "SECURITY.md", + "mcp-manifest.json" + ], "bin": { "codesensei": "./bin/nirmiq.mjs", "nirmiqcodesensei": "./bin/nirmiq.mjs", @@ -15,7 +28,10 @@ "dev": "next dev --turbopack --hostname 127.0.0.1", "build": "next build", "start": "next start --hostname 127.0.0.1", - "lint": "eslint .", + "pack:standalone": "node scripts/pack-standalone.mjs", + "build:mcp": "esbuild mcp-server/index.ts --bundle --platform=node --format=cjs --target=node20 --outfile=dist/mcp-server.cjs --external:better-sqlite3 --tsconfig=tsconfig.json", + "prepack": "npm run build && npm run pack:standalone && npm run build:mcp", + "lint": "eslint . --ignore-pattern dist/ --ignore-pattern data/", "typecheck": "tsc --noEmit", "test": "tsx --test tests/import-pipeline.test.mts", "db:generate": "drizzle-kit generate", @@ -38,8 +54,11 @@ "three": "^0.184.0", "zod": "^4.4.3" }, + "optionalDependencies": { + "sharp": "^0.34.0" + }, "overrides": { - "esbuild": ">=0.25.0" + "esbuild": "$esbuild" }, "devDependencies": { "@eslint/eslintrc": "^3", @@ -49,6 +68,7 @@ "@types/react": "^19", "@types/react-dom": "^19", "drizzle-kit": "^0.31.10", + "esbuild": "^0.28.1", "eslint": "^9", "eslint-config-next": "16.2.7", "postcss": "^8.4.31", diff --git a/scripts/pack-standalone.mjs b/scripts/pack-standalone.mjs new file mode 100644 index 0000000..143e7b8 --- /dev/null +++ b/scripts/pack-standalone.mjs @@ -0,0 +1,236 @@ +#!/usr/bin/env node +/** + * pack-standalone — assemble the distributable server into dist/. + * + * Runs from `prepack`, so `npm pack` / `npm publish` always ship a fresh build. + * + * Next's standalone output is deliberately incomplete: it emits server.js and the + * traced runtime deps, but leaves out .next/static and public/ — they must be + * copied alongside it or every asset 404s. + * + * It also traces the *build* machine's native binaries (better-sqlite3, sharp). + * Those are platform-specific, so a Windows-built tarball would hard-fail on + * Linux. We strip them here and declare better-sqlite3 a real dependency instead: + * Node resolves upward from dist/server.js and finds the copy npm installed for + * the *install* platform. One tarball, every OS. + */ + +import { + cpSync, + rmSync, + existsSync, + mkdirSync, + readdirSync, + statSync, + writeFileSync, +} from "node:fs"; +import { join, relative, sep } from "node:path"; +import { fileURLToPath } from "node:url"; + +const ROOT = fileURLToPath(new URL("..", import.meta.url)); +const STANDALONE = join(ROOT, ".next", "standalone"); +const DIST = join(ROOT, "dist"); + +const log = (msg) => console.log(` ${msg}`); +const fail = (msg) => { + console.error(`\n \x1b[31mpack-standalone failed:\x1b[0m ${msg}\n`); + process.exit(1); +}; + +// ── Preconditions ───────────────────────────────────────────────────────────── +if (!existsSync(join(STANDALONE, "server.js"))) { + fail( + `.next/standalone/server.js not found — run \`npm run build\` first.\n` + + ` (next.config.ts must keep output: "standalone".)` + ); +} + +// ── Assemble ────────────────────────────────────────────────────────────────── +rmSync(DIST, { recursive: true, force: true }); +mkdirSync(DIST, { recursive: true }); + +// File tracing sweeps the whole project root into .next/standalone — the database, +// the user's imported projects, and the repo's own .git and .gitignore. Filter at +// copy time rather than copying then deleting: it avoids duplicating ~1300 .git +// files and a multi-MB database only to remove them, and avoids Windows' +// delete-pending semantics, where a directory stays enumerable after rmSync returns. +// +// Two entries are load-bearing, not tidiness: +// .gitignore — npm honours ignore files nested inside the packed directory. This +// copy lists `node_modules/` and `.next/`, so shipping it makes npm silently +// drop the Next runtime and compiled app: publish succeeds, every install +// boots a server.js with nothing behind it. +// .git — the repository's full history. +// dist — our own output from a previous run. `next build` traces it back into +// .next/standalone, so without this the bundle nests dist/dist/… and each +// release drags in the last one (including the .git copy it used to contain). +const SKIP_TOP_LEVEL = new Set([ + "data", + "docs", + "tests", + "scripts", + "graphify-out", + "dist", + ".claude", + ".github", + ".git", + ".gitignore", + ".npmignore", +]); + +const skipped = new Set(); +cpSync(STANDALONE, DIST, { + recursive: true, + // Next symlinks serverExternalPackages into .next/node_modules. Recreating a + // symlink on Windows needs elevation (EPERM without it), and a tarball can't + // carry links out to the build machine's node_modules anyway — copy the real + // files so the bundle stands alone. + dereference: true, + filter: (src) => { + const rel = relative(STANDALONE, src); + if (rel === "") return true; + const top = rel.split(sep)[0]; + if (SKIP_TOP_LEVEL.has(top)) { + skipped.add(top); + return false; + } + return true; + }, +}); +log("copied .next/standalone → dist/"); +if (skipped.size > 0) log(`excluded non-runtime paths: ${[...skipped].sort().join(", ")}`); + +// Static assets: tracing never includes these — without them the app renders unstyled. +const staticSrc = join(ROOT, ".next", "static"); +if (!existsSync(staticSrc)) fail(".next/static missing — the build did not complete."); +cpSync(staticSrc, join(DIST, ".next", "static"), { recursive: true }); +log("copied .next/static → dist/.next/static"); + +const publicSrc = join(ROOT, "public"); +if (existsSync(publicSrc)) { + cpSync(publicSrc, join(DIST, "public"), { recursive: true }); + log("copied public/ → dist/public"); +} + +// ── Strip platform-specific native modules ──────────────────────────────────── +// better-sqlite3 → resolved from the install's own node_modules (a real dependency). +// sharp → optionalDependency; Next falls back to an unoptimised image path without it. +// +// These live in two places. Plain copies land in node_modules/, but anything +// in serverExternalPackages is *also* copied to .next/node_modules/- +// (e.g. better-sqlite3-90e2652d1716b047). Only the .nft.json trace manifests refer +// to the hashed path — runtime require() resolves normally — so both can go. +const NATIVE = ["better-sqlite3", "sharp", "@img"]; + +const stripNativeFrom = (modulesDir, label) => { + if (!existsSync(modulesDir)) return; + for (const entry of readdirSync(modulesDir, { withFileTypes: true })) { + const isNative = NATIVE.some( + (n) => entry.name === n || entry.name.startsWith(`${n}-`) + ); + if (!isNative) continue; + rmSync(join(modulesDir, entry.name), { recursive: true, force: true }); + log(`stripped native module: ${label}/${entry.name}`); + } +}; + +stripNativeFrom(join(DIST, "node_modules"), "node_modules"); + +// .next/node_modules is different: it cannot simply be emptied. Turbopack compiles +// serverExternalPackages to a require() of the *hashed* directory name — the chunk +// literally calls require("better-sqlite3-90e2652d1716b047") — so deleting it makes +// the server boot and then fail every request with "Failed to load external module". +// +// Keep the module requirable, but strip its platform-locked payload and delegate to +// the copy npm installed for the *install* platform. Node resolves the bare specifier +// upward from here and finds the real better-sqlite3. That is what lets one tarball +// serve Windows, Linux and macOS. +// +// Rewrite in place rather than remove-and-recreate: on Windows a deleted directory +// stays enumerable until its handles close, so recreating the same path in the same +// process is unreliable. +const shimNativeIn = (modulesDir, label) => { + if (!existsSync(modulesDir)) return; + for (const entry of readdirSync(modulesDir, { withFileTypes: true })) { + const real = NATIVE.find((n) => entry.name === n || entry.name.startsWith(`${n}-`)); + if (!real) continue; + + const dir = join(modulesDir, entry.name); + for (const payload of readdirSync(dir)) { + rmSync(join(dir, payload), { recursive: true, force: true }); + } + writeFileSync( + join(dir, "package.json"), + `${JSON.stringify({ name: entry.name, version: "0.0.0", main: "index.js" }, null, 2)}\n` + ); + writeFileSync( + join(dir, "index.js"), + `// Generated by scripts/pack-standalone.mjs.\n` + + `// Turbopack requires this package by its hashed name; the real, platform-correct\n` + + `// ${real} is the one npm installed alongside this package.\n` + + `module.exports = require(${JSON.stringify(real)});\n` + ); + log(`shimmed native module → ${label}/${entry.name} → require("${real}")`); + } +}; + +shimNativeIn(join(DIST, ".next", "node_modules"), ".next/node_modules"); + +// ── Assert the bundle is safe to publish ────────────────────────────────────── +// This is not paranoia: file tracing really does copy data/ (database + the user's +// imported project trees) into .next/standalone, and outputFileTracingExcludes does +// not stop it — see the note in next.config.ts. The strip above is the fix; this is +// the proof. Last gate before a tarball goes out, so it fails the build rather than +// warning. +const FORBIDDEN = [ + { test: (p) => p.split(sep)[0] === "data", why: "local database / imported user projects" }, + { test: (p) => /\.db(-wal|-shm)?$/.test(p), why: "SQLite database" }, + { test: (p) => p.split(sep).some((s) => s.startsWith(".env")), why: "environment file" }, + { test: (p) => p.endsWith(".node"), why: "platform-specific native binary" }, + { test: (p) => p.split(sep).includes(".git"), why: "git history" }, + // Only a dist-root ignore file is dangerous — it is the one npm applies to the + // whole tarball. The .gitignore files individual npm packages ship inside + // node_modules are inert here, so don't fail on those. + { test: (p) => p === ".gitignore" || p === ".npmignore", why: "root ignore file — would gut the tarball" }, +]; + +const offenders = []; +const walk = (dir) => { + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const full = join(dir, entry.name); + if (entry.isDirectory()) { + walk(full); + continue; + } + const rel = relative(DIST, full); + for (const rule of FORBIDDEN) { + if (rule.test(rel)) offenders.push(`${rel} (${rule.why})`); + } + } +}; +walk(DIST); + +if (offenders.length > 0) { + fail( + `dist/ contains ${offenders.length} file(s) that must never be published:\n` + + offenders.slice(0, 20).map((o) => ` - ${o}`).join("\n") + + (offenders.length > 20 ? `\n … and ${offenders.length - 20} more` : "") + ); +} + +// ── Report ──────────────────────────────────────────────────────────────────── +let bytes = 0; +let files = 0; +const measure = (dir) => { + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const full = join(dir, entry.name); + if (entry.isDirectory()) measure(full); + else { + bytes += statSync(full).size; + files += 1; + } + } +}; +measure(DIST); + +log(`\x1b[32m✓\x1b[0m dist/ ready — ${files} files, ${(bytes / 1024 / 1024).toFixed(1)} MB, no forbidden content`); From a7df2dd7933f4d6a8f10f3963cb65433b4900cfd Mon Sep 17 00:00:00 2001 From: SheeshDarth Date: Thu, 16 Jul 2026 21:56:02 +0530 Subject: [PATCH 2/2] docs: CHANGELOG, scaling-N/A ADR (REVIEW-013), MS7 record MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Completes MS7's documentation criteria and closes out the REVIEW-011 megasprint program. - CHANGELOG.md (Keep a Changelog) — first release entry for 1.0.0, leading with `npx nirmiqcodesensei@latest` and naming the distribution bugs that would otherwise have shipped. - REVIEW-013 — the scaling-N/A ADR REVIEW-011 asked for: load balancing, horizontal scaling and traffic handling are out of scope *by design*, not deferred. A single-user, loopback-bound tool over a single-writer embedded SQLite database has no traffic to distribute and no replicas to balance; a second instance would contend for the WAL lock or fragment the user's history. Scaling here means per-machine analysis performance, which is measured (BENCHMARKS.md). Revisit only on a move to a hosted model — and re-architect the data tier first, which is the actual blocker. - README leads with the npx path; MCP config uses `npx nirmiqcodesensei mcp` (the old `npm run mcp` + cwd form only works from a clone). - Roadmap MS7 → ✅; REVIEW-011 → complete. - PROJECT_DOSSIER: MS7 outcome, status, and a new §7 subsection on the distribution class of bugs — none were visible to lint, typecheck, build or 24 passing tests; all five surfaced only from installing the real tarball. That is now the recorded verification step for packaging changes. Gate: lint + typecheck + build + 24/24 green. Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 79 ++++++++++++++++++++++++++++++++++++++ README.md | 46 +++++++++++++--------- docs/COUNCIL_REVIEW_LOG.md | 28 +++++++++++++- docs/MEGASPRINT_ROADMAP.md | 2 +- docs/PROJECT_DOSSIER.md | 53 +++++++++++++++++++------ 5 files changed, 176 insertions(+), 32 deletions(-) create mode 100644 CHANGELOG.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..d9b9090 --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,79 @@ +# Changelog + +All notable changes to NirmiqCodeSensei are documented here. + +Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). +Versioning: [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +--- + +## [1.0.0] — 2026-07-16 + +First public release. NirmiqCodeSensei is a local-first learning OS: point it at a project +and it produces a learning map, an 8-lens senior-engineer review, a code-grounded DSA +breakdown, and explain-back questions. It binds to 127.0.0.1, stores everything in a local +SQLite database, and sends no telemetry. + +```bash +npx nirmiqcodesensei@latest +``` + +### Added + +- **Installable distribution.** `npx nirmiqcodesensei` runs the app on a machine that has + never seen the repo. The published package carries a prebuilt Next.js standalone server; + native modules resolve per-platform at install time, so one tarball serves Windows, Linux + and macOS. Aliases: `codesensei`, `nirmiq`. +- **MCP server in the published package** (`npx nirmiqcodesensei mcp`) — 12 tools over stdio + for Claude Code, Cursor and Windsurf. Bundled at pack time, so no dev toolchain is needed. +- **Import & analysis.** Local path or GitHub URL → learning map, architecture graph, + code-grounded DSA concepts, explain-back questions. Offline by default; an optional + `ANTHROPIC_API_KEY` enriches the analysis and sends only computed findings, never source. +- **8-lens senior review** with an overall grade, scored on code *density* rather than project + size, plus an incremental re-analysis path that skips unchanged sources. +- **Learning surfaces:** knowledge graph, explain-back with confidence tracking, DSA bridge, + debug lab, daily log, session log, BM25 search, and Markdown export of the whole pipeline. +- **Data ownership:** the database lives in `data/` in the directory you launch from — your + project, not the install — and can be downloaded as a backup at any time. +- `CHANGELOG.md`, and a scaling-N/A architecture decision (REVIEW-013) recording that + horizontal scaling is a deliberate non-goal for a single-user local-first tool. + +### Fixed + +Found by installing the real tarball into an empty directory — each would have shipped a +broken or unsafe 1.0: + +- Next.js file tracing copied `data/` into the build output — the local database *and* the + user's imported projects, including their `.git` history. Publishing would have leaked + third-party source. The bundle is now filtered at copy time and the build hard-fails if a + database, dotenv file, git history or native binary reaches it. +- The standalone server `chdir`s to its own directory, so a published install resolved the + database to `node_modules/…/dist/data/`. Every `npx …@latest` would have silently destroyed + the user's learning history. The launcher now pins the data directory to your working + directory. +- The standalone server defaulted to `0.0.0.0`, exposing a 127.0.0.1-only product across the + local network. The launcher pins the loopback interface. +- The repo's own `.gitignore` was traced into the bundle; npm honours nested ignore files, so + the Next runtime and compiled app were dropped from the tarball — a package that installs + and cannot boot. +- Turbopack requires external packages by a hashed name, so removing the build machine's + `better-sqlite3` made every request fail with `Failed to load external module`. +- The MCP server printed its banner to stdout, corrupting the JSON-RPC stream that stdio + transport requires; it now goes to stderr. +- Install paths containing spaces (`C:\Program Files\…`) broke the launcher. + +### Security + +- Ingests private source safely: symlink-confined tree walk, shell-free git, realpath and + credential-directory blocks, per-file size caps. Self-scanned security lens: **A/100**. +- Production CSP drops `unsafe-eval` (dev-only). CI gates on critical `npm audit` advisories. +- No telemetry, no analytics, no cloud dependency. + +### Known limitations + +- Image optimisation needs the optional `sharp` dependency; without it Next serves unoptimised + images. +- macOS is untested in CI (Windows + Linux are covered); it is expected to work via the same + per-platform native resolution. + +[1.0.0]: https://github.com/SheeshDarth/NirmiqCodeSensei/releases/tag/v1.0.0 diff --git a/README.md b/README.md index f8ee9fc..8ed9572 100644 --- a/README.md +++ b/README.md @@ -33,23 +33,34 @@ Everything is stored locally in SQLite. Nothing leaves your machine. ## Quick start -### 1. Clone and install +```bash +cd your-project +npx nirmiqcodesensei@latest +``` + +That's it. The dashboard opens at [http://127.0.0.1:3000](http://127.0.0.1:3000) — bound to localhost only. Requires Node.js 20+. + +Your learning data lives in `data/` **in the directory you launch from**, so each project keeps its own history, and it's yours to back up, move or delete. Migrations are applied automatically on first start. ```bash -git clone https://github.com/SheeshDarth/NirmiqCodeSensei.git -cd NirmiqCodeSensei -npm install +npx nirmiqcodesensei # start the app + open the dashboard +npx nirmiqcodesensei mcp # start the MCP server (stdio) +npx nirmiqcodesensei open # just open the dashboard +npx nirmiqcodesensei --help # all commands ``` -### 2. Run the app +Aliases: `npx codesensei` and `npx nirmiq` do the same thing. + +### From a clone (contributors) ```bash +git clone https://github.com/SheeshDarth/NirmiqCodeSensei.git +cd NirmiqCodeSensei +npm install npm run dev ``` -Database migrations are applied automatically on first start — no manual setup needed. Requires Node.js 20+. - -Open [http://127.0.0.1:3000](http://127.0.0.1:3000) — the app binds to localhost only. +The CLI detects which shape it's running in: a published install boots the prebuilt server, a checkout runs `next dev` against the source. --- @@ -58,7 +69,7 @@ Open [http://127.0.0.1:3000](http://127.0.0.1:3000) — the app binds to localho Start the MCP server: ```bash -npm run mcp +npx nirmiqcodesensei mcp ``` ### Claude Code @@ -69,9 +80,8 @@ Add to `.claude/mcp.json` in your project (or user MCP settings): { "mcpServers": { "nirmiqcodesensei": { - "command": "npm", - "args": ["run", "mcp"], - "cwd": "/absolute/path/to/NirmiqCodeSensei" + "command": "npx", + "args": ["nirmiqcodesensei", "mcp"] } } } @@ -84,9 +94,8 @@ Add to Cursor MCP settings (`Settings → Features → MCP`): ```json { "nirmiqcodesensei": { - "command": "npm", - "args": ["run", "mcp"], - "cwd": "/absolute/path/to/NirmiqCodeSensei" + "command": "npx", + "args": ["nirmiqcodesensei", "mcp"] } } ``` @@ -99,14 +108,15 @@ Add to `~/.codeium/windsurf/mcp_config.json`: { "mcpServers": { "nirmiqcodesensei": { - "command": "npm", - "args": ["run", "mcp"], - "cwd": "/absolute/path/to/NirmiqCodeSensei" + "command": "npx", + "args": ["nirmiqcodesensei", "mcp"] } } } ``` +> Running from a clone instead? Use `"command": "npm", "args": ["run", "mcp"]` with `"cwd"` set to the repo root. + --- ## MCP Tools diff --git a/docs/COUNCIL_REVIEW_LOG.md b/docs/COUNCIL_REVIEW_LOG.md index e3c84e6..7ee78d8 100644 --- a/docs/COUNCIL_REVIEW_LOG.md +++ b/docs/COUNCIL_REVIEW_LOG.md @@ -555,6 +555,31 @@ Given 44 findings across two audits (many overlapping), what is the correct orde --- +### REVIEW-013 — Scaling, load balancing and traffic handling for 1.0: build, stub, or record as N/A? + +**Date:** 2026-07-16 +**Trigger:** MS7 ships 1.0 as installable software. A conventional "deploy" checklist expects load balancing, horizontal scaling, a traffic handler, health checks and a rollout strategy. None of it exists here. Before tagging 1.0, decide whether that is a gap to close, a stub to add, or a deliberate non-goal to record — so the absence is a documented decision rather than something a future reader (or a reviewer) reads as an oversight. + +**Council Synthesis:** + +**Recommendation:** **Record it as N/A. Build nothing.** The premise of the checklist does not hold for this product's deployment model. NirmiqCodeSensei is not a service anyone deploys — it is software a single developer runs on their own machine, bound to 127.0.0.1, against an embedded single-writer SQLite database, with the user's private source code on local disk. The concurrency ceiling is *one person*. Load balancing distributes traffic across replicas; there is no traffic and there are no replicas. Horizontal scaling adds instances behind a shared data tier; a second instance would either contend for the same WAL lock or fragment the user's learning history across databases — strictly worse than one. A health-check endpoint monitors a process no operator is watching; the user sees the browser tab. Each of these would add surface area, dependencies and failure modes to buy capacity that cannot be consumed. + +This is a *deployment-model* judgement, not an admission of immaturity. The scaling work 1.0 actually needed was done, and it was per-machine, not per-fleet: analysis holds 300 files under 200 ms, incremental re-analysis short-circuits unchanged sources (~8× cheaper), and the review lens pass is flat (~4 ms) — all recorded in [BENCHMARKS.md](BENCHMARKS.md). Local-first shifts the scaling question from "how many users per node" to "how large a project per machine", and that question has a measured answer. + +**Risks:** +- *Read as an oversight later* → mitigated by this entry: the absence is a decision with a rationale and an explicit trigger for revisiting. +- *The deployment model changes (hosted/multi-tenant/team mode)* → this decision is scoped to the local-first, single-user model and would be void immediately. That pivot's blocking problem is not load balancing anyway; it is that SQLite-on-local-disk plus reading the user's private source code from local paths is the wrong data and trust architecture for a shared service. Revisit **only** on a deliberate move to a hosted model, and revisit the data tier first. +- *Genuine local performance problems get filed under "scaling N/A"* → they don't: per-machine performance is a live concern with a benchmark suite behind it. N/A covers fleet concerns only. + +**What NOT to Build Yet:** load balancer or reverse-proxy config; multi-instance/clustering support; a `/health` endpoint; PM2/systemd/Docker orchestration; blue-green or canary rollout; connection pooling (better-sqlite3 is synchronous and single-process by design); rate limiting (there is one user, on loopback); autoscaling policies; distributed tracing or APM. + +**Decision:** +> Load balancing, horizontal scaling and traffic handling are **out of scope for 1.0 by design**, not deferred — they solve problems a single-user, loopback-bound, local-first tool does not have. Scaling for this product means per-machine analysis performance, which is measured and gated in [BENCHMARKS.md](BENCHMARKS.md). Revisit only if the deployment model itself changes to a hosted or multi-tenant one, and re-architect the data tier before touching traffic. + +**Status:** ✅ Logged (MS7). Satisfies the REVIEW-011 exit criterion "load balancing recorded N/A". + +--- + ## Architecture Decisions Summary | ID | Decision | Outcome | Phase | @@ -569,5 +594,6 @@ Given 44 findings across two audits (many overlapping), what is the correct orde | REVIEW-008 | Whole-project review — polish sprint: GitHub-pull on refresh, blended progress formula (#26), conceptType form enum (#30); defer cohesion/search/graph | ✅ Implemented — 10/10 tests | Pre-1.0 | | REVIEW-009 | Landing strategy — preserve 15 commits (rebase-and-merge, no squash); skip paid AI smoke test; F2/F4/F5 cleanup as 3 free commits | ✅ Implemented — cleanup done | Pre-1.0 | | REVIEW-010 | CodeSensei program — visible-identity rename, 8-lens local senior-review engine (findings-only optional AI), Obsidian-grade graph | ✅ Implemented — 16/16 tests | v0.2 | -| REVIEW-011 | Road to deployed 1.0 — single-problem megasprints (MS1–MS7) + full deep rename to NirmiqCodeSensei; load-balancing recorded N/A | 🔄 In progress (MS1✅ MS2✅ MS3✅ MS4✅) | v0.2→1.0 | +| REVIEW-011 | Road to deployed 1.0 — single-problem megasprints (MS1–MS7) + full deep rename to NirmiqCodeSensei; load-balancing recorded N/A | ✅ Complete (MS1–MS7 ✅; N/A recorded in REVIEW-013) | v0.2→1.0 | | REVIEW-012 | MS3 cross-feature module links (#27/#28) — deferred from MS3, then **built in MS4** as a soft `module_key` column with deterministic tagging on both AI + offline paths (no AI-schema dependency) | ✅ Implemented (MS4) | v0.2→1.0 | +| REVIEW-013 | Scaling/load balancing/traffic handling — **N/A by design** for a single-user, loopback-bound, local-first tool; scaling here means per-machine analysis performance (measured in BENCHMARKS.md). Revisit only on a move to a hosted model, data tier first | ✅ Recorded (MS7) — build nothing | 1.0 | diff --git a/docs/MEGASPRINT_ROADMAP.md b/docs/MEGASPRINT_ROADMAP.md index f7d02ec..ebf8387 100644 --- a/docs/MEGASPRINT_ROADMAP.md +++ b/docs/MEGASPRINT_ROADMAP.md @@ -22,7 +22,7 @@ free + BYOK path (`ANTHROPIC_API_KEY` optional; offline analyzer is the default) | **MS4** ✅ | Algorithms & Analysis Depth | The analysis *is* the product — make it rigorous and calibrated | **DONE** — ✅ codeHealth calibrated to code *density* not project size (self-scan F→**B(76)**, overall **A(95)**); `computeCodeHealthScore` + relativity test. ✅ incremental re-analysis — `computeSourceFingerprint` (path\|size\|mtime sha256) + `learning_maps.source_fingerprint` (0009); reanalyze short-circuits `{unchanged:true}`. ✅ [benchmarks](BENCHMARKS.md) — 300-file analysis <200ms, flat ~4ms lens pass, incremental ~8× cheaper. ✅ #27/#28 module associations — soft `module_key` on questions/concepts (0010), deterministic tagging on both AI + offline paths, module cards link to their questions/concepts | | **MS5** 🔄 | Quality & Reliability (QA) | 16 tests is a foundation, not production confidence | **In progress** — ✅ E2E smoke of the critical path (import→analyze→deep-review→export), which also closed a real gap: the deep review now reaches the export. ✅ broadened coverage — BM25 search (indexing/ranking/rebuild). ✅ cross-platform CI — gate now runs on Windows + Linux (macOS deferred: fs/path-equivalent to Linux, 10× runner cost; Windows is the divergent platform). Suite 24/24. Remaining: watch first Windows CI run land green | | **MS6** ✅ | Framework & Performance | Production Next.js build quality | **DONE** — ✅ `output: "standalone"` self-contained server (migrations + native better-sqlite3 traced in); verified: boots on 127.0.0.1, HTTP 200, runtime DB migrated. ✅ a11y — accessible names (aria-label) on icon-only delete buttons. ✅ no UI-blocking analysis already met (import + reanalyze show pending/disabled states) | -| **MS7** | Distribution & Release | Turn the repo into installable, versioned software — the actual "deploy" | `npx nirmiqcodesensei@latest` runs on a fresh machine; tagged v1.0.0 GitHub Release; CHANGELOG current; scaling-N/A ADR recorded | +| **MS7** ✅ | Distribution & Release | Turn the repo into installable, versioned software — the actual "deploy" | **DONE** — ✅ `npx nirmiqcodesensei` verified by installing the real tarball into an empty dir: HTTP 200, DB created in the *user's* cwd and runtime-migrated (11), MCP handshake clean (12 tools). Ships MS6's standalone build; native modules resolve per-platform at install (one tarball, every OS) via `scripts/pack-standalone.mjs`, which also hard-fails if a database, dotenv, git history or `.node` binary reaches the bundle. Five ship-blockers found only by installing it — incl. tracing copying `data/` (user projects + `.git`) into the bundle, and the DB landing in `node_modules` where `npx …@latest` would wipe it. ✅ CHANGELOG. ✅ scaling-N/A ADR (REVIEW-013). ⏳ v1.0.0 tag + Release pending explicit go-ahead; `npm publish` is the maintainer's to run | | MS8 | Launch (stretch) | Discoverability & polish | Onboarding, docs, MCP-directory submission, desktop-installer spike | Full detail, per-megasprint scope, and the MS1 rename mechanics live in the approved plan diff --git a/docs/PROJECT_DOSSIER.md b/docs/PROJECT_DOSSIER.md index 5b317d8..0e9f457 100644 --- a/docs/PROJECT_DOSSIER.md +++ b/docs/PROJECT_DOSSIER.md @@ -183,7 +183,7 @@ Each solves **one** major problem deeply and ships as its own gated PR. | **MS4 ✅** Algorithms & Analysis Depth | make the analysis rigorous and calibrated | codeHealth calibrated to *density* not size (self-scan F→**B**, overall **A**) via `computeCodeHealthScore`; **incremental re-analysis** (`computeSourceFingerprint` + `source_fingerprint` 0009, `{unchanged:true}` short-circuit); documented **benchmarks** (300-file < 200 ms, ~8× incremental savings); **#27/#28 module associations** — soft `module_key` (0010) on questions/concepts, deterministic tagging on both AI + offline paths, module cards link to their questions/concepts | | **MS5 ✅** Quality & Reliability | a handful of tests isn't production confidence | **E2E smoke** of the critical path (import→analyze→deep-review→export) — which closed a real gap: the deep review now reaches the export; **BM25 search coverage**; **cross-platform CI** (Windows + Linux matrix). Suite 24/24 | | **MS6 ✅** Framework & Performance | production Next.js build quality | **`output: "standalone"`** self-contained server (migrations + native `better-sqlite3` force-traced in) — verified boots on 127.0.0.1, HTTP 200, runtime DB migrated; **a11y** accessible names on icon-only buttons; **no UI-blocking analysis** (import + reanalyze already show pending/disabled states) | -| **MS7** Distribution & Release | turn the repo into installable, versioned software — the actual "deploy" | *(remaining)* `npx nirmiqcodesensei@latest` on a fresh machine; tagged **v1.0.0** GitHub Release; CHANGELOG; **scaling-N/A ADR** (load balancing / horizontal scaling intentionally out of scope for a single-user local-first tool) | +| **MS7 ✅** Distribution & Release | turn the repo into installable, versioned software — the actual "deploy" | **`npx nirmiqcodesensei`** — ships MS6's standalone build as a published package; native modules resolve **per-platform at install** (one tarball serves Windows/Linux/macOS) via `scripts/pack-standalone.mjs`, which assembles `dist/` and **hard-fails** if a database, dotenv, git history or `.node` binary reaches the bundle. **Verified by installing the real tarball into an empty directory**: HTTP 200 on `/dashboard`, `/workspaces`, `/workspaces/import`; DB created in the *user's* cwd and runtime-migrated (11); MCP handshake clean (12 tools, stdout pure JSON-RPC). MCP bundled at prepack (esbuild, CJS). **CHANGELOG** + **scaling-N/A ADR (REVIEW-013)**. Gate green, 24/24. *v1.0.0 tag/Release pending explicit go-ahead; `npm publish` is the maintainer's to run* | Monetization (Pro/Gumroad) is **deferred/dormant** — 1.0 runs fully on the free + BYOK path. @@ -204,6 +204,23 @@ Monetization (Pro/Gumroad) is **deferred/dormant** — 1.0 runs fully on the fre | Stacked PR closed instead of retargeting | deleting the base branch on merge closed the child PR | `git rebase --onto ` to drop the duplicated commits, re-open against master | | `git commit -m` mangled multiline bodies | PowerShell 5.1 here-string interpolation | commit via `-F -` heredoc / quote-free bodies | +### MS7 — the distribution class of bugs + +None of these are visible from source, a green build, or a passing test suite. Every one was +found by building the real tarball and installing it into an empty directory — which is why +that is now the verification step for any packaging change, not `npm run build`. + +| Problem | Root cause | Fix | +|---|---|---| +| The bundle contained the local **database and the user's imported projects, incl. their `.git`** — publishing would have leaked third-party source | file tracing resolves `path.join(process.cwd(), "data")` in `lib/db/client.ts` to a literal dir and sweeps it in. `outputFileTracingExcludes` **does not stop this** (tried for data/docs/tests/scripts — zero effect on Next 16.2.7) | `scripts/pack-standalone.mjs` filters at copy time and **asserts** — the build fails if a DB, dotenv, git history or `.node` binary reaches `dist/` | +| A published install wrote the user's DB into `node_modules/…/dist/data/`, so every `npx …@latest` would **silently wipe their learning history** | standalone `server.js` runs `process.chdir(__dirname)`; `lib/db/client.ts` resolves the DB from `process.cwd()` | launcher pins `NCS_DATA_DIR` to the invoking cwd — reusing the env hook already built for out-of-tree entry points | +| A 127.0.0.1-only product would have been **exposed across the LAN** | standalone `server.js` defaults `HOSTNAME` to `0.0.0.0` (the `next start --hostname` flag doesn't apply — there is no Next CLI) | launcher pins `HOSTNAME=127.0.0.1` | +| Tarball installed but **could not boot** — no Next runtime, no compiled app | the tracer copied the repo's own `.gitignore` into the bundle; npm honours ignore files *nested inside* the packed dir, so `node_modules/` + `.next/` were dropped. `npm publish` would have succeeded | exclude `.gitignore`/`.git` from the bundle; assert on a dist-root ignore file | +| Stripping the build machine's `better-sqlite3` **500'd every request** | Turbopack compiles `serverExternalPackages` to `require()` of the **hashed** dir name (`better-sqlite3-90e2652d…`) — not the bare specifier | keep the hashed dir requirable but replace its payload with a shim that delegates to the platform-correct copy npm installs | +| MCP handshake returned nothing from a published install | the CLI banner printed to **stdout**, which stdio transport reserves for JSON-RPC; separately, an ESM bundle crashed on `typescript-estree`'s `__filename` | banner → stderr; bundle as **CJS** | +| `eslint .` died with a **V8 OOM crash** | flat config only ignores `node_modules` by default, so it walked `dist/` (~2000 files, one 12 MB) | `--ignore-pattern dist/ --ignore-pattern data/` in the lint script | +| `npm install` failed `EOVERRIDE` | the `esbuild` security pin collided with esbuild as a direct devDependency | `"esbuild": "$esbuild"` — reference the direct dep, keeping the pin's intent | + --- ## 8. Verification, testing & CI @@ -237,17 +254,29 @@ Monetization (Pro/Gumroad) is **deferred/dormant** — 1.0 runs fully on the fre ## 10. Current status & what remains -**Done & merged:** MS1–MS6 (identity, security, architecture/data-integrity, analysis depth, -QA, framework/perf). Master is green on Windows + Linux; the standalone build boots and self-migrates. - -**Remaining — MS7 (Distribution & Release):** -1. `npx nirmiqcodesensei@latest` runs on a fresh machine (package the standalone build + a launcher - that copies `.next/static` + `public` alongside `server.js`). -2. Tagged **v1.0.0** GitHub Release with a current CHANGELOG. -3. A **scaling-N/A ADR** recording that load balancing / horizontal scaling is a deliberate - non-goal for a single-user local-first tool (not an oversight). -4. Optional stretch (MS8): onboarding polish, MCP-directory submission, desktop-installer spike. +**Done & merged: MS1–MS7** — identity, security, architecture/data-integrity, analysis depth, +QA, framework/perf, and distribution. Master is green on Windows + Linux. The product is +**installable software**: `npx nirmiqcodesensei` boots the app and `npx nirmiqcodesensei mcp` +serves 12 MCP tools, both verified from a real tarball installed into an empty directory. +Version **1.0.0** across the package, the MCP `serverInfo` and the manifest. Suite 24/24; +migrations at 0010. REVIEW-011's megasprint program is complete, including its +"load-balancing recorded N/A" criterion ([REVIEW-013](COUNCIL_REVIEW_LOG.md)). + +**Remaining for the 1.0 release itself** (outward-facing, deliberately gated): +1. Tag **v1.0.0** and cut the GitHub Release — pending the maintainer's explicit go-ahead. +2. `npm publish` — the maintainer runs this; it needs npm credentials. +3. After publishing, confirm `npx nirmiqcodesensei@latest` resolves from the registry (local + verification covers everything up to the registry itself). + +**Optional stretch (MS8):** onboarding polish, MCP-directory submission, desktop-installer +spike, macOS CI leg. + +**The MS7 lesson worth carrying forward:** a green gate says the code is correct; it says +nothing about what you ship. Five ship-blockers — one of them a privacy leak of a third +party's source, one a silent destroyer of user data — were invisible to lint, typecheck, +build and 24 passing tests, and surfaced the moment the tarball was installed for real. For +packaging changes, `npm pack` + install into an empty directory **is** the test. --- -*This dossier is a living document — update it as MS7 lands and 1.0 ships.* +*This dossier is a living document — update it as 1.0 ships and MS8 is considered.*