Add API benchmark toolkit

[viqueen]

Summary

• Adds a Node.js CLI toolkit under toolkit/ for benchmarking Docker Compose-based API services
• Captures build time, deploy/start time, and k6 load test metrics (throughput, latency percentiles, error rate)
• Publishes results to Grafana Cloud via OTLP/HTTP for cross-implementation comparison
• Includes comparison reporting with terminal table output and best-value highlighting
• Uses npm workspaces so everything installs and runs from the repo root

Commands

Command	Description
npm run benchmark -- run <target>	Full pipeline: build, deploy, loadtest, collect
npm run benchmark -- build <target>	Docker compose build + timing
npm run benchmark -- deploy <target>	Docker compose up + readiness wait + timing
npm run benchmark -- loadtest <target>	k6 load test against running target
npm run benchmark -- publish <results>	Push results to Grafana Cloud
npm run benchmark -- compare <files...>	Compare results in terminal table
npm run benchmark -- list	List configured targets
npm run benchmark -- clean <target>	Docker compose down

Test plan

• Run npm install from repo root
• Run npm run benchmark -- list to verify config loading
• Run npm run benchmark -- --help to verify all commands registered
• Add a sample Docker Compose project and run full pipeline
• Verify results JSON is written to results/
• Configure Grafana Cloud credentials and test --publish
• Run compare with multiple result files

🤖 Generated with Claude Code

[Copilot]

Pull request overview

Adds a new Node.js-based benchmarking toolkit to this repo for running a Docker Compose build/deploy pipeline, executing k6 load tests, writing results to JSON, comparing runs in the terminal, and optionally publishing metrics to Grafana Cloud via OTLP/HTTP.

Changes:

• Introduces toolkit/ as an npm workspace providing a benchmark CLI with commands for build/deploy/loadtest/run/publish/compare/list/clean.
• Implements metrics collection/parsing plus reporting (JSON output + terminal comparison table).
• Adds Grafana Cloud OTLP/HTTP publishing and updates repo docs/config to support running benchmarks from the root.

Reviewed changes

Copilot reviewed 31 out of 35 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
toolkit/src/util/logger.js	Adds pino-based pretty logger helper.
toolkit/src/util/exec.js	Adds execa wrapper with debug logging.
toolkit/src/report/table.js	Renders terminal comparison tables with best-value highlighting.
toolkit/src/report/json.js	Writes benchmark results JSON files to disk.
toolkit/src/report/compare.js	Loads result files and computes “best” values for comparison.
toolkit/src/publish/grafana-cloud.js	Publishes metrics to Grafana Cloud via OTLP/HTTP with retry logic.
toolkit/src/publish/formatter.js	Formats results into OTLP/HTTP JSON metric payloads.
toolkit/src/metrics/k6-parser.js	Parses k6 summary-export JSON into normalized metrics.
toolkit/src/metrics/collector.js	Collects environment metadata and assembles final results object.
toolkit/src/core/timer.js	Provides high-resolution timing utility.
toolkit/src/core/k6.js	Runs k6 with --summary-export and returns parsed output.
toolkit/src/core/health.js	Polls readiness probe endpoint until healthy/timeout.
toolkit/src/core/docker.js	Wraps docker compose build/up/down/ps execution.
toolkit/src/config/schema.js	Defines Zod schema for benchmark config.
toolkit/src/config/loader.js	Loads, interpolates env vars, validates, and resolves targets.
toolkit/src/config/defaults.js	Adds defaults object (currently not integrated elsewhere).
toolkit/src/cli/index.js	Defines top-level CLI program and global options.
toolkit/src/cli/commands/run.js	Implements full pipeline command (build → deploy → loadtest → collect → cleanup).
toolkit/src/cli/commands/publish.js	Implements publish command for existing results JSON.
toolkit/src/cli/commands/loadtest.js	Implements loadtest-only command against a running target.
toolkit/src/cli/commands/list.js	Lists configured benchmark targets.
toolkit/src/cli/commands/deploy.js	Implements deploy timing + readiness wait.
toolkit/src/cli/commands/compare.js	Implements comparison of multiple results files.
toolkit/src/cli/commands/clean.js	Implements docker compose down teardown command.
toolkit/src/cli/commands/build.js	Implements build timing command.
toolkit/package.json	Declares toolkit workspace package, deps, and benchmark bin.
toolkit/package-lock.json	Locks toolkit dependencies.
toolkit/k6/scripts/default-http.js	Adds default HTTP k6 script.
toolkit/k6/scripts/default-grpc.js	Adds default gRPC k6 script.
toolkit/bin/benchmark.js	Adds CLI entrypoint script.
package.json	Adds npm workspaces and root benchmark script.
package-lock.json	Updates lockfile for workspace linkage and deps.
benchmark.config.json	Adds example benchmark configuration and Grafana env interpolation.
README.md	Adds full documentation and usage instructions for toolkit.
.gitignore	Ignores results/ output directory.

Files not reviewed (1)

• toolkit/package-lock.json: Language not supported

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.