CONTRIBUTING.md

Contributing

We welcome your code contributions. Learn all about how to get involved on the https://mariadb.org/contribute/ web page.

Bugs for MariaDB AI Plugins are filed using Jira at https://jira.mariadb.org/ for the AIPL (AI Plugins) project https://jira.mariadb.org/projects/AIPL.

Development and maintenance

Everything below is for people working on the plugins rather than with them.

Harness	Plugin Folder	Test suite	CI workflow
Claude Code	claude	claude/dev-plugin-tests	claude-test.yml
Codex	codex	codex/dev-plugin-test	codex-test.yml
OpenCode	opencode	opencode/dev-plugin-test	opencode-test.yml
Pi	pi	pi/dev-plugin-tests	pi-test.yml
Hermes Agent	hermes	hermes/dev-plugin-tests	hermes-test.yml

How the plugins are built

1. Skills — MariaDB agent skills (SKILL.md docs) covering SQL statements, functions, client tools, connectors, and topical deep-dives. Most are vendored from mariadb-corporation/mariadb-docs/agent-skills; additional repo-local skills are maintained in additional-skills/, grouped into sql/ (e.g. mariadb-schema-create-script), rest/ (MariaDB REST Service) and schema-management/ (MSM lifecycle) subfolders. The dev plugin vendors all of them; the sql plugin vendors only additional-skills/sql/. The agent surfaces the right skill by its description "Use when …" trigger.
2. A native MCP server — the mariadb-shell binary, launched by scripts/mariadb-mcp-launcher.sh (and .cmd for native Windows). On first use it runs the first mariadb-shell that satisfies MARIADB_SHELL_VERSION — $MARIADB_SHELL_BIN, one on PATH, or an install in ~/.local/bin (%LOCALAPPDATA%\Programs\mariadb-shell\bin on Windows) — and otherwise runs the shell's own install.sh / install.ps1 to put the newest release there first. Either way it execs that binary as the MCP server over stdio, and later runs reuse the install. Pi uses the same launcher, which its extension pi/dev-plugin/src/index.ts registers with Pi's built-in MCP support (pi.registerMcpServer). Hermes runs it too: each Hermes plugin is a portable Agent Plugins v1 package whose root mcp.json declares the mariadb server with ./scripts/mariadb-mcp-launcher.sh as its command. mcp.json cannot branch per OS, so native Windows users add the .cmd launcher in Hermes' config.yaml instead (see hermes/dev-plugin/README.md).

Repository layout

ai-plugins/
├── .claude-plugin/marketplace.json    # Claude Code marketplace entry
├── .agents/plugins/marketplace.json   # Codex marketplace entry (Codex reads this, not .codex-plugin/)
├── additional-skills/                 # repo-local skills, grouped by topic subfolder
│   ├── sql/                            # vendored into dev + sql (e.g. mariadb-schema-create-script)
│   ├── rest/                           # MariaDB REST Service skills (dev only)
│   └── schema-management/              # MSM lifecycle skills (dev only)
├── claude/
│   ├── dev-plugin/                    # the Claude Code plugin (skills/, scripts/, .mcp.json)
│   ├── sql-plugin/                    # SQL-focused variant (no client-tool/connector skills)
│   ├── contributor-plugin/           # mariadb-shell contributor skills (skills only)
│   └── dev-plugin-tests/              # its pytest suite (4 tiers — adds e2e)
├── codex/
│   ├── dev-plugin/
│   ├── sql-plugin/
│   ├── contributor-plugin/
│   └── dev-plugin-test/
├── opencode/
│   ├── dev-plugin/
│   ├── sql-plugin/
│   ├── contributor-plugin/
│   └── dev-plugin-test/
├── pi/                                # Pi (pi.dev) extension sources — dev only for now
│   ├── dev-plugin/                    # src/index.ts extension (registers the MCP server), scripts/ (launchers), skills/
│   └── dev-plugin-tests/              # its pytest suite (static + db + e2e; no eval tier)
├── hermes/                            # Hermes Agent — portable Agent Plugins v1 packages
│   ├── dev-plugin/                    # plugin.json, mcp.json, scripts/ (launchers), skills/ → mariadb-dev
│   ├── sql-plugin/                    # → mariadb-sql
│   ├── contributor-plugin/            # → mariadb-contributor (plugin.json + skills/ only)
│   └── dev-plugin-tests/              # its pytest suite (static + db + e2e; no eval tier)
├── package.json                       # repo-root pi manifest (`pi` field → pi/dev-plugin/) + test scripts
├── run_tests.py                        # runs every suite with the mariadb-shell Python + coverage
├── pytest-coverage.ini                 # shared pytest config used by run_tests.py
├── .coveragerc                         # coverage config (reports in test-results/ + htmlcov/)
├── docs/                              # the DevHub site (GitHub Pages, built-in Jekyll, no build step)
│   ├── _tutorials/                    # one file per tutorial; filename = URL slug
│   ├── _data/skills.yml               # GENERATED by docs/regenerate-skills-data.py
│   └── README.md                      # how to publish it and add a tutorial
├── scripts/
│   └── sync-skills.sh                 # vendors skills into every plugin
└── .github/workflows/                 # one CI workflow per harness (pi and hermes have no eval job)

Skills sync

The last thing it does is regenerate the DevHub's skill data (docs/_data/skills.yml and the skill_count in docs/_config.yml) so the published catalog and counts cannot drift from what actually ships. That step needs python3; without it the sync still succeeds and warns, and npm run docs:skills catches up.

Skills are vendored into each plugin — the plugins are self-contained and are never edited by hand. scripts/sync-skills.sh is the single source of truth:

scripts/sync-skills.sh          # sync the latest upstream (default branch head)
scripts/sync-skills.sh <ref>    # or a specific tag / branch / commit

It downloads the upstream agent-skills/ tree once at the chosen ref — the latest commit on the upstream default branch unless you pass one — and copies the selected skills into each dev/sql plugin's skills/ dir, writing per-plugin provenance (including the exact commit SHA synced) to skills-source.json. The dev plugins get every upstream layer plus all additional-skills/ subfolders; the sql plugins get the SQL-focused upstream layers plus only additional-skills/sql/ (rest/ and schema-management/ are dev-only — see SQL_INCLUDE_LAYERS in the script). It also vendors the contributor plugins from a separate source — the mariadb-shell repo's .claude/skills/ tree. That repo is public, so this needs no credentials; a GH_TOKEN (or gh auth token) is still used when present, purely for the higher GitHub API rate limit.

Flat layout. Every plugin — including pi/dev-plugin — vendors skills flat, skills/<skill>/SKILL.md, regardless of how they are grouped upstream. This is what OpenCode requires (it discovers skills only one directory deep) and it keeps every plugin identical on disk. It also keeps pi's loader happy: skills/ must contain nothing but skill directories plus the manifest and skills-source.json — a stray dir without a SKILL.md makes pi's loader fail. The vendored .skills-manifest.json is rewritten to flat paths while preserving each skill's layer grouping, so the manifest-driven test suites still resolve. Requires jq and curl.

Testing

All five harness plugins have a parallel pytest suite; run_tests.py discovers them by */dev-plugin-test*/conftest.py:

Tier	Marker	Needs	Checks
1. Static / structural	static	nothing	frontmatter, manifest↔disk consistency, cross-references, SQL fences, statement-skill contract
2. SQL execution	db	mariadb-shell + a server binary	the skills' recommended DDL runs on a live server with the documented effect
3. Behavioral evals	eval	an LLM API key	the skill steers the model toward the MariaDB-preferred form (opt-in; deselected by default)
4. End-to-end	e2e	the harness CLI, authenticated	drives the real CLI with the plugin loaded, then asserts the side effects (opt-in)

Tiers 1 and 2 are the same everywhere. The opt-in tiers differ by harness, because what there is to drive differs:

• Claude — eval (Anthropic SDK) plus two e2e modules: the REST workflow (sandbox, schema, REST DDL) and the MSM schema-lifecycle workflow.
• Codex — eval (OpenAI SDK) plus the same two e2e workflows, and two token-free checks: that Codex resolves this repo's Codex plugin, and that setup-codex-mcp.sh leaves Codex a server it can spawn.
• OpenCode — eval only; no e2e yet.
• Pi — e2e only, and no eval tier: pi has no SDK of its own to prompt, so driving the pi binary is the behavioural test. Its e2e installs this repo as a project-local pi package (pi install -l, which pi honours only when the run passes --approve), checks that the vendored skills reached the model and that the generated schema script carries the Start Block its skill mandates, and that a codemode script reaches a tool of the mariadb MCP server the extension registers. Both runs use a throwaway PI_CODING_AGENT_DIR with only the provider setup copied in, so the user's own packages (a global pi-mcp-adapter would replace the built-in MCP) stay out of the way.
• Hermes — e2e only, and no eval tier, for the same reason as pi. Its static tier also runs hermes plugins validate on the dev plugin when the CLI is present. The e2e creates a temporary Hermes profile (hermes profile create … --no-skills), copies in the user's provider setup (config.yaml without its mcp_servers, .env, auth.json), installs the working tree's plugin with hermes plugins install file://…#hermes/dev-plugin --enable and drives hermes chat -q … --format stream-json --yolo. It checks that the model loaded a skill under the plugin's namespace through skill_view, that the generated schema script carries the mandated Start Block, and that a mcp__mariadb__db_list_connections call succeeded; the profile is deleted afterwards. HERMES_BIN, HERMES_HOME and E2E_TIMEOUT override the binary, the Hermes home the provider setup is copied from, and the per-run timeout.

The e2e tiers self-skip rather than fail when their toolchain is absent — no CLI, no authenticated provider, or no resolvable mariadb-shell.

The db tier needs no server running beforehand: it deploys a throwaway sandbox instance on a free port through the mariadb-shell MCP server's sandbox.* tools, the way mariadb-shell-plugins/mcp_plugin/tests does, and deletes it afterwards. The shell's user config home is isolated to a temp dir for the run (with the real one's plugins symlinked in, and the sandbox dir allow-listed), so nothing touches ~/.mariadb-shell. Setting any of MARIADB_HOST / MARIADB_PORT / MARIADB_USER / MARIADB_PASSWORD switches the tier onto that already-running server instead — which is what CI and docker-compose.yml do. MARIADB_SANDBOX_MARIADBD pins the server binary the sandbox starts. The tier skips only when no sandbox can be deployed at all (no mariadb-shell, no mariadbd/mysqld, or an MCP server without the sandbox tools).

The unified runner

run_tests.py runs every suite with the Python that ships inside mariadb-shell (mariadb-shell --pym pytest), the same way mysql-shell-plugins/mcp_plugin/run_tests.py does, so the tests execute against the same interpreter and packages as the MCP server they exercise. It installs each suite's requirements.txt into that Python, runs one pytest process per suite (their same-named modules and lib packages can't share a process), and appends coverage across the runs into one combined report.

./run_tests.py                  # every suite, default tiers (static + db)
./run_tests.py claude           # one suite (named after its top-level dir)
./run_tests.py -m static        # a single tier, all suites
./run_tests.py claude -m e2e    # the opt-in end-to-end tier
./run_tests.py -k manifest      # only tests matching a pattern
./run_tests.py -- --lf -x       # anything after `--` is passed to pytest

npm test                        # same, via package.json (also test:static/db/eval/e2e)

The binary comes from --shell, else $MARIADB_SHELL, else PATH, and its directory is prepended to the PATH the suites see so the e2e tier's MCP launcher resolves the same one. --no-install skips the dependency install, --no-coverage the measurement, --userhome relocates the shell's user config home (by default the real one is left alone — the e2e tests that need isolation create their own).

Config lives in pytest-coverage.ini (markers, default tier selection, report formats) and .coveragerc. Reports land in test-results/ (coverage.xml, <suite>-tests.xml) and htmlcov/, all git-ignored. Coverage measures the suites' own Python — the lib/ helpers and conftest.py fixtures; the skills themselves are Markdown, which the static tier checks instead.

Per-suite, without the runner

cd claude/dev-plugin-tests      # or pi/ , hermes/dev-plugin-tests , codex/ , opencode/ dev-plugin-test
pip install -r requirements.txt
pytest -m static                # fast, no services
pytest -m db                    # needs a MariaDB 11.8 server (docker-compose.yml provided)
pytest -m eval                  # opt-in, calls the LLM API      (not in the pi/hermes suites)
pytest -m e2e                   # opt-in, drives the harness CLI (claude, codex, pi, hermes)

The eval tier uses the SDK matching its agent: the Claude and OpenCode suites call the Anthropic SDK (claude-opus-4-8, needs ANTHROPIC_API_KEY); the Codex suite calls the OpenAI SDK. The pi and Hermes suites have no eval tier at all — see the tier notes above. CI runs static on every push/PR and db on PRs for all five suites, and eval nightly / on demand for the three that have it; the e2e tiers are local-only, since they need an authenticated CLI and a mariadb-shell on the machine.