From e2c3c9b00eb4c27d7710e607f4a57b1416afd104 Mon Sep 17 00:00:00 2001 From: Tin Dang Date: Mon, 24 Aug 2026 03:20:52 +0700 Subject: [PATCH] fix(test): scripts/test-commands.sh runs to completion and reports totals (#679) The suite CLAUDE.md points at for every new command had never printed a result. It died partway through, silently, and the operator saw a truncated log instead of a summary. It now finishes: 504 rows, 478 passing. The 26 remaining failures are pre-existing and are filed as #683 -- they became visible for the first time because the run now reaches them. Nothing here weakens an assertion to get a green. Every abort was the same bash shape: a command whose non-zero status `set -euo pipefail` converts into a silent exit. grep exits 1 when it matches nothing, lsof exits 1 when a port is free, pkill exits 1 when nothing matched, and a shell function whose last statement is a false `if` returns 1 as well. Because that failure prints nothing at all, each instance hid the next -- which is why this took several rounds, and why two of the guards below are for aborts introduced by earlier ones in this same commit. Guarded: * 26 command substitutions ending in grep. The reported NUMERIC-07 site was one of a class, so the class is fixed rather than the instance. * 6 raw redis-cli pipelines and all four client wrappers, so a dead server yields failing rows plus a summary instead of a truncated log. Nothing in the file branches on those wrappers' exit status (checked: no `if mcli` and no `mcli ... &&` anywhere), so `|| true` costs no signal. * `cargo build ... 2>/dev/null`, which threw away the reason a build failed and left the log reading "Building moon..." and nothing else. * the cleanup trap, which returned its last kill's status rather than the script's -- reporting a clean run as a failure. Three defects that produced wrong results rather than aborts: * `grep -Pzo "(?s)A.*B"` at 13 call sites is GNU-only. On a macOS host grep is ugrep, which rejects -P and exits 2; since those rows compare output rather than status, that 2 was being reported as moon's answer. Replaced with a portable spans() helper. * No --dir, so moon treated the CWD as its data dir: the suite wrote appendonlydir/ and moon.lock into the repo root and reloaded the previous run's FT index definitions, so a second run failed with "Index already exists". Each run now gets a fresh mktemp dir, removed on exit. * No port pre-flight. A leftover server from an unrelated run answers and every row silently compares against it. Not hypothetical: it produced a full run of MOONERR diskfull failures traced to another session's moon on the port. The suite now refuses to start on an occupied port and names the holder. The `FT.CREATE ... VECTOR FLAT` row expected OK, but moon has only ever implemented HNSW (ERR expected HNSW algorithm, in ft_create.rs since #27), so it had failed from the day it was written and took four dependent rows with it. It now builds an HNSW index, and the FLAT gap is asserted explicitly instead of hiding inside a row that expected success. Added a regression row for #681 asserting the server is still alive after a truncated FT.CREATE. Proven in both directions: it fails against a pre-#682 binary and passes after. CLAUDE.md's "190 tests" was stale by more than half. Fixes #679 Refs #681, #682, #683 author: Tin Dang --- CHANGELOG.md | 46 ++++++++ CLAUDE.md | 2 +- scripts/test-commands.sh | 233 ++++++++++++++++++++++++++++++--------- 3 files changed, 227 insertions(+), 54 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1b1f28fd..e836309e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -190,6 +190,52 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 as an answer. That shard also logs a warning, because cold keys with no manifest will not survive a restart — `rebuild_from_manifest` is the only thing that re-indexes them. ### Fixed +- **`scripts/test-commands.sh` runs to completion and prints its summary** (#679). The + suite CLAUDE.md points at for every new command had never reported totals: it died + partway through, silently, and the operator saw a truncated log rather than a result. + It now finishes — **504 rows, 478 passing** — which is also how the 26 long-standing + failures in it became visible for the first time (#683 tracks them; none are + regressions from this change). + + Every abort was the same bash shape: a command whose non-zero status `set -euo + pipefail` turns into a silent exit. `grep` exits 1 when it matches nothing, `lsof` + exits 1 when a port is free, `pkill` exits 1 when nothing matched, and a shell + function whose last statement is a false `if` returns 1 too. Because the failure + prints nothing at all, each instance hid the next — which is why this took several + rounds and why two of the fixes are for aborts introduced *by* the earlier fixes: + + * 26 command substitutions ending in `grep` (the reported NUMERIC-07 site was one of + a class, so the whole class is guarded); + * 6 raw `redis-cli` pipelines and all four client wrappers, so a dead server produces + failing rows and a summary instead of a truncated log; + * `cargo build ... 2>/dev/null`, which discarded the reason a build failed and left + the log reading `Building moon...` and nothing else; + * the `cleanup` trap, which returned the status of its last `kill` rather than the + script's — reporting a clean run as a failure. + + Three defects behind wrong *results* rather than aborts: + + * **`grep -Pzo "(?s)A.*B"` (13 call sites) is GNU-only.** On a macOS host `grep` is + ugrep, which rejects `-P` and exits 2 — and since the rows compared output rather + than status, that `2` was reported as moon's answer. Replaced with a portable + `spans()` helper. + * **No `--dir`, so moon used the CWD.** The suite wrote `appendonlydir/` and + `moon.lock` into the repo root and reloaded the previous run's FT indexes, so a + second run failed with `ERR Index already exists`. Each run now gets a fresh temp + dir, removed on exit. + * **No port pre-flight.** A leftover server from an unrelated run answers, and every + row silently compares against it. This is not hypothetical — it produced a full run + of `MOONERR diskfull` failures traced to another session's moon holding the port. + The suite now refuses to start on an occupied port and names the holder. + + The `FT.CREATE ... VECTOR FLAT` row expected `OK`, but moon has only ever implemented + HNSW (`ERR expected HNSW algorithm`, in `ft_create.rs` since #27), so it had failed + from the day it was written and took four dependent rows down with it. It now creates + an HNSW index, and the FLAT gap is asserted explicitly instead of hiding inside a row + that expected success. Added a regression row for #681 that checks the server is still + alive after a truncated `FT.CREATE` — proven in both directions: it fails against a + pre-#682 binary and passes after. + - **A truncated `FT.CREATE` no longer aborts the server** (#681). `FT.CREATE idx ON HASH PREFIX 1 d: SCHEMA v VECTOR HNSW` — the argument list cut off right after the algorithm keyword — indexed one past the end of argv and panicked. The panic ran on a shard diff --git a/CLAUDE.md b/CLAUDE.md index 4c753daf..85592c0c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -83,7 +83,7 @@ orb run -m moon-dev bash -c 'sudo apt-get update -qq && sudo apt-get install -y - `scripts/bench-compare.sh` — Moon vs Redis side-by-side (all commands, pipeline 1–128, data 8B–64KB). Use `--requests 200000` for stable numbers. - `scripts/bench-production.sh` — 10 production scenarios (session, ratelimit, leaderboard, cache, queue, hash, connections, datasizes, memory, pipeline). - `scripts/bench-resources.sh` — RSS/memory bench. Starts a fresh server per row (RSS is a high-water mark; delta within one process is unreliable). -- `scripts/test-commands.sh` — 190 tests across 13 categories (correctness vs Redis + throughput). `--skip-bench` for fast correctness-only. +- `scripts/test-commands.sh` — 504 rows across 13 categories (correctness vs Redis + throughput). `--skip-bench` for fast correctness-only. Honours `PORT_REDIS`/`PORT_RUST`, and refuses to start if either port is already held (it would otherwise compare against somebody else's server). - `scripts/test-consistency.sh` — 132 data-consistency tests across 1/4/12 shard configs. - All scripts run inside `moon-dev` and need `redis-server` / `redis-benchmark` on PATH. diff --git a/scripts/test-commands.sh b/scripts/test-commands.sh index 755bf131..2eb56275 100755 --- a/scripts/test-commands.sh +++ b/scripts/test-commands.sh @@ -88,30 +88,64 @@ done log() { echo "[$(date '+%H:%M:%S')] $*" >&2; } cleanup() { + # `rc` is captured and re-exited at the end (moon#679): a trap's exit + # status replaces the script's, and the `[[ -n ... ]] && kill` lines below + # return 1 whenever a PID is unset, which would report a clean run as a + # failure (and a failed run as whatever the last kill happened to return). + local rc=$? log "Cleaning up..." [[ -n "${RUST_PID:-}" ]] && kill "$RUST_PID" 2>/dev/null; wait "$RUST_PID" 2>/dev/null || true [[ -n "${REDIS_PID:-}" ]] && kill "$REDIS_PID" 2>/dev/null; wait "$REDIS_PID" 2>/dev/null || true pkill -f "redis-server.*${PORT_REDIS}" 2>/dev/null || true pkill -f "moon.*${PORT_RUST}" 2>/dev/null || true + [[ -n "${MOON_DATA_DIR:-}" ]] && rm -rf "$MOON_DATA_DIR" + return "$rc" } trap cleanup EXIT +# Multiline "A appears somewhere before B" match, reading stdin. +# +# Replaces `grep -Pzo "(?s)A.*B"` (moon#679). That idiom is GNU-only: on a +# macOS host `grep` is ugrep, which rejects -P and exits 2 -- and because the +# rows compared command output rather than exit status, that 2 leaked through +# as `got: 2`, so every AGG-*/TAG-*/NUMERIC-04 row reported a moon answer of +# "2" that moon never gave. Flattening newlines to spaces and using a plain +# BRE works on GNU grep, BSD grep and ugrep alike. +# +# The match is deliberately as loose as the idiom it replaces (`open.*5` also +# matches "open" followed by a later "25"); tightening it is a separate change +# from making it run at all. +spans() { + tr '\n' ' ' | grep -q "$1" +} + +# The `|| true` on every client wrapper is load-bearing (moon#679). +# +# `redis-cli` exits non-zero when it cannot reach the server, and under +# `set -euo pipefail` that makes `VAR=$(mcli ...)` end the whole run. So the +# moment a server died -- which is exactly when the suite has something +# important to say -- the script stopped without printing its summary, and the +# operator saw a truncated log rather than "N rows failed". A row that gets an +# empty answer should FAIL and let the run continue to the totals. +# +# Nothing branches on these functions' exit status (checked: no `if mcli ...` +# and no `mcli ... &&` anywhere in this file), so swallowing it costs nothing. rcli() { # Run redis-cli against Redis - redis-cli -p "$PORT_REDIS" "$@" 2>/dev/null + redis-cli -p "$PORT_REDIS" "$@" 2>/dev/null || true } mcli() { # Run redis-cli against moon - redis-cli -p "$PORT_RUST" "$@" 2>/dev/null + redis-cli -p "$PORT_RUST" "$@" 2>/dev/null || true } rcli_raw() { - redis-cli -p "$PORT_REDIS" --no-auth-warning "$@" 2>/dev/null + redis-cli -p "$PORT_REDIS" --no-auth-warning "$@" 2>/dev/null || true } mcli_raw() { - redis-cli -p "$PORT_RUST" --no-auth-warning "$@" 2>/dev/null + redis-cli -p "$PORT_RUST" --no-auth-warning "$@" 2>/dev/null || true } # Compare redis-cli output between Redis and moon @@ -344,10 +378,14 @@ assert_bench() { TOTAL=$((TOTAL + 1)) local raw rps raw=$(redis-benchmark -p "$PORT_RUST" -n 5000 -c 50 $cmd "$@" 2>&1 | tr '\r' '\n') - rps=$(echo "$raw" | grep -i "requests per second" | tail -1 | awk '{for(i=1;i<=NF;i++) if($i ~ /^[0-9]/ && $(i+1) ~ /requests/) print $i}' | sed 's/,//g') + # `|| true`: when redis-benchmark prints no summary line (a command it + # cannot drive, a server that went away) grep exits 1, pipefail propagates + # it, and set -e ends the run -- before the `-z "$rps"` fallback just below + # ever gets to do its job (moon#679). + rps=$(echo "$raw" | grep -i "requests per second" | tail -1 | awk '{for(i=1;i<=NF;i++) if($i ~ /^[0-9]/ && $(i+1) ~ /requests/) print $i}' | sed 's/,//g' || true) # Fallback: try -q mode format "COMMAND: NNN.NN requests per second" if [[ -z "$rps" ]]; then - rps=$(echo "$raw" | grep "requests per second" | tail -1 | sed 's/.*: \([0-9.]*\) requests.*/\1/' | sed 's/,//g') + rps=$(echo "$raw" | grep "requests per second" | tail -1 | sed 's/.*: \([0-9.]*\) requests.*/\1/' | sed 's/,//g' || true) fi if [[ -n "$rps" ]] && [[ "$rps" != "0" ]] && [[ "$rps" != "0.00" ]]; then BENCH_PASS=$((BENCH_PASS + 1)) @@ -377,7 +415,50 @@ log "=== Moon Command Coverage Test ===" if [[ "$SKIP_BUILD" == "false" ]]; then log "Building moon..." - cargo build --release --features text-index --quiet 2>/dev/null + # Do NOT swallow stderr here (moon#679). This was + # `cargo build ... --quiet 2>/dev/null`, so a failed build ended the run + # under `set -e` with the log reading "Building moon..." and nothing else + # -- no error, no summary, no clue. Whatever cargo has to say about why it + # could not build the binary this suite is about to test, the operator + # needs to see. + if ! cargo build --release --features text-index --quiet; then + echo "FATAL: cargo build failed; the suite has nothing to test." >&2 + exit 2 + fi +fi + +# Refuse to run on a port somebody else is already listening on (moon#679). +# +# Without this the suite silently measures the WRONG SERVER: a leftover moon or +# redis from another run answers, every row compares against it, and the report +# looks authoritative. This is not hypothetical -- it produced a full run of +# `MOONERR diskfull` failures that had nothing to do with the code under test, +# because a moon from an unrelated session (a different binary, different +# flags, different data dir) held the port. Judge by the LISTENER, not by +# whether a PING comes back: a PING coming back is exactly the symptom. +require_free_port() { + local port="$1" what="$2" owner + # `|| true`: lsof exits 1 when nothing matches -- i.e. when the port is + # FREE, the common case -- and pipefail + set -e would end the run there. + owner=$(lsof -nP -iTCP:"$port" -sTCP:LISTEN 2>/dev/null | awk 'NR==2{print $1" (pid "$2")"}' || true) + if [[ -n "$owner" ]]; then + echo "FATAL: port $port ($what) is already held by $owner." >&2 + echo " This suite would have compared against that process instead of" >&2 + echo " the server it started. Stop it, or re-run with a free port:" >&2 + echo " PORT_REDIS= PORT_RUST= $0" >&2 + exit 2 + fi + # Explicit: without it the function's status is that of the `if` test + # above, which is 1 when the port IS free -- and under `set -e` a function + # returning 1 at top level ends the script. The first draft of this very + # guard did exactly that, killing the run with no output at all. + return 0 +} +require_free_port "$PORT_RUST" "moon" +# A plain `[[ ... ]] && cmd` would be the same trap: false test, status 1, +# `set -e` ends the run. Use an if. +if [[ "$MOON_ONLY" == "false" ]]; then + require_free_port "$PORT_REDIS" "redis" fi if [[ "$MOON_ONLY" == "false" ]]; then @@ -387,7 +468,16 @@ if [[ "$MOON_ONLY" == "false" ]]; then fi log "Starting moon on port $PORT_RUST ($SHARDS shards)..." -RUST_LOG=warn "$RUST_BINARY" --port "$PORT_RUST" --shards "$SHARDS" --protected-mode no & +# A fresh --dir per run (moon#679). Without it moon treats the CWD as its data +# directory: it writes `appendonlydir/` and `moon.lock` into the repo root, and +# -- because FLUSHALL deliberately keeps index DEFINITIONS -- it reloads the +# previous run's FT indexes on start. The visible symptom is `FT.CREATE basic` +# failing with `ERR Index already exists` on the second and every later run, +# which makes the suite non-reproducible: a clean checkout passes, the same +# checkout run twice does not. +MOON_DATA_DIR=$(mktemp -d "${TMPDIR:-/tmp}/moon-test-commands.XXXXXX") +RUST_LOG=warn "$RUST_BINARY" --port "$PORT_RUST" --shards "$SHARDS" --protected-mode no \ + --dir "$MOON_DATA_DIR" --disk-free-min-pct 0 & RUST_PID=$! sleep 1 @@ -1073,7 +1163,7 @@ if should_run "transaction"; then # Test MULTI/EXEC via pipe (using \n not \r\n for redis-cli pipe mode) TOTAL=$((TOTAL + 1)) - tx_moon=$(printf 'MULTI\nSET tx:k1 v1\nSET tx:k2 v2\nGET tx:k1\nEXEC\n' | redis-cli -p "$PORT_RUST" 2>/dev/null) + tx_moon=$(printf 'MULTI\nSET tx:k1 v1\nSET tx:k2 v2\nGET tx:k1\nEXEC\n' | redis-cli -p "$PORT_RUST" 2>/dev/null || true) if echo "$tx_moon" | grep -q "v1"; then PASS=$((PASS + 1)) else @@ -1084,7 +1174,7 @@ if should_run "transaction"; then # DISCARD (must be inside MULTI) TOTAL=$((TOTAL + 1)) - tx_discard=$(printf 'MULTI\nDISCARD\n' | redis-cli -p "$PORT_RUST" 2>/dev/null) + tx_discard=$(printf 'MULTI\nDISCARD\n' | redis-cli -p "$PORT_RUST" 2>/dev/null || true) if echo "$tx_discard" | grep -q "OK"; then PASS=$((PASS + 1)) else @@ -1247,16 +1337,49 @@ if should_run "vector"; then echo "=== VECTOR SEARCH COMMANDS ===" mcli FLUSHALL >/dev/null 2>&1 - # FT.CREATE — create a vector index - assert_moon "FT.CREATE basic" "OK" FT.CREATE myidx ON HASH PREFIX 1 doc: SCHEMA embedding VECTOR FLAT 6 DIM 4 DISTANCE_METRIC L2 TYPE FLOAT32 + # FT.CREATE — create a vector index. + # + # HNSW, not FLAT (moon#679). This row asked for `VECTOR FLAT` and expected + # `OK` from the day it was written, but moon has only ever implemented + # HNSW: `ERR expected HNSW algorithm` has been in `ft_create.rs` since #27. + # So the row never passed, and the five rows below it -- FT.INFO, + # FT.SEARCH, FT.DROPINDEX, FT.INFO-after-drop -- all fell over behind it + # on an index that was never created. Four of the six "vector" rows in this + # category were reporting a failure that told you nothing. + # + # Note this is not a redis-parity assertion and never was: the + # `redis-server` this suite runs against has no query engine, so FT.* is + # `unknown command` on the Redis side. These are moon-only rows, which is + # why nothing flagged the expectation as unsupported. + assert_moon "FT.CREATE basic" "OK" FT.CREATE myidx ON HASH PREFIX 1 doc: SCHEMA embedding VECTOR HNSW 6 DIM 4 DISTANCE_METRIC L2 TYPE FLOAT32 + + # FLAT is a real RediSearch algorithm moon does not implement. Assert the + # refusal explicitly rather than letting it hide inside a row that expected + # success -- a gap that is asserted is a gap someone can find. + TOTAL=$((TOTAL + 1)); FT_FLAT=$(mcli FT.CREATE flatidx ON HASH PREFIX 1 f: SCHEMA v VECTOR FLAT 6 DIM 4 DISTANCE_METRIC L2 TYPE FLOAT32 2>&1) + if echo "$FT_FLAT" | grep -q "expected HNSW algorithm"; then + PASS=$((PASS + 1)); echo " PASS: FT.CREATE VECTOR FLAT refused (moon implements HNSW only)" + else + FAIL=$((FAIL + 1)); echo " FAIL: FT.CREATE VECTOR FLAT expected 'expected HNSW algorithm', got: $FT_FLAT" + fi + + # moon#681: a VECTOR clause truncated after the algorithm keyword used to + # panic the shard thread and abort the whole process. The row that matters + # is the PING after it -- an error reply is fine, a dead server is not. + TOTAL=$((TOTAL + 1)); FT_TRUNC=$(mcli FT.CREATE truncidx ON HASH PREFIX 1 t: SCHEMA v VECTOR HNSW 2>&1) + if [ "$(mcli PING 2>&1)" = "PONG" ]; then + PASS=$((PASS + 1)); echo " PASS: moon#681 truncated FT.CREATE answers ($FT_TRUNC) and the server survives" + else + FAIL=$((FAIL + 1)); echo " FAIL: moon#681 truncated FT.CREATE killed the server (reply was: $FT_TRUNC)" + fi # FT.INFO — index metadata TOTAL=$((TOTAL + 1)); FT_INFO=$(mcli FT.INFO myidx 2>&1) if echo "$FT_INFO" | grep -q "myidx"; then PASS=$((PASS + 1)); echo " PASS: FT.INFO returns index name"; else FAIL=$((FAIL + 1)); echo " FAIL: FT.INFO returns index name"; fi # Insert vectors via HSET (auto-indexed) — use python3 to avoid null byte stripping in bash - python3 -c "import struct,sys; sys.stdout.buffer.write(struct.pack('<4f',1.0,0.0,0.0,0.0))" | redis-cli -x -p "$PORT_RUST" HSET doc:1 embedding >/dev/null 2>&1 - python3 -c "import struct,sys; sys.stdout.buffer.write(struct.pack('<4f',0.0,1.0,0.0,0.0))" | redis-cli -x -p "$PORT_RUST" HSET doc:2 embedding >/dev/null 2>&1 + python3 -c "import struct,sys; sys.stdout.buffer.write(struct.pack('<4f',1.0,0.0,0.0,0.0))" | redis-cli -x -p "$PORT_RUST" HSET doc:1 embedding >/dev/null 2>&1 || true + python3 -c "import struct,sys; sys.stdout.buffer.write(struct.pack('<4f',0.0,1.0,0.0,0.0))" | redis-cli -x -p "$PORT_RUST" HSET doc:2 embedding >/dev/null 2>&1 || true # FT.SEARCH — verify command doesn't error (redis-cli can't pass binary args directly) TOTAL=$((TOTAL + 1)); FT_SEARCH=$(mcli FT.SEARCH myidx "*" 2>&1) @@ -1435,7 +1558,7 @@ if should_run "vector"; then # KNNFILT-01 baseline: the prefilter is applied at all (150 only). TOTAL=$((TOTAL + 1)) KF_INC=$(mcli FT.SEARCH knnfilt '@vt:[100 200]=>[KNN 4 @vec $q]' PARAMS 2 q "$KF_VEC" DIALECT 2 2>&1) - KF_N=$(echo "$KF_INC" | grep -c '^kf:') + KF_N=$(echo "$KF_INC" | grep -c '^kf:' || true) if [ "$KF_N" -eq 1 ]; then PASS=$((PASS + 1)); echo " PASS: KNNFILT-01 prefilter [100 200] -> 1 key" else @@ -1445,7 +1568,7 @@ if should_run "vector"; then # KNNFILT-02: exclusive upper bound. Pre-fix this returned 3 (unfiltered). TOTAL=$((TOTAL + 1)) KF_EXCL=$(mcli FT.SEARCH knnfilt '@vt:[100 (300]=>[KNN 4 @vec $q]' PARAMS 2 q "$KF_VEC" DIALECT 2 2>&1) - KF_NE=$(echo "$KF_EXCL" | grep -c '^kf:') + KF_NE=$(echo "$KF_EXCL" | grep -c '^kf:' || true) if [ "$KF_NE" -eq 2 ]; then PASS=$((PASS + 1)); echo " PASS: KNNFILT-02 prefilter [100 (300] -> 2 keys (exclusive honoured)" else @@ -1492,9 +1615,9 @@ if should_run "vector"; then # every category after VECTOR SEARCH (MQ, txn_kv, eviction, benchmark) and # the result summary never executed. Same class as moon#634. printf '\x00\x00\x80\x3f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' \ - | redis-cli -x -p "$PORT_RUST" HSET dd:1 vec >/dev/null 2>&1 + | redis-cli -x -p "$PORT_RUST" HSET dd:1 vec >/dev/null 2>&1 || true printf '\x00\x00\x80\x3f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' \ - | redis-cli -x -p "$PORT_RUST" HSET dd:2 vec >/dev/null 2>&1 + | redis-cli -x -p "$PORT_RUST" HSET dd:2 vec >/dev/null 2>&1 || true # Verify documents exist TOTAL=$((TOTAL + 1)); DD_EXISTS=$(mcli EXISTS dd:1 dd:2 2>&1) @@ -1510,7 +1633,7 @@ if should_run "vector"; then # Test case insensitivity: create another index assert_moon_ok "FT.CREATE dd_test2" FT.CREATE ddtest2 ON HASH PREFIX 1 dd2: SCHEMA vec VECTOR HNSW 6 DIM 4 TYPE FLOAT32 DISTANCE_METRIC L2 printf '\x00\x00\x80\x3f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' \ - | redis-cli -x -p "$PORT_RUST" HSET dd2:1 vec >/dev/null 2>&1 + | redis-cli -x -p "$PORT_RUST" HSET dd2:1 vec >/dev/null 2>&1 || true # Drop with lowercase dd flag assert_moon_ok "FT.DROPINDEX dd (lowercase)" FT.DROPINDEX ddtest2 dd @@ -1521,7 +1644,7 @@ if should_run "vector"; then # Test without DD — documents should remain assert_moon_ok "FT.CREATE no_dd_test" FT.CREATE noddtest ON HASH PREFIX 1 ndd: SCHEMA vec VECTOR HNSW 6 DIM 4 TYPE FLOAT32 DISTANCE_METRIC L2 printf '\x00\x00\x80\x3f\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00' \ - | redis-cli -x -p "$PORT_RUST" HSET ndd:1 vec >/dev/null 2>&1 + | redis-cli -x -p "$PORT_RUST" HSET ndd:1 vec >/dev/null 2>&1 || true assert_moon_ok "FT.DROPINDEX no DD" FT.DROPINDEX noddtest @@ -1561,7 +1684,7 @@ if should_run "vector"; then FT_TEXT_INFO=$(mcli FT.INFO textidx 2>&1) TOTAL=$((TOTAL + 1)) - TEXT_NUM_DOCS=$(echo "$FT_TEXT_INFO" | grep -A1 "num_docs" | tail -1 | tr -d '[:space:]') + TEXT_NUM_DOCS=$(echo "$FT_TEXT_INFO" | grep -A1 "num_docs" | tail -1 | tr -d '[:space:]' || true) if [ -n "$TEXT_NUM_DOCS" ] && [ "$TEXT_NUM_DOCS" != "0" ] && [ "$TEXT_NUM_DOCS" -gt 0 ] 2>/dev/null; then PASS=$((PASS + 1)); echo " PASS: FT.INFO text num_docs = $TEXT_NUM_DOCS (should be > 0)" else @@ -1569,7 +1692,7 @@ if should_run "vector"; then fi TOTAL=$((TOTAL + 1)) - TEXT_NUM_TERMS=$(echo "$FT_TEXT_INFO" | grep -A1 "num_terms" | tail -1 | tr -d '[:space:]') + TEXT_NUM_TERMS=$(echo "$FT_TEXT_INFO" | grep -A1 "num_terms" | tail -1 | tr -d '[:space:]' || true) if [ -n "$TEXT_NUM_TERMS" ] && [ "$TEXT_NUM_TERMS" != "0" ] && [ "$TEXT_NUM_TERMS" -gt 0 ] 2>/dev/null; then PASS=$((PASS + 1)); echo " PASS: FT.INFO text num_terms = $TEXT_NUM_TERMS (should be > 0)" else @@ -1662,7 +1785,7 @@ if should_run "vector"; then # 6. LIMIT clause: FT.SEARCH textidx "document" LIMIT 0 1 — returns exactly 1 doc entry TOTAL=$((TOTAL + 1)) FT_LIMIT=$(mcli FT.SEARCH textidx "document" LIMIT 0 1 2>&1) - FT_LIMIT_DOC_COUNT=$(echo "$FT_LIMIT" | grep -c "doc:") + FT_LIMIT_DOC_COUNT=$(echo "$FT_LIMIT" | grep -c "doc:" || true) if [ "$FT_LIMIT_DOC_COUNT" -le 1 ] && echo "$FT_LIMIT" | grep -q "doc:"; then PASS=$((PASS + 1)); echo " PASS: FT.SEARCH LIMIT 0 1 returns at most 1 result" else @@ -1945,7 +2068,7 @@ if should_run "vector"; then # AGG-01: GROUPBY + COUNT — assert specific counts (open=5 closed=2) TOTAL=$((TOTAL + 1)) AGG_COUNT=$(mcli FT.AGGREGATE aggidx '*' GROUPBY 1 @status REDUCE COUNT 0 AS cnt SORTBY 2 @cnt DESC 2>&1) - if echo "$AGG_COUNT" | grep -Pzo "(?s)open.*5" >/dev/null && echo "$AGG_COUNT" | grep -Pzo "(?s)closed.*2" >/dev/null; then + if echo "$AGG_COUNT" | spans "open.*5" && echo "$AGG_COUNT" | spans "closed.*2"; then PASS=$((PASS + 1)); echo " PASS: AGG-01 GROUPBY+COUNT (open=5 closed=2)" else FAIL=$((FAIL + 1)); echo " FAIL: AGG-01 expected open=5 closed=2, got: $AGG_COUNT" @@ -1954,7 +2077,7 @@ if should_run "vector"; then # AGG-02: GROUPBY @priority (high=3 low=4) TOTAL=$((TOTAL + 1)) AGG_PRIORITY=$(mcli FT.AGGREGATE aggidx '*' GROUPBY 1 @priority REDUCE COUNT 0 AS cnt SORTBY 2 @cnt DESC 2>&1) - if echo "$AGG_PRIORITY" | grep -Pzo "(?s)low.*4" >/dev/null && echo "$AGG_PRIORITY" | grep -Pzo "(?s)high.*3" >/dev/null; then + if echo "$AGG_PRIORITY" | spans "low.*4" && echo "$AGG_PRIORITY" | spans "high.*3"; then PASS=$((PASS + 1)); echo " PASS: AGG-02 GROUPBY @priority (high=3 low=4)" else FAIL=$((FAIL + 1)); echo " FAIL: AGG-02 expected high=3 low=4, got: $AGG_PRIORITY" @@ -1965,7 +2088,7 @@ if should_run "vector"; then TOTAL=$((TOTAL + 1)) AGG_SUM=$(mcli FT.AGGREGATE aggidx '*' GROUPBY 1 @status REDUCE SUM 1 @score AS total 2>&1) # Expected: status=open → 10+20+30+40+50=150; status=closed → 60+70=130. - if echo "$AGG_SUM" | grep -Pzo "(?s)open.*150" >/dev/null && echo "$AGG_SUM" | grep -Pzo "(?s)closed.*130" >/dev/null; then + if echo "$AGG_SUM" | spans "open.*150" && echo "$AGG_SUM" | spans "closed.*130"; then PASS=$((PASS + 1)); echo " PASS: AGG-03 GROUPBY+SUM exact (open=150 closed=130)" else FAIL=$((FAIL + 1)); echo " FAIL: AGG-03 expected open=150 closed=130, got: $AGG_SUM" @@ -1982,7 +2105,7 @@ if should_run "vector"; then TOTAL=$((TOTAL + 1)) AGG_MIN=$(mcli FT.AGGREGATE aggidx '*' GROUPBY 1 @status REDUCE MIN 1 @score AS min_score 2>&1) # Expected: status=open → min=10; status=closed → min=60. - if echo "$AGG_MIN" | grep -Pzo "(?s)open.*10" >/dev/null && echo "$AGG_MIN" | grep -Pzo "(?s)closed.*60" >/dev/null; then + if echo "$AGG_MIN" | spans "open.*10" && echo "$AGG_MIN" | spans "closed.*60"; then PASS=$((PASS + 1)); echo " PASS: AGG-03 GROUPBY+MIN exact (open=10 closed=60)" else FAIL=$((FAIL + 1)); echo " FAIL: AGG-03 expected MIN open=10 closed=60, got: $AGG_MIN" @@ -1991,7 +2114,7 @@ if should_run "vector"; then TOTAL=$((TOTAL + 1)) AGG_MAX=$(mcli FT.AGGREGATE aggidx '*' GROUPBY 1 @status REDUCE MAX 1 @score AS max_score 2>&1) # Expected: status=open → max=50; status=closed → max=70. - if echo "$AGG_MAX" | grep -Pzo "(?s)open.*50" >/dev/null && echo "$AGG_MAX" | grep -Pzo "(?s)closed.*70" >/dev/null; then + if echo "$AGG_MAX" | spans "open.*50" && echo "$AGG_MAX" | spans "closed.*70"; then PASS=$((PASS + 1)); echo " PASS: AGG-03 GROUPBY+MAX exact (open=50 closed=70)" else FAIL=$((FAIL + 1)); echo " FAIL: AGG-03 expected MAX open=50 closed=70, got: $AGG_MAX" @@ -2028,7 +2151,7 @@ if should_run "vector"; then # TAG-01: @status:{open} GROUPBY @priority — expect high=3 low=2 TOTAL=$((TOTAL + 1)) AGG_TAG_FILTER=$(mcli FT.AGGREGATE aggidx '@status:{open}' GROUPBY 1 @priority REDUCE COUNT 0 AS cnt SORTBY 2 @cnt DESC 2>&1) - if echo "$AGG_TAG_FILTER" | grep -Pzo "(?s)high.*3" >/dev/null && echo "$AGG_TAG_FILTER" | grep -Pzo "(?s)low.*2" >/dev/null; then + if echo "$AGG_TAG_FILTER" | spans "high.*3" && echo "$AGG_TAG_FILTER" | spans "low.*2"; then PASS=$((PASS + 1)); echo " PASS: TAG-01 @status:{open} GROUPBY @priority (high=3 low=2)" else FAIL=$((FAIL + 1)); echo " FAIL: TAG-01 expected high=3 low=2, got: $AGG_TAG_FILTER" @@ -2037,7 +2160,7 @@ if should_run "vector"; then # TAG-02: FT.SEARCH @status:{open} — 5 keys TOTAL=$((TOTAL + 1)) SEARCH_TAG=$(mcli FT.SEARCH aggidx '@status:{open}' LIMIT 0 10 2>&1) - HIT_COUNT=$(echo "$SEARCH_TAG" | grep -c '^agg:') + HIT_COUNT=$(echo "$SEARCH_TAG" | grep -c '^agg:' || true) if [ "$HIT_COUNT" -eq 5 ]; then PASS=$((PASS + 1)); echo " PASS: TAG-02 FT.SEARCH @status:{open} returned 5 keys" else @@ -2047,7 +2170,7 @@ if should_run "vector"; then # TAG-03: @Status:{open} (mixed-case field) must match @status:{open} TOTAL=$((TOTAL + 1)) SEARCH_TAG_CASE=$(mcli FT.SEARCH aggidx '@Status:{open}' LIMIT 0 10 2>&1) - HIT_COUNT_CASE=$(echo "$SEARCH_TAG_CASE" | grep -c '^agg:') + HIT_COUNT_CASE=$(echo "$SEARCH_TAG_CASE" | grep -c '^agg:' || true) if [ "$HIT_COUNT_CASE" -eq 5 ]; then PASS=$((PASS + 1)); echo " PASS: TAG-03 case-insensitive field @Status:{open} → 5 keys" else @@ -2081,7 +2204,7 @@ if should_run "vector"; then # NUMERIC-01: @score:[20 40] inclusive — agg:2 agg:3 agg:4 = 3 keys TOTAL=$((TOTAL + 1)) NUM_INC=$(mcli FT.SEARCH aggidx '@score:[20 40]' LIMIT 0 10 2>&1) - HIT_NUM=$(echo "$NUM_INC" | grep -c '^agg:') + HIT_NUM=$(echo "$NUM_INC" | grep -c '^agg:' || true) if [ "$HIT_NUM" -eq 3 ]; then PASS=$((PASS + 1)); echo " PASS: NUMERIC-01 @score:[20 40] inclusive → 3 keys" else @@ -2091,7 +2214,7 @@ if should_run "vector"; then # NUMERIC-02: exclusive bounds — (20 40] → agg:3 agg:4 = 2 keys TOTAL=$((TOTAL + 1)) NUM_EXCL=$(mcli FT.SEARCH aggidx '@score:[(20 40]' LIMIT 0 10 2>&1) - HIT_EXCL=$(echo "$NUM_EXCL" | grep -c '^agg:') + HIT_EXCL=$(echo "$NUM_EXCL" | grep -c '^agg:' || true) if [ "$HIT_EXCL" -eq 2 ]; then PASS=$((PASS + 1)); echo " PASS: NUMERIC-02 @score:[(20 40] exclusive-low → 2 keys" else @@ -2101,7 +2224,7 @@ if should_run "vector"; then # NUMERIC-03: full range [-inf +inf] → all 7 keys TOTAL=$((TOTAL + 1)) NUM_FULL=$(mcli FT.SEARCH aggidx '@score:[-inf +inf]' LIMIT 0 20 2>&1) - HIT_FULL=$(echo "$NUM_FULL" | grep -c '^agg:') + HIT_FULL=$(echo "$NUM_FULL" | grep -c '^agg:' || true) if [ "$HIT_FULL" -eq 7 ]; then PASS=$((PASS + 1)); echo " PASS: NUMERIC-03 @score:[-inf +inf] → 7 keys" else @@ -2112,7 +2235,7 @@ if should_run "vector"; then # agg:1..3 (scores 10,20,30) all open → cnt=3 on status=open only TOTAL=$((TOTAL + 1)) NUM_AGG=$(mcli FT.AGGREGATE aggidx '@score:[10 30]' GROUPBY 1 @status REDUCE COUNT 0 AS cnt 2>&1) - if echo "$NUM_AGG" | grep -Pzo "(?s)open.*3" >/dev/null; then + if echo "$NUM_AGG" | spans "open.*3"; then PASS=$((PASS + 1)); echo " PASS: NUMERIC-04 FT.AGGREGATE @score:[10 30] GROUPBY status (open=3)" else FAIL=$((FAIL + 1)); echo " FAIL: NUMERIC-04 expected open=3, got: $NUM_AGG" @@ -2152,18 +2275,22 @@ if should_run "vector"; then pkill -f 'moon --port 6411' 2>/dev/null || true pkill -f 'moon --port 6414' 2>/dev/null || true sleep 1 - ./target/release/moon --port 6411 --shards 1 --protected-mode no > /tmp/moon-6411.log 2>&1 & - ./target/release/moon --port 6414 --shards 4 --protected-mode no > /tmp/moon-6414.log 2>&1 & + # Fresh dirs here too -- these two would otherwise reload `nidx` from the + # repo root and report a cross-shard "match" that came from disk. + N7_DIR1=$(mktemp -d "${TMPDIR:-/tmp}/moon-n7-1.XXXXXX") + N7_DIR2=$(mktemp -d "${TMPDIR:-/tmp}/moon-n7-4.XXXXXX") + ./target/release/moon --port 6411 --shards 1 --protected-mode no --dir "$N7_DIR1" --disk-free-min-pct 0 > /tmp/moon-6411.log 2>&1 & + ./target/release/moon --port 6414 --shards 4 --protected-mode no --dir "$N7_DIR2" --disk-free-min-pct 0 > /tmp/moon-6414.log 2>&1 & sleep 2 for PORT in 6411 6414; do - redis-cli -p $PORT FT.CREATE nidx ON HASH PREFIX 1 n: SCHEMA status TAG score NUMERIC > /dev/null 2>&1 + redis-cli -p $PORT FT.CREATE nidx ON HASH PREFIX 1 n: SCHEMA status TAG score NUMERIC > /dev/null 2>&1 || true for i in $(seq 0 19); do - redis-cli -p $PORT HSET n:$i status open score $i > /dev/null 2>&1 + redis-cli -p $PORT HSET n:$i status open score $i > /dev/null 2>&1 || true done done sleep 1 - N1=$(redis-cli -p 6411 FT.SEARCH nidx '@score:[5 15]' LIMIT 0 100 2>&1 | grep '^n:' | sort) - N4=$(redis-cli -p 6414 FT.SEARCH nidx '@score:[5 15]' LIMIT 0 100 2>&1 | grep '^n:' | sort) + N1=$(redis-cli -p 6411 FT.SEARCH nidx '@score:[5 15]' LIMIT 0 100 2>&1 | grep '^n:' | sort || true) + N4=$(redis-cli -p 6414 FT.SEARCH nidx '@score:[5 15]' LIMIT 0 100 2>&1 | grep '^n:' | sort || true) if [ "$N1" = "$N4" ] && [ -n "$N1" ]; then COUNT_N=$(echo "$N1" | wc -l | tr -d ' ') PASS=$((PASS + 1)); echo " PASS: NUMERIC-07 1-shard and 4-shard return identical keys for @score:[5 15] (count=$COUNT_N)" @@ -2304,7 +2431,7 @@ if should_run "temporal"; then mcli GRAPH.CREATE testgraph >/dev/null 2>&1 ADDNODE_OUT=$(mcli GRAPH.ADDNODE testgraph :TestLabel 2>&1) # Extract numeric node_id from ADDNODE response (format: "(integer) " or just "") - NODE_ID=$(echo "$ADDNODE_OUT" | grep -oE '[0-9]+' | head -1) + NODE_ID=$(echo "$ADDNODE_OUT" | grep -oE '[0-9]+' | head -1 || true) if [[ -n "$NODE_ID" ]]; then TOTAL=$((TOTAL + 1)) INV_OK=$(mcli TEMPORAL.INVALIDATE "$NODE_ID" NODE testgraph 2>&1) @@ -2406,10 +2533,10 @@ print(f"ERR_MSG={err_msg}") PYEOF ) - AS_OF_COUNT=$(echo "$FT_ASOF_OUT" | grep '^AS_OF_COUNT=' | cut -d= -f2) - AS_OF_KEYS=$(echo "$FT_ASOF_OUT" | grep '^AS_OF_KEYS=' | cut -d= -f2) - LATEST_COUNT=$(echo "$FT_ASOF_OUT" | grep '^LATEST_COUNT=' | cut -d= -f2) - ERR_MSG=$(echo "$FT_ASOF_OUT" | grep '^ERR_MSG=' | cut -d= -f2-) + AS_OF_COUNT=$(echo "$FT_ASOF_OUT" | grep '^AS_OF_COUNT=' | cut -d= -f2 || true) + AS_OF_KEYS=$(echo "$FT_ASOF_OUT" | grep '^AS_OF_KEYS=' | cut -d= -f2 || true) + LATEST_COUNT=$(echo "$FT_ASOF_OUT" | grep '^LATEST_COUNT=' | cut -d= -f2 || true) + ERR_MSG=$(echo "$FT_ASOF_OUT" | grep '^ERR_MSG=' | cut -d= -f2- || true) TOTAL=$((TOTAL + 1)) if [[ "$AS_OF_COUNT" == "1" && "$AS_OF_KEYS" == "as:1" ]]; then @@ -2481,10 +2608,10 @@ print(f"POST_KEYS={','.join(sorted(post_keys))}") PYEOF ) - INSIDE_COUNT=$(echo "$FT_TXN_OUT" | grep '^INSIDE_COUNT=' | cut -d= -f2) - INSIDE_KEYS=$(echo "$FT_TXN_OUT" | grep '^INSIDE_KEYS=' | cut -d= -f2) - POST_COUNT=$(echo "$FT_TXN_OUT" | grep '^POST_COUNT=' | cut -d= -f2) - POST_KEYS=$(echo "$FT_TXN_OUT" | grep '^POST_KEYS=' | cut -d= -f2) + INSIDE_COUNT=$(echo "$FT_TXN_OUT" | grep '^INSIDE_COUNT=' | cut -d= -f2 || true) + INSIDE_KEYS=$(echo "$FT_TXN_OUT" | grep '^INSIDE_KEYS=' | cut -d= -f2 || true) + POST_COUNT=$(echo "$FT_TXN_OUT" | grep '^POST_COUNT=' | cut -d= -f2 || true) + POST_KEYS=$(echo "$FT_TXN_OUT" | grep '^POST_KEYS=' | cut -d= -f2 || true) # Inside TXN: must see tx:a (pre-TXN) but NOT tx:b (post-snapshot). TOTAL=$((TOTAL + 1)) @@ -2521,9 +2648,9 @@ if should_run "temporal"; then # rate the older direct edge pays lambda * age_seconds and the fresh # detour wins. Real wall-clock sleep creates the age gap. mcli GRAPH.CREATE decayg >/dev/null 2>&1 - DECAY_A=$(mcli GRAPH.ADDNODE decayg Person name A 2>&1 | grep -oE '[0-9]+' | head -1) - DECAY_B=$(mcli GRAPH.ADDNODE decayg Person name B 2>&1 | grep -oE '[0-9]+' | head -1) - DECAY_C=$(mcli GRAPH.ADDNODE decayg Person name C 2>&1 | grep -oE '[0-9]+' | head -1) + DECAY_A=$(mcli GRAPH.ADDNODE decayg Person name A 2>&1 | grep -oE '[0-9]+' | head -1 || true) + DECAY_B=$(mcli GRAPH.ADDNODE decayg Person name B 2>&1 | grep -oE '[0-9]+' | head -1 || true) + DECAY_C=$(mcli GRAPH.ADDNODE decayg Person name C 2>&1 | grep -oE '[0-9]+' | head -1 || true) mcli GRAPH.ADDEDGE decayg "$DECAY_A" "$DECAY_C" KNOWS WEIGHT 1.0 >/dev/null 2>&1 sleep 2