FT.SEARCH <vector-only-index> "*" still cannot enumerate — the open half of #693.
#693 made * the match-all query. It is answered by the inverted index, which is
where the document registry (TextIndex::doc_id_to_key) lives, so it now works on every
index with a TEXT, TAG or NUMERIC field — including mixed VECTOR+TEXT schemas.
An index built from VECTOR fields alone has no inverted index at all: FT.CREATE
only constructs a TextIndex when the schema has at least one TEXT/TAG/NUMERIC field
(src/command/vector_search/ft_create.rs). So * there falls through to
run_text_query, finds no text index, and answers ERR no such index — for an index
that plainly exists.
Measured
moon at 296a81ac + the #690/#691/#693 branch, --shards 1, host build:
FT.CREATE myidx ON HASH PREFIX 1 doc: SCHEMA embedding VECTOR HNSW 6 DIM 4 DISTANCE_METRIC L2 TYPE FLOAT32
FT.SEARCH myidx "*" -> ERR no such index # the index exists; FT._LIST lists it
Before #693 the same call answered ERR invalid KNN query syntax. Both are wrong; neither
tells the user their index has no way to be enumerated.
For contrast, on an index with an inverted schema it now works, verified at --shards 4
(so the scatter path is covered, not just the local one):
FT.CREATE txt ON HASH PREFIX 1 t: SCHEMA title TEXT body TEXT
8 x HSET t:N ...
FT.SEARCH txt "*" LIMIT 0 0 -> 8
Why it was not done in #693
The registry to enumerate exists and is suitable: VectorIndex.key_hash_to_key is
maintained as a live map — src/vector/store.rs:2395 prunes it on delete, with a
comment saying so explicitly ("prune the key-hash maps so they track LIVE keys, not
historical inserts"). BucketedKeyMap::iter() walks it.
What makes this its own change rather than a tail of #693 is the routing, not the data:
is_text_query("*") is now true, so * reaches the text engine at all four
FT.SEARCH routing sites. A vector fallback therefore has to sit behind the text
lookup, at a point that can see both stores.
- There are three local handler call sites (
handler_single, handler_monoio/ft.rs,
handler_sharded/ft.rs), each inside a with_shard closure that does have
s.vector_store in scope — so those are reachable.
- But the multi-shard path is
scatter_text_search -> run_text_query_on_index,
which is handed a &TextIndex and has no vector store at all. A vector-only * at
--shards > 1 needs its own scatter + merge, and the merge has to agree with the
reply shape the local path produces.
That last point is the real work: a fix that only covers the local path would pass at
--shards 1 and silently return nothing at --shards 4, which is worse than the current
honest error.
Suggested direction
Give the vector engine a match_all(index, db) -> Vec<(key_hash, key)> over the live key
map, then wire it through the same two-level structure the text path already has — local
in the three handler sites, scattered in a scatter_* sibling — and verify at BOTH
--shards 1 and --shards 4 before closing. Tombstoned-but-not-yet-pruned entries need
checking too: the prune at store.rs:2395 covers the delete path, but segment tombstones
(tombstone_key_in_holder, same function) are a separate mechanism and it is worth
confirming the two cannot disagree.
Coverage
scripts/test-commands.sh row FT.SEARCH "*" on a VECTOR-only index is red and names
this issue. It asserts the RediSearch behaviour rather than moon's current error, so it
goes green on its own when this lands. The passing half is covered by the sibling row
FT.SEARCH "*" enumerates a TEXT index (3 docs).
FT.SEARCH <vector-only-index> "*"still cannot enumerate — the open half of #693.#693 made
*the match-all query. It is answered by the inverted index, which iswhere the document registry (
TextIndex::doc_id_to_key) lives, so it now works on everyindex with a TEXT, TAG or NUMERIC field — including mixed VECTOR+TEXT schemas.
An index built from VECTOR fields alone has no inverted index at all:
FT.CREATEonly constructs a
TextIndexwhen the schema has at least one TEXT/TAG/NUMERIC field(
src/command/vector_search/ft_create.rs). So*there falls through torun_text_query, finds no text index, and answersERR no such index— for an indexthat plainly exists.
Measured
moon at
296a81ac+ the #690/#691/#693 branch,--shards 1, host build:Before #693 the same call answered
ERR invalid KNN query syntax. Both are wrong; neithertells the user their index has no way to be enumerated.
For contrast, on an index with an inverted schema it now works, verified at
--shards 4(so the scatter path is covered, not just the local one):
Why it was not done in #693
The registry to enumerate exists and is suitable:
VectorIndex.key_hash_to_keyismaintained as a live map —
src/vector/store.rs:2395prunes it on delete, with acomment saying so explicitly ("prune the key-hash maps so they track LIVE keys, not
historical inserts").
BucketedKeyMap::iter()walks it.What makes this its own change rather than a tail of #693 is the routing, not the data:
is_text_query("*")is nowtrue, so*reaches the text engine at all fourFT.SEARCH routing sites. A vector fallback therefore has to sit behind the text
lookup, at a point that can see both stores.
handler_single,handler_monoio/ft.rs,handler_sharded/ft.rs), each inside awith_shardclosure that does haves.vector_storein scope — so those are reachable.scatter_text_search->run_text_query_on_index,which is handed a
&TextIndexand has no vector store at all. A vector-only*at--shards > 1needs its own scatter + merge, and the merge has to agree with thereply shape the local path produces.
That last point is the real work: a fix that only covers the local path would pass at
--shards 1and silently return nothing at--shards 4, which is worse than the currenthonest error.
Suggested direction
Give the vector engine a
match_all(index, db) -> Vec<(key_hash, key)>over the live keymap, then wire it through the same two-level structure the text path already has — local
in the three handler sites, scattered in a
scatter_*sibling — and verify at BOTH--shards 1and--shards 4before closing. Tombstoned-but-not-yet-pruned entries needchecking too: the prune at
store.rs:2395covers the delete path, but segment tombstones(
tombstone_key_in_holder, same function) are a separate mechanism and it is worthconfirming the two cannot disagree.
Coverage
scripts/test-commands.shrowFT.SEARCH "*" on a VECTOR-only indexis red and namesthis issue. It asserts the RediSearch behaviour rather than moon's current error, so it
goes green on its own when this lands. The passing half is covered by the sibling row
FT.SEARCH "*" enumerates a TEXT index (3 docs).