Skip to content

FT.SEARCH "*" cannot enumerate a VECTOR-only index — the open half of #693 #695

Description

@TinDang97

FT.SEARCH <vector-only-index> "*" still cannot enumerate — the open half of #693.

#693 made * the match-all query. It is answered by the inverted index, which is
where the document registry (TextIndex::doc_id_to_key) lives, so it now works on every
index with a TEXT, TAG or NUMERIC field — including mixed VECTOR+TEXT schemas.

An index built from VECTOR fields alone has no inverted index at all: FT.CREATE
only constructs a TextIndex when the schema has at least one TEXT/TAG/NUMERIC field
(src/command/vector_search/ft_create.rs). So * there falls through to
run_text_query, finds no text index, and answers ERR no such index — for an index
that plainly exists.

Measured

moon at 296a81ac + the #690/#691/#693 branch, --shards 1, host build:

FT.CREATE myidx ON HASH PREFIX 1 doc: SCHEMA embedding VECTOR HNSW 6 DIM 4 DISTANCE_METRIC L2 TYPE FLOAT32
FT.SEARCH myidx "*"      -> ERR no such index      # the index exists; FT._LIST lists it

Before #693 the same call answered ERR invalid KNN query syntax. Both are wrong; neither
tells the user their index has no way to be enumerated.

For contrast, on an index with an inverted schema it now works, verified at --shards 4
(so the scatter path is covered, not just the local one):

FT.CREATE txt ON HASH PREFIX 1 t: SCHEMA title TEXT body TEXT
8 x HSET t:N ...
FT.SEARCH txt "*" LIMIT 0 0   -> 8

Why it was not done in #693

The registry to enumerate exists and is suitable: VectorIndex.key_hash_to_key is
maintained as a live map — src/vector/store.rs:2395 prunes it on delete, with a
comment saying so explicitly ("prune the key-hash maps so they track LIVE keys, not
historical inserts"). BucketedKeyMap::iter() walks it.

What makes this its own change rather than a tail of #693 is the routing, not the data:

  1. is_text_query("*") is now true, so * reaches the text engine at all four
    FT.SEARCH routing sites. A vector fallback therefore has to sit behind the text
    lookup, at a point that can see both stores.
  2. There are three local handler call sites (handler_single, handler_monoio/ft.rs,
    handler_sharded/ft.rs), each inside a with_shard closure that does have
    s.vector_store in scope — so those are reachable.
  3. But the multi-shard path is scatter_text_search -> run_text_query_on_index,
    which is handed a &TextIndex and has no vector store at all. A vector-only * at
    --shards > 1 needs its own scatter + merge, and the merge has to agree with the
    reply shape the local path produces.

That last point is the real work: a fix that only covers the local path would pass at
--shards 1 and silently return nothing at --shards 4, which is worse than the current
honest error.

Suggested direction

Give the vector engine a match_all(index, db) -> Vec<(key_hash, key)> over the live key
map, then wire it through the same two-level structure the text path already has — local
in the three handler sites, scattered in a scatter_* sibling — and verify at BOTH
--shards 1 and --shards 4 before closing. Tombstoned-but-not-yet-pruned entries need
checking too: the prune at store.rs:2395 covers the delete path, but segment tombstones
(tombstone_key_in_holder, same function) are a separate mechanism and it is worth
confirming the two cannot disagree.

Coverage

scripts/test-commands.sh row FT.SEARCH "*" on a VECTOR-only index is red and names
this issue. It asserts the RediSearch behaviour rather than moon's current error, so it
goes green on its own when this lands. The passing half is covered by the sibling row
FT.SEARCH "*" enumerates a TEXT index (3 docs).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions