Skip to content

Typesense sink hardcodes action=emplace, so omitted fields persist in the index #2153

Description

@osaton

The Typesense sink always writes with action=emplace, which merges into an
existing document. Any key not present in the payload keeps its old value, so a
field that disappears from the source row stays in the index and keeps matching
in search.

Stored document:

{ "id": "1", "name": "Shoe", "attrs": { "color": "red", "size": 42 } }

The row is edited to drop color, and the sink sends:

{ "id": "1", "name": "Shoe", "attrs": { "size": 42 } }

attrs.color is still "red" in the index, and attrs.color:=red still
matches it.

action=upsert replaces the whole document and is what CDC semantics want, but
it's unreachable: params: %{action: "emplace"} is a literal in
lib/sequin/sinks/typesense/client.ex, and Consumers.TypesenseSink has no
action field for sequin.yml to set.

Emitting explicit nulls instead isn't a general fix as it requires knowing
which keys the stored document has, which a transform over a single CDC row
doesn't.

Ask

An optional sink setting, defaulting to emplace so nothing changes for
existing users:

destination:
  type: typesense
  import_action: upsert   # default: emplace

Sequin v0.14.6, unchanged on main. Happy to open a PR if you're open to it.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions