The Typesense sink always writes with action=emplace, which merges into an
existing document. Any key not present in the payload keeps its old value, so a
field that disappears from the source row stays in the index and keeps matching
in search.
Stored document:
{ "id": "1", "name": "Shoe", "attrs": { "color": "red", "size": 42 } }
The row is edited to drop color, and the sink sends:
{ "id": "1", "name": "Shoe", "attrs": { "size": 42 } }
attrs.color is still "red" in the index, and attrs.color:=red still
matches it.
action=upsert replaces the whole document and is what CDC semantics want, but
it's unreachable: params: %{action: "emplace"} is a literal in
lib/sequin/sinks/typesense/client.ex, and Consumers.TypesenseSink has no
action field for sequin.yml to set.
Emitting explicit nulls instead isn't a general fix as it requires knowing
which keys the stored document has, which a transform over a single CDC row
doesn't.
Ask
An optional sink setting, defaulting to emplace so nothing changes for
existing users:
destination:
type: typesense
import_action: upsert # default: emplace
Sequin v0.14.6, unchanged on main. Happy to open a PR if you're open to it.
The Typesense sink always writes with
action=emplace, which merges into anexisting document. Any key not present in the payload keeps its old value, so a
field that disappears from the source row stays in the index and keeps matching
in search.
Stored document:
{ "id": "1", "name": "Shoe", "attrs": { "color": "red", "size": 42 } }The row is edited to drop
color, and the sink sends:{ "id": "1", "name": "Shoe", "attrs": { "size": 42 } }attrs.coloris still"red"in the index, andattrs.color:=redstillmatches it.
action=upsertreplaces the whole document and is what CDC semantics want, butit's unreachable:
params: %{action: "emplace"}is a literal inlib/sequin/sinks/typesense/client.ex, andConsumers.TypesenseSinkhas noactionfield forsequin.ymlto set.Emitting explicit
nulls instead isn't a general fix as it requires knowingwhich keys the stored document has, which a transform over a single CDC row
doesn't.
Ask
An optional sink setting, defaulting to
emplaceso nothing changes forexisting users:
Sequin v0.14.6, unchanged on
main. Happy to open a PR if you're open to it.