Skip to content

[P1] Close the entity deduplication race in resolveEntityCore #79

Description

@jusso-dev

Found during adversarial review of #55 (PR #77). Not a blocker for that PR, which merged with the two organisation-isolation defects fixed.

Problem

resolveEntityCore (src/lib/investigations/entities-core.ts) deduplicates with a read-then-conditionally-insert sequence rather than a transaction. The entities uniqueness constraint is (organisationId, type, canonicalKey), and canonicalKey is derived only from identifiers[0] when no existing identifier matches.

Two concurrent calls resolving the same real-world entity with non-overlapping identifier sets in different orders — for example one connector reporting [device_id, upn] and another reporting [upn, mailbox_smtp] — can both observe "not found" and each insert a row with a different canonicalKey. The entity_identifiers constraint (organisationId, kind, value) then silently no-ops via onConflictDoNothing for whichever call loses, leaving one duplicate entity missing the shared identifier that would have linked them.

This defeats the "entities deduplicate per organisation" acceptance criterion from #55 under real parallel ingestion.

Why the existing tests miss it

scripts/test-investigations-core.ts exercises resolution strictly sequentially (await first; await second). There is no Promise.all anywhere in that file, so the concurrency claim in #55 is not actually covered for this path.

Suggested approach

  • Resolve and insert inside a single transaction, relying on the unique constraint to serialise rather than application-level checking.
  • Retry on unique violation and re-resolve, so the loser of the race adopts the winner's entity.
  • Consider deriving canonicalKey deterministically from the sorted identifier set rather than identifiers[0], so ordering cannot change the key.

Acceptance criteria

  • Concurrent resolution of the same entity with differently-ordered, partially-overlapping identifier sets yields exactly one entity row.
  • Identifier rows are never silently dropped by onConflictDoNothing in a way that orphans a duplicate.
  • A regression test drives genuine parallelism (Promise.all), not sequential awaits.
  • Organisation isolation is preserved.

Metadata

Metadata

Assignees

No one assigned

    Labels

    execution:readyNo unresolved issue dependency blocks starting this workpriority:p1High: core workflow or operational usability gap

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions