Skip to content

fix(channels): archive group messages before the pending buffer drops them - #1514

Merged
clark-cant merged 1 commit into
nextlevelbuilder:devfrom
minhdang03:fix/pending-message-archive
Aug 15, 2026
Merged

fix(channels): archive group messages before the pending buffer drops them#1514
clark-cant merged 1 commit into
nextlevelbuilder:devfrom
minhdang03:fix/pending-message-archive

Conversation

@minhdang03

Copy link
Copy Markdown
Contributor

channel_pending_messages is a buffer, not an archive. Rows leave it on two paths, both plain deletes:

  • the bot being mentioned hands the whole buffer to the agent, then Clear() wipes the key;
  • LLM compaction replaces old rows with a summary.

For group capture that buffer holds the only stored copy of the raw text, so a single mention or one compaction pass permanently destroyed every message nothing had read yet. Observed in production: a compaction run erased ~4.5 days of a Zalo group's history, and a later mention erased the rest along with the summary that replaced it.

Change

  • New table channel_message_archive (migration 97; SQLite schema 60).
  • DeleteByKey, Compact and DeleteStale copy rows into the archive inside the same transaction as the delete, tagged with archive_reason (consumed, compacted, stale).
  • Archived rows keep their original id, so a replayed delete is a no-op.
  • New ListArchivedByKey(channel, key, since, limit) for consumers that need full history; tenant-scoped like every other store read.

Wiring lives in the store layer, so every channel (Telegram, Discord, Slack, Feishu, WhatsApp, Zalo) and the HTTP compact endpoint are covered with no call-site changes.

Tests

  • 4 SQLite unit tests (internal/store/sqlitestore/pending_message_archive_test.go)
  • 4 PG integration tests (tests/integration/pending_message_archive_test.go), run against a clean pgvector instance migrating 1 → 97

Covers clear, compaction, idempotency across compact-then-clear, the since filter, and tenant scoping.

go build and go vet clean under both the default and sqliteonly tags. The 6 pre-existing Ollama failures in internal/http reproduce on a clean origin/dev checkout and are unrelated.

… them

channel_pending_messages is a buffer, not an archive. Rows were deleted
outright on two paths: the bot being mentioned hands the buffer to the
agent and clears the key, and LLM compaction replaces old rows with a
summary. For group capture that buffer held the only copy of the raw
text, so a single mention or compaction pass destroyed days of messages
nothing had read yet.

Copy every row into channel_message_archive inside the same transaction
as the delete, tagged with the reason (consumed, compacted, stale).
Archived rows keep their original id, so a replayed delete is a no-op.
Add ListArchivedByKey so consumers that need full history read the
archive instead of the buffer.

@clark-cant clark-cant left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved: archive-before-delete is transactionally implemented for PostgreSQL and SQLite, tenant-scoped reads are covered, migration/schema parity is present, duplicate gate is clear, and all required CI checks pass. No critical or important findings.

@clark-cant
clark-cant merged commit 1d68a21 into nextlevelbuilder:dev Aug 15, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants