Skip to content

fix(xchat): a KeyChange refreshes its sender's public-key roster - #191

Closed
salch-cred wants to merge 2 commits into
NousResearch:mainfrom
salch-cred:fix/xchat-keychange-verification
Closed

salch-cred wants to merge 2 commits into
NousResearch:mainfrom
salch-cred:fix/xchat-keychange-verification

Conversation

@salch-cred

Copy link
Copy Markdown
Contributor

fix(xchat): a KeyChange refreshes its sender's public-key roster

Root cause

Signing keys are fetched only for unknown senders (senders -= self._known_senders), and _register_signing_keys is only given the page's messages. When another device enables X Chat encryption it registers a second identity and signs the resulting KeyChange with the new version's signing key ? but the sender is already known, so that key never reaches the XDK roster. The SDK rejects the change with no matching signing key, no conversation key is extracted, and every following message fails with no matching key found until restart.

Fix

  • Decode each KeyChange blob for its sender_id (_key_event_senders, best-effort) and refresh exactly those senders regardless of _known_senders; keys are still fetched for all message senders as before.
  • Deduplicate the pushed roster by (user_id, public_key_version), so repeated key changes do not grow it, and keep pushing the full roster because the SDK store is replaced wholesale.

Tests

tests/test_xchat_adapter.py:

  • test_a_keychange_refreshes_a_known_senders_roster: sender known with v3, phone registers v4 and sends a KeyChange ? roster becomes {3, 4}. Red on main (24d49a4: roster stays {3}), green here.
  • test_the_roster_does_not_grow_on_repeat_keychanges: two polls, one version ? still one entry. Red on main only because the refresh never happens.

test_xchat_adapter.py + test_xchat_api.py: 53 passed.

Draft: the reporter's live X account cannot be driven from this host; the offline fakes exercise the adapter path, and the SDK-level ordering the issue describes (set_signing_keys before decrypt_events) is pinned by the existing batch test.

Closes #173.

@dacheah

dacheah commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Independent reproduction of #173 from a live install, in case it strengthens #191.

Setup: xchat 0.1.0, chatxdk 0.4.1, Linux x86_64, Python 3.14.9. The counterparty account re-initialised encrypted DMs on its own device, registering a third identity version — GET /2/users/{id}/public_keys then returned three records, oldest first. The adapter connected and polled normally, and every inbound event failed to verify:

WARNING hermes_plugins.xchat.adapter: [xchat] decrypt failed conv=… event=…
  Crypto error: Signature verification failed: Message signature could not be
  verified: signature missing or no matching signing key

Five events, all of them, with no recovery over several minutes — consistent with _register_signing_keys skipping senders already in _known_senders. Restarting the gateway, which clears that set and re-fetches every version, restored decryption immediately; that is the same effect the refresh= path in this PR should give without a restart. I have not run this PR itself, so treat this as supporting evidence for the diagnosis rather than a test of the patch.

Scope note, and no overlap intended: this PR touches _register_signing_keys and does not go near _initiate_conversation. The outbound handshake still selects records[0], the oldest identity, and X refuses a conversation key built on anything but the account's latest version:

400 Invalid identity key: one or more participant keys do not use the latest public key version.

So a recipient with more than one registered identity cannot be messaged at all — the conversation key never seeds, and their client falls back to an unencrypted thread, which this adapter does not read. I have a separate issue and patch staged for that rather than widening this one.

A second device enabling X Chat encryption registers a new identity and rotates the conversation key; the KeyChange is signed with that new version's signing key. Signing keys were only fetched for senders not already known, so the new version never entered the XDK roster, every KeyChange failed verification ("no matching signing key"), no conversation key was extracted, and all later messages stayed undecryptable until restart (NousResearch#173).

Decrypt the KeyChange blobs for their sender ids and fetch those senders unconditionally, deduplicating the roster by (user, version) so repeats do not grow it.
@salch-cred
salch-cred force-pushed the fix/xchat-keychange-verification branch from 4e64b8d to 1b8347a Compare November 15, 2026 05:52
@salch-cred

Copy link
Copy Markdown
Contributor Author

Rebased onto main (3b721f9). Conflict was the same README bullet both sides rewrote: kept main's inbound-media-cap wording for Media and merged the KeyChange sentence into Senders. xchat/tests/test_xchat_adapter.py: 32 passed / 12 failed, identical to main's 30 passed / 12 failed (the 12 are pre-existing here), so the two new tests are the only delta.

@dacheah

dacheah commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

Confirmed against the current head of main, not just the PR's rebase base.

main @ 3b721f9, xchat/adapter.py:816: senders -= self._known_senders is still present, and _register_signing_keys still derives its sender set from the page's messages alone, so a KeyChange from an already-known sender still never triggers a key fetch. The defect described in #173 is live in main today.

No overlap with #220 either. That PR (merged 6 Nov, +13/-1 in xchat/adapter.py) changes one expression inside _initiate_conversation — records[0] to the record with the highest public_key_version — and does not touch _register_signing_keys or _known_senders. So the rebase resolves the README conflict in text only; the two fixes stay independent and this one is still needed.

Live-install evidence for the inbound failure is in my earlier comment above: three identity versions on the counterparty, every inbound event failing verification, and decryption restored immediately by a restart that cleared _known_senders.

On the test counts: I could not reproduce the 12 failures on either ref, so naming them would not help a reviewer. Under the workflow's own conditions — hermes-agent checkout on PYTHONPATH at fd8a6c41, Python 3.14.9, chatxdk==0.4.1, set up exactly as .github/workflows/tests.yml does — xchat/tests/test_xchat_adapter.py gives 42 passed / 0 failed on main @ 3b721f9 and 44 passed / 0 failed at 1b8347a, the +2 being the two new tests, with the whole xchat/tests directory at 62 passed. The xchat CI job agrees, green on ubuntu, macOS and Windows at that head. So the delta is 42 to 44 from a clean baseline; I would state it that way rather than subtracting from a failing one, in case the 12 are local to your environment.

KeyChange blobs are Thrift-encoded (checked against chatxdk 0.4.1's
prepare_conversation_key_change output), so json.loads(b64decode(blob)) never
yields a sender_id and the refresh set was always empty in production; the
tests only passed because they fed JSON blobs. Refresh the conversation's
participants (both ids of a 1:1, the page's senders, and the account itself)
once per KeyChange blob not seen before.
@teknium1

Copy link
Copy Markdown
Contributor

Thank you @salch-cred! This landed on main through #446, which stacked the 34 reviewed official-plugin PRs on current main so their shared files (CHANGELOGs, versions, the README grid) conflicted only once. Your commits are kept with your authorship; this PR's tip is 594b485. To get it now: hermes plugins install NousResearch/hermes-official-plugins#<plugin> --force; hermes plugins update follows once the catalog pin moves (in progress).

@teknium1 teknium1 closed this Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

xchat: conversation keys never extracted after the account enables X Chat encryption (PIN) on another device — KeyChange signature verification fails

3 participants