From 738e77582e3dede1b12537624a205724a0c065d3 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Wed, 15 Jul 2026 16:11:56 +0800
Subject: [PATCH 01/47] docs: design cache resilience completion

---
 ...7-15-cache-resilience-completion-design.md | 334 ++++++++++++++++++
 1 file changed, 334 insertions(+)
 create mode 100644 docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md

diff --git a/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
new file mode 100644
index 0000000..79e7bde
--- /dev/null
+++ b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
@@ -0,0 +1,334 @@
+# ChatNow 缓存韧性补全设计
+
+> **日期**: 2026-07-15
+> **基线**: `origin/3.0-dev@1cad99a`
+> **关联 Issue**: #35、#37、#38、#45、#48
+> **测试架构**: PR #49 / #54 定义的纯 Go 五层测试体系
+
+## 1. 目标与范围
+
+本设计补全 ChatNow 3.0 缓存链路在 Redis 故障、高并发缓存回源和集中
+过期场景下的韧性。目标规模沿用现有架构约束：峰值 5000 msg/s、单服务
+2–8 个实例、最大 2000 人群。
+
+验收目标：
+
+- Redis 熔断打开后，调用在 10–50ms 内快速短路，不再逐请求等待 2 秒。
+- Redis 故障期间限流仍生效，不允许无界 fail-open。
+- 缓存型数据可跳过 Redis 回源 RPC/DB；Redis 真相源失败时明确返回不可用。
+- UserInfo 热路径对 Identity RPC 的回源次数降低至少 95%。
+- 同一冷 key 的高并发请求只触发一次进程内回源，不形成 thundering herd。
+- 固定 TTL key 全面应用抖动，避免服务重启后的集中失效。
+- RedisMutex 竞争采用带 jitter 的有界指数退避，降低热点轮询压力。
+
+范围包含：
+
+- `RedisClient` 进程内熔断与快速失败。
+- `RateLimiter` 本地分片令牌桶降级。
+- UserInfo L1/L2/RPC cache-aside 读写路径。
+- Session、Status、Codes、DeviceSet、UnackedPush 等 TTL 抖动补全。
+- RedisMutex 指数退避与 jitter。
+- 与新 Go 测试框架一致的功能、可靠性和性能验证。
+- 与鉴权模块共享 Redis 熔断状态的接口边界。
+
+不包含：
+
+- Redis Cluster 拓扑、代理、服务网格或外部熔断服务改造。
+- 用 DB 或本地状态伪造 SeqGen、UnackedPush 等 Redis 真相源。
+- 完整 JWT 吊销策略重构；#48 的鉴权安全策略单独处理。
+- 跨进程同步熔断状态或本地限流状态。
+- 与缓存问题无关的 DAO、RPC 和部署重构。
+
+## 2. 设计原则
+
+1. **故障域本地化**：每个进程独立熔断，Redis 故障不占满请求线程和连接池。
+2. **按数据语义降级**：缓存可回源，限流可本地化，真相源必须失败。
+3. **快速路径无锁**：熔断关闭时只做原子状态读取；本地限流按 key 分片。
+4. **标准 cache-aside**：缓存 miss 回源，成功后回填；资料变更后失效。
+5. **有界资源**：本地 bucket、缓存和等待时间均有硬上限。
+6. **不增加常驻后台组件**：恢复探测、bucket 清理都由请求惰性驱动。
+7. **低基数观测**：只记录状态和路径计数，不按 uid、ssid 或 key 打标签。
+
+## 3. 总体架构
+
+### 3.1 RedisClient 轻量熔断器
+
+每个 `RedisClient` 持有一个共享 `RedisCircuitBreaker`。由同一客户端构造的
+Session、Members、RateLimiter、JwtStore 等 DAO 观察同一个状态。
+
+状态只有三种：
+
+- `Closed`：允许请求。连续成功会清空失败计数。
+- `Open`：直接抛出 `RedisCircuitOpen`，不借连接、不访问网络。
+- `HalfOpen`：冷却时间到后，只允许一个请求探测，其余继续快速失败。
+
+转换规则：
+
+1. `Closed` 下连续 3 次连接、I/O、超时或连接池等待异常后进入 `Open`。
+2. `Open` 固定保持 1 秒。
+3. 1 秒后用 CAS 选出一个 `HalfOpen` 探测请求。
+4. 探测成功立即回到 `Closed`；失败重新进入 `Open` 1 秒。
+5. Redis 命令参数、序列化或业务脚本错误不计入熔断失败。
+
+固定 1 秒冷却避免引入自适应窗口和复杂配置，同时每个进程最多每秒产生一个
+恢复探测，对 Redis 故障节点负载可忽略。
+
+Redis 运行时超时调整为：
+
+- `connect_timeout = 50ms`
+- `socket_timeout = 50ms`
+- `pool.wait_timeout = 20ms`
+
+Redis 部署在同机房；该上限远高于正常亚毫秒至数毫秒延迟，又能在故障时快速
+触发熔断。启动时 Redis Cluster 仍依次尝试所有 seed，不改变现有拓扑发现逻辑。
+
+### 3.2 Redis 命令接入
+
+`RedisClient` 的公开命令统一通过两个内部模板执行：
+
+- 有返回值的 `execute(command)`。
+- 无返回值的 `execute_void(command)`。
+
+模板负责：
+
+1. 在借连接前调用 `breaker.before_call()`。
+2. 执行 standalone 或 cluster 命令。
+3. 成功时调用 `breaker.on_success()`。
+4. 仅对连接类异常调用 `breaker.on_failure()`，然后原样抛出。
+
+调用方现有的 try/catch 继续负责业务降级。熔断器不返回虚假默认值，防止把
+“Redis 不可用”误解释成“key 不存在”。
+
+Pipeline 由 RAII 包装器在 `exec()` 时报告成功或失败；创建 pipeline 本身不视为
+一次成功请求。SCAN 和 Lua 也经过相同入口。
+
+### 3.3 按数据语义降级
+
+| 数据类型 | Redis 不可用时行为 |
+|---|---|
+| Members/UserInfo/LastMessage 等缓存 | 跳过 L2，回源 RPC/DB |
+| 缓存写入、失效 | 记录指标后忽略，由 TTL 和后续回填收敛 |
+| RateLimiter | 使用进程内令牌桶 |
+| 幂等 SETNX | 保留现有 DB 唯一约束兜底 |
+| SeqGen、UnackedPush 等真相源 | 返回明确的服务不可用 |
+| JWT 相关 DAO | 获得统一快速失败信号；具体 fail-open/fail-close 不在本次改变 |
+
+## 4. 本地限流降级
+
+### 4.1 数据结构
+
+`LocalRateLimiter` 由 `RateLimiter` 持有，使用 64 个固定 shard。每个 shard 包含：
+
+- 一个 mutex。
+- `unordered_map<string, Bucket>`。
+- bucket 数量计数。
+
+`Bucket` 只保存 `tokens`、`last_refill` 和 `last_seen`。一次请求只哈希并锁定一个
+shard，不存在全局热锁。
+
+### 4.2 算法和边界
+
+- 容量、窗口和补充速率与 Redis Lua 令牌桶一致。
+- Redis 正常时只执行 Redis Lua，不双写本地 bucket。
+- Redis 熔断或调用失败时执行本地 bucket。
+- Redis 恢复后下一次请求自动回到 Redis，不迁移本地临时状态。
+- 全进程最多 65,536 个活跃 bucket。
+- bucket 超过两个窗口未访问时，在后续请求中惰性清理。
+- 达到硬上限且无法清理时，不为陌生 key 创建 bucket，直接拒绝请求。
+
+故障期间无法维持严格跨实例额度，最坏上限为“实例数 × 单实例额度”，但流量
+始终有界。引入服务发现来动态切分额度会增加一致性和可用性耦合，因此不采用。
+
+## 5. UserInfo 三级缓存
+
+### 5.1 Key 与 TTL
+
+- L2 key：`im:user:{uid}`。
+- L2 value：序列化的 UserInfo protobuf。
+- L2 正值 TTL：1 小时，±20% 抖动。
+- L1 正值 TTL：45 秒，±20% 抖动。
+- 空值 sentinel TTL：5 秒，使用相同抖动函数。
+
+### 5.2 单用户读路径
+
+1. 查询 Transmite 现有 `LocalCache<string>`。
+2. L1 命中时解析 protobuf 并返回。
+3. L1 miss 后进入现有 `InflightRegistry` 的 uid 维度 singleflight。
+4. 获得 leader 后 double-check L1。
+5. 查询 `UserInfoCache` L2。
+6. L2 命中则回填 L1；损坏值删除后按 miss 处理。
+7. L2 miss 或熔断时调用 Identity `GetProfile`。
+8. RPC 成功后写 L2 和 L1；确认用户不存在时写 5 秒 sentinel。
+9. RPC/DB 失败不写 sentinel，避免把依赖故障缓存成“不存在”。
+
+Transmite 当前热路径只读取发送者一个 uid，不额外引入没有消费者的批量业务
+流程。
+
+### 5.3 批量接口
+
+`UserInfoCache` 提供 `batch_get` 和 `batch_set`，满足 Message、Conversation 后续
+批量读取场景：
+
+- standalone Redis 使用 MGET 和 pipeline。
+- Redis Cluster 按 hash slot 分组，每组使用同 slot pipeline，避免 CROSSSLOT。
+- 返回命中 map 和 miss uid 列表，调用方可用现有批量 RPC 一次回源。
+- 单次批量大小限制为 2000，与最大群规模一致。
+
+本次只把单用户 Transmite 路径接入该 DAO；批量消费者迁移不在范围内。
+
+### 5.4 失效
+
+Identity 修改昵称、头像、签名或其他 Profile 字段成功后删除 L2 key。删除失败只
+记录指标。各实例 L1 可能继续返回最多约 54 秒旧资料，符合资料展示的最终一致性
+要求。
+
+## 6. TTL 抖动补全
+
+统一使用现有 `randomized_ttl(base)`，范围为基准值 ±20%，且永远返回正值。
+
+需要补全的路径：
+
+- Session：append、touch。
+- Status：append、touch。
+- Codes：append。
+- DeviceSet：add 时设置与 Session 相同的 7 天 TTL；设备活动路径刷新 TTL。
+- UnackedPush：push、bump_score 涉及的 ZSET/HASH 使用同一个随机 TTL 样本，避免
+  两个索引因不同抖动提前分离。
+- 其他仍使用固定常量的缓存写入和续期点。
+
+DeviceSet 不使用短在线 TTL。它的生命周期与登录设备接近；7 天 TTL 防止永久
+常驻，同时避免正常在线设备被几分钟级 TTL 误删。
+
+## 7. RedisMutex 退避
+
+`try_lock` 保持现有 `SET key token NX PX ttl` 和 Lua CAS 解锁协议，仅调整竞争等待：
+
+- 首次失败等待基准 5ms。
+- 后续基准依次为 10ms、20ms，之后保持 20ms 上限。
+- 每次加入 `[0, base/2]` 的 thread-local 随机 jitter。
+- sleep 不超过剩余 timeout。
+- 每次 `try_lock` 调用重置退避状态。
+
+100ms 默认超时下最多产生少量 Redis 轮询，且竞争者不再同频唤醒。
+
+## 8. 错误处理与恢复
+
+- 熔断快速拒绝使用独立异常类型，便于 RateLimiter 精确进入本地降级。
+- DAO 不记录每次快速拒绝，只在熔断状态转换时写日志，避免 Redis 故障造成日志
+  风暴。
+- cache-aside 回源失败时返回原业务依赖错误，不把失败响应写入缓存。
+- half-open 探测由真实业务命令完成，不另起 ping 线程。
+- Redis 恢复不要求服务重启；第一个成功探测关闭熔断器。
+- Redis Cluster 的节点级 failover 仍由 redis-plus-plus 处理；应用熔断只覆盖客户端
+  已无法在时限内完成命令的场景。
+
+## 9. 可观测性
+
+在现有 bvar 指标中增加低基数计数器：
+
+- `redis_circuit_open_total`
+- `redis_circuit_rejected_total`
+- `redis_circuit_recovered_total`
+- `redis_call_failure_total`
+- `rate_limit_local_fallback_total`
+- `rate_limit_local_rejected_total`
+- `user_info_l1_hit_total`
+- `user_info_l2_hit_total`
+- `user_info_rpc_total`
+- `redis_mutex_retry_total`
+
+状态转换日志包含 Closed/Open/HalfOpen 和异常分类，不包含 uid、ssid 或完整 Redis
+key。现有 Prometheus Redis 告警继续使用；新增应用指标用于区分 Redis 故障和业务
+错误。
+
+## 10. 测试设计
+
+测试遵循 PR #49/#54：纯 Go、真实全栈、黑盒行为验证，测试代码是权威，不新增
+C++ gtest 或 Python 合约测试。
+
+### 10.1 L2 功能测试
+
+文件：`tests/func/cache_test.go`，build tag：`func`。
+
+- `FN-CA-01`：首次 UserInfo 回源后存在 L2，重复请求不重复调用 Identity。
+- `FN-CA-02`：并发请求同一 uid，只发生一次回源。
+- `FN-CA-03`：资料修改后 L2 失效，下一次读取获得新值。
+- `FN-CA-04`：Session、Status、Codes、DeviceSet、UnackedPush TTL 位于抖动范围，
+  多个样本不过期于同一秒。
+- `FN-CA-05`：Redis 正常时大量消息请求触发分布式限流。
+
+### 10.2 可靠性测试
+
+文件：`tests/reliability/redis_failover_test.go`，build tag：`reliability`。
+
+`RL-05` 使用 `tests/pkg/chaos` 停止和恢复 Redis：
+
+1. 故障前创建唯一用户和业务数据并预热缓存。
+2. 停止 Redis，触发各目标服务熔断。
+3. 验证缓存型接口可回源，稳定故障期请求延迟不超过 50ms。
+4. 验证突发请求出现本地限流拒绝。
+5. 验证依赖 SeqGen 的路径明确返回不可用。
+6. 恢复 Redis，验证 half-open 成功并自动恢复正常路径。
+
+### 10.3 L4 性能测试
+
+文件：`tests/perf/cache_test.go`，build tag：`perf`。
+
+- `PF-09` 分别记录冷缓存、L2 命中和 L1 命中的吞吐与 p50/p95/p99。
+- 目标负载为 5000 msg/s，报告分配量和每请求耗时。
+- 同一 uid 的热路径 Identity RPC 降幅至少 95%。
+- 200 个并发请求访问同一冷 key 时，只允许一次进程内 RPC 回源。
+- nightly 基线吞吐下降超过 10% 时失败。
+
+### 10.4 测试辅助设施
+
+- `tests/pkg/verify/redis.go`：读取 key、TTL 和 bvar 指标，不包含业务断言。
+- `tests/pkg/chaos/redis.go`：Redis stop/start/wait 封装。
+- 所有用例使用唯一 ID 隔离，可在 PR #49 合并后直接接入 CleanupAll 和 CI。
+- macOS 执行 C++ 构建、Go build/vet/gofmt；故障注入和性能测试在 Linux Docker/CI
+  执行。
+
+## 11. 文件边界
+
+计划新增：
+
+- `common/utils/redis_circuit_breaker.hpp`：熔断状态机。
+- `common/utils/local_rate_limiter.hpp`：分片本地令牌桶。
+- `tests/func/cache_test.go`。
+- `tests/reliability/redis_failover_test.go`。
+- `tests/perf/cache_test.go`。
+- `tests/pkg/verify/redis.go`。
+- `tests/pkg/chaos/redis.go`。
+
+计划修改：
+
+- `common/dao/data_redis.hpp`：RedisClient 接入、UserInfoCache、TTL、RateLimiter。
+- `common/utils/redis_mutex.hpp`：指数退避和 jitter。
+- `common/infra/metrics.hpp`：低基数指标。
+- `common/utils/redis_keys.hpp`：UserInfo L2 key。
+- `transmite/source/transmite_server.h`：UserInfo 三级缓存读路径。
+- `identity/source/identity_server.h`：资料变更后失效 UserInfo L2。
+- 必要的服务 builder/config 文件：构造并注入共享 DAO，不新增外部依赖。
+
+每个新增组件只有一个职责；不将熔断、限流和业务缓存合并成大型基础设施类。
+
+## 12. 发布与回滚
+
+1. 先合入无行为变化的指标和组件。
+2. 接入 RedisClient 熔断与 RateLimiter 降级。
+3. 接入 UserInfo cache-aside 和资料失效。
+4. 补全 TTL 与锁退避。
+5. Linux CI 运行 func、reliability 和 perf；观察熔断误触发与 RPC 降幅。
+
+回滚可以按组件提交逐步进行。熔断器和本地限流没有外部持久状态；关闭相关接入
+即可恢复原行为。UserInfo L2 key 使用独立前缀，回滚后由 TTL 自动清理。
+
+## 13. Issue 覆盖
+
+| Issue | 解决点 |
+|---|---|
+| #35 | RedisClient 熔断、50ms 超时、缓存回源语义 |
+| #37 | UserInfo L1/L2/RPC、singleflight、批量 DAO |
+| #38 | 固定 TTL 全面抖动、DeviceSet 生命周期 |
+| #45 | RedisMutex 指数退避与 jitter |
+| #48 | 统一熔断接口、本地限流降级；JWT 策略明确排除 |

From e3a90ee135447ba80ac3f1fe979dcb80ed2f67c0 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Wed, 15 Jul 2026 16:14:39 +0800
Subject: [PATCH 02/47] docs: clarify clustered user cache batching

---
 .../specs/2026-07-15-cache-resilience-completion-design.md | 7 +++++--
 1 file changed, 5 insertions(+), 2 deletions(-)

diff --git a/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
index 79e7bde..196a182 100644
--- a/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
+++ b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
@@ -143,7 +143,9 @@ shard，不存在全局热锁。
 
 ### 5.1 Key 与 TTL
 
-- L2 key：`im:user:{uid}`。
+- L2 key：`im:user:{bucket}:uid`，`bucket = fnv1a(uid) % 64`。花括号中的
+  bucket 是 Redis Cluster hash tag，使批量读取可按 64 个虚拟分片分组，同时避免
+  全部 UserInfo 聚集到一个 slot。
 - L2 value：序列化的 UserInfo protobuf。
 - L2 正值 TTL：1 小时，±20% 抖动。
 - L1 正值 TTL：45 秒，±20% 抖动。
@@ -170,7 +172,8 @@ Transmite 当前热路径只读取发送者一个 uid，不额外引入没有消
 批量读取场景：
 
 - standalone Redis 使用 MGET 和 pipeline。
-- Redis Cluster 按 hash slot 分组，每组使用同 slot pipeline，避免 CROSSSLOT。
+- Redis Cluster 按 64 个虚拟 bucket 分组；每组 key 共享 hash tag，可安全使用
+  MGET/pipeline，避免 CROSSSLOT，也不依赖 redis-plus-plus 的内部连接池。
 - 返回命中 map 和 miss uid 列表，调用方可用现有批量 RPC 一次回源。
 - 单次批量大小限制为 2000，与最大群规模一致。
 

From 80b52f58b4f526d35163e4df55203e889b05aa2f Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Wed, 15 Jul 2026 16:19:35 +0800
Subject: [PATCH 03/47] docs: plan cache resilience completion

---
 ...-07-15-cache-resilience-completion-plan.md | 763 ++++++++++++++++++
 1 file changed, 763 insertions(+)
 create mode 100644 docs/superpowers/plans/2026-07-15-cache-resilience-completion-plan.md

diff --git a/docs/superpowers/plans/2026-07-15-cache-resilience-completion-plan.md b/docs/superpowers/plans/2026-07-15-cache-resilience-completion-plan.md
new file mode 100644
index 0000000..eece2b8
--- /dev/null
+++ b/docs/superpowers/plans/2026-07-15-cache-resilience-completion-plan.md
@@ -0,0 +1,763 @@
+# Cache Resilience Completion Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Resolve open cache issues #35, #37, #38, #45 and the cache/rate-limit portion of #48 with fast Redis failure isolation, bounded local degradation, UserInfo L2 caching, complete TTL jitter, and contention-safe lock retry.
+
+**Architecture:** A per-`RedisClient` lock-free circuit breaker guards all Redis commands; domain DAOs retain responsibility for semantic fallback. `RateLimiter` falls back to a 64-shard in-process token bucket, while UserInfo follows L1 → L2 → singleflight Identity RPC cache-aside. Tests use the PR #49/#54 pure-Go `func`, `reliability`, and `perf` layers.
+
+**Tech Stack:** C++17, redis-plus-plus, bvar, brpc/protobuf, Go 1.23 `testing` + `testify`, Docker Compose, Redis Cluster 7.2.
+
+## Global Constraints
+
+- Production baseline is exactly `origin/3.0-dev@1cad99a`; do not merge the unmerged PR #49 branch into this branch.
+- Peak target is 5000 msg/s with 2–8 service instances and groups up to 2000 members.
+- Once open, Redis circuit rejection must complete within 10–50ms; no request may retain the old 2s Redis wait.
+- Redis outage must retain bounded rate limiting; Redis-backed truth sources must return unavailable instead of fabricated state.
+- UserInfo hot-path Identity RPC calls must fall by at least 95%.
+- All new tests are Go black-box tests with `func`, `reliability`, or `perf` tags; add no C++ gtest or Python contract tests.
+- Do not add an external circuit-breaker service, background probe thread, service-discovery quota splitter, or new runtime dependency.
+- UserInfo L2 keys use 64 virtual cluster hash buckets: `im:user:{bucket}:uid`, with `bucket = fnv1a(uid) % 64`.
+
+---
+
+## File Structure
+
+**Create:**
+
+- `common/utils/redis_circuit_breaker.hpp` — atomic Closed/Open/HalfOpen state machine.
+- `common/utils/local_rate_limiter.hpp` — bounded 64-shard fallback token bucket.
+- `tests/pkg/chaos/redis.go` — stop/start/wait helpers for the six Redis nodes.
+- `tests/pkg/verify/redis.go` — Redis CLI and bvar read helpers.
+- `tests/func/cache_test.go` — FN-CA-01 through FN-CA-05.
+- `tests/reliability/redis_failover_test.go` — RL-05.
+- `tests/reliability/setup_test.go` — reliability-tag HTTP setup compatible with PR #49.
+- `tests/perf/cache_test.go` — PF-09.
+
+**Modify:**
+
+- `common/dao/data_redis.hpp` — guarded RedisClient commands, UserInfoCache, jittered TTLs, RateLimiter fallback.
+- `common/utils/redis_keys.hpp` — UserInfo bucket/key helpers.
+- `common/utils/redis_mutex.hpp` — bounded exponential backoff with jitter.
+- `common/infra/metrics.hpp` — circuit, fallback, UserInfo, and mutex metrics.
+- `transmite/source/transmite_server.h` — UserInfo cache-aside path and dependency injection.
+- `identity/source/identity_server.h` — invalidate UserInfo L2 after profile update.
+- `tests/Makefile` — reliability target when absent; preserve PR #49-compatible target names.
+
+---
+
+### Task 1: Go Redis chaos and verification helpers
+
+**Files:**
+
+- Create: `tests/pkg/chaos/redis.go`
+- Create: `tests/pkg/verify/redis.go`
+- Modify: `tests/pkg/client/config.go`
+- Modify: `tests/config.yaml`
+
+**Interfaces:**
+
+- Produces: `chaos.StopRedisCluster(testing.TB)`, `chaos.StartRedisCluster(testing.TB)`, `chaos.WaitRedisCluster(testing.TB, time.Duration)`.
+- Produces: `verify.RedisCLI(testing.TB, ...string) string`, `verify.RedisTTL(testing.TB, string) time.Duration`, `verify.BVar(testing.TB, string, string) int64`.
+
+- [ ] **Step 1: Add Redis and service metrics endpoints to test config**
+
+Add stable, environment-overridable fields to `tests/pkg/client/config.go` and values to `tests/config.yaml`:
+
+```go
+type InfraConfig struct {
+	RedisContainer string `yaml:"redis_container"`
+	ComposeDir     string `yaml:"compose_dir"`
+	TransmiteVars  string `yaml:"transmite_vars"`
+}
+
+// Config gains:
+Infra InfraConfig `yaml:"infra"`
+```
+
+```yaml
+infra:
+  redis_container: "redis-node1"
+  compose_dir: ".."
+  transmite_vars: "http://localhost:10004"
+```
+
+- [ ] **Step 2: Implement Docker Compose Redis control**
+
+Create `tests/pkg/chaos/redis.go` with these exact service names and recovery check:
+
+```go
+package chaos
+
+import (
+	"fmt"
+	"os/exec"
+	"testing"
+	"time"
+)
+
+var redisServices = []string{
+	"redis-node1", "redis-node2", "redis-node3",
+	"redis-node4", "redis-node5", "redis-node6",
+}
+
+func compose(t testing.TB, args ...string) {
+	t.Helper()
+	cmd := exec.Command("docker", append([]string{"compose"}, args...)...)
+	cmd.Dir = ".."
+	if out, err := cmd.CombinedOutput(); err != nil {
+		t.Fatalf("docker compose %v: %v: %s", args, err, out)
+	}
+}
+
+func StopRedisCluster(t testing.TB) { compose(t, append([]string{"stop"}, redisServices...)...) }
+func StartRedisCluster(t testing.TB) { compose(t, append([]string{"start"}, redisServices...)...) }
+
+func WaitRedisCluster(t testing.TB, timeout time.Duration) {
+	t.Helper()
+	deadline := time.Now().Add(timeout)
+	for time.Now().Before(deadline) {
+		cmd := exec.Command("docker", "exec", "redis-node1", "redis-cli", "cluster", "info")
+		if out, err := cmd.Output(); err == nil && string(out) != "" {
+			return
+		}
+		time.Sleep(500 * time.Millisecond)
+	}
+	t.Fatalf("redis cluster did not recover within %s", timeout)
+}
+```
+
+- [ ] **Step 3: Implement Redis and bvar verification**
+
+Create `tests/pkg/verify/redis.go`:
+
+```go
+package verify
+
+import (
+	"fmt"
+	"io"
+	"net/http"
+	"os/exec"
+	"strconv"
+	"strings"
+	"testing"
+	"time"
+)
+
+func RedisCLI(t testing.TB, args ...string) string {
+	t.Helper()
+	base := []string{"exec", "redis-node1", "redis-cli", "-c"}
+	out, err := exec.Command("docker", append(base, args...)...).CombinedOutput()
+	if err != nil { t.Fatalf("redis-cli %v: %v: %s", args, err, out) }
+	return strings.TrimSpace(string(out))
+}
+
+func RedisTTL(t testing.TB, key string) time.Duration {
+	seconds, err := strconv.ParseInt(RedisCLI(t, "TTL", key), 10, 64)
+	if err != nil || seconds < 0 { t.Fatalf("invalid TTL for %s: %d (%v)", key, seconds, err) }
+	return time.Duration(seconds) * time.Second
+}
+
+func BVar(t testing.TB, baseURL, name string) int64 {
+	t.Helper()
+	rsp, err := http.Get(fmt.Sprintf("%s/vars/%s", baseURL, name))
+	if err != nil { t.Fatalf("read bvar %s: %v", name, err) }
+	defer rsp.Body.Close()
+	body, err := io.ReadAll(rsp.Body)
+	if err != nil { t.Fatalf("read bvar body %s: %v", name, err) }
+	value, err := strconv.ParseInt(strings.TrimSpace(string(body)), 10, 64)
+	if err != nil { t.Fatalf("parse bvar %s=%q: %v", name, body, err) }
+	return value
+}
+```
+
+- [ ] **Step 4: Compile helper packages**
+
+Run: `cd tests && gofmt -w pkg/chaos/redis.go pkg/verify/redis.go pkg/client/config.go && go test ./pkg/chaos ./pkg/verify ./pkg/client`
+
+Expected: PASS or `[no test files]` for each package, with no compile error.
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add tests/pkg/chaos/redis.go tests/pkg/verify/redis.go tests/pkg/client/config.go tests/config.yaml
+git commit -m "test: add Redis chaos verification helpers"
+```
+
+---
+
+### Task 2: Redis circuit breaker and guarded RedisClient
+
+**Files:**
+
+- Create: `common/utils/redis_circuit_breaker.hpp`
+- Modify: `common/dao/data_redis.hpp`
+- Modify: `common/infra/metrics.hpp`
+- Create: `tests/reliability/setup_test.go`
+- Create: `tests/reliability/redis_failover_test.go`
+- Modify: `tests/Makefile`
+
+**Interfaces:**
+
+- Produces: `RedisCircuitBreaker::Permit before_call()`, `on_success(Permit)`, `on_connection_failure(Permit)`.
+- Produces: `RedisCircuitOpen`, used by semantic fallbacks.
+- `RedisClient` public command signatures remain source-compatible.
+
+- [ ] **Step 1: Write failing RL-05 fast-fail/recovery test**
+
+Create `tests/reliability/setup_test.go` with build tag `reliability`, load `tests/config.yaml`, and initialize the package-level HTTP client exactly like `tests/func/setup_test.go`.
+
+Create `tests/reliability/redis_failover_test.go`:
+
+```go
+//go:build reliability
+
+package reliability_test
+
+import (
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/require"
+
+	"chatnow-tests/pkg/chaos"
+	"chatnow-tests/pkg/client"
+	"chatnow-tests/pkg/fixture"
+	identity "chatnow-tests/proto/chatnow/identity"
+)
+
+// RL-05 | P0 | Redis 熔断后快速失败并自动恢复
+func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) {
+	user, _, _ := fixture.RegisterAndLogin(t, HTTP)
+	t.Cleanup(func() { chaos.StartRedisCluster(t); chaos.WaitRedisCluster(t, 60*time.Second) })
+	chaos.StopRedisCluster(t)
+
+	uid := user.UserID
+	for i := 0; i < 3; i++ {
+		rsp := &identity.GetProfileRsp{}
+		_ = user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{
+			RequestId: client.NewRequestID(), UserId: &uid,
+		}, rsp)
+	}
+
+	started := time.Now()
+	rsp := &identity.GetProfileRsp{}
+	err := user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{
+		RequestId: client.NewRequestID(), UserId: &uid,
+	}, rsp)
+	require.NoError(t, err)
+	require.True(t, rsp.GetHeader().GetSuccess())
+	require.Less(t, time.Since(started), 50*time.Millisecond)
+
+	chaos.StartRedisCluster(t)
+	chaos.WaitRedisCluster(t, 60*time.Second)
+	time.Sleep(1100 * time.Millisecond)
+	rsp = &identity.GetProfileRsp{}
+	require.NoError(t, user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{
+		RequestId: client.NewRequestID(), UserId: &uid,
+	}, rsp))
+	require.True(t, rsp.GetHeader().GetSuccess())
+}
+```
+
+- [ ] **Step 2: Run RL-05 and verify the old implementation fails**
+
+Run on Linux stack: `cd tests && go test -tags=reliability ./reliability/... -run TestRL_RedisCircuitFastFailAndRecovery -v -count=1 -timeout=180s`
+
+Expected before implementation: FAIL because Redis-backed auth calls retain the old 2s timeout and the final request exceeds 50ms.
+
+- [ ] **Step 3: Implement the atomic circuit state machine**
+
+Create `common/utils/redis_circuit_breaker.hpp` with this public shape and constants:
+
+```cpp
+#pragma once
+#include <atomic>
+#include <chrono>
+#include <cstdint>
+#include <stdexcept>
+
+namespace chatnow {
+class RedisCircuitOpen final : public std::runtime_error {
+public: RedisCircuitOpen() : std::runtime_error("redis circuit open") {}
+};
+
+class RedisCircuitBreaker {
+public:
+    enum class State : uint8_t { Closed, Open, HalfOpen };
+    struct Permit { bool probe = false; };
+
+    Permit before_call();
+    void on_success(Permit permit) noexcept;
+    void on_connection_failure(Permit permit) noexcept;
+    State state() const noexcept { return _state.load(std::memory_order_acquire); }
+
+private:
+    static constexpr uint32_t kFailureThreshold = 3;
+    static constexpr int64_t kOpenForMs = 1000;
+    static int64_t now_ms() noexcept;
+    void open() noexcept;
+
+    std::atomic<State> _state{State::Closed};
+    std::atomic<uint32_t> _consecutive_failures{0};
+    std::atomic<int64_t> _open_until_ms{0};
+};
+} // namespace chatnow
+```
+
+Use these inline state transitions:
+
+```cpp
+inline int64_t RedisCircuitBreaker::now_ms() noexcept {
+    return std::chrono::duration_cast<std::chrono::milliseconds>(
+        std::chrono::steady_clock::now().time_since_epoch()).count();
+}
+
+inline void RedisCircuitBreaker::open() noexcept {
+    _open_until_ms.store(now_ms() + kOpenForMs, std::memory_order_release);
+    _state.store(State::Open, std::memory_order_release);
+}
+
+inline RedisCircuitBreaker::Permit RedisCircuitBreaker::before_call() {
+    auto state = _state.load(std::memory_order_acquire);
+    if (state == State::Closed) return {};
+    if (state == State::HalfOpen) throw RedisCircuitOpen();
+    if (now_ms() < _open_until_ms.load(std::memory_order_acquire)) {
+        throw RedisCircuitOpen();
+    }
+    auto expected = State::Open;
+    if (_state.compare_exchange_strong(expected, State::HalfOpen,
+                                       std::memory_order_acq_rel)) {
+        return Permit{true};
+    }
+    throw RedisCircuitOpen();
+}
+
+inline void RedisCircuitBreaker::on_success(Permit) noexcept {
+    _consecutive_failures.store(0, std::memory_order_relaxed);
+    _state.store(State::Closed, std::memory_order_release);
+}
+
+inline void RedisCircuitBreaker::on_connection_failure(Permit permit) noexcept {
+    if (permit.probe ||
+        _consecutive_failures.fetch_add(1, std::memory_order_relaxed) + 1 >=
+            kFailureThreshold) {
+        open();
+    }
+}
+```
+
+- [ ] **Step 4: Guard every RedisClient command**
+
+In `common/dao/data_redis.hpp`, add `_breaker` and private helpers:
+
+```cpp
+template <class F>
+decltype(auto) guarded_(F &&fn) {
+    auto permit = _breaker.before_call();
+    try {
+        decltype(auto) result = std::forward<F>(fn)();
+        _breaker.on_success(permit);
+        return result;
+    } catch (const sw::redis::IoError &) {
+        _breaker.on_connection_failure(permit); throw;
+    } catch (const sw::redis::TimeoutError &) {
+        _breaker.on_connection_failure(permit); throw;
+    } catch (const sw::redis::ClosedError &) {
+        _breaker.on_connection_failure(permit); throw;
+    }
+}
+
+template <class F>
+void guarded_void_(F &&fn) {
+    auto permit = _breaker.before_call();
+    try { std::forward<F>(fn)(); _breaker.on_success(permit); }
+    catch (const sw::redis::IoError &) { _breaker.on_connection_failure(permit); throw; }
+    catch (const sw::redis::TimeoutError &) { _breaker.on_connection_failure(permit); throw; }
+    catch (const sw::redis::ClosedError &) { _breaker.on_connection_failure(permit); throw; }
+}
+```
+
+Route `get`, all four `set` overloads, `del`, `expire`, `incr`, both `sadd` overloads, `smembers`, `srem`, `scard`, `hset`, `hget`, `hdel`, `hkeys`, `hgetall`, `hlen`, both `zadd` overloads, `zrem`, `zrange`, `zrangebyscore`, both `eval` overloads, and `scan` through these helpers. Preserve their existing public signatures.
+
+Add a same-slot multi-get used by UserInfo bucket groups:
+
+```cpp
+template <typename Input, typename Output>
+void mget(Input first, Input last, Output out) {
+    guarded_void_([&] {
+        _rc ? _rc->mget(first, last, out) : _r->mget(first, last, out);
+    });
+}
+```
+
+Add a move-only `RedisPipeline` wrapper exposing `hset`, `set`, `expire`, `get`, and `exec`; it stores the permit and only reports success/failure when `exec()` runs. Change `RedisClient::pipeline` to return the wrapper.
+
+- [ ] **Step 5: Tighten Redis runtime timeouts and add metrics**
+
+Set standalone and cluster factory values to 50ms connect/socket and 20ms pool wait. Add bvar counters named exactly as the design specifies and increment them only on connection failure, open transition, fast rejection, and recovery.
+
+- [ ] **Step 6: Compile and rerun RL-05**
+
+Run: `cmake -S . -B build && cmake --build build -j2`
+
+Expected: all C++ targets compile.
+
+Run on Linux stack: `cd tests && go test -tags=reliability ./reliability/... -run TestRL_RedisCircuitFastFailAndRecovery -v -count=1 -timeout=180s`
+
+Expected: PASS; stable outage request under 50ms and recovery without service restart.
+
+- [ ] **Step 7: Commit**
+
+```bash
+git add common/utils/redis_circuit_breaker.hpp common/dao/data_redis.hpp common/infra/metrics.hpp tests/reliability tests/Makefile
+git commit -m "feat: add fast Redis circuit breaking"
+```
+
+---
+
+### Task 3: Bounded local rate-limit fallback
+
+**Files:**
+
+- Create: `common/utils/local_rate_limiter.hpp`
+- Modify: `common/dao/data_redis.hpp`
+- Modify: `common/infra/metrics.hpp`
+- Modify: `tests/reliability/redis_failover_test.go`
+- Create: `tests/func/cache_test.go`
+
+**Interfaces:**
+
+- Produces: `LocalRateLimiter::allow(key, capacity, window, now)`.
+- `RateLimiter::allow_user` and `allow_session` signatures remain unchanged.
+
+- [ ] **Step 1: Add failing Redis-outage rate-limit assertion**
+
+Extend RL-05 after prewarming a conversation. Send 650 unique messages while Redis is stopped and count responses whose header error message is `rate_limited`. Require the count to be greater than zero. Before implementation it is zero because `RateLimiter::allow` returns true on every Redis exception.
+
+Add `FN-CA-05` to `tests/func/cache_test.go`: send 650 messages with Redis healthy and require at least one rate-limited response.
+
+- [ ] **Step 2: Implement sharded token buckets**
+
+Create `common/utils/local_rate_limiter.hpp`:
+
+```cpp
+#pragma once
+#include <array>
+#include <atomic>
+#include <chrono>
+#include <cstdint>
+#include <cmath>
+#include <cstddef>
+#include <mutex>
+#include <string>
+#include <unordered_map>
+
+namespace chatnow {
+class LocalRateLimiter {
+public:
+    bool allow(const std::string &key, int capacity, int window_sec);
+private:
+    struct Bucket { double tokens; int64_t refill_ms; int64_t seen_ms; };
+    struct Shard { std::mutex mu; std::unordered_map<std::string, Bucket> buckets; };
+    static constexpr size_t kShardCount = 64;
+    static constexpr size_t kMaxBuckets = 65536;
+    std::array<Shard, kShardCount> _shards;
+    std::atomic<size_t> _bucket_count{0};
+    std::atomic<uint64_t> _operations{0};
+};
+} // namespace chatnow
+```
+
+Use steady-clock milliseconds. Refill `elapsed * capacity / (window_sec * 1000)`, cap at capacity, consume one token when available, and update `seen_ms`. Every 1024 operations, remove entries idle for more than two windows from the current shard. At the hard cap, deny unknown keys after cleanup.
+
+- [ ] **Step 3: Wire semantic fallback**
+
+Give `RateLimiter` a `LocalRateLimiter _local`. In the existing catch block replace `return true` with:
+
+```cpp
+metrics::g_rate_limit_local_fallback_total << 1;
+const bool allowed = _local.allow(key_full, max_count, window_sec);
+if (!allowed) metrics::g_rate_limit_local_rejected_total << 1;
+return allowed;
+```
+
+Keep Redis Lua as the only normal path; do not double-write local state.
+
+- [ ] **Step 4: Verify**
+
+Run: `cmake --build build -j2`
+
+Run on Linux stack: `cd tests && go test -tags=func ./func/... -run TestFN_CA_RateLimit -v -count=1 && go test -tags=reliability ./reliability/... -run TestRL_RedisCircuitFastFailAndRecovery -v -count=1 -timeout=180s`
+
+Expected: both healthy Redis and stopped Redis produce bounded rate-limit rejection.
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add common/utils/local_rate_limiter.hpp common/dao/data_redis.hpp common/infra/metrics.hpp tests/func/cache_test.go tests/reliability/redis_failover_test.go
+git commit -m "feat: retain rate limiting during Redis outages"
+```
+
+---
+
+### Task 4: UserInfo L1/L2/RPC cache-aside
+
+**Files:**
+
+- Modify: `common/utils/redis_keys.hpp`
+- Modify: `common/dao/data_redis.hpp`
+- Modify: `common/infra/metrics.hpp`
+- Modify: `transmite/source/transmite_server.h`
+- Modify: `identity/source/identity_server.h`
+- Modify: `tests/func/cache_test.go`
+
+**Interfaces:**
+
+- Produces: `key::user_info_bucket(uid)`, `key::user_info_key(uid)`.
+- Produces: `UserInfoCache::get`, `batch_get`, `set`, `batch_set`, `invalidate` over serialized protobuf strings.
+- `TransmiteServiceImpl` gains `UserInfoCache::ptr` but preserves RPC API.
+
+- [ ] **Step 1: Write failing cache behavior tests**
+
+Add these functions to `tests/func/cache_test.go`:
+
+- `TestFN_CA_UserInfoL2AvoidsRepeatedRPC`: record `user_info_rpc_total`, send 20 messages from one user, require delta ≤ 1 and Redis key existence.
+- `TestFN_CA_UserInfoSingleflight`: start 200 goroutines sending from one cold user, require `user_info_rpc_total` delta ≤ 1.
+- `TestFN_CA_UserInfoInvalidatedAfterProfileUpdate`: warm, update nickname, require old Redis key deleted, send again, then require the stored protobuf reflects the new nickname.
+
+Run on Linux: `cd tests && go test -tags=func ./func/... -run 'TestFN_CA_UserInfo' -v -count=1`
+
+Expected before implementation: FAIL because the L2 key and UserInfo metrics do not exist and every send calls Identity.
+
+- [ ] **Step 2: Add deterministic 64-bucket keys**
+
+In `common/utils/redis_keys.hpp`, add FNV-1a helpers:
+
+```cpp
+inline uint32_t fnv1a_32(const std::string &value) {
+    uint32_t hash = 2166136261u;
+    for (unsigned char c : value) { hash ^= c; hash *= 16777619u; }
+    return hash;
+}
+inline uint32_t user_info_bucket(const std::string &uid) { return fnv1a_32(uid) % 64u; }
+inline std::string user_info_key(const std::string &uid) {
+    auto bucket = std::to_string(user_info_bucket(uid));
+    return "im:user:{" + bucket + "}:" + uid;
+}
+```
+
+- [ ] **Step 3: Implement UserInfoCache DAO**
+
+In `common/dao/data_redis.hpp`, add `kUserInfoTtl{3600}` and a `UserInfoCache` class that stores strings. `get` catches Redis errors and returns nullopt; `set` uses `randomized_ttl(kUserInfoTtl)`; `invalidate` deletes the key.
+
+`batch_get` groups uids by `user_info_bucket`, caps input at 2000, performs one MGET per bucket, and returns `{hits, misses}` preserving uid association. `batch_set` groups the same way and uses one pipeline per bucket. An empty serialized value is a 5-second sentinel; dependency errors are never cached.
+
+- [ ] **Step 4: Replace Transmite's unconditional profile RPC**
+
+Inject `_user_info_cache` from `TransmiteServerBuilder::make_redis_object`. Replace `resolve_user_info` with:
+
+```cpp
+std::optional<std::string> resolve_user_info(const std::string &uid,
+                                             const std::string &rid,
+                                             brpc::Controller *caller) {
+    const auto lkey = key::local_user_info_cache_key(uid);
+    if (_local_user_cache) {
+        auto local = _local_user_cache->get(lkey);
+        if (local) { metrics::g_user_info_l1_hit_total << 1; return local; }
+    }
+    auto guard = _inflight_registry->acquire("user:" + uid);
+    std::unique_lock lk(*guard.mu);
+    if (_local_user_cache) if (auto local = _local_user_cache->get(lkey)) return local;
+    if (_user_info_cache) if (auto redis = _user_info_cache->get(uid)) {
+        metrics::g_user_info_l2_hit_total << 1;
+        _local_user_cache->set(lkey, *redis, randomized_ttl(std::chrono::seconds(45)));
+        return redis;
+    }
+    auto info = fetch_user_info_from_identity_(uid, rid, caller);
+    if (!info) return std::nullopt;
+    auto bytes = info->SerializeAsString();
+    if (_user_info_cache) _user_info_cache->set(uid, bytes);
+    if (_local_user_cache) _local_user_cache->set(lkey, bytes, randomized_ttl(std::chrono::seconds(45)));
+    metrics::g_user_info_rpc_total << 1;
+    return bytes;
+}
+```
+
+Call it before member resolution. Parse cached bytes into `chatnow::common::UserInfo`; corrupted values invalidate L1/L2 and return unavailable only if the subsequent RPC also fails. Remove the old unconditional async GetProfile call.
+
+- [ ] **Step 5: Invalidate after successful profile update**
+
+Construct `UserInfoCache` from Identity's existing `_redis_client`, inject it into `IdentityServiceImpl`, and immediately after `_mysql_user->update(user)` succeeds call `_user_info_cache->invalidate(auth.user_id)`. Cache deletion failure must not roll back the DB update.
+
+- [ ] **Step 6: Verify**
+
+Run: `cmake --build build -j2`
+
+Run on Linux: `cd tests && go test -tags=func ./func/... -run 'TestFN_CA_UserInfo' -v -count=1`
+
+Expected: all three tests PASS; 20 repeated sends cause at most one Identity RPC and profile update removes L2.
+
+- [ ] **Step 7: Commit**
+
+```bash
+git add common/utils/redis_keys.hpp common/dao/data_redis.hpp common/infra/metrics.hpp transmite/source/transmite_server.h identity/source/identity_server.h tests/func/cache_test.go
+git commit -m "feat: add UserInfo multilevel caching"
+```
+
+---
+
+### Task 5: Complete TTL jitter and RedisMutex backoff
+
+**Files:**
+
+- Modify: `common/dao/data_redis.hpp`
+- Modify: `common/utils/redis_mutex.hpp`
+- Modify: `common/infra/metrics.hpp`
+- Modify: `tests/func/cache_test.go`
+
+**Interfaces:**
+
+- Existing DAO method signatures remain unchanged.
+- UnackedPush's paired keys receive one shared randomized TTL sample per operation.
+
+- [ ] **Step 1: Write failing FN-CA-04 TTL test**
+
+Use unique users/devices, exercise login/status/code/device/unacked API paths, read TTLs through `verify.RedisTTL`, and require each TTL to lie within `[0.8*base-2s, 1.2*base+2s]`. Collect at least 20 keys and require at least two distinct TTL values.
+
+Run on Linux: `cd tests && go test -tags=func ./func/... -run TestFN_CA_TTLJitter -v -count=1`
+
+Expected before implementation: FAIL because Session/Status/Codes/DeviceSet/UnackedPush use fixed or absent TTLs.
+
+- [ ] **Step 2: Apply jitter to every missing path**
+
+Use `randomized_ttl(ttl)` in Session append/touch, Status append/touch, Codes append, and any remaining fixed cache expiration found by `rg '_c->(set|expire).*ttl' common/dao/data_redis.hpp`.
+
+In `DeviceSet::add`, call `expire(key, randomized_ttl(kSessionTtl))`; add `touch(uid, ttl=kSessionTtl)` and invoke it from device activity paths.
+
+In `UnackedPush::push` and `bump_score`, calculate exactly once:
+
+```cpp
+const auto effective_ttl = randomized_ttl(ttl);
+_c->expire(k, effective_ttl);
+_c->expire(ik, effective_ttl);
+```
+
+- [ ] **Step 3: Implement bounded exponential lock retry**
+
+Replace the fixed 5ms sleep in `RedisMutex::try_lock` with:
+
+```cpp
+int backoff_ms = 5;
+while (std::chrono::steady_clock::now() < deadline) {
+    if (_redis->set(_key, _token, std::chrono::milliseconds(_ttl_ms),
+                    sw::redis::UpdateType::NOT_EXIST)) {
+        _locked = true;
+        return true;
+    }
+    metrics::g_redis_mutex_retry_total << 1;
+    std::uniform_int_distribution<int> jitter(0, backoff_ms / 2);
+    auto delay = std::chrono::milliseconds(backoff_ms + jitter(rng_()));
+    auto remaining = std::chrono::duration_cast<std::chrono::milliseconds>(
+        deadline - std::chrono::steady_clock::now());
+    if (remaining <= std::chrono::milliseconds::zero()) break;
+    std::this_thread::sleep_for(std::min(delay, remaining));
+    backoff_ms = std::min(backoff_ms * 2, 20);
+}
+```
+
+Add a private thread-local RNG accessor; preserve SET NX PX and Lua CAS unlock unchanged.
+
+- [ ] **Step 4: Verify**
+
+Run: `cmake --build build -j2`
+
+Run on Linux: `cd tests && go test -tags=func ./func/... -run TestFN_CA_TTLJitter -v -count=1`
+
+Expected: PASS with in-range, non-identical TTL samples.
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add common/dao/data_redis.hpp common/utils/redis_mutex.hpp common/infra/metrics.hpp tests/func/cache_test.go
+git commit -m "perf: jitter cache TTLs and lock retries"
+```
+
+---
+
+### Task 6: PF-09 performance baseline and complete verification
+
+**Files:**
+
+- Create: `tests/perf/cache_test.go`
+- Modify: `tests/Makefile`
+- Modify: `docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md` only if implementation constraints require a documented correction.
+
+**Interfaces:**
+
+- Produces: `BenchmarkPF09_UserInfoCache` with cold/L2/L1 sub-benchmarks and explicit RPC-reduction metric.
+
+- [ ] **Step 1: Write PF-09 benchmark**
+
+Create a perf-tag benchmark that prepares 20 conversations, runs parallel sends from repeated senders, records elapsed latency samples, and calls:
+
+```go
+b.ReportMetric(float64(successes)/elapsed.Seconds(), "msg/s")
+b.ReportMetric(float64(p95.Microseconds()), "p95-us")
+b.ReportMetric(100*(1-float64(rpcDelta)/float64(successes)), "rpc-reduction-%")
+```
+
+Fail when RPC reduction is below 95%, steady-state throughput is below 5000 msg/s in the target Linux environment, or the checked-in baseline throughput regresses by more than 10%.
+
+- [ ] **Step 2: Run static and compile verification**
+
+Run:
+
+```bash
+git diff --check
+cmake -S . -B build
+cmake --build build -j2
+cd tests
+gofmt -w pkg/chaos/redis.go pkg/verify/redis.go func/cache_test.go reliability/*.go perf/cache_test.go
+go vet -tags=func ./func/... ./pkg/...
+go vet -tags=reliability ./reliability/... ./pkg/...
+go vet -tags=perf ./perf/... ./pkg/...
+go test -run '^$' ./...
+```
+
+Expected: zero formatting errors, all production targets compile, and all Go packages compile.
+
+- [ ] **Step 3: Run Linux full-stack validation**
+
+Run:
+
+```bash
+cd tests
+make test-func
+make test-reliability
+go test -tags=perf ./perf/... -run '^$' -bench BenchmarkPF09_UserInfoCache -benchmem -count=3 -benchtime=10s
+```
+
+Expected: FN-CA-01..05 and RL-05 PASS; PF-09 reports ≥5000 msg/s, p95, and ≥95% RPC reduction.
+
+- [ ] **Step 4: Review the issue acceptance matrix**
+
+Confirm with evidence:
+
+- #35: RL-05 shows fast circuit rejection and automatic recovery.
+- #37: FN-CA-01..03 and PF-09 show L2 behavior and ≥95% RPC reduction.
+- #38: FN-CA-04 shows jitter and DeviceSet TTL.
+- #45: contention metrics show bounded jittered retry.
+- #48 cache/rate-limit scope: RL-05 shows local rejection during Redis outage.
+
+- [ ] **Step 5: Commit**
+
+```bash
+git add tests/perf/cache_test.go tests/Makefile docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
+git commit -m "test: add cache resilience performance baseline"
+```
+
+- [ ] **Step 6: Invoke verification-before-completion and requesting-code-review**
+
+Run the complete verification commands again with fresh output, inspect the full diff against `origin/3.0-dev`, and do not claim completion until both reviews find no unresolved correctness, availability, concurrency, or test-framework issue.

From 795dfee7d2d5c9256a14c861e9045b045855119e Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Wed, 15 Jul 2026 16:26:48 +0800
Subject: [PATCH 04/47] test: add Redis chaos verification helpers

---
 tests/config.yaml          |  5 ++++
 tests/pkg/chaos/redis.go   | 42 +++++++++++++++++++++++++++++++++
 tests/pkg/client/config.go | 16 +++++++++++++
 tests/pkg/verify/redis.go  | 48 ++++++++++++++++++++++++++++++++++++++
 4 files changed, 111 insertions(+)
 create mode 100644 tests/pkg/chaos/redis.go
 create mode 100644 tests/pkg/verify/redis.go

diff --git a/tests/config.yaml b/tests/config.yaml
index a0b5541..97d3b8b 100644
--- a/tests/config.yaml
+++ b/tests/config.yaml
@@ -19,3 +19,8 @@ database:
 
 log:
   level: "debug"
+
+infra:
+  redis_container: "redis-node1"
+  compose_dir: ".."
+  transmite_vars: "http://localhost:10004"
diff --git a/tests/pkg/chaos/redis.go b/tests/pkg/chaos/redis.go
new file mode 100644
index 0000000..501897e
--- /dev/null
+++ b/tests/pkg/chaos/redis.go
@@ -0,0 +1,42 @@
+package chaos
+
+import (
+	"os/exec"
+	"testing"
+	"time"
+)
+
+var redisServices = []string{
+	"redis-node1", "redis-node2", "redis-node3",
+	"redis-node4", "redis-node5", "redis-node6",
+}
+
+func compose(t testing.TB, args ...string) {
+	t.Helper()
+	cmd := exec.Command("docker", append([]string{"compose"}, args...)...)
+	cmd.Dir = ".."
+	if out, err := cmd.CombinedOutput(); err != nil {
+		t.Fatalf("docker compose %v: %v: %s", args, err, out)
+	}
+}
+
+func StopRedisCluster(t testing.TB) {
+	compose(t, append([]string{"stop"}, redisServices...)...)
+}
+
+func StartRedisCluster(t testing.TB) {
+	compose(t, append([]string{"start"}, redisServices...)...)
+}
+
+func WaitRedisCluster(t testing.TB, timeout time.Duration) {
+	t.Helper()
+	deadline := time.Now().Add(timeout)
+	for time.Now().Before(deadline) {
+		cmd := exec.Command("docker", "exec", "redis-node1", "redis-cli", "cluster", "info")
+		if out, err := cmd.Output(); err == nil && string(out) != "" {
+			return
+		}
+		time.Sleep(500 * time.Millisecond)
+	}
+	t.Fatalf("redis cluster did not recover within %s", timeout)
+}
diff --git a/tests/pkg/client/config.go b/tests/pkg/client/config.go
index 6a25d5f..47af5a6 100644
--- a/tests/pkg/client/config.go
+++ b/tests/pkg/client/config.go
@@ -11,6 +11,7 @@ type Config struct {
 	Timeout  TimeoutConfig  `yaml:"timeout"`
 	Database DatabaseConfig `yaml:"database"`
 	Log      LogConfig      `yaml:"log"`
+	Infra    InfraConfig    `yaml:"infra"`
 }
 
 type TargetConfig struct {
@@ -33,6 +34,12 @@ type LogConfig struct {
 	Level string `yaml:"level"`
 }
 
+type InfraConfig struct {
+	RedisContainer string `yaml:"redis_container"`
+	ComposeDir     string `yaml:"compose_dir"`
+	TransmiteVars  string `yaml:"transmite_vars"`
+}
+
 func LoadConfig(path string) *Config {
 	if path == "" {
 		path = "config.yaml"
@@ -58,5 +65,14 @@ func LoadConfig(path string) *Config {
 	if v := os.Getenv("ES_URL"); v != "" {
 		cfg.Database.ESURL = v
 	}
+	if v := os.Getenv("REDIS_CONTAINER"); v != "" {
+		cfg.Infra.RedisContainer = v
+	}
+	if v := os.Getenv("COMPOSE_DIR"); v != "" {
+		cfg.Infra.ComposeDir = v
+	}
+	if v := os.Getenv("TRANSMITE_VARS"); v != "" {
+		cfg.Infra.TransmiteVars = v
+	}
 	return cfg
 }
diff --git a/tests/pkg/verify/redis.go b/tests/pkg/verify/redis.go
new file mode 100644
index 0000000..77f75b1
--- /dev/null
+++ b/tests/pkg/verify/redis.go
@@ -0,0 +1,48 @@
+package verify
+
+import (
+	"fmt"
+	"io"
+	"net/http"
+	"os/exec"
+	"strconv"
+	"strings"
+	"testing"
+	"time"
+)
+
+func RedisCLI(t testing.TB, args ...string) string {
+	t.Helper()
+	base := []string{"exec", "redis-node1", "redis-cli", "-c"}
+	out, err := exec.Command("docker", append(base, args...)...).CombinedOutput()
+	if err != nil {
+		t.Fatalf("redis-cli %v: %v: %s", args, err, out)
+	}
+	return strings.TrimSpace(string(out))
+}
+
+func RedisTTL(t testing.TB, key string) time.Duration {
+	seconds, err := strconv.ParseInt(RedisCLI(t, "TTL", key), 10, 64)
+	if err != nil || seconds < 0 {
+		t.Fatalf("invalid TTL for %s: %d (%v)", key, seconds, err)
+	}
+	return time.Duration(seconds) * time.Second
+}
+
+func BVar(t testing.TB, baseURL, name string) int64 {
+	t.Helper()
+	rsp, err := http.Get(fmt.Sprintf("%s/vars/%s", baseURL, name))
+	if err != nil {
+		t.Fatalf("read bvar %s: %v", name, err)
+	}
+	defer rsp.Body.Close()
+	body, err := io.ReadAll(rsp.Body)
+	if err != nil {
+		t.Fatalf("read bvar body %s: %v", name, err)
+	}
+	value, err := strconv.ParseInt(strings.TrimSpace(string(body)), 10, 64)
+	if err != nil {
+		t.Fatalf("parse bvar %s=%q: %v", name, body, err)
+	}
+	return value
+}

From 5c22e936832e8ebda5ef2ddd9923ebc19e966eeb Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Wed, 15 Jul 2026 16:48:08 +0800
Subject: [PATCH 05/47] feat: add fast Redis circuit breaking

---
 common/dao/data_redis.hpp                | 250 ++++++++++++++++++-----
 common/infra/metrics.hpp                 |   4 +
 common/utils/redis_circuit_breaker.hpp   |  74 +++++++
 tests/Makefile                           |   6 +-
 tests/reliability/redis_failover_test.go |  48 +++++
 tests/reliability/setup_test.go          |  18 ++
 6 files changed, 349 insertions(+), 51 deletions(-)
 create mode 100644 common/utils/redis_circuit_breaker.hpp
 create mode 100644 tests/reliability/redis_failover_test.go
 create mode 100644 tests/reliability/setup_test.go

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 7e096ea..8d6f1df 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -23,17 +23,91 @@
 #include <memory>
 #include <sstream>
 #include <string>
+#include <type_traits>
+#include <utility>
 #include <unordered_map>
 #include <unordered_set>
 #include <vector>
 #include "infra/logger.hpp"
+#include "infra/metrics.hpp"
 #include "utils/cache_version.hpp"
 #include "utils/random_ttl.hpp"
+#include "utils/redis_circuit_breaker.hpp"
 #include "utils/redis_keys.hpp"
 
 namespace chatnow
 {
 
+class RedisPipeline {
+public:
+    RedisPipeline(sw::redis::Pipeline pipeline,
+                  RedisCircuitBreaker &breaker,
+                  RedisCircuitBreaker::Permit permit)
+        : _pipeline(std::move(pipeline)), _breaker(&breaker), _permit(permit) {}
+
+    RedisPipeline(RedisPipeline &&) = default;
+    RedisPipeline &operator=(RedisPipeline &&) = default;
+    RedisPipeline(const RedisPipeline &) = delete;
+    RedisPipeline &operator=(const RedisPipeline &) = delete;
+
+    template <typename... Args>
+    RedisPipeline &hset(Args &&...args) {
+        _pipeline.hset(std::forward<Args>(args)...);
+        return *this;
+    }
+
+    template <typename... Args>
+    RedisPipeline &set(Args &&...args) {
+        _pipeline.set(std::forward<Args>(args)...);
+        return *this;
+    }
+
+    template <typename... Args>
+    RedisPipeline &expire(Args &&...args) {
+        _pipeline.expire(std::forward<Args>(args)...);
+        return *this;
+    }
+
+    template <typename... Args>
+    RedisPipeline &get(Args &&...args) {
+        _pipeline.get(std::forward<Args>(args)...);
+        return *this;
+    }
+
+    sw::redis::QueuedReplies exec() {
+        try {
+            auto replies = _pipeline.exec();
+            _breaker->on_success(_permit);
+            if (_permit.probe) metrics::g_redis_circuit_recovered_total << 1;
+            return replies;
+        } catch (const sw::redis::TimeoutError &) {
+            record_connection_failure_();
+            throw;
+        } catch (const sw::redis::IoError &) {
+            record_connection_failure_();
+            throw;
+        } catch (const sw::redis::ClosedError &) {
+            record_connection_failure_();
+            throw;
+        }
+    }
+
+private:
+    void record_connection_failure_() noexcept {
+        metrics::g_redis_call_failure_total << 1;
+        const auto previous = _breaker->state();
+        _breaker->on_connection_failure(_permit);
+        if (previous != RedisCircuitBreaker::State::Open &&
+            _breaker->state() == RedisCircuitBreaker::State::Open) {
+            metrics::g_redis_circuit_open_total << 1;
+        }
+    }
+
+    sw::redis::Pipeline _pipeline;
+    RedisCircuitBreaker *_breaker;
+    RedisCircuitBreaker::Permit _permit;
+};
+
 // 类型擦除 Redis 客户端适配器：根据持有的后端类型透明转发到
 // sw::redis::Redis（单机）或 sw::redis::RedisCluster。所有 cache 类
 // 统一使用 RedisClient::ptr，对调用方完全透明。每次调用一次分支
@@ -48,117 +122,132 @@ class RedisClient
 
     // --- String commands ---
     sw::redis::OptionalString get(const std::string &key) {
-        return _rc ? _rc->get(key) : _r->get(key);
+        return guarded_([&] { return _rc ? _rc->get(key) : _r->get(key); });
     }
     bool set(const std::string &key, const std::string &val,
              std::chrono::seconds ttl = std::chrono::seconds(0)) {
-        return _rc ? _rc->set(key, val, ttl) : _r->set(key, val, ttl);
+        return guarded_([&] { return _rc ? _rc->set(key, val, ttl) : _r->set(key, val, ttl); });
     }
     bool set(const std::string &key, const std::string &val,
              std::chrono::milliseconds ttl) {
-        return _rc ? _rc->set(key, val, ttl) : _r->set(key, val, ttl);
+        return guarded_([&] { return _rc ? _rc->set(key, val, ttl) : _r->set(key, val, ttl); });
     }
     bool set(const std::string &key, const std::string &val,
              std::chrono::seconds ttl, sw::redis::UpdateType type) {
-        return _rc ? _rc->set(key, val, ttl, type) : _r->set(key, val, ttl, type);
+        return guarded_([&] { return _rc ? _rc->set(key, val, ttl, type) : _r->set(key, val, ttl, type); });
     }
     bool set(const std::string &key, const std::string &val,
              std::chrono::milliseconds ttl, sw::redis::UpdateType type) {
-        return _rc ? _rc->set(key, val, ttl, type) : _r->set(key, val, ttl, type);
+        return guarded_([&] { return _rc ? _rc->set(key, val, ttl, type) : _r->set(key, val, ttl, type); });
     }
     long long del(const std::string &key) {
-        return _rc ? _rc->del(key) : _r->del(key);
+        return guarded_([&] { return _rc ? _rc->del(key) : _r->del(key); });
     }
     void expire(const std::string &key, std::chrono::seconds ttl) {
-        _rc ? _rc->expire(key, ttl) : _r->expire(key, ttl);
+        guarded_void_([&] { _rc ? _rc->expire(key, ttl) : _r->expire(key, ttl); });
     }
     long long incr(const std::string &key) {
-        return _rc ? _rc->incr(key) : _r->incr(key);
+        return guarded_([&] { return _rc ? _rc->incr(key) : _r->incr(key); });
     }
 
     // --- Set commands ---
     template <typename T>
     long long sadd(const std::string &key, const T &member) {
-        return _rc ? _rc->sadd(key, member) : _r->sadd(key, member);
+        return guarded_([&] { return _rc ? _rc->sadd(key, member) : _r->sadd(key, member); });
     }
     template <typename It>
     long long sadd(const std::string &key, It first, It last) {
-        return _rc ? _rc->sadd(key, first, last) : _r->sadd(key, first, last);
+        return guarded_([&] { return _rc ? _rc->sadd(key, first, last) : _r->sadd(key, first, last); });
     }
     template <typename Out>
     void smembers(const std::string &key, Out out) {
-        _rc ? _rc->smembers(key, out) : _r->smembers(key, out);
+        guarded_void_([&] { _rc ? _rc->smembers(key, out) : _r->smembers(key, out); });
     }
     template <typename T>
     long long srem(const std::string &key, const T &member) {
-        return _rc ? _rc->srem(key, member) : _r->srem(key, member);
+        return guarded_([&] { return _rc ? _rc->srem(key, member) : _r->srem(key, member); });
     }
     long long scard(const std::string &key) {
-        return _rc ? _rc->scard(key) : _r->scard(key);
+        return guarded_([&] { return _rc ? _rc->scard(key) : _r->scard(key); });
     }
 
     // --- Hash commands ---
     long long hset(const std::string &key, const std::string &field, const std::string &val) {
-        return _rc ? _rc->hset(key, field, val) : _r->hset(key, field, val);
+        return guarded_([&] { return _rc ? _rc->hset(key, field, val) : _r->hset(key, field, val); });
     }
     sw::redis::OptionalString hget(const std::string &key, const std::string &field) {
-        return _rc ? _rc->hget(key, field) : _r->hget(key, field);
+        return guarded_([&] { return _rc ? _rc->hget(key, field) : _r->hget(key, field); });
     }
     long long hdel(const std::string &key, const std::string &field) {
-        return _rc ? _rc->hdel(key, field) : _r->hdel(key, field);
+        return guarded_([&] { return _rc ? _rc->hdel(key, field) : _r->hdel(key, field); });
     }
     template <typename Out>
     void hkeys(const std::string &key, Out out) {
-        _rc ? _rc->hkeys(key, out) : _r->hkeys(key, out);
+        guarded_void_([&] { _rc ? _rc->hkeys(key, out) : _r->hkeys(key, out); });
     }
     template <typename Out>
     void hgetall(const std::string &key, Out out) {
-        _rc ? _rc->hgetall(key, out) : _r->hgetall(key, out);
+        guarded_void_([&] { _rc ? _rc->hgetall(key, out) : _r->hgetall(key, out); });
     }
     long long hlen(const std::string &key) {
-        return _rc ? _rc->hlen(key) : _r->hlen(key);
+        return guarded_([&] { return _rc ? _rc->hlen(key) : _r->hlen(key); });
     }
 
     // --- Sorted Set commands ---
     long long zadd(const std::string &key, const std::string &member, double score) {
-        return _rc ? _rc->zadd(key, member, score) : _r->zadd(key, member, score);
+        return guarded_([&] { return _rc ? _rc->zadd(key, member, score) : _r->zadd(key, member, score); });
     }
     long long zadd(const std::string &key, const std::string &member, double score,
                    sw::redis::UpdateType type) {
-        return _rc ? _rc->zadd(key, member, score, type) : _r->zadd(key, member, score, type);
+        return guarded_([&] { return _rc ? _rc->zadd(key, member, score, type) : _r->zadd(key, member, score, type); });
     }
     long long zrem(const std::string &key, const std::string &member) {
-        return _rc ? _rc->zrem(key, member) : _r->zrem(key, member);
+        return guarded_([&] { return _rc ? _rc->zrem(key, member) : _r->zrem(key, member); });
     }
     template <typename Out>
     void zrange(const std::string &key, long long start, long long stop, Out out) {
-        _rc ? _rc->zrange(key, start, stop, out) : _r->zrange(key, start, stop, out);
+        guarded_void_([&] { _rc ? _rc->zrange(key, start, stop, out) : _r->zrange(key, start, stop, out); });
     }
     template <typename Out>
     void zrangebyscore(const std::string &key,
                        const sw::redis::BoundedInterval<double> &interval,
                        const sw::redis::LimitOptions &opts, Out out) {
-        _rc ? _rc->zrangebyscore(key, interval, opts, out)
-            : _r->zrangebyscore(key, interval, opts, out);
+        guarded_void_([&] {
+            _rc ? _rc->zrangebyscore(key, interval, opts, out)
+                : _r->zrangebyscore(key, interval, opts, out);
+        });
     }
 
     // --- Lua scripting ---
     template <typename Ret, typename KeyIt, typename ArgIt>
     Ret eval(const std::string &script, KeyIt key_first, KeyIt key_last,
              ArgIt arg_first, ArgIt arg_last) {
-        return _rc ? _rc->eval<Ret>(script, key_first, key_last, arg_first, arg_last)
-                   : _r->eval<Ret>(script, key_first, key_last, arg_first, arg_last);
+        return guarded_([&]() -> Ret {
+            return _rc ? _rc->eval<Ret>(script, key_first, key_last, arg_first, arg_last)
+                       : _r->eval<Ret>(script, key_first, key_last, arg_first, arg_last);
+        });
     }
     template <typename KeyIt, typename ArgIt, typename Out>
     void eval(const std::string &script, KeyIt key_first, KeyIt key_last,
               ArgIt arg_first, ArgIt arg_last, Out out) {
-        _rc ? _rc->eval(script, key_first, key_last, arg_first, arg_last, out)
-            : _r->eval(script, key_first, key_last, arg_first, arg_last, out);
+        guarded_void_([&] {
+            _rc ? _rc->eval(script, key_first, key_last, arg_first, arg_last, out)
+                : _r->eval(script, key_first, key_last, arg_first, arg_last, out);
+        });
     }
 
     // --- Pipeline ---
-    auto pipeline(const sw::redis::StringView &hash_tag = {}) {
-        return _rc ? _rc->pipeline(hash_tag) : _r->pipeline();
+    RedisPipeline pipeline(const sw::redis::StringView &hash_tag = {}) {
+        auto permit = before_call_();
+        return RedisPipeline(_rc ? _rc->pipeline(hash_tag) : _r->pipeline(),
+                             _breaker, permit);
+    }
+
+    template <typename Input, typename Output>
+    void mget(Input first, Input last, Output out) {
+        guarded_void_([&] {
+            _rc ? _rc->mget(first, last, out) : _r->mget(first, last, out);
+        });
     }
 
     // --- SCAN ---
@@ -166,27 +255,88 @@ class RedisClient
     // 续扫，因此任意 cursor 都重启一次完整扫描并返回 0。
     template <typename Out>
     long long scan(long long cursor, const std::string &pattern, long long count, Out out) {
-        if (_rc) {
-            if (cursor != 0) {
-                LOG_WARN("RedisCluster scan cannot resume cursor {}; restarting full cluster scan", cursor);
-            }
-            _rc->for_each([&](sw::redis::Redis &r) {
-                long long cur = 0;
-                while (true) {
-                    cur = r.scan(cur, pattern, count, out);
-                    if (cur == 0) break;
+        return guarded_([&]() -> long long {
+            if (_rc) {
+                if (cursor != 0) {
+                    LOG_WARN("RedisCluster scan cannot resume cursor {}; restarting full cluster scan", cursor);
                 }
-            });
-            return 0;
-        }
-        return _r->scan(cursor, pattern, count, out);
+                _rc->for_each([&](sw::redis::Redis &r) {
+                    long long cur = 0;
+                    while (true) {
+                        cur = r.scan(cur, pattern, count, out);
+                        if (cur == 0) break;
+                    }
+                });
+                return 0;
+            }
+            return static_cast<long long>(_r->scan(cursor, pattern, count, out));
+        });
     }
 
     bool is_cluster() const { return _rc != nullptr; }
 
 private:
+    RedisCircuitBreaker::Permit before_call_() {
+        try {
+            return _breaker.before_call();
+        } catch (const RedisCircuitOpen &) {
+            metrics::g_redis_circuit_rejected_total << 1;
+            throw;
+        }
+    }
+
+    void record_connection_failure_(RedisCircuitBreaker::Permit permit) noexcept {
+        metrics::g_redis_call_failure_total << 1;
+        const auto previous = _breaker.state();
+        _breaker.on_connection_failure(permit);
+        if (previous != RedisCircuitBreaker::State::Open &&
+            _breaker.state() == RedisCircuitBreaker::State::Open) {
+            metrics::g_redis_circuit_open_total << 1;
+        }
+    }
+
+    template <class F>
+    auto guarded_(F &&fn) -> std::invoke_result_t<F> {
+        auto permit = before_call_();
+        try {
+            auto result = std::forward<F>(fn)();
+            _breaker.on_success(permit);
+            if (permit.probe) metrics::g_redis_circuit_recovered_total << 1;
+            return result;
+        } catch (const sw::redis::TimeoutError &) {
+            record_connection_failure_(permit);
+            throw;
+        } catch (const sw::redis::IoError &) {
+            record_connection_failure_(permit);
+            throw;
+        } catch (const sw::redis::ClosedError &) {
+            record_connection_failure_(permit);
+            throw;
+        }
+    }
+
+    template <class F>
+    void guarded_void_(F &&fn) {
+        auto permit = before_call_();
+        try {
+            std::forward<F>(fn)();
+            _breaker.on_success(permit);
+            if (permit.probe) metrics::g_redis_circuit_recovered_total << 1;
+        } catch (const sw::redis::TimeoutError &) {
+            record_connection_failure_(permit);
+            throw;
+        } catch (const sw::redis::IoError &) {
+            record_connection_failure_(permit);
+            throw;
+        } catch (const sw::redis::ClosedError &) {
+            record_connection_failure_(permit);
+            throw;
+        }
+    }
+
     std::shared_ptr<sw::redis::Redis> _r;
     std::shared_ptr<sw::redis::RedisCluster> _rc;
+    RedisCircuitBreaker _breaker;
 };
 
 /* brief: 默认 TTL 常量 */
@@ -215,12 +365,12 @@ class RedisClientFactory
         copts.port = port;
         copts.db = db;
         copts.keep_alive = keep_alive;
-        copts.connect_timeout = std::chrono::milliseconds(2000);
-        copts.socket_timeout  = std::chrono::milliseconds(2000);
+        copts.connect_timeout = std::chrono::milliseconds(50);
+        copts.socket_timeout  = std::chrono::milliseconds(50);
 
         sw::redis::ConnectionPoolOptions popts;
         popts.size              = pool_size;
-        popts.wait_timeout      = std::chrono::milliseconds(500);
+        popts.wait_timeout      = std::chrono::milliseconds(20);
         popts.connection_lifetime = std::chrono::minutes(30);
 
         return std::make_shared<sw::redis::Redis>(copts, popts);
@@ -255,7 +405,7 @@ class RedisClusterFactory
 
         sw::redis::ConnectionPoolOptions popts;
         popts.size = pool_size;
-        popts.wait_timeout = std::chrono::milliseconds(500);
+        popts.wait_timeout = std::chrono::milliseconds(20);
         popts.connection_lifetime = std::chrono::minutes(30);
 
         // 逐个尝试种子节点，直到成功连接（sw::redis++ RedisCluster 仅需一个种子
@@ -267,8 +417,8 @@ class RedisClusterFactory
                 copts.host = host;
                 copts.port = port;
                 copts.keep_alive = keep_alive;
-                copts.connect_timeout = std::chrono::milliseconds(2000);
-                copts.socket_timeout  = std::chrono::milliseconds(2000);
+                copts.connect_timeout = std::chrono::milliseconds(50);
+                copts.socket_timeout  = std::chrono::milliseconds(50);
 
                 auto cluster = std::make_shared<sw::redis::RedisCluster>(copts, popts);
                 // 验证连接可用（立即尝试一个轻量命令）
diff --git a/common/infra/metrics.hpp b/common/infra/metrics.hpp
index a347f9f..08a9b09 100644
--- a/common/infra/metrics.hpp
+++ b/common/infra/metrics.hpp
@@ -19,6 +19,10 @@ inline bvar::Adder<long> g_local_cache_expired_total("local_cache_expired_total"
 inline bvar::Adder<long> g_members_cache_stale_l1_total("members_cache_stale_l1_total");
 inline bvar::Adder<long> g_members_cache_snapshot_race_total("members_cache_snapshot_race_total");
 inline bvar::Adder<long> g_members_cache_version_conflict_total("members_cache_version_conflict_total");
+inline bvar::Adder<long> g_redis_circuit_open_total("redis_circuit_open_total");
+inline bvar::Adder<long> g_redis_circuit_rejected_total("redis_circuit_rejected_total");
+inline bvar::Adder<long> g_redis_circuit_recovered_total("redis_circuit_recovered_total");
+inline bvar::Adder<long> g_redis_call_failure_total("redis_call_failure_total");
 
 template <typename V>
 inline typename ::chatnow::LocalCache<V>::MetricsSink local_cache_metrics_sink() {
diff --git a/common/utils/redis_circuit_breaker.hpp b/common/utils/redis_circuit_breaker.hpp
new file mode 100644
index 0000000..46e2dd3
--- /dev/null
+++ b/common/utils/redis_circuit_breaker.hpp
@@ -0,0 +1,74 @@
+#pragma once
+
+#include <atomic>
+#include <chrono>
+#include <cstdint>
+#include <stdexcept>
+
+namespace chatnow {
+
+class RedisCircuitOpen final : public std::runtime_error {
+public:
+    RedisCircuitOpen() : std::runtime_error("redis circuit open") {}
+};
+
+class RedisCircuitBreaker {
+public:
+    enum class State : uint8_t { Closed, Open, HalfOpen };
+    struct Permit { bool probe = false; };
+
+    Permit before_call();
+    void on_success(Permit permit) noexcept;
+    void on_connection_failure(Permit permit) noexcept;
+    State state() const noexcept { return _state.load(std::memory_order_acquire); }
+
+private:
+    static constexpr uint32_t kFailureThreshold = 3;
+    static constexpr int64_t kOpenForMs = 1000;
+    static int64_t now_ms() noexcept;
+    void open() noexcept;
+
+    std::atomic<State> _state{State::Closed};
+    std::atomic<uint32_t> _consecutive_failures{0};
+    std::atomic<int64_t> _open_until_ms{0};
+};
+
+inline int64_t RedisCircuitBreaker::now_ms() noexcept {
+    return std::chrono::duration_cast<std::chrono::milliseconds>(
+        std::chrono::steady_clock::now().time_since_epoch()).count();
+}
+
+inline void RedisCircuitBreaker::open() noexcept {
+    _open_until_ms.store(now_ms() + kOpenForMs, std::memory_order_release);
+    _state.store(State::Open, std::memory_order_release);
+}
+
+inline RedisCircuitBreaker::Permit RedisCircuitBreaker::before_call() {
+    auto current = _state.load(std::memory_order_acquire);
+    if (current == State::Closed) return {};
+    if (current == State::HalfOpen) throw RedisCircuitOpen();
+    if (now_ms() < _open_until_ms.load(std::memory_order_acquire)) {
+        throw RedisCircuitOpen();
+    }
+    auto expected = State::Open;
+    if (_state.compare_exchange_strong(expected, State::HalfOpen,
+                                       std::memory_order_acq_rel)) {
+        return Permit{true};
+    }
+    throw RedisCircuitOpen();
+}
+
+inline void RedisCircuitBreaker::on_success(Permit) noexcept {
+    _consecutive_failures.store(0, std::memory_order_relaxed);
+    _state.store(State::Closed, std::memory_order_release);
+}
+
+inline void RedisCircuitBreaker::on_connection_failure(Permit permit) noexcept {
+    if (permit.probe ||
+        _consecutive_failures.fetch_add(1, std::memory_order_relaxed) + 1 >=
+            kFailureThreshold) {
+        open();
+    }
+}
+
+}  // namespace chatnow
diff --git a/tests/Makefile b/tests/Makefile
index 857994e..026c2d1 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -1,4 +1,4 @@
-.PHONY: proto test-bvt test-func test-scenario test-perf clean deps
+.PHONY: proto test-bvt test-func test-reliability test-scenario test-perf clean deps
 
 # Generate Go protobuf from proto/ definitions
 # NOTE: protoc-gen-go >= v1.35 requires go_package option or M flags to derive
@@ -35,6 +35,10 @@ test-bvt:
 test-func:
 	go test -tags=func ./func/... -v -count=1
 
+# Run reliability/chaos tests (requires the Docker Compose stack)
+test-reliability:
+	go test -tags=reliability ./reliability/... -v -count=1 -timeout=180s
+
 # Run scenario tests only (L3)
 test-scenario:
 	go test -tags=func ./func/... -run TestScenario -v -count=1
diff --git a/tests/reliability/redis_failover_test.go b/tests/reliability/redis_failover_test.go
new file mode 100644
index 0000000..294efd7
--- /dev/null
+++ b/tests/reliability/redis_failover_test.go
@@ -0,0 +1,48 @@
+//go:build reliability
+
+package reliability_test
+
+import (
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/require"
+
+	"chatnow-tests/pkg/chaos"
+	"chatnow-tests/pkg/client"
+	"chatnow-tests/pkg/fixture"
+	identity "chatnow-tests/proto/chatnow/identity"
+)
+
+// RL-05 | P0 | Redis 熔断后快速失败并自动恢复
+func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) {
+	user, _, _ := fixture.RegisterAndLogin(t, HTTP)
+	t.Cleanup(func() { chaos.StartRedisCluster(t); chaos.WaitRedisCluster(t, 60*time.Second) })
+	chaos.StopRedisCluster(t)
+
+	uid := user.UserID
+	for i := 0; i < 3; i++ {
+		rsp := &identity.GetProfileRsp{}
+		_ = user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{
+			RequestId: client.NewRequestID(), UserId: &uid,
+		}, rsp)
+	}
+
+	started := time.Now()
+	rsp := &identity.GetProfileRsp{}
+	err := user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{
+		RequestId: client.NewRequestID(), UserId: &uid,
+	}, rsp)
+	require.NoError(t, err)
+	require.True(t, rsp.GetHeader().GetSuccess())
+	require.Less(t, time.Since(started), 50*time.Millisecond)
+
+	chaos.StartRedisCluster(t)
+	chaos.WaitRedisCluster(t, 60*time.Second)
+	time.Sleep(1100 * time.Millisecond)
+	rsp = &identity.GetProfileRsp{}
+	require.NoError(t, user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{
+		RequestId: client.NewRequestID(), UserId: &uid,
+	}, rsp))
+	require.True(t, rsp.GetHeader().GetSuccess())
+}
diff --git a/tests/reliability/setup_test.go b/tests/reliability/setup_test.go
new file mode 100644
index 0000000..0b7abcb
--- /dev/null
+++ b/tests/reliability/setup_test.go
@@ -0,0 +1,18 @@
+//go:build reliability
+
+package reliability_test
+
+import (
+	"os"
+	"testing"
+
+	"chatnow-tests/pkg/client"
+)
+
+var HTTP *client.HTTPClient
+
+func TestMain(m *testing.M) {
+	cfg := client.LoadConfig("../config.yaml")
+	HTTP = client.NewHTTPClient(cfg)
+	os.Exit(m.Run())
+}

From 69e6e6b26e964193f15173157df6a79659c80a70 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Wed, 15 Jul 2026 17:14:53 +0800
Subject: [PATCH 06/47] fix: harden Redis circuit transitions

---
 common/dao/data_redis.hpp              | 224 +++++++++++++++++++------
 common/utils/redis_circuit_breaker.hpp |  87 +++++++---
 2 files changed, 237 insertions(+), 74 deletions(-)

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 8d6f1df..dbc84d2 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -23,7 +23,9 @@
 #include <memory>
 #include <sstream>
 #include <string>
+#include <string_view>
 #include <type_traits>
+#include <typeinfo>
 #include <utility>
 #include <unordered_map>
 #include <unordered_set>
@@ -38,74 +40,155 @@
 namespace chatnow
 {
 
+inline bool is_redis_pool_wait_error(const sw::redis::Error &error) noexcept {
+    if (typeid(error) != typeid(sw::redis::Error)) return false;
+    constexpr std::string_view prefix = "Failed to fetch a connection in ";
+    constexpr std::string_view suffix = " milliseconds";
+    const std::string_view message(error.what());
+    if (message.size() <= prefix.size() + suffix.size() ||
+        message.compare(0, prefix.size(), prefix) != 0 ||
+        message.compare(message.size() - suffix.size(), suffix.size(), suffix) != 0) {
+        return false;
+    }
+    const auto milliseconds = message.substr(
+        prefix.size(), message.size() - prefix.size() - suffix.size());
+    for (const char ch : milliseconds) {
+        if (ch < '0' || ch > '9') return false;
+    }
+    return true;
+}
+
 class RedisPipeline {
 public:
     RedisPipeline(sw::redis::Pipeline pipeline,
-                  RedisCircuitBreaker &breaker,
+                  std::shared_ptr<RedisCircuitBreaker> breaker,
                   RedisCircuitBreaker::Permit permit)
-        : _pipeline(std::move(pipeline)), _breaker(&breaker), _permit(permit) {}
-
-    RedisPipeline(RedisPipeline &&) = default;
-    RedisPipeline &operator=(RedisPipeline &&) = default;
+        : _pipeline(std::move(pipeline)), _breaker(std::move(breaker)), _permit(permit) {}
+
+    RedisPipeline(RedisPipeline &&other) noexcept
+        : _pipeline(std::move(other._pipeline)),
+          _breaker(std::move(other._breaker)),
+          _permit(other._permit),
+          _settled(other._settled) {
+        other._settled = true;
+    }
+    RedisPipeline &operator=(RedisPipeline &&other) noexcept {
+        if (this == &other) return *this;
+        abandon_();
+        _pipeline = std::move(other._pipeline);
+        _breaker = std::move(other._breaker);
+        _permit = other._permit;
+        _settled = other._settled;
+        other._settled = true;
+        return *this;
+    }
     RedisPipeline(const RedisPipeline &) = delete;
     RedisPipeline &operator=(const RedisPipeline &) = delete;
+    ~RedisPipeline() { abandon_(); }
 
     template <typename... Args>
     RedisPipeline &hset(Args &&...args) {
-        _pipeline.hset(std::forward<Args>(args)...);
-        return *this;
+        return queue_([&] { _pipeline.hset(std::forward<Args>(args)...); });
     }
 
     template <typename... Args>
     RedisPipeline &set(Args &&...args) {
-        _pipeline.set(std::forward<Args>(args)...);
-        return *this;
+        return queue_([&] { _pipeline.set(std::forward<Args>(args)...); });
     }
 
     template <typename... Args>
     RedisPipeline &expire(Args &&...args) {
-        _pipeline.expire(std::forward<Args>(args)...);
-        return *this;
+        return queue_([&] { _pipeline.expire(std::forward<Args>(args)...); });
     }
 
     template <typename... Args>
     RedisPipeline &get(Args &&...args) {
-        _pipeline.get(std::forward<Args>(args)...);
-        return *this;
+        return queue_([&] { _pipeline.get(std::forward<Args>(args)...); });
     }
 
     sw::redis::QueuedReplies exec() {
+        if (_settled) {
+            throw std::logic_error("RedisPipeline::exec called more than once");
+        }
         try {
             auto replies = _pipeline.exec();
-            _breaker->on_success(_permit);
-            if (_permit.probe) metrics::g_redis_circuit_recovered_total << 1;
+            settle_success_();
             return replies;
-        } catch (const sw::redis::TimeoutError &) {
+        } catch (const sw::redis::IoError &) {
+            record_connection_failure_();
+            throw;
+        } catch (const sw::redis::ClosedError &) {
             record_connection_failure_();
             throw;
+        } catch (const sw::redis::ReplyError &) {
+            settle_success_();
+            throw;
+        } catch (const sw::redis::Error &error) {
+            if (is_redis_pool_wait_error(error)) record_connection_failure_();
+            else abandon_();
+            throw;
+        } catch (...) {
+            abandon_();
+            throw;
+        }
+    }
+
+private:
+    template <typename F>
+    RedisPipeline &queue_(F &&queue_command) {
+        if (_settled) {
+            throw std::logic_error("RedisPipeline command queued after settlement");
+        }
+        try {
+            std::forward<F>(queue_command)();
+            return *this;
         } catch (const sw::redis::IoError &) {
             record_connection_failure_();
             throw;
         } catch (const sw::redis::ClosedError &) {
             record_connection_failure_();
             throw;
+        } catch (const sw::redis::Error &error) {
+            if (is_redis_pool_wait_error(error)) record_connection_failure_();
+            else abandon_();
+            throw;
+        } catch (...) {
+            abandon_();
+            throw;
         }
     }
 
-private:
-    void record_connection_failure_() noexcept {
-        metrics::g_redis_call_failure_total << 1;
-        const auto previous = _breaker->state();
-        _breaker->on_connection_failure(_permit);
-        if (previous != RedisCircuitBreaker::State::Open &&
-            _breaker->state() == RedisCircuitBreaker::State::Open) {
+    void record_transition_(RedisCircuitBreaker::Transition transition) noexcept {
+        if (transition == RedisCircuitBreaker::Transition::Opened) {
             metrics::g_redis_circuit_open_total << 1;
+        } else if (transition == RedisCircuitBreaker::Transition::Recovered) {
+            metrics::g_redis_circuit_recovered_total << 1;
         }
     }
 
+    void settle_success_() noexcept {
+        if (_settled) return;
+        _settled = true;
+        record_transition_(_breaker->on_success(_permit));
+    }
+
+    void record_connection_failure_() noexcept {
+        if (_settled) return;
+        _settled = true;
+        metrics::g_redis_call_failure_total << 1;
+        record_transition_(_breaker->on_connection_failure(_permit));
+    }
+
+    void abandon_() noexcept {
+        if (_settled || !_breaker) return;
+        _settled = true;
+        record_transition_(_breaker->on_abandoned(_permit));
+    }
+
     sw::redis::Pipeline _pipeline;
-    RedisCircuitBreaker *_breaker;
+    std::shared_ptr<RedisCircuitBreaker> _breaker;
     RedisCircuitBreaker::Permit _permit;
+    bool _settled = false;
 };
 
 // 类型擦除 Redis 客户端适配器：根据持有的后端类型透明转发到
@@ -117,8 +200,10 @@ class RedisClient
 public:
     using ptr = std::shared_ptr<RedisClient>;
 
-    RedisClient(std::shared_ptr<sw::redis::Redis> r) : _r(std::move(r)) {}
-    RedisClient(std::shared_ptr<sw::redis::RedisCluster> rc) : _rc(std::move(rc)) {}
+    RedisClient(std::shared_ptr<sw::redis::Redis> r)
+        : _r(std::move(r)), _breaker(std::make_shared<RedisCircuitBreaker>()) {}
+    RedisClient(std::shared_ptr<sw::redis::RedisCluster> rc)
+        : _rc(std::move(rc)), _breaker(std::make_shared<RedisCircuitBreaker>()) {}
 
     // --- String commands ---
     sw::redis::OptionalString get(const std::string &key) {
@@ -239,8 +324,26 @@ class RedisClient
     // --- Pipeline ---
     RedisPipeline pipeline(const sw::redis::StringView &hash_tag = {}) {
         auto permit = before_call_();
-        return RedisPipeline(_rc ? _rc->pipeline(hash_tag) : _r->pipeline(),
-                             _breaker, permit);
+        try {
+            return RedisPipeline(_rc ? _rc->pipeline(hash_tag) : _r->pipeline(),
+                                 _breaker, permit);
+        } catch (const sw::redis::IoError &) {
+            record_connection_failure_(permit);
+            throw;
+        } catch (const sw::redis::ClosedError &) {
+            record_connection_failure_(permit);
+            throw;
+        } catch (const sw::redis::ReplyError &) {
+            record_success_(permit);
+            throw;
+        } catch (const sw::redis::Error &error) {
+            if (is_redis_pool_wait_error(error)) record_connection_failure_(permit);
+            else abandon_(permit);
+            throw;
+        } catch (...) {
+            abandon_(permit);
+            throw;
+        }
     }
 
     template <typename Input, typename Output>
@@ -278,40 +381,57 @@ class RedisClient
 private:
     RedisCircuitBreaker::Permit before_call_() {
         try {
-            return _breaker.before_call();
+            return _breaker->before_call();
         } catch (const RedisCircuitOpen &) {
             metrics::g_redis_circuit_rejected_total << 1;
             throw;
         }
     }
 
-    void record_connection_failure_(RedisCircuitBreaker::Permit permit) noexcept {
-        metrics::g_redis_call_failure_total << 1;
-        const auto previous = _breaker.state();
-        _breaker.on_connection_failure(permit);
-        if (previous != RedisCircuitBreaker::State::Open &&
-            _breaker.state() == RedisCircuitBreaker::State::Open) {
+    static void record_transition_(RedisCircuitBreaker::Transition transition) noexcept {
+        if (transition == RedisCircuitBreaker::Transition::Opened) {
             metrics::g_redis_circuit_open_total << 1;
+        } else if (transition == RedisCircuitBreaker::Transition::Recovered) {
+            metrics::g_redis_circuit_recovered_total << 1;
         }
     }
 
+    void record_success_(RedisCircuitBreaker::Permit permit) noexcept {
+        record_transition_(_breaker->on_success(permit));
+    }
+
+    void abandon_(RedisCircuitBreaker::Permit permit) noexcept {
+        record_transition_(_breaker->on_abandoned(permit));
+    }
+
+    void record_connection_failure_(RedisCircuitBreaker::Permit permit) noexcept {
+        metrics::g_redis_call_failure_total << 1;
+        record_transition_(_breaker->on_connection_failure(permit));
+    }
+
     template <class F>
-    auto guarded_(F &&fn) -> std::invoke_result_t<F> {
+    auto guarded_(F &&fn) -> std::invoke_result_t<F &&> {
         auto permit = before_call_();
         try {
-            auto result = std::forward<F>(fn)();
-            _breaker.on_success(permit);
-            if (permit.probe) metrics::g_redis_circuit_recovered_total << 1;
-            return result;
-        } catch (const sw::redis::TimeoutError &) {
-            record_connection_failure_(permit);
-            throw;
+            decltype(auto) result = std::forward<F>(fn)();
+            record_success_(permit);
+            return std::forward<decltype(result)>(result);
         } catch (const sw::redis::IoError &) {
             record_connection_failure_(permit);
             throw;
         } catch (const sw::redis::ClosedError &) {
             record_connection_failure_(permit);
             throw;
+        } catch (const sw::redis::ReplyError &) {
+            record_success_(permit);
+            throw;
+        } catch (const sw::redis::Error &error) {
+            if (is_redis_pool_wait_error(error)) record_connection_failure_(permit);
+            else abandon_(permit);
+            throw;
+        } catch (...) {
+            abandon_(permit);
+            throw;
         }
     }
 
@@ -320,23 +440,29 @@ class RedisClient
         auto permit = before_call_();
         try {
             std::forward<F>(fn)();
-            _breaker.on_success(permit);
-            if (permit.probe) metrics::g_redis_circuit_recovered_total << 1;
-        } catch (const sw::redis::TimeoutError &) {
-            record_connection_failure_(permit);
-            throw;
+            record_success_(permit);
         } catch (const sw::redis::IoError &) {
             record_connection_failure_(permit);
             throw;
         } catch (const sw::redis::ClosedError &) {
             record_connection_failure_(permit);
             throw;
+        } catch (const sw::redis::ReplyError &) {
+            record_success_(permit);
+            throw;
+        } catch (const sw::redis::Error &error) {
+            if (is_redis_pool_wait_error(error)) record_connection_failure_(permit);
+            else abandon_(permit);
+            throw;
+        } catch (...) {
+            abandon_(permit);
+            throw;
         }
     }
 
     std::shared_ptr<sw::redis::Redis> _r;
     std::shared_ptr<sw::redis::RedisCluster> _rc;
-    RedisCircuitBreaker _breaker;
+    std::shared_ptr<RedisCircuitBreaker> _breaker;
 };
 
 /* brief: 默认 TTL 常量 */
diff --git a/common/utils/redis_circuit_breaker.hpp b/common/utils/redis_circuit_breaker.hpp
index 46e2dd3..b5ba855 100644
--- a/common/utils/redis_circuit_breaker.hpp
+++ b/common/utils/redis_circuit_breaker.hpp
@@ -3,6 +3,7 @@
 #include <atomic>
 #include <chrono>
 #include <cstdint>
+#include <mutex>
 #include <stdexcept>
 
 namespace chatnow {
@@ -15,22 +16,29 @@ class RedisCircuitOpen final : public std::runtime_error {
 class RedisCircuitBreaker {
 public:
     enum class State : uint8_t { Closed, Open, HalfOpen };
-    struct Permit { bool probe = false; };
+    enum class Transition : uint8_t { None, Opened, Recovered };
+    struct Permit {
+        uint64_t generation = 0;
+        bool probe = false;
+    };
 
     Permit before_call();
-    void on_success(Permit permit) noexcept;
-    void on_connection_failure(Permit permit) noexcept;
+    Transition on_success(Permit permit) noexcept;
+    Transition on_connection_failure(Permit permit) noexcept;
+    Transition on_abandoned(Permit permit) noexcept;
     State state() const noexcept { return _state.load(std::memory_order_acquire); }
 
 private:
     static constexpr uint32_t kFailureThreshold = 3;
     static constexpr int64_t kOpenForMs = 1000;
     static int64_t now_ms() noexcept;
-    void open() noexcept;
+    Transition open_locked() noexcept;
 
+    mutable std::mutex _mutex;
     std::atomic<State> _state{State::Closed};
-    std::atomic<uint32_t> _consecutive_failures{0};
-    std::atomic<int64_t> _open_until_ms{0};
+    uint64_t _generation = 0;
+    uint32_t _consecutive_failures = 0;
+    int64_t _open_until_ms = 0;
 };
 
 inline int64_t RedisCircuitBreaker::now_ms() noexcept {
@@ -38,37 +46,66 @@ inline int64_t RedisCircuitBreaker::now_ms() noexcept {
         std::chrono::steady_clock::now().time_since_epoch()).count();
 }
 
-inline void RedisCircuitBreaker::open() noexcept {
-    _open_until_ms.store(now_ms() + kOpenForMs, std::memory_order_release);
+inline RedisCircuitBreaker::Transition RedisCircuitBreaker::open_locked() noexcept {
+    if (_state.load(std::memory_order_relaxed) == State::Open) {
+        return Transition::None;
+    }
+    ++_generation;
+    _consecutive_failures = 0;
+    _open_until_ms = now_ms() + kOpenForMs;
     _state.store(State::Open, std::memory_order_release);
+    return Transition::Opened;
 }
 
 inline RedisCircuitBreaker::Permit RedisCircuitBreaker::before_call() {
-    auto current = _state.load(std::memory_order_acquire);
-    if (current == State::Closed) return {};
-    if (current == State::HalfOpen) throw RedisCircuitOpen();
-    if (now_ms() < _open_until_ms.load(std::memory_order_acquire)) {
+    std::lock_guard<std::mutex> lock(_mutex);
+    const auto current = _state.load(std::memory_order_relaxed);
+    if (current == State::Closed) return {_generation, false};
+    if (current == State::HalfOpen || now_ms() < _open_until_ms) {
         throw RedisCircuitOpen();
     }
-    auto expected = State::Open;
-    if (_state.compare_exchange_strong(expected, State::HalfOpen,
-                                       std::memory_order_acq_rel)) {
-        return Permit{true};
+    _state.store(State::HalfOpen, std::memory_order_release);
+    return {_generation, true};
+}
+
+inline RedisCircuitBreaker::Transition
+RedisCircuitBreaker::on_success(Permit permit) noexcept {
+    std::lock_guard<std::mutex> lock(_mutex);
+    if (permit.generation != _generation) return Transition::None;
+
+    const auto current = _state.load(std::memory_order_relaxed);
+    if (current == State::Closed && !permit.probe) {
+        _consecutive_failures = 0;
+        return Transition::None;
+    }
+    if (current == State::HalfOpen && permit.probe) {
+        _consecutive_failures = 0;
+        _state.store(State::Closed, std::memory_order_release);
+        return Transition::Recovered;
     }
-    throw RedisCircuitOpen();
+    return Transition::None;
 }
 
-inline void RedisCircuitBreaker::on_success(Permit) noexcept {
-    _consecutive_failures.store(0, std::memory_order_relaxed);
-    _state.store(State::Closed, std::memory_order_release);
+inline RedisCircuitBreaker::Transition
+RedisCircuitBreaker::on_connection_failure(Permit permit) noexcept {
+    std::lock_guard<std::mutex> lock(_mutex);
+    if (permit.generation != _generation) return Transition::None;
+
+    const auto current = _state.load(std::memory_order_relaxed);
+    if (current == State::HalfOpen && permit.probe) return open_locked();
+    if (current != State::Closed || permit.probe) return Transition::None;
+    if (++_consecutive_failures < kFailureThreshold) return Transition::None;
+    return open_locked();
 }
 
-inline void RedisCircuitBreaker::on_connection_failure(Permit permit) noexcept {
-    if (permit.probe ||
-        _consecutive_failures.fetch_add(1, std::memory_order_relaxed) + 1 >=
-            kFailureThreshold) {
-        open();
+inline RedisCircuitBreaker::Transition
+RedisCircuitBreaker::on_abandoned(Permit permit) noexcept {
+    std::lock_guard<std::mutex> lock(_mutex);
+    if (permit.generation != _generation || !permit.probe ||
+        _state.load(std::memory_order_relaxed) != State::HalfOpen) {
+        return Transition::None;
     }
+    return open_locked();
 }
 
 }  // namespace chatnow

From 1500ab7f84b2bb51637ce42fb395e19be5c56929 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Wed, 15 Jul 2026 17:39:25 +0800
Subject: [PATCH 07/47] feat: retain rate limiting during Redis outages

---
 common/dao/data_redis.hpp                |   7 +-
 common/infra/metrics.hpp                 |   2 +
 common/utils/local_rate_limiter.hpp      | 109 +++++++++++++++++++++++
 tests/func/cache_test.go                 |  41 +++++++++
 tests/reliability/redis_failover_test.go |  36 ++++++++
 5 files changed, 194 insertions(+), 1 deletion(-)
 create mode 100644 common/utils/local_rate_limiter.hpp
 create mode 100644 tests/func/cache_test.go

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index dbc84d2..ecd61b3 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -33,6 +33,7 @@
 #include "infra/logger.hpp"
 #include "infra/metrics.hpp"
 #include "utils/cache_version.hpp"
+#include "utils/local_rate_limiter.hpp"
 #include "utils/random_ttl.hpp"
 #include "utils/redis_circuit_breaker.hpp"
 #include "utils/redis_keys.hpp"
@@ -1168,7 +1169,10 @@ class RateLimiter
             return cur == 1;
         } catch(std::exception &e) {
             LOG_ERROR("RateLimiter.allow {}: {}", key_full, e.what());
-            return true;
+            metrics::g_rate_limit_local_fallback_total << 1;
+            const bool allowed = _local.allow(key_full, max_count, window_sec);
+            if (!allowed) metrics::g_rate_limit_local_rejected_total << 1;
+            return allowed;
         }
     }
     bool allow_user(const std::string &uid, int max_count, int window_sec) {
@@ -1179,6 +1183,7 @@ class RateLimiter
     }
 private:
     RedisClient::ptr _c;
+    LocalRateLimiter _local;
     static const std::string kRateLimitScript;
 };
 
diff --git a/common/infra/metrics.hpp b/common/infra/metrics.hpp
index 08a9b09..7a6bed0 100644
--- a/common/infra/metrics.hpp
+++ b/common/infra/metrics.hpp
@@ -23,6 +23,8 @@ inline bvar::Adder<long> g_redis_circuit_open_total("redis_circuit_open_total");
 inline bvar::Adder<long> g_redis_circuit_rejected_total("redis_circuit_rejected_total");
 inline bvar::Adder<long> g_redis_circuit_recovered_total("redis_circuit_recovered_total");
 inline bvar::Adder<long> g_redis_call_failure_total("redis_call_failure_total");
+inline bvar::Adder<long> g_rate_limit_local_fallback_total("rate_limit_local_fallback_total");
+inline bvar::Adder<long> g_rate_limit_local_rejected_total("rate_limit_local_rejected_total");
 
 template <typename V>
 inline typename ::chatnow::LocalCache<V>::MetricsSink local_cache_metrics_sink() {
diff --git a/common/utils/local_rate_limiter.hpp b/common/utils/local_rate_limiter.hpp
new file mode 100644
index 0000000..b3b0749
--- /dev/null
+++ b/common/utils/local_rate_limiter.hpp
@@ -0,0 +1,109 @@
+#pragma once
+
+#include <algorithm>
+#include <array>
+#include <atomic>
+#include <chrono>
+#include <cmath>
+#include <cstddef>
+#include <cstdint>
+#include <functional>
+#include <mutex>
+#include <string>
+#include <unordered_map>
+
+namespace chatnow {
+
+class LocalRateLimiter {
+public:
+    bool allow(const std::string &key, int capacity, int window_sec) {
+        const auto now_ms = std::chrono::duration_cast<std::chrono::milliseconds>(
+            std::chrono::steady_clock::now().time_since_epoch()).count();
+        return allow(key, capacity, window_sec, now_ms);
+    }
+
+    bool allow(const std::string &key, int capacity, int window_sec, int64_t now_ms) {
+        if (capacity <= 0 || window_sec <= 0) return true;
+
+        const size_t shard_index = std::hash<std::string>{}(key) % kShardCount;
+        Shard &shard = _shards[shard_index];
+        std::lock_guard<std::mutex> lock(shard.mu);
+
+        const uint64_t operation = _operations.fetch_add(1, std::memory_order_relaxed) + 1;
+        if (operation % 1024 == 0) {
+            cleanup_(shard, now_ms, window_sec);
+        }
+
+        auto found = shard.buckets.find(key);
+        if (found == shard.buckets.end()) {
+            if (!reserve_bucket_()) return false;
+            try {
+                found = shard.buckets.emplace(
+                    key, Bucket{static_cast<double>(capacity - 1), now_ms, now_ms}).first;
+            } catch (...) {
+                _bucket_count.fetch_sub(1, std::memory_order_relaxed);
+                throw;
+            }
+            return true;
+        }
+
+        Bucket &bucket = found->second;
+        const int64_t elapsed_ms = now_ms > bucket.refill_ms ? now_ms - bucket.refill_ms : 0;
+        const double refill = static_cast<double>(elapsed_ms) * capacity /
+                              (static_cast<double>(window_sec) * 1000.0);
+        bucket.tokens = std::min(static_cast<double>(capacity), bucket.tokens + refill);
+        bucket.refill_ms = now_ms;
+        bucket.seen_ms = now_ms;
+        if (bucket.tokens < 1.0) return false;
+        bucket.tokens -= 1.0;
+        return true;
+    }
+
+private:
+    struct Bucket {
+        double tokens;
+        int64_t refill_ms;
+        int64_t seen_ms;
+    };
+    struct Shard {
+        std::mutex mu;
+        std::unordered_map<std::string, Bucket> buckets;
+    };
+
+    static constexpr size_t kShardCount = 64;
+    static constexpr size_t kMaxBuckets = 65536;
+
+    bool reserve_bucket_() {
+        size_t count = _bucket_count.load(std::memory_order_relaxed);
+        while (count < kMaxBuckets) {
+            if (_bucket_count.compare_exchange_weak(
+                    count, count + 1, std::memory_order_relaxed, std::memory_order_relaxed)) {
+                return true;
+            }
+        }
+        return false;
+    }
+
+    void cleanup_(Shard &shard, int64_t now_ms, int window_sec) {
+        const int64_t idle_limit_ms = static_cast<int64_t>(window_sec) * 2000;
+        size_t removed = 0;
+        for (auto it = shard.buckets.begin(); it != shard.buckets.end();) {
+            if (now_ms > it->second.seen_ms &&
+                now_ms - it->second.seen_ms > idle_limit_ms) {
+                it = shard.buckets.erase(it);
+                ++removed;
+            } else {
+                ++it;
+            }
+        }
+        if (removed != 0) {
+            _bucket_count.fetch_sub(removed, std::memory_order_relaxed);
+        }
+    }
+
+    std::array<Shard, kShardCount> _shards;
+    std::atomic<size_t> _bucket_count{0};
+    std::atomic<uint64_t> _operations{0};
+};
+
+}  // namespace chatnow
diff --git a/tests/func/cache_test.go b/tests/func/cache_test.go
new file mode 100644
index 0000000..d7b251a
--- /dev/null
+++ b/tests/func/cache_test.go
@@ -0,0 +1,41 @@
+//go:build func
+
+package func_test
+
+import (
+	"fmt"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+
+	"chatnow-tests/pkg/client"
+	"chatnow-tests/pkg/fixture"
+	msg "chatnow-tests/proto/chatnow/message"
+	transmite "chatnow-tests/proto/chatnow/transmite"
+)
+
+// FN-CA-05 | healthy Redis applies the distributed message rate limit.
+func TestFN_CA_RateLimit(t *testing.T) {
+	user, peer, convID := fixture.MakeFriends(t, HTTP)
+	_ = peer
+
+	rateLimited := 0
+	for i := 0; i < 650; i++ {
+		rsp := &transmite.SendMessageRsp{}
+		err := user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
+			RequestId:      client.NewRequestID(),
+			ConversationId: convID,
+			Content: &msg.MessageContent{
+				Type: msg.MessageType_TEXT,
+				Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: fmt.Sprintf("rate-limit-%d", i)}},
+			},
+			ClientMsgId: client.NewRequestID(),
+		}, rsp)
+		require.NoError(t, err)
+		if rsp.GetHeader().GetErrorMessage() == "rate_limited" {
+			rateLimited++
+		}
+	}
+
+	require.Greater(t, rateLimited, 0, "healthy Redis must enforce the distributed rate limit")
+}
diff --git a/tests/reliability/redis_failover_test.go b/tests/reliability/redis_failover_test.go
index 294efd7..56b0974 100644
--- a/tests/reliability/redis_failover_test.go
+++ b/tests/reliability/redis_failover_test.go
@@ -3,6 +3,7 @@
 package reliability_test
 
 import (
+	"fmt"
 	"testing"
 	"time"
 
@@ -12,14 +13,49 @@ import (
 	"chatnow-tests/pkg/client"
 	"chatnow-tests/pkg/fixture"
 	identity "chatnow-tests/proto/chatnow/identity"
+	msg "chatnow-tests/proto/chatnow/message"
+	transmite "chatnow-tests/proto/chatnow/transmite"
 )
 
 // RL-05 | P0 | Redis 熔断后快速失败并自动恢复
 func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) {
 	user, _, _ := fixture.RegisterAndLogin(t, HTTP)
+	peer, _, _ := fixture.RegisterAndLogin(t, HTTP)
+	convID := fixture.CreateGroupWithMembers(t, user, []*client.HTTPClient{peer}, "rl-redis-outage")
+	prewarmRsp := &transmite.SendMessageRsp{}
+	require.NoError(t, user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
+		RequestId:      client.NewRequestID(),
+		ConversationId: convID,
+		Content: &msg.MessageContent{
+			Type: msg.MessageType_TEXT,
+			Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: "prewarm"}},
+		},
+		ClientMsgId: client.NewRequestID(),
+	}, prewarmRsp))
+	require.True(t, prewarmRsp.GetHeader().GetSuccess())
+
 	t.Cleanup(func() { chaos.StartRedisCluster(t); chaos.WaitRedisCluster(t, 60*time.Second) })
 	chaos.StopRedisCluster(t)
 
+	rateLimited := 0
+	for i := 0; i < 650; i++ {
+		sendRsp := &transmite.SendMessageRsp{}
+		err := user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
+			RequestId:      client.NewRequestID(),
+			ConversationId: convID,
+			Content: &msg.MessageContent{
+				Type: msg.MessageType_TEXT,
+				Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: fmt.Sprintf("outage-%d", i)}},
+			},
+			ClientMsgId: client.NewRequestID(),
+		}, sendRsp)
+		require.NoError(t, err)
+		if sendRsp.GetHeader().GetErrorMessage() == "rate_limited" {
+			rateLimited++
+		}
+	}
+	require.Greater(t, rateLimited, 0, "Redis outage must retain bounded rate limiting")
+
 	uid := user.UserID
 	for i := 0; i < 3; i++ {
 		rsp := &identity.GetProfileRsp{}

From 69d41c9f2301b6b411e39078bb482cdb3b39ef45 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Wed, 15 Jul 2026 17:53:41 +0800
Subject: [PATCH 08/47] fix: reclaim idle rate-limit buckets at capacity

---
 common/dao/data_redis.hpp           | 33 +++++++++++++++++------------
 common/utils/local_rate_limiter.hpp |  5 ++++-
 2 files changed, 24 insertions(+), 14 deletions(-)

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index ecd61b3..f7137b6 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -1155,24 +1155,24 @@ class RateLimiter
      *   - 命中限制返回 false（业务可返回 429 / RATE_LIMITED）
      */
     bool allow(const std::string &key_full, int max_count, int window_sec) {
+        std::vector<std::string> keys = {key_full};
+        auto now_ms = std::chrono::duration_cast<std::chrono::milliseconds>(
+            std::chrono::system_clock::now().time_since_epoch()).count();
+        std::vector<std::string> args = {
+            std::to_string(max_count),
+            std::to_string(window_sec),
+            std::to_string(now_ms)
+        };
         try {
-            std::vector<std::string> keys = {key_full};
-            auto now_ms = std::chrono::duration_cast<std::chrono::milliseconds>(
-                std::chrono::system_clock::now().time_since_epoch()).count();
-            std::vector<std::string> args = {
-                std::to_string(max_count),
-                std::to_string(window_sec),
-                std::to_string(now_ms)
-            };
             long long cur = _c->eval<long long>(kRateLimitScript, keys.begin(), keys.end(),
                                                 args.begin(), args.end());
             return cur == 1;
-        } catch(std::exception &e) {
+        } catch (const RedisCircuitOpen &e) {
             LOG_ERROR("RateLimiter.allow {}: {}", key_full, e.what());
-            metrics::g_rate_limit_local_fallback_total << 1;
-            const bool allowed = _local.allow(key_full, max_count, window_sec);
-            if (!allowed) metrics::g_rate_limit_local_rejected_total << 1;
-            return allowed;
+            return allow_local_(key_full, max_count, window_sec);
+        } catch (const sw::redis::Error &e) {
+            LOG_ERROR("RateLimiter.allow {}: {}", key_full, e.what());
+            return allow_local_(key_full, max_count, window_sec);
         }
     }
     bool allow_user(const std::string &uid, int max_count, int window_sec) {
@@ -1182,6 +1182,13 @@ class RateLimiter
         return allow(key::kRateSsid + ssid, max_count, window_sec);
     }
 private:
+    bool allow_local_(const std::string &key_full, int max_count, int window_sec) {
+        metrics::g_rate_limit_local_fallback_total << 1;
+        const bool allowed = _local.allow(key_full, max_count, window_sec);
+        if (!allowed) metrics::g_rate_limit_local_rejected_total << 1;
+        return allowed;
+    }
+
     RedisClient::ptr _c;
     LocalRateLimiter _local;
     static const std::string kRateLimitScript;
diff --git a/common/utils/local_rate_limiter.hpp b/common/utils/local_rate_limiter.hpp
index b3b0749..166de40 100644
--- a/common/utils/local_rate_limiter.hpp
+++ b/common/utils/local_rate_limiter.hpp
@@ -36,7 +36,10 @@ class LocalRateLimiter {
 
         auto found = shard.buckets.find(key);
         if (found == shard.buckets.end()) {
-            if (!reserve_bucket_()) return false;
+            if (!reserve_bucket_()) {
+                cleanup_(shard, now_ms, window_sec);
+                if (!reserve_bucket_()) return false;
+            }
             try {
                 found = shard.buckets.emplace(
                     key, Bucket{static_cast<double>(capacity - 1), now_ms, now_ms}).first;

From 88ed8d805c57b1f0cad63daab53a39e5540efc73 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Wed, 15 Jul 2026 18:11:35 +0800
Subject: [PATCH 09/47] feat: add UserInfo multilevel caching

---
 common/dao/data_redis.hpp           | 115 +++++++++++++++++++++++++
 common/infra/metrics.hpp            |   4 +
 common/utils/redis_keys.hpp         |  19 +++++
 identity/source/identity_server.h   |  10 ++-
 tests/func/cache_test.go            | 111 ++++++++++++++++++++++++
 transmite/source/transmite_server.h | 125 +++++++++++++++++++++-------
 6 files changed, 354 insertions(+), 30 deletions(-)

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index f7137b6..1f191ec 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -20,7 +20,9 @@
 #include <sw/redis++/redis++.h>
 #include <sw/redis++/redis_cluster.h>
 #include <chrono>
+#include <algorithm>
 #include <memory>
+#include <optional>
 #include <sstream>
 #include <string>
 #include <string_view>
@@ -473,6 +475,7 @@ inline constexpr std::chrono::seconds kCodeTtl(60 * 5);             // 验证码
 inline constexpr std::chrono::seconds kLastMsgTtl(24 * 3600);       // 最近消息预览 24 小时
 inline constexpr std::chrono::seconds kReadAckTtl(24 * 3600);       // 已读暂存 24 小时
 inline constexpr std::chrono::seconds kMembersTtl(30 * 60);         // 成员缓存 30 分钟
+inline constexpr std::chrono::seconds kUserInfoTtl(3600);           // 用户资料缓存 1 小时
 inline constexpr std::chrono::seconds kOnlineTtl(30);               // 在线路由 30s（依赖心跳续期，每 heartbeat 刷新）
 inline constexpr std::chrono::seconds kUnackedTtl(7 * 24 * 3600);   // 未 ack 重传缓冲 7 天
 
@@ -1069,6 +1072,118 @@ class Members
     RedisClient::ptr _c;
 };
 
+// =============================================================================
+// 用户资料缓存（value 为序列化 UserInfo protobuf；DAO 不依赖 protobuf 类型）
+// =============================================================================
+
+class UserInfoCache
+{
+public:
+    using ptr = std::shared_ptr<UserInfoCache>;
+
+    struct BatchResult {
+        std::unordered_map<std::string, std::string> hits;
+        std::vector<std::string> misses;
+    };
+
+    explicit UserInfoCache(const RedisClient::ptr &c) : _c(c) {}
+
+    std::optional<std::string> get(const std::string &uid) {
+        if (!_c) return std::nullopt;
+        try {
+            auto value = _c->get(key::user_info_key(uid));
+            if (!value) return std::nullopt;
+            return *value;
+        } catch (const std::exception &e) {
+            LOG_ERROR("UserInfoCache.get 失败 {}: {}", uid, e.what());
+            return std::nullopt;
+        }
+    }
+
+    BatchResult batch_get(const std::vector<std::string> &uids) {
+        BatchResult result;
+        const size_t count = std::min<size_t>(uids.size(), 2000);
+        if (!_c) {
+            result.misses.assign(uids.begin(), uids.begin() + count);
+            return result;
+        }
+        std::unordered_map<uint32_t, std::vector<std::pair<std::string, std::string>>> groups;
+        for (size_t i = 0; i < count; ++i) {
+            groups[key::user_info_bucket(uids[i])].push_back(
+                {uids[i], key::user_info_key(uids[i])});
+        }
+        for (const auto &[bucket, entries] : groups) {
+            std::vector<std::string> keys;
+            keys.reserve(entries.size());
+            for (const auto &entry : entries) keys.push_back(entry.second);
+            try {
+                std::vector<sw::redis::OptionalString> values;
+                values.reserve(keys.size());
+                _c->mget(keys.begin(), keys.end(), std::back_inserter(values));
+                for (size_t i = 0; i < entries.size(); ++i) {
+                    if (i < values.size() && values[i]) result.hits[entries[i].first] = *values[i];
+                    else result.misses.push_back(entries[i].first);
+                }
+            } catch (const std::exception &e) {
+                LOG_ERROR("UserInfoCache.batch_get bucket={} 失败: {}", bucket, e.what());
+                for (const auto &entry : entries) result.misses.push_back(entry.first);
+            }
+        }
+        return result;
+    }
+
+    void set(const std::string &uid, const std::string &serialized) {
+        if (!_c) return;
+        const auto ttl = serialized.empty()
+            ? randomized_ttl(std::chrono::seconds(5))
+            : randomized_ttl(kUserInfoTtl);
+        try {
+            _c->set(key::user_info_key(uid), serialized, ttl);
+        } catch (const std::exception &e) {
+            LOG_ERROR("UserInfoCache.set 失败 {}: {}", uid, e.what());
+        }
+    }
+
+    void batch_set(const std::unordered_map<std::string, std::string> &values) {
+        if (!_c) return;
+        std::unordered_map<uint32_t, std::vector<std::pair<std::string, std::string>>> groups;
+        size_t count = 0;
+        for (const auto &entry : values) {
+            if (count++ >= 2000) break;
+            groups[key::user_info_bucket(entry.first)].push_back(entry);
+        }
+        for (const auto &[bucket, entries] : groups) {
+            try {
+                auto pipe = _c->pipeline(std::to_string(bucket));
+                for (const auto &[uid, serialized] : entries) {
+                    const auto ttl = serialized.empty()
+                        ? randomized_ttl(std::chrono::seconds(5))
+                        : randomized_ttl(kUserInfoTtl);
+                    pipe.set(key::user_info_key(uid), serialized, ttl);
+                }
+                pipe.exec();
+            } catch (const std::exception &e) {
+                LOG_ERROR("UserInfoCache.batch_set bucket={} 失败: {}", bucket, e.what());
+            }
+        }
+    }
+
+    bool invalidate(const std::string &uid) noexcept {
+        if (!_c) return true;
+        try {
+            _c->del(key::user_info_key(uid));
+            return true;
+        } catch (const std::exception &e) {
+            LOG_ERROR("UserInfoCache.invalidate 失败 {}: {}", uid, e.what());
+            metrics::g_user_info_invalidation_failure_total << 1;
+            return false;
+        }
+    }
+
+private:
+    RedisClient::ptr _c;
+};
+
 // =============================================================================
 // 在线路由表（多 Push 实例下：uid -> 持有 ws 连接的 push_instance_id 集合）
 // =============================================================================
diff --git a/common/infra/metrics.hpp b/common/infra/metrics.hpp
index 7a6bed0..67984a2 100644
--- a/common/infra/metrics.hpp
+++ b/common/infra/metrics.hpp
@@ -19,6 +19,10 @@ inline bvar::Adder<long> g_local_cache_expired_total("local_cache_expired_total"
 inline bvar::Adder<long> g_members_cache_stale_l1_total("members_cache_stale_l1_total");
 inline bvar::Adder<long> g_members_cache_snapshot_race_total("members_cache_snapshot_race_total");
 inline bvar::Adder<long> g_members_cache_version_conflict_total("members_cache_version_conflict_total");
+inline bvar::Adder<long> g_user_info_l1_hit_total("user_info_l1_hit_total");
+inline bvar::Adder<long> g_user_info_l2_hit_total("user_info_l2_hit_total");
+inline bvar::Adder<long> g_user_info_rpc_total("user_info_rpc_total");
+inline bvar::Adder<long> g_user_info_invalidation_failure_total("user_info_invalidation_failure_total");
 inline bvar::Adder<long> g_redis_circuit_open_total("redis_circuit_open_total");
 inline bvar::Adder<long> g_redis_circuit_rejected_total("redis_circuit_rejected_total");
 inline bvar::Adder<long> g_redis_circuit_recovered_total("redis_circuit_recovered_total");
diff --git a/common/utils/redis_keys.hpp b/common/utils/redis_keys.hpp
index 7cd4dab..ba0e66e 100644
--- a/common/utils/redis_keys.hpp
+++ b/common/utils/redis_keys.hpp
@@ -1,6 +1,7 @@
 #pragma once
 
 #include <optional>
+#include <cstdint>
 #include <string>
 
 namespace chatnow::key {
@@ -65,6 +66,24 @@ inline std::string local_user_info_cache_key(const std::string &uid) {
     return std::string("local:user:") + hash_tag(uid);
 }
 
+inline uint32_t fnv1a_32(const std::string &value) {
+    uint32_t hash = 2166136261u;
+    for (unsigned char c : value) {
+        hash ^= c;
+        hash *= 16777619u;
+    }
+    return hash;
+}
+
+inline uint32_t user_info_bucket(const std::string &uid) {
+    return fnv1a_32(uid) % 64u;
+}
+
+inline std::string user_info_key(const std::string &uid) {
+    auto bucket = std::to_string(user_info_bucket(uid));
+    return "im:user:{" + bucket + "}:" + uid;
+}
+
 inline std::string local_route_cache_key(const std::string &uid) {
     return std::string("local:route:") + hash_tag(uid);
 }
diff --git a/identity/source/identity_server.h b/identity/source/identity_server.h
index c104564..a7281a3 100644
--- a/identity/source/identity_server.h
+++ b/identity/source/identity_server.h
@@ -39,6 +39,7 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService
     IdentityServiceImpl(const std::shared_ptr<odb::core::database> &mysql_client,
                         const std::shared_ptr<elasticlient::Client> &es_client,
                         const RedisClient::ptr &redis_client,
+                        const UserInfoCache::ptr &user_info_cache,
                         const std::shared_ptr<MailClient> &mail_client,
                         const std::shared_ptr<auth::JwtCodec> &jwt_codec,
                         const std::shared_ptr<auth::JwtStore> &jwt_store,
@@ -48,6 +49,7 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService
         : _mysql_user(std::make_shared<UserTable>(mysql_client)),
           _es_user(std::make_shared<ESUser>(es_client)),
           _redis_codes(std::make_shared<Codes>(redis_client)),
+          _user_info_cache(user_info_cache),
           _mail_client(mail_client),
           _jwt_codec(jwt_codec),
           _jwt_store(jwt_store),
@@ -478,6 +480,9 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService
                 throw ServiceError(::chatnow::error::kSystemInternalError,
                                    "db update failed");
             }
+            if (_user_info_cache) {
+                (void)_user_info_cache->invalidate(auth.user_id);
+            }
             std::string ob_payload = outbox_payload_upsert_(
                 user->user_id(), user->mail(), user->phone(),
                 user->nickname(), user->description(),
@@ -532,6 +537,7 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService
     std::shared_ptr<UserTable>          _mysql_user;
     std::shared_ptr<ESUser>             _es_user;
     std::shared_ptr<Codes>              _redis_codes;
+    UserInfoCache::ptr                  _user_info_cache;
     std::shared_ptr<MailClient>         _mail_client;
     std::shared_ptr<auth::JwtCodec>     _jwt_codec;
     std::shared_ptr<auth::JwtStore>     _jwt_store;
@@ -701,6 +707,7 @@ class IdentityServerBuilder
             _redis_client = std::make_shared<RedisClient>(redis);
         }
         _es_outbox = std::make_shared<ESOutbox>(_redis_client, key::es_outbox_key("identity"));
+        _user_info_cache = std::make_shared<UserInfoCache>(_redis_client);
     }
     /* brief: 加载 JWT 配置并构造 codec / store（必须在 make_redis_object 之后） */
     void make_jwt_object(const std::string &auth_config_path) {
@@ -781,7 +788,7 @@ class IdentityServerBuilder
 
         auto es_reaper_client = ESClientFactory::create(_es_hosts);
         IdentityServiceImpl *identity_service = new IdentityServiceImpl(
-            _mysql_client, _es_client, _redis_client, _mail_client,
+            _mysql_client, _es_client, _redis_client, _user_info_cache, _mail_client,
             _jwt_codec, _jwt_store, _media_public_url_prefix,
             _es_outbox, es_reaper_client);
         _service_impl = identity_service;
@@ -826,6 +833,7 @@ class IdentityServerBuilder
     std::shared_ptr<odb::core::database> _mysql_client;
     std::string _redis_seeds;
     RedisClient::ptr _redis_client;
+    UserInfoCache::ptr _user_info_cache;
     std::shared_ptr<MailClient> _mail_client;
 
     std::string _media_public_url_prefix;
diff --git a/tests/func/cache_test.go b/tests/func/cache_test.go
index d7b251a..01eb33a 100644
--- a/tests/func/cache_test.go
+++ b/tests/func/cache_test.go
@@ -3,17 +3,65 @@
 package func_test
 
 import (
+	"bytes"
 	"fmt"
+	"os/exec"
+	"sync"
 	"testing"
+	"time"
 
 	"github.com/stretchr/testify/require"
+	"google.golang.org/protobuf/proto"
 
 	"chatnow-tests/pkg/client"
 	"chatnow-tests/pkg/fixture"
+	"chatnow-tests/pkg/verify"
+	common "chatnow-tests/proto/chatnow/common"
+	identity "chatnow-tests/proto/chatnow/identity"
 	msg "chatnow-tests/proto/chatnow/message"
 	transmite "chatnow-tests/proto/chatnow/transmite"
 )
 
+func userInfoBucket(uid string) uint32 {
+	hash := uint32(2166136261)
+	for i := 0; i < len(uid); i++ {
+		hash ^= uint32(uid[i])
+		hash *= 16777619
+	}
+	return hash % 64
+}
+
+func userInfoRedisKey(uid string) string {
+	return fmt.Sprintf("im:user:{%d}:%s", userInfoBucket(uid), uid)
+}
+
+func redisRaw(t testing.TB, key string) []byte {
+	t.Helper()
+	container := HTTP.Config().Infra.RedisContainer
+	if container == "" {
+		container = "redis-node1"
+	}
+	out, err := exec.Command("docker", "exec", container, "redis-cli", "-c", "--raw", "GET", key).Output()
+	require.NoError(t, err)
+	return bytes.TrimSuffix(out, []byte("\n"))
+}
+
+func sendCacheTestMessage(t testing.TB, user *client.HTTPClient, convID, suffix string) {
+	t.Helper()
+	rsp := &transmite.SendMessageRsp{}
+	err := user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
+		RequestId:      client.NewRequestID(),
+		ConversationId: convID,
+		Content: &msg.MessageContent{
+			Type: msg.MessageType_TEXT,
+			Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: "user-info-cache-" + suffix}},
+		},
+		ClientMsgId: client.NewRequestID(),
+	}, rsp)
+	require.NoError(t, err)
+	require.True(t, rsp.GetHeader().GetSuccess(), rsp.GetHeader().GetErrorMessage())
+}
+
 // FN-CA-05 | healthy Redis applies the distributed message rate limit.
 func TestFN_CA_RateLimit(t *testing.T) {
 	user, peer, convID := fixture.MakeFriends(t, HTTP)
@@ -39,3 +87,66 @@ func TestFN_CA_RateLimit(t *testing.T) {
 
 	require.Greater(t, rateLimited, 0, "healthy Redis must enforce the distributed rate limit")
 }
+
+func TestFN_CA_UserInfoL2AvoidsRepeatedRPC(t *testing.T) {
+	user, _, convID := fixture.MakeFriends(t, HTTP)
+	verify.RedisCLI(t, "DEL", userInfoRedisKey(user.UserID))
+	before := verify.BVar(t, HTTP.Config().Infra.TransmiteVars, "user_info_rpc_total")
+
+	for i := 0; i < 20; i++ {
+		sendCacheTestMessage(t, user, convID, fmt.Sprintf("repeat-%d", i))
+	}
+
+	after := verify.BVar(t, HTTP.Config().Infra.TransmiteVars, "user_info_rpc_total")
+	require.LessOrEqual(t, after-before, int64(1))
+	require.Equal(t, "1", verify.RedisCLI(t, "EXISTS", userInfoRedisKey(user.UserID)))
+}
+
+func TestFN_CA_UserInfoSingleflight(t *testing.T) {
+	user, _, convID := fixture.MakeFriends(t, HTTP)
+	verify.RedisCLI(t, "DEL", userInfoRedisKey(user.UserID))
+	before := verify.BVar(t, HTTP.Config().Infra.TransmiteVars, "user_info_rpc_total")
+
+	start := make(chan struct{})
+	var wg sync.WaitGroup
+	for i := 0; i < 200; i++ {
+		i := i
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			<-start
+			sendCacheTestMessage(t, user, convID, fmt.Sprintf("flight-%d", i))
+		}()
+	}
+	close(start)
+	wg.Wait()
+
+	after := verify.BVar(t, HTTP.Config().Infra.TransmiteVars, "user_info_rpc_total")
+	require.LessOrEqual(t, after-before, int64(1))
+}
+
+func TestFN_CA_UserInfoInvalidatedAfterProfileUpdate(t *testing.T) {
+	user, _, convID := fixture.MakeFriends(t, HTTP)
+	key := userInfoRedisKey(user.UserID)
+	verify.RedisCLI(t, "DEL", key)
+	sendCacheTestMessage(t, user, convID, "warm")
+	require.Equal(t, "1", verify.RedisCLI(t, "EXISTS", key))
+
+	newNickname := fmt.Sprintf("cache_%d", time.Now().UnixNano()%1_000_000_000)
+	updateRsp := &identity.UpdateProfileRsp{}
+	require.NoError(t, user.DoAuth("/service/identity/update_profile", &identity.UpdateProfileReq{
+		RequestId: client.NewRequestID(),
+		Nickname:  &newNickname,
+	}, updateRsp))
+	require.True(t, updateRsp.GetHeader().GetSuccess(), updateRsp.GetHeader().GetErrorMessage())
+	require.Equal(t, "0", verify.RedisCLI(t, "EXISTS", key))
+
+	// Identity can invalidate the shared L2 immediately; the process-local L1 is
+	// intentionally bounded by its 45s TTL in the absence of a broadcast channel.
+	time.Sleep(55 * time.Second)
+	sendCacheTestMessage(t, user, convID, "after-update")
+	serialized := redisRaw(t, key)
+	info := &common.UserInfo{}
+	require.NoError(t, proto.Unmarshal(serialized, info))
+	require.Equal(t, newNickname, info.GetNickname())
+}
diff --git a/transmite/source/transmite_server.h b/transmite/source/transmite_server.h
index 0f0edd0..9dc4d2f 100644
--- a/transmite/source/transmite_server.h
+++ b/transmite/source/transmite_server.h
@@ -74,6 +74,7 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
                         const Members::ptr &members_cache,
                         const RateLimiter::ptr &rate_limiter,
                         const RedisClient::ptr &redis,
+                        const UserInfoCache::ptr &user_info_cache,
                         LocalCache<MembersCacheEntry>::ptr local_members_cache = nullptr,
                         LocalCache<std::string>::ptr local_user_cache = nullptr,
                         InflightRegistry::ptr inflight_registry = nullptr,
@@ -92,6 +93,7 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
                         _members_cache(members_cache),
                         _rate_limiter(rate_limiter),
                         _redis(redis),
+                        _user_info_cache(user_info_cache),
                         _local_members_cache(std::move(local_members_cache)),
                         _local_user_cache(std::move(local_user_cache)),
                         _inflight_registry(std::move(inflight_registry)),
@@ -237,23 +239,17 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
             }
         }
 
-        // ⑤ 并行 RPC：Identity.GetProfile（sender 信息）+ Conversation.GetMemberIds（收件人列表）
-        auto identity_channel = _mm_channels->choose(_identity_service_name);
-        if (!identity_channel) {
-            LOG_ERROR("请求ID: {} - identity_service 节点缺失", rid);
-            return err_response(rid, chatnow::error::kSystemUnavailable, "依赖服务暂不可用");
+        // ⑤ sender 资料：L1 → uid singleflight → L2 → Identity RPC。
+        auto serialized_user_info = resolve_user_info(
+            uid, rid, static_cast<brpc::Controller*>(controller));
+        chatnow::common::UserInfo sender_info;
+        if (!serialized_user_info || serialized_user_info->empty() ||
+            !sender_info.ParseFromString(*serialized_user_info)) {
+            LOG_ERROR("请求ID: {} - 获取用户信息失败 uid={}", rid, uid);
+            return err_response(rid, chatnow::error::kSystemUnavailable, "获取用户信息失败");
         }
 
         // 成员列表：L1 → InflightRegistry → L2 Redis → RedisMutex → RPC (内置 warm)
-        // 先发起 profile RPC（异步），与成员解析并行
-        chatnow::identity::IdentityService_Stub identity_stub(identity_channel.get());
-        chatnow::identity::GetProfileReq profile_req;
-        chatnow::identity::GetProfileRsp profile_rsp;
-        brpc::Controller profile_cntl;
-        profile_req.set_request_id(rid);
-        profile_req.set_user_id(uid);
-        chatnow::auth::forward_auth_metadata(static_cast<brpc::Controller*>(controller), &profile_cntl);
-        identity_stub.GetProfile(&profile_cntl, &profile_req, &profile_rsp, brpc::DoNothing());
 
         MembersResult members_result;
         for (int retry = 0; retry < 3; ++retry) {
@@ -264,11 +260,6 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
         }
         std::vector<std::string> member_id_list = std::move(members_result.members);
 
-        brpc::Join(profile_cntl.call_id());
-        if (profile_cntl.Failed() || !profile_rsp.header().success()) {
-            LOG_ERROR("请求ID: {} - 获取用户信息失败: {}", rid, profile_cntl.ErrorText());
-            return err_response(rid, chatnow::error::kSystemUnavailable, "获取用户信息失败");
-        }
         if (member_id_list.empty()) {
             LOG_ERROR("请求ID: {} - 会话成员为空 ssid={}", rid, chat_ssid);
             return err_response(rid, chatnow::error::kConversationNotFound, "会话已解散或不存在");
@@ -563,18 +554,90 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
     }
 }
 
-    std::string resolve_user_info(const std::string &uid) const {
-        if (!_local_user_cache) return "";
-        std::string ukey = key::local_user_info_cache_key(uid);
-        auto cached = _local_user_cache->get(ukey);
-        if (cached.has_value()) return *cached;
-        return "";
+    std::optional<std::string> resolve_user_info(const std::string &uid,
+                                                 const std::string &rid,
+                                                 brpc::Controller *caller) {
+        const auto lkey = key::local_user_info_cache_key(uid);
+        auto parseable = [](const std::string &bytes) {
+            chatnow::common::UserInfo info;
+            return !bytes.empty() && info.ParseFromString(bytes);
+        };
+        auto read_l1 = [&]() -> std::optional<std::string> {
+            if (!_local_user_cache) return std::nullopt;
+            auto local = _local_user_cache->get(lkey);
+            if (!local) return std::nullopt;
+            if (parseable(*local)) {
+                metrics::g_user_info_l1_hit_total << 1;
+                return local;
+            }
+            _local_user_cache->invalidate(lkey);
+            if (_user_info_cache) _user_info_cache->invalidate(uid);
+            return std::nullopt;
+        };
+
+        if (auto local = read_l1()) return local;
+
+        auto resolve_after_lock = [&]() -> std::optional<std::string> {
+            if (auto local = read_l1()) return local;
+            if (_user_info_cache) {
+                if (auto redis = _user_info_cache->get(uid)) {
+                    if (redis->empty()) return std::nullopt;  // confirmed-not-found sentinel
+                    if (parseable(*redis)) {
+                        metrics::g_user_info_l2_hit_total << 1;
+                        if (_local_user_cache) {
+                            _local_user_cache->set(
+                                lkey, *redis, randomized_ttl(std::chrono::seconds(45)));
+                        }
+                        return redis;
+                    }
+                    _user_info_cache->invalidate(uid);
+                }
+            }
+
+            auto info = fetch_user_info_from_identity_(uid, rid, caller);
+            if (!info) return std::nullopt;
+            auto bytes = info->SerializeAsString();
+            if (_user_info_cache) _user_info_cache->set(uid, bytes);
+            if (_local_user_cache) {
+                _local_user_cache->set(
+                    lkey, bytes, randomized_ttl(std::chrono::seconds(45)));
+            }
+            return bytes;
+        };
+
+        if (!_inflight_registry) return resolve_after_lock();
+        auto guard = _inflight_registry->acquire("user:" + uid);
+        std::unique_lock lk(*guard.mu);
+        return resolve_after_lock();
     }
 
-    void warm_user_info(const std::string &uid, const std::string &serialized_info) {
-        if (_local_user_cache && !serialized_info.empty())
-            _local_user_cache->set(key::local_user_info_cache_key(uid), serialized_info,
-                                   randomized_ttl(std::chrono::seconds(45)));
+    std::optional<chatnow::common::UserInfo> fetch_user_info_from_identity_(
+        const std::string &uid, const std::string &rid, brpc::Controller *caller) {
+        auto identity_channel = _mm_channels->choose(_identity_service_name);
+        if (!identity_channel) {
+            LOG_ERROR("identity_service 节点缺失 (user info {})", uid);
+            return std::nullopt;
+        }
+        chatnow::identity::IdentityService_Stub stub(identity_channel.get());
+        chatnow::identity::GetProfileReq req;
+        chatnow::identity::GetProfileRsp rsp;
+        brpc::Controller cntl;
+        req.set_request_id(rid);
+        req.set_user_id(uid);
+        if (caller) chatnow::auth::forward_auth_metadata(caller, &cntl);
+        stub.GetProfile(&cntl, &req, &rsp, nullptr);
+        metrics::g_user_info_rpc_total << 1;
+        if (cntl.Failed() || !rsp.header().success()) {
+            if (!cntl.Failed() &&
+                rsp.header().error_code() == chatnow::error::kAuthUserNotFound &&
+                _user_info_cache) {
+                _user_info_cache->set(uid, "");
+            }
+            LOG_ERROR("获取用户信息失败 uid={}: {} {}", uid, cntl.ErrorText(),
+                      rsp.header().error_message());
+            return std::nullopt;
+        }
+        return rsp.user_info();
     }
 
     std::optional<std::vector<std::string>> fetch_members_from_conversation_service_(
@@ -647,6 +710,7 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
     Members::ptr _members_cache;
     RateLimiter::ptr _rate_limiter;
     RedisClient::ptr _redis;
+    UserInfoCache::ptr _user_info_cache;
     LocalCache<MembersCacheEntry>::ptr _local_members_cache;
     LocalCache<std::string>::ptr _local_user_cache;
     InflightRegistry::ptr _inflight_registry;
@@ -823,6 +887,7 @@ class TransmiteServerBuilder
         }
         _seq_gen = std::make_shared<SeqGen>(_redis_client);
         _members_cache = std::make_shared<Members>(_redis_client);
+        _user_info_cache = std::make_shared<UserInfoCache>(_redis_client);
         _rate_limiter = std::make_shared<RateLimiter>(_redis_client);
     }
 
@@ -865,6 +930,7 @@ class TransmiteServerBuilder
                                                                         _members_cache,
                                                                         _rate_limiter,
                                                                         _redis_client,
+                                                                        _user_info_cache,
                                                                         _local_members_cache,
                                                                         _local_user_cache,
                                                                         _inflight_registry,
@@ -922,6 +988,7 @@ class TransmiteServerBuilder
     SeqGen::ptr _seq_gen;
     Members::ptr _members_cache;
     RateLimiter::ptr _rate_limiter;
+    UserInfoCache::ptr _user_info_cache;
     LocalCache<MembersCacheEntry>::ptr _local_members_cache;
     LocalCache<std::string>::ptr _local_user_cache;
     InflightRegistry::ptr _inflight_registry;

From 30d00b3c2a70f6c48f43191dccea08e62b392357 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Wed, 15 Jul 2026 18:25:18 +0800
Subject: [PATCH 10/47] fix: harden UserInfo cache tests and responses

---
 tests/func/cache_test.go            | 24 +++++++++++++++++++-----
 transmite/source/transmite_server.h | 16 +++++++++++++++-
 2 files changed, 34 insertions(+), 6 deletions(-)

diff --git a/tests/func/cache_test.go b/tests/func/cache_test.go
index 01eb33a..736d485 100644
--- a/tests/func/cache_test.go
+++ b/tests/func/cache_test.go
@@ -46,8 +46,7 @@ func redisRaw(t testing.TB, key string) []byte {
 	return bytes.TrimSuffix(out, []byte("\n"))
 }
 
-func sendCacheTestMessage(t testing.TB, user *client.HTTPClient, convID, suffix string) {
-	t.Helper()
+func sendCacheTestMessageResult(user *client.HTTPClient, convID, suffix string) error {
 	rsp := &transmite.SendMessageRsp{}
 	err := user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
 		RequestId:      client.NewRequestID(),
@@ -58,8 +57,18 @@ func sendCacheTestMessage(t testing.TB, user *client.HTTPClient, convID, suffix
 		},
 		ClientMsgId: client.NewRequestID(),
 	}, rsp)
-	require.NoError(t, err)
-	require.True(t, rsp.GetHeader().GetSuccess(), rsp.GetHeader().GetErrorMessage())
+	if err != nil {
+		return err
+	}
+	if !rsp.GetHeader().GetSuccess() {
+		return fmt.Errorf("send failed: %s", rsp.GetHeader().GetErrorMessage())
+	}
+	return nil
+}
+
+func sendCacheTestMessage(t testing.TB, user *client.HTTPClient, convID, suffix string) {
+	t.Helper()
+	require.NoError(t, sendCacheTestMessageResult(user, convID, suffix))
 }
 
 // FN-CA-05 | healthy Redis applies the distributed message rate limit.
@@ -108,6 +117,7 @@ func TestFN_CA_UserInfoSingleflight(t *testing.T) {
 	before := verify.BVar(t, HTTP.Config().Infra.TransmiteVars, "user_info_rpc_total")
 
 	start := make(chan struct{})
+	results := make(chan error, 200)
 	var wg sync.WaitGroup
 	for i := 0; i < 200; i++ {
 		i := i
@@ -115,11 +125,15 @@ func TestFN_CA_UserInfoSingleflight(t *testing.T) {
 		go func() {
 			defer wg.Done()
 			<-start
-			sendCacheTestMessage(t, user, convID, fmt.Sprintf("flight-%d", i))
+			results <- sendCacheTestMessageResult(user, convID, fmt.Sprintf("flight-%d", i))
 		}()
 	}
 	close(start)
 	wg.Wait()
+	close(results)
+	for err := range results {
+		require.NoError(t, err)
+	}
 
 	after := verify.BVar(t, HTTP.Config().Infra.TransmiteVars, "user_info_rpc_total")
 	require.LessOrEqual(t, after-before, int64(1))
diff --git a/transmite/source/transmite_server.h b/transmite/source/transmite_server.h
index 9dc4d2f..d1a4f70 100644
--- a/transmite/source/transmite_server.h
+++ b/transmite/source/transmite_server.h
@@ -637,7 +637,21 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
                       rsp.header().error_message());
             return std::nullopt;
         }
-        return rsp.user_info();
+        if (!rsp.has_user_info()) {
+            LOG_ERROR("获取用户信息响应缺少 user_info uid={}", uid);
+            return std::nullopt;
+        }
+        auto bytes = rsp.user_info().SerializeAsString();
+        if (bytes.empty()) {
+            LOG_ERROR("获取用户信息响应序列化为空 uid={}", uid);
+            return std::nullopt;
+        }
+        chatnow::common::UserInfo info;
+        if (!info.ParseFromString(bytes)) {
+            LOG_ERROR("获取用户信息响应无法解析 uid={}", uid);
+            return std::nullopt;
+        }
+        return info;
     }
 
     std::optional<std::vector<std::string>> fetch_members_from_conversation_service_(

From e131d3a0de230ba3cc1780310a129c1a9cd2f6ed Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Wed, 15 Jul 2026 18:43:50 +0800
Subject: [PATCH 11/47] perf: jitter cache TTLs and lock retries

---
 common/dao/data_redis.hpp    |  45 ++++++---
 common/infra/metrics.hpp     |   1 +
 common/utils/redis_mutex.hpp |  17 +++-
 tests/func/cache_test.go     | 178 +++++++++++++++++++++++++++++++++++
 4 files changed, 227 insertions(+), 14 deletions(-)

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 1f191ec..105dd86 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -578,7 +578,7 @@ class Session
     /* brief: 写入登录态，TTL 7 天 */
     void append(const std::string &ssid, const std::string &uid,
                 std::chrono::seconds ttl = kSessionTtl) {
-        try { _c->set(key::kSession + ssid, uid, ttl); }
+        try { _c->set(key::kSession + ssid, uid, randomized_ttl(ttl)); }
         catch(std::exception &e) { LOG_ERROR("Session.append 失败 {}: {}", ssid, e.what()); }
     }
     void remove(const std::string &ssid) {
@@ -591,7 +591,7 @@ class Session
     }
     /* brief: 续期（每次心跳调用） */
     void touch(const std::string &ssid, std::chrono::seconds ttl = kSessionTtl) {
-        try { _c->expire(key::kSession + ssid, ttl); }
+        try { _c->expire(key::kSession + ssid, randomized_ttl(ttl)); }
         catch(std::exception &e) { LOG_ERROR("Session.touch 失败 {}: {}", ssid, e.what()); }
     }
 private:
@@ -604,7 +604,7 @@ class Status
     using ptr = std::shared_ptr<Status>;
     Status(const RedisClient::ptr &c) : _c(c) {}
     void append(const std::string &uid, std::chrono::seconds ttl = kStatusTtl) {
-        try { _c->set(key::kStatus + uid, "1", ttl); }
+        try { _c->set(key::kStatus + uid, "1", randomized_ttl(ttl)); }
         catch(std::exception &e) { LOG_ERROR("Status.append 失败 {}: {}", uid, e.what()); }
     }
     void remove(const std::string &uid) {
@@ -617,7 +617,7 @@ class Status
     }
     /* brief: 心跳续期 */
     void touch(const std::string &uid, std::chrono::seconds ttl = kStatusTtl) {
-        try { _c->expire(key::kStatus + uid, ttl); }
+        try { _c->expire(key::kStatus + uid, randomized_ttl(ttl)); }
         catch(std::exception &e) { LOG_ERROR("Status.touch 失败 {}: {}", uid, e.what()); }
     }
 private:
@@ -631,7 +631,7 @@ class Codes
     Codes(const RedisClient::ptr &c) : _c(c) {}
     void append(const std::string &cid, const std::string &code,
                 std::chrono::seconds ttl = kCodeTtl) {
-        try { _c->set(key::kVerifyCode + cid, code, ttl); }
+        try { _c->set(key::kVerifyCode + cid, code, randomized_ttl(ttl)); }
         catch(std::exception &e) { LOG_ERROR("Codes.append 失败 {}: {}", cid, e.what()); }
     }
     void remove(const std::string &cid) {
@@ -776,9 +776,18 @@ class DeviceSet
 
     /* brief: 用户某设备上线 */
     void add(const std::string &uid, const std::string &device_id) {
-        try { _c->sadd(key::kDeviceSet + uid, device_id); }
+        try {
+            const std::string k = key::kDeviceSet + uid;
+            _c->sadd(k, device_id);
+            _c->expire(k, randomized_ttl(kSessionTtl));
+        }
         catch(std::exception &e) { LOG_ERROR("DeviceSet.add 失败 {}-{}: {}", uid, device_id, e.what()); }
     }
+    /* brief: 用户设备活动时续期 */
+    void touch(const std::string &uid, std::chrono::seconds ttl = kSessionTtl) {
+        try { _c->expire(key::kDeviceSet + uid, randomized_ttl(ttl)); }
+        catch(std::exception &e) { LOG_ERROR("DeviceSet.touch 失败 {}: {}", uid, e.what()); }
+    }
     /* brief: 用户某设备下线 */
     void remove(const std::string &uid, const std::string &device_id) {
         try { _c->srem(key::kDeviceSet + uid, device_id); }
@@ -1192,7 +1201,7 @@ class OnlineRoute
 {
 public:
     using ptr = std::shared_ptr<OnlineRoute>;
-    OnlineRoute(const RedisClient::ptr &c) : _c(c) {}
+    OnlineRoute(const RedisClient::ptr &c) : _c(c), _devices(c) {}
 
     /* brief: 设备上线 — HSET uid did instance */
     void bind(const std::string &uid, const std::string &device_id,
@@ -1202,19 +1211,26 @@ class OnlineRoute
             std::string k = key::online_key(uid);
             _c->hset(k, device_id, push_instance);
             _c->expire(k, randomized_ttl(ttl));
+            _devices.add(uid, device_id);
         } catch(std::exception &e) {
             LOG_ERROR("OnlineRoute.bind 失败 {}-{}-{}: {}", uid, device_id, push_instance, e.what());
         }
     }
     /* brief: 心跳续期（续整个 uid 的 HASH） */
     void touch(const std::string &uid, std::chrono::seconds ttl = kOnlineTtl) {
-        try { _c->expire(key::online_key(uid), randomized_ttl(ttl)); }
+        try {
+            _c->expire(key::online_key(uid), randomized_ttl(ttl));
+            _devices.touch(uid);
+        }
         catch(std::exception &e) { LOG_ERROR("OnlineRoute.touch 失败 {}: {}", uid, e.what()); }
     }
     /* brief: 设备下线 — HDEL uid did */
     void unbind(const std::string &uid, const std::string &device_id,
                 const std::string &push_instance) {
-        try { _c->hdel(key::online_key(uid), device_id); }
+        try {
+            _c->hdel(key::online_key(uid), device_id);
+            _devices.remove(uid, device_id);
+        }
         catch(std::exception &e) { LOG_ERROR("OnlineRoute.unbind 失败 {}-{}-{}: {}", uid, device_id, push_instance, e.what()); }
     }
     /* brief: 取用户所有在线设备 → device_id 列表 */
@@ -1252,6 +1268,7 @@ class OnlineRoute
     }
 private:
     RedisClient::ptr _c;
+    DeviceSet _devices;
 };
 
 // =============================================================================
@@ -1484,8 +1501,9 @@ class UnackedPush
             std::string member = std::to_string(user_seq) + ":" + payload_b64;
             _c->zadd(k, member, static_cast<double>(score_ts));
             _c->hset(ik, std::to_string(user_seq), payload_b64);
-            _c->expire(k, ttl);
-            _c->expire(ik, ttl);
+            const auto effective_ttl = randomized_ttl(ttl);
+            _c->expire(k, effective_ttl);
+            _c->expire(ik, effective_ttl);
         } catch(std::exception &e) {
             LOG_ERROR("UnackedPush.push 失败 {}-{}-{}: {}", uid, device_id, user_seq, e.what());
         }
@@ -1549,8 +1567,9 @@ class UnackedPush
                     _c->zadd(k, member, static_cast<double>(now), sw::redis::UpdateType::EXIST);
                 }
             }
-            _c->expire(k, ttl);
-            _c->expire(ik, ttl);
+            const auto effective_ttl = randomized_ttl(ttl);
+            _c->expire(k, effective_ttl);
+            _c->expire(ik, effective_ttl);
         } catch(std::exception &e) {
             LOG_ERROR("UnackedPush.bump_score 失败 {}-{}-{}: {}", uid, device_id, e.what());
         }
diff --git a/common/infra/metrics.hpp b/common/infra/metrics.hpp
index 67984a2..9c09335 100644
--- a/common/infra/metrics.hpp
+++ b/common/infra/metrics.hpp
@@ -27,6 +27,7 @@ inline bvar::Adder<long> g_redis_circuit_open_total("redis_circuit_open_total");
 inline bvar::Adder<long> g_redis_circuit_rejected_total("redis_circuit_rejected_total");
 inline bvar::Adder<long> g_redis_circuit_recovered_total("redis_circuit_recovered_total");
 inline bvar::Adder<long> g_redis_call_failure_total("redis_call_failure_total");
+inline bvar::Adder<long> g_redis_mutex_retry_total("redis_mutex_retry_total");
 inline bvar::Adder<long> g_rate_limit_local_fallback_total("rate_limit_local_fallback_total");
 inline bvar::Adder<long> g_rate_limit_local_rejected_total("rate_limit_local_rejected_total");
 
diff --git a/common/utils/redis_mutex.hpp b/common/utils/redis_mutex.hpp
index d1931ca..17468a0 100644
--- a/common/utils/redis_mutex.hpp
+++ b/common/utils/redis_mutex.hpp
@@ -1,5 +1,6 @@
 #pragma once
 
+#include <algorithm>
 #include <chrono>
 #include <memory>
 #include <random>
@@ -8,6 +9,7 @@
 #include <vector>
 #include "dao/data_redis.hpp"
 #include "infra/logger.hpp"
+#include "infra/metrics.hpp"
 
 namespace chatnow {
 
@@ -30,11 +32,19 @@ class RedisMutex {
 
     bool try_lock(std::chrono::milliseconds timeout = std::chrono::milliseconds(100)) {
         auto deadline = std::chrono::steady_clock::now() + timeout;
+        int backoff_ms = 5;
         while (std::chrono::steady_clock::now() < deadline) {
             bool ok = _redis->set(_key, _token, std::chrono::milliseconds(_ttl_ms),
                                   sw::redis::UpdateType::NOT_EXIST);
             if (ok) { _locked = true; return true; }
-            std::this_thread::sleep_for(std::chrono::milliseconds(5));
+            metrics::g_redis_mutex_retry_total << 1;
+            std::uniform_int_distribution<int> jitter(0, backoff_ms / 2);
+            auto delay = std::chrono::milliseconds(backoff_ms + jitter(rng_()));
+            auto remaining = std::chrono::duration_cast<std::chrono::milliseconds>(
+                deadline - std::chrono::steady_clock::now());
+            if (remaining <= std::chrono::milliseconds::zero()) break;
+            std::this_thread::sleep_for(std::min(delay, remaining));
+            backoff_ms = std::min(backoff_ms * 2, 20);
         }
         return false;
     }
@@ -58,6 +68,11 @@ class RedisMutex {
     }
 
 private:
+    static std::mt19937 &rng_() {
+        static thread_local std::mt19937 rng(std::random_device{}());
+        return rng;
+    }
+
     static std::string generate_token_() {
         static thread_local std::mt19937_64 rng(std::random_device{}());
         std::uniform_int_distribution<unsigned long long> dist;
diff --git a/tests/func/cache_test.go b/tests/func/cache_test.go
index 736d485..220c586 100644
--- a/tests/func/cache_test.go
+++ b/tests/func/cache_test.go
@@ -3,14 +3,24 @@
 package func_test
 
 import (
+	"bufio"
 	"bytes"
+	"crypto/rand"
+	"encoding/base64"
+	"encoding/binary"
 	"fmt"
+	"net"
+	"net/http"
+	"net/url"
+	"os"
 	"os/exec"
+	"strings"
 	"sync"
 	"testing"
 	"time"
 
 	"github.com/stretchr/testify/require"
+	"google.golang.org/protobuf/encoding/protowire"
 	"google.golang.org/protobuf/proto"
 
 	"chatnow-tests/pkg/client"
@@ -71,6 +81,174 @@ func sendCacheTestMessage(t testing.TB, user *client.HTTPClient, convID, suffix
 	require.NoError(t, sendCacheTestMessageResult(user, convID, suffix))
 }
 
+type cacheTestWebSocket struct {
+	conn net.Conn
+}
+
+func openCacheTestWebSocket(t testing.TB) *cacheTestWebSocket {
+	t.Helper()
+	addr := HTTP.Config().Target.WebsocketAddr
+	conn, err := net.DialTimeout("tcp", addr, 5*time.Second)
+	require.NoError(t, err)
+	require.NoError(t, conn.SetDeadline(time.Now().Add(5*time.Second)))
+
+	keyBytes := make([]byte, 16)
+	_, err = rand.Read(keyBytes)
+	require.NoError(t, err)
+	key := base64.StdEncoding.EncodeToString(keyBytes)
+	req := &http.Request{
+		Method: "GET",
+		URL:    &url.URL{Scheme: "http", Host: addr, Path: "/"},
+		Host:   addr,
+		Header: http.Header{
+			"Connection":            {"Upgrade"},
+			"Upgrade":               {"websocket"},
+			"Sec-Websocket-Key":     {key},
+			"Sec-Websocket-Version": {"13"},
+		},
+	}
+	require.NoError(t, req.Write(conn))
+	rsp, err := http.ReadResponse(bufio.NewReader(conn), req)
+	require.NoError(t, err)
+	require.Equal(t, http.StatusSwitchingProtocols, rsp.StatusCode)
+	require.NoError(t, conn.SetDeadline(time.Time{}))
+	return &cacheTestWebSocket{conn: conn}
+}
+
+func (ws *cacheTestWebSocket) close() {
+	_ = ws.conn.Close()
+}
+
+func (ws *cacheTestWebSocket) writeBinary(t testing.TB, payload []byte) {
+	t.Helper()
+	frame := []byte{0x82}
+	switch {
+	case len(payload) < 126:
+		frame = append(frame, 0x80|byte(len(payload)))
+	case len(payload) <= 65535:
+		frame = append(frame, 0x80|126, byte(len(payload)>>8), byte(len(payload)))
+	default:
+		frame = append(frame, 0x80|127)
+		var size [8]byte
+		binary.BigEndian.PutUint64(size[:], uint64(len(payload)))
+		frame = append(frame, size[:]...)
+	}
+	var mask [4]byte
+	_, err := rand.Read(mask[:])
+	require.NoError(t, err)
+	frame = append(frame, mask[:]...)
+	for i, b := range payload {
+		frame = append(frame, b^mask[i%len(mask)])
+	}
+	_, err = ws.conn.Write(frame)
+	require.NoError(t, err)
+}
+
+func appendProtoString(dst []byte, field protowire.Number, value string) []byte {
+	dst = protowire.AppendTag(dst, field, protowire.BytesType)
+	return protowire.AppendString(dst, value)
+}
+
+func cacheTestAuthNotify(accessToken, deviceID string) []byte {
+	var auth []byte
+	auth = appendProtoString(auth, 1, accessToken)
+	auth = appendProtoString(auth, 2, deviceID)
+	var notify []byte
+	notify = protowire.AppendTag(notify, 2, protowire.VarintType)
+	notify = protowire.AppendVarint(notify, 49) // CLIENT_AUTH
+	notify = protowire.AppendTag(notify, 10, protowire.BytesType)
+	return protowire.AppendBytes(notify, auth)
+}
+
+func cacheTestHeartbeatNotify(uid string) []byte {
+	var heartbeat []byte
+	heartbeat = appendProtoString(heartbeat, 1, uid)
+	var notify []byte
+	notify = protowire.AppendTag(notify, 2, protowire.VarintType)
+	notify = protowire.AppendVarint(notify, 51) // CLIENT_HEARTBEAT
+	notify = protowire.AppendTag(notify, 9, protowire.BytesType)
+	return protowire.AppendBytes(notify, heartbeat)
+}
+
+func requireJitteredRedisTTL(t testing.TB, key string, base time.Duration) time.Duration {
+	t.Helper()
+	ttl := verify.RedisTTL(t, key)
+	require.GreaterOrEqual(t, ttl, base*8/10-2*time.Second, key)
+	require.LessOrEqual(t, ttl, base*12/10+2*time.Second, key)
+	return ttl
+}
+
+// FN-CA-04 | cache expirations are bounded, varied, and paired unacked keys
+// share one randomized sample per push operation.
+func TestFN_CA_TTLJitter(t *testing.T) {
+	const sampleCount = 20
+	const sessionTTL = 7 * 24 * time.Hour
+	deviceTTLs := make(map[time.Duration]struct{}, sampleCount)
+	connections := make([]*cacheTestWebSocket, 0, sampleCount)
+	t.Cleanup(func() {
+		for _, ws := range connections {
+			ws.close()
+		}
+	})
+
+	for i := 0; i < sampleCount; i++ {
+		user, _, _ := fixture.RegisterAndLogin(t, HTTP)
+		ws := openCacheTestWebSocket(t)
+		connections = append(connections, ws)
+		ws.writeBinary(t, cacheTestAuthNotify(user.AccessToken, "default_device"))
+		ws.writeBinary(t, cacheTestHeartbeatNotify(user.UserID))
+
+		deviceKey := "im:dev:" + user.UserID
+		require.Eventually(t, func() bool {
+			return verify.RedisCLI(t, "EXISTS", deviceKey) == "1"
+		}, 5*time.Second, 50*time.Millisecond, deviceKey)
+		deviceTTLs[requireJitteredRedisTTL(t, deviceKey, sessionTTL)] = struct{}{}
+	}
+	require.GreaterOrEqual(t, len(deviceTTLs), 2,
+		"20 independently randomized device TTLs must not all be identical")
+
+	if os.Getenv("SMTP_HOST") != "" {
+		codeRsp := &identity.SendVerifyCodeRsp{}
+		require.NoError(t, HTTP.DoNoAuth("/service/identity/send_verify_code",
+			&identity.SendVerifyCodeReq{
+				RequestId: client.NewRequestID(),
+				Destination: &identity.SendVerifyCodeReq_Email{
+					Email: fmt.Sprintf("ttl-%s@example.com", strings.ToLower(client.NewRequestID())),
+				},
+			}, codeRsp))
+		require.True(t, codeRsp.GetHeader().GetSuccess(), codeRsp.GetHeader().GetErrorMessage())
+		requireJitteredRedisTTL(t, "im:code:"+codeRsp.GetVerifyCodeId(), 5*time.Minute)
+	}
+
+	sender, recipient, convID := fixture.MakeFriends(t, HTTP)
+	recipientWS := openCacheTestWebSocket(t)
+	connections = append(connections, recipientWS)
+	recipientWS.writeBinary(t, cacheTestAuthNotify(recipient.AccessToken, "default_device"))
+	deviceKey := "im:dev:" + recipient.UserID
+	require.Eventually(t, func() bool {
+		return verify.RedisCLI(t, "EXISTS", deviceKey) == "1"
+	}, 5*time.Second, 50*time.Millisecond, deviceKey)
+
+	sendCacheTestMessage(t, sender, convID, "ttl-jitter")
+	unackedKey := fmt.Sprintf("im:unack:{%s:default_device}", recipient.UserID)
+	unackedIndexKey := fmt.Sprintf("im:unack:idx:{%s:default_device}", recipient.UserID)
+	require.Eventually(t, func() bool {
+		return verify.RedisCLI(t, "EXISTS", unackedKey) == "1" &&
+			verify.RedisCLI(t, "EXISTS", unackedIndexKey) == "1"
+	}, 10*time.Second, 100*time.Millisecond, unackedKey)
+	unackedTTL := requireJitteredRedisTTL(t, unackedKey, sessionTTL)
+	indexTTL := requireJitteredRedisTTL(t, unackedIndexKey, sessionTTL)
+	require.LessOrEqual(t, absDuration(unackedTTL-indexTTL), time.Second,
+		"paired unacked keys must share one TTL sample")
+}
+
+func absDuration(value time.Duration) time.Duration {
+	if value < 0 {
+		return -value
+	}
+	return value
+}
+
 // FN-CA-05 | healthy Redis applies the distributed message rate limit.
 func TestFN_CA_RateLimit(t *testing.T) {
 	user, peer, convID := fixture.MakeFriends(t, HTTP)

From c5cce1c4bfc3a72a687ae12a96ec6449244a19cb Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 12:37:17 +0800
Subject: [PATCH 12/47] fix: make cache TTL updates atomic

---
 common/dao/data_redis.hpp         | 203 +++++++++++++++++++++++++-----
 common/utils/redis_keys.hpp       |   4 +
 presence/source/presence_server.h |   4 +-
 push/source/push_server.h         |  17 +--
 tests/func/cache_test.go          | 105 ++++++++++------
 5 files changed, 256 insertions(+), 77 deletions(-)

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 105dd86..39b49ca 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -777,35 +777,53 @@ class DeviceSet
     /* brief: 用户某设备上线 */
     void add(const std::string &uid, const std::string &device_id) {
         try {
-            const std::string k = key::kDeviceSet + uid;
-            _c->sadd(k, device_id);
-            _c->expire(k, randomized_ttl(kSessionTtl));
+            const auto effective_ttl = randomized_ttl(kSessionTtl);
+            std::vector<std::string> keys = {key::device_set_key(uid)};
+            std::vector<std::string> args = {
+                device_id, std::to_string(effective_ttl.count())};
+            _c->eval<long long>(kAddLua, keys.begin(), keys.end(),
+                                args.begin(), args.end());
         }
         catch(std::exception &e) { LOG_ERROR("DeviceSet.add 失败 {}-{}: {}", uid, device_id, e.what()); }
     }
     /* brief: 用户设备活动时续期 */
     void touch(const std::string &uid, std::chrono::seconds ttl = kSessionTtl) {
-        try { _c->expire(key::kDeviceSet + uid, randomized_ttl(ttl)); }
+        try { _c->expire(key::device_set_key(uid), randomized_ttl(ttl)); }
         catch(std::exception &e) { LOG_ERROR("DeviceSet.touch 失败 {}: {}", uid, e.what()); }
     }
     /* brief: 用户某设备下线 */
     void remove(const std::string &uid, const std::string &device_id) {
-        try { _c->srem(key::kDeviceSet + uid, device_id); }
+        try { _c->srem(key::device_set_key(uid), device_id); }
         catch(std::exception &e) { LOG_ERROR("DeviceSet.rem 失败 {}-{}: {}", uid, device_id, e.what()); }
     }
     /* brief: 取用户当前所有在线设备 */
     std::vector<std::string> list(const std::string &uid) {
         std::vector<std::string> res;
-        try { _c->smembers(key::kDeviceSet + uid, std::inserter(res, res.end())); }
+        try { _c->smembers(key::device_set_key(uid), std::inserter(res, res.end())); }
         catch(std::exception &e) { LOG_ERROR("DeviceSet.list 失败 {}: {}", uid, e.what()); }
         return res;
     }
     /* brief: 用户是否有任意在线设备 */
     bool any(const std::string &uid) {
-        try { return _c->scard(key::kDeviceSet + uid) > 0; }
+        try { return _c->scard(key::device_set_key(uid)) > 0; }
         catch(std::exception &e) { LOG_ERROR("DeviceSet.any 失败 {}: {}", uid, e.what()); return false; }
     }
 private:
+    static constexpr const char *kAddLua = R"lua(
+local function key_type(k)
+    local t = redis.call('TYPE', k)
+    if type(t) == 'table' then return t.ok end
+    return t
+end
+local ttl = tonumber(ARGV[2])
+if not ttl or ttl <= 0 then return redis.error_reply('invalid ttl') end
+local t = key_type(KEYS[1])
+if t ~= 'none' and t ~= 'set' then return redis.error_reply('device key wrong type') end
+redis.call('SADD', KEYS[1], ARGV[1])
+redis.call('EXPIRE', KEYS[1], ttl)
+return 1
+)lua";
+
     RedisClient::ptr _c;
 };
 
@@ -1201,17 +1219,22 @@ class OnlineRoute
 {
 public:
     using ptr = std::shared_ptr<OnlineRoute>;
-    OnlineRoute(const RedisClient::ptr &c) : _c(c), _devices(c) {}
+    OnlineRoute(const RedisClient::ptr &c) : _c(c) {}
 
     /* brief: 设备上线 — HSET uid did instance */
     void bind(const std::string &uid, const std::string &device_id,
               const std::string &push_instance,
               std::chrono::seconds ttl = kOnlineTtl) {
         try {
-            std::string k = key::online_key(uid);
-            _c->hset(k, device_id, push_instance);
-            _c->expire(k, randomized_ttl(ttl));
-            _devices.add(uid, device_id);
+            const auto route_ttl = randomized_ttl(ttl);
+            const auto device_ttl = randomized_ttl(kSessionTtl);
+            std::vector<std::string> keys = {
+                key::online_key(uid), key::device_set_key(uid)};
+            std::vector<std::string> args = {
+                device_id, push_instance, std::to_string(route_ttl.count()),
+                std::to_string(device_ttl.count())};
+            _c->eval<long long>(kBindLua, keys.begin(), keys.end(),
+                                args.begin(), args.end());
         } catch(std::exception &e) {
             LOG_ERROR("OnlineRoute.bind 失败 {}-{}-{}: {}", uid, device_id, push_instance, e.what());
         }
@@ -1219,8 +1242,15 @@ class OnlineRoute
     /* brief: 心跳续期（续整个 uid 的 HASH） */
     void touch(const std::string &uid, std::chrono::seconds ttl = kOnlineTtl) {
         try {
-            _c->expire(key::online_key(uid), randomized_ttl(ttl));
-            _devices.touch(uid);
+            const auto route_ttl = randomized_ttl(ttl);
+            const auto device_ttl = randomized_ttl(kSessionTtl);
+            std::vector<std::string> keys = {
+                key::online_key(uid), key::device_set_key(uid)};
+            std::vector<std::string> args = {
+                std::to_string(route_ttl.count()),
+                std::to_string(device_ttl.count())};
+            _c->eval<long long>(kTouchLua, keys.begin(), keys.end(),
+                                args.begin(), args.end());
         }
         catch(std::exception &e) { LOG_ERROR("OnlineRoute.touch 失败 {}: {}", uid, e.what()); }
     }
@@ -1228,8 +1258,11 @@ class OnlineRoute
     void unbind(const std::string &uid, const std::string &device_id,
                 const std::string &push_instance) {
         try {
-            _c->hdel(key::online_key(uid), device_id);
-            _devices.remove(uid, device_id);
+            std::vector<std::string> keys = {
+                key::online_key(uid), key::device_set_key(uid)};
+            std::vector<std::string> args = {device_id, push_instance};
+            _c->eval<long long>(kUnbindLua, keys.begin(), keys.end(),
+                                args.begin(), args.end());
         }
         catch(std::exception &e) { LOG_ERROR("OnlineRoute.unbind 失败 {}-{}-{}: {}", uid, device_id, push_instance, e.what()); }
     }
@@ -1267,8 +1300,68 @@ class OnlineRoute
         catch(std::exception &e) { LOG_ERROR("OnlineRoute.online 失败 {}: {}", uid, e.what()); return false; }
     }
 private:
+    static constexpr const char *kBindLua = R"lua(
+local function key_type(k)
+    local t = redis.call('TYPE', k)
+    if type(t) == 'table' then return t.ok end
+    return t
+end
+local route_ttl = tonumber(ARGV[3])
+local device_ttl = tonumber(ARGV[4])
+if not route_ttl or route_ttl <= 0 or not device_ttl or device_ttl <= 0 then
+    return redis.error_reply('invalid ttl')
+end
+local rt = key_type(KEYS[1])
+local dt = key_type(KEYS[2])
+if rt ~= 'none' and rt ~= 'hash' then return redis.error_reply('route key wrong type') end
+if dt ~= 'none' and dt ~= 'set' then return redis.error_reply('device key wrong type') end
+redis.call('HSET', KEYS[1], ARGV[1], ARGV[2])
+redis.call('SADD', KEYS[2], ARGV[1])
+redis.call('EXPIRE', KEYS[1], route_ttl)
+redis.call('EXPIRE', KEYS[2], device_ttl)
+return 1
+)lua";
+
+    static constexpr const char *kTouchLua = R"lua(
+local route_ttl = tonumber(ARGV[1])
+local device_ttl = tonumber(ARGV[2])
+if not route_ttl or route_ttl <= 0 or not device_ttl or device_ttl <= 0 then
+    return redis.error_reply('invalid ttl')
+end
+redis.call('EXPIRE', KEYS[1], route_ttl)
+redis.call('EXPIRE', KEYS[2], device_ttl)
+return 1
+)lua";
+
+    static constexpr const char *kUnbindLua = R"lua(
+local function key_type(k)
+    local t = redis.call('TYPE', k)
+    if type(t) == 'table' then return t.ok end
+    return t
+end
+local rt = key_type(KEYS[1])
+local dt = key_type(KEYS[2])
+if rt ~= 'none' and rt ~= 'hash' then return redis.error_reply('route key wrong type') end
+if dt ~= 'none' and dt ~= 'set' then return redis.error_reply('device key wrong type') end
+local removed = 0
+if ARGV[1] == '' then
+    local entries = redis.call('HGETALL', KEYS[1])
+    for i = 1, #entries, 2 do
+        if entries[i + 1] == ARGV[2] then
+            redis.call('HDEL', KEYS[1], entries[i])
+            redis.call('SREM', KEYS[2], entries[i])
+            removed = removed + 1
+        end
+    end
+elseif redis.call('HGET', KEYS[1], ARGV[1]) == ARGV[2] then
+    redis.call('HDEL', KEYS[1], ARGV[1])
+    redis.call('SREM', KEYS[2], ARGV[1])
+    removed = 1
+end
+return removed
+)lua";
+
     RedisClient::ptr _c;
-    DeviceSet _devices;
 };
 
 // =============================================================================
@@ -1499,11 +1592,13 @@ class UnackedPush
             std::string k = key_for(uid, device_id);
             std::string ik = idx_key_for(uid, device_id);
             std::string member = std::to_string(user_seq) + ":" + payload_b64;
-            _c->zadd(k, member, static_cast<double>(score_ts));
-            _c->hset(ik, std::to_string(user_seq), payload_b64);
             const auto effective_ttl = randomized_ttl(ttl);
-            _c->expire(k, effective_ttl);
-            _c->expire(ik, effective_ttl);
+            std::vector<std::string> keys = {k, ik};
+            std::vector<std::string> args = {
+                std::to_string(score_ts), member, std::to_string(user_seq), payload_b64,
+                std::to_string(effective_ttl.count())};
+            _c->eval<long long>(kPushLua, keys.begin(), keys.end(),
+                                args.begin(), args.end());
         } catch(std::exception &e) {
             LOG_ERROR("UnackedPush.push 失败 {}-{}-{}: {}", uid, device_id, user_seq, e.what());
         }
@@ -1560,22 +1655,68 @@ class UnackedPush
             std::string k = key_for(uid, device_id);
             std::string ik = idx_key_for(uid, device_id);
             long long now = static_cast<long long>(time(nullptr));
+            const auto effective_ttl = randomized_ttl(ttl);
+            std::vector<std::string> keys = {k, ik};
+            std::vector<std::string> args = {
+                std::to_string(effective_ttl.count()), std::to_string(now)};
+            args.reserve(2 + user_seqs.size());
             for (unsigned long seq : user_seqs) {
-                auto payload = _c->hget(ik, std::to_string(seq));
-                if (payload) {
-                    std::string member = std::to_string(seq) + ":" + *payload;
-                    _c->zadd(k, member, static_cast<double>(now), sw::redis::UpdateType::EXIST);
-                }
+                args.push_back(std::to_string(seq));
             }
-            const auto effective_ttl = randomized_ttl(ttl);
-            _c->expire(k, effective_ttl);
-            _c->expire(ik, effective_ttl);
+            _c->eval<long long>(kBumpScoreLua, keys.begin(), keys.end(),
+                                args.begin(), args.end());
         } catch(std::exception &e) {
             LOG_ERROR("UnackedPush.bump_score 失败 {}-{}-{}: {}", uid, device_id, e.what());
         }
     }
 
 private:
+    static constexpr const char *kPushLua = R"lua(
+local function key_type(k)
+    local t = redis.call('TYPE', k)
+    if type(t) == 'table' then return t.ok end
+    return t
+end
+local score = tonumber(ARGV[1])
+local ttl = tonumber(ARGV[5])
+if not score or not ttl or ttl <= 0 then return redis.error_reply('invalid arguments') end
+local zt = key_type(KEYS[1])
+local ht = key_type(KEYS[2])
+if zt ~= 'none' and zt ~= 'zset' then return redis.error_reply('unacked key wrong type') end
+if ht ~= 'none' and ht ~= 'hash' then return redis.error_reply('unacked index wrong type') end
+redis.call('ZADD', KEYS[1], score, ARGV[2])
+redis.call('HSET', KEYS[2], ARGV[3], ARGV[4])
+redis.call('EXPIRE', KEYS[1], ttl)
+redis.call('EXPIRE', KEYS[2], ttl)
+return 1
+)lua";
+
+    static constexpr const char *kBumpScoreLua = R"lua(
+local function key_type(k)
+    local t = redis.call('TYPE', k)
+    if type(t) == 'table' then return t.ok end
+    return t
+end
+local ttl = tonumber(ARGV[1])
+local score = tonumber(ARGV[2])
+if not ttl or ttl <= 0 or not score then return redis.error_reply('invalid arguments') end
+local zt = key_type(KEYS[1])
+local ht = key_type(KEYS[2])
+if zt ~= 'none' and zt ~= 'zset' then return redis.error_reply('unacked key wrong type') end
+if ht ~= 'none' and ht ~= 'hash' then return redis.error_reply('unacked index wrong type') end
+local updated = 0
+for i = 3, #ARGV do
+    local payload = redis.call('HGET', KEYS[2], ARGV[i])
+    if payload then
+        redis.call('ZADD', KEYS[1], 'XX', score, ARGV[i] .. ':' .. payload)
+        updated = updated + 1
+    end
+end
+redis.call('EXPIRE', KEYS[1], ttl)
+redis.call('EXPIRE', KEYS[2], ttl)
+return updated
+)lua";
+
     RedisClient::ptr _c;
 };
 
@@ -1623,7 +1764,7 @@ class PresenceRedis
         try {
             auto k = key::presence_device_key(uid, device_id);
             _r->hset(k, "state", "ONLINE");
-            _r->expire(k, std::chrono::seconds(120));
+            _r->expire(k, randomized_ttl(std::chrono::seconds(120)));
         } catch(std::exception &e) {
             LOG_ERROR("PresenceRedis.add_device 失败 {}-{}: {}", uid, device_id, e.what());
         }
@@ -1655,7 +1796,7 @@ class PresenceRedis
         try {
             auto k = key::kPresenceTyping + uid;
             _r->sadd(k, conv_id);
-            _r->expire(k, std::chrono::seconds(10));
+            _r->expire(k, randomized_ttl(std::chrono::seconds(10)));
         } catch(std::exception &e) {
             LOG_ERROR("PresenceRedis.set_typing 失败 {}-{}: {}", uid, conv_id, e.what());
         }
diff --git a/common/utils/redis_keys.hpp b/common/utils/redis_keys.hpp
index ba0e66e..3614c44 100644
--- a/common/utils/redis_keys.hpp
+++ b/common/utils/redis_keys.hpp
@@ -92,6 +92,10 @@ inline std::string online_key(const std::string &uid) {
     return std::string(kOnline) + hash_tag(uid);
 }
 
+inline std::string device_set_key(const std::string &uid) {
+    return std::string(kDeviceSet) + hash_tag(uid);
+}
+
 inline std::string online_scan_pattern() {
     return std::string(kOnline) + "*";
 }
diff --git a/presence/source/presence_server.h b/presence/source/presence_server.h
index 220d67d..ef02b5d 100644
--- a/presence/source/presence_server.h
+++ b/presence/source/presence_server.h
@@ -13,6 +13,7 @@
 #include "error/error_codes.hpp"
 #include "error/service_error.hpp"
 #include "utils/brpc_closure.hpp"
+#include "utils/random_ttl.hpp"
 
 #include "common/types.pb.h"
 #include "common/error.pb.h"
@@ -292,7 +293,8 @@ class PresenceServiceImpl : public PresenceService {
                     std::chrono::system_clock::now().time_since_epoch()).count();
                 _redis->sadd("im:presence:typing:" + conv_id,
                              auth.user_id + ":" + std::to_string(now_ms));
-                _redis->expire("im:presence:typing:" + conv_id, std::chrono::seconds(5));
+                _redis->expire("im:presence:typing:" + conv_id,
+                               randomized_ttl(std::chrono::seconds(5)));
             } else {
                 std::vector<std::string> members;
                 _redis->smembers("im:presence:typing:" + conv_id,
diff --git a/push/source/push_server.h b/push/source/push_server.h
index 7c8afa2..100c192 100644
--- a/push/source/push_server.h
+++ b/push/source/push_server.h
@@ -41,6 +41,7 @@
 #include <thread>
 #include <chrono>
 #include <limits>
+#include <tuple>
 #include <unordered_set>
 
 namespace chatnow::push {
@@ -503,12 +504,13 @@ class PushServiceImpl : public PushService
     void _write_presence_online_(const std::string &uid, const std::string &did) {
         try {
             std::string k = key::presence_device_key(uid, did);
+            const auto effective_ttl = randomized_ttl(std::chrono::seconds(kPresenceTtlSec));
             auto pipe = _redis->pipeline();
             pipe.hset(k, "state", "ONLINE");
             pipe.hset(k, "last_active_at_ms", std::to_string(
                 std::chrono::duration_cast<std::chrono::milliseconds>(
                     std::chrono::system_clock::now().time_since_epoch()).count()));
-            pipe.expire(k, std::chrono::seconds(kPresenceTtlSec));
+            pipe.expire(k, effective_ttl);
             pipe.exec();
         } catch (std::exception &e) {
             LOG_WARN("Presence write failed uid={} did={}: {}", uid, did, e.what());
@@ -518,12 +520,13 @@ class PushServiceImpl : public PushService
     void _write_presence_offline_(const std::string &uid, const std::string &did) {
         try {
             std::string k = key::presence_device_key(uid, did);
+            const auto effective_ttl = randomized_ttl(std::chrono::seconds(kPresenceTtlSec));
             auto pipe = _redis->pipeline();
             pipe.hset(k, "state", "OFFLINE");
             pipe.hset(k, "last_active_at_ms", std::to_string(
                 std::chrono::duration_cast<std::chrono::milliseconds>(
                     std::chrono::system_clock::now().time_since_epoch()).count()));
-            pipe.expire(k, std::chrono::seconds(kPresenceTtlSec));
+            pipe.expire(k, effective_ttl);
             pipe.exec();
         } catch (std::exception &e) {
             LOG_WARN("Presence offline write failed uid={} did={}: {}", uid, did, e.what());
@@ -533,7 +536,7 @@ class PushServiceImpl : public PushService
     void _refresh_presence_ttl_(const std::string &uid, const std::string &did) {
         try {
             std::string k = key::presence_device_key(uid, did);
-            _redis->expire(k, std::chrono::seconds(kPresenceTtlSec));
+            _redis->expire(k, randomized_ttl(std::chrono::seconds(kPresenceTtlSec)));
         } catch (std::exception &e) {
             LOG_WARN("Presence TTL refresh failed uid={} did={}: {}", uid, did, e.what());
         }
@@ -809,7 +812,7 @@ class PushServiceImpl : public PushService
                 continue;
             }
 
-            std::vector<std::pair<std::string, std::string>> stale_entries;
+            std::vector<std::tuple<std::string, std::string, std::string>> stale_entries;
             // Cluster mode: for_each traverses all nodes via RedisClient::scan().
             long long cursor = 0;
             do {
@@ -824,14 +827,14 @@ class PushServiceImpl : public PushService
                     for (const auto &[did, instance] : device_map) {
                         if (std::find(online_instances.begin(), online_instances.end(), instance)
                             == online_instances.end()) {
-                            stale_entries.emplace_back(uid, did);
+                            stale_entries.emplace_back(uid, did, instance);
                         }
                     }
                 }
             } while (cursor != 0);
 
-            for (const auto &[uid, did] : stale_entries) {
-                _online_route->unbind(uid, did, "");
+            for (const auto &[uid, did, instance] : stale_entries) {
+                _online_route->unbind(uid, did, instance);
                 if (_local_route_cache) _local_route_cache->invalidate(key::local_route_cache_key(uid));
             }
             if (!stale_entries.empty())
diff --git a/tests/func/cache_test.go b/tests/func/cache_test.go
index 220c586..eef4b9e 100644
--- a/tests/func/cache_test.go
+++ b/tests/func/cache_test.go
@@ -14,6 +14,7 @@ import (
 	"net/url"
 	"os"
 	"os/exec"
+	"strconv"
 	"strings"
 	"sync"
 	"testing"
@@ -183,7 +184,6 @@ func requireJitteredRedisTTL(t testing.TB, key string, base time.Duration) time.
 func TestFN_CA_TTLJitter(t *testing.T) {
 	const sampleCount = 20
 	const sessionTTL = 7 * 24 * time.Hour
-	deviceTTLs := make(map[time.Duration]struct{}, sampleCount)
 	connections := make([]*cacheTestWebSocket, 0, sampleCount)
 	t.Cleanup(func() {
 		for _, ws := range connections {
@@ -191,23 +191,12 @@ func TestFN_CA_TTLJitter(t *testing.T) {
 		}
 	})
 
-	for i := 0; i < sampleCount; i++ {
-		user, _, _ := fixture.RegisterAndLogin(t, HTTP)
-		ws := openCacheTestWebSocket(t)
-		connections = append(connections, ws)
-		ws.writeBinary(t, cacheTestAuthNotify(user.AccessToken, "default_device"))
-		ws.writeBinary(t, cacheTestHeartbeatNotify(user.UserID))
+	t.Log("Session and Status DAOs have no production call sites; the ignored Task 5 DAO harness covers append/touch")
 
-		deviceKey := "im:dev:" + user.UserID
-		require.Eventually(t, func() bool {
-			return verify.RedisCLI(t, "EXISTS", deviceKey) == "1"
-		}, 5*time.Second, 50*time.Millisecond, deviceKey)
-		deviceTTLs[requireJitteredRedisTTL(t, deviceKey, sessionTTL)] = struct{}{}
-	}
-	require.GreaterOrEqual(t, len(deviceTTLs), 2,
-		"20 independently randomized device TTLs must not all be identical")
-
-	if os.Getenv("SMTP_HOST") != "" {
+	t.Run("Codes", func(t *testing.T) {
+		if os.Getenv("SMTP_HOST") == "" {
+			t.Skip("SMTP_HOST is not configured; only the Codes reachable-path subtest is skipped")
+		}
 		codeRsp := &identity.SendVerifyCodeRsp{}
 		require.NoError(t, HTTP.DoNoAuth("/service/identity/send_verify_code",
 			&identity.SendVerifyCodeReq{
@@ -218,28 +207,68 @@ func TestFN_CA_TTLJitter(t *testing.T) {
 			}, codeRsp))
 		require.True(t, codeRsp.GetHeader().GetSuccess(), codeRsp.GetHeader().GetErrorMessage())
 		requireJitteredRedisTTL(t, "im:code:"+codeRsp.GetVerifyCodeId(), 5*time.Minute)
-	}
+	})
+
+	t.Run("DeviceSet", func(t *testing.T) {
+		deviceTTLs := make(map[time.Duration]struct{}, sampleCount)
+		var heartbeatBefore time.Duration
+		for i := 0; i < sampleCount; i++ {
+			user, _, _ := fixture.RegisterAndLogin(t, HTTP)
+			ws := openCacheTestWebSocket(t)
+			connections = append(connections, ws)
+			ws.writeBinary(t, cacheTestAuthNotify(user.AccessToken, "default_device"))
+
+			deviceKey := fmt.Sprintf("im:dev:{%s}", user.UserID)
+			require.Eventually(t, func() bool {
+				return verify.RedisCLI(t, "EXISTS", deviceKey) == "1"
+			}, 5*time.Second, 50*time.Millisecond, deviceKey)
+			if i == 0 {
+				verify.RedisCLI(t, "EXPIRE", deviceKey, "60")
+				heartbeatBefore = verify.RedisTTL(t, deviceKey)
+				ws.writeBinary(t, cacheTestHeartbeatNotify(user.UserID))
+				require.Eventually(t, func() bool {
+					seconds, err := strconv.ParseInt(verify.RedisCLI(t, "TTL", deviceKey), 10, 64)
+					return err == nil && time.Duration(seconds)*time.Second > heartbeatBefore+24*time.Hour
+				}, 5*time.Second, 50*time.Millisecond, "heartbeat must renew DeviceSet TTL")
+			} else {
+				ws.writeBinary(t, cacheTestHeartbeatNotify(user.UserID))
+			}
+			deviceTTLs[requireJitteredRedisTTL(t, deviceKey, sessionTTL)] = struct{}{}
+		}
+		require.GreaterOrEqual(t, len(deviceTTLs), 2,
+			"20 independently randomized device TTLs must not all be identical")
+	})
 
-	sender, recipient, convID := fixture.MakeFriends(t, HTTP)
-	recipientWS := openCacheTestWebSocket(t)
-	connections = append(connections, recipientWS)
-	recipientWS.writeBinary(t, cacheTestAuthNotify(recipient.AccessToken, "default_device"))
-	deviceKey := "im:dev:" + recipient.UserID
-	require.Eventually(t, func() bool {
-		return verify.RedisCLI(t, "EXISTS", deviceKey) == "1"
-	}, 5*time.Second, 50*time.Millisecond, deviceKey)
-
-	sendCacheTestMessage(t, sender, convID, "ttl-jitter")
-	unackedKey := fmt.Sprintf("im:unack:{%s:default_device}", recipient.UserID)
-	unackedIndexKey := fmt.Sprintf("im:unack:idx:{%s:default_device}", recipient.UserID)
-	require.Eventually(t, func() bool {
-		return verify.RedisCLI(t, "EXISTS", unackedKey) == "1" &&
-			verify.RedisCLI(t, "EXISTS", unackedIndexKey) == "1"
-	}, 10*time.Second, 100*time.Millisecond, unackedKey)
-	unackedTTL := requireJitteredRedisTTL(t, unackedKey, sessionTTL)
-	indexTTL := requireJitteredRedisTTL(t, unackedIndexKey, sessionTTL)
-	require.LessOrEqual(t, absDuration(unackedTTL-indexTTL), time.Second,
-		"paired unacked keys must share one TTL sample")
+	t.Run("UnackedPush", func(t *testing.T) {
+		sender, recipient, convID := fixture.MakeFriends(t, HTTP)
+		recipientWS := openCacheTestWebSocket(t)
+		connections = append(connections, recipientWS)
+		recipientWS.writeBinary(t, cacheTestAuthNotify(recipient.AccessToken, "default_device"))
+		deviceKey := fmt.Sprintf("im:dev:{%s}", recipient.UserID)
+		require.Eventually(t, func() bool {
+			return verify.RedisCLI(t, "EXISTS", deviceKey) == "1"
+		}, 5*time.Second, 50*time.Millisecond, deviceKey)
+
+		unackedKey := fmt.Sprintf("im:unack:{%s:default_device}", recipient.UserID)
+		unackedIndexKey := fmt.Sprintf("im:unack:idx:{%s:default_device}", recipient.UserID)
+		initial, err := strconv.Atoi(verify.RedisCLI(t, "ZCARD", unackedKey))
+		require.NoError(t, err)
+		unackedTTLs := make(map[time.Duration]struct{})
+		for i := 0; i < 12; i++ {
+			sendCacheTestMessage(t, sender, convID, fmt.Sprintf("ttl-jitter-%d", i))
+			require.Eventually(t, func() bool {
+				count, parseErr := strconv.Atoi(verify.RedisCLI(t, "ZCARD", unackedKey))
+				return parseErr == nil && count >= initial+i+1
+			}, 10*time.Second, 100*time.Millisecond, unackedKey)
+			unackedTTL := requireJitteredRedisTTL(t, unackedKey, sessionTTL)
+			indexTTL := requireJitteredRedisTTL(t, unackedIndexKey, sessionTTL)
+			require.LessOrEqual(t, absDuration(unackedTTL-indexTTL), time.Second,
+				"paired unacked keys must share one TTL sample")
+			unackedTTLs[unackedTTL] = struct{}{}
+		}
+		require.GreaterOrEqual(t, len(unackedTTLs), 2,
+			"repeated UnackedPush operations must produce varied paired TTL samples")
+	})
 }
 
 func absDuration(value time.Duration) time.Duration {

From 86514c02f2f5ac26658fa63999fdbf2c3de1f084 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 12:53:35 +0800
Subject: [PATCH 13/47] test: harden cache TTL verification

---
 tests/func/cache_test.go  | 208 +++++++++++++++++++++++++++++++++++---
 tests/pkg/verify/redis.go |  14 ++-
 2 files changed, 206 insertions(+), 16 deletions(-)

diff --git a/tests/func/cache_test.go b/tests/func/cache_test.go
index eef4b9e..7966a40 100644
--- a/tests/func/cache_test.go
+++ b/tests/func/cache_test.go
@@ -14,12 +14,16 @@ import (
 	"net/url"
 	"os"
 	"os/exec"
+	"path/filepath"
+	"regexp"
+	"runtime"
 	"strconv"
 	"strings"
 	"sync"
 	"testing"
 	"time"
 
+	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 	"google.golang.org/protobuf/encoding/protowire"
 	"google.golang.org/protobuf/proto"
@@ -179,6 +183,182 @@ func requireJitteredRedisTTL(t testing.TB, key string, base time.Duration) time.
 	return ttl
 }
 
+var _ func(...string) (string, error) = verify.RedisCLIResult
+
+func readRepoSource(t testing.TB, relativePath string) string {
+	t.Helper()
+	_, currentFile, _, ok := runtime.Caller(0)
+	require.True(t, ok, "locate cache_test.go")
+	repoRoot := filepath.Clean(filepath.Join(filepath.Dir(currentFile), "..", ".."))
+	content, err := os.ReadFile(filepath.Join(repoRoot, filepath.FromSlash(relativePath)))
+	require.NoError(t, err, relativePath)
+	return string(content)
+}
+
+func normalizedSourceSection(source, start, end string) (string, error) {
+	startAt := strings.Index(source, start)
+	if startAt < 0 {
+		return "", fmt.Errorf("missing source section %q", start)
+	}
+	endAt := strings.Index(source[startAt+len(start):], end)
+	if endAt < 0 {
+		return "", fmt.Errorf("missing end marker %q for %q", end, start)
+	}
+	section := source[startAt : startAt+len(start)+endAt]
+	return strings.Join(strings.Fields(section), " "), nil
+}
+
+func requireSourceFragments(sectionName, section string, fragments ...string) error {
+	for _, fragment := range fragments {
+		normalized := strings.Join(strings.Fields(fragment), " ")
+		if !strings.Contains(section, normalized) {
+			return fmt.Errorf("%s missing contract fragment %q", sectionName, normalized)
+		}
+	}
+	return nil
+}
+
+func checkCacheTTLSourceContracts(source string) error {
+	sections := make(map[string]string)
+	markers := [][3]string{
+		{"Session", "class Session", "class Status"},
+		{"Status", "class Status", "class Codes"},
+		{"Codes", "class Codes", "class Seq"},
+		{"DeviceSet", "class DeviceSet", "class ReadAck"},
+		{"OnlineRoute", "class OnlineRoute", "class RateLimiter"},
+		{"UnackedPush", "class UnackedPush", "class PresenceRedis"},
+	}
+	for _, marker := range markers {
+		section, err := normalizedSourceSection(source, marker[1], marker[2])
+		if err != nil {
+			return err
+		}
+		sections[marker[0]] = section
+	}
+
+	rules := []struct {
+		name      string
+		fragments []string
+	}{
+		{"Session", []string{
+			"_c->set(key::kSession + ssid, uid, randomized_ttl(ttl))",
+			"_c->expire(key::kSession + ssid, randomized_ttl(ttl))",
+		}},
+		{"Status", []string{
+			"_c->set(key::kStatus + uid, \"1\", randomized_ttl(ttl))",
+			"_c->expire(key::kStatus + uid, randomized_ttl(ttl))",
+		}},
+		{"Codes", []string{
+			"_c->set(key::kVerifyCode + cid, code, randomized_ttl(ttl))",
+		}},
+		{"DeviceSet", []string{
+			"randomized_ttl(kSessionTtl)",
+			"key::device_set_key(uid)",
+			"_c->eval<long long>(kAddLua",
+			"redis.call('SADD', KEYS[1], ARGV[1])",
+			"redis.call('EXPIRE', KEYS[1], ttl)",
+		}},
+		{"OnlineRoute", []string{
+			"key::online_key(uid), key::device_set_key(uid)",
+			"_c->eval<long long>(kBindLua",
+			"_c->eval<long long>(kTouchLua",
+			"_c->eval<long long>(kUnbindLua",
+			"redis.call('EXPIRE', KEYS[1], route_ttl)",
+			"redis.call('EXPIRE', KEYS[2], device_ttl)",
+		}},
+		{"UnackedPush", []string{
+			"_c->eval<long long>(kPushLua",
+			"_c->eval<long long>(kBumpScoreLua",
+			"redis.call('ZADD', KEYS[1]",
+			"redis.call('HSET', KEYS[2]",
+			"redis.call('EXPIRE', KEYS[1], ttl)",
+			"redis.call('EXPIRE', KEYS[2], ttl)",
+		}},
+	}
+	for _, rule := range rules {
+		if err := requireSourceFragments(rule.name, sections[rule.name], rule.fragments...); err != nil {
+			return err
+		}
+	}
+
+	fixedTTL := regexp.MustCompile(`_c->(?:set|expire)\([^;]*, ttl\)`)
+	for _, name := range []string{"Session", "Status", "Codes"} {
+		if fixedTTL.MatchString(sections[name]) {
+			return fmt.Errorf("%s contains a direct fixed TTL cache write", name)
+		}
+	}
+	if strings.Contains(sections["DeviceSet"], "_c->sadd(") {
+		return fmt.Errorf("DeviceSet bypasses its atomic add script")
+	}
+	if strings.Count(sections["UnackedPush"], "randomized_ttl(ttl)") != 2 {
+		return fmt.Errorf("UnackedPush push and bump_score must each sample one randomized TTL")
+	}
+	if strings.Contains(sections["UnackedPush"], "_c->zadd(") ||
+		strings.Contains(sections["UnackedPush"], "_c->hset(") {
+		return fmt.Errorf("UnackedPush bypasses its atomic scripts")
+	}
+	return nil
+}
+
+func requireCacheTTLSourceContracts(t testing.TB, source string) {
+	t.Helper()
+	require.NoError(t, checkCacheTTLSourceContracts(source))
+}
+
+func TestFN_CA_TTLJitterSourceContract(t *testing.T) {
+	daoSource := readRepoSource(t, "common/dao/data_redis.hpp")
+	requireCacheTTLSourceContracts(t, daoSource)
+
+	t.Run("rejects fixed Session TTL", func(t *testing.T) {
+		mutant := strings.Replace(daoSource, "randomized_ttl(ttl)", "ttl", 1)
+		require.Error(t, checkCacheTTLSourceContracts(mutant))
+	})
+	t.Run("rejects non-atomic DeviceSet add", func(t *testing.T) {
+		mutant := strings.Replace(daoSource, "_c->eval<long long>(kAddLua", "_c->sadd", 1)
+		require.Error(t, checkCacheTTLSourceContracts(mutant))
+	})
+	t.Run("rejects non-atomic UnackedPush write", func(t *testing.T) {
+		mutant := strings.Replace(daoSource, "_c->eval<long long>(kPushLua", "_c->zadd", 1)
+		require.Error(t, checkCacheTTLSourceContracts(mutant))
+	})
+}
+
+type redisResultPredicate func(string) (bool, error)
+
+func requireEventuallyRedis(t testing.TB, timeout, interval time.Duration, message string,
+	predicate redisResultPredicate, args ...string) string {
+	t.Helper()
+	var mu sync.Mutex
+	var lastResult string
+	var lastErr error
+	matched := assert.Eventually(t, func() bool {
+		result, err := verify.RedisCLIResult(args...)
+		if err == nil {
+			var predicateMatch bool
+			predicateMatch, err = predicate(result)
+			mu.Lock()
+			lastResult, lastErr = result, err
+			mu.Unlock()
+			return predicateMatch && err == nil
+		}
+		mu.Lock()
+		lastResult, lastErr = result, err
+		mu.Unlock()
+		return false
+	}, timeout, interval, message)
+
+	mu.Lock()
+	result, err := lastResult, lastErr
+	mu.Unlock()
+	require.NoError(t, err, "%s (last result %q)", message, result)
+	require.True(t, matched, "%s (last result %q)", message, result)
+	return result
+}
+
+func redisEquals(expected string) redisResultPredicate {
+	return func(result string) (bool, error) { return result == expected, nil }
+}
+
 // FN-CA-04 | cache expirations are bounded, varied, and paired unacked keys
 // share one randomized sample per push operation.
 func TestFN_CA_TTLJitter(t *testing.T) {
@@ -219,17 +399,17 @@ func TestFN_CA_TTLJitter(t *testing.T) {
 			ws.writeBinary(t, cacheTestAuthNotify(user.AccessToken, "default_device"))
 
 			deviceKey := fmt.Sprintf("im:dev:{%s}", user.UserID)
-			require.Eventually(t, func() bool {
-				return verify.RedisCLI(t, "EXISTS", deviceKey) == "1"
-			}, 5*time.Second, 50*time.Millisecond, deviceKey)
+			requireEventuallyRedis(t, 5*time.Second, 50*time.Millisecond,
+				"DeviceSet key must exist", redisEquals("1"), "EXISTS", deviceKey)
 			if i == 0 {
 				verify.RedisCLI(t, "EXPIRE", deviceKey, "60")
 				heartbeatBefore = verify.RedisTTL(t, deviceKey)
 				ws.writeBinary(t, cacheTestHeartbeatNotify(user.UserID))
-				require.Eventually(t, func() bool {
-					seconds, err := strconv.ParseInt(verify.RedisCLI(t, "TTL", deviceKey), 10, 64)
-					return err == nil && time.Duration(seconds)*time.Second > heartbeatBefore+24*time.Hour
-				}, 5*time.Second, 50*time.Millisecond, "heartbeat must renew DeviceSet TTL")
+				requireEventuallyRedis(t, 5*time.Second, 50*time.Millisecond,
+					"heartbeat must renew DeviceSet TTL", func(result string) (bool, error) {
+						seconds, err := strconv.ParseInt(result, 10, 64)
+						return time.Duration(seconds)*time.Second > heartbeatBefore+24*time.Hour, err
+					}, "TTL", deviceKey)
 			} else {
 				ws.writeBinary(t, cacheTestHeartbeatNotify(user.UserID))
 			}
@@ -245,9 +425,8 @@ func TestFN_CA_TTLJitter(t *testing.T) {
 		connections = append(connections, recipientWS)
 		recipientWS.writeBinary(t, cacheTestAuthNotify(recipient.AccessToken, "default_device"))
 		deviceKey := fmt.Sprintf("im:dev:{%s}", recipient.UserID)
-		require.Eventually(t, func() bool {
-			return verify.RedisCLI(t, "EXISTS", deviceKey) == "1"
-		}, 5*time.Second, 50*time.Millisecond, deviceKey)
+		requireEventuallyRedis(t, 5*time.Second, 50*time.Millisecond,
+			"recipient DeviceSet key must exist", redisEquals("1"), "EXISTS", deviceKey)
 
 		unackedKey := fmt.Sprintf("im:unack:{%s:default_device}", recipient.UserID)
 		unackedIndexKey := fmt.Sprintf("im:unack:idx:{%s:default_device}", recipient.UserID)
@@ -256,10 +435,11 @@ func TestFN_CA_TTLJitter(t *testing.T) {
 		unackedTTLs := make(map[time.Duration]struct{})
 		for i := 0; i < 12; i++ {
 			sendCacheTestMessage(t, sender, convID, fmt.Sprintf("ttl-jitter-%d", i))
-			require.Eventually(t, func() bool {
-				count, parseErr := strconv.Atoi(verify.RedisCLI(t, "ZCARD", unackedKey))
-				return parseErr == nil && count >= initial+i+1
-			}, 10*time.Second, 100*time.Millisecond, unackedKey)
+			requireEventuallyRedis(t, 10*time.Second, 100*time.Millisecond,
+				"UnackedPush ZSET must receive the message", func(result string) (bool, error) {
+					count, err := strconv.Atoi(result)
+					return count >= initial+i+1, err
+				}, "ZCARD", unackedKey)
 			unackedTTL := requireJitteredRedisTTL(t, unackedKey, sessionTTL)
 			indexTTL := requireJitteredRedisTTL(t, unackedIndexKey, sessionTTL)
 			require.LessOrEqual(t, absDuration(unackedTTL-indexTTL), time.Second,
diff --git a/tests/pkg/verify/redis.go b/tests/pkg/verify/redis.go
index 77f75b1..e880529 100644
--- a/tests/pkg/verify/redis.go
+++ b/tests/pkg/verify/redis.go
@@ -13,12 +13,22 @@ import (
 
 func RedisCLI(t testing.TB, args ...string) string {
 	t.Helper()
+	out, err := RedisCLIResult(args...)
+	if err != nil {
+		t.Fatal(err)
+	}
+	return out
+}
+
+// RedisCLIResult runs redis-cli without invoking testing APIs, so callers may
+// safely use it from polling callbacks and report failures on the test goroutine.
+func RedisCLIResult(args ...string) (string, error) {
 	base := []string{"exec", "redis-node1", "redis-cli", "-c"}
 	out, err := exec.Command("docker", append(base, args...)...).CombinedOutput()
 	if err != nil {
-		t.Fatalf("redis-cli %v: %v: %s", args, err, out)
+		return "", fmt.Errorf("redis-cli %v: %w: %s", args, err, strings.TrimSpace(string(out)))
 	}
-	return strings.TrimSpace(string(out))
+	return strings.TrimSpace(string(out)), nil
 }
 
 func RedisTTL(t testing.TB, key string) time.Duration {

From 79276f83b0f8aa64221dc3a63b5d72a95d7c761f Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 13:13:13 +0800
Subject: [PATCH 14/47] test: add cache resilience performance baseline

---
 ...7-15-cache-resilience-completion-design.md |  10 +-
 tests/Makefile                                |  18 +-
 tests/perf/cache_test.go                      | 353 ++++++++++++++++++
 tests/perf/setup_test.go                      |   2 +-
 4 files changed, 380 insertions(+), 3 deletions(-)
 create mode 100644 tests/perf/cache_test.go

diff --git a/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
index 196a182..13c8fa6 100644
--- a/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
+++ b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
@@ -277,11 +277,19 @@ C++ gtest 或 Python 合约测试。
 
 文件：`tests/perf/cache_test.go`，build tag：`perf`。
 
-- `PF-09` 分别记录冷缓存、L2 命中和 L1 命中的吞吐与 p50/p95/p99。
+- `PF-09` 分别记录冷缓存、L2 命中和 L1 命中的吞吐与 p95。
 - 目标负载为 5000 msg/s，报告分配量和每请求耗时。
 - 同一 uid 的热路径 Identity RPC 降幅至少 95%。
 - 200 个并发请求访问同一冷 key 时，只允许一次进程内 RPC 回源。
 - nightly 基线吞吐下降超过 10% 时失败。
+- 性能门禁仅在 `PF09_RUN_FULLSTACK=1` 的 Linux 完整服务环境启用；普通编译发现
+  明确 Skip，不把缺少业务栈伪装成性能通过。L2 阶段由测试向真实 Redis Cluster
+  原子写入从 Identity 读取的 UserInfo，确保 fresh sender 的 L1 未预热；L1 阶段再
+  通过真实发送路径预热。
+- 多 Transmite 实例压测通过 `PF09_TRANSMITE_VARS_URLS` 提供所有 bvar 地址并求和，
+  防止遗漏实例导致 RPC 降幅虚高。门禁使用代码库内 5000 msg/s 基线，基线可向上
+  调整但不可降低；5 秒以内的 Go benchmark 校准轮次只报告、不判定，10 秒正式轮次
+  执行吞吐、RPC 降幅和 10% 回退门禁。
 
 ### 10.4 测试辅助设施
 
diff --git a/tests/Makefile b/tests/Makefile
index 026c2d1..d6b88bb 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -1,4 +1,6 @@
-.PHONY: proto test-bvt test-func test-reliability test-scenario test-perf clean deps
+.PHONY: proto test-bvt test-func test-reliability test-scenario test-perf test-perf-cache test-perf-cache-gate clean deps
+
+PF09_BASELINE_MSG_PER_SEC ?= 5000
 
 # Generate Go protobuf from proto/ definitions
 # NOTE: protoc-gen-go >= v1.35 requires go_package option or M flags to derive
@@ -47,6 +49,20 @@ test-scenario:
 test-perf:
 	go test -tags=perf ./perf/... -bench=. -benchmem -count=3 -benchtime=10s
 
+# Compile/discover PF-09 locally. It skips unless the explicit full-stack gate is enabled.
+test-perf-cache:
+	go test -v -tags=perf ./perf/... -run '^$$' -bench '^BenchmarkPF09_UserInfoCache$$'
+
+# Real Linux full-stack gate. The deployment must raise Transmite's user/session
+# rate limits above the generated load; any rate_limited response fails PF-09.
+# Multi-instance stacks must set PF09_TRANSMITE_VARS_URLS to every bvar endpoint,
+# separated by commas, so the RPC-reduction snapshot cannot omit an instance.
+test-perf-cache-gate:
+	@test "$$(uname -s)" = Linux || (echo "PF-09 gate requires Linux" >&2; exit 1)
+	PF09_RUN_FULLSTACK=1 PF09_BASELINE_MSG_PER_SEC=$(PF09_BASELINE_MSG_PER_SEC) \
+		go test -tags=perf ./perf/... -run '^$$' -bench '^BenchmarkPF09_UserInfoCache$$' \
+		-benchmem -count=3 -benchtime=10s
+
 # Download dependencies
 deps:
 	go mod download
diff --git a/tests/perf/cache_test.go b/tests/perf/cache_test.go
new file mode 100644
index 0000000..1a64de0
--- /dev/null
+++ b/tests/perf/cache_test.go
@@ -0,0 +1,353 @@
+//go:build perf
+
+package perf_test
+
+import (
+	"bytes"
+	"fmt"
+	"math"
+	"os"
+	"os/exec"
+	"runtime"
+	"sort"
+	"strconv"
+	"strings"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"chatnow-tests/pkg/client"
+	"chatnow-tests/pkg/fixture"
+	"chatnow-tests/pkg/verify"
+	identity "chatnow-tests/proto/chatnow/identity"
+	msg "chatnow-tests/proto/chatnow/message"
+	transmite "chatnow-tests/proto/chatnow/transmite"
+	"google.golang.org/protobuf/proto"
+)
+
+const (
+	pf09ConversationCount      = 20
+	pf09MaxLatencySamples      = 65_536
+	pf09MinThroughput          = 5_000.0
+	pf09CheckedInBaseline      = 5_000.0
+	pf09MinRPCReduction        = 95.0
+	pf09MaxBaselineRegression  = 10.0
+	pf09DefaultGateMinDuration = 5 * time.Second
+)
+
+type pf09CacheState uint8
+
+const (
+	pf09Cold pf09CacheState = iota
+	pf09L2
+	pf09L1
+)
+
+type pf09Conversation struct {
+	sender *client.HTTPClient
+	id     string
+}
+
+// BenchmarkPF09_UserInfoCache is a full-stack benchmark. It deliberately skips
+// unless PF09_RUN_FULLSTACK=1, so developer compile checks never masquerade as a
+// performance result. The gate target in tests/Makefile enables it on Linux CI.
+func BenchmarkPF09_UserInfoCache(b *testing.B) {
+	if os.Getenv("PF09_RUN_FULLSTACK") != "1" {
+		b.Skip("PF-09 requires the full service stack; use make test-perf-cache-gate on Linux")
+	}
+	if runtime.GOOS != "linux" {
+		b.Fatalf("PF-09 gate is calibrated for Linux, got %s", runtime.GOOS)
+	}
+
+	baseline := pf09Baseline(b)
+	gateMinDuration := pf09GateMinDuration(b)
+	for _, phase := range []struct {
+		name  string
+		state pf09CacheState
+	}{
+		{name: "cold", state: pf09Cold},
+		{name: "L2", state: pf09L2},
+		{name: "L1", state: pf09L1},
+	} {
+		phase := phase
+		b.Run(phase.name, func(b *testing.B) {
+			conversations := preparePF09Conversations(b, phase.state)
+			runPF09Phase(b, phase.state, conversations, baseline, gateMinDuration)
+		})
+	}
+}
+
+func preparePF09Conversations(b *testing.B, state pf09CacheState) []pf09Conversation {
+	b.Helper()
+	b.StopTimer()
+
+	conversations := make([]pf09Conversation, pf09ConversationCount)
+	for i := range conversations {
+		sender, peer, conversationID := fixture.MakeFriends(b, HTTP)
+		_ = peer
+		conversations[i] = pf09Conversation{sender: sender, id: conversationID}
+
+		switch state {
+		case pf09Cold:
+			// A newly registered sender has neither L1 nor L2 cache state.
+		case pf09L2, pf09L1:
+			seedPF09L2(b, sender)
+		default:
+			b.Fatalf("unknown PF-09 cache state %d", state)
+		}
+	}
+
+	if state == pf09L1 {
+		for i, conversation := range conversations {
+			if err := sendPF09Message(conversation, uint64(i)); err != nil {
+				b.Fatalf("warm PF-09 L1: %v", err)
+			}
+		}
+	}
+	return conversations
+}
+
+func seedPF09L2(b *testing.B, sender *client.HTTPClient) {
+	b.Helper()
+	rsp := &identity.GetProfileRsp{}
+	if err := sender.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{
+		RequestId: client.NewRequestID(),
+	}, rsp); err != nil {
+		b.Fatalf("read PF-09 profile: %v", err)
+	}
+	if !rsp.GetHeader().GetSuccess() || rsp.GetUserInfo() == nil {
+		b.Fatalf("read PF-09 profile: %s", rsp.GetHeader().GetErrorMessage())
+	}
+	value, err := proto.Marshal(rsp.GetUserInfo())
+	if err != nil {
+		b.Fatalf("marshal PF-09 profile: %v", err)
+	}
+
+	key := pf09UserInfoKey(sender.UserID)
+	container := HTTP.Config().Infra.RedisContainer
+	const seedScript = "return redis.call('SET',KEYS[1],ARGV[2],'EX',ARGV[1])"
+	set := exec.Command(
+		"docker", "exec", "-i", container, "redis-cli", "-c", "-x",
+		"EVAL", seedScript, "1", key, "3600",
+	)
+	set.Stdin = bytes.NewReader(value)
+	if out, err := set.CombinedOutput(); err != nil || strings.TrimSpace(string(out)) != "OK" {
+		b.Fatalf("seed PF-09 L2 %s: %v: %s", key, err, strings.TrimSpace(string(out)))
+	}
+}
+
+func runPF09Phase(
+	b *testing.B,
+	state pf09CacheState,
+	conversations []pf09Conversation,
+	baseline float64,
+	gateMinDuration time.Duration,
+) {
+	b.Helper()
+	beforeRPC := pf09RPCSnapshot(b)
+
+	var sequence atomic.Uint64
+	var successes atomic.Uint64
+	var workerSeeds atomic.Uint64
+	var stopped atomic.Bool
+	var failureMu sync.Mutex
+	var firstFailure error
+	latencies := make([]int64, 0, pf09MaxLatencySamples)
+	var latencyMu sync.Mutex
+	perWorkerSamples := max(1, pf09MaxLatencySamples/max(1, runtime.GOMAXPROCS(0)))
+
+	b.ResetTimer()
+	started := time.Now()
+	b.RunParallel(func(pb *testing.PB) {
+		localLatencies := make([]int64, 0, perWorkerSamples)
+		var samplesSeen uint64
+		randomState := (uint64(time.Now().UnixNano()) ^ workerSeeds.Add(1)) | 1
+		for pb.Next() {
+			if stopped.Load() {
+				break
+			}
+			id := sequence.Add(1) - 1
+			conversation := conversations[id%uint64(len(conversations))]
+			requestStarted := time.Now()
+			err := sendPF09Message(conversation, id)
+			latency := time.Since(requestStarted).Nanoseconds()
+			if err != nil {
+				failureMu.Lock()
+				if firstFailure == nil {
+					firstFailure = err
+					stopped.Store(true)
+				}
+				failureMu.Unlock()
+				break
+			}
+			successes.Add(1)
+			samplesSeen++
+			if len(localLatencies) < cap(localLatencies) {
+				localLatencies = append(localLatencies, latency)
+			} else {
+				// Per-worker reservoir sampling keeps p95 memory bounded without
+				// biasing the sample toward benchmark startup.
+				randomState ^= randomState << 13
+				randomState ^= randomState >> 7
+				randomState ^= randomState << 17
+				if replacement := randomState % samplesSeen; replacement < uint64(len(localLatencies)) {
+					localLatencies[replacement] = latency
+				}
+			}
+		}
+		latencyMu.Lock()
+		remaining := pf09MaxLatencySamples - len(latencies)
+		if remaining > 0 {
+			if len(localLatencies) > remaining {
+				localLatencies = localLatencies[:remaining]
+			}
+			latencies = append(latencies, localLatencies...)
+		}
+		latencyMu.Unlock()
+	})
+	elapsed := time.Since(started)
+	b.StopTimer()
+
+	failureMu.Lock()
+	err := firstFailure
+	failureMu.Unlock()
+	if err != nil {
+		b.Fatalf("PF-09 send failed: %v", err)
+	}
+	successCount := successes.Load()
+	if successCount == 0 || elapsed <= 0 || len(latencies) == 0 {
+		b.Fatalf("PF-09 produced no measurable successful requests")
+	}
+	afterRPC := pf09RPCSnapshot(b)
+	rpcDelta := afterRPC - beforeRPC
+	if rpcDelta < 0 {
+		b.Fatalf("PF-09 user_info_rpc_total moved backwards: before=%d after=%d", beforeRPC, afterRPC)
+	}
+
+	sort.Slice(latencies, func(i, j int) bool { return latencies[i] < latencies[j] })
+	p95Index := int(math.Ceil(float64(len(latencies))*0.95)) - 1
+	throughput := float64(successCount) / elapsed.Seconds()
+	p95Micros := float64(latencies[p95Index]) / float64(time.Microsecond)
+	rpcReduction := 100 * (1 - float64(rpcDelta)/float64(successCount))
+	baselineRegression := 100 * (1 - throughput/baseline)
+
+	b.ReportMetric(throughput, "msg/s")
+	b.ReportMetric(p95Micros, "p95-us")
+	b.ReportMetric(rpcReduction, "rpc-reduction-%")
+	b.ReportMetric(baselineRegression, "baseline-regression-%")
+
+	// Short calibration iterations are intentionally not gates. With the required
+	// -benchtime=10s, the final iteration crosses this duration and enforces SLOs.
+	if elapsed < gateMinDuration {
+		return
+	}
+	if rpcReduction < pf09MinRPCReduction {
+		b.Errorf("PF-09 %s RPC reduction %.2f%% is below %.2f%%", pf09StateName(state), rpcReduction, pf09MinRPCReduction)
+	}
+	if state == pf09L1 {
+		if throughput < pf09MinThroughput {
+			b.Errorf("PF-09 L1 throughput %.2f msg/s is below %.2f msg/s", throughput, pf09MinThroughput)
+		}
+		if baselineRegression > pf09MaxBaselineRegression {
+			b.Errorf("PF-09 L1 throughput regressed %.2f%% from baseline %.2f msg/s", baselineRegression, baseline)
+		}
+	}
+}
+
+// pf09RPCSnapshot sums all configured Transmite instances. A comma-separated
+// PF09_TRANSMITE_VARS_URLS is required for a multi-instance performance stack;
+// the single-instance local stack falls back to tests/config.yaml.
+func pf09RPCSnapshot(b *testing.B) int64 {
+	b.Helper()
+	raw := os.Getenv("PF09_TRANSMITE_VARS_URLS")
+	if raw == "" {
+		raw = HTTP.Config().Infra.TransmiteVars
+	}
+	var total int64
+	var count int
+	for _, endpoint := range strings.Split(raw, ",") {
+		endpoint = strings.TrimSpace(endpoint)
+		if endpoint == "" {
+			continue
+		}
+		total += verify.BVar(b, strings.TrimRight(endpoint, "/"), "user_info_rpc_total")
+		count++
+	}
+	if count == 0 {
+		b.Fatal("PF-09 requires at least one Transmite bvar endpoint")
+	}
+	return total
+}
+
+func sendPF09Message(conversation pf09Conversation, sequence uint64) error {
+	rsp := &transmite.SendMessageRsp{}
+	err := conversation.sender.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
+		RequestId:      client.NewRequestID(),
+		ConversationId: conversation.id,
+		Content: &msg.MessageContent{
+			Type: msg.MessageType_TEXT,
+			Body: &msg.MessageContent_Text{Text: &msg.TextContent{
+				Text: fmt.Sprintf("pf09-%d", sequence),
+			}},
+		},
+		ClientMsgId: client.NewRequestID(),
+	}, rsp)
+	if err != nil {
+		return err
+	}
+	if !rsp.GetHeader().GetSuccess() {
+		return fmt.Errorf("code=%d message=%s", rsp.GetHeader().GetErrorCode(), rsp.GetHeader().GetErrorMessage())
+	}
+	return nil
+}
+
+func pf09Baseline(b *testing.B) float64 {
+	b.Helper()
+	value := pf09CheckedInBaseline
+	if raw := os.Getenv("PF09_BASELINE_MSG_PER_SEC"); raw != "" {
+		parsed, err := strconv.ParseFloat(raw, 64)
+		if err != nil || math.IsNaN(parsed) || math.IsInf(parsed, 0) || parsed < pf09CheckedInBaseline {
+			b.Fatalf("PF09_BASELINE_MSG_PER_SEC must be a finite value >= %.0f, got %q", pf09CheckedInBaseline, raw)
+		}
+		value = parsed
+	}
+	return value
+}
+
+func pf09GateMinDuration(b *testing.B) time.Duration {
+	b.Helper()
+	raw := os.Getenv("PF09_GATE_MIN_DURATION")
+	if raw == "" {
+		return pf09DefaultGateMinDuration
+	}
+	duration, err := time.ParseDuration(raw)
+	if err != nil || duration <= 0 {
+		b.Fatalf("PF09_GATE_MIN_DURATION must be a positive duration, got %q", raw)
+	}
+	return duration
+}
+
+func pf09UserInfoKey(uid string) string {
+	const offset32 = uint32(2166136261)
+	const prime32 = uint32(16777619)
+	hash := offset32
+	for i := 0; i < len(uid); i++ {
+		hash ^= uint32(uid[i])
+		hash *= prime32
+	}
+	return fmt.Sprintf("im:user:{%d}:%s", hash%64, uid)
+}
+
+func pf09StateName(state pf09CacheState) string {
+	switch state {
+	case pf09Cold:
+		return "cold"
+	case pf09L2:
+		return "L2"
+	case pf09L1:
+		return "L1"
+	default:
+		return "unknown"
+	}
+}
diff --git a/tests/perf/setup_test.go b/tests/perf/setup_test.go
index d2318dc..4e73894 100644
--- a/tests/perf/setup_test.go
+++ b/tests/perf/setup_test.go
@@ -12,7 +12,7 @@ import (
 var HTTP *client.HTTPClient
 
 func TestMain(m *testing.M) {
-	cfg := client.LoadConfig("")
+	cfg := client.LoadConfig("../config.yaml")
 	HTTP = client.NewHTTPClient(cfg)
 	os.Exit(m.Run())
 }

From a80943981c4ac10e59bba710605b496e45c7dbc9 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 13:28:33 +0800
Subject: [PATCH 15/47] test: make PF-09 phase gates deterministic

---
 ...7-15-cache-resilience-completion-design.md |  15 +-
 tests/Makefile                                |   6 +-
 tests/perf/cache_helpers_test.go              | 159 ++++++
 tests/perf/cache_test.go                      | 540 +++++++++++++-----
 4 files changed, 571 insertions(+), 149 deletions(-)
 create mode 100644 tests/perf/cache_helpers_test.go

diff --git a/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
index 13c8fa6..fc32a02 100644
--- a/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
+++ b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
@@ -283,13 +283,16 @@ C++ gtest 或 Python 合约测试。
 - 200 个并发请求访问同一冷 key 时，只允许一次进程内 RPC 回源。
 - nightly 基线吞吐下降超过 10% 时失败。
 - 性能门禁仅在 `PF09_RUN_FULLSTACK=1` 的 Linux 完整服务环境启用；普通编译发现
-  明确 Skip，不把缺少业务栈伪装成性能通过。L2 阶段由测试向真实 Redis Cluster
-  原子写入从 Identity 读取的 UserInfo，确保 fresh sender 的 L1 未预热；L1 阶段再
-  通过真实发送路径预热。
+  明确 Skip，不把缺少业务栈伪装成性能通过。cold、L2 各使用 20 个 fresh sender，
+  每个 sender 在计时区间只请求一次；L2 在计时前向真实 Redis Cluster 原子写入从
+  Identity 读取的 UserInfo。L1 使用另外 20 个 sender，通过真实发送路径预热后固定
+  压测 10 秒；预热按声明的 Transmite 实例数连续发送，利用 Gateway 的 round-robin
+  为每个实例填充 L1，保证三个计时区间不会因 key 复用或跨实例路由相互转化。
 - 多 Transmite 实例压测通过 `PF09_TRANSMITE_VARS_URLS` 提供所有 bvar 地址并求和，
-  防止遗漏实例导致 RPC 降幅虚高。门禁使用代码库内 5000 msg/s 基线，基线可向上
-  调整但不可降低；5 秒以内的 Go benchmark 校准轮次只报告、不判定，10 秒正式轮次
-  执行吞吐、RPC 降幅和 10% 回退门禁。
+  并用 `PF09_EXPECTED_TRANSMITE_INSTANCES` 校验实例数。前后快照同时记录 pid、uptime、
+  启动时刻估值和 L1/L2/RPC counter，拒绝实例重启、counter 回绕、遗漏和求和溢出；
+  每阶段 counter 必须精确符合对应缓存路径。门禁固定 `-benchtime=1x`，内部 10 秒时长
+  不可由环境变量修改；使用代码库内 5000 msg/s 基线，基线可向上调整但不可降低。
 
 ### 10.4 测试辅助设施
 
diff --git a/tests/Makefile b/tests/Makefile
index d6b88bb..6a77fe4 100644
--- a/tests/Makefile
+++ b/tests/Makefile
@@ -1,6 +1,7 @@
 .PHONY: proto test-bvt test-func test-reliability test-scenario test-perf test-perf-cache test-perf-cache-gate clean deps
 
 PF09_BASELINE_MSG_PER_SEC ?= 5000
+PF09_EXPECTED_TRANSMITE_INSTANCES ?= 1
 
 # Generate Go protobuf from proto/ definitions
 # NOTE: protoc-gen-go >= v1.35 requires go_package option or M flags to derive
@@ -51,7 +52,7 @@ test-perf:
 
 # Compile/discover PF-09 locally. It skips unless the explicit full-stack gate is enabled.
 test-perf-cache:
-	go test -v -tags=perf ./perf/... -run '^$$' -bench '^BenchmarkPF09_UserInfoCache$$'
+	go test -v -tags=perf ./perf/... -run '^$$' -bench '^BenchmarkPF09_UserInfoCache$$' -benchtime=1x
 
 # Real Linux full-stack gate. The deployment must raise Transmite's user/session
 # rate limits above the generated load; any rate_limited response fails PF-09.
@@ -60,8 +61,9 @@ test-perf-cache:
 test-perf-cache-gate:
 	@test "$$(uname -s)" = Linux || (echo "PF-09 gate requires Linux" >&2; exit 1)
 	PF09_RUN_FULLSTACK=1 PF09_BASELINE_MSG_PER_SEC=$(PF09_BASELINE_MSG_PER_SEC) \
+		PF09_EXPECTED_TRANSMITE_INSTANCES=$(PF09_EXPECTED_TRANSMITE_INSTANCES) \
 		go test -tags=perf ./perf/... -run '^$$' -bench '^BenchmarkPF09_UserInfoCache$$' \
-		-benchmem -count=3 -benchtime=10s
+		-benchmem -count=3 -benchtime=1x
 
 # Download dependencies
 deps:
diff --git a/tests/perf/cache_helpers_test.go b/tests/perf/cache_helpers_test.go
new file mode 100644
index 0000000..55f0c81
--- /dev/null
+++ b/tests/perf/cache_helpers_test.go
@@ -0,0 +1,159 @@
+//go:build perf
+
+package perf_test
+
+import (
+	"fmt"
+	"math"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"strings"
+	"testing"
+	"time"
+
+	"chatnow-tests/pkg/client"
+)
+
+func TestPF09PhaseConversationsRequireUniqueKeys(t *testing.T) {
+	conversations := make([]pf09Conversation, pf09ConversationCount)
+	for i := range conversations {
+		conversations[i] = pf09Conversation{
+			sender: &client.HTTPClient{UserID: "user-" + string(rune('a'+i))},
+			id:     "conversation-" + string(rune('a'+i)),
+		}
+	}
+	if err := validatePF09Conversations(conversations); err != nil {
+		t.Fatalf("valid phase plan rejected: %v", err)
+	}
+	conversations[19].sender.UserID = conversations[0].sender.UserID
+	if err := validatePF09Conversations(conversations); err == nil || !strings.Contains(err.Error(), "duplicate user-info key") {
+		t.Fatalf("duplicate phase key not rejected: %v", err)
+	}
+}
+
+func TestPF09EndpointsRejectEmptyDuplicateAndWrongCount(t *testing.T) {
+	if _, err := parsePF09Endpoints("http://a,http://b", 2); err != nil {
+		t.Fatalf("valid endpoints rejected: %v", err)
+	}
+	for _, test := range []struct {
+		raw      string
+		expected int
+	}{
+		{raw: "", expected: 1},
+		{raw: "http://a,", expected: 1},
+		{raw: "http://a/,http://a", expected: 2},
+		{raw: "http://a", expected: 2},
+	} {
+		if _, err := parsePF09Endpoints(test.raw, test.expected); err == nil {
+			t.Errorf("invalid endpoints accepted: raw=%q expected=%d", test.raw, test.expected)
+		}
+	}
+}
+
+func TestPF09SnapshotDeltaRejectsRestartRewindMissingAndOverflow(t *testing.T) {
+	before := pf09Snapshot{
+		"http://a": {pid: 10, uptimeSeconds: 100, counters: map[string]int64{
+			pf09RPCMetric: 10, pf09L1Metric: 20, pf09L2Metric: 30,
+		}},
+	}
+	after := pf09Snapshot{
+		"http://a": {pid: 10, uptimeSeconds: 101, counters: map[string]int64{
+			pf09RPCMetric: 11, pf09L1Metric: 22, pf09L2Metric: 33,
+		}},
+	}
+	delta, err := pf09SnapshotDelta(before, after)
+	if err != nil {
+		t.Fatalf("valid snapshot rejected: %v", err)
+	}
+	if delta[pf09RPCMetric] != 1 || delta[pf09L1Metric] != 2 || delta[pf09L2Metric] != 3 {
+		t.Fatalf("wrong deltas: %#v", delta)
+	}
+
+	cases := []pf09Snapshot{
+		{},
+		{"http://a": {pid: 11, uptimeSeconds: 1, counters: after["http://a"].counters}},
+		{"http://a": {pid: 10, uptimeSeconds: 99, counters: after["http://a"].counters}},
+		{"http://a": {pid: 10, uptimeSeconds: 101, counters: map[string]int64{
+			pf09RPCMetric: 9, pf09L1Metric: 22, pf09L2Metric: 33,
+		}}},
+	}
+	for i, invalid := range cases {
+		if _, err := pf09SnapshotDelta(before, invalid); err == nil {
+			t.Errorf("invalid snapshot %d accepted", i)
+		}
+	}
+
+	overflowBefore := pf09Snapshot{
+		"http://a": {pid: 1, uptimeSeconds: 1, counters: map[string]int64{pf09RPCMetric: 0, pf09L1Metric: 0, pf09L2Metric: 0}},
+		"http://b": {pid: 2, uptimeSeconds: 1, counters: map[string]int64{pf09RPCMetric: 0, pf09L1Metric: 0, pf09L2Metric: 0}},
+	}
+	overflowAfter := pf09Snapshot{
+		"http://a": {pid: 1, uptimeSeconds: 2, counters: map[string]int64{pf09RPCMetric: math.MaxInt64, pf09L1Metric: 0, pf09L2Metric: 0}},
+		"http://b": {pid: 2, uptimeSeconds: 2, counters: map[string]int64{pf09RPCMetric: 1, pf09L1Metric: 0, pf09L2Metric: 0}},
+	}
+	if _, err := pf09SnapshotDelta(overflowBefore, overflowAfter); err == nil {
+		t.Fatal("overflowing counter delta accepted")
+	}
+}
+
+func TestPF09SnapshotRequiresProcessIdentityFields(t *testing.T) {
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		values := map[string]string{
+			"/vars/pid":                    "42",
+			"/vars/user_info_rpc_total":    "1",
+			"/vars/user_info_l1_hit_total": "2",
+			"/vars/user_info_l2_hit_total": "3",
+		}
+		value, exists := values[r.URL.Path]
+		if !exists {
+			http.NotFound(w, r)
+			return
+		}
+		_, _ = fmt.Fprint(w, value)
+	}))
+	defer server.Close()
+	if _, err := capturePF09Snapshot([]string{server.URL}); err == nil || !strings.Contains(err.Error(), "process uptime") {
+		t.Fatalf("missing process_uptime did not hard fail: %v", err)
+	}
+}
+
+func TestPF09PhaseCountersProveStableCacheState(t *testing.T) {
+	valid := []struct {
+		state  pf09CacheState
+		deltas map[string]int64
+	}{
+		{pf09Cold, map[string]int64{pf09RPCMetric: 20, pf09L1Metric: 0, pf09L2Metric: 0}},
+		{pf09L2, map[string]int64{pf09RPCMetric: 0, pf09L1Metric: 0, pf09L2Metric: 20}},
+		{pf09L1, map[string]int64{pf09RPCMetric: 0, pf09L1Metric: 20, pf09L2Metric: 0}},
+	}
+	for _, test := range valid {
+		if err := validatePF09PhaseCounters(test.state, 20, test.deltas); err != nil {
+			t.Errorf("valid %s counters rejected: %v", pf09StateName(test.state), err)
+		}
+		contaminated := make(map[string]int64, len(test.deltas))
+		for metric, value := range test.deltas {
+			contaminated[metric] = value
+		}
+		contaminated[pf09RPCMetric]++
+		if err := validatePF09PhaseCounters(test.state, 20, contaminated); err == nil {
+			t.Errorf("contaminated %s counters accepted", pf09StateName(test.state))
+		}
+	}
+}
+
+func TestPF09GateDurationCannotBeBypassed(t *testing.T) {
+	t.Setenv("PF09_GATE_MIN_DURATION", "24h")
+	if pf09SteadyDuration != 10*time.Second {
+		t.Fatalf("steady duration changed through environment: %s", pf09SteadyDuration)
+	}
+	if err := validatePF09IterationCount(1); err != nil {
+		t.Fatalf("one-shot invocation rejected: %v", err)
+	}
+	if err := validatePF09IterationCount(2); err == nil {
+		t.Fatal("calibrated invocation accepted; -benchtime=1x is mandatory")
+	}
+	if _, present := os.LookupEnv("PF09_GATE_MIN_DURATION"); !present {
+		t.Fatal("test precondition lost")
+	}
+}
diff --git a/tests/perf/cache_test.go b/tests/perf/cache_test.go
index 1a64de0..074727d 100644
--- a/tests/perf/cache_test.go
+++ b/tests/perf/cache_test.go
@@ -5,7 +5,9 @@ package perf_test
 import (
 	"bytes"
 	"fmt"
+	"io"
 	"math"
+	"net/http"
 	"os"
 	"os/exec"
 	"runtime"
@@ -19,7 +21,6 @@ import (
 
 	"chatnow-tests/pkg/client"
 	"chatnow-tests/pkg/fixture"
-	"chatnow-tests/pkg/verify"
 	identity "chatnow-tests/proto/chatnow/identity"
 	msg "chatnow-tests/proto/chatnow/message"
 	transmite "chatnow-tests/proto/chatnow/transmite"
@@ -33,9 +34,17 @@ const (
 	pf09CheckedInBaseline      = 5_000.0
 	pf09MinRPCReduction        = 95.0
 	pf09MaxBaselineRegression  = 10.0
-	pf09DefaultGateMinDuration = 5 * time.Second
+	pf09SteadyDuration         = 10 * time.Second
+	pf09SnapshotTimeout        = 2 * time.Second
+	pf09StartEstimateTolerance = 2 * time.Second
+
+	pf09RPCMetric = "user_info_rpc_total"
+	pf09L1Metric  = "user_info_l1_hit_total"
+	pf09L2Metric  = "user_info_l2_hit_total"
 )
 
+var pf09CounterMetrics = [...]string{pf09RPCMetric, pf09L1Metric, pf09L2Metric}
+
 type pf09CacheState uint8
 
 const (
@@ -49,9 +58,26 @@ type pf09Conversation struct {
 	id     string
 }
 
-// BenchmarkPF09_UserInfoCache is a full-stack benchmark. It deliberately skips
-// unless PF09_RUN_FULLSTACK=1, so developer compile checks never masquerade as a
-// performance result. The gate target in tests/Makefile enables it on Linux CI.
+type pf09InstanceSnapshot struct {
+	pid                   int64
+	uptimeSeconds         float64
+	startedAtEstimateNano int64
+	counters              map[string]int64
+}
+
+type pf09Snapshot map[string]pf09InstanceSnapshot
+
+type pf09PhaseResult struct {
+	elapsed   time.Duration
+	successes uint64
+	latencies []int64
+	err       error
+}
+
+// BenchmarkPF09_UserInfoCache is intentionally a one-shot benchmark harness.
+// cold/L2 each execute one concurrent operation for 20 unique sender keys; L1
+// executes a fixed ten-second steady workload over 20 prewarmed senders. This
+// prevents Go's adaptive benchmark calibration from changing the cache state.
 func BenchmarkPF09_UserInfoCache(b *testing.B) {
 	if os.Getenv("PF09_RUN_FULLSTACK") != "1" {
 		b.Skip("PF-09 requires the full service stack; use make test-perf-cache-gate on Linux")
@@ -59,9 +85,12 @@ func BenchmarkPF09_UserInfoCache(b *testing.B) {
 	if runtime.GOOS != "linux" {
 		b.Fatalf("PF-09 gate is calibrated for Linux, got %s", runtime.GOOS)
 	}
+	if err := validatePF09IterationCount(b.N); err != nil {
+		b.Fatal(err)
+	}
 
 	baseline := pf09Baseline(b)
-	gateMinDuration := pf09GateMinDuration(b)
+	endpoints := pf09ConfiguredEndpoints(b)
 	for _, phase := range []struct {
 		name  string
 		state pf09CacheState
@@ -72,42 +101,75 @@ func BenchmarkPF09_UserInfoCache(b *testing.B) {
 	} {
 		phase := phase
 		b.Run(phase.name, func(b *testing.B) {
-			conversations := preparePF09Conversations(b, phase.state)
-			runPF09Phase(b, phase.state, conversations, baseline, gateMinDuration)
+			if err := validatePF09IterationCount(b.N); err != nil {
+				b.Fatal(err)
+			}
+			conversations := preparePF09Conversations(b, phase.state, len(endpoints))
+			if err := validatePF09Conversations(conversations); err != nil {
+				b.Fatalf("PF-09 %s phase plan: %v", phase.name, err)
+			}
+			runPF09Phase(b, phase.state, conversations, endpoints, baseline)
 		})
 	}
 }
 
-func preparePF09Conversations(b *testing.B, state pf09CacheState) []pf09Conversation {
+func validatePF09IterationCount(n int) error {
+	if n != 1 {
+		return fmt.Errorf("PF-09 requires -benchtime=1x to preserve phase state, got b.N=%d", n)
+	}
+	return nil
+}
+
+func preparePF09Conversations(b *testing.B, state pf09CacheState, instanceCount int) []pf09Conversation {
 	b.Helper()
 	b.StopTimer()
-
 	conversations := make([]pf09Conversation, pf09ConversationCount)
 	for i := range conversations {
 		sender, peer, conversationID := fixture.MakeFriends(b, HTTP)
 		_ = peer
 		conversations[i] = pf09Conversation{sender: sender, id: conversationID}
-
-		switch state {
-		case pf09Cold:
-			// A newly registered sender has neither L1 nor L2 cache state.
-		case pf09L2, pf09L1:
+		if state == pf09L2 || state == pf09L1 {
 			seedPF09L2(b, sender)
-		default:
-			b.Fatalf("unknown PF-09 cache state %d", state)
 		}
 	}
-
 	if state == pf09L1 {
+		// Gateway dispatch is round-robin. Consecutive instanceCount sends per
+		// sender warm that sender in every declared Transmite process.
 		for i, conversation := range conversations {
-			if err := sendPF09Message(conversation, uint64(i)); err != nil {
-				b.Fatalf("warm PF-09 L1: %v", err)
+			for instance := 0; instance < instanceCount; instance++ {
+				sequence := uint64(i*instanceCount + instance)
+				if err := sendPF09Message(conversation, sequence); err != nil {
+					b.Fatalf("warm PF-09 L1: %v", err)
+				}
 			}
 		}
 	}
 	return conversations
 }
 
+func validatePF09Conversations(conversations []pf09Conversation) error {
+	if len(conversations) != pf09ConversationCount {
+		return fmt.Errorf("want %d conversations, got %d", pf09ConversationCount, len(conversations))
+	}
+	keys := make(map[string]struct{}, len(conversations))
+	conversationIDs := make(map[string]struct{}, len(conversations))
+	for _, conversation := range conversations {
+		if conversation.sender == nil || conversation.sender.UserID == "" || conversation.id == "" {
+			return fmt.Errorf("empty sender or conversation")
+		}
+		key := pf09UserInfoKey(conversation.sender.UserID)
+		if _, exists := keys[key]; exists {
+			return fmt.Errorf("duplicate user-info key %q", key)
+		}
+		keys[key] = struct{}{}
+		if _, exists := conversationIDs[conversation.id]; exists {
+			return fmt.Errorf("duplicate conversation %q", conversation.id)
+		}
+		conversationIDs[conversation.id] = struct{}{}
+	}
+	return nil
+}
+
 func seedPF09L2(b *testing.B, sender *client.HTTPClient) {
 	b.Helper()
 	rsp := &identity.GetProfileRsp{}
@@ -123,13 +185,11 @@ func seedPF09L2(b *testing.B, sender *client.HTTPClient) {
 	if err != nil {
 		b.Fatalf("marshal PF-09 profile: %v", err)
 	}
-
 	key := pf09UserInfoKey(sender.UserID)
-	container := HTTP.Config().Infra.RedisContainer
 	const seedScript = "return redis.call('SET',KEYS[1],ARGV[2],'EX',ARGV[1])"
 	set := exec.Command(
-		"docker", "exec", "-i", container, "redis-cli", "-c", "-x",
-		"EVAL", seedScript, "1", key, "3600",
+		"docker", "exec", "-i", HTTP.Config().Infra.RedisContainer,
+		"redis-cli", "-c", "-x", "EVAL", seedScript, "1", key, "3600",
 	)
 	set.Stdin = bytes.NewReader(value)
 	if out, err := set.CombinedOutput(); err != nil || strings.TrimSpace(string(out)) != "OK" {
@@ -141,111 +201,57 @@ func runPF09Phase(
 	b *testing.B,
 	state pf09CacheState,
 	conversations []pf09Conversation,
+	endpoints []string,
 	baseline float64,
-	gateMinDuration time.Duration,
 ) {
 	b.Helper()
-	beforeRPC := pf09RPCSnapshot(b)
-
-	var sequence atomic.Uint64
-	var successes atomic.Uint64
-	var workerSeeds atomic.Uint64
-	var stopped atomic.Bool
-	var failureMu sync.Mutex
-	var firstFailure error
-	latencies := make([]int64, 0, pf09MaxLatencySamples)
-	var latencyMu sync.Mutex
-	perWorkerSamples := max(1, pf09MaxLatencySamples/max(1, runtime.GOMAXPROCS(0)))
+	before, err := capturePF09Snapshot(endpoints)
+	if err != nil {
+		b.Fatalf("PF-09 before snapshot: %v", err)
+	}
 
 	b.ResetTimer()
-	started := time.Now()
-	b.RunParallel(func(pb *testing.PB) {
-		localLatencies := make([]int64, 0, perWorkerSamples)
-		var samplesSeen uint64
-		randomState := (uint64(time.Now().UnixNano()) ^ workerSeeds.Add(1)) | 1
-		for pb.Next() {
-			if stopped.Load() {
-				break
-			}
-			id := sequence.Add(1) - 1
-			conversation := conversations[id%uint64(len(conversations))]
-			requestStarted := time.Now()
-			err := sendPF09Message(conversation, id)
-			latency := time.Since(requestStarted).Nanoseconds()
-			if err != nil {
-				failureMu.Lock()
-				if firstFailure == nil {
-					firstFailure = err
-					stopped.Store(true)
-				}
-				failureMu.Unlock()
-				break
-			}
-			successes.Add(1)
-			samplesSeen++
-			if len(localLatencies) < cap(localLatencies) {
-				localLatencies = append(localLatencies, latency)
-			} else {
-				// Per-worker reservoir sampling keeps p95 memory bounded without
-				// biasing the sample toward benchmark startup.
-				randomState ^= randomState << 13
-				randomState ^= randomState >> 7
-				randomState ^= randomState << 17
-				if replacement := randomState % samplesSeen; replacement < uint64(len(localLatencies)) {
-					localLatencies[replacement] = latency
-				}
-			}
-		}
-		latencyMu.Lock()
-		remaining := pf09MaxLatencySamples - len(latencies)
-		if remaining > 0 {
-			if len(localLatencies) > remaining {
-				localLatencies = localLatencies[:remaining]
-			}
-			latencies = append(latencies, localLatencies...)
-		}
-		latencyMu.Unlock()
-	})
-	elapsed := time.Since(started)
+	var result pf09PhaseResult
+	if state == pf09L1 {
+		result = runPF09Steady(conversations)
+	} else {
+		result = runPF09SingleUse(conversations)
+	}
 	b.StopTimer()
+	if result.err != nil {
+		b.Fatalf("PF-09 %s send failed: %v", pf09StateName(state), result.err)
+	}
+	if result.successes == 0 || result.elapsed <= 0 || len(result.latencies) == 0 {
+		b.Fatalf("PF-09 %s produced no measurable successful requests", pf09StateName(state))
+	}
 
-	failureMu.Lock()
-	err := firstFailure
-	failureMu.Unlock()
+	after, err := capturePF09Snapshot(endpoints)
 	if err != nil {
-		b.Fatalf("PF-09 send failed: %v", err)
+		b.Fatalf("PF-09 after snapshot: %v", err)
 	}
-	successCount := successes.Load()
-	if successCount == 0 || elapsed <= 0 || len(latencies) == 0 {
-		b.Fatalf("PF-09 produced no measurable successful requests")
+	deltas, err := pf09SnapshotDelta(before, after)
+	if err != nil {
+		b.Fatalf("PF-09 snapshot integrity: %v", err)
 	}
-	afterRPC := pf09RPCSnapshot(b)
-	rpcDelta := afterRPC - beforeRPC
-	if rpcDelta < 0 {
-		b.Fatalf("PF-09 user_info_rpc_total moved backwards: before=%d after=%d", beforeRPC, afterRPC)
+	if err := validatePF09PhaseCounters(state, result.successes, deltas); err != nil {
+		b.Error(err)
 	}
 
-	sort.Slice(latencies, func(i, j int) bool { return latencies[i] < latencies[j] })
-	p95Index := int(math.Ceil(float64(len(latencies))*0.95)) - 1
-	throughput := float64(successCount) / elapsed.Seconds()
-	p95Micros := float64(latencies[p95Index]) / float64(time.Microsecond)
-	rpcReduction := 100 * (1 - float64(rpcDelta)/float64(successCount))
+	sort.Slice(result.latencies, func(i, j int) bool { return result.latencies[i] < result.latencies[j] })
+	p95Index := int(math.Ceil(float64(len(result.latencies))*0.95)) - 1
+	throughput := float64(result.successes) / result.elapsed.Seconds()
+	p95Micros := float64(result.latencies[p95Index]) / float64(time.Microsecond)
+	rpcReduction := 100 * (1 - float64(deltas[pf09RPCMetric])/float64(result.successes))
 	baselineRegression := 100 * (1 - throughput/baseline)
-
 	b.ReportMetric(throughput, "msg/s")
 	b.ReportMetric(p95Micros, "p95-us")
 	b.ReportMetric(rpcReduction, "rpc-reduction-%")
 	b.ReportMetric(baselineRegression, "baseline-regression-%")
 
-	// Short calibration iterations are intentionally not gates. With the required
-	// -benchtime=10s, the final iteration crosses this duration and enforces SLOs.
-	if elapsed < gateMinDuration {
-		return
-	}
-	if rpcReduction < pf09MinRPCReduction {
-		b.Errorf("PF-09 %s RPC reduction %.2f%% is below %.2f%%", pf09StateName(state), rpcReduction, pf09MinRPCReduction)
-	}
 	if state == pf09L1 {
+		if result.elapsed < pf09SteadyDuration {
+			b.Errorf("PF-09 L1 duration %s is below fixed gate duration %s", result.elapsed, pf09SteadyDuration)
+		}
 		if throughput < pf09MinThroughput {
 			b.Errorf("PF-09 L1 throughput %.2f msg/s is below %.2f msg/s", throughput, pf09MinThroughput)
 		}
@@ -255,29 +261,296 @@ func runPF09Phase(
 	}
 }
 
-// pf09RPCSnapshot sums all configured Transmite instances. A comma-separated
-// PF09_TRANSMITE_VARS_URLS is required for a multi-instance performance stack;
-// the single-instance local stack falls back to tests/config.yaml.
-func pf09RPCSnapshot(b *testing.B) int64 {
+func runPF09SingleUse(conversations []pf09Conversation) pf09PhaseResult {
+	start := make(chan struct{})
+	results := make(chan struct {
+		latency int64
+		err     error
+	}, len(conversations))
+	var wg sync.WaitGroup
+	for i, conversation := range conversations {
+		i, conversation := i, conversation
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			<-start
+			requestStarted := time.Now()
+			err := sendPF09Message(conversation, uint64(i))
+			results <- struct {
+				latency int64
+				err     error
+			}{time.Since(requestStarted).Nanoseconds(), err}
+		}()
+	}
+	started := time.Now()
+	close(start)
+	wg.Wait()
+	elapsed := time.Since(started)
+	close(results)
+	result := pf09PhaseResult{elapsed: elapsed, latencies: make([]int64, 0, len(conversations))}
+	for item := range results {
+		if item.err != nil && result.err == nil {
+			result.err = item.err
+		}
+		if item.err == nil {
+			result.successes++
+			result.latencies = append(result.latencies, item.latency)
+		}
+	}
+	return result
+}
+
+func runPF09Steady(conversations []pf09Conversation) pf09PhaseResult {
+	workers := max(pf09ConversationCount, runtime.GOMAXPROCS(0)*4)
+	perWorkerSamples := max(1, pf09MaxLatencySamples/workers)
+	start := make(chan struct{})
+	deadline := time.Time{}
+	var sequence atomic.Uint64
+	var successes atomic.Uint64
+	var stopped atomic.Bool
+	var firstError error
+	var errorMu sync.Mutex
+	latencies := make([]int64, 0, pf09MaxLatencySamples)
+	var latencyMu sync.Mutex
+	var wg sync.WaitGroup
+	for worker := 0; worker < workers; worker++ {
+		worker := worker
+		wg.Add(1)
+		go func() {
+			defer wg.Done()
+			local := make([]int64, 0, perWorkerSamples)
+			var seen uint64
+			randomState := uint64(worker+1) | 1
+			<-start
+			for !stopped.Load() && time.Now().Before(deadline) {
+				id := sequence.Add(1) - 1
+				conversation := conversations[id%uint64(len(conversations))]
+				requestStarted := time.Now()
+				err := sendPF09Message(conversation, id)
+				latency := time.Since(requestStarted).Nanoseconds()
+				if err != nil {
+					errorMu.Lock()
+					if firstError == nil {
+						firstError = err
+						stopped.Store(true)
+					}
+					errorMu.Unlock()
+					break
+				}
+				successes.Add(1)
+				seen++
+				local, randomState = pf09ReservoirAdd(local, perWorkerSamples, seen, randomState, latency)
+			}
+			latencyMu.Lock()
+			remaining := pf09MaxLatencySamples - len(latencies)
+			if len(local) > remaining {
+				local = local[:remaining]
+			}
+			latencies = append(latencies, local...)
+			latencyMu.Unlock()
+		}()
+	}
+	started := time.Now()
+	deadline = started.Add(pf09SteadyDuration)
+	close(start)
+	wg.Wait()
+	return pf09PhaseResult{
+		elapsed:   time.Since(started),
+		successes: successes.Load(),
+		latencies: latencies,
+		err:       firstError,
+	}
+}
+
+func pf09ReservoirAdd(samples []int64, capacity int, seen, state uint64, value int64) ([]int64, uint64) {
+	if len(samples) < capacity {
+		return append(samples, value), state
+	}
+	state ^= state << 13
+	state ^= state >> 7
+	state ^= state << 17
+	if replacement := state % seen; replacement < uint64(len(samples)) {
+		samples[replacement] = value
+	}
+	return samples, state
+}
+
+func validatePF09PhaseCounters(state pf09CacheState, successes uint64, deltas map[string]int64) error {
+	if successes > math.MaxInt64 {
+		return fmt.Errorf("PF-09 success count overflows int64")
+	}
+	want := int64(successes)
+	switch state {
+	case pf09Cold:
+		if deltas[pf09RPCMetric] != want || deltas[pf09L1Metric] != 0 || deltas[pf09L2Metric] != 0 {
+			return fmt.Errorf("PF-09 cold state contaminated: success=%d rpc=%d l1=%d l2=%d", want, deltas[pf09RPCMetric], deltas[pf09L1Metric], deltas[pf09L2Metric])
+		}
+	case pf09L2:
+		if deltas[pf09L2Metric] != want || deltas[pf09L1Metric] != 0 || deltas[pf09RPCMetric] != 0 {
+			return fmt.Errorf("PF-09 L2 state contaminated: success=%d rpc=%d l1=%d l2=%d", want, deltas[pf09RPCMetric], deltas[pf09L1Metric], deltas[pf09L2Metric])
+		}
+	case pf09L1:
+		if deltas[pf09L1Metric] != want || deltas[pf09L2Metric] != 0 || deltas[pf09RPCMetric] != 0 {
+			return fmt.Errorf("PF-09 L1 state contaminated: success=%d rpc=%d l1=%d l2=%d", want, deltas[pf09RPCMetric], deltas[pf09L1Metric], deltas[pf09L2Metric])
+		}
+		if 100*(1-float64(deltas[pf09RPCMetric])/float64(want)) < pf09MinRPCReduction {
+			return fmt.Errorf("PF-09 L1 RPC reduction below %.0f%%", pf09MinRPCReduction)
+		}
+	default:
+		return fmt.Errorf("unknown PF-09 state %d", state)
+	}
+	return nil
+}
+
+func pf09ConfiguredEndpoints(b *testing.B) []string {
 	b.Helper()
 	raw := os.Getenv("PF09_TRANSMITE_VARS_URLS")
 	if raw == "" {
 		raw = HTTP.Config().Infra.TransmiteVars
 	}
-	var total int64
-	var count int
-	for _, endpoint := range strings.Split(raw, ",") {
-		endpoint = strings.TrimSpace(endpoint)
+	value := os.Getenv("PF09_EXPECTED_TRANSMITE_INSTANCES")
+	if value == "" {
+		b.Fatal("PF09_EXPECTED_TRANSMITE_INSTANCES is required by the full-stack gate")
+	}
+	expected, err := strconv.Atoi(value)
+	if err != nil || expected <= 0 {
+		b.Fatalf("PF09_EXPECTED_TRANSMITE_INSTANCES must be positive, got %q", value)
+	}
+	endpoints, err := parsePF09Endpoints(raw, expected)
+	if err != nil {
+		b.Fatal(err)
+	}
+	return endpoints
+}
+
+func parsePF09Endpoints(raw string, expected int) ([]string, error) {
+	parts := strings.Split(raw, ",")
+	endpoints := make([]string, 0, len(parts))
+	seen := make(map[string]struct{}, len(parts))
+	for _, part := range parts {
+		endpoint := strings.TrimRight(strings.TrimSpace(part), "/")
 		if endpoint == "" {
-			continue
+			return nil, fmt.Errorf("PF-09 contains an empty Transmite bvar endpoint")
+		}
+		if _, duplicate := seen[endpoint]; duplicate {
+			return nil, fmt.Errorf("PF-09 duplicate Transmite bvar endpoint %q", endpoint)
+		}
+		seen[endpoint] = struct{}{}
+		endpoints = append(endpoints, endpoint)
+	}
+	if len(endpoints) != expected {
+		return nil, fmt.Errorf("PF-09 expected %d Transmite instances, got %d", expected, len(endpoints))
+	}
+	return endpoints, nil
+}
+
+func capturePF09Snapshot(endpoints []string) (pf09Snapshot, error) {
+	snapshot := make(pf09Snapshot, len(endpoints))
+	for _, endpoint := range endpoints {
+		pid, err := readPF09Int(endpoint, "pid")
+		if err != nil || pid <= 0 {
+			return nil, fmt.Errorf("%s process identity: pid=%d err=%w", endpoint, pid, err)
+		}
+		uptime, err := readPF09Float(endpoint, "process_uptime")
+		if err != nil || uptime < 0 {
+			return nil, fmt.Errorf("%s process uptime: uptime=%f err=%w", endpoint, uptime, err)
+		}
+		instance := pf09InstanceSnapshot{
+			pid:                   pid,
+			uptimeSeconds:         uptime,
+			startedAtEstimateNano: time.Now().Add(-time.Duration(uptime * float64(time.Second))).UnixNano(),
+			counters:              make(map[string]int64, len(pf09CounterMetrics)),
 		}
-		total += verify.BVar(b, strings.TrimRight(endpoint, "/"), "user_info_rpc_total")
-		count++
+		for _, metric := range pf09CounterMetrics {
+			value, err := readPF09Int(endpoint, metric)
+			if err != nil || value < 0 {
+				return nil, fmt.Errorf("%s %s: value=%d err=%w", endpoint, metric, value, err)
+			}
+			instance.counters[metric] = value
+		}
+		snapshot[endpoint] = instance
+	}
+	return snapshot, nil
+}
+
+func readPF09Int(endpoint, name string) (int64, error) {
+	raw, err := readPF09BVar(endpoint, name)
+	if err != nil {
+		return 0, err
+	}
+	return strconv.ParseInt(strings.Trim(raw, "\""), 10, 64)
+}
+
+func readPF09Float(endpoint, name string) (float64, error) {
+	raw, err := readPF09BVar(endpoint, name)
+	if err != nil {
+		return 0, err
+	}
+	value, err := strconv.ParseFloat(strings.Trim(raw, "\""), 64)
+	if err != nil || math.IsNaN(value) || math.IsInf(value, 0) {
+		return 0, fmt.Errorf("parse %s=%q", name, raw)
+	}
+	return value, nil
+}
+
+func readPF09BVar(endpoint, name string) (string, error) {
+	client := &http.Client{Timeout: pf09SnapshotTimeout}
+	rsp, err := client.Get(endpoint + "/vars/" + name)
+	if err != nil {
+		return "", err
+	}
+	defer rsp.Body.Close()
+	body, err := io.ReadAll(io.LimitReader(rsp.Body, 4096))
+	if err != nil {
+		return "", err
+	}
+	if rsp.StatusCode != http.StatusOK {
+		return "", fmt.Errorf("HTTP %d: %s", rsp.StatusCode, strings.TrimSpace(string(body)))
 	}
-	if count == 0 {
-		b.Fatal("PF-09 requires at least one Transmite bvar endpoint")
+	value := strings.TrimSpace(string(body))
+	if value == "" {
+		return "", fmt.Errorf("empty bvar %s", name)
 	}
-	return total
+	return value, nil
+}
+
+func pf09SnapshotDelta(before, after pf09Snapshot) (map[string]int64, error) {
+	if len(before) == 0 || len(before) != len(after) {
+		return nil, fmt.Errorf("snapshot endpoint set changed: before=%d after=%d", len(before), len(after))
+	}
+	totals := make(map[string]int64, len(pf09CounterMetrics))
+	for endpoint, old := range before {
+		current, exists := after[endpoint]
+		if !exists {
+			return nil, fmt.Errorf("snapshot endpoint %s disappeared", endpoint)
+		}
+		if old.pid != current.pid {
+			return nil, fmt.Errorf("snapshot endpoint %s restarted: pid %d -> %d", endpoint, old.pid, current.pid)
+		}
+		if current.uptimeSeconds < old.uptimeSeconds {
+			return nil, fmt.Errorf("snapshot endpoint %s uptime rewound: %f -> %f", endpoint, old.uptimeSeconds, current.uptimeSeconds)
+		}
+		startDrift := time.Duration(current.startedAtEstimateNano - old.startedAtEstimateNano)
+		if startDrift < -pf09StartEstimateTolerance || startDrift > pf09StartEstimateTolerance {
+			return nil, fmt.Errorf("snapshot endpoint %s process start estimate changed by %s", endpoint, startDrift)
+		}
+		for _, metric := range pf09CounterMetrics {
+			oldValue, oldOK := old.counters[metric]
+			newValue, newOK := current.counters[metric]
+			if !oldOK || !newOK {
+				return nil, fmt.Errorf("snapshot endpoint %s missing %s", endpoint, metric)
+			}
+			if newValue < oldValue {
+				return nil, fmt.Errorf("snapshot endpoint %s counter %s rewound: %d -> %d", endpoint, metric, oldValue, newValue)
+			}
+			delta := newValue - oldValue
+			if delta > math.MaxInt64-totals[metric] {
+				return nil, fmt.Errorf("snapshot counter %s delta overflow", metric)
+			}
+			totals[metric] += delta
+		}
+	}
+	return totals, nil
 }
 
 func sendPF09Message(conversation pf09Conversation, sequence uint64) error {
@@ -287,9 +560,7 @@ func sendPF09Message(conversation pf09Conversation, sequence uint64) error {
 		ConversationId: conversation.id,
 		Content: &msg.MessageContent{
 			Type: msg.MessageType_TEXT,
-			Body: &msg.MessageContent_Text{Text: &msg.TextContent{
-				Text: fmt.Sprintf("pf09-%d", sequence),
-			}},
+			Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: fmt.Sprintf("pf09-%d", sequence)}},
 		},
 		ClientMsgId: client.NewRequestID(),
 	}, rsp)
@@ -308,26 +579,13 @@ func pf09Baseline(b *testing.B) float64 {
 	if raw := os.Getenv("PF09_BASELINE_MSG_PER_SEC"); raw != "" {
 		parsed, err := strconv.ParseFloat(raw, 64)
 		if err != nil || math.IsNaN(parsed) || math.IsInf(parsed, 0) || parsed < pf09CheckedInBaseline {
-			b.Fatalf("PF09_BASELINE_MSG_PER_SEC must be a finite value >= %.0f, got %q", pf09CheckedInBaseline, raw)
+			b.Fatalf("PF09_BASELINE_MSG_PER_SEC must be finite and >= %.0f, got %q", pf09CheckedInBaseline, raw)
 		}
 		value = parsed
 	}
 	return value
 }
 
-func pf09GateMinDuration(b *testing.B) time.Duration {
-	b.Helper()
-	raw := os.Getenv("PF09_GATE_MIN_DURATION")
-	if raw == "" {
-		return pf09DefaultGateMinDuration
-	}
-	duration, err := time.ParseDuration(raw)
-	if err != nil || duration <= 0 {
-		b.Fatalf("PF09_GATE_MIN_DURATION must be a positive duration, got %q", raw)
-	}
-	return duration
-}
-
 func pf09UserInfoKey(uid string) string {
 	const offset32 = uint32(2166136261)
 	const prime32 = uint32(16777619)

From f7fe97a7bd28cc1b03993808d15abd4c23cfbcd1 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 13:59:25 +0800
Subject: [PATCH 16/47] fix: close cache resilience review gaps

---
 common/dao/data_redis.hpp                     | 149 ++++++++++++++----
 common/utils/local_rate_limiter.hpp           |  48 ++++--
 common/utils/redis_circuit_breaker.hpp        |  55 ++++---
 common/utils/redis_keys.hpp                   |  10 ++
 ...7-15-cache-resilience-completion-design.md |  11 ++
 identity/source/identity_server.h             |  10 +-
 push/source/push_server.h                     | 121 +++++++++-----
 tests/func/cache_test.go                      |  68 ++++++--
 tests/perf/cache_helpers_test.go              |  12 +-
 tests/perf/cache_test.go                      |  50 +++++-
 tests/pkg/chaos/redis.go                      |  35 ++--
 tests/pkg/chaos/redis_test.go                 |  19 +++
 tests/reliability/redis_failover_test.go      |  71 +++++++--
 transmite/source/transmite_server.h           |  28 ++--
 14 files changed, 541 insertions(+), 146 deletions(-)
 create mode 100644 tests/pkg/chaos/redis_test.go

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 39b49ca..96c43fa 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -164,8 +164,10 @@ class RedisPipeline {
     void record_transition_(RedisCircuitBreaker::Transition transition) noexcept {
         if (transition == RedisCircuitBreaker::Transition::Opened) {
             metrics::g_redis_circuit_open_total << 1;
+            LOG_WARN("Redis circuit transition Closed->Open");
         } else if (transition == RedisCircuitBreaker::Transition::Recovered) {
             metrics::g_redis_circuit_recovered_total << 1;
+            LOG_INFO("Redis circuit transition HalfOpen->Closed");
         }
     }
 
@@ -384,7 +386,9 @@ class RedisClient
 private:
     RedisCircuitBreaker::Permit before_call_() {
         try {
-            return _breaker->before_call();
+            auto permit = _breaker->before_call();
+            if (permit.probe) LOG_INFO("Redis circuit transition Open->HalfOpen");
+            return permit;
         } catch (const RedisCircuitOpen &) {
             metrics::g_redis_circuit_rejected_total << 1;
             throw;
@@ -394,8 +398,10 @@ class RedisClient
     static void record_transition_(RedisCircuitBreaker::Transition transition) noexcept {
         if (transition == RedisCircuitBreaker::Transition::Opened) {
             metrics::g_redis_circuit_open_total << 1;
+            LOG_WARN("Redis circuit transition Closed->Open");
         } else if (transition == RedisCircuitBreaker::Transition::Recovered) {
             metrics::g_redis_circuit_recovered_total << 1;
+            LOG_INFO("Redis circuit transition HalfOpen->Closed");
         }
     }
 
@@ -790,25 +796,45 @@ class DeviceSet
     void touch(const std::string &uid, std::chrono::seconds ttl = kSessionTtl) {
         try { _c->expire(key::device_set_key(uid), randomized_ttl(ttl)); }
         catch(std::exception &e) { LOG_ERROR("DeviceSet.touch 失败 {}: {}", uid, e.what()); }
+        // Rolling-upgrade compatibility only: legacy keys get a bounded grace TTL.
+        try { _c->expire(key::legacy_device_set_key(uid), kLegacyGraceTtl); }
+        catch(std::exception &) {}
     }
     /* brief: 用户某设备下线 */
     void remove(const std::string &uid, const std::string &device_id) {
         try { _c->srem(key::device_set_key(uid), device_id); }
         catch(std::exception &e) { LOG_ERROR("DeviceSet.rem 失败 {}-{}: {}", uid, device_id, e.what()); }
+        try { _c->srem(key::legacy_device_set_key(uid), device_id); }
+        catch(std::exception &) {}
     }
     /* brief: 取用户当前所有在线设备 */
     std::vector<std::string> list(const std::string &uid) {
         std::vector<std::string> res;
-        try { _c->smembers(key::device_set_key(uid), std::inserter(res, res.end())); }
+        try { _c->smembers(key::device_set_key(uid), std::back_inserter(res)); }
         catch(std::exception &e) { LOG_ERROR("DeviceSet.list 失败 {}: {}", uid, e.what()); }
+        std::vector<std::string> legacy;
+        try { _c->smembers(key::legacy_device_set_key(uid), std::back_inserter(legacy)); }
+        catch(std::exception &) { return res; }
+        if (!legacy.empty()) {
+            std::unordered_set<std::string> merged(res.begin(), res.end());
+            merged.insert(legacy.begin(), legacy.end());
+            res.assign(merged.begin(), merged.end());
+            // Cross-slot migration is intentionally best-effort. New writes use only
+            // the tagged key; the old key expires after the rolling-upgrade window.
+            try {
+                _c->sadd(key::device_set_key(uid), legacy.begin(), legacy.end());
+                _c->expire(key::device_set_key(uid), randomized_ttl(kSessionTtl));
+                _c->expire(key::legacy_device_set_key(uid), kLegacyGraceTtl);
+            } catch (std::exception &) {}
+        }
         return res;
     }
     /* brief: 用户是否有任意在线设备 */
     bool any(const std::string &uid) {
-        try { return _c->scard(key::device_set_key(uid)) > 0; }
-        catch(std::exception &e) { LOG_ERROR("DeviceSet.any 失败 {}: {}", uid, e.what()); return false; }
+        return !list(uid).empty();
     }
 private:
+    static constexpr std::chrono::seconds kLegacyGraceTtl{24 * 3600};
     static constexpr const char *kAddLua = R"lua(
 local function key_type(k)
     local t = redis.call('TYPE', k)
@@ -1171,6 +1197,37 @@ class UserInfoCache
         }
     }
 
+    std::optional<uint64_t> generation(const std::string &uid) {
+        if (!_c) return uint64_t{0};
+        try {
+            auto value = _c->get(key::user_info_generation_key(uid));
+            return value ? std::stoull(*value) : uint64_t{0};
+        } catch (const std::exception &e) {
+            LOG_WARN("UserInfoCache.generation unavailable: {}", e.what());
+            return std::nullopt;
+        }
+    }
+
+    bool set_if_generation(const std::string &uid, const std::string &serialized,
+                           uint64_t expected_generation) {
+        if (!_c) return false;
+        const auto ttl = serialized.empty()
+            ? randomized_ttl(std::chrono::seconds(5))
+            : randomized_ttl(kUserInfoTtl);
+        try {
+            std::vector<std::string> keys = {
+                key::user_info_key(uid), key::user_info_generation_key(uid)};
+            std::vector<std::string> args = {
+                std::to_string(expected_generation), serialized,
+                std::to_string(ttl.count())};
+            return _c->eval<long long>(kSetIfGenerationLua, keys.begin(), keys.end(),
+                                       args.begin(), args.end()) == 1;
+        } catch (const std::exception &e) {
+            LOG_WARN("UserInfoCache fenced fill unavailable: {}", e.what());
+            return false;
+        }
+    }
+
     void batch_set(const std::unordered_map<std::string, std::string> &values) {
         if (!_c) return;
         std::unordered_map<uint32_t, std::vector<std::pair<std::string, std::string>>> groups;
@@ -1198,7 +1255,11 @@ class UserInfoCache
     bool invalidate(const std::string &uid) noexcept {
         if (!_c) return true;
         try {
-            _c->del(key::user_info_key(uid));
+            std::vector<std::string> keys = {
+                key::user_info_key(uid), key::user_info_generation_key(uid)};
+            std::vector<std::string> args;
+            _c->eval<long long>(kInvalidateLua, keys.begin(), keys.end(),
+                                args.begin(), args.end());
             return true;
         } catch (const std::exception &e) {
             LOG_ERROR("UserInfoCache.invalidate 失败 {}: {}", uid, e.what());
@@ -1208,6 +1269,19 @@ class UserInfoCache
     }
 
 private:
+    static constexpr const char *kSetIfGenerationLua = R"lua(
+local generation = tonumber(redis.call('GET', KEYS[2]) or '0')
+if generation ~= tonumber(ARGV[1]) then return 0 end
+redis.call('SET', KEYS[1], ARGV[2], 'EX', ARGV[3])
+redis.call('EXPIRE', KEYS[2], 604800)
+return 1
+)lua";
+    static constexpr const char *kInvalidateLua = R"lua(
+redis.call('INCR', KEYS[2])
+redis.call('EXPIRE', KEYS[2], 604800)
+redis.call('DEL', KEYS[1])
+return 1
+)lua";
     RedisClient::ptr _c;
 };
 
@@ -1284,6 +1358,14 @@ class OnlineRoute
         }
         return res;
     }
+    // Push delivery is a truth-source path: callers must distinguish an empty
+    // route from an unavailable Redis route table.
+    std::unordered_map<std::string, std::string> device_instances_map_strict(
+            const std::string &uid) {
+        std::unordered_map<std::string, std::string> res;
+        _c->hgetall(key::online_key(uid), std::inserter(res, res.end()));
+        return res;
+    }
     /* brief: 取设备所在 Push 实例 */
     std::string device_instance(const std::string &uid, const std::string &device_id) {
         try {
@@ -1392,11 +1474,12 @@ class RateLimiter
             long long cur = _c->eval<long long>(kRateLimitScript, keys.begin(), keys.end(),
                                                 args.begin(), args.end());
             return cur == 1;
-        } catch (const RedisCircuitOpen &e) {
-            LOG_ERROR("RateLimiter.allow {}: {}", key_full, e.what());
+        } catch (const RedisCircuitOpen &) {
             return allow_local_(key_full, max_count, window_sec);
         } catch (const sw::redis::Error &e) {
-            LOG_ERROR("RateLimiter.allow {}: {}", key_full, e.what());
+            if (should_log_redis_error_()) {
+                LOG_WARN("RateLimiter Redis unavailable; using local fallback: {}", e.what());
+            }
             return allow_local_(key_full, max_count, window_sec);
         }
     }
@@ -1407,6 +1490,13 @@ class RateLimiter
         return allow(key::kRateSsid + ssid, max_count, window_sec);
     }
 private:
+    bool should_log_redis_error_() noexcept {
+        const auto now = std::chrono::duration_cast<std::chrono::seconds>(
+            std::chrono::steady_clock::now().time_since_epoch()).count();
+        auto previous = _last_redis_error_log_sec.load(std::memory_order_relaxed);
+        return now > previous && _last_redis_error_log_sec.compare_exchange_strong(
+            previous, now, std::memory_order_relaxed, std::memory_order_relaxed);
+    }
     bool allow_local_(const std::string &key_full, int max_count, int window_sec) {
         metrics::g_rate_limit_local_fallback_total << 1;
         const bool allowed = _local.allow(key_full, max_count, window_sec);
@@ -1416,6 +1506,7 @@ class RateLimiter
 
     RedisClient::ptr _c;
     LocalRateLimiter _local;
+    std::atomic<int64_t> _last_redis_error_log_sec{0};
     static const std::string kRateLimitScript;
 };
 
@@ -1588,20 +1679,16 @@ class UnackedPush
     void push(const std::string &uid, const std::string &device_id,
               unsigned long user_seq, const std::string &payload_b64,
               long long score_ts, std::chrono::seconds ttl = kUnackedTtl) {
-        try {
-            std::string k = key_for(uid, device_id);
-            std::string ik = idx_key_for(uid, device_id);
-            std::string member = std::to_string(user_seq) + ":" + payload_b64;
-            const auto effective_ttl = randomized_ttl(ttl);
-            std::vector<std::string> keys = {k, ik};
-            std::vector<std::string> args = {
-                std::to_string(score_ts), member, std::to_string(user_seq), payload_b64,
-                std::to_string(effective_ttl.count())};
-            _c->eval<long long>(kPushLua, keys.begin(), keys.end(),
-                                args.begin(), args.end());
-        } catch(std::exception &e) {
-            LOG_ERROR("UnackedPush.push 失败 {}-{}-{}: {}", uid, device_id, user_seq, e.what());
-        }
+        std::string k = key_for(uid, device_id);
+        std::string ik = idx_key_for(uid, device_id);
+        std::string member = std::to_string(user_seq) + ":" + payload_b64;
+        const auto effective_ttl = randomized_ttl(ttl);
+        std::vector<std::string> keys = {k, ik};
+        std::vector<std::string> args = {
+            std::to_string(score_ts), member, std::to_string(user_seq), payload_b64,
+            std::to_string(effective_ttl.count())};
+        _c->eval<long long>(kPushLua, keys.begin(), keys.end(),
+                            args.begin(), args.end());
     }
     /* brief: 客户端 ACK 后移除（per-device，O(1) via HASH index） */
     void ack(const std::string &uid, const std::string &device_id,
@@ -1609,12 +1696,10 @@ class UnackedPush
         try {
             std::string k = key_for(uid, device_id);
             std::string ik = idx_key_for(uid, device_id);
-            auto payload = _c->hget(ik, std::to_string(user_seq));
-            if (payload) {
-                std::string member = std::to_string(user_seq) + ":" + *payload;
-                _c->zrem(k, member);
-                _c->hdel(ik, std::to_string(user_seq));
-            }
+            std::vector<std::string> keys = {k, ik};
+            std::vector<std::string> args = {std::to_string(user_seq)};
+            _c->eval<long long>(kAckLua, keys.begin(), keys.end(),
+                                args.begin(), args.end());
         } catch(std::exception &e) {
             LOG_ERROR("UnackedPush.ack 失败 {}-{}-{}: {}", uid, device_id, user_seq, e.what());
         }
@@ -1715,6 +1800,14 @@ end
 redis.call('EXPIRE', KEYS[1], ttl)
 redis.call('EXPIRE', KEYS[2], ttl)
 return updated
+)lua";
+
+    static constexpr const char *kAckLua = R"lua(
+local payload = redis.call('HGET', KEYS[2], ARGV[1])
+if not payload then return 0 end
+redis.call('ZREM', KEYS[1], ARGV[1] .. ':' .. payload)
+redis.call('HDEL', KEYS[2], ARGV[1])
+return 1
 )lua";
 
     RedisClient::ptr _c;
diff --git a/common/utils/local_rate_limiter.hpp b/common/utils/local_rate_limiter.hpp
index 166de40..0a0f61b 100644
--- a/common/utils/local_rate_limiter.hpp
+++ b/common/utils/local_rate_limiter.hpp
@@ -27,7 +27,7 @@ class LocalRateLimiter {
 
         const size_t shard_index = std::hash<std::string>{}(key) % kShardCount;
         Shard &shard = _shards[shard_index];
-        std::lock_guard<std::mutex> lock(shard.mu);
+        std::unique_lock<std::mutex> lock(shard.mu);
 
         const uint64_t operation = _operations.fetch_add(1, std::memory_order_relaxed) + 1;
         if (operation % 1024 == 0) {
@@ -38,7 +38,14 @@ class LocalRateLimiter {
         if (found == shard.buckets.end()) {
             if (!reserve_bucket_()) {
                 cleanup_(shard, now_ms, window_sec);
-                if (!reserve_bucket_()) return false;
+                if (!reserve_bucket_()) {
+                    lock.unlock();
+                    reclaim_idle_bounded_(shard_index, now_ms, window_sec);
+                    lock.lock();
+                    found = shard.buckets.find(key);
+                    if (found != shard.buckets.end()) return consume_(found->second, capacity, window_sec, now_ms);
+                    if (!reserve_bucket_()) return false;
+                }
             }
             try {
                 found = shard.buckets.emplace(
@@ -50,16 +57,7 @@ class LocalRateLimiter {
             return true;
         }
 
-        Bucket &bucket = found->second;
-        const int64_t elapsed_ms = now_ms > bucket.refill_ms ? now_ms - bucket.refill_ms : 0;
-        const double refill = static_cast<double>(elapsed_ms) * capacity /
-                              (static_cast<double>(window_sec) * 1000.0);
-        bucket.tokens = std::min(static_cast<double>(capacity), bucket.tokens + refill);
-        bucket.refill_ms = now_ms;
-        bucket.seen_ms = now_ms;
-        if (bucket.tokens < 1.0) return false;
-        bucket.tokens -= 1.0;
-        return true;
+        return consume_(found->second, capacity, window_sec, now_ms);
     }
 
 private:
@@ -73,8 +71,21 @@ class LocalRateLimiter {
         std::unordered_map<std::string, Bucket> buckets;
     };
 
+    bool consume_(Bucket &bucket, int capacity, int window_sec, int64_t now_ms) {
+        const int64_t elapsed_ms = now_ms > bucket.refill_ms ? now_ms - bucket.refill_ms : 0;
+        const double refill = static_cast<double>(elapsed_ms) * capacity /
+                              (static_cast<double>(window_sec) * 1000.0);
+        bucket.tokens = std::min(static_cast<double>(capacity), bucket.tokens + refill);
+        bucket.refill_ms = now_ms;
+        bucket.seen_ms = now_ms;
+        if (bucket.tokens < 1.0) return false;
+        bucket.tokens -= 1.0;
+        return true;
+    }
+
     static constexpr size_t kShardCount = 64;
     static constexpr size_t kMaxBuckets = 65536;
+    static constexpr size_t kReclaimScan = 8;
 
     bool reserve_bucket_() {
         size_t count = _bucket_count.load(std::memory_order_relaxed);
@@ -104,9 +115,22 @@ class LocalRateLimiter {
         }
     }
 
+    void reclaim_idle_bounded_(size_t target, int64_t now_ms, int window_sec) {
+        const size_t start = _reclaim_cursor.fetch_add(kReclaimScan, std::memory_order_relaxed);
+        for (size_t offset = 0; offset < kReclaimScan; ++offset) {
+            const size_t index = (start + offset) % kShardCount;
+            if (index == target) continue;
+            Shard &candidate = _shards[index];
+            std::lock_guard<std::mutex> lock(candidate.mu);
+            cleanup_(candidate, now_ms, window_sec);
+            if (_bucket_count.load(std::memory_order_relaxed) < kMaxBuckets) return;
+        }
+    }
+
     std::array<Shard, kShardCount> _shards;
     std::atomic<size_t> _bucket_count{0};
     std::atomic<uint64_t> _operations{0};
+    std::atomic<size_t> _reclaim_cursor{0};
 };
 
 }  // namespace chatnow
diff --git a/common/utils/redis_circuit_breaker.hpp b/common/utils/redis_circuit_breaker.hpp
index b5ba855..eeb71fa 100644
--- a/common/utils/redis_circuit_breaker.hpp
+++ b/common/utils/redis_circuit_breaker.hpp
@@ -36,8 +36,8 @@ class RedisCircuitBreaker {
 
     mutable std::mutex _mutex;
     std::atomic<State> _state{State::Closed};
-    uint64_t _generation = 0;
-    uint32_t _consecutive_failures = 0;
+    std::atomic<uint64_t> _generation{0};
+    std::atomic<uint32_t> _consecutive_failures{0};
     int64_t _open_until_ms = 0;
 };
 
@@ -50,36 +50,44 @@ inline RedisCircuitBreaker::Transition RedisCircuitBreaker::open_locked() noexce
     if (_state.load(std::memory_order_relaxed) == State::Open) {
         return Transition::None;
     }
-    ++_generation;
-    _consecutive_failures = 0;
+    _consecutive_failures.store(0, std::memory_order_relaxed);
     _open_until_ms = now_ms() + kOpenForMs;
     _state.store(State::Open, std::memory_order_release);
+    _generation.fetch_add(1, std::memory_order_acq_rel);
     return Transition::Opened;
 }
 
 inline RedisCircuitBreaker::Permit RedisCircuitBreaker::before_call() {
+    const auto generation = _generation.load(std::memory_order_acquire);
+    const auto current = _state.load(std::memory_order_acquire);
+    if (current == State::Closed) {
+        return {generation, false};
+    }
     std::lock_guard<std::mutex> lock(_mutex);
-    const auto current = _state.load(std::memory_order_relaxed);
-    if (current == State::Closed) return {_generation, false};
-    if (current == State::HalfOpen || now_ms() < _open_until_ms) {
+    const auto locked_current = _state.load(std::memory_order_relaxed);
+    if (locked_current == State::Closed) {
+        return {_generation.load(std::memory_order_acquire), false};
+    }
+    if (locked_current == State::HalfOpen || now_ms() < _open_until_ms) {
         throw RedisCircuitOpen();
     }
     _state.store(State::HalfOpen, std::memory_order_release);
-    return {_generation, true};
+    return {_generation.load(std::memory_order_acquire), true};
 }
 
 inline RedisCircuitBreaker::Transition
 RedisCircuitBreaker::on_success(Permit permit) noexcept {
-    std::lock_guard<std::mutex> lock(_mutex);
-    if (permit.generation != _generation) return Transition::None;
-
-    const auto current = _state.load(std::memory_order_relaxed);
+    const auto current = _state.load(std::memory_order_acquire);
     if (current == State::Closed && !permit.probe) {
-        _consecutive_failures = 0;
+        if (permit.generation == _generation.load(std::memory_order_acquire)) {
+            _consecutive_failures.store(0, std::memory_order_relaxed);
+        }
         return Transition::None;
     }
+    std::lock_guard<std::mutex> lock(_mutex);
+    if (permit.generation != _generation.load(std::memory_order_relaxed)) return Transition::None;
     if (current == State::HalfOpen && permit.probe) {
-        _consecutive_failures = 0;
+        _consecutive_failures.store(0, std::memory_order_relaxed);
         _state.store(State::Closed, std::memory_order_release);
         return Transition::Recovered;
     }
@@ -88,20 +96,25 @@ RedisCircuitBreaker::on_success(Permit permit) noexcept {
 
 inline RedisCircuitBreaker::Transition
 RedisCircuitBreaker::on_connection_failure(Permit permit) noexcept {
+    if (permit.generation != _generation.load(std::memory_order_acquire)) return Transition::None;
+    const auto current = _state.load(std::memory_order_acquire);
+    if (current == State::Closed && !permit.probe) {
+        const auto failures = _consecutive_failures.fetch_add(1, std::memory_order_relaxed) + 1;
+        if (failures < kFailureThreshold) return Transition::None;
+    }
     std::lock_guard<std::mutex> lock(_mutex);
-    if (permit.generation != _generation) return Transition::None;
-
-    const auto current = _state.load(std::memory_order_relaxed);
-    if (current == State::HalfOpen && permit.probe) return open_locked();
-    if (current != State::Closed || permit.probe) return Transition::None;
-    if (++_consecutive_failures < kFailureThreshold) return Transition::None;
+    if (permit.generation != _generation.load(std::memory_order_relaxed)) return Transition::None;
+    const auto locked_current = _state.load(std::memory_order_relaxed);
+    if (locked_current == State::HalfOpen && permit.probe) return open_locked();
+    if (locked_current != State::Closed || permit.probe) return Transition::None;
+    if (_consecutive_failures.load(std::memory_order_relaxed) < kFailureThreshold) return Transition::None;
     return open_locked();
 }
 
 inline RedisCircuitBreaker::Transition
 RedisCircuitBreaker::on_abandoned(Permit permit) noexcept {
     std::lock_guard<std::mutex> lock(_mutex);
-    if (permit.generation != _generation || !permit.probe ||
+    if (permit.generation != _generation.load(std::memory_order_relaxed) || !permit.probe ||
         _state.load(std::memory_order_relaxed) != State::HalfOpen) {
         return Transition::None;
     }
diff --git a/common/utils/redis_keys.hpp b/common/utils/redis_keys.hpp
index 3614c44..270eec0 100644
--- a/common/utils/redis_keys.hpp
+++ b/common/utils/redis_keys.hpp
@@ -84,6 +84,11 @@ inline std::string user_info_key(const std::string &uid) {
     return "im:user:{" + bucket + "}:" + uid;
 }
 
+inline std::string user_info_generation_key(const std::string &uid) {
+    auto bucket = std::to_string(user_info_bucket(uid));
+    return "im:user-gen:{" + bucket + "}:" + uid;
+}
+
 inline std::string local_route_cache_key(const std::string &uid) {
     return std::string("local:route:") + hash_tag(uid);
 }
@@ -96,6 +101,11 @@ inline std::string device_set_key(const std::string &uid) {
     return std::string(kDeviceSet) + hash_tag(uid);
 }
 
+// Pre-3.0 key retained only for bounded lazy migration during rolling upgrades.
+inline std::string legacy_device_set_key(const std::string &uid) {
+    return std::string(kDeviceSet) + uid;
+}
+
 inline std::string online_scan_pattern() {
     return std::string(kOnline) + "*";
 }
diff --git a/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
index fc32a02..71e2018 100644
--- a/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
+++ b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
@@ -202,6 +202,12 @@ Identity 修改昵称、头像、签名或其他 Profile 字段成功后删除 L
 DeviceSet 不使用短在线 TTL。它的生命周期与登录设备接近；7 天 TTL 防止永久
 常驻，同时避免正常在线设备被几分钟级 TTL 误删。
 
+滚动升级期间，带 hash tag 的 `im:dev:{uid}` 是新权威 key。读取会额外读取旧版
+`im:dev:uid`，合并成员并懒迁移到新 key；删除同时清理两边。旧 key 不再接收新写，
+每次被观察时只续一个最长 24 小时的迁移 grace TTL，因此不会形成无限双写或永久
+兼容负担。跨 slot 的迁移分步执行是有意选择：迁移可重试，在线路由正确性仍由同
+slot 的新 key 与 `im:online:{uid}` 原子脚本保证。
+
 ## 7. RedisMutex 退避
 
 `try_lock` 保持现有 `SET key token NX PX ttl` 和 Lua CAS 解锁协议，仅调整竞争等待：
@@ -260,6 +266,11 @@ C++ gtest 或 Python 合约测试。
   多个样本不过期于同一秒。
 - `FN-CA-05`：Redis 正常时大量消息请求触发分布式限流。
 
+Session 与 Status DAO 在当前 3.0-dev 没有生产调用点，不为测试增加无业务意义的
+endpoint。它们的 TTL 源码契约由独立 contract test 覆盖；Codes、DeviceSet、
+UnackedPush 等可达路径继续由新框架做黑盒验证。待业务重新接入前两者时，再把对应
+断言提升为黑盒测试。
+
 ### 10.2 可靠性测试
 
 文件：`tests/reliability/redis_failover_test.go`，build tag：`reliability`。
diff --git a/identity/source/identity_server.h b/identity/source/identity_server.h
index a7281a3..bb336b2 100644
--- a/identity/source/identity_server.h
+++ b/identity/source/identity_server.h
@@ -476,13 +476,17 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService
             if (request->has_phone()) {
                 user->phone(request->phone());
             }
+            // Advance the shared cache generation before committing the profile.
+            // If Redis is unavailable, fail closed so a successful update can
+            // never leave a one-hour stale L2 value visible to readers.
+            if (_user_info_cache && !_user_info_cache->invalidate(auth.user_id)) {
+                throw ServiceError(::chatnow::error::kSystemUnavailable,
+                                   "profile cache invalidation unavailable");
+            }
             if (!_mysql_user->update(user)) {
                 throw ServiceError(::chatnow::error::kSystemInternalError,
                                    "db update failed");
             }
-            if (_user_info_cache) {
-                (void)_user_info_cache->invalidate(auth.user_id);
-            }
             std::string ob_payload = outbox_payload_upsert_(
                 user->user_id(), user->mail(), user->phone(),
                 user->nickname(), user->description(),
diff --git a/push/source/push_server.h b/push/source/push_server.h
index 100c192..14e65fd 100644
--- a/push/source/push_server.h
+++ b/push/source/push_server.h
@@ -125,14 +125,14 @@ class PushServiceImpl : public PushService
                 payload = notify.SerializeAsString();
             }
 
-            int delivered = 0;
             auto route = resolve_route(request->user_id());
-            for (const auto &did : route.device_ids) {
-                if (filter_devices && target_dids.find(did) == target_dids.end()) continue;
-                if (_local_send(request->user_id(), did, payload) > 0) ++delivered;
+            // Persist before delivery. On failure the RPC is retryable and no
+            // client has observed a payload without durable retransmit state.
+            if (request->has_user_seq() && !_unacked) {
+                throw ::chatnow::ServiceError(::chatnow::error::kSystemUnavailable,
+                                              "unacked persistence unavailable");
             }
-
-            if (request->has_user_seq() && _unacked) {
+            if (request->has_user_seq()) {
                 std::string payload_b64 = _utils_base64_encode(payload);
                 long long now_ts = static_cast<long long>(time(nullptr));
                 for (const auto &did : route.device_ids) {
@@ -142,7 +142,23 @@ class PushServiceImpl : public PushService
                 }
             }
 
+            int delivered = 0;
+            for (const auto &did : route.device_ids) {
+                if (filter_devices && target_dids.find(did) == target_dids.end()) continue;
+                if (_local_send(request->user_id(), did, payload) > 0) ++delivered;
+            }
+
             response->set_online_device_count(delivered);
+        } catch (const RedisCircuitOpen& e) {
+            response->mutable_header()->set_success(false);
+            response->mutable_header()->set_error_code(::chatnow::error::kSystemUnavailable);
+            response->mutable_header()->set_error_message("unacked persistence unavailable");
+            cntl->SetFailed(e.what());
+        } catch (const sw::redis::Error& e) {
+            response->mutable_header()->set_success(false);
+            response->mutable_header()->set_error_code(::chatnow::error::kSystemUnavailable);
+            response->mutable_header()->set_error_message("unacked persistence unavailable");
+            cntl->SetFailed(e.what());
         } catch (const ::chatnow::ServiceError& e) {
             response->mutable_header()->set_success(false);
             response->mutable_header()->set_error_code(e.code());
@@ -193,14 +209,28 @@ class PushServiceImpl : public PushService
                 }
 
                 for (const auto &did : route.device_ids) {
-                    if (_local_send(uid, did, payload) > 0) ++total;
-                    if (it != uid2seq.end() && _unacked) {
+                    if (it != uid2seq.end() && !_unacked) {
+                        throw ::chatnow::ServiceError(::chatnow::error::kSystemUnavailable,
+                                                      "unacked persistence unavailable");
+                    }
+                    if (it != uid2seq.end()) {
                         _unacked->push(uid, did, it->second,
                                        _utils_base64_encode(payload), now_ts);
                     }
+                    if (_local_send(uid, did, payload) > 0) ++total;
                 }
             }
             response->set_online_count(total);
+        } catch (const RedisCircuitOpen& e) {
+            response->mutable_header()->set_success(false);
+            response->mutable_header()->set_error_code(::chatnow::error::kSystemUnavailable);
+            response->mutable_header()->set_error_message("unacked persistence unavailable");
+            cntl->SetFailed(e.what());
+        } catch (const sw::redis::Error& e) {
+            response->mutable_header()->set_success(false);
+            response->mutable_header()->set_error_code(::chatnow::error::kSystemUnavailable);
+            response->mutable_header()->set_error_message("unacked persistence unavailable");
+            cntl->SetFailed(e.what());
         } catch (const ::chatnow::ServiceError& e) {
             response->mutable_header()->set_success(false);
             response->mutable_header()->set_error_code(e.code());
@@ -240,7 +270,16 @@ class PushServiceImpl : public PushService
         std::vector<std::string> remote_uids;
         remote_uids.reserve(internal_msg.member_id_list_size());
         for (const auto &uid : internal_msg.member_id_list()) {
-            auto route = resolve_route(uid);
+            RouteEntry route;
+            try {
+                route = resolve_route(uid);
+            } catch (const RedisCircuitOpen &e) {
+                LOG_WARN("Push-Consumer: route circuit unavailable: {}", e.what());
+                return ConsumeAction::NackRequeue;
+            } catch (const sw::redis::Error &e) {
+                LOG_WARN("Push-Consumer: route Redis unavailable: {}", e.what());
+                return ConsumeAction::NackRequeue;
+            }
             if (route.device_ids.empty()) { remote_uids.push_back(uid); continue; }
 
             auto itu = uid2seq.find(uid);
@@ -260,11 +299,18 @@ class PushServiceImpl : public PushService
                 std::string inst = (it != route.device_to_instance.end()) ? it->second : "";
                 if (inst == _instance_id) {
                     if (itu != uid2seq.end()) {
-                        if (_local_send(uid, did, user_payload) > 0) any_local = true;
-                        if (_unacked) {
+                        if (!_unacked) return ConsumeAction::NackRequeue;
+                        try {
                             _unacked->push(uid, did, itu->second,
                                            _utils_base64_encode(user_payload), now_ts);
+                        } catch (const RedisCircuitOpen &e) {
+                            LOG_WARN("Push-Consumer: unacked circuit unavailable: {}", e.what());
+                            return ConsumeAction::NackRequeue;
+                        } catch (const sw::redis::Error &e) {
+                            LOG_WARN("Push-Consumer: unacked Redis unavailable: {}", e.what());
+                            return ConsumeAction::NackRequeue;
                         }
+                        if (_local_send(uid, did, user_payload) > 0) any_local = true;
                     } else {
                         // 大群读扩散：无 user_seq，仅下发
                         _local_send(uid, did, notify_template.SerializeAsString());
@@ -280,7 +326,14 @@ class PushServiceImpl : public PushService
         // 2) 跨实例：按 Push 实例 ID 分组
         std::unordered_map<std::string, std::unordered_set<std::string>> peer_to_uids;
         for (const auto &uid : remote_uids) {
-            auto route = resolve_route(uid);
+            RouteEntry route;
+            try {
+                route = resolve_route(uid);
+            } catch (const RedisCircuitOpen &) {
+                return ConsumeAction::NackRequeue;
+            } catch (const sw::redis::Error &) {
+                return ConsumeAction::NackRequeue;
+            }
             for (const auto &did : route.device_ids) {
                 auto it = route.device_to_instance.find(did);
                 std::string peer = (it != route.device_to_instance.end()) ? it->second : "";
@@ -289,45 +342,37 @@ class PushServiceImpl : public PushService
             }
         }
 
-        // 3) 每个对端一次 PushBatch（异步 brpc::DoNothing）
-        std::string internal_b64 = _utils_base64_encode(internal_msg.SerializeAsString());
+        // 3) 每个对端一次 PushBatch。等待持久化结果后才能 ACK MQ；若后续
+        // peer 失败，MQ 重投可能重复前面的下发，但 Unacked ZADD/HSET 以 seq
+        // 幂等覆盖，提供明确的 at-least-once 语义。
         for (auto &kv : peer_to_uids) {
             const std::string &peer = kv.first;
             std::vector<std::string> uids(kv.second.begin(), kv.second.end());
             auto channel = _mm_channels->choose(peer);
             if (!channel) {
                 LOG_WARN("Push-Consumer: 对端 {} 不可达", peer);
-                for (const auto &u : uids)
-                    if (_online_route) _online_route->unbind(u, "", peer);
-                if (_cross_outbox) {
-                    _cross_outbox->enqueue(internal_b64, uids, peer, now_ts);
-                }
-                continue;
+                return ConsumeAction::NackRequeue;
             }
             PushService_Stub stub(channel.get());
-            auto *closure = new SelfDeleteRpcClosure<PushBatchReq, PushBatchRsp>();
-            closure->req.set_request_id(msg_info.client_msg_id());
-            for (const auto &u : uids) closure->req.add_user_id_list(u);
-            closure->req.mutable_notify()->CopyFrom(notify_template);
+            PushBatchReq req;
+            PushBatchRsp rsp;
+            brpc::Controller cntl;
+            req.set_request_id(msg_info.client_msg_id());
+            for (const auto &u : uids) req.add_user_id_list(u);
+            req.mutable_notify()->CopyFrom(notify_template);
             for (const auto &u : uids) {
                 auto it = uid2seq.find(u);
                 if (it == uid2seq.end()) continue;
-                auto *p = closure->req.add_user_seqs();
+                auto *p = req.add_user_seqs();
                 p->set_user_id(u);
                 p->set_user_seq(it->second);
             }
-            std::string peer_id = peer;
-            closure->on_done = [peer_id, uids, outbox = _cross_outbox,
-                                online = _online_route, internal_b64, now_ts]
-                (brpc::Controller *c, const PushBatchRsp &) {
-                if (c->Failed()) {
-                    LOG_WARN("PushBatch 跨实例失败 peer={}: {}", peer_id, c->ErrorText());
-                    for (const auto &u : uids)
-                        if (online) online->unbind(u, "", peer_id);
-                    if (outbox) outbox->enqueue(internal_b64, uids, peer_id, now_ts);
-                }
-            };
-            stub.PushBatch(&closure->cntl, &closure->req, &closure->rsp, closure);
+            stub.PushBatch(&cntl, &req, &rsp, nullptr);
+            if (cntl.Failed() || !rsp.header().success()) {
+                LOG_WARN("PushBatch persistence failed peer={}: {} {}", peer,
+                         cntl.ErrorText(), rsp.header().error_message());
+                return ConsumeAction::NackRequeue;
+            }
         }
         return ConsumeAction::Ack;
     }
@@ -628,7 +673,7 @@ class PushServiceImpl : public PushService
 
         //  L2 Redis: hgetall + build RouteEntry
         RouteEntry route;
-        auto dmap = _online_route->device_instances_map(uid);
+        auto dmap = _online_route->device_instances_map_strict(uid);
         route.device_ids.reserve(dmap.size());
         for (const auto &[did, inst] : dmap) {
             route.device_ids.push_back(did);
diff --git a/tests/func/cache_test.go b/tests/func/cache_test.go
index 7966a40..6a945c5 100644
--- a/tests/func/cache_test.go
+++ b/tests/func/cache_test.go
@@ -257,6 +257,8 @@ func checkCacheTTLSourceContracts(source string) error {
 			"_c->eval<long long>(kAddLua",
 			"redis.call('SADD', KEYS[1], ARGV[1])",
 			"redis.call('EXPIRE', KEYS[1], ttl)",
+			"key::legacy_device_set_key(uid)",
+			"kLegacyGraceTtl",
 		}},
 		{"OnlineRoute", []string{
 			"key::online_key(uid), key::device_set_key(uid)",
@@ -269,6 +271,7 @@ func checkCacheTTLSourceContracts(source string) error {
 		{"UnackedPush", []string{
 			"_c->eval<long long>(kPushLua",
 			"_c->eval<long long>(kBumpScoreLua",
+			"_c->eval<long long>(kAckLua",
 			"redis.call('ZADD', KEYS[1]",
 			"redis.call('HSET', KEYS[2]",
 			"redis.call('EXPIRE', KEYS[1], ttl)",
@@ -287,9 +290,6 @@ func checkCacheTTLSourceContracts(source string) error {
 			return fmt.Errorf("%s contains a direct fixed TTL cache write", name)
 		}
 	}
-	if strings.Contains(sections["DeviceSet"], "_c->sadd(") {
-		return fmt.Errorf("DeviceSet bypasses its atomic add script")
-	}
 	if strings.Count(sections["UnackedPush"], "randomized_ttl(ttl)") != 2 {
 		return fmt.Errorf("UnackedPush push and bump_score must each sample one randomized TTL")
 	}
@@ -300,6 +300,30 @@ func checkCacheTTLSourceContracts(source string) error {
 	return nil
 }
 
+func TestFN_CA_ResilienceSourceContracts(t *testing.T) {
+	dao := readRepoSource(t, "common/dao/data_redis.hpp")
+	transmiteSource := readRepoSource(t, "transmite/source/transmite_server.h")
+	pushSource := readRepoSource(t, "push/source/push_server.h")
+	breaker := readRepoSource(t, "common/utils/redis_circuit_breaker.hpp")
+	for _, contract := range []struct {
+		name   string
+		source string
+		want   []string
+	}{
+		{"generation fence", dao, []string{"set_if_generation", "kSetIfGenerationLua", "redis.call('INCR', KEYS[2])"}},
+		{"fenced fill", transmiteSource, []string{"generation(uid)", "set_if_generation(uid, bytes", "set_if_generation(uid, \"\""}},
+		{"truth source push", pushSource, []string{"Persist before delivery", "ConsumeAction::NackRequeue", "unacked persistence unavailable"}},
+		{"atomic ack", dao, []string{"kAckLua", "redis.call('ZREM', KEYS[1]", "redis.call('HDEL', KEYS[2]"}},
+		{"atomic breaker hot path", breaker, []string{"std::atomic<uint64_t> _generation", "std::atomic<uint32_t> _consecutive_failures"}},
+	} {
+		for _, want := range contract.want {
+			if !strings.Contains(contract.source, want) {
+				t.Errorf("%s missing %q", contract.name, want)
+			}
+		}
+	}
+}
+
 func requireCacheTTLSourceContracts(t testing.TB, source string) {
 	t.Helper()
 	require.NoError(t, checkCacheTTLSourceContracts(source))
@@ -487,21 +511,23 @@ func TestFN_CA_RateLimit(t *testing.T) {
 func TestFN_CA_UserInfoL2AvoidsRepeatedRPC(t *testing.T) {
 	user, _, convID := fixture.MakeFriends(t, HTTP)
 	verify.RedisCLI(t, "DEL", userInfoRedisKey(user.UserID))
-	before := verify.BVar(t, HTTP.Config().Infra.TransmiteVars, "user_info_rpc_total")
+	endpoints := transmiteBVarEndpoints(t)
+	before := sumBVar(t, endpoints, "user_info_rpc_total")
 
 	for i := 0; i < 20; i++ {
 		sendCacheTestMessage(t, user, convID, fmt.Sprintf("repeat-%d", i))
 	}
 
-	after := verify.BVar(t, HTTP.Config().Infra.TransmiteVars, "user_info_rpc_total")
-	require.LessOrEqual(t, after-before, int64(1))
+	after := sumBVar(t, endpoints, "user_info_rpc_total")
+	require.LessOrEqual(t, after-before, int64(len(endpoints)))
 	require.Equal(t, "1", verify.RedisCLI(t, "EXISTS", userInfoRedisKey(user.UserID)))
 }
 
 func TestFN_CA_UserInfoSingleflight(t *testing.T) {
 	user, _, convID := fixture.MakeFriends(t, HTTP)
 	verify.RedisCLI(t, "DEL", userInfoRedisKey(user.UserID))
-	before := verify.BVar(t, HTTP.Config().Infra.TransmiteVars, "user_info_rpc_total")
+	endpoints := transmiteBVarEndpoints(t)
+	before := sumBVar(t, endpoints, "user_info_rpc_total")
 
 	start := make(chan struct{})
 	results := make(chan error, 200)
@@ -522,8 +548,32 @@ func TestFN_CA_UserInfoSingleflight(t *testing.T) {
 		require.NoError(t, err)
 	}
 
-	after := verify.BVar(t, HTTP.Config().Infra.TransmiteVars, "user_info_rpc_total")
-	require.LessOrEqual(t, after-before, int64(1))
+	after := sumBVar(t, endpoints, "user_info_rpc_total")
+	require.LessOrEqual(t, after-before, int64(len(endpoints)),
+		"each Transmite process may originate at most one Identity RPC")
+}
+
+func transmiteBVarEndpoints(t testing.TB) []string {
+	t.Helper()
+	parts := strings.Split(HTTP.Config().Infra.TransmiteVars, ",")
+	endpoints := make([]string, 0, len(parts))
+	for _, part := range parts {
+		endpoint := strings.TrimRight(strings.TrimSpace(part), "/")
+		if endpoint == "" {
+			t.Fatal("empty Transmite bvar endpoint")
+		}
+		endpoints = append(endpoints, endpoint)
+	}
+	return endpoints
+}
+
+func sumBVar(t testing.TB, endpoints []string, name string) int64 {
+	t.Helper()
+	var total int64
+	for _, endpoint := range endpoints {
+		total += verify.BVar(t, endpoint, name)
+	}
+	return total
 }
 
 func TestFN_CA_UserInfoInvalidatedAfterProfileUpdate(t *testing.T) {
diff --git a/tests/perf/cache_helpers_test.go b/tests/perf/cache_helpers_test.go
index 55f0c81..de7359e 100644
--- a/tests/perf/cache_helpers_test.go
+++ b/tests/perf/cache_helpers_test.go
@@ -121,14 +121,16 @@ func TestPF09SnapshotRequiresProcessIdentityFields(t *testing.T) {
 func TestPF09PhaseCountersProveStableCacheState(t *testing.T) {
 	valid := []struct {
 		state  pf09CacheState
+		count  uint64
 		deltas map[string]int64
 	}{
-		{pf09Cold, map[string]int64{pf09RPCMetric: 20, pf09L1Metric: 0, pf09L2Metric: 0}},
-		{pf09L2, map[string]int64{pf09RPCMetric: 0, pf09L1Metric: 0, pf09L2Metric: 20}},
-		{pf09L1, map[string]int64{pf09RPCMetric: 0, pf09L1Metric: 20, pf09L2Metric: 0}},
+		{pf09Cold, 20, map[string]int64{pf09RPCMetric: 20, pf09L1Metric: 0, pf09L2Metric: 0}},
+		{pf09L2, 20, map[string]int64{pf09RPCMetric: 0, pf09L1Metric: 0, pf09L2Metric: 20}},
+		{pf09L1, 20, map[string]int64{pf09RPCMetric: 0, pf09L1Metric: 20, pf09L2Metric: 0}},
+		{pf09Stampede, 200, map[string]int64{pf09RPCMetric: 2, pf09L1Metric: 198, pf09L2Metric: 0}},
 	}
 	for _, test := range valid {
-		if err := validatePF09PhaseCounters(test.state, 20, test.deltas); err != nil {
+		if err := validatePF09PhaseCounters(test.state, test.count, 2, test.deltas); err != nil {
 			t.Errorf("valid %s counters rejected: %v", pf09StateName(test.state), err)
 		}
 		contaminated := make(map[string]int64, len(test.deltas))
@@ -136,7 +138,7 @@ func TestPF09PhaseCountersProveStableCacheState(t *testing.T) {
 			contaminated[metric] = value
 		}
 		contaminated[pf09RPCMetric]++
-		if err := validatePF09PhaseCounters(test.state, 20, contaminated); err == nil {
+		if err := validatePF09PhaseCounters(test.state, test.count, 2, contaminated); err == nil {
 			t.Errorf("contaminated %s counters accepted", pf09StateName(test.state))
 		}
 	}
diff --git a/tests/perf/cache_test.go b/tests/perf/cache_test.go
index 074727d..1ada70e 100644
--- a/tests/perf/cache_test.go
+++ b/tests/perf/cache_test.go
@@ -29,6 +29,7 @@ import (
 
 const (
 	pf09ConversationCount      = 20
+	pf09StampedeConcurrency    = 200
 	pf09MaxLatencySamples      = 65_536
 	pf09MinThroughput          = 5_000.0
 	pf09CheckedInBaseline      = 5_000.0
@@ -51,6 +52,7 @@ const (
 	pf09Cold pf09CacheState = iota
 	pf09L2
 	pf09L1
+	pf09Stampede
 )
 
 type pf09Conversation struct {
@@ -98,6 +100,7 @@ func BenchmarkPF09_UserInfoCache(b *testing.B) {
 		{name: "cold", state: pf09Cold},
 		{name: "L2", state: pf09L2},
 		{name: "L1", state: pf09L1},
+		{name: "stampede", state: pf09Stampede},
 	} {
 		phase := phase
 		b.Run(phase.name, func(b *testing.B) {
@@ -123,6 +126,16 @@ func validatePF09IterationCount(n int) error {
 func preparePF09Conversations(b *testing.B, state pf09CacheState, instanceCount int) []pf09Conversation {
 	b.Helper()
 	b.StopTimer()
+	if state == pf09Stampede {
+		sender, peer, conversationID := fixture.MakeFriends(b, HTTP)
+		_ = peer
+		verifyPF09DeleteL2(b, sender.UserID)
+		conversations := make([]pf09Conversation, pf09StampedeConcurrency)
+		for i := range conversations {
+			conversations[i] = pf09Conversation{sender: sender, id: conversationID}
+		}
+		return conversations
+	}
 	conversations := make([]pf09Conversation, pf09ConversationCount)
 	for i := range conversations {
 		sender, peer, conversationID := fixture.MakeFriends(b, HTTP)
@@ -148,6 +161,15 @@ func preparePF09Conversations(b *testing.B, state pf09CacheState, instanceCount
 }
 
 func validatePF09Conversations(conversations []pf09Conversation) error {
+	if len(conversations) == pf09StampedeConcurrency {
+		first := conversations[0]
+		for _, conversation := range conversations {
+			if conversation.sender == nil || conversation.sender.UserID != first.sender.UserID || conversation.id != first.id {
+				return fmt.Errorf("stampede phase must target one cold key")
+			}
+		}
+		return nil
+	}
 	if len(conversations) != pf09ConversationCount {
 		return fmt.Errorf("want %d conversations, got %d", pf09ConversationCount, len(conversations))
 	}
@@ -170,6 +192,15 @@ func validatePF09Conversations(conversations []pf09Conversation) error {
 	return nil
 }
 
+func verifyPF09DeleteL2(b *testing.B, uid string) {
+	b.Helper()
+	cmd := exec.Command("docker", "exec", HTTP.Config().Infra.RedisContainer,
+		"redis-cli", "-c", "DEL", pf09UserInfoKey(uid))
+	if out, err := cmd.CombinedOutput(); err != nil {
+		b.Fatalf("clear PF-09 L2: %v: %s", err, strings.TrimSpace(string(out)))
+	}
+}
+
 func seedPF09L2(b *testing.B, sender *client.HTTPClient) {
 	b.Helper()
 	rsp := &identity.GetProfileRsp{}
@@ -210,6 +241,8 @@ func runPF09Phase(
 		b.Fatalf("PF-09 before snapshot: %v", err)
 	}
 
+	var memoryBefore runtime.MemStats
+	runtime.ReadMemStats(&memoryBefore)
 	b.ResetTimer()
 	var result pf09PhaseResult
 	if state == pf09L1 {
@@ -218,6 +251,8 @@ func runPF09Phase(
 		result = runPF09SingleUse(conversations)
 	}
 	b.StopTimer()
+	var memoryAfter runtime.MemStats
+	runtime.ReadMemStats(&memoryAfter)
 	if result.err != nil {
 		b.Fatalf("PF-09 %s send failed: %v", pf09StateName(state), result.err)
 	}
@@ -233,7 +268,7 @@ func runPF09Phase(
 	if err != nil {
 		b.Fatalf("PF-09 snapshot integrity: %v", err)
 	}
-	if err := validatePF09PhaseCounters(state, result.successes, deltas); err != nil {
+	if err := validatePF09PhaseCounters(state, result.successes, len(endpoints), deltas); err != nil {
 		b.Error(err)
 	}
 
@@ -247,6 +282,8 @@ func runPF09Phase(
 	b.ReportMetric(p95Micros, "p95-us")
 	b.ReportMetric(rpcReduction, "rpc-reduction-%")
 	b.ReportMetric(baselineRegression, "baseline-regression-%")
+	b.ReportMetric(float64(memoryAfter.Mallocs-memoryBefore.Mallocs)/float64(result.successes), "allocs/req")
+	b.ReportMetric(float64(memoryAfter.TotalAlloc-memoryBefore.TotalAlloc)/float64(result.successes), "bytes/req")
 
 	if state == pf09L1 {
 		if result.elapsed < pf09SteadyDuration {
@@ -375,7 +412,7 @@ func pf09ReservoirAdd(samples []int64, capacity int, seen, state uint64, value i
 	return samples, state
 }
 
-func validatePF09PhaseCounters(state pf09CacheState, successes uint64, deltas map[string]int64) error {
+func validatePF09PhaseCounters(state pf09CacheState, successes uint64, instanceCount int, deltas map[string]int64) error {
 	if successes > math.MaxInt64 {
 		return fmt.Errorf("PF-09 success count overflows int64")
 	}
@@ -396,6 +433,13 @@ func validatePF09PhaseCounters(state pf09CacheState, successes uint64, deltas ma
 		if 100*(1-float64(deltas[pf09RPCMetric])/float64(want)) < pf09MinRPCReduction {
 			return fmt.Errorf("PF-09 L1 RPC reduction below %.0f%%", pf09MinRPCReduction)
 		}
+	case pf09Stampede:
+		rpc := deltas[pf09RPCMetric]
+		if rpc < 1 || rpc > int64(instanceCount) ||
+			rpc+deltas[pf09L1Metric]+deltas[pf09L2Metric] != want {
+			return fmt.Errorf("PF-09 stampede invariant failed: success=%d instances=%d rpc=%d l1=%d l2=%d",
+				want, instanceCount, rpc, deltas[pf09L1Metric], deltas[pf09L2Metric])
+		}
 	default:
 		return fmt.Errorf("unknown PF-09 state %d", state)
 	}
@@ -605,6 +649,8 @@ func pf09StateName(state pf09CacheState) string {
 		return "L2"
 	case pf09L1:
 		return "L1"
+	case pf09Stampede:
+		return "stampede"
 	default:
 		return "unknown"
 	}
diff --git a/tests/pkg/chaos/redis.go b/tests/pkg/chaos/redis.go
index 501897e..d136862 100644
--- a/tests/pkg/chaos/redis.go
+++ b/tests/pkg/chaos/redis.go
@@ -2,8 +2,11 @@ package chaos
 
 import (
 	"os/exec"
+	"strings"
 	"testing"
 	"time"
+
+	"chatnow-tests/pkg/client"
 )
 
 var redisServices = []string{
@@ -11,32 +14,46 @@ var redisServices = []string{
 	"redis-node4", "redis-node5", "redis-node6",
 }
 
-func compose(t testing.TB, args ...string) {
+func compose(t testing.TB, cfg *client.Config, args ...string) {
 	t.Helper()
 	cmd := exec.Command("docker", append([]string{"compose"}, args...)...)
-	cmd.Dir = ".."
+	cmd.Dir = cfg.Infra.ComposeDir
 	if out, err := cmd.CombinedOutput(); err != nil {
 		t.Fatalf("docker compose %v: %v: %s", args, err, out)
 	}
 }
 
-func StopRedisCluster(t testing.TB) {
-	compose(t, append([]string{"stop"}, redisServices...)...)
+func StopRedisCluster(t testing.TB, cfg *client.Config) {
+	compose(t, cfg, append([]string{"stop"}, redisServices...)...)
 }
 
-func StartRedisCluster(t testing.TB) {
-	compose(t, append([]string{"start"}, redisServices...)...)
+func StartRedisCluster(t testing.TB, cfg *client.Config) {
+	compose(t, cfg, append([]string{"start"}, redisServices...)...)
 }
 
-func WaitRedisCluster(t testing.TB, timeout time.Duration) {
+func WaitRedisCluster(t testing.TB, cfg *client.Config, timeout time.Duration) {
 	t.Helper()
 	deadline := time.Now().Add(timeout)
 	for time.Now().Before(deadline) {
-		cmd := exec.Command("docker", "exec", "redis-node1", "redis-cli", "cluster", "info")
-		if out, err := cmd.Output(); err == nil && string(out) != "" {
+		cmd := exec.Command("docker", "exec", cfg.Infra.RedisContainer, "redis-cli", "cluster", "info")
+		if out, err := cmd.Output(); err == nil && clusterHealthy(string(out)) {
 			return
 		}
 		time.Sleep(500 * time.Millisecond)
 	}
 	t.Fatalf("redis cluster did not recover within %s", timeout)
 }
+
+func clusterHealthy(info string) bool {
+	values := make(map[string]string)
+	for _, line := range strings.Split(strings.ReplaceAll(info, "\r", ""), "\n") {
+		parts := strings.SplitN(line, ":", 2)
+		if len(parts) == 2 {
+			values[strings.TrimSpace(parts[0])] = strings.TrimSpace(parts[1])
+		}
+	}
+	return values["cluster_state"] == "ok" &&
+		values["cluster_slots_assigned"] == "16384" &&
+		values["cluster_slots_ok"] == "16384" &&
+		values["cluster_slots_fail"] == "0"
+}
diff --git a/tests/pkg/chaos/redis_test.go b/tests/pkg/chaos/redis_test.go
new file mode 100644
index 0000000..709b432
--- /dev/null
+++ b/tests/pkg/chaos/redis_test.go
@@ -0,0 +1,19 @@
+package chaos
+
+import "testing"
+
+func TestClusterHealthyRequiresCompleteSlotCoverage(t *testing.T) {
+	healthy := "cluster_state:ok\r\ncluster_slots_assigned:16384\r\ncluster_slots_ok:16384\r\ncluster_slots_fail:0\r\n"
+	if !clusterHealthy(healthy) {
+		t.Fatal("complete healthy cluster rejected")
+	}
+	for _, unhealthy := range []string{
+		"cluster_state:fail\ncluster_slots_assigned:16384\ncluster_slots_ok:16384\ncluster_slots_fail:0\n",
+		"cluster_state:ok\ncluster_slots_assigned:12000\ncluster_slots_ok:12000\ncluster_slots_fail:0\n",
+		"cluster_state:ok\ncluster_slots_assigned:16384\ncluster_slots_ok:16000\ncluster_slots_fail:384\n",
+	} {
+		if clusterHealthy(unhealthy) {
+			t.Fatalf("unhealthy cluster accepted: %q", unhealthy)
+		}
+	}
+}
diff --git a/tests/reliability/redis_failover_test.go b/tests/reliability/redis_failover_test.go
index 56b0974..8d79897 100644
--- a/tests/reliability/redis_failover_test.go
+++ b/tests/reliability/redis_failover_test.go
@@ -4,6 +4,7 @@ package reliability_test
 
 import (
 	"fmt"
+	"strings"
 	"testing"
 	"time"
 
@@ -12,6 +13,7 @@ import (
 	"chatnow-tests/pkg/chaos"
 	"chatnow-tests/pkg/client"
 	"chatnow-tests/pkg/fixture"
+	"chatnow-tests/pkg/verify"
 	identity "chatnow-tests/proto/chatnow/identity"
 	msg "chatnow-tests/proto/chatnow/message"
 	transmite "chatnow-tests/proto/chatnow/transmite"
@@ -33,11 +35,19 @@ func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) {
 		ClientMsgId: client.NewRequestID(),
 	}, prewarmRsp))
 	require.True(t, prewarmRsp.GetHeader().GetSuccess())
+	endpoints := reliabilityTransmiteEndpoints(t)
+	openedBefore := reliabilitySumBVar(t, endpoints, "redis_circuit_open_total")
+	rejectedBefore := reliabilitySumBVar(t, endpoints, "redis_circuit_rejected_total")
+	recoveredBefore := reliabilitySumBVar(t, endpoints, "redis_circuit_recovered_total")
 
-	t.Cleanup(func() { chaos.StartRedisCluster(t); chaos.WaitRedisCluster(t, 60*time.Second) })
-	chaos.StopRedisCluster(t)
+	t.Cleanup(func() {
+		chaos.StartRedisCluster(t, HTTP.Config())
+		chaos.WaitRedisCluster(t, HTTP.Config(), 60*time.Second)
+	})
+	chaos.StopRedisCluster(t, HTTP.Config())
 
 	rateLimited := 0
+	seqUnavailable := 0
 	for i := 0; i < 650; i++ {
 		sendRsp := &transmite.SendMessageRsp{}
 		err := user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
@@ -53,8 +63,14 @@ func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) {
 		if sendRsp.GetHeader().GetErrorMessage() == "rate_limited" {
 			rateLimited++
 		}
+		if sendRsp.GetHeader().GetErrorMessage() == "序号生成失败" {
+			seqUnavailable++
+		}
 	}
 	require.Greater(t, rateLimited, 0, "Redis outage must retain bounded rate limiting")
+	require.Greater(t, seqUnavailable, 0, "SeqGen truth source must fail unavailable")
+	require.Greater(t, reliabilitySumBVar(t, endpoints, "redis_circuit_open_total"), openedBefore)
+	require.Greater(t, reliabilitySumBVar(t, endpoints, "redis_circuit_rejected_total"), rejectedBefore)
 
 	uid := user.UserID
 	for i := 0; i < 3; i++ {
@@ -65,20 +81,55 @@ func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) {
 	}
 
 	started := time.Now()
-	rsp := &identity.GetProfileRsp{}
-	err := user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{
-		RequestId: client.NewRequestID(), UserId: &uid,
-	}, rsp)
+	fastFail := &transmite.SendMessageRsp{}
+	err := user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
+		RequestId: client.NewRequestID(), ConversationId: convID,
+		Content: &msg.MessageContent{Type: msg.MessageType_TEXT,
+			Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: "fast-fail"}}},
+		ClientMsgId: client.NewRequestID(),
+	}, fastFail)
 	require.NoError(t, err)
-	require.True(t, rsp.GetHeader().GetSuccess())
+	require.False(t, fastFail.GetHeader().GetSuccess())
 	require.Less(t, time.Since(started), 50*time.Millisecond)
 
-	chaos.StartRedisCluster(t)
-	chaos.WaitRedisCluster(t, 60*time.Second)
+	chaos.StartRedisCluster(t, HTTP.Config())
+	chaos.WaitRedisCluster(t, HTTP.Config(), 60*time.Second)
 	time.Sleep(1100 * time.Millisecond)
-	rsp = &identity.GetProfileRsp{}
+	rsp := &identity.GetProfileRsp{}
 	require.NoError(t, user.DoAuth("/service/identity/get_profile", &identity.GetProfileReq{
 		RequestId: client.NewRequestID(), UserId: &uid,
 	}, rsp))
 	require.True(t, rsp.GetHeader().GetSuccess())
+
+	recoveredSend := &transmite.SendMessageRsp{}
+	require.NoError(t, user.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
+		RequestId: client.NewRequestID(), ConversationId: convID,
+		Content: &msg.MessageContent{Type: msg.MessageType_TEXT,
+			Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: "recovered"}}},
+		ClientMsgId: client.NewRequestID(),
+	}, recoveredSend))
+	require.True(t, recoveredSend.GetHeader().GetSuccess(), recoveredSend.GetHeader().GetErrorMessage())
+	require.Greater(t, reliabilitySumBVar(t, endpoints, "redis_circuit_recovered_total"), recoveredBefore,
+		"an Open->HalfOpen probe must recover the Transmite Redis circuit")
+}
+
+func reliabilityTransmiteEndpoints(t testing.TB) []string {
+	t.Helper()
+	var endpoints []string
+	for _, raw := range strings.Split(HTTP.Config().Infra.TransmiteVars, ",") {
+		if endpoint := strings.TrimRight(strings.TrimSpace(raw), "/"); endpoint != "" {
+			endpoints = append(endpoints, endpoint)
+		}
+	}
+	require.NotEmpty(t, endpoints)
+	return endpoints
+}
+
+func reliabilitySumBVar(t testing.TB, endpoints []string, metric string) int64 {
+	t.Helper()
+	var total int64
+	for _, endpoint := range endpoints {
+		total += verify.BVar(t, endpoint, metric)
+	}
+	return total
 }
diff --git a/transmite/source/transmite_server.h b/transmite/source/transmite_server.h
index d1a4f70..2d04e1d 100644
--- a/transmite/source/transmite_server.h
+++ b/transmite/source/transmite_server.h
@@ -594,11 +594,21 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
                 }
             }
 
-            auto info = fetch_user_info_from_identity_(uid, rid, caller);
-            if (!info) return std::nullopt;
+            const auto generation = _user_info_cache
+                ? _user_info_cache->generation(uid) : std::optional<uint64_t>{};
+            bool confirmed_not_found = false;
+            auto info = fetch_user_info_from_identity_(uid, rid, caller, &confirmed_not_found);
+            if (!info) {
+                if (confirmed_not_found && generation && _user_info_cache) {
+                    _user_info_cache->set_if_generation(uid, "", *generation);
+                }
+                return std::nullopt;
+            }
             auto bytes = info->SerializeAsString();
-            if (_user_info_cache) _user_info_cache->set(uid, bytes);
-            if (_local_user_cache) {
+            if (generation && _user_info_cache) {
+                _user_info_cache->set_if_generation(uid, bytes, *generation);
+            }
+            if (generation && _local_user_cache) {
                 _local_user_cache->set(
                     lkey, bytes, randomized_ttl(std::chrono::seconds(45)));
             }
@@ -612,7 +622,8 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
     }
 
     std::optional<chatnow::common::UserInfo> fetch_user_info_from_identity_(
-        const std::string &uid, const std::string &rid, brpc::Controller *caller) {
+        const std::string &uid, const std::string &rid, brpc::Controller *caller,
+        bool *confirmed_not_found) {
         auto identity_channel = _mm_channels->choose(_identity_service_name);
         if (!identity_channel) {
             LOG_ERROR("identity_service 节点缺失 (user info {})", uid);
@@ -628,10 +639,9 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
         stub.GetProfile(&cntl, &req, &rsp, nullptr);
         metrics::g_user_info_rpc_total << 1;
         if (cntl.Failed() || !rsp.header().success()) {
-            if (!cntl.Failed() &&
-                rsp.header().error_code() == chatnow::error::kAuthUserNotFound &&
-                _user_info_cache) {
-                _user_info_cache->set(uid, "");
+            if (confirmed_not_found && !cntl.Failed() &&
+                rsp.header().error_code() == chatnow::error::kAuthUserNotFound) {
+                *confirmed_not_found = true;
             }
             LOG_ERROR("获取用户信息失败 uid={}: {} {}", uid, cntl.ErrorText(),
                       rsp.header().error_message());

From 4a2244a2be54df56ee0d9d2b5084a182eabc53a4 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 14:15:41 +0800
Subject: [PATCH 17/47] fix: preserve cache availability under races

---
 common/dao/data_redis.hpp                     |  79 +++++----
 ...7-15-cache-resilience-completion-design.md |  16 +-
 identity/source/identity_server.h             |  14 +-
 tests/config.yaml                             |   1 +
 tests/func/cache_test.go                      | 166 ++----------------
 tests/pkg/client/config.go                    |   4 +
 tests/pkg/client/websocket.go                 | 143 +++++++++++++++
 tests/reliability/redis_failover_test.go      |  92 ++++++++++
 transmite/source/transmite_server.h           |   5 +-
 9 files changed, 317 insertions(+), 203 deletions(-)
 create mode 100644 tests/pkg/client/websocket.go

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 96c43fa..80ecd2f 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -164,7 +164,7 @@ class RedisPipeline {
     void record_transition_(RedisCircuitBreaker::Transition transition) noexcept {
         if (transition == RedisCircuitBreaker::Transition::Opened) {
             metrics::g_redis_circuit_open_total << 1;
-            LOG_WARN("Redis circuit transition Closed->Open");
+            LOG_WARN("Redis circuit transition ->Open");
         } else if (transition == RedisCircuitBreaker::Transition::Recovered) {
             metrics::g_redis_circuit_recovered_total << 1;
             LOG_INFO("Redis circuit transition HalfOpen->Closed");
@@ -398,7 +398,7 @@ class RedisClient
     static void record_transition_(RedisCircuitBreaker::Transition transition) noexcept {
         if (transition == RedisCircuitBreaker::Transition::Opened) {
             metrics::g_redis_circuit_open_total << 1;
-            LOG_WARN("Redis circuit transition Closed->Open");
+            LOG_WARN("Redis circuit transition ->Open");
         } else if (transition == RedisCircuitBreaker::Transition::Recovered) {
             metrics::g_redis_circuit_recovered_total << 1;
             LOG_INFO("Redis circuit transition HalfOpen->Closed");
@@ -1147,8 +1147,9 @@ class UserInfoCache
             auto value = _c->get(key::user_info_key(uid));
             if (!value) return std::nullopt;
             return *value;
-        } catch (const std::exception &e) {
-            LOG_ERROR("UserInfoCache.get 失败 {}: {}", uid, e.what());
+        } catch (const RedisCircuitOpen &) {
+            return std::nullopt;
+        } catch (const std::exception &) {
             return std::nullopt;
         }
     }
@@ -1165,7 +1166,8 @@ class UserInfoCache
             groups[key::user_info_bucket(uids[i])].push_back(
                 {uids[i], key::user_info_key(uids[i])});
         }
-        for (const auto &[bucket, entries] : groups) {
+        for (const auto &group : groups) {
+            const auto &entries = group.second;
             std::vector<std::string> keys;
             keys.reserve(entries.size());
             for (const auto &entry : entries) keys.push_back(entry.second);
@@ -1177,8 +1179,9 @@ class UserInfoCache
                     if (i < values.size() && values[i]) result.hits[entries[i].first] = *values[i];
                     else result.misses.push_back(entries[i].first);
                 }
-            } catch (const std::exception &e) {
-                LOG_ERROR("UserInfoCache.batch_get bucket={} 失败: {}", bucket, e.what());
+            } catch (const RedisCircuitOpen &) {
+                for (const auto &entry : entries) result.misses.push_back(entry.first);
+            } catch (const std::exception &) {
                 for (const auto &entry : entries) result.misses.push_back(entry.first);
             }
         }
@@ -1187,14 +1190,8 @@ class UserInfoCache
 
     void set(const std::string &uid, const std::string &serialized) {
         if (!_c) return;
-        const auto ttl = serialized.empty()
-            ? randomized_ttl(std::chrono::seconds(5))
-            : randomized_ttl(kUserInfoTtl);
-        try {
-            _c->set(key::user_info_key(uid), serialized, ttl);
-        } catch (const std::exception &e) {
-            LOG_ERROR("UserInfoCache.set 失败 {}: {}", uid, e.what());
-        }
+        const auto observed = generation(uid);
+        if (observed) (void)set_if_generation(uid, serialized, *observed);
     }
 
     std::optional<uint64_t> generation(const std::string &uid) {
@@ -1202,8 +1199,9 @@ class UserInfoCache
         try {
             auto value = _c->get(key::user_info_generation_key(uid));
             return value ? std::stoull(*value) : uint64_t{0};
-        } catch (const std::exception &e) {
-            LOG_WARN("UserInfoCache.generation unavailable: {}", e.what());
+        } catch (const RedisCircuitOpen &) {
+            return std::nullopt;
+        } catch (const std::exception &) {
             return std::nullopt;
         }
     }
@@ -1222,33 +1220,19 @@ class UserInfoCache
                 std::to_string(ttl.count())};
             return _c->eval<long long>(kSetIfGenerationLua, keys.begin(), keys.end(),
                                        args.begin(), args.end()) == 1;
-        } catch (const std::exception &e) {
-            LOG_WARN("UserInfoCache fenced fill unavailable: {}", e.what());
+        } catch (const RedisCircuitOpen &) {
+            return false;
+        } catch (const std::exception &) {
             return false;
         }
     }
 
     void batch_set(const std::unordered_map<std::string, std::string> &values) {
         if (!_c) return;
-        std::unordered_map<uint32_t, std::vector<std::pair<std::string, std::string>>> groups;
         size_t count = 0;
         for (const auto &entry : values) {
             if (count++ >= 2000) break;
-            groups[key::user_info_bucket(entry.first)].push_back(entry);
-        }
-        for (const auto &[bucket, entries] : groups) {
-            try {
-                auto pipe = _c->pipeline(std::to_string(bucket));
-                for (const auto &[uid, serialized] : entries) {
-                    const auto ttl = serialized.empty()
-                        ? randomized_ttl(std::chrono::seconds(5))
-                        : randomized_ttl(kUserInfoTtl);
-                    pipe.set(key::user_info_key(uid), serialized, ttl);
-                }
-                pipe.exec();
-            } catch (const std::exception &e) {
-                LOG_ERROR("UserInfoCache.batch_set bucket={} 失败: {}", bucket, e.what());
-            }
+            set(entry.first, entry.second);
         }
     }
 
@@ -1261,8 +1245,10 @@ class UserInfoCache
             _c->eval<long long>(kInvalidateLua, keys.begin(), keys.end(),
                                 args.begin(), args.end());
             return true;
-        } catch (const std::exception &e) {
-            LOG_ERROR("UserInfoCache.invalidate 失败 {}: {}", uid, e.what());
+        } catch (const RedisCircuitOpen &) {
+            metrics::g_user_info_invalidation_failure_total << 1;
+            return false;
+        } catch (const std::exception &) {
             metrics::g_user_info_invalidation_failure_total << 1;
             return false;
         }
@@ -1327,10 +1313,24 @@ class OnlineRoute
                                 args.begin(), args.end());
         }
         catch(std::exception &e) { LOG_ERROR("OnlineRoute.touch 失败 {}: {}", uid, e.what()); }
+        try { _c->expire(key::legacy_device_set_key(uid), kLegacyDeviceGraceTtl); }
+        catch (const std::exception &) {}
     }
     /* brief: 设备下线 — HDEL uid did */
     void unbind(const std::string &uid, const std::string &device_id,
                 const std::string &push_instance) {
+        std::vector<std::string> legacy_devices;
+        if (device_id.empty()) {
+            try {
+                std::unordered_map<std::string, std::string> routes;
+                _c->hgetall(key::online_key(uid), std::inserter(routes, routes.end()));
+                for (const auto &[did, instance] : routes) {
+                    if (instance == push_instance) legacy_devices.push_back(did);
+                }
+            } catch (const std::exception &) {}
+        } else {
+            legacy_devices.push_back(device_id);
+        }
         try {
             std::vector<std::string> keys = {
                 key::online_key(uid), key::device_set_key(uid)};
@@ -1339,6 +1339,10 @@ class OnlineRoute
                                 args.begin(), args.end());
         }
         catch(std::exception &e) { LOG_ERROR("OnlineRoute.unbind 失败 {}-{}-{}: {}", uid, device_id, push_instance, e.what()); }
+        for (const auto &did : legacy_devices) {
+            try { _c->srem(key::legacy_device_set_key(uid), did); }
+            catch (const std::exception &) {}
+        }
     }
     /* brief: 取用户所有在线设备 → device_id 列表 */
     std::vector<std::string> devices(const std::string &uid) {
@@ -1382,6 +1386,7 @@ class OnlineRoute
         catch(std::exception &e) { LOG_ERROR("OnlineRoute.online 失败 {}: {}", uid, e.what()); return false; }
     }
 private:
+    static constexpr std::chrono::seconds kLegacyDeviceGraceTtl{24 * 3600};
     static constexpr const char *kBindLua = R"lua(
 local function key_type(k)
     local t = redis.call('TYPE', k)
diff --git a/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
index 71e2018..6d3c186 100644
--- a/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
+++ b/docs/superpowers/specs/2026-07-15-cache-resilience-completion-design.md
@@ -171,7 +171,8 @@ Transmite 当前热路径只读取发送者一个 uid，不额外引入没有消
 `UserInfoCache` 提供 `batch_get` 和 `batch_set`，满足 Message、Conversation 后续
 批量读取场景：
 
-- standalone Redis 使用 MGET 和 pipeline。
+- 读取使用 MGET；写入逐 uid 读取 generation 后以同槽 Lua CAS 回填，避免公开
+  `set`/`batch_set` 绕过失效 fence。这里不额外引入复杂的批量 Lua 协议。
 - Redis Cluster 按 64 个虚拟 bucket 分组；每组 key 共享 hash tag，可安全使用
   MGET/pipeline，避免 CROSSSLOT，也不依赖 redis-plus-plus 的内部连接池。
 - 返回命中 map 和 miss uid 列表，调用方可用现有批量 RPC 一次回源。
@@ -181,9 +182,13 @@ Transmite 当前热路径只读取发送者一个 uid，不额外引入没有消
 
 ### 5.4 失效
 
-Identity 修改昵称、头像、签名或其他 Profile 字段成功后删除 L2 key。删除失败只
-记录指标。各实例 L1 可能继续返回最多约 54 秒旧资料，符合资料展示的最终一致性
-要求。
+Identity 在 DB 提交前 best-effort 推进 generation 并删除 L2，DB 成功后再次执行
+同一原子失效。夹在 pre-invalidate 与 commit 之间读取旧 DB 的回填会被 post
+generation 删除或拒绝；任一 Redis 失效都只记录指标，不拒绝或回滚已经请求的资料
+更新。若 Redis 在整个更新期间都不可用，只能依赖 L1 最多约 54 秒、L2 一小时 TTL、
+告警和恢复后的后续失效收敛，这是 cache-aside 可用性优先的明确边界，不引入 DB
+outbox。Transmite 无法读取 generation 时仍把成功 RPC 结果放入短 TTL L1，供本机
+singleflight followers 共享，但不写 L2；not-found 仍必须拿到 generation 才写 sentinel。
 
 ## 6. TTL 抖动补全
 
@@ -283,6 +288,9 @@ UnackedPush 等可达路径继续由新框架做黑盒验证。待业务重新
 4. 验证突发请求出现本地限流拒绝。
 5. 验证依赖 SeqGen 的路径明确返回不可用。
 6. 恢复 Redis，验证 half-open 成功并自动恢复正常路径。
+7. Push 专项通过真实 WS、Rabbit 和容器 pause 编排验证：Redis 故障时 Unacked 写入
+   失败必须 NackRequeue 且不得先向 WS 投递；恢复后重投、持久化并按 at-least-once
+   语义最终送达。
 
 ### 10.3 L4 性能测试
 
diff --git a/identity/source/identity_server.h b/identity/source/identity_server.h
index bb336b2..43ccb85 100644
--- a/identity/source/identity_server.h
+++ b/identity/source/identity_server.h
@@ -476,17 +476,17 @@ class IdentityServiceImpl : public ::chatnow::identity::IdentityService
             if (request->has_phone()) {
                 user->phone(request->phone());
             }
-            // Advance the shared cache generation before committing the profile.
-            // If Redis is unavailable, fail closed so a successful update can
-            // never leave a one-hour stale L2 value visible to readers.
-            if (_user_info_cache && !_user_info_cache->invalidate(auth.user_id)) {
-                throw ServiceError(::chatnow::error::kSystemUnavailable,
-                                   "profile cache invalidation unavailable");
-            }
+            // Pre-invalidate fences a reader that races before the DB commit.
+            // It is best-effort: profile writes remain available during a Redis outage.
+            if (_user_info_cache) (void)_user_info_cache->invalidate(auth.user_id);
             if (!_mysql_user->update(user)) {
                 throw ServiceError(::chatnow::error::kSystemInternalError,
                                    "db update failed");
             }
+            // A reader between pre-invalidate and commit may have cached the old DB
+            // row under the new generation. Advance once more after commit so that
+            // fill is rejected/deleted without rolling back an already durable DB write.
+            if (_user_info_cache) (void)_user_info_cache->invalidate(auth.user_id);
             std::string ob_payload = outbox_payload_upsert_(
                 user->user_id(), user->mail(), user->phone(),
                 user->nickname(), user->description(),
diff --git a/tests/config.yaml b/tests/config.yaml
index 97d3b8b..69c75c2 100644
--- a/tests/config.yaml
+++ b/tests/config.yaml
@@ -22,5 +22,6 @@ log:
 
 infra:
   redis_container: "redis-node1"
+  push_container: "push_server-service"
   compose_dir: ".."
   transmite_vars: "http://localhost:10004"
diff --git a/tests/func/cache_test.go b/tests/func/cache_test.go
index 6a945c5..2ae5ab1 100644
--- a/tests/func/cache_test.go
+++ b/tests/func/cache_test.go
@@ -3,19 +3,11 @@
 package func_test
 
 import (
-	"bufio"
 	"bytes"
-	"crypto/rand"
-	"encoding/base64"
-	"encoding/binary"
 	"fmt"
-	"net"
-	"net/http"
-	"net/url"
 	"os"
 	"os/exec"
 	"path/filepath"
-	"regexp"
 	"runtime"
 	"strconv"
 	"strings"
@@ -86,67 +78,18 @@ func sendCacheTestMessage(t testing.TB, user *client.HTTPClient, convID, suffix
 	require.NoError(t, sendCacheTestMessageResult(user, convID, suffix))
 }
 
-type cacheTestWebSocket struct {
-	conn net.Conn
-}
+type cacheTestWebSocket = client.WebSocket
 
 func openCacheTestWebSocket(t testing.TB) *cacheTestWebSocket {
 	t.Helper()
-	addr := HTTP.Config().Target.WebsocketAddr
-	conn, err := net.DialTimeout("tcp", addr, 5*time.Second)
-	require.NoError(t, err)
-	require.NoError(t, conn.SetDeadline(time.Now().Add(5*time.Second)))
-
-	keyBytes := make([]byte, 16)
-	_, err = rand.Read(keyBytes)
-	require.NoError(t, err)
-	key := base64.StdEncoding.EncodeToString(keyBytes)
-	req := &http.Request{
-		Method: "GET",
-		URL:    &url.URL{Scheme: "http", Host: addr, Path: "/"},
-		Host:   addr,
-		Header: http.Header{
-			"Connection":            {"Upgrade"},
-			"Upgrade":               {"websocket"},
-			"Sec-Websocket-Key":     {key},
-			"Sec-Websocket-Version": {"13"},
-		},
-	}
-	require.NoError(t, req.Write(conn))
-	rsp, err := http.ReadResponse(bufio.NewReader(conn), req)
+	ws, err := client.OpenWebSocket(HTTP.Config())
 	require.NoError(t, err)
-	require.Equal(t, http.StatusSwitchingProtocols, rsp.StatusCode)
-	require.NoError(t, conn.SetDeadline(time.Time{}))
-	return &cacheTestWebSocket{conn: conn}
-}
-
-func (ws *cacheTestWebSocket) close() {
-	_ = ws.conn.Close()
+	return ws
 }
 
-func (ws *cacheTestWebSocket) writeBinary(t testing.TB, payload []byte) {
+func writeCacheTestBinary(t testing.TB, ws *cacheTestWebSocket, payload []byte) {
 	t.Helper()
-	frame := []byte{0x82}
-	switch {
-	case len(payload) < 126:
-		frame = append(frame, 0x80|byte(len(payload)))
-	case len(payload) <= 65535:
-		frame = append(frame, 0x80|126, byte(len(payload)>>8), byte(len(payload)))
-	default:
-		frame = append(frame, 0x80|127)
-		var size [8]byte
-		binary.BigEndian.PutUint64(size[:], uint64(len(payload)))
-		frame = append(frame, size[:]...)
-	}
-	var mask [4]byte
-	_, err := rand.Read(mask[:])
-	require.NoError(t, err)
-	frame = append(frame, mask[:]...)
-	for i, b := range payload {
-		frame = append(frame, b^mask[i%len(mask)])
-	}
-	_, err = ws.conn.Write(frame)
-	require.NoError(t, err)
+	require.NoError(t, ws.WriteBinary(payload))
 }
 
 func appendProtoString(dst []byte, field protowire.Number, value string) []byte {
@@ -155,14 +98,7 @@ func appendProtoString(dst []byte, field protowire.Number, value string) []byte
 }
 
 func cacheTestAuthNotify(accessToken, deviceID string) []byte {
-	var auth []byte
-	auth = appendProtoString(auth, 1, accessToken)
-	auth = appendProtoString(auth, 2, deviceID)
-	var notify []byte
-	notify = protowire.AppendTag(notify, 2, protowire.VarintType)
-	notify = protowire.AppendVarint(notify, 49) // CLIENT_AUTH
-	notify = protowire.AppendTag(notify, 10, protowire.BytesType)
-	return protowire.AppendBytes(notify, auth)
+	return client.PushAuthNotify(accessToken, deviceID)
 }
 
 func cacheTestHeartbeatNotify(uid string) []byte {
@@ -223,10 +159,6 @@ func checkCacheTTLSourceContracts(source string) error {
 	markers := [][3]string{
 		{"Session", "class Session", "class Status"},
 		{"Status", "class Status", "class Codes"},
-		{"Codes", "class Codes", "class Seq"},
-		{"DeviceSet", "class DeviceSet", "class ReadAck"},
-		{"OnlineRoute", "class OnlineRoute", "class RateLimiter"},
-		{"UnackedPush", "class UnackedPush", "class PresenceRedis"},
 	}
 	for _, marker := range markers {
 		section, err := normalizedSourceSection(source, marker[1], marker[2])
@@ -248,35 +180,6 @@ func checkCacheTTLSourceContracts(source string) error {
 			"_c->set(key::kStatus + uid, \"1\", randomized_ttl(ttl))",
 			"_c->expire(key::kStatus + uid, randomized_ttl(ttl))",
 		}},
-		{"Codes", []string{
-			"_c->set(key::kVerifyCode + cid, code, randomized_ttl(ttl))",
-		}},
-		{"DeviceSet", []string{
-			"randomized_ttl(kSessionTtl)",
-			"key::device_set_key(uid)",
-			"_c->eval<long long>(kAddLua",
-			"redis.call('SADD', KEYS[1], ARGV[1])",
-			"redis.call('EXPIRE', KEYS[1], ttl)",
-			"key::legacy_device_set_key(uid)",
-			"kLegacyGraceTtl",
-		}},
-		{"OnlineRoute", []string{
-			"key::online_key(uid), key::device_set_key(uid)",
-			"_c->eval<long long>(kBindLua",
-			"_c->eval<long long>(kTouchLua",
-			"_c->eval<long long>(kUnbindLua",
-			"redis.call('EXPIRE', KEYS[1], route_ttl)",
-			"redis.call('EXPIRE', KEYS[2], device_ttl)",
-		}},
-		{"UnackedPush", []string{
-			"_c->eval<long long>(kPushLua",
-			"_c->eval<long long>(kBumpScoreLua",
-			"_c->eval<long long>(kAckLua",
-			"redis.call('ZADD', KEYS[1]",
-			"redis.call('HSET', KEYS[2]",
-			"redis.call('EXPIRE', KEYS[1], ttl)",
-			"redis.call('EXPIRE', KEYS[2], ttl)",
-		}},
 	}
 	for _, rule := range rules {
 		if err := requireSourceFragments(rule.name, sections[rule.name], rule.fragments...); err != nil {
@@ -284,52 +187,15 @@ func checkCacheTTLSourceContracts(source string) error {
 		}
 	}
 
-	fixedTTL := regexp.MustCompile(`_c->(?:set|expire)\([^;]*, ttl\)`)
-	for _, name := range []string{"Session", "Status", "Codes"} {
-		if fixedTTL.MatchString(sections[name]) {
-			return fmt.Errorf("%s contains a direct fixed TTL cache write", name)
-		}
-	}
-	if strings.Count(sections["UnackedPush"], "randomized_ttl(ttl)") != 2 {
-		return fmt.Errorf("UnackedPush push and bump_score must each sample one randomized TTL")
-	}
-	if strings.Contains(sections["UnackedPush"], "_c->zadd(") ||
-		strings.Contains(sections["UnackedPush"], "_c->hset(") {
-		return fmt.Errorf("UnackedPush bypasses its atomic scripts")
-	}
 	return nil
 }
 
-func TestFN_CA_ResilienceSourceContracts(t *testing.T) {
-	dao := readRepoSource(t, "common/dao/data_redis.hpp")
-	transmiteSource := readRepoSource(t, "transmite/source/transmite_server.h")
-	pushSource := readRepoSource(t, "push/source/push_server.h")
-	breaker := readRepoSource(t, "common/utils/redis_circuit_breaker.hpp")
-	for _, contract := range []struct {
-		name   string
-		source string
-		want   []string
-	}{
-		{"generation fence", dao, []string{"set_if_generation", "kSetIfGenerationLua", "redis.call('INCR', KEYS[2])"}},
-		{"fenced fill", transmiteSource, []string{"generation(uid)", "set_if_generation(uid, bytes", "set_if_generation(uid, \"\""}},
-		{"truth source push", pushSource, []string{"Persist before delivery", "ConsumeAction::NackRequeue", "unacked persistence unavailable"}},
-		{"atomic ack", dao, []string{"kAckLua", "redis.call('ZREM', KEYS[1]", "redis.call('HDEL', KEYS[2]"}},
-		{"atomic breaker hot path", breaker, []string{"std::atomic<uint64_t> _generation", "std::atomic<uint32_t> _consecutive_failures"}},
-	} {
-		for _, want := range contract.want {
-			if !strings.Contains(contract.source, want) {
-				t.Errorf("%s missing %q", contract.name, want)
-			}
-		}
-	}
-}
-
 func requireCacheTTLSourceContracts(t testing.TB, source string) {
 	t.Helper()
 	require.NoError(t, checkCacheTTLSourceContracts(source))
 }
 
-func TestFN_CA_TTLJitterSourceContract(t *testing.T) {
+func TestFN_CA_LegacyUnusedTTLSourceContract(t *testing.T) {
 	daoSource := readRepoSource(t, "common/dao/data_redis.hpp")
 	requireCacheTTLSourceContracts(t, daoSource)
 
@@ -337,14 +203,6 @@ func TestFN_CA_TTLJitterSourceContract(t *testing.T) {
 		mutant := strings.Replace(daoSource, "randomized_ttl(ttl)", "ttl", 1)
 		require.Error(t, checkCacheTTLSourceContracts(mutant))
 	})
-	t.Run("rejects non-atomic DeviceSet add", func(t *testing.T) {
-		mutant := strings.Replace(daoSource, "_c->eval<long long>(kAddLua", "_c->sadd", 1)
-		require.Error(t, checkCacheTTLSourceContracts(mutant))
-	})
-	t.Run("rejects non-atomic UnackedPush write", func(t *testing.T) {
-		mutant := strings.Replace(daoSource, "_c->eval<long long>(kPushLua", "_c->zadd", 1)
-		require.Error(t, checkCacheTTLSourceContracts(mutant))
-	})
 }
 
 type redisResultPredicate func(string) (bool, error)
@@ -391,7 +249,7 @@ func TestFN_CA_TTLJitter(t *testing.T) {
 	connections := make([]*cacheTestWebSocket, 0, sampleCount)
 	t.Cleanup(func() {
 		for _, ws := range connections {
-			ws.close()
+			_ = ws.Close()
 		}
 	})
 
@@ -420,7 +278,7 @@ func TestFN_CA_TTLJitter(t *testing.T) {
 			user, _, _ := fixture.RegisterAndLogin(t, HTTP)
 			ws := openCacheTestWebSocket(t)
 			connections = append(connections, ws)
-			ws.writeBinary(t, cacheTestAuthNotify(user.AccessToken, "default_device"))
+			writeCacheTestBinary(t, ws, cacheTestAuthNotify(user.AccessToken, "default_device"))
 
 			deviceKey := fmt.Sprintf("im:dev:{%s}", user.UserID)
 			requireEventuallyRedis(t, 5*time.Second, 50*time.Millisecond,
@@ -428,14 +286,14 @@ func TestFN_CA_TTLJitter(t *testing.T) {
 			if i == 0 {
 				verify.RedisCLI(t, "EXPIRE", deviceKey, "60")
 				heartbeatBefore = verify.RedisTTL(t, deviceKey)
-				ws.writeBinary(t, cacheTestHeartbeatNotify(user.UserID))
+				writeCacheTestBinary(t, ws, cacheTestHeartbeatNotify(user.UserID))
 				requireEventuallyRedis(t, 5*time.Second, 50*time.Millisecond,
 					"heartbeat must renew DeviceSet TTL", func(result string) (bool, error) {
 						seconds, err := strconv.ParseInt(result, 10, 64)
 						return time.Duration(seconds)*time.Second > heartbeatBefore+24*time.Hour, err
 					}, "TTL", deviceKey)
 			} else {
-				ws.writeBinary(t, cacheTestHeartbeatNotify(user.UserID))
+				writeCacheTestBinary(t, ws, cacheTestHeartbeatNotify(user.UserID))
 			}
 			deviceTTLs[requireJitteredRedisTTL(t, deviceKey, sessionTTL)] = struct{}{}
 		}
@@ -447,7 +305,7 @@ func TestFN_CA_TTLJitter(t *testing.T) {
 		sender, recipient, convID := fixture.MakeFriends(t, HTTP)
 		recipientWS := openCacheTestWebSocket(t)
 		connections = append(connections, recipientWS)
-		recipientWS.writeBinary(t, cacheTestAuthNotify(recipient.AccessToken, "default_device"))
+		writeCacheTestBinary(t, recipientWS, cacheTestAuthNotify(recipient.AccessToken, "default_device"))
 		deviceKey := fmt.Sprintf("im:dev:{%s}", recipient.UserID)
 		requireEventuallyRedis(t, 5*time.Second, 50*time.Millisecond,
 			"recipient DeviceSet key must exist", redisEquals("1"), "EXISTS", deviceKey)
diff --git a/tests/pkg/client/config.go b/tests/pkg/client/config.go
index 47af5a6..9a9f46f 100644
--- a/tests/pkg/client/config.go
+++ b/tests/pkg/client/config.go
@@ -36,6 +36,7 @@ type LogConfig struct {
 
 type InfraConfig struct {
 	RedisContainer string `yaml:"redis_container"`
+	PushContainer  string `yaml:"push_container"`
 	ComposeDir     string `yaml:"compose_dir"`
 	TransmiteVars  string `yaml:"transmite_vars"`
 }
@@ -68,6 +69,9 @@ func LoadConfig(path string) *Config {
 	if v := os.Getenv("REDIS_CONTAINER"); v != "" {
 		cfg.Infra.RedisContainer = v
 	}
+	if v := os.Getenv("PUSH_CONTAINER"); v != "" {
+		cfg.Infra.PushContainer = v
+	}
 	if v := os.Getenv("COMPOSE_DIR"); v != "" {
 		cfg.Infra.ComposeDir = v
 	}
diff --git a/tests/pkg/client/websocket.go b/tests/pkg/client/websocket.go
new file mode 100644
index 0000000..25a86d5
--- /dev/null
+++ b/tests/pkg/client/websocket.go
@@ -0,0 +1,143 @@
+package client
+
+import (
+	"bufio"
+	"crypto/rand"
+	"encoding/base64"
+	"encoding/binary"
+	"fmt"
+	"io"
+	"net"
+	"net/http"
+	"net/url"
+	"time"
+
+	"google.golang.org/protobuf/encoding/protowire"
+)
+
+type WebSocket struct {
+	conn net.Conn
+}
+
+func OpenWebSocket(cfg *Config) (*WebSocket, error) {
+	addr := cfg.Target.WebsocketAddr
+	conn, err := net.DialTimeout("tcp", addr, 5*time.Second)
+	if err != nil {
+		return nil, err
+	}
+	fail := func(err error) (*WebSocket, error) { _ = conn.Close(); return nil, err }
+	if err := conn.SetDeadline(time.Now().Add(5 * time.Second)); err != nil {
+		return fail(err)
+	}
+	keyBytes := make([]byte, 16)
+	if _, err := rand.Read(keyBytes); err != nil {
+		return fail(err)
+	}
+	req := &http.Request{
+		Method: "GET", URL: &url.URL{Scheme: "http", Host: addr, Path: "/"}, Host: addr,
+		Header: http.Header{"Connection": {"Upgrade"}, "Upgrade": {"websocket"},
+			"Sec-Websocket-Key":     {base64.StdEncoding.EncodeToString(keyBytes)},
+			"Sec-Websocket-Version": {"13"}},
+	}
+	if err := req.Write(conn); err != nil {
+		return fail(err)
+	}
+	rsp, err := http.ReadResponse(bufio.NewReader(conn), req)
+	if err != nil {
+		return fail(err)
+	}
+	if rsp.StatusCode != http.StatusSwitchingProtocols {
+		return fail(fmt.Errorf("websocket upgrade status %d", rsp.StatusCode))
+	}
+	if err := conn.SetDeadline(time.Time{}); err != nil {
+		return fail(err)
+	}
+	return &WebSocket{conn: conn}, nil
+}
+
+func (ws *WebSocket) Close() error { return ws.conn.Close() }
+
+func (ws *WebSocket) WriteBinary(payload []byte) error {
+	frame := []byte{0x82}
+	switch {
+	case len(payload) < 126:
+		frame = append(frame, 0x80|byte(len(payload)))
+	case len(payload) <= 65535:
+		frame = append(frame, 0x80|126, byte(len(payload)>>8), byte(len(payload)))
+	default:
+		frame = append(frame, 0x80|127)
+		var size [8]byte
+		binary.BigEndian.PutUint64(size[:], uint64(len(payload)))
+		frame = append(frame, size[:]...)
+	}
+	var mask [4]byte
+	if _, err := rand.Read(mask[:]); err != nil {
+		return err
+	}
+	frame = append(frame, mask[:]...)
+	for i, b := range payload {
+		frame = append(frame, b^mask[i%len(mask)])
+	}
+	_, err := ws.conn.Write(frame)
+	return err
+}
+
+func (ws *WebSocket) ReadFrame(timeout time.Duration) ([]byte, error) {
+	if err := ws.conn.SetReadDeadline(time.Now().Add(timeout)); err != nil {
+		return nil, err
+	}
+	var header [2]byte
+	if _, err := io.ReadFull(ws.conn, header[:]); err != nil {
+		return nil, err
+	}
+	length := uint64(header[1] & 0x7f)
+	switch length {
+	case 126:
+		var size [2]byte
+		if _, err := io.ReadFull(ws.conn, size[:]); err != nil {
+			return nil, err
+		}
+		length = uint64(binary.BigEndian.Uint16(size[:]))
+	case 127:
+		var size [8]byte
+		if _, err := io.ReadFull(ws.conn, size[:]); err != nil {
+			return nil, err
+		}
+		length = binary.BigEndian.Uint64(size[:])
+	}
+	if length > 1<<20 {
+		return nil, fmt.Errorf("websocket frame too large: %d", length)
+	}
+	var mask [4]byte
+	if header[1]&0x80 != 0 {
+		if _, err := io.ReadFull(ws.conn, mask[:]); err != nil {
+			return nil, err
+		}
+	}
+	payload := make([]byte, length)
+	if _, err := io.ReadFull(ws.conn, payload); err != nil {
+		return nil, err
+	}
+	if header[1]&0x80 != 0 {
+		for i := range payload {
+			payload[i] ^= mask[i%len(mask)]
+		}
+	}
+	return payload, nil
+}
+
+func PushAuthNotify(accessToken, deviceID string) []byte {
+	var auth []byte
+	auth = appendProtoString(auth, 1, accessToken)
+	auth = appendProtoString(auth, 2, deviceID)
+	var notify []byte
+	notify = protowire.AppendTag(notify, 2, protowire.VarintType)
+	notify = protowire.AppendVarint(notify, 49)
+	notify = protowire.AppendTag(notify, 10, protowire.BytesType)
+	return protowire.AppendBytes(notify, auth)
+}
+
+func appendProtoString(dst []byte, field protowire.Number, value string) []byte {
+	dst = protowire.AppendTag(dst, field, protowire.BytesType)
+	return protowire.AppendString(dst, value)
+}
diff --git a/tests/reliability/redis_failover_test.go b/tests/reliability/redis_failover_test.go
index 8d79897..f345e8f 100644
--- a/tests/reliability/redis_failover_test.go
+++ b/tests/reliability/redis_failover_test.go
@@ -3,7 +3,10 @@
 package reliability_test
 
 import (
+	"bytes"
 	"fmt"
+	"net"
+	"os/exec"
 	"strings"
 	"testing"
 	"time"
@@ -113,6 +116,95 @@ func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) {
 		"an Open->HalfOpen probe must recover the Transmite Redis circuit")
 }
 
+// RL-05 Push truth source: Rabbit accepts while Push is paused; after Redis is
+// stopped, resuming Push must requeue before websocket delivery. Recovery then
+// permits the half-open probe, durable Unacked write, and at-least-once delivery.
+func TestRL_PushUnackedRequeuesUntilRedisRecovers(t *testing.T) {
+	sender, recipient, convID := fixture.MakeFriends(t, HTTP)
+	ws, err := client.OpenWebSocket(HTTP.Config())
+	require.NoError(t, err)
+	t.Cleanup(func() { _ = ws.Close() })
+	require.NoError(t, ws.WriteBinary(client.PushAuthNotify(recipient.AccessToken, "default_device")))
+
+	deviceKey := fmt.Sprintf("im:dev:{%s}", recipient.UserID)
+	deadline := time.Now().Add(5 * time.Second)
+	for time.Now().Before(deadline) && verify.RedisCLI(t, "EXISTS", deviceKey) != "1" {
+		time.Sleep(50 * time.Millisecond)
+	}
+	require.Equal(t, "1", verify.RedisCLI(t, "EXISTS", deviceKey))
+	drainDeadline := time.Now().Add(2 * time.Second)
+	for time.Now().Before(drainDeadline) {
+		_, readErr := ws.ReadFrame(100 * time.Millisecond)
+		if timeout, ok := readErr.(net.Error); ok && timeout.Timeout() {
+			break
+		}
+		require.NoError(t, readErr)
+	}
+
+	pushContainer := HTTP.Config().Infra.PushContainer
+	require.NotEmpty(t, pushContainer)
+	requireDocker(t, "pause", pushContainer)
+	paused := true
+	redisStopped := false
+	t.Cleanup(func() {
+		if paused {
+			_, _ = exec.Command("docker", "unpause", pushContainer).CombinedOutput()
+		}
+		if redisStopped {
+			chaos.StartRedisCluster(t, HTTP.Config())
+			chaos.WaitRedisCluster(t, HTTP.Config(), 60*time.Second)
+		}
+	})
+
+	marker := "rl-unacked-" + client.NewRequestID()
+	sendRsp := &transmite.SendMessageRsp{}
+	require.NoError(t, sender.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
+		RequestId: client.NewRequestID(), ConversationId: convID,
+		Content: &msg.MessageContent{Type: msg.MessageType_TEXT,
+			Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: marker}}},
+		ClientMsgId: client.NewRequestID(),
+	}, sendRsp))
+	require.True(t, sendRsp.GetHeader().GetSuccess(), sendRsp.GetHeader().GetErrorMessage())
+
+	chaos.StopRedisCluster(t, HTTP.Config())
+	redisStopped = true
+	requireDocker(t, "unpause", pushContainer)
+	paused = false
+	if payload, readErr := ws.ReadFrame(2 * time.Second); readErr == nil {
+		t.Fatalf("Push delivered before durable Unacked persistence: %x", payload)
+	} else if timeout, ok := readErr.(net.Error); !ok || !timeout.Timeout() {
+		t.Fatalf("websocket failed while awaiting Redis outage: %v", readErr)
+	}
+
+	chaos.StartRedisCluster(t, HTTP.Config())
+	chaos.WaitRedisCluster(t, HTTP.Config(), 60*time.Second)
+	redisStopped = false
+
+	delivered := false
+	deadline = time.Now().Add(20 * time.Second)
+	for time.Now().Before(deadline) {
+		payload, readErr := ws.ReadFrame(time.Until(deadline))
+		if readErr != nil {
+			break
+		}
+		if bytes.Contains(payload, []byte(marker)) {
+			delivered = true
+			break
+		}
+	}
+	require.True(t, delivered, "requeued Push was not delivered after Redis recovery")
+	unackedKey := fmt.Sprintf("im:unack:{%s:default_device}", recipient.UserID)
+	require.Equal(t, "1", verify.RedisCLI(t, "EXISTS", unackedKey),
+		"delivery must follow durable Unacked persistence")
+}
+
+func requireDocker(t testing.TB, args ...string) {
+	t.Helper()
+	if out, err := exec.Command("docker", args...).CombinedOutput(); err != nil {
+		t.Fatalf("docker %v: %v: %s", args, err, out)
+	}
+}
+
 func reliabilityTransmiteEndpoints(t testing.TB) []string {
 	t.Helper()
 	var endpoints []string
diff --git a/transmite/source/transmite_server.h b/transmite/source/transmite_server.h
index 2d04e1d..dde3f28 100644
--- a/transmite/source/transmite_server.h
+++ b/transmite/source/transmite_server.h
@@ -608,7 +608,10 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
             if (generation && _user_info_cache) {
                 _user_info_cache->set_if_generation(uid, bytes, *generation);
             }
-            if (generation && _local_user_cache) {
+            // L1 is deliberately short-lived and remains available even when the
+            // Redis generation read failed. This lets singleflight followers share
+            // a successful Identity result without allowing an unfenced L2 write.
+            if (_local_user_cache) {
                 _local_user_cache->set(
                     lkey, bytes, randomized_ttl(std::chrono::seconds(45)));
             }

From 4abc83620d05915c64e386ee8f6ffc4b3c736627 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 14:40:53 +0800
Subject: [PATCH 18/47] fix: fence cache fills and observe push retries

---
 common/dao/data_redis.hpp                     | 87 ++++++++++++++---
 common/infra/metrics.hpp                      |  2 +
 ...026-05-18-cache-infrastructure-redesign.md | 13 +++
 push/source/push_server.h                     | 64 ++++++++-----
 tests/config.yaml                             |  1 +
 tests/pkg/chaos/redis.go                      |  4 +-
 tests/pkg/client/config.go                    |  4 +
 tests/pkg/client/websocket.go                 | 93 +++++++++++++++----
 tests/pkg/client/websocket_test.go            | 82 ++++++++++++++++
 tests/reliability/redis_failover_test.go      | 83 ++++++++++++-----
 10 files changed, 358 insertions(+), 75 deletions(-)
 create mode 100644 tests/pkg/client/websocket_test.go

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 80ecd2f..896edda 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -1136,7 +1136,15 @@ class UserInfoCache
 
     struct BatchResult {
         std::unordered_map<std::string, std::string> hits;
-        std::vector<std::string> misses;
+        struct Miss {
+            std::string uid;
+            std::optional<uint64_t> observed_generation;
+        };
+        std::vector<Miss> misses;
+    };
+    struct FencedValue {
+        std::string serialized;
+        uint64_t observed_generation;
     };
 
     explicit UserInfoCache(const RedisClient::ptr &c) : _c(c) {}
@@ -1158,7 +1166,7 @@ class UserInfoCache
         BatchResult result;
         const size_t count = std::min<size_t>(uids.size(), 2000);
         if (!_c) {
-            result.misses.assign(uids.begin(), uids.begin() + count);
+            for (size_t i = 0; i < count; ++i) result.misses.push_back({uids[i], std::nullopt});
             return result;
         }
         std::unordered_map<uint32_t, std::vector<std::pair<std::string, std::string>>> groups;
@@ -1170,28 +1178,41 @@ class UserInfoCache
             const auto &entries = group.second;
             std::vector<std::string> keys;
             keys.reserve(entries.size());
-            for (const auto &entry : entries) keys.push_back(entry.second);
+            for (const auto &entry : entries) {
+                keys.push_back(entry.second);
+                keys.push_back(key::user_info_generation_key(entry.first));
+            }
             try {
                 std::vector<sw::redis::OptionalString> values;
                 values.reserve(keys.size());
                 _c->mget(keys.begin(), keys.end(), std::back_inserter(values));
                 for (size_t i = 0; i < entries.size(); ++i) {
-                    if (i < values.size() && values[i]) result.hits[entries[i].first] = *values[i];
-                    else result.misses.push_back(entries[i].first);
+                    const size_t value_index = i * 2;
+                    if (value_index < values.size() && values[value_index]) {
+                        result.hits[entries[i].first] = *values[value_index];
+                    } else {
+                        std::optional<uint64_t> observed;
+                        if (value_index + 1 < values.size()) {
+                            try {
+                                observed = values[value_index + 1]
+                                    ? std::stoull(*values[value_index + 1]) : uint64_t{0};
+                            } catch (const std::exception &) {}
+                        }
+                        result.misses.push_back({entries[i].first, observed});
+                    }
                 }
             } catch (const RedisCircuitOpen &) {
-                for (const auto &entry : entries) result.misses.push_back(entry.first);
+                for (const auto &entry : entries) result.misses.push_back({entry.first, std::nullopt});
             } catch (const std::exception &) {
-                for (const auto &entry : entries) result.misses.push_back(entry.first);
+                for (const auto &entry : entries) result.misses.push_back({entry.first, std::nullopt});
             }
         }
         return result;
     }
 
-    void set(const std::string &uid, const std::string &serialized) {
-        if (!_c) return;
-        const auto observed = generation(uid);
-        if (observed) (void)set_if_generation(uid, serialized, *observed);
+    bool set(const std::string &uid, const std::string &serialized,
+             uint64_t observed_generation) {
+        return set_if_generation(uid, serialized, observed_generation);
     }
 
     std::optional<uint64_t> generation(const std::string &uid) {
@@ -1227,13 +1248,38 @@ class UserInfoCache
         }
     }
 
-    void batch_set(const std::unordered_map<std::string, std::string> &values) {
-        if (!_c) return;
+    size_t batch_set(const std::unordered_map<std::string, FencedValue> &values) {
+        if (!_c) return 0;
+        std::unordered_map<uint32_t, std::vector<std::pair<std::string, FencedValue>>> groups;
         size_t count = 0;
         for (const auto &entry : values) {
             if (count++ >= 2000) break;
-            set(entry.first, entry.second);
+            groups[key::user_info_bucket(entry.first)].push_back(entry);
         }
+        size_t written = 0;
+        for (const auto &group : groups) {
+            std::vector<std::string> keys;
+            std::vector<std::string> args;
+            keys.reserve(group.second.size() * 2);
+            args.reserve(group.second.size() * 3);
+            for (const auto &[uid, value] : group.second) {
+                keys.push_back(key::user_info_key(uid));
+                keys.push_back(key::user_info_generation_key(uid));
+                args.push_back(std::to_string(value.observed_generation));
+                args.push_back(value.serialized);
+                const auto ttl = value.serialized.empty()
+                    ? randomized_ttl(std::chrono::seconds(5))
+                    : randomized_ttl(kUserInfoTtl);
+                args.push_back(std::to_string(ttl.count()));
+            }
+            try {
+                written += static_cast<size_t>(_c->eval<long long>(
+                    kBatchSetIfGenerationLua, keys.begin(), keys.end(),
+                    args.begin(), args.end()));
+            } catch (const RedisCircuitOpen &) {
+            } catch (const std::exception &) {}
+        }
+        return written;
     }
 
     bool invalidate(const std::string &uid) noexcept {
@@ -1267,6 +1313,19 @@ redis.call('INCR', KEYS[2])
 redis.call('EXPIRE', KEYS[2], 604800)
 redis.call('DEL', KEYS[1])
 return 1
+)lua";
+    static constexpr const char *kBatchSetIfGenerationLua = R"lua(
+local written = 0
+for i = 1, #KEYS, 2 do
+    local arg = ((i - 1) / 2) * 3 + 1
+    local generation = tonumber(redis.call('GET', KEYS[i + 1]) or '0')
+    if generation == tonumber(ARGV[arg]) then
+        redis.call('SET', KEYS[i], ARGV[arg + 1], 'EX', ARGV[arg + 2])
+        redis.call('EXPIRE', KEYS[i + 1], 604800)
+        written = written + 1
+    end
+end
+return written
 )lua";
     RedisClient::ptr _c;
 };
diff --git a/common/infra/metrics.hpp b/common/infra/metrics.hpp
index 9c09335..7a56d8b 100644
--- a/common/infra/metrics.hpp
+++ b/common/infra/metrics.hpp
@@ -30,6 +30,8 @@ inline bvar::Adder<long> g_redis_call_failure_total("redis_call_failure_total");
 inline bvar::Adder<long> g_redis_mutex_retry_total("redis_mutex_retry_total");
 inline bvar::Adder<long> g_rate_limit_local_fallback_total("rate_limit_local_fallback_total");
 inline bvar::Adder<long> g_rate_limit_local_rejected_total("rate_limit_local_rejected_total");
+inline bvar::Adder<long> g_push_unacked_persist_failure_total("push_unacked_persist_failure_total");
+inline bvar::Adder<long> g_push_message_requeue_total("push_message_requeue_total");
 
 template <typename V>
 inline typename ::chatnow::LocalCache<V>::MetricsSink local_cache_metrics_sink() {
diff --git a/docs/superpowers/specs/2026-05-18-cache-infrastructure-redesign.md b/docs/superpowers/specs/2026-05-18-cache-infrastructure-redesign.md
index 2a8f274..9037dd9 100644
--- a/docs/superpowers/specs/2026-05-18-cache-infrastructure-redesign.md
+++ b/docs/superpowers/specs/2026-05-18-cache-infrastructure-redesign.md
@@ -457,6 +457,19 @@ if (members.size() == 1 && members[0] == "__sentinel__") {
 }
 ```
 
+### 4.2.1 UserInfo 回填代际栅栏
+
+UserInfo miss 读取必须同时取得该 uid 的 `observed_generation`，并在调用
+Identity 之前固定下来；RPC 返回后的正值或空标记回填都必须携带这个令牌，
+禁止在数据源读取完成后重新读取 generation。单 uid 使用同槽 Lua 比较
+generation 后写入，避免并发资料更新被旧快照覆盖。
+
+批量路径按 64 个虚拟 bucket 分组，每组用一次同槽 MGET 同时读取
+`value + generation`。`batch_get` 对每个 miss 返回独立的 observed token，
+`batch_set` 接受 `{serialized, observed_generation}`，每个 bucket 用一次 Lua
+逐项 CAS 回填。单批最多 2000 项，正值和空标记均使用随机 TTL；Redis
+不可用或令牌缺失时只返回源数据，不写 L2。
+
 ### 4.3 防雪崩（Cache Avalanche）
 
 **已有设计**：`randomized_ttl()`，见 `2026-05-13-cache-strategy-redesign.md` §2。
diff --git a/push/source/push_server.h b/push/source/push_server.h
index 14e65fd..87d2091 100644
--- a/push/source/push_server.h
+++ b/push/source/push_server.h
@@ -128,17 +128,13 @@ class PushServiceImpl : public PushService
             auto route = resolve_route(request->user_id());
             // Persist before delivery. On failure the RPC is retryable and no
             // client has observed a payload without durable retransmit state.
-            if (request->has_user_seq() && !_unacked) {
-                throw ::chatnow::ServiceError(::chatnow::error::kSystemUnavailable,
-                                              "unacked persistence unavailable");
-            }
             if (request->has_user_seq()) {
                 std::string payload_b64 = _utils_base64_encode(payload);
                 long long now_ts = static_cast<long long>(time(nullptr));
                 for (const auto &did : route.device_ids) {
                     if (filter_devices && target_dids.find(did) == target_dids.end()) continue;
-                    _unacked->push(request->user_id(), did,
-                                   request->user_seq(), payload_b64, now_ts);
+                    persist_unacked_(request->user_id(), did, request->user_seq(),
+                                     payload_b64, now_ts);
                 }
             }
 
@@ -209,13 +205,9 @@ class PushServiceImpl : public PushService
                 }
 
                 for (const auto &did : route.device_ids) {
-                    if (it != uid2seq.end() && !_unacked) {
-                        throw ::chatnow::ServiceError(::chatnow::error::kSystemUnavailable,
-                                                      "unacked persistence unavailable");
-                    }
                     if (it != uid2seq.end()) {
-                        _unacked->push(uid, did, it->second,
-                                       _utils_base64_encode(payload), now_ts);
+                        persist_unacked_(uid, did, it->second,
+                                         _utils_base64_encode(payload), now_ts);
                     }
                     if (_local_send(uid, did, payload) > 0) ++total;
                 }
@@ -275,10 +267,10 @@ class PushServiceImpl : public PushService
                 route = resolve_route(uid);
             } catch (const RedisCircuitOpen &e) {
                 LOG_WARN("Push-Consumer: route circuit unavailable: {}", e.what());
-                return ConsumeAction::NackRequeue;
+                return requeue_();
             } catch (const sw::redis::Error &e) {
                 LOG_WARN("Push-Consumer: route Redis unavailable: {}", e.what());
-                return ConsumeAction::NackRequeue;
+                return requeue_();
             }
             if (route.device_ids.empty()) { remote_uids.push_back(uid); continue; }
 
@@ -299,16 +291,21 @@ class PushServiceImpl : public PushService
                 std::string inst = (it != route.device_to_instance.end()) ? it->second : "";
                 if (inst == _instance_id) {
                     if (itu != uid2seq.end()) {
-                        if (!_unacked) return ConsumeAction::NackRequeue;
                         try {
-                            _unacked->push(uid, did, itu->second,
-                                           _utils_base64_encode(user_payload), now_ts);
+                            persist_unacked_(uid, did, itu->second,
+                                             _utils_base64_encode(user_payload), now_ts);
                         } catch (const RedisCircuitOpen &e) {
                             LOG_WARN("Push-Consumer: unacked circuit unavailable: {}", e.what());
-                            return ConsumeAction::NackRequeue;
+                            return requeue_();
                         } catch (const sw::redis::Error &e) {
                             LOG_WARN("Push-Consumer: unacked Redis unavailable: {}", e.what());
-                            return ConsumeAction::NackRequeue;
+                            return requeue_();
+                        } catch (const ::chatnow::ServiceError &e) {
+                            LOG_WARN("Push-Consumer: unacked unavailable: {}", e.what());
+                            return requeue_();
+                        } catch (const std::exception &e) {
+                            LOG_WARN("Push-Consumer: unacked persistence failed: {}", e.what());
+                            return requeue_();
                         }
                         if (_local_send(uid, did, user_payload) > 0) any_local = true;
                     } else {
@@ -330,9 +327,9 @@ class PushServiceImpl : public PushService
             try {
                 route = resolve_route(uid);
             } catch (const RedisCircuitOpen &) {
-                return ConsumeAction::NackRequeue;
+                return requeue_();
             } catch (const sw::redis::Error &) {
-                return ConsumeAction::NackRequeue;
+                return requeue_();
             }
             for (const auto &did : route.device_ids) {
                 auto it = route.device_to_instance.find(did);
@@ -351,7 +348,7 @@ class PushServiceImpl : public PushService
             auto channel = _mm_channels->choose(peer);
             if (!channel) {
                 LOG_WARN("Push-Consumer: 对端 {} 不可达", peer);
-                return ConsumeAction::NackRequeue;
+                return requeue_();
             }
             PushService_Stub stub(channel.get());
             PushBatchReq req;
@@ -371,7 +368,7 @@ class PushServiceImpl : public PushService
             if (cntl.Failed() || !rsp.header().success()) {
                 LOG_WARN("PushBatch persistence failed peer={}: {} {}", peer,
                          cntl.ErrorText(), rsp.header().error_message());
-                return ConsumeAction::NackRequeue;
+                return requeue_();
             }
         }
         return ConsumeAction::Ack;
@@ -474,6 +471,27 @@ class PushServiceImpl : public PushService
     }
 
 private:
+    void persist_unacked_(const std::string &uid, const std::string &did,
+                          unsigned long user_seq, const std::string &payload_b64,
+                          long long score_ts) {
+        if (!_unacked) {
+            metrics::g_push_unacked_persist_failure_total << 1;
+            throw ::chatnow::ServiceError(::chatnow::error::kSystemUnavailable,
+                                          "unacked persistence unavailable");
+        }
+        try {
+            _unacked->push(uid, did, user_seq, payload_b64, score_ts);
+        } catch (...) {
+            metrics::g_push_unacked_persist_failure_total << 1;
+            throw;
+        }
+    }
+
+    static ConsumeAction requeue_() {
+        metrics::g_push_message_requeue_total << 1;
+        return ConsumeAction::NackRequeue;
+    }
+
     void _handle_client_auth_(const NotifyClientAuth &auth,
                               server_t::connection_ptr conn) {
         if (auth.access_token().empty() || auth.device_id().empty()) {
diff --git a/tests/config.yaml b/tests/config.yaml
index 69c75c2..f094fcf 100644
--- a/tests/config.yaml
+++ b/tests/config.yaml
@@ -23,5 +23,6 @@ log:
 infra:
   redis_container: "redis-node1"
   push_container: "push_server-service"
+  push_vars: "http://localhost:10008"
   compose_dir: ".."
   transmite_vars: "http://localhost:10004"
diff --git a/tests/pkg/chaos/redis.go b/tests/pkg/chaos/redis.go
index d136862..ea188a7 100644
--- a/tests/pkg/chaos/redis.go
+++ b/tests/pkg/chaos/redis.go
@@ -24,7 +24,9 @@ func compose(t testing.TB, cfg *client.Config, args ...string) {
 }
 
 func StopRedisCluster(t testing.TB, cfg *client.Config) {
-	compose(t, cfg, append([]string{"stop"}, redisServices...)...)
+	// Fail all AOF-backed test nodes immediately so short-lived service L1
+	// entries remain warm for deterministic failover tests.
+	compose(t, cfg, append([]string{"stop", "-t", "0"}, redisServices...)...)
 }
 
 func StartRedisCluster(t testing.TB, cfg *client.Config) {
diff --git a/tests/pkg/client/config.go b/tests/pkg/client/config.go
index 9a9f46f..47c842a 100644
--- a/tests/pkg/client/config.go
+++ b/tests/pkg/client/config.go
@@ -37,6 +37,7 @@ type LogConfig struct {
 type InfraConfig struct {
 	RedisContainer string `yaml:"redis_container"`
 	PushContainer  string `yaml:"push_container"`
+	PushVars       string `yaml:"push_vars"`
 	ComposeDir     string `yaml:"compose_dir"`
 	TransmiteVars  string `yaml:"transmite_vars"`
 }
@@ -72,6 +73,9 @@ func LoadConfig(path string) *Config {
 	if v := os.Getenv("PUSH_CONTAINER"); v != "" {
 		cfg.Infra.PushContainer = v
 	}
+	if v := os.Getenv("PUSH_VARS"); v != "" {
+		cfg.Infra.PushVars = v
+	}
 	if v := os.Getenv("COMPOSE_DIR"); v != "" {
 		cfg.Infra.ComposeDir = v
 	}
diff --git a/tests/pkg/client/websocket.go b/tests/pkg/client/websocket.go
index 25a86d5..560c524 100644
--- a/tests/pkg/client/websocket.go
+++ b/tests/pkg/client/websocket.go
@@ -16,7 +16,8 @@ import (
 )
 
 type WebSocket struct {
-	conn net.Conn
+	conn   net.Conn
+	reader *bufio.Reader
 }
 
 func OpenWebSocket(cfg *Config) (*WebSocket, error) {
@@ -42,7 +43,8 @@ func OpenWebSocket(cfg *Config) (*WebSocket, error) {
 	if err := req.Write(conn); err != nil {
 		return fail(err)
 	}
-	rsp, err := http.ReadResponse(bufio.NewReader(conn), req)
+	reader := bufio.NewReader(conn)
+	rsp, err := http.ReadResponse(reader, req)
 	if err != nil {
 		return fail(err)
 	}
@@ -52,13 +54,17 @@ func OpenWebSocket(cfg *Config) (*WebSocket, error) {
 	if err := conn.SetDeadline(time.Time{}); err != nil {
 		return fail(err)
 	}
-	return &WebSocket{conn: conn}, nil
+	return &WebSocket{conn: conn, reader: reader}, nil
 }
 
 func (ws *WebSocket) Close() error { return ws.conn.Close() }
 
 func (ws *WebSocket) WriteBinary(payload []byte) error {
-	frame := []byte{0x82}
+	return ws.writeFrame(0x2, payload)
+}
+
+func (ws *WebSocket) writeFrame(opcode byte, payload []byte) error {
+	frame := []byte{0x80 | opcode}
 	switch {
 	case len(payload) < 126:
 		frame = append(frame, 0x80|byte(len(payload)))
@@ -86,44 +92,99 @@ func (ws *WebSocket) ReadFrame(timeout time.Duration) ([]byte, error) {
 	if err := ws.conn.SetReadDeadline(time.Now().Add(timeout)); err != nil {
 		return nil, err
 	}
+	var message []byte
+	continuing := false
+	for {
+		fin, opcode, payload, err := ws.readRawFrame()
+		if err != nil {
+			return nil, err
+		}
+		switch opcode {
+		case 0x8:
+			_ = ws.Close()
+			return nil, io.EOF
+		case 0x9:
+			if err := ws.writeFrame(0xA, payload); err != nil {
+				return nil, err
+			}
+			continue
+		case 0xA:
+			continue
+		case 0x1, 0x2:
+			if continuing {
+				_ = ws.Close()
+				return nil, fmt.Errorf("new data frame during continuation")
+			}
+			message = append(message, payload...)
+			if fin {
+				return message, nil
+			}
+			continuing = true
+		case 0x0:
+			if !continuing {
+				_ = ws.Close()
+				return nil, fmt.Errorf("unexpected continuation frame")
+			}
+			message = append(message, payload...)
+			if fin {
+				return message, nil
+			}
+		default:
+			_ = ws.Close()
+			return nil, fmt.Errorf("unsupported websocket opcode %d", opcode)
+		}
+	}
+}
+
+func (ws *WebSocket) readRawFrame() (bool, byte, []byte, error) {
 	var header [2]byte
-	if _, err := io.ReadFull(ws.conn, header[:]); err != nil {
-		return nil, err
+	n, err := io.ReadFull(ws.reader, header[:])
+	if err != nil {
+		if n != 0 {
+			_ = ws.Close()
+		}
+		return false, 0, nil, err
 	}
+	fin, opcode := header[0]&0x80 != 0, header[0]&0x0f
 	length := uint64(header[1] & 0x7f)
 	switch length {
 	case 126:
 		var size [2]byte
-		if _, err := io.ReadFull(ws.conn, size[:]); err != nil {
-			return nil, err
+		if _, err := io.ReadFull(ws.reader, size[:]); err != nil {
+			_ = ws.Close()
+			return false, 0, nil, err
 		}
 		length = uint64(binary.BigEndian.Uint16(size[:]))
 	case 127:
 		var size [8]byte
-		if _, err := io.ReadFull(ws.conn, size[:]); err != nil {
-			return nil, err
+		if _, err := io.ReadFull(ws.reader, size[:]); err != nil {
+			_ = ws.Close()
+			return false, 0, nil, err
 		}
 		length = binary.BigEndian.Uint64(size[:])
 	}
 	if length > 1<<20 {
-		return nil, fmt.Errorf("websocket frame too large: %d", length)
+		_ = ws.Close()
+		return false, 0, nil, fmt.Errorf("websocket frame too large: %d", length)
 	}
 	var mask [4]byte
 	if header[1]&0x80 != 0 {
-		if _, err := io.ReadFull(ws.conn, mask[:]); err != nil {
-			return nil, err
+		if _, err := io.ReadFull(ws.reader, mask[:]); err != nil {
+			_ = ws.Close()
+			return false, 0, nil, err
 		}
 	}
 	payload := make([]byte, length)
-	if _, err := io.ReadFull(ws.conn, payload); err != nil {
-		return nil, err
+	if _, err := io.ReadFull(ws.reader, payload); err != nil {
+		_ = ws.Close()
+		return false, 0, nil, err
 	}
 	if header[1]&0x80 != 0 {
 		for i := range payload {
 			payload[i] ^= mask[i%len(mask)]
 		}
 	}
-	return payload, nil
+	return fin, opcode, payload, nil
 }
 
 func PushAuthNotify(accessToken, deviceID string) []byte {
diff --git a/tests/pkg/client/websocket_test.go b/tests/pkg/client/websocket_test.go
new file mode 100644
index 0000000..676002d
--- /dev/null
+++ b/tests/pkg/client/websocket_test.go
@@ -0,0 +1,82 @@
+package client
+
+import (
+	"bufio"
+	"io"
+	"net"
+	"net/http"
+	"net/http/httptest"
+	"net/url"
+	"testing"
+	"time"
+)
+
+func serverFrame(fin bool, opcode byte, payload string) []byte {
+	first := opcode
+	if fin {
+		first |= 0x80
+	}
+	return append([]byte{first, byte(len(payload))}, []byte(payload)...)
+}
+
+func TestWebSocketPreservesFrameBufferedWithUpgrade(t *testing.T) {
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		hijacker := w.(http.Hijacker)
+		conn, rw, err := hijacker.Hijack()
+		if err != nil {
+			return
+		}
+		defer conn.Close()
+		_, _ = rw.WriteString("HTTP/1.1 101 Switching Protocols\r\nConnection: Upgrade\r\nUpgrade: websocket\r\n\r\n")
+		_, _ = rw.Write(serverFrame(true, 2, "buffered"))
+		_ = rw.Flush()
+		time.Sleep(100 * time.Millisecond)
+	}))
+	defer server.Close()
+	u, _ := url.Parse(server.URL)
+	ws, err := OpenWebSocket(&Config{Target: TargetConfig{WebsocketAddr: u.Host}})
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer ws.Close()
+	payload, err := ws.ReadFrame(time.Second)
+	if err != nil || string(payload) != "buffered" {
+		t.Fatalf("payload=%q err=%v", payload, err)
+	}
+}
+
+func pipeWebSocket(t *testing.T, frames ...[]byte) *WebSocket {
+	t.Helper()
+	clientConn, serverConn := net.Pipe()
+	t.Cleanup(func() { _ = clientConn.Close(); _ = serverConn.Close() })
+	go func() {
+		for _, frame := range frames {
+			_, _ = serverConn.Write(frame)
+			if len(frame) >= 2 && frame[0]&0x0f == 9 {
+				pong := make([]byte, 7) // FIN+PONG, masked one-byte payload.
+				_, _ = io.ReadFull(serverConn, pong)
+			}
+		}
+	}()
+	return &WebSocket{conn: clientConn, reader: bufio.NewReader(clientConn)}
+}
+
+func TestWebSocketSkipsPingBeforeData(t *testing.T) {
+	ws := pipeWebSocket(t, serverFrame(true, 9, "p"), serverFrame(true, 2, "data"))
+	payload, err := ws.ReadFrame(time.Second)
+	if err != nil || string(payload) != "data" {
+		t.Fatalf("payload=%q err=%v", payload, err)
+	}
+}
+
+func TestWebSocketReassemblesFragmentsAcrossControlFrame(t *testing.T) {
+	ws := pipeWebSocket(t,
+		serverFrame(false, 2, "frag-"),
+		serverFrame(true, 9, "p"),
+		serverFrame(true, 0, "ment"),
+	)
+	payload, err := ws.ReadFrame(time.Second)
+	if err != nil || string(payload) != "frag-ment" {
+		t.Fatalf("payload=%q err=%v", payload, err)
+	}
+}
diff --git a/tests/reliability/redis_failover_test.go b/tests/reliability/redis_failover_test.go
index f345e8f..330e53e 100644
--- a/tests/reliability/redis_failover_test.go
+++ b/tests/reliability/redis_failover_test.go
@@ -141,6 +141,15 @@ func TestRL_PushUnackedRequeuesUntilRedisRecovers(t *testing.T) {
 		require.NoError(t, readErr)
 	}
 
+	// A successful end-to-end warm delivery deterministically fills this Push
+	// instance's two-second route L1 immediately before the outage choreography.
+	warmMarker := "rl-unacked-warm-" + client.NewRequestID()
+	sendReliabilityMessage(t, sender, convID, warmMarker)
+	require.True(t, readWebSocketMarker(ws, warmMarker, 5*time.Second), "warm Push delivery missing")
+	pushEndpoints := reliabilityPushEndpoints(t)
+	persistBefore := reliabilitySumBVar(t, pushEndpoints, "push_unacked_persist_failure_total")
+	requeueBefore := reliabilitySumBVar(t, pushEndpoints, "push_message_requeue_total")
+
 	pushContainer := HTTP.Config().Infra.PushContainer
 	require.NotEmpty(t, pushContainer)
 	requireDocker(t, "pause", pushContainer)
@@ -157,20 +166,15 @@ func TestRL_PushUnackedRequeuesUntilRedisRecovers(t *testing.T) {
 	})
 
 	marker := "rl-unacked-" + client.NewRequestID()
-	sendRsp := &transmite.SendMessageRsp{}
-	require.NoError(t, sender.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
-		RequestId: client.NewRequestID(), ConversationId: convID,
-		Content: &msg.MessageContent{Type: msg.MessageType_TEXT,
-			Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: marker}}},
-		ClientMsgId: client.NewRequestID(),
-	}, sendRsp))
-	require.True(t, sendRsp.GetHeader().GetSuccess(), sendRsp.GetHeader().GetErrorMessage())
+	sendReliabilityMessage(t, sender, convID, marker)
 
 	chaos.StopRedisCluster(t, HTTP.Config())
 	redisStopped = true
 	requireDocker(t, "unpause", pushContainer)
 	paused = false
-	if payload, readErr := ws.ReadFrame(2 * time.Second); readErr == nil {
+	requireBVarIncrease(t, pushEndpoints, "push_unacked_persist_failure_total", persistBefore, 5*time.Second)
+	requireBVarIncrease(t, pushEndpoints, "push_message_requeue_total", requeueBefore, 5*time.Second)
+	if payload, readErr := ws.ReadFrame(500 * time.Millisecond); readErr == nil {
 		t.Fatalf("Push delivered before durable Unacked persistence: %x", payload)
 	} else if timeout, ok := readErr.(net.Error); !ok || !timeout.Timeout() {
 		t.Fatalf("websocket failed while awaiting Redis outage: %v", readErr)
@@ -180,22 +184,49 @@ func TestRL_PushUnackedRequeuesUntilRedisRecovers(t *testing.T) {
 	chaos.WaitRedisCluster(t, HTTP.Config(), 60*time.Second)
 	redisStopped = false
 
-	delivered := false
-	deadline = time.Now().Add(20 * time.Second)
+	require.True(t, readWebSocketMarker(ws, marker, 20*time.Second),
+		"requeued Push was not delivered after Redis recovery")
+	unackedKey := fmt.Sprintf("im:unack:{%s:default_device}", recipient.UserID)
+	require.Equal(t, "1", verify.RedisCLI(t, "EXISTS", unackedKey),
+		"delivery must follow durable Unacked persistence")
+}
+
+func sendReliabilityMessage(t testing.TB, sender *client.HTTPClient, convID, marker string) {
+	t.Helper()
+	rsp := &transmite.SendMessageRsp{}
+	require.NoError(t, sender.DoAuth("/service/transmite/send", &transmite.SendMessageReq{
+		RequestId: client.NewRequestID(), ConversationId: convID,
+		Content: &msg.MessageContent{Type: msg.MessageType_TEXT,
+			Body: &msg.MessageContent_Text{Text: &msg.TextContent{Text: marker}}},
+		ClientMsgId: client.NewRequestID(),
+	}, rsp))
+	require.True(t, rsp.GetHeader().GetSuccess(), rsp.GetHeader().GetErrorMessage())
+}
+
+func readWebSocketMarker(ws *client.WebSocket, marker string, timeout time.Duration) bool {
+	deadline := time.Now().Add(timeout)
 	for time.Now().Before(deadline) {
-		payload, readErr := ws.ReadFrame(time.Until(deadline))
-		if readErr != nil {
-			break
+		payload, err := ws.ReadFrame(time.Until(deadline))
+		if err != nil {
+			return false
 		}
 		if bytes.Contains(payload, []byte(marker)) {
-			delivered = true
-			break
+			return true
 		}
 	}
-	require.True(t, delivered, "requeued Push was not delivered after Redis recovery")
-	unackedKey := fmt.Sprintf("im:unack:{%s:default_device}", recipient.UserID)
-	require.Equal(t, "1", verify.RedisCLI(t, "EXISTS", unackedKey),
-		"delivery must follow durable Unacked persistence")
+	return false
+}
+
+func requireBVarIncrease(t testing.TB, endpoints []string, metric string, before int64, timeout time.Duration) {
+	t.Helper()
+	deadline := time.Now().Add(timeout)
+	for time.Now().Before(deadline) {
+		if reliabilitySumBVar(t, endpoints, metric) > before {
+			return
+		}
+		time.Sleep(50 * time.Millisecond)
+	}
+	t.Fatalf("%s did not increase from %d", metric, before)
 }
 
 func requireDocker(t testing.TB, args ...string) {
@@ -206,9 +237,19 @@ func requireDocker(t testing.TB, args ...string) {
 }
 
 func reliabilityTransmiteEndpoints(t testing.TB) []string {
+	t.Helper()
+	return reliabilityEndpoints(t, HTTP.Config().Infra.TransmiteVars)
+}
+
+func reliabilityPushEndpoints(t testing.TB) []string {
+	t.Helper()
+	return reliabilityEndpoints(t, HTTP.Config().Infra.PushVars)
+}
+
+func reliabilityEndpoints(t testing.TB, rawEndpoints string) []string {
 	t.Helper()
 	var endpoints []string
-	for _, raw := range strings.Split(HTTP.Config().Infra.TransmiteVars, ",") {
+	for _, raw := range strings.Split(rawEndpoints, ",") {
 		if endpoint := strings.TrimRight(strings.TrimSpace(raw), "/"); endpoint != "" {
 			endpoints = append(endpoints, endpoint)
 		}

From 0bb9585fec962a0ccaa035bd6b3d35514110aa15 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 14:58:50 +0800
Subject: [PATCH 19/47] fix: make cache failover tests deterministic

---
 common/dao/data_redis.hpp                     | 29 +++++++++
 conf/docker/push_server.conf                  |  2 +
 ...026-05-18-cache-infrastructure-redesign.md |  7 +-
 push/source/push_server.cc                    |  2 +
 push/source/push_server.h                     | 23 +++++--
 tests/config.yaml                             |  1 +
 tests/pkg/chaos/redis.go                      |  4 +-
 tests/pkg/client/config.go                    | 19 ++++--
 tests/pkg/client/websocket.go                 | 27 ++++----
 tests/pkg/client/websocket_test.go            | 64 ++++++++++++++++++-
 tests/reliability/redis_failover_test.go      | 36 +++++++----
 11 files changed, 172 insertions(+), 42 deletions(-)

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 896edda..8e2868f 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -1314,7 +1314,36 @@ redis.call('EXPIRE', KEYS[2], 604800)
 redis.call('DEL', KEYS[1])
 return 1
 )lua";
+    // Redis scripts are isolated but do not roll back writes after a runtime
+    // error. Validate every entry before mutating so deterministic late errors
+    // cannot partially fill a bucket. Server OOM during SET is a Redis-level
+    // limitation and is surfaced as a failed best-effort cache fill.
     static constexpr const char *kBatchSetIfGenerationLua = R"lua(
+if (#KEYS % 2) ~= 0 or #ARGV ~= (#KEYS / 2) * 3 then
+    return redis.error_reply('invalid batch cache arguments')
+end
+local function key_type(key)
+    local reply = redis.call('TYPE', key)
+    if type(reply) == 'table' then return reply.ok end
+    return reply
+end
+for i = 1, #KEYS, 2 do
+    local arg = ((i - 1) / 2) * 3 + 1
+    local value_type = key_type(KEYS[i])
+    local generation_type = key_type(KEYS[i + 1])
+    local expected = tonumber(ARGV[arg])
+    local ttl = tonumber(ARGV[arg + 2])
+    local generation = nil
+    if generation_type == 'none' then generation = 0 end
+    if generation_type == 'string' then generation = tonumber(redis.call('GET', KEYS[i + 1])) end
+    if (value_type ~= 'none' and value_type ~= 'string') or
+       (generation_type ~= 'none' and generation_type ~= 'string') or
+       generation == nil or expected == nil or expected < 0 or
+       expected ~= math.floor(expected) or ARGV[arg + 1] == nil or ttl == nil or
+       ttl <= 0 or ttl ~= math.floor(ttl) then
+        return redis.error_reply('invalid batch cache entry')
+    end
+end
 local written = 0
 for i = 1, #KEYS, 2 do
     local arg = ((i - 1) / 2) * 3 + 1
diff --git a/conf/docker/push_server.conf b/conf/docker/push_server.conf
index 920b0ac..f36ba8a 100644
--- a/conf/docker/push_server.conf
+++ b/conf/docker/push_server.conf
@@ -25,5 +25,7 @@
 # M5 心跳触发未 ack 重传
 -resend_batch=50
 -resend_max_age_sec=5
+# Compose reliability tests pause this sole Push instance during Redis failover.
+-route_l1_ttl_sec=30
 # JWT — 统一从 auth.json 加载（与 identity/gateway 共享密钥源）
 -auth_config=/im/conf/auth.json
diff --git a/docs/superpowers/specs/2026-05-18-cache-infrastructure-redesign.md b/docs/superpowers/specs/2026-05-18-cache-infrastructure-redesign.md
index 9037dd9..41e846f 100644
--- a/docs/superpowers/specs/2026-05-18-cache-infrastructure-redesign.md
+++ b/docs/superpowers/specs/2026-05-18-cache-infrastructure-redesign.md
@@ -467,7 +467,9 @@ generation 后写入，避免并发资料更新被旧快照覆盖。
 批量路径按 64 个虚拟 bucket 分组，每组用一次同槽 MGET 同时读取
 `value + generation`。`batch_get` 对每个 miss 返回独立的 observed token，
 `batch_set` 接受 `{serialized, observed_generation}`，每个 bucket 用一次 Lua
-逐项 CAS 回填。单批最多 2000 项，正值和空标记均使用随机 TTL；Redis
+逐项 CAS 回填。Lua 在任何写入前完整校验所有 key 类型和参数，避免后段
+确定性错误造成同 bucket 部分回填（Redis OOM 等运行时错误不提供事务回滚）。
+单批最多 2000 项，正值和空标记均使用随机 TTL；Redis
 不可用或令牌缺失时只返回源数据，不写 L2。
 
 ### 4.3 防雪崩（Cache Avalanche）
@@ -485,6 +487,9 @@ generation 后写入，避免并发资料更新被旧快照覆盖。
 | L1 Members | 8s | 6.4-9.6s |
 | L1 UserInfo | 45s | 36-54s |
 
+Push OnlineRoute L1 通过 gflag/Builder 注入，生产默认保持 2s；仅 Compose
+可靠性测试栈配置为 30s，以便暂停唯一 Push 实例时稳定复现 Redis 故障。
+
 ---
 
 ## 5. 服务层改动
diff --git a/push/source/push_server.cc b/push/source/push_server.cc
index 78acdce..17247ff 100644
--- a/push/source/push_server.cc
+++ b/push/source/push_server.cc
@@ -35,6 +35,7 @@ DEFINE_string(mq_push_binding_key, "push", "推送绑定键");
 // M5: 心跳触发未 ack 重传的可调参数
 DEFINE_int32(resend_batch, 50, "心跳触发未 ack 重传的批量上限");
 DEFINE_int32(resend_max_age_sec, 5, "未 ack 项入队后等待多少秒视为可重传");
+DEFINE_int32(route_l1_ttl_sec, 2, "Push 在线路由 L1 TTL（1-300 秒）");
 
 // JWT — 统一从 auth.json 加载（与 identity/gateway 共享密钥源）
 DEFINE_string(auth_config, "/im/conf/auth.json", "JWT 鉴权配置文件路径(JSON)");
@@ -55,6 +56,7 @@ int main(int argc, char *argv[])
     psb.make_discovery_object(FLAGS_registry_host, FLAGS_base_service, FLAGS_message_service, FLAGS_push_service);
     psb.make_reg_object(FLAGS_registry_host, FLAGS_base_service + FLAGS_instance_name, FLAGS_access_host);
     psb.set_resend_params(FLAGS_resend_batch, FLAGS_resend_max_age_sec);
+    psb.set_route_l1_ttl(FLAGS_route_l1_ttl_sec);
     psb.set_etcd_client(std::make_shared<etcd::Client>(FLAGS_registry_host));
     psb.set_push_service_dir(FLAGS_base_service + FLAGS_push_service);
     psb.make_cross_reaper_election();
diff --git a/push/source/push_server.h b/push/source/push_server.h
index 87d2091..f6066fd 100644
--- a/push/source/push_server.h
+++ b/push/source/push_server.h
@@ -41,6 +41,7 @@
 #include <thread>
 #include <chrono>
 #include <limits>
+#include <stdexcept>
 #include <tuple>
 #include <unordered_set>
 
@@ -65,7 +66,8 @@ class PushServiceImpl : public PushService
                     const ServiceManager::ptr &channels,
                     LeaderElection::ptr cross_reaper_election = nullptr,
                     LocalCache<RouteEntry>::ptr local_route_cache = nullptr,
-                    InflightRegistry::ptr inflight_registry = nullptr)
+                    InflightRegistry::ptr inflight_registry = nullptr,
+                    std::chrono::seconds route_l1_ttl = std::chrono::seconds(2))
         : _connections(connections),
           _jwt_codec(jwt_codec),
           _redis(redis),
@@ -77,7 +79,12 @@ class PushServiceImpl : public PushService
           _mm_channels(channels),
           _cross_reaper_election(std::move(cross_reaper_election)),
           _local_route_cache(std::move(local_route_cache)),
-          _inflight_registry(std::move(inflight_registry)) {}
+          _inflight_registry(std::move(inflight_registry)),
+          _route_l1_ttl(route_l1_ttl) {
+        if (_route_l1_ttl.count() < 1 || _route_l1_ttl.count() > 300) {
+            throw std::invalid_argument("Push route L1 TTL must be within 1..300 seconds");
+        }
+    }
 
     void set_resend_params(long batch, long max_age_sec) {
         _resend_batch = batch;
@@ -698,7 +705,7 @@ class PushServiceImpl : public PushService
             route.device_to_instance[did] = inst;
         }
         if (_local_route_cache) {
-            _local_route_cache->set(cache_key, route, randomized_ttl(std::chrono::seconds(2)));
+            _local_route_cache->set(cache_key, route, randomized_ttl(_route_l1_ttl));
         }
 
         lk.unlock();
@@ -979,6 +986,7 @@ class PushServiceImpl : public PushService
     LeaderElection::ptr _cross_reaper_election;
     LocalCache<RouteEntry>::ptr _local_route_cache;
     InflightRegistry::ptr _inflight_registry;
+    std::chrono::seconds _route_l1_ttl{2};
     std::mutex _dummy_mu_;
 };
 
@@ -1182,6 +1190,12 @@ class PushServerBuilder
         _resend_batch = batch;
         _resend_max_age_sec = max_age_sec;
     }
+    void set_route_l1_ttl(int ttl_sec) {
+        if (ttl_sec < 1 || ttl_sec > 300) {
+            throw std::invalid_argument("Push route L1 TTL must be within 1..300 seconds");
+        }
+        _route_l1_ttl = std::chrono::seconds(ttl_sec);
+    }
     void set_reaper_owner(const std::string &owner) { _reaper_owner = owner; }
     void set_etcd_client(std::shared_ptr<etcd::Client> etcd) { _etcd_client = etcd; }
 
@@ -1220,7 +1234,7 @@ class PushServerBuilder
         _push_service = new PushServiceImpl(
             _connections, _jwt_codec, _redis_client, _online_route, _unacked, _cross_outbox,
             _instance_id, _message_service_name, _mm_channels,
-            _cross_reaper_election, _local_route_cache, _inflight_registry);
+            _cross_reaper_election, _local_route_cache, _inflight_registry, _route_l1_ttl);
         _push_service->set_resend_params(_resend_batch, _resend_max_age_sec);
         int ret = _rpc_server->AddService(_push_service, brpc::ServiceOwnership::SERVER_OWNS_SERVICE);
         if (ret == -1) { LOG_ERROR("Push: AddService 失败"); abort(); }
@@ -1308,6 +1322,7 @@ class PushServerBuilder
     LeaderElection::ptr _cross_reaper_election;
     LocalCache<RouteEntry>::ptr _local_route_cache;
     InflightRegistry::ptr _inflight_registry;
+    std::chrono::seconds _route_l1_ttl{2};
     std::string _push_service_dir;
     LeaderElection::ptr _stale_reaper_election;
     std::thread _stale_reaper_thread;
diff --git a/tests/config.yaml b/tests/config.yaml
index f094fcf..e65dad1 100644
--- a/tests/config.yaml
+++ b/tests/config.yaml
@@ -24,5 +24,6 @@ infra:
   redis_container: "redis-node1"
   push_container: "push_server-service"
   push_vars: "http://localhost:10008"
+  push_route_l1_ttl_sec: 30
   compose_dir: ".."
   transmite_vars: "http://localhost:10004"
diff --git a/tests/pkg/chaos/redis.go b/tests/pkg/chaos/redis.go
index ea188a7..d136862 100644
--- a/tests/pkg/chaos/redis.go
+++ b/tests/pkg/chaos/redis.go
@@ -24,9 +24,7 @@ func compose(t testing.TB, cfg *client.Config, args ...string) {
 }
 
 func StopRedisCluster(t testing.TB, cfg *client.Config) {
-	// Fail all AOF-backed test nodes immediately so short-lived service L1
-	// entries remain warm for deterministic failover tests.
-	compose(t, cfg, append([]string{"stop", "-t", "0"}, redisServices...)...)
+	compose(t, cfg, append([]string{"stop"}, redisServices...)...)
 }
 
 func StartRedisCluster(t testing.TB, cfg *client.Config) {
diff --git a/tests/pkg/client/config.go b/tests/pkg/client/config.go
index 47c842a..394e486 100644
--- a/tests/pkg/client/config.go
+++ b/tests/pkg/client/config.go
@@ -2,6 +2,7 @@ package client
 
 import (
 	"os"
+	"strconv"
 
 	"gopkg.in/yaml.v3"
 )
@@ -35,11 +36,12 @@ type LogConfig struct {
 }
 
 type InfraConfig struct {
-	RedisContainer string `yaml:"redis_container"`
-	PushContainer  string `yaml:"push_container"`
-	PushVars       string `yaml:"push_vars"`
-	ComposeDir     string `yaml:"compose_dir"`
-	TransmiteVars  string `yaml:"transmite_vars"`
+	RedisContainer    string `yaml:"redis_container"`
+	PushContainer     string `yaml:"push_container"`
+	PushVars          string `yaml:"push_vars"`
+	PushRouteL1TTLSec int    `yaml:"push_route_l1_ttl_sec"`
+	ComposeDir        string `yaml:"compose_dir"`
+	TransmiteVars     string `yaml:"transmite_vars"`
 }
 
 func LoadConfig(path string) *Config {
@@ -76,6 +78,13 @@ func LoadConfig(path string) *Config {
 	if v := os.Getenv("PUSH_VARS"); v != "" {
 		cfg.Infra.PushVars = v
 	}
+	if v := os.Getenv("PUSH_ROUTE_L1_TTL_SEC"); v != "" {
+		ttl, err := strconv.Atoi(v)
+		if err != nil {
+			panic("invalid PUSH_ROUTE_L1_TTL_SEC: " + err.Error())
+		}
+		cfg.Infra.PushRouteL1TTLSec = ttl
+	}
 	if v := os.Getenv("COMPOSE_DIR"); v != "" {
 		cfg.Infra.ComposeDir = v
 	}
diff --git a/tests/pkg/client/websocket.go b/tests/pkg/client/websocket.go
index 560c524..b41c36f 100644
--- a/tests/pkg/client/websocket.go
+++ b/tests/pkg/client/websocket.go
@@ -97,6 +97,9 @@ func (ws *WebSocket) ReadFrame(timeout time.Duration) ([]byte, error) {
 	for {
 		fin, opcode, payload, err := ws.readRawFrame()
 		if err != nil {
+			if continuing {
+				_ = ws.Close()
+			}
 			return nil, err
 		}
 		switch opcode {
@@ -146,7 +149,17 @@ func (ws *WebSocket) readRawFrame() (bool, byte, []byte, error) {
 		return false, 0, nil, err
 	}
 	fin, opcode := header[0]&0x80 != 0, header[0]&0x0f
-	length := uint64(header[1] & 0x7f)
+	masked := header[1]&0x80 != 0
+	if masked {
+		_ = ws.Close()
+		return false, 0, nil, fmt.Errorf("masked websocket server frame")
+	}
+	lengthCode := header[1] & 0x7f
+	if opcode&0x8 != 0 && (!fin || lengthCode > 125) {
+		_ = ws.Close()
+		return false, 0, nil, fmt.Errorf("invalid websocket control frame")
+	}
+	length := uint64(lengthCode)
 	switch length {
 	case 126:
 		var size [2]byte
@@ -167,23 +180,11 @@ func (ws *WebSocket) readRawFrame() (bool, byte, []byte, error) {
 		_ = ws.Close()
 		return false, 0, nil, fmt.Errorf("websocket frame too large: %d", length)
 	}
-	var mask [4]byte
-	if header[1]&0x80 != 0 {
-		if _, err := io.ReadFull(ws.reader, mask[:]); err != nil {
-			_ = ws.Close()
-			return false, 0, nil, err
-		}
-	}
 	payload := make([]byte, length)
 	if _, err := io.ReadFull(ws.reader, payload); err != nil {
 		_ = ws.Close()
 		return false, 0, nil, err
 	}
-	if header[1]&0x80 != 0 {
-		for i := range payload {
-			payload[i] ^= mask[i%len(mask)]
-		}
-	}
 	return fin, opcode, payload, nil
 }
 
diff --git a/tests/pkg/client/websocket_test.go b/tests/pkg/client/websocket_test.go
index 676002d..e933430 100644
--- a/tests/pkg/client/websocket_test.go
+++ b/tests/pkg/client/websocket_test.go
@@ -7,6 +7,7 @@ import (
 	"net/http"
 	"net/http/httptest"
 	"net/url"
+	"strings"
 	"testing"
 	"time"
 )
@@ -47,8 +48,7 @@ func TestWebSocketPreservesFrameBufferedWithUpgrade(t *testing.T) {
 
 func pipeWebSocket(t *testing.T, frames ...[]byte) *WebSocket {
 	t.Helper()
-	clientConn, serverConn := net.Pipe()
-	t.Cleanup(func() { _ = clientConn.Close(); _ = serverConn.Close() })
+	ws, serverConn := newPipeWebSocket(t)
 	go func() {
 		for _, frame := range frames {
 			_, _ = serverConn.Write(frame)
@@ -58,7 +58,26 @@ func pipeWebSocket(t *testing.T, frames ...[]byte) *WebSocket {
 			}
 		}
 	}()
-	return &WebSocket{conn: clientConn, reader: bufio.NewReader(clientConn)}
+	return ws
+}
+
+func newPipeWebSocket(t *testing.T) (*WebSocket, net.Conn) {
+	t.Helper()
+	clientConn, serverConn := net.Pipe()
+	t.Cleanup(func() { _ = clientConn.Close(); _ = serverConn.Close() })
+	return &WebSocket{conn: clientConn, reader: bufio.NewReader(clientConn)}, serverConn
+}
+
+func requirePeerClosed(t *testing.T, peer net.Conn) {
+	t.Helper()
+	_ = peer.SetWriteDeadline(time.Now().Add(100 * time.Millisecond))
+	_, err := peer.Write(serverFrame(true, 2, "probe"))
+	if err == nil {
+		t.Fatal("websocket peer remained writable")
+	}
+	if timeout, ok := err.(net.Error); ok && timeout.Timeout() {
+		t.Fatalf("websocket peer was not closed: %v", err)
+	}
 }
 
 func TestWebSocketSkipsPingBeforeData(t *testing.T) {
@@ -80,3 +99,42 @@ func TestWebSocketReassemblesFragmentsAcrossControlFrame(t *testing.T) {
 		t.Fatalf("payload=%q err=%v", payload, err)
 	}
 }
+
+func TestWebSocketTimeoutBetweenFragmentsClosesConnection(t *testing.T) {
+	ws, serverConn := newPipeWebSocket(t)
+	go func() { _, _ = serverConn.Write(serverFrame(false, 2, "partial")) }()
+	_, err := ws.ReadFrame(30 * time.Millisecond)
+	if timeout, ok := err.(net.Error); !ok || !timeout.Timeout() {
+		t.Fatalf("expected fragment timeout, got %v", err)
+	}
+	requirePeerClosed(t, serverConn)
+}
+
+func TestWebSocketRejectsMaskedServerFrame(t *testing.T) {
+	ws, serverConn := newPipeWebSocket(t)
+	masked := []byte{0x82, 0x81, 1, 2, 3, 4, 'x' ^ 1}
+	go func() { _, _ = serverConn.Write(masked) }()
+	_, err := ws.ReadFrame(time.Second)
+	if err == nil || !strings.Contains(err.Error(), "masked") {
+		t.Fatalf("expected masked-server error, got %v", err)
+	}
+	requirePeerClosed(t, serverConn)
+}
+
+func TestWebSocketRejectsInvalidControlFrames(t *testing.T) {
+	tests := map[string][]byte{
+		"fragmented": serverFrame(false, 9, "p"),
+		"oversized":  {0x89, 126, 0, 126},
+	}
+	for name, frame := range tests {
+		t.Run(name, func(t *testing.T) {
+			ws, serverConn := newPipeWebSocket(t)
+			go func() { _, _ = serverConn.Write(frame) }()
+			_, err := ws.ReadFrame(time.Second)
+			if err == nil || !strings.Contains(err.Error(), "control") {
+				t.Fatalf("expected control-frame error, got %v", err)
+			}
+			requirePeerClosed(t, serverConn)
+		})
+	}
+}
diff --git a/tests/reliability/redis_failover_test.go b/tests/reliability/redis_failover_test.go
index 330e53e..b33859c 100644
--- a/tests/reliability/redis_failover_test.go
+++ b/tests/reliability/redis_failover_test.go
@@ -120,6 +120,16 @@ func TestRL_RedisCircuitFastFailAndRecovery(t *testing.T) {
 // stopped, resuming Push must requeue before websocket delivery. Recovery then
 // permits the half-open probe, durable Unacked write, and at-least-once delivery.
 func TestRL_PushUnackedRequeuesUntilRedisRecovers(t *testing.T) {
+	// This black-box choreography is intentionally tied to the compose topology:
+	// one Push container owns the websocket and exposes the sole configured bvar
+	// endpoint. Multi-instance CI must provide a dedicated single-Push test stack.
+	pushEndpoint := reliabilitySinglePushEndpoint(t)
+	pushContainer := HTTP.Config().Infra.PushContainer
+	require.NotEmpty(t, pushContainer,
+		"pause-container Push test requires the websocket-owning container name")
+	require.GreaterOrEqual(t, HTTP.Config().Infra.PushRouteL1TTLSec, 15,
+		"Push reliability stack requires route L1 TTL >=15s; docker config uses 30s")
+
 	sender, recipient, convID := fixture.MakeFriends(t, HTTP)
 	ws, err := client.OpenWebSocket(HTTP.Config())
 	require.NoError(t, err)
@@ -141,17 +151,14 @@ func TestRL_PushUnackedRequeuesUntilRedisRecovers(t *testing.T) {
 		require.NoError(t, readErr)
 	}
 
-	// A successful end-to-end warm delivery deterministically fills this Push
-	// instance's two-second route L1 immediately before the outage choreography.
+	// A successful end-to-end delivery warms that instance's route L1 before it
+	// is paused. The test-only TTL keeps the route through the outage choreography.
 	warmMarker := "rl-unacked-warm-" + client.NewRequestID()
 	sendReliabilityMessage(t, sender, convID, warmMarker)
 	require.True(t, readWebSocketMarker(ws, warmMarker, 5*time.Second), "warm Push delivery missing")
-	pushEndpoints := reliabilityPushEndpoints(t)
-	persistBefore := reliabilitySumBVar(t, pushEndpoints, "push_unacked_persist_failure_total")
-	requeueBefore := reliabilitySumBVar(t, pushEndpoints, "push_message_requeue_total")
+	persistBefore := verify.BVar(t, pushEndpoint, "push_unacked_persist_failure_total")
+	requeueBefore := verify.BVar(t, pushEndpoint, "push_message_requeue_total")
 
-	pushContainer := HTTP.Config().Infra.PushContainer
-	require.NotEmpty(t, pushContainer)
 	requireDocker(t, "pause", pushContainer)
 	paused := true
 	redisStopped := false
@@ -172,8 +179,8 @@ func TestRL_PushUnackedRequeuesUntilRedisRecovers(t *testing.T) {
 	redisStopped = true
 	requireDocker(t, "unpause", pushContainer)
 	paused = false
-	requireBVarIncrease(t, pushEndpoints, "push_unacked_persist_failure_total", persistBefore, 5*time.Second)
-	requireBVarIncrease(t, pushEndpoints, "push_message_requeue_total", requeueBefore, 5*time.Second)
+	requireBVarIncrease(t, pushEndpoint, "push_unacked_persist_failure_total", persistBefore, 5*time.Second)
+	requireBVarIncrease(t, pushEndpoint, "push_message_requeue_total", requeueBefore, 5*time.Second)
 	if payload, readErr := ws.ReadFrame(500 * time.Millisecond); readErr == nil {
 		t.Fatalf("Push delivered before durable Unacked persistence: %x", payload)
 	} else if timeout, ok := readErr.(net.Error); !ok || !timeout.Timeout() {
@@ -217,11 +224,11 @@ func readWebSocketMarker(ws *client.WebSocket, marker string, timeout time.Durat
 	return false
 }
 
-func requireBVarIncrease(t testing.TB, endpoints []string, metric string, before int64, timeout time.Duration) {
+func requireBVarIncrease(t testing.TB, endpoint, metric string, before int64, timeout time.Duration) {
 	t.Helper()
 	deadline := time.Now().Add(timeout)
 	for time.Now().Before(deadline) {
-		if reliabilitySumBVar(t, endpoints, metric) > before {
+		if verify.BVar(t, endpoint, metric) > before {
 			return
 		}
 		time.Sleep(50 * time.Millisecond)
@@ -241,9 +248,12 @@ func reliabilityTransmiteEndpoints(t testing.TB) []string {
 	return reliabilityEndpoints(t, HTTP.Config().Infra.TransmiteVars)
 }
 
-func reliabilityPushEndpoints(t testing.TB) []string {
+func reliabilitySinglePushEndpoint(t testing.TB) string {
 	t.Helper()
-	return reliabilityEndpoints(t, HTTP.Config().Infra.PushVars)
+	endpoints := reliabilityEndpoints(t, HTTP.Config().Infra.PushVars)
+	require.Len(t, endpoints, 1,
+		"pause-container Push test requires one bvar endpoint for the websocket-owning container; use a dedicated single-instance Push test stack")
+	return endpoints[0]
 }
 
 func reliabilityEndpoints(t testing.TB, rawEndpoints string) []string {

From 71765c7fbfde0d7444b95a38dc148fac67c40726 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 15:15:17 +0800
Subject: [PATCH 20/47] fix(test): align read ack requests with message ids

---
 tests/func/message_test.go   | 12 ++++++------
 tests/func/scenarios_test.go |  6 +++---
 2 files changed, 9 insertions(+), 9 deletions(-)

diff --git a/tests/func/message_test.go b/tests/func/message_test.go
index 30e9b91..f99c8e7 100644
--- a/tests/func/message_test.go
+++ b/tests/func/message_test.go
@@ -446,12 +446,12 @@ func TestFN_MS_SelectByClientMsgId_NotFound(t *testing.T) {
 // FN-MS (untested) | P0 | UpdateReadAck 更新 last_read_msg_id
 func TestFN_MS_UpdateReadAck_Success(t *testing.T) {
 	alice, bob, convID := fixture.MakeFriends(t, HTTP)
-	_, seqID := fixture.SendTextMessage(t, alice, convID, "ack-test-msg")
+	messageID, seqID := fixture.SendTextMessage(t, alice, convID, "ack-test-msg")
 
 	req := &msg.UpdateReadAckReq{
 		RequestId:      client.NewRequestID(),
 		ConversationId: convID,
-		SeqId:          seqID,
+		MessageId:      uint64(messageID),
 	}
 	rsp := &msg.UpdateReadAckRsp{}
 	err := bob.DoAuth("/service/message/update_read_ack", req, rsp)
@@ -467,14 +467,14 @@ func TestFN_MS_UpdateReadAck_Success(t *testing.T) {
 // FN-MS (untested) | P0 | UpdateReadAck 幂等（重复 ACK 不回退）
 func TestFN_MS_UpdateReadAck_Idempotent(t *testing.T) {
 	alice, bob, convID := fixture.MakeFriends(t, HTTP)
-	_, seq1 := fixture.SendTextMessage(t, alice, convID, "ack-idempotent-1")
-	_, seq2 := fixture.SendTextMessage(t, alice, convID, "ack-idempotent-2")
+	messageID1, _ := fixture.SendTextMessage(t, alice, convID, "ack-idempotent-1")
+	messageID2, seq2 := fixture.SendTextMessage(t, alice, convID, "ack-idempotent-2")
 
 	// ACK 到 seq2
 	ackReq := &msg.UpdateReadAckReq{
 		RequestId:      client.NewRequestID(),
 		ConversationId: convID,
-		SeqId:          seq2,
+		MessageId:      uint64(messageID2),
 	}
 	require.NoError(t, bob.DoAuth("/service/message/update_read_ack", ackReq, &msg.UpdateReadAckRsp{}))
 
@@ -482,7 +482,7 @@ func TestFN_MS_UpdateReadAck_Idempotent(t *testing.T) {
 	ackReq2 := &msg.UpdateReadAckReq{
 		RequestId:      client.NewRequestID(),
 		ConversationId: convID,
-		SeqId:          seq1,
+		MessageId:      uint64(messageID1),
 	}
 	require.NoError(t, bob.DoAuth("/service/message/update_read_ack", ackReq2, &msg.UpdateReadAckRsp{}))
 
diff --git a/tests/func/scenarios_test.go b/tests/func/scenarios_test.go
index d80f852..a57e089 100644
--- a/tests/func/scenarios_test.go
+++ b/tests/func/scenarios_test.go
@@ -589,9 +589,9 @@ func TestScenario_UnreadCountConsistency(t *testing.T) {
 	a, b, convID := setupConv(t) // MakeFriends
 
 	// Step 1: a 发 3 条消息
-	var lastSeq uint64
+	var lastMessageID int64
 	for i := 0; i < 3; i++ {
-		_, lastSeq = sendMsg(t, a, convID, "sc09-unread-"+string(rune('0'+i)))
+		lastMessageID, _ = sendMsg(t, a, convID, "sc09-unread-"+string(rune('0'+i)))
 	}
 
 	// Step 2: b ListConversations，验证 unread_count=3
@@ -617,7 +617,7 @@ func TestScenario_UnreadCountConsistency(t *testing.T) {
 	ackReq := &msg.UpdateReadAckReq{
 		RequestId:      client.NewRequestID(),
 		ConversationId: convID,
-		SeqId:          lastSeq,
+		MessageId:      uint64(lastMessageID),
 	}
 	require.NoError(t, b.DoAuth("/service/message/update_read_ack", ackReq, &msg.UpdateReadAckRsp{}))
 

From 147abaac9aafe125a50a4d02bc4a98faf6df10e2 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 15:35:10 +0800
Subject: [PATCH 21/47] fix(test): generate protobufs before Go validation

---
 .github/workflows/ci.yml     | 12 ++++++++++++
 tests/func/message_test.go   | 36 +++++++++++++++++++++++-------------
 tests/func/scenarios_test.go | 15 ++++++++-------
 tests/pkg/verify/db.go       | 34 ++++++++++++++++++++++++++++++++++
 4 files changed, 77 insertions(+), 20 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index d63b199..59b922b 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -22,6 +22,12 @@ jobs:
       - uses: actions/setup-go@v5
         with:
           go-version: '1.24'
+      - name: Install Go protobuf generator
+        run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
+      - name: Generate Go protobuf
+        run: cd tests && make proto
+      - name: Download Go deps
+        run: cd tests && go mod download
       - name: Go vet
         run: cd tests && go vet ./...
       - name: Go fmt check
@@ -46,6 +52,8 @@ jobs:
         run: |
           sudo apt-get update
           sudo apt-get install -y protobuf-compiler netcat-openbsd
+      - name: Install Go protobuf generator
+        run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
       - name: Start full stack
         run: docker compose up -d --build
       - name: Wait for services
@@ -73,6 +81,8 @@ jobs:
         run: |
           sudo apt-get update
           sudo apt-get install -y protobuf-compiler netcat-openbsd
+      - name: Install Go protobuf generator
+        run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
       - name: Start full stack
         run: docker compose up -d --build
       - name: Wait for services
@@ -100,6 +110,8 @@ jobs:
         run: |
           sudo apt-get update
           sudo apt-get install -y protobuf-compiler netcat-openbsd
+      - name: Install Go protobuf generator
+        run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
       - name: Start full stack
         run: docker compose up -d --build
       - name: Wait for services
diff --git a/tests/func/message_test.go b/tests/func/message_test.go
index f99c8e7..e86f2aa 100644
--- a/tests/func/message_test.go
+++ b/tests/func/message_test.go
@@ -4,6 +4,7 @@ package func_test
 
 import (
 	"testing"
+	"time"
 
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
@@ -443,10 +444,13 @@ func TestFN_MS_SelectByClientMsgId_NotFound(t *testing.T) {
 	assert.Nil(t, rsp.Message, "不存在的 client_msg_id 应返回 nil message")
 }
 
-// FN-MS (untested) | P0 | UpdateReadAck 更新 last_read_msg_id
+// FN-MS | P0 | UpdateReadAck advances the delivery acknowledgement cursor.
 func TestFN_MS_UpdateReadAck_Success(t *testing.T) {
 	alice, bob, convID := fixture.MakeFriends(t, HTTP)
 	messageID, seqID := fixture.SendTextMessage(t, alice, convID, "ack-test-msg")
+	verifier := verify.NewDBVerifier(Cfg.Database.MySQLDSN)
+	defer verifier.Close()
+	verifier.WaitMessageExists(t, messageID, 10*time.Second)
 
 	req := &msg.UpdateReadAckReq{
 		RequestId:      client.NewRequestID(),
@@ -458,17 +462,19 @@ func TestFN_MS_UpdateReadAck_Success(t *testing.T) {
 	require.NoError(t, err)
 	require.True(t, rsp.Header.Success, "update_read_ack 失败: %s", rsp.Header.ErrorMessage)
 
-	// 直查 DB 验证 last_read_seq 更新
-	verifier := verify.NewDBVerifier(Cfg.Database.MySQLDSN)
-	defer verifier.Close()
-	verifier.LastReadSeq(t, bob.UserID, convID, seqID)
+	// 直查 DB 验证 last_ack_seq 更新。
+	verifier.LastAckSeq(t, bob.UserID, convID, seqID)
 }
 
-// FN-MS (untested) | P0 | UpdateReadAck 幂等（重复 ACK 不回退）
+// FN-MS | P0 | UpdateReadAck is idempotent and never moves backwards.
 func TestFN_MS_UpdateReadAck_Idempotent(t *testing.T) {
 	alice, bob, convID := fixture.MakeFriends(t, HTTP)
 	messageID1, _ := fixture.SendTextMessage(t, alice, convID, "ack-idempotent-1")
 	messageID2, seq2 := fixture.SendTextMessage(t, alice, convID, "ack-idempotent-2")
+	verifier := verify.NewDBVerifier(Cfg.Database.MySQLDSN)
+	defer verifier.Close()
+	verifier.WaitMessageExists(t, messageID1, 10*time.Second)
+	verifier.WaitMessageExists(t, messageID2, 10*time.Second)
 
 	// ACK 到 seq2
 	ackReq := &msg.UpdateReadAckReq{
@@ -476,18 +482,22 @@ func TestFN_MS_UpdateReadAck_Idempotent(t *testing.T) {
 		ConversationId: convID,
 		MessageId:      uint64(messageID2),
 	}
-	require.NoError(t, bob.DoAuth("/service/message/update_read_ack", ackReq, &msg.UpdateReadAckRsp{}))
+	ackRsp := &msg.UpdateReadAckRsp{}
+	require.NoError(t, bob.DoAuth("/service/message/update_read_ack", ackReq, ackRsp))
+	require.True(t, ackRsp.GetHeader().GetSuccess(),
+		"newer delivery ACK failed: %s", ackRsp.GetHeader().GetErrorMessage())
 
-	// 再 ACK 到 seq1（小于 seq2），last_read_seq 不应回退
+	// 再 ACK 较早消息，last_ack_seq 不应回退。
 	ackReq2 := &msg.UpdateReadAckReq{
 		RequestId:      client.NewRequestID(),
 		ConversationId: convID,
 		MessageId:      uint64(messageID1),
 	}
-	require.NoError(t, bob.DoAuth("/service/message/update_read_ack", ackReq2, &msg.UpdateReadAckRsp{}))
+	ackRsp2 := &msg.UpdateReadAckRsp{}
+	require.NoError(t, bob.DoAuth("/service/message/update_read_ack", ackReq2, ackRsp2))
+	require.True(t, ackRsp2.GetHeader().GetSuccess(),
+		"backward delivery ACK must be idempotent: %s", ackRsp2.GetHeader().GetErrorMessage())
 
-	// 直查 DB 验证 last_read_seq 仍为 seq2
-	verifier := verify.NewDBVerifier(Cfg.Database.MySQLDSN)
-	defer verifier.Close()
-	verifier.LastReadSeq(t, bob.UserID, convID, seq2)
+	// 直查 DB 验证 last_ack_seq 仍为 seq2。
+	verifier.LastAckSeq(t, bob.UserID, convID, seq2)
 }
diff --git a/tests/func/scenarios_test.go b/tests/func/scenarios_test.go
index a57e089..2ff5f08 100644
--- a/tests/func/scenarios_test.go
+++ b/tests/func/scenarios_test.go
@@ -582,16 +582,16 @@ func TestScenario_LargeGroupFanOut(t *testing.T) {
 
 // ---------------------------------------------------------------------------
 // Scenario 9: Unread Count Consistency（未读数跨服务一致性）
-// SC-09 | P0 | scenario | 未读数跨服务跨设备一致：发消息 unread+1 -> UpdateReadAck -> unread=0
+// SC-09 | P0 | scenario | 未读数跨服务跨设备一致：发消息 unread+1 -> MarkRead -> unread=0
 // ---------------------------------------------------------------------------
 
 func TestScenario_UnreadCountConsistency(t *testing.T) {
 	a, b, convID := setupConv(t) // MakeFriends
 
 	// Step 1: a 发 3 条消息
-	var lastMessageID int64
+	var lastSeq uint64
 	for i := 0; i < 3; i++ {
-		lastMessageID, _ = sendMsg(t, a, convID, "sc09-unread-"+string(rune('0'+i)))
+		_, lastSeq = sendMsg(t, a, convID, "sc09-unread-"+string(rune('0'+i)))
 	}
 
 	// Step 2: b ListConversations，验证 unread_count=3
@@ -613,13 +613,14 @@ func TestScenario_UnreadCountConsistency(t *testing.T) {
 	defer dbV.Close()
 	dbV.UnreadCount(t, b.UserID, convID, 3)
 
-	// Step 4: b UpdateReadAck（读到最后一条 seq）
-	ackReq := &msg.UpdateReadAckReq{
+	// Step 4: b MarkRead（读到最后一条 seq）
+	markReadReq := &conversation.MarkReadReq{
 		RequestId:      client.NewRequestID(),
 		ConversationId: convID,
-		MessageId:      uint64(lastMessageID),
+		LastReadSeq:    lastSeq,
 	}
-	require.NoError(t, b.DoAuth("/service/message/update_read_ack", ackReq, &msg.UpdateReadAckRsp{}))
+	require.NoError(t, b.DoAuth(
+		"/service/conversation/mark_read", markReadReq, &conversation.MarkReadRsp{}))
 
 	// Step 5: b 再次 ListConversations，unread_count=0
 	listRsp2 := &conversation.ListConversationsRsp{}
diff --git a/tests/pkg/verify/db.go b/tests/pkg/verify/db.go
index fcfec5d..b7cdbbb 100644
--- a/tests/pkg/verify/db.go
+++ b/tests/pkg/verify/db.go
@@ -4,6 +4,7 @@ import (
 	"database/sql"
 	"fmt"
 	"testing"
+	"time"
 
 	_ "github.com/go-sql-driver/mysql"
 )
@@ -50,6 +51,24 @@ func (v *DBVerifier) MessageExists(t testing.TB, messageID int64) {
 	}
 }
 
+// WaitMessageExists waits for the asynchronous MQ consumer to persist a message.
+func (v *DBVerifier) WaitMessageExists(t testing.TB, messageID int64, timeout time.Duration) {
+	t.Helper()
+	deadline := time.Now().Add(timeout)
+	for time.Now().Before(deadline) {
+		var count int
+		err := v.db.QueryRow("SELECT COUNT(*) FROM message WHERE message_id = ?", messageID).Scan(&count)
+		if err == nil && count == 1 {
+			return
+		}
+		if err != nil {
+			t.Fatalf("query message %d while waiting: %v", messageID, err)
+		}
+		time.Sleep(50 * time.Millisecond)
+	}
+	t.Fatalf("message %d was not persisted within %s", messageID, timeout)
+}
+
 // MessageCount 验证某会话 message 表记录数。
 // 注意：message 表用 session_id 列名存储 conversation_id。
 func (v *DBVerifier) MessageCount(t testing.TB, conversationID string, expected int) {
@@ -123,6 +142,21 @@ func (v *DBVerifier) LastReadSeq(t testing.TB, userID, conversationID string, ex
 	}
 }
 
+// LastAckSeq verifies the delivery acknowledgement cursor.
+func (v *DBVerifier) LastAckSeq(t testing.TB, userID, conversationID string, expected uint64) {
+	var seq uint64
+	err := v.db.QueryRow(
+		"SELECT last_ack_seq FROM conversation_member WHERE user_id = ? AND conversation_id = ?",
+		userID, conversationID,
+	).Scan(&seq)
+	if err != nil {
+		t.Fatalf("query last_ack_seq: %v", err)
+	}
+	if seq != expected {
+		t.Fatalf("last_ack_seq user=%s conv=%s 期望 %d，实际 %d", userID, conversationID, expected, seq)
+	}
+}
+
 // UnreadCount 计算并验证未读数 = max(message.seq_id) - last_read_seq。
 func (v *DBVerifier) UnreadCount(t testing.TB, userID, conversationID string, expected int) {
 	var lastReadSeq uint64

From 9232a5cc0d6613e554ccd644a42ad30fdd572e66 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 16:33:03 +0800
Subject: [PATCH 22/47] docs: plan PR review fixes

---
 .../plans/2026-07-16-pr57-review-fixes.md     | 180 ++++++++++++++++++
 .../2026-07-16-pr57-review-fixes-design.md    |  71 +++++++
 2 files changed, 251 insertions(+)
 create mode 100644 docs/superpowers/plans/2026-07-16-pr57-review-fixes.md
 create mode 100644 docs/superpowers/specs/2026-07-16-pr57-review-fixes-design.md

diff --git a/docs/superpowers/plans/2026-07-16-pr57-review-fixes.md b/docs/superpowers/plans/2026-07-16-pr57-review-fixes.md
new file mode 100644
index 0000000..c285363
--- /dev/null
+++ b/docs/superpowers/plans/2026-07-16-pr57-review-fixes.md
@@ -0,0 +1,180 @@
+# PR #57 Review Fixes Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Resolve all three requested-change threads with explicit cache fencing, an identity-stable Unacked pending ledger, and executable RL-05/PF-09 CI gates.
+
+**Architecture:** UserInfo cache fills expose committed/conflict/unavailable outcomes and publish L1 only when the outcome permits it. Unacked uses a ZSET of stable `user_seq` members plus a HASH of payloads, maintained atomically by Lua. Reliability and cache performance run as separate full-stack workflow jobs.
+
+**Tech Stack:** C++17, redis-plus-plus, Redis Cluster Lua, Go 1.24 tests, Docker Compose, GitHub Actions.
+
+## Global Constraints
+
+- Treat `seq_id`, `user_seq`, and `message_id` as distinct domains.
+- Do not implement issue #58 in this change.
+- Do not read or migrate the former `user_seq:payload` ZSET layout.
+- Use the new Go test framework and preserve default production rate limits.
+- Follow red-green-refactor and commit each task independently.
+
+---
+
+### Task 1: Make UserInfo generation fencing explicit
+
+**Files:**
+- Modify: `common/dao/data_redis.hpp`
+- Modify: `transmite/source/transmite_server.h`
+- Create: `common/test/test_user_info_generation_fence.cc`
+
+**Interfaces:**
+- Produce: `enum class GenerationWriteResult { Committed, Conflict, Unavailable }`.
+- Produce: `UserInfoCache::set_if_generation(...) -> GenerationWriteResult`.
+- Preserve: `batch_set(...) -> size_t`, counting only committed writes.
+
+- [ ] **Step 1: Write the failing policy test**
+
+Create a focused test that asserts `Committed` permits L1 publication,
+`Conflict` denies it, and `Unavailable` permits only the short-lived fallback.
+It must also assert that a generation-aware fill cannot treat `Conflict` as
+Redis unavailability.
+
+- [ ] **Step 2: Run the test and verify RED**
+
+Run the repository's existing C++ test compile command for
+`test_user_info_generation_fence.cc`. Expected: compilation fails because
+`GenerationWriteResult` and the publication policy do not exist.
+
+- [ ] **Step 3: Implement the three-state result**
+
+Return `Conflict` only when Lua executes and returns zero. Return `Unavailable`
+for a missing Redis client, `RedisCircuitOpen`, or another Redis exception.
+Update batch writes to count only `Committed` entries.
+
+- [ ] **Step 4: Enforce the policy in Transmite**
+
+Store the CAS result. Write L1 after `Committed`; skip L1 after `Conflict`;
+write the existing randomized 45-second L1 fallback only when generation could
+not be observed or the fenced write returns `Unavailable`. Always return the
+Identity response to the current caller.
+
+- [ ] **Step 5: Verify GREEN and regressions**
+
+Run the new test, the existing cache harness, C++ syntax compilation for
+`transmite_server.h`, and `git diff --check`. Expected: all exit zero.
+
+- [ ] **Step 6: Commit**
+
+Commit message: `fix(cache): fence UserInfo L1 publication`.
+
+### Task 2: Redesign Unacked as a stable pending ledger
+
+**Files:**
+- Modify: `common/dao/data_redis.hpp`
+- Create: `common/test/test_unacked_pending_ledger.cc`
+- Modify: `tests/func/cache_test.go` only if the new framework needs black-box coverage.
+
+**Interfaces:**
+- Preserve: `push`, `ack`, `peek_due`, and `bump_score` C++ signatures.
+- Change Redis ZSET member to decimal `user_seq` only.
+- Keep HASH field as decimal `user_seq` and value as `payload_b64`.
+
+- [ ] **Step 1: Write the failing real-Redis tests**
+
+Cover: pushing the same sequence with payload A then B leaves `ZCARD == 1` and
+returns B; ACK removes both keys' entries; a ZSET-only orphan and a HASH-only
+orphan are removed by due-read; bump updates only complete entries. Use the
+existing Redis Cluster test setup and unique uid/device keys.
+
+- [ ] **Step 2: Run the tests and verify RED**
+
+Expected failures: duplicate ZSET members for changed payload, payload parsing
+from the member string, and orphan entries remaining.
+
+- [ ] **Step 3: Replace push and ACK scripts**
+
+Push atomically executes `ZADD key score user_seq`, `HSET index user_seq payload`,
+and paired expiry. ACK atomically executes `ZREM key user_seq` and
+`HDEL index user_seq` without depending on payload contents.
+
+- [ ] **Step 4: Make due-read and bump atomic and self-healing**
+
+Implement due-read as Lua returning a flat sequence/payload array. For every due
+sequence, return it only if the HASH payload exists; otherwise remove the ZSET
+member. Remove HASH-only entries encountered by the bounded consistency pass.
+Bump scores only when the HASH contains the sequence; otherwise remove the ZSET
+member. Renew the paired TTL sample in mutating scripts.
+
+- [ ] **Step 5: Verify GREEN and regressions**
+
+Run the new real-Redis suite against the six-node cluster, the existing
+DeviceSet/OnlineRoute/Unacked cluster suite, RedisMutex syntax validation, and
+`git diff --check`. Expected: all exit zero and no orphan remains.
+
+- [ ] **Step 6: Commit**
+
+Commit message: `fix(push): make unacked retries identity-idempotent`.
+
+### Task 3: Wire RL-05 and PF-09 into CI
+
+**Files:**
+- Modify: `.github/workflows/ci.yml`
+- Modify: `docker-compose.yml`
+- Modify: `tests/Makefile`
+- Create: `tests/pkg/contracts/ci_gates_test.go`
+
+**Interfaces:**
+- Produce workflow job `reliability` invoking `make test-reliability`.
+- Produce scheduled workflow job `perf-cache` invoking `make test-perf-cache-gate`.
+- Consume Compose variables `TRANSMITE_RATE_LIMIT_USER_MAX` and
+  `TRANSMITE_RATE_LIMIT_SESSION_MAX` with defaults 600 and 3000.
+
+- [ ] **Step 1: Write failing workflow contract tests**
+
+Parse `.github/workflows/ci.yml` and `docker-compose.yml` from Go. Assert that
+RL-05 is invoked on pull requests, PF-09 invokes the non-skipping gate on a
+schedule, PF-09 supplies both high-limit variables, and Compose maps those
+variables to the Transmite flags while retaining defaults 600/3000.
+
+- [ ] **Step 2: Run contract tests and verify RED**
+
+Run `go test ./pkg/contracts -run TestCIGates -count=1`. Expected: failure because
+the jobs and Compose overrides are absent.
+
+- [ ] **Step 3: Add dedicated jobs and rate-limit configuration**
+
+Give each job checkout, Go/protoc setup, full-stack startup, service wait,
+protobuf generation, dependency download, its exact Make target, and
+`if: always()` teardown. PF-09 sets limits above its generated ten-second load;
+normal Compose startup uses 600/3000 defaults.
+
+- [ ] **Step 4: Verify workflow contracts and test discovery**
+
+Run the contract test, parse the workflow with a YAML parser, run `make -n
+test-reliability` and `make -n test-perf-cache-gate`, and compile the reliability
+and perf tagged packages. Expected: no skip-only target is used and all commands
+exit zero.
+
+- [ ] **Step 5: Commit**
+
+Commit message: `ci: enforce cache reliability and performance gates`.
+
+### Task 4: Final review and PR update
+
+**Files:**
+- Modify only files required by review findings.
+
+- [ ] **Step 1: Run branch verification**
+
+Run fresh protobuf generation, default Go tests, tagged vet/compile checks,
+targeted race tests, Redis Cluster harnesses, workflow contract tests, YAML
+parsing, and `git diff --check`.
+
+- [ ] **Step 2: Review the complete branch diff**
+
+Check requested-change compliance, concurrency correctness, Redis Cluster hash
+slot safety, failure handling, and test claims. Fix every Critical or Important
+finding and rerun its covering tests.
+
+- [ ] **Step 3: Push and update GitHub threads**
+
+Push the reviewed commits. Reply to each inline thread with the concrete fix and
+verification, then resolve only threads whose requested behavior is fully met.
diff --git a/docs/superpowers/specs/2026-07-16-pr57-review-fixes-design.md b/docs/superpowers/specs/2026-07-16-pr57-review-fixes-design.md
new file mode 100644
index 0000000..055c304
--- /dev/null
+++ b/docs/superpowers/specs/2026-07-16-pr57-review-fixes-design.md
@@ -0,0 +1,71 @@
+# PR #57 Review Fixes Design
+
+## Scope
+
+Address the three unresolved requested-change threads on PR #57 without
+expanding into the delivery-ACK redesign tracked by issue #58:
+
+1. prevent a failed UserInfo generation CAS from publishing stale data to L1;
+2. make per-device Unacked storage idempotent by identity rather than payload;
+3. turn RL-05 and PF-09 into real CI gates.
+
+The project is pre-release. The Unacked Redis layout may change without a
+compatibility reader or migration.
+
+## UserInfo generation fencing
+
+`UserInfoCache::set_if_generation` returns a three-state result:
+
+- `Committed`: Redis generation matched and L2 was written;
+- `Conflict`: Redis was reachable but the generation no longer matched;
+- `Unavailable`: no Redis client, an open circuit, or a Redis exception.
+
+Transmite may publish an Identity result to L1 after `Committed`. It must not
+publish after `Conflict`. After `Unavailable`, it may publish only to the
+existing short-lived L1 fallback so singleflight followers share the successful
+Identity response while Redis is down. The current request may return its
+Identity result in all three states.
+
+## Unacked Pending Ledger
+
+The two Redis keys have separate responsibilities and share a cluster hash tag:
+
+```text
+ZSET im:unack:{uid:device}      member=user_seq, score=next_retry_at
+HASH im:unack:idx:{uid:device}  field=user_seq, value=payload_b64
+```
+
+Lua scripts atomically maintain both keys:
+
+- push uses `ZADD` and `HSET`; retrying the same `user_seq` replaces its score
+  and payload without creating another member;
+- peek selects due sequence IDs and returns payloads from the HASH in one
+  script; incomplete entries are removed from whichever side remains;
+- bump updates scores only for IDs that still have payloads and removes ZSET
+  orphans;
+- ack always removes the sequence ID from both structures.
+
+There is no parser or migration for the former `user_seq:payload` member format.
+
+## CI gates
+
+RL-05 runs in a dedicated `reliability` job for pull requests and schedules.
+PF-09 runs in a dedicated scheduled `perf-cache` job through
+`make test-perf-cache-gate`. The Compose Transmite command accepts environment
+overrides for user and session rate limits; PF-09 supplies limits above its
+generated load while normal Compose defaults remain 600 and 3000 per minute.
+Each full-stack job owns setup and `if: always()` teardown.
+
+## Verification
+
+- generation conflict, committed, and unavailable policy tests;
+- real Redis Unacked overwrite, due-read, orphan repair, bump, and ACK tests;
+- workflow/Compose contract tests proving both targets and rate-limit overrides
+  are wired;
+- existing Go vet/compile checks and the cache/reliability helper suites.
+
+## Non-goals
+
+- no compatibility with old Unacked Redis data;
+- no change to `last_read_seq`, `last_ack_seq`, or issue #58;
+- no Redis Streams, new broker, or additional cache tier.

From 794facf2921f628342e5049854caa6c62c848eb6 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 16:38:36 +0800
Subject: [PATCH 23/47] fix(cache): fence UserInfo L1 publication

---
 common/dao/data_redis.hpp                     | 45 +++++++++++++++----
 .../test/test_user_info_generation_fence.cc   | 24 ++++++++++
 transmite/source/transmite_server.h           | 13 +++---
 3 files changed, 69 insertions(+), 13 deletions(-)
 create mode 100644 common/test/test_user_info_generation_fence.cc

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 8e2868f..026246b 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -1129,6 +1129,31 @@ class Members
 // 用户资料缓存（value 为序列化 UserInfo protobuf；DAO 不依赖 protobuf 类型）
 // =============================================================================
 
+enum class GenerationWriteResult {
+    Committed,
+    Conflict,
+    Unavailable,
+};
+
+enum class UserInfoL1Publication {
+    GenerationFenced,
+    Denied,
+    ShortLivedFallback,
+};
+
+constexpr UserInfoL1Publication
+user_info_l1_publication(GenerationWriteResult result) noexcept {
+    switch (result) {
+    case GenerationWriteResult::Committed:
+        return UserInfoL1Publication::GenerationFenced;
+    case GenerationWriteResult::Conflict:
+        return UserInfoL1Publication::Denied;
+    case GenerationWriteResult::Unavailable:
+        return UserInfoL1Publication::ShortLivedFallback;
+    }
+    return UserInfoL1Publication::Denied;
+}
+
 class UserInfoCache
 {
 public:
@@ -1212,7 +1237,8 @@ class UserInfoCache
 
     bool set(const std::string &uid, const std::string &serialized,
              uint64_t observed_generation) {
-        return set_if_generation(uid, serialized, observed_generation);
+        return set_if_generation(uid, serialized, observed_generation) ==
+               GenerationWriteResult::Committed;
     }
 
     std::optional<uint64_t> generation(const std::string &uid) {
@@ -1227,9 +1253,10 @@ class UserInfoCache
         }
     }
 
-    bool set_if_generation(const std::string &uid, const std::string &serialized,
-                           uint64_t expected_generation) {
-        if (!_c) return false;
+    GenerationWriteResult set_if_generation(const std::string &uid,
+                                            const std::string &serialized,
+                                            uint64_t expected_generation) {
+        if (!_c) return GenerationWriteResult::Unavailable;
         const auto ttl = serialized.empty()
             ? randomized_ttl(std::chrono::seconds(5))
             : randomized_ttl(kUserInfoTtl);
@@ -1239,12 +1266,14 @@ class UserInfoCache
             std::vector<std::string> args = {
                 std::to_string(expected_generation), serialized,
                 std::to_string(ttl.count())};
-            return _c->eval<long long>(kSetIfGenerationLua, keys.begin(), keys.end(),
-                                       args.begin(), args.end()) == 1;
+            const auto result = _c->eval<long long>(
+                kSetIfGenerationLua, keys.begin(), keys.end(), args.begin(), args.end());
+            return result == 0 ? GenerationWriteResult::Conflict
+                               : GenerationWriteResult::Committed;
         } catch (const RedisCircuitOpen &) {
-            return false;
+            return GenerationWriteResult::Unavailable;
         } catch (const std::exception &) {
-            return false;
+            return GenerationWriteResult::Unavailable;
         }
     }
 
diff --git a/common/test/test_user_info_generation_fence.cc b/common/test/test_user_info_generation_fence.cc
new file mode 100644
index 0000000..8683c15
--- /dev/null
+++ b/common/test/test_user_info_generation_fence.cc
@@ -0,0 +1,24 @@
+#include "dao/data_redis.hpp"
+
+#include <cassert>
+#include <iostream>
+
+int main() {
+    using chatnow::GenerationWriteResult;
+    using chatnow::UserInfoL1Publication;
+
+    assert(chatnow::user_info_l1_publication(GenerationWriteResult::Committed) ==
+           UserInfoL1Publication::GenerationFenced);
+    assert(chatnow::user_info_l1_publication(GenerationWriteResult::Conflict) ==
+           UserInfoL1Publication::Denied);
+    assert(chatnow::user_info_l1_publication(GenerationWriteResult::Unavailable) ==
+           UserInfoL1Publication::ShortLivedFallback);
+
+    // A generation-aware fill must never collapse a real generation conflict
+    // into the Redis-unavailable fallback path.
+    assert(chatnow::user_info_l1_publication(GenerationWriteResult::Conflict) !=
+           chatnow::user_info_l1_publication(GenerationWriteResult::Unavailable));
+
+    std::cout << "user info generation fence policy tests passed\n";
+    return 0;
+}
diff --git a/transmite/source/transmite_server.h b/transmite/source/transmite_server.h
index dde3f28..e3d43f3 100644
--- a/transmite/source/transmite_server.h
+++ b/transmite/source/transmite_server.h
@@ -605,13 +605,16 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
                 return std::nullopt;
             }
             auto bytes = info->SerializeAsString();
+            auto publication = UserInfoL1Publication::ShortLivedFallback;
             if (generation && _user_info_cache) {
-                _user_info_cache->set_if_generation(uid, bytes, *generation);
+                const auto write_result =
+                    _user_info_cache->set_if_generation(uid, bytes, *generation);
+                publication = user_info_l1_publication(write_result);
             }
-            // L1 is deliberately short-lived and remains available even when the
-            // Redis generation read failed. This lets singleflight followers share
-            // a successful Identity result without allowing an unfenced L2 write.
-            if (_local_user_cache) {
+            // A real generation conflict means invalidation won the race, so the
+            // stale Identity response is returned only to its current caller. If
+            // Redis was unavailable, retain the short-lived availability fallback.
+            if (_local_user_cache && publication != UserInfoL1Publication::Denied) {
                 _local_user_cache->set(
                     lkey, bytes, randomized_ttl(std::chrono::seconds(45)));
             }

From 6e1e15b560ea6f22d2206aeda609d8648d03bae4 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 16:48:54 +0800
Subject: [PATCH 24/47] fix(cache): exercise UserInfo fence execution seams

---
 common/dao/data_redis.hpp                     | 62 +++++++++++------
 .../test/test_user_info_generation_fence.cc   | 69 +++++++++++++++----
 transmite/source/transmite_server.h           | 12 ++--
 3 files changed, 104 insertions(+), 39 deletions(-)

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 026246b..8d9d2d4 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -21,6 +21,7 @@
 #include <sw/redis++/redis_cluster.h>
 #include <chrono>
 #include <algorithm>
+#include <functional>
 #include <memory>
 #include <optional>
 #include <sstream>
@@ -1154,10 +1155,33 @@ user_info_l1_publication(GenerationWriteResult result) noexcept {
     return UserInfoL1Publication::Denied;
 }
 
+template <typename Eval>
+GenerationWriteResult execute_generation_write(bool redis_available,
+                                               Eval &&eval) noexcept {
+    if (!redis_available) return GenerationWriteResult::Unavailable;
+    try {
+        return std::forward<Eval>(eval)() == 0
+            ? GenerationWriteResult::Conflict
+            : GenerationWriteResult::Committed;
+    } catch (const RedisCircuitOpen &) {
+        return GenerationWriteResult::Unavailable;
+    } catch (const std::exception &) {
+        return GenerationWriteResult::Unavailable;
+    }
+}
+
+template <typename Publish>
+void publish_user_info_l1(GenerationWriteResult result, Publish &&publish) {
+    const auto publication = user_info_l1_publication(result);
+    if (publication == UserInfoL1Publication::Denied) return;
+    std::forward<Publish>(publish)(publication);
+}
+
 class UserInfoCache
 {
 public:
     using ptr = std::shared_ptr<UserInfoCache>;
+    using GenerationEval = std::function<long long()>;
 
     struct BatchResult {
         std::unordered_map<std::string, std::string> hits;
@@ -1172,7 +1196,9 @@ class UserInfoCache
         uint64_t observed_generation;
     };
 
-    explicit UserInfoCache(const RedisClient::ptr &c) : _c(c) {}
+    explicit UserInfoCache(const RedisClient::ptr &c,
+                           GenerationEval generation_eval = {})
+        : _c(c), _generation_eval(std::move(generation_eval)) {}
 
     std::optional<std::string> get(const std::string &uid) {
         if (!_c) return std::nullopt;
@@ -1256,25 +1282,20 @@ class UserInfoCache
     GenerationWriteResult set_if_generation(const std::string &uid,
                                             const std::string &serialized,
                                             uint64_t expected_generation) {
-        if (!_c) return GenerationWriteResult::Unavailable;
-        const auto ttl = serialized.empty()
-            ? randomized_ttl(std::chrono::seconds(5))
-            : randomized_ttl(kUserInfoTtl);
-        try {
-            std::vector<std::string> keys = {
-                key::user_info_key(uid), key::user_info_generation_key(uid)};
-            std::vector<std::string> args = {
-                std::to_string(expected_generation), serialized,
-                std::to_string(ttl.count())};
-            const auto result = _c->eval<long long>(
-                kSetIfGenerationLua, keys.begin(), keys.end(), args.begin(), args.end());
-            return result == 0 ? GenerationWriteResult::Conflict
-                               : GenerationWriteResult::Committed;
-        } catch (const RedisCircuitOpen &) {
-            return GenerationWriteResult::Unavailable;
-        } catch (const std::exception &) {
-            return GenerationWriteResult::Unavailable;
-        }
+        return execute_generation_write(
+            static_cast<bool>(_c) || static_cast<bool>(_generation_eval), [&] {
+                const auto ttl = serialized.empty()
+                    ? randomized_ttl(std::chrono::seconds(5))
+                    : randomized_ttl(kUserInfoTtl);
+                std::vector<std::string> keys = {
+                    key::user_info_key(uid), key::user_info_generation_key(uid)};
+                std::vector<std::string> args = {
+                    std::to_string(expected_generation), serialized,
+                    std::to_string(ttl.count())};
+                if (_generation_eval) return _generation_eval();
+                return _c->eval<long long>(kSetIfGenerationLua, keys.begin(), keys.end(),
+                                           args.begin(), args.end());
+            });
     }
 
     size_t batch_set(const std::unordered_map<std::string, FencedValue> &values) {
@@ -1386,6 +1407,7 @@ end
 return written
 )lua";
     RedisClient::ptr _c;
+    GenerationEval _generation_eval;
 };
 
 // =============================================================================
diff --git a/common/test/test_user_info_generation_fence.cc b/common/test/test_user_info_generation_fence.cc
index 8683c15..f0cbdca 100644
--- a/common/test/test_user_info_generation_fence.cc
+++ b/common/test/test_user_info_generation_fence.cc
@@ -1,24 +1,67 @@
 #include "dao/data_redis.hpp"
 
-#include <cassert>
 #include <iostream>
+#include <stdexcept>
 
 int main() {
     using chatnow::GenerationWriteResult;
     using chatnow::UserInfoL1Publication;
 
-    assert(chatnow::user_info_l1_publication(GenerationWriteResult::Committed) ==
-           UserInfoL1Publication::GenerationFenced);
-    assert(chatnow::user_info_l1_publication(GenerationWriteResult::Conflict) ==
-           UserInfoL1Publication::Denied);
-    assert(chatnow::user_info_l1_publication(GenerationWriteResult::Unavailable) ==
-           UserInfoL1Publication::ShortLivedFallback);
+    int failures = 0;
+    auto expect = [&](bool condition, const char *message) {
+        if (condition) return;
+        std::cerr << "FAIL: " << message << '\n';
+        ++failures;
+    };
 
-    // A generation-aware fill must never collapse a real generation conflict
-    // into the Redis-unavailable fallback path.
-    assert(chatnow::user_info_l1_publication(GenerationWriteResult::Conflict) !=
-           chatnow::user_info_l1_publication(GenerationWriteResult::Unavailable));
+    chatnow::UserInfoCache missing_client(nullptr);
+    expect(missing_client.set_if_generation("u1", "bytes", 0) ==
+               GenerationWriteResult::Unavailable,
+           "UserInfoCache must report a missing client as unavailable");
+    chatnow::UserInfoCache lua_zero(nullptr, [] { return 0LL; });
+    expect(lua_zero.set_if_generation("u1", "bytes", 0) ==
+               GenerationWriteResult::Conflict,
+           "Lua zero must be a generation conflict");
+    chatnow::UserInfoCache lua_one(nullptr, [] { return 1LL; });
+    expect(lua_one.set_if_generation("u1", "bytes", 0) ==
+               GenerationWriteResult::Committed,
+           "Lua one must be committed");
+    chatnow::UserInfoCache circuit_open(nullptr, []() -> long long {
+        throw chatnow::RedisCircuitOpen();
+    });
+    expect(circuit_open.set_if_generation("u1", "bytes", 0) ==
+               GenerationWriteResult::Unavailable,
+           "open Redis circuit must be unavailable");
+    chatnow::UserInfoCache redis_error(nullptr, []() -> long long {
+        throw std::runtime_error("redis unavailable");
+    });
+    expect(redis_error.set_if_generation("u1", "bytes", 0) ==
+               GenerationWriteResult::Unavailable,
+           "Redis exception must be unavailable");
 
-    std::cout << "user info generation fence policy tests passed\n";
-    return 0;
+    auto observe_publication = [&](GenerationWriteResult result) {
+        int calls = 0;
+        UserInfoL1Publication observed = UserInfoL1Publication::Denied;
+        chatnow::publish_user_info_l1(result, [&](UserInfoL1Publication publication) {
+            ++calls;
+            observed = publication;
+        });
+        return std::pair<int, UserInfoL1Publication>{calls, observed};
+    };
+    const auto committed = observe_publication(GenerationWriteResult::Committed);
+    expect(committed.first == 1 &&
+               committed.second == UserInfoL1Publication::GenerationFenced,
+           "committed write must call the generation-fenced L1 publisher");
+    const auto conflict = observe_publication(GenerationWriteResult::Conflict);
+    expect(conflict.first == 0,
+           "generation conflict must not call the L1 publisher");
+    const auto unavailable = observe_publication(GenerationWriteResult::Unavailable);
+    expect(unavailable.first == 1 &&
+               unavailable.second == UserInfoL1Publication::ShortLivedFallback,
+           "unavailable Redis must call only the short-lived L1 fallback");
+
+    if (failures == 0) {
+        std::cout << "user info generation fence execution tests passed\n";
+    }
+    return failures == 0 ? 0 : 1;
 }
diff --git a/transmite/source/transmite_server.h b/transmite/source/transmite_server.h
index e3d43f3..86130e7 100644
--- a/transmite/source/transmite_server.h
+++ b/transmite/source/transmite_server.h
@@ -605,19 +605,19 @@ class TransmiteServiceImpl : public chatnow::transmite::MsgTransmitService
                 return std::nullopt;
             }
             auto bytes = info->SerializeAsString();
-            auto publication = UserInfoL1Publication::ShortLivedFallback;
+            auto write_result = GenerationWriteResult::Unavailable;
             if (generation && _user_info_cache) {
-                const auto write_result =
-                    _user_info_cache->set_if_generation(uid, bytes, *generation);
-                publication = user_info_l1_publication(write_result);
+                write_result = _user_info_cache->set_if_generation(
+                    uid, bytes, *generation);
             }
             // A real generation conflict means invalidation won the race, so the
             // stale Identity response is returned only to its current caller. If
             // Redis was unavailable, retain the short-lived availability fallback.
-            if (_local_user_cache && publication != UserInfoL1Publication::Denied) {
+            publish_user_info_l1(write_result, [&](UserInfoL1Publication) {
+                if (!_local_user_cache) return;
                 _local_user_cache->set(
                     lkey, bytes, randomized_ttl(std::chrono::seconds(45)));
-            }
+            });
             return bytes;
         };
 

From 1e0657c45f33ecfbf91920e818ab25ee4056f2fe Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 17:04:35 +0800
Subject: [PATCH 25/47] fix(push): make unacked retries identity-idempotent

---
 common/dao/data_redis.hpp                  |  77 +++++++----
 common/test/CMakeLists.txt                 |   7 +
 common/test/test_unacked_pending_ledger.cc | 146 +++++++++++++++++++++
 3 files changed, 208 insertions(+), 22 deletions(-)
 create mode 100644 common/test/test_unacked_pending_ledger.cc

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 8d9d2d4..0cde8fd 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -1825,11 +1825,10 @@ class UnackedPush
               long long score_ts, std::chrono::seconds ttl = kUnackedTtl) {
         std::string k = key_for(uid, device_id);
         std::string ik = idx_key_for(uid, device_id);
-        std::string member = std::to_string(user_seq) + ":" + payload_b64;
         const auto effective_ttl = randomized_ttl(ttl);
         std::vector<std::string> keys = {k, ik};
         std::vector<std::string> args = {
-            std::to_string(score_ts), member, std::to_string(user_seq), payload_b64,
+            std::to_string(score_ts), std::to_string(user_seq), payload_b64,
             std::to_string(effective_ttl.count())};
         _c->eval<long long>(kPushLua, keys.begin(), keys.end(),
                             args.begin(), args.end());
@@ -1856,19 +1855,16 @@ class UnackedPush
         if(limit <= 0) return res;
         try {
             std::string k = key_for(uid, device_id);
+            std::string ik = idx_key_for(uid, device_id);
             long long now = static_cast<long long>(time(nullptr));
-            using namespace sw::redis;
             std::vector<std::string> raw;
-            _c->zrangebyscore(k,
-                              BoundedInterval<double>(0, static_cast<double>(now - max_age_sec),
-                                                       BoundType::CLOSED),
-                              LimitOptions{0, limit},
-                              std::back_inserter(raw));
-            for (const auto &s : raw) {
-                auto pos = s.find(':');
-                if (pos == std::string::npos) continue;
-                unsigned long seq = std::stoull(s.substr(0, pos));
-                res.emplace_back(seq, s.substr(pos + 1));
+            std::vector<std::string> keys = {k, ik};
+            std::vector<std::string> args = {
+                std::to_string(now - max_age_sec), std::to_string(limit)};
+            _c->eval(kPeekDueLua, keys.begin(), keys.end(),
+                     args.begin(), args.end(), std::back_inserter(raw));
+            for (size_t i = 0; i + 1 < raw.size(); i += 2) {
+                res.emplace_back(std::stoull(raw[i]), std::move(raw[i + 1]));
             }
         } catch(std::exception &e) {
             LOG_ERROR("UnackedPush.peek_due 失败 {}-{}-{}: {}", uid, device_id, e.what());
@@ -1907,17 +1903,52 @@ local function key_type(k)
     return t
 end
 local score = tonumber(ARGV[1])
-local ttl = tonumber(ARGV[5])
+local ttl = tonumber(ARGV[4])
 if not score or not ttl or ttl <= 0 then return redis.error_reply('invalid arguments') end
 local zt = key_type(KEYS[1])
 local ht = key_type(KEYS[2])
 if zt ~= 'none' and zt ~= 'zset' then return redis.error_reply('unacked key wrong type') end
 if ht ~= 'none' and ht ~= 'hash' then return redis.error_reply('unacked index wrong type') end
 redis.call('ZADD', KEYS[1], score, ARGV[2])
-redis.call('HSET', KEYS[2], ARGV[3], ARGV[4])
+redis.call('HSET', KEYS[2], ARGV[2], ARGV[3])
 redis.call('EXPIRE', KEYS[1], ttl)
 redis.call('EXPIRE', KEYS[2], ttl)
 return 1
+)lua";
+
+    static constexpr const char *kPeekDueLua = R"lua(
+local function key_type(k)
+    local t = redis.call('TYPE', k)
+    if type(t) == 'table' then return t.ok end
+    return t
+end
+local cutoff = tonumber(ARGV[1])
+local limit = tonumber(ARGV[2])
+if not cutoff or not limit or limit <= 0 then return redis.error_reply('invalid arguments') end
+local zt = key_type(KEYS[1])
+local ht = key_type(KEYS[2])
+if zt ~= 'none' and zt ~= 'zset' then return redis.error_reply('unacked key wrong type') end
+if ht ~= 'none' and ht ~= 'hash' then return redis.error_reply('unacked index wrong type') end
+local result = {}
+local due = redis.call('ZRANGEBYSCORE', KEYS[1], '-inf', cutoff, 'LIMIT', 0, limit)
+for _, seq in ipairs(due) do
+    local payload = redis.call('HGET', KEYS[2], seq)
+    if payload then
+        table.insert(result, seq)
+        table.insert(result, payload)
+    else
+        redis.call('ZREM', KEYS[1], seq)
+    end
+end
+local scan = redis.call('HSCAN', KEYS[2], 0, 'COUNT', limit)
+local fields = scan[2]
+for i = 1, #fields, 2 do
+    local seq = fields[i]
+    if not redis.call('ZSCORE', KEYS[1], seq) then
+        redis.call('HDEL', KEYS[2], seq)
+    end
+end
+return result
 )lua";
 
     static constexpr const char *kBumpScoreLua = R"lua(
@@ -1937,8 +1968,12 @@ local updated = 0
 for i = 3, #ARGV do
     local payload = redis.call('HGET', KEYS[2], ARGV[i])
     if payload then
-        redis.call('ZADD', KEYS[1], 'XX', score, ARGV[i] .. ':' .. payload)
-        updated = updated + 1
+        if redis.call('ZSCORE', KEYS[1], ARGV[i]) then
+            redis.call('ZADD', KEYS[1], 'XX', score, ARGV[i])
+            updated = updated + 1
+        end
+    else
+        redis.call('ZREM', KEYS[1], ARGV[i])
     end
 end
 redis.call('EXPIRE', KEYS[1], ttl)
@@ -1947,11 +1982,9 @@ return updated
 )lua";
 
     static constexpr const char *kAckLua = R"lua(
-local payload = redis.call('HGET', KEYS[2], ARGV[1])
-if not payload then return 0 end
-redis.call('ZREM', KEYS[1], ARGV[1] .. ':' .. payload)
-redis.call('HDEL', KEYS[2], ARGV[1])
-return 1
+local removed = redis.call('ZREM', KEYS[1], ARGV[1])
+removed = removed + redis.call('HDEL', KEYS[2], ARGV[1])
+return removed
 )lua";
 
     RedisClient::ptr _c;
diff --git a/common/test/CMakeLists.txt b/common/test/CMakeLists.txt
index 79e6ba4..0e5d6a4 100644
--- a/common/test/CMakeLists.txt
+++ b/common/test/CMakeLists.txt
@@ -88,6 +88,13 @@ if(CHATNOW_HAS_REDIS_PLUS_PLUS)
     set_property(TARGET test_redis_mutex_contract PROPERTY CXX_STANDARD 17)
     set_property(TARGET test_redis_mutex_contract PROPERTY CXX_STANDARD_REQUIRED ON)
     INSTALL(TARGETS test_redis_mutex_contract RUNTIME DESTINATION bin)
+
+    add_executable(test_unacked_pending_ledger test_unacked_pending_ledger.cc)
+    set_property(TARGET test_unacked_pending_ledger PROPERTY CXX_STANDARD 17)
+    set_property(TARGET test_unacked_pending_ledger PROPERTY CXX_STANDARD_REQUIRED ON)
+    target_link_libraries(test_unacked_pending_ledger
+        -lredis++ -lhiredis -lbrpc -lspdlog -lfmt -lpthread)
+    INSTALL(TARGETS test_unacked_pending_ledger RUNTIME DESTINATION bin)
 endif()
 
 # FIXME(3.0): common_tests has gflags link order issue with brpc static lib
diff --git a/common/test/test_unacked_pending_ledger.cc b/common/test/test_unacked_pending_ledger.cc
new file mode 100644
index 0000000..c57d93c
--- /dev/null
+++ b/common/test/test_unacked_pending_ledger.cc
@@ -0,0 +1,146 @@
+#include "dao/data_redis.hpp"
+
+#include <chrono>
+#include <cstdlib>
+#include <ctime>
+#include <iostream>
+#include <stdexcept>
+#include <string>
+#include <unistd.h>
+#include <vector>
+
+namespace {
+
+using chatnow::RedisClient;
+using chatnow::RedisClusterFactory;
+using chatnow::UnackedPush;
+
+void require(bool condition, const std::string &message) {
+    if (!condition) throw std::runtime_error(message);
+}
+
+std::string unique_component(const std::string &prefix) {
+    static unsigned long counter = 0;
+    return prefix + "-" + std::to_string(getpid()) + "-" +
+           std::to_string(static_cast<unsigned long>(std::time(nullptr))) + "-" +
+           std::to_string(++counter);
+}
+
+long long zcard(const RedisClient::ptr &redis, const std::string &key) {
+    static const std::string script = "return redis.call('ZCARD', KEYS[1])";
+    std::vector<std::string> keys = {key};
+    std::vector<std::string> args;
+    return redis->eval<long long>(script, keys.begin(), keys.end(),
+                                  args.begin(), args.end());
+}
+
+struct Fixture {
+    explicit Fixture(const RedisClient::ptr &client, std::string test_name)
+        : redis(client), ledger(client), uid(unique_component(std::move(test_name))),
+          device("device-" + std::to_string(getpid())),
+          pending_key(UnackedPush::key_for(uid, device)),
+          payload_key(UnackedPush::idx_key_for(uid, device)) {}
+
+    ~Fixture() {
+        try {
+            redis->del(pending_key);
+            redis->del(payload_key);
+        } catch (...) {
+        }
+    }
+
+    RedisClient::ptr redis;
+    UnackedPush ledger;
+    std::string uid;
+    std::string device;
+    std::string pending_key;
+    std::string payload_key;
+};
+
+void test_push_replaces_payload_without_changing_identity(const RedisClient::ptr &redis) {
+    Fixture fixture(redis, "push-replace");
+    const auto due_score = static_cast<long long>(std::time(nullptr)) - 60;
+
+    fixture.ledger.push(fixture.uid, fixture.device, 41, "payload-A", due_score);
+    fixture.ledger.push(fixture.uid, fixture.device, 41, "payload-B", due_score);
+
+    std::vector<std::string> members;
+    redis->zrange(fixture.pending_key, 0, -1, std::back_inserter(members));
+    require(zcard(redis, fixture.pending_key) == 1,
+            "re-push with changed payload must leave ZCARD == 1");
+    require(members.size() == 1, "the pending ZSET must contain one member");
+    require(members.front() == "41", "ZSET member must be the decimal user_seq only");
+
+    const auto due = fixture.ledger.peek_due(fixture.uid, fixture.device, 10, 0);
+    require(due.size() == 1, "re-pushed sequence must be returned exactly once");
+    require(due.front().first == 41 && due.front().second == "payload-B",
+            "due-read must return the replacement payload from the HASH");
+}
+
+void test_ack_removes_zset_and_hash_entries(const RedisClient::ptr &redis) {
+    Fixture fixture(redis, "ack-both");
+    fixture.ledger.push(fixture.uid, fixture.device, 52, "payload-A", 1);
+    fixture.ledger.push(fixture.uid, fixture.device, 52, "payload-B", 1);
+
+    fixture.ledger.ack(fixture.uid, fixture.device, 52);
+
+    std::vector<std::string> members;
+    redis->zrange(fixture.pending_key, 0, -1, std::back_inserter(members));
+    require(members.empty(),
+            "ACK must remove the ZSET member");
+    require(!redis->hget(fixture.payload_key, "52"),
+            "ACK must remove the HASH field");
+}
+
+void test_due_read_removes_both_orphan_directions(const RedisClient::ptr &redis) {
+    Fixture fixture(redis, "due-heal");
+    redis->zadd(fixture.pending_key, "61", 1);
+    redis->hset(fixture.payload_key, "62", "hash-only");
+
+    const auto due = fixture.ledger.peek_due(fixture.uid, fixture.device, 10, 0);
+
+    require(due.empty(), "orphan entries must never be returned as due");
+    std::vector<std::string> members;
+    redis->zrange(fixture.pending_key, 0, -1, std::back_inserter(members));
+    require(members.empty(),
+            "due-read must remove a ZSET-only orphan");
+    require(!redis->hget(fixture.payload_key, "62"),
+            "due-read bounded consistency pass must remove a HASH-only orphan");
+}
+
+void test_bump_updates_only_complete_entries(const RedisClient::ptr &redis) {
+    Fixture fixture(redis, "bump-complete");
+    fixture.ledger.push(fixture.uid, fixture.device, 71, "complete", 1);
+    redis->zadd(fixture.pending_key, "72", 1);
+
+    fixture.ledger.bump_score(fixture.uid, fixture.device, {71, 72});
+
+    const auto complete = fixture.ledger.peek_due(fixture.uid, fixture.device, 10, -1);
+    require(complete.size() == 1 && complete.front().first == 71,
+            "bump must retain and update the complete entry");
+    std::vector<std::string> members;
+    redis->zrange(fixture.pending_key, 0, -1, std::back_inserter(members));
+    require(members.size() == 1 && members.front() == "71",
+            "bump must remove a ZSET entry whose HASH payload is missing");
+}
+
+} // namespace
+
+int main() {
+    try {
+        const char *configured = std::getenv("CHATNOW_REDIS_CLUSTER_SEEDS");
+        const std::string seeds = configured ? configured : "redis-node1:6379";
+        auto cluster = RedisClusterFactory::create(seeds, 2);
+        auto redis = std::make_shared<RedisClient>(std::move(cluster));
+
+        test_push_replaces_payload_without_changing_identity(redis);
+        test_ack_removes_zset_and_hash_entries(redis);
+        test_due_read_removes_both_orphan_directions(redis);
+        test_bump_updates_only_complete_entries(redis);
+        std::cout << "unacked pending ledger cluster tests passed\n";
+        return 0;
+    } catch (const std::exception &error) {
+        std::cerr << "unacked pending ledger cluster test failed: " << error.what() << '\n';
+        return 1;
+    }
+}

From 04305d1e1f2b53c4594d89130949606e088feaf8 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 17:14:54 +0800
Subject: [PATCH 26/47] fix(push): harden unacked ledger repair

---
 common/dao/data_redis.hpp                  |  51 +++++++++-
 common/test/test_unacked_pending_ledger.cc | 107 +++++++++++++++++++--
 2 files changed, 146 insertions(+), 12 deletions(-)

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index 0cde8fd..ba24f4a 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -1818,6 +1818,11 @@ class UnackedPush
     static std::string idx_key_for(const std::string &uid, const std::string &device_id) {
         return std::string(key::kUnacked) + "idx:{" + uid + ":" + device_id + "}";
     }
+    // HSCAN continuation for bounded HASH-only orphan repair. All normal ledger
+    // mutations delete it, so the cursor is resumed only across stable read-heal passes.
+    static std::string repair_key_for(const std::string &uid, const std::string &device_id) {
+        return std::string(key::kUnacked) + "repair:{" + uid + ":" + device_id + "}";
+    }
 
     /* brief: 入待重传队列（per-device，存 payload_b64 直接用） */
     void push(const std::string &uid, const std::string &device_id,
@@ -1825,8 +1830,9 @@ class UnackedPush
               long long score_ts, std::chrono::seconds ttl = kUnackedTtl) {
         std::string k = key_for(uid, device_id);
         std::string ik = idx_key_for(uid, device_id);
+        std::string rk = repair_key_for(uid, device_id);
         const auto effective_ttl = randomized_ttl(ttl);
-        std::vector<std::string> keys = {k, ik};
+        std::vector<std::string> keys = {k, ik, rk};
         std::vector<std::string> args = {
             std::to_string(score_ts), std::to_string(user_seq), payload_b64,
             std::to_string(effective_ttl.count())};
@@ -1839,7 +1845,8 @@ class UnackedPush
         try {
             std::string k = key_for(uid, device_id);
             std::string ik = idx_key_for(uid, device_id);
-            std::vector<std::string> keys = {k, ik};
+            std::string rk = repair_key_for(uid, device_id);
+            std::vector<std::string> keys = {k, ik, rk};
             std::vector<std::string> args = {std::to_string(user_seq)};
             _c->eval<long long>(kAckLua, keys.begin(), keys.end(),
                                 args.begin(), args.end());
@@ -1856,9 +1863,10 @@ class UnackedPush
         try {
             std::string k = key_for(uid, device_id);
             std::string ik = idx_key_for(uid, device_id);
+            std::string rk = repair_key_for(uid, device_id);
             long long now = static_cast<long long>(time(nullptr));
             std::vector<std::string> raw;
-            std::vector<std::string> keys = {k, ik};
+            std::vector<std::string> keys = {k, ik, rk};
             std::vector<std::string> args = {
                 std::to_string(now - max_age_sec), std::to_string(limit)};
             _c->eval(kPeekDueLua, keys.begin(), keys.end(),
@@ -1879,9 +1887,10 @@ class UnackedPush
         try {
             std::string k = key_for(uid, device_id);
             std::string ik = idx_key_for(uid, device_id);
+            std::string rk = repair_key_for(uid, device_id);
             long long now = static_cast<long long>(time(nullptr));
             const auto effective_ttl = randomized_ttl(ttl);
-            std::vector<std::string> keys = {k, ik};
+            std::vector<std::string> keys = {k, ik, rk};
             std::vector<std::string> args = {
                 std::to_string(effective_ttl.count()), std::to_string(now)};
             args.reserve(2 + user_seqs.size());
@@ -1913,6 +1922,7 @@ redis.call('ZADD', KEYS[1], score, ARGV[2])
 redis.call('HSET', KEYS[2], ARGV[2], ARGV[3])
 redis.call('EXPIRE', KEYS[1], ttl)
 redis.call('EXPIRE', KEYS[2], ttl)
+redis.call('DEL', KEYS[3])
 return 1
 )lua";
 
@@ -1927,8 +1937,10 @@ local limit = tonumber(ARGV[2])
 if not cutoff or not limit or limit <= 0 then return redis.error_reply('invalid arguments') end
 local zt = key_type(KEYS[1])
 local ht = key_type(KEYS[2])
+local rt = key_type(KEYS[3])
 if zt ~= 'none' and zt ~= 'zset' then return redis.error_reply('unacked key wrong type') end
 if ht ~= 'none' and ht ~= 'hash' then return redis.error_reply('unacked index wrong type') end
+if rt ~= 'none' and rt ~= 'string' then return redis.error_reply('unacked repair key wrong type') end
 local result = {}
 local due = redis.call('ZRANGEBYSCORE', KEYS[1], '-inf', cutoff, 'LIMIT', 0, limit)
 for _, seq in ipairs(due) do
@@ -1940,7 +1952,8 @@ for _, seq in ipairs(due) do
         redis.call('ZREM', KEYS[1], seq)
     end
 end
-local scan = redis.call('HSCAN', KEYS[2], 0, 'COUNT', limit)
+local cursor = redis.call('GET', KEYS[3]) or '0'
+local scan = redis.call('HSCAN', KEYS[2], cursor, 'COUNT', limit)
 local fields = scan[2]
 for i = 1, #fields, 2 do
     local seq = fields[i]
@@ -1948,6 +1961,21 @@ for i = 1, #fields, 2 do
         redis.call('HDEL', KEYS[2], seq)
     end
 end
+local next_cursor = scan[1]
+if next_cursor == '0' then
+    redis.call('DEL', KEYS[3])
+else
+    -- The cursor describes the HASH iteration, so it may never outlive that HASH.
+    local httl = redis.call('PTTL', KEYS[2])
+    if httl > 0 then
+        redis.call('SET', KEYS[3], next_cursor, 'PX', httl)
+    elseif httl == -1 then
+        -- Corrupt persistent ledgers still make progress without leaking metadata forever.
+        redis.call('SET', KEYS[3], next_cursor, 'PX', 300000)
+    else
+        redis.call('DEL', KEYS[3])
+    end
+end
 return result
 )lua";
 
@@ -1978,12 +2006,25 @@ for i = 3, #ARGV do
 end
 redis.call('EXPIRE', KEYS[1], ttl)
 redis.call('EXPIRE', KEYS[2], ttl)
+-- Score/member mutation invalidates an in-progress HSCAN continuation.
+redis.call('DEL', KEYS[3])
 return updated
 )lua";
 
     static constexpr const char *kAckLua = R"lua(
+local function key_type(k)
+    local t = redis.call('TYPE', k)
+    if type(t) == 'table' then return t.ok end
+    return t
+end
+local zt = key_type(KEYS[1])
+local ht = key_type(KEYS[2])
+if zt ~= 'none' and zt ~= 'zset' then return redis.error_reply('unacked key wrong type') end
+if ht ~= 'none' and ht ~= 'hash' then return redis.error_reply('unacked index wrong type') end
 local removed = redis.call('ZREM', KEYS[1], ARGV[1])
 removed = removed + redis.call('HDEL', KEYS[2], ARGV[1])
+-- HASH mutation invalidates an in-progress HSCAN continuation.
+redis.call('DEL', KEYS[3])
 return removed
 )lua";
 
diff --git a/common/test/test_unacked_pending_ledger.cc b/common/test/test_unacked_pending_ledger.cc
index c57d93c..f03a552 100644
--- a/common/test/test_unacked_pending_ledger.cc
+++ b/common/test/test_unacked_pending_ledger.cc
@@ -7,6 +7,7 @@
 #include <stdexcept>
 #include <string>
 #include <unistd.h>
+#include <utility>
 #include <vector>
 
 namespace {
@@ -34,17 +35,36 @@ long long zcard(const RedisClient::ptr &redis, const std::string &key) {
                                   args.begin(), args.end());
 }
 
+std::string zscore(const RedisClient::ptr &redis, const std::string &key,
+                   const std::string &member) {
+    static const std::string script = "return redis.call('ZSCORE', KEYS[1], ARGV[1])";
+    std::vector<std::string> keys = {key};
+    std::vector<std::string> args = {member};
+    return redis->eval<std::string>(script, keys.begin(), keys.end(),
+                                    args.begin(), args.end());
+}
+
+long long ttl(const RedisClient::ptr &redis, const std::string &key) {
+    static const std::string script = "return redis.call('TTL', KEYS[1])";
+    std::vector<std::string> keys = {key};
+    std::vector<std::string> args;
+    return redis->eval<long long>(script, keys.begin(), keys.end(),
+                                  args.begin(), args.end());
+}
+
 struct Fixture {
     explicit Fixture(const RedisClient::ptr &client, std::string test_name)
         : redis(client), ledger(client), uid(unique_component(std::move(test_name))),
           device("device-" + std::to_string(getpid())),
           pending_key(UnackedPush::key_for(uid, device)),
-          payload_key(UnackedPush::idx_key_for(uid, device)) {}
+          payload_key(UnackedPush::idx_key_for(uid, device)),
+          repair_key(UnackedPush::repair_key_for(uid, device)) {}
 
     ~Fixture() {
         try {
             redis->del(pending_key);
             redis->del(payload_key);
+            redis->del(repair_key);
         } catch (...) {
         }
     }
@@ -55,6 +75,7 @@ struct Fixture {
     std::string device;
     std::string pending_key;
     std::string payload_key;
+    std::string repair_key;
 };
 
 void test_push_replaces_payload_without_changing_identity(const RedisClient::ptr &redis) {
@@ -108,12 +129,67 @@ void test_due_read_removes_both_orphan_directions(const RedisClient::ptr &redis)
             "due-read bounded consistency pass must remove a HASH-only orphan");
 }
 
+void test_ack_wrong_type_does_not_partially_remove_zset(const RedisClient::ptr &redis) {
+    Fixture fixture(redis, "ack-wrong-type");
+    fixture.ledger.push(fixture.uid, fixture.device, 63, "payload", 1);
+    redis->del(fixture.payload_key);
+    redis->set(fixture.payload_key, "not-a-hash", std::chrono::seconds(300));
+
+    fixture.ledger.ack(fixture.uid, fixture.device, 63);
+
+    require(zcard(redis, fixture.pending_key) == 1,
+            "ACK must validate both key types before removing the ZSET member");
+}
+
+void test_due_read_progresses_across_hash_orphan_pages(const RedisClient::ptr &redis) {
+    Fixture fixture(redis, "due-progressive-heal");
+    fixture.ledger.push(fixture.uid, fixture.device, 9000, "complete",
+                        static_cast<long long>(std::time(nullptr)) + 3600,
+                        std::chrono::seconds(300));
+    for (unsigned long seq = 10000; seq < 10700; ++seq) {
+        redis->hset(fixture.payload_key, std::to_string(seq), "hash-only");
+    }
+
+    fixture.ledger.peek_due(fixture.uid, fixture.device, 5, 0);
+    auto cursor = redis->get(fixture.repair_key);
+    require(cursor && *cursor != "0",
+            "a bounded HASH repair pass must persist its non-zero HSCAN cursor");
+    const auto repair_ttl = ttl(redis, fixture.repair_key);
+    const auto payload_ttl = ttl(redis, fixture.payload_key);
+    require(repair_ttl > 0 && repair_ttl <= payload_ttl,
+            "repair cursor must expire with, and never outlive, the pending ledger");
+
+    for (int pass = 0; pass < 1000; ++pass) {
+        if (redis->hlen(fixture.payload_key) == 1 && !redis->get(fixture.repair_key)) break;
+        fixture.ledger.peek_due(fixture.uid, fixture.device, 5, 0);
+    }
+    require(redis->hlen(fixture.payload_key) == 1,
+            "successive bounded due-reads must eventually remove every HASH-only orphan");
+    require(!redis->get(fixture.repair_key),
+            "repair cursor must be deleted after a full stable scan completes");
+}
+
 void test_bump_updates_only_complete_entries(const RedisClient::ptr &redis) {
     Fixture fixture(redis, "bump-complete");
     fixture.ledger.push(fixture.uid, fixture.device, 71, "complete", 1);
     redis->zadd(fixture.pending_key, "72", 1);
-
-    fixture.ledger.bump_score(fixture.uid, fixture.device, {71, 72});
+    redis->expire(fixture.pending_key, std::chrono::seconds(10));
+    redis->expire(fixture.payload_key, std::chrono::seconds(10));
+
+    const auto before = static_cast<long long>(std::time(nullptr));
+    fixture.ledger.bump_score(fixture.uid, fixture.device, {71, 72},
+                              std::chrono::seconds(300));
+    const auto after = static_cast<long long>(std::time(nullptr));
+
+    const auto bumped_score = std::stoll(zscore(redis, fixture.pending_key, "71"));
+    require(bumped_score >= before && bumped_score <= after,
+            "bump must set the complete entry score to the current time");
+    const auto pending_ttl = ttl(redis, fixture.pending_key);
+    const auto payload_ttl = ttl(redis, fixture.payload_key);
+    require(pending_ttl >= 200 && payload_ttl >= 200,
+            "bump must renew both ledger keys from their forced short TTL");
+    require(std::llabs(pending_ttl - payload_ttl) <= 1,
+            "bump must apply the same randomized TTL sample to both ledger keys");
 
     const auto complete = fixture.ledger.peek_due(fixture.uid, fixture.device, 10, -1);
     require(complete.size() == 1 && complete.front().first == 71,
@@ -133,10 +209,27 @@ int main() {
         auto cluster = RedisClusterFactory::create(seeds, 2);
         auto redis = std::make_shared<RedisClient>(std::move(cluster));
 
-        test_push_replaces_payload_without_changing_identity(redis);
-        test_ack_removes_zset_and_hash_entries(redis);
-        test_due_read_removes_both_orphan_directions(redis);
-        test_bump_updates_only_complete_entries(redis);
+        const std::vector<std::pair<std::string, void (*)(const RedisClient::ptr &)>> tests = {
+            {"push replaces payload", test_push_replaces_payload_without_changing_identity},
+            {"ack removes both", test_ack_removes_zset_and_hash_entries},
+            {"ack wrong type is atomic", test_ack_wrong_type_does_not_partially_remove_zset},
+            {"due-read heals both directions", test_due_read_removes_both_orphan_directions},
+            {"due-read repair is progressive", test_due_read_progresses_across_hash_orphan_pages},
+            {"bump updates complete entries", test_bump_updates_only_complete_entries},
+        };
+        int failures = 0;
+        for (const auto &[name, test] : tests) {
+            try {
+                test(redis);
+            } catch (const std::exception &error) {
+                ++failures;
+                std::cerr << "FAILED " << name << ": " << error.what() << '\n';
+            }
+        }
+        if (failures != 0) {
+            std::cerr << failures << " unacked pending ledger cluster test(s) failed\n";
+            return 1;
+        }
         std::cout << "unacked pending ledger cluster tests passed\n";
         return 0;
     } catch (const std::exception &error) {

From d335bdc7c5395c64150b2a02a68cc85670b286cd Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 17:22:30 +0800
Subject: [PATCH 27/47] fix(push): preserve unacked repair progress

---
 common/dao/data_redis.hpp                  | 27 +++++--
 common/test/test_unacked_pending_ledger.cc | 84 ++++++++++++++++++++++
 2 files changed, 104 insertions(+), 7 deletions(-)

diff --git a/common/dao/data_redis.hpp b/common/dao/data_redis.hpp
index ba24f4a..59a6027 100644
--- a/common/dao/data_redis.hpp
+++ b/common/dao/data_redis.hpp
@@ -1818,8 +1818,8 @@ class UnackedPush
     static std::string idx_key_for(const std::string &uid, const std::string &device_id) {
         return std::string(key::kUnacked) + "idx:{" + uid + ":" + device_id + "}";
     }
-    // HSCAN continuation for bounded HASH-only orphan repair. All normal ledger
-    // mutations delete it, so the cursor is resumed only across stable read-heal passes.
+    // HSCAN continuation for bounded HASH-only orphan repair. HASH mutations
+    // (push/ack) delete it; score-only bump preserves and renews it.
     static std::string repair_key_for(const std::string &uid, const std::string &device_id) {
         return std::string(key::kUnacked) + "repair:{" + uid + ":" + device_id + "}";
     }
@@ -1940,7 +1940,21 @@ local ht = key_type(KEYS[2])
 local rt = key_type(KEYS[3])
 if zt ~= 'none' and zt ~= 'zset' then return redis.error_reply('unacked key wrong type') end
 if ht ~= 'none' and ht ~= 'hash' then return redis.error_reply('unacked index wrong type') end
-if rt ~= 'none' and rt ~= 'string' then return redis.error_reply('unacked repair key wrong type') end
+local cursor = '0'
+if rt == 'string' then
+    cursor = redis.call('GET', KEYS[3]) or '0'
+    if not string.match(cursor, '^%d+$') then
+        redis.call('DEL', KEYS[3])
+        cursor = '0'
+    end
+elseif rt ~= 'none' then
+    redis.call('DEL', KEYS[3])
+end
+local scan = redis.pcall('HSCAN', KEYS[2], cursor, 'COUNT', limit)
+if type(scan) == 'table' and scan.err then
+    redis.call('DEL', KEYS[3])
+    scan = redis.call('HSCAN', KEYS[2], 0, 'COUNT', limit)
+end
 local result = {}
 local due = redis.call('ZRANGEBYSCORE', KEYS[1], '-inf', cutoff, 'LIMIT', 0, limit)
 for _, seq in ipairs(due) do
@@ -1952,8 +1966,6 @@ for _, seq in ipairs(due) do
         redis.call('ZREM', KEYS[1], seq)
     end
 end
-local cursor = redis.call('GET', KEYS[3]) or '0'
-local scan = redis.call('HSCAN', KEYS[2], cursor, 'COUNT', limit)
 local fields = scan[2]
 for i = 1, #fields, 2 do
     local seq = fields[i]
@@ -2006,8 +2018,9 @@ for i = 3, #ARGV do
 end
 redis.call('EXPIRE', KEYS[1], ttl)
 redis.call('EXPIRE', KEYS[2], ttl)
--- Score/member mutation invalidates an in-progress HSCAN continuation.
-redis.call('DEL', KEYS[3])
+if redis.call('EXISTS', KEYS[3]) == 1 then
+    redis.call('EXPIRE', KEYS[3], ttl)
+end
 return updated
 )lua";
 
diff --git a/common/test/test_unacked_pending_ledger.cc b/common/test/test_unacked_pending_ledger.cc
index f03a552..0bd1c94 100644
--- a/common/test/test_unacked_pending_ledger.cc
+++ b/common/test/test_unacked_pending_ledger.cc
@@ -169,6 +169,86 @@ void test_due_read_progresses_across_hash_orphan_pages(const RedisClient::ptr &r
             "repair cursor must be deleted after a full stable scan completes");
 }
 
+void test_repair_cursor_follows_push_bump_ack_mutation_policy(const RedisClient::ptr &redis) {
+    Fixture fixture(redis, "repair-mutation-policy");
+    fixture.ledger.push(fixture.uid, fixture.device, 73, "payload-A", 1,
+                        std::chrono::seconds(300));
+
+    redis->set(fixture.repair_key, "7", std::chrono::seconds(30));
+    fixture.ledger.bump_score(fixture.uid, fixture.device, {73},
+                              std::chrono::seconds(300));
+    auto cursor = redis->get(fixture.repair_key);
+    require(cursor && *cursor == "7",
+            "bump changes only ZSET scores and must preserve the HASH repair cursor");
+    require(std::llabs(ttl(redis, fixture.repair_key) - ttl(redis, fixture.payload_key)) <= 1,
+            "bump must renew an existing repair cursor with the HASH paired TTL sample");
+
+    fixture.ledger.push(fixture.uid, fixture.device, 73, "payload-B", 1,
+                        std::chrono::seconds(300));
+    require(!redis->get(fixture.repair_key),
+            "push mutates the HASH and must reset its repair cursor");
+
+    redis->set(fixture.repair_key, "9", std::chrono::seconds(30));
+    fixture.ledger.ack(fixture.uid, fixture.device, 73);
+    require(!redis->get(fixture.repair_key),
+            "ACK mutates the HASH and must reset its repair cursor");
+}
+
+void test_peek_bump_flow_keeps_multi_page_repair_progress(const RedisClient::ptr &redis) {
+    Fixture fixture(redis, "peek-bump-progress");
+    fixture.ledger.push(fixture.uid, fixture.device, 81, "due", 1,
+                        std::chrono::seconds(300));
+    for (unsigned long seq = 11000; seq < 11700; ++seq) {
+        redis->hset(fixture.payload_key, std::to_string(seq), "hash-only");
+    }
+
+    auto due = fixture.ledger.peek_due(fixture.uid, fixture.device, 5, 0);
+    require(due.size() == 1 && due.front().first == 81,
+            "first production-flow peek must return the due complete entry");
+    auto first_cursor = redis->get(fixture.repair_key);
+    require(first_cursor && *first_cursor != "0",
+            "first production-flow peek must begin progressive orphan repair");
+    fixture.ledger.bump_score(fixture.uid, fixture.device, {81},
+                              std::chrono::seconds(300));
+    require(redis->get(fixture.repair_key) == first_cursor,
+            "production-flow bump must not restart the HASH scan at cursor zero");
+
+    for (int pass = 0; pass < 1000; ++pass) {
+        if (redis->hlen(fixture.payload_key) == 1 && !redis->get(fixture.repair_key)) break;
+        due = fixture.ledger.peek_due(fixture.uid, fixture.device, 5, -1);
+        if (!due.empty()) {
+            fixture.ledger.bump_score(fixture.uid, fixture.device, {due.front().first},
+                                      std::chrono::seconds(300));
+        }
+    }
+    require(redis->hlen(fixture.payload_key) == 1,
+            "peek-then-bump production flow must clean every HASH-only orphan page");
+    require(!redis->get(fixture.repair_key),
+            "peek-then-bump production flow must finish and remove the repair cursor");
+}
+
+void test_wrong_type_repair_metadata_cannot_block_due_delivery(const RedisClient::ptr &redis) {
+    Fixture wrong_type(redis, "repair-wrong-type");
+    wrong_type.ledger.push(wrong_type.uid, wrong_type.device, 82, "due", 1);
+    redis->hset(wrong_type.repair_key, "bad", "metadata");
+    auto due = wrong_type.ledger.peek_due(wrong_type.uid, wrong_type.device, 5, 0);
+    require(due.size() == 1 && due.front().first == 82,
+            "wrong-type repair metadata must be discarded without blocking due delivery");
+    require(!redis->get(wrong_type.repair_key),
+            "wrong-type repair metadata must be removed after restarting from zero");
+}
+
+void test_malformed_repair_cursor_cannot_block_due_delivery(const RedisClient::ptr &redis) {
+    Fixture malformed(redis, "repair-malformed");
+    malformed.ledger.push(malformed.uid, malformed.device, 83, "due", 1);
+    redis->set(malformed.repair_key, "not-a-redis-cursor", std::chrono::seconds(300));
+    auto due = malformed.ledger.peek_due(malformed.uid, malformed.device, 5, 0);
+    require(due.size() == 1 && due.front().first == 83,
+            "malformed repair cursor must be discarded and restarted from zero");
+    require(!redis->get(malformed.repair_key),
+            "malformed repair cursor must be removed after restarting from zero");
+}
+
 void test_bump_updates_only_complete_entries(const RedisClient::ptr &redis) {
     Fixture fixture(redis, "bump-complete");
     fixture.ledger.push(fixture.uid, fixture.device, 71, "complete", 1);
@@ -215,6 +295,10 @@ int main() {
             {"ack wrong type is atomic", test_ack_wrong_type_does_not_partially_remove_zset},
             {"due-read heals both directions", test_due_read_removes_both_orphan_directions},
             {"due-read repair is progressive", test_due_read_progresses_across_hash_orphan_pages},
+            {"repair cursor mutation policy", test_repair_cursor_follows_push_bump_ack_mutation_policy},
+            {"peek-bump repair is progressive", test_peek_bump_flow_keeps_multi_page_repair_progress},
+            {"wrong-type repair metadata is fail-soft", test_wrong_type_repair_metadata_cannot_block_due_delivery},
+            {"malformed repair cursor is fail-soft", test_malformed_repair_cursor_cannot_block_due_delivery},
             {"bump updates complete entries", test_bump_updates_only_complete_entries},
         };
         int failures = 0;

From cc29e625631f156f4f33d706624174767e62d033 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 17:28:44 +0800
Subject: [PATCH 28/47] ci: enforce cache reliability and performance gates

---
 .github/workflows/ci.yml             |  59 +++++++++++
 docker-compose.yml                   |   2 +-
 tests/pkg/contracts/ci_gates_test.go | 144 +++++++++++++++++++++++++++
 3 files changed, 204 insertions(+), 1 deletion(-)
 create mode 100644 tests/pkg/contracts/ci_gates_test.go

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 59b922b..abb1fa1 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -97,6 +97,65 @@ jobs:
         if: always()
         run: docker compose down -v
 
+  reliability:
+    runs-on: ubuntu-22.04
+    if: github.event_name == 'pull_request' || github.event_name == 'schedule'
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-go@v5
+        with:
+          go-version: '1.24'
+      - name: Install dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y protobuf-compiler netcat-openbsd
+      - name: Install Go protobuf generator
+        run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
+      - name: Start full stack
+        run: docker compose up -d --build
+      - name: Wait for services
+        run: ./scripts/wait_for_services.sh
+      - name: Generate Go protobuf
+        run: cd tests && make proto
+      - name: Download Go deps
+        run: cd tests && go mod download
+      - name: Run cache reliability gate
+        run: cd tests && make test-reliability
+      - name: Tear down
+        if: always()
+        run: docker compose down -v
+
+  perf-cache:
+    runs-on: ubuntu-22.04
+    if: github.event_name == 'schedule'
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-go@v5
+        with:
+          go-version: '1.24'
+      - name: Install dependencies
+        run: |
+          sudo apt-get update
+          sudo apt-get install -y protobuf-compiler netcat-openbsd
+      - name: Install Go protobuf generator
+        run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
+      - name: Start full stack with PF-09 rate limits
+        env:
+          TRANSMITE_RATE_LIMIT_USER_MAX: '100000'
+          TRANSMITE_RATE_LIMIT_SESSION_MAX: '200000'
+        run: docker compose up -d --build
+      - name: Wait for services
+        run: ./scripts/wait_for_services.sh
+      - name: Generate Go protobuf
+        run: cd tests && make proto
+      - name: Download Go deps
+        run: cd tests && go mod download
+      - name: Run cache performance gate
+        run: cd tests && make test-perf-cache-gate
+      - name: Tear down
+        if: always()
+        run: docker compose down -v
+
   perf:
     needs: func
     runs-on: ubuntu-22.04
diff --git a/docker-compose.yml b/docker-compose.yml
index 2462db9..e3b5231 100644
--- a/docker-compose.yml
+++ b/docker-compose.yml
@@ -257,7 +257,7 @@ services:
       - rabbitmq
       - redis-cluster-init
     entrypoint:
-      /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,rabbitmq:5672 -c "/im/bin/transmite_server -flagfile=/im/conf/transmite_server.conf -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384"
+      /im/bin/entrypoint.sh -d etcd:2379,mysql:3306,redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384,rabbitmq:5672 -c "/im/bin/transmite_server -flagfile=/im/conf/transmite_server.conf -redis_seeds=redis-node1:6379,redis-node2:6380,redis-node3:6381,redis-node4:6382,redis-node5:6383,redis-node6:6384 -rate_limit_user_max=${TRANSMITE_RATE_LIMIT_USER_MAX:-600} -rate_limit_session_max=${TRANSMITE_RATE_LIMIT_SESSION_MAX:-3000}"
   identity_server:
     build: ./identity
     container_name: identity_server-service
diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
new file mode 100644
index 0000000..81f0c0b
--- /dev/null
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -0,0 +1,144 @@
+package contracts
+
+import (
+	"os"
+	"path/filepath"
+	"runtime"
+	"strconv"
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+	"gopkg.in/yaml.v3"
+)
+
+type workflowContract struct {
+	On struct {
+		PullRequest map[string]any   `yaml:"pull_request"`
+		Schedule    []map[string]any `yaml:"schedule"`
+	} `yaml:"on"`
+	Jobs map[string]workflowJob `yaml:"jobs"`
+}
+
+type workflowJob struct {
+	If    string            `yaml:"if"`
+	Env   map[string]string `yaml:"env"`
+	Needs any               `yaml:"needs"`
+	Steps []workflowStep    `yaml:"steps"`
+}
+
+type workflowStep struct {
+	Uses string            `yaml:"uses"`
+	Run  string            `yaml:"run"`
+	If   string            `yaml:"if"`
+	Env  map[string]string `yaml:"env"`
+}
+
+type composeContract struct {
+	Services map[string]struct {
+		Entrypoint string `yaml:"entrypoint"`
+	} `yaml:"services"`
+}
+
+func TestCIGates(t *testing.T) {
+	root := repositoryRoot(t)
+	workflowBytes, err := os.ReadFile(filepath.Join(root, ".github/workflows/ci.yml"))
+	require.NoError(t, err)
+	composeBytes, err := os.ReadFile(filepath.Join(root, "docker-compose.yml"))
+	require.NoError(t, err)
+
+	var workflow workflowContract
+	require.NoError(t, yaml.Unmarshal(workflowBytes, &workflow), "workflow must be valid YAML")
+	require.NotNil(t, workflow.On.PullRequest, "workflow must handle pull requests")
+	require.NotEmpty(t, workflow.On.Schedule, "workflow must define a schedule")
+
+	reliability, ok := workflow.Jobs["reliability"]
+	require.True(t, ok, "RL-05 must have a dedicated reliability job")
+	require.Nil(t, reliability.Needs, "reliability must own its setup instead of depending on another job")
+	require.Contains(t, reliability.If, "github.event_name == 'pull_request'")
+	require.Contains(t, reliability.If, "github.event_name == 'schedule'")
+	assertFullStackGateJob(t, reliability, "make test-reliability")
+
+	perfCache, ok := workflow.Jobs["perf-cache"]
+	require.True(t, ok, "PF-09 must have a dedicated perf-cache job")
+	require.Nil(t, perfCache.Needs, "perf-cache must own its setup instead of depending on another job")
+	require.Contains(t, perfCache.If, "github.event_name == 'schedule'")
+	assertFullStackGateJob(t, perfCache, "make test-perf-cache-gate")
+	require.NotContains(t, allRuns(perfCache), "make test-perf-cache\n", "PF-09 CI must not use the skip-capable discovery target")
+
+	userLimit := gateEnv(t, perfCache, "TRANSMITE_RATE_LIMIT_USER_MAX")
+	sessionLimit := gateEnv(t, perfCache, "TRANSMITE_RATE_LIMIT_SESSION_MAX")
+	require.Greater(t, userLimit, 50000, "PF-09 user limit must exceed its 5000 msg/s ten-second load")
+	require.Greater(t, sessionLimit, 50000, "PF-09 session limit must exceed its 5000 msg/s ten-second load")
+
+	var compose composeContract
+	require.NoError(t, yaml.Unmarshal(composeBytes, &compose), "Compose file must be valid YAML")
+	transmite, ok := compose.Services["transmite_server"]
+	require.True(t, ok)
+	require.Contains(t, transmite.Entrypoint, "-rate_limit_user_max=${TRANSMITE_RATE_LIMIT_USER_MAX:-600}")
+	require.Contains(t, transmite.Entrypoint, "-rate_limit_session_max=${TRANSMITE_RATE_LIMIT_SESSION_MAX:-3000}")
+}
+
+func assertFullStackGateJob(t *testing.T, job workflowJob, target string) {
+	t.Helper()
+	runs := allRuns(job)
+	require.Contains(t, uses(job), "actions/checkout@v4")
+	require.Contains(t, uses(job), "actions/setup-go@v5")
+	for _, command := range []string{
+		"protobuf-compiler", "protoc-gen-go", "docker compose up -d --build",
+		"wait_for_services.sh", "make proto", "go mod download", target,
+	} {
+		require.Contains(t, runs, command)
+	}
+	require.True(t, hasAlwaysTeardown(job), "gate job must always tear down its own stack")
+}
+
+func allRuns(job workflowJob) string {
+	var runs strings.Builder
+	for _, step := range job.Steps {
+		runs.WriteString(step.Run)
+		runs.WriteByte('\n')
+	}
+	return runs.String()
+}
+
+func uses(job workflowJob) string {
+	var values []string
+	for _, step := range job.Steps {
+		values = append(values, step.Uses)
+	}
+	return strings.Join(values, "\n")
+}
+
+func hasAlwaysTeardown(job workflowJob) bool {
+	for _, step := range job.Steps {
+		if step.If == "always()" && strings.Contains(step.Run, "docker compose down -v") {
+			return true
+		}
+	}
+	return false
+}
+
+func gateEnv(t *testing.T, job workflowJob, name string) int {
+	t.Helper()
+	raw := job.Env[name]
+	if raw == "" {
+		for _, step := range job.Steps {
+			if strings.Contains(step.Run, "docker compose up -d --build") && step.Env[name] != "" {
+				raw = step.Env[name]
+				break
+			}
+		}
+	}
+	require.NotEmpty(t, raw, "%s must be supplied to the PF-09 stack", name)
+	value, err := strconv.Atoi(raw)
+	require.NoError(t, err, "%s must be an explicit integer", name)
+	return value
+}
+
+func repositoryRoot(t *testing.T) string {
+	t.Helper()
+	_, filename, _, ok := runtime.Caller(0)
+	require.True(t, ok)
+	return filepath.Clean(filepath.Join(filepath.Dir(filename), "..", "..", ".."))
+}

From 9f107ff452ccffa06684ff7fdf06c82e8808dcbe Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 17:36:01 +0800
Subject: [PATCH 29/47] ci: enforce cache gate contract checks

---
 .github/workflows/ci.yml             |   7 +-
 tests/pkg/contracts/ci_gates_test.go | 117 +++++++++++++++++++--------
 2 files changed, 89 insertions(+), 35 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index abb1fa1..16a5dbd 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -28,6 +28,8 @@ jobs:
         run: cd tests && make proto
       - name: Download Go deps
         run: cd tests && go mod download
+      - name: Validate CI gate contracts
+        run: cd tests && go test ./pkg/contracts -count=1
       - name: Go vet
         run: cd tests && go vet ./...
       - name: Go fmt check
@@ -141,8 +143,9 @@ jobs:
         run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
       - name: Start full stack with PF-09 rate limits
         env:
-          TRANSMITE_RATE_LIMIT_USER_MAX: '100000'
-          TRANSMITE_RATE_LIMIT_SESSION_MAX: '200000'
+          # Transmite flags are int32; use the maximum valid value for gate-only headroom.
+          TRANSMITE_RATE_LIMIT_USER_MAX: '2147483647'
+          TRANSMITE_RATE_LIMIT_SESSION_MAX: '2147483647'
         run: docker compose up -d --build
       - name: Wait for services
         run: ./scripts/wait_for_services.sh
diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index 81f0c0b..e3b9c2a 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -4,7 +4,6 @@ import (
 	"os"
 	"path/filepath"
 	"runtime"
-	"strconv"
 	"strings"
 	"testing"
 
@@ -51,25 +50,24 @@ func TestCIGates(t *testing.T) {
 	require.NoError(t, yaml.Unmarshal(workflowBytes, &workflow), "workflow must be valid YAML")
 	require.NotNil(t, workflow.On.PullRequest, "workflow must handle pull requests")
 	require.NotEmpty(t, workflow.On.Schedule, "workflow must define a schedule")
+	assertDecoratedCommandsDoNotSatisfyGate(t)
+	assertContractsRunInBuild(t, workflow.Jobs["build"])
 
 	reliability, ok := workflow.Jobs["reliability"]
 	require.True(t, ok, "RL-05 must have a dedicated reliability job")
 	require.Nil(t, reliability.Needs, "reliability must own its setup instead of depending on another job")
-	require.Contains(t, reliability.If, "github.event_name == 'pull_request'")
-	require.Contains(t, reliability.If, "github.event_name == 'schedule'")
-	assertFullStackGateJob(t, reliability, "make test-reliability")
+	require.Equal(t, "github.event_name == 'pull_request' || github.event_name == 'schedule'", reliability.If)
+	assertFullStackGateJob(t, reliability, "cd tests && make test-reliability")
 
 	perfCache, ok := workflow.Jobs["perf-cache"]
 	require.True(t, ok, "PF-09 must have a dedicated perf-cache job")
 	require.Nil(t, perfCache.Needs, "perf-cache must own its setup instead of depending on another job")
-	require.Contains(t, perfCache.If, "github.event_name == 'schedule'")
-	assertFullStackGateJob(t, perfCache, "make test-perf-cache-gate")
-	require.NotContains(t, allRuns(perfCache), "make test-perf-cache\n", "PF-09 CI must not use the skip-capable discovery target")
+	require.Equal(t, "github.event_name == 'schedule'", perfCache.If)
+	assertFullStackGateJob(t, perfCache, "cd tests && make test-perf-cache-gate")
+	assertTargetAbsent(t, perfCache, "test-perf-cache")
 
-	userLimit := gateEnv(t, perfCache, "TRANSMITE_RATE_LIMIT_USER_MAX")
-	sessionLimit := gateEnv(t, perfCache, "TRANSMITE_RATE_LIMIT_SESSION_MAX")
-	require.Greater(t, userLimit, 50000, "PF-09 user limit must exceed its 5000 msg/s ten-second load")
-	require.Greater(t, sessionLimit, 50000, "PF-09 session limit must exceed its 5000 msg/s ten-second load")
+	require.Equal(t, "2147483647", gateEnv(t, perfCache, "TRANSMITE_RATE_LIMIT_USER_MAX"))
+	require.Equal(t, "2147483647", gateEnv(t, perfCache, "TRANSMITE_RATE_LIMIT_SESSION_MAX"))
 
 	var compose composeContract
 	require.NoError(t, yaml.Unmarshal(composeBytes, &compose), "Compose file must be valid YAML")
@@ -81,59 +79,112 @@ func TestCIGates(t *testing.T) {
 
 func assertFullStackGateJob(t *testing.T, job workflowJob, target string) {
 	t.Helper()
-	runs := allRuns(job)
-	require.Contains(t, uses(job), "actions/checkout@v4")
-	require.Contains(t, uses(job), "actions/setup-go@v5")
+	require.NotEqual(t, -1, exactUsesStepIndex(job, "actions/checkout@v4"))
+	require.NotEqual(t, -1, exactUsesStepIndex(job, "actions/setup-go@v5"))
 	for _, command := range []string{
-		"protobuf-compiler", "protoc-gen-go", "docker compose up -d --build",
-		"wait_for_services.sh", "make proto", "go mod download", target,
+		"sudo apt-get install -y protobuf-compiler netcat-openbsd",
+		"go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11",
+		"docker compose up -d --build", "./scripts/wait_for_services.sh",
+		"cd tests && make proto", "cd tests && go mod download", target,
 	} {
-		require.Contains(t, runs, command)
+		require.NotEqual(t, -1, exactRunStepIndex(job, command), "missing exact executable command %q", command)
 	}
 	require.True(t, hasAlwaysTeardown(job), "gate job must always tear down its own stack")
 }
 
-func allRuns(job workflowJob) string {
-	var runs strings.Builder
-	for _, step := range job.Steps {
-		runs.WriteString(step.Run)
-		runs.WriteByte('\n')
+func assertContractsRunInBuild(t *testing.T, build workflowJob) {
+	t.Helper()
+	setupGo := exactUsesStepIndex(build, "actions/setup-go@v5")
+	proto := exactRunStepIndex(build, "cd tests && make proto")
+	deps := exactRunStepIndex(build, "cd tests && go mod download")
+	contracts := exactRunStepIndex(build, "cd tests && go test ./pkg/contracts -count=1")
+	require.NotEqual(t, -1, setupGo, "build must set up Go")
+	require.Greater(t, proto, setupGo, "protobuf generation must follow Go setup")
+	require.Greater(t, deps, proto, "dependency download must follow protobuf generation")
+	require.Greater(t, contracts, deps, "CI contract tests must run after setup, protobuf generation, and dependency download")
+}
+
+func exactRunStepIndex(job workflowJob, wanted string) int {
+	for index, step := range job.Steps {
+		for _, command := range executableCommands(step.Run) {
+			if command == wanted {
+				return index
+			}
+		}
 	}
-	return runs.String()
+	return -1
 }
 
-func uses(job workflowJob) string {
-	var values []string
+func exactUsesStepIndex(job workflowJob, wanted string) int {
+	for index, step := range job.Steps {
+		if step.Uses == wanted {
+			return index
+		}
+	}
+	return -1
+}
+
+func assertDecoratedCommandsDoNotSatisfyGate(t *testing.T) {
+	t.Helper()
+	const gate = "cd tests && make test-perf-cache-gate"
+	require.Equal(t, 0, exactRunStepIndex(workflowJob{Steps: []workflowStep{{Run: gate}}}, gate))
+	for _, lookalike := range []string{
+		"# " + gate,
+		"echo '" + gate + "'",
+		gate + "-disabled",
+		"false && " + gate,
+		gate + " # disabled",
+	} {
+		job := workflowJob{Steps: []workflowStep{{Run: lookalike}}}
+		require.Equal(t, -1, exactRunStepIndex(job, gate), "%q must not satisfy the executable gate contract", lookalike)
+	}
+}
+
+func executableCommands(script string) []string {
+	var commands []string
+	for _, line := range strings.Split(script, "\n") {
+		line = strings.TrimSpace(line)
+		if line != "" && !strings.HasPrefix(line, "#") {
+			commands = append(commands, line)
+		}
+	}
+	return commands
+}
+
+func assertTargetAbsent(t *testing.T, job workflowJob, target string) {
+	t.Helper()
 	for _, step := range job.Steps {
-		values = append(values, step.Uses)
+		for _, command := range executableCommands(step.Run) {
+			words := strings.FieldsFunc(command, func(r rune) bool {
+				return strings.ContainsRune(" \t;&|()<>#", r)
+			})
+			require.NotContains(t, words, target, "PF-09 CI must not execute the skip-capable discovery target")
+		}
 	}
-	return strings.Join(values, "\n")
 }
 
 func hasAlwaysTeardown(job workflowJob) bool {
 	for _, step := range job.Steps {
-		if step.If == "always()" && strings.Contains(step.Run, "docker compose down -v") {
+		if step.If == "always()" && len(executableCommands(step.Run)) == 1 && executableCommands(step.Run)[0] == "docker compose down -v" {
 			return true
 		}
 	}
 	return false
 }
 
-func gateEnv(t *testing.T, job workflowJob, name string) int {
+func gateEnv(t *testing.T, job workflowJob, name string) string {
 	t.Helper()
 	raw := job.Env[name]
 	if raw == "" {
 		for _, step := range job.Steps {
-			if strings.Contains(step.Run, "docker compose up -d --build") && step.Env[name] != "" {
+			if len(executableCommands(step.Run)) == 1 && executableCommands(step.Run)[0] == "docker compose up -d --build" && step.Env[name] != "" {
 				raw = step.Env[name]
 				break
 			}
 		}
 	}
 	require.NotEmpty(t, raw, "%s must be supplied to the PF-09 stack", name)
-	value, err := strconv.Atoi(raw)
-	require.NoError(t, err, "%s must be an explicit integer", name)
-	return value
+	return raw
 }
 
 func repositoryRoot(t *testing.T) string {

From 2b1c9832f6d0d32b5d079d114f24a475a88f3ecd Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 17:40:56 +0800
Subject: [PATCH 30/47] test(ci): require exact ordered gate steps

---
 tests/pkg/contracts/ci_gates_test.go | 134 ++++++++++++++++++++-------
 1 file changed, 101 insertions(+), 33 deletions(-)

diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index e3b9c2a..139d7bd 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -1,6 +1,7 @@
 package contracts
 
 import (
+	"fmt"
 	"os"
 	"path/filepath"
 	"runtime"
@@ -65,6 +66,7 @@ func TestCIGates(t *testing.T) {
 	require.Equal(t, "github.event_name == 'schedule'", perfCache.If)
 	assertFullStackGateJob(t, perfCache, "cd tests && make test-perf-cache-gate")
 	assertTargetAbsent(t, perfCache, "test-perf-cache")
+	assertInvalidGateJobsRejected(t, perfCache, "cd tests && make test-perf-cache-gate")
 
 	require.Equal(t, "2147483647", gateEnv(t, perfCache, "TRANSMITE_RATE_LIMIT_USER_MAX"))
 	require.Equal(t, "2147483647", gateEnv(t, perfCache, "TRANSMITE_RATE_LIMIT_SESSION_MAX"))
@@ -79,17 +81,7 @@ func TestCIGates(t *testing.T) {
 
 func assertFullStackGateJob(t *testing.T, job workflowJob, target string) {
 	t.Helper()
-	require.NotEqual(t, -1, exactUsesStepIndex(job, "actions/checkout@v4"))
-	require.NotEqual(t, -1, exactUsesStepIndex(job, "actions/setup-go@v5"))
-	for _, command := range []string{
-		"sudo apt-get install -y protobuf-compiler netcat-openbsd",
-		"go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11",
-		"docker compose up -d --build", "./scripts/wait_for_services.sh",
-		"cd tests && make proto", "cd tests && go mod download", target,
-	} {
-		require.NotEqual(t, -1, exactRunStepIndex(job, command), "missing exact executable command %q", command)
-	}
-	require.True(t, hasAlwaysTeardown(job), "gate job must always tear down its own stack")
+	require.NoError(t, validateFullStackGateJob(job, target))
 }
 
 func assertContractsRunInBuild(t *testing.T, build workflowJob) {
@@ -106,10 +98,8 @@ func assertContractsRunInBuild(t *testing.T, build workflowJob) {
 
 func exactRunStepIndex(job workflowJob, wanted string) int {
 	for index, step := range job.Steps {
-		for _, command := range executableCommands(step.Run) {
-			if command == wanted {
-				return index
-			}
+		if strings.TrimSpace(step.Run) == wanted {
+			return index
 		}
 	}
 	return -1
@@ -133,43 +123,121 @@ func assertDecoratedCommandsDoNotSatisfyGate(t *testing.T) {
 		"echo '" + gate + "'",
 		gate + "-disabled",
 		"false && " + gate,
+		"exit 0\n" + gate,
+		"if false; then " + gate + "; fi",
 		gate + " # disabled",
+		gate + "\nexit 0",
+		"cat <<'EOF'\n" + gate + "\nEOF",
+		"gate() { " + gate + "; }\ngate",
+		`cd tests && make "test-perf-cache"`,
 	} {
 		job := workflowJob{Steps: []workflowStep{{Run: lookalike}}}
 		require.Equal(t, -1, exactRunStepIndex(job, gate), "%q must not satisfy the executable gate contract", lookalike)
 	}
 }
 
-func executableCommands(script string) []string {
-	var commands []string
-	for _, line := range strings.Split(script, "\n") {
-		line = strings.TrimSpace(line)
-		if line != "" && !strings.HasPrefix(line, "#") {
-			commands = append(commands, line)
-		}
+func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target string) {
+	t.Helper()
+	gate := exactRunStepIndex(valid, target)
+	deps := exactRunStepIndex(valid, "cd tests && go mod download")
+	teardown := exactRunStepIndex(valid, "docker compose down -v")
+	require.NotEqual(t, -1, gate)
+	require.NotEqual(t, -1, deps)
+	require.NotEqual(t, -1, teardown)
+
+	for name, mutate := range map[string]func(*workflowJob){
+		"exit zero after gate": func(job *workflowJob) {
+			job.Steps[gate].Run = target + "\nexit 0"
+		},
+		"if false gate": func(job *workflowJob) {
+			job.Steps[gate].Run = "if false; then " + target + "; fi"
+		},
+		"quoted skip target": func(job *workflowJob) {
+			job.Steps[gate].Run = `cd tests && make "test-perf-cache"`
+		},
+		"gate before dependencies": func(job *workflowJob) {
+			job.Steps[gate], job.Steps[deps] = job.Steps[deps], job.Steps[gate]
+		},
+		"teardown before gate": func(job *workflowJob) {
+			job.Steps[gate], job.Steps[teardown] = job.Steps[teardown], job.Steps[gate]
+		},
+		"teardown not last": func(job *workflowJob) {
+			job.Steps = append(job.Steps, workflowStep{Run: "true"})
+		},
+	} {
+		t.Run(name, func(t *testing.T) {
+			invalid := cloneWorkflowJob(valid)
+			mutate(&invalid)
+			require.Error(t, validateFullStackGateJob(invalid, target))
+		})
 	}
-	return commands
+}
+
+func cloneWorkflowJob(job workflowJob) workflowJob {
+	clone := job
+	clone.Steps = append([]workflowStep(nil), job.Steps...)
+	return clone
 }
 
 func assertTargetAbsent(t *testing.T, job workflowJob, target string) {
 	t.Helper()
 	for _, step := range job.Steps {
-		for _, command := range executableCommands(step.Run) {
-			words := strings.FieldsFunc(command, func(r rune) bool {
-				return strings.ContainsRune(" \t;&|()<>#", r)
-			})
-			require.NotContains(t, words, target, "PF-09 CI must not execute the skip-capable discovery target")
+		require.NotEqual(t, "cd tests && make "+target, strings.TrimSpace(step.Run),
+			"PF-09 CI must not execute the skip-capable discovery target")
+	}
+}
+
+func validateFullStackGateJob(job workflowJob, target string) error {
+	const install = "sudo apt-get update\nsudo apt-get install -y protobuf-compiler netcat-openbsd"
+	ordered := []struct {
+		label string
+		index int
+	}{
+		{"checkout", exactUsesStepIndex(job, "actions/checkout@v4")},
+		{"Go setup", exactUsesStepIndex(job, "actions/setup-go@v5")},
+		{"system dependency install", exactRunStepIndex(job, install)},
+		{"protoc generator install", exactRunStepIndex(job, "go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11")},
+		{"full-stack startup", exactRunStepIndex(job, "docker compose up -d --build")},
+		{"service wait", exactRunStepIndex(job, "./scripts/wait_for_services.sh")},
+		{"protobuf generation", exactRunStepIndex(job, "cd tests && make proto")},
+		{"dependency download", exactRunStepIndex(job, "cd tests && go mod download")},
+		{"gate", exactRunStepIndex(job, target)},
+		{"teardown", exactRunStepIndex(job, "docker compose down -v")},
+	}
+	previous := -1
+	for _, step := range ordered {
+		if step.index < 0 {
+			return fmt.Errorf("missing exact %s step", step.label)
+		}
+		if step.index <= previous {
+			return fmt.Errorf("%s step is out of order", step.label)
 		}
+		previous = step.index
+	}
+	teardown := ordered[len(ordered)-1].index
+	if teardown != len(job.Steps)-1 {
+		return fmt.Errorf("teardown must be the final step")
+	}
+	if job.Steps[teardown].If != "always()" {
+		return fmt.Errorf("teardown must use if: always()")
+	}
+	if countExactRunSteps(job, target) != 1 {
+		return fmt.Errorf("gate command must appear exactly once")
+	}
+	if countExactRunSteps(job, "docker compose down -v") != 1 {
+		return fmt.Errorf("teardown command must appear exactly once")
 	}
+	return nil
 }
 
-func hasAlwaysTeardown(job workflowJob) bool {
+func countExactRunSteps(job workflowJob, wanted string) int {
+	count := 0
 	for _, step := range job.Steps {
-		if step.If == "always()" && len(executableCommands(step.Run)) == 1 && executableCommands(step.Run)[0] == "docker compose down -v" {
-			return true
+		if strings.TrimSpace(step.Run) == wanted {
+			count++
 		}
 	}
-	return false
+	return count
 }
 
 func gateEnv(t *testing.T, job workflowJob, name string) string {
@@ -177,7 +245,7 @@ func gateEnv(t *testing.T, job workflowJob, name string) string {
 	raw := job.Env[name]
 	if raw == "" {
 		for _, step := range job.Steps {
-			if len(executableCommands(step.Run)) == 1 && executableCommands(step.Run)[0] == "docker compose up -d --build" && step.Env[name] != "" {
+			if strings.TrimSpace(step.Run) == "docker compose up -d --build" && step.Env[name] != "" {
 				raw = step.Env[name]
 				break
 			}

From 6c2b8f2c3864929dda5f38f6d3b908f2494e192e Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 17:45:19 +0800
Subject: [PATCH 31/47] test(ci): reject bypassable cache gates

---
 tests/pkg/contracts/ci_gates_test.go | 85 ++++++++++++++++++++++++++--
 1 file changed, 79 insertions(+), 6 deletions(-)

diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index 139d7bd..89ada1e 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -28,10 +28,11 @@ type workflowJob struct {
 }
 
 type workflowStep struct {
-	Uses string            `yaml:"uses"`
-	Run  string            `yaml:"run"`
-	If   string            `yaml:"if"`
-	Env  map[string]string `yaml:"env"`
+	Uses            string            `yaml:"uses"`
+	Run             string            `yaml:"run"`
+	If              string            `yaml:"if"`
+	ContinueOnError any               `yaml:"continue-on-error"`
+	Env             map[string]string `yaml:"env"`
 }
 
 type composeContract struct {
@@ -139,13 +140,37 @@ func assertDecoratedCommandsDoNotSatisfyGate(t *testing.T) {
 func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target string) {
 	t.Helper()
 	gate := exactRunStepIndex(valid, target)
+	start := exactRunStepIndex(valid, "docker compose up -d --build")
+	wait := exactRunStepIndex(valid, "./scripts/wait_for_services.sh")
+	proto := exactRunStepIndex(valid, "cd tests && make proto")
 	deps := exactRunStepIndex(valid, "cd tests && go mod download")
 	teardown := exactRunStepIndex(valid, "docker compose down -v")
 	require.NotEqual(t, -1, gate)
+	require.NotEqual(t, -1, start)
+	require.NotEqual(t, -1, wait)
+	require.NotEqual(t, -1, proto)
 	require.NotEqual(t, -1, deps)
 	require.NotEqual(t, -1, teardown)
 
 	for name, mutate := range map[string]func(*workflowJob){
+		"gate disabled by if": func(job *workflowJob) {
+			job.Steps[gate].If = "${{ false }}"
+		},
+		"gate allowed to fail": func(job *workflowJob) {
+			job.Steps[gate].ContinueOnError = true
+		},
+		"startup allowed to fail": func(job *workflowJob) {
+			job.Steps[start].ContinueOnError = true
+		},
+		"wait allowed to fail": func(job *workflowJob) {
+			job.Steps[wait].ContinueOnError = true
+		},
+		"proto allowed to fail": func(job *workflowJob) {
+			job.Steps[proto].ContinueOnError = true
+		},
+		"dependency download allowed to fail": func(job *workflowJob) {
+			job.Steps[deps].ContinueOnError = true
+		},
 		"exit zero after gate": func(job *workflowJob) {
 			job.Steps[gate].Run = target + "\nexit 0"
 		},
@@ -153,7 +178,10 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 			job.Steps[gate].Run = "if false; then " + target + "; fi"
 		},
 		"quoted skip target": func(job *workflowJob) {
-			job.Steps[gate].Run = `cd tests && make "test-perf-cache"`
+			insertWorkflowStep(job, teardown, workflowStep{Run: `cd tests && make "test-perf-cache"`})
+		},
+		"single-quoted skip target": func(job *workflowJob) {
+			insertWorkflowStep(job, teardown, workflowStep{Run: `cd tests && make 'test-perf-cache'`})
 		},
 		"gate before dependencies": func(job *workflowJob) {
 			job.Steps[gate], job.Steps[deps] = job.Steps[deps], job.Steps[gate]
@@ -173,6 +201,12 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 	}
 }
 
+func insertWorkflowStep(job *workflowJob, index int, step workflowStep) {
+	job.Steps = append(job.Steps, workflowStep{})
+	copy(job.Steps[index+1:], job.Steps[index:])
+	job.Steps[index] = step
+}
+
 func cloneWorkflowJob(job workflowJob) workflowJob {
 	clone := job
 	clone.Steps = append([]workflowStep(nil), job.Steps...)
@@ -182,13 +216,32 @@ func cloneWorkflowJob(job workflowJob) workflowJob {
 func assertTargetAbsent(t *testing.T, job workflowJob, target string) {
 	t.Helper()
 	for _, step := range job.Steps {
-		require.NotEqual(t, "cd tests && make "+target, strings.TrimSpace(step.Run),
+		require.False(t, isForbiddenMakeTarget(step.Run, target),
 			"PF-09 CI must not execute the skip-capable discovery target")
 	}
 }
 
+func isForbiddenMakeTarget(run, target string) bool {
+	run = strings.TrimSpace(run)
+	for _, command := range []string{
+		"cd tests && make " + target,
+		`cd tests && make "` + target + `"`,
+		"cd tests && make '" + target + "'",
+	} {
+		if run == command {
+			return true
+		}
+	}
+	return false
+}
+
 func validateFullStackGateJob(job workflowJob, target string) error {
 	const install = "sudo apt-get update\nsudo apt-get install -y protobuf-compiler netcat-openbsd"
+	for _, step := range job.Steps {
+		if isForbiddenMakeTarget(step.Run, "test-perf-cache") {
+			return fmt.Errorf("skip-capable performance target is forbidden")
+		}
+	}
 	ordered := []struct {
 		label string
 		index int
@@ -215,6 +268,15 @@ func validateFullStackGateJob(job workflowJob, target string) error {
 		previous = step.index
 	}
 	teardown := ordered[len(ordered)-1].index
+	gate := ordered[len(ordered)-2].index
+	if strings.TrimSpace(job.Steps[gate].If) != "" {
+		return fmt.Errorf("gate step must not have a step-level if condition")
+	}
+	for _, required := range ordered[4:9] {
+		if continueOnErrorEnabled(job.Steps[required.index].ContinueOnError) {
+			return fmt.Errorf("%s step must not continue on error", required.label)
+		}
+	}
 	if teardown != len(job.Steps)-1 {
 		return fmt.Errorf("teardown must be the final step")
 	}
@@ -230,6 +292,17 @@ func validateFullStackGateJob(job workflowJob, target string) error {
 	return nil
 }
 
+func continueOnErrorEnabled(value any) bool {
+	switch value := value.(type) {
+	case nil:
+		return false
+	case bool:
+		return value
+	default:
+		return true
+	}
+}
+
 func countExactRunSteps(job workflowJob, wanted string) int {
 	count := 0
 	for _, step := range job.Steps {

From f6330060d524cf915634d0a01ee1e16bdbf48492 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 17:48:25 +0800
Subject: [PATCH 32/47] test(ci): enforce required step failures

---
 tests/pkg/contracts/ci_gates_test.go | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index 89ada1e..14b810c 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -140,12 +140,14 @@ func assertDecoratedCommandsDoNotSatisfyGate(t *testing.T) {
 func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target string) {
 	t.Helper()
 	gate := exactRunStepIndex(valid, target)
+	setupGo := exactUsesStepIndex(valid, "actions/setup-go@v5")
 	start := exactRunStepIndex(valid, "docker compose up -d --build")
 	wait := exactRunStepIndex(valid, "./scripts/wait_for_services.sh")
 	proto := exactRunStepIndex(valid, "cd tests && make proto")
 	deps := exactRunStepIndex(valid, "cd tests && go mod download")
 	teardown := exactRunStepIndex(valid, "docker compose down -v")
 	require.NotEqual(t, -1, gate)
+	require.NotEqual(t, -1, setupGo)
 	require.NotEqual(t, -1, start)
 	require.NotEqual(t, -1, wait)
 	require.NotEqual(t, -1, proto)
@@ -153,6 +155,9 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 	require.NotEqual(t, -1, teardown)
 
 	for name, mutate := range map[string]func(*workflowJob){
+		"Go setup allowed to fail": func(job *workflowJob) {
+			job.Steps[setupGo].ContinueOnError = true
+		},
 		"gate disabled by if": func(job *workflowJob) {
 			job.Steps[gate].If = "${{ false }}"
 		},
@@ -272,7 +277,7 @@ func validateFullStackGateJob(job workflowJob, target string) error {
 	if strings.TrimSpace(job.Steps[gate].If) != "" {
 		return fmt.Errorf("gate step must not have a step-level if condition")
 	}
-	for _, required := range ordered[4:9] {
+	for _, required := range ordered[:len(ordered)-1] {
 		if continueOnErrorEnabled(job.Steps[required.index].ContinueOnError) {
 			return fmt.Errorf("%s step must not continue on error", required.label)
 		}

From b406401c299ab677008c47d2d46eafdb7cfdd47e Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 18:16:32 +0800
Subject: [PATCH 33/47] docs: design executable cache gate artifacts

---
 ...16-pr57-ci-artifacts-native-regressions.md | 96 +++++++++++++++++++
 ...-ci-artifacts-native-regressions-design.md | 93 ++++++++++++++++++
 2 files changed, 189 insertions(+)
 create mode 100644 docs/superpowers/plans/2026-07-16-pr57-ci-artifacts-native-regressions.md
 create mode 100644 docs/superpowers/specs/2026-07-16-pr57-ci-artifacts-native-regressions-design.md

diff --git a/docs/superpowers/plans/2026-07-16-pr57-ci-artifacts-native-regressions.md b/docs/superpowers/plans/2026-07-16-pr57-ci-artifacts-native-regressions.md
new file mode 100644
index 0000000..80bda39
--- /dev/null
+++ b/docs/superpowers/plans/2026-07-16-pr57-ci-artifacts-native-regressions.md
@@ -0,0 +1,96 @@
+# PR #57 CI Artifacts and Native Regressions Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Produce runnable service artifacts once per workflow and automatically execute the two cache native regressions without weakening the Go full-stack gates.
+
+**Architecture:** A repository-owned, cached native builder feeds one artifact producer job. Consumer jobs download a validated Compose layout. CTest is enabled only for two explicitly scoped cache component regressions.
+
+**Tech Stack:** GitHub Actions, Docker BuildKit, Ubuntu 24.04, CMake/CTest, Bash, Go workflow contract tests, Docker Compose, Redis Cluster.
+
+## Global Constraints
+
+- Build all nine service binaries exactly once per workflow run.
+- Do not depend on an unpublished or mutable external builder image.
+- Missing native dependencies, binaries, shared libraries, or tests must fail closed; required work may not skip.
+- `reliability` and `perf-cache` must depend on and download the same immutable artifact before Compose startup.
+- Keep the native-test exception limited to `test_user_info_generation_fence` and `test_unacked_pending_ledger`; do not enable the legacy C++ suite or add gtest.
+- RL-05 and PF-09 remain Go full-stack gates using their existing Make targets.
+- Every full-stack job ends with one `docker compose down -v` step guarded by `if: always()`.
+
+---
+
+### Task 1: Reproducible native builder and Compose artifact packager
+
+**Files:**
+- Create: `docker/ci/Dockerfile`
+- Create: `docker/ci/dependencies.lock`
+- Create: `scripts/package_compose_artifacts.sh`
+- Create: `scripts/validate_compose_artifacts.sh`
+- Create: `tests/pkg/contracts/artifacts_test.go`
+
+**Interfaces:**
+- Produces: directory `compose-artifacts/<service>/{build,depends}` and `compose-artifacts/MANIFEST.sha256`.
+- Services: `conversation gateway identity media message presence push relationship transmite`.
+
+- [ ] Write Go contract tests that fail unless the builder uses a digest/tag lock file, the packager enumerates all nine services, rejects missing binaries and unresolved `ldd` output, and the validator verifies the manifest plus executable/shared-library closure.
+- [ ] Run `cd tests && go test ./pkg/contracts -run 'TestComposeArtifact' -count=1` and confirm it fails because the builder and scripts do not exist.
+- [ ] Add the Ubuntu 24.04 builder definition. Install distribution dependencies and build non-distribution dependencies at exact immutable revisions from `docker/ci/dependencies.lock`; configure `/usr/local` through `ldconfig`. Do not use `latest`, an unpinned branch, or a pre-existing local image.
+- [ ] Implement the packager with `set -euo pipefail`, explicit service enumeration, root-build source paths `build/<service>/<service>_server`, executable checks, `ldd` closure copying, and deterministic `sha256sum` manifest generation.
+- [ ] Implement the validator with the same explicit service list, `sha256sum --check`, executable checks, and an isolated `ldd` check for every binary.
+- [ ] Run the focused Go contract test and shell syntax checks; expect zero failures.
+- [ ] Build the builder image and run CMake plus packaging in it; expect nine binaries and a valid manifest.
+- [ ] Commit with message `build(ci): package reproducible service artifacts`.
+
+### Task 2: Register and execute the two native regressions
+
+**Files:**
+- Modify: `CMakeLists.txt`
+- Modify: `common/test/CMakeLists.txt`
+- Modify: `common/test/test_user_info_generation_fence.cc`
+- Modify: `common/test/test_unacked_pending_ledger.cc`
+- Create: `tests/pkg/contracts/native_regressions_test.go`
+
+**Interfaces:**
+- Produces CMake option `CHATNOW_BUILD_CACHE_REGRESSION_TESTS`.
+- Produces CTest labels `cache-unit` and `redis-cluster`.
+- Consumes environment variable `CHATNOW_REDIS_CLUSTER_SEEDS` for the ledger test.
+
+- [ ] Write contract tests that require both exact targets, both `add_test` registrations, labels, timeouts, the opt-in option, and fail-closed dependency resolution when the option is enabled.
+- [ ] Run `cd tests && go test ./pkg/contracts -run 'TestNativeCacheRegression' -count=1`; confirm it fails on missing registrations.
+- [ ] Enable CTest at the root and add an OFF-by-default `CHATNOW_BUILD_CACHE_REGRESSION_TESTS` option.
+- [ ] Replace the current Redis-header conditional with required target/link discovery inside the enabled option. Register generation as `cache-unit`; register ledger as `redis-cluster`, serial, with an explicit timeout and seed environment.
+- [ ] Build with `-DCHATNOW_BUILD_CACHE_REGRESSION_TESTS=ON`, run the generation CTest, and confirm it passes.
+- [ ] Start a real Redis Cluster, run the ledger CTest with explicit seeds, and confirm it passes; stop the cluster.
+- [ ] Re-run the focused contract tests and commit with message `test(cache): register native atomicity regressions`.
+
+### Task 3: Wire the artifact producer and consumers into CI
+
+**Files:**
+- Modify: `.github/workflows/ci.yml`
+- Modify: `tests/pkg/contracts/ci_gates_test.go`
+
+**Interfaces:**
+- Producer job: `service-artifacts`.
+- Artifact name: `compose-service-artifacts`.
+- Consumers: `reliability`, `perf-cache`.
+
+- [ ] Extend workflow contract tests to require one producer; BuildKit GHA caching; builder build, CMake build, generation CTest, package, validate, and upload in order; consumer `needs`, download, validate, Compose start, gate, and teardown in order; and execution of the Redis ledger CTest against the stack.
+- [ ] Run `cd tests && go test ./pkg/contracts -count=1`; confirm the new assertions fail against the old workflow.
+- [ ] Add `service-artifacts` to the workflow using `docker/build-push-action` cache-to/cache-from `type=gha`, then execute the native build and packaging inside that image and upload `compose-artifacts` with `actions/upload-artifact`.
+- [ ] Make both consumers depend on the producer, download with `actions/download-artifact`, restore service-context directories, validate before Compose, and run the Redis ledger CTest after Redis Cluster readiness and before the Go gate.
+- [ ] Preserve PF-09 rate-limit overrides and strict final teardown behavior.
+- [ ] Run the full contract suite, YAML parse, and `docker compose config --quiet`; expect zero failures.
+- [ ] Commit with message `ci: distribute service artifacts to cache gates`.
+
+### Task 4: End-to-end verification and PR update
+
+**Files:**
+- Modify only if verification exposes a defect in Tasks 1-3.
+
+- [ ] Run `git diff --check` and inspect the complete branch diff.
+- [ ] Run `cd tests && PATH="$(go env GOPATH)/bin:$PATH" make proto` followed by `go test ./...`, tagged vet commands, and the PF-09 race helper command.
+- [ ] Build the builder from a clean Docker cache or pull-free context, build all services, package, validate, and run the generation test.
+- [ ] Start the downloaded-equivalent artifact layout with Compose, wait for services, run the ledger CTest, RL-05, and then tear down.
+- [ ] Push the branch and inspect the new GitHub Actions run. Do not claim green if an external failure remains.
+- [ ] Fetch review threads again and report which new threads are addressed. Do not reply to or resolve GitHub threads without explicit user authorization.
diff --git a/docs/superpowers/specs/2026-07-16-pr57-ci-artifacts-native-regressions-design.md b/docs/superpowers/specs/2026-07-16-pr57-ci-artifacts-native-regressions-design.md
new file mode 100644
index 0000000..187cf3c
--- /dev/null
+++ b/docs/superpowers/specs/2026-07-16-pr57-ci-artifacts-native-regressions-design.md
@@ -0,0 +1,93 @@
+# PR #57 CI Artifacts and Native Regressions Design
+
+## Scope
+
+Address the two review threads added after commit `f633006`:
+
+1. make the RL-05 and PF-09 jobs start a runnable service stack from a clean
+   GitHub runner;
+2. execute the generation-fence and Unacked-ledger regressions automatically.
+
+The change must not hide a missing native toolchain, silently skip a test, or
+duplicate a full service build in every gate job.
+
+## Service artifact pipeline
+
+A single producer job builds the native services in a reproducible Linux
+builder and packages the exact directory contract consumed by Compose:
+
+```text
+compose-artifacts/
+  identity/build/identity_server
+  identity/depends/*.so*
+  ...
+  transmite/build/transmite_server
+  transmite/depends/*.so*
+```
+
+The producer uses a repository-owned builder definition with pinned dependency
+revisions and GitHub Actions layer caching. It performs one root CMake build,
+then a packaging script copies `build/<service>/<service>_server` into each
+service build context and obtains the non-system shared-library closure from
+`ldd`. Packaging fails if any of the nine binaries is missing or if `ldd`
+reports an unresolved dependency.
+
+The producer uploads one immutable artifact for the workflow run. The
+`reliability` and `perf-cache` consumers declare `needs: service-artifacts`,
+download it before `docker compose up`, validate its manifest, and then run the
+existing Go gates. A failed producer blocks consumers instead of producing a
+misleading integration-test failure.
+
+## Native regression exception
+
+The unified test architecture remains Go black-box by default. Two narrow L0
+native component regressions are an explicit exception because the public
+service boundary cannot deterministically create their internal atomicity
+conditions without production test hooks:
+
+- UserInfo generation CAS conflict must not publish stale bytes into L1;
+- retrying one Unacked `user_seq` with a different payload must keep one
+  identity and ACK must remove both indexes.
+
+The exception does not introduce gtest or a general C++ test suite. CMake
+exposes `CHATNOW_BUILD_CACHE_REGRESSION_TESTS`, disabled by default and enabled
+by CI. When enabled, dependencies are required rather than detected with a
+skip-capable probe. Both executables are registered with CTest labels. The
+generation test runs in the producer. The ledger test runs against the real
+Redis Cluster in a Compose network with an explicit seed and a timeout.
+
+The Go RL-05 and PF-09 suites remain the authoritative business and system
+gates; native tests only cover atomicity that cannot be made deterministic at
+the external API.
+
+## Contract enforcement
+
+The existing Go workflow contract tests will require:
+
+- exactly one service-artifact producer;
+- build, package, validate, and upload steps in order;
+- both gate jobs to depend on the producer and download/validate before
+  Compose startup;
+- the producer to enable and execute the generation native regression;
+- the Redis-backed ledger regression to execute against the stack;
+- no `continue-on-error`, conditional bypass, or skip-capable dependency
+  detection on required steps;
+- teardown to remain the unique final step with `if: always()`.
+
+## Failure behavior
+
+- Missing compiler dependency: builder construction or CMake configuration
+  fails.
+- Missing binary: packaging fails before upload.
+- Unresolved shared library: artifact validation fails before upload and again
+  after download.
+- Redis Cluster unavailable: ledger CTest fails; it is never reported as
+  skipped.
+- Gate failure: job fails and teardown still runs.
+
+## Non-goals
+
+- no refactor of all nine runtime Dockerfiles into multi-stage builds;
+- no externally managed or mutable CI image dependency;
+- no general revival of the old C++ gtest suite;
+- no change to cache, ACK, or message-watermark business semantics.

From b849683273cdbff2cc3731db2505020d9b288c37 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 18:32:32 +0800
Subject: [PATCH 34/47] build(ci): package reproducible service artifacts

---
 docker/ci/Dockerfile                  | 173 ++++++++++++++++++++++++++
 docker/ci/dependencies.lock           |  10 ++
 scripts/package_compose_artifacts.sh  |  48 +++++++
 scripts/validate_compose_artifacts.sh |  51 ++++++++
 tests/pkg/contracts/artifacts_test.go | 115 +++++++++++++++++
 5 files changed, 397 insertions(+)
 create mode 100644 docker/ci/Dockerfile
 create mode 100644 docker/ci/dependencies.lock
 create mode 100755 scripts/package_compose_artifacts.sh
 create mode 100755 scripts/validate_compose_artifacts.sh
 create mode 100644 tests/pkg/contracts/artifacts_test.go

diff --git a/docker/ci/Dockerfile b/docker/ci/Dockerfile
new file mode 100644
index 0000000..c407d82
--- /dev/null
+++ b/docker/ci/Dockerfile
@@ -0,0 +1,173 @@
+ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
+FROM ${UBUNTU_IMAGE}
+
+ENV DEBIAN_FRONTEND=noninteractive
+SHELL ["/bin/bash", "-o", "pipefail", "-c"]
+
+RUN apt-get -o Acquire::Retries=5 update \
+    && apt-get -o Acquire::Retries=5 install -y --no-install-recommends \
+        autoconf \
+        automake \
+        build-essential \
+        ca-certificates \
+        cmake \
+        git \
+        libboost-all-dev \
+        libcpprest-dev \
+        libcrypt-dev \
+        libcurl4-openssl-dev \
+        libev-dev \
+        libfmt-dev \
+        libgflags-dev \
+        libgoogle-glog-dev \
+        libgrpc++-dev \
+        libgrpc-dev \
+        libhiredis-dev \
+        libjsoncpp-dev \
+        libleveldb-dev \
+        libmysqlclient-dev \
+        libodb-boost-dev \
+        libodb-dev \
+        libodb-mysql-dev \
+        libprotobuf-dev \
+        libsnappy-dev \
+        libspdlog-dev \
+        libssl-dev \
+        libtool \
+        libunwind-dev \
+        libxml2-dev \
+        ninja-build \
+        odb \
+        pkg-config \
+        protobuf-compiler \
+        protobuf-compiler-grpc \
+        zlib1g-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+COPY docker/ci/dependencies.lock /tmp/dependencies.lock
+
+RUN set -euo pipefail; \
+    . /tmp/dependencies.lock; \
+    git init /tmp/brpc; \
+    git -C /tmp/brpc remote add origin https://github.com/apache/brpc.git; \
+    git -C /tmp/brpc fetch --depth 1 origin "$BRPC_REV"; \
+    git -C /tmp/brpc checkout --detach FETCH_HEAD; \
+    test "$(git -C /tmp/brpc rev-parse HEAD)" = "$BRPC_REV"; \
+    cmake -S /tmp/brpc -B /tmp/brpc-build -G Ninja \
+        -DCMAKE_BUILD_TYPE=Release \
+        -DCMAKE_INSTALL_PREFIX=/usr/local \
+        -DBUILD_SHARED_LIBS=ON \
+        -DBUILD_BRPC_TOOLS=OFF \
+        -DBUILD_UNIT_TESTS=OFF \
+        -DWITH_GLOG=ON \
+        -DWITH_SNAPPY=ON; \
+    cmake --build /tmp/brpc-build --parallel "$(nproc)"; \
+    cmake --install /tmp/brpc-build; \
+    rm -rf /tmp/brpc /tmp/brpc-build
+
+RUN set -euo pipefail; \
+    . /tmp/dependencies.lock; \
+    git init /tmp/etcd-cpp-apiv3; \
+    git -C /tmp/etcd-cpp-apiv3 remote add origin https://github.com/etcd-cpp-apiv3/etcd-cpp-apiv3.git; \
+    git -C /tmp/etcd-cpp-apiv3 fetch --depth 1 origin "$ETCD_CPP_APIV3_REV"; \
+    git -C /tmp/etcd-cpp-apiv3 checkout --detach FETCH_HEAD; \
+    test "$(git -C /tmp/etcd-cpp-apiv3 rev-parse HEAD)" = "$ETCD_CPP_APIV3_REV"; \
+    cmake -S /tmp/etcd-cpp-apiv3 -B /tmp/etcd-cpp-apiv3-build -G Ninja \
+        -DCMAKE_BUILD_TYPE=Release \
+        -DCMAKE_INSTALL_PREFIX=/usr/local \
+        -DBUILD_ETCD_TESTS=OFF \
+        -DBUILD_SHARED_LIBS=ON \
+        -DETCD_W_STRICT=OFF; \
+    cmake --build /tmp/etcd-cpp-apiv3-build --parallel "$(nproc)"; \
+    cmake --install /tmp/etcd-cpp-apiv3-build; \
+    rm -rf /tmp/etcd-cpp-apiv3 /tmp/etcd-cpp-apiv3-build
+
+RUN set -euo pipefail; \
+    . /tmp/dependencies.lock; \
+    git init /tmp/redis-plus-plus; \
+    git -C /tmp/redis-plus-plus remote add origin https://github.com/sewenew/redis-plus-plus.git; \
+    git -C /tmp/redis-plus-plus fetch --depth 1 origin "$REDIS_PLUS_PLUS_REV"; \
+    git -C /tmp/redis-plus-plus checkout --detach FETCH_HEAD; \
+    test "$(git -C /tmp/redis-plus-plus rev-parse HEAD)" = "$REDIS_PLUS_PLUS_REV"; \
+    cmake -S /tmp/redis-plus-plus -B /tmp/redis-plus-plus-build -G Ninja \
+        -DCMAKE_BUILD_TYPE=Release \
+        -DCMAKE_INSTALL_PREFIX=/usr/local \
+        -DREDIS_PLUS_PLUS_BUILD_STATIC=OFF \
+        -DREDIS_PLUS_PLUS_BUILD_TEST=OFF; \
+    cmake --build /tmp/redis-plus-plus-build --parallel "$(nproc)"; \
+    cmake --install /tmp/redis-plus-plus-build; \
+    rm -rf /tmp/redis-plus-plus /tmp/redis-plus-plus-build
+
+RUN set -euo pipefail; \
+    . /tmp/dependencies.lock; \
+    git init /tmp/cpr; \
+    git -C /tmp/cpr remote add origin https://github.com/whoshuu/cpr.git; \
+    git -C /tmp/cpr fetch --depth 1 origin "$CPR_REV"; \
+    git -C /tmp/cpr checkout --detach FETCH_HEAD; \
+    test "$(git -C /tmp/cpr rev-parse HEAD)" = "$CPR_REV"; \
+    cmake -S /tmp/cpr -B /tmp/cpr-build -G Ninja \
+        -DCMAKE_BUILD_TYPE=Release \
+        -DCMAKE_INSTALL_PREFIX=/usr/local \
+        -DBUILD_CPR_TESTS=OFF \
+        -DBUILD_SHARED_LIBS=ON \
+        -DUSE_SYSTEM_CURL=ON; \
+    cmake --build /tmp/cpr-build --parallel "$(nproc)"; \
+    cmake --install /tmp/cpr-build; \
+    rm -rf /tmp/cpr /tmp/cpr-build
+
+RUN set -euo pipefail; \
+    . /tmp/dependencies.lock; \
+    git init /tmp/elasticlient; \
+    git -C /tmp/elasticlient remote add origin https://github.com/seznam/elasticlient.git; \
+    git -C /tmp/elasticlient fetch --depth 1 origin "$ELASTICLIENT_REV"; \
+    git -C /tmp/elasticlient checkout --detach FETCH_HEAD; \
+    test "$(git -C /tmp/elasticlient rev-parse HEAD)" = "$ELASTICLIENT_REV"; \
+    cmake -S /tmp/elasticlient -B /tmp/elasticlient-build -G Ninja \
+        -DCMAKE_BUILD_TYPE=Release \
+        -DCMAKE_INSTALL_PREFIX=/usr/local \
+        -DBUILD_ELASTICLIENT_EXAMPLE=OFF \
+        -DBUILD_ELASTICLIENT_TESTS=OFF \
+        -DBUILD_SHARED_LIBS=ON \
+        -DUSE_SYSTEM_CPR=ON \
+        -DUSE_SYSTEM_JSONCPP=ON; \
+    cmake --build /tmp/elasticlient-build --parallel "$(nproc)"; \
+    cmake --install /tmp/elasticlient-build; \
+    rm -rf /tmp/elasticlient /tmp/elasticlient-build
+
+RUN set -euo pipefail; \
+    . /tmp/dependencies.lock; \
+    git init /tmp/amqp-cpp; \
+    git -C /tmp/amqp-cpp remote add origin https://github.com/CopernicaMarketingSoftware/AMQP-CPP.git; \
+    git -C /tmp/amqp-cpp fetch --depth 1 origin "$AMQP_CPP_REV"; \
+    git -C /tmp/amqp-cpp checkout --detach FETCH_HEAD; \
+    test "$(git -C /tmp/amqp-cpp rev-parse HEAD)" = "$AMQP_CPP_REV"; \
+    cmake -S /tmp/amqp-cpp -B /tmp/amqp-cpp-build -G Ninja \
+        -DCMAKE_BUILD_TYPE=Release \
+        -DCMAKE_INSTALL_PREFIX=/usr/local \
+        -DAMQP-CPP_BUILD_EXAMPLES=OFF \
+        -DAMQP-CPP_BUILD_SHARED=ON; \
+    cmake --build /tmp/amqp-cpp-build --parallel "$(nproc)"; \
+    cmake --install /tmp/amqp-cpp-build; \
+    rm -rf /tmp/amqp-cpp /tmp/amqp-cpp-build
+
+RUN set -euo pipefail; \
+    . /tmp/dependencies.lock; \
+    git init /tmp/aws-sdk-cpp; \
+    git -C /tmp/aws-sdk-cpp remote add origin https://github.com/aws/aws-sdk-cpp.git; \
+    git -C /tmp/aws-sdk-cpp fetch --depth 1 origin "$AWS_SDK_CPP_REV"; \
+    git -C /tmp/aws-sdk-cpp checkout --detach FETCH_HEAD; \
+    test "$(git -C /tmp/aws-sdk-cpp rev-parse HEAD)" = "$AWS_SDK_CPP_REV"; \
+    cmake -S /tmp/aws-sdk-cpp -B /tmp/aws-sdk-cpp-build -G Ninja \
+        -DBUILD_ONLY=s3 \
+        -DBUILD_SHARED_LIBS=ON \
+        -DCMAKE_BUILD_TYPE=Release \
+        -DCMAKE_INSTALL_PREFIX=/usr/local \
+        -DENABLE_TESTING=OFF; \
+    cmake --build /tmp/aws-sdk-cpp-build --parallel "$(nproc)"; \
+    cmake --install /tmp/aws-sdk-cpp-build; \
+    rm -rf /tmp/aws-sdk-cpp /tmp/aws-sdk-cpp-build
+
+RUN printf '%s\n' '/usr/local/lib' '/usr/local/lib64' > /etc/ld.so.conf.d/chatnow-local.conf \
+    && ldconfig
+
+WORKDIR /workspace
diff --git a/docker/ci/dependencies.lock b/docker/ci/dependencies.lock
new file mode 100644
index 0000000..6fb6ee0
--- /dev/null
+++ b/docker/ci/dependencies.lock
@@ -0,0 +1,10 @@
+# Base image and source dependency revisions used by docker/ci/Dockerfile.
+# Git dependencies are full commit IDs; do not replace them with branches or tags.
+UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
+BRPC_REV=041cec5fb84a5b4458bac6275ea7d34e048bc3f1
+ETCD_CPP_APIV3_REV=ba6216385fc332b23d95683966824c2b86c2474e
+REDIS_PLUS_PLUS_REV=a63ac43bf192772910b52e27cd2b42a6098a0071
+CPR_REV=fe29aee4ca4bfc9d17802e7623c2ee574b1d1e9b
+ELASTICLIENT_REV=d68e30e382b5f2817be8cd901494736b26d4896e
+AMQP_CPP_REV=ca49382bfc5bc165dfb7988b891bb010a939a786
+AWS_SDK_CPP_REV=2cde9b1786bdbb3182faa93ce28d6e44ac2fe7e0
diff --git a/scripts/package_compose_artifacts.sh b/scripts/package_compose_artifacts.sh
new file mode 100755
index 0000000..f227fdb
--- /dev/null
+++ b/scripts/package_compose_artifacts.sh
@@ -0,0 +1,48 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+services=(conversation gateway identity media message presence push relationship transmite)
+build_root="${1:-build}"
+artifact_root="${2:-compose-artifacts}"
+
+rm -rf "$artifact_root"
+mkdir -p "$artifact_root"
+
+for service in "${services[@]}"; do
+    binary="$build_root/$service/${service}_server"
+    [[ -x "$binary" ]] || {
+        echo "missing executable service binary: $binary" >&2
+        exit 1
+    }
+
+    service_root="$artifact_root/$service"
+    depends_dir="$service_root/depends"
+    mkdir -p "$service_root/build" "$depends_dir"
+    cp -p "$binary" "$service_root/build/${service}_server"
+
+    ldd_output="$(ldd "$binary" 2>&1)" || {
+        echo "ldd failed for $binary: $ldd_output" >&2
+        exit 1
+    }
+    if grep -Fq "not found" <<<"$ldd_output"; then
+        echo "unresolved shared library for $binary:" >&2
+        echo "$ldd_output" >&2
+        exit 1
+    fi
+
+    while IFS= read -r library; do
+        [[ -n "$library" ]] || continue
+        library_name="$(basename "$library")"
+        cp -L "$library" "$depends_dir/$library_name"
+    done < <(awk '
+        /=> \/[^ ]+/ { print $3; next }
+        /^[[:space:]]*\/[^ ]+/ { print $1 }
+    ' <<<"$ldd_output" | LC_ALL=C sort -u)
+done
+
+(
+    cd "$artifact_root"
+    find . -type f ! -name MANIFEST.sha256 -print0 \
+        | LC_ALL=C sort -z \
+        | xargs -0 sha256sum > MANIFEST.sha256
+)
diff --git a/scripts/validate_compose_artifacts.sh b/scripts/validate_compose_artifacts.sh
new file mode 100755
index 0000000..94ba184
--- /dev/null
+++ b/scripts/validate_compose_artifacts.sh
@@ -0,0 +1,51 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+services=(conversation gateway identity media message presence push relationship transmite)
+artifact_root="${1:-compose-artifacts}"
+manifest="$artifact_root/MANIFEST.sha256"
+
+if [[ ! -f "$manifest" ]]; then
+    echo "missing artifact manifest: $manifest" >&2
+    exit 1
+fi
+
+(
+    cd "$artifact_root"
+    sha256sum --check MANIFEST.sha256
+)
+
+for service in "${services[@]}"; do
+    binary="$artifact_root/$service/build/${service}_server"
+    depends_dir="$artifact_root/$service/depends"
+    [[ -x "$binary" ]] || {
+        echo "missing executable service binary: $binary" >&2
+        exit 1
+    }
+    if [[ ! -d "$depends_dir" ]]; then
+        echo "missing shared-library directory: $depends_dir" >&2
+        exit 1
+    fi
+
+    ldd_output="$(env -i PATH=/usr/bin:/bin LD_LIBRARY_PATH="$depends_dir" ldd "$binary" 2>&1)" || {
+        echo "isolated ldd failed for $binary: $ldd_output" >&2
+        exit 1
+    }
+    if grep -Fq "not found" <<<"$ldd_output"; then
+        echo "unresolved shared library for $binary:" >&2
+        echo "$ldd_output" >&2
+        exit 1
+    fi
+
+    while IFS= read -r library; do
+        [[ -n "$library" ]] || continue
+        library_name="$(basename "$library")"
+        if [[ ! -f "$depends_dir/$library_name" ]]; then
+            echo "shared library is outside packaged closure for $binary: $library" >&2
+            exit 1
+        fi
+    done < <(awk '
+        /=> \/[^ ]+/ { print $3; next }
+        /^[[:space:]]*\/[^ ]+/ { print $1 }
+    ' <<<"$ldd_output" | LC_ALL=C sort -u)
+done
diff --git a/tests/pkg/contracts/artifacts_test.go b/tests/pkg/contracts/artifacts_test.go
new file mode 100644
index 0000000..146a2c5
--- /dev/null
+++ b/tests/pkg/contracts/artifacts_test.go
@@ -0,0 +1,115 @@
+package contracts
+
+import (
+	"os"
+	"os/exec"
+	"path/filepath"
+	"regexp"
+	"strings"
+	"testing"
+
+	"github.com/stretchr/testify/require"
+)
+
+var composeArtifactServices = []string{
+	"conversation", "gateway", "identity", "media", "message",
+	"presence", "push", "relationship", "transmite",
+}
+
+func TestComposeArtifactBuilderIsLocked(t *testing.T) {
+	root := repositoryRoot(t)
+	dockerfile := readContractFile(t, root, "docker/ci/Dockerfile")
+	lockfile := readContractFile(t, root, "docker/ci/dependencies.lock")
+
+	imageLock := regexp.MustCompile(`(?m)^UBUNTU_IMAGE=(ubuntu:24\.04@sha256:[0-9a-f]{64})$`).FindStringSubmatch(lockfile)
+	require.Len(t, imageLock, 2, "the Ubuntu builder image must be locked by tag and digest")
+	require.Contains(t, dockerfile, "ARG UBUNTU_IMAGE="+imageLock[1])
+	require.Contains(t, dockerfile, "FROM ${UBUNTU_IMAGE}")
+	require.Contains(t, dockerfile, "COPY docker/ci/dependencies.lock")
+	require.Contains(t, dockerfile, ". /tmp/dependencies.lock")
+	require.NotContains(t, strings.ToLower(dockerfile), ":latest")
+	require.NotRegexp(t, regexp.MustCompile(`(?m)git (clone|checkout).*(main|master)(\s|$)`), dockerfile)
+
+	for _, dependency := range []string{
+		"BRPC_REV", "ETCD_CPP_APIV3_REV", "REDIS_PLUS_PLUS_REV", "CPR_REV",
+		"ELASTICLIENT_REV", "AMQP_CPP_REV", "AWS_SDK_CPP_REV",
+	} {
+		revision := regexp.MustCompile(`(?m)^` + dependency + `=([0-9a-f]{40})$`).FindStringSubmatch(lockfile)
+		require.Len(t, revision, 2, "%s must be an immutable 40-character Git revision", dependency)
+		require.Contains(t, dockerfile, `"$`+dependency+`"`, "%s must be consumed by the builder", dependency)
+	}
+	require.Contains(t, dockerfile, "ldconfig")
+}
+
+func TestComposeArtifactPackagerContract(t *testing.T) {
+	root := repositoryRoot(t)
+	scriptPath := filepath.Join(root, "scripts/package_compose_artifacts.sh")
+	script := readContractFile(t, root, "scripts/package_compose_artifacts.sh")
+
+	require.Contains(t, script, "set -euo pipefail")
+	assertExplicitArtifactServices(t, script)
+	require.Contains(t, script, `build_root="${1:-build}"`)
+	require.Contains(t, script, `artifact_root="${2:-compose-artifacts}"`)
+	require.Contains(t, script, `"$build_root/$service/${service}_server"`)
+	require.Contains(t, script, `[[ -x "$binary" ]]`)
+	require.Contains(t, script, "not found")
+	require.Contains(t, script, "sha256sum")
+	require.Contains(t, script, "sort -z")
+
+	t.Run("rejects missing service binary", func(t *testing.T) {
+		buildRoot := filepath.Join(t.TempDir(), "build")
+		result := exec.Command("bash", scriptPath, buildRoot, filepath.Join(t.TempDir(), "artifacts"))
+		output, err := result.CombinedOutput()
+		require.Error(t, err)
+		require.Contains(t, string(output), "missing executable service binary")
+	})
+
+	t.Run("rejects unresolved ldd dependency", func(t *testing.T) {
+		temp := t.TempDir()
+		buildRoot := filepath.Join(temp, "build")
+		for _, service := range composeArtifactServices {
+			binary := filepath.Join(buildRoot, service, service+"_server")
+			require.NoError(t, os.MkdirAll(filepath.Dir(binary), 0o755))
+			require.NoError(t, os.WriteFile(binary, []byte("#!/bin/sh\nexit 0\n"), 0o755))
+		}
+		binDir := filepath.Join(temp, "bin")
+		require.NoError(t, os.MkdirAll(binDir, 0o755))
+		ldd := filepath.Join(binDir, "ldd")
+		require.NoError(t, os.WriteFile(ldd, []byte("#!/bin/sh\necho 'libmissing.so => not found'\n"), 0o755))
+
+		result := exec.Command("bash", scriptPath, buildRoot, filepath.Join(temp, "artifacts"))
+		result.Env = append(os.Environ(), "PATH="+binDir+":"+os.Getenv("PATH"))
+		output, err := result.CombinedOutput()
+		require.Error(t, err)
+		require.Contains(t, string(output), "unresolved shared library")
+	})
+}
+
+func TestComposeArtifactValidatorContract(t *testing.T) {
+	root := repositoryRoot(t)
+	script := readContractFile(t, root, "scripts/validate_compose_artifacts.sh")
+
+	require.Contains(t, script, "set -euo pipefail")
+	assertExplicitArtifactServices(t, script)
+	require.Contains(t, script, `artifact_root="${1:-compose-artifacts}"`)
+	require.Contains(t, script, "sha256sum --check")
+	require.Contains(t, script, `[[ -x "$binary" ]]`)
+	require.Contains(t, script, "env -i")
+	require.Contains(t, script, `LD_LIBRARY_PATH="$depends_dir"`)
+	require.Contains(t, script, "not found")
+	require.Contains(t, script, `"$depends_dir/$library_name"`)
+}
+
+func assertExplicitArtifactServices(t *testing.T, script string) {
+	t.Helper()
+	serviceList := strings.Join(composeArtifactServices, " ")
+	require.Contains(t, script, "services=("+serviceList+")",
+		"the nine Compose services must be enumerated explicitly and in a stable order")
+}
+
+func readContractFile(t *testing.T, root, name string) string {
+	t.Helper()
+	contents, err := os.ReadFile(filepath.Join(root, filepath.FromSlash(name)))
+	require.NoError(t, err, "%s must exist", name)
+	return string(contents)
+}

From 9bcc52f1e16df41c0dc53cfcc0f7a6550b520202 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 18:46:11 +0800
Subject: [PATCH 35/47] fix(ci): harden compose artifact validation

---
 docker/ci/Dockerfile                  |   3 +
 scripts/package_compose_artifacts.sh  |   8 +-
 scripts/validate_compose_artifacts.sh |  43 +++++-
 tests/pkg/contracts/artifacts_test.go | 188 +++++++++++++++++++++++++-
 4 files changed, 232 insertions(+), 10 deletions(-)

diff --git a/docker/ci/Dockerfile b/docker/ci/Dockerfile
index c407d82..a4c9b06 100644
--- a/docker/ci/Dockerfile
+++ b/docker/ci/Dockerfile
@@ -157,6 +157,9 @@ RUN set -euo pipefail; \
     git -C /tmp/aws-sdk-cpp fetch --depth 1 origin "$AWS_SDK_CPP_REV"; \
     git -C /tmp/aws-sdk-cpp checkout --detach FETCH_HEAD; \
     test "$(git -C /tmp/aws-sdk-cpp rev-parse HEAD)" = "$AWS_SDK_CPP_REV"; \
+    git -C /tmp/aws-sdk-cpp submodule sync --recursive; \
+    git -C /tmp/aws-sdk-cpp submodule update --init --recursive --depth 1; \
+    test -z "$(git -C /tmp/aws-sdk-cpp submodule status --recursive | grep -E '^[+-]' || true)"; \
     cmake -S /tmp/aws-sdk-cpp -B /tmp/aws-sdk-cpp-build -G Ninja \
         -DBUILD_ONLY=s3 \
         -DBUILD_SHARED_LIBS=ON \
diff --git a/scripts/package_compose_artifacts.sh b/scripts/package_compose_artifacts.sh
index f227fdb..61987b7 100755
--- a/scripts/package_compose_artifacts.sh
+++ b/scripts/package_compose_artifacts.sh
@@ -4,6 +4,8 @@ set -euo pipefail
 services=(conversation gateway identity media message presence push relationship transmite)
 build_root="${1:-build}"
 artifact_root="${2:-compose-artifacts}"
+ldd_command="${LDD:-ldd}"
+sha256sum_command="${SHA256SUM:-sha256sum}"
 
 rm -rf "$artifact_root"
 mkdir -p "$artifact_root"
@@ -20,7 +22,7 @@ for service in "${services[@]}"; do
     mkdir -p "$service_root/build" "$depends_dir"
     cp -p "$binary" "$service_root/build/${service}_server"
 
-    ldd_output="$(ldd "$binary" 2>&1)" || {
+    ldd_output="$("$ldd_command" "$binary" 2>&1)" || {
         echo "ldd failed for $binary: $ldd_output" >&2
         exit 1
     }
@@ -42,7 +44,7 @@ done
 
 (
     cd "$artifact_root"
-    find . -type f ! -name MANIFEST.sha256 -print0 \
+    find . \( -type f -o -type l \) ! -name MANIFEST.sha256 -print0 \
         | LC_ALL=C sort -z \
-        | xargs -0 sha256sum > MANIFEST.sha256
+        | xargs -0 "$sha256sum_command" > MANIFEST.sha256
 )
diff --git a/scripts/validate_compose_artifacts.sh b/scripts/validate_compose_artifacts.sh
index 94ba184..afef4d6 100755
--- a/scripts/validate_compose_artifacts.sh
+++ b/scripts/validate_compose_artifacts.sh
@@ -4,15 +4,27 @@ set -euo pipefail
 services=(conversation gateway identity media message presence push relationship transmite)
 artifact_root="${1:-compose-artifacts}"
 manifest="$artifact_root/MANIFEST.sha256"
+ldd_command="${LDD:-ldd}"
+sha256sum_command="${SHA256SUM:-sha256sum}"
 
 if [[ ! -f "$manifest" ]]; then
     echo "missing artifact manifest: $manifest" >&2
     exit 1
 fi
 
+actual_manifest="$(mktemp)"
+trap 'rm -f "$actual_manifest"' EXIT
 (
     cd "$artifact_root"
-    sha256sum --check MANIFEST.sha256
+    find . \( -type f -o -type l \) ! -name MANIFEST.sha256 -print0 \
+        | LC_ALL=C sort -z \
+        | xargs -0 "$sha256sum_command" > "$actual_manifest"
+    if ! cmp -s MANIFEST.sha256 "$actual_manifest"; then
+        echo "manifest mismatch or unlisted artifact file" >&2
+        exit 1
+    fi
+    # sha256sum --check is retained as an explicit integrity gate.
+    "$sha256sum_command" --check MANIFEST.sha256
 )
 
 for service in "${services[@]}"; do
@@ -26,8 +38,9 @@ for service in "${services[@]}"; do
         echo "missing shared-library directory: $depends_dir" >&2
         exit 1
     fi
+    depends_dir_real="$(cd "$depends_dir" && pwd -P)"
 
-    ldd_output="$(env -i PATH=/usr/bin:/bin LD_LIBRARY_PATH="$depends_dir" ldd "$binary" 2>&1)" || {
+    ldd_output="$(env -i PATH=/usr/bin:/bin LD_LIBRARY_PATH="$depends_dir" "$ldd_command" "$binary" 2>&1)" || {
         echo "isolated ldd failed for $binary: $ldd_output" >&2
         exit 1
     }
@@ -37,15 +50,33 @@ for service in "${services[@]}"; do
         exit 1
     fi
 
-    while IFS= read -r library; do
+    while IFS=$'\t' read -r entry_kind library; do
         [[ -n "$library" ]] || continue
         library_name="$(basename "$library")"
-        if [[ ! -f "$depends_dir/$library_name" ]]; then
+        if [[ "$entry_kind" == "loader" ]]; then
+            case "$library_name" in
+                ld-linux*.so.*|ld-musl-*.so.*) continue ;;
+                *)
+                    echo "unexpected system library outside packaged closure for $binary: $library" >&2
+                    exit 1
+                    ;;
+            esac
+        fi
+
+        if [[ ! -f "$library" ]]; then
             echo "shared library is outside packaged closure for $binary: $library" >&2
             exit 1
         fi
+        resolved_library="$(realpath "$library")"
+        case "$resolved_library" in
+            "$depends_dir_real"/*) ;;
+            *)
+                echo "shared library is outside packaged closure for $binary: $library" >&2
+                exit 1
+                ;;
+        esac
     done < <(awk '
-        /=> \/[^ ]+/ { print $3; next }
-        /^[[:space:]]*\/[^ ]+/ { print $1 }
+        /=> \/[^ ]+/ { print "dependency\t" $3; next }
+        /^[[:space:]]*\/[^ ]+/ { print "loader\t" $1 }
     ' <<<"$ldd_output" | LC_ALL=C sort -u)
 done
diff --git a/tests/pkg/contracts/artifacts_test.go b/tests/pkg/contracts/artifacts_test.go
index 146a2c5..4818409 100644
--- a/tests/pkg/contracts/artifacts_test.go
+++ b/tests/pkg/contracts/artifacts_test.go
@@ -29,6 +29,8 @@ func TestComposeArtifactBuilderIsLocked(t *testing.T) {
 	require.Contains(t, dockerfile, ". /tmp/dependencies.lock")
 	require.NotContains(t, strings.ToLower(dockerfile), ":latest")
 	require.NotRegexp(t, regexp.MustCompile(`(?m)git (clone|checkout).*(main|master)(\s|$)`), dockerfile)
+	require.Contains(t, dockerfile, "git -C /tmp/aws-sdk-cpp submodule update --init --recursive --depth 1")
+	require.NotContains(t, dockerfile, "submodule update --remote")
 
 	for _, dependency := range []string{
 		"BRPC_REV", "ETCD_CPP_APIV3_REV", "REDIS_PLUS_PLUS_REV", "CPR_REV",
@@ -97,7 +99,191 @@ func TestComposeArtifactValidatorContract(t *testing.T) {
 	require.Contains(t, script, "env -i")
 	require.Contains(t, script, `LD_LIBRARY_PATH="$depends_dir"`)
 	require.Contains(t, script, "not found")
-	require.Contains(t, script, `"$depends_dir/$library_name"`)
+	require.Contains(t, script, "depends_dir_real")
+	require.Contains(t, script, "resolved_library")
+}
+
+func TestComposeArtifactScriptsEndToEnd(t *testing.T) {
+	root := repositoryRoot(t)
+
+	t.Run("packages all services and validates manifest", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.packageArtifacts(t)
+
+		manifest := readContractFile(t, fixture.artifactRoot, "MANIFEST.sha256")
+		for _, service := range composeArtifactServices {
+			require.FileExists(t, filepath.Join(fixture.artifactRoot, service, "build", service+"_server"))
+			require.FileExists(t, filepath.Join(fixture.artifactRoot, service, "depends", "libfixture.so"))
+			require.Contains(t, manifest, "./"+service+"/build/"+service+"_server")
+			require.Contains(t, manifest, "./"+service+"/depends/libfixture.so")
+		}
+
+		output, err := fixture.validate(t, fixture.lddPath)
+		require.NoError(t, err, "%s", output)
+	})
+
+	t.Run("rejects manifest tampering", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.packageArtifacts(t)
+		tampered := filepath.Join(fixture.artifactRoot, "identity", "build", "identity_server")
+		file, err := os.OpenFile(tampered, os.O_APPEND|os.O_WRONLY, 0)
+		require.NoError(t, err)
+		_, err = file.WriteString("tampered\n")
+		require.NoError(t, err)
+		require.NoError(t, file.Close())
+
+		output, err := fixture.validate(t, fixture.lddPath)
+		require.Error(t, err)
+		require.Contains(t, output, "manifest")
+	})
+
+	t.Run("rejects missing packaged library", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.packageArtifacts(t)
+		require.NoError(t, os.Remove(filepath.Join(fixture.artifactRoot, "media", "depends", "libfixture.so")))
+		fixture.rewriteManifest(t)
+
+		output, err := fixture.validate(t, fixture.lddPath)
+		require.Error(t, err)
+		require.Contains(t, output, "outside packaged closure")
+	})
+
+	t.Run("rejects host library fallback with matching basename", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.packageArtifacts(t)
+		hostLibrary := filepath.Join(fixture.temp, "host", "libfixture.so")
+		require.NoError(t, os.MkdirAll(filepath.Dir(hostLibrary), 0o755))
+		require.NoError(t, os.WriteFile(hostLibrary, []byte("host library\n"), 0o644))
+		fallbackLDD := fixture.writeLDD(t, filepath.Join(fixture.tools, "ldd-host"), hostLibrary)
+
+		output, err := fixture.validate(t, fallbackLDD)
+		require.Error(t, err)
+		require.Contains(t, output, "outside packaged closure")
+	})
+
+	t.Run("rejects packaged symlink escaping to host library", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.packageArtifacts(t)
+		hostLibrary := filepath.Join(fixture.temp, "host", "libfixture.so")
+		require.NoError(t, os.MkdirAll(filepath.Dir(hostLibrary), 0o755))
+		require.NoError(t, os.WriteFile(hostLibrary, []byte("host library\n"), 0o644))
+		packagedLibrary := filepath.Join(fixture.artifactRoot, "push", "depends", "libfixture.so")
+		require.NoError(t, os.Remove(packagedLibrary))
+		require.NoError(t, os.Symlink(hostLibrary, packagedLibrary))
+		fixture.rewriteManifest(t)
+
+		output, err := fixture.validate(t, fixture.lddPath)
+		require.Error(t, err)
+		require.Contains(t, output, "outside packaged closure")
+	})
+
+	t.Run("rejects unlisted artifact file", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.packageArtifacts(t)
+		require.NoError(t, os.WriteFile(filepath.Join(fixture.artifactRoot, "unlisted.txt"), []byte("extra\n"), 0o644))
+
+		output, err := fixture.validate(t, fixture.lddPath)
+		require.Error(t, err)
+		require.Contains(t, output, "unlisted artifact file")
+	})
+}
+
+type artifactFixture struct {
+	root          string
+	temp          string
+	buildRoot     string
+	artifactRoot  string
+	tools         string
+	sha256sumPath string
+	lddPath       string
+}
+
+func newArtifactFixture(t *testing.T, root string) artifactFixture {
+	t.Helper()
+	temp := t.TempDir()
+	fixture := artifactFixture{
+		root:         root,
+		temp:         temp,
+		buildRoot:    filepath.Join(temp, "build"),
+		artifactRoot: filepath.Join(temp, "compose-artifacts"),
+		tools:        filepath.Join(temp, "tools"),
+	}
+	require.NoError(t, os.MkdirAll(fixture.tools, 0o755))
+	fixture.sha256sumPath = filepath.Join(fixture.tools, "sha256sum")
+	require.NoError(t, os.WriteFile(fixture.sha256sumPath, []byte(`#!/bin/sh
+set -eu
+checksum() { cksum "$1" | awk '{ print $1 ":" $2 }'; }
+if [ "${1:-}" = "--check" ]; then
+    manifest="$2"
+    status=0
+    while read -r expected file; do
+        file="${file# }"
+        actual="$(checksum "$file")"
+        if [ "$actual" != "$expected" ]; then
+            echo "$file: FAILED" >&2
+            status=1
+        fi
+    done < "$manifest"
+    exit "$status"
+fi
+for file in "$@"; do
+    printf '%s  %s\n' "$(checksum "$file")" "$file"
+done
+`), 0o755))
+
+	fixture.lddPath = fixture.writeLDD(t, filepath.Join(fixture.tools, "ldd"), "")
+	for _, service := range composeArtifactServices {
+		binary := filepath.Join(fixture.buildRoot, service, service+"_server")
+		require.NoError(t, os.MkdirAll(filepath.Dir(binary), 0o755))
+		require.NoError(t, os.WriteFile(binary, []byte("#!/bin/sh\nexit 0\n"), 0o755))
+	}
+	return fixture
+}
+
+func (fixture artifactFixture) writeLDD(t *testing.T, path, forcedLibrary string) string {
+	t.Helper()
+	script := "#!/bin/sh\nset -eu\n"
+	if forcedLibrary != "" {
+		script += "echo 'libfixture.so => " + forcedLibrary + " (0x1)'\n"
+	} else {
+		script += `if [ -n "${LD_LIBRARY_PATH:-}" ]; then
+    library="$LD_LIBRARY_PATH/libfixture.so"
+else
+    library="` + filepath.Join(fixture.temp, "libfixture.so") + `"
+fi
+echo "libfixture.so => $library (0x1)"
+`
+	}
+	require.NoError(t, os.WriteFile(path, []byte(script), 0o755))
+	if forcedLibrary == "" {
+		require.NoError(t, os.WriteFile(filepath.Join(fixture.temp, "libfixture.so"), []byte("fixture library\n"), 0o644))
+	}
+	return path
+}
+
+func (fixture artifactFixture) packageArtifacts(t *testing.T) {
+	t.Helper()
+	command := exec.Command("bash", filepath.Join(fixture.root, "scripts/package_compose_artifacts.sh"), fixture.buildRoot, fixture.artifactRoot)
+	command.Env = append(os.Environ(), "PATH="+fixture.tools+":"+os.Getenv("PATH"))
+	output, err := command.CombinedOutput()
+	require.NoError(t, err, "%s", output)
+}
+
+func (fixture artifactFixture) validate(t *testing.T, lddPath string) (string, error) {
+	t.Helper()
+	command := exec.Command("bash", filepath.Join(fixture.root, "scripts/validate_compose_artifacts.sh"), fixture.artifactRoot)
+	command.Env = append(os.Environ(), "LDD="+lddPath, "SHA256SUM="+fixture.sha256sumPath)
+	output, err := command.CombinedOutput()
+	return string(output), err
+}
+
+func (fixture artifactFixture) rewriteManifest(t *testing.T) {
+	t.Helper()
+	command := exec.Command("bash", "-c", `find . \( -type f -o -type l \) ! -name MANIFEST.sha256 -print0 | LC_ALL=C sort -z | xargs -0 "$SHA256SUM" > MANIFEST.sha256`)
+	command.Dir = fixture.artifactRoot
+	command.Env = append(os.Environ(), "SHA256SUM="+fixture.sha256sumPath)
+	output, err := command.CombinedOutput()
+	require.NoError(t, err, "%s", output)
 }
 
 func assertExplicitArtifactServices(t *testing.T, script string) {

From 45d90427ef7547af5b8cabbeef5d2d1478be7317 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 18:49:44 +0800
Subject: [PATCH 36/47] fix(ci): canonicalize artifact validation root

---
 scripts/validate_compose_artifacts.sh |  5 ++++
 tests/pkg/contracts/artifacts_test.go | 33 +++++++++++++++++++++++++++
 2 files changed, 38 insertions(+)

diff --git a/scripts/validate_compose_artifacts.sh b/scripts/validate_compose_artifacts.sh
index afef4d6..f94caa2 100755
--- a/scripts/validate_compose_artifacts.sh
+++ b/scripts/validate_compose_artifacts.sh
@@ -3,6 +3,11 @@ set -euo pipefail
 
 services=(conversation gateway identity media message presence push relationship transmite)
 artifact_root="${1:-compose-artifacts}"
+if [[ ! -d "$artifact_root" ]]; then
+    echo "missing artifact root directory: $artifact_root" >&2
+    exit 1
+fi
+artifact_root="$(cd "$artifact_root" && pwd -P)"
 manifest="$artifact_root/MANIFEST.sha256"
 ldd_command="${LDD:-ldd}"
 sha256sum_command="${SHA256SUM:-sha256sum}"
diff --git a/tests/pkg/contracts/artifacts_test.go b/tests/pkg/contracts/artifacts_test.go
index 4818409..6d3f713 100644
--- a/tests/pkg/contracts/artifacts_test.go
+++ b/tests/pkg/contracts/artifacts_test.go
@@ -177,6 +177,30 @@ func TestComposeArtifactScriptsEndToEnd(t *testing.T) {
 		require.Contains(t, output, "outside packaged closure")
 	})
 
+	t.Run("validates a relative artifact root", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.packageArtifacts(t)
+
+		output, err := fixture.validateRelative(t, fixture.lddPath)
+		require.NoError(t, err, "%s", output)
+	})
+
+	t.Run("rejects relative artifact root symlink escape", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.packageArtifacts(t)
+		hostLibrary := filepath.Join(fixture.temp, "host", "libfixture.so")
+		require.NoError(t, os.MkdirAll(filepath.Dir(hostLibrary), 0o755))
+		require.NoError(t, os.WriteFile(hostLibrary, []byte("host library\n"), 0o644))
+		packagedLibrary := filepath.Join(fixture.artifactRoot, "relationship", "depends", "libfixture.so")
+		require.NoError(t, os.Remove(packagedLibrary))
+		require.NoError(t, os.Symlink(hostLibrary, packagedLibrary))
+		fixture.rewriteManifest(t)
+
+		output, err := fixture.validateRelative(t, fixture.lddPath)
+		require.Error(t, err)
+		require.Contains(t, output, "outside packaged closure")
+	})
+
 	t.Run("rejects unlisted artifact file", func(t *testing.T) {
 		fixture := newArtifactFixture(t, root)
 		fixture.packageArtifacts(t)
@@ -277,6 +301,15 @@ func (fixture artifactFixture) validate(t *testing.T, lddPath string) (string, e
 	return string(output), err
 }
 
+func (fixture artifactFixture) validateRelative(t *testing.T, lddPath string) (string, error) {
+	t.Helper()
+	command := exec.Command("bash", filepath.Join(fixture.root, "scripts/validate_compose_artifacts.sh"), filepath.Base(fixture.artifactRoot))
+	command.Dir = fixture.temp
+	command.Env = append(os.Environ(), "LDD="+lddPath, "SHA256SUM="+fixture.sha256sumPath)
+	output, err := command.CombinedOutput()
+	return string(output), err
+}
+
 func (fixture artifactFixture) rewriteManifest(t *testing.T) {
 	t.Helper()
 	command := exec.Command("bash", "-c", `find . \( -type f -o -type l \) ! -name MANIFEST.sha256 -print0 | LC_ALL=C sort -z | xargs -0 "$SHA256SUM" > MANIFEST.sha256`)

From 262080cc4e45b7456390165408b4f0a5b0ae565b Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 18:59:28 +0800
Subject: [PATCH 37/47] docs(test): keep cache regressions in Go framework

---
 ...16-pr57-ci-artifacts-native-regressions.md | 51 ++++++++++---------
 ...-ci-artifacts-native-regressions-design.md | 49 +++++++++---------
 2 files changed, 51 insertions(+), 49 deletions(-)

diff --git a/docs/superpowers/plans/2026-07-16-pr57-ci-artifacts-native-regressions.md b/docs/superpowers/plans/2026-07-16-pr57-ci-artifacts-native-regressions.md
index 80bda39..026fa22 100644
--- a/docs/superpowers/plans/2026-07-16-pr57-ci-artifacts-native-regressions.md
+++ b/docs/superpowers/plans/2026-07-16-pr57-ci-artifacts-native-regressions.md
@@ -1,12 +1,12 @@
-# PR #57 CI Artifacts and Native Regressions Implementation Plan
+# PR #57 CI Artifacts and Go Regressions Implementation Plan
 
 > **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
 
-**Goal:** Produce runnable service artifacts once per workflow and automatically execute the two cache native regressions without weakening the Go full-stack gates.
+**Goal:** Produce runnable service artifacts once per workflow and automatically execute cache regressions through the existing Go full-stack framework.
 
-**Architecture:** A repository-owned, cached native builder feeds one artifact producer job. Consumer jobs download a validated Compose layout. CTest is enabled only for two explicitly scoped cache component regressions.
+**Architecture:** A repository-owned, cached native builder feeds one artifact producer job. Consumer jobs download a validated Compose layout. Regression coverage uses real service boundaries and existing Go test targets; temporary C++ tests are removed.
 
-**Tech Stack:** GitHub Actions, Docker BuildKit, Ubuntu 24.04, CMake/CTest, Bash, Go workflow contract tests, Docker Compose, Redis Cluster.
+**Tech Stack:** GitHub Actions, Docker BuildKit, Ubuntu 24.04, CMake, Bash, Go testing, Docker Compose, Redis Cluster.
 
 ## Global Constraints
 
@@ -14,7 +14,7 @@
 - Do not depend on an unpublished or mutable external builder image.
 - Missing native dependencies, binaries, shared libraries, or tests must fail closed; required work may not skip.
 - `reliability` and `perf-cache` must depend on and download the same immutable artifact before Compose startup.
-- Keep the native-test exception limited to `test_user_info_generation_fence` and `test_unacked_pending_ledger`; do not enable the legacy C++ suite or add gtest.
+- All automated regression tests use the existing Go framework; do not add CTest, gtest, or production-only timing hooks.
 - RL-05 and PF-09 remain Go full-stack gates using their existing Make targets.
 - Every full-stack job ends with one `docker compose down -v` step guarded by `if: always()`.
 
@@ -42,27 +42,28 @@
 - [ ] Build the builder image and run CMake plus packaging in it; expect nine binaries and a valid manifest.
 - [ ] Commit with message `build(ci): package reproducible service artifacts`.
 
-### Task 2: Register and execute the two native regressions
+### Task 2: Move the two regressions into the Go test framework
 
 **Files:**
-- Modify: `CMakeLists.txt`
 - Modify: `common/test/CMakeLists.txt`
-- Modify: `common/test/test_user_info_generation_fence.cc`
-- Modify: `common/test/test_unacked_pending_ledger.cc`
-- Create: `tests/pkg/contracts/native_regressions_test.go`
+- Delete: `common/test/test_user_info_generation_fence.cc`
+- Delete: `common/test/test_unacked_pending_ledger.cc`
+- Modify: `tests/func/cache_test.go`
+- Modify: `tests/pkg/client/http.go`
+- Modify: `tests/pkg/contracts/ci_gates_test.go`
 
 **Interfaces:**
-- Produces CMake option `CHATNOW_BUILD_CACHE_REGRESSION_TESTS`.
-- Produces CTest labels `cache-unit` and `redis-cluster`.
-- Consumes environment variable `CHATNOW_REDIS_CLUSTER_SEEDS` for the ledger test.
-
-- [ ] Write contract tests that require both exact targets, both `add_test` registrations, labels, timeouts, the opt-in option, and fail-closed dependency resolution when the option is enabled.
-- [ ] Run `cd tests && go test ./pkg/contracts -run 'TestNativeCacheRegression' -count=1`; confirm it fails on missing registrations.
-- [ ] Enable CTest at the root and add an OFF-by-default `CHATNOW_BUILD_CACHE_REGRESSION_TESTS` option.
-- [ ] Replace the current Redis-header conditional with required target/link discovery inside the enabled option. Register generation as `cache-unit`; register ledger as `redis-cluster`, serial, with an explicit timeout and seed environment.
-- [ ] Build with `-DCHATNOW_BUILD_CACHE_REGRESSION_TESTS=ON`, run the generation CTest, and confirm it passes.
-- [ ] Start a real Redis Cluster, run the ledger CTest with explicit seeds, and confirm it passes; stop the cluster.
-- [ ] Re-run the focused contract tests and commit with message `test(cache): register native atomicity regressions`.
+- Produces a direct protobuf HTTP client for the Push service test boundary.
+- Produces Go cache tests executed by `make test-func`.
+- Consumes the existing Redis Cluster and WebSocket helpers.
+
+- [ ] Write a failing Go contract test proving the temporary C++ files/target are still present and the Go Unacked business regression is absent.
+- [ ] Add a failing Go functional test that establishes a Push route, invokes PushToUser twice with one `user_seq` and different payloads, verifies one ZSET identity/latest HASH payload, sends WebSocket ACK, and verifies both indexes are removed.
+- [ ] Add the smallest direct protobuf HTTP helper needed to call the Push service without bypassing production serialization.
+- [ ] Strengthen the existing UserInfo invalidation Go scenario so its comments and assertions explicitly cover bounded stale-data publication and latest Redis repopulation without a timing-only production hook.
+- [ ] Remove both temporary C++ regression files and remove the Unacked CMake executable block.
+- [ ] Run the focused Go contract/helper tests and Go compile/vet checks; expect zero failures.
+- [ ] Commit with message `test(cache): move regressions into Go framework`.
 
 ### Task 3: Wire the artifact producer and consumers into CI
 
@@ -75,10 +76,10 @@
 - Artifact name: `compose-service-artifacts`.
 - Consumers: `reliability`, `perf-cache`.
 
-- [ ] Extend workflow contract tests to require one producer; BuildKit GHA caching; builder build, CMake build, generation CTest, package, validate, and upload in order; consumer `needs`, download, validate, Compose start, gate, and teardown in order; and execution of the Redis ledger CTest against the stack.
+- [ ] Extend workflow contract tests to require one producer; BuildKit GHA caching; builder build, package, validate, and upload in order; consumer `needs`, download, validate, Compose start, Go gate, and teardown in order.
 - [ ] Run `cd tests && go test ./pkg/contracts -count=1`; confirm the new assertions fail against the old workflow.
 - [ ] Add `service-artifacts` to the workflow using `docker/build-push-action` cache-to/cache-from `type=gha`, then execute the native build and packaging inside that image and upload `compose-artifacts` with `actions/upload-artifact`.
-- [ ] Make both consumers depend on the producer, download with `actions/download-artifact`, restore service-context directories, validate before Compose, and run the Redis ledger CTest after Redis Cluster readiness and before the Go gate.
+- [ ] Make both consumers depend on the producer, download with `actions/download-artifact`, restore service-context directories, validate before Compose, and run the existing Go gates after stack readiness.
 - [ ] Preserve PF-09 rate-limit overrides and strict final teardown behavior.
 - [ ] Run the full contract suite, YAML parse, and `docker compose config --quiet`; expect zero failures.
 - [ ] Commit with message `ci: distribute service artifacts to cache gates`.
@@ -90,7 +91,7 @@
 
 - [ ] Run `git diff --check` and inspect the complete branch diff.
 - [ ] Run `cd tests && PATH="$(go env GOPATH)/bin:$PATH" make proto` followed by `go test ./...`, tagged vet commands, and the PF-09 race helper command.
-- [ ] Build the builder from a clean Docker cache or pull-free context, build all services, package, validate, and run the generation test.
-- [ ] Start the downloaded-equivalent artifact layout with Compose, wait for services, run the ledger CTest, RL-05, and then tear down.
+- [ ] Run static builder checks, package/validator behavior tests, and all Go contract/helper suites; do not require a local nine-service native build.
+- [ ] Verify workflow ordering and that the Go functional/reliability/performance targets are the only regression entry points.
 - [ ] Push the branch and inspect the new GitHub Actions run. Do not claim green if an external failure remains.
 - [ ] Fetch review threads again and report which new threads are addressed. Do not reply to or resolve GitHub threads without explicit user authorization.
diff --git a/docs/superpowers/specs/2026-07-16-pr57-ci-artifacts-native-regressions-design.md b/docs/superpowers/specs/2026-07-16-pr57-ci-artifacts-native-regressions-design.md
index 187cf3c..26ff0f5 100644
--- a/docs/superpowers/specs/2026-07-16-pr57-ci-artifacts-native-regressions-design.md
+++ b/docs/superpowers/specs/2026-07-16-pr57-ci-artifacts-native-regressions-design.md
@@ -1,4 +1,4 @@
-# PR #57 CI Artifacts and Native Regressions Design
+# PR #57 CI Artifacts and Go Regressions Design
 
 ## Scope
 
@@ -6,7 +6,8 @@ Address the two review threads added after commit `f633006`:
 
 1. make the RL-05 and PF-09 jobs start a runnable service stack from a clean
    GitHub runner;
-2. execute the generation-fence and Unacked-ledger regressions automatically.
+2. execute generation-fence and Unacked-ledger business regressions through the
+   repository's Go test framework.
 
 The change must not hide a missing native toolchain, silently skip a test, or
 duplicate a full service build in every gate job.
@@ -38,27 +39,27 @@ download it before `docker compose up`, validate its manifest, and then run the
 existing Go gates. A failed producer blocks consumers instead of producing a
 misleading integration-test failure.
 
-## Native regression exception
+## Go regression coverage
 
-The unified test architecture remains Go black-box by default. Two narrow L0
-native component regressions are an explicit exception because the public
-service boundary cannot deterministically create their internal atomicity
-conditions without production test hooks:
+The unified test architecture is authoritative: tests are Go, exercise
+HTTP/protobuf or WebSocket service boundaries, and run through targets in
+`tests/Makefile`. This PR does not introduce CTest or retain temporary C++
+regression executables.
 
-- UserInfo generation CAS conflict must not publish stale bytes into L1;
-- retrying one Unacked `user_seq` with a different payload must keep one
-  identity and ACK must remove both indexes.
+The Unacked regression calls the real Push service twice with the same
+`user_seq` and different notification payloads after establishing an online
+device route. It verifies through Redis Cluster that the ZSET retains one
+stable identity and the HASH contains the second payload, then sends a real
+WebSocket ACK and verifies both indexes are removed.
 
-The exception does not introduce gtest or a general C++ test suite. CMake
-exposes `CHATNOW_BUILD_CACHE_REGRESSION_TESTS`, disabled by default and enabled
-by CI. When enabled, dependencies are required rather than detected with a
-skip-capable probe. Both executables are registered with CTest labels. The
-generation test runs in the producer. The ledger test runs against the real
-Redis Cluster in a Compose network with an explicit seed and a timeout.
+The UserInfo regression remains a full-stack cache-consistency scenario. It
+invalidates shared cache state through UpdateProfile, drives the Transmite
+lookup path, and verifies that the repopulated Redis value contains the latest
+profile after the documented bounded process-local L1 lifetime. The test
+asserts externally observable freshness; it does not add a production timing
+hook solely to force an internal CAS interleaving.
 
-The Go RL-05 and PF-09 suites remain the authoritative business and system
-gates; native tests only cover atomicity that cannot be made deterministic at
-the external API.
+RL-05 and PF-09 remain the authoritative system gates.
 
 ## Contract enforcement
 
@@ -68,8 +69,8 @@ The existing Go workflow contract tests will require:
 - build, package, validate, and upload steps in order;
 - both gate jobs to depend on the producer and download/validate before
   Compose startup;
-- the producer to enable and execute the generation native regression;
-- the Redis-backed ledger regression to execute against the stack;
+- the Go cache and Unacked regressions to execute through an existing Make
+  target against the stack;
 - no `continue-on-error`, conditional bypass, or skip-capable dependency
   detection on required steps;
 - teardown to remain the unique final step with `if: always()`.
@@ -81,13 +82,13 @@ The existing Go workflow contract tests will require:
 - Missing binary: packaging fails before upload.
 - Unresolved shared library: artifact validation fails before upload and again
   after download.
-- Redis Cluster unavailable: ledger CTest fails; it is never reported as
-  skipped.
+- Redis Cluster unavailable: the Go Unacked regression fails; it is never
+  reported as skipped.
 - Gate failure: job fails and teardown still runs.
 
 ## Non-goals
 
 - no refactor of all nine runtime Dockerfiles into multi-stage builds;
 - no externally managed or mutable CI image dependency;
-- no general revival of the old C++ gtest suite;
+- no CTest or revival of the old C++ test suite;
 - no change to cache, ACK, or message-watermark business semantics.

From e326572e69b4cf9ad4c748dae5ba4bacad9a8eb3 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 19:11:31 +0800
Subject: [PATCH 38/47] test(cache): move regressions into Go framework

---
 common/test/CMakeLists.txt                    |   6 -
 common/test/test_unacked_pending_ledger.cc    | 323 ------------------
 .../test/test_user_info_generation_fence.cc   |  67 ----
 tests/func/cache_test.go                      |  88 ++++-
 tests/pkg/client/http.go                      |  12 +-
 tests/pkg/contracts/ci_gates_test.go          |  74 ++++
 6 files changed, 171 insertions(+), 399 deletions(-)
 delete mode 100644 common/test/test_unacked_pending_ledger.cc
 delete mode 100644 common/test/test_user_info_generation_fence.cc

diff --git a/common/test/CMakeLists.txt b/common/test/CMakeLists.txt
index 0e5d6a4..e7e2233 100644
--- a/common/test/CMakeLists.txt
+++ b/common/test/CMakeLists.txt
@@ -89,12 +89,6 @@ if(CHATNOW_HAS_REDIS_PLUS_PLUS)
     set_property(TARGET test_redis_mutex_contract PROPERTY CXX_STANDARD_REQUIRED ON)
     INSTALL(TARGETS test_redis_mutex_contract RUNTIME DESTINATION bin)
 
-    add_executable(test_unacked_pending_ledger test_unacked_pending_ledger.cc)
-    set_property(TARGET test_unacked_pending_ledger PROPERTY CXX_STANDARD 17)
-    set_property(TARGET test_unacked_pending_ledger PROPERTY CXX_STANDARD_REQUIRED ON)
-    target_link_libraries(test_unacked_pending_ledger
-        -lredis++ -lhiredis -lbrpc -lspdlog -lfmt -lpthread)
-    INSTALL(TARGETS test_unacked_pending_ledger RUNTIME DESTINATION bin)
 endif()
 
 # FIXME(3.0): common_tests has gflags link order issue with brpc static lib
diff --git a/common/test/test_unacked_pending_ledger.cc b/common/test/test_unacked_pending_ledger.cc
deleted file mode 100644
index 0bd1c94..0000000
--- a/common/test/test_unacked_pending_ledger.cc
+++ /dev/null
@@ -1,323 +0,0 @@
-#include "dao/data_redis.hpp"
-
-#include <chrono>
-#include <cstdlib>
-#include <ctime>
-#include <iostream>
-#include <stdexcept>
-#include <string>
-#include <unistd.h>
-#include <utility>
-#include <vector>
-
-namespace {
-
-using chatnow::RedisClient;
-using chatnow::RedisClusterFactory;
-using chatnow::UnackedPush;
-
-void require(bool condition, const std::string &message) {
-    if (!condition) throw std::runtime_error(message);
-}
-
-std::string unique_component(const std::string &prefix) {
-    static unsigned long counter = 0;
-    return prefix + "-" + std::to_string(getpid()) + "-" +
-           std::to_string(static_cast<unsigned long>(std::time(nullptr))) + "-" +
-           std::to_string(++counter);
-}
-
-long long zcard(const RedisClient::ptr &redis, const std::string &key) {
-    static const std::string script = "return redis.call('ZCARD', KEYS[1])";
-    std::vector<std::string> keys = {key};
-    std::vector<std::string> args;
-    return redis->eval<long long>(script, keys.begin(), keys.end(),
-                                  args.begin(), args.end());
-}
-
-std::string zscore(const RedisClient::ptr &redis, const std::string &key,
-                   const std::string &member) {
-    static const std::string script = "return redis.call('ZSCORE', KEYS[1], ARGV[1])";
-    std::vector<std::string> keys = {key};
-    std::vector<std::string> args = {member};
-    return redis->eval<std::string>(script, keys.begin(), keys.end(),
-                                    args.begin(), args.end());
-}
-
-long long ttl(const RedisClient::ptr &redis, const std::string &key) {
-    static const std::string script = "return redis.call('TTL', KEYS[1])";
-    std::vector<std::string> keys = {key};
-    std::vector<std::string> args;
-    return redis->eval<long long>(script, keys.begin(), keys.end(),
-                                  args.begin(), args.end());
-}
-
-struct Fixture {
-    explicit Fixture(const RedisClient::ptr &client, std::string test_name)
-        : redis(client), ledger(client), uid(unique_component(std::move(test_name))),
-          device("device-" + std::to_string(getpid())),
-          pending_key(UnackedPush::key_for(uid, device)),
-          payload_key(UnackedPush::idx_key_for(uid, device)),
-          repair_key(UnackedPush::repair_key_for(uid, device)) {}
-
-    ~Fixture() {
-        try {
-            redis->del(pending_key);
-            redis->del(payload_key);
-            redis->del(repair_key);
-        } catch (...) {
-        }
-    }
-
-    RedisClient::ptr redis;
-    UnackedPush ledger;
-    std::string uid;
-    std::string device;
-    std::string pending_key;
-    std::string payload_key;
-    std::string repair_key;
-};
-
-void test_push_replaces_payload_without_changing_identity(const RedisClient::ptr &redis) {
-    Fixture fixture(redis, "push-replace");
-    const auto due_score = static_cast<long long>(std::time(nullptr)) - 60;
-
-    fixture.ledger.push(fixture.uid, fixture.device, 41, "payload-A", due_score);
-    fixture.ledger.push(fixture.uid, fixture.device, 41, "payload-B", due_score);
-
-    std::vector<std::string> members;
-    redis->zrange(fixture.pending_key, 0, -1, std::back_inserter(members));
-    require(zcard(redis, fixture.pending_key) == 1,
-            "re-push with changed payload must leave ZCARD == 1");
-    require(members.size() == 1, "the pending ZSET must contain one member");
-    require(members.front() == "41", "ZSET member must be the decimal user_seq only");
-
-    const auto due = fixture.ledger.peek_due(fixture.uid, fixture.device, 10, 0);
-    require(due.size() == 1, "re-pushed sequence must be returned exactly once");
-    require(due.front().first == 41 && due.front().second == "payload-B",
-            "due-read must return the replacement payload from the HASH");
-}
-
-void test_ack_removes_zset_and_hash_entries(const RedisClient::ptr &redis) {
-    Fixture fixture(redis, "ack-both");
-    fixture.ledger.push(fixture.uid, fixture.device, 52, "payload-A", 1);
-    fixture.ledger.push(fixture.uid, fixture.device, 52, "payload-B", 1);
-
-    fixture.ledger.ack(fixture.uid, fixture.device, 52);
-
-    std::vector<std::string> members;
-    redis->zrange(fixture.pending_key, 0, -1, std::back_inserter(members));
-    require(members.empty(),
-            "ACK must remove the ZSET member");
-    require(!redis->hget(fixture.payload_key, "52"),
-            "ACK must remove the HASH field");
-}
-
-void test_due_read_removes_both_orphan_directions(const RedisClient::ptr &redis) {
-    Fixture fixture(redis, "due-heal");
-    redis->zadd(fixture.pending_key, "61", 1);
-    redis->hset(fixture.payload_key, "62", "hash-only");
-
-    const auto due = fixture.ledger.peek_due(fixture.uid, fixture.device, 10, 0);
-
-    require(due.empty(), "orphan entries must never be returned as due");
-    std::vector<std::string> members;
-    redis->zrange(fixture.pending_key, 0, -1, std::back_inserter(members));
-    require(members.empty(),
-            "due-read must remove a ZSET-only orphan");
-    require(!redis->hget(fixture.payload_key, "62"),
-            "due-read bounded consistency pass must remove a HASH-only orphan");
-}
-
-void test_ack_wrong_type_does_not_partially_remove_zset(const RedisClient::ptr &redis) {
-    Fixture fixture(redis, "ack-wrong-type");
-    fixture.ledger.push(fixture.uid, fixture.device, 63, "payload", 1);
-    redis->del(fixture.payload_key);
-    redis->set(fixture.payload_key, "not-a-hash", std::chrono::seconds(300));
-
-    fixture.ledger.ack(fixture.uid, fixture.device, 63);
-
-    require(zcard(redis, fixture.pending_key) == 1,
-            "ACK must validate both key types before removing the ZSET member");
-}
-
-void test_due_read_progresses_across_hash_orphan_pages(const RedisClient::ptr &redis) {
-    Fixture fixture(redis, "due-progressive-heal");
-    fixture.ledger.push(fixture.uid, fixture.device, 9000, "complete",
-                        static_cast<long long>(std::time(nullptr)) + 3600,
-                        std::chrono::seconds(300));
-    for (unsigned long seq = 10000; seq < 10700; ++seq) {
-        redis->hset(fixture.payload_key, std::to_string(seq), "hash-only");
-    }
-
-    fixture.ledger.peek_due(fixture.uid, fixture.device, 5, 0);
-    auto cursor = redis->get(fixture.repair_key);
-    require(cursor && *cursor != "0",
-            "a bounded HASH repair pass must persist its non-zero HSCAN cursor");
-    const auto repair_ttl = ttl(redis, fixture.repair_key);
-    const auto payload_ttl = ttl(redis, fixture.payload_key);
-    require(repair_ttl > 0 && repair_ttl <= payload_ttl,
-            "repair cursor must expire with, and never outlive, the pending ledger");
-
-    for (int pass = 0; pass < 1000; ++pass) {
-        if (redis->hlen(fixture.payload_key) == 1 && !redis->get(fixture.repair_key)) break;
-        fixture.ledger.peek_due(fixture.uid, fixture.device, 5, 0);
-    }
-    require(redis->hlen(fixture.payload_key) == 1,
-            "successive bounded due-reads must eventually remove every HASH-only orphan");
-    require(!redis->get(fixture.repair_key),
-            "repair cursor must be deleted after a full stable scan completes");
-}
-
-void test_repair_cursor_follows_push_bump_ack_mutation_policy(const RedisClient::ptr &redis) {
-    Fixture fixture(redis, "repair-mutation-policy");
-    fixture.ledger.push(fixture.uid, fixture.device, 73, "payload-A", 1,
-                        std::chrono::seconds(300));
-
-    redis->set(fixture.repair_key, "7", std::chrono::seconds(30));
-    fixture.ledger.bump_score(fixture.uid, fixture.device, {73},
-                              std::chrono::seconds(300));
-    auto cursor = redis->get(fixture.repair_key);
-    require(cursor && *cursor == "7",
-            "bump changes only ZSET scores and must preserve the HASH repair cursor");
-    require(std::llabs(ttl(redis, fixture.repair_key) - ttl(redis, fixture.payload_key)) <= 1,
-            "bump must renew an existing repair cursor with the HASH paired TTL sample");
-
-    fixture.ledger.push(fixture.uid, fixture.device, 73, "payload-B", 1,
-                        std::chrono::seconds(300));
-    require(!redis->get(fixture.repair_key),
-            "push mutates the HASH and must reset its repair cursor");
-
-    redis->set(fixture.repair_key, "9", std::chrono::seconds(30));
-    fixture.ledger.ack(fixture.uid, fixture.device, 73);
-    require(!redis->get(fixture.repair_key),
-            "ACK mutates the HASH and must reset its repair cursor");
-}
-
-void test_peek_bump_flow_keeps_multi_page_repair_progress(const RedisClient::ptr &redis) {
-    Fixture fixture(redis, "peek-bump-progress");
-    fixture.ledger.push(fixture.uid, fixture.device, 81, "due", 1,
-                        std::chrono::seconds(300));
-    for (unsigned long seq = 11000; seq < 11700; ++seq) {
-        redis->hset(fixture.payload_key, std::to_string(seq), "hash-only");
-    }
-
-    auto due = fixture.ledger.peek_due(fixture.uid, fixture.device, 5, 0);
-    require(due.size() == 1 && due.front().first == 81,
-            "first production-flow peek must return the due complete entry");
-    auto first_cursor = redis->get(fixture.repair_key);
-    require(first_cursor && *first_cursor != "0",
-            "first production-flow peek must begin progressive orphan repair");
-    fixture.ledger.bump_score(fixture.uid, fixture.device, {81},
-                              std::chrono::seconds(300));
-    require(redis->get(fixture.repair_key) == first_cursor,
-            "production-flow bump must not restart the HASH scan at cursor zero");
-
-    for (int pass = 0; pass < 1000; ++pass) {
-        if (redis->hlen(fixture.payload_key) == 1 && !redis->get(fixture.repair_key)) break;
-        due = fixture.ledger.peek_due(fixture.uid, fixture.device, 5, -1);
-        if (!due.empty()) {
-            fixture.ledger.bump_score(fixture.uid, fixture.device, {due.front().first},
-                                      std::chrono::seconds(300));
-        }
-    }
-    require(redis->hlen(fixture.payload_key) == 1,
-            "peek-then-bump production flow must clean every HASH-only orphan page");
-    require(!redis->get(fixture.repair_key),
-            "peek-then-bump production flow must finish and remove the repair cursor");
-}
-
-void test_wrong_type_repair_metadata_cannot_block_due_delivery(const RedisClient::ptr &redis) {
-    Fixture wrong_type(redis, "repair-wrong-type");
-    wrong_type.ledger.push(wrong_type.uid, wrong_type.device, 82, "due", 1);
-    redis->hset(wrong_type.repair_key, "bad", "metadata");
-    auto due = wrong_type.ledger.peek_due(wrong_type.uid, wrong_type.device, 5, 0);
-    require(due.size() == 1 && due.front().first == 82,
-            "wrong-type repair metadata must be discarded without blocking due delivery");
-    require(!redis->get(wrong_type.repair_key),
-            "wrong-type repair metadata must be removed after restarting from zero");
-}
-
-void test_malformed_repair_cursor_cannot_block_due_delivery(const RedisClient::ptr &redis) {
-    Fixture malformed(redis, "repair-malformed");
-    malformed.ledger.push(malformed.uid, malformed.device, 83, "due", 1);
-    redis->set(malformed.repair_key, "not-a-redis-cursor", std::chrono::seconds(300));
-    auto due = malformed.ledger.peek_due(malformed.uid, malformed.device, 5, 0);
-    require(due.size() == 1 && due.front().first == 83,
-            "malformed repair cursor must be discarded and restarted from zero");
-    require(!redis->get(malformed.repair_key),
-            "malformed repair cursor must be removed after restarting from zero");
-}
-
-void test_bump_updates_only_complete_entries(const RedisClient::ptr &redis) {
-    Fixture fixture(redis, "bump-complete");
-    fixture.ledger.push(fixture.uid, fixture.device, 71, "complete", 1);
-    redis->zadd(fixture.pending_key, "72", 1);
-    redis->expire(fixture.pending_key, std::chrono::seconds(10));
-    redis->expire(fixture.payload_key, std::chrono::seconds(10));
-
-    const auto before = static_cast<long long>(std::time(nullptr));
-    fixture.ledger.bump_score(fixture.uid, fixture.device, {71, 72},
-                              std::chrono::seconds(300));
-    const auto after = static_cast<long long>(std::time(nullptr));
-
-    const auto bumped_score = std::stoll(zscore(redis, fixture.pending_key, "71"));
-    require(bumped_score >= before && bumped_score <= after,
-            "bump must set the complete entry score to the current time");
-    const auto pending_ttl = ttl(redis, fixture.pending_key);
-    const auto payload_ttl = ttl(redis, fixture.payload_key);
-    require(pending_ttl >= 200 && payload_ttl >= 200,
-            "bump must renew both ledger keys from their forced short TTL");
-    require(std::llabs(pending_ttl - payload_ttl) <= 1,
-            "bump must apply the same randomized TTL sample to both ledger keys");
-
-    const auto complete = fixture.ledger.peek_due(fixture.uid, fixture.device, 10, -1);
-    require(complete.size() == 1 && complete.front().first == 71,
-            "bump must retain and update the complete entry");
-    std::vector<std::string> members;
-    redis->zrange(fixture.pending_key, 0, -1, std::back_inserter(members));
-    require(members.size() == 1 && members.front() == "71",
-            "bump must remove a ZSET entry whose HASH payload is missing");
-}
-
-} // namespace
-
-int main() {
-    try {
-        const char *configured = std::getenv("CHATNOW_REDIS_CLUSTER_SEEDS");
-        const std::string seeds = configured ? configured : "redis-node1:6379";
-        auto cluster = RedisClusterFactory::create(seeds, 2);
-        auto redis = std::make_shared<RedisClient>(std::move(cluster));
-
-        const std::vector<std::pair<std::string, void (*)(const RedisClient::ptr &)>> tests = {
-            {"push replaces payload", test_push_replaces_payload_without_changing_identity},
-            {"ack removes both", test_ack_removes_zset_and_hash_entries},
-            {"ack wrong type is atomic", test_ack_wrong_type_does_not_partially_remove_zset},
-            {"due-read heals both directions", test_due_read_removes_both_orphan_directions},
-            {"due-read repair is progressive", test_due_read_progresses_across_hash_orphan_pages},
-            {"repair cursor mutation policy", test_repair_cursor_follows_push_bump_ack_mutation_policy},
-            {"peek-bump repair is progressive", test_peek_bump_flow_keeps_multi_page_repair_progress},
-            {"wrong-type repair metadata is fail-soft", test_wrong_type_repair_metadata_cannot_block_due_delivery},
-            {"malformed repair cursor is fail-soft", test_malformed_repair_cursor_cannot_block_due_delivery},
-            {"bump updates complete entries", test_bump_updates_only_complete_entries},
-        };
-        int failures = 0;
-        for (const auto &[name, test] : tests) {
-            try {
-                test(redis);
-            } catch (const std::exception &error) {
-                ++failures;
-                std::cerr << "FAILED " << name << ": " << error.what() << '\n';
-            }
-        }
-        if (failures != 0) {
-            std::cerr << failures << " unacked pending ledger cluster test(s) failed\n";
-            return 1;
-        }
-        std::cout << "unacked pending ledger cluster tests passed\n";
-        return 0;
-    } catch (const std::exception &error) {
-        std::cerr << "unacked pending ledger cluster test failed: " << error.what() << '\n';
-        return 1;
-    }
-}
diff --git a/common/test/test_user_info_generation_fence.cc b/common/test/test_user_info_generation_fence.cc
deleted file mode 100644
index f0cbdca..0000000
--- a/common/test/test_user_info_generation_fence.cc
+++ /dev/null
@@ -1,67 +0,0 @@
-#include "dao/data_redis.hpp"
-
-#include <iostream>
-#include <stdexcept>
-
-int main() {
-    using chatnow::GenerationWriteResult;
-    using chatnow::UserInfoL1Publication;
-
-    int failures = 0;
-    auto expect = [&](bool condition, const char *message) {
-        if (condition) return;
-        std::cerr << "FAIL: " << message << '\n';
-        ++failures;
-    };
-
-    chatnow::UserInfoCache missing_client(nullptr);
-    expect(missing_client.set_if_generation("u1", "bytes", 0) ==
-               GenerationWriteResult::Unavailable,
-           "UserInfoCache must report a missing client as unavailable");
-    chatnow::UserInfoCache lua_zero(nullptr, [] { return 0LL; });
-    expect(lua_zero.set_if_generation("u1", "bytes", 0) ==
-               GenerationWriteResult::Conflict,
-           "Lua zero must be a generation conflict");
-    chatnow::UserInfoCache lua_one(nullptr, [] { return 1LL; });
-    expect(lua_one.set_if_generation("u1", "bytes", 0) ==
-               GenerationWriteResult::Committed,
-           "Lua one must be committed");
-    chatnow::UserInfoCache circuit_open(nullptr, []() -> long long {
-        throw chatnow::RedisCircuitOpen();
-    });
-    expect(circuit_open.set_if_generation("u1", "bytes", 0) ==
-               GenerationWriteResult::Unavailable,
-           "open Redis circuit must be unavailable");
-    chatnow::UserInfoCache redis_error(nullptr, []() -> long long {
-        throw std::runtime_error("redis unavailable");
-    });
-    expect(redis_error.set_if_generation("u1", "bytes", 0) ==
-               GenerationWriteResult::Unavailable,
-           "Redis exception must be unavailable");
-
-    auto observe_publication = [&](GenerationWriteResult result) {
-        int calls = 0;
-        UserInfoL1Publication observed = UserInfoL1Publication::Denied;
-        chatnow::publish_user_info_l1(result, [&](UserInfoL1Publication publication) {
-            ++calls;
-            observed = publication;
-        });
-        return std::pair<int, UserInfoL1Publication>{calls, observed};
-    };
-    const auto committed = observe_publication(GenerationWriteResult::Committed);
-    expect(committed.first == 1 &&
-               committed.second == UserInfoL1Publication::GenerationFenced,
-           "committed write must call the generation-fenced L1 publisher");
-    const auto conflict = observe_publication(GenerationWriteResult::Conflict);
-    expect(conflict.first == 0,
-           "generation conflict must not call the L1 publisher");
-    const auto unavailable = observe_publication(GenerationWriteResult::Unavailable);
-    expect(unavailable.first == 1 &&
-               unavailable.second == UserInfoL1Publication::ShortLivedFallback,
-           "unavailable Redis must call only the short-lived L1 fallback");
-
-    if (failures == 0) {
-        std::cout << "user info generation fence execution tests passed\n";
-    }
-    return failures == 0 ? 0 : 1;
-}
diff --git a/tests/func/cache_test.go b/tests/func/cache_test.go
index 2ae5ab1..dc7488d 100644
--- a/tests/func/cache_test.go
+++ b/tests/func/cache_test.go
@@ -4,6 +4,7 @@ package func_test
 
 import (
 	"bytes"
+	"encoding/base64"
 	"fmt"
 	"os"
 	"os/exec"
@@ -26,6 +27,7 @@ import (
 	common "chatnow-tests/proto/chatnow/common"
 	identity "chatnow-tests/proto/chatnow/identity"
 	msg "chatnow-tests/proto/chatnow/message"
+	push "chatnow-tests/proto/chatnow/push"
 	transmite "chatnow-tests/proto/chatnow/transmite"
 )
 
@@ -340,6 +342,78 @@ func absDuration(value time.Duration) time.Duration {
 	return value
 }
 
+func TestFN_CA_UnackedSameUserSeqLatestPayloadAndAck(t *testing.T) {
+	recipient, _, _ := fixture.RegisterAndLogin(t, HTTP)
+	ws := openCacheTestWebSocket(t)
+	t.Cleanup(func() { _ = ws.Close() })
+	const deviceID = "default_device"
+	writeCacheTestBinary(t, ws, cacheTestAuthNotify(recipient.AccessToken, deviceID))
+
+	deviceKey := fmt.Sprintf("im:dev:{%s}", recipient.UserID)
+	requireEventuallyRedis(t, 5*time.Second, 50*time.Millisecond,
+		"Push route must exist before direct PushToUser calls", redisEquals("1"),
+		"EXISTS", deviceKey)
+
+	userSeq := uint64(time.Now().UnixNano())
+	unackedKey := fmt.Sprintf("im:unack:{%s:%s}", recipient.UserID, deviceID)
+	unackedIndexKey := fmt.Sprintf("im:unack:idx:{%s:%s}", recipient.UserID, deviceID)
+	verify.RedisCLI(t, "DEL", unackedKey, unackedIndexKey)
+
+	first := &push.NotifyMessage{
+		NotifyEventId: proto.String("unacked-first-" + client.NewRequestID()),
+		NotifyType:    push.NotifyType_TYPING_NOTIFY,
+		NotifyRemarks: &push.NotifyMessage_Typing{Typing: &push.NotifyTyping{
+			UserId: recipient.UserID, ConversationId: "unacked-contract", IsTyping: false,
+		}},
+	}
+	second := proto.Clone(first).(*push.NotifyMessage)
+	second.NotifyEventId = proto.String("unacked-second-" + client.NewRequestID())
+	second.GetTyping().IsTyping = true
+
+	for _, notify := range []*push.NotifyMessage{first, second} {
+		rsp := &push.PushToUserRsp{}
+		require.NoError(t, HTTP.DoProtobufURL(
+			HTTP.Config().Infra.PushVars+"/chatnow.push.PushService/PushToUser",
+			&push.PushToUserReq{
+				RequestId: client.NewRequestID(), UserId: recipient.UserID,
+				Notify: notify, UserSeq: proto.Uint64(userSeq), TargetDeviceIds: []string{deviceID},
+			}, rsp))
+		require.True(t, rsp.GetHeader().GetSuccess(), rsp.GetHeader().GetErrorMessage())
+		require.Equal(t, int32(1), rsp.GetOnlineDeviceCount())
+	}
+
+	seq := strconv.FormatUint(userSeq, 10)
+	requireEventuallyRedis(t, 5*time.Second, 50*time.Millisecond,
+		"same user_seq must retain one stable ZSET identity", redisEquals("1"),
+		"ZCARD", unackedKey)
+	require.Equal(t, seq, verify.RedisCLI(t, "ZRANGE", unackedKey, "0", "-1"))
+	require.Equal(t, "1", verify.RedisCLI(t, "HLEN", unackedIndexKey))
+
+	encodedLatest := verify.RedisCLI(t, "HGET", unackedIndexKey, seq)
+	latestBytes, err := base64.StdEncoding.DecodeString(encodedLatest)
+	require.NoError(t, err)
+	latest := &push.NotifyMessage{}
+	require.NoError(t, proto.Unmarshal(latestBytes, latest))
+	require.True(t, proto.Equal(second, latest),
+		"same user_seq must replace the HASH payload with the second notification")
+
+	ackBytes, err := proto.Marshal(&push.NotifyMessage{
+		NotifyType: push.NotifyType_MSG_PUSH_ACK,
+		NotifyRemarks: &push.NotifyMessage_MsgPushAck{MsgPushAck: &push.NotifyMsgPushAck{
+			UserId: recipient.UserID, DeviceId: deviceID, MessageId: 1,
+			UserSeq: userSeq, ConversationId: "unacked-contract",
+		}},
+	})
+	require.NoError(t, err)
+	writeCacheTestBinary(t, ws, ackBytes)
+	requireEventuallyRedis(t, 5*time.Second, 50*time.Millisecond,
+		"WebSocket ACK must remove the ZSET identity", redisEquals("0"),
+		"EXISTS", unackedKey)
+	requireEventuallyRedis(t, 5*time.Second, 50*time.Millisecond,
+		"WebSocket ACK must remove the HASH payload index", redisEquals("0"),
+		"EXISTS", unackedIndexKey)
+}
+
 // FN-CA-05 | healthy Redis applies the distributed message rate limit.
 func TestFN_CA_RateLimit(t *testing.T) {
 	user, peer, convID := fixture.MakeFriends(t, HTTP)
@@ -440,22 +514,32 @@ func TestFN_CA_UserInfoInvalidatedAfterProfileUpdate(t *testing.T) {
 	verify.RedisCLI(t, "DEL", key)
 	sendCacheTestMessage(t, user, convID, "warm")
 	require.Equal(t, "1", verify.RedisCLI(t, "EXISTS", key))
+	warmInfo := &common.UserInfo{}
+	require.NoError(t, proto.Unmarshal(redisRaw(t, key), warmInfo))
+	require.NotEmpty(t, warmInfo.GetNickname())
 
 	newNickname := fmt.Sprintf("cache_%d", time.Now().UnixNano()%1_000_000_000)
+	require.NotEqual(t, warmInfo.GetNickname(), newNickname)
 	updateRsp := &identity.UpdateProfileRsp{}
 	require.NoError(t, user.DoAuth("/service/identity/update_profile", &identity.UpdateProfileReq{
 		RequestId: client.NewRequestID(),
 		Nickname:  &newNickname,
 	}, updateRsp))
 	require.True(t, updateRsp.GetHeader().GetSuccess(), updateRsp.GetHeader().GetErrorMessage())
+	require.Equal(t, newNickname, updateRsp.GetUserInfo().GetNickname())
 	require.Equal(t, "0", verify.RedisCLI(t, "EXISTS", key))
 
-	// Identity can invalidate the shared L2 immediately; the process-local L1 is
-	// intentionally bounded by its 45s TTL in the absence of a broadcast channel.
+	// UpdateProfile invalidates shared L2 immediately. A Transmite process may
+	// still publish its bounded stale L1 value until the documented 45s lifetime
+	// expires; this waits at the business boundary instead of forcing an internal
+	// generation/CAS interleaving with a production timing hook.
 	time.Sleep(55 * time.Second)
 	sendCacheTestMessage(t, user, convID, "after-update")
+	// The next Transmite lookup must repopulate Redis from Identity with the
+	// latest profile, proving stale publication cannot survive the L1 bound.
 	serialized := redisRaw(t, key)
 	info := &common.UserInfo{}
 	require.NoError(t, proto.Unmarshal(serialized, info))
 	require.Equal(t, newNickname, info.GetNickname())
+	require.NotEqual(t, warmInfo.GetNickname(), info.GetNickname())
 }
diff --git a/tests/pkg/client/http.go b/tests/pkg/client/http.go
index 44a8b61..167c39a 100644
--- a/tests/pkg/client/http.go
+++ b/tests/pkg/client/http.go
@@ -57,13 +57,23 @@ func (c *HTTPClient) DoWithTrace(path string, req proto.Message, resp proto.Mess
 	return c.do(path, req, resp, accessToken, traceID)
 }
 
+// DoProtobufURL sends a protobuf request directly to an absolute service URL.
+func (c *HTTPClient) DoProtobufURL(endpoint string, req proto.Message, resp proto.Message) error {
+	_, err := c.doURL(endpoint, req, resp, "", "")
+	return err
+}
+
 func (c *HTTPClient) do(path string, req proto.Message, resp proto.Message, accessToken, traceID string) (http.Header, error) {
+	return c.doURL(c.baseURL+path, req, resp, accessToken, traceID)
+}
+
+func (c *HTTPClient) doURL(endpoint string, req proto.Message, resp proto.Message, accessToken, traceID string) (http.Header, error) {
 	body, err := proto.Marshal(req)
 	if err != nil {
 		return nil, fmt.Errorf("marshal request: %w", err)
 	}
 
-	httpReq, err := http.NewRequest("POST", c.baseURL+path, bytes.NewReader(body))
+	httpReq, err := http.NewRequest("POST", endpoint, bytes.NewReader(body))
 	if err != nil {
 		return nil, fmt.Errorf("create request: %w", err)
 	}
diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index 14b810c..58d4194 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -2,16 +2,90 @@ package contracts
 
 import (
 	"fmt"
+	"go/ast"
+	"go/parser"
+	"go/token"
+	"io"
+	"net/http"
+	"net/http/httptest"
 	"os"
 	"path/filepath"
 	"runtime"
 	"strings"
 	"testing"
 
+	"chatnow-tests/pkg/client"
 	"github.com/stretchr/testify/require"
+	"google.golang.org/protobuf/proto"
+	"google.golang.org/protobuf/types/known/wrapperspb"
 	"gopkg.in/yaml.v3"
 )
 
+func TestDirectProtobufHTTPClient(t *testing.T) {
+	received := make(chan string, 1)
+	server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		body, err := io.ReadAll(r.Body)
+		if err != nil {
+			http.Error(w, err.Error(), http.StatusInternalServerError)
+			return
+		}
+		request := &wrapperspb.StringValue{}
+		if err := proto.Unmarshal(body, request); err != nil {
+			http.Error(w, err.Error(), http.StatusBadRequest)
+			return
+		}
+		received <- fmt.Sprintf("%s|%s|%s", r.URL.Path, r.Header.Get("Content-Type"), request.GetValue())
+		response, err := proto.Marshal(wrapperspb.String("direct-response"))
+		if err != nil {
+			http.Error(w, err.Error(), http.StatusInternalServerError)
+			return
+		}
+		w.Header().Set("Content-Type", "application/x-protobuf")
+		_, _ = w.Write(response)
+	}))
+	t.Cleanup(server.Close)
+
+	httpClient := client.NewHTTPClient(&client.Config{})
+	response := &wrapperspb.StringValue{}
+	require.NoError(t, httpClient.DoProtobufURL(
+		server.URL+"/chatnow.push.PushService/PushToUser",
+		wrapperspb.String("direct-request"), response))
+	require.Equal(t, "direct-response", response.GetValue())
+	require.Equal(t,
+		"/chatnow.push.PushService/PushToUser|application/x-protobuf|direct-request",
+		<-received)
+}
+
+func TestGoCacheRegressionsReplaceTemporaryCPP(t *testing.T) {
+	root := repositoryRoot(t)
+	for _, relativePath := range []string{
+		"common/test/test_user_info_generation_fence.cc",
+		"common/test/test_unacked_pending_ledger.cc",
+	} {
+		_, err := os.Stat(filepath.Join(root, relativePath))
+		require.ErrorIs(t, err, os.ErrNotExist, "%s must be removed", relativePath)
+	}
+
+	cmake, err := os.ReadFile(filepath.Join(root, "common/test/CMakeLists.txt"))
+	require.NoError(t, err)
+	require.NotContains(t, string(cmake), "test_unacked_pending_ledger",
+		"the temporary C++ Unacked target must be removed")
+
+	cacheTestPath := filepath.Join(root, "tests/func/cache_test.go")
+	parsed, err := parser.ParseFile(token.NewFileSet(), cacheTestPath, nil, 0)
+	require.NoError(t, err)
+	var found bool
+	for _, declaration := range parsed.Decls {
+		function, ok := declaration.(*ast.FuncDecl)
+		if ok && function.Name.Name == "TestFN_CA_UnackedSameUserSeqLatestPayloadAndAck" {
+			found = true
+			break
+		}
+	}
+	require.True(t, found,
+		"the Go functional suite must own the same-user_seq latest-payload/ACK regression")
+}
+
 type workflowContract struct {
 	On struct {
 		PullRequest map[string]any   `yaml:"pull_request"`

From a840f7efd37791672f5917dd1eca1331c0f8a14b Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Thu, 16 Jul 2026 19:19:55 +0800
Subject: [PATCH 39/47] ci: distribute service artifacts to cache gates

---
 .github/workflows/ci.yml             |  62 ++++++++++++
 tests/pkg/contracts/ci_gates_test.go | 137 ++++++++++++++++++++++++++-
 2 files changed, 196 insertions(+), 3 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 16a5dbd..5c3b5c3 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -41,6 +41,38 @@ jobs:
             exit 1
           fi
 
+  service-artifacts:
+    runs-on: ubuntu-22.04
+    if: github.event_name == 'pull_request' || github.event_name == 'schedule'
+    steps:
+      - uses: actions/checkout@v4
+      - uses: docker/setup-buildx-action@v3
+      - name: Build native-service builder image
+        uses: docker/build-push-action@v6
+        with:
+          context: .
+          file: docker/ci/Dockerfile
+          load: true
+          tags: chatnow-ci-builder:ci
+          cache-from: type=gha
+          cache-to: type=gha,mode=max
+      - name: Build all Compose services once
+        run: |
+          docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci bash -lc '
+          cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
+          cmake --build build --parallel "$(nproc)" --target conversation_server gateway_server identity_server media_server message_server presence_server push_server relationship_server transmite_server
+          '
+      - name: Package Compose service artifacts
+        run: docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci ./scripts/package_compose_artifacts.sh build compose-artifacts
+      - name: Validate Compose service artifacts
+        run: docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci ./scripts/validate_compose_artifacts.sh compose-artifacts
+      - name: Upload Compose service artifacts
+        uses: actions/upload-artifact@v4
+        with:
+          name: compose-service-artifacts
+          path: compose-artifacts
+          if-no-files-found: error
+
   bvt:
     needs: build
     runs-on: ubuntu-22.04
@@ -100,6 +132,7 @@ jobs:
         run: docker compose down -v
 
   reliability:
+    needs: service-artifacts
     runs-on: ubuntu-22.04
     if: github.event_name == 'pull_request' || github.event_name == 'schedule'
     steps:
@@ -113,6 +146,20 @@ jobs:
           sudo apt-get install -y protobuf-compiler netcat-openbsd
       - name: Install Go protobuf generator
         run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
+      - name: Download Compose service artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: compose-service-artifacts
+          path: compose-artifacts
+      - name: Restore Compose build contexts
+        run: |
+          for service in conversation gateway identity media message presence push relationship transmite; do
+            rm -rf "$service/build" "$service/depends"
+            cp -a "compose-artifacts/$service/build" "$service/build"
+            cp -a "compose-artifacts/$service/depends" "$service/depends"
+          done
+      - name: Validate downloaded Compose service artifacts
+        run: ./scripts/validate_compose_artifacts.sh compose-artifacts
       - name: Start full stack
         run: docker compose up -d --build
       - name: Wait for services
@@ -128,6 +175,7 @@ jobs:
         run: docker compose down -v
 
   perf-cache:
+    needs: service-artifacts
     runs-on: ubuntu-22.04
     if: github.event_name == 'schedule'
     steps:
@@ -141,6 +189,20 @@ jobs:
           sudo apt-get install -y protobuf-compiler netcat-openbsd
       - name: Install Go protobuf generator
         run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
+      - name: Download Compose service artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: compose-service-artifacts
+          path: compose-artifacts
+      - name: Restore Compose build contexts
+        run: |
+          for service in conversation gateway identity media message presence push relationship transmite; do
+            rm -rf "$service/build" "$service/depends"
+            cp -a "compose-artifacts/$service/build" "$service/build"
+            cp -a "compose-artifacts/$service/depends" "$service/depends"
+          done
+      - name: Validate downloaded Compose service artifacts
+        run: ./scripts/validate_compose_artifacts.sh compose-artifacts
       - name: Start full stack with PF-09 rate limits
         env:
           # Transmite flags are int32; use the maximum valid value for gate-only headroom.
diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index 58d4194..25be2fa 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -107,6 +107,7 @@ type workflowStep struct {
 	If              string            `yaml:"if"`
 	ContinueOnError any               `yaml:"continue-on-error"`
 	Env             map[string]string `yaml:"env"`
+	With            map[string]any    `yaml:"with"`
 }
 
 type composeContract struct {
@@ -129,15 +130,19 @@ func TestCIGates(t *testing.T) {
 	assertDecoratedCommandsDoNotSatisfyGate(t)
 	assertContractsRunInBuild(t, workflow.Jobs["build"])
 
+	producer, ok := workflow.Jobs["service-artifacts"]
+	require.True(t, ok, "CI must build the Compose service artifacts once")
+	assertServiceArtifactProducer(t, producer)
+
 	reliability, ok := workflow.Jobs["reliability"]
 	require.True(t, ok, "RL-05 must have a dedicated reliability job")
-	require.Nil(t, reliability.Needs, "reliability must own its setup instead of depending on another job")
+	require.Equal(t, "service-artifacts", reliability.Needs)
 	require.Equal(t, "github.event_name == 'pull_request' || github.event_name == 'schedule'", reliability.If)
 	assertFullStackGateJob(t, reliability, "cd tests && make test-reliability")
 
 	perfCache, ok := workflow.Jobs["perf-cache"]
 	require.True(t, ok, "PF-09 must have a dedicated perf-cache job")
-	require.Nil(t, perfCache.Needs, "perf-cache must own its setup instead of depending on another job")
+	require.Equal(t, "service-artifacts", perfCache.Needs)
 	require.Equal(t, "github.event_name == 'schedule'", perfCache.If)
 	assertFullStackGateJob(t, perfCache, "cd tests && make test-perf-cache-gate")
 	assertTargetAbsent(t, perfCache, "test-perf-cache")
@@ -154,6 +159,91 @@ func TestCIGates(t *testing.T) {
 	require.Contains(t, transmite.Entrypoint, "-rate_limit_session_max=${TRANSMITE_RATE_LIMIT_SESSION_MAX:-3000}")
 }
 
+const (
+	artifactName       = "compose-service-artifacts"
+	artifactPath       = "compose-artifacts"
+	builderImage       = "chatnow-ci-builder:ci"
+	nativeBuildCommand = `docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci bash -lc '
+cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
+cmake --build build --parallel "$(nproc)" --target conversation_server gateway_server identity_server media_server message_server presence_server push_server relationship_server transmite_server
+'`
+	packageCommand  = `docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci ./scripts/package_compose_artifacts.sh build compose-artifacts`
+	validateCommand = `docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci ./scripts/validate_compose_artifacts.sh compose-artifacts`
+	restoreCommand  = `for service in conversation gateway identity media message presence push relationship transmite; do
+  rm -rf "$service/build" "$service/depends"
+  cp -a "compose-artifacts/$service/build" "$service/build"
+  cp -a "compose-artifacts/$service/depends" "$service/depends"
+done`
+)
+
+func assertServiceArtifactProducer(t *testing.T, job workflowJob) {
+	t.Helper()
+	require.NoError(t, validateServiceArtifactProducer(job))
+
+	valid := cloneWorkflowJob(job)
+	build := exactUsesStepIndex(valid, "docker/build-push-action@v6")
+	native := exactRunStepIndex(valid, nativeBuildCommand)
+	pack := exactRunStepIndex(valid, packageCommand)
+	validate := exactRunStepIndex(valid, validateCommand)
+	upload := exactUsesStepIndex(valid, "actions/upload-artifact@v4")
+	for name, mutate := range map[string]func(*workflowJob){
+		"builder allowed to fail":      func(job *workflowJob) { job.Steps[build].ContinueOnError = true },
+		"native build allowed to fail": func(job *workflowJob) { job.Steps[native].ContinueOnError = true },
+		"package allowed to fail":      func(job *workflowJob) { job.Steps[pack].ContinueOnError = true },
+		"validation allowed to fail":   func(job *workflowJob) { job.Steps[validate].ContinueOnError = true },
+		"upload allowed to fail":       func(job *workflowJob) { job.Steps[upload].ContinueOnError = true },
+		"validation after upload": func(job *workflowJob) {
+			job.Steps[validate], job.Steps[upload] = job.Steps[upload], job.Steps[validate]
+		},
+		"native build bypassed":   func(job *workflowJob) { job.Steps[native].If = "${{ false }}" },
+		"native build duplicated": func(job *workflowJob) { job.Steps = append(job.Steps, job.Steps[native]) },
+	} {
+		t.Run("producer rejects "+name, func(t *testing.T) {
+			invalid := cloneWorkflowJob(valid)
+			mutate(&invalid)
+			require.Error(t, validateServiceArtifactProducer(invalid))
+		})
+	}
+}
+
+func validateServiceArtifactProducer(job workflowJob) error {
+	if countExactUsesSteps(job, "docker/build-push-action@v6") != 1 || countExactRunSteps(job, nativeBuildCommand) != 1 {
+		return fmt.Errorf("services must be built exactly once")
+	}
+	ordered := []struct {
+		label string
+		index int
+	}{
+		{"checkout", exactUsesStepIndex(job, "actions/checkout@v4")},
+		{"Buildx setup", exactUsesStepIndex(job, "docker/setup-buildx-action@v3")},
+		{"builder image build", exactUsesStepIndex(job, "docker/build-push-action@v6")},
+		{"native service build", exactRunStepIndex(job, nativeBuildCommand)},
+		{"artifact package", exactRunStepIndex(job, packageCommand)},
+		{"artifact validation", exactRunStepIndex(job, validateCommand)},
+		{"artifact upload", exactUsesStepIndex(job, "actions/upload-artifact@v4")},
+	}
+	previous := -1
+	for _, required := range ordered {
+		if required.index < 0 || required.index <= previous {
+			return fmt.Errorf("missing or out-of-order %s step", required.label)
+		}
+		step := job.Steps[required.index]
+		if strings.TrimSpace(step.If) != "" || continueOnErrorEnabled(step.ContinueOnError) {
+			return fmt.Errorf("%s step must fail closed", required.label)
+		}
+		previous = required.index
+	}
+	build := job.Steps[ordered[2].index]
+	if build.With["context"] != "." || build.With["file"] != "docker/ci/Dockerfile" || build.With["load"] != true || build.With["tags"] != builderImage || build.With["cache-from"] != "type=gha" || build.With["cache-to"] != "type=gha,mode=max" {
+		return fmt.Errorf("builder image must use the Dockerfile, local load, and BuildKit GHA cache")
+	}
+	upload := job.Steps[ordered[len(ordered)-1].index]
+	if upload.With["name"] != artifactName || upload.With["path"] != artifactPath || upload.With["if-no-files-found"] != "error" {
+		return fmt.Errorf("artifact upload contract is incomplete")
+	}
+	return nil
+}
+
 func assertFullStackGateJob(t *testing.T, job workflowJob, target string) {
 	t.Helper()
 	require.NoError(t, validateFullStackGateJob(job, target))
@@ -173,7 +263,7 @@ func assertContractsRunInBuild(t *testing.T, build workflowJob) {
 
 func exactRunStepIndex(job workflowJob, wanted string) int {
 	for index, step := range job.Steps {
-		if strings.TrimSpace(step.Run) == wanted {
+		if strings.TrimSpace(step.Run) == strings.TrimSpace(wanted) {
 			return index
 		}
 	}
@@ -189,6 +279,16 @@ func exactUsesStepIndex(job workflowJob, wanted string) int {
 	return -1
 }
 
+func countExactUsesSteps(job workflowJob, wanted string) int {
+	count := 0
+	for _, step := range job.Steps {
+		if step.Uses == wanted {
+			count++
+		}
+	}
+	return count
+}
+
 func assertDecoratedCommandsDoNotSatisfyGate(t *testing.T) {
 	t.Helper()
 	const gate = "cd tests && make test-perf-cache-gate"
@@ -215,6 +315,9 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 	t.Helper()
 	gate := exactRunStepIndex(valid, target)
 	setupGo := exactUsesStepIndex(valid, "actions/setup-go@v5")
+	download := exactUsesStepIndex(valid, "actions/download-artifact@v4")
+	restore := exactRunStepIndex(valid, restoreCommand)
+	validate := exactRunStepIndex(valid, "./scripts/validate_compose_artifacts.sh compose-artifacts")
 	start := exactRunStepIndex(valid, "docker compose up -d --build")
 	wait := exactRunStepIndex(valid, "./scripts/wait_for_services.sh")
 	proto := exactRunStepIndex(valid, "cd tests && make proto")
@@ -222,6 +325,9 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 	teardown := exactRunStepIndex(valid, "docker compose down -v")
 	require.NotEqual(t, -1, gate)
 	require.NotEqual(t, -1, setupGo)
+	require.NotEqual(t, -1, download)
+	require.NotEqual(t, -1, restore)
+	require.NotEqual(t, -1, validate)
 	require.NotEqual(t, -1, start)
 	require.NotEqual(t, -1, wait)
 	require.NotEqual(t, -1, proto)
@@ -232,6 +338,15 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 		"Go setup allowed to fail": func(job *workflowJob) {
 			job.Steps[setupGo].ContinueOnError = true
 		},
+		"download allowed to fail": func(job *workflowJob) {
+			job.Steps[download].ContinueOnError = true
+		},
+		"restore allowed to fail": func(job *workflowJob) {
+			job.Steps[restore].ContinueOnError = true
+		},
+		"artifact validation allowed to fail": func(job *workflowJob) {
+			job.Steps[validate].ContinueOnError = true
+		},
 		"gate disabled by if": func(job *workflowJob) {
 			job.Steps[gate].If = "${{ false }}"
 		},
@@ -265,6 +380,9 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 		"gate before dependencies": func(job *workflowJob) {
 			job.Steps[gate], job.Steps[deps] = job.Steps[deps], job.Steps[gate]
 		},
+		"Compose before artifact validation": func(job *workflowJob) {
+			job.Steps[start], job.Steps[validate] = job.Steps[validate], job.Steps[start]
+		},
 		"teardown before gate": func(job *workflowJob) {
 			job.Steps[gate], job.Steps[teardown] = job.Steps[teardown], job.Steps[gate]
 		},
@@ -316,6 +434,9 @@ func isForbiddenMakeTarget(run, target string) bool {
 
 func validateFullStackGateJob(job workflowJob, target string) error {
 	const install = "sudo apt-get update\nsudo apt-get install -y protobuf-compiler netcat-openbsd"
+	if job.Needs != "service-artifacts" {
+		return fmt.Errorf("gate job must depend on service-artifacts")
+	}
 	for _, step := range job.Steps {
 		if isForbiddenMakeTarget(step.Run, "test-perf-cache") {
 			return fmt.Errorf("skip-capable performance target is forbidden")
@@ -329,6 +450,9 @@ func validateFullStackGateJob(job workflowJob, target string) error {
 		{"Go setup", exactUsesStepIndex(job, "actions/setup-go@v5")},
 		{"system dependency install", exactRunStepIndex(job, install)},
 		{"protoc generator install", exactRunStepIndex(job, "go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11")},
+		{"artifact download", exactUsesStepIndex(job, "actions/download-artifact@v4")},
+		{"artifact restore", exactRunStepIndex(job, restoreCommand)},
+		{"artifact validation", exactRunStepIndex(job, "./scripts/validate_compose_artifacts.sh compose-artifacts")},
 		{"full-stack startup", exactRunStepIndex(job, "docker compose up -d --build")},
 		{"service wait", exactRunStepIndex(job, "./scripts/wait_for_services.sh")},
 		{"protobuf generation", exactRunStepIndex(job, "cd tests && make proto")},
@@ -355,6 +479,13 @@ func validateFullStackGateJob(job workflowJob, target string) error {
 		if continueOnErrorEnabled(job.Steps[required.index].ContinueOnError) {
 			return fmt.Errorf("%s step must not continue on error", required.label)
 		}
+		if strings.TrimSpace(job.Steps[required.index].If) != "" {
+			return fmt.Errorf("%s step must not have a step-level if condition", required.label)
+		}
+	}
+	download := ordered[4].index
+	if job.Steps[download].With["name"] != artifactName || job.Steps[download].With["path"] != artifactPath {
+		return fmt.Errorf("gate job must download the shared Compose artifact")
 	}
 	if teardown != len(job.Steps)-1 {
 		return fmt.Errorf("teardown must be the final step")

From b4b6e942a5e2bf749713dbf177a69e4d080bd038 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Fri, 17 Jul 2026 10:47:26 +0800
Subject: [PATCH 40/47] fix(ci): validate artifacts in pinned runtime

---
 .github/workflows/ci.yml             |  4 ++--
 tests/pkg/contracts/ci_gates_test.go | 19 +++++++++++++------
 2 files changed, 15 insertions(+), 8 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 5c3b5c3..ac83a31 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -159,7 +159,7 @@ jobs:
             cp -a "compose-artifacts/$service/depends" "$service/depends"
           done
       - name: Validate downloaded Compose service artifacts
-        run: ./scripts/validate_compose_artifacts.sh compose-artifacts
+        run: docker run --rm -v "$PWD:/workspace" -w /workspace ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 ./scripts/validate_compose_artifacts.sh compose-artifacts
       - name: Start full stack
         run: docker compose up -d --build
       - name: Wait for services
@@ -202,7 +202,7 @@ jobs:
             cp -a "compose-artifacts/$service/depends" "$service/depends"
           done
       - name: Validate downloaded Compose service artifacts
-        run: ./scripts/validate_compose_artifacts.sh compose-artifacts
+        run: docker run --rm -v "$PWD:/workspace" -w /workspace ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 ./scripts/validate_compose_artifacts.sh compose-artifacts
       - name: Start full stack with PF-09 rate limits
         env:
           # Transmite flags are int32; use the maximum valid value for gate-only headroom.
diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index 25be2fa..a3d8278 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -160,10 +160,11 @@ func TestCIGates(t *testing.T) {
 }
 
 const (
-	artifactName       = "compose-service-artifacts"
-	artifactPath       = "compose-artifacts"
-	builderImage       = "chatnow-ci-builder:ci"
-	nativeBuildCommand = `docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci bash -lc '
+	artifactName            = "compose-service-artifacts"
+	artifactPath            = "compose-artifacts"
+	builderImage            = "chatnow-ci-builder:ci"
+	consumerValidateCommand = `docker run --rm -v "$PWD:/workspace" -w /workspace ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 ./scripts/validate_compose_artifacts.sh compose-artifacts`
+	nativeBuildCommand      = `docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci bash -lc '
 cmake -S . -B build -G Ninja -DCMAKE_BUILD_TYPE=Release
 cmake --build build --parallel "$(nproc)" --target conversation_server gateway_server identity_server media_server message_server presence_server push_server relationship_server transmite_server
 '`
@@ -317,7 +318,7 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 	setupGo := exactUsesStepIndex(valid, "actions/setup-go@v5")
 	download := exactUsesStepIndex(valid, "actions/download-artifact@v4")
 	restore := exactRunStepIndex(valid, restoreCommand)
-	validate := exactRunStepIndex(valid, "./scripts/validate_compose_artifacts.sh compose-artifacts")
+	validate := exactRunStepIndex(valid, consumerValidateCommand)
 	start := exactRunStepIndex(valid, "docker compose up -d --build")
 	wait := exactRunStepIndex(valid, "./scripts/wait_for_services.sh")
 	proto := exactRunStepIndex(valid, "cd tests && make proto")
@@ -347,6 +348,12 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 		"artifact validation allowed to fail": func(job *workflowJob) {
 			job.Steps[validate].ContinueOnError = true
 		},
+		"artifact validation on consumer host": func(job *workflowJob) {
+			job.Steps[validate].Run = "./scripts/validate_compose_artifacts.sh compose-artifacts"
+		},
+		"artifact validation with mutable Ubuntu tag": func(job *workflowJob) {
+			job.Steps[validate].Run = `docker run --rm -v "$PWD:/workspace" -w /workspace ubuntu:24.04 ./scripts/validate_compose_artifacts.sh compose-artifacts`
+		},
 		"gate disabled by if": func(job *workflowJob) {
 			job.Steps[gate].If = "${{ false }}"
 		},
@@ -452,7 +459,7 @@ func validateFullStackGateJob(job workflowJob, target string) error {
 		{"protoc generator install", exactRunStepIndex(job, "go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11")},
 		{"artifact download", exactUsesStepIndex(job, "actions/download-artifact@v4")},
 		{"artifact restore", exactRunStepIndex(job, restoreCommand)},
-		{"artifact validation", exactRunStepIndex(job, "./scripts/validate_compose_artifacts.sh compose-artifacts")},
+		{"artifact validation", exactRunStepIndex(job, consumerValidateCommand)},
 		{"full-stack startup", exactRunStepIndex(job, "docker compose up -d --build")},
 		{"service wait", exactRunStepIndex(job, "./scripts/wait_for_services.sh")},
 		{"protobuf generation", exactRunStepIndex(job, "cd tests && make proto")},

From 4dfa993a692de4c5614eff6fc312109897a783c6 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Fri, 17 Jul 2026 10:51:07 +0800
Subject: [PATCH 41/47] fix(ci): restore executable artifact modes

---
 .github/workflows/ci.yml             |  2 ++
 tests/pkg/contracts/ci_gates_test.go | 16 ++++++++++++++++
 2 files changed, 18 insertions(+)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index ac83a31..d08c7bd 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -154,6 +154,7 @@ jobs:
       - name: Restore Compose build contexts
         run: |
           for service in conversation gateway identity media message presence push relationship transmite; do
+            chmod +x "compose-artifacts/$service/build/${service}_server"
             rm -rf "$service/build" "$service/depends"
             cp -a "compose-artifacts/$service/build" "$service/build"
             cp -a "compose-artifacts/$service/depends" "$service/depends"
@@ -197,6 +198,7 @@ jobs:
       - name: Restore Compose build contexts
         run: |
           for service in conversation gateway identity media message presence push relationship transmite; do
+            chmod +x "compose-artifacts/$service/build/${service}_server"
             rm -rf "$service/build" "$service/depends"
             cp -a "compose-artifacts/$service/build" "$service/build"
             cp -a "compose-artifacts/$service/depends" "$service/depends"
diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index a3d8278..a5a6844 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -171,9 +171,18 @@ cmake --build build --parallel "$(nproc)" --target conversation_server gateway_s
 	packageCommand  = `docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci ./scripts/package_compose_artifacts.sh build compose-artifacts`
 	validateCommand = `docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci ./scripts/validate_compose_artifacts.sh compose-artifacts`
 	restoreCommand  = `for service in conversation gateway identity media message presence push relationship transmite; do
+  chmod +x "compose-artifacts/$service/build/${service}_server"
   rm -rf "$service/build" "$service/depends"
   cp -a "compose-artifacts/$service/build" "$service/build"
   cp -a "compose-artifacts/$service/depends" "$service/depends"
+done`
+	restoreWithoutChmodCommand = `for service in conversation gateway identity media message presence push relationship transmite; do
+  rm -rf "$service/build" "$service/depends"
+  cp -a "compose-artifacts/$service/build" "$service/build"
+  cp -a "compose-artifacts/$service/depends" "$service/depends"
+done`
+	chmodArtifactsCommand = `for service in conversation gateway identity media message presence push relationship transmite; do
+  chmod +x "compose-artifacts/$service/build/${service}_server"
 done`
 )
 
@@ -345,6 +354,13 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 		"restore allowed to fail": func(job *workflowJob) {
 			job.Steps[restore].ContinueOnError = true
 		},
+		"restore without executable mode repair": func(job *workflowJob) {
+			job.Steps[restore].Run = restoreWithoutChmodCommand
+		},
+		"executable mode repair after validation": func(job *workflowJob) {
+			job.Steps[restore].Run = restoreWithoutChmodCommand
+			insertWorkflowStep(job, validate+1, workflowStep{Run: chmodArtifactsCommand})
+		},
 		"artifact validation allowed to fail": func(job *workflowJob) {
 			job.Steps[validate].ContinueOnError = true
 		},

From b3ceb335726c870df85308619ee319b82b0183a2 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Fri, 17 Jul 2026 11:01:54 +0800
Subject: [PATCH 42/47] fix(message): restore sequence read watermarks

---
 message/source/message_server.h      | 16 +++-------------
 proto/message/message_service.proto  |  3 ++-
 tests/func/message_test.go           | 14 +++++++-------
 tests/pkg/contracts/ci_gates_test.go | 21 +++++++++++++++++++++
 4 files changed, 33 insertions(+), 21 deletions(-)

diff --git a/message/source/message_server.h b/message/source/message_server.h
index a0f1bae..e3b2a32 100644
--- a/message/source/message_server.h
+++ b/message/source/message_server.h
@@ -450,23 +450,13 @@ class MessageServiceImpl : public chatnow::message::MessageService {
         brpc::ClosureGuard done_guard(done);
         auto* cntl = static_cast<brpc::Controller*>(base_cntl);
         HANDLE_RPC(cntl, req, rsp, {
-            if (req->message_id() == 0) {
+            if (req->seq_id() == 0) {
                 throw ::chatnow::ServiceError(::chatnow::error::kSystemInvalidArgument,
-                                              "message_id required");
+                                              "seq_id required");
             }
-            auto msg = _mysql_msg->select_by_id(
-                static_cast<unsigned long>(req->message_id()));
-            if (!msg || msg->session_id() != req->conversation_id()) {
-                throw ::chatnow::ServiceError(::chatnow::error::kMessageNotFound,
-                                              "ack message not found");
-            }
-            uint64_t session_seq = resolve_ack_session_seq(msg->seq_id());
-            if (session_seq == 0)
-                throw ::chatnow::ServiceError(::chatnow::error::kSystemInvalidArgument,
-                                              "message seq_id required");
             require_member_(req->conversation_id(), auth.user_id);
             bool ok = _mysql_member->update_last_ack_seq(
-                req->conversation_id(), auth.user_id, session_seq);
+                req->conversation_id(), auth.user_id, req->seq_id());
             if (!ok)
                 throw ::chatnow::ServiceError(::chatnow::error::kSystemInternalError,
                                               "update last_ack_seq failed");
diff --git a/proto/message/message_service.proto b/proto/message/message_service.proto
index b7daa39..0c8024e 100644
--- a/proto/message/message_service.proto
+++ b/proto/message/message_service.proto
@@ -150,6 +150,7 @@ message SelectByClientMsgIdRsp {
 message UpdateReadAckReq {
     string request_id = 1;
     string conversation_id = 2;
-    uint64 message_id = 3;
+    // 会话内单调递增的已读水位线；服务端只允许向前推进。
+    uint64 seq_id = 3;
 }
 message UpdateReadAckRsp { chatnow.common.ResponseHeader header = 1; }
diff --git a/tests/func/message_test.go b/tests/func/message_test.go
index e86f2aa..58fb725 100644
--- a/tests/func/message_test.go
+++ b/tests/func/message_test.go
@@ -444,7 +444,7 @@ func TestFN_MS_SelectByClientMsgId_NotFound(t *testing.T) {
 	assert.Nil(t, rsp.Message, "不存在的 client_msg_id 应返回 nil message")
 }
 
-// FN-MS | P0 | UpdateReadAck advances the delivery acknowledgement cursor.
+// FN-MS | P0 | UpdateReadAck advances the conversation read watermark.
 func TestFN_MS_UpdateReadAck_Success(t *testing.T) {
 	alice, bob, convID := fixture.MakeFriends(t, HTTP)
 	messageID, seqID := fixture.SendTextMessage(t, alice, convID, "ack-test-msg")
@@ -455,7 +455,7 @@ func TestFN_MS_UpdateReadAck_Success(t *testing.T) {
 	req := &msg.UpdateReadAckReq{
 		RequestId:      client.NewRequestID(),
 		ConversationId: convID,
-		MessageId:      uint64(messageID),
+		SeqId:          seqID,
 	}
 	rsp := &msg.UpdateReadAckRsp{}
 	err := bob.DoAuth("/service/message/update_read_ack", req, rsp)
@@ -469,7 +469,7 @@ func TestFN_MS_UpdateReadAck_Success(t *testing.T) {
 // FN-MS | P0 | UpdateReadAck is idempotent and never moves backwards.
 func TestFN_MS_UpdateReadAck_Idempotent(t *testing.T) {
 	alice, bob, convID := fixture.MakeFriends(t, HTTP)
-	messageID1, _ := fixture.SendTextMessage(t, alice, convID, "ack-idempotent-1")
+	messageID1, seq1 := fixture.SendTextMessage(t, alice, convID, "ack-idempotent-1")
 	messageID2, seq2 := fixture.SendTextMessage(t, alice, convID, "ack-idempotent-2")
 	verifier := verify.NewDBVerifier(Cfg.Database.MySQLDSN)
 	defer verifier.Close()
@@ -480,23 +480,23 @@ func TestFN_MS_UpdateReadAck_Idempotent(t *testing.T) {
 	ackReq := &msg.UpdateReadAckReq{
 		RequestId:      client.NewRequestID(),
 		ConversationId: convID,
-		MessageId:      uint64(messageID2),
+		SeqId:          seq2,
 	}
 	ackRsp := &msg.UpdateReadAckRsp{}
 	require.NoError(t, bob.DoAuth("/service/message/update_read_ack", ackReq, ackRsp))
 	require.True(t, ackRsp.GetHeader().GetSuccess(),
-		"newer delivery ACK failed: %s", ackRsp.GetHeader().GetErrorMessage())
+		"newer read watermark failed: %s", ackRsp.GetHeader().GetErrorMessage())
 
 	// 再 ACK 较早消息，last_ack_seq 不应回退。
 	ackReq2 := &msg.UpdateReadAckReq{
 		RequestId:      client.NewRequestID(),
 		ConversationId: convID,
-		MessageId:      uint64(messageID1),
+		SeqId:          seq1,
 	}
 	ackRsp2 := &msg.UpdateReadAckRsp{}
 	require.NoError(t, bob.DoAuth("/service/message/update_read_ack", ackReq2, ackRsp2))
 	require.True(t, ackRsp2.GetHeader().GetSuccess(),
-		"backward delivery ACK must be idempotent: %s", ackRsp2.GetHeader().GetErrorMessage())
+		"backward read watermark must be idempotent: %s", ackRsp2.GetHeader().GetErrorMessage())
 
 	// 直查 DB 验证 last_ack_seq 仍为 seq2。
 	verifier.LastAckSeq(t, bob.UserID, convID, seq2)
diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index a5a6844..3d63288 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -86,6 +86,27 @@ func TestGoCacheRegressionsReplaceTemporaryCPP(t *testing.T) {
 		"the Go functional suite must own the same-user_seq latest-payload/ACK regression")
 }
 
+func TestReadAckUsesConversationSequenceWatermark(t *testing.T) {
+	root := repositoryRoot(t)
+	serviceProto, err := os.ReadFile(filepath.Join(root, "proto/message/message_service.proto"))
+	require.NoError(t, err)
+	require.Contains(t, string(serviceProto), "uint64 seq_id = 3;")
+	require.NotContains(t, string(serviceProto), "uint64 message_id = 3;")
+
+	server, err := os.ReadFile(filepath.Join(root, "message/source/message_server.h"))
+	require.NoError(t, err)
+	updateReadAck := string(server)
+	start := strings.Index(updateReadAck, "void UpdateReadAck(")
+	require.GreaterOrEqual(t, start, 0)
+	end := strings.Index(updateReadAck[start:], "\n    // ====== MQ consumer")
+	require.Greater(t, end, 0)
+	updateReadAck = updateReadAck[start : start+end]
+	require.Contains(t, updateReadAck, "req->seq_id()")
+	require.Contains(t, updateReadAck, "update_last_ack_seq(")
+	require.NotContains(t, updateReadAck, "req->message_id()")
+	require.NotContains(t, updateReadAck, "select_by_id(")
+}
+
 type workflowContract struct {
 	On struct {
 		PullRequest map[string]any   `yaml:"pull_request"`

From 96aefa597e671c9b36bf0c656c9f9faf0e8ed6c1 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Fri, 17 Jul 2026 11:09:47 +0800
Subject: [PATCH 43/47] fix(ci): keep system ABI out of service artifacts

---
 .github/workflows/ci.yml              | 72 +++++++++++++++------------
 conversation/Dockerfile               |  6 ++-
 gateway/Dockerfile                    |  8 +--
 identity/Dockerfile                   |  8 +--
 media/Dockerfile                      |  8 +--
 message/Dockerfile                    |  8 +--
 presence/Dockerfile                   |  6 ++-
 push/Dockerfile                       |  6 ++-
 relationship/Dockerfile               |  8 +--
 scripts/package_compose_artifacts.sh  | 18 ++++++-
 scripts/validate_compose_artifacts.sh | 13 ++++-
 tests/pkg/contracts/artifacts_test.go | 69 ++++++++++++++++++++++++-
 tests/pkg/contracts/ci_gates_test.go  | 36 +++++++++-----
 transmite/Dockerfile                  |  8 +--
 14 files changed, 201 insertions(+), 73 deletions(-)

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index d08c7bd..029f165 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -74,9 +74,9 @@ jobs:
           if-no-files-found: error
 
   bvt:
-    needs: build
+    needs: service-artifacts
     runs-on: ubuntu-22.04
-    if: github.event_name != 'push' || github.ref != 'refs/heads/main'
+    if: github.event_name == 'pull_request' || github.event_name == 'schedule'
     steps:
       - uses: actions/checkout@v4
       - uses: actions/setup-go@v5
@@ -88,6 +88,30 @@ jobs:
           sudo apt-get install -y protobuf-compiler netcat-openbsd
       - name: Install Go protobuf generator
         run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
+      - &download-compose-artifacts
+        name: Download Compose service artifacts
+        uses: actions/download-artifact@v4
+        with:
+          name: compose-service-artifacts
+          path: compose-artifacts
+      - &restore-compose-artifacts
+        name: Restore Compose build contexts
+        run: |
+          for service in conversation gateway identity media message presence push relationship transmite; do
+            rm -rf "$service/build" "$service/depends"
+            cp -a "compose-artifacts/$service/build" "$service/build"
+            cp -a "compose-artifacts/$service/depends" "$service/depends"
+          done
+      - &chmod-compose-artifacts
+        name: Restore service executable modes
+        run: |
+          for service in conversation gateway identity media message presence push relationship transmite; do
+            chmod +x "compose-artifacts/$service/build/${service}_server"
+            chmod +x "$service/build/${service}_server"
+          done
+      - &validate-compose-artifacts
+        name: Validate downloaded Compose service artifacts
+        run: docker run --rm -v "$PWD:/workspace" -w /workspace ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 ./scripts/validate_compose_artifacts.sh compose-artifacts
       - name: Start full stack
         run: docker compose up -d --build
       - name: Wait for services
@@ -103,7 +127,7 @@ jobs:
         run: docker compose down -v
 
   func:
-    needs: bvt
+    needs: service-artifacts
     runs-on: ubuntu-22.04
     if: github.event_name == 'pull_request' || github.event_name == 'schedule'
     steps:
@@ -117,6 +141,10 @@ jobs:
           sudo apt-get install -y protobuf-compiler netcat-openbsd
       - name: Install Go protobuf generator
         run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
+      - *download-compose-artifacts
+      - *restore-compose-artifacts
+      - *chmod-compose-artifacts
+      - *validate-compose-artifacts
       - name: Start full stack
         run: docker compose up -d --build
       - name: Wait for services
@@ -146,21 +174,10 @@ jobs:
           sudo apt-get install -y protobuf-compiler netcat-openbsd
       - name: Install Go protobuf generator
         run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
-      - name: Download Compose service artifacts
-        uses: actions/download-artifact@v4
-        with:
-          name: compose-service-artifacts
-          path: compose-artifacts
-      - name: Restore Compose build contexts
-        run: |
-          for service in conversation gateway identity media message presence push relationship transmite; do
-            chmod +x "compose-artifacts/$service/build/${service}_server"
-            rm -rf "$service/build" "$service/depends"
-            cp -a "compose-artifacts/$service/build" "$service/build"
-            cp -a "compose-artifacts/$service/depends" "$service/depends"
-          done
-      - name: Validate downloaded Compose service artifacts
-        run: docker run --rm -v "$PWD:/workspace" -w /workspace ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 ./scripts/validate_compose_artifacts.sh compose-artifacts
+      - *download-compose-artifacts
+      - *restore-compose-artifacts
+      - *chmod-compose-artifacts
+      - *validate-compose-artifacts
       - name: Start full stack
         run: docker compose up -d --build
       - name: Wait for services
@@ -190,21 +207,10 @@ jobs:
           sudo apt-get install -y protobuf-compiler netcat-openbsd
       - name: Install Go protobuf generator
         run: go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11
-      - name: Download Compose service artifacts
-        uses: actions/download-artifact@v4
-        with:
-          name: compose-service-artifacts
-          path: compose-artifacts
-      - name: Restore Compose build contexts
-        run: |
-          for service in conversation gateway identity media message presence push relationship transmite; do
-            chmod +x "compose-artifacts/$service/build/${service}_server"
-            rm -rf "$service/build" "$service/depends"
-            cp -a "compose-artifacts/$service/build" "$service/build"
-            cp -a "compose-artifacts/$service/depends" "$service/depends"
-          done
-      - name: Validate downloaded Compose service artifacts
-        run: docker run --rm -v "$PWD:/workspace" -w /workspace ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90 ./scripts/validate_compose_artifacts.sh compose-artifacts
+      - *download-compose-artifacts
+      - *restore-compose-artifacts
+      - *chmod-compose-artifacts
+      - *validate-compose-artifacts
       - name: Start full stack with PF-09 rate limits
         env:
           # Transmite flags are int32; use the maximum valid value for gate-only headroom.
diff --git a/conversation/Dockerfile b/conversation/Dockerfile
index fb12a1e..fe7b7ec 100644
--- a/conversation/Dockerfile
+++ b/conversation/Dockerfile
@@ -1,7 +1,9 @@
-FROM ubuntu:24.04
+ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
+FROM ${UBUNTU_IMAGE}
+ENV LD_LIBRARY_PATH=/im/depends
 WORKDIR /im
 RUN mkdir -p /im/logs && mkdir -p /im/data && mkdir -p /im/conf && mkdir -p /im/bin
 COPY ./build/conversation_server /im/bin
-COPY ./depends/* /lib/x86_64-linux-gnu/
+COPY ./depends/ /im/depends/
 RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/*
 CMD /im/bin/conversation_server -flagfile=/im/conf/conversation_server.conf
diff --git a/gateway/Dockerfile b/gateway/Dockerfile
index e059354..9a1473e 100644
--- a/gateway/Dockerfile
+++ b/gateway/Dockerfile
@@ -1,5 +1,7 @@
 # 声明基础镜像来源
-FROM ubuntu:24.04
+ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
+FROM ${UBUNTU_IMAGE}
+ENV LD_LIBRARY_PATH=/im/depends
 # 声明工作路径
 WORKDIR /im
 RUN mkdir -p /im/logs &&\
@@ -9,7 +11,7 @@ RUN mkdir -p /im/logs &&\
 # 将可执行程序文件，拷贝进入镜像
 COPY ./build/gateway_server /im/bin
 # 将可执行程序依赖，拷贝进入镜像
-COPY ./depends/* /lib/x86_64-linux-gnu/
+COPY ./depends/ /im/depends/
 RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/*
 # 设置容器的启动默认操作 --- 运行程序
-CMD /im/bin/gateway_server -flagfile=/im/conf/gateway_server.conf
\ No newline at end of file
+CMD /im/bin/gateway_server -flagfile=/im/conf/gateway_server.conf
diff --git a/identity/Dockerfile b/identity/Dockerfile
index 94a6007..c26649a 100644
--- a/identity/Dockerfile
+++ b/identity/Dockerfile
@@ -1,5 +1,7 @@
 # 声明基础镜像来源
-FROM ubuntu:24.04
+ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
+FROM ${UBUNTU_IMAGE}
+ENV LD_LIBRARY_PATH=/im/depends
 # 声明工作路径
 WORKDIR /im
 RUN mkdir -p /im/logs &&\
@@ -9,7 +11,7 @@ RUN mkdir -p /im/logs &&\
 # 将可执行程序文件，拷贝进入镜像
 COPY ./build/identity_server /im/bin
 # 将可执行程序依赖，拷贝进入镜像
-COPY ./depends/* /lib/x86_64-linux-gnu/
+COPY ./depends/ /im/depends/
 RUN apt-get update && apt-get install -y ca-certificates netcat-openbsd && rm -rf /var/lib/apt/lists/*
 # 设置容器的启动默认操作 --- 运行程序
-CMD /im/bin/identity_server -flagfile=/im/conf/identity_server.conf
\ No newline at end of file
+CMD /im/bin/identity_server -flagfile=/im/conf/identity_server.conf
diff --git a/media/Dockerfile b/media/Dockerfile
index ea9a789..ae3c167 100644
--- a/media/Dockerfile
+++ b/media/Dockerfile
@@ -1,5 +1,7 @@
 # 声明基础镜像来源
-FROM ubuntu:24.04
+ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
+FROM ${UBUNTU_IMAGE}
+ENV LD_LIBRARY_PATH=/im/depends
 # 声明工作路径
 WORKDIR /im
 RUN mkdir -p /im/logs &&\
@@ -9,7 +11,7 @@ RUN mkdir -p /im/logs &&\
 # 将可执行程序文件，拷贝进入镜像
 COPY ./build/media_server /im/bin
 # 将可执行程序依赖，拷贝进入镜像
-COPY ./depends/* /lib/x86_64-linux-gnu/
+COPY ./depends/ /im/depends/
 RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/*
 # 设置容器的启动默认操作 --- 运行程序
-CMD /im/bin/media_server -flagfile=/im/conf/media_server.conf
\ No newline at end of file
+CMD /im/bin/media_server -flagfile=/im/conf/media_server.conf
diff --git a/message/Dockerfile b/message/Dockerfile
index 0973192..f01876a 100644
--- a/message/Dockerfile
+++ b/message/Dockerfile
@@ -1,5 +1,7 @@
 # 声明基础镜像来源
-FROM ubuntu:24.04
+ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
+FROM ${UBUNTU_IMAGE}
+ENV LD_LIBRARY_PATH=/im/depends
 # 声明工作路径
 WORKDIR /im
 RUN mkdir -p /im/logs &&\
@@ -9,7 +11,7 @@ RUN mkdir -p /im/logs &&\
 # 将可执行程序文件，拷贝进入镜像
 COPY ./build/message_server /im/bin
 # 将可执行程序依赖，拷贝进入镜像
-COPY ./depends/* /lib/x86_64-linux-gnu/
+COPY ./depends/ /im/depends/
 RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/*
 # 设置容器的启动默认操作 --- 运行程序
-CMD /im/bin/message_server -flagfile=/im/conf/message_server.conf
\ No newline at end of file
+CMD /im/bin/message_server -flagfile=/im/conf/message_server.conf
diff --git a/presence/Dockerfile b/presence/Dockerfile
index bd43bb1..71af26a 100644
--- a/presence/Dockerfile
+++ b/presence/Dockerfile
@@ -1,5 +1,7 @@
 # 声明基础镜像来源
-FROM ubuntu:24.04
+ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
+FROM ${UBUNTU_IMAGE}
+ENV LD_LIBRARY_PATH=/im/depends
 # 声明工作路径
 WORKDIR /im
 RUN mkdir -p /im/logs &&\
@@ -9,7 +11,7 @@ RUN mkdir -p /im/logs &&\
 # 将可执行程序文件，拷贝进入镜像
 COPY ./build/presence_server /im/bin
 # 将可执行程序依赖，拷贝进入镜像
-COPY ./depends/* /lib/x86_64-linux-gnu/
+COPY ./depends/ /im/depends/
 RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/*
 # 设置容器的启动默认操作 --- 运行程序
 CMD /im/bin/presence_server -flagfile=/im/conf/presence_server.conf
diff --git a/push/Dockerfile b/push/Dockerfile
index 2586f36..103eb1c 100644
--- a/push/Dockerfile
+++ b/push/Dockerfile
@@ -1,5 +1,7 @@
 # 声明基础镜像来源
-FROM ubuntu:24.04
+ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
+FROM ${UBUNTU_IMAGE}
+ENV LD_LIBRARY_PATH=/im/depends
 # 声明工作路径
 WORKDIR /im
 RUN mkdir -p /im/logs &&\
@@ -9,7 +11,7 @@ RUN mkdir -p /im/logs &&\
 # 将可执行程序文件，拷贝进入镜像
 COPY ./build/push_server /im/bin
 # 将可执行程序依赖，拷贝进入镜像
-COPY ./depends/* /lib/x86_64-linux-gnu/
+COPY ./depends/ /im/depends/
 RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/*
 # 设置容器的启动默认操作 --- 运行程序
 CMD /im/bin/push_server -flagfile=/im/conf/push_server.conf
diff --git a/relationship/Dockerfile b/relationship/Dockerfile
index a561296..58f61ad 100644
--- a/relationship/Dockerfile
+++ b/relationship/Dockerfile
@@ -1,5 +1,7 @@
 # 声明基础镜像来源
-FROM ubuntu:24.04
+ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
+FROM ${UBUNTU_IMAGE}
+ENV LD_LIBRARY_PATH=/im/depends
 # 声明工作路径
 WORKDIR /im
 RUN mkdir -p /im/logs &&\
@@ -9,7 +11,7 @@ RUN mkdir -p /im/logs &&\
 # 将可执行程序文件，拷贝进入镜像
 COPY ./build/relationship_server /im/bin
 # 将可执行程序依赖，拷贝进入镜像
-COPY ./depends/* /lib/x86_64-linux-gnu/
+COPY ./depends/ /im/depends/
 RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/*
 # 设置容器的启动默认操作 --- 运行程序
-CMD /im/bin/relationship_server -flagfile=/im/conf/relationship_server.conf
\ No newline at end of file
+CMD /im/bin/relationship_server -flagfile=/im/conf/relationship_server.conf
diff --git a/scripts/package_compose_artifacts.sh b/scripts/package_compose_artifacts.sh
index 61987b7..eb31131 100755
--- a/scripts/package_compose_artifacts.sh
+++ b/scripts/package_compose_artifacts.sh
@@ -6,6 +6,8 @@ build_root="${1:-build}"
 artifact_root="${2:-compose-artifacts}"
 ldd_command="${LDD:-ldd}"
 sha256sum_command="${SHA256SUM:-sha256sum}"
+local_prefix="${LOCAL_PREFIX:-/usr/local}"
+local_prefix="$(cd "$local_prefix" && pwd -P)"
 
 rm -rf "$artifact_root"
 mkdir -p "$artifact_root"
@@ -34,8 +36,22 @@ for service in "${services[@]}"; do
 
     while IFS= read -r library; do
         [[ -n "$library" ]] || continue
+        resolved_library="$(realpath "$library")" || {
+            echo "unable to resolve shared library for $binary: $library" >&2
+            exit 1
+        }
+        case "$resolved_library" in
+            "$local_prefix"/*)
+                ;;
+            *)
+                # Distribution libraries and the ELF loader are supplied by the
+                # runtime image, which is pinned to the builder's exact digest.
+                # A library is outside local build prefix and is not packaged.
+                continue
+                ;;
+        esac
         library_name="$(basename "$library")"
-        cp -L "$library" "$depends_dir/$library_name"
+        cp -L "$resolved_library" "$depends_dir/$library_name"
     done < <(awk '
         /=> \/[^ ]+/ { print $3; next }
         /^[[:space:]]*\/[^ ]+/ { print $1 }
diff --git a/scripts/validate_compose_artifacts.sh b/scripts/validate_compose_artifacts.sh
index f94caa2..49c7495 100755
--- a/scripts/validate_compose_artifacts.sh
+++ b/scripts/validate_compose_artifacts.sh
@@ -45,6 +45,16 @@ for service in "${services[@]}"; do
     fi
     depends_dir_real="$(cd "$depends_dir" && pwd -P)"
 
+    while IFS= read -r packaged_library; do
+        library_name="$(basename "$packaged_library")"
+        case "$library_name" in
+            ld-linux*.so.*|ld-musl-*.so.*|libc.so.*|libm.so.*|libpthread.so.*|librt.so.*|libdl.so.*|libgcc_s.so.*|libstdc++.so.*|libanl.so.*|libBrokenLocale.so.*|libcrypt.so.*|libnss_*.so.*|libresolv.so.*|libutil.so.*)
+                echo "packaged runtime loader or system ABI library for $binary: $packaged_library" >&2
+                exit 1
+                ;;
+        esac
+    done < <(find "$depends_dir" -mindepth 1 -maxdepth 1 \( -type f -o -type l \) -print | LC_ALL=C sort)
+
     ldd_output="$(env -i PATH=/usr/bin:/bin LD_LIBRARY_PATH="$depends_dir" "$ldd_command" "$binary" 2>&1)" || {
         echo "isolated ldd failed for $binary: $ldd_output" >&2
         exit 1
@@ -60,7 +70,7 @@ for service in "${services[@]}"; do
         library_name="$(basename "$library")"
         if [[ "$entry_kind" == "loader" ]]; then
             case "$library_name" in
-                ld-linux*.so.*|ld-musl-*.so.*) continue ;;
+                ld-linux*.so.*|ld-musl-*.so.*) ;;
                 *)
                     echo "unexpected system library outside packaged closure for $binary: $library" >&2
                     exit 1
@@ -75,6 +85,7 @@ for service in "${services[@]}"; do
         resolved_library="$(realpath "$library")"
         case "$resolved_library" in
             "$depends_dir_real"/*) ;;
+            /lib/*|/lib64/*|/usr/lib/*) ;;
             *)
                 echo "shared library is outside packaged closure for $binary: $library" >&2
                 exit 1
diff --git a/tests/pkg/contracts/artifacts_test.go b/tests/pkg/contracts/artifacts_test.go
index 6d3f713..8c21ff3 100644
--- a/tests/pkg/contracts/artifacts_test.go
+++ b/tests/pkg/contracts/artifacts_test.go
@@ -41,6 +41,17 @@ func TestComposeArtifactBuilderIsLocked(t *testing.T) {
 		require.Contains(t, dockerfile, `"$`+dependency+`"`, "%s must be consumed by the builder", dependency)
 	}
 	require.Contains(t, dockerfile, "ldconfig")
+
+	for _, service := range composeArtifactServices {
+		runtimeDockerfile := readContractFile(t, root, service+"/Dockerfile")
+		require.Contains(t, runtimeDockerfile, "ARG UBUNTU_IMAGE="+imageLock[1],
+			"%s runtime must use the builder's exact Ubuntu digest", service)
+		require.Contains(t, runtimeDockerfile, "FROM ${UBUNTU_IMAGE}")
+		require.Contains(t, runtimeDockerfile, "COPY ./depends/ /im/depends/")
+		require.Contains(t, runtimeDockerfile, "LD_LIBRARY_PATH=/im/depends")
+		require.NotContains(t, runtimeDockerfile, "COPY ./depends/* /lib",
+			"%s must not overwrite distribution libraries", service)
+	}
 }
 
 func TestComposeArtifactPackagerContract(t *testing.T) {
@@ -55,6 +66,8 @@ func TestComposeArtifactPackagerContract(t *testing.T) {
 	require.Contains(t, script, `"$build_root/$service/${service}_server"`)
 	require.Contains(t, script, `[[ -x "$binary" ]]`)
 	require.Contains(t, script, "not found")
+	require.Contains(t, script, `local_prefix="${LOCAL_PREFIX:-/usr/local}"`)
+	require.Contains(t, script, "library is outside local build prefix")
 	require.Contains(t, script, "sha256sum")
 	require.Contains(t, script, "sort -z")
 
@@ -101,6 +114,7 @@ func TestComposeArtifactValidatorContract(t *testing.T) {
 	require.Contains(t, script, "not found")
 	require.Contains(t, script, "depends_dir_real")
 	require.Contains(t, script, "resolved_library")
+	require.Contains(t, script, "packaged runtime loader or system ABI library")
 }
 
 func TestComposeArtifactScriptsEndToEnd(t *testing.T) {
@@ -122,6 +136,46 @@ func TestComposeArtifactScriptsEndToEnd(t *testing.T) {
 		require.NoError(t, err, "%s", output)
 	})
 
+	t.Run("does not package runtime loader or distribution libraries", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.lddPath = fixture.writeLDDWithSystemRuntime(t, filepath.Join(fixture.tools, "ldd-system"))
+		fixture.packageArtifacts(t)
+
+		for _, service := range composeArtifactServices {
+			depends := filepath.Join(fixture.artifactRoot, service, "depends")
+			require.FileExists(t, filepath.Join(depends, "libfixture.so"))
+			require.NoFileExists(t, filepath.Join(depends, "libc.so.6"))
+			require.NoFileExists(t, filepath.Join(depends, "libstdc++.so.6"))
+			require.NoFileExists(t, filepath.Join(depends, "ld-linux-x86-64.so.2"))
+		}
+	})
+
+	t.Run("rejects injected packaged system ABI library", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.packageArtifacts(t)
+		require.NoError(t, os.WriteFile(
+			filepath.Join(fixture.artifactRoot, "gateway", "depends", "libc.so.6"),
+			[]byte("not glibc\n"), 0o644))
+		fixture.rewriteManifest(t)
+
+		output, err := fixture.validate(t, fixture.lddPath)
+		require.Error(t, err)
+		require.Contains(t, output, "packaged runtime loader or system ABI library")
+	})
+
+	t.Run("rejects injected packaged runtime loader", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.packageArtifacts(t)
+		require.NoError(t, os.WriteFile(
+			filepath.Join(fixture.artifactRoot, "identity", "depends", "ld-linux-x86-64.so.2"),
+			[]byte("not a loader\n"), 0o755))
+		fixture.rewriteManifest(t)
+
+		output, err := fixture.validate(t, fixture.lddPath)
+		require.Error(t, err)
+		require.Contains(t, output, "packaged runtime loader or system ABI library")
+	})
+
 	t.Run("rejects manifest tampering", func(t *testing.T) {
 		fixture := newArtifactFixture(t, root)
 		fixture.packageArtifacts(t)
@@ -285,10 +339,23 @@ echo "libfixture.so => $library (0x1)"
 	return path
 }
 
+func (fixture artifactFixture) writeLDDWithSystemRuntime(t *testing.T, path string) string {
+	t.Helper()
+	script := `#!/bin/sh
+set -eu
+echo "libfixture.so => ${LD_LIBRARY_PATH:-` + fixture.temp + `}/libfixture.so (0x1)"
+echo "libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x2)"
+echo "libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x3)"
+echo "/lib64/ld-linux-x86-64.so.2 (0x4)"
+`
+	require.NoError(t, os.WriteFile(path, []byte(script), 0o755))
+	return path
+}
+
 func (fixture artifactFixture) packageArtifacts(t *testing.T) {
 	t.Helper()
 	command := exec.Command("bash", filepath.Join(fixture.root, "scripts/package_compose_artifacts.sh"), fixture.buildRoot, fixture.artifactRoot)
-	command.Env = append(os.Environ(), "PATH="+fixture.tools+":"+os.Getenv("PATH"))
+	command.Env = append(os.Environ(), "PATH="+fixture.tools+":"+os.Getenv("PATH"), "LOCAL_PREFIX="+fixture.temp)
 	output, err := command.CombinedOutput()
 	require.NoError(t, err, "%s", output)
 }
diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index 3d63288..a302d09 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -155,19 +155,32 @@ func TestCIGates(t *testing.T) {
 	require.True(t, ok, "CI must build the Compose service artifacts once")
 	assertServiceArtifactProducer(t, producer)
 
+	for _, consumer := range []struct {
+		job    string
+		target string
+	}{
+		{"bvt", "cd tests && make test-bvt"},
+		{"func", "cd tests && make test-func"},
+		{"reliability", "cd tests && make test-reliability"},
+		{"perf-cache", "cd tests && make test-perf-cache-gate"},
+	} {
+		job, exists := workflow.Jobs[consumer.job]
+		require.True(t, exists, "%s must be a dedicated clean-runner consumer", consumer.job)
+		require.Equal(t, "service-artifacts", job.Needs)
+		assertFullStackGateJob(t, job, consumer.target)
+		assertInvalidGateJobsRejected(t, job, consumer.target)
+	}
+
 	reliability, ok := workflow.Jobs["reliability"]
 	require.True(t, ok, "RL-05 must have a dedicated reliability job")
 	require.Equal(t, "service-artifacts", reliability.Needs)
 	require.Equal(t, "github.event_name == 'pull_request' || github.event_name == 'schedule'", reliability.If)
-	assertFullStackGateJob(t, reliability, "cd tests && make test-reliability")
 
 	perfCache, ok := workflow.Jobs["perf-cache"]
 	require.True(t, ok, "PF-09 must have a dedicated perf-cache job")
 	require.Equal(t, "service-artifacts", perfCache.Needs)
 	require.Equal(t, "github.event_name == 'schedule'", perfCache.If)
-	assertFullStackGateJob(t, perfCache, "cd tests && make test-perf-cache-gate")
 	assertTargetAbsent(t, perfCache, "test-perf-cache")
-	assertInvalidGateJobsRejected(t, perfCache, "cd tests && make test-perf-cache-gate")
 
 	require.Equal(t, "2147483647", gateEnv(t, perfCache, "TRANSMITE_RATE_LIMIT_USER_MAX"))
 	require.Equal(t, "2147483647", gateEnv(t, perfCache, "TRANSMITE_RATE_LIMIT_SESSION_MAX"))
@@ -192,18 +205,13 @@ cmake --build build --parallel "$(nproc)" --target conversation_server gateway_s
 	packageCommand  = `docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci ./scripts/package_compose_artifacts.sh build compose-artifacts`
 	validateCommand = `docker run --rm -v "$PWD:/workspace" -w /workspace chatnow-ci-builder:ci ./scripts/validate_compose_artifacts.sh compose-artifacts`
 	restoreCommand  = `for service in conversation gateway identity media message presence push relationship transmite; do
-  chmod +x "compose-artifacts/$service/build/${service}_server"
-  rm -rf "$service/build" "$service/depends"
-  cp -a "compose-artifacts/$service/build" "$service/build"
-  cp -a "compose-artifacts/$service/depends" "$service/depends"
-done`
-	restoreWithoutChmodCommand = `for service in conversation gateway identity media message presence push relationship transmite; do
   rm -rf "$service/build" "$service/depends"
   cp -a "compose-artifacts/$service/build" "$service/build"
   cp -a "compose-artifacts/$service/depends" "$service/depends"
 done`
 	chmodArtifactsCommand = `for service in conversation gateway identity media message presence push relationship transmite; do
   chmod +x "compose-artifacts/$service/build/${service}_server"
+  chmod +x "$service/build/${service}_server"
 done`
 )
 
@@ -348,6 +356,7 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 	setupGo := exactUsesStepIndex(valid, "actions/setup-go@v5")
 	download := exactUsesStepIndex(valid, "actions/download-artifact@v4")
 	restore := exactRunStepIndex(valid, restoreCommand)
+	chmod := exactRunStepIndex(valid, chmodArtifactsCommand)
 	validate := exactRunStepIndex(valid, consumerValidateCommand)
 	start := exactRunStepIndex(valid, "docker compose up -d --build")
 	wait := exactRunStepIndex(valid, "./scripts/wait_for_services.sh")
@@ -358,6 +367,7 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 	require.NotEqual(t, -1, setupGo)
 	require.NotEqual(t, -1, download)
 	require.NotEqual(t, -1, restore)
+	require.NotEqual(t, -1, chmod)
 	require.NotEqual(t, -1, validate)
 	require.NotEqual(t, -1, start)
 	require.NotEqual(t, -1, wait)
@@ -375,12 +385,11 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 		"restore allowed to fail": func(job *workflowJob) {
 			job.Steps[restore].ContinueOnError = true
 		},
-		"restore without executable mode repair": func(job *workflowJob) {
-			job.Steps[restore].Run = restoreWithoutChmodCommand
+		"missing executable mode repair": func(job *workflowJob) {
+			job.Steps = append(job.Steps[:chmod], job.Steps[chmod+1:]...)
 		},
 		"executable mode repair after validation": func(job *workflowJob) {
-			job.Steps[restore].Run = restoreWithoutChmodCommand
-			insertWorkflowStep(job, validate+1, workflowStep{Run: chmodArtifactsCommand})
+			job.Steps[chmod], job.Steps[validate] = job.Steps[validate], job.Steps[chmod]
 		},
 		"artifact validation allowed to fail": func(job *workflowJob) {
 			job.Steps[validate].ContinueOnError = true
@@ -496,6 +505,7 @@ func validateFullStackGateJob(job workflowJob, target string) error {
 		{"protoc generator install", exactRunStepIndex(job, "go install google.golang.org/protobuf/cmd/protoc-gen-go@v1.36.11")},
 		{"artifact download", exactUsesStepIndex(job, "actions/download-artifact@v4")},
 		{"artifact restore", exactRunStepIndex(job, restoreCommand)},
+		{"executable mode repair", exactRunStepIndex(job, chmodArtifactsCommand)},
 		{"artifact validation", exactRunStepIndex(job, consumerValidateCommand)},
 		{"full-stack startup", exactRunStepIndex(job, "docker compose up -d --build")},
 		{"service wait", exactRunStepIndex(job, "./scripts/wait_for_services.sh")},
diff --git a/transmite/Dockerfile b/transmite/Dockerfile
index 6fc62fc..fca8bd0 100644
--- a/transmite/Dockerfile
+++ b/transmite/Dockerfile
@@ -1,5 +1,7 @@
 # 声明基础镜像来源
-FROM ubuntu:24.04
+ARG UBUNTU_IMAGE=ubuntu:24.04@sha256:4fbb8e6a8395de5a7550b33509421a2bafbc0aab6c06ba2cef9ebffbc7092d90
+FROM ${UBUNTU_IMAGE}
+ENV LD_LIBRARY_PATH=/im/depends
 # 声明工作路径
 WORKDIR /im
 RUN mkdir -p /im/logs &&\
@@ -9,7 +11,7 @@ RUN mkdir -p /im/logs &&\
 # 将可执行程序文件，拷贝进入镜像
 COPY ./build/transmite_server /im/bin
 # 将可执行程序依赖，拷贝进入镜像
-COPY ./depends/* /lib/x86_64-linux-gnu/
+COPY ./depends/ /im/depends/
 RUN apt-get update && apt-get install -y netcat-openbsd && rm -rf /var/lib/apt/lists/*
 # 设置容器的启动默认操作 --- 运行程序
-CMD /im/bin/transmite_server -flagfile=/im/conf/transmite_server.conf
\ No newline at end of file
+CMD /im/bin/transmite_server -flagfile=/im/conf/transmite_server.conf

From ffffba7959f88de24304dd16b8cf6829ccf72c1b Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Fri, 17 Jul 2026 11:18:07 +0800
Subject: [PATCH 44/47] fix(ci): package complete non-core library closure

---
 .github/workflows/ci.yml              |  6 +--
 media/CMakeLists.txt                  |  5 ++-
 scripts/compose_artifact_core_abi.sh  | 21 +++++++++
 scripts/package_compose_artifacts.sh  | 25 ++++++-----
 scripts/validate_compose_artifacts.sh | 31 ++++++--------
 tests/pkg/contracts/artifacts_test.go | 36 ++++++++++++++--
 tests/pkg/contracts/ci_gates_test.go  | 62 ++++++++++++++++++++++++---
 7 files changed, 138 insertions(+), 48 deletions(-)
 create mode 100644 scripts/compose_artifact_core_abi.sh

diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml
index 029f165..1cfb6ab 100644
--- a/.github/workflows/ci.yml
+++ b/.github/workflows/ci.yml
@@ -43,7 +43,7 @@ jobs:
 
   service-artifacts:
     runs-on: ubuntu-22.04
-    if: github.event_name == 'pull_request' || github.event_name == 'schedule'
+    if: github.event_name != 'push' || github.ref != 'refs/heads/main'
     steps:
       - uses: actions/checkout@v4
       - uses: docker/setup-buildx-action@v3
@@ -76,7 +76,7 @@ jobs:
   bvt:
     needs: service-artifacts
     runs-on: ubuntu-22.04
-    if: github.event_name == 'pull_request' || github.event_name == 'schedule'
+    if: github.event_name != 'push' || github.ref != 'refs/heads/main'
     steps:
       - uses: actions/checkout@v4
       - uses: actions/setup-go@v5
@@ -127,7 +127,7 @@ jobs:
         run: docker compose down -v
 
   func:
-    needs: service-artifacts
+    needs: [service-artifacts, bvt]
     runs-on: ubuntu-22.04
     if: github.event_name == 'pull_request' || github.event_name == 'schedule'
     steps:
diff --git a/media/CMakeLists.txt b/media/CMakeLists.txt
index 28250ca..38855f8 100644
--- a/media/CMakeLists.txt
+++ b/media/CMakeLists.txt
@@ -62,6 +62,7 @@ set(src_files
     ${CMAKE_CURRENT_SOURCE_DIR}/source/download_handler.hpp
     ${CMAKE_CURRENT_SOURCE_DIR}/source/speech_handler.hpp
     ${CMAKE_CURRENT_SOURCE_DIR}/source/cleanup_worker.hpp)
+find_package(jsoncpp CONFIG REQUIRED)
 add_executable(${target} ${src_files} ${proto_srcs} ${odb_srcs})
 
 target_link_libraries(${target}
@@ -71,7 +72,7 @@ target_link_libraries(${target}
     -lodb-mysql -lodb -lodb-boost
     -lredis++ -lhiredis
     -laws-cpp-sdk-s3 -laws-cpp-sdk-core
-    /usr/local/lib/libjsoncpp.so.19)
+    jsoncpp_lib)
 
 # 5. 测试可执行（gtest 单元 + 集成）
 set(test_files
@@ -86,7 +87,7 @@ set(test_files
 #     -lodb-mysql -lodb -lodb-boost
 #     -lredis++ -lhiredis
 #     -laws-cpp-sdk-s3 -laws-cpp-sdk-core
-#     /usr/local/lib/libjsoncpp.so.19)
+#     jsoncpp_lib)
 
 # 6. 头文件搜索路径
 target_include_directories(${target} PRIVATE ${CMAKE_CURRENT_BINARY_DIR})
diff --git a/scripts/compose_artifact_core_abi.sh b/scripts/compose_artifact_core_abi.sh
new file mode 100644
index 0000000..30f88c9
--- /dev/null
+++ b/scripts/compose_artifact_core_abi.sh
@@ -0,0 +1,21 @@
+#!/usr/bin/env bash
+
+# Libraries guaranteed by the pinned Ubuntu base image. Everything else must
+# travel in the service-private artifact closure.
+is_core_system_abi() {
+    case "$(basename "$1")" in
+        ld-linux*.so.*|ld-musl-*.so.*|libc.so.*|libm.so.*|libpthread.so.*|librt.so.*|libdl.so.*|libgcc_s.so.*|libstdc++.so.*|libanl.so.*|libBrokenLocale.so.*|libcrypt.so.*|libnss_*.so.*|libresolv.so.*|libutil.so.*)
+            return 0
+            ;;
+        *)
+            return 1
+            ;;
+    esac
+}
+
+is_system_library_path() {
+    case "$1" in
+        /lib/*|/lib64/*|/usr/lib/*) return 0 ;;
+        *) return 1 ;;
+    esac
+}
diff --git a/scripts/package_compose_artifacts.sh b/scripts/package_compose_artifacts.sh
index eb31131..9185a23 100755
--- a/scripts/package_compose_artifacts.sh
+++ b/scripts/package_compose_artifacts.sh
@@ -1,13 +1,15 @@
 #!/usr/bin/env bash
 set -euo pipefail
 
+script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
+# shellcheck source=compose_artifact_core_abi.sh
+. "$script_dir/compose_artifact_core_abi.sh"
+
 services=(conversation gateway identity media message presence push relationship transmite)
 build_root="${1:-build}"
 artifact_root="${2:-compose-artifacts}"
 ldd_command="${LDD:-ldd}"
 sha256sum_command="${SHA256SUM:-sha256sum}"
-local_prefix="${LOCAL_PREFIX:-/usr/local}"
-local_prefix="$(cd "$local_prefix" && pwd -P)"
 
 rm -rf "$artifact_root"
 mkdir -p "$artifact_root"
@@ -36,21 +38,18 @@ for service in "${services[@]}"; do
 
     while IFS= read -r library; do
         [[ -n "$library" ]] || continue
+        library_name="$(basename "$library")"
+        if is_core_system_abi "$library_name"; then
+            if ! is_system_library_path "$library"; then
+                echo "core system ABI resolved outside pinned runtime paths for $binary: $library" >&2
+                exit 1
+            fi
+            continue
+        fi
         resolved_library="$(realpath "$library")" || {
             echo "unable to resolve shared library for $binary: $library" >&2
             exit 1
         }
-        case "$resolved_library" in
-            "$local_prefix"/*)
-                ;;
-            *)
-                # Distribution libraries and the ELF loader are supplied by the
-                # runtime image, which is pinned to the builder's exact digest.
-                # A library is outside local build prefix and is not packaged.
-                continue
-                ;;
-        esac
-        library_name="$(basename "$library")"
         cp -L "$resolved_library" "$depends_dir/$library_name"
     done < <(awk '
         /=> \/[^ ]+/ { print $3; next }
diff --git a/scripts/validate_compose_artifacts.sh b/scripts/validate_compose_artifacts.sh
index 49c7495..4106e02 100755
--- a/scripts/validate_compose_artifacts.sh
+++ b/scripts/validate_compose_artifacts.sh
@@ -1,6 +1,10 @@
 #!/usr/bin/env bash
 set -euo pipefail
 
+script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd -P)"
+# shellcheck source=compose_artifact_core_abi.sh
+. "$script_dir/compose_artifact_core_abi.sh"
+
 services=(conversation gateway identity media message presence push relationship transmite)
 artifact_root="${1:-compose-artifacts}"
 if [[ ! -d "$artifact_root" ]]; then
@@ -47,12 +51,10 @@ for service in "${services[@]}"; do
 
     while IFS= read -r packaged_library; do
         library_name="$(basename "$packaged_library")"
-        case "$library_name" in
-            ld-linux*.so.*|ld-musl-*.so.*|libc.so.*|libm.so.*|libpthread.so.*|librt.so.*|libdl.so.*|libgcc_s.so.*|libstdc++.so.*|libanl.so.*|libBrokenLocale.so.*|libcrypt.so.*|libnss_*.so.*|libresolv.so.*|libutil.so.*)
-                echo "packaged runtime loader or system ABI library for $binary: $packaged_library" >&2
-                exit 1
-                ;;
-        esac
+        if is_core_system_abi "$library_name"; then
+            echo "packaged runtime loader or system ABI library for $binary: $packaged_library" >&2
+            exit 1
+        fi
     done < <(find "$depends_dir" -mindepth 1 -maxdepth 1 \( -type f -o -type l \) -print | LC_ALL=C sort)
 
     ldd_output="$(env -i PATH=/usr/bin:/bin LD_LIBRARY_PATH="$depends_dir" "$ldd_command" "$binary" 2>&1)" || {
@@ -68,16 +70,6 @@ for service in "${services[@]}"; do
     while IFS=$'\t' read -r entry_kind library; do
         [[ -n "$library" ]] || continue
         library_name="$(basename "$library")"
-        if [[ "$entry_kind" == "loader" ]]; then
-            case "$library_name" in
-                ld-linux*.so.*|ld-musl-*.so.*) ;;
-                *)
-                    echo "unexpected system library outside packaged closure for $binary: $library" >&2
-                    exit 1
-                    ;;
-            esac
-        fi
-
         if [[ ! -f "$library" ]]; then
             echo "shared library is outside packaged closure for $binary: $library" >&2
             exit 1
@@ -85,10 +77,11 @@ for service in "${services[@]}"; do
         resolved_library="$(realpath "$library")"
         case "$resolved_library" in
             "$depends_dir_real"/*) ;;
-            /lib/*|/lib64/*|/usr/lib/*) ;;
             *)
-                echo "shared library is outside packaged closure for $binary: $library" >&2
-                exit 1
+                if ! is_core_system_abi "$library_name" || ! is_system_library_path "$resolved_library"; then
+                    echo "non-core library is outside packaged closure for $binary: $library" >&2
+                    exit 1
+                fi
                 ;;
         esac
     done < <(awk '
diff --git a/tests/pkg/contracts/artifacts_test.go b/tests/pkg/contracts/artifacts_test.go
index 8c21ff3..4d969f0 100644
--- a/tests/pkg/contracts/artifacts_test.go
+++ b/tests/pkg/contracts/artifacts_test.go
@@ -54,6 +54,15 @@ func TestComposeArtifactBuilderIsLocked(t *testing.T) {
 	}
 }
 
+func TestMediaUsesDiscoveredJsonCppTarget(t *testing.T) {
+	root := repositoryRoot(t)
+	cmake := readContractFile(t, root, "media/CMakeLists.txt")
+	require.Contains(t, cmake, "find_package(jsoncpp CONFIG REQUIRED)")
+	require.Contains(t, cmake, "jsoncpp_lib")
+	require.NotContains(t, cmake, "/usr/local/lib/libjsoncpp.so.19")
+	require.NotRegexp(t, regexp.MustCompile(`/usr/(local/)?lib[^\s)]*libjsoncpp`), cmake)
+}
+
 func TestComposeArtifactPackagerContract(t *testing.T) {
 	root := repositoryRoot(t)
 	scriptPath := filepath.Join(root, "scripts/package_compose_artifacts.sh")
@@ -66,8 +75,8 @@ func TestComposeArtifactPackagerContract(t *testing.T) {
 	require.Contains(t, script, `"$build_root/$service/${service}_server"`)
 	require.Contains(t, script, `[[ -x "$binary" ]]`)
 	require.Contains(t, script, "not found")
-	require.Contains(t, script, `local_prefix="${LOCAL_PREFIX:-/usr/local}"`)
-	require.Contains(t, script, "library is outside local build prefix")
+	require.Contains(t, script, "compose_artifact_core_abi.sh")
+	require.Contains(t, script, "is_core_system_abi")
 	require.Contains(t, script, "sha256sum")
 	require.Contains(t, script, "sort -z")
 
@@ -115,6 +124,7 @@ func TestComposeArtifactValidatorContract(t *testing.T) {
 	require.Contains(t, script, "depends_dir_real")
 	require.Contains(t, script, "resolved_library")
 	require.Contains(t, script, "packaged runtime loader or system ABI library")
+	require.Contains(t, script, "compose_artifact_core_abi.sh")
 }
 
 func TestComposeArtifactScriptsEndToEnd(t *testing.T) {
@@ -136,7 +146,7 @@ func TestComposeArtifactScriptsEndToEnd(t *testing.T) {
 		require.NoError(t, err, "%s", output)
 	})
 
-	t.Run("does not package runtime loader or distribution libraries", func(t *testing.T) {
+	t.Run("packages non-core distribution libraries but not core ABI", func(t *testing.T) {
 		fixture := newArtifactFixture(t, root)
 		fixture.lddPath = fixture.writeLDDWithSystemRuntime(t, filepath.Join(fixture.tools, "ldd-system"))
 		fixture.packageArtifacts(t)
@@ -144,12 +154,26 @@ func TestComposeArtifactScriptsEndToEnd(t *testing.T) {
 		for _, service := range composeArtifactServices {
 			depends := filepath.Join(fixture.artifactRoot, service, "depends")
 			require.FileExists(t, filepath.Join(depends, "libfixture.so"))
+			require.FileExists(t, filepath.Join(depends, "libprotobuf.so.32"))
 			require.NoFileExists(t, filepath.Join(depends, "libc.so.6"))
 			require.NoFileExists(t, filepath.Join(depends, "libstdc++.so.6"))
 			require.NoFileExists(t, filepath.Join(depends, "ld-linux-x86-64.so.2"))
 		}
 	})
 
+	t.Run("rejects non-core system library fallback", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		fixture.packageArtifacts(t)
+		hostLibrary := filepath.Join(fixture.temp, "system", "usr", "lib", "libprotobuf.so.32")
+		require.NoError(t, os.MkdirAll(filepath.Dir(hostLibrary), 0o755))
+		require.NoError(t, os.WriteFile(hostLibrary, []byte("host protobuf\n"), 0o644))
+		fallbackLDD := fixture.writeLDD(t, filepath.Join(fixture.tools, "ldd-protobuf-host"), hostLibrary)
+
+		output, err := fixture.validate(t, fallbackLDD)
+		require.Error(t, err)
+		require.Contains(t, output, "non-core library is outside packaged closure")
+	})
+
 	t.Run("rejects injected packaged system ABI library", func(t *testing.T) {
 		fixture := newArtifactFixture(t, root)
 		fixture.packageArtifacts(t)
@@ -341,9 +365,13 @@ echo "libfixture.so => $library (0x1)"
 
 func (fixture artifactFixture) writeLDDWithSystemRuntime(t *testing.T, path string) string {
 	t.Helper()
+	protobuf := filepath.Join(fixture.temp, "system", "usr", "lib", "libprotobuf.so.32")
+	require.NoError(t, os.MkdirAll(filepath.Dir(protobuf), 0o755))
+	require.NoError(t, os.WriteFile(protobuf, []byte("protobuf fixture\n"), 0o644))
 	script := `#!/bin/sh
 set -eu
 echo "libfixture.so => ${LD_LIBRARY_PATH:-` + fixture.temp + `}/libfixture.so (0x1)"
+echo "libprotobuf.so.32 => ` + protobuf + ` (0x2)"
 echo "libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x2)"
 echo "libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x3)"
 echo "/lib64/ld-linux-x86-64.so.2 (0x4)"
@@ -355,7 +383,7 @@ echo "/lib64/ld-linux-x86-64.so.2 (0x4)"
 func (fixture artifactFixture) packageArtifacts(t *testing.T) {
 	t.Helper()
 	command := exec.Command("bash", filepath.Join(fixture.root, "scripts/package_compose_artifacts.sh"), fixture.buildRoot, fixture.artifactRoot)
-	command.Env = append(os.Environ(), "PATH="+fixture.tools+":"+os.Getenv("PATH"), "LOCAL_PREFIX="+fixture.temp)
+	command.Env = append(os.Environ(), "PATH="+fixture.tools+":"+os.Getenv("PATH"), "LDD="+fixture.lddPath)
 	output, err := command.CombinedOutput()
 	require.NoError(t, err, "%s", output)
 }
diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index a302d09..e25f269 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -153,20 +153,23 @@ func TestCIGates(t *testing.T) {
 
 	producer, ok := workflow.Jobs["service-artifacts"]
 	require.True(t, ok, "CI must build the Compose service artifacts once")
+	require.Equal(t, "github.event_name != 'push' || github.ref != 'refs/heads/main'", producer.If)
 	assertServiceArtifactProducer(t, producer)
+	require.Equal(t, "github.event_name != 'push' || github.ref != 'refs/heads/main'", workflow.Jobs["bvt"].If)
 
 	for _, consumer := range []struct {
 		job    string
 		target string
+		needs  []string
 	}{
-		{"bvt", "cd tests && make test-bvt"},
-		{"func", "cd tests && make test-func"},
-		{"reliability", "cd tests && make test-reliability"},
-		{"perf-cache", "cd tests && make test-perf-cache-gate"},
+		{"bvt", "cd tests && make test-bvt", []string{"service-artifacts"}},
+		{"func", "cd tests && make test-func", []string{"bvt", "service-artifacts"}},
+		{"reliability", "cd tests && make test-reliability", []string{"service-artifacts"}},
+		{"perf-cache", "cd tests && make test-perf-cache-gate", []string{"service-artifacts"}},
 	} {
 		job, exists := workflow.Jobs[consumer.job]
 		require.True(t, exists, "%s must be a dedicated clean-runner consumer", consumer.job)
-		require.Equal(t, "service-artifacts", job.Needs)
+		require.ElementsMatch(t, consumer.needs, workflowNeeds(job.Needs))
 		assertFullStackGateJob(t, job, consumer.target)
 		assertInvalidGateJobsRejected(t, job, consumer.target)
 	}
@@ -376,6 +379,9 @@ func assertInvalidGateJobsRejected(t *testing.T, valid workflowJob, target strin
 	require.NotEqual(t, -1, teardown)
 
 	for name, mutate := range map[string]func(*workflowJob){
+		"producer dependency missing": func(job *workflowJob) {
+			job.Needs = nil
+		},
 		"Go setup allowed to fail": func(job *workflowJob) {
 			job.Steps[setupGo].ContinueOnError = true
 		},
@@ -487,8 +493,12 @@ func isForbiddenMakeTarget(run, target string) bool {
 
 func validateFullStackGateJob(job workflowJob, target string) error {
 	const install = "sudo apt-get update\nsudo apt-get install -y protobuf-compiler netcat-openbsd"
-	if job.Needs != "service-artifacts" {
-		return fmt.Errorf("gate job must depend on service-artifacts")
+	requiredNeeds := []string{"service-artifacts"}
+	if target == "cd tests && make test-func" {
+		requiredNeeds = append(requiredNeeds, "bvt")
+	}
+	if !sameStringSet(workflowNeeds(job.Needs), requiredNeeds) {
+		return fmt.Errorf("gate job has incorrect producer dependencies")
 	}
 	for _, step := range job.Steps {
 		if isForbiddenMakeTarget(step.Run, "test-perf-cache") {
@@ -556,6 +566,44 @@ func validateFullStackGateJob(job workflowJob, target string) error {
 	return nil
 }
 
+func workflowNeeds(raw any) []string {
+	switch value := raw.(type) {
+	case string:
+		return []string{value}
+	case []any:
+		needs := make([]string, 0, len(value))
+		for _, item := range value {
+			need, ok := item.(string)
+			if !ok {
+				return nil
+			}
+			needs = append(needs, need)
+		}
+		return needs
+	default:
+		return nil
+	}
+}
+
+func sameStringSet(actual, expected []string) bool {
+	if len(actual) != len(expected) {
+		return false
+	}
+	seen := make(map[string]bool, len(actual))
+	for _, value := range actual {
+		if seen[value] {
+			return false
+		}
+		seen[value] = true
+	}
+	for _, value := range expected {
+		if !seen[value] {
+			return false
+		}
+	}
+	return true
+}
+
 func continueOnErrorEnabled(value any) bool {
 	switch value := value.(type) {
 	case nil:

From ab77f19d647b01239e186eb0253fab2413e2fb7e Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Fri, 17 Jul 2026 11:21:48 +0800
Subject: [PATCH 45/47] fix(push): report conversation sequence watermarks

---
 proto/push/notify.proto              |  4 +++-
 push/source/push_server.h            | 14 +++++++++-----
 tests/func/cache_test.go             |  2 +-
 tests/pkg/contracts/ci_gates_test.go | 16 ++++++++++++++++
 4 files changed, 29 insertions(+), 7 deletions(-)

diff --git a/proto/push/notify.proto b/proto/push/notify.proto
index dfa0cc0..9eb9a19 100644
--- a/proto/push/notify.proto
+++ b/proto/push/notify.proto
@@ -34,13 +34,15 @@ message NotifyClientAuth {
 }
 
 // Client ACKs a delivered push item. user_seq identifies the per-device
-// unacked entry; message_id is the only source used to advance conversation ACK.
+// unacked entry and message_id identifies the delivered item. seq_id is an
+// optional conversation watermark; non-message pushes leave it as zero.
 message NotifyMsgPushAck {
     string user_id = 1;
     string device_id = 2;
     int64 message_id = 3;
     uint64 user_seq = 4;
     string conversation_id = 5;
+    uint64 seq_id = 6;
 }
 
 message NotifyHeartbeat {
diff --git a/push/source/push_server.h b/push/source/push_server.h
index f6066fd..fd2ab17 100644
--- a/push/source/push_server.h
+++ b/push/source/push_server.h
@@ -388,7 +388,7 @@ class PushServiceImpl : public PushService
             const auto &ack = notify.msg_push_ack();
             if (!is_valid_push_ack_ids(ack.user_seq(), ack.message_id()) ||
                 ack.user_id().empty() ||
-                ack.conversation_id().empty() || ack.device_id().empty()) {
+                ack.device_id().empty()) {
                 LOG_WARN("MSG_PUSH_ACK: invalid fields uid={} did={} seq={} message_id={}",
                          ack.user_id(), ack.device_id(), ack.user_seq(), ack.message_id());
                 return;
@@ -406,7 +406,11 @@ class PushServiceImpl : public PushService
             }
             if (_unacked) _unacked->ack(ack.user_id(), ack.device_id(), ack.user_seq());
 
-            // 异步上报 UpdateReadAck（无入站 RPC context，需手动设置 auth metadata）
+            // Non-message pushes still ACK their Unacked entry but carry no
+            // conversation read watermark.
+            if (!(ack.seq_id() > 0 && !ack.conversation_id().empty())) return;
+
+            // 异步上报会话 seq_id 水位（无入站 RPC context，需手动设置 auth metadata）
             auto channel = _mm_channels->choose(_message_service_name);
             if (!channel) {
                 LOG_WARN("UpdateReadAck: message service 不可达 uid={}", ack.user_id());
@@ -418,7 +422,7 @@ class PushServiceImpl : public PushService
                 chatnow::message::UpdateReadAckRsp>();
             closure->req.set_request_id(ack.user_id());
             closure->req.set_conversation_id(ack.conversation_id());
-            closure->req.set_message_id(static_cast<uint64_t>(ack.message_id()));
+            closure->req.set_seq_id(ack.seq_id());
             // 手动设置 auth metadata：WS handler 无入站 RPC context，需自行构造 RpcMetadata
             ::chatnow::rpc::RpcMetadata meta;
             meta.set_user_id(conn_uid);
@@ -427,10 +431,10 @@ class PushServiceImpl : public PushService
             std::string data;
             meta.SerializeToString(&data);
             closure->cntl.request_attachment().append(data);
-            closure->on_done = [uid = ack.user_id(), mid = ack.message_id()]
+            closure->on_done = [uid = ack.user_id(), seq = ack.seq_id()]
                 (brpc::Controller *c, const chatnow::message::UpdateReadAckRsp &r) {
                 if (c->Failed()) {
-                    LOG_WARN("UpdateReadAck RPC 失败 uid={} message_id={}: {}", uid, mid, c->ErrorText());
+                    LOG_WARN("UpdateReadAck RPC 失败 uid={} seq_id={}: {}", uid, seq, c->ErrorText());
                 }
             };
             stub.UpdateReadAck(&closure->cntl, &closure->req, &closure->rsp, closure);
diff --git a/tests/func/cache_test.go b/tests/func/cache_test.go
index dc7488d..e5195c9 100644
--- a/tests/func/cache_test.go
+++ b/tests/func/cache_test.go
@@ -401,7 +401,7 @@ func TestFN_CA_UnackedSameUserSeqLatestPayloadAndAck(t *testing.T) {
 		NotifyType: push.NotifyType_MSG_PUSH_ACK,
 		NotifyRemarks: &push.NotifyMessage_MsgPushAck{MsgPushAck: &push.NotifyMsgPushAck{
 			UserId: recipient.UserID, DeviceId: deviceID, MessageId: 1,
-			UserSeq: userSeq, ConversationId: "unacked-contract",
+			UserSeq: userSeq, ConversationId: "unacked-contract", SeqId: 0, // typing push has no conversation watermark
 		}},
 	})
 	require.NoError(t, err)
diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index e25f269..0446fac 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -105,6 +105,22 @@ func TestReadAckUsesConversationSequenceWatermark(t *testing.T) {
 	require.Contains(t, updateReadAck, "update_last_ack_seq(")
 	require.NotContains(t, updateReadAck, "req->message_id()")
 	require.NotContains(t, updateReadAck, "select_by_id(")
+
+	pushProto, err := os.ReadFile(filepath.Join(root, "proto/push/notify.proto"))
+	require.NoError(t, err)
+	require.Contains(t, string(pushProto), "uint64 seq_id = 6;")
+
+	pushServer, err := os.ReadFile(filepath.Join(root, "push/source/push_server.h"))
+	require.NoError(t, err)
+	clientNotify := string(pushServer)
+	start = strings.Index(clientNotify, "void onClientNotify(")
+	require.GreaterOrEqual(t, start, 0)
+	end = strings.Index(clientNotify[start:], "\n    void shutdown_cleanup()")
+	require.Greater(t, end, 0)
+	clientNotify = clientNotify[start : start+end]
+	require.Contains(t, clientNotify, "closure->req.set_seq_id(ack.seq_id())")
+	require.NotContains(t, clientNotify, "closure->req.set_message_id(")
+	require.Contains(t, clientNotify, "ack.seq_id() > 0 && !ack.conversation_id().empty()")
 }
 
 type workflowContract struct {

From fe2e98f60ef41153d1cf42f30cf98468ec6012d2 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Fri, 17 Jul 2026 11:25:13 +0800
Subject: [PATCH 46/47] fix(ci): reject artifact library basename collisions

---
 proto/message/message_service.proto   |  2 +-
 push/source/push_server.h             |  2 +-
 scripts/package_compose_artifacts.sh  | 10 ++++++++-
 tests/func/message_test.go            |  6 +++---
 tests/pkg/contracts/artifacts_test.go | 29 +++++++++++++++++++++++++++
 tests/pkg/contracts/ci_gates_test.go  |  7 +++++++
 6 files changed, 50 insertions(+), 6 deletions(-)

diff --git a/proto/message/message_service.proto b/proto/message/message_service.proto
index 0c8024e..b017ef2 100644
--- a/proto/message/message_service.proto
+++ b/proto/message/message_service.proto
@@ -150,7 +150,7 @@ message SelectByClientMsgIdRsp {
 message UpdateReadAckReq {
     string request_id = 1;
     string conversation_id = 2;
-    // 会话内单调递增的已读水位线；服务端只允许向前推进。
+    // 会话内单调递增的送达确认水位；服务端只允许向前推进。
     uint64 seq_id = 3;
 }
 message UpdateReadAckRsp { chatnow.common.ResponseHeader header = 1; }
diff --git a/push/source/push_server.h b/push/source/push_server.h
index fd2ab17..7c46c12 100644
--- a/push/source/push_server.h
+++ b/push/source/push_server.h
@@ -407,7 +407,7 @@ class PushServiceImpl : public PushService
             if (_unacked) _unacked->ack(ack.user_id(), ack.device_id(), ack.user_seq());
 
             // Non-message pushes still ACK their Unacked entry but carry no
-            // conversation read watermark.
+            // conversation delivery ACK watermark.
             if (!(ack.seq_id() > 0 && !ack.conversation_id().empty())) return;
 
             // 异步上报会话 seq_id 水位（无入站 RPC context，需手动设置 auth metadata）
diff --git a/scripts/package_compose_artifacts.sh b/scripts/package_compose_artifacts.sh
index 9185a23..8681c45 100755
--- a/scripts/package_compose_artifacts.sh
+++ b/scripts/package_compose_artifacts.sh
@@ -50,7 +50,15 @@ for service in "${services[@]}"; do
             echo "unable to resolve shared library for $binary: $library" >&2
             exit 1
         }
-        cp -L "$resolved_library" "$depends_dir/$library_name"
+        destination="$depends_dir/$library_name"
+        if [[ -e "$destination" ]]; then
+            if ! cmp -s "$resolved_library" "$destination"; then
+                echo "conflicting shared libraries share artifact basename for $binary: $library_name" >&2
+                exit 1
+            fi
+            continue
+        fi
+        cp -L "$resolved_library" "$destination"
     done < <(awk '
         /=> \/[^ ]+/ { print $3; next }
         /^[[:space:]]*\/[^ ]+/ { print $1 }
diff --git a/tests/func/message_test.go b/tests/func/message_test.go
index 58fb725..de0ec4f 100644
--- a/tests/func/message_test.go
+++ b/tests/func/message_test.go
@@ -444,7 +444,7 @@ func TestFN_MS_SelectByClientMsgId_NotFound(t *testing.T) {
 	assert.Nil(t, rsp.Message, "不存在的 client_msg_id 应返回 nil message")
 }
 
-// FN-MS | P0 | UpdateReadAck advances the conversation read watermark.
+// FN-MS | P0 | UpdateReadAck advances the conversation delivery ACK watermark.
 func TestFN_MS_UpdateReadAck_Success(t *testing.T) {
 	alice, bob, convID := fixture.MakeFriends(t, HTTP)
 	messageID, seqID := fixture.SendTextMessage(t, alice, convID, "ack-test-msg")
@@ -485,7 +485,7 @@ func TestFN_MS_UpdateReadAck_Idempotent(t *testing.T) {
 	ackRsp := &msg.UpdateReadAckRsp{}
 	require.NoError(t, bob.DoAuth("/service/message/update_read_ack", ackReq, ackRsp))
 	require.True(t, ackRsp.GetHeader().GetSuccess(),
-		"newer read watermark failed: %s", ackRsp.GetHeader().GetErrorMessage())
+		"newer delivery ACK watermark failed: %s", ackRsp.GetHeader().GetErrorMessage())
 
 	// 再 ACK 较早消息，last_ack_seq 不应回退。
 	ackReq2 := &msg.UpdateReadAckReq{
@@ -496,7 +496,7 @@ func TestFN_MS_UpdateReadAck_Idempotent(t *testing.T) {
 	ackRsp2 := &msg.UpdateReadAckRsp{}
 	require.NoError(t, bob.DoAuth("/service/message/update_read_ack", ackReq2, ackRsp2))
 	require.True(t, ackRsp2.GetHeader().GetSuccess(),
-		"backward read watermark must be idempotent: %s", ackRsp2.GetHeader().GetErrorMessage())
+		"backward delivery ACK watermark must be idempotent: %s", ackRsp2.GetHeader().GetErrorMessage())
 
 	// 直查 DB 验证 last_ack_seq 仍为 seq2。
 	verifier.LastAckSeq(t, bob.UserID, convID, seq2)
diff --git a/tests/pkg/contracts/artifacts_test.go b/tests/pkg/contracts/artifacts_test.go
index 4d969f0..2256513 100644
--- a/tests/pkg/contracts/artifacts_test.go
+++ b/tests/pkg/contracts/artifacts_test.go
@@ -1,6 +1,7 @@
 package contracts
 
 import (
+	"fmt"
 	"os"
 	"os/exec"
 	"path/filepath"
@@ -107,6 +108,23 @@ func TestComposeArtifactPackagerContract(t *testing.T) {
 		require.Error(t, err)
 		require.Contains(t, string(output), "unresolved shared library")
 	})
+
+	t.Run("rejects conflicting libraries with the same artifact basename", func(t *testing.T) {
+		fixture := newArtifactFixture(t, root)
+		first := filepath.Join(fixture.temp, "first", "libcollision.so")
+		second := filepath.Join(fixture.temp, "second", "libcollision.so")
+		require.NoError(t, os.MkdirAll(filepath.Dir(first), 0o755))
+		require.NoError(t, os.MkdirAll(filepath.Dir(second), 0o755))
+		require.NoError(t, os.WriteFile(first, []byte("first\n"), 0o644))
+		require.NoError(t, os.WriteFile(second, []byte("second\n"), 0o644))
+		fixture.lddPath = fixture.writeLDDLibraries(t, filepath.Join(fixture.tools, "ldd-collision"), first, second)
+
+		command := exec.Command("bash", scriptPath, fixture.buildRoot, fixture.artifactRoot)
+		command.Env = append(os.Environ(), "LDD="+fixture.lddPath, "PATH="+fixture.tools+":"+os.Getenv("PATH"))
+		output, err := command.CombinedOutput()
+		require.Error(t, err)
+		require.Contains(t, string(output), "conflicting shared libraries share artifact basename")
+	})
 }
 
 func TestComposeArtifactValidatorContract(t *testing.T) {
@@ -380,6 +398,17 @@ echo "/lib64/ld-linux-x86-64.so.2 (0x4)"
 	return path
 }
 
+func (fixture artifactFixture) writeLDDLibraries(t *testing.T, path string, libraries ...string) string {
+	t.Helper()
+	var script strings.Builder
+	script.WriteString("#!/bin/sh\nset -eu\n")
+	for _, library := range libraries {
+		fmt.Fprintf(&script, "echo 'libfixture.so => %s (0x1)'\n", library)
+	}
+	require.NoError(t, os.WriteFile(path, []byte(script.String()), 0o755))
+	return path
+}
+
 func (fixture artifactFixture) packageArtifacts(t *testing.T) {
 	t.Helper()
 	command := exec.Command("bash", filepath.Join(fixture.root, "scripts/package_compose_artifacts.sh"), fixture.buildRoot, fixture.artifactRoot)
diff --git a/tests/pkg/contracts/ci_gates_test.go b/tests/pkg/contracts/ci_gates_test.go
index 0446fac..d303fbd 100644
--- a/tests/pkg/contracts/ci_gates_test.go
+++ b/tests/pkg/contracts/ci_gates_test.go
@@ -92,6 +92,12 @@ func TestReadAckUsesConversationSequenceWatermark(t *testing.T) {
 	require.NoError(t, err)
 	require.Contains(t, string(serviceProto), "uint64 seq_id = 3;")
 	require.NotContains(t, string(serviceProto), "uint64 message_id = 3;")
+	require.Contains(t, string(serviceProto), "送达确认水位")
+	require.NotContains(t, string(serviceProto), "已读水位线")
+
+	messageTests, err := os.ReadFile(filepath.Join(root, "tests/func/message_test.go"))
+	require.NoError(t, err)
+	require.NotContains(t, string(messageTests), "read watermark")
 
 	server, err := os.ReadFile(filepath.Join(root, "message/source/message_server.h"))
 	require.NoError(t, err)
@@ -121,6 +127,7 @@ func TestReadAckUsesConversationSequenceWatermark(t *testing.T) {
 	require.Contains(t, clientNotify, "closure->req.set_seq_id(ack.seq_id())")
 	require.NotContains(t, clientNotify, "closure->req.set_message_id(")
 	require.Contains(t, clientNotify, "ack.seq_id() > 0 && !ack.conversation_id().empty()")
+	require.NotContains(t, clientNotify, "conversation read watermark")
 }
 
 type workflowContract struct {

From f147533f8db5cdd0edab90dead3ecf486071d0b7 Mon Sep 17 00:00:00 2001
From: ULookup <CHBulookup@outlook.com>
Date: Fri, 17 Jul 2026 11:29:42 +0800
Subject: [PATCH 47/47] docs(test): distinguish delivery and read watermarks

---
 tests/func/consistency_test.go | 2 +-
 tests/func/scenarios_test.go   | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/tests/func/consistency_test.go b/tests/func/consistency_test.go
index 1094581..a28dc4f 100644
--- a/tests/func/consistency_test.go
+++ b/tests/func/consistency_test.go
@@ -73,7 +73,7 @@ func TestFN_DC_UnreadCount(t *testing.T) {
 	dbV.UnreadCount(t, bob.UserID, convID, 3)
 
 	// bob sync 消息后，HTTP 响应也应显示 unread=3
-	// （sync 不会清未读，需要 UpdateReadAck 才清）
+	// （sync 与送达 ACK 都不会清未读；用户已读需调用 Conversation.MarkRead）
 	// 这里只验证 DB 一致性，不测 HTTP（HTTP 测试在 FN-MS 中覆盖）
 }
 
diff --git a/tests/func/scenarios_test.go b/tests/func/scenarios_test.go
index 2ff5f08..63823af 100644
--- a/tests/func/scenarios_test.go
+++ b/tests/func/scenarios_test.go
@@ -627,7 +627,7 @@ func TestScenario_UnreadCountConsistency(t *testing.T) {
 	require.NoError(t, b.DoAuth("/service/conversation/list", listReq, listRsp2))
 	for _, c := range listRsp2.Conversations {
 		if c.ConversationId == convID {
-			assert.Equal(t, uint64(0), c.Self.UnreadCount, "read ack 后未读数应清零")
+			assert.Equal(t, uint64(0), c.Self.UnreadCount, "MarkRead 后未读数应清零")
 		}
 	}