Skip to content

Commit e22b318

Browse files
fix(lgtm-datamesh): smoke-kafka polls a dead tunnel under the KEDA scaler
Same defect 764aa3f fixed in smoke-avro and smoke-notifications, missed in smoke-kafka: it polls notification-service /received through a pinned port-forward with a ~60s window. With the bootstrap-applied ScaledObject, helm's replicas=1 is reconciled to 0 within KEDA's 5s poll, the tested event wakes a NEW pod, and the smoke polls the dead tunnel while '|| echo []' masks every failure as 'not consumed yet'. It passed two full-suite runs on lucky timing and failed the third (shuffled order). Apply the same re-attach-on-curl-failure and widen the window to ~180s. Verified live with the scaler active: passes after the patch. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 4bfcd61 commit e22b318

1 file changed

Lines changed: 15 additions & 3 deletions

File tree

‎examples/lgtm-datamesh/demos/smoke-kafka.sh‎

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -117,16 +117,28 @@ printf ' order id=%s\n' "$ORDER_ID"
117117

118118
# ── 7. poll notification /received for the event ──────────────────────────────
119119
step "Polling notification-service /received for the order.placed event"
120+
# ~180s window, not ~60s: when the bootstrap-applied KEDA ScaledObject has
121+
# notification-service scaled to zero, this very event is what wakes it —
122+
# KEDA's kafka lag poll + pod start + consumer-group join must fit here.
120123
seen=0
121-
for i in $(seq 1 30); do
122-
RECV="$(curl -fsS "http://127.0.0.1:${LOCAL_NOTIF}/received" 2>/dev/null || echo '[]')"
124+
for i in $(seq 1 90); do
125+
if ! RECV="$(curl -fsS "http://127.0.0.1:${LOCAL_NOTIF}/received" 2>/dev/null)"; then
126+
# A port-forward pins the pod it attached to at start. With the KEDA
127+
# ScaledObject applied, that pod can be replaced or scaled away at any
128+
# moment (helm sets replicas=1, KEDA reconciles to 0, the tested event
129+
# wakes a NEW pod) — leaving us polling a dead tunnel forever. Re-attach
130+
# to the Service and treat this attempt as "not yet".
131+
kill "$PF_N" 2>/dev/null
132+
kubectl port-forward -n "$NS" service/notification-service "${LOCAL_NOTIF}:80" >/dev/null 2>&1 & PF_N=$!
133+
RECV='[]'
134+
fi
123135
if printf '%s' "$RECV" | python3 -c "import sys,json; d=json.load(sys.stdin); sys.exit(0 if any(e.get('order_id')=='$ORDER_ID' and e.get('event_type')=='order.placed' for e in d) else 1)" 2>/dev/null; then
124136
printf ' ✓ notification consumed order.placed for %s (after ~%ds)\n' "$ORDER_ID" "$((i*2))"
125137
seen=1; break
126138
fi
127139
sleep 2
128140
done
129-
(( seen )) || fail "notification-service did not consume the order.placed event within ~60s"
141+
(( seen )) || fail "notification-service did not consume the order.placed event within ~180s"
130142

131143
printf '\n✓ SUCCESS — async spine verified (order.placed: order-service → Kafka → notification-service)\n'
132144

0 commit comments

Comments
 (0)