Cache the routing decision, not the colour it resolves to

[partouf]

Follow-up to compiler-explorer/infra#2372, which fixed the deploy side of this. This is the remaining half.

routingCache stored the fully resolved queue URL, active colour already substituted, in a map with no expiry:

const queueName = item.queueName?.S;           // stable
const activeColor = await getActiveColor();    // volatile - changes every deploy
routingCache.set(cacheKey, {type: 'queue', target: buildQueueUrl(queueName, activeColor), ...});

The colour changes on every deploy, so an entry that was correct when written silently became wrong. And because routingCache.get() returns before getActiveColor() is ever reached, the 30s TTL on activeColorCache can never fire for a compiler already cached — three comments describe /admin/clear-cache as an optimisation over that TTL while it is in fact the only mechanism. A push that does not arrive leaves a router on the old colour until its process restarts.

Change

Cache what the routing table said — type, plus queueName or targetUrl — and resolve the colour per request in resolveRouting(). Everything cached is now stable, so caching it forever is correct, and the existing TTL becomes reachable.

RoutingInfo and the lookupCompilerRouting() signature are unchanged, so compiler-explorer-router.ts and the test mocks don't move. Resolving per request is a Map lookup and string manipulation; no extra AWS calls within the TTL. The four near-identical result/cache blocks collapse into decisionFromItem() plus one cache write, which is most of the deleted lines.

What this does and doesn't fix

It does not remove the need for the clear: a routing-table change mid-deploy (routingType flipping, a changed targetUrl) is still only invalidated by the push, and resolving the colour later does nothing for that. What it removes is the permanence of the colour half — a router the push misses is now wrong for 30 seconds rather than until it restarts.

Tests

Three added; two fail against current main:

• a colour switch on an already-cached compiler is picked up once the TTL expires, with no new DynamoDB call — the routing row stays cached, only the colour is re-read
• within the TTL there is still no SSM call per request, so the caching this exists for is intact
• a URL-routed compiler resolves no colour at all

124 tests pass, typecheck and biome check clean.

🤖 Generated with Claude Code

[partouf]

A concrete trace of what this changes, with values taken from prod: active colour blue, two routers i-09146a0c000f4cbfc (172.30.0.37) and i-093f89bb3ba9d55b9 (172.30.1.222), and the real routing row for g141 (GCC 14.1) — routingType: queue, queueName: prod-compilation-queue.

Steady state, before a deploy

SSM /compiler-explorer/prod/active-color = blue. Someone compiles with g141, and the router that takes it does:

routingCache.get("prod#g141")                      -> miss
DynamoDB GetItem CompilerRouting, compilerId="prod#g141"
  -> { routingType: "queue", queueName: "prod-compilation-queue", environment: "prod" }
getActiveColor()                                    -> SSM -> "blue"
buildQueueUrl("prod-compilation-queue", "blue")
  templateUrl = https://sqs.us-east-1.amazonaws.com/052730242331/prod-compilation-queue-blue.fifo
  baseUrl     = https://sqs.us-east-1.amazonaws.com/052730242331/
  queueName has no -blue/-green, so append "-blue", then ".fifo"

and stores, with no expiry:

routingCache["prod#g141"] = {
  type: "queue",
  target: "https://sqs.us-east-1.amazonaws.com/052730242331/prod-compilation-queue-blue.fifo",
  environment: "prod"
}

blue is now baked into that entry. Both routers end up with their own copy.

The deploy, blue -> green

Step 4 writes green to SSM. Step 6.5 POSTs /admin/clear-cache to both routers. 172.30.0.37 returns 200. 172.30.1.222 does not answer in time — that is the whole failure, one HTTP request.

What each router does next

172.30.0.37, cache empty, next g141 request:

routingCache.get("prod#g141")  -> miss
getActiveColor()               -> SSM -> "green"
target = .../prod-compilation-queue-green.fifo     correct

172.30.1.222, cache intact:

routingCache.get("prod#g141")  -> HIT, returns immediately
target = .../prod-compilation-queue-blue.fifo      stale

Note where the hit returns in the current code: it is the first thing lookupCompilerRouting does, and getActiveColor() sits below it. So for g141 on this process, getActiveColor() is never called again — not in 30 seconds, not tomorrow. The 30s TTL on activeColorCache is real but unreachable from here.

The ALB spreads requests across both routers, so g141 is now version-correct or version-stale roughly 50/50, per request, with no error anywhere.

Two hours later

Someone runs ce --env prod blue-green cleanup: reset_asg_min_size(prod-blue, 0) then scale_asg(prod-blue, 0). Nothing is reading prod-compilation-queue-blue.fifo any more.

172.30.1.222 keeps sending there. resultWaiter.waitForResult(guid, 60) fires its timer and the user gets:

408  Compilation timeout: No response received within 60 seconds for GUID: ...

Half the g141 requests, an hour after a deploy that reported success, triggered by a cleanup command that looks unrelated.

With this PR

172.30.1.222 caches {type: "queue", queueName: "prod-compilation-queue", environment: "prod"} — no colour in it. The cache hit still short-circuits DynamoDB, but resolveRouting() then calls getActiveColor(), served from activeColorCache until its 30 seconds are up and then re-read from SSM. Same missed POST, same everything else: the router is wrong for up to 30 seconds and then corrects itself.