fix(backend): let operator-set RESEND_API_KEY survive ArgoCD sync#154
Merged
Conversation
Preview + prod email (org invites, password resets) silently failed: the backend logged "API key is invalid" from Resend and returned 201/200 anyway (send is best-effort, non-fatal), so the UI showed success but nothing was delivered. Root cause: the chart rendered RESEND_API_KEY unconditionally from .Values.secrets.resendApiKey (default ""), so ArgoCD owned /data/RESEND_API_KEY and rewrote it to empty on every sync. The Railway->GKE migration never carried a key into the chart, and any out-of-band operator-set value would be reverted on the next sync. An empty key is what Resend rejects as invalid. (Confirmed both trakrf-preview and trakrf-prod secrets held an empty RESEND_API_KEY.) Fix mirrors the JWT_SECRET TRA-860 pattern: - secret.yaml omits RESEND_API_KEY when empty, so ArgoCD never manages the key and an out-of-band value persists across all sync paths. - trakrf-backend Application adds /data/RESEND_API_KEY to the ignoreDifferences jsonPointers (belt-and-suspenders, per-env so prod inherits the carve-out at cutover). The real key is still set out-of-band (operator kubectl today; ESO + GCP Secret Manager later, TRA-375). Verified: helm template omits RESEND_API_KEY when unset and includes it when set; root app renders both /data/JWT_SECRET + /data/RESEND_API_KEY pointers per env. (argocd/root helm lint failure is pre-existing on main — multi-doc separator quirk — not from this change.) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Symptom
Preview (and at-risk prod) email — org invitations + password resets — silently fail. The backend logs
[ERROR]: API key is invalidfrom Resend on every send, but the HTTP request still returns 201/200 (email send is best-effort / logged as a non-fatal warning), so the UI shows success while nothing is delivered. Flagged via the live preview repro in org "Organized Chaos".Root cause (config, not code)
RESEND_API_KEYis empty in bothtrakrf-previewandtrakrf-prodbackend Secrets (verified live, len 0). Resend rejects an empty key as "invalid".The chart rendered
RESEND_API_KEYunconditionally from.Values.secrets.resendApiKey(default""), so ArgoCD owned/data/RESEND_API_KEYand rewrote it to empty on every sync. The Railway→GKE migration never carried a key into the chart values, and — unlikeJWT_SECRET— there was no omit-when-empty +ignoreDifferencestreatment, so even an out-of-band operator-set value would be clobbered on the next sync.This is not an ESO/GSM miss — Secret Manager isn't wired (or even enabled) on this project; the key is a plain chart value.
Fix (mirrors the
JWT_SECRETTRA-860 pattern)helm/trakrf-backend/templates/secret.yaml— omitRESEND_API_KEYwhen empty, so ArgoCD never manages the key and an out-of-band value persists across all sync paths.argocd/root/templates/trakrf-backend.yaml— add/data/RESEND_API_KEYto theignoreDifferencesjsonPointers (belt-and-suspenders; per-env, so prod inherits the carve-out at cutover).The real key stays set out-of-band (operator
kubectltoday; ESO + GCP Secret Manager later — TRA-375).Follow-up (not in this PR)
Inject a valid
re_...key into the preview + prod Secrets out-of-band once Mike provides it. Preview can go as soon as the key is in hand; prod apply timing to be coordinated with Mike (it's a prod-secret change). platform-live-reads will confirm with a test invite on preview after the roll.Verification
helm templateomitsRESEND_API_KEYwhen unset, includes it when set (re_testkey123)./data/JWT_SECRET+/data/RESEND_API_KEYpointers per env (preview + prod).helm lint helm/trakrf-backendclean. (argocd/rootlint failure is pre-existing on main — multi-doc---separator quirk — not from this change.)🤖 Generated with Claude Code