[INFRA-1336] kustomize: opt-in Redis Sentinel component

kustomize/README.md

@@ -14,6 +14,7 @@ Kubernetes deployment for Plane Commercial using Kustomize.
 - [Deployment Scenarios](#deployment-scenarios)
 - [Deploying with IRSA or EKS Pod Identity](#deploying-with-irsa-or-eks-pod-identity)
 - [Running workloads as non-root](#running-workloads-as-non-root)
+- [Redis Sentinel](#redis-sentinel)
 - [Configuration](#configuration)
 - [Production Considerations](#production-considerations)
 - [Security Best Practices](#security-best-practices)
@@ -111,7 +112,8 @@ Choose to deploy these locally OR provide external managed services. Each compon
 **redis** (Redis cache)
 - Adds: StatefulSet, Service
 - Patches: `plane-app-secrets` and `plane-live-secrets` with Redis URLs
-- Alternative: AWS ElastiCache, Redis Cloud, Azure Cache
+- Alternative: AWS ElastiCache, Redis Cloud, Azure Cache, or a Redis Sentinel
+  set you run yourself (see [Redis Sentinel](#redis-sentinel))
 
 **rabbitmq** (RabbitMQ message broker)
 - Adds: StatefulSet, Service, Secret
@@ -340,6 +342,7 @@ spec:
 |-----------|-------------|-------------------|
 | **postgres** | Use local PostgreSQL | Using AWS RDS, Cloud SQL, etc. |
 | **redis** | Use local Redis | Using AWS ElastiCache, Redis Cloud |
+| **redis-sentinel** | Using a Redis Sentinel set you run (instead of **redis**) | Using a single Redis endpoint |
 | **rabbitmq** | Use local RabbitMQ | Using AWS MQ, CloudAMQP |
 | **minio** | Use local S3 storage | Using AWS S3, Google Cloud Storage |
 | **opensearch** | Use local OpenSearch | Using AWS OpenSearch Service |
@@ -397,6 +400,10 @@ components:
 - Resources: 50Mi memory, 50m CPU
 - Storage: 1Gi
 
+**Redis Sentinel:** this component always runs a single node. To use a Sentinel set
+you run, leave it out and use the `redis-sentinel` component instead (see
+[Redis Sentinel](#redis-sentinel)).
+
 #### RabbitMQ Component
 
 **What it adds:**
@@ -669,6 +676,88 @@ components:
 
 No extra configuration is required. The component patches all base Deployments (api, web, space, admin, live, worker, etc.) with the same non-root security context. Ensure your container images support running as the configured user (e.g. UID 1000); Plane Commercial images are built to run as non-root.
 
+## Redis Sentinel
+
+Plane can connect to a Redis primary that a **Redis Sentinel set you run** manages.
+Plane does not deploy Sentinel; running it (quorum, failover timeouts, topology) is
+your job. This needs a Plane release with Redis Sentinel support. The Plane docs
+page "Connect to your own Redis Sentinel" lists the Sentinel requirements and what
+happens during a failover.
+
+The opt-in `redis-sentinel` component copies the settings from `secrets-vars.yaml`
+into the Secrets the Redis clients read:
+
+| Secret | Workloads |
+|--------|-----------|
+| `plane-app-secrets` | api, workers, beat, consumers, migrator, silo, flux |
+| `plane-live-secrets` | live, live-exporter |
+| `plane-silo-secrets` | silo (api and worker also load it) |
+| `plane-flux-secrets` | flux, flux-consumer (`flux` component) |
+| `plane-mcp-server-secrets` | mcp-server (`mcp-server` component) |
+
+Secrets that aren't in your overlay are skipped. pi is not covered.
+
+### Settings
+
+| Key | Meaning | Default / notes |
+|-----|---------|-----------------|
+| `REDIS_SENTINELS` | `host1:26379,host2:26379,host3:26379` | A non-empty value turns on Sentinel mode. Port defaults to 26379. |
+| `REDIS_SENTINEL_MASTER` | Sentinel master (service) name | Required in Sentinel mode; services fail at startup without it. |
+| `REDIS_SENTINEL_USERNAME` / `REDIS_SENTINEL_PASSWORD` | Auth to the sentinels | Optional. ACL usernames need Redis >= 6.2. |
+| `REDIS_USERNAME` / `REDIS_PASSWORD` | Auth to the data nodes | Optional. |
+| `REDIS_DB` | Database index | `0` |
+| `REDIS_SSL` | TLS to the data nodes | `true`/`1`/`yes`/`on` turn it on. |
+| `REDIS_SENTINEL_SSL` | TLS to the sentinels | Empty = same as `REDIS_SSL`. |
+| `REDIS_SSL_VERIFY` | Verify server certificates against the system CA bundle | Off by default (today's behaviour). The certificate must match the address the sentinels report for the primary. |
+
+### Enable it
+
+1. In `kustomization.yaml`, remove `../../components/redis` (Sentinel always means
+   an external Redis) and add the component **after** `flux`, `flux-consumer` and
+   `mcp-server`, since it only patches Secrets that are already in the build:
+
+   ```yaml
+   components:
+     # ...
+     - ../../components/flux
+     - ../../components/mcp-server
+     # ...
+     - ../../components/redis-sentinel
+   ```
+
+2. In `secrets-vars.yaml`, set `REDIS_URL: ""` and add **all ten** keys (they are
+   replacement sources, so a missing key fails the whole overlay render). Leave
+   optional ones as `""`. Each overlay's `secrets-vars.yaml.example` has the block
+   commented out:
+
+   ```yaml
+   stringData:
+     REDIS_URL: ""
+     REDIS_SENTINELS: "sentinel-0.redis.internal:26379,sentinel-1.redis.internal:26379,sentinel-2.redis.internal:26379"
+     REDIS_SENTINEL_MASTER: "mymaster"
+     REDIS_SENTINEL_USERNAME: ""
+     REDIS_SENTINEL_PASSWORD: ""
+     REDIS_USERNAME: ""
+     REDIS_PASSWORD: "<data-node-password>"
+     REDIS_DB: "0"
+     REDIS_SSL: "false"
+     REDIS_SENTINEL_SSL: ""
+     REDIS_SSL_VERIFY: "false"
+   ```
+
+3. If the overlay uses ElastiCache (`ELASTICACHE_SECRET_ARN`, mcp-server
+   `REDIS_HOST`), Sentinel mode takes precedence and the services log a warning.
+   Clear those values to silence it. The `iframely` cache settings are separate
+   and are not affected.
+
+4. Render and check before applying:
+
+   ```bash
+   kubectl kustomize overlays/my-env | grep -n "REDIS_SENTINEL"
+   ```
+
+Overlays that don't include `redis-sentinel` render exactly as before.
+
 ## Configuration
 
 ### Configuration System
@@ -770,6 +859,8 @@ These files are marked with `config.kubernetes.io/local-config: "true"` so they'
 **Configured via secrets-vars.yaml (sensitive):**
 - `DATABASE_URL` - PostgreSQL connection string
 - `REDIS_URL` - Redis connection string (main)
+- `REDIS_SENTINELS`, `REDIS_SENTINEL_*`, `REDIS_USERNAME`, `REDIS_PASSWORD`, `REDIS_DB`, `REDIS_SSL*` -
+  only with the `redis-sentinel` component; see [Redis Sentinel](#redis-sentinel)
 - `AMQP_URL` - RabbitMQ connection string
 - `OPENSEARCH_*` - OpenSearch configuration
 - `AWS_*` - S3/MinIO configuration
@@ -832,7 +923,8 @@ For production, configure HA for critical components:
 
 2. **Use Managed Services:** For true HA, use cloud-managed services:
    - AWS RDS Multi-AZ for PostgreSQL
-   - AWS ElastiCache with Redis replication
+   - AWS ElastiCache with Redis replication, or your own Redis Sentinel
+     (see [Redis Sentinel](#redis-sentinel))
    - AWS MQ with active/standby brokers
    - S3 with versioning and cross-region replication
    - AWS OpenSearch with 3+ nodes across AZs

kustomize/components/flux/secret.yaml

@@ -5,7 +5,8 @@ metadata:
   name: plane-flux-secrets
   namespace: plane
 stringData:
-  # Will be filled by redis component or overlay
+  # Will be filled by redis component or overlay. For a Redis Sentinel set,
+  # leave it empty: the redis-sentinel component adds the REDIS_SENTINEL* keys.
   REDIS_URL: ""
   # Will be filled by rabbitmq component or overlay
   AMQP_URL: ""

kustomize/components/redis-sentinel/kustomization.yaml

@@ -0,0 +1,233 @@
+apiVersion: kustomize.config.k8s.io/v1alpha1
+kind: Component
+
+# ============================================================
+# REDIS SENTINEL (customer-run, bring your own)
+# ============================================================
+# Points Plane at a Redis primary that is managed by a Redis Sentinel set the
+# customer runs. Plane does NOT deploy Sentinel; this component only copies the
+# connection settings into the Secrets the Redis consumers read:
+#
+#   plane-app-secrets         api, workers, beat, consumers, migrator, silo, flux
+#   plane-live-secrets        live, live-exporter
+#   plane-silo-secrets        silo (also loaded by api/worker after app secrets)
+#   plane-flux-secrets        flux, flux-consumer (flux component only)
+#   plane-mcp-server-secrets  mcp-server (mcp-server component only)
+#
+# A target Secret that is not part of the overlay (flux, mcp-server) is skipped.
+# pi is not covered.
+#
+# Opt-in. To use it:
+#   1. Remove ../../components/redis from the overlay (Sentinel always means an
+#      external Redis).
+#   2. Add ALL ten REDIS_* keys below to secrets-vars.yaml (see the commented
+#      block in secrets-vars.yaml.example). They are replacement sources, so a
+#      missing key fails the WHOLE overlay render. Leave optional ones "".
+#   3. Set REDIS_URL: "" in secrets-vars.yaml (and leave ELASTICACHE_SECRET_ARN
+#      empty). Sentinel mode wins over both, but the services log a warning when
+#      both are set.
+#   4. Add - ../../components/redis-sentinel to the overlay's components.
+#
+# Requires a Plane release with Redis Sentinel support: older images ignore
+# these keys and, with REDIS_URL empty, have no Redis to connect to.
+# See kustomize/README.md, "Redis Sentinel".
+# ============================================================
+#
+# Keys (all from overlay-secret-vars / secrets-vars.yaml):
+#   REDIS_SENTINELS          host1:26379,host2:26379,host3:26379; non-empty turns on Sentinel mode
+#   REDIS_SENTINEL_MASTER    Sentinel master (service) name; required
+#   REDIS_SENTINEL_USERNAME  auth to the sentinels (optional)
+#   REDIS_SENTINEL_PASSWORD  auth to the sentinels (optional)
+#   REDIS_USERNAME           auth to the data nodes (optional, ACL user)
+#   REDIS_PASSWORD           auth to the data nodes (optional)
+#   REDIS_DB                 database index
+#   REDIS_SSL                TLS to the data nodes
+#   REDIS_SENTINEL_SSL       TLS to the sentinels; empty = same as REDIS_SSL
+#   REDIS_SSL_VERIFY         verify server certificates
+
+# Declare the keys as empty strings first (component patches run before
+# component replacements). Replacing into an existing string keeps it a string;
+# letting the replacement create the key would turn "" into null and "false"
+# into a YAML boolean, which a Secret rejects. A target Secret that is not in
+# the overlay matches nothing and is skipped.
+patches:
+  - target:
+      kind: Secret
+      name: ^plane-(app|live|silo|flux|mcp-server)-secrets$
+    patch: |-
+      apiVersion: v1
+      kind: Secret
+      metadata:
+        name: placeholder
+      stringData:
+        REDIS_SENTINELS: ""
+        REDIS_SENTINEL_MASTER: ""
+        REDIS_SENTINEL_USERNAME: ""
+        REDIS_SENTINEL_PASSWORD: ""
+        REDIS_USERNAME: ""
+        REDIS_PASSWORD: ""
+        REDIS_DB: ""
+        REDIS_SSL: ""
+        REDIS_SENTINEL_SSL: ""
+        REDIS_SSL_VERIFY: ""
+
+replacements:
+  - source:
+      kind: Secret
+      name: overlay-secret-vars
+      fieldPath: stringData.REDIS_SENTINELS
+    targets:
+      - select: { kind: Secret, name: plane-app-secrets }
+        fieldPaths: [stringData.REDIS_SENTINELS]
+      - select: { kind: Secret, name: plane-live-secrets }
+        fieldPaths: [stringData.REDIS_SENTINELS]
+      - select: { kind: Secret, name: plane-silo-secrets }
+        fieldPaths: [stringData.REDIS_SENTINELS]
+      - select: { kind: Secret, name: plane-flux-secrets }
+        fieldPaths: [stringData.REDIS_SENTINELS]
+      - select: { kind: Secret, name: plane-mcp-server-secrets }
+        fieldPaths: [stringData.REDIS_SENTINELS]
+
+  - source:
+      kind: Secret
+      name: overlay-secret-vars
+      fieldPath: stringData.REDIS_SENTINEL_MASTER
+    targets:
+      - select: { kind: Secret, name: plane-app-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_MASTER]
+      - select: { kind: Secret, name: plane-live-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_MASTER]
+      - select: { kind: Secret, name: plane-silo-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_MASTER]
+      - select: { kind: Secret, name: plane-flux-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_MASTER]
+      - select: { kind: Secret, name: plane-mcp-server-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_MASTER]
+
+  - source:
+      kind: Secret
+      name: overlay-secret-vars
+      fieldPath: stringData.REDIS_SENTINEL_USERNAME
+    targets:
+      - select: { kind: Secret, name: plane-app-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_USERNAME]
+      - select: { kind: Secret, name: plane-live-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_USERNAME]
+      - select: { kind: Secret, name: plane-silo-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_USERNAME]
+      - select: { kind: Secret, name: plane-flux-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_USERNAME]
+      - select: { kind: Secret, name: plane-mcp-server-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_USERNAME]
+
+  - source:
+      kind: Secret
+      name: overlay-secret-vars
+      fieldPath: stringData.REDIS_SENTINEL_PASSWORD
+    targets:
+      - select: { kind: Secret, name: plane-app-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_PASSWORD]
+      - select: { kind: Secret, name: plane-live-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_PASSWORD]
+      - select: { kind: Secret, name: plane-silo-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_PASSWORD]
+      - select: { kind: Secret, name: plane-flux-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_PASSWORD]
+      - select: { kind: Secret, name: plane-mcp-server-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_PASSWORD]
+
+  - source:
+      kind: Secret
+      name: overlay-secret-vars
+      fieldPath: stringData.REDIS_USERNAME
+    targets:
+      - select: { kind: Secret, name: plane-app-secrets }
+        fieldPaths: [stringData.REDIS_USERNAME]
+      - select: { kind: Secret, name: plane-live-secrets }
+        fieldPaths: [stringData.REDIS_USERNAME]
+      - select: { kind: Secret, name: plane-silo-secrets }
+        fieldPaths: [stringData.REDIS_USERNAME]
+      - select: { kind: Secret, name: plane-flux-secrets }
+        fieldPaths: [stringData.REDIS_USERNAME]
+      - select: { kind: Secret, name: plane-mcp-server-secrets }
+        fieldPaths: [stringData.REDIS_USERNAME]
+
+  - source:
+      kind: Secret
+      name: overlay-secret-vars
+      fieldPath: stringData.REDIS_PASSWORD
+    targets:
+      - select: { kind: Secret, name: plane-app-secrets }
+        fieldPaths: [stringData.REDIS_PASSWORD]
+      - select: { kind: Secret, name: plane-live-secrets }
+        fieldPaths: [stringData.REDIS_PASSWORD]
+      - select: { kind: Secret, name: plane-silo-secrets }
+        fieldPaths: [stringData.REDIS_PASSWORD]
+      - select: { kind: Secret, name: plane-flux-secrets }
+        fieldPaths: [stringData.REDIS_PASSWORD]
+      - select: { kind: Secret, name: plane-mcp-server-secrets }
+        fieldPaths: [stringData.REDIS_PASSWORD]
+
+  - source:
+      kind: Secret
+      name: overlay-secret-vars
+      fieldPath: stringData.REDIS_DB
+    targets:
+      - select: { kind: Secret, name: plane-app-secrets }
+        fieldPaths: [stringData.REDIS_DB]
+      - select: { kind: Secret, name: plane-live-secrets }
+        fieldPaths: [stringData.REDIS_DB]
+      - select: { kind: Secret, name: plane-silo-secrets }
+        fieldPaths: [stringData.REDIS_DB]
+      - select: { kind: Secret, name: plane-flux-secrets }
+        fieldPaths: [stringData.REDIS_DB]
+      - select: { kind: Secret, name: plane-mcp-server-secrets }
+        fieldPaths: [stringData.REDIS_DB]
+
+  - source:
+      kind: Secret
+      name: overlay-secret-vars
+      fieldPath: stringData.REDIS_SSL
+    targets:
+      - select: { kind: Secret, name: plane-app-secrets }
+        fieldPaths: [stringData.REDIS_SSL]
+      - select: { kind: Secret, name: plane-live-secrets }
+        fieldPaths: [stringData.REDIS_SSL]
+      - select: { kind: Secret, name: plane-silo-secrets }
+        fieldPaths: [stringData.REDIS_SSL]
+      - select: { kind: Secret, name: plane-flux-secrets }
+        fieldPaths: [stringData.REDIS_SSL]
+      - select: { kind: Secret, name: plane-mcp-server-secrets }
+        fieldPaths: [stringData.REDIS_SSL]
+
+  - source:
+      kind: Secret
+      name: overlay-secret-vars
+      fieldPath: stringData.REDIS_SENTINEL_SSL
+    targets:
+      - select: { kind: Secret, name: plane-app-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_SSL]
+      - select: { kind: Secret, name: plane-live-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_SSL]
+      - select: { kind: Secret, name: plane-silo-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_SSL]
+      - select: { kind: Secret, name: plane-flux-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_SSL]
+      - select: { kind: Secret, name: plane-mcp-server-secrets }
+        fieldPaths: [stringData.REDIS_SENTINEL_SSL]
+
+  - source:
+      kind: Secret
+      name: overlay-secret-vars
+      fieldPath: stringData.REDIS_SSL_VERIFY
+    targets:
+      - select: { kind: Secret, name: plane-app-secrets }
+        fieldPaths: [stringData.REDIS_SSL_VERIFY]
+      - select: { kind: Secret, name: plane-live-secrets }
+        fieldPaths: [stringData.REDIS_SSL_VERIFY]
+      - select: { kind: Secret, name: plane-silo-secrets }
+        fieldPaths: [stringData.REDIS_SSL_VERIFY]
+      - select: { kind: Secret, name: plane-flux-secrets }
+        fieldPaths: [stringData.REDIS_SSL_VERIFY]
+      - select: { kind: Secret, name: plane-mcp-server-secrets }
+        fieldPaths: [stringData.REDIS_SSL_VERIFY]

kustomize/components/redis/kustomization.yaml

@@ -1,6 +1,10 @@
 apiVersion: kustomize.config.k8s.io/v1alpha1
 kind: Component
 
+# Single-node in-cluster Redis (Valkey). To connect to a Redis Sentinel set you
+# run instead, leave this component out and use ../redis-sentinel (see
+# kustomize/README.md, "Redis Sentinel").
+
 resources:
   - statefulset.yaml
   - service.yaml