Repository navigation
Conversation
Add a Kubernetes lab that routes a Redpanda shadow link through a Kroxylicious virtual cluster instead of reading the source cluster directly. A small custom filter presents every topic and consumer group under a "p_" prefix, so the rename is visible end to end on the shadow cluster while the source cluster keeps its real names. A healthy shadow cluster is also proof that shadowing works through a Kafka proxy. Structure follows kubernetes/shadow-linking, which this extends: - README.adoc with the lab attributes, the rpk commands inline rather than as wrapper scripts, and Limitations covering the filter's deliberate gaps - setup.sh in the same style, plus the image build and kind load steps the proxy needs - The filter's own docs converted from Markdown to AsciiDoc - resources/syslog.avsc and syslogs.json reused from the sibling lab, so the sample records stay readable and use documentation IP ranges - docs/modules symlink so Antora publishes the page - kubernetes/README.adoc entries for this lab and for the two existing labs the index had not yet picked up Two fixes carried over from the source demo: the shadow-side Schema Registry URL had a stray "b." in its hostname, and the kind cluster is now named, so `kind load docker-image` and the clean-up step can target it unambiguously. Based on https://github.com/pmw-rp/kroxy-linking-demo Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
👷 Deploy request for redpanda-labs-preview pending review.Visit the deploys page to approve it
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a Kubernetes lab that routes a Redpanda shadow link through a Kroxylicious virtual cluster instead of reading the source cluster directly.
A small custom Kroxylicious filter presents every topic and consumer group under a
p_prefix, without renaming anything on the source cluster. A topic created asbasicon the source arrives on the shadow cluster asp_basic; a consumer group created asconsumer-group-fooarrives asp_consumer-group-foo. Because the shadow cluster ends up healthy and consistent with those renamed topics and groups intact — and fails over correctly — the lab doubles as proof that shadowing works end to end through a Kafka protocol proxy.Based on https://github.com/pmw-rp/kroxy-linking-demo.
Where it lives
kubernetes/kroxylicious-shadow-linking/, next tokubernetes/shadow-linking, which this is a variant of. It deploys via kind + Helm + the Redpanda operator, so it belongs underkubernetes/rather thandocker-compose/.Conventions
README.adocat the lab root with the required header attributes (:page-layout: lab,:env-kubernetes: true,:page-categories:,:description:) and the same section shape as the sibling lab, plus aLimitationssection covering the filter's deliberate gaps.Disaster Recovery, Data Replication, Integration— all three already in use elsewhere in the repo.rpkcommands are inline in the README rather than wrapper scripts, following the simplification made tokubernetes/shadow-linkingin Add shadow linking lab for Kubernetes disaster recovery #284. The source demo's numbereddemos/*.shscripts are not carried over.setup.shfollows the sibling lab's style (set -e, progress echoes, quiet Helm), with the proxy image build andkind loadsteps added.docs/modules/kubernetes/pages/kroxylicious-shadow-linking.adocsymlink added so Antora publishes the page.test-docs.ymlwiring: the Kubernetes labs aren't covered there, since they need a cluster.All four AsciiDoc files render clean under
asciidoctor --failure-level=WARN.Points for review
resources/syslog.avscandresources/syslogs.jsonare copied fromkubernetes/shadow-linkingrather than from the source demo, which ships 1000 generated records with real-looking public IPs. The repo's version is 3 readable records using RFC 5737 documentation addresses.b.in its hostname (redpanda.shadow.b.svc.cluster.local). Andresources/kind.yamlnow names the clusterredpanda-kroxy, withkind load docker-image --nameand the clean-up step targeting it — previouslykind loadwould have silently targeted a cluster namedkind.kubernetes/README.adocwas stale, listing onlygitops-helm. This adds entries foricebergandshadow-linkingalongside the new lab, matching howdocker-compose/README.adocindexes everything. Happy to drop those two lines if you'd rather keep the diff to the new lab only.v26.2.2, which is what the demo was validated against, versusv25.3.4/v25.3.1inkubernetes/shadow-linking.Limitationssection and the filter's own README: producing through the proxy, transactional producers,DeleteGroupsand the KIP-848 consumer group protocol are all out of scope.kroxylicious/topic-prefixer/consumer-group-renaming.adocrecords the wire-level tracing behind the consumer group half, including the finding that shadow linking correlates topics by ID rather than by name.resources/reverse.yamlfrom the demo is deliberately left out: nothing references it, it isn't documented, and its comments are mislabeled.🤖 Generated with Claude Code