feat(serenity): mirror each Semrush market as a SpaceCat Site

[rainer-friederich]

1. Abstract

Every Semrush market now gets a resolvable SpaceCat Site mirroring its primary domain, linked to the owning brand via a brand_sites row tagged type='serenity'.

2. Reasoning

A Serenity brand is a domain-less shell (like its Semrush sub-workspace); the real domains live on its markets. Integrations and lookups that resolve a site by its base URL had nothing to find for these brands. Mirroring each market's domain as a Site restores base-URL resolvability and backwards compatibility without changing the brand model.

3. High-level overview of the changes

• Before: a Serenity brand had no Site, so a base-URL lookup of a market's domain resolved to nothing.
• After: ensureMarketSite idempotently ensures one Site per market domain (deliveryType other) and links it to the brand with a brand_sites row marked type='serenity'. Global sites.base_url uniqueness means at most one Site per domain; a brand whose markets span distinct domains owns several Sites.
• Wired into the sub-workspace-mode hooks only: brand creation (initial market), activation (activated markets), and market creation (that market). Best-effort and same-org guarded, so it never fails a live market. Sites/links are never auto-deleted; deleting a market leaves them in place.
• syncBrandSites now preserves type='serenity' (and NULL-type) rows so the delete-then-reinsert brand-edit rebuild no longer wipes these links, and a re-upsert cannot downgrade the marker.
• mapDbBrandToV2 excludes type='serenity' rows from urls[] and siteIds, so they stay a pure backend linkage and the Brand V2 response is unchanged.
• Per-market domain is exposed on the markets API + OpenAPI; docs/serenity.md gains a "Site mirroring" section.

4. Required information

• Spec: docs/serenity.md "Site mirroring" section (design-of-record, updated in this PR)

5. Affected / used mysticat-workspace projects

• mysticat-data-service — depends-on: requires the migration allowing brand_sites.type='serenity'; the link insert is silently rejected by the CHECK constraint until that migration is deployed.
• project-elmo-ui — consumed: surfaces the new per-market SerenityMarket.domain field.

7. Test plan

(a) Local: exercise the Serenity through-API local e2e (activate brand -> create market -> deactivate) against the local stack and confirm each market domain resolves to a Site and a brand_sites row with type='serenity' exists for the brand. Run only after the data-service migration is applied to the local DB.
(b) Per-env: after the data-service migration deploys to a given env, create or activate a Serenity market and verify (1) a Site exists for the market's domain, (2) a brand_sites row tagged type='serenity' links it to the brand, (3) the brand V2 response still omits that domain from urls[]/siteIds, and (4) the markets API returns the market domain.

8. Deployment & merge order

• mysticat-data-service (branch feat/semrush-project-site-entities, migration brand_sites.type='serenity') — depends-on: MUST merge and deploy first; until then the link insert is silently rejected (migration-first rollout).
• project-elmo-ui PR 2078 — related: the UI consuming the new per-market domain.
• Order: deploy the data-service migration first, then merge this PR, then the elmo UI change.

🤖 Generated with Claude Code

[github-actions]

This PR will trigger a minor release when merged.

[MysticatBot]

Hey @rainer-friederich,

Verdict: Request changes - one blocking concern about brand URL visibility when a brand URL and market domain resolve to the same site.
Changes: mirrors each Semrush market as a SpaceCat Site entity with a brand_sites row tagged type='serenity', preserving it through brand edits and excluding it from the brand V2 response (12 files).
Note: CI checks are currently failing - resolve before merge.

Must fix before merge

1. [Important] Brand URL silently vanishes from V2 response when it resolves to the same site as a market domain - src/support/brands-storage.js:329 (details inline)
2. [Important] syncBrandSites does not check for error on the protected-rows SELECT - a transient DB failure silently empties protectedSiteIds, causing the re-INSERT to downgrade serenity rows - src/support/brands-storage.js:269 (details inline)

Non-blocking (5): minor issues and suggestions

• nit: race window between syncBrandSites (SELECT then DELETE then INSERT) and concurrent ensureMarketSite could temporarily downgrade a serenity row's type; self-healing on next market write but worth a comment acknowledging the eventual-consistency guarantee - src/support/brands-storage.js:275
• suggestion: validate brandDomain format before passing to ensureMarketSite (hostname pattern in OpenAPI or runtime check) to prevent junk/internal hostnames from creating Site records - src/support/serenity/site-linkage.js:68
• nit: ensureMarketSite returns a non-null siteId when the existing site belongs to another org (no link created), indistinguishable from full success for callers - src/support/serenity/site-linkage.js:85
• nit: brandDomainFromPayload(brandData) re-derives the domain instead of capturing the earlier brandDomain in a shared scope - src/controllers/brands.js:1486
• nit: the .or() filter type.is.null,type.neq.serenity relies on PostgREST NULL semantics that are non-obvious; a one-line comment explaining why is.null is needed alongside neq would help future readers - src/support/brands-storage.js:282

---

_{Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 3m 7s | Cost: $8.81 | Commit: 828dfaaf01e79dd36000a2b3d4baf34be2ac50f7
If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

[rainer-friederich]

Thanks for the review. Addressed in fc2943dd (the review ran on 828dfaa; several non-blocking items were already fixed in the intervening f6d2c170/a97155ce). Also: the CI failure you noted is fixed in a97155ce (an activate mirror hook evaluated brand.getOrganizationId() outside ensureMarketSite's try/catch → 500; now optional-chained).

Must fix before merge:

• [Important] Brand URL vanishes when it shares a site with a market domain (brands-storage.js) — fixed. mapDbBrandToV2 now surfaces a type='serenity' row when the brand ALSO lists that exact domain as a brand URL, so it keeps onboarded:true/siteId. Pure market domains (not in brand_urls) stay excluded. New test: "surfaces a Serenity row when its domain is ALSO an explicit brand URL".
• [Important] syncBrandSites ignores the protected-rows SELECT error — fixed. Now throws (fails closed), consistent with the delete/upsert error pattern; a swallowed error would empty protectedSiteIds and downgrade serenity rows. Replaced the earlier "silently proceeds" test with a fail-closed one.

Non-blocking:

• nit: race-window comment (brands-storage.js) — done (added in f6d2c170): the SELECT→DELETE→INSERT vs concurrent ensureMarketSite downgrade window is now documented as doubly-rare + self-healing on next market write.
• suggestion: validate brandDomain before ensureMarketSite — partially done / rest declined. ensureMarketSite now normalizes via hostnameFromUrlString (added in f6d2c170), so unparseable/junk input is skipped. A stage/internal-hostname blocklist is declined as out of scope here: the market domain is provisioned to Semrush upstream, which is its source of truth.
• nit: cross-org ensureMarketSite returns a non-null siteId — done (f6d2c170): returns null now when the site belongs to another org (no link created).
• nit: brandDomainFromPayload re-derives the domain — done (f6d2c170): lifted to a shared provisionedBrandDomain.
• nit: .or() filter needs a comment on the is.null+neq semantics — done: the comment ("a bare .neq would skip NULLs; type IS DISTINCT FROM 'serenity'") is in place.

[MysticatBot]

Hey @rainer-friederich,

Verdict: Approve - both prior blocking findings are addressed with tests; remaining observations are non-blocking.
Changes: mirrors each Semrush market as a SpaceCat Site entity with a brand_sites row tagged type='serenity', preserving it through brand edits and excluding it from the brand V2 response (13 files).

Non-blocking (2): minor issues and suggestions

• nit: ensureMarketSite returns siteId on brand_sites upsert failure, which contradicts the JSDoc ("null when skipped/failed"); no caller depends on this today, but updating the return or the doc would keep the contract honest - src/support/serenity/site-linkage.js:120
• suggestion: the ensureMarketSite call in createBrandForOrg sits inside the try/catch that triggers workspace-release compensation; the never-throws contract holds today (outer catch-all in site-linkage.js), but a one-line comment at the call site like "MUST NOT throw - catch below would release a live workspace" would make the invariant explicit for future editors - src/controllers/brands.js:1483

Previously flagged, now resolved

• Brand URL vanishing when it shares a site with a market domain - mapDbBrandToV2 now surfaces a serenity row when the domain is also an explicit brand URL
• syncBrandSites protected-rows SELECT error handling - now throws (fails closed), consistent with delete/upsert pattern
• Race-window comment on SELECT/DELETE/INSERT vs concurrent ensureMarketSite - documented
• brandDomainFromPayload re-derivation - lifted to shared provisionedBrandDomain
• Cross-org ensureMarketSite return value - now returns null when site belongs to another org
• .or() filter NULL semantics comment - added
• Domain normalization via hostnameFromUrlString - added

---

_{Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 0m 4s | Cost: $9.79 | Commit: fc2943dd387c7cf53eb27785efb06a7410bdc443
If this code review was useful, please react with 👍. Otherwise, react with 👎.}

[rainer-friederich]

Both non-blocking nits addressed in 27b02fef (leaving your approval standing — doc tweak + return-value honesty fix, no logic rework):

• ensureMarketSite now returns the site id only when the link is established; null otherwise (no client / failed link write). Tests updated.
• Added the explicit MUST-NOT-throw invariant comment at the brands.js call site.

[MysticatBot]

Hey @rainer-friederich,

Verdict: Approve - both post-approval nits addressed cleanly; no new concerns.
Changes: mirrors each Semrush market as a SpaceCat Site entity with a brand_sites row tagged type='serenity', preserving it through brand edits and excluding it from the brand V2 response (20 files).

Previously flagged, now resolved

• ensureMarketSite now returns null when the link is not established (upsert failure or missing postgrest client), honoring the JSDoc contract that non-null means "linked"
• MUST-NOT-throw invariant documented at the brands.js call site
• IT seed data and end-to-end assertions wired for the serenity market-mirror exclusion (brand GET single + list, delivery-type filter)
• Data-service image bumped to v5.44.0 (required for brand_sites CHECK constraint)

---

_{Skill: pr-review | Model: us.anthropic.claude-opus-4-6-v1[1m] | Duration: 0m 27s | Cost: $4.53 | Commit: a4436387bd59bdd94d0208ddd0327824849751ab
If this code review was useful, please react with 👍. Otherwise, react with 👎.}