Skip to content

feat: synchronize upstream model catalogs safely - #680

Merged
chenyme merged 2 commits into
devfrom
models
Aug 29, 2026
Merged

feat: synchronize upstream model catalogs safely#680
chenyme merged 2 commits into
devfrom
models

Conversation

@chenyme

@chenyme chenyme commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Summary

Improve upstream model synchronization with a preview-first and transactional catalog reconciliation workflow.

  • Classify remote models as added, updated, reactivated, inactivated, unchanged, or protected before applying changes.
  • Soft-inactivate managed models no longer returned by the upstream without deleting platform models or route configuration.
  • Preserve manually managed catalog entries and automatically reactivate models that return upstream.
  • Reject stale previews using catalog snapshots and require explicit confirmation for empty upstream catalogs.
  • Fetch all Anthropic and Gemini model catalog pages to prevent incomplete snapshots.
  • Redesign the synchronization dialog with status summaries, model tooltips, confirmation handling, cancellable requests, and a table that grows naturally up to ten rows.
  • Keep synchronization and route-binding failures separate so partial completion is reported accurately.

Change type

  • Bug fix
  • Feature
  • Documentation
  • Refactor
  • Configuration / deployment
  • Security hardening
  • Other

Affected areas

  • Frontend / UI
  • Backend / API
  • Authentication / authorization
  • Conversations / streaming
  • Files / RAG / extraction
  • Model routing / providers
  • MCP / tools
  • Billing / payments
  • Admin console
  • Deployment / Docker / configuration
  • Documentation

Verification

  • env GOCACHE=/private/tmp/deeix-go-cache go test ./...

  • env GOCACHE=/private/tmp/deeix-go-cache go vet ./...

  • env GOCACHE=/private/tmp/deeix-go-cache go test -count=1 -race ./internal/application/channel ./internal/infra/llm ./internal/infra/persistence/postgres/channel ./internal/transport/http/channel

  • pnpm api:check

  • pnpm --filter @deeix/web check

  • pnpm --filter @deeix/web build

  • pnpm build

  • git diff --check

  • Not run; reason: Browser-based manual verification was not performed.

Screenshots, API examples, or logs

Screenshots are not included in this draft.

The updated admin dialog provides:

  • A complete catalog status summary with scrollable model-name tooltips.
  • Search and permission-group selection on one responsive row.
  • A model table using its natural height up to ten rows before enabling internal scrolling.
  • Explicit confirmation when synchronization will deactivate missing upstream models.
  • Clear handling for stale previews, empty catalogs, catalog-only synchronization, and partial binding failures.

Configuration, migration, and compatibility notes

  • No database migration is required.
  • No environment variable or deployment configuration changes are required.
  • Missing sync-managed models are soft-inactivated instead of permanently deleted.
  • Platform models, route records, and route configuration are preserved.
  • Manual catalog entries are protected from synchronization updates and inactivation.
  • Legacy imported catalog entries remain managed and migrate to the synchronized lifecycle when encountered.
  • Models that reappear upstream are automatically reactivated.
  • Empty upstream catalogs require explicit confirmation before any managed models are inactivated.
  • Synchronization now rejects a changed remote snapshot with HTTP 409 and requires a new preview.
  • The sync endpoint adds optional expected_snapshot and allow_empty query parameters.
  • Existing response fields remain available; synchronization plan and result fields are additive.
  • Anthropic and Gemini catalog discovery may issue multiple paginated requests when required.
  • Generated Swagger files and TypeScript API contracts are included.

Documentation

  • Documentation is not needed for this change.
  • Documentation was updated.
  • Documentation still needs to be updated.

Security and privacy

  • No secrets, tokens, credentials, local config, or personal data are included.
  • User data access remains scoped by authenticated user context unless an admin-only path explicitly requires broader access.
  • Security-sensitive behavior was reviewed, including authentication, authorization, provider routing, file processing, billing, and admin APIs where relevant.

Checklist

  • I searched existing issues and pull requests.
  • Changes are focused and do not include unrelated refactors.
  • Tests or static verification were run where practical.
  • User-facing behavior, deployment steps, API contracts, or configuration changes are documented.
  • Generated artifacts are included only when this project explicitly requires them.
  • Caches, build output, .pyc files, .env files, and local storage data are not committed.

@chenyme chenyme self-assigned this Aug 29, 2026
@chenyme chenyme linked an issue Aug 29, 2026 that may be closed by this pull request
3 tasks
# Conflicts:
#	frontend/features/admin/components/sections/upstreams/upstreams-models-dialog.tsx
@chenyme
chenyme merged commit 508b96a into dev Aug 29, 2026
11 checks passed
@chenyme
chenyme deleted the models branch August 29, 2026 04:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature]: Reconcile removed upstream models during synchronization

1 participant