Skip to content

Migrate to self-hosted server never actually deploys (3 stacked bugs: missing deploy trigger, release-dist path/packaging mismatch, PGlite assets crash) #869

Description

@PratikGolecha

Summary

Migrating a self-hosted Compose instance to another self-hosted server via Settings → Team → Migrate never completes — the wizard's preflight checks all pass, "Start migration" spins, and it eventually fails deep inside the data-restore step with no user-facing error. I dug into the source to find out why and found three separate, stackable bugs in the migration path. Filing them together since each one just exposes the next.

Repro environment: v0.7.2, migrating from a Compose instance (source) to a fresh RackNerd VPS (target), custom domain, SSH key auth. Target tried both --bare and Compose installs — same failure both times.


Bug 1 — the deploy step is a documented no-op; the DB-restore step assumes it ran

apps/api/src/modules/system/migration/migrate-instance.service.ts goes straight from creating the project row (step 2) to the sealed data restore (step 5). Step 3 — actually deploying the release to the target — is explicitly a TODO:

* 3. (TODO when deploy-engine integration lands) Trigger the deploy
*    via the standard project pipeline. The pipeline streams the
*    release dist to the target server, installs, starts.
...
* The "actual deploy" in step 3 is intentionally wired to the existing
* deployment-pipeline call shape but left as a TODO marker...

Meanwhile db-migrate-remote.service.ts (dumpRemoteRestore / sealedRemoteImport) hardcodes:

const remoteProjectDir = `/var/lib/openship/projects/${input.projectSlug}/current`;
...
`cd ${remoteProjectDir} && ...`

Since nothing in step 3 ever creates /var/lib/openship/projects/<slug>/ (verified: the directory doesn't exist on the target at all, on either a blank server or one already running openship up), this cd always fails:

sh: 1: cd: can't cd to /var/lib/openship/projects/openship-instance-org_.../current

The desktop UI just shows a spinner that completes with no error and no success state — the failure is swallowed somewhere above this (the SSH journal at ~/.openship/ops/migrate:sealed-import:.../{cmd,exit,stdout,stderr} is the only place the real error is visible).

Workaround used to get further: manually create the directory structure the restore step expects (releases/1/, a current symlink to it) and populate it with a build-release output.


Bug 2 — even with that workaround, release-dist's layout doesn't match what the migration commands assume

Two separate mismatches here:

  • release-dist's packaged layout is flat — api/, dashboard/, packages/ at the root (per apps/api/scripts/build-release.ts's own output). But dumpRemoteRestore/sealedRemoteImport both hardcode bun --cwd apps/api scripts/... — the monorepo-source path, not the release layout. apps/api doesn't exist in a release-dist checkout at all.
  • scripts/import-instance.ts (and presumably packages/db/scripts/restore.ts, used by the non-sealed dumpRemoteRestore path) is not copied into release-dist by build-release.ts — only api/src, dashboard, and packages get shipped; the scripts/ directory is dropped entirely.

So even once bug 1's directory exists, bun --cwd apps/api scripts/import-instance.ts fails two different ways depending on what's missing: no apps/api directory, and the script itself isn't there to run.

Workaround: symlink apps/api → api, manually copy scripts/import-instance.ts from the monorepo into the release checkout.


Bug 3 — OPENSHIP_PGLITE_ASSETS_DIR breaks PGlite when set against a plain bun install of the package

Getting past bugs 1 and 2, import-instance.ts now runs — and crashes inside PGlite's WASM init:

TypeError: ASM_CONSTS[e] is not a function. (In 'ASM_CONSTS[e](...a)', 'ASM_CONSTS[e]' is undefined)
  at .../@electric-sql+pglite@0.3.16/.../index.js
error: Failed query: CREATE SCHEMA IF NOT EXISTS "drizzle"

Root cause, in packages/db/src/client.ts:

// OPENSHIP_PGLITE_ASSETS_DIR points at copies shipped alongside the binary; ...
const dir = process.env.OPENSHIP_PGLITE_ASSETS_DIR;
...
const wasmModule = await WebAssembly.compile(readFileSync(join(dir, "pglite.wasm")));
const fsBundle = new Blob([readFileSync(join(dir, "pglite.data"))]);
...
const client = assets ? new PGlite({ dataDir, ...assets }) : new PGlite(dataDir);

When OPENSHIP_PGLITE_ASSETS_DIR is set (as it is for a real openship up --bare install — it points at the CLI's own bundled pglite.wasm/pglite.data), this manually loads those pre-staged asset files and feeds them into new PGlite(), bypassing PGlite's own internal asset resolution. Those staged assets are built for the officially bundled CLI server, not a plain bun install of @electric-sql/pglite@0.3.16 from a release-dist checkout — the two don't agree on ASM_CONSTS/build format, and PGlite's WASM init crashes.

Workaround: unset OPENSHIP_PGLITE_ASSETS_DIR for this script's environment, letting PGlite resolve its own bundled assets from its own node_modules copy. Confirmed working with a standalone reproduction:

import { db } from '../packages/db/src/client';
console.log('DB connected OK', !!db);

— fails with the assets-dir env var set, succeeds cleanly without it.


Net effect

With all three worked around by hand (create the directory structure, patch in the missing script + path symlink, drop the assets-dir env var), the migration's data-restore step completes and the source instance correctly reports "This instance moved to your server." But since bug 1 means the actual deploy/service-start never happens through the real pipeline, nothing is left serving the migrated instance on the target — a human still has to separately bring up a working install (openship up) pointed at the now-migrated data directory to actually see it live.

Suggested fix direction

  • Bug 1 is the root cause — wiring the actual POST /api/deployments + POST /api/deployments/:id/build calls into the wizard (as the code comment already describes) before calling into sealedRemoteImport/dumpRemoteRestore would make bugs 2 and 3 surface immediately in a normal dev/test pass instead of only in a fully-manual repro.
  • Bug 2's path mismatch suggests the restore commands were written against a monorepo dev checkout and never actually exercised against a real release-dist output.
  • Bug 3 is the most standalone fix: either don't pass assets into new PGlite() when running from a release-dist context (detect via DEPLOY_MODE or similar), or ship pglite.wasm/pglite.data inside release-dist itself matching the package version actually installed there.

Happy to open a PR for any of these if useful — have a working manual repro/fix for all three.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions