On September 29 and 30, twenty-one commits over two days moved nav-site from Vercel + Neon to a Singapore VPS: Cloudflare Tunnel in, the origin exposes no public ports, postgres binds only to 127.0.0.1:5432, the app runs in Docker Compose, and builds are done on the server with a dedicated buildx builder.

The migration ultimately succeeded. But those earlier commits were almost all patching the same category of thing: assumptions the platform used to make on my behalf. This post records them in order of discovery, not in commit order.

One hop lands on 0.0.0.0:3000

Before switching production, I tested in preview: the 302 landing point for /r/<渠道码> was https://0.0.0.0:3000/, and the same applied to the callback redirects after successful and failed Google login. If the redirect entry point had only been discovered on production switch day, every short link on the site would have broken at once.

The root cause is in attachRequestMeta of next-server.js: in standalone mode, next-server holds the hostname/port it listens on and prioritizes them when constructing the request URL, so the experimental.trustHostHeader branch is never reached. On Vercel, the platform automatically trusts Host (config.js is enabled at hasNextSupport), so this was never exposed before the migration. In testing, neither Host nor X-Forwarded-Host affected req.url; only the protocol followed X-Forwarded-Proto.

The fix is a newly added lib/public-origin.ts: take the Host header (the origin is reachable only through the tunnel, and Cloudflare writes Host based on the domain the visitor requested), and for the protocol accept only https from X-Forwarded-Proto. Deliberately do not trust X-Forwarded-Host—visitors can supply it themselves, Cloudflare passes it through unchanged, and trusting it is an open redirect.

I also replaced openapi.json and llms.txt along the way: they used to prioritize reading XFH to build the site root, which is forgeable after self-hosting, and the endpoint is cached at the edge for an hour, which amounts to cache poisoning.

Same commit, prod and preview fight over one image tag

Both sides build nav-site:<sha>. But the same commit does not produce identical artifacts on both sides: the database connected at build time, prerendered data, and NEXT_PUBLIC_* can all differ. Whichever builds later takes over the tag—in testing, when production built later, the preview container’s image became an unnamed sha256:…; conversely, a production restart or rollback would end up on a build containing preview data.

Changed to nav-site:<env>-<sha>, so build, release, rollback, and cleanup all carry an environment prefix. Image identity cannot rely on commit alone; things with different artifacts should not share one name.

Secrets don’t enter the cache key

When releasing via CI on the VPS for the first time, between two commits only the deploy/ script changed (it does not enter the build context), and the whole pipeline hit cache and completed in 5 seconds. The cache itself was not the problem, but it exposed a quieter hazard: build-time environment variables are injected via BuildKit secrets, and the secret contents do not participate in the cache key. When only env changes and the code does not, pnpm build hits the old cache and the new NEXT_PUBLIC_* silently fails to take effect.

The fix is that deploy.sh passes the first 16 characters of the build env file’s sha256 as a build arg, and echo it in Dockerfile. As soon as env changes, the cache is invalidated.

EXIT trap references a local variable; a successful release exits 1

The EXIT trap in cmd_build_deploy / cmd_migrate used a function-local variable. The trap only fires when the script exits after the function returns normally; under set -u it reports “unbound”, turning a successful release into exit code 1—CI then marks every success red. After cleanup completes, trap - EXIT is removed.

pg-verify treats “empty vs. empty” as a match

When running the import against Neon for the first time, the source database failed to connect because of a certificate parameter. Yet pg-verify output “✓ sequence values match”—both sides were empty results, and since the table loop was empty there was no ✗ either. Although the script ultimately exited non-zero because of set -e, it reached no clear conclusion; on the night of the switch, it could easily have been read as a pass.

Now it first confirms both ends are reachable and that the source database really has tables under public; otherwise it exits 2 immediately. One more note: for libpq with sslmode=verify-full, sslrootcert=system must be added; Node’s pg driver does not recognize this parameter, so the connection string used by the app should contain only sslmode=verify-full.

Backups in a public bucket

The original recommendation was to create a new private backup bucket; later we decided to use the primary bucket nav-bucket directly—it has a public CDN domain attached, is publicly readable, and its object keys contain timestamps that make them guessable. The database contains user emails, password hashes, and partner keys; uploading it in plaintext would be equivalent to making it public.

So everything is age-encrypted before upload; if BACKUP_AGE_RECIPIENT is empty, the upload is refused outright (fail-closed), and ciphertext is placed under a prefix with a random suffix. The decryption private key is in /opt/nav-site/env/backup-age.key; a separate copy must be stored outside the machine: if the machine is lost entirely, the backups on R2 can only be decrypted with it.

Splitting one key into three

CHANNEL_REPORT_SECRET, USER_SESSION_SECRET, ADMIN_SESSION_SECRET, and CRON_SECRET all used to fall back to the same value, so the validity of externally shared links ended up tied to a key whose name gave no hint of the connection. When we split them apart during the migration, deploy.sh refuses to publish if any one of them is missing.

Among them, changing CHANNEL_REPORT_SECRET invalidates every externally shared link and QR code already sent to partners, so it must be stored somewhere other than the server; the others can be rotated at any time, and the only cost is that whoever happens to be logging in fails once. Keys that can be rotated at any time and keys that can’t must not look the same.

A few more that look harmless but bite

  • fail2ban on this machine bans you for 24 hours after 3 authentication failures. When you can’t connect, find out why first — don’t go guessing usernames.
  • sshd has port forwarding disabled, so migrations, script runs, and database queries all happen on the server itself; there is no SSH tunnel to fall back on.
  • The migrator keeps no record table. It reruns everything by filename every time, so migration files must be idempotent.
  • A deploy normally takes about 7 seconds with zero failed requests, and a bad version that won’t start is rolled back automatically. But Docker’s service-name resolution ignores health status: a version that can listen on the port yet returns wrong responses will receive roughly half the traffic during the health-check window, which lasts up to 30 seconds. That boundary is written into the runbook; we didn’t pretend it doesn’t exist.
  • Run a restore drill at least once a quarter — a backup that has never been restored isn’t a backup.

The defaults after the move

Vercel trusted the Host for you, gave every build its own identity, guaranteed that your backup bucket was private, and said “consistent” for you when a result was empty. Self-hosting doesn’t make these things disappear; it turns them from the platform’s defaults into your own defaults: one publicOrigin, one <env>-<sha> label, one sha256 build arg, two preflight checks, and one age public key.

What really matters in a migration checklist is never the commands — it’s whether each of these assumptions has been caught, one by one.