Migrations

Schema-change SQL for global tables and online, resumable data migrations.

Last updated:

Lunora has two distinct migration surfaces, because it has two distinct storage backends:

  • Sharded / root tables live in each Durable Object's own SQLite. There is no schema-migration step — editing lunora/schema.ts and re-running codegen is the migration. The DO provisions tables and indexes on demand.
  • Global tables (.global()) live in D1. A relational store needs explicit CREATE TABLE / ADD COLUMN SQL, so lunora migrate generate diffs your schema and emits a timestamped .sql file that is applied at deploy time.

Separately, data migrations (defineMigration) rewrite existing rows in a sharded table: a backfill or shape change applied to live documents, running inside each shard's DO in resumable batches.

There is no migrations DSL for sharded-table schema. If all your tables are sharded, the only migration surface you'll touch is defineMigration for data backfills.

Schema migrations (global tables)

lunora migrate generate parses lunora/schema.ts, filters to .global() tables, and diffs the result against lunora/migrations/.snapshot.json. When the diff is non-empty it writes a timestamped SQL file and updates the snapshot.

lunora migrate generate                    # name defaults to "auto"
lunora migrate generate add_email_index    # name the migration

This emits lunora/migrations/<timestamp>_<slug>.sql and rewrites lunora/migrations/.snapshot.json. Commit both — they are deterministic, and the snapshot is what the next generate diffs against.

Supported diffs: CREATE TABLE (new global table), DROP TABLE (removed table), ALTER TABLE … ADD COLUMN (added column), and CREATE INDEX (added index). Unsupported deltas (column rename, column type change, dropping a column or index) are surfaced as a commented unsupported block in the generated file with a warning; write that SQL by hand.

The generated SQL is applied through @lunora/d1's MigrationRunner, which tracks applied migrations in a journal table and skips ones already run, so re-applying is safe.

Data migrations (sharded tables)

A data migration is a per-document transform over one table. Declare it with defineMigration from @lunora/server anywhere under lunora/ — codegen discovers it through the type checker and keys a registry on its id.

// lunora/migrations.ts
import { defineMigration } from "lunorash/server";

export const backfillReadBy = defineMigration({
    id: "backfill-read-by",
    table: "messages",
    up: (document) => ({ ...document, readBy: [] }),
});
FieldRequiredDescription
idyesStable, unique string. The registry key and the per-shard run-state key.
tableyesTable whose documents the migration iterates.
upyesForward transform applied to every row by migrate up.
downnoReverse transform, applied by migrate down.
batchSizenoRows fetched and rewritten per batch (defaults to the runner's batch size).

The transform receives the stored document (including _id and _creationTime). Return a new document to rewrite the row, or undefined (or nothing) to leave it untouched; untouched rows are counted as processed, not changed. The runner always preserves the original _id and _creationTime, so a transform should not change row identity.

id and table must be static string literals. Codegen lifts them at build time to key the registry and resolve the target table, and rejects duplicate ids across the project.

Scaffolding

lunora migrate create <name> writes a defineMigration stub into lunora/migrations.ts, appending to the file (and adding the import) when it already exists. It refuses to clobber a migration with the same id or export name.

lunora migrate create backfill_read_by --table messages

The free-text name is slugified into a kebab-case id and a camelCase export. --table must be a bare identifier; omit it and the stub gets a TODO_table placeholder you'll need to fill in.

Running a data migration

lunora migrate up | down | status <id> drives the cross-shard orchestrator. It resolves the migration's table locally, then POSTs an admin RPC to the worker's /_lunora/migrate endpoint, which fans the run out to every live shard of that table and rolls the per-shard outcomes up.

lunora migrate up backfill-read-by             # run forward across shards
lunora migrate up backfill-read-by --dry-run   # preview counts, rewrite nothing
lunora migrate down backfill-read-by           # apply the down transform
lunora migrate status backfill-read-by         # per-shard run-state
FlagDescription
--dry-runScan and count without rewriting rows or persisting run-state.
--batch-sizeRows per batch, overriding the migration's own batchSize and the default.
--stepsCap on batches processed this invocation (the runner's maxBatches). The run stays resumable.
--urlWorker URL (defaults to http://localhost:8787).
--tokenAdmin bearer token. Falls back to LUNORA_ADMIN_TOKEN.
--prodTarget production. Requires an explicit --url.
--yesRequired alongside --prod for up/down — confirms running against production.

Every run is admin-gated: pass --token or set LUNORA_ADMIN_TOKEN. Against production, --prod requires an explicit --url, and up/down additionally require --yes.

You can also apply pending data migrations as part of a deploy:

lunora deploy --migrate --migrate-token $LUNORA_ADMIN_TOKEN

Resumable and idempotent

Each shard's runner tracks progress (cursor, processed/changed counts, status) in a reserved __lunora_migrations table, persisting after every batch. Two properties follow:

  • Resumable. An interrupted run, or one stopped by --steps, resumes from the stored cursor instead of rescanning. Iteration uses a stable _creationTime ASC, _id ASC keyset order, and rewrites preserve row identity, so each row is visited exactly once even as the batch ahead is rewritten.
  • Idempotent on completion. Re-running a migration already completed in the same direction is a no-op that returns the recorded counts.

The rolled-up status across shards is completed only when every shard finished cleanly; failed if any shard's runner reported failure; and in_progress if any shard is incomplete or unreachable (the run stays resumable). The roll-up also reports summed processed / changed counts and the number of ok vs failed shards.

Inspecting run-state in the Studio

The Studio's Database → Migrations panel (see Studio › Database) inspects and drives data migrations on a single shard. Enter a shard key to read its persisted run-state table (id, direction, status, processed, changed, last updated, and any error) over a live channel, so an in-progress migration's counts climb in place. You can kick off a migration by id with a direction and an optional dry-run toggle; a real (non-dry-run) run is behind a confirm gate. The panel issues no credentials of its own; it relies on the worker's LUNORA_ADMIN_TOKEN gate like the CLI.

Resetting local state

lunora reset is not a migration command, but it's the companion when a local schema change leaves stale dev data. It clears the local Miniflare state directory (.wrangler/state), and with --all also removes .lunora-cache.

lunora reset          # clear .wrangler/state (prompts to confirm)
lunora reset --all    # also remove .lunora-cache
lunora reset --yes    # skip the prompt (required when stdin is not a TTY)

This only touches local state; it never reaches a deployed worker or D1.

See also

  • Schema — tables, validators, indexes, sharding, and .global().
  • CLI — full lunora migrate, lunora deploy, and lunora reset reference.
  • Studio › Database — the Migrations panel and the rest of the data console.