Migrations
Schema-change SQL for global tables and online, resumable data migrations.
Last updated:
Lunora has two distinct migration surfaces, because it has two distinct storage backends:
- Sharded / root tables live in each Durable Object's own SQLite. There is
no schema-migration step — editing
lunora/schema.tsand re-running codegen is the migration. The DO provisions tables and indexes on demand. - Global tables (
.global()) live in D1. A relational store needs explicitCREATE TABLE/ADD COLUMNSQL, solunora migrate generatediffs your schema and emits a timestamped.sqlfile that is applied at deploy time.
Separately, data migrations (defineMigration) rewrite existing rows in a
sharded table: a backfill or shape change applied to live documents, running
inside each shard's DO in resumable batches.
There is no migrations DSL for sharded-table schema. If all your tables are sharded, the only migration surface you'll touch is defineMigration for data
backfills.
Schema migrations (global tables)
lunora migrate generate parses lunora/schema.ts, filters to .global()
tables, and diffs the result against lunora/migrations/.snapshot.json. When
the diff is non-empty it writes a timestamped SQL file and updates the snapshot.
lunora migrate generate # name defaults to "auto"
lunora migrate generate add_email_index # name the migrationThis emits lunora/migrations/<timestamp>_<slug>.sql and rewrites
lunora/migrations/.snapshot.json. Commit both — they are deterministic, and
the snapshot is what the next generate diffs against.
Supported diffs: CREATE TABLE (new global table), DROP TABLE (removed
table), ALTER TABLE … ADD COLUMN (added column), and CREATE INDEX (added
index). Unsupported deltas (column rename, column type change, dropping a
column or index) are surfaced as a commented unsupported block in the
generated file with a warning; write that SQL by hand.
The generated SQL is applied through @lunora/d1's MigrationRunner, which
tracks applied migrations in a journal table and skips ones already run, so
re-applying is safe.
Data migrations (sharded tables)
A data migration is a per-document transform over one table. Declare it with
defineMigration from @lunora/server anywhere under lunora/ — codegen
discovers it through the type checker and keys a registry on its id.
// lunora/migrations.ts
import { defineMigration } from "lunorash/server";
export const backfillReadBy = defineMigration({
id: "backfill-read-by",
table: "messages",
up: (document) => ({ ...document, readBy: [] }),
});| Field | Required | Description |
|---|---|---|
id | yes | Stable, unique string. The registry key and the per-shard run-state key. |
table | yes | Table whose documents the migration iterates. |
up | yes | Forward transform applied to every row by migrate up. |
down | no | Reverse transform, applied by migrate down. |
batchSize | no | Rows fetched and rewritten per batch (defaults to the runner's batch size). |
The transform receives the stored document (including _id and
_creationTime). Return a new document to rewrite the row, or undefined (or
nothing) to leave it untouched; untouched rows are counted as processed, not
changed. The runner always preserves the original _id and _creationTime, so
a transform should not change row identity.
id and table must be static string literals. Codegen lifts them at build time to key the registry and resolve the target table, and rejects
duplicate ids across the project.
Scaffolding
lunora migrate create <name> writes a defineMigration stub into
lunora/migrations.ts, appending to the file (and adding the import) when it
already exists. It refuses to clobber a migration with the same id or export
name.
lunora migrate create backfill_read_by --table messagesThe free-text name is slugified into a kebab-case id and a camelCase export.
--table must be a bare identifier; omit it and the stub gets a TODO_table
placeholder you'll need to fill in.
Running a data migration
lunora migrate up | down | status <id> drives the cross-shard orchestrator. It
resolves the migration's table locally, then POSTs an admin RPC to the worker's
/_lunora/migrate endpoint, which fans the run out to every live shard of that
table and rolls the per-shard outcomes up.
lunora migrate up backfill-read-by # run forward across shards
lunora migrate up backfill-read-by --dry-run # preview counts, rewrite nothing
lunora migrate down backfill-read-by # apply the down transform
lunora migrate status backfill-read-by # per-shard run-state| Flag | Description |
|---|---|
--dry-run | Scan and count without rewriting rows or persisting run-state. |
--batch-size | Rows per batch, overriding the migration's own batchSize and the default. |
--steps | Cap on batches processed this invocation (the runner's maxBatches). The run stays resumable. |
--url | Worker URL (defaults to http://localhost:8787). |
--token | Admin bearer token. Falls back to LUNORA_ADMIN_TOKEN. |
--prod | Target production. Requires an explicit --url. |
--yes | Required alongside --prod for up/down — confirms running against production. |
Every run is admin-gated: pass --token or set LUNORA_ADMIN_TOKEN. Against
production, --prod requires an explicit --url, and up/down additionally
require --yes.
You can also apply pending data migrations as part of a deploy:
lunora deploy --migrate --migrate-token $LUNORA_ADMIN_TOKENResumable and idempotent
Each shard's runner tracks progress (cursor, processed/changed counts, status)
in a reserved __lunora_migrations table, persisting after every batch. Two
properties follow:
- Resumable. An interrupted run, or one stopped by
--steps, resumes from the stored cursor instead of rescanning. Iteration uses a stable_creationTime ASC, _id ASCkeyset order, and rewrites preserve row identity, so each row is visited exactly once even as the batch ahead is rewritten. - Idempotent on completion. Re-running a migration already
completedin the same direction is a no-op that returns the recorded counts.
The rolled-up status across shards is completed only when every shard finished
cleanly; failed if any shard's runner reported failure; and in_progress if
any shard is incomplete or unreachable (the run stays resumable). The roll-up
also reports summed processed / changed counts and the number of ok vs
failed shards.
Inspecting run-state in the Studio
The Studio's Database → Migrations panel (see
Studio › Database) inspects and drives data
migrations on a single shard. Enter a shard key to read its persisted run-state
table (id, direction, status, processed, changed, last updated, and
any error) over a live channel, so an in-progress migration's counts climb in
place. You can kick off a migration by id with a direction and an optional
dry-run toggle; a real (non-dry-run) run is behind a confirm gate. The panel
issues no credentials of its own; it relies on the worker's LUNORA_ADMIN_TOKEN
gate like the CLI.
Resetting local state
lunora reset is not a migration command, but it's the companion when a local
schema change leaves stale dev data. It clears the local Miniflare state
directory (.wrangler/state), and with --all also removes .lunora-cache.
lunora reset # clear .wrangler/state (prompts to confirm)
lunora reset --all # also remove .lunora-cache
lunora reset --yes # skip the prompt (required when stdin is not a TTY)This only touches local state; it never reaches a deployed worker or D1.
See also
- Schema — tables, validators, indexes, sharding, and
.global(). - CLI — full
lunora migrate,lunora deploy, andlunora resetreference. - Studio › Database — the Migrations panel and the rest of the data console.