Migrations

Lunora has two distinct migration surfaces, because it has two distinct storage backends:

• Sharded / root tables live in each Durable Object's own SQLite. There is no schema-migration step: editing lunora/schema.ts and re-running codegen is the migration. The DO provisions tables and indexes on demand.
• Global tables (.global()) live in D1. A relational store needs explicit CREATE TABLE / ADD COLUMN SQL. The runtime writes the additive part itself on first use, and lunora migrate generate emits a timestamped .sql file recording the diff — a review artifact and the place to hand-write what the generator cannot express. No command applies that file, lunora deploy included.

Separately, data migrations (defineMigration) rewrite existing rows in a sharded table: a backfill or shape change applied to live documents, running inside each shard's DO in resumable batches.

The deploy gate tells you when you need one

You do not have to work out which schema edits need a backfill. lunora deploy (and prepare / verify) diffs your schema against the committed baseline in lunora/.lunora-schema.json and classifies every change. Additive edits — a new optional field, a new table, an added index — are reported and let through. A change existing rows cannot satisfy blocks the deploy, names the field, and prints the command that scaffolds the migration it wants:

deploy blocked: 1 unresolved breaking schema change(s) since the last blessed schema baseline:
  - added required field users.displayName — existing rows have no value; add a data migration to backfill it

To fix:
  • Add a `defineMigration({ id, table, up })` in lunora/ for each affected table, then re-run this command — the baseline is re-blessed on a successful deploy, not by `lunora migrate` itself.

    Scaffold the missing migration(s) — the generated `up` is an identity placeholder you must fill in:
      lunora migrate create backfill_users --table users

  • For backward-compatible changes (e.g. adding an optional field): pass `--allow-schema-drift` to skip the block.
  • To accept the new shape without a migration (you know data is compatible): pass `--update-schema-baseline`.
Docs: https://lunora.dev/docs/migrations

A migration only clears the drift on the table it iterates, so a backfill on messages will not wave through a required field added to users.

Nor is a migration offered for every breaking change — only for the ones a per-document transform can actually fix (a new required field, a type change, an optional field made required, a dropped field). The rest name their own fix instead:

If you know the stored data is already compatible, --allow-schema-drift skips the block for one run and lunora prepare --update-schema-baseline accepts the new shape permanently.

Schema migrations (global tables)

lunora migrate generate parses lunora/schema.ts, filters to .global() tables, and diffs the result against lunora/migrations/.snapshot.json. When the diff is non-empty it writes a timestamped SQL file and updates the snapshot.

lunora migrate generate                    # name defaults to "auto"
lunora migrate generate add_email_index    # name the migration

This emits lunora/migrations/<timestamp>_<slug>.sql and rewrites lunora/migrations/.snapshot.json. Commit both: they are deterministic, and the snapshot is what the next generate diffs against.

Supported diffs: CREATE TABLE (new global table), DROP TABLE (removed table), ALTER TABLE … ADD COLUMN (added column), and CREATE INDEX (added index). Unsupported deltas (column rename, column type change, dropping a column or index) are surfaced as a commented unsupported block in the generated file with a warning; write that SQL by hand.

Nothing applies the generated file

The .sql file is a record of the diff, not a step in any pipeline. What actually provisions a global table is the runtime: on the first request that touches one it runs CREATE TABLE IF NOT EXISTS for the declared columns and indexes, and ALTER TABLE … ADD COLUMN for fields the table lacks. That covers every additive change, which is most of them, with the schema as the single source of truth.

It does not cover anything the generator flags as unsupported, or SQL you add by hand: a backfilling UPDATE, a column rename written as add/copy/drop, an index the schema does not declare, a DROP TABLE. Apply those yourself, once per environment, before the deploy that depends on them:

wrangler d1 execute <DB> --remote --file=lunora/migrations/<timestamp>_<slug>.sql

Data migrations (sharded tables)

A data migration is a per-document transform over one table. Declare it with defineMigration from @lunora/server anywhere under lunora/; codegen discovers it through the type checker and keys a registry on its id.

// lunora/migrations.ts
import { defineMigration } from "lunorash/server";

export const backfillReadBy = defineMigration({
    id: "backfill-read-by",
    table: "messages",
    up: (document) => ({ ...document, readBy: [] }),
});

The transform receives the stored document (including _id and _creationTime). Return a new document to rewrite the row, or undefined (or nothing) to leave it untouched; untouched rows are counted as processed, not changed. The runner always preserves the original _id and _creationTime, so a transform should not change row identity.

What a data migration cannot do

The runner executes inside one shard's Durable Object, and the only write it performs is a replace of the row it was handed. Two consequences are worth knowing before you plan a change around it:

• It cannot delete a row. Returning undefined means "leave this one alone", not "drop it". Removing rows is a job for an ordinary mutation.
• It cannot move a row to another DO. The orchestrator runs the transform once per shard, but each run only ever rewrites its own shard's rows.

That second one rules it out for adopting .shardBy() on a table that already has rows — re-homing them is an export/import round trip, not a transform. See Migrating a populated table, and Migrating a populated table to D1 for the .global() variant. That one runs in a different order: the root DO's copies must be cleared before the schema edit, because once a table is declared .global() the shard's writer refuses to touch it.

The admin surface is the exception, and the reason that procedure works: admin reads and writes are addressed to a shard rather than routed by the schema, so the Studio can still reach a root DO's copies of a table the schema has since sharded. A data migration is the one path that always follows the schema.

Scaffolding

lunora migrate create <name> writes a defineMigration stub into lunora/migrations.ts, appending to the file (and adding the import) when it already exists. It refuses to clobber a migration with the same id or export name.

lunora migrate create backfill_read_by --table messages

The free-text name is slugified into a kebab-case id and a camelCase export. The table must be a bare identifier. Omit --table and the command prompts for it interactively (offering the tables declared in lunora/schema.ts); in a non-interactive context (CI / piped) it fails instead, so pass --table explicitly there.

Running a data migration

lunora migrate up | down | status <id> drives the cross-shard orchestrator. It resolves the migration's table locally, then POSTs an admin RPC to the worker's /_lunora/migrate endpoint, which fans the run out to every live shard of that table and rolls the per-shard outcomes up.

lunora migrate up backfill-read-by             # run forward across shards
lunora migrate up backfill-read-by --dry-run   # preview counts, rewrite nothing
lunora migrate down backfill-read-by           # apply the down transform
lunora migrate status backfill-read-by         # per-shard run-state

Every run is admin-gated: pass --token or set LUNORA_ADMIN_TOKEN. Against production, --prod requires an explicit --url, and up/down additionally require --yes.

You can also apply pending data migrations as part of a deploy:

lunora deploy --migrate --migrate-yes --migrate-url https://app.example.com --migrate-token $LUNORA_ADMIN_TOKEN

Resumable and idempotent

Each shard's runner tracks progress (cursor, processed/changed counts, status) in a reserved __lunora_migrations table, persisting after every batch. Two properties follow:

• Resumable. An interrupted run, or one stopped by --steps, resumes from the stored cursor instead of rescanning. Iteration uses a stable _creationTime ASC, _id ASC keyset order, and rewrites preserve row identity, so each row is visited exactly once even as the batch ahead is rewritten.
• Idempotent on completion. Re-running a migration already completed in the same direction is a no-op that returns the recorded counts.

The rolled-up status across shards is completed only when every shard finished cleanly; failed if any shard's runner reported failure; and in_progress if any shard is incomplete or unreachable (the run stays resumable). The roll-up also reports summed processed / changed counts and the number of ok vs failed shards.

The built-in re-projection backfill

One migration id is reserved by the framework rather than declared in your lunora/: __lunora_reproject__<table>. It rewrites rows whose v.bigint() / v.bytes() columns are still stored in the pre-projection form. Such a row reads back correctly, but SQL never matches it, so filter, withIndex, ORDER BY and SUM silently skip it. Only rows written before the storage codec was fixed are affected, and any ordinary write heals a row, so what is left is whatever nobody rewrites.

--dry-run answers whether you need it, per shard, without rewriting anything:

lunora migrate up __lunora_reproject__paymentSessions --dry-run
lunora migrate up __lunora_reproject__paymentSessions

The transform skips a row already in the current projection, and the runner counts only the rows it would rewrite, so a dry run's changed is the legacy-row count for that shard. migrate status reports run-state for a backfill that has already run; use the dry run before one, since a shard that never ran it has no run-state to report.

One caveat on reading changed: 0 as "done": the transform also skips a row it cannot re-project at all (a bigint wider than the projection's 39-digit key, which is reachable for a uint256-shaped value), and such a row is counted processed, not changed. It logs a warning naming the row id, and that warning is the only signal. So changed: 0 means "nothing left that this backfill can fix", which is the same thing as "fully re-projected" only when the shard's logs are clean.

The backfill is deliberately not automatic on deploy: every rewritten row pokes its live subscribers, so an all-legacy shard would turn a deploy into a broadcast storm on a path nobody asked for.

Inspecting run-state in the Studio

The Studio's Database → Migrations panel (see Studio › Database) inspects and drives data migrations on a single shard. Enter a shard key to read its persisted run-state table (id, direction, status, processed, changed, last updated, and any error) over a live channel, so an in-progress migration's counts climb in place. You can kick off a migration by id with a direction and an optional dry-run toggle; a real (non-dry-run) run is behind a confirm gate. The panel issues no credentials of its own; it relies on the worker's LUNORA_ADMIN_TOKEN gate like the CLI.

One-off data edits without a deploy

A defineMigration is the right tool for a backfill you want reviewed, versioned and re-runnable. It is the wrong tool for "mark these 800 rows seen" — that shouldn't need a deploy at all.

The Studio's data browser has Set column on N matching beside Delete N matching. Filter the table to the rows you mean, pick a column, give it a JSON value, and every matching row is patched through the same schema-aware writer a mutation uses: validators run, indexes and FTS/aggregate/rank shadow tables stay in sync, and live subscribers are poked.

The Studio is always local: lunora view opens the studio the running dev server serves, and a deployed worker serves none — so these edits land in whatever storage that dev server is bound to. lunora dev --remote proxies the D1/KV/R2 bindings to the deployed worker, which puts the data browser on production .global() tables. DO shards stay local either way, so a sharded table's rows are only ever edited against the dev server's own storage.

The value is parsed as JSON, not taken as text, so true, 42 and null arrive as themselves — a v.boolean() column takes true, not "true". A v.bigint() or v.bytes() column takes the same tagged form the grid displays.

Things worth knowing before you run one:

• It is scoped to one shard. The data browser is shard-addressed, so on a .shardBy() table "every matching row" means every matching row in the shard you are viewing. The dialog names the shard. Other shards are untouched — repeat per shard, or write a defineMigration, which runs across all of them.
• It is bounded. The shard writes at most one page per round-trip and the browser loops it. A very large set stops with "rows still match"; running it again resumes from where it stopped rather than rescanning from the top.
• A unique column is refused. Setting a single-column .unique() index to the same value on more than one row cannot succeed, so the dialog blocks it rather than letting the writer fail partway.
• A failure mid-run is partial, not rolled back. Each row commits on its own, so the message reports how many rows had already been written.
• Every call is audited — table, the field names set, and the row count. The values themselves are never recorded.

For anything you want to keep, review or run again later, write a defineMigration instead.

Resetting local state

lunora reset is not a migration command, but it's the companion when a local schema change leaves stale dev data. It clears the local Miniflare state directory (.wrangler/state), and with --all also removes .lunora-cache.

lunora reset          # clear .wrangler/state (prompts to confirm)
lunora reset --all    # also remove .lunora-cache
lunora reset --yes    # skip the prompt (required when stdin is not a TTY)

This only touches local state; it never reaches a deployed worker or D1.