Skip to content
DocsconceptsDocumentation

Sharding

Default __root__ DO, opt-in .shardBy(), and .global() escape hatch.

Last updated:

Lunora scales out, but starts on a single shard. New apps get one Durable Object, addressable as __root__, that holds every table without an explicit tier modifier. This is the Zeroback shape, and the right answer for the first 80% of an app's life.

When to shard

The per-DO ceiling is 10 GB of SQLite and roughly 1 000 sustained req/s. When the __root__ DO crosses 1 GB, the runtime emits a console.warn once per DO lifetime so you can plan the migration before you hit the wall. The threshold (1 073 741 824 bytes) is 10% of the per-DO ceiling, far enough below it that a .shardBy() migration has runway to land.

[@lunora/do] __root__ Durable Object SQLite size is 1075000000 bytes
(>= 1 GiB, 10% of the 10 GiB per-DO ceiling). Plan a `.shardBy()` migration
before you hit the wall. See https://lunora.sh/docs/concepts/sharding for
guidance.

The warning fires from ShardDO after an RPC write, and deduplicates itself with a flag that is static on the class — so it shows up at most once per isolate, not once per DO instance, and several shards sharing an isolate see one warning between them. It is also only on the RPC write path: admin, WebSocket, and alarm writes never trigger the check. Treat it as a nudge to go look, not as a per-shard alert. Non-__root__ DOs are never affected, even when they exceed 1 GB on their own.

On an empty table, sharding is a single edit:

messages: defineTable({ channelId: v.id("channels"), text: v.string() }).shardBy("channelId");

After codegen, every call to ctx.db.messages.* routes by channelId. A chat with 5 000 active channels now spreads across 5 000 DOs, each with its own SQLite, CPU budget, and hibernation timer.

Migrating a populated table

The edit above only moves future rows. Rows written before it stay physically in __root__'s SQLite, and once the table routes by channelId your app never reads them again — they are invisible to ctx.db and still occupying the bytes you sharded to reclaim. Since the 1 GB warning fires precisely when the table is already full, that is the case you will actually be in.

Move them with an export/import round trip. lunora import resolves each row's shard key from the deployed schema and fans the batches out per shard, so a dump taken before the edit lands on the right DOs after it.

This is not an online migration. The table is unavailable from step 1 until writes resume at step 5, and the dump is inside that window rather than before it. Take the window deliberately: a maintenance flag, or a quiet hour. Snapshot first: Backups covers point-in-time recovery, and a sharding migration is exactly the change you want it for.

  1. Stop writes to the table. Do this before the dump, not after. Export walks the table with successive live reads rather than one snapshot, and step 6 deletes the root DO's copies — so anything written during the procedure and missed by the dump is gone. The widest window is between the dump finishing and the deploy taking effect: those writes still land in __root__ and are deleted later. (A multi-table export widens it further — tables are walked one after another, so a write to an already-finished table is missed too.)

  2. Dump the table while it still lives in the root DO. _id and _creationTime survive the round trip, so relations and _creationTime-ordered pagination are preserved.

    lunora export --tables messages --out messages.ndjson --prod --url <url>
  3. Add .shardBy() and deploy. The schema-drift gate blocks this deploy on purpose — a shard-mode change is breaking — so pass --allow-schema-drift once you are running these steps deliberately.

    lunora codegen && lunora deploy --allow-schema-drift
  4. Load the dump back. --verify checks row parity and fails non-zero on a mismatch, so a partial import is loud.

    lunora import messages.ndjson --prod --url <url> --yes --verify
  5. Resume writes. Confirm placement in the Studio's Data page — its shard picker shows what actually landed where.

  6. Reclaim the root DO's copies. They are still there, and they are what you sharded to get rid of. In the Studio's Data page set the shard key to __root__, open the table, and use Clear table; it deletes through the schema-aware writer in bounded batches, so search/aggregate/rank companions stay consistent. This step is off the critical path — do it after writes resume, not during the window.

    Only for .shardBy(). A .global() migration has to clear the root DO before the schema edit, because afterwards the writer refuses the table outright — see Migrating a populated table to D1.

Admin reads and writes are addressed to a shard, not routed by the schema, which is why step 6 still reaches rows your application can no longer see. lunora migrate is the exception and cannot help here: a defineMigration transform runs inside one shard's Durable Object and can only replace the row it was handed, so it can neither write to another shard nor delete a row. It is the right tool for a backfill within a shard, and the wrong one for moving rows between shards.

Deleting rows returns their pages to that DO's SQLite freelist for reuse rather than shrinking the file, so the size the runtime reports may not fall right away. The headroom is real either way.

What stays in the root DO

Anything without .shardBy() or .global() stays in __root__. That keeps the common case (app config, feature flags, small per-app state) cheap and strongly consistent, with no routing overhead.

Going global

For cross-tenant data (identities, billing, account-wide audit logs) use .global(). The table lives in Cloudflare D1 instead of a DO. Reads can hit a regional replica; writes go through the primary. Pass the D1Session bookmark via the x-d1-bookmark header for read-your-writes consistency.

users: defineTable({ email: v.string(), name: v.string() }).global().index("by_email", ["email"], { unique: true });

Migrating a populated table to D1

Adding .global() to a table that already has rows moves it to D1, and — like adopting .shardBy() — the edit only changes where future rows go. The export/import round trip moves the existing ones, because lunora import routes each row by the deployed schema: a table the schema now declares .global() goes to D1 instead of a shard.

The order differs from the .shardBy() procedure, and the difference matters. There, the root DO's leftover copies are cleared last, from the Studio. Here they cannot be cleared at all once the edit lands: the shard's writer refuses to touch a table the schema calls global (GLOBAL_TABLE_NOT_EDITABLE). The orphaned rows are still listed under __root__ — the table list reads SQLite directly — so the Studio will let you try, and the clear is what surfaces the refusal. So the root copies have to go before the schema edit, while the table is still shard-local and the writer still owns it.

Writes are unavailable from step 1 until they resume at step 6. Export is keyset pagination over live reads, not a snapshot, and step 3 deletes the source rows — so a write that lands mid-procedure is lost. Snapshot first: Backups covers point-in-time recovery.

  1. Stop writes to the table.

  2. Dump it while it still lives in the root DO.

    lunora export --tables users --out users.ndjson --prod --url <url>
  3. Delete the root DO's rows now, before the schema edit — this is the only window in which anything can. In the Studio's Data page set the shard key to __root__, open the table, and use Clear table.

    Check inbound relations first. Clear table deletes through the schema-aware writer, so onDelete fires. Any table declaring r.one("users", { onDelete: "cascade" }) loses its rows too — and those are not in a --tables users dump, so they are gone for good. With onDelete: "restrict" the clear aborts part-way instead, leaving the table half-emptied. Either dump the dependent tables in the same export and re-import them, or drop the onDelete on inbound relations for the migration. Confirm the dump's row count before you clear: between this step and step 5 the export file is the only copy.

  4. Add .global() to lunora/schema.ts, then generate and deploy. The schema-drift gate blocks a shard-mode change on purpose, so pass --allow-schema-drift once you are running these steps deliberately.

    lunora migrate generate add_users_global   # keeps the committed D1 migration history in step
    lunora codegen && lunora deploy --allow-schema-drift
  5. Load the dump back. Rows now route to D1.

    lunora import users.ndjson --prod --url <url> --yes --verify
  6. Resume writes.

migrate generate is bookkeeping here, not a prerequisite: ensureMigrated provisions a .global() table and its declared indexes — including the UNIQUE ones — before the first read or write, so the import would succeed without it. Run it anyway to keep lunora/migrations/ and its snapshot in step with the schema, which is what the next generate diffs against.

Going global is a one-way trip for atomicity, not just a storage move: a mutation that writes both a sharded table and a .global() one is not writing both atomically. Make that call before the migration, not after — reversing it is the same round trip again.

Cross-shard reads

ctx.db.query("messages").collect() against a .shardBy() table that doesn't pin a shard is a fan-out: codegen routes it through the Query Coordinator Worker, which dispatches to every shard and merges the results. Avoid this in hot paths.

Securing shard access

Once you shard, the shard key is a client-supplied address, so one tenant could ask for another tenant's DO. Lunora default-denies cross-shard access: a request that names a shard other than the default (shardKey !== defaultShard, default __root__), or a fan-out, is rejected with 403 (FORBIDDEN_SHARD / FORBIDDEN_FANOUT) unless the worker opts in. A single-DO app never names a non-default shard, so it is unaffected.

Configure authorizeShard({ identity, shardKey }) on createWorker to gate it: return true to allow, false to reject. Derive the decision from the verified identity, so a caller can only reach shards it owns:

createWorker({
    shardDO: env.SHARD,
    // Once configured, this runs for EVERY shard the caller names — including the
    // default one (`__root__`, or your `defaultShardKey`), which is what an
    // unsharded table resolves to. So let the default through explicitly:
    // `identity?.userId === shardKey` alone would reject every unsharded RPC in
    // the app, because no user id equals `"__root__"`.
    authorizeShard: ({ identity, shardKey }) => shardKey === "__root__" || identity?.userId === shardKey,
    // Fan-out is a separate, more privileged grant when `authorizeShard` is set.
    authorizeFanOut: (identity) => identity?.role === "admin",
});

The simpler gate, when every shard is per-user and you only need "is this caller signed in", is ({ identity }) => identity?.userId !== undefined — it admits the default shard without a special case because it never looks at the key.

The alternative is allowUnauthenticatedShardAccess: true, which opts into open shard/fan-out access. Only reach for it when every table is protected by per-row RLS. The DO address is then no longer a security boundary; each row authorizes itself. The runtime logs a one-time security warning while it is on and no authorizeShard/authorizeFanOut is configured.