@lunora/hyperdrive

@lunora/hyperdrive lets an action read and write an existing Postgres/MySQL database through Cloudflare Hyperdrive: pooled, cached connections from the edge. It surfaces the binding's connection string and a driver-agnostic ctx.sql client.

Install

pnpm add @lunora/hyperdrive

No driver is bundled: postgres, pg, and mysql2 are heavy and the choice is yours. They are declared as optional peer dependencies; install the one you use:

pnpm add postgres        # postgres.js  → fromPostgresJs
# or
pnpm add pg              # node-postgres → fromNodePg
# or
pnpm add mysql2          # mysql2        → fromMysql2

Set up the binding

1. Create a Hyperdrive config pointing at your origin database:

   wrangler hyperdrive create my-db --connection-string="postgres://user:pass@host:5432/db" # gitleaks:allow -- placeholder, not a real secret

   This prints an id.

2. Add the binding to wrangler.jsonc. Use localConnectionString so local dev (lunora dev) connects straight to your DB without the edge proxy:

   {
       "hyperdrive": [
           {
               "binding": "HYPERDRIVE",
               "id": "<the id from step 1>",
               "localConnectionString": "postgres://user:pass@localhost:5432/db", // gitleaks:allow -- placeholder, not a real secret
           },
       ],
   }

Lunora validates the binding (it errors when binding is missing and warns when id is empty, since a placeholder id can't connect), but it does not auto-provision the id: that's a remote resource only wrangler hyperdrive create can mint, so importing @lunora/hyperdrive surfaces a hint rather than writing the binding for you.

The canonical recipe

Construct your driver from the connection string and wrap it with the matching adapter to get a ctx.sql. Do this only inside an action:

import { createHyperdrive, fromPostgresJs } from "@lunora/hyperdrive";
import postgres from "postgres";

import { action, v } from "@/lunora/_generated/server";

export const importLegacyOrders = action.input({ orgId: v.string() }).action(async ({ ctx, args: { orgId } }) => {
    const { connectionString } = createHyperdrive(ctx.env.HYPERDRIVE);
    ctx.sql = fromPostgresJs(postgres(connectionString));

    // Read from external Postgres ($1, $2, … placeholders).
    const orders = await ctx.sql.query<{ id: string; total: number }>("select id, total from orders where org = $1", [orgId]);

    return orders;
});

When the codegen detects ctx.sql usage it adds sql: SqlClient to ActionCtx only (never QueryCtx or MutationCtx), with a JSDoc restating the determinism/realtime caveat.

Drivers & placeholders

The package never rewrites SQL; use your driver's native positional syntax.

Making external data reactive

Lunora cannot observe external writes, so a subscription over a defineSchema table won't re-fire when Postgres changes. To make external data reactive, project it into a DO/D1 table inside the same action. That write is on the change-feed, so live queries reading the projection re-run:

import { createHyperdrive, fromPostgresJs } from "@lunora/hyperdrive";
import postgres from "postgres";

import { api } from "@/lunora/_generated/api";
import { action, v } from "@/lunora/_generated/server";

export const syncOrder = action.input({ id: v.string() }).action(async ({ ctx, args: { id } }) => {
    const { connectionString } = createHyperdrive(ctx.env.HYPERDRIVE);
    ctx.sql = fromPostgresJs(postgres(connectionString));

    const [row] = await ctx.sql.query<{ id: string; total: number }>("select id, total from orders where id = $1", [id]);

    // This write is tracked — a `query` over `orders` re-runs for subscribers.
    await ctx.runMutation(api.orders.upsert, { id: row.id, total: row.total });
});

Per-agent shape ingest (multitenant Postgres → per-tenant DOs)

A common shape of the pattern above: a multitenant Postgres is the source of truth, and you want each tenant (or agent) to work against only its own slice, materialized into its own sharded Durable Object, with clients riding the same live slice. .shardBy() gives each tenant a private DO + SQLite; pullSourceRows (read side) and materializeExternalRows (write side) bridge the Postgres slice into it; defineShape carries it to clients with no extra wiring.

// lunora/schema.ts — one DO per tenant, plus a shape clients subscribe to
export default defineSchema({
    documents: defineTable({ title: v.string(), body: v.string(), orgId: v.string() }).shardBy("orgId").externallyManaged(), // rows are written by the ingest bridge, not user mutations
});

export const tenantDocs = defineShape({ table: "documents", where: () => ({}) });

// The action pulls THIS tenant's slice (the shard key binds into the WHERE — the
// tenant-isolation boundary), then a mutation materializes it.
import { createHyperdrive, fromPostgresJs, pullSourceRows } from "@lunora/hyperdrive";
import postgres from "postgres";

import { api } from "@/lunora/_generated/api";
import { action, mutation, v } from "@/lunora/_generated/server";
import { materializeExternalRows } from "@lunora/shard-engine";

export const refreshTenant = action.input({ orgId: v.string() }).action(async ({ ctx, args: { orgId } }) => {
    const { connectionString } = createHyperdrive(ctx.env.HYPERDRIVE);
    ctx.sql = fromPostgresJs(postgres(connectionString));

    const docs = await pullSourceRows(ctx.sql, {
        query: 'select id, title, body, org_id as "orgId" from documents where org_id = $1',
        params: [orgId], // ← tenant scope. NEVER omit this on a sharded source.
        map: (row) => ({ body: row.body, orgId: row.orgId, title: row.title }),
    });

    await ctx.runMutation(api.documents.ingest, { orgId, docs });
});

export const ingest = mutation.input({ orgId: v.string(), docs: v.array(v.any()) }).mutation(async ({ ctx, args: { docs } }) => {
    // Empty baseline = upsert-only (inserts + updates, no deletes). The materialized
    // rows append to the CDC log, so `tenantDocs` subscribers are poked live.
    await materializeExternalRows(ctx.db, docs, new Map(), { table: "documents" });
});

Clients consume the same slice with the shape they'd use over any table:

const docs = useShape(api.shapes.tenantDocs, {}); // RLS-filtered, live, per-tenant

Refresh on a schedule with @lunora/scheduler (ctx.scheduler.runAfter(...) / a cron) so each tenant's slice stays current.

Tenant scoping is the correctness boundary. Per-shard SQLite isolation only controls where rows land, not what the query pulls. The shard key MUST bind into the source WHERE (as a parameter), or every tenant's DO would replicate the whole table. Deletes: the empty-baseline call above is upsert-only; to propagate upstream deletes, pass the table's current membership as the baseline so materializeExternalRows can diff it.

Or skip the boilerplate: the declarative .source() modifier

The manual bridge above is the escape hatch. For the common case, declare the source on the table and Lunora runs the whole loop on the DO's alarm: full-pull diff, tenant scoping, and a poll cadence, with no action, mutation, or cron to write:

// lunora/schema.ts
export default defineSchema({
    documents: defineTable({ title: v.string(), body: v.string(), orgId: v.string() })
        .shardBy("orgId")
        .source({
            binding: "HYPERDRIVE_DOCS",
            query: 'select id, title, body, org_id as "orgId" from documents where org_id = $1',
            tenantBy: (shardKey) => [shardKey], // mandatory under .shardBy() — the tenant boundary
        }),
});

export const tenantDocs = defineShape({ table: "documents", where: () => ({}) });

You provide the driver once, when constructing the shard DO: one resolver for every sourced binding (build the SqlClient exactly as above). Lunora memoizes it per binding and polls each tenant's slice on the alarm:

createShardDO({
    sourceClient: (env, binding) => fromPostgresJs(postgres((env[binding] as { connectionString: string }).connectionString)),
});

Tenant scoping is enforced two ways: defineSchema throws at load if a sourced .shardBy() table omits tenantBy (the runtime fail-safe), and the external_source_unscoped advisor lint flags it earlier, at build time, in your terminal + Studio. defineSchema likewise rejects combining .source() with .global() (contradictory tiers). Clients consume the slice with useShape(api.shapes.tenantDocs), with no client change, since it is an ordinary table.

Refresh cadence. Omit refresh to poll on every DO alarm tick (the floor), or pass refresh: { everyMs } to throttle a large slice to at most one pull per interval. refresh: "manual" tells Lunora not to auto-poll; use it when you drive the refresh yourself (the manual pullSourceRows → materializeExternalRows bridge above, on your own schedule).

Delete detection: mode. The default mode: "full-pull" reads the whole tenant slice each tick and diffs it, so upstream deletes are observed for free, but it costs a full read per tick (bench ceiling ~10k rows). For a large, low-churn slice past that cap, mode: "incremental" pulls only rows changed since a durable watermark:

documents: defineTable({ title: v.string(), body: v.string(), orgId: v.string(), updatedAt: v.number() })
    .shardBy("orgId")
    .source({
        binding: "HYPERDRIVE_DOCS",
        mode: "incremental",
        query: 'select id, title, body, org_id as "orgId", updated_at as "updatedAt" from documents where org_id = $1',
        // cursor.query pulls only rows past the watermark — tenantBy params bind first ($1), the watermark last ($2).
        cursor: {
            column: "updatedAt",
            query: 'select id, title, body, org_id as "orgId", updated_at as "updatedAt" from documents where org_id = $1 and updated_at >= $2 order by updated_at',
        },
        // Delete visibility — REQUIRED for incremental (pick one):
        reconcileEveryMs: 3_600_000, // periodic full-pull sweep GCs upstream deletes, OR:
        // softDeleteColumn: "deletedAt", // upstream tombstone column the cursor query returns
        tenantBy: (shardKey) => [shardKey],
    }),

How it runs: the first poll (and every reconcileEveryMs sweep) does a full-pull to seed membership + the watermark and GC deletes; every other tick binds the stored watermark as the cursor query's trailing param and upserts the returned slice. The watermark (the max cursor.column value seen) and the last-reconcile time persist per (table, shard) in the DO's reserved __lunora_source_cursor table, so they survive hibernation. Use >= in the cursor query (rows sharing a boundary timestamp re-pull idempotently rather than being skipped).

Because an incremental slice can't see a delete (an absent row means "unchanged", not "deleted"), incremental requires a delete-visibility path: reconcileEveryMs (a periodic full-pull sweep) or softDeleteColumn (an upstream tombstone column the cursor query returns, turned into a local delete; don't filter it out of the query). defineSchema throws, and the external_source_incremental_no_delete_path advisor lint fails the build, if an incremental source declares neither.

The cursor column must be strictly commit-monotonic: a value that only ever increases as rows become visible, never assigned below the current max after a later commit. A plain updated_at set from wall-clock time can violate this under clock skew or a long transaction (a row commits with a timestamp below a watermark already advanced past it), and an incremental slice would then miss it permanently. A reconcileEveryMs sweep is the backstop that re-establishes such rows; softDeleteColumn alone does not (it only catches deletes, not missed inserts), so prefer reconcileEveryMs unless your cursor is provably monotonic (e.g. a gapless sequence or a logical replication LSN).

Reactive .global() over Hyperdrive

The @lunora/hyperdrive/global subpath is the inverse trade-off: instead of an escape hatch onto a DB Lunora doesn't own, it makes a Postgres/MySQL database a reactive .global() storage backend, alongside D1. Lunora owns the schema: a .global() table gets a real column-per-field layout and every write routes through the shared store core, so live queries stay reactive with no extra wiring.

You build the writer inside the Durable Object that hosts the .global() store (the HYPERDRIVE binding is reachable there) and inject it as globalDb. Cache the driver on the DO instance and rebuild it lazily after hibernation.

import { createPostgresGlobalCtxDb } from "@lunora/hyperdrive/global";
import postgres from "postgres";

const sql = postgres(env.HYPERDRIVE.connectionString);

const globalDb = createPostgresGlobalCtxDb({ query: (text, params) => sql.unsafe(text, params) }, { schema });

Full-text search

.searchIndex() works on a Hyperdrive-backed .global() table exactly as it does on D1: same tokenizer, same matching rules, same relevance order (see Full-text search). Neither Postgres nor MySQL ships SQLite's FTS5, so the store maintains a portable inverted companion instead: one indexed (token, document, occurrences) row per distinct token, updated with each row write and read back in a single indexed query. On Postgres its token index declares text_pattern_ops so the prefix match of a query's final term stays indexed under any collation; on MySQL both columns take the InnoDB key prefix.

Rows that predate the index are backfilled a bounded page per request, with progress recorded in __lunora_search_state, so a large table becomes searchable progressively rather than stalling the first request after a deploy. Pass staged: true to keep that work out of the request path entirely and drive it yourself with backfillSqlSearchIndexes.

Ranking aggregates over every matching token row, so the 1024-document limit bounds what you get back, not what the database reads. On Postgres you can trade that away with .searchIndex({ strategy: "native" }), which stores a tsvector per document and lets a GIN index answer the match: same documents, ordered by the engine rather than by Lunora's scorer.

The convenience constructors cover the common path. For custom wiring, the lower-level pieces are also exported:

• buildPgExec(client) / buildMysqlExec(connection) turn a driver into the store's SqlExec.
• postgresDialect / mysqlDialect are the engine dialects.
• createHyperdriveGlobalCtxDb({ engine, exec, ...storeOptions }) is the general factory the two convenience constructors call.

Non-goals

• No CDC / logical replication. Lunora does not ingest your Postgres write-ahead log; the projection pattern above is the supported path to reactivity.
• No ctx.sql in query/mutation. Enforced by the hyperdrive_outside_action advisor lint.
• No bundled driver / ORM. You own driver choice and lifecycle.

Public API

From @lunora/hyperdrive/global (reactive .global() backend):

See also

• Concepts: queries & mutations: why actions are the only non-deterministic context
• Concepts: advisors: the hyperdrive_outside_action lint
• Concepts: real-time: what the change-feed tracks