@lunora/hyperdrive lets an action read and write an existing
Postgres/MySQL database through Cloudflare
Hyperdrive: pooled, cached
connections from the edge. It surfaces the binding's connection string and a
driver-agnostic ctx.sql client.
Integrate an existing database; don't replace the Lunora data layer. Hyperdrive points at a database Lunora has no visibility into, so two invariants the rest of Lunora relies on do not hold for it:
- Non-deterministic. A SQL query over the network is an external,
mutable read, exactly like
fetch. It is therefore forbidden inquery/mutationand available only onActionCtx. Thehyperdrive_outside_actionadvisor lint flags anyctx.sqlreached from a query or mutation. - Non-reactive. Live queries track writes to the Durable Object (DO)'s SQLite / D1. An
UPDATEissued over Hyperdrive produces no Lunora change event, so subscriptions will not re-run when external rows change.
Hyperdrive is the right tool for "read/write my legacy Postgres from an
action," and the wrong tool for "make my Postgres reactive." If you want
external data to be reactive, write a projection of it into a
defineSchema DO/D1 table (see Making external data
reactive).
Install
pnpm add @lunora/hyperdriveNo driver is bundled: postgres, pg, and mysql2 are heavy and the choice
is yours. They are declared as optional peer dependencies; install the one
you use:
pnpm add postgres # postgres.js → fromPostgresJs
# or
pnpm add pg # node-postgres → fromNodePg
# or
pnpm add mysql2 # mysql2 → fromMysql2Set up the binding
Importing @lunora/hyperdrive (or reading ctx.sql) is enough: lunora dev and
lunora deploy add a HYPERDRIVE binding to wrangler.jsonc for you, with a
placeholder id.
-
In dev,
ctx.sqlruns against an embedded Postgres (PGlite), with its data in.wrangler/state/lunora-pglite, so there is nothing to run. It starts underlunora dev,vite devandvite preview. It needs two optional packages, andlunora devsays so when they are missing:pnpm add -D @electric-sql/pglite @electric-sql/pglite-socketTo use your own database instead, set
CLOUDFLARE_HYPERDRIVE_LOCAL_CONNECTION_STRING_HYPERDRIVE(the last part is the binding name) orlocalConnectionStringon the binding. Neither is overridden. The embedded database speaks PostgreSQL: a project whose only SQL driver ismysql2gets a warning and must set that variable to a local MySQL of its own. A plainwrangler devthat Lunora does not start has no embedded database either: set the variable (orlocalConnectionString) there too. -
On deploy, set the origin's connection string and a token that may edit Hyperdrive (
CLOUDFLARE_API_TOKENwith Hyperdrive: Edit, plusCLOUDFLARE_ACCOUNT_ID):LUNORA_HYPERDRIVE_URL="postgres://user:pass@host:5432/db" lunora deploy # gitleaks:allow -- placeholder, not a real secretThe variable is
LUNORA_<BINDING>_URL. Lunora creates a config named after the deployed Worker (so--env staginggets its own) and the binding, or reuses one with that name that reaches the same database, and writes itsidintowrangler.jsonc. A same-named config for another database stops the deploy. The connection string is only sent to Cloudflare: it is never written to a file or logged. Once theidis real, later deploys need neither variable.
Prefer to manage it yourself? Run wrangler hyperdrive create my-db --connection-string=...
and put the id in the binding; Lunora then leaves it alone. Lunora validates the binding
(it errors when binding is missing).
Hyperdrive caches read results for up to a minute by default, and a write does not
invalidate them, so a ctx.sql read can miss a row your own action just wrote. For reads
that must be fresh, follow Cloudflare's advice and add a second config created with
--caching-disabled, bound under another name. A .global() backend always needs caching
off: see below.
The canonical recipe
ctx.sql is wired once, at the app level, not inside a handler: the
generated ctx.sql is a readonly property (assigning to it is a TS2540).
Codegen cannot build the client for you — createHyperdrive returns connection
info, and turning that into a SqlClient needs the driver you chose — so it
emits a sql config thunk instead. Wire it on the app builder:
import type { HyperdriveLike } from "@lunora/hyperdrive";
import { createHyperdrive, fromPostgresJs } from "@lunora/hyperdrive";
import { defineApp } from "../../lunora/_generated/app";
const app = defineApp<Env>()
.shard((env) => env.SHARD)
// Called once per shard construction, not per request.
.hyperdrive((env) =>
fromPostgresJs(async () => {
const { default: postgres } = await import("postgres");
return postgres(createHyperdrive(env.HYPERDRIVE as HyperdriveLike).connectionString, { fetch_types: false, max: 5 });
}),
)
.build();
export const ShardDO = app.ShardDO;Hand-composing the worker instead? The same thunk is createShardDO({ sql }):
export const ShardDO = createShardDO({
sql: (env) =>
fromPostgresJs(async () => {
const { default: postgres } = await import("postgres");
return postgres(createHyperdrive(env.HYPERDRIVE as HyperdriveLike).connectionString, { fetch_types: false, max: 5 });
}),
});The adapter takes a loader, not a driver: it runs on the first query and
its client is reused, so the driver stays out of the Worker's startup graph. A
static import postgres from "postgres" in the entry is evaluated at startup and
defeats this. A failed load rejects that query; the next one retries. For pg,
the loader may await client.connect() before returning the client.
Without the thunk ctx.sql is a stub whose every method throws a message
pointing back at this wiring.
Handlers then just read it. Only inside an action:
import { action, v } from "@/lunora/_generated/server";
export const importLegacyOrders = action.input({ orgId: v.string() }).action(async ({ ctx, args: { orgId } }) => {
// Read from external Postgres ($1, $2, … placeholders).
const orders = await ctx.sql.query<{ id: string; total: number }>("select id, total from orders where org = $1", [orgId]);
return orders;
});When the codegen detects ctx.sql usage it adds sql: SqlClient to ActionCtx
only (never QueryCtx or MutationCtx), with a JSDoc restating the
determinism/realtime caveat.
Drivers & placeholders
| Driver | Adapter | Placeholders |
|---|---|---|
postgres (postgres.js) | fromPostgresJs | $1, $2, … |
pg (node-postgres) | fromNodePg | $1, $2, … |
mysql2/promise | fromMysql2 | ? |
The package never rewrites SQL; use your driver's native positional syntax.
Build postgres.js with { fetch_types: false, max: 5 }, Cloudflare's recommended
Hyperdrive setup: max stays under the Workers limit on concurrent connections, and
fetch_types: false saves a round trip on connect. fromPostgresJs runs every statement
as a named prepared statement (unsafe's own default is unnamed, which Hyperdrive's
transaction pooling does not reliably support). Prepared reads are cacheable, so a
ctx.sql read through postgres.js can be served from Hyperdrive's query cache like one
through pg.
fetch_types: false leaves postgres.js without array types, so a JS array bound to
= ANY($1) goes over as a malformed literal. Bind a list as JSON text instead
(IN (SELECT jsonb_array_elements_text($1::text::jsonb)) with JSON.stringify(list)), or
drop the option if you rely on array params. Cast a JSON parameter $1::text::jsonb, not
bare $1::jsonb: postgres.js types the parameter from the server and would JSON-encode
your already-serialised string a second time.
fromMysql2 (and buildMysqlExec) call mysql2's query, not execute: Hyperdrive does
not support MySQL protocol-level prepared statements, so the ? values are escaped
client-side and the statement goes over as text. An array or object value binds as its
JSON text (mysql2 alone would expand it into SQL), and an undefined value throws.
Strings are quoted with a doubled '' for a quote and \\ for a backslash, so the server
must not run with NO_BACKSLASH_ESCAPES in its sql_mode (MySQL's default doesn't
include it): under that mode a backslash would be stored twice. A quote can't end the literal
under either mode. A ? inside a quoted string or a comment is left alone, but ?? is an
identifier placeholder (the value is quoted with backticks). Create the mysql2 connection with disableEval: true, which Workers require.
Telemetry
With an observability sink configured, ctx.sql is instrumented automatically
at the same level as ctx.db — the sink's instrumentDatabase field (see
Observability: database telemetry):
"summary"(default) foldssql.calls,sql.duration_ms,sql.errorsand a per-operation count (sql.op.SELECT,sql.op.INSERT, …) onto the dispatch's root span. These are separate fromctx.db'sdb.*counters, so time spent in your external database stays distinguishable from the shard's own SQLite."spans"adds one CLIENT span perquery(sql.SELECT, …), capped at 100 per dispatch (sql.spans_truncatedpast that), carryingdb.system.name(postgresqlforfromPostgresJs/fromNodePg,mysqlforfromMysql2,other_sqlfor a hand-built client),db.operation.nameanddb.response.returned_rows."off"hands the action your client untouched.
The operation is the statement's leading keyword only. Statement text and
parameters are never recorded, there is no db.collection.name, and a failed
query records its error type rather than the driver's message (outside
development), since Postgres and MySQL errors quote row values. SqlClient has
a single query method, so there are no transaction or batch spans — a
BEGIN/COMMIT you issue yourself is counted under its own keyword.
Queries you run through your own driver or ORM instance outside ctx.sql
are not seen. For a Drizzle instance on Hyperdrive,
cloudflare-drizzle-tracing's
instrumentDrizzle() adds Cloudflare-native drizzle.execute / transaction /
batch spans — be aware it records query text and parameters.
Making external data reactive
Lunora cannot observe external writes, so a subscription over a defineSchema
table won't re-fire when Postgres changes. To make external data reactive,
project it into a DO/D1 table inside the same action. That write is on
the change-feed, so live queries reading the projection re-run:
import { api } from "@/lunora/_generated/api";
import { action, v } from "@/lunora/_generated/server";
export const syncOrder = action.input({ id: v.string() }).action(async ({ ctx, args: { id } }) => {
const [row] = await ctx.sql.query<{ id: string; total: number }>("select id, total from orders where id = $1", [id]);
// This write is tracked — a `query` over `orders` re-runs for subscribers.
await ctx.runMutation(api.orders.upsert, { id: row.id, total: row.total });
});Per-agent shape ingest (multitenant Postgres → per-tenant DOs)
A common shape of the pattern above: a multitenant Postgres is the source of
truth, and you want each tenant (or agent) to work against only its own slice,
materialized into its own sharded Durable Object, with clients riding the same
live slice. .shardBy() gives each tenant a private DO + SQLite; pullSourceRows
(read side) and materializeExternalRows (write side) bridge the Postgres slice
into it; defineShape carries it to clients with no extra wiring.
// lunora/schema.ts — one DO per tenant, plus a shape clients subscribe to
export default defineSchema({
documents: defineTable({ title: v.string(), body: v.string(), orgId: v.string() }).shardBy("orgId").externallyManaged(), // rows are written by the ingest bridge, not user mutations
});
export const tenantDocs = defineShape({ table: "documents", where: () => ({}) });// The action pulls THIS tenant's slice (the shard key binds into the WHERE — the
// tenant-isolation boundary), then a mutation materializes it.
import { pullSourceRows } from "@lunora/hyperdrive";
import { api } from "@/lunora/_generated/api";
import { action, mutation, v } from "@/lunora/_generated/server";
import { materializeExternalRows } from "@lunora/shard-engine";
export const refreshTenant = action.input({ orgId: v.string() }).action(async ({ ctx, args: { orgId } }) => {
const docs = await pullSourceRows(ctx.sql, {
query: 'select id, title, body, org_id as "orgId" from documents where org_id = $1',
params: [orgId], // ← tenant scope. NEVER omit this on a sharded source.
map: (row) => ({ body: row.body, orgId: row.orgId, title: row.title }),
});
// One mutation, so the whole slice materializes atomically: a failure part
// way through leaves the tenant's DO on its previous snapshot, not half-updated.
await ctx.runMutation(api.documents.ingest, { orgId, docs });
});
export const ingest = mutation.input({ orgId: v.string(), docs: v.array(v.any()) }).mutation(async ({ ctx, args: { docs } }) => {
// Empty baseline = upsert-only (inserts + updates, no deletes). The materialized
// rows append to the CDC log, so `tenantDocs` subscribers are poked live.
await materializeExternalRows(ctx.db, docs, new Map(), { table: "documents" });
});Clients consume the same slice with the shape they'd use over any table:
const docs = useShape(api.shapes.tenantDocs, {}); // RLS-filtered, live, per-tenantRefresh on a schedule with @lunora/scheduler (ctx.scheduler.runAfter(...) /
a cron) so each tenant's slice stays current.
Tenant scoping is the correctness boundary. Per-shard SQLite isolation only controls where rows land, not what the query pulls. The shard key MUST bind into the source
WHERE(as a parameter), or every tenant's DO would replicate the whole table. Deletes: the empty-baseline call above is upsert-only; to propagate upstream deletes, pass the table's current membership as the baseline somaterializeExternalRowscan diff it.
Or skip the boilerplate: the declarative .source() modifier
The manual bridge above is the escape hatch. For the common case, declare the source on the table and Lunora runs the whole loop on the DO's alarm: full-pull diff, tenant scoping, and a poll cadence, with no action, mutation, or cron to write:
// lunora/schema.ts
export default defineSchema({
documents: defineTable({ title: v.string(), body: v.string(), orgId: v.string() })
.shardBy("orgId")
.source({
binding: "HYPERDRIVE_DOCS",
query: 'select id, title, body, org_id as "orgId" from documents where org_id = $1',
tenantBy: (shardKey) => [shardKey], // mandatory under .shardBy() — the tenant boundary
}),
});
export const tenantDocs = defineShape({ table: "documents", where: () => ({}) });You provide the driver once, on the app builder: one resolver for every sourced
binding (build the SqlClient exactly as above). Lunora memoizes it per binding and
polls each tenant's slice on the alarm. It is required — without it every tick
records no sourceClient resolved for binding "…" and the table stays empty.
// worker.ts
export default defineApp<Env>()
.shard((env) => env.SHARD)
.sourceClient((env, binding) =>
fromPostgresJs(async () => {
const { default: postgres } = await import("postgres");
return postgres((env[binding] as { connectionString: string }).connectionString, { fetch_types: false, max: 5 });
}),
)
.build();.sourceClient() appears on the builder only when your schema declares a
.source(...) table. Composing the shard DO by hand instead, pass the same resolver
as createShardDO({ sourceClient }).
Tenant scoping is enforced two ways: defineSchema throws at load if a sourced
.shardBy() table omits tenantBy (the runtime fail-safe), and the
external_source_unscoped advisor lint flags it earlier, at build time, in your
terminal + Studio. defineSchema likewise rejects combining .source() with
.global() (contradictory tiers). Clients consume the slice with
useShape(api.shapes.tenantDocs), with no client change, since it is an ordinary table.
Refresh cadence. Omit refresh to poll on every DO alarm tick (the floor), or
pass refresh: { everyMs } to throttle a large slice to at most one pull per
interval. refresh: "manual" tells Lunora not to auto-poll; use it when you
drive the refresh yourself (the manual pullSourceRows → materializeExternalRows
bridge above, on your own schedule).
Delete detection: mode. The default mode: "full-pull" reads the whole tenant
slice each tick and diffs it, so upstream deletes are observed for free, but it
costs a full read per tick. For a large, low-churn slice,
mode: "incremental" pulls only rows changed since a durable
watermark:
documents: defineTable({ title: v.string(), body: v.string(), orgId: v.string(), updatedAt: v.number() })
.shardBy("orgId")
.source({
binding: "HYPERDRIVE_DOCS",
mode: "incremental",
query: 'select id, title, body, org_id as "orgId", updated_at as "updatedAt" from documents where org_id = $1',
// cursor.query pulls only rows past the watermark — tenantBy params bind first ($1), the watermark last ($2).
cursor: {
column: "updatedAt",
query: 'select id, title, body, org_id as "orgId", updated_at as "updatedAt" from documents where org_id = $1 and updated_at >= $2 order by updated_at',
},
// Delete visibility — REQUIRED for incremental (pick one):
reconcileEveryMs: 3_600_000, // periodic full-pull sweep GCs upstream deletes, OR:
// softDeleteColumn: "deletedAt", // upstream tombstone column the cursor query returns
tenantBy: (shardKey) => [shardKey],
}),How it runs: the first poll (and every reconcileEveryMs sweep) does a full-pull to
seed membership + the watermark and GC deletes; every other tick binds the stored
watermark as the cursor query's trailing param and upserts the returned slice.
The watermark (the max cursor.column value seen) and the last-reconcile time persist
per (table, shard) in the DO's reserved __lunora_source_cursor table, so they
survive hibernation. Use >= in the cursor query (rows sharing a boundary timestamp
re-pull idempotently rather than being skipped).
Because an incremental slice can't see a delete (an absent row means "unchanged", not
"deleted"), incremental requires a delete-visibility path: reconcileEveryMs (a
periodic full-pull sweep) or softDeleteColumn (an upstream tombstone column the
cursor query returns, turned into a local delete; don't filter it out of the query).
defineSchema throws, and the external_source_incremental_no_delete_path advisor
lint fails the build, if an incremental source declares neither.
The cursor column must be strictly commit-monotonic: a value that only ever
increases as rows become visible, never assigned below the current max after a later
commit. A plain updated_at set from wall-clock time can violate this under clock
skew or a long transaction (a row commits with a timestamp below a watermark already
advanced past it), and an incremental slice would then miss it permanently. A
reconcileEveryMs sweep is the backstop that re-establishes such rows; softDeleteColumn
alone does not (it only catches deletes, not missed inserts), so prefer
reconcileEveryMs unless your cursor is provably monotonic (e.g. a gapless sequence
or a logical replication LSN).
Reactive .global() over Hyperdrive
The @lunora/hyperdrive/global subpath is the inverse trade-off: instead of an
escape hatch onto a DB Lunora doesn't own, it makes a Postgres/MySQL database a
reactive .global() storage backend, alongside D1. Lunora owns
the schema: a .global() table gets a real column-per-field layout and every
write routes through the shared store core, so live queries stay reactive with no
extra wiring.
You build the writer inside the Durable Object that hosts the .global() store
(the HYPERDRIVE binding is reachable there) and inject it as globalDb. Cache
the driver on the DO instance and rebuild it lazily after hibernation.
import { fromPostgresJs } from "@lunora/hyperdrive";
import { createPostgresGlobalCtxDb } from "@lunora/hyperdrive/global";
// The loader runs on the first statement, so postgres.js is not evaluated at Worker startup.
const globalDb = createPostgresGlobalCtxDb(
fromPostgresJs(async () => {
const { default: postgres } = await import("postgres");
return postgres(env.HYPERDRIVE.connectionString, { fetch_types: false, max: 5 });
}),
{ schema },
);A .global() write commits its row and companion rows in one transaction, so the
driver must be able to open one: a postgres.js client with begin(), or a mysql2
pool (createPool, not a single createConnection). Anything else is refused
with GLOBAL_TRANSACTION_UNSUPPORTED instead of committing statement by statement.
Turn Hyperdrive's query cache off
The Hyperdrive config behind a .global() store must have query caching
disabled. Hyperdrive caches read results by default (max_age 60 s plus 15 s
of stale_while_revalidate), and a write does not invalidate that cache. The
store's own reads would then lag its writes: an optimistic-concurrency re-read
returns the old version, so the conditional write misses and every retry rereads
the same cached row (spurious ConflictErrors), and change-log polls and
live-query re-runs serve rows up to 75 s old.
lunora deploy creates (or updates) its configs with caching off when the
schema has .global({ backend: "hyperdrive" }) tables. For a binding whose id is
already set, it doesn't change the config; with CLOUDFLARE_API_TOKEN and
CLOUDFLARE_ACCOUNT_ID in the environment it warns when caching is still on. A
config you made yourself needs it set once:
wrangler hyperdrive create my-db --connection-string="postgres://user:pass@host/db" --caching-disabled # gitleaks:allow -- placeholder
# or, for an existing config:
wrangler hyperdrive update <id> --caching-disabledLocal dev (localConnectionString, the embedded Postgres) has no cache, so it
won't reproduce the problem. Node and celld have no Hyperdrive: their
.global() store is SQLite (a local file on Node, a D1 binding on celld), so
this does not apply there.
Tables provisioned by an earlier version
The store creates missing tables and adds missing columns on its own, but it does not rewrite a column that already exists. Two gaps can remain on a table provisioned before a release changed the column it would declare:
-
A field that accepts
nullbut whose column isNOT NULL. Requiredv.any(),v.null(),v.literal(null)and unions with a nullable member used to be provisionedNOT NULL, so storingnullthere fails. Postgres drops the constraint itself at startup (a catalog-only change). MySQL does not: its only way to relax a column isMODIFY COLUMN, which restates the whole column and can rebuild the table. The store logs one warning per column with the exact statement to run instead, built from the column's current type and collation, for example:ALTER TABLE `notes` MODIFY COLUMN `note` longtext CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_ai_ci NULL;Run it once per table in a maintenance window.
-
MySQL collation. Text columns are now declared
utf8mb4_0900_bin, which compares byte for byte like SQLite and Postgres. A column created earlier keeps the server default (usuallyutf8mb4_0900_ai_ci, case- and accent-insensitive). Convert a table withALTER TABLE <t> CONVERT TO CHARACTER SET utf8mb4 COLLATE utf8mb4_0900_bin. Until then, aninfilter too long for one statement's placeholders compares under the column's own collation onutf8mb4_0900_ai_ciand_bincolumns. On any other legacy collation (utf8mb4_general_ci,utf8mb4_unicode_ci) MySQL refuses such a filter with "Illegal mix of collations" instead of answering it — convert the table.
Full-text search
.searchIndex() works on a Hyperdrive-backed .global() table exactly as it
does on D1: same tokenizer, same matching rules, same relevance order (see
Full-text search). Neither Postgres nor MySQL ships
SQLite's FTS5, so the store maintains a portable inverted companion instead: one
indexed (token, document, occurrences) row per distinct token, updated with
each row write and read back in a single indexed query. On Postgres its token
index declares text_pattern_ops so the prefix match of a query's final term
stays indexed under any collation; on MySQL both columns take the InnoDB key
prefix.
Rows that predate the index are backfilled a bounded page per request, with
progress recorded in __lunora_search_state, so a large table becomes
searchable progressively rather than stalling the first request after a deploy.
Pass staged: true to keep that work out of the request path entirely and drive
it yourself with backfillSqlSearchIndexes.
Ranking aggregates over every matching token row, so the 1024-document limit
bounds what you get back, not what the database reads. On Postgres you can trade
that away with .searchIndex({ strategy: "native" }), which stores a tsvector
per document and lets a GIN index answer the match: same documents, ordered by
the engine rather than by Lunora's scorer.
The convenience constructors cover the common path. For custom wiring, the lower-level pieces are also exported:
buildPgExec(client)/buildMysqlExec(load)turn a driver into the store'sSqlExec.postgresDialect/mysqlDialectare the engine dialects.createHyperdriveGlobalCtxDb({ engine, exec, ...storeOptions })is the general factory the two convenience constructors call.
Vector search on your own Postgres
.vectorize() normally writes through to a Cloudflare Vectorize binding. If you
have already moved the relational tier onto your own Postgres, that split is
awkward: full-text search runs natively (see strategy: "native" above) while
vectors alone still reach for a Cloudflare service.
createPgVectorIndex closes it. The shard takes
vectors: (env) => Record<string, VectorizeIndexLike>, and everything above that
— the write-through sync hook, ctx.vectors, codegen — is written against the
interface rather than the provider, so a pgvector index drops in where the
binding went:
import { createHyperdrive, fromPostgresJs } from "@lunora/hyperdrive";
import { createPgVectorIndex } from "@lunora/hyperdrive/global";
// Built once, not per request: `vectors` is called on every dispatch, and a fresh
// index object means a fresh provisioning memo — five extra round trips each time.
let index: ReturnType<typeof createPgVectorIndex> | undefined;
createShardDO({
vectors: (env) => {
index ??= createPgVectorIndex({
client: fromPostgresJs(async () => {
const { default: postgres } = await import("postgres");
return postgres(createHyperdrive((env as Env).HYPERDRIVE).connectionString, { fetch_types: false, max: 5 });
}),
dimensions: 768,
metric: "cosine",
name: "posts_search",
});
return { posts_search: index };
},
});Nothing else changes: .vectorize() in the schema, ctx.vectors.query(...) in a
function, and the automatic write-through sync all behave the same.
The table (__vec_<name>), the vector extension, an HNSW index in the metric's
operator class, and btree/GIN indexes on the two filterable columns are
provisioned lazily on first use — every
statement is IF NOT EXISTS, so it is safe against a live table. The connecting
role needs to be able to run CREATE EXTENSION vector; on a managed Postgres
where it cannot, install the extension once out-of-band and the statement becomes
a no-op.
Four deliberate differences from Vectorize. Metadata filters are JSONB
containment — equality for scalars, but for containers it is containment, not
equality: { tags: ["x"] } also matches a row whose tags are ["x", "y"], and
{ author: {} } matches every row carrying an author key. A filter value
JSON cannot represent (undefined, a function, a symbol) is rejected, because
dropping it would widen the filter to match every row. A comparison operator
like { views: { $gt: 10 } } and Vectorize's dot-addressed "author.role"
syntax throw for the same reason. returnMetadata: "indexed" behaves as
"all", since Postgres indexes the whole JSONB document — note this defeats
a deliberate default: ctx.vectors asks for "indexed" precisely so a query
that opted into nothing does not return every stored field. And dimensions
is fixed at first provisioning: CREATE TABLE IF NOT EXISTS no-ops on a
later change, so writes fail against the original width. Finally mutationId
is a local counter: Vectorize's is a handle for an async mutation queue,
while a Postgres write is already durable when its promise resolves.
Two ceilings worth knowing. dimensions is capped at 2000, pgvector's HNSW limit — rejected at construction rather than failing opaquely at CREATE INDEX
(Vectorize itself tops out at 1536, so most ports are unaffected). And name is capped at 34 characters, because the longest index name derived from it
(__vec_ + name + __ann_ + the operator class) has to stay under Postgres' 63-byte identifier truncation — past that, two indexes would collapse onto one
name and the second CREATE INDEX IF NOT EXISTS would silently no-op.
Pass the same metric the schema's .vectorize(...) declares. Nothing
cross-checks the two: mismatch them and the index is built for one operator and
queried with another, so the ordering is wrong for the declared metric and every
score is on a different scale — with no error at any layer.
The indexes on namespace and metadata are not an optimisation. pgvector
applies WHERE after scanning the ANN index, so without a usable index on the
filtered column the planner's only path is the approximate one, and a filter
matching a small slice of the table returns a short page with no error. Indexing
both lets the planner answer the filter exactly instead. (hnsw.iterative_scan is
pgvector's own answer to this, but it is a session setting, and a pooled client can
run the SET and the query on different sessions.)
metric picks the distance operator and the matching index operator class —
cosine (default), euclidean, or dot-product. score is reported the way
Vectorize reports it: cosine similarity (1 = identical), raw L2 distance
(0 = identical), or the NEGATIVE inner
dot product — Cloudflare's own convention, where a score of -1000 is more similar
than -500.
Non-goals
- No CDC / logical replication. Lunora does not ingest your Postgres write-ahead log; the projection pattern above is the supported path to reactivity.
- No
ctx.sqlinquery/mutation. Enforced by thehyperdrive_outside_actionadvisor lint. - No bundled driver / ORM. You own driver choice and lifecycle.
Public API
| Export | Purpose |
|---|---|
createHyperdrive(binding) | Lift connectionString + discrete parts off the binding |
fromPostgresJs(load) | Wrap a postgres.js client, built by load on first query, as a SqlClient |
fromNodePg(load) | Wrap a pg Client/Pool, built by load on first query, as a SqlClient |
fromMysql2(load) | Wrap a mysql2/promise connection/pool, built by load, as a SqlClient |
pullSourceRows(sql, { query, params, idColumn?, map? }) | Run a tenant query + project rows to Lunora docs (read side of the ingest bridge) |
projectSourceRow(row, { idColumn?, map? }) | Project one external row → a document with _id |
SqlClient, HyperdriveLike, HyperdriveConnection, PostgresJsLike, NodePgLike, Mysql2Like, ProjectOptions, PullSourceOptions | Type-only |
From @lunora/hyperdrive/global (reactive .global() backend):
| Export | Purpose |
|---|---|
createPostgresGlobalCtxDb(client, options) | Build a reactive Postgres .global() writer (globalDb) |
createMysqlGlobalCtxDb(load, options) | Build a reactive MySQL .global() writer (needs FOUND_ROWS) |
createHyperdriveGlobalCtxDb({ engine, exec, … }) | General factory the two convenience constructors call |
buildPgExec(client) / buildMysqlExec(load) | Wrap a driver as the store's SqlExec |
postgresDialect / mysqlDialect | The engine dialects |
createPgVectorIndex({ client, name, dimensions }) | A pgvector-backed index for ctx.vectors (no Vectorize binding); name is the SQL table identifier, not the vectors map key |
exportHyperdriveGlobalRows / importHyperdriveGlobalRows | The admin export/import of .global() rows (lunora export/import, migrate d1-to-hyperdrive and back); codegen wires them |
createHyperdriveGlobalIntrospector(engine, exec, schema) | The Studio's global data browser over the Hyperdrive database; codegen wires it |
readHyperdriveGlobalCdc / applyHyperdriveGlobalCdc | The global changelog read (admin CDC sync) and replay (point-in-time apply); codegen wires them |
See also
- Concepts: queries & mutations: why actions are the only non-deterministic context
- Concepts: advisors: the
hyperdrive_outside_actionlint - Concepts: real-time: what the change-feed tracks