@lunora/testing is the toolkit for testing a Lunora app. It ships two
surfaces: an in-memory function harness (lunoraTest) that runs your
query / mutation / action functions against a fresh node:sqlite
database, with no Durable Object, no wrangler, and no network; and the
mail-catcher helpers re-exported from
@lunora/mail/testing for driving end-to-end email
flows.
pnpm add -D @lunora/testingIn-memory harness
lunoraTest(schema) runs the migrations against a fresh in-memory SQLite
database, builds the same ctx.db writer the real Durable Object builds, and
returns a harness whose methods execute a registered function's handler
directly. It follows the shape of Convex's convexTest, and all
of its methods share one database, so a write from a mutation is visible to a later
query.
import { defineSchema, defineTable, initLunora, v } from "lunorash/server";
import { lunoraTest } from "@lunora/testing";
import { afterEach, expect, test } from "vitest";
const schema = defineSchema({
messages: defineTable({
author: v.string(),
body: v.string(),
}),
});
// A self-contained test derives its builders from the inline schema. In your
// app you instead import `query` / `mutation` from `@/lunora/_generated/server`.
const { mutation, query } = initLunora.dataModel().create();
const send = mutation.input({ author: v.string(), body: v.string() }).mutation(({ ctx, args }) => ctx.db.insert("messages", args));
const list = query.query(({ ctx }) => ctx.db.query("messages").collect());
test("sends and lists a message", async () => {
const t = lunoraTest(schema);
await t.mutation(send, { author: "ada", body: "hi" });
expect(await t.query(list, {})).toHaveLength(1);
});Each lunoraTest(...) opens one in-memory database backed by a native handle.
Call t.close() (typically in an afterEach) to release it when a test
finishes. It is idempotent, so closing any view of the harness is safe.
const open: ReturnType<typeof lunoraTest>[] = [];
afterEach(() => {
while (open.length > 0) {
open.pop()?.close();
}
});lunoraTest options
lunoraTest(schema, options?) accepts these options. Each one is optional.
| Option | Default | Description |
|---|---|---|
enforceRls | true | Runs procedures through the RLS guard that production uses. Under .rls("required"), a procedure without .use(rls(...)) rejects with RlsRequiredError. Set false to restore the permissive behavior. |
env | undefined | The object that ctx.env returns in every context. |
fetch | throwing stub | The implementation ctx.fetch calls inside actions. See Injecting ctx.fetch. |
services | undefined | A fake for each lunora.config service an action calls. See Injectable ctx.services. |
functions | undefined | The registered functions by path, which the scheduler dispatches to. See Scheduler. |
queues / topics | undefined | The export names of the queues and topics that ctx.queues and ctx.topics accept. |
notify | undefined | The lunora/notify.ts default export that builds ctx.notify and ctx.push. |
Harness API
lunoraTest(schema) returns a TestHarness:
| Method | Description |
|---|---|
query(reference, args) | Run a registered query. Internal queries are rejected on this external surface. |
mutation(reference, args) | Run a registered mutation. Internal mutations are rejected on this external surface. |
action(reference, args) | Run a registered action. Internal actions are rejected on this external surface. |
query(inline) | Run an inline (ctx) => … with a QueryCtx. Read-only in the type only — at runtime it holds the same writer mutation gets, so a cast can write (outside any transaction, and without notifying subscribe). |
mutation(inline) | Run an inline (ctx) => … with a read-write MutationCtx. |
action(inline) | Run an inline (ctx) => … with an ActionCtx. |
run(fn) | Direct db access at mutation level (read + write), mirroring convexTest's run. |
subscribe(reference, args) | Async iterable yielding the query result now, then re-emitting after each mutation. |
scheduler | Controls for the in-memory scheduler (list / advance / runPending). |
queues | What handlers enqueued through ctx.queues (sent(name?) / clear()). |
topics | What handlers published through ctx.topics (published(name?) / clear()). |
notify | What handlers delivered through ctx.notify / ctx.push (sent(channel?) / clear()). |
withIdentity(identity) | Return a harness view sharing this db but reporting the given identity on ctx.auth. |
close() | Close the underlying in-memory SQLite database. Idempotent. |
Each query / mutation / action accepts either a registered function
reference plus its args, or a bare inline handler:
test("inline access to the db", async () => {
const t = lunoraTest(schema);
await t.run(async (ctx) => ctx.db.insert("messages", { author: "grace", body: "from run" }));
const count = await t.query(async (ctx) => {
const rows = await ctx.db.query("messages").collect();
return rows.length;
});
expect(count).toBe(1);
t.close();
});Auth and identity
withIdentity(identity) returns a harness view that shares the same database
but reports the given identity on ctx.auth. userId is the subject handlers
read from ctx.auth.userId; any additional fields are returned verbatim from
ctx.auth.getIdentity() (mirroring a decoded JWT). Writes made under the
scoped view persist for every accessor.
const whoAmI = query.query(({ ctx }) => ctx.auth.userId);
test("reflects the injected identity", async () => {
const t = lunoraTest(schema);
await expect(t.query(whoAmI, {})).resolves.toBeNull();
const scoped = t.withIdentity({ userId: "u1" });
await expect(scoped.query(whoAmI, {})).resolves.toBe("u1");
// A write under the scoped view is visible from the base harness.
await scoped.mutation(send, { author: "u1", body: "scoped" });
expect(await t.query(list, {})).toHaveLength(1);
t.close();
});This is how you test row-level security: drive a procedure with and without an identity and assert on the rows it returns or the errors it throws.
Internal functions
Internal functions (internalQuery / internalMutation / internalAction)
are unreachable from the external RPC boundary in production. The harness
enforces that contract: calling one through t.query / t.mutation /
t.action throws, while calling it through ctx.runQuery / ctx.runMutation /
ctx.runAction from another function succeeds. That mirrors prod's trusted
system dispatch.
import { initLunora, v } from "lunorash/server";
const { internalMutation, mutation } = initLunora.dataModel().create();
const internalSend = internalMutation.input({ author: v.string(), body: v.string() }).mutation(({ ctx, args }) => ctx.db.insert("messages", args));
// A public mutation routes through ctx.runMutation to the internal one.
const sendViaInternal = mutation.input({ author: v.string(), body: v.string() }).mutation(({ ctx, args }) => ctx.runMutation(internalSend, args));
test("internal functions are only reachable via ctx.run*", async () => {
const t = lunoraTest(schema);
// Direct call on the external surface is rejected.
await expect(t.mutation(internalSend, { author: "ada", body: "leak" })).rejects.toThrow("is an internal function");
// Reached through a public mutation, it runs.
await t.mutation(sendViaInternal, { author: "grace", body: "ok" });
expect(await t.query(list, {})).toHaveLength(1);
t.close();
});Injecting ctx.fetch
Actions can call ctx.fetch. Pass a fetch option to lunoraTest to replace
the throwing stub with your own implementation:
import { vi } from "vitest";
const fakeFetch = vi.fn<typeof globalThis.fetch>(async () => Response.json({ ok: true }));
const t = lunoraTest(schema, { fetch: fakeFetch });
// `ctx.fetch` inside any action now calls `fakeFetch`.Without the option, ctx.fetch throws the v1 error the first time an action
reaches for it.
Injectable ctx.services
Pass a services option with a fake per lunora.config service an action
calls. Like at runtime, only actions get ctx.services; a service with no fake
throws on use, naming the option:
const t = lunoraTest(schema, {
services: { documentParser: { fetch: vi.fn(async () => Response.json({ text: "…" })) } },
});Queues
List the export names of your lunora/queues.ts queues in the queues option
and mutations and actions get a recording ctx.queues. Sends go through
@lunora/queue's own validation, so a batch over 100 messages, a delay past
24 hours, or a queue name you did not list rejects as it would in production.
Each body is recorded as the consumer decodes it: encoded with its content type
(Cloudflare's default json, unless the send names one) and decoded back, so a
Date arrives as its ISO string and a BigInt rejects. Encoded sizes are checked
against the 128 KB message and 256 KB batch limits. The harness cannot read a
queue's content type from wrangler config, so name contentType on the send when
the queue is not json. Nothing is consumed — assert on what was enqueued:
const t = lunoraTest(schema, { queues: ["jobs"] });
const orderId = await t.mutation(placeOrder, { item: "lamp" });
expect(t.queues.sent("jobs")).toEqual([{ body: { kind: "fulfil", orderId }, queue: "jobs" }]);As in production a send is not transactional: one made by a mutation that then throws stays recorded even though the mutation's writes roll back.
Topics
List the export names of your lunora/queues.ts topics in the topics option
and mutations and actions get a recording ctx.topics, built by
@lunora/queue's own createTopicContext — so the batch cap, the delay
ceiling, the reserved key and a topic name you did not list reject as they do in
production. Bodies are encoded, decoded and size-checked exactly as for
queues. Each published message is recorded once, however many subscriptions
the app declares, and no subscription handler runs — assert on what was
published:
const t = lunoraTest(schema, { topics: ["signups"] });
const id = await t.mutation(signUp, { email: "ada@example.test" });
expect(t.topics.published("signups")).toEqual([{ body: { email: "ada@example.test", id }, topic: "signups" }]);Notifications and push
Pass your lunora/notify.ts default export as the notify option and every
context gets ctx.notify and its ctx.push alias. They are built by
@lunora/notify's own createNotify, so register / list / unregister /
broadcast behave as in production. Channels are wired exactly when production
wires them: each factory is called with the env option, and a channel whose
factory returns undefined is refused; a push to a device whose transport
(webPush / fcm) resolved to nothing fails, as it does in production.
The differences: deliveries are recorded and report accepted instead of
reaching a push service — so there is no send-time DNS re-check, retry or
circuit breaker — and subscriptions live in an in-memory store owned by the
harness, so the definition's store is never called. For Web Push,
payload.to is the JSON-stringified subscription, keys included.
import notify from "../lunora/notify";
const t = lunoraTest(schema, { notify });
const alice = t.withIdentity({ userId: "alice" });
await alice.mutation(registerDevice, { token: "alice-phone" });
await alice.action(notifyShipped, { title: "Shipped" });
expect(t.notify.sent("push")).toEqual([{ channel: "push", payload: expect.objectContaining({ title: "Shipped", to: "alice-phone" }) }]);Without the option, touching ctx.notify or ctx.push throws naming it, and
so does t.notify.sent() — a handler that swallows the error cannot make "no
notification was sent" pass by accident. queues and topics behave the same
way.
Scheduler
ctx.scheduler is a working fake with a virtual clock. Jobs enqueue
synchronously but only execute when you advance the clock. Map each scheduled
function path to its registered function via the functions option so the
scheduler can dispatch it:
const { mutation } = initLunora.dataModel().create();
const sendMessage = mutation.input({ author: v.string(), body: v.string() }).mutation(({ args, ctx }) => ctx.db.insert("messages", args));
const enqueue = mutation.mutation(({ ctx }) => ctx.scheduler.runAfter(5_000, "messages:send", { author: "ada", body: "hello" }));
test("scheduled mutation runs after advance", async () => {
const t = lunoraTest(schema, {
functions: { "messages:send": sendMessage },
});
await t.mutation(enqueue, {});
expect(t.scheduler.list()).toHaveLength(1); // queued, not yet run
await t.scheduler.advance(10_000); // tick past scheduledFor
expect(await t.query(list, {})).toHaveLength(1); // sendMessage ran
t.close();
});| Control | Description |
|---|---|
t.scheduler.list() | Snapshot of pending jobs in enqueue order. |
t.scheduler.advance(ms, options?) | Advance the virtual clock by ms, running every job due at or before the new now. Returns the count executed. |
t.scheduler.runPending(options?) | Run all pending jobs regardless of scheduledFor. Returns the count executed. |
The virtual clock is per-harness, so parallel tests are isolated. A path not in
the functions map (or pointing at a query) fails the job with
FUNCTION_NOT_FOUND; it is then retried and finally recorded on
t.scheduler.failures(), matching production's dead-letter park.
Two things that catch tests out. A failure with retries left is silently
re-enqueued with exponential backoff on the virtual clock, so failures()
stays empty until the whole schedule has elapsed — 30s + 60s + 120s + 240s +
480s = 930s of virtual clock at today's defaults, not one tick. And once it does
land, advance() / runPending() reject: a terminal failure is re-thrown
verbatim (several become an AggregateError). Pass
{ throwOnError: false } to suppress the re-throw and inspect failures()
instead:
await t.scheduler.advance(1_000_000, { throwOnError: false });
expect(t.scheduler.failures()).toHaveLength(1);Jobs run through
the same dispatch as ctx.runMutation, so they share the harness database, and
ctx.now inside a job is the virtual clock at fire time.
Subscriptions
t.subscribe(query, args) (or t.subscribe(inline)) returns an async iterable
that yields the query's current result immediately, then re-emits after every
mutation or run:
test("subscription re-emits after a mutation", async () => {
const t = lunoraTest(schema);
const sub = t.subscribe(list, {});
expect((await sub.next()).value).toHaveLength(0); // initial snapshot
await t.mutation(send, { author: "ada", body: "hi" });
expect((await sub.next()).value).toHaveLength(1); // updated snapshot
await sub.return(); // unsubscribe
t.close();
});Subscriptions are table-agnostic: any mutation triggers a re-evaluation. The
iterable buffers at most one pending result, so if several mutations fire
between two next() calls the next read reflects the most recent state.
v1 scope
ctx.storage, ctx.vectors, and ctx.workflows are clearly-throwing stubs. A
handler that touches one fails with a descriptive error the first time it
reaches for it:
const touchesStorage = mutation.mutation(({ ctx }) => ctx.storage.bucket("avatars"));
test("stubbed surfaces throw a clear error", async () => {
const t = lunoraTest(schema);
await expect(t.mutation(touchesStorage, {})).rejects.toThrow("ctx.storage is not available in the in-memory @lunora/testing harness (v1)");
t.close();
});The stub only throws when a handler actually touches it: functions that never reach for ctx.storage / ctx.vectors / ctx.workflows still run. Real R2
storage, Vectorize, and Cloudflare Workflows are deferred to a follow-up.
Mail-catcher helpers (E2E)
In lunora dev, @lunora/mail captures every outbound
email (sign-up verification, forgot-password, magic links) into the studio's
root-shard inbox instead of hitting a real provider. The mail-catcher helpers
read that inbox over the admin RPC, so a Playwright (or any HTTP) test can drive
"request reset → read the email → follow the link → set a new password"
deterministically. They are re-exported from @lunora/testing:
| Export | Description |
|---|---|
listCapturedMail | Read the captured-mail inbox, newest first. |
waitForMail | Poll the inbox until a message matching to (and optional subjectMatch) appears. |
extractLink | Pull the first link out of a captured message (html first, then text). |
InboxOptions | Type: options for listCapturedMail. |
WaitForMailOptions | Type: options for waitForMail. |
waitForMail takes the worker baseUrl, the admin bearer token (the worker
gates introspection behind LUNORA_ADMIN_TOKEN), the recipient to, and an
optional subjectMatch. It polls (default every 250ms, timing out after 10s)
and returns the matching CapturedMail, or throws on timeout. extractLink
takes an optional match substring to disambiguate the action link from a logo
or footer URL.
import { extractLink, waitForMail } from "@lunora/testing";
import { expect, test } from "vitest";
test("password reset email carries a working link", async () => {
const baseUrl = "http://localhost:8787";
const adminToken = process.env.LUNORA_ADMIN_TOKEN!;
// Trigger the flow (e.g. POST /api/auth/forgot-password).
await fetch(`${baseUrl}/api/auth/forgot-password`, {
body: JSON.stringify({ email: "alice@example.test" }),
headers: { "content-type": "application/json" },
method: "POST",
});
const mail = await waitForMail({
adminToken,
baseUrl,
to: "alice@example.test",
subjectMatch: "Reset your password",
});
const resetLink = extractLink(mail, { match: "/reset-password" });
expect(resetLink).toContain("/reset-password");
// → visit `resetLink`, set a new password, assert success.
});Options
waitForMail accepts everything in InboxOptions plus a few polling knobs:
| Option | Default | Description |
|---|---|---|
adminToken | — | Admin bearer token (LUNORA_ADMIN_TOKEN). |
baseUrl | — | App base URL, e.g. http://localhost:8787. |
to | — | Recipient address the message must be addressed to. |
subjectMatch | — | Only match a subject containing this substring. |
limit | 50 | Newest-N messages to read per poll. |
pollMs | 250 | Poll interval in ms. |
timeoutMs | 10000 | Give up after this many ms. |
fetch | globalThis.fetch | Inject a fetch implementation (e.g. a stub). |
Evals — scoring model output
evaluate runs a dataset through a producer and scores each output. The
built-in scorers are heuristic (containsScorer, absentScorer,
regexScorer, exactMatchScorer, keywordScorer) plus llmScorer, a large
language model (LLM) used as judge, whose judge is injected — a (prompt) => Promise<string>
you wire to your own model — so this package stays model-agnostic and the
judge is mockable. absentScorer is the one that can express a refusal: "the
write was never dispatched", "the tool above this tier was never offered".
import { evaluate, keywordScorer } from "@lunora/testing";
const result = await evaluate(
[{ expected: "shipped", input: "where is my order?" }],
async (input) => (await harness.run({ input, threadKey: input })).text ?? "",
[keywordScorer(["shipped"])],
);
expect(result.average).toBeGreaterThan(0.5);Retrieval scorers
Scoring only the final answer tells you it was wrong, not which half broke: whether retrieval never surfaced the right passage, or surfaced it and the model ignored it. Those have opposite fixes — chunking and hybrid search versus the prompt — so an answer-only eval sends you to the wrong one half the time.
The retrieval scorers judge the ranked ids instead. The producer returns
{ output, metadata } so it can report what it actually retrieved; that
metadata is merged over the case's own, which declares the gold set:
import { evaluate, groundednessScorer, mrrScorer, ndcgAtK, recallAtK } from "@lunora/testing";
const result = await evaluate(
[{ input: "how do I rotate keys?", metadata: { relevant: ["security#3"] } }],
async (input) => {
const { chunks, context } = await docs(ctx).retrieve(input);
return { metadata: { context, retrieved: chunks.map((chunk) => chunk.id) }, output: context };
},
[recallAtK(5), ndcgAtK(5), mrrScorer()],
);| Scorer | Measures |
|---|---|
recallAtK(k?) | Fraction of gold passages inside the top k. The ceiling on everything downstream. |
precisionAtK(k?) | Fraction of the top k that is gold. The counterweight — padding topK buys recall with noise. |
mrrScorer() | 1 / rank of the first gold passage. The metric that notices ordering, which recall cannot. |
ndcgAtK(k?) | Rank-discounted gain over the ideal arrangement. Gate on this when comparing retrieval strategies. |
groundednessScorer({judge}) | Whether the answer asserts only what the retrieved context supports. |
All of them fail closed: a case with no gold ids scores 0 with a reason, never 1. A mis-wired eval that reads as a perfect score is the most dangerous failure a quality gate has.
recallAtK and ndcgAtK answer different questions and you generally want
both. Recall@10 cannot distinguish a gold passage at rank 1 from one at rank 10,
but only the first survives a context-window trim — that difference is exactly
what normalized discounted cumulative gain (nDCG) and mean reciprocal rank (MRR) measure, and it is how you tell whether a reranker or a
hybrid leg actually helped rather than just moved results around.