char/avril
(wip) why'd you have to go and make things so complicaaaaated?
git clone https://git.t4t.associates/char/avril
78f4748
main
avril:kv
avril:kv is an embedded persistent key-value store: named keyspaces of
ordered byte-string keys mapping to byte-string values, with cross-keyspace
atomic batches guarded by per-entry versionstamps. Think "a SQLite-shaped
hole for durable program state" — user profiles with a secondary email
index, a build cache, a job queue's bookkeeping — without leaving the
avril:std world.
The canonical API surface is the generated avril:kv declarations (from
exts/kv/src/public.ts); what follows describes the model, not the
signatures.
import { openKv } from "avril:kv" ; const db = await openKv ( "users.kv" , { access :"exclusive" , create :true }); const users = db . keyspace ( "users" ); await users . set ( "u1" , JSON . stringify ({ name :"Ada" })); console . log ( await users . getText ( "u1" )); await db . close ();
Storage is a redb file. The on-disk format is an internal detail, not a public contract: the engine and its encoding may change between releases, so don't open the file with other tools.
Opening
openKv(path | file URL, { access, create }) resolves path against the
working directory (a file: URL works too, via its href). create
defaults to true unless access is "read-only".
shared(default) opens and closes the file per operation. Cheap to hold and safe to share, but other processes may interleave between your operations — use atomic batches when that matters.exclusiveholds one read-write handle. Opening fails promptly if another process holds the file, so this is the right choice for servers, daemons, and hot paths that want stable, uncontended access.read-onlynever creates or modifies the file. Opening a missing file succeeds and simply reads empty.
Authority is capability-checked at open and re-checked live on every
operation, like file handles: narrowing capabilities with pledge() (even
pledge({ read: false }) after opening) makes subsequent operations on
already-open handles fail. close() is synchronous and idempotent;
await using db = await openKv(...) or an explicit
db[Symbol.asyncDispose]() awaits the actual close.
Keyspaces
A database holds any number of named keyspaces — partitions that behave
like one table each. db.keyspace(name) returns a lightweight handle
(identity-stable per database: the same name gives the same object);
db.keyspaces() lists names sorted; clear() empties one keyspace while
leaving the others alone, and a cleared keyspace keeps existing as an
empty space. Closing the database invalidates its keyspace handles.
Keys
A key is an arbitrary byte string: a string is its UTF-8 encoding, a
Uint8Array is copied as-is. "abc" and new Uint8Array([97, 98, 99])
name the very same key. The empty string and the empty byte array are
valid keys and sort before every other key.
Keys order lexicographically by byte. Strings therefore sort by their
UTF-8 encoding, and byte keys interleave with string keys purely by byte
content — there is no cross-type ordering because there are no types. A
shorter key sorts before any key extending it. list({ prefix }) scans
exactly the keys starting with those bytes, so prefix: "user:" scans the
whole "user:1", "user:2", … family while excluding "admin:1". An empty
prefix (the default) scans everything.
Limits, enforced with TypeError: strings must be well-formed (lone
surrogates rejected, not replaced) for both keys and values.
Values
A value is an arbitrary byte string, exactly as stored: get and list
return the raw bytes, byte-for-byte, with no decoding and no envelope.
set with a string stores its UTF-8 encoding; set with a Uint8Array
stores a copy of its bytes. getText decodes the stored bytes as UTF-8
(and throws a TypeError when they are not valid UTF-8).
Richer values are userland's job: serialize to bytes first — JSON.stringify
into a string, or a binary codec such as jsr:@char/cbor into a
Uint8Array — and store the bytes. What comes back is the same bytes.
Limits: values are capped at 4 MiB (RangeError). Anything that is
neither a string nor a Uint8Array — numbers, objects, functions, symbols,
undefined, null — is rejected with a TypeError.
Operations and versionstamps
get returns an entry { keyspace, key, value, versionstamp } or null;
getText is the same entry with the value decoded as UTF-8;
getMany answers in input order; set resolves with the new
{ versionstamp }; delete and clear resolve when committed (deleting
a missing key changes nothing, not even the versionstamp). Entry keys are
returned as Uint8Arrays. list takes prefix, start, end,
limit (default and cap 10000), reverse, and yields one snapshot per
iteration — breaking out early just stops yielding from the
already-materialized rows.
A versionstamp is a bigint: the database's commit sequence number for
the commit that last wrote the entry. Equality is the intended use; ordering
is also meaningful, but only between stamps of the same database. Every
write mints a new one, and all writes in one atomic commit share theirs.
Atomic batches
db.atomic((tx) => { ... }) runs check/set/delete across any of
the database's keyspaces in one transaction:
tx.check(space, key, stamp | null)asserts the entry's current stamp — or, withnull, that the key must not exist.- On any failed check the whole batch aborts:
{ ok: false }, and nothing is written — not even keyspace registrations. - Success resolves
{ ok: true, versionstamp }(shared by every write;nullwhen the batch was empty or check-only, which burns no stamp). - Repeated writes to one key resolve last-write-wins.
- Batches are capped at 1000 ops / 4 MiB total (keyspace names, keys and
values). The
txcall that would breach either cap throwsRangeError.
The callback must be synchronous: returning a promise, throwing, or
reusing the transaction afterwards all fail (TypeError on the async and
retained cases; a throw discards the batch and propagates). Each call
validates its arguments immediately; the ops are collected during the
synchronous callback and committed in one operation.
The canonical use is a secondary index maintained atomically with its primary data, with a conflict-retry loop on the versionstamp:
import { openKv } from "avril:kv" ; const db = await openKv ( "users.kv" , { access :"exclusive" , create :true }); const users = db . keyspace ( "users" ); const byEmail = db . keyspace ( "usersByEmail" ); async function createUser ( id :string , string , name :string ) :Promise < void > { const value = JSON . stringify ({ id, email, name}); for (;;) { if ( await byEmail . get ( ) !== null ) throw new Error ( `email taken: ${ } ` ); const result = await db . atomic (( tx ) => { tx . check ( byEmail , , null ); // still absent tx . set ( users , "user:" + id , value ); tx . set ( byEmail , , id ); }); if ( result . ok ) return ; // Lost a race: someone claimed the address first. Loop back and re-check. } } await createUser ( "u1" , "a@example.com" , "Ada" ); for await ( const entry of users . list ({ prefix :"user:" })) { console . log ( entry . key , entry . value ); } await db . close ();
Non-goals for v1
TTL/expiry, watch/change notification, queues, cross-keyspace listing,
backup/export, and a synchronous API. list is per-keyspace; there is no
way to scan the whole database in one call.
1# `avril:kv` 2 3`avril:kv` is an embedded persistent key-value store: named keyspaces of 4ordered byte-string keys mapping to byte-string values, with cross-keyspace 5atomic batches guarded by per-entry versionstamps. Think "a SQLite-shaped 6hole for durable program state" — user profiles with a secondary email 7index, a build cache, a job queue's bookkeeping — without leaving the 8`avril:std` world. 9 10The canonical API surface is the generated `avril:kv` declarations (from 11`exts/kv/src/public.ts`); what follows describes the model, not the 12signatures. 13 14``` ts 15import { openKv } from "avril:kv"; 16 17const db = await openKv("users.kv", { access: "exclusive", create: true }); 18const users = db.keyspace("users"); 19await users.set("u1", JSON.stringify({ name: "Ada" })); 20console.log(await users.getText("u1")); 21await db.close(); 22``` 23 24Storage is a redb file. The on-disk format is an internal detail, not a 25public contract: the engine and its encoding may change between releases, 26so don't open the file with other tools. 27 28## Opening 29 30`openKv(path | file URL, { access, create })` resolves `path` against the 31working directory (a `file:` URL works too, via its `href`). `create` 32defaults to true unless `access` is `"read-only"`. 33 34- `shared` (default) opens and closes the file per operation. Cheap to 35hold and safe to share, but other processes may interleave between your 36operations — use atomic batches when that matters. 37- `exclusive` holds one read-write handle. Opening fails promptly if 38another process holds the file, so this is the right choice for 39servers, daemons, and hot paths that want stable, uncontended access. 40- `read-only` never creates or modifies the file. Opening a missing file 41succeeds and simply reads empty. 42 43Authority is capability-checked at open **and** re-checked live on every 44operation, like file handles: narrowing capabilities with `pledge()` (even 45`pledge({ read: false })` after opening) makes subsequent operations on 46already-open handles fail. `close()` is synchronous and idempotent; 47`await using db = await openKv(...)` or an explicit 48`db[Symbol.asyncDispose]()` awaits the actual close. 49 50## Keyspaces 51 52A database holds any number of named keyspaces — partitions that behave 53like one table each. `db.keyspace(name)` returns a lightweight handle 54(identity-stable per database: the same name gives the same object); 55`db.keyspaces()` lists names sorted; `clear()` empties one keyspace while 56leaving the others alone, and a cleared keyspace keeps existing as an 57empty space. Closing the database invalidates its keyspace handles. 58 59## Keys 60 61A key is an arbitrary byte string: a `string` is its UTF-8 encoding, a 62`Uint8Array` is copied as-is. `"abc"` and `new Uint8Array([97, 98, 99])` 63name the very same key. The empty string and the empty byte array are 64valid keys and sort before every other key. 65 66Keys order lexicographically by byte. Strings therefore sort by their 67UTF-8 encoding, and byte keys interleave with string keys purely by byte 68content — there is no cross-type ordering because there are no types. A 69shorter key sorts before any key extending it. `list({ prefix })` scans 70exactly the keys starting with those bytes, so `prefix: "user:"` scans the 71whole `"user:1", "user:2", …` family while excluding `"admin:1"`. An empty 72prefix (the default) scans everything. 73 74Limits, enforced with `TypeError`: strings must be well-formed (lone 75surrogates rejected, not replaced) for both keys and values. 76 77## Values 78 79A value is an arbitrary byte string, exactly as stored: `get` and `list` 80return the raw bytes, byte-for-byte, with no decoding and no envelope. 81`set` with a string stores its UTF-8 encoding; `set` with a `Uint8Array` 82stores a copy of its bytes. `getText` decodes the stored bytes as UTF-8 83(and throws a `TypeError` when they are not valid UTF-8). 84 85Richer values are userland's job: serialize to bytes first — `JSON.stringify` 86into a string, or a binary codec such as `jsr:@char/cbor` into a 87`Uint8Array` — and store the bytes. What comes back is the same bytes. 88 89Limits: values are capped at 4 MiB (`RangeError`). Anything that is 90neither a string nor a `Uint8Array` — numbers, objects, functions, symbols, 91`undefined`, `null` — is rejected with a `TypeError`. 92 93## Operations and versionstamps 94 95`get` returns an entry `{ keyspace, key, value, versionstamp }` or `null`; 96`getText` is the same entry with the value decoded as UTF-8; 97`getMany` answers in input order; `set` resolves with the new 98`{ versionstamp }`; `delete` and `clear` resolve when committed (deleting 99a missing key changes nothing, not even the versionstamp). Entry keys are 100returned as `Uint8Array`s. `list` takes `prefix`, `start`, `end`, 101`limit` (default and cap 10000), `reverse`, and yields one snapshot per 102iteration — breaking out early just stops yielding from the 103already-materialized rows. 104 105A versionstamp is a `bigint`: the database's commit sequence number for 106the commit that last wrote the entry. Equality is the intended use; ordering 107is also meaningful, but only between stamps of the same database. Every 108write mints a new one, and all writes in one atomic commit share theirs. 109 110## Atomic batches 111 112`db.atomic((tx) => { ... })` runs `check`/`set`/`delete` across any of 113the database's keyspaces in **one** transaction: 114 115- `tx.check(space, key, stamp | null)` asserts the entry's current stamp — 116or, with `null`, that the key must not exist. 117- On any failed check the whole batch aborts: `{ ok: false }`, and nothing 118is written — not even keyspace registrations. 119- Success resolves `{ ok: true, versionstamp }` (shared by every write; 120`null` when the batch was empty or check-only, which burns no stamp). 121- Repeated writes to one key resolve last-write-wins. 122- Batches are capped at 1000 ops / 4 MiB total (keyspace names, keys and 123values). The `tx` call that would breach either cap throws `RangeError`. 124 125The callback must be synchronous: returning a promise, throwing, or 126reusing the transaction afterwards all fail (`TypeError` on the async and 127retained cases; a throw discards the batch and propagates). Each call 128validates its arguments immediately; the ops are collected during the 129synchronous callback and committed in one operation. 130 131The canonical use is a secondary index maintained atomically with its 132primary data, with a conflict-retry loop on the versionstamp: 133 134``` ts 135import { openKv } from "avril:kv"; 136 137const db = await openKv("users.kv", { access: "exclusive", create: true }); 138const users = db.keyspace("users"); 139const byEmail = db.keyspace("usersByEmail"); 140 141async function createUser(id: string, email: string, name: string): Promise<void> { 142const value = JSON.stringify({ id, email, name }); 143for (;;) { 144if (await byEmail.get(email) !== null) throw new Error(`email taken: ${email}`); 145const result = await db.atomic((tx) => { 146tx.check(byEmail, email, null); // still absent 147tx.set(users, "user:" + id, value); 148tx.set(byEmail, email, id); 149}); 150if (result.ok) return; 151// Lost a race: someone claimed the address first. Loop back and re-check. 152} 153} 154 155await createUser("u1", "a@example.com", "Ada"); 156 157for await (const entry of users.list({ prefix: "user:" })) { 158console.log(entry.key, entry.value); 159} 160 161await db.close(); 162``` 163 164## Non-goals for v1 165 166TTL/expiry, watch/change notification, queues, cross-keyspace listing, 167backup/export, and a synchronous API. `list` is per-keyspace; there is no 168way to scan the whole database in one call.