RFC 0004: Commit and transaction protocol

Date: 2026-07-09

Summary

Defines how a DuckLake catalog mutation becomes a durable, atomic commit in moraine: how a committer reads the head snapshot, allocates ids, detects conflicts against concurrent commits, and lands everything in a single atomic SlateDB write. Builds directly on RFC 0002’s atomicity invariant (one commit = one batch) and treats RFC 0005 inlined writes as first-class commit participants. The design is optimistic and single-writer-oriented: commits race on the head pointer under SlateDB’s transactional write-write conflict detection, disjoint concurrent commits retry internally, and genuine conflicts surface as a typed error for the caller — DuckLake or the application — to re-drive. The protocol has two front doors over one commit core: the verb/closure API of RFC 0003, whose members are closures moraine re-runs, and the staged-row path of RFC 0006, whose members are rows DuckLake authored against the head it read. Both join the same batch, and both are re-driven internally when they lose a benign race — a verb member by re-running its closure, a staged member by rebasing the head-derived values DuckLake stamped on it, which is the re-derivation DuckLake’s own retry would perform one layer up. A true conflict does the same on both: a typed error to whoever authored the operation. This RFC specifies both doors and the rebase that lets them share one batch.

Both front doors land commits as one atomic SlateDB batch over the shared core. Multi-statement, cross-table ACID transactions work end to end through DuckLake: a BEGIN … COMMIT spanning tables stages every statement’s writes into one moraine staged tx (opened lazily on the first write, reused across the transaction) and commits them as one snapshot; ROLLBACK discards them and mints none (ducklake_load.rs’s ducklake_multi_statement_transaction_commits_atomically pins that two writes across two tables advance the head by exactly one snapshot).

Concurrent DuckLake transactions are reachable, and they race: several connections on one DuckDB instance each hold a DuckLake transaction over one attached lake, and their commits overlap (test/sql/commit_concurrency.test). A second read-write process does not race — it fences (Topology). The staged-row path once surfaced every lost race to DuckLake, whose loop re-drives after a jittered sleep from 100 ms. Measured end to end (cargo xtask bench --workloads concurrent_connections), that composition forfeited most of the batching the verb path gets: at eight writers half the commits lost at least one race, each loss paid the sleep and a further flush, and commits per flush saturated near three where the verb path reaches the writer count. Removing the sleep alone livelocks at sixteen writers — per-commit races on one key with instant retry starve writers until DuckLake’s budget runs out. The design below batches staged commits and re-drives them inside the catalog, so the sleep is never entered for them.

Goals

Every commit is exactly one SlateDB WriteBatch (RFC 0002). Snapshot allocation, id allocation, conflict detection, and the write itself all fit inside that envelope — no read-modify-write spanning batches.
Inlined inserts/deletes (RFC 0005) ride the same batch as their catalog metadata. Inlining adds no round trips to the commit path.
On both front doors, concurrent commits touching disjoint tables — and concurrent pure appends to the same table — both succeed without caller involvement; incompatible commits resolve to a typed conflict. The staged-row path (RFC 0006) reaches this by rebasing DuckLake’s head-derived values rather than re-running anything of DuckLake’s, and only for the commit shapes the rebase re-derives exactly; every other shape surfaces its lost race to DuckLake as before.
Acknowledgement waits for WAL durability and checkpoint publication. Group commit amortizes both across transactions; publication is not per query.
Faithful to DuckLake’s own concurrency semantics — moraine implements DuckLake’s conflict model, it does not invent a stricter or looser one.

Non-goals:

Multi-writer coordination. The supported topology is single-writer, many-readers (see Topology). Concurrent commit coordination across processes is out of scope; the escape hatch is funneling commits through one process.
Key layout and value codec — RFC 0002.
Inlined data record formats — RFC 0005. This RFC covers only how inlined writes participate in a commit.
Snapshot expiry / GC of history and flushed inline data — RFC 0007.

Background

RFC 0002 established the keyspace: sys/head holds the latest committed snapshot_id; the snapshot subspace is append-only snapshot records; current and history split live from ended entity versions; commits are atomic WriteBatch writes under a single fenced SlateDB writer.

SlateDB (pinned at 0.14.x) provides the concrete primitives this protocol uses, verified against its source rather than assumed from prose: Db::begin(IsolationLevel::Snapshot) opens a transaction whose commit performs write-write conflict detection on the keys it writes and fails with a typed TransactionConflict; Db::snapshot() gives a pinned consistent read handle (RFC 0009); DbReader opens a store read-only without becoming a writer; and writer fencing is a monotonic writer_epoch bumped by manifest CAS on Db::open. There is no bespoke key-level compare-and-swap API, and none is needed — the “head CAS” in this RFC is realized as conflict detection on sys/head (see the commit sequence).

DuckLake allocates ids from counters: next_catalog_id and next_file_id live in the ducklake_snapshot record; next_row_id is per-table in ducklake_table_stats (next_row_id += record_count on insert), preserved across UPDATE and compaction for row lineage. The ducklake_snapshot record also carries schema_version — a counter DuckLake advances only when a commit changes the catalog schema, and that DuckDB uses as its schema-cache key. This RFC defines the protocol that maintains all of these under optimistic concurrency.

Design

Checkpoint publication before acknowledgement

Demand-loaded independent readers use the rotating checkpoint readers in RFC 0009. A successful commit acknowledgement guarantees that the writes are durable and a published checkpoint includes them. Direct writes share the flush pacer’s publisher; commit-log writes wait for publication covering the won slot before reporting success.

For each commit group:

Submit its atomic catalog batch and await WAL durability.
Publish a durable checkpoint through the writer’s existing Db::create_checkpoint(CheckpointScope::Durable, ...) API. Publication must finish before success is reported; requesting it is insufficient.
Acknowledge every group covered by the completed publication. Groups whose writes became durable later await a subsequent publication.

The checkpoint adds manifest metadata, not another catalog mutation or DuckLake snapshot. The catalog batch remains the atomicity boundary. Readers discover published checkpoints and retain their own leases; discovery may lag publication, so a reader requiring a particular commit position verifies that its chosen checkpoint covers that position.

If publication fails after the catalog write became durable, retry publication without resubmitting the mutation, or return CommitOutcomeUnknown for reconciliation. Cancellation and writer fencing after submission must also preserve the potentially committed outcome; neither authorizes replaying a mutation whose durable write may exist. Recovery publishes a checkpoint covering recovered durable progress before acknowledging further groups.

Commit-log clients need the same coverage: a durable slot can precede its fold into SlateDB. Their acknowledgement gate must await a published checkpoint covering that slot, including fold progress or recovery when needed. A temporary publisher replays the log and closes without an L0 flush, so publication does not require room for another L0 file. A coalesced group advances the batch counter once, even when its members mint several DuckLake snapshot ids; otherwise it would skip a log slot.

Validation must exercise publication stalls and failures after WAL durability, cancellation, recovery after writer loss, grouped acknowledgements, and readers requiring a commit position newer than their discovered checkpoint. No successful acknowledgement may escape before the covering checkpoint is published, and no publication-only failure may re-run the catalog mutation.

Topology

Single writer, many readers. Readers open the store through SlateDB’s DbReader — a handle that follows the manifest, never becomes a writer, and never participates in fencing (only Db::open and the compactor bump manifest epochs); there is no bound on their number. Writers are expected to be a single logical committer holding the one read-write Db.

One source-verified caveat keeps “read-only” from being oversimplified: DbReader::open in latest mode writes a checkpoint into the manifest (a manifest CAS update) and maintains it from a background poller for the reader’s lifetime. So a reader does not fence and does not write data, but it does need write access to the manifest object and generates manifest traffic, and its checkpoint pins SSTs against SlateDB’s own GC for the checkpoint lifetime. The default “follow latest” mode trades that small manifest write for consistent WAL-inclusive reads.

Deployments whose reader credentials cannot write at all open against an existing checkpoint instead. A checkpoint id given at open selects a DbReader that resolves a fixed cut: it neither creates a checkpoint, nor replays WALs, nor polls the manifest, so the open and everything after it write nothing whatsoever — verified by opening one against a store that refuses every write and reading through it. The cost is the other half of the trade: a pinned reader never sees a commit made after its checkpoint, however long it stays open. Checkpoints are minted by the writer (which alone can write the manifest) and released explicitly or by their lifetime, since one that is neither pins the objects it references against SlateDB’s garbage collection for as long as it exists.

Be precise about what fencing does, because its direction is the opposite of a lock: SlateDB’s writer_epoch means the newest writer wins. A second process opening the store read-write bumps the epoch and fences the incumbent — the first writer’s next durable write fails, not the second’s. Safety is absolute (a fenced writer writes nothing, never corrupts — RFC 0011’s FencedWriterResumes), but availability is not: two processes attaching read-write do not degrade to “second one fails,” they take turns killing each other’s committer. The operational rule follows: exactly one process attaches read-write; every other process attaches read-only (DbReader), and RFC 0006 exposes a read-only attach option so a DuckDB process can opt into the reader role explicitly.

This is a real product limitation relative to the multi-client SQL catalogs DuckLake otherwise uses — many DuckDB processes writing through one PostgreSQL catalog is the common lakehouse shape, and moraine does not support it: there is no funnel component in the DuckLake deployment model, so “run one long-lived committer process and route commits through it” is an escape hatch for embedding hosts, not something a fleet of independent DuckDB clients gets for free. This constraint is stated here once, surfaced in README-level documentation, and holds for the only implemented topology. A replacement topology is specified with the separate commit-log library; no multi-writer mode exists in the current implementation.

Submission outcomes

The coalescer distinguishes a forming batch from a submitted one. Dropping a forming batch proves that it wrote nothing; its members may retry. A submitted batch reports either an acknowledged commit or a rejected head race. A write error or a task that exits without acknowledgement returns Error::CommitOutcomeUnknown to every member, without replaying their closures. A fenced writer retains the terminal Error::Fenced result and must be reattached before it can write. The staged path returns the same error for an unacknowledged write.

The caller must retain external files and reconcile the operation against catalog state before resubmitting an unknown outcome. There is no durable client operation id that makes arbitrary resubmission idempotent. A changed head alone does not resolve a particular operation’s fate. Cancellation of the waiting caller leaves a submitted batch running; RFC 0006 defines how that uncertainty crosses the C ABI.

Bootstrap — the commit that creates the store

Opening a store that has never been written is itself a commit-shaped operation and rides the same machinery. On open, the catalog reads sys/format through a transaction:

Present → validate: an unknown format version is refused (Corruption today; Migration once RFC 0015’s variant exists), and a present sys/migration marker is refused outright (RFC 0002 reserves that check from format v1).
Absent (empty store) → bootstrap: stage one atomic batch writing sys/format (format version 1 plus the writing moraine version), snapshot 0 (empty catalog: no entities, counters at their initial values so the first allocation yields id 0, schema_version 0, empty changes_made), and sys/head → 0 at batch count 0, then commit it under the same head conflict detection as any commit. A same-process race between two bootstrapping opens arbitrates on the head write like any other commit — the loser re-reads and finds the initialized store; across processes the single-writer topology and fencing already exclude a second writer.

Bootstrap mints the default main schema in the same batch, matching the initial catalog state a fresh DuckLake metadata store carries: a moraine store is DuckLake-attachable from birth, without a DuckLake-side bootstrap pass (whose exists-probe always succeeds against moraine — see the extension-surface RFC). The main row’s id comes from the ordinary counter, so the first user allocation follows it.

Mutation-sized verb preparation

Demand-loaded commit translation reads changed entities and the dependency ranges needed for validation, applying staged changes through an operation-owned overlay. Allocation, unique constraints, conflict detection, read-your-writes, retry classification, and atomic publication retain their existing semantics. Cold small mutations must not load unrelated catalog metadata. A partial operation view must never be installed as a complete catalog snapshot.

Public asynchronous verb callbacks begin with only the head and counters. Each operation loads its dependency keys or ranges, keeping loaded-key identity so subsequent dependency reads cannot overwrite staged changes. Original and staged projections grow together; absent point reads are also remembered. Pending group writes are folded before the next callback starts. A commit-log reader’s head is checked before and after the callback, and a move refuses the attempt. Eager verb preparation exists only as a unit-test reference.

Verb transactions clone only their operation-owned CatalogSnapshot maps. Mutations copy affected tree paths; untouched maps and nested subtrees remain shared with the premise and held readers. Group formation and folding use the same representation, so each member avoids copying unrelated catalog rows.

The verb diff discovers changed ids through ordered-tree differences that skip shared nodes, descending into changed table-scoped maps. Those ids use the same history/version staging as the staged path’s explicit touched ids. Debug builds compare both paths against a whole-catalog diff to detect omitted writes. Inserts, deletes, cascades, and chained edits retain the existing write order and history semantics. No additional mutation journal is required.

Require imbl 7.0.2 or newer: the diff iterator in 6.0 through 7.0.1 missed an update at a shared-leaf boundary (64 sequential keys, then replace key 24). A missed diff would omit the entity write from a successful release-mode commit. This is a correctness constraint, not a performance choice. A saved property-test seed pins this failure for future dependency upgrades.

Upstream issue #161 reported the failure and fix #166, which also corrected map equality and deletion results from the same cursor bug, merged on 2026-09-09 and shipped in 7.0.2. The Cargo.toml floor is that release, so a fresh resolve cannot pick a broken 7.0.x. Any upgrade must still pass the full-diff equivalence and shared-boundary regressions.

Index maintenance resolves affected definitions through an in-memory index-id-to-table lookup. A known current format floor bypasses feature scans. These indexes and shared nodes are derived memory state, with no new on-disk encoding or migration. Commit conflict detection and WAL durability are unchanged.

For a warm premise, a fixed metadata mutation pays for changed tree paths and its own encoded writes, rather than unrelated tables or files. Explicit whole-table operations, large individual records, and projection readers retaining a copy-on-write dump are outside this bound. The release benchmark in crates/moraine/examples/verb_commit_bench.rs varies unrelated tables and files while updating one table’s statistics. It reports process CPU, allocation calls and bytes, total latency, durability-phase latency, and object PUT time separately. Debug builds intentionally retain the full-diff correctness oracle.

The commit sequence

A commit takes a set of staged catalog mutations (entity inserts, entity ends, inlined writes) and lands them atomically:

Load head. Open the SlateDB transaction, then read sys/head → snapshot id N, the snapshot N record for the global counters, and the tstat records for any tables this commit inserts rows into — all through the transaction handle, so the premise view and the conflict-detection window share one start sequence and no commit can land unseen between them.
Allocate ids locally by bumping in-memory copies of the counters: next_catalog_id/next_file_id from the snapshot record, next_row_id from each touched table’s tstat. No store writes yet.
Stage the batch. Assemble one atomic write set:
- new entity records into current (and, for ended versions, delete the current key + write the history key — RFC 0002);
- inlined inline/insert chunks and inline/inline_delete/inline/file_delete records (RFC 0005);
- updated tstat records carrying advanced next_row_id;
- the new snapshot record N+1 with advanced global counters, schema_version (bumped or carried forward per the rule below), and merged snapshot_changes;
- the sys/head update N → N+1, with the batch count incremented.
Commit under head conflict detection. The staged writes commit as one SlateDB transaction (begin(IsolationLevel::Snapshot)) opened at step 1. Every batch writes sys/head — including a maintenance batch, which mints no snapshot and so reuses the standing id while still moving the batch count (RFC 0002) — so SlateDB’s write-write conflict detection on that one key is the head CAS: if a concurrent commit advanced the head since this transaction began, commit() returns TransactionConflict and nothing was written — go to conflict handling. If it lands, the commit is durable once the WAL flush lands — done. (This is deliberately not a bespoke compare-and-swap: SlateDB exposes no key-level CAS API, and the transaction primitive it does expose is exactly the required shape. A transaction’s write set is applied as one atomic batch, so RFC 0002’s one-commit-one-batch invariant is preserved verbatim. Two source-verified mechanics worth knowing: the conflict check runs inside SlateDB’s single batch-writer event loop immediately before the write is sequenced, so check and apply cannot interleave; and the detection state is in-process — which is sufficient precisely because the topology admits one writer process, with cross-process safety carried by fencing, not by conflict detection. Plain non-transactional writes are tracked too, modeled as single-op transactions, so a stray direct write cannot slip past a concurrent commit’s conflict check.)

The read in step 1 is “load head,” not a lock; the write in step 4 is the whole commit. Nothing spans two batches. Within one process, concurrent commit attempts (multiple connections) may run steps 1–3 optimistically in parallel — the transaction conflict at step 4 arbitrates — or the committer may serialize them (see Group commit); across processes, the topology above guarantees there is no second writer to race.

Conflict detection — table-level

The changes_made field of the snaphot record is written in DuckLake’s own grammar, not a moraine dialect — the field is DuckLake-visible on the staged-row path (DuckLake re-parses it mid-retry, and its parser throws on unknown entry kinds), so the wire format is DuckLake’s to define. Source-verified (writer WriteSnapshotChanges in ducklake_transaction_state.cpp, parser ParseChangesMade in ducklake_transaction_changes.cpp): comma-separated kind:payload entries; created entries carry SQL-quoted names (created_schema:"s", created_table:"s"."t", quotes doubled for embedded quotes), all other entries carry numeric ids (dropped_schema:1, dropped_table:2, altered_table:3, inserted_into_table:4, …). moraine emits entries in DuckLake’s writer order and, when parsing, treats any entry kind it does not model as an unclassifiable change — conservatively a true conflict — rather than erroring.

The verb-path mapping for data-plane operations, chosen so every emitted kind is one DuckLake’s conflict matrix actually checks (the legacy compacted_table kind is parse-only in DuckLake’s source and unchecked by its matrix — emitting it would make these commits invisible to a DuckLake retry): register_data_file → inserted_into_table:<id>; register_delete_file → deleted_from_table:<id>; expire_data_file → merge_adjacent:<id>; expire_delete_file → rewrite_delete:<id> (the two compaction kinds — an expiry’s conflict profile is a compaction’s: true against concurrent deletes, drops, and compaction of the table, benign against appends and alters). Statistics-only mutations emit no entry: stats are re-derived on retry and conflict with nothing.

When the head conflict fires, the committer compares the set of table_ids it touched against those touched by the intervening commits N+1 … N+k (recoverable from their snapshot_changes):

Disjoint table sets → benign race. The concurrent commits do not invalidate this commit’s premise. Retry from step 1 against the new head (what a retry re-runs is defined in Conflict resolution below — on the verb path it includes re-running the caller’s closure, not just re-stamping ids). This is bounded internal retry inside transaction; no caller involvement.

Retries wait before re-running: the delay starts at 2ms, doubles, and caps at 50ms, with jitter of up to the base delay so two writers that just collided do not back off in lockstep and collide again. The whole budget spans a few hundred milliseconds. A commit that lands on its first attempt never waits, so the backoff costs the uncontended path nothing.

The budget is deliberately not DuckLake’s, whose own loop waits 100 ms × 1.5ᵏ over 10 attempts. Two reasons, both consequences of what moraine’s races actually are. First, a benign race here clears in one WAL flush — the shape group commit made visible, where concurrent commits inside the process meet in a batch rather than racing at all, leaving only a staged-path commit colliding with a verb-path batch. A budget of ten attempts capped at 50 ms spans several flush intervals at the 100 ms default, which is several times more than the collision it waits out. Second, the two budgets compose: DuckLake re-drives a retryable moraine error up to ten times, so moraine’s own waits are multiplied by ten before an application sees anything. At DuckLake’s cadence that product is tens of seconds of invisible waiting; at moraine’s it is a few seconds worst case, and typically one retry that waits 2 ms. Ten attempts is kept: losing ten consecutive races requires a pathology the topology does not otherwise admit, and past that RetryBudgetExhausted is the honest answer.
Overlapping table set → true conflict (subject to the append-append refinement below). A concurrent commit mutated a table this commit also mutated. The premise may be invalid (the table was dropped, a column altered, files this commit references were compacted away). Abort with a typed CommitConflict.
Budget spent → terminal. A commit that loses MAX_COMMIT_ATTEMPTS benign races in a row aborts with RetryBudgetExhausted, not CommitConflict. The distinction is a wire contract, not cosmetics: DuckLake decides retryability in DuckLakeTransaction::RetryOnError, a substring match on the error message — it re-runs a failed commit iff the lowercased text contains primary key, unique, conflict, or concurrent — and CommitConflict’s Display carries conflict in its prefix. The text moraine surfaces for a lost commit is part of the interface, not wording. A transient conflict should stay retryable and keeps that wording; an exhausted budget must not, or the two bounded loops compose into ducklake_max_retry_count × MAX_COMMIT_ATTEMPTS re-runs of a commit that already failed to settle ten times. RetryBudgetExhausted’s text is therefore free of all four substrings, exactly as the equality-index uniqueness rejection already is. Every exhaustion is logged at warn with the head it started from and the commits it lost to — without it the failure is a slow commit with no stated reason.

Counter advancement is never, by itself, a conflict. Every commit reads and bumps next_catalog_id/next_file_id; that shared read is precisely what benign-race retry re-derives. Only overlapping table_ids are a conflict. This is the rule that keeps table-level detection from collapsing into serialize-everything.

Append-append refinement. A same-table overlap is not automatically a true conflict when both sides touched the shared table only as pure appends: register_data_file / inline_insert records plus the tstat counter/stat advance they imply. Data-file registrations are commutative — neither invalidates the other’s premise — and the next_row_id/stats updates are mechanically re-derivable on retry exactly like the global counters. So append-append overlap classifies as benign on the verb path: retry re-reads the advanced next_row_id, re-allocates row ids, re-merges stats, and re-commits.

This mirrors DuckLake’s own conflict matrix, verified in its source (DuckLakeTransactionState::CheckForConflicts, src/storage/ducklake_transaction_state.cpp): an insert conflicts with a concurrent drop, alter, or delete on the same table — and with nothing else. Its tables_inserted_into and tables_inserted_inlined sets are checked against the dropped, altered, and deleted-from tables only, never against another transaction’s inserts, and an insert is likewise compatible with a concurrent inline flush or compaction of the same table (their file sets are disjoint: the insert’s new files are not the maintenance operation’s inputs). moraine’s verb path adopts the same classification: an append is benign against other appends, flushes, and compactions of its table; drop/alter/delete on that table are true conflicts, in both directions.

The refinement is required for verb-path fidelity, not an optimization: plain table-grain overlap would call a same-table append pair a true conflict and make moraine strictly stricter than the catalog it serves.

DuckLake is finer-grained in exactly one place — two transactions deleting from the same table conflict only if they touched the same data files (its CheckForConflicts fetches the files deleted after the transaction’s snapshot and conflict-checks at data_file_id grain) — and moraine matches it. A register_delete_file records the data file it targets, and two commits deleting from one table classify benign when those targets are disjoint. The premise holds because moraine allows one live delete file per data file: disjoint targets write disjoint records, so neither commit invalidates the other’s.

Two cases fall back to table grain, both because the file set is not knowable rather than as a policy choice. An inlined delete names a row, not a file. And a DuckLake-authored change set arrives through the changes_made grammar, which carries deleted_from_table:<id> and no file — so a commit whose file ids moraine did not author is treated as deleting from all of them. Since data-file ids are global, a shared id is a shared file and the comparison needs no table key; the file ids ride the snapshot record as moraine-internal state, exactly as schema_changed_table_ids does, and are invisible to DuckLake.

The snapshot also persists deletes_untargeted_files: a mixed commit that targets files and deletes inlined rows still requires the table-grain fallback after replay. With a non-empty file list the flag controls that fallback; with an empty list it applies regardless of the flag. Older records default the flag to false, so readers cannot recover mixed-delete information omitted by older writers.

Name collisions are invisible to id-set comparison. Two concurrent commits each creating a table named orders in the same schema allocate fresh, disjoint table ids — the set comparison classifies the race benign. That is correct as classification (neither commit’s staged state is invalidated), but uniqueness must still be enforced: on the verb path it is the closure re-run on retry that catches it — the re-run resolves names against the new head, sees the winner’s table, and returns AlreadyExists; on the staged-row path every lost head surfaces to DuckLake (below), which re-drives with full knowledge of the new catalog state. Set comparison alone never establishes name uniqueness; re-validation against the new head does. This is stated explicitly because it is the tempting shortcut: a purely mechanical re-stage-and-retry, with no re-validation, would commit duplicate names.

One operation does not fit table grain and is handled at schema-list granularity: CREATE/DROP SCHEMA (mutates the set of schemas). Concurrent commits that both touch the schema list conflict with each other — matching DuckLake’s matrix, where created/dropped schemas conflict-check against each other and against creations inside a dropped schema. Rare enough that coarse handling costs nothing. Option changes need no grain at all: DuckLake neither versions nor conflict-detects them (set_option writes ducklake_metadata outside the snapshot protocol, last-write-wins — see Schema-version tracking); the verb path mirrors that, treating set_option/unset_option as non-conflicting overwrites.

The benign/true classification is mechanical — set comparison over table_ids and mutation kinds — but classification only decides whether to retry; the retry itself re-validates (next section). tx still never needs to understand what SQL produced the mutations.

Conflict resolution — abort, with internal benign-race retry

What a benign-race retry re-runs (verb path). A retry is a full re-run of the commit cycle, exactly as RFC 0003 specifies for commit: re-read a fresh demand-loaded premise at the new head, re-run the caller’s closure against it, re-allocate ids from the advanced counters, re-stage, re-commit. The closure re-run is load-bearing, not an implementation nicety: it is what re-resolves names and re-validates logical premises against the state that actually won (the name-collision case above), and it is why RFC 0003 requires the closure to be pure and re-runnable. A retry that mechanically re-stamped ids onto the previously staged records — without re-running the closure — would be cheaper and wrong (see Alternatives).

The core resolves benign races itself (bounded retry, above) and aborts true conflicts with a typed CommitConflict. It does not attempt to rebase a true conflict. Rebasing a true conflict means re-running the DuckLake operation — which moraine cannot do, because by RFC 0001 the core is DuckLake-agnostic and never held the originating SQL. Re-driving a true conflict belongs to whoever authored the operation:

DuckLake, whose commit loop retries retryable conflict signals internally and re-checks its own conflict matrix (source-verified; see Staged-row commits), or
the application, re-running its statement when DuckLake classifies the conflict as true and throws — as a PostgreSQL SERIALIZABLE transaction retries on a 40001 serialization failure.

The core’s responsibility ends at a truthful typed error.

The internal benign-race retry is bounded (a small fixed attempt count); exceeding it returns CommitConflict rather than looping unbounded, so a pathological write storm degrades to caller-visible conflicts instead of a hang.

Id allocation

next_catalog_id, next_file_id — global, in the snapshot record, matching DuckLake. Allocated by bumping the in-memory copy from head and writing the advanced value in the new snapshot record. Shared reads of these counters are benign (re-derived on retry), never conflicts.
next_row_id — per-table, in the tstat record, matching DuckLake’s ducklake_table_stats.next_row_id. Because the counter lives in table-scoped state, bumping it is a mutation to that table and is inside the table’s conflict set: two commits inserting into different tables share no row-id counter, so they never contend; two inserting into the same table overlap on that table_id — and resolve under the append-append refinement above: the shared counter read is re-derivable, so both commits land via benign-race retry: on the verb path the loser re-allocates its row ids from the winner’s advanced next_row_id, and on the staged-row path the rebase moves the loser’s row_id_start/row_id values and its next_row_id by the same advance (Staged-row commits). Row ids stay dense per table for RFC 0005’s inlined chunks. Verified against the DuckLake spec; the e2e suite validates the increment mechanics (next_row_id += record_count) and DuckLake’s own behavior under concurrent same-table inserts.

Schema-version tracking

schema_version is a counter separate from the id counters: DuckLake advances it only when a commit changes the catalog schema, and carries it forward unchanged for data-only commits. DuckDB keys its schema-metadata cache on it, so getting this wrong is not a storage bug but a client-visible one — in both directions. Always incrementing defeats the cache: every INSERT looks like a schema change and forces DuckDB to re-fetch every table’s columns, multiplying catalog read traffic on exactly the small-append workload moraine targets. Never incrementing serves a stale cache: after add_column, DuckDB keeps using its cached column list and returns wrong results or errors on the schema mismatch.

The committer tracks a schema_changed flag over the staged mutations. On commit, the new snapshot record’s schema_version is prev + 1 if the flag is set, else copied from snapshot N. The flag is set by the structural mutators — those that alter the catalog’s shape rather than its data (RFC 0003 verb names):

Schema-changing (schema_version bumps): create/drop schema; create/rename/drop table and move-to-schema; create/alter/rename/drop view; add/remove/rename column, column type change, default and null-constraint changes; set partition key; create/drop macros; comments and tags on tables and columns. The last of these bump despite changing no column: DuckLake models them as AlterEntry variants, and the rewritten entry enters new_tables.
Altered but not schema-versioned: set sort key. DuckLake marks the table altered — conflict detection sees the alter — but does not bump schema_version (a sort change never invalidates cross-file compaction, so the ducklake_schema_versions grouping is unaffected).
Data-only (schema_version carried forward): register_data_file / expire_data_file, register_delete_file / expire_delete_file, statistics updates, column/name-mapping registration, and the RFC 0005 inlined inline_insert / inline_delete / flush_inlined_data.
Outside the snapshot protocol entirely: option changes. set_option in DuckLake is an immediate check-then-write on ducklake_metadata within the transaction’s metadata connection — it mints no snapshot, bumps no schema_version, and is not recorded in snapshot_changes (so it is invisible to conflict detection; options are last-write-wins). Both paths implement exactly that. On the staged-row path a ducklake_metadata row translates to a write of the scope’s option record, and because such a commit carries no ducklake_snapshot insert it takes the head-preserving path — the same one snapshot expiry and file cleanup take. So CALL <lake>.set_option(…) lands the option and moves nothing else: no snapshot, no schema_version, no snapshot_changes entry, and so nothing for conflict detection to classify. It does still write sys/head at the standing snapshot id with the batch count advanced, as every head-preserving batch does — that is the anchor a concurrent commit conflicts on, and the stamp that tells a reader the state moved under an unchanged snapshot id.

This classification is source-verified against DuckLake (DuckLakeTransactionState::SchemaChangesMade, which tests exactly the new/dropped table, schema, view, and macro sets; per-table alters route through GetTransactionTableChanges, which decides per change type whether the alter also bumps schema_version — SET_SORT_KEY lands in altered_tables only).

The flag is an explicit property of the mutation set, not something inferred from the batch’s key set after the fact — an inference would misclassify (e.g. a stats-only touch of a table’s state) and silently re-introduce one of the two failure modes. The exact schema-mutating set is pinned against DuckLake’s own behaviour in the e2e suite.

The flag machinery above belongs to the verb path only. On the staged-row path (next section) DuckLake computes schema_version itself and writes it in the ducklake_snapshot row it authors; moraine stores that value row-faithfully and neither tracks nor second-guesses a flag. The two paths must agree — the e2e schema-version matrix pins both.

Staged-row commits — the extension path

RFC 0006’s extension surface does not call RFC 0003’s verbs. DuckLake drives moraine with row-level DML against the ducklake_* tables: a DuckLake transaction stages row inserts/updates/deletes, and — critically — DuckLake itself authors the catalog mechanics. It reads next_catalog_id / next_file_id from the snapshot row, computes the new ids, computes schema_version, and writes the new ducklake_snapshot row, with the new snapshot id and begin_snapshot values embedded in every row it stages. moraine’s job on this path is narrower and different from the verb path:

Translate, don’t author. moraine maps each staged row onto the RFC 0002 keyed layout — an INSERT becomes a current record; an UPDATE that sets end_snapshot becomes the end-version bookkeeping (delete current key, write history key); the snapshot row becomes the snapshot record and the sys/head advance. This begin/end-lifecycle translation is the one semantic convention moraine interprets (RFC 0006 states and bounds it); ids, counters, and schema_version are DuckLake’s values, stored verbatim.
Same commit core. The translated records land through steps 3–4 above: one transaction, one atomic batch, head conflict detection. Nothing about atomicity or durability differs between the paths.
Rebase, then retry — for the shapes the rebase re-derives exactly. A retry on the verb path re-runs the closure and re-allocates ids moraine minted. On this path there is no closure, and the snapshot id, begin_snapshot, counters and schema_version are DuckLake’s values embedded in the rows. But each is a function of the head DuckLake read, and for a data-only commit — one that creates, alters or drops no schema, table, view or macro, so it allocates no catalog id and carries schema_version forward unchanged — that function is arithmetic moraine reapplies against the head the commit actually lands on:
- the snapshot id, and every snapshot_id/begin_snapshot the member stamped equal to it, moves to the premise’s head plus one;
- file ids allocated from next_file_id move by the premise’s advance of that counter, in every row that names them;
- for a table an earlier commit also appended to, row_id_start on the member’s data files, row_id on its inlined rows and the next_row_id it writes move by that table’s advance, and the ducklake_table_stats row it overwrites is re-derived as the member’s delta over the premise, column stats merged by min and max.
This is the re-derivation DuckLake performs when its own loop re-drives such a commit, done without the round trip: it rewrites nothing of DuckLake’s semantics, only the values DuckLake would itself recompute. Every value a data-only commit derives from the head is one of these, so the rule covers that shape entirely; what it cannot re-derive — a catalog id, a schema version — is exactly what makes a commit not data-only. The rebase is therefore refused for every other shape, and for a member that appends rows to a table without carrying that table’s statistics row, since the row-id counter the shift is taken from lives there and nowhere else. A refused member does not join a batch: it commits alone, and a lost race there surfaces as CommitConflict. Index upkeep derives a member’s entries from the moved values, so an indexed table is no exception.
The base is the view the rows were authored against. DuckLake stamps its rows from the snapshot its own transaction read, and the catalog transaction those rows reach may open later, after other commits landed. The member’s base for the rebase is therefore the view the caller read, handed in when the transaction opens, and only absent that the transaction’s own view. Measured against the transaction’s view instead, every such commit read as a lost race: half of all commits at eight writers, each re-driven by DuckLake after its backoff.
A head-preserving commit re-drives its own lost race. Expiry, cleanup and the deletion schedule mint no snapshot and derive nothing from the head: their rows are as good against the head they find as against the one they read. Losing the head race to a session that commits without pause is therefore moraine’s to re-drive, a bounded number of times with the batch cycle’s backoff, re-staged against the head it finds; surfaced instead, DuckLake’s retry would sleep into the same session and the pass would never land. A minting commit’s rows are DuckLake’s, stamped against the head it read, and its race stays DuckLake’s.
The batch yields to a commit it could not take. A minting commit the rebase refuses — DDL, an inline table’s creation, a flush — commits alone, and a race it loses is DuckLake’s to re-drive, after a backoff of at least fifty milliseconds and a fresh read. A session committing without pause moves the head every few milliseconds and would beat every retry, where under a spacing-paced flush it left gaps the retry landed in. So a lone commit that loses tells the coalescer, and arrivals hold for a quarter second before staging: one hiccup for the writers per schema change, against a schema change that never lands. Measured: a CREATE TABLE under a session inserting every twelve milliseconds took the whole session’s length, 5.1 s, to land before the yield, and 63 to 200 ms with it, against 45 ms idle; the session paid half a second over five of them. A loss is found either while the rows are staged, when the transaction’s view is already past the head the rows were stamped from, or at the head swap, and both yield.
A member’s own lost race declines, never poisons. A CommitConflict raised while a member stages — its stamped snapshot at or below the head it is measured against — is that member’s to re-drive. It leaves the forming batch as it found it; any other error a member raises still poisons the batch, since a half-staged member cannot be unstaged.
Same classifier, same outcomes. A staged member’s change set is parsed from the changes_made DuckLake authors, so the table-level classifier that decides benign against true for verb members decides it for staged members by the same matrix: a race lost to disjoint commits or to same-table appends is rebased and re-driven under the same bounded budget; a race lost to an incompatible commit, or one the rebase cannot absorb, returns CommitConflict carrying the text DuckLake re-drives on. DuckLake’s RunCommitLoop remains the re-driver for every commit the shim keeps on its client-side path (RFC 0006: DDL, and anything not data-only), re-checking its own matrix against ducklake_snapshot_changes before each attempt — so the read surface stays live while a commit is in flight, as before.

Two of the staged path’s read phases dominate its latency — the inline translation (chunk walks for flush-deletes and drops) and equality-index maintenance (scoped Parquet reads and entry staging) — and they run beside each other rather than in sequence. Both read the transaction’s pre-commit cut, so ordering could only matter through writes: index maintenance stages only index/* keys as it streams — a subspace the inline translation never reads — and hands its chunk-directory repairs back unstaged, while the translation stages nothing at all. Neither can observe the other, so running them together is indistinguishable from running them in order. The operation-owned dependency projection resolves first; maintenance still completes before the batch is assembled, so a poisoned definition rides the writes it produces; and its directory repairs are ordered ahead of the batch’s own inline writes, so a flush draining a repaired chunk still deletes the locator the repair added.

The two front doors share the commit core (staging, one-batch atomicity, head conflict detection, durability, the batch) and differ in what a retry re-runs: a closure on one, a rebase on the other. Holding the rebase to values DuckLake itself re-derives is what lets the staged path be re-driven at all without risking a silent rewrite of DuckLake-authored state.

Inlined writes as commit participants

Per RFC 0005, an inlined insert/delete is not a separate path: its inline/* records are added to the same WriteBatch in step 3, allocating row ids from the same per-table next_row_id bump as a Parquet write would. An inline flush is itself a commit — Parquet PUTs first, then one batch that creates the file/delfile records and deletes the consumed inline/* records outright (no history; inline/* is append-then-delete, not begin/end-versioned). Nothing about inlining changes the commit sequence or the conflict rules; it only adds record kinds to the batch.

Group commit

Because the topology funnels concurrent transactions through one committer, the committer may batch several pending catalog commits into one WAL flush: allocate ids for each, stage their records into one WriteBatch, one CAS, one durable flush. Per-commit latency stays one-PUT-bound (RFC 0002 / README); throughput under load stops being one-flush-per-commit and PUT costs amortize. Group commit is an optimization the protocol permits, not a correctness requirement — a batch of one is the normal path. Ordering within a group is the committer’s choice; conflicts within a group are impossible because a single committer serializes them before staging.

Implemented two ways over one core — a lone commit is a group of one, so there is one commit path, not two. Catalog::commit_group batches what one caller hands it; concurrent callers of the ordinary Catalog::commit are batched without asking. A commit that finds the store free opens a batch, one that finds a batch forming joins it, one that finds a batch being submitted waits for the next, and a batch seals as soon as no caller is on its way into it — so the submission in progress is the batching window and an uncontended commit waits for nobody. A batch also seals at a bounded member count, since under saturation the arrival count never falls to zero. Submission ends when the batch’s write is visible and folded into the projections, before it is durable: the store then admits the next batch, which forms and submits while the first awaits its flush, and the two share that flush under the pacing below. Every member still returns only once its own batch is durable.

A member that stages no writes still shares the batch’s outcome if its premise includes uncommitted writes. It must wait for durability or retry the lost batch before reporting that premise’s snapshot id. A no-op against committed state alone may return immediately.

Steps 1–3 run once per member, against a premise folded forward from the member before it, so member i allocates its ids and detects its name collisions against everything members 1..i did. That is what makes intra-batch conflicts unreachable rather than unlikely. Step 4 then runs once.

The store’s one submission slot is released by ownership, not by a statement: the slot is claimed under the lock that moves a batch out of forming, and the claim is dropped when the submitting task lets go of it. A submission that panics, or whose task is dropped before it finishes, therefore still admits the next batch. Nothing else clears the slot, so were it released by a statement the submitter could fail to reach, one lost submission would close the store to every later commit for the life of the process. A waiter subscribes to the slot’s generation before reading the slot, so a submission that ends between the two wakes it rather than leaving it parked for the following one.

For the same reason the write that holds the slot is bounded, where the durability wait that follows it is not. A write that never returns would hold the slot forever, so after five minutes — thirty stall reports — its batch is given up on and reported as an unknown outcome, which is what it is: the write may still land, so its members must not retry. The durability wait needs no bound because it runs after the slot is released.

Every await a commit can park on inside the coalescer is named in the log if it runs long: admitting, opening the batch, staging onto it, submitting it, and awaiting durability. A commit that parks cannot be read from a thread dump — the caller blocks on the runtime, so the future’s state machine is on the heap and the await it is suspended at appears on no thread’s stack — and the waits that are not object-store requests, such as the store’s own flush and capacity backpressure, are bounded by nothing and reported by nothing else. These records are what make a stalled commit say which phase it is stuck in. They name a slow phase; they never cut one short.

The process-wide work pools a commit passes through are named the same way, on both sides: the wait for a permit and the hold of one. A pool is shared by every catalog in the process and hands permits out in arrival order, so it cannot starve a waiter on its own — but a permit is held across blocking work that cannot be cancelled, so work that never returns takes a permit out of the process permanently. Reporting the hold names the holder, which no waiter’s record can.

A batch batches commits; it does not merge them. Each member mints its own snapshot, which time travel resolves separately. What batching changes is durability granularity, not catalog shape.

Measured, the coalescing is exact: a batch carries as many commits as there are concurrent committers, so throughput is concurrency / flush_interval (the flush spacing, under which back-to-back batches each wait one spacing) with no overhead visible at any level, up to the member ceiling — where 128 concurrent commits land as two full batches at the same rate 64 do (BENCHMARK.md, “Commit throughput vs. concurrency”). The ceiling, not the flush cadence, is what binds first under saturation, and it is the lever if a workload ever needs past it. End to end through DuckDB the same burst is measured by cargo xtask bench --workloads concurrent_connections; with staged commits outside the batch it saturates near three commits per flush, half the commits losing a race — the gap the rebase exists to close.

The single-commit floor is measured against AWS S3 as well as the local and injected-latency stores. From a worker in the bucket’s us-west-2 region, 1 ms and 25 ms flush cadences both land at roughly 28–30 ms median, while a 100 ms cadence lands at 101.9 ms (BENCHMARK.md, “Durable-commit latency against a real endpoint”). The endpoint round trip therefore replaces the short cadence as the binding term; it does not add a second serial wait.

The batch is the unit of failure. A member that fails discards the batch, and its other members re-run; a lost race re-runs every member, conflicting if any member’s change set does; a crash leaves every member committed or none (RFC 0011, GroupCommit). A sealed batch commits on a task of its own rather than on whichever caller sealed it, because a host interrupt drops that caller’s future (RFC 0010) and the other members are owed an answer.

Both front doors land through one function, so the durable write, the head-race classification and the projection bookkeeping exist once — and both join the forming batch. A verb member stages by running its closure against the premise folded forward from the member before it; a staged member stages by translating its rows against that same premise, rebasing the head-derived values DuckLake stamped on them (Staged-row commits) so member i’s snapshot id, file ids and row ids sit above everything members 1..i allocated. The rebase stands in for the fold’s re-allocation; the fold itself is unchanged. A staged member the rebase refuses — one that is not data-only — does not join: it commits as a batch of one, its id checked rather than trusted, and a staged commit whose id does not advance the head it lands on is refused as the lost race it is, with CommitConflict’s text — the substring DuckLake re-drives on. Reporting it as corruption instead would be a wire-contract bug, not a wording one.

Batching therefore reaches DuckLake’s own DML: several connections on one instance, each holding a DuckLake transaction, commit into one batch, one WAL object, one flush. It does not reach its DDL, which stays on the client-side path and commits alone. A BEGIN … COMMIT still lands as one snapshot — statements merged — and several such transactions committing at once land as one batch — commits batched.

Sequence numbers are the store’s, not moraine’s

A commit writes with SlateDB’s generated sequence number; moraine never pins one. The knob exists — a non-zero WriteOptions::seqnum overrides the generated value, and a write below the store’s current maximum is refused — so it reads at first like an exactly-once primitive: pin the same number on a re-drive and the duplicate is rejected.

It is not one here, for two reasons. The sequence space is global to the store, not per catalog commit: index build steps, entry reclamation, inlined writes, and migration batches all advance it while minting no snapshot, so no function of a snapshot id stays in step with it, and any mapping moraine chose would be stale the moment a non-committing write landed. And the number moraine would need to pin does not exist at re-drive time anyway. A re-drive re-runs the caller’s closure against the advanced head, so it is a different logical commit computed from a different premise — there is nothing stable across the crash to pin to.

Exactly-once across a process death needs a caller-supplied idempotency token, recorded in the commit and checked before staging. That is an API change this RFC does not make, and pinning seqnum would not substitute for it. So a re-driven commit is an ordinary commit: guarded operations surface their guard, and a data-only re-drive lands a second time (RFC 0011, CommitDurableNotAcknowledged).

Reader visibility after commit

Step 4 makes a commit durable; a separate reader observing it is a second, distinct property. Readers are uncoordinated (Topology) and resolve snapshots by following the manifest from object storage. A reader that opened before the commit — or a fresh reader whose manifest view is a poll interval stale — therefore does not see snapshot N+1 the instant commit returns. For moraine’s own committer this is a non-issue: it holds the advanced sys/head in memory and reads its own writes. It matters at the boundary where another process must observe a just-returned commit — read-your-writes across processes, and writer takeover, where a new writer must publish the last committed state before it serves.

The contract: commit does not return success until the new snapshot is both durable and reflected in the manifest that readers follow, so a reader opened after commit returns resolves N+1. (A reader opened before may still lag by its manifest poll interval — that is ordinary snapshot isolation, not a violation.)

SlateDB 0.16 returns a WriteHandle once the transaction is visible in memory. Moraine then either flushes the WAL itself or awaits WriteHandle::await_durable() on a flush another commit is performing, per the pacing below, before reporting success; an empty transaction returns no handle and needs no wait. Both routes retain shutdown and fencing errors and report stalls without abandoning the durability wait.

Flush pacing

Every WAL flush is one object-store PUT, so the flush rate is the PUT rate a commit stream costs. The writer opens its store with SlateDB’s flush timer off and paces the flushes itself, under one setting, flush_interval, read as the minimum spacing between two flushes:

A commit that finds the spacing elapsed since the last flush flushes at once and waits only on its own PUT.
A commit that finds no recent flush carried another commit waits only the floor, a tenth of the spacing, since the last flush started: nothing rode those flushes, so nobody is coming whom waiting longer would coalesce with. A writer without siblings pays no spacing. The floor is the rate cap that remains — ten flushes per spacing at most from one writer — and the window in which commits arriving just behind a lone one still share a flush.
A commit that finds the spacing running behind a flush that carried riders joins one flush deferred to the moment it elapses, or to the moment as many commits have joined it as the most a recent flush carried, whichever comes first; the first such commit performs it, the rest await their handles on it. The spacing is the longest a flush waits, not the only time it closes: once the population a recent flush saw has arrived, waiting longer coalesces nothing. The population is counted in commits: a coalescer batch is one handle carrying many, and weighs its members. No early close starts a flush before the floor, and none happens without a recent measurement: a flush that waited the whole spacing out carries every writer and so measures the population, where an early close carries only the mark and measures nothing; a measurement is trusted for ten spacings, after which the next flush waits the spacing out again. That is how the mark grows to the population and what the latency of the scheme costs, one spacing in ten at most. The mark is the most of the last few flushes, so a flush that closed early, carrying fewer than there are, cannot lower it; only flushes that waited the spacing out and still carried less do, a few in a row, which is how a writer left alone gets its lone rule back.
A commit that finds a flush in the air, and so not carrying it, schedules the next flush for when the spacing elapses from that flush’s start.

So a lone commit costs one PUT of latency and one PUT, which is what it cost waiting the spacing out; a burst costs at most the spacing plus one PUT, and usually only the time its writers take to arrive; and the objects a burst costs are one per round of its writers either way. What the rules give up is nine tenths of the PUT rate cap: flushes never exceed one per floor rather than one per spacing, so a writer population can flush as fast as it commits in rounds, down to the floor — the same objects, sooner. The timer is off rather than kept as a ceiling because a tick landing inside a spacing window would carry the pending commits early, one extra PUT per tick, and the cap would only be approximate; it is safe to turn off because every Moraine write reaches the WAL through a commit that waits for durability, so nothing is left for a timer to sweep up. flush_on_commit is the same setting at zero. A store opened for a migration keeps the timer, since its few commits do not warrant pacing.

Group commit and pacing compose: the coalescer batches concurrent commits into one write, a batch submitted while an earlier one awaits its deferred flush rides on that flush, and so under a burst every commit that lands inside one spacing is durable together at its end, in one PUT.

The durability watcher advances only after the WAL reaches object storage. Both fresh-open paths — Db::open and DbReader::open in latest mode — replay WALs from object storage past the manifest state. So a handle opened after Moraine’s commit returns resolves N+1 by construction.

A validation test in the store harness (open object_store, commit, open a fresh reader, assert N+1 is visible) pins that behavior against real SlateDB and measures its per-backend latency.

The log’s object store

A commit’s latency floor is one PUT of the log, and that PUT need not go to the bucket the catalog lives in. CatalogOptions::wal_store writes the log to an object store of its own — an S3 Express One Zone directory bucket beside a standard one — while every other object, manifests and sorted-string tables alike, stays in the catalog store. The keys do not move: SlateDB lays the log out under the catalog’s own path within whatever store it is handed, so both stores address the same key space and only the endpoint differs. The log store must be as durable as the catalog store, since a commit no sorted-string table holds yet lives only there.

The store a catalog records. SlateDB persists nothing usable about it — the manifest field that carried it is dropped by the manifest version SlateDB now writes — so moraine records the log store’s name itself, as the global wal_path option the bootstrap writes beside encrypted and data_path, and every open reads it back in the point read that already validates the format stamp. An open whose log store is not the recorded one is refused: one missing, one that differs, or one supplied where the record names none. Replaying an empty log over live state loses every commit no sorted-string table carries yet, silently, and the next flush buries the evidence. A migration is held to the same record, since it opens the writer itself and rewrites keys from whatever the log replayed.

That record cannot be left in the log it describes. A bootstrap that configures an external log therefore writes its memtable out before the open returns, so the catalog store itself carries the stamp and the name from the moment the catalog exists — otherwise an open handed the wrong store, or none, would find no catalog at all and bootstrap a second one over the first, stranding everything the real log holds. The window that remains is inside creation, before any caller holds a handle, and is the genesis window RFC 0004 already allows for: a crash there leaves either a catalog whose store is recorded or no catalog at all.

Moving it. Catalog::move_wal_store takes a catalog nothing holds and moves its log in two opens, whose order is the whole design:

Open the writer against the log store the catalog records — refusing any other — and write the memtable out as a sorted-string table. That carries the log’s contents into the catalog store and moves the replay point past them, so the old log holds nothing a replay needs.
Open the writer against the new log store and record its name there — the only write that reaches it — then write that out too.

A destination inside the lake’s recorded data root is refused before the first open drains anything, for the reason the attach surface refuses one: the orphaned-file cleanup that lists that root would delete log objects holding commits no sorted-string table carries yet. The comparison is lexical and shared with the attach’s, so both refuse the same layouts.

Every crash point leaves the catalog openable with nothing lost. Before the second open’s commit is durable the record still names the old store, whose log is drained: an open against it serves everything, and the move can be run again. After it, the record names the new store and the commit saying so is in the new log, which an open against that store replays. An open against the old store in that window is accepted too — the record it can see names it — and merely leaves the move un-applied. No window refuses both stores, which is what the ordering buys: recording the new name through the old log would produce exactly that.

Test obligations

Per RFC 0001, integration tests exercise the protocol against real SlateDB on in-memory object_store — no mocks of the store:

Disjoint concurrent commits both succeed (benign-race retry).
Concurrent pure appends to the same table both succeed (append-append refinement); the loser’s rows carry row ids re-allocated above the winner’s advanced next_row_id, with no collision or gap.
Concurrent deletes from the same table are benign when their target data files are disjoint and a conflict when they meet on one; an inlined delete, and a delete parsed from DuckLake’s grammar, fall back to table grain.
Concurrent same-name creates (two commits each creating table orders in one schema): exactly one succeeds; the other’s retry re-runs the closure and returns AlreadyExists — never two live tables with one name.
Incompatible overlapping concurrent commits: one wins, the other returns CommitConflict.
Staged-row commits rebase and re-drive: a data-only staged commit that loses a benign race lands with its snapshot id, file ids and row ids moved above the winner’s, every stamped begin_snapshot moved with the id, and its table stats equal to what a sequential commit of the same rows after the winner writes — differential against that sequential run. A staged commit the rebase refuses (DDL, a catalog id, a schema change) returns CommitConflict with the store unchanged by the loser, its text carrying conflict.
Staged members batch: a burst of concurrent data-only staged commits lands in fewer WAL objects than the same commits made one at a time, each minting its own consecutive snapshot; same-table appenders in one batch leave row ids dense with no collision; a member the rebase refuses lands alone without stranding the batch it declined.
Crash-shaped sequences: owned by RFC 0011 rather than restated here. CommitNotDurable covers the batch whose flush never landed — the head does not move and no record of it is visible — and CommitDurableNotAcknowledged the flush that landed before the process died, where the head and snapshot resolve consistently on reopen and a re-drive surfaces its guard.
Counter monotonicity: next_catalog_id/next_file_id/per-table next_row_id never regress or collide across concurrent commits.
Bounded retry terminates: a forced write storm returns CommitConflict rather than looping.
Schema-version matrix: every schema-changing operation increments schema_version; every data-only operation — including inline insert/delete, flush, and mapping registration — carries it forward; comment/tag changes increment; set_option mints no snapshot and changes no version.
Bootstrap: the first open of an empty store lands format stamp, snapshot 0, and head in one batch; a second open (concurrent or later) finds the initialized store rather than re-initializing; an unknown-format or mid-migration store is refused.
Fresh-reader visibility: after commit returns, a newly opened reader resolves the new snapshot (the store-harness validation above, run against real SlateDB on in-memory object_store).
Checkpoint readers: a reader pinned to a checkpoint serves the state at it and never a later commit; it opens, reads, and closes against a store that refuses every write, attempting none, where a latest-following reader on the same store writes; a writer refuses a checkpoint, a read-only catalog refuses to mint one, and a malformed, unknown, or deleted id is refused rather than silently followed.
DuckLake-facing pins, differential against a stock DuckLake catalog fed the identical statements: row ids allocate as next_row_id += record_count per table, stay dense, survive an UPDATE of other rows, and never contend across tables; schema_version bumps for column DDL and for comments and tags, and is carried forward by inserts (file and inlined) and by name-mapping registration; and DuckLake’s retry budget (ducklake_max_retry_count 10, ducklake_retry_wait_ms 100, ducklake_retry_backoff 1.5) is what the composed budget above assumes for the commits the shim keeps on DuckLake’s client-side path. A genuine concurrent DuckLake race needs more than one connection on one DuckDB instance, which the CLI cannot give: test/sql drives DuckDB’s own sqllogictest runner for that, and pins the loop end to end — two overlapping DuckLake transactions where a compatible loser re-drives and lands (with row ids dense across the race), and an incompatible one throws. What a retry reads mid-flight is pinned alongside it: the ducklake_snapshot_changes rows above a snapshot id, and the exact changes_made entries moraine authors on the verb path.
Options through the staged-row path: set_option records the option, global or table-scoped, and mints no snapshot; a second write of one key wins over the first; a delete removes it and the scope resolves to its parent’s value again.
Group commit: a group costs fewer object-store writes than the same members committed separately; every member mints its own consecutive snapshot and time travel resolves each one to that member’s state; a member stages against what the members before it left; a failing member aborts the group; a group that loses a disjoint race re-runs and lands every member.
Coalescing: a burst of concurrent commit calls costs fewer object-store writes than the same calls made one at a time, while each still mints its own snapshot; and a commit whose future is dropped part way — the shape a host interrupt takes, swept across the commit’s life — never strands the commits sharing its batch.

E2E testing validates the protocol against real DuckLake SQL.

Alternatives considered

Mechanical re-stage on benign-race retry (re-derive ids, re-stamp the previously staged records, re-commit — without re-running the closure). Rejected: cheaper per retry, but it re-validates nothing. Logical premises the closure established against the old head — name uniqueness above all — are never re-checked against the commits that won the race, so two concurrent same-name creates would both land. The closure re-run costs one in-memory pass and is the only retry form that is correct by construction. The staged-row rebase is not this alternative: it re-stamps nothing a closure established — DuckLake’s logical premises are its conflict matrix, which the classifier re-checks on every retry — and it moves only the values DuckLake’s own re-drive recomputes, refusing the rest.
Internal rebase of true conflicts (replay staged mutations against the new head). Rejected: replaying a true conflict requires re-running the DuckLake operation, which lives above the DuckLake-agnostic core. It would put DuckLake transaction semantics in the wrong layer and can silently produce wrong results if the replay logic is incomplete. Abort
- typed error is honest; benign races — safely re-drivable because the verb path re-runs the caller’s closure against the new head — are the only thing retried internally.
Snapshot-level conflict detection (any concurrent commit conflicts). Rejected: serializes all commits and defeats the benign-race retry — disjoint-table writers would falsely conflict. The single-WriteBatch CAS already serializes the physical write; there is no reason to also serialize logically-disjoint commits.
Entity-level conflict detection (per-file/column) everywhere. Rejected, but with one exception now built. DuckLake’s matrix is operation-grained over tables everywhere except delete-vs-delete on one table, which it checks at data_file_id grain — and moraine reproduces exactly that one check (above), because a delete already names its target file and carrying the id costs a set comparison. Generalizing the idea to columns or to per-file inserts is what stays rejected: it buys no concurrency the append-append refinement does not already give, and it would put entity-shaped knowledge in a classifier whose whole virtue is not needing any.
Global next_row_id in the snapshot record. Rejected: contradicts DuckLake (next_row_id is per-table in ducklake_table_stats) and would make every insert into any table contend on one counter, turning benign cross-table concurrency into counter churn. Per-table placement aligns allocation with conflict granularity.
Leave staged commits to DuckLake’s retry loop (the design until concurrent connections were measured). Rejected: the loop sleeps a jittered 100 ms × 1.5ⁱ before every re-drive, tuned for SQL catalogs whose commits take milliseconds; against a commit whose floor is one flush, every lost race costs the sleep and a further flush, and because staged commits did not batch, every concurrent commit raced. At eight writers half the commits lost and commits per flush saturated near three.
Shorten or remove DuckLake’s retry sleep (ducklake_retry_wait_ms, or a patch skipping the sleep on a benign-race marker). Rejected on measurement: zero wait recovers most of the loss at eight writers and livelocks at sixteen — per-commit optimistic races on one key with instant retry starve writers until ducklake_max_retry_count runs out, and that budget is what bounds the storm. The sleep is load-bearing for a loop that races per commit; the fix is not to race per commit.
Rebase every staged shape, DDL included. Rejected: a catalog id or a schema version embedded across a member’s rows and inlined-table names is not a delta against the head but an identity other rows reference; moving it is the re-modeling of DuckLake-authored state RFC 0006 forbids. Data-only commits are the shape that batches, and the shape under load.
Pessimistic locking (a lease/lock record in system a writer must hold). Rejected: reintroduces coordination state and a liveness problem (crashed lock-holder) that the optimistic head-CAS avoids. SlateDB fencing already provides the safety a lock would; optimism provides the concurrency.