CIP-9: Runner-Backed Storage
Status: Draft
Type: Standards Track
Category: Core
Created: 2026-03-04
Updated: 2026-11-05
Cowboy Cloud and Cowboy Chain. This CIP specifies the Cowboy protocol. Cowboy Cloud runs on this protocol today. Cowboy Chain, a public network on the same protocol, is under consideration for a future release. Where the two differ, this CIP says so.
1. Background and goal
CIP-2 Runners are stateless and short-lived. A job that needs secrets, large outputs, data carried between jobs, or a persistent container disk has nowhere durable to keep it (§2). This CIP specifies off-chain storage volumes that belong to a Cowboy account and that Runners attach during jobs. It extends CIP-2 (Verifiable Off-Chain Compute) with storage that outlives a single job. CBFS (the Cowboy File System) is the data plane: encrypted object writes, manifests, erasure coding, Relay Node RPCs, placement records, repair, and the FUSE and object clients. Cowboy adds the control plane: on-chainStorageCommitment records, CapToken issuance and revocation, Relay Registry state, billing, and task attachment. Together they are Runner Attached Storage (RAS).
- A Cowboy account (EOA or Actor) owns and pays for every volume.
- Volumes are private by default. A private volume is encrypted by the Runner before data leaves it, and only the owning account can decrypt it. A public volume (
visibility = PUBLIC) is plaintext, and anyone can read it without a CapToken; it serves web assets (CIP-15) and public datasets. - A CapToken grants a Runner read-only, write-only, or read-write access to a volume. Several can be active on one volume at once. Public volumes need one only for writes.
- Every object has a deterministic path, so the owner can read or delete it directly.
- The client erasure-codes each object (Reed-Solomon) across several Relay Nodes.
- Storage is billed in CBY per byte per epoch, including erasure overhead, at CIP-31 rates in nano-CBY. This extends CIP-3 Cells to off-chain blobs.
- What a volume, object, chunk, shard, stripe, manifest and root are: §3 and §3.1.
- How a write flows (upload shards, publish manifest nodes, commit the root on chain): §4.2, §5.3, §5.4, §5.4.1, §12.1.3, §12.2 and §12.2.1.
- How a read flows (resolve the root, fetch the manifest, fetch shards, decode): §5.3.2, §6.2 and §7.5.
- What a reader is guaranteed to see: §12.1.5.
- Failure handling and repair: §5.5 and §5.6.
- Rent and fees: §10 and CIP-31.
Genesis signing and material identity
CIP-40 §§7.6–7.7 define the normative signature slots, domains and tagged keys for this CIP. These keys useSigningPublicKey restricted to Ed25519:
RasOwnerDelegation.delegated_pubkey, owner-authorization keys,
DelegationRevocation keys, relay identity, mount-grant signing keys, and the CIP-10
workload release and commit recipient keys. Their authorization fields use
SignatureSlot under the table’s policy. A storage key that contains a signing key
uses storage_key_fragment() (algorithm ‖ key). This includes ras:delegation:,
ras:revocation: and ras:svrn:. Decoding is structural. Consensus charges for
per-key validation before it validates and stores the key.
Material and volume-DEK hashes bind the real chain identity in the length-framed
BLAKE3 construction (u16le domain length ‖ domain, then each field as
u32le length ‖ field). The first two identity fields are u64le(chain_id) and
then chain_instance_id[32]. These two fields take the place of a single
material_chain_id field in the release-key-material and volume-DEK hashes, and of
the pair chain_id, material_chain_id in the service-volume request ID. There is no
protocol-owned material chain ID 1. The domain labels and the order and width of
every other field are as specified; these codec APIs take the chain identity as input.
Owner-key-wrap HKDF info and grant-DEK AAD are not delegations. They keep their
specified chain_id ‖ network framing. The AAD encodes the
grantee’s signing key in the canonical tagged-key encoding and the grantee’s
encryption key as specified.
2. Motivation
CIP-2 Runners are stateless: each job runs in isolation and terminates. That leaves four gaps:- Runners need secrets (API keys, certificates, credentials) that only authorized Runners can decrypt, optionally after TEE attestation.
- Jobs produce artifacts larger than the 64 KiB on-chain inline cap (whitepaper §7), such as images, datasets or model weights. These need off-chain storage with a hash of the content recorded on chain.
- Training loops, agents with history, pipelines and swarms need addressable storage that persists between jobs.
- CIP-10 containers need persistent storage across restarts.
2.1 Why not existing systems?
Existing systems each fall short for Runners:
RAS takes Storj’s Macaroon-style scoped tokens, Sia and Storj’s erasure coding, S3’s path addressing and Sia’s contract-expiry cleanup, and fits them to Cowboy’s accounts, Runners and fees.
3. Definitions
- A Volume is a named, account-scoped namespace of objects. An account may own several.
- An Object is one blob in a Volume, identified by a path key.
- CBFS is the off-chain storage engine. CBFS nodes are the Relay Node data plane, and CBFS clients implement the object API, manifest handling and FUSE mount.
- A Storage Commitment is the on-chain record of a Volume’s existence, owner, size, shard placement, creation epoch and billing state.
- A Capability Token (CapToken) is a bearer token that carries a client’s permissions (read-only, write-only or read-write), scope (volume and path prefix), time bounds and size quota for one Volume. The client may be a Runner, the owner’s CLI or SDK, or another authorized caller.
- A Relay Node is a CBFS node that stores erasure-coded shards. It is a separate network role from Runners and Validators, with its own staking and incentives.
- A Shard is one erasure-coded fragment of an object chunk or manifest node; any K of its K+M shards reconstruct that chunk or node.
- A Stripe is all
w = K+Merasure shards of one object chunk or manifest node (leaf or interior), sharing oneshard_idand containing each index in0..w(0 throughw−1) exactly once. An object can span several stripes. The chain’s own record of stored data (its native inventory) and its rent assignments both count in stripes (§11.1). - An ObjectDescriptor is the immutable identity of one object: path, optional content type, write ID, aggregate plaintext hash and length, and an ordered vector of
ChunkDescriptorcommitments. Each chunk carries its own plaintext and stored-byte hashes, nonce, stored length, erasure grid and shard commitments. The manifest holds it inManifestEntry::File, encrypted for private volumes. - A PlacementRecord is the mutable assignment record for one stripe (one object chunk or one manifest DAG node). It holds the
shard_id, assignees, duplicated erasure parameters, shard hashes and proof-of-retrievability roots (PoR, §5.6), the stored-byte length before erasure padding (ciphertext_size) and a compare-and-swap (CAS)version, which a writer must match to replace the record. Relays replicate it separately from the manifest, so repair needs no manifest decryption. - The Volume DEK is the per-volume AES-256 data-encryption key. Depending on the access class (§9.1), the client wraps it to the owner’s wallet-derived key, to the CBSS committee, or both.
- CBSS (the Cowboy Secret Service) is the threshold committee that can unwrap a committee-wrapped Volume DEK (using IBE) for an authorized Runner (§9.2).
- The Dispatcher is the CIP-2 Job Dispatcher system actor (
0x02). It runs inside block processing, selects Runners (VRF plus committee filtering), issues and revokes CapTokens, and manages the job lifecycle. For CIP-9 it also applies the storage-capability filter (§5.1.1), scopes CapTokens to mount grants (§7.7), and picks the private-volume delivery path (§9.2), constructing and authorizing the CBSSSealRequestfor a committee seal. It never handles a plaintext DEK.
3.1 Canonical manifest DAG
The manifest is a content-chunked Merkle DAG ordered by path. Every read verifies against the on-chain root (StorageCommitment.manifest_root, §11.1). For a public volume any party can recompute the root from the entries with the algorithm below. For a private volume each node’s locator is the hash of its encrypted envelope, so a reader verifies each fetched envelope against its locator.
All integers below are big-endian. string(s) and bytes(b) are a u64 length followed by the raw bytes. opt(x) is u8 0 for absent, or u8 1 followed by x. The constants are:
-
The manifest is the set of
ManifestEntryvalues (File,Symlink,Directory), sorted by path in UTF-8 byte order, one entry per path. An empty manifest has the all-zero root and no nodes. -
Each entry is encoded as a leaf pair, and each child pointer as a child pair:
-
A pair hits the boundary when
h = BLAKE3(domain ‖ pair)gives(u16_be(h[0], h[1]) & 0x03FF) == 0. The test hashes each pair on its own; there is no rolling hash. -
Leaf chunking walks the sorted entries with a running byte count that starts at 14. For each leaf pair:
- A pair longer than
MANIFEST_NODE_MAX_BYTES − 14fails the build. - If the current chunk is non-empty and already holds 2,048 entries, or adding the pair would push the byte count past 262,144, the chunk is cut before the pair and the count resets to 14.
- The pair is appended and its length added to the count.
- If the chunk now holds at least 64 entries and the pair hits the boundary under
ENTRY_BOUNDARY_DOMAIN, the chunk is cut after the pair and the count resets to 14.
- A pair longer than
- A merge pass then runs over the chunk list from the second chunk on. A chunk with fewer than 64 entries is merged into its predecessor when the merged node encodes to at most 262,144 bytes and holds at most 2,048 entries. After a merge, the chunk that moves into that position is tested against the grown predecessor. A chunk that cannot merge is left as it is. The first chunk is never merged.
-
Each chunk becomes a node, encoded as:
A node’s
min_pathis its first entry’s path or first child’smin_path. Itssubtree_countis its entry count for a leaf, or the sum of its children’s counts for an interior node. -
While a level holds more than one node, the next level is built from that level’s child pairs with steps 3–5, using
CHILD_BOUNDARY_DOMAIN. The single remaining node is the root. -
A node’s stored blob is its encoding for a public volume, or for a private volume the envelope
"CBME" ‖ u8 version=1 ‖ nonce[12] ‖ AES-256-GCM(dek, nonce, node, aad). ItslocatorisBLAKE3(stored blob), andmanifest_rootis the root node’s locator.
MANIFEST_NODE_AAD_DOMAIN is b"cbfs.manifest-node-aad.v1"; its version is not part of the magic.
An insert, update or delete usually rewrites only its leaf and the interior nodes above it, and every other node keeps its locator. A byte-bound cut or a merge can shift every later leaf, so one insert or delete can repack every following leaf. The builder takes the prior DAG snapshot and returns the new root plus added_nodes and removed_nodes, so a commit publishes and bills only changed nodes. A private node whose plaintext encoding is unchanged keeps its previous envelope and locator. The set of added and removed nodes and shards is the commit’s delta. The PoR shard-hash records (§5.6) and commit authority, which is checked against the delta (§7.2), depend on this.
For private volumes the client encrypts each node with AES-256-GCM using the volume DEK directly as the key. The AAD is the literal domain bytes followed by the 32-byte volume_id, without delimiters or a terminating NUL, so manifest_root commits to ciphertext. Public nodes are plaintext, so an unauthenticated Gateway can verify the root (§5.3.2, §7.6.3).
4. Design overview
4.1 Architecture
Ephemeral Runners read and write over the network to CBFS Relay Nodes, which hold the data.4.2 Lifecycle
- The owner creates a Volume through the Storage Manager with a name, optional size quota and replication parameters. The Storage Manager writes an on-chain Storage Commitment.
- The owner attaches volumes in a CIP-2 task definition, each with a name, access mode and optional path prefix.
- The Dispatcher (or a delegated Storage Manager) issues each selected Runner a CapToken scoped to the volume, access mode, path prefix, job duration and byte quota.
- The Runner writes by splitting the object into bounded chunks, encrypting private chunks, erasure-coding each chunk into K+M shards and sending each stripe to Relay Nodes. Other token holders can fetch the shards once relays store them.
- The Runner reads each chunk in order: it fetches K valid shards, reconstructs and verifies the stored bytes, decrypts a private chunk and checks its plaintext hash, then checks the whole-object hash.
- At job completion (or periodically), the Runner commits a manifest root over all objects written to the Storage Commitment.
- The owner can list, read and delete objects or whole Volumes at any time through the Storage Manager.
4.2.1 Committee composition for storage-attached jobs
A job that mounts a volume uses the normal CIP-2 verification committee, with one prerequisite: every runner considered for a storage-attached job MUST be storage-capable (§5.1.1), meaningstorage_support = true with a valid x25519 encryption_pubkey. The Dispatcher filters before sizing the committee. If the verification committee size M exceeds the number of eligible storage-capable runners, dispatch fails with InsufficientStorageCapableRunners { required, available, reasons } (§8.2.1) and does not proceed.
CIP-2 has not settled whether storage reads may use a smaller committee (for example M = 1) than verified compute; until it does, a storage-attached job uses the compute committee size. Implementations MUST expose the storage-capable count at dispatch, so an operator can tell when committee size is the limiting factor and storage availability is not.
4.2.2 Dependency on runner-backed continuations
Except for CLI and client-app mounts, a mount comes from a runner-backed continuation (an actor’srunner.agent or runner.http job), which CIP-2 and CIP-5 materialize on chain as a deferred job_submit transaction. If that transaction is dropped, for example by a speculative-execution cache eviction, the mount job never exists and storage never sees it. Appendix D lists every check a storage job must pass.
4.3 Canonical implementation boundary
Both the CBFS data plane and the Cowboy control plane are normative parts of a conforming implementation. The CBFS data plane is responsible for:- Object encryption/decryption for
PRIVATEvolumes and plaintext handling forPUBLICvolumes. - Reed-Solomon erasure coding, shard hashing, and shard placement records.
- Relay Node RPCs (
PUT_SHARD,GET_SHARD, placement replication, repair traffic). - Manifest storage, reconstruction, Merkle root computation, and verification against the authoritative root.
- FUSE mount behavior, local cache, sync daemon, and direct object API.
- Shard repair, garbage collection of orphan shards, and local usage reporting hooks.
StorageCommitmentlifecycle, authoritativemanifest_root, and volume ownership semantics.- CapToken issuance, revocation, and task-scoped attachment semantics.
- Relay Registry membership, staking, health, and repair coordination triggers.
- Billing, fee settlement, storage grace periods, and slashing policy.
- Integration with CIP-2 task submission and the Runner execution flow.
4.4 Relationship to existing CIPs
- CIP-9 adds volume attachments to the CIP-2
OffchainTask, and the CIP-2 Runner Submission Contract accepts storage manifests alongsideresult_data. - CIP-9 adds a CIP-3 fee dimension, storage billed per byte per epoch including erasure overhead. Relay Nodes meter it outside the VM, and the chain settles it through attestation-based billing.
- Storage Commitments live in the CIP-4
STORAGEkey space under the Storage Manager’s address. Volume data is not in the MPT trie. - CIP-10 runtimes mount the volumes, attachment semantics and object API defined here. Container images, cgroups, network policy and GPU passthrough belong to CIP-10.
5. Relay nodes
5.1 Role and responsibilities
Relay Nodes are CBFS storage nodes, a role separate from Runners and Validators. A Relay Node:- Stores erasure-coded shards of encrypted Volume data.
- Stores
PlacementRecordentries that map shard IDs to their assigned nodes (see §5.3.1). - Serves shards and placement records to authorized requesters (Runners with valid CapTokens, account owners).
- Keeps its profile in the onchain Relay Registry current (§5.2).
- Runs autonomous two-phase repair (self-heal and redundancy repair) without Runner involvement (see §5.5).
- Replicates PlacementRecords to peer nodes via
ReplicatePlacementRPCs.
5.1.1 Runner storage capability
The dispatcher assigns a storage-attached job only to a Runner that advertises a storage capability (§4.2.1, §8.1). The capability is two fields in the CIP-2RunnerCapabilities record:
storage_support: bool: the Runner is willing and configured to mount volumes.encryption_pubkey: EncryptionPublicKey: the Runner’s algorithm-tagged X25519 public key, used as the recipient key for sealed DEK delivery (§9.2).
storage_support = true and publish a valid encryption_pubkey to be eligible. The dispatcher’s storage filter rejects any runner missing either. Both default to off (storage_support is #[serde(default)] false). Auto-registration derives them from the Runner’s storage configuration (a CBSS/secrets endpoint and an x25519 key). Auto-registration MUST NOT set storage_support = true unless a valid encryption_pubkey is present in the same registration.
Capabilities are fixed at registration. A second register_runner for an already-registered identity fails with RunnerAlreadyRegistered. To change storage capability, the operator MUST be able to deregister_runner (which refunds stake) and register again.
5.2 Relay registry
The Relay Registry is a system contract, like the CIP-2 Runner Registry. It manages Relay Node registration, staking, health and capacity.- The Relay Node registers with a
RegisterRelayinstruction carryingowner_address,node_id,relay_identity_pubkey, the owner and relay authorizations,endpoint,capacity_bytes, an optionalregionand an optionalstatusz_url. Registration requiresMIN_RELAY_STAKE. UpdateRelaychanges the endpoint, capacity, region or status URL and refreshesupdated_epoch. There is no separate heartbeat instruction. The auto-drain scan treats everyACTIVErelay as live, soupdated_epochdoes not trigger a drain (§5.8).- The auto-drain policy (§5.8) moves an over-capacity, under-staked or governance-targeted relay out of
ACTIVEtoDRAININGorINACTIVE, and its shards are moved or flagged for repair (§5.5). - A Relay Node may unstake after a cooldown (
RELAY_UNSTAKE_DELAY_EPOCHS) if it has no active shard assignments or has transferred them.
5.3 Shard assignment
Writes are index-aligned. Shard indexi of every stripe goes to relay_nodes[i], the relay the volume’s StorageCommitment names for that index, and to no other relay. The client does no selection, rotation or fallback:
- The volume’s
relay_nodeslist is fixed at creation and changed only by epoch-sweep rotation (§5.6.1). It holds exactly K+M distinct relays. - Each chunk’s assignment is recorded in its PlacementRecord as
relay_nodes[i]for every indexi, including an index whose shard the write could not place (§5.4.1). The record is replicated to Relay Nodes (§5.3.1). Readers and repair workers use it to find which nodes hold which shards.
chunks: Vec<ChunkDescriptor> holds one entry per chunk, each naming its own stripe with its own shard_id, shard_hashes and shard_chunk_roots. The chunk ID is object_chunk_shard_id(volume_id, write_id, chunk_index) = BLAKE3("cbfs.object-chunk.v1" ‖ volume_id ‖ write_id ‖ u32_le(chunk_index)), with the zero-based chunk position encoded as a little-endian u32. It survives renames and differs for each chunk position within one write.
Every chunk of an object MUST use the volume’s K and M. The per-chunk erasure_k and erasure_m fields duplicate the volume grid, and descriptor validation rejects mixed grids. The descriptor fields are:
ciphertext_size let them verify, reconstruct and reassign shards anyway. The CAS version makes reassignment atomic: a worker reads the current version, computes a new assignment, and writes with expected_version = current; if another node wrote first, the write fails and the worker retries.
The write_id gives each overwrite of a path distinct shard addresses, so an overwrite never replaces the shards of the previous version. The old shards stay on Relay Nodes until the new manifest commits, which keeps rollback, concurrent writers and reads from old manifests working. After the commit they are orphans, and garbage collection removes them after ORPHAN_SHARD_TTL_SECS.
A volume has two relay lists, and the protocol does not keep them in agreement. The volume’s relay_nodes is set at creation. The epoch sweep, the block phase after transactions that processes bounded storage settlement and cleanup work, replaces a departed relay in it (§5.6.1). It is the only record the chain has of which relay is responsible for each shard index. An object’s PlacementRecord.assignments records where its shards are. A write records relay_nodes[i] at every index, and repair Phase 2 (§5.5) can later move an index to another relay without telling the chain. §5.6.1 describes how the two diverge; this is a known defect.
relay_nodes is a Vec<RelayEndpoint> on the StorageCommitment. An explicit
list at create_volume MUST name exactly K + M relays with pairwise-distinct
node_ids. Any other length, or a repeated node_id, fails the whole
instruction as E_VOLUME_PARAM_INVALID (receipt code 2002) before any
instruction state read; the chain never deduplicates the list. An empty list requests
auto-assignment: the chain takes the first K + M relays in the registry’s
active-relay list that are Active and have used_bytes < capacity_bytes, and
refuses creation with E_VOLUME_RELAY_FLOOR (§12.3.1) when fewer exist. Explicit order is preserved:
relay_nodes[i] is the relay the chain holds responsible for shard index i.
5.3.1 Placement persistence
Relay Nodes store PlacementRecords independently of the manifest. Three RPCs move them:
The same
AuthProvider that gates shard operations gates all three RPCs (§7.1.1).
A PlacementRecord MUST carry its volume_id. When a Relay Node accepts a placement update or replication for a shard assigned to itself, it MUST reject a record whose volume conflicts with that shard’s existing local volume binding.
At commit, the SDK publishes each PlacementRecord to all its assignees via PutPlacement. For manifest-node records, every assignee that holds its shard MUST accept the record before the SDK submits CommitManifest; the relay at an owed index (§5.4.1) is skipped, and repair hands it the record from a peer. Acceptance is Ok, or ErrConflict for a record the assignee already holds at that version or newer. A refusal or an unreachable assignee aborts the commit before the chain call, and the client’s local commit journal (§12.1.3) keeps the commit marked “not ready” so the next attempt republishes. Object-shard records are best-effort: the SDK logs failures, and placement sync and repair fix them later.
At open, the SDK calls GetPlacement for each shard ID in the manifest. If no Relay Node returns a record for a shard (for example, on the first open of a moved volume), the reader asks every relay in the volume’s relay set for every index of that shard.
The Relay Node stores PlacementRecords in a versioned envelope: magic: [u8; 2] = [0xCB, 0xD3] || schema_version: u8 || bincode(PlacementRecord). bincode is not self-describing, so a field added without a version change makes old records decode as truncated or garbage, and #[serde(default)] does not help. On a schema_version mismatch the node treats the record as unreadable and re-derives placement from peers. The schema_version MUST be incremented on any change to the PlacementRecord layout.
Placement records name node_ids and volume_ids that are meaningful only within one chain instance. A Relay Node MUST record the genesis hash alongside its placement store and, on first boot against a different genesis, wipe the placement store. A relay that keeps placements from an old chain crash-loops on repairs it can never satisfy (§5.5 dead-placement backoff).
5.3.2 GET_MANIFEST RPC
A client can always read the manifest by traversing the DAG (§3.1) from the on-chain manifest_root, fetching each node via GET_SHARD at manifest_node_shard_id(volume_id, locator) = BLAKE3("cbfs.manifest-node.v2" || volume_id || locator). For public volumes, GET_MANIFEST returns the whole manifest in one round trip. It is the recommended path for Gateways, indexers and other public-volume consumers:
erasure_k and erasure_m because handle_get_manifest
reaches assembly holding only the AuthDecision ({allowed, max_bytes, metadata}),
without the StorageCommitment that carries them (§11.1). An implementation that
keeps the commitment in hand MAY take k/m from it and ignore the request’s
values. This CIP states no preference, because a SHOULD either way would make two
conforming relays answer one request differently. The reference implementation
passes the request’s values through.
A Relay Node
MUST range-check erasure_k and erasure_m before any peer fetch, because the
caller controls them on an unauthenticated RPC and k + m bounds the per-node
fanout of assembly. This bound is per DAG node. The number of nodes an assembly
walks depends on the manifest the caller named, and this section specifies no
total budget per request (node count, response size or rate limit). A Relay Node
that serves this RPC at volume will want one.
A relay serves GET_MANIFEST without a CapToken for a volume whose StorageCommitment has visibility PUBLIC and status ACTIVE or GRACE_PERIOD. The decision goes through the same AuthProvider entry point that authorizes an anonymous GET_SHARD and GET_PLACEMENT (§7.6.3), so the three read paths cannot disagree.
The decision never comes from stored shard metadata, although PUT_SHARD persists whatever metadata the AuthDecision carries. Metadata written at commit time cannot track later status changes: on disk, a DELETED or GARBAGE_COLLECTING volume looks like an active one until the volume-event poller removes its shards. The commitment path also lags, through a bounded-staleness cache, but by orders of magnitude less. GET_MANIFEST also names no shard, so there is no shard metadata to consult. Shard metadata therefore carries no visibility marker. It carries the shard’s volume binding, commit state, kind and tentative deadline (the time after which an uncommitted shard may be deleted).
An anonymous GET_SHARD must match the shard’s stored volume binding against the volume named in the request, so naming a public volume cannot fetch a shard of a different, possibly private, volume. When a token is present, the handler takes the authorized volume from the shard’s recorded binding and uses the caller’s claim only for a shard that carries no binding.
A relay does not serve GET_MANIFEST for private volumes. Without the DEK it cannot decrypt a node to learn its child locators (§3.1), and the privacy model forbids it from inspecting the manifest (§9.3). A DEK holder reads a private manifest node by node via GET_SHARD from the on-chain root. A relay also treats the manifest of a DELETED or GARBAGE_COLLECTING volume as absent.
The access-mode check admits GET_MANIFEST for the volume owner’s READ_WRITE token under the owner-authority rules (§7.2, §12.2.1), and does not admit it for a grant-scoped token (§7.7.1). The handler performs no visibility check of its own, so an owner’s request for a private volume reaches assembly. It fails when a manifest node decodes as an encrypted envelope, after the placement lookup and peer shard fetches, and returns no private manifest.
A Relay Node SHOULD therefore refuse GET_MANIFEST at the authorization layer for a volume whose commitment is not PUBLIC, on the token path as well as the anonymous one. Until relays do this, the access-mode check lets the request through and assembly is what fails.
The relay is untrusted, and its entry map is bound to no root. The caller MUST rebuild the manifest from those entries, recompute its DAG root, and reject unless that root byte-matches the manifest_root it supplied. The caller MUST have taken that root from the on-chain StorageCommitment. It MUST NOT cache or serve content from a manifest that fails this check. On mismatch, the caller refetches from another Relay Node and marks the offending node suspect (§15.3 reputation). Clients treat a Relay Node that does not implement GET_MANIFEST as outdated and fall back to node-by-node GET_SHARD traversal.
For the entry map, senders MUST encode each map key as its value’s entry.path(), with one entry per path. Receivers MUST bound bincode decoding by the received payload length, using the fixed-width integer encoding of bincode::serialize. Malformed or truncated payloads and DAG-construction errors MUST be rejected. Decoding permits trailing bytes.
A receiver MUST construct the canonical lookup map from the entry values’ entry.path() fields and use that same map for both DAG-root verification and subsequent path lookup. A receiver MAY reject mismatched wire keys or duplicate entry paths. If it accepts them, it MUST normalize as the reference SDK does, before computing the root: it iterates the decoded BTreeMap in wire-key order and inserts each value under entry.path(), replacing any earlier value at that path. Only this normalized, root-verified map may be cached or used to serve content. Looking up content through the original wire keys is non-conforming.
GET_MANIFEST is an optimization. Placement does not guarantee that any one Relay Node holds the whole manifest, because each DAG node is an erasure-coded object of its own. A relay answers only if it can assemble the public DAG, by reconstructing each node from K shards, from a cache it built at commit time, or both. Assembly is possible only for public volumes, because private nodes are opaque to the relay. A relay that cannot assemble the DAG does not implement the RPC, and clients fall back to node-by-node traversal. Because the caller always recomputes the root, a stale or partial cache fails verification and the caller refetches.
5.4 Erasure coding
RAS spreads each object chunk as a Reed-Solomon stripe across Relay Nodes. Governance can tune the defaults, and each volume can override them:
The writing client performs the write path:
- Split the plaintext into the volume grid’s object chunks (§14); an empty object still has one chunk. Generate one
write_id = CSPRNG(16), compute the aggregate plaintextcontent_hash, and derive each chunk’sshard_id(§5.3). - Encrypt each private chunk independently with AES-256-GCM, a fresh nonce, and AAD equal to its
shard_id(§9). Public chunks stay plaintext with a zero nonce. Record the chunk’s plaintextcontent_hash, stored-byteciphertext_hashand pre-paddingsize_bytes(which includes the 16-byte tag for a private chunk). - Reed-Solomon encode each chunk’s stored bytes into K data shards and M parity shards.
- Compute each shard’s BLAKE3 hash and small-chunk PoR Merkle root.
- For each stripe, PUT shard
itorelay_nodes[i]for every index, with the bounded concurrency of §5.4.1, authenticated by the CapToken (§5.4.1 covers refusals, owed indexes and durability). - Produce one
ObjectDescriptorwith the orderedChunkDescriptorvector and the whole object’s plaintext hash and length for the manifest, and onePlacementRecordper chunk for publication at commit (§5.3.1).
- Resolve the
ObjectDescriptorfrom the verified manifest and validate its path, lengths, uniform chunk grid and position-bound chunk IDs before allocating or transferring data. Get aPlacementRecordfor each chunk’sshard_id, from cache or viaGetPlacement. - For each chunk in order, request K valid shards from the assignees, verify each against that chunk’s manifest-committed shard hash, and fail over on unavailable or corrupt shards.
- Reconstruct the chunk’s stored bytes and strip padding using
ChunkDescriptor.size_bytes(duplicated asPlacementRecord.ciphertext_size). - Verify
ciphertext_hash, then decrypt a private chunk with its recorded nonce andshard_idAAD. - Verify the chunk’s plaintext
content_hashand deliver chunks in order. - After the full read, verify the aggregate plaintext against
ObjectDescriptor.content_hash. A failed streaming read may already have written a prefix, and callers must discard it.
Accounts may choose higher redundancy at volume creation, for example 6/10 for 1.67x overhead or 10/16 for 1.6x.
5.4.1 Write durability
Placing a write’s shards, as this section describes, comes before the write commits. A write is committed, and readable by other mounts, only once the commit carrying its new manifest root lands on chain through the parent-root compare-and-set (§7.3.1), which is also when a default--fsync commit fsync() returns (§12.1.3).
Writes are index-aligned. The client sends shard i of each stripe to
relay_nodes[i] and to no other relay, authenticated by the CapToken. Uploads
run concurrently up to max(1, floor(8 MiB / shard size)) at a time, capped at
the stripe width, so an 8 MiB shard uploads one index at a time. Each answer has one of three outcomes:
Okplaces the shard.ErrCapacity, or a relay that cannot be reached, leaves indexiowed.PutShardreturnsErrCapacityfor the node-wide cap, a per-authorization byte cap and a failed fsync.- Any other refusal (authorization, an invalid request, a conflict) fails the write whatever the other indexes did, so the real cause is reported. The client lets the attempts already started finish before it returns the refusal.
K of its K + M shards are placed. With fewer
than K placed shards the write fails. A stripe with owed indexes is a
degraded write. Its PlacementRecord still names relay_nodes[i] at every
index, so the relay at an owed index knows what it has to reconstruct.
The commit carries the owed indexes as owed_shards, a list of
OwedShardRef { shard_id, shard_index }. The chain requires each owed index to
name an added shard, at most once, and at most M owed indexes per stripe, so
every committed stripe has at least K placed shards. A degraded commit lands
like any other commit. The manifest-node stripes that commit publishes
(§5.3.1) follow the same rule, and their owed indexes join the same list.
An owed index is credited to no relay (§5.6.1), earns no rent and is never
challenged by PoR. Repair Phase 1 (§5.5) on the relay at that index reconstructs
the shard from K peers. That relay then submits SettleOwedShards, signed by
its identity key over the volume’s current relay_set_revision, its node_id
and, for each shard, the stripe incarnation (added_at_event_seq) and the
index’s chunk_root. The chain checks that the signer is relay_nodes[i], that
the index is still owed and that the incarnation and root match the stripe row.
It then credits the relay with the shard and admits the index to PoR.
A landed write, including one that a successful fsync acknowledged
(§12.1.5), therefore rests on at least K shards per stripe. A stripe with o
owed indexes survives the loss of M − o more shards until repair re-creates
the owed ones. A stripe committed with exactly K shards has no loss margin
until then. Repair runs every REPAIR_CHECK_INTERVAL_SECS (300 s) and can
re-create an owed shard only while the relay at that index is up and K peers
serve their shards.
The record must also reach a relay. Repair walks only records that relays hold,
and object-shard record publication is best-effort (§5.3.1, §12.2). An owed
index is repaired only after at least one relay holds the record.
§5.4.2 has to work around three properties of the built system:
- The client holds every shard, so it computes every
shard_hashand PoRchunk_rootand commits them in theObjectDescriptor.PutShardcarries only{volume_id, bytes}, so the receiving relay has no hash to check and stores what it is given. The descriptor, which the on-chain manifest root commits to, is the only unforgeable copy of those hashes. Repair, drain andGetPlacementverify againstPlacementRecord.shard_hashes, an unsigned duplicate that relays serve. - Only a client creates a new assignment. Relays write records (the repair Phase 2
leader CAS-updates one, §5.5 step 5; the drain/rebalance worker builds a
successor;
ReplicatePlacementaccepts a peer copy if the sender already appears in itsassignments, §7.1.2), but no relay adds an index or a position it did not already hold. In the drain push (§7.1.2) the draining relay supplies theold_record, so the assignee check proves only self-consistency. Two rules contain the push: §7.1.2’s MUST that the sender’s registry status beDraining, and the destination writing the shard as bound to the volume but uncommitted. Only the volume’s own on-chain add event, from a commit that adds the shard, promotes it to committed. - Relays do not check record coverage.
validate_placement_recordboundsassignments.len()atK + Mand eachshard_indexbelowK + M, but permits duplicateshard_indexandnode_idvalues. A writer’s record names eachrelay_nodes[i]once, and create-time validation makes those relays distinct (§5.3).
5.4.2 Server-side fan-out (proposed, not implemented)
The write path costs the clientK + M shard uploads of
ceil(ciphertext_size / K) bytes each, (K + M) / K of the padded ciphertext
(1.5× at K=4/M=2, more for small objects). Relays already rebuild shards from
peers during repair. The proposal reuses that on the write path: the client
uploads once, and relays send the remaining shards to each other.
Uploading only the K data shards makes every stripe a degraded write with M
owed indexes (§5.4.1), at zero parity until repair re-creates each one from K
peers. Any design in this space MUST produce a record that names all K + M
assignees at publication time, whatever subset of them the client itself talked to.
A design that fits the existing invariants keeps the client as author of the
descriptor and the complete record, and moves only the bytes. Per object-chunk
stripe:
- The client encrypts private chunks and erasure-codes each chunk locally, so it holds every shard’s bytes, hash and chunk-root without uploading them.
- It takes the
K + Massigneesrelay_nodes[0..K+M]as today and designates exactly one as the ingest relay. Several ingest relays would either each hold fewer than K shards, too few to reconstruct, or cost the client more uplink than today. - It uploads the K data shards to the ingest relay, together with the complete assignment list, the per-shard hashes and the erasure parameters.
- The ingest relay reconstructs the
K + M − 1shards it is not itself keeping, verifies each against the client-suppliedshard_hash, and pushes it to the named assignee. - The client publishes the
PlacementRecordnaming allK + Massignees, and the chunk descriptor with allK + Mhashes and chunk-roots, exactly as today.
shard_size = ceil(ciphertext_size / K),
zero-filled to K × shard_size), the empty-chunk case, and the Reed-Solomon
generator matrix. §5.4 does not pin it, and fan-out cannot be specified until it
is pinned.
Today the write returns once at least K shards of each stripe are on their
relays (§5.4.1). With fan-out that becomes two states. A stripe is accepted when the ingest relay holds its K
data shards and owes delivery of the rest. The object may still be unreadable:
readers fetch only from the record’s assignments, one address per index, so the
ingest relay can serve one shard per stripe, and a readable object needs delivery
to at least K assignees in every stripe. A stripe is placed when every named
assignee holds its shard. A commit of an accepted but
unplaced stripe asserts a placement that does not exist yet. A proposal must say
whether that is permitted and what bounds the window.
5.4.2.1 Unsolved problems
A proposal must address each of these obstacles in the built system:- PoR liability does not follow the bytes, even today. A challenge binds
responsible_node_idtocommitment.relay_nodes[shard_index](including epoch-sweep replacements), not to the record’s assignee, and §5.6.1 lists the ways the two lists diverge. Every repair Phase-2 reassignment already aims liability at a relay that may not hold the shard. Fan-out widens this gap and must decide which list is authoritative. - An assignee cannot reject a mis-delivered shard, because
PutShardcarries no hash. It stores the bytes and fails its next challenge. That lands on the fraud row of §5.6, an order of magnitude above the miss row, although the implementation has no fraud branch and folds an invalid proof into the miss counter. Either way, wrong bytes from the ingest relay leave the assignee accountable. - There is no authorization for the push, and one existing MUST forbids it.
§7.1.2 states that the sender of a
PeerRelayPushShardMUST beDraining, which excludes anActiveingest relay; relaxing that gate would let any registered relay push. ReusingPutShardwould hand the ingest relay the client’s CapToken, a bearer credential scoped to the whole volume (including deletes) whose path prefix relays do not enforce. No shard-scoped, single-use credential exists. - An undelivered index self-heals at a cost. Phase 1 pulls K shards from peers for each undelivered index, on top of the fan-out traffic, and a client can trigger this with one upload to a slow ingest relay. PoR then avoids slashing only if self-heal wins the race, which the repair budget and a 300 s cycle bound by timing alone.
- Surplus copies on the ingest relay become undeletable and unpaid. It holds K
data shards but is the named assignee for at most one. At commit its
volume-event poller promotes any local copy matching
(shard_id, shard_index)and the volume binding tocommittedwithout checking node id. GC never deletes a shard marked committed, so those bytes stay indefinitely. Rent flows only to named assignees. - Today a relay receives at most one shard of each stripe (about
1/Kof that chunk’s stored bytes), because each index goes to its own relay. An ingest relay with K distinct indexes can reconstruct the chunk, and with K for every stripe, the whole object, in plaintext for a PUBLIC volume. With (5) it may keep them permanently. The current threat model does not cover this. effective_size_deltais committed for allK + Mshards, so the owner is billed from the commit for placement that may not exist yet. Relays earn rent only with a valid PoR response, so an assignee that never received its shard earns nothing.- Nothing bounds the relay’s retries: how long an ingest relay retries a delivery, how many it may owe at once, or what it does with bytes for a delivery it can never complete. A capacity refusal during fan-out has no fallback, because the ingest relay cannot rewrite the assignment list.
- If Phase 2 reassigns an index while its delivery is in flight, the bytes land on a relay the record no longer names. By (5) the commit pins them there, and the new assignee self-heals by (4).
- The client has no usable evidence.
write_objectreturns a receipt the client builds itself, and no relay signs anything on the write path, so an ingest relay can acknowledge and discard undetectably. The drain path has the needed shape, a destination-signed absorption receipt settled on chain; fan-out needs one at ingest and at delivery. - With one ingest relay the client sends K shard-units and relays send
K + M − 1, so the network moves2K + M − 1: 9 units at K=4/M=2, against 6 today. §10.1 charges data transfer “at read/write time” and §10.4 routes it to the relay serving the shard; neither covers a relay-to-relay leg. The ingest relay also picks the delivery order, so it decides which assignees can serve reads, and earn fees, during the accepted window.
5.5 Autonomous shard repair
Relay Nodes run repair themselves, with no Runner and no onchain timer. Each Relay Node periodically walks the PlacementRecords it holds and runs two phases. Phase 1 (self-heal) checks local shards. For each PlacementRecord that names this node as a holder:- Verify the local shard bytes against the expected
shard_hashfrom the PlacementRecord. - If the shard is missing or corrupt, fetch K healthy shards from peer Relay Nodes listed in the same PlacementRecord.
- Reconstruct the ciphertext using Reed-Solomon decoding and truncate it to
ciphertext_size. Truncating to the padded shard length produces garbage. - Re-encode and extract the needed shard. Verify its hash. Store it locally.
- For each PlacementRecord, probe all assigned nodes. A node counts as dead if it is unreachable after the configured timeout.
- Only the live assigned node with the lowest
shard_indexdrives reassignment for that PlacementRecord, so nodes do not race to repair the same shard. - The leader selects replacement node(s) from the known peer list, excluding already-assigned nodes.
- The leader reconstructs the missing shard(s) from K surviving shards, as in Phase 1, and uploads them via
PutShardto the replacement node(s). - The leader writes an updated PlacementRecord with the new assignments and an incremented
versionthrough a CAS write: thePutPlacementRPC specifiesexpected_version = current_version. If another node already updated the record, the CAS fails and the leader retries from step 1 on the next cycle. - The leader replicates the updated PlacementRecord to all assigned nodes via
ReplicatePlacement.
REPAIR_CHECK_INTERVAL_SECS = 300 s.
A PlacementRecord whose assigned nodes are all unreachable (0 of K+M live) cannot be repaired, because no shard survives to reconstruct from. The repair worker MUST NOT re-attempt such a record every cycle. It applies exponential backoff (2^n cycles, capped at MAX_REPAIR_BACKOFF). After LOST_PLACEMENT_CYCLES consecutive all-dead observations, it marks the placement LOST and stops until a new placement supersedes it. Without this rule, stale placements (for example, across a re-genesis, §5.3.1) make the worker log “0/N shards available” every cycle without bound.
Repair takes everything it needs from PlacementRecords: erasure params, shard hashes, ciphertext_size and assignments. StorageCommitment.degraded_shards is a reporting counter and does not trigger repair. Repair Phase 1 also re-creates owed shards (§5.4.1).
5.6 Proof of retrievability (PoR) challenges
A Relay Node that staysActive can still have lost data, or withhold it (keep it and refuse to serve it). RAS issues random PoR challenges to check that Relay Nodes hold the shards they claim.
A challenge runs in three steps:
- A periodic onchain timer (CIP-5, every
POR_CHALLENGE_INTERVALblocks) selects a random set of(shard_id, shard_index, byte_offset, byte_length, nonce)challenges targeting active Relay Nodes. Both the direct global sampler and the bounded volume-directory builder MUST excludeGARBAGE_COLLECTINGvolumes before drawing, even while their native inventory rows remain for cleanup (§13.3). Thenonceis fresh per challenge. Consensus reads the stripe’s current row in the chain’s native RAS inventory, keyed by(volume_id, shard_id), and copies the selected index’schunk_root, the stripe’sshard_size, and the sequence number of the event that published the stripe into the challenge (§11.1). The PoR weight page, the paged table that PoR draws its random targets from, carriesw = K+M, so choosing targets at block start needs five reads and no extra commitment read. - The challenged Relay Node must respond within
POR_RESPONSE_WINDOWblocks with:- The requested byte range of the specified shard. The
nonceseeds which range is asked, so a relay cannot precompute or replay a response. - A range-inclusion proof: a Merkle path from the fixed-size chunks (e.g. 4 KiB) covering the range to the shard’s chunk-root, the Merkle root over that shard’s chunks.
- The requested byte range of the specified shard. The
- The onchain verifier checks against the issue-time
snapshot_chunk_rootcarried in the challenge:- the returned bytes hash into the chunk leaves of the range-inclusion proof, which resolves to a chunk-root;
- that chunk-root equals the selected stripe index’s snapshotted chunk-root;
- if either fails, the response is invalid.
POR_RESPONSE_WINDOW = 50 blocks, plus a congestion grace
of POR_RESPONSE_GRACE = 20 additional blocks.
Set deadline_block = saturating_add(issued_at_block, 50) and
grace_deadline = saturating_add(deadline_block, 20) using u64 saturation.
A still-open challenge accepts an otherwise valid response at
block_height <= grace_deadline. Unanswered-challenge settlement is permitted
only at block_height > grace_deadline: through issue height + 70 inclusive
for responses, with settlement first possible at +71. At u64::MAX saturation
no later settlement height exists. The grace is a fixed consensus value that no
node-local estimate or per-challenge extension changes.
A relay that discarded the shard cannot produce valid chunks against the snapshotted chunk-root, and cannot reuse an old response because the nonce moves the challenged range. The per-shard chunk-root is public and covers erasure-coded ciphertext, so the chain verifies private and public volumes the same way without the DEK. manifest_root cannot anchor PoR, because it commits to ciphertext manifest nodes (§3.1). Only the challenged range is transferred.
If the volume commits between issuance and response, the relay still proves against the issue-time snapshot_chunk_root. If a commit removed the challenged shard, or the volume enters GARBAGE_COLLECTING before an unanswered challenge settles, settlement MUST void that challenge, even if its native row remains during bounded GC. A later re-addition does not rewrite the pending snapshot. DELETED remains challengeable, because it is reversible and the relay must still retain the shard.
The slashing ladder runs only when governance has set
cip31.cbfs.por_slashing_enabled, which defaults to unset. While it is unset, a
settled miss emits a ras.por.miss_alarm event and records nothing else, since
the consecutive-miss counter is written inside the slashing path. The table
gives the behavior with slashing enabled; §5.6.2 lists what must hold first.
PoR spot-checks random byte ranges and does not seal sectors or re-prove continuously as Filecoin’s Proof-of-Spacetime does.
1% of every per-epoch storage-fee batch funds the PoR challenge pool at
0x0B (STORAGE_FEE_CHALLENGE_POOL_BPS = 100; split in §10.4 and CIP-31 §4).
Submitting IssuePorChallenge { volume_id, shard_id, shard_index, byte_offset, byte_length } requires the caller to escrow RELAY_CHALLENGE_BOND (default 66 CBY, Tier-0 tunable; CIP-31 §7) at 0x0B for the challenge window. The bond settles as follows:
- After a valid response or unconfirmed miss, the bond is refunded minus
POR_CHALLENGE_FEE(default 7 CBY, retained by0x0B). A rejected invalid response does not resolve the challenge. - After a confirmed third consecutive miss with slashing enabled, the bond is refunded in full, and the challenger may receive
CHALLENGER_BOUNTY(default 33 CBY) from the PoR challenge pool, bounded by the pool balance and its epoch cap. TheRELAY_EVICTION_PENALTYslash is capped by actual stake and distributed under CIP-31 §8. This path does not chargePOR_MISS_PENALTYorPOR_FRAUD_PENALTY. The slashed-stake 10% share is burned; the storage-fee 10% share goes to the Platform Fee Account (0x18). - After a void, the full bond is refunded, with no
POR_CHALLENGE_FEEand no bounty.
released = min(challenge.bond, escrow_balance),
fee = min(configured_fee, released) on fee-retaining outcomes, and
refund = released - fee. Void and confirmed-slash outcomes retain zero fee.
The escrow debit is released, and refund + fee == released, so a fee raised
above an older challenge’s bond cannot underflow its refund.
CIP-31 §7 defines one nano-CBY scale for balances, escrow, pool reserves and
bounty caps, with checked conversion from whole-CBY parameters before
arithmetic. The current handlers still use raw 66/7/33 base units and must be
reconciled before slashing is enabled. The CBY defaults here are not
already-scaled on-chain amounts.
Before paying a bounty, settlement MUST keep
effective_reserve = max(configured_pool_reserve, next_fire_max_cost) in the
pool, with next_fire_max_cost computed from the scheduled timer’s resource
limits and applicable fee prices, recomputed when either changes. When the pool
balance is below that reserve the bounty is zero. The reserve does not protect against future price
increases or an unfunded pool. The current reserve defaults to zero (CIP-31 §7).
The genesis-seeded PoR timer names fee_payer = STORAGE_MANAGER (0x0A) as a
marker that selects native funding; it does not debit the Storage Manager’s account.
Before handler execution, the block timer phase reserves max_cost directly
from 0x0B/CHALLENGE_POOL_BALANCE_KEY, and unused reservation returns to that
pool, including after handler failure. The net pool debit is the consumed timer
fee, with the ordinary burn/tip distribution. Challenger bond escrow cannot fund
timer gas or bounties.
max_cost includes all active timer resource meters, including the Access
reservation in the Access-meter release. If the pool is insufficient or a
relevant system actor is paused, the timer phase skips the handler, emits
por.challenge_paused, and re-arms the same timer for the next interval. A
successful fire is also re-armed natively (or continued next block for pending
bounded work). These rules override the ordinary removal of a timer after it fires and
its deletion for insufficient funds. Both re-arms preserve the timer ID and economics
and reset skip/retry counters. Ordinary actors cannot select this funding
source, because CIP-5 §4.2 permits a system payer only for its own actor.
5.6.1 Who a challenge holds liable
The chain names the liable relay fromcommitment.relay_nodes[shard_index] and
binds it into the challenge at issue time as responsible_node_id, for both the
manual and automatic issuers. An already-issued challenge MUST retain its issue-time
relay: response authorization and miss settlement use that recorded identity,
even if the epoch sweep later rotates the relay at that index.
After settling a volume’s storage epoch against the pre-rotation commitment,
the sweep decides relay replacement within its remaining bounded work budget.
Work that does not fit waits for a later epoch.
- Only an
ACTIVEvolume is eligible. An index needs replacement when its relay profile is missing or its registry status is notActive. An assigned relay that isActiveis not replaced, even when full. - For each such index, in index order, the sweep takes the first entry in the
registry’s active-relay list that exists, is
Active, hasused_bytes < capacity_bytes, and is not already held by this volume or selected earlier in the pass. The sweep reserves no capacity on the replacement for the shard. Without a replacement, the old assignment stays responsible and gets no new repair grace. - The sweep does not apply its decision. It stores it on the commitment as
pending_rotation { decided_epoch, replacements }, wherereplacementslists(shard_index, RelayEndpoint)pairs. The volume’s nextFinalizeVolumeCheckpointapplies it in the same transaction, after every update anchored at the old relay set is checkpointed. If none comes, the sweep applies it itself in a later epoch oncecurrent_epoch > decided_epoch. - The handoff (applying the rotation) replaces each index’s
RelayEndpoint, moves the departing node’sper_relay_bytesentry (zero if absent) to the replacement, incrementsrelay_set_revisionand records the rotation block for each(volume_id, shard_index). ACommitManifestorSettleOwedShardsthat names an olderrelay_set_revisionis refused. The handoff moves commitment attribution only. It does not copy shard bytes or update thePlacementRecord, volume sizes, or native shard inventory; repair still has to place the bytes.
POR_RELAY_REPAIR_GRACE_BLOCKS = 7,200 blocks without new PoR
challenges for that volume/index. While
height < rotated_at.saturating_add(7_200), manual issuance rejects with
ras.por_relay_repair_grace before debiting a bond, and automatic
issuance skips the target. Both read the current rotation marker, including
when automatic sampling used the parent state. The exclusion ends at the exact
end height, and a later rotation resets the marker. The grace does not certify
that repair completed or cancel an already-issued challenge.
relay_nodes[shard_index] is the chain’s only per-index notion of
responsibility. A commit carries no per-relay byte deltas. The chain derives
them under the commit’s (prev_root, relay_set_revision) fence: an added shard
at index i credits relay_nodes[i] with its shard_size unless the commit
owes it (§5.4.1), and a removed shard debits the relay now at index i unless
that index is still owed. Writes are index-aligned (§5.3), so a write’s
PlacementRecord names the same relay at each index. This section is the
definitive statement of how the two can still differ (see also §5.4.2.1(1)).
The two lists diverge in two ways:
- Repair reassigns. Phase 2 selects replacements from the repairing relay’s
own peer table, unrelated to the volume’s relay set, so a repaired index can
leave
relay_nodesentirely. The chain is not told. - A relay leaves. An index naming a
node_idthe registry never held settles with only a miss alarm. A drained, self-disabled or evicted relay’s profile still loads. Until an eligible volume’s sweep finds a replacement within budget, new challenges bind that relay. During the replacement’s repair grace, issuance excludes that index. After the grace, new challenges bind the new relay. Existing challenges keep their issue-time relay. Rotation does not establish which relay physically holds the shard.
- The named relay is challenged for a shard it does not hold, cannot answer, and builds up consecutive misses toward eviction.
- The relay that holds the shard is never challenged for that index, so the shard is unaudited.
- The chain-derived
per_relay_bytescredit followsrelay_nodes[i]and does not prove which relay holds a shard. Rent payout uses prior-epoch valid PoR and assignment evidence and ignores the byte ledger.
CBFS_MANIFEST_COMMIT_WIRE = 1 in the node’s fingerprinted activation.policy
map. A missing or mismatched capability refuses new mutations, signatures,
and submissions. A future incompatible manifest wire change MUST
advance this policy value, which is not an activation height.
5.6.2 Before slashing is enabled
The slashing ladder in §5.6 (penalties that escalate with consecutive misses) has no effect untilcip31.cbfs.por_slashing_enabled is set.
These requirements apply:
- How a relay consents to an assignment is not yet settled. Explicit relay lists require registered identities and use registry endpoints. Registration does not prove that the relay accepted this volume or received its shards, so an owner can name another registered relay and challenge bytes never delivered to it.
- Relay-authorized responses are implemented and active from genesis. The
submitter may be a relayer, but the answer MUST carry the responsible relay’s
authorization. The verifier reconstructs the signed payload from chain state
(
chain_id,challenge_id,shard_id,shard_index, under the response operation domain) and checks the registered relay identity key, owner, node ID and epoch. It rejects an unbound challenge or invalid authorization. Anonymous access to public shards does not bypass this check. The range proof establishes retrievability and the signature establishes relay consent; neither establishes continuous dedicated storage. - CIP-31 §7’s unit-scale, timer-reserve, and chain-observable, relay-consented
assignment requirements MUST be enforced before the slashing flag can take
effect. Merely setting the governance parameter MUST NOT be sufficient in the
release implementation. The current flag reader only checks for
1. - Per-index liability must correspond to something the chain can observe (§5.6.1).
- Make repair placement follow
relay_nodesas writes already do, so the chain’s model is true by construction. This gives up repair’s free choice of replacement, which keeps one unreachable relay from stranding a repair. - Add reassignment by an instruction the incoming relay signs, linking repair
to the on-chain assignment. The epoch sweep already mutates
relay_nodes(§5.6.1) without consent, and the chain already keeps a per-volume relay-assignment snapshot. - Put relay identity in the commit’s shard references. The commit is
owner-signed, so the relay must counter-sign, as the drain path’s
destination-signed absorption receipt already does. Any such attestation MUST bind
volume_id,shard_id,shard_index,chunk_root, the chain id and a freshness window, or one attestation is replayable into another volume. - Drop per-index liability: challenge the volume and let any holder answer. Payout already requires a valid response. This gives up naming a specific relay as having lost specific bytes.
5.7 Relay node incentives
Relay Nodes earn fees from two sources:- The relay share of the per-epoch storage fee paid by the account owner, weighted by valid PoR responses and assignment age (CIP-31 §5).
- Reader-authorized transfer fees for complete logical shards, counted per rounded MiB and paid from dedicated funded channels (§10.4.1). A relay’s own byte report does not set them.
used_bytes counts stored, erasure-expanded bytes. The first unsettled epoch is billed at epoch_peak_used_bytes, the largest used_bytes since the last settlement, and each later unsettled epoch at the current used_bytes. Both use the storage fee in the fee snapshot the epoch sweep recorded when it queued the epoch. The rent-floor policy stamped on the volume sets only its minimum escrow, not the rent rate. shards_lost (§5.6) is a counter only. No assignment reads it: auto-assignment and rotation take the first Active relay with free capacity.
5.8 Relay drain governance
Draining a relay moves its shards to other relays and releases it. Three triggers start a drain:- The operator initiates a manual drain, posts a drain bond, and gets a window to move the shards.
- The per-epoch storage-settlement scan auto-drains over-capacity or under-staked relays under the governance policy.
- Governance forces a drain of a silent or misbehaving relay whose operator will not cooperate.
SubmitProposal=45 / CastVote=46 / ExecuteProposal=47) and share the Proposal storage schema at GOVERNANCE_SYSTEM_ACTOR=0x09, each with its own submit-proposal opcode. The payload discriminator ProposalPayloadKind (CIP-12 §5.4) includes DrainRelay (opcode 85) and UpdateAutoDrainPolicy (opcode 86) among its variants.
The Proposal record carries two payload fields, populated by variant: payload_relay_node_id: Option<[u8; 32]> (DrainRelay) and payload_auto_drain_policy: Option<AutoDrainPolicyConfig> (UpdateAutoDrainPolicy). Both are None for every other variant.
SubmitDrainRelayProposal (opcode 85) has the wire format { description_hash: [u8; 32], voting_blocks: u64, node_id: [u8; 32] }. It requires MIN_VOTING_BLOCKS ≤ voting_blocks ≤ MAX_VOTING_BLOCKS (else InvalidData) and standard base gas, and any account may submit it. The target node_id is validated only at execution, where an unknown node_id yields InvalidData. Submit writes an Active Proposal (payload_kind = DrainRelay, payload_relay_node_id = Some(node_id), deadline = block + voting_blocks) and emits governance.proposal.submitted. ExecuteProposal after a passing tally reads the relay’s RelayNodeProfile from RELAY_REGISTRY=0x0B and writes the node into its auto_drain_governance_scan slot; the next epoch’s storage-settlement loop performs the drain. A governance-initiated drain has no operator drain bond. It reuses the drain windowing, shard-absorption receipts, and stake-reservation and slashing logic, with the relay’s existing stake as the slashable collateral.
SubmitAutoDrainPolicyProposal (opcode 86) has the wire format { description_hash, voting_blocks, enabled, high_water_mark_bps, over_capacity_epochs, stale_heartbeat_epochs, liveness_window_epochs, min_active_stake, drain_bond_base, drain_bond_per_gib, drain_window_epochs }. It writes the network-wide AutoDrainPolicyConfig:
Submit and
ExecuteProposal both validate the ranges above; enabled and the stake and bond fields may be zero. Execution writes the policy to 0x0B:AUTO_DRAIN_POLICY_KEY, and later epochs apply it in the auto-drain scan.
Drain-relay and auto-drain-policy proposals are Tier-0 governance actions under CIP-12 §5.1, with no proposer-selected tier field. Opcodes 85/86 are pinned in the CIP-13 master opcode table. A draining relay’s outstanding CBFS rent obligations follow the moving shards to the absorbing relay; CIP-31 owns the per-shard rent accrual the drain window is sized against.
6. Storage addressing
6.1 Path-based namespace
Every object in RAS is addressed by a three-part key:account_addressis the owning account’s 20-byte protocolAddress, hex-encoded. It is the sameAddresstype used protocol-wide and carried byVolumeAttachment.owner(§8.1).volume_nameis a UTF-8 string of at most 64 bytes, restricted to[a-zA-Z0-9_\-.].object_pathis a UTF-8 string of at most 512 bytes that uses/as a logical separator. It has no leading or trailing/.
6.2 Content integrity
The manifest commits a whole-object plaintext hash and separate hashes for every ordered object chunkj:
ChunkDescriptor.shard_hashes[i] is the whole-shard integrity hash that repair uses. shard_chunk_roots[i] is the Merkle root over shard i’s small fixed-size PoR chunks (e.g. 4 KiB). These small PoR chunks are distinct from the bounded object chunks. For public object chunks the stored bytes equal the plaintext, so the two per-chunk content hashes are equal. There is no single object-level ciphertext hash.
A successful commit_manifest delta writes the chunk root into the shard index’s entry in the stripe’s row of the chain’s native RAS inventory, and PoR snapshots that row when it issues a challenge (§5.6). The corresponding ChunkDescriptor carries both shard commitments, and its PlacementRecord duplicates them. Every added and removed ShardRef in a commit delta MUST carry the shard’s real, nonzero chunk_root and shard_size. For a removal, the Storage Manager MUST require the existing stripe row and reject a zero or mismatched chunk_root at any index, or a mismatched shard_size. A removal cannot identify a shard by (shard_id, shard_index) alone. Validation MUST carry the loaded rows into mutation rather than reading them again. The worst-case Access reservation for inventory work is 3 reads and 4 writes (3R+4W) per removed stripe and 1 read and 2 writes (1R+2W) per added stripe, before rent-assignment work (§11.1; CIP-43 §5.5).
Content hashing uses BLAKE3 throughout. The integrity checks run off chain, so onchain hash compatibility does not matter.
The onchain manifest root (§3.1) lets any party verify each stage of a read. Shard hashes verify shards, per-chunk stored-byte hashes verify reconstruction, per-chunk plaintext hashes verify decryption, and the aggregate plaintext hash verifies the whole object. A path from a leaf node to the root is O(log N) in the number of objects. For a public volume this path proves object membership. For a private volume the nodes are ciphertext, so the chain cannot verify membership from manifest_root alone, and PoR instead checks shard-level retrievability against the public per-shard chunk roots in the native RAS inventory (§5.6).
6.3 Addressing for deletion
The account owner deletes objects by their full path:delete_object marks the object’s shards for removal on the assigned Relay Nodes and updates the onchain manifest, and the relays garbage-collect the data. delete_volume is a soft delete: it sets the status to DELETED and starts the GC grace timer (§13.2), and the owner may call undelete_volume() until the window ends. The StorageCommitment and rent persist through the grace window. After it, the volume is garbage-collected and fees cease. Only the account owner, or an Actor acting for the account, may delete. Runners never receive delete permission, even in read-write mode.
7. Access control
7.1 Capability tokens
Access to a Volume goes through Capability Tokens (CapTokens), modeled on Storj’s Macaroons and UCANs:AuthProvider) and the validator (through validate_token) honor a presented token only if it matches that stored copy byte for byte, so forging a token requires compromising consensus. The signature field is reserved for a future client-verifiable token form (§17.3).
Ending a job (completion, cancellation or timeout failure) does not immediately revoke its CapTokens or release its write-prefix reservations. Issuance stores a bounded per-job token-hash index (job_cap_tokens:, at most MAX_ATTACHMENTS_PER_JOB = 64 hashes, each 32 bytes). The CIP-2 admission-owned retained-record reap, subject to its pause and damaged-state qualifications, deletes each indexed ras:token:{hash} row and then the index. A token stops being authoritative when the reap deletes it (including after cancellation during private key delivery) or when valid_until passes. Public M=1 retry deletes the outgoing tokens before replacing the runner handle. Native route-session revocation has its own immediate lifecycle, and the later reap’s deletion is idempotent.
Until then, the runner of a terminated job may still use the token’s scope, and overlapping write prefixes stay reserved until a later issuance finds them expired and releases them. Immediate revocation and prefix release at job termination are not yet specified; adding them requires re-deriving the BlockStart timeout budget.
7.1.1 Auth enforcement on placement RPCs
The sameAuthProvider that gates shard operations also gates GetPlacement and PutPlacement. A request without a valid token, or with an expired or revoked token, MUST be rejected before the Relay Node reads or writes any placement data.
A leaked PlacementRecord would tell an attacker which relays to target with denial of service. For public volumes, placement reads follow the open-access rule for shard reads (§7.6.3).
7.1.2 Node-to-node operations are not CapToken-gated
Relays send each otherReplicatePlacement, SyncPeers, the drain operations PeerRelayPushShard and PeerPlacementUpdate, and the audit read AuditGetShard. The calling relay’s identity authorizes these operations, so they are not gated by AuthProvider and MUST NOT be. They carry an empty auth_token.
Each carries a relay assertion instead. The calling relay signs a domain-tagged digest with the identity key registered in its on-chain RelayNodeProfile, and the receiver verifies it against relay_identity_pubkey from the relay registry. The receiver MUST check all of:
- the signing timestamp is within its accepted clock-skew window;
- the sender resolves to a registered relay whose on-chain status is
ActiveorDraining.DisabledandInactiveMUST be refused, since revocation can only come from the registry and never from the signature; - the signature verifies against that profile’s
relay_identity_pubkey; - the digest covers the payload being acted on. A signature that binds only the sender can be replayed with attacker-chosen content.
ReplicatePlacementandSyncPeersare both writes, so such a replay would corrupt placement records.
ReplicatePlacement: the sender MUST already appear in the record’sassignments. Without this rule any registered relay could write any record for any volume.SyncPeers: the advertisedsender.node_idMUST equal the signing identity, so a registered relay cannot advertise another node’s id at an address it controls.PeerRelayPushShard/PeerPlacementUpdate: the sender MUST additionally beDraining, since only a draining relay sends a drain push.
7.2 Access modes (CapToken scopes)
The following modes apply to CapToken-gated access on private volumes:
WRITE_ONLY is the default for most Runner jobs: the Runner produces output and cannot inspect existing data. On a private volume the Relay Node refuses a
WRITE_ONLY token for GET_SHARD, GET_PLACEMENT and listings. On a public volume anyone can read without a token (§7.6), so a WRITE_ONLY mount can still refresh the manifest and learn the object namespace (§12.1.3); a grant that must hide a volume’s contents needs a private volume. READ_ONLY suits a Runner that only consumes data, such as loading model weights for inference. READ_WRITE suits a Runner that reads and updates the same volume, such as an agent updating its memory.
Writing shards (PUT_SHARD) and committing the manifest need separate authority, because a commit replaces the volume’s single manifest_root. §7.3.1 specifies commit authority.
7.3 Concurrent CapTokens
Multiple CapTokens may be active on the same volume at once. In the agent swarm pattern, a coordinator Runner holds a READ_WRITE token while sub-agent Runners hold WRITE_ONLY tokens scoped to disjoint path prefixes. These rules apply:- If two CapTokens grant WRITE access, their
path_prefixvalues MUST NOT overlap. The Dispatcher enforces this at token issuance.- The Dispatcher canonicalizes a prefix to end in
/, soagent-1is stored asagent-1/. Prefix A overlaps prefix B if, after canonicalization, either is a prefix of the other:agent-1/andagent-1/sub/overlap, andagent-1/andagent-10/do not. - An empty prefix (
"") means full volume access. If any WRITE token has an empty prefix, no other WRITE CapToken may be active on the volume at the same time. READ_ONLY tokens are not counted.
- The Dispatcher canonicalizes a prefix to end in
- Reads never conflict. READ_ONLY tokens may coexist with any other tokens, and a READ_WRITE token may read paths that other tokens are writing.
- Concurrent writes to different paths are independent. There is no global write ordering across CapTokens.
- There is no token revocation list. A token stops being authoritative when the job’s reap deletes its stored copy, when
valid_untilpasses, or when an ownership transfer advanceslast_owner_transfer_blockpast itsvalid_from(§7.1, §13.4).
7.3.1 Prefix enforcement boundaries
Prefix enforcement happens at three layers. Layer 1 is the onchain check at issuance. Before issuing a WRITE CapToken, the Dispatcher checks the requestedpath_prefix against all active WRITE CapTokens on the volume and rejects an overlap.
Layer 2 is commit authority, also onchain. A sub-agent’s commit authority limits what it can damage:
- Parent-root CAS applies to all volumes. Every commit names the
prev_rootit extends, and the chain rejects the commit unlessprev_rootequals the volume’s currentmanifest_root(enforced today). A stale or racing committer must re-apply its changes on top of the current root and retry, so no commit overwrites an intervening one. - CIP-33 §2.1.2 adds a commit guard for a Public volume registered as a Store serving artifact. Direct
CommitManifestand stagedCommitManifestFinalizeMUST reject while that volume’s CIP-33 pin count is nonzero. A staged commit prepared before registration still checks the pin at finalization. CIP-33 defines the references and release rule, including the future Forked spawn case.CommitManifestStagedoes not change the canonical root. - Prefix-scoped writers stage, and full-volume holders publish. A commit carries the new root and its added and removed shards but not the changed paths, so the chain cannot tell whether it stays inside a prefix. The chain therefore rejects
CommitManifestfrom a writer scoped to a non-emptypath_prefix, in any access mode and on either visibility. Such a CapToken may only stage its delta (CommitManifestStage, attributed tostaged_by), and the stage carries no native-route finalize authority. A prefix-scoped mount grant confers no authority to commit. Only a full-volume holder (the owner, an owner delegation, or an empty-prefix grant or CapToken) issuesCommitManifestFinalizeor a directCommitManifest. Before finalizing, the publisher confirms the staged delta touches only paths under the stager’s prefix; on a PRIVATE volume that review needs the DEK. The publisher is usually the coordinator that issued the sub-agent’s CapToken.
PUT_SHARD requests are keyed by opaque shard_id values (BLAKE3("cbfs.object-chunk.v1" ‖ volume_id ‖ write_id ‖ u32_le(chunk_index))). A Relay Node can verify that the CapToken is valid (signature, expiry, volume ID, write permission) but cannot see the object path, so it cannot enforce prefixes (§16.3). A Runner scoped to agent-1/ can therefore upload shards for agent-2/poison.dat. Those shards are referenced only if the full-volume publisher finalizes a stage that names them. The token’s max_bytes quota bounds the waste, and Relay Nodes garbage-collect orphan shards after ORPHAN_SHARD_TTL_SECS (§14).
7.5 Read consistency
All reads are READ_COMMITTED. An object becomes visible only after the writing client has committed onchain a manifest that includes the object’s ObjectDescriptor. When a client fetches a manifest from Relay Nodes, it MUST:- Fetch the manifest DAG nodes from Relay Nodes, starting at the root locator, and reconstruct them.
- Recompute the root locator of the reconstructed DAG (§3.1).
- Compare the computed root to the onchain
manifest_rootin the volume’sStorageCommitment. - Reject the manifest if the roots differ, since it is stale, partially published or corrupted.
manifest_root is the single source of truth. A manifest that a writer published but has not committed onchain fails step 3, so no reader sees uncommitted data.
§12.1.5 specifies what a mount guarantees on top of this: read-your-writes, monotonic reads, the cross-mount visibility bound, and how failed and concurrent commits resolve.
A READ_UNCOMMITTED mode, where an object is visible once relays store its shards, is deferred to a future CIP. A reader cannot predict an uncommitted shard_id without the writer’s random write_id (§5.3), and prefix confinement is enforced only at commit (§7.3.1).
7.6 Public volumes (PUBLIC)
7.6.1 Overview
Any party can read a volume withvisibility = PUBLIC without a CapToken. Public volumes serve DNS-addressable actors (CIP-14), static web assets (CIP-15), public datasets and shared artifacts. The owner sets visibility at creation (§12.3), and it never changes afterward.
7.6.2 Properties
- Relay Nodes store objects in
PUBLICvolumes unencrypted. The volume has no DEK, and thewrapped_dekfield in theStorageCommitmentis empty. - Any party can fetch shards from Relay Nodes without presenting a CapToken. Relay Nodes serve
GET_SHARDfor a public volume’s shards without authenticating the caller, under two conditions: the volume must beACTIVEorGRACE_PERIOD(§13.5), and the shard’s recorded volume must be the one the request names (§7.6.3). - Writes still need a CapToken. Before issuing a
WRITE_ONLYorREAD_WRITEattachment token or reserving a write prefix, the Dispatcher MUST verify that the job submitter owns the volume or has an owner-issued mount grant permitting that mode and prefix. This applies equally to PRIVATE volumes. A supplied volume-owner address only locates the volume and proves no authority. - The client still computes and stores
content_hash(BLAKE3) for every object. Readers MUST verify the content hash after shard reconstruction to detect corruption or tampering. - Erasure coding and storage fees are the same as for private volumes; the erasure input is plaintext.
list_objectsneeds no CapToken. Anyone can read the plaintext manifest from the on-chain root (§3.1) or withGET_MANIFEST(§5.3.2).
7.6.3 Relay node behavior
A Relay Node decides whether a shard is publicly readable from the volume’sStorageCommitment: visibility must be PUBLIC and status ACTIVE or GRACE_PERIOD (§13.5). Anonymous GET_SHARD, GET_PLACEMENT and GET_MANIFEST (§5.3.2) all go through one AuthProvider entry point that applies this predicate. Relays do not take visibility from shard metadata, because a volume that has since become DELETED or GARBAGE_COLLECTING looks identical on disk (§5.3.2). The commitment lookup reads a bounded-staleness cache.
On an anonymous GET_SHARD, the AuthProvider also requires the shard’s recorded volume (from the stored shard_metadata) to equal the volume named in the request. Otherwise a caller could name a public volume and receive a shard of a private one. LIST_SHARDS is not exposed; clients list objects from the manifest.
PUT_SHARD requests require a valid CapToken with write access. The relay persists the shard’s volume binding, taken from the authenticated payload’s volume_id (the value the CapToken is checked against), together with any metadata the AuthDecision carries and the shard’s commit state, kind and tentative deadline. The relay writes the binding; the AuthProvider’s decision is what lets it trust the payload’s claim.
For private volumes the Relay Node refuses unauthenticated GET_SHARD requests. A Relay Node MUST perform the same work (metadata read, placement lookup, authorization round-trip) whatever the outcome, and collapse every refusal to one indistinguishable response. Otherwise latency reveals whether a refused shard exists.
A payer may choose the funded-channel operation (§10.4.1) for public or private data. Its payment requirements apply only to that operation and leave GET_SHARD, manifest and repair behavior unchanged. A selected paid strategy MUST NOT fall back to an unpaid operation on error. A saved channel ACK handoff delivers no new data. It checks its durable request/voucher binding and payer signature, and it does not check a current data token, relay registration or volume reauthorization. The same separation applies to recovery after a token expires or serving eligibility changes.
A Gateway (CIP-15) is an ordinary unauthenticated public-volume reader. There is no Gateway storage role, key or capability.
7.6.4 Shard ID opacity
Public volumes use the same opaque shard IDs as private ones (§16.3). Their manifest is plaintext, so anyone can read the object paths.7.6.5 Content-type metadata
A public volume may store a content-type map at_meta/content_types.json:
Content-Type. Without a map, they infer types from file extensions.
7.6.6 Cache headers
A public volume may store a cache configuration at_meta/cache_config.json:
Cache-Control. An object’s ETag is the quoted string "b3_" followed by the lowercase hex content_hash (CIP-15 §8.8), for If-None-Match requests. A change in the onchain manifest_root signals that the volume’s contents changed.
7.7 Cross-owner volume access (mount allowlist)
By default only a volume’s owner can attach or open it. The owner lets other principals in through an on-chain mount allowlist on theStorageCommitment (§11.1), changed only by an owner-signed UpdateMountAllowlist transaction (§12.3). The allowlist is the only cross-owner authority, for both the Dispatcher-issued CapToken (§8) and the grantee’s self-issued owner-auth token (§7.7.1). A DelegationCert (§12.2.1) never carries volume authority of its own.
Each grant is a MountGrant:
grantee_signing_key only lets the grantee attach the volume from a job it submits (§8). A grant with grantee_signing_key also authorizes direct client access under §7.7.1 by exactly that key. On a PRIVATE volume, direct reads also require grantee_encryption_key and grantee_wrapped_dek. On a PUBLIC volume these fields are absent.
At dispatch, when the task submitter is not the volume owner, the Dispatcher resolves the submitter to a principal and looks up a matching unexpired grant. It scopes the issued CapToken to the grant’s max_access_mode and canonical_path_prefix, intersected with the attachment request. Delete is never grantable (§6.3). The Dispatcher skips grants whose valid_until has passed.
Removing a grant deletes the grant row, including its grantee_wrapped_dek. The Dispatcher stops issuing that principal CapTokens, and the direct-client path, which resolves the grant on every request, stops serving it. A party that already decrypted its package, completed a CBSS seal or holds the owner’s wrapped copy of the DEK keeps the unchanged DEK. It can still read any ciphertext or plaintext it kept. Because path_tag = HMAC(dek, seq, path) and every commit publishes its path tags on chain, it can also test candidate paths against the whole commit history offline. Only rotating the DEK and re-encrypting the volume (§9.4) withdraws the key. An implementation MUST NOT present grant removal as key revocation.
A grantee may be an Account or a deployed Actor. The mount check derives the submitter’s type from chain state (Actor(addr) if it carries an ActorManifest, else Account(addr)), so a manifest-less actor granted as Actor(addr) would fail an exact-variant match. UpdateMountAllowlist therefore keys grants by bare address, and the mount check matches the canonical address regardless of variant. An implementation MAY also require deployed actors to carry a manifest; the address-keyed match remains the normative rule. A grant MUST NOT be silently dropped on a variant mismatch. The MountPrincipal wire encoding is unchanged.
Direct-client grants have these admission rules. UpdateMountAllowlist MUST reject a grant whose grantee_signing_key is present and whose canonical_path_prefix is non-empty, for every access mode. Shard addressing is opaque (§7.3.1, Layer 3) and a private volume has one DEK (§9.1), so a prefix cannot bound a direct client, and the protocol MUST NOT present a scope it cannot enforce. grantee_encryption_key and grantee_wrapped_dek MUST be both present or both absent, and MUST be absent on a PUBLIC volume. The package MUST be a canonical tagged v1 package within MAX_GRANT_PACKAGE_BYTES, with the selected suite’s exact field lengths (§9.2.1, §14), and MUST name the commitment’s current dek_version.
A WRITE_ONLY or READ_WRITE direct grant lets the grantee upload shards and placements. Chain commit authority comes from the grant row itself, with the grantee’s account as the transaction sender (§7.7.1). A READ_WRITE grantee may stage, commit directly and finalize, and a WRITE_ONLY grantee may only stage, when the grant is whole-volume, not tee_required and unexpired, and on a PRIVATE volume carries a grantee_wrapped_dek. Otherwise uploaded data becomes visible only when a full-volume holder commits a manifest that references it. On a PRIVATE volume the grantee holds the whole DEK, so a write grant limits only what relays allow. A grantee holding the package can still read the volume’s contents.
TransferVolume clears mount_allowed_actors atomically with the owner change (§13.2), and a recreated volume starts with an empty allowlist, so old grants never revive.
7.7.1 Grant-scoped owner tokens
A grantee exercises a direct-client grant with its own delegation cert (§12.2.1). It self-issues anOwnerCapTokenV1 exactly as an owner does, except that owner_address names the volume’s owner instead of the grantee’s wallet. Relay validation (validate_owner_cap_token) accepts such a token iff every ordinary condition holds (cert validity; the cert registered on chain and unrevoked for the grantee’s wallet; token signature by the cert’s key; volume, TTL and status bounds) and a grant on the commitment’s allowlist matches on every one of:
granteeequals the cert’swallet_addressby bare address;grantee_signing_keyis present and equals the cert’scbfs_public_key;token.access_modeis permitted bymax_access_modeas an operation set:READ_ONLYpermits onlyREAD_ONLYtokens,WRITE_ONLYonlyWRITE_ONLYtokens,READ_WRITEany of the three;token.path_prefixis"", because a grant-scoped token is always whole-volume;valid_until, if present, has not passed at the relay’s block-height view; a relay without a height view MUST refuse an expiring grant;tee_requiredis false. A client holder is never a TEE, so atee_requiredgrant does not authorize an owner-auth token.
cert.wallet_address == commitment.owner) and the grant rule are alternatives, and the relay refuses a token that satisfies neither. An owner-admitted token keeps every owner operation (§7.2, §12.2.1). A grant-admitted token gets only these operation sets:
- A
READ_ONLYtoken permits the metadata needed to open the volume, committed-manifest discovery, shard and placement reads, and read-only proofs. Each is permitted only when the relay resolved the target volume from authoritative stored state, so it refuses grantees a shard that is present but not recorded as belonging to a volume, and aPROVE_SHARDnot tied to a volume; the owner keeps its exceptions. The relay MUST deny aREAD_ONLYtoken shard, placement and replication writes, staging, finalization, manifest commit, deletion and every administrative operation. - A
WRITE_ONLYtoken permits shard and placement writes. It denies reads, deletion, replication and every commit or administrative operation. READ_WRITEis the union of the two.
tee_required, unexpired grant (with a grantee_wrapped_dek on a PRIVATE volume) lets a READ_WRITE grantee stage, commit directly and finalize, and a WRITE_ONLY grantee stage. Allowlist changes, deposits, transfer and delete require the owner’s authorization, which a grantee’s delegation cannot produce.
Grant revocation (UpdateMountAllowlist remove), delegation revocation, expiry, and ownership transfer all end access within the relay’s chain-state refresh bound, which MUST be at most 60 seconds. When commitment, delegation or revocation state is stale or unavailable, the relay refuses access. Issued tokens are bearer credentials, as owner tokens are (§12.2.1), and the short owner-token TTLs bound a copied token. Request signing for both is future work (§17).
GET /ras/volumes/{name}/open resolves names in the caller’s namespace. A grantee names the owner: GET /ras/volumes/{name}/open?owner={address}. The node serves the request iff the caller is the owner or its authenticated delegated key matches a grant under the rules above. The response carries the caller’s grantee_wrapped_dek in place of owner_wrapped_dek, never the owner wrap, and is otherwise identical. GET /ras/volumes lists only volumes the caller owns.
8. Volume attachment
8.1 Attachment at job dispatch
A CIP-2 task definition lists volume attachments.VolumeAttachment is the normative wire structure; the same 10 fields, in order, back both the dispatch request and OffchainTask.volume_attachments (§12.4):
The RAS
MountGrant enum uses a different tag order (§9.2.1). Implementations MUST map between the two by variant name, never by ordinal.
relay_endpoints empty. When the Dispatcher builds the runner-facing OffchainTask.volume_attachments, it resolves relays and confirms the erasure parameters against the volume’s PlacementRecord (§5.3). Per CIP-11 §12.6, job_spec_hash covers the attachment exactly as the client submitted it. Dispatcher-side resolution MUST NOT rewrite the stored spec or move the hash, so changes to the relay set between submission and dispatch never invalidates an assignment.
A job may attach several volumes, each authorized independently. A dispatched grant is a mount-allowlist grant (§7.7) whose grantee is the selected Runner’s account and keys (§9.2). A matching dispatched grant issues no CapToken, and other attachments each get a CapToken (§9.2). Only storage-capable runners receive such a task (§4.2.1, §5.1.1).
8.1.1 Per-job attachment admission caps
The task codec’svalidate() runs three stateless checks on volume_attachments, using the task bytes alone, before any volume, owner or grant lookup:
attachments.len() <= 64.- Each
volume_nameMUST satisfy the §6.1 volume-name rule. - The
canonical_path_prefixMUST already be in canonical form:""for whole-volume access, or a non-empty prefix ending in a single/with no leading/.validate()rejects a non-canonical prefix and does not normalize it.
8.2 Attachment process
Attachment moves no data to the Runner. It has three steps:- The Dispatcher authorizes the attachment. It either selects a matching dispatched grant (§9.2), issuing no CapToken on that path, or issues a scoped CapToken for the attachment.
- For private volumes only, the Runner obtains the volume key. On the dispatched-grant path, the Runner opens the grant’s package locally with no CBSS involvement. Otherwise the volume must carry a committee wrap (a copy of the DEK encrypted to the CBSS committee, §9.1), and the Runner completes the CBSS
SealRequestthat the Dispatcher constructs and authorizes (§9.2). Anowner-key-only volume is runner-mountable only through a matching grant. The Dispatcher never handles a plaintext DEK. - The Runner fetches the manifest DAG from the on-chain root locator (§3.1) by node-addressed
GET_SHARD, recomputes the root, and verifies that it matches the onchainmanifest_root(§7.5). On a public volume it MAY useGET_MANIFEST(§5.3.2) instead and verify the same way. The Runner trusts the manifest only after verification.
8.3 Detachment
When a Runner job completes or times out:- The Runner commits the final storage manifest onchain if it wrote any objects.
- The CapToken becomes invalid once
valid_untilpasses. - Shards written during the job persist on Relay Nodes, so the Runner may terminate immediately after commit.
8.2.1 Dispatch eligibility diagnostics
A healthy runner can still be excluded from a storage job by reputation, job type,storage_support or DNS constraints (Appendix D). The Dispatcher MUST emit a structured, per-gate exclusion summary when selecting for a storage-attached job:
selected is below the required committee size, the dispatch error MUST be the structured InsufficientStorageCapableRunners { required, available, reasons: [...] }. A runner excluded for reputation (for example, freshly bootstrapped, or with a reputation that decayed below the threshold when it was last recomputed) MUST appear as its own exclusion in the summary.
9. Encryption and privacy
9.1 Encryption at rest
The client that writes to a private Volume encrypts the data before erasure coding and distribution to Relay Nodes. That client is a Runner during a job, or the owner through the CLI. Relay Nodes never see plaintext. Public volumes (visibility = PUBLIC) skip encryption. There is no DEK and no wrapping key, and readers verify integrity with BLAKE3 content hashes (§7.6).
Private volumes use envelope encryption with a random Data Encryption Key (DEK) per volume:
-
The client generates the volume DEK, a random 256-bit key, at volume creation. It is never derived from the account’s signing key.
The commitment carries a
dek_version: u32alongside the wraps. It is an input to the CBSS volume-DEK identityvolume_dek_identity(chain_identity, owner, volume_id, dek_version, committee_epoch, release_key_material_hash, committee_override_hash)(CIP-24 §9.3), hashed as 4 little-endian bytes, and torelease_context. Every direct-clientGrantDekPackagev1 (§7.7) carries it as 4 big-endian bytes after the version and algorithm tags. Three preimages and the package layout fix its width, so an implementation may not widen it locally.create_volumetakes an owner-suppliedinitial_dek_versionand applies only a lower bound of 1. The chain does not assign it or count rotations. No operation advances it today (§9.4). The client then wraps the DEK (encrypts it under another key) one or more ways according to the volume’s access class (wrapping_key_policy, stored on-chain in theStorageCommitment, §11.1). A volume may carry either wrap or both:-
The owner wrap (
owner-key, v2) wraps the DEK to a key derived from a deterministic wallet signature. The owner can then read and write directly from the CLI or browser, including from extension wallets that never expose their private key. Native wallets, CLI wallets, and extensionsignMessageimplementations MUST use this same derivation:The chain ID is unsigned canonical decimal with no leading zeroes. The volume hex is exactly 64 lowercase digits without0x. Lengths count UTF-8 bytes, and the two length-framed strings MUST fit inu16. Both signature scalars MUST satisfy0 < r,s < secp256k1_order. The client normalizes high-s before HKDF and excludes the recovery bytev. Extension wallets signHthroughwindow.cowboy.signMessage("0x" + lowercase_hex(H)). This API accepts exactly 32 bytes, shows the user the hex ofH, and adds the Cowboy personal-message prefix once. Native wallets and the cbfs CLI signdigestdirectly. The sharedpersonal_message_digest(bytes)stays generic (prefix || decimal(byte_length(bytes)) || bytes); the owner-wrap path always passes the 32-byteH. The caller MUST establish that the signing wallet is the volume owner. The signature is secret key material. The client never sends the signature, the wrapping key or the plaintext DEK to a server, a web origin, telemetry or a clipboard, never logs them, and wipes its buffers after use. It never offers this signature as a proof of wallet ownership, and it explains the key-derivation purpose before requesting approval. Shared cross-client test vectors use deterministic nonces for tests only; production wrapping samples a fresh nonce. RFC 6979 lets the owner recover the same key and gives no secrecy once the signature is disclosed. The stored owner envelope is this canonical type, separate from the CLI’s local-developmentWrappedDektype:Algorithm 1 names the RFC 6979 secp256k1, HKDF-SHA256 and AES-256-GCM derivation above. The envelope is exactly 62 bytes, and the ciphertext includes its 16-byte authentication tag. Create and transfer admission check the version, the algorithm, the exact size and the absence of trailing bytes before storing it. CLI and WASM share one decoder, which rejects unknown tags, truncated envelopes and untagged bincode blobs and does no format sniffing. Version and algorithm are fixed validated constants for this suite, and changing either cannot reinterpret the ciphertext. The secret derivation and its AAD remain exactly as specified above. The deterministic(r,s)output is a per-volume secret and is not a publishable authentication signature. This design assumes a trusted owner client and signing endpoint. The client keeps the result inside that trusted client, never returns it to an untrusted web origin, telemetry, server or shared clipboard, and never reuses this derivation request as a public proof of wallet ownership. A wallet displays the requested hash; that display does not publish the signature. The client must explain the key derivation purpose before requesting approval. RFC 6979 permits recovery of the same key and provides no secrecy if the signature itself is disclosed. If the signature or the DEK leaks, the volume is compromised, including recorded ciphertext. A fresh nonce, a rewrap or an owner transfer cannot revoke a leaked DEK, and DEK rotation is not implemented. For future confidentiality, the owner creates a new volume with a fresh random DEK and re-encrypts its content; this does not erase plaintext or ciphertext already exposed. Algorithm tags do not change this threat model. A client unable to keep this secret MUST NOT use owner-key wrapping. Owner-wrap v2 is active from block 0 of the next regenesis. There is no derivation from a raw wallet secret. -
The committee wrap (
committee-only) IBE-encrypts the DEK to the CBSS committee under domainCIP9_VOLUME_DEK_IBE_DOMAIN = "cbss/ibe/cip9-volume-dek/v1". An authorized Runner obtains the DEK at mount through a threshold seal (§9.2), and no wallet key material reaches the runner. Release-key discovery MUST supplyrelease_key_suite = 1(BLS12-381 G2). Clients MUST refuse a missing tag,0,2, or any other unsupported tag before using committee material, as specified in CIP-24 §6.4. AtCreateVolumeandRewrapVolume, consensus MUST decode this envelope with the 512-byte bound and the exact bincode 1 fixed-integer, little-endian vector layout specified in CIP-24 §3.1, and reject trailing bytes, any version other than 2, a ciphertext length other than 48 bytes, or anephemeral_ulength other than 96 bytes. G2 decompression and full AAD and identity binding happen at release time (CIP-24 §3.1, §9.3).
CIP9_HASH_DOMAIN_VOLUME_DEK = "cowboy.cip-9.volume-dek.v1"(with AAD domain…volume-dek-aad.v1) binds each wrap to itsvolume_id. The wraps are stored in the on-chainStorageCommitment(§11.1). The access classes (wrapping_key_policy) are:A matching dispatched grant (§9.2) also lets a Runner read and write anowner-keyorcommittee-onlyvolume without committee-seal delivery.--owner-keyat create stores the owner wrap in addition to any committee wrap, and the CLI can write any volume that carries an owner wrap. -
The owner wrap (
-
The client encrypts each private object chunk independently with AES-256-GCM, using a fresh random 12-byte nonce per chunk:
Each nonce is stored in that chunk’s
ChunkDescriptor.encryption_nonce, andchunks[].size_bytesrecords the ciphertext length including its tag, before erasure padding. A(volume_dek, nonce)pair MUST NOT be reused across chunks or writes. Deterministic nonces derived from paths MUST NOT be used. The AAD binds the volume, write ID and chunk index and does not change on rename. - The client erasure-codes the ciphertext, so Relay Nodes hold only ciphertext shards.
- The client encrypts the manifest (object paths, sizes, content hashes, ObjectDescriptors) with the volume DEK. Relay Nodes store PlacementRecords separately (§5.3.1), outside the encrypted manifest.
9.2 Runner key access
A Runner receives the volume DEK at job attachment (§8.2); the chain never stores it in plaintext. A matching dispatched grant selects grant delivery. Without one, runner access requires the volume’s committee wrap (§9.1). The Dispatcher never holds the owner wrapping key or a plaintext DEK. On the committee-seal path, each CBSS committee member returns a partial decryption of the committee wrap and the Runner combines them into the DEK. The Runner holds an x25519 recipient keypair whose public key is theencryption_pubkey in its registry capabilities (§5.1.1). The Dispatcher constructs and authorizes the SealRequest. At mount:
- The Runner issues a
SealRequestfor the volume’s committee-wrapped DEK, presenting its CapToken (and, fortee-gated, its attestation). - Each CBSS committee member verifies authorization and returns a partial decryption.
- The Runner combines the threshold partials and completes the IBE decryption (domain
cbss/ibe/cip9-volume-dek/v1) to recover the DEK, sealed in transit to itsencryption_pubkey. - The Runner holds the DEK in memory for the job and zeroizes it on completion.
owner-key-only volume. For a dispatched grant, the volume MUST be PRIVATE and ACTIVE, the
attachment’s path prefix MUST be empty, and exactly one grant row MUST name the
Runner’s account. That grant MUST be READ_WRITE, whole-volume, not tee_required,
unexpired, and carry a valid grantee_wrapped_dek. Its signing and encryption
keys MUST equal the Runner’s registered keys (§9.2.1). This path issues no
CapToken (token = None) and needs no CBSS release-key material.
cbss_rewrap_required gates only committee-seal delivery. Ownership transfer
clears all grants, so the new owner must authorize a new grant to use this path.
A persistent CIP-10 workload’s host requests the committee-wrapped DEK for one attempt (CIP-10 §17.5) with no CapToken and no owner delegation. The committee releases it to the recipient tuple {owner: Address, delegated_ed25519_pubkey: SigningPublicKey, mount_policy_hash: [u8; 32], recipient_key_id: u64, recipient_x25519_pubkey: [u8; 32]}, encoded in that order: the volume’s current owner, the attempt’s release key, the hash of the owner’s current CIP-10 mount policy for that workload and volume, the attempt number, and the host’s X25519 key. The release key uses the tagged SigningPublicKey encoding u8 algorithm ‖ varint length ‖ key, 34 bytes for an Ed25519 key. Execution fills owner and mount_policy_hash from current state at admission and requires recipient_key_id to equal the attempt. Each committee member releases a partial only when proofs against one finalized state root show that the attempt records that release key and recipient, that the stored policy still carries mount_policy_hash and covers the workload, and that the workload’s reporter still holds its host-pool membership. A revoked or replaced policy has no row with the old hash, so its recipients are never served again.
Each committee member submits its partial in its own KeyDelivery(Cip9ServiceVolumeDek) transaction, signed by the member’s operator. The Storage Manager keeps one release row per request, recording the accepted partials in arrival order. The row becomes Ready at the release key’s threshold. Its discovery queue index, which tells committee members the request still needs partials, stays until accepted partials reach min(committee size, 64) or the request expires. The delivery that completes that count removes the queue index, not the release row. An authenticated fetch of a Ready row stays live until the request expires or loses authority. Before it accepts a partial, a delivery authenticates the sender against the committee operator at the partial’s position and verifies the partial’s signature. It also re-checks the request’s full liveness: the attempt, slot, volume state, mount policy, owner policy and release key. For every partial already in the row, it re-checks the partial’s binding to the request and its committee position and index. While the row is Pending, it also re-checks that each accepted proxy is still active: registered, eligible, not suspended, not unbonding, with a public key that derives its operator. A failed check makes the row Invalidated. Once the row is Ready, an accepted partial stays fetchable after its proxy is suspended or unbonds: only its binding and signature are re-checked. Every new delivery still requires its own proxy to be active. From SERVICE_RELEASE_DELIVERY_COST_ACTIVATION_HEIGHT (a CIP-50 not-before gate), a delivery verifies the new partial’s signature once and no longer re-verifies the signatures of partials already in the row. Only the delivery itself writes partials into a release row, and only after verifying each one against its operator, and the request fixes the committee epoch whose operators sign. Every other check above still runs on each accepted partial. RPC polling, the CBSS host’s authorizer and the Runner recipient still verify every partial. From the same gate, the delivery’s fixed access row is the CBSS pause read, the release-row read and the two writes. Once the delivery has decoded the row, it reserves its remaining reads for the row’s real status and accepted count k: 18 + 2k for a Pending row and 26 + k for a Ready row. Before the gate, it reserved them for the 64-partial maximum (CIP-43).
The owner of an owner-wrapped volume unwraps the owner wrap locally with its wallet-derived key, with no runner or committee involved.
The DEK never appears in plaintext on-chain or in any persistent store. READ_ONLY CapTokens on private volumes still require DEK delivery, because the Runner must decrypt to serve reads. For tee-gated volumes, the committee releases partials only against a valid attestation, and the DEK is sealed to the enclave and never exposed to the host OS. Public volumes skip key delivery, and the job assignment payload has no encrypted_dek field.
9.2.1 Delegated client key access
Pre-release format changes do not advance version numbers. This grant package’s layout at genesis is version 1. Other tags keep their values, including the committee envelope’s version 2 (§9.1). A direct-client grant (§7.7) authorizes storage operations and gives the grantee no owner unwrap key. Direct access to a PRIVATE volume therefore requires a recipient-sealed copy of the DEK, carried in the grant asgrantee_wrapped_dek.
The grantee generates a separate X25519 encryption keypair, never derived from its Ed25519 signing key, and gives the owner the public half, recorded as grantee_encryption_key. The sealer is the owner or, for a committee-wrapped volume, an authorized DEK holder acting under the applicable CBSS release policy. It seals the current DEK with the CBSS HPKE suite: DHKEM(X25519, HKDF-SHA256), HKDF-SHA256, AES-256-GCM, single-shot base mode, info = "cbfs/grant-volume-dek/v1".
EncryptionPublicKey has one encoding: the
commonware codec u8 algorithm ‖ Vec<u8> key, with a commonware varint length.
Algorithm 1 encodes as 01 20 ‖ key[32] (34 bytes). Its strict commonware decoder
MUST reject an unknown algorithm before reading the length, check the 2,048-byte
key bound before allocation, and require exactly 32 contributory X25519 bytes
for algorithm 1. A standalone value MUST have no trailing bytes.
On every JSON interface listed below, the key is one string: "0x" followed by lowercase
hex of those codec bytes, so algorithm 1 is "0x0120" followed by 64 hex digits.
The list is consensus state, node RPC, relayer HTTP, runner registration,
runner-node grant-identity output, cowboy runner register --grant-identity-json
input, cbfs CLI --json grant rows and the cbss-wasm grant_dek_aad input. Decode
requires the 0x prefix and lowercase, even-length hex, then the strict commonware
decoder, and accepts no object form, integer array or bare 32-byte key. Off-chain
APIs outside this list keep their own encodings; the harness
/v1/workspace/grantee-key response stays 64 hex characters.
Signing and AAD preimages use a different form,
u8 algorithm ‖ u64be byte_length ‖ key, following the §12.2.1 Encoder
convention. MountGrantPayload preimages and the grant AAD use it, the AAD in the exact field order below. This form applies only to signing and AAD preimages; stored and transported keys keep the encoding above. Runner
registration is signed over keccak256 of its exact JSON bytes, which carry the
key as the hex-codec string above.
Runner registration is JSON carried in RunnerRegister { registration }, with a
10 KiB cap on registration. The algorithm-1 key string is 70 characters. A
2,048-byte key would take about 4.1 KB as hex, so enabling such a suite must
revisit that cap.
At genesis only algorithm 1 is enabled:
HPKE DHKEM(X25519, HKDF-SHA256), HKDF-SHA256, AES-256-GCM, base mode, with a key of
exactly 32 bytes. Decoders reject low-order and non-contributory X25519 keys. The runner’s
registered grant_recipient_pubkey uses this same type. The dispatcher MUST match
the registry’s (algorithm, key) against the grant’s grantee_encryption_key
exactly, and the package’s recipient MUST equal it. The runner opens the package
according to the registered algorithm. There is no implicit X25519 conversion and
no algorithm fallback.
The package bytes (grantee_wrapped_dek, GrantDekPackage v1) are:
enc 2,048 and ciphertext 4,096 bytes; MAX_GRANT_PACKAGE_BYTES=8,204. Decoders check each
prefix before allocation. They reject trailing bytes, unknown versions or algorithms,
wrong suite lengths, non-contributory recipient or enc keys, a recipient
algorithm or key that differs from the grant, and a dek_version that differs from the commitment.
Consensus and clients both apply these rules to constructed objects and to
decoded wire data. Only this package format is accepted.
The ceilings fit an X-Wing public key (1,216 bytes) and KEM ciphertext
(enc, 1,120 bytes), but no X-Wing suite is allocated. A new suite needs a suite
registry entry, exact lengths and an implementation. A post-quantum package of
roughly 2.4 KB would not let a 64-grant roster fit in the 64 KiB StorageCommitment
row or in the per-job issuance census bounded by MAX_ACTOR_KV_VALUE_BYTES, so
such a suite must also change roster storage.
The owner-wrap, HPKE-grant and committee namespaces are separate types. A decoder
cannot parse an owner algorithm 1 envelope as a grant algorithm 1 envelope.
The seal’s authenticated data is built with the §12.2.1 Encoder (big-endian
fixed-width integers, u64-BE byte and string lengths, checksum address strings,
and the tags defined below), in this exact order:
max_access_mode is one u8 in the RAS MountGrant tag order:
WriteOnly = 0, ReadOnly = 1, ReadWrite = 2. The VolumeAttachment enum of §8.1 uses
ReadOnly = 0, WriteOnly = 1,
ReadWrite = 2. The two enums are separate types and MUST be converted by
variant name, never by ordinal. The RAS order applies wherever a MountGrant
field is encoded: the grant wire, the MountGrantPayload owner-action preimage
(§12.2.1) and this AAD. tee_required is one u8:
0 = false, 1 = true. valid_until is u8 0 for None, or u8 1 followed
by the u64be height for Some.
The package’s recipient algorithm and key must equal the grant’s before the AAD
is built, and the rest of the context comes from the target commitment. The AAD
excludes the HPKE package ciphertext. Clients and the node share one helper in
protocol-codec. Substituting a tag or key therefore cannot downgrade the seal or move it to another grant.
The grantee persists only its own encrypted private keys and unseals the package locally. Under a WRITE_ONLY or READ_WRITE direct grant, the grantee encrypts uploads with the same DEK.
Revoking a grant or delegation stops later authorized requests once cached authorization decisions expire, within the bound in §7.7.1, and cannot recall a DEK or plaintext already delivered. A DEK rotation (dek_version bump) invalidates every package on the allowlist. The owner’s rewrap flow MUST re-seal for each direct-client grant, and a grantee holding a stale package reads nothing newer than the rotation.
9.3 Privacy guarantees
For private volumes (visibility = PRIVATE):
- Relay Nodes see only ciphertext shards indexed by opaque shard IDs (§16.3). Object paths are encrypted inside the manifest.
- Runners not assigned to the job cannot access the volume DEK.
- On-chain observers see only the Storage Commitment (volume ID, wrapped DEK, encrypted manifest hash, total size, shard assignments).
- The account owner can read all of its volume data by unwrapping the DEK with its wrapping key.
visibility = PUBLIC) have no confidentiality: anyone can read the unencrypted data. BLAKE3 content hashes detect tampering, and only the account owner or Runners authorized by a CapToken can write.
9.4 Rewrapping and rotation
Rewrapping changes who can obtain the key. Rotation changes what the key opens, and only rotation revokes access. The protocol has rewrapping and does not have rotation.9.4.1 Rewrapping keeps the DEK and replaces the wrap (built)
RewrapVolume replaces a private volume’s committee wrap (wrapped_dek) with one
the owner supplies. It first checks that the accompanying cbss_committee_epoch and
cbss_release_key_material_hash equal the owner’s current release key material.
It leaves dek_version, owner_wrapped_dek and every stored byte unchanged, and it
clears cbss_rewrap_required.
TransferVolume sets cbss_rewrap_required on any volume that has a committee
wrap, and the Dispatcher refuses committee-seal delivery while it is set (§13.4).
An ordinary committee reshare does not set it. A CBSS force-rekey of an account walks the owner’s volume index and sets it on every private volume of that owner, at most 64 volumes per call (CIP-24). The new owner must rebind the committee wrap
to their own release key before using the committee-seal path, or authorize a new
matching grant (§9.2).
The chain verifies the epoch and the material hash, never the wrapped key, so an
owner can substitute a different key with no dek_version change. Rewrapping
also revokes nothing: every party that has held the DEK still holds a working key.
9.4.2 Rotation is not implemented
Rotation MUST re-encrypt. While the ciphertext is unchanged the old DEK still opens it, so bumpingdek_version and republishing wraps alone would assert a
rotation that did not occur (§17.2 rules this out). Until rotation exists, the
available operations can withdraw permission (§7.7) and none can take back a key already delivered, so an owner facing a
compromised grantee must create a new volume and copy into it. This CIP does not
specify the rotation instruction.
What a rotation MUST achieve, whatever shape it takes:
- R1, total re-encryption: every object and every manifest node is re-encrypted under the new DEK. A partial rotation does not count.
- R2, no split state: no commitment is reachable in which
dek_versionhas advanced but the root has not, or the reverse. - R3, owner authority: a principal that can rotate can lock the owner out, so rotation is not delegable to any grant, CapToken or delegation scope that is otherwise sufficient to commit.
- R4, end of pre-rotation authority: after a rotation, no principal authorized before it retains read or write access.
9.4.3 What any rotation design has to solve
Each obstacle below exists in the current chain.- A full-volume rewrite does not fit one commit.
apply_manifest_commitrefuses when added + removed shard references exceedMAX_SHARDS_PER_COMMIT(4,096), and the check that a commit fits its Access-meter budget (CIP-43) can refuse smaller commits, while a volume may holdMAX_OBJECTS_PER_VOLUME = 1,000,000objects. R2 then needs a rekey that is durable across a multi-commit rewrite or an explicit mid-rotation state that readers can see. CommitManifestFinalizeis{ volume_id, stage_id }and carries no authorization. The chain derives its authority from the stage header, which admits a full-volumeReadWritemount grantee and a staged owner delegation as well as the owner. Meeting R3 with an existing commit instruction needs a field that marks the commit as a rekey and narrows who may submit it.- Clearing
mount_allowed_actorsdoes not invalidate CapTokens.TransferVolumeinvalidates them by advancinglast_owner_transfer_block, which the chain compares againsttoken.valid_from; R4 needs an equivalent marker that a rotation advances. Without one, a runner attached before the rotation can commit old-DEK objects onto the post-rotation root. Adek_versionbump does make direct-clientGrantDekPackages unusable, but keyless grants and issued CapTokens survive. - The party being rotated away can keep the rotation from ever finishing. A staged rewrite spans
many transactions and expires, and the stage header fixes the root the rewrite started from (
prev_root). Any intervening commit by a principal that still holds write authority fails the compare-and-swap, and the owner re-uploads the whole volume. Authority has to be withdrawn before or during the rewrite. - The owner of a
committee-onlyvolume cannot read without a runner (§9.2), so it cannot always re-encrypt client-side. That class needs a runner-mediated rotation, which conflicts with R3, or must be declared not rotatable. dek_versionis owner-chosen and 32 bits.volume_idis deterministic in(owner, volume_name), the identity carries no per-creation nonce, andinitial_dek_versionis caller-supplied, so deleting and recreating a volume can reproduce an identity already used, and a party that combined threshold partials for it keeps its key. Overflow atu32::MAXis undefined.wrapping_key_policydoes not exist on chain. A validator cannot read §9.1’s access classes as state (§17.3), so no rotation rule may depend on them.- A rekey changes the commit payloads in the shared protocol codec, with golden vectors and a fixed-size owner-operation table, so every signer and verifier changes together.
9.4.4 What rotation would and would not revoke
- Objects written after a rotation are opaque to a holder of the old DEK, and earlier objects only because R1 re-encrypted them.
- Plaintext already extracted stays extracted.
path_taghistory (§7.7) is computed under the old DEK and stays on chain, linkable by the old DEK.- Relays that retain superseded ciphertext hold bytes the old DEK opens until shard lifecycle (§13) removes them.
- A rotation re-encrypts, re-erasure-codes and re-uploads every object, and the byte delta counts twice. Rotation MUST NOT be presented as a background operation.
10. Billing and fees
10.1 Fee components
RAS charges two fees and holds one refundable bond, all in CBY:create_volume debits the creation bond from the owner in addition to any initial escrow, and holds it inside the volume’s escrow_balance as deposit_wei. The bond is not available for rent, withdrawals, gas or the rent floor. When the volume is garbage-collected, the bond returns with the remaining escrow to the current owner’s account storage reserve.
10.2 Effective size and erasure overhead
A volume’s effective size is its raw size times the erasure overhead:effective_size = raw_size * 1.5. The account pays for the effective size, which is what Relay Nodes store.
10.3 Persistent storage billing
Storage costs accrue every epoch. Relay Nodes hold the bytes, so the chain settles storage from attestations and checks them with Proof of Retrievability challenges (§5.6).- Rent settles once per epoch, per volume, at that epoch’s fee snapshot (CIP-31).
- Each settlement first funds the volume’s escrow toward
fee × (elapsed + E)epochs of rent, whereEismax(min_storage_balance_epochs, target)andtargetis the owner’s auto top-up target. Funding draws from the owner’s account storage reserve, then from the owner’s spendable balance. Settlement never draws vesting-locked balance. Settlement then charges the elapsed rent from escrow, then from the reserve. - Auto top-up is on for every wallet by default (
Default: the governed targetauto_top_up_default_target_epochsfrom that segment’s fee snapshot, genesis 7, and no cap). An owner may setCustom(own target and a total cap) orOff(Eismin_storage_balance_epochsand settlement never draws the spendable balance). - Residual escrow at garbage collection returns to the owner’s account storage reserve. §10.3.1 gives the grace and expiry transitions.
10.3.1 Escrow and rent lifecycle
Atcreate_volume() the owner MAY prepay storage with --initial-escrow.
- A volume may be created at zero bytes with any escrow, including none; a zero-byte volume’s rent floor (the minimum escrow a volume of its size must hold, CIP-31 §3) is 0. If the requested initial escrow exceeds the owner’s spendable balance, creation MUST fail with a distinct, named error (§12.3.1). Create records on the volume the rent-floor policy in force in its epoch (CIP-31 §3).
- A commit that grows
used_bytesMUST leave escrow at or above the rent floor of the new size, under the policy recorded on the volume. If escrow is short, the Storage Manager tops it up from the owner’s account storage reserve only, never from the wallet. If the reserve cannot cover the shortfall, the commit fails withE_STORAGE_GROWTH_BELOW_FLOOR. A commit that shrinks the volume always passes. - When escrow plus reserve falls below the rent floor at settlement, the volume enters
GRACE_PERIODforSTORAGE_GRACE_EPOCHS. Funding at a later settlement, by deposit, reserve or auto top-up, returns it toACTIVE. - If funds are still short after the grace window, the Storage Manager marks the volume
GARBAGE_COLLECTING(§13).
DepositVolumeEscrow, which moves the sender’s spendable balance into escrow the owner controls. The owner may withdraw escrow above the rent floor (§12.3).
Two instructions act on the sender’s own account with no delegated authorization, so only the owner’s wallet can issue them:
SetStorageAutoTopUp { setting }(RAS kind 31) replaces the auto top-up setting withDefault,Off, orCustom { cap_remaining_wei, target_epochs }(tags 0, 1, 2; any other tag is invalid).Customrequires a non-zero cap andtarget_epochs ≥ 1. EachCustomdraw reduces the cap, caps do not accumulate, and a spent cap becomesOff. An explicitOffpersists until the owner changes it. No setting releases escrow or reserve.WithdrawStorageFunds { volume_id?, amount_wei }(RAS kind 32) moves unused storage money to the sender’s spendable balance. With no volume it draws the account storage reserve. With a volume, the sender must own it, it must be live (notDELETEDorGARBAGE_COLLECTING), and it settles first. An amount that would leave escrow below the rent floor is refused withE_STORAGE_WITHDRAW_BELOW_FLOOR, which names the maximum withdrawable amount.amount_weiis at least 1 and fits in u64.
10.4 Fee distribution
STORAGE_FEE_PLATFORM_BPS (default 1000 = 10%) goes to the Platform Fee Account 0x18. STORAGE_FEE_CHALLENGE_POOL_BPS (default 100 = 1%) goes to the PoR challenge pool at 0x0B; it is the POR_CHALLENGE_FEE_SHARE row in §14 and funds the challenger bounty (CIP-31 §7). STORAGE_FEE_RELAY_BPS (default 8900 = 89%) goes to active Relay Nodes in proportion to a sum over their counted valid PoR responses, where each response adds how long that relay has been assigned that shard index, up to a cap (CIP-31 §5).
The three must satisfy STORAGE_FEE_PLATFORM_BPS + STORAGE_FEE_CHALLENGE_POOL_BPS + STORAGE_FEE_RELAY_BPS == 10000, and Tier-0 governance MAY rebalance them under that sum. Transfer fees (TRANSFER_FEE_PER_MIB) go entirely to the serving Relay Node.
10.4.1 Funded cumulative read channels
The version-1 funded-channel mechanism is active from genesis. Nodes MUST publishactivation_height: 0, active: true in
the funded-channel pricing capability and 0 in the corresponding
/chain-info activation-height field. Operator services are opt-in. The advertised authorization is
funded_payer_cumulative_read_channel_v1. Support for kind-27 read transfers
MUST NOT imply channel support. The kind-27 instruction, its domains and records
are separate from channels, and none of its signatures or fee caps count as
channel authority.
A channel opens with RAS kind 28 (OpenReadChannel), which MUST be a direct,
externally signed transaction from the configured payer. Actor authority, whether
exercised through a deferred transaction or held implicitly, is insufficient. The terms fix version, chain ID, genesis beacon,
payer, relay wallet, relay node ID, volume ID, a nonzero funding nonce, maximum
rate R, maximum cumulative units, deposit B, read-stop height, and a later
claim deadline. Identities, deposit and unit bound are nonzero. The deadline is
strictly later than read-stop and less than u64::MAX.
The channel ID binds network, payer and funding nonce. A separate terms hash
binds the complete fixed terms. Reusing a nonce with changed terms MUST collide
with the permanent channel or its tombstone (the permanent record kept after the channel closes) and create no new channel. Opening
requires an Active volume and an Active registered fixed relay with the matching
wallet. Opening debits exactly B from the payer into a dedicated per-channel ledger
under 0x0A / ras:read-channel:v1:, plus transaction gas. It MUST NOT fund reads from volume rent
escrow, rent rewards, challenge pools, or an attachment/job budget.
To read, the payer sends a signed off-chain request that binds channel ID, terms
hash, a permanent read ID, shard ID/index, the complete logical payload length and
BLAKE3, the previous sequence, and a deadline no later than read-stop. One complete
logical shard is at most 256 MiB. A request lets the relay try to serve bytes and
carries no spending authority. Every new data request
MUST pass current serving-registration, manifest/placement,
pinned transport-identity, and data-access checks (including a capability where
the private-data policy requires one). Payment grants no data access, and an
Actor, job, CapToken or DEK grants no payment authority. Active or Draining fixed relays may serve valid new reads.
The payer signs a cumulative ACK only after the complete logical payload matches
the independently trusted manifest. Each ACK contains:
- channel ID and terms hash;
- strictly advancing sequence;
- cumulative units
U = sum(ceil(logical_shard_bytes / 2^20)); - cumulative fee ceiling
F, read count, payload-byte count and receipt root.
keccak256(READ_CHANNEL_RECEIPT_DOMAIN || previous_root || canonical_request_bytes).
The domain is the bytes cowboy.cbfs.read-channel-receipt.v1 followed by one NUL,
and the initial root is zero. Sequence, units, read count and payload bytes advance together under the
codec’s rules on how those counters may change. Fee ceilings are monotone and bounded by
B. An object-level cap bounds incremental exposure only. Verifiers MUST check network/domain separation and canonical low-S payer
signatures. Transport chunks MUST NOT be individually rounded or acknowledged
as extra logical shards.
The relay claims with RAS kind 29 (SettleReadChannels), which carries an epoch label
and 1..32 vouchers for one fixed relay wallet/node, sorted by unique channel ID.
The epoch label is scheduling metadata; it neither namespaces replay nor resets a budget.
Each selected channel needs only its latest cumulative ACK. A checkpoint settles one
channel at the block height where the claim executes, and charges:
r <= R, the signed cumulative ceiling
delta <= F - paid_atomic, and the remaining dedicated escrow. It MUST advance
the settled-units counter (settled_units) exactly once, even when r == 0. All arithmetic is checked.
Only the fixed relay wallet receives delta, whatever the current registry
owner or volume ownership and status. A kind-29 transaction MUST NOT be
deferred, and the page’s relay wallet MUST be one of the transaction’s signers,
so no other party can submit the relay’s claim. The escrow is released to that
wallet. The Storage Manager validates the entire page, meaning the batch of vouchers
in one claim transaction (signatures, channels, aggregate payout), before
any write, and one invalid entry rejects the page. Settlement at the claim deadline
is permitted. Existing ACKs and claims MUST NOT depend on current serving
registration or reauthorization of the volume.
Any gas-paying account may submit RAS kind 30 (RefundReadChannel) strictly
after the claim deadline. The remaining principal goes only to the fixed original
payer, and the closed channel’s tombstone is permanent. Every valid state satisfies:
AccountSignature (a varint policy revision and 1..16 signature slots), so its
length varies. With one slot and a policy revision of 0..127 (a one-byte
varint), a voucher is 209 bytes and a request 258 bytes; each further varint
byte adds one. A settlement page is 62 + Σ signed voucher lengths bytes,
for example 6,750 bytes at 32 such vouchers. CBFS operations 16
and 17 carry channel shard delivery and ACK handoff; operations 0..15 and the existing payment domains are unchanged. The canonical codec and its
golden vectors define the signed bytes; JSON and host-native serialization do not.
10.4.2 Bounded journals, economics and recovery
Payers MUST reserve a channel’s fullB and worst-case opening gas under
explicit lifetime budgets, saved durably across restarts, before signing any opening. All payment APIs
that share an identity or journal (the durable local record of signed and pending payment operations) share that allowance; independent APIs MUST NOT each
allocate a fresh copy. A relay MUST reserve, per channel, room for one logical shard delivered but not
yet acknowledged before serving bytes, enforce global limits on bytes and value
not yet covered by a signed ACK and lifetime limits on all delivery at R, and
retain permanent records of read IDs and settled cursors.
If a payer refuses to ACK, the relay’s loss is bounded by that window; there is no
fair-exchange guarantee for the last chunk. A full journal MUST refuse new operations, without dropping existing authority
or resetting budgets.
Exact signed opening/page bytes and nonce ownership MUST be durable before
submission. When admission or finality is unknown, the client may only replay those
exact bytes, never repriced, repacked, re-signed or under a new nonce. Release requires
trusted finalized evidence for the exact transaction and committed nonce
consumption; an HTTP status, a missing transaction or elapsed time does not count.
After a page finalizes as reverted, a new budgeted page may reuse still-unconsumed
vouchers; after a page whose outcome is unknown, it may not. A lost signed ACK is resent as saved.
A crash between payload verification and signing may resume only the exact record of what it
intended to sign, saved durably before the crash. It cannot create an ACK for bytes it never verified.
A relay may need several budgeted pages and may postpone claiming small remaining amounts. The relay’s settlement submitter runs under three explicit, nonzero operator thresholds: minimum_revenue_atomic (a settlement earning less is not submitted), maximum_gas_share_bps (the most of a settlement’s revenue it may reserve for gas, 1..9,999 bps) and maximum_gas_atomic (the most any one settlement may reserve for gas). The gas budget for a settlement earning revenue is min(revenue × maximum_gas_share_bps / 10,000, maximum_gas_atomic). Current native accounting
debits the SUM of all three tariff components from the same CBY balance. The
CBY/DBY component field labels name one asset and MUST NOT justify a discount.
Lifecycle cost includes opening, checkpoints, charged failed attempts, refunds,
locked capital and bytes delivered but not yet covered by a signed ACK. The protocol sets no production lock
windows, deposit tiers, gas/revenue ratios, carry-over or loss budgets. Pricing
requires measured 1/8/32-channel Cycle/Cell/Access costs and equal-total-unit
checkpoint-partition comparisons from execution receipts.
Operators may stop new reads without invalidating funded claims or expiry refunds,
and reverting deployment code must not strand escrow. Enabling a new client
implementation implies no operator enablement, free fallback or journal deletion. Pinned trusted-node JSON
state/finalized archives MUST be labelled as observations; they are not
cryptographic state proofs or relay delivery evidence.
10.5 Relationship to CIP-3
RAS adds no meter to CIP-3. Onchain operations (creating Storage Commitments, writing manifests, opening, claiming or refunding read channels) consume Cycles, Cells and Accesses as normal transactions, separately from escrow principal. The Storage Manager debits off-chain storage fees from a separate ledger each epoch.11. Onchain state
11.1 Storage Manager system actor
The Storage Manager system actor atSTORAGE_MANAGER = 0x0A keeps one StorageCommitment per volume:
StorageCommitment. One stripe row, keyed by (volume_id, shard_id), holds all w = K+M per-index chunk_root values, the common shard_size, the publication event sequence, and a dense stripe ordinal (the stripe’s position in a gap-free, zero-based numbering of the volume’s stripes). An ordinal row keyed by (volume_id, stripe_ordinal) identifies the stripe. live_shard_count counts shard refs, w per stripe, so the stripe count and next append ordinal are live_shard_count / w, and MAX_LIVE_SHARDS_PER_VOLUME = 1,048,576 caps refs. A manifest commit updates the delta’s stripe rows and the ref count atomically with manifest_root. Removal moves the tail stripe into the removed slot. The whole-stripe rule (below: a commit adds or removes every index of a stripe together) reads only TaggedShardRef fields (kind, path_tag, shard_id, shard_index, chunk_root, shard_size).
MAX_SHARDS_PER_COMMIT = 4,096 is a consensus cap on added plus removed refs across a complete commit. A direct CommitManifest and each CommitManifestStage page MUST also fit the stricter COMMIT_EVENT_SHARD_REFS_PER_CHUNK = 512 combined-ref cap. Stage checks the aggregate before writing a new page; an identical replay is not recounted. Finalize checks each page and the running aggregate. The shared apply path checks it again. Violations reject with RasInvalidDataSite::QuotaExceeded. The check that a commit fits its Access-meter budget (CIP-43) is a separate limit.
Every object chunk MUST use the volume’s K and M, duplicated in its descriptor (§5.3), and consensus derives w from it. For each shard_id in each of the added and removed sets, a commit MUST carry indexes 0..w (0 through K+M−1) exactly once, with equal shard_size, kind, and path_tag across those indexes. For staged commits the rule applies to the concatenation of all stages, so a stage boundary may split a stripe but finalization rejects a partial or duplicated one. Shape validation runs after the manifest-root check and before inventory reads. A violation rejects with RasInvalidDataSite::CommitWholeStripeInvalid (slug ras.commit_whole_stripe_invalid). Every removed stripe MUST already exist and match the supplied root at every index and the supplied size (§6.2). Every added stripe MUST have no existing (volume_id, shard_id) inventory row, so a reused write_id fails the commit and cannot overwrite another object’s stripe.
Rent evidence uses one history row per stripe inventory incarnation. An incarnation is one stay of a stripe in the inventory, from the event that adds it until its removal. The row is keyed by ras:rent:v1:history:<volume><shard><incarnation_be>. incarnation_be is the big-endian u64 encoding of the stripe’s publication event sequence (RasStripeRow.added_at_event_seq), which the challenge carries as snapshot_added_at_event_seq. The row’s stripe-wide generation is that sequence, with (node_id, chunk_root, assigned_at_block, closed_at_block, origin) per index. Re-adding a removed stripe creates a new incarnation. Assignment time MUST never be backdated for any index. PoR reads the history row for the challenge’s snapshot_added_at_event_seq, and FirstValidPor bootstrap checks that sequence against inventory. Proof keys and counted keys remain per index (CIP-31 §5).
There is no separate ras:rent:v1:assignment: row; other rent keys, including rrp:, rrw: and rrf:, keep their formats. Rent rows use a key-aware binary codec that omits identifiers implied by the key, stripe header or index position and reconstructs them on decode. Remaining fields use fixed-width big-endian integers, raw arrays, one-byte Option/bool/enum tags, and bounded vectors with u32 lengths. FeeAmount is raw u128. The decoder rejects malformed keys, unknown tags, invalid booleans, truncation, trailing bytes and oversized vectors. The encoder rejects values that disagree with implied fields. There is no JSON fallback. /fees/cbfs-rent/epochs/{epoch} returns JSON whose assignment_generation field carries the inventory incarnation event sequence. PoR weight-page JSON stores {weight,w} entries, and the raw-state RPC exposes value hex.
Direct commits and Finalize charge an upper bound of 1 Cell per logical persisted key+value byte for all state growth, plus full rewrites of rows sized by the transaction inputs, as specified in CIP-43 §5.5 and mirrored in cbfs_registry_proto. Inventory, rent, event, snapshot, funding and deferred rows count full writes, including overwrites; only the in-place remainder of StorageCommitment, the global stage index and existing PoR rows is covered by the flat base charge, and their growth is charged. Direct and staged paths have the same application charge for the same delta and settlement context. Each Stage separately reserves its full body, header, uploaded ordinals and accumulated totals, plus global-index growth. Clients price direct/Finalize with fresh, durably saved information about the storage epochs queued for rent settlement, and reserve width 32 for empty deltas, whose refs reveal no width; native execution uses the stored width. Width 32 bounds Cells metering and row decoding; the creatable grid is 2 ≤ K ≤ 16, 1 ≤ M ≤ 8 (K+M ∈ 3..=24).
Shared read-only settlement reserves queued rent Cells before writes for DepositVolumeEscrow, WithdrawStorageFunds(Some(volume)), manifest application, and escrow-paid write admission other than CommitManifestStage. Escrow-paid Stage admission charges zero queued-rent Cells even when it writes rent rows. A catch-up settles storage epochs that are queued and not yet settled. On the charged paths, the first catch-up in an epoch writes the queued rent rows and charges 334 × unsettled_rows; a later catch-up in the same epoch, in any transaction, charges 334 × 0 = 0. In the native application formula, K is unsettled_rows. Clients budget the conservative pending_rows. A withdrawal from the account reserve without a volume does not catch up rent. A tight signed Cells limit can fail before the post-settlement terminal-state, balance or withdrawal-floor checks (E_STORAGE_WITHDRAW_BELOW_FLOOR), though an already-terminal volume is refused before settlement. On OutOfCells the full Cells limit counts as used and is not refunded, and the node rolls back the attempted rent and manifest writes.
Stage header, uploaded-ordinal and totals values use a strict big-endian codec under the same keys. The decoder checks lengths before allocating and rejects unknown tags, unsorted or duplicate IDs or ordinals, truncation and trailing bytes. Bodies are bincode, and the shared global stage index is JSON. Wire pins are CBFS_MANIFEST_COMMIT_WIRE = 1 and RULESET_REVISION = 1; the codec was introduced at access table 25, and the current access table is 43.
Computed direct/Finalize signers refresh K after their last await and reserve K+1, adding 334 Cells of inclusion headroom. Manual gas overrides MUST include their own K+1 margin. Partial or Finalize-only batches recover missing committed pages, then refresh queued-epoch context to price the complete delta. SDK, CLI and hooks clients compute their own limits.
Computed direct/Finalize CLI and runner signers read K immediately before and after /estimate_gas, and again at the final signing await. Any change discards the estimate and repeats the check, up to three attempts; if K keeps changing, the signer returns a retryable “queued epochs are changing” error without signing. A stable estimate supplies access authority and Cells at that K+1. The signed access limit is min(ceiling, max(suggested, used + reserved) + client_access_limit(12)), where the extra inclusion epoch converts 12 structural accesses to charged units at the maximum governed scalar. Optional padding is capped at the lane ceiling. The signer never refuses locally on epoch-growth arithmetic; the node decides whether the transaction fits.
Runner retries follow these rules. An uncertain retry keeps the nonce and the exact signed bytes. A definitive rejection before inclusion (non-admission), including an initial nonce_too_low, releases the nonce reservation without recording a failure, and the next submit re-reads the account nonce. During reconciliation, a low-nonce replay of exact bytes stays uncertain until its receipt resolves, and a missing receipt plus an advanced account nonce never permits a new signature. A failed Cells budget survives a successor’s non-admission. An executed Cells shortfall requires a strictly larger Cells limit; any other executed failure permits a freshly priced limit. Immediate success records nothing. Only an uncertain attempt later reconciled as success is answered once without re-signing, and its matching read consumes the success record. Any other repeat of Stage, Finalize or direct CommitManifest is signed and the chain decides. The account keeps at most 16,384 FIFO operation outcomes, refreshed on re-recording; eviction permits another submission. Byte-identical manifest changes can be distinct operations after the root returns to an earlier value. Uncertain pending bytes remain separate.
Current Stage funding uses the source-derived auth-v3 reservation, whose ordinary-System structural capacity is 1,116.
The shards RPC expands stripe ordinal c and index i into the computed shard number c·w+i; its total is live_shard_count, and the inventory revision detects concurrent mutations during pagination. PoR selects a shard index from a stripe row and records its issue-time commitment (§5.6).
The Storage Manager also keeps a relay-rotation side table:
Expiry needs no row deletion.
On chain the committee wrap is
wrapped_dek and the owner wrap is owner_wrapped_dek; there is no field named committee_wrapped_dek. CreateVolume requires wrapped_dek and a nonzero cbss_committee_epoch on a PRIVATE volume, and accepts owner_wrapped_dek only on a PRIVATE volume. Each wrap is bound to volume_id through the canonical hash domain cowboy.cip-9.volume-dek.v1.
dek_version, the two cbss_* binding fields, cbss_rewrap_required and last_owner_transfer_block exist on the chain StorageCommitment. wrapping_key_policy exists nowhere in the node. §9.1’s access classes are a target, so no rule may depend on reading them from state. The block above is a partial view: the chain commitment also carries cbss_committee_override, the MPK-setup fields, relay_set_revision, pending_rotation, deleted_at_epoch, deposit_wei and the whole rent/escrow group. See §17.3.
It keeps one AccountStorageSummary per account:
MAX_VOLUMES_PER_OWNER = 1,024 volumes. CreateVolume, and a TransferVolume naming the account as new_owner, MUST reject a 1,025th volume with RasInvalidDataSite::QuotaExceeded (E1213) before any write. Both also reject while the account’s CBSS force-rekey cursor is live. The cap bounds the owner-scoped volume index, one bincode Vec<VolumeId> value (32,776 bytes at the cap, inside the 64 KiB actor value limit), and the CBSS force-rekey walk over it, which marks at most 64 volumes per call (16 calls at the cap; CIP-24).
11.2 Relay registry
The Relay Registry is the system actor atRELAY_REGISTRY = 0x0B. It manages:
RelayNodeProfileentries (see §5.2)- Active relay list, stored as two fixed pages
ras:active-relays:0andras:active-relays:1of raw 20-byte addresses (2,048 per page). Liveness isRelayStatus; there is no health decay. - Shard assignment index:
volume_id → list[PlacementAssignment] - The network-wide
AUTO_DRAIN_POLICY_KEYslot written by governance (§5.8)
11.3 Key space
The Storage Manager stores Storage Commitments in the CIP-4STORAGE key space under its own address, with ASCII-prefixed keys and JSON values:
(owner, name) → volume_id lookup (§12.3), written at create_volume(). It is keyed by the owner’s chain address, the same owner_address used to derive volume_id. The derivation is owner_volume_name_key(owner, volume_name):
0x0B), indexed both ways:
12. Client interfaces
RAS gives Runner workloads two client interfaces:- The filesystem interface (§12.1) is a FUSE-mounted directory that presents the volume as an ordinary filesystem. It is the primary interface for agentic workloads, where an LLM works through tool calls, and the model’s filesystem tools work on it unchanged.
- The Object API (§12.2) is a programmatic interface for orchestration code and non-agentic workloads. The FUSE layer calls it internally.
12.1 Filesystem interface (FUSE mount)
12.1.1 Design rationale
The main consumer of Runner storage is an AI model making tool calls such asRead, Write and Bash, which operate on filesystem paths. A custom put_object / get_object API would need custom tool definitions for every model and would cut storage off from standard Unix tools. A mounted volume lets the model’s existing tools work directly. Each tool call maps to FUSE operations as follows:
12.1.2 Mount point layout
The Runner mounts each attached volume at a deterministic path inside its execution environment:path_prefix, only that subtree is visible:
12.1.3 Sync strategy (hybrid)
The FUSE mount keeps an in-process memory cache (each file a stub, clean or dirty) with a shared size budget, which every filesystem operation reads and writes first. A background sync daemon moves data between the local layer and the Relay Nodes in two cycles:- The push cycle (local to Relay Nodes) detects files written or modified locally (via inotify/fswatch), encrypts them, erasure-codes them, and distributes the shards to Relay Nodes.
- The pull cycle (Relay Nodes to local) re-fetches the volume’s committed manifest from Relay Nodes, verifies it against the onchain
manifest_root(§7.5), and copies into the local cache the new objects that are not already there. Reads are READ_COMMITTED (§7.5): a reader sees only what a committed root contains, so another writer’s objects appear only after that writer callscommit_manifest().
SYNC_INTERVAL_SECONDS = 5. The journal is a log on the Runner’s local disk of manifest changes not yet committed on chain.
Manifest commits are batched: one commit carries every change waiting on the volume. A change is staged once its shards are uploaded and it is journaled, and it has landed once the writer has verified that the chain finalized the commit that contains it (§12.1.5 gives the exact definitions). A write reaches the chain as follows:
- The push half of a sync cycle stages every dirty file: the file’s Relay Nodes accept its shards (§5.4.1) and the Runner durably journals the manifest change.
- Each volume has one commit loop. The loop waits out the commit window (
COMMIT_WINDOW_SECONDS = 5), then submits everything staged since the last landed root as a singlecommit_manifest()and waits for finality. If nothing is staged, the loop submits nothing. - A change staged while a commit is in flight joins the next commit, one window later.
fsync) does not wait for the commit window. It submits the waiting batch immediately and waits for it to land. This document calls that an anchoring commit.
fsync() on a file stages that file, then submits the batch as an anchoring commit and returns once the commit has landed. This is the default --fsync commit mode, which the CLI, standalone mounts and every Runner mount use. With the opt-in --fsync local, fsync() returns once the file is staged, without putting the root on chain.
A bare sync does nothing for the mount: the mount has no syncfs handler. sync DIR becomes fsyncdir(DIR), which publishes only names created directly in DIR that are not yet in the manifest. It does not push overwrites of existing files or anything in subdirectories, and it still returns success. In commit mode it then submits and waits for an anchoring commit. Per-file fsync() is the only durability barrier for a file’s contents.
Some refusals cannot be fixed by retrying: the volume’s escrow is too low, or the signer lacks commit authority. On such a refusal the commit loop stops submitting and keeps the batch staged until someone acts, and a later fsync reports it as EIO, or as EACCES while the batch waits for the volume owner to finalize it (a prefix-scoped writer stages a commit and the owner finalizes it, §7.3.1). The commit loop retries transient failures with backoff, and the Runner’s next fsync recovers a failed commit before staging anything new. An anchoring commit retries a failed commit once before reporting any failure other than such a refusal.
If a Runner dies after a change is staged and before its batch lands, it replays the journaled batch when it reopens the volume with the same local journal. The journal stays on that Runner, so if a different Runner mounts the volume next, staged writes that had not landed are lost.
Relay Nodes delete shards that no committed root references after ORPHAN_SHARD_TTL_SECS (24 hours). The Runner uploads again any upload older than half the TTL before a root references it. If relays have already deleted an upload and it is still missing after retries, its path is dropped from the batch; the batch publishes the rest and reports the lost paths, and a Runner that still holds those bytes uploads them again. Otherwise those writes are lost.
On container shutdown, the sync daemon pushes all dirty files a final time, then submits the batch as an anchoring commit. If the container crashes before that push, data written since the last push cycle is lost, unless fsync acknowledged it.
Reads behave as follows:
Writes behave as follows:
Every mount schedules the pull cycle. The push cycle runs only under a grant that permits writes. The refresh is deliberately not a MUST, because §7.2 gives
WRITE_ONLY no read authority. Whether a write-only mount’s pull succeeds depends on visibility:
- On a public volume, reads and listings are open (§7.6), so the refresh succeeds and the mounter sees the object namespace (paths, sizes and object descriptors). The mount still refuses every POSIX read of object contents (§12.1.4). Any party can already read that namespace.
- On a private volume, the refresh is
GET_PLACEMENTfollowed by per-nodeGET_SHARD(§5.3.2), and §7.2 grantsWRITE_ONLYneither. The Relay Node refuses, and the client cannot tell the refusal from transient unavailability. The mount has no committed manifest to compute its changes against, so in practice it cannot commit. The parent-root compare-and-swap of §7.3.1 (the chain accepts a commit only if the root it extends is the volume’s current root) prevents a destructive commit.
GET_SHARD for manifest-node shards only, and a way for a Relay Node to recognise such a shard, which opaque shard IDs do not give (§7.3.1 Layer 3, §7.6.4). The write-time shard metadata of §7.6.3 is the nearest existing mechanism.
The gap MUST NOT be closed by admitting object GET_SHARD for WRITE_ONLY tokens wholesale. A write-only mount already holds the volume DEK (§9.2), so relay-side refusal of object GET_SHARD is the only thing between a write-only Runner and full plaintext. Such a change would belong in §15.2 as a deliberate change of the threat model.
Object materialization (fetching object bytes into the local cache) is an object GET_SHARD and is checked as one. GET_SHARD, PUT_SHARD and COMMIT_MANIFEST are checked where they are performed: at the shard fetch, the shard put and the commit. §7.2 separates commit authority from write authority, so a PUT_SHARD answer must not be taken as a COMMIT_MANIFEST answer.
A long-lived mount can outlive its CapToken’s valid_until. The sync daemon MUST re-mint the CapToken (obtain a fresh one) before expiry (via the mount’s set_auth_token hook) and continue without interrupting in-flight reads or writes. A write sent with a token that expires before the relay checks it retries with the refreshed token and does not surface a 403 to the workload.
12.1.4 FUSE operation mapping
This section defines access modes (§7.2) over POSIX mode bits, so that two implementations refuse the same requests. Depending on its grant, a mount withholds some mode bits: it reports them as
0 and refuses to set them. A grant is partial when it does not permit both reads and writes. For a given inode, a mount reports, and may change, only the bits its grant does not withhold:
Linux always reports a symlink’s mode as
0o777, so a mount does not withhold the read and write bits on a symlink. A partial grant still withholds setuid, setgid and sticky on every inode, symlinks included, so it can never write those bits into the volume. A READ_WRITE grant withholds nothing.
Under a partial grant, a withheld bit reads back as 0. A tool echoing that view (cp -p, rsync -a, tar -p, chmod --reference) then sends the same bytes as a caller asking to clear the bit:
- A request that leaves the withheld bits at
0is ambiguous, and the implementation MUST preserve the stored value of those bits. Storing the zeros would write them into the manifest and clear those bits for every mount. - A request that sets a withheld bit is unambiguous, because no caller read a
1there from this mount. The implementation MUST refuse it withEPERM, and MUST NOT apply any other field of the samesetattr, sincechmod(2)applies the requestedS_IALLUGObits or fails.
WRITE_ONLY the withheld set is 0o7444, so any request naming one of those bits fails, including cp -p, rsync -a, install -m 0644, chmod u+r and chmod 0600. The file contents are still written; only the mode change is refused. A full grant has no withheld bits, so neither rule applies to it.
On a WRITE_ONLY mount, no request can clear the group and other read bits. chmod 0600 over a stored 0o644 sets the withheld bit 0o400 and is refused. chmod 0200 succeeds and preserves the stored 0o644, so it reports success while the durable state keeps the group and other read bits.
Implementations MAY materialize explicit directory entries in the manifest, to carry POSIX mode, ownership, timestamps and xattrs for directories that contain no files. Where they do, those entries are manifest state covered by §6.3. A Runner MUST NOT be able to remove one, in any access mode, because removing it mutates the volume index and produces a new manifest_root committed under a Runner CapToken with no owner authorization. Such an extension MUST NOT create a deletion capability that §6.3 and §7.2 withhold. The account owner removes directory entries through the onchain API.
12.1.5 Consistency model
This section states what a mount guarantees to the workload and to every other reader of the volume. It applies to every FUSE mount (CLI and Runner). An Object API client (§12.2) holds the root it last refreshed and sees a newer one only when it refreshes. A change is staged when the push half of a sync cycle has uploaded its shards and journaled it on the writing host. It has landed when the writer has verified a finalized-state proof in which the volume’smanifest_root and head sequence are the ones its commit produced. The head
sequence is the volume’s commit counter. Only landed
changes are visible to other mounts (§7.5).
The unit of ordering is the volume: each commit replaces the root and advances
the head sequence by one through the parent-root compare-and-swap (the chain accepts a commit only
if the root it extends is the current root), and commits on
one volume are totally ordered. The unit of conflict is the manifest entry at
one path (a file, an explicit directory or a symlink). Commits that change
disjoint paths all land: a writer whose commit is refused re-applies its
changes on top of the new root (rebases) and submits again. A commit is atomic;
a mount’s batch is not, because a conflict removes only the conflicting paths
from it. A file is replaced whole: writes from two mounts to one file never
merge byte ranges, so O_APPEND from two mounts keeps one writer’s file.
Nothing is ordered across volumes.
The guarantees are:
The visibility bound is as follows. A change written on mount A at time
t is visible on
every healthy mount B no later than t + V, where
With the defaults at 1 s blocks,
V ≈ 28 s + R. After a successful
--fsync commit fsync() returns at t_f, the change is visible on every
healthy mount by t_f + A + S + R (15 s + R). A staged batch larger than
4 MiB is submitted without waiting for W.
The bound holds while the writer’s publications and the reader’s refreshes
succeed. A transient publication failure backs off as W · 2^min(f, 4) after
f consecutive failures (at most 80 s), and the change becomes visible within
V of the publication that succeeds. A refresh that cannot obtain a proof
younger than A, or cannot fetch the manifest, leaves the mount on the root it
has; it never serves an unverified root (§7.5). An external tester SHOULD
treat a path missing from B less than V after the write as a stale read, and
one still missing after V with both mounts healthy as a lost write.
A mount MUST keep pulling while it has changes it has not published or cannot
publish, overlaying them on each pulled root, so that a writer that is failing
to publish still converges on everyone else’s writes. The pull runs every sync
cycle, and the pulled root is merged under the mount’s unpublished changes.
A conditional publication is a commit transaction that is valid only against
one approved root. While one awaits explicit recovery, the mount neither
publishes nor pulls, so that the mount’s root stays the approved one.
When shards or manifest nodes cannot be published,
nothing is submitted. The writer keeps its changes in cache and journal, keeps
showing them, and retries on the next cycle or backoff; an fsync() waiting on
the batch fails with EIO. On a refusal that retrying cannot fix (the
volume’s escrow is below its minimum balance, the owner must finalize the
commit, or the client’s relay list is out of date), the writer stops
submitting the batch: the next fsync() reports the refusal, and the batch
stays staged until someone acts.
Other readers see the previous root throughout, and published nodes of a
root that never commits are never accepted by a reader.
On commit failure, the chain refuses a commit whose parent root or head
sequence is no longer current. The writer reads the new root from a finalized
proof and re-applies its batch on top of it. Changes to paths the other commit did not touch
are carried over and published in the next attempt. A path the other commit
changed, where the batch’s base for that path is no longer the committed
entry, is a conflict, resolved as below. A submission whose outcome is unknown
(the volume gas nonce was taken, or the response was lost) is resolved from
the next proof: if the commit landed, the writer accepts it as its own; if it
did not, the writer re-applies the batch on the current root and resubmits. Readers never see a refused root.
For concurrent writes to one path, the first commit to land for the path wins.
A later batch changes that path only if its base for the path is still the
committed entry. Otherwise its change to that path is dropped from the batch:
the losing mount shows the winner’s version, the conflict is logged, and the next
fsync() of the affected file (or the one already waiting) and fsyncdir() of
its directory fail with EIO once. The losing bytes are not committed. A
write made after the change is dropped is based on the winner and lands normally, so
the last writer to write after seeing the winner wins. No write() returns an
error for a conflict, and a write that was never fsync()ed can be lost this
way without any syscall reporting it.
Where directories are explicit
manifest entries (§12.1.4), two mounts that create the same directory merge:
the committed directory stands, both succeed, and the children both mounts
create under it land. When two mounts create one path as different kinds (a
file on one, a directory or symlink on the other, including creation under a
path the other made a file), the committed entry wins. The losing mount moves
its uncommitted entry, with everything under it that it has not committed, to
<name>.conflict-<mount id> in the same directory and publishes it there. The
mount id is eight hex digits, random per mount session; a taken name gets
-2, -3, and so on, and the name is shortened to stay within
MAX_OBJECT_PATH_LENGTH. Every mount then shows the committed entry at the
original path and the moved entry at the conflict path. The next fsync() of
each moved inode and fsyncdir() of the directory fail with EIO once.
The moved entry MUST be journaled like any staged change, so that a crash or
remount before it lands replays it at the conflict path.
Current implementation: the move is held in the mount’s memory until the conflict entry lands. A crash in that interval, or a conflict met by a batch restored from the journal after a remount, loses the moved bytes.
12.2 Object API (programmatic)
The Object API is the low-level interface the FUSE layer uses internally. Scripts, lightweight jobs and orchestration code outside a container call it directly.put_object splits the plaintext into chunks and encrypts private chunks. It erasure-codes each chunk and distributes it separately. The result is one ObjectDescriptor with ordered chunk commitments and one PlacementRecord per chunk, which the client publishes at commit time. get_object locates, reconstructs and verifies each chunk, decrypts private chunks, and checks the aggregate object hash at the end of the full read.
Committing a volume takes two steps, one in the data plane and one in the control plane (§4.3):
-
The CBFS commit (
commit) publishes the delta’sadded_nodesto Relay Nodes, each addressed bymanifest_node_shard_id(volume_id, locator)(§3.1), and publishes PlacementRecords to all assigned Relay Nodes viaPutPlacement(§5.3.1). It does not touch the chain. Manifest-node PlacementRecord publication MUST succeed on every assignee that holds its shard before step 2. An owed index’s relay is skipped; repair gives it the record (§5.3.1). Object-shard record publication is best-effort: the client logs failures, and they do not block the commit. -
The Cowboy commit (
commit_manifest) submits the new manifest DAG root, theprev_rootit extends, and theadded_shards/removed_shardsdeltas to the Storage Manager in a transaction the client signs and submits (§12.2.1). The chain rejects the commit unlessprev_rootequals the volume’s currentmanifest_root(parent-root compare-and-swap). In the same state transition, the chain updateslive_shard_countand its own inventory of the volume’s stripes (the native stripe rows and dense stripe-ordinal rows, §11.1) for the added and removed stripes. READ_COMMITTED reads (§7.5) read from the committedmanifest_root. The live inventory rows supply PoR’s per-shard issue-time commitments (§5.6). Individual ObjectDescriptors stay off-chain in the manifest DAG. For a public volume, the plaintext root yields O(log N) object-inclusion proofs. For a private volume, object contents stay opaque, and PoR checks shard retrievability against the selected native row.
GARBAGE_COLLECTING still MUST preserve shards in the current authoritative committed set. A relay establishes that set from the chain’s native RAS inventory (the chain’s own per-stripe record of committed shards) at the same finalized state as the current authoritative manifest_root (§11.1). The cursor-paged /ras/chain/volumes/{id}/shards view expands each stripe into computed shard numbers c·w+i (stripe ordinal c, stripe width w, shard index i) and exposes a change token. A scan MUST restart if that token changes between pages. The inventory includes manifest-node shards and works for private volumes without decrypting the DAG. A stale removal event, or a lagging event-derived live set, is not by itself proof of absence. If the relay cannot establish that a shard is currently absent from the committed set, or that the local publication may be deleted, it MUST defer deletion. Once the finalized volume status is GARBAGE_COLLECTING, that terminal status authorizes the relay to delete the whole volume’s data (§13.3), even while native rows remain until the chain finishes removing them in bounded steps. A shard shared with another volume still needs protection against deletion while that volume references it. Relays apply publication holds, the current-set check and a publication guard on every relay write and delete.
On every successful commit_manifest, the Storage Manager emits a chain event:
manifest_root every MANIFEST_POLL_INTERVAL blocks remains valid as a fallback.
12.2.1 Manifest commit transport
Control-plane writes (create_volume, commit_manifest, UpdateMountAllowlist, relay registration, escrow deposits) are account transactions. The client, or the relay for its own duties, signs the transaction with its own account and submits it to a node, and that account pays gas unless the payer rules below select workload funding. Owner-authorized instructions also carry the owner authorization below.
From CBFS_WORKLOAD_COMMIT_PAYER_ACTIVATION_HEIGHT, eligible workload-bound
owner-delegation commits use C3’s payer rule.
In this section a floor is a minimum resource budget that the signed transaction
must carry. Admission (the node’s check before it accepts the transaction for
inclusion) and classification use constant work only: a supported kind, a signed
binding-presence hint where available, and fixed cycles and Cells floors. Admission MUST
NOT scan refs or derive a payload-sized access floor. The live selector (the payer
selection that runs when the transaction executes) prepays
payload_length * SHAPE_SCAN_CYCLES_PER_BYTE, with that rate equal to one,
before it reads the payload and counts every added or removed ref as a separate stripe.
For A added refs, R removed refs, H chunks, O owed refs and payload length L,
the width w is one plus the largest shard index (or MAX_STRIPE_WIDTH for no refs),
and D is zero for an empty delta and w otherwise. At K = 0, base Cells are
1,048 + 4L + manifest_cells(w,D,A,R,A,R,0) for Direct and
1,048 + 4L + stage_cells(H,O,A,R) for Stage. The scan-derived access row
reserves PoR net-stripe change for any nonempty delta. It uses the maximum token,
bound-workload and owner-nonce branches, without typed vectors or stripe sets.
Only after that does the selector compare all three signed dimensions with the
complete base plus the C1 and C3 allowances. An initial-floor failure performs no
scan or C3 work. A later shortfall selects signer payment and keeps the scan charge.
A Finalize commit carries only volume and stage ids. Its live selector first
prepays one retained header point read, within C3’s five-read allowance, before
the scan, the complete comparison or any other C3 work. An unbound header selects
signer payment, with only that read charged. A bound header continues with the
scan and complete comparison, and reserves only four later discovery reads.
Other unbound commit kinds pay for no C3 work. The complete live Finalize access
comparison then reuses the paid volume and header records and storage epoch to apply
C3’s persisted volume/chunk bound.
That bound covers commits accepted by apply_manifest_commit. Malformed retained
stages still fail both below and above the gate, and selected funding pays for an
eligible failure. The live bound check follows six discovery accesses. A valid
non-hooks Finalize whose signed/live structural budget runs from its exact
native need through native need plus five can therefore become signer-paid
OutOfGas at the gate. Direct and Stage typed decoding and payload hashing stay
inside the fixed discovery allowance, regardless of a larger signed budget.
Deterministic C3 decode and authority failures select signer payment without
waiving ordinary RAS validation. Local storage and PVM faults escape before a
receipt. Off-chain estimates may compute the complete signing floor, while
admission uses only the constant-work possible-candidate check. Estimates and
final signing include each overhead once. Below the gate, existing payer
selection applies, and explicit volume-escrow transactions keep their
separate funding rules.
Payer advice is optional for both hooks and production runner signing. On a
transport, HTTP or parse failure, on missing or malformed required-limit or
overhead fields, on success: false, and on over-cap advice, signing MUST
retain all original signer budgets atomically. Valid suggestions are inclusive
totals. Signing consumes each one once and retains the exact Cells floor that the
planner computes for the storage epochs waiting to be settled. The production runner uses reporter-scoped submission
behind C3. It shares the Reporter’s nonce journal and preserves an existing
stage’s signer across activation. Unknown Finalize stages keep the node signer.
On shutdown, the runner keeps the Busy notifications it has already consumed so it can reconcile them later,
with bounded retry delays. These client rules leave the consensus payer
predicate unchanged. Only the current workload.reporter can select
owning-actor funding, and other signers keep signer payment.
C3’s payer rule gives
the complete bound, eligibility examples and signing contract.
Signers have three further requirements:
- The owner authorization MUST canonicalize addresses as EIP-55 in both the payload hash and the signing bytes. The owner-action payload is signed over an explicit, language-neutral, big-endian byte layout independent of Rust struct encoding. Every signer and verifier, native or WASM, MUST produce these exact bytes. Golden vectors fix the layout.
- Owner delegations, owner and relay authorizations, acknowledgements and relay-evidence signatures use CIP-40 §7.6’s
object_signing_preimagewith the runningChainIdentity. Delegations have nonetworkfield; the instance commits to the genesis network. A verifier never trusts a chain ID or network supplied by the signed object as its own expected identity, and chain ID zero has no exemption.RasOwnerDelegationand workloadDelegationCerthave distinct domains. Owner-authorization nonces are consumed once. Relay self-disable timestamps obey the drain commitment freshness window. - Processes that sign with the same gas key MUST select nonces through one shared account journal under an exclusive lock. Processes that share a key without it race on nonce selection.
put_many) and other long operations MUST refresh the CapToken mid-stream. The writer MUST re-mint before expiry, through the same set_auth_token path the FUSE mount uses (§12.1.3), and continue without aborting the batch.
12.3 Account owner API (via Storage Manager actor)
12.4 CIP-2 task definition extension
TheOffchainTask struct from CIP-2 gains an optional volume_attachments field:
VolumeAttachment normatively: the 10-field wire-canonical structure, its RelayEndpoint shape, and the AccessMode tag values (ReadOnly = 0, WriteOnly = 1, ReadWrite = 2). This extension carries the same structure. There is no read_consistency field, because all reads are READ_COMMITTED (§7.5).
12.3.1 Volume creation and lookup errors
Volume-create and name-lookup failures MUST be distinct, named errors, replacing the opaqueE1900 "invalid data". The errors are:
E_VOLUME_NAME_DUPLICATE:(owner, volume_name)is already present in the name index (§11.3).E_VOLUME_ESCROW_INSUFFICIENT:initial_escrowexceeds the owner’s spendable balance (§10.3.1).
-
E_STORAGE_WITHDRAW_BELOW_FLOOR: a volume withdraw would leave escrow below the rent floor. The detail names the maximum withdrawable. -
E_STORAGE_WITHDRAW_EXCEEDS_RESERVE: an account-reserve withdraw exceeds the reserve. -
E_STORAGE_AUTO_TOP_UP_INVALID: aCustomauto top-up with a zero cap ortarget_epochs0. -
E_STORAGE_GROWTH_BELOW_FLOOR: a commit that grows a volume left escrow below the rent floor of the new size after the reserve draw. The detail names the floor, the escrow and the shortfall. -
E_VOLUME_PARAM_INVALID(receipt code2002): erasurekoutside2..=16ormoutside1..=8(the retainedk+m ≤ 32check is a metering-bound parity guard), unknownvisibilityorwrapping_key_policy, avolume_nameviolating the[a-zA-Z0-9_\-.]/ 64-byte rule (§6.1), or an explicitly suppliedrelay_nodeslist whose length is noterasure_k + erasure_mor that repeats anode_id(§5.3). -
E_VOLUME_NOT_FOUND_FOR_OWNER: a name lookup missed the index for this owner-domain (§11.3). This MUST NOT be reported as a bare 404 that hides whether the caller queried under the wrong owner. -
E_VOLUME_RELAY_FLOOR(receipt code2005): auto-assignment finds fewer thanerasure_k + erasure_mdistinct usable relays (§5.3). An explicit list of the wrong length, or with a repeatednode_id, fails earlier asE_VOLUME_PARAM_INVALID(§5.3). Creation MUST be refused and never half-completed, because each shard index needs its own relay (§5.4.1). -
E_VOLUME_RELAY_UNKNOWN: an explicitly suppliedrelay_nodesentry names anode_idthe relay registry has never held.relay_nodes[shard_index]is the per-index responsibility that PoR challenges (§7.6) and rent assignment intervals (§5.6.1) bind to, and a challenge bound to an unknown node settles with only a miss alarm. Creation MUST reject unknown identities and not rely on a later epoch-sweep replacement (§5.6.1). The explicit branch requires registry membership only, without theActive-and-has-room filter used for automatic assignment. The check promises nothing about later status, capacity or possession. The epoch sweep later replaces missing or non-Activeassignments onACTIVEvolumes with anActive, non-full relay and starts its PoR repair grace (§5.6.1); if none fits the eligibility and work-budget checks, the assignment remains. An assigned relay that is full does not by itself trigger this rotation. A full relay’s index is owed at write time (§5.4.1). An owner can still name a registered relay it does not own, without that relay’s consent. §5.6.2 addresses this defect.
13. Garbage collection
13.1 Triggers
The protocol garbage-collects volume data when:- The account owner calls
delete_volume()(soft-delete plus a grace window, §13.2) ordelete_object(). - Escrow plus the account storage reserve stays below the volume’s rent floor (CIP-31 §3) through
STORAGE_GRACE_EPOCHS, and settlement funding cannot restore it (§10.3).
expiry_height; escrow and rent alone decide how long a volume lives.
13.2 Process
13.3 Deletion semantics
delete_objecthas no onchain instruction of its own. It submits a manifest commit and uses that commit’s CIP-43 access row. On a Public volume with a nonzero CIP-33 Store pin count, any object deletion that changes the committed manifest is rejected (§2.1.2 of CIP-33). Otherwise the commit removes the object from the onchain manifest, CapToken holders can no longer reach it, and Relay Nodes garbage-collect its shards asynchronously.delete_volumeon a Public volume with a nonzero CIP-33 Store pin count is rejected (§2.1.2 of CIP-33; the pinned Trading Post implementation cited in §7.3.1). Otherwise the volume enters DELETED (soft-delete): all active CapTokens are revoked, no new ones can be issued, and storage fees keep accruing forVOLUME_DELETE_GRACE_EPOCHS(default: same asSTORAGE_GRACE_EPOCHS). During this window the owner may callundelete_volume()to return the volume to ACTIVE.- After the grace window the volume moves to
GARBAGE_COLLECTING. Both PoR sampling builders exclude it before drawing, even while its native rows remain. The epoch sweep prunes acknowledged event history. - The event-prune gate is passed once the volume’s acknowledged event history has been pruned. After that, a separate garbage-collection pass over the chain’s native inventory (the native-inventory GC lane) advances across blocks, doing a bounded amount of work in each. Each item deletes at most one stripe row and its dense stripe ordinal and durably decrements
live_shard_countbyw. The lane’s cursor lives in consensus state, survives restart and rotates among volumes inGARBAGE_COLLECTING. With healthy queue state and no rows frozen by a fault, a volume near the 1,048,576 live-shard-ref cap drains across blocks without waiting one daily rent epoch per batch. - Rows frozen by a fault stay in the queue until a reviewed recovery. Corruption that cannot be traced to one volume blocks all native-inventory cleanup at the head of the queue until repair, as CIP-31 specifies. Healthy native GC MUST NOT block later rent epochs or settle one epoch twice.
- Committed-data faults follow CIP-31 native inventory GC fault isolation. A fault that belongs to one item quarantines only that item’s native cleanup: its records are kept and automatic processing stops touching it. Undecodable queue rows and faults in shared control state stop processing with the original savepoint intact. Infrastructure failures propagate. If adding to the queue hits corrupt shared GC control state, the storage cursor is kept and storage settlement waits for repair while other kinds of sweep work continue.
- The
StorageCommitmentstays inGARBAGE_COLLECTINGwhile any native row remains. Control-plane cleanup removes the volume from the live index only after the count reaches zero and the event-prune gate is satisfied. Relay Nodes may purge the volume’s bytes without waiting for its native rows to drain (§12.2). Storage fees stop under the terminal status. - The
StorageCommitmentsurvives removal from the live index as a terminalvolume_idtombstone.CreateVolumerejects an occupied ID regardless of status, so garbage collection never makes an ID reusable. - Rent exhaustion can still move a pinned Public Store volume to
GARBAGE_COLLECTINGand remove its bytes. The Trading Post pin count (in separate actor storage) and the RAS ID tombstone both survive, and neither preserves the bytes. CIP-33 §2.1.2 records the unresolved availability and OneShot serving rule. - Garbage collection is irreversible. Once a volume enters
GARBAGE_COLLECTING, its data cannot be recovered.
MAX_PRUNE_PER_PASS = 1,024 events for one volume in each daily storage epoch. A volume with a large acknowledged event history can therefore wait many epochs before the native-inventory GC lane becomes eligible, and a live volume that adds more than 1,024 events per day can grow its backlog without bound. Closing the gap needs:
- A durable, fair per-block event-prune queue. A queue alone does not bound the backlog: the current sweep handles at most 32 items per block, shared with other work, and valid manifest traffic can create more event-prune work than that.
- Before any claim of indefinite production readiness, a separate CIP-10 follow-up that bounds outstanding prune work with deterministic commit admission and backpressure, reserves prune capacity when a staged commit is accepted so that a full prune queue alone never blocks that commit at finalize, and makes temporary admission refusals automatically retryable by clients.
- A recovery path that safely removes the need for a signed ACK from a relay that was assigned when the event was emitted and is now permanently unavailable. The prune gate still requires that relay’s signed ACK, even after eviction, assignment rotation or terminal volume status.
13.4 Ownership transfer
The account owner may transfer a volume to another Cowboy account withtransfer_volume(volume_name, new_owner). Transfer:
- Revokes all active CapTokens. It advances
last_owner_transfer_block, and a token whosevalid_fromis below it is refused. It clearsmount_allowed_actorsin the same transaction. - Updates
StorageCommitment.owneratomically. The new owner takes on billing from the next epoch. TransferVolume.new_owner_wrapped_dekMUST be present if and only if the commitment already carries an owner wrap. When present, it MUST decode as a validOwnerWrappedDek(§9.1), with the required version, algorithm and exact size and no trailing bytes. It re-encrypts the owner wrap to the new owner’s wallet-derived key, so the old owner must cooperate to unwrap and re-wrap. A commitment without an owner wrap MUST NOT gain one through transfer.- Sets
cbss_rewrap_requiredif a committee wrap is present, which blocks committee-seal delivery until the new owner callsRewrapVolume(§9.4.1). A new dispatched grant from the new owner enables runner access without a rewrap (§9.2). - Rewraps the DEK without rotating it (§9.4). An old owner who held the DEK can still decrypt everything written before and after the transfer, and the new owner has no protocol remedy (§9.4.2). Public volumes have no DEK to rewrap.
- Is rejected for a volume in
DELETEDorGARBAGE_COLLECTINGstatus, and for a Public volume with a CIP-33 Store artifact pin. The pin outlives artifact retirement while a pinned hire can execute; CIP-33 §2.1.2 defines its release. The hire-revocation path releases the hire-lifetime reference only at its terminal transition.
13.5 Gateway HTTP serving by status
A public-volume Gateway (CIP-15) mapsStorageCommitment.status to HTTP behavior using the existing lifecycle states. There is no DELINQUENT status.
The Gateway keeps serving in
GRACE_PERIOD because the owner may top up at any moment. DELETED is recoverable during the deletion grace window but is not served. GARBAGE_COLLECTING is irreversible, so the Gateway returns 410 Gone.
14. Parameters
Object chunk sizes
object_chunk_bytes(k, m) is 32 MiB for K+M ≤ 3, 32 MiB for widths 4–6, 64 MiB for 7–12, and 128 MiB for widths ≥13. CBFS-supported grids have 2 ≤ K ≤ 16 and 1 ≤ M ≤ 8, so K+M ∈ 3..24. These object chunks are separate from the small fixed-size chunks PoR uses inside a shard. Shards stay below MAX_SHARD_TRANSFER_SIZE = 256 MiB.
Chunk-tier compatibility
SDK and CLI readers accept the current tiers and, forK+M=3 only, the older 16 MiB tier, and validate each object against its stored chunk sizes. Each object must match one complete allowed tier with uniform visibility overhead: zero for public data, or one 16-byte AEAD tag per private chunk. Ranged reads, including FUSE cache reads, map byte offsets to chunk indexes using the descriptor’s validated plaintext tier, excluding any AEAD tag. Writers always use the current tier, so a reader without 32 MiB support cannot read a new K=2,M=1 object over 16 MiB. Readers must be upgraded before new writers are enabled, including on standalone and dev stores (which have no wipe) and for relay manifest blobs and manifest-root records (which survive the chain-derived-state wipe).
Staged-manifest funding
Current signing funds the source-derived auth-v3 staged-manifest reservation, including pending-epoch and route context. Its structural capacity is 1,116.Historical object-fit audit
The fit costs, headroom and examples below use the pre-auth-v3 rent-inclusive formula and a capacity of 1,114. “Then-current-tier” means the object tiers this audit measured. Every legal then-current-tier object up to 1 GiB, for either visibility and every CBFS-supported K/M grid (2 ≤ K ≤ 16, 1 ≤ M ≤ 8), fits create, delete and replace in one atomic manifest commit, as measured by the rent-inclusive stripe formula in CIP-43 §5.5. This holds for path-local updates under the single-path model: manifest depth 1–3 with no DAG repartitioning, one rewritten node per level, and sufficient pending-epoch headroom. An object’s shard set is atomic; staging can carry a large delta but never splits one object’s publication across commits. Multi-object batches still need a complete-delta check: in volumes of many multi-chunk objects, one insert or delete can repack every following manifest leaf and exceed one commit even at shallow depth. Before signing, clients refuse a change set whose cost one commit cannot cover.
Take a path-local update to a then-current-tier 1 GiB object, with manifest depth 1–3 and no DAG repartitioning, relay-delta count D ∈ {w, 2w, 64}, and no queued storage epochs or native-route surcharge. The worst signed structural costs are 644 create, 772 delete, and 1,084 replace, below the 1,114 ceiling. Empty-volume creation costs at most 580. The worst replace leaves 30 units. At 12 units per queued storage epoch, it admits at most two queued epochs, including the native-route surcharge of two units where applicable. With three or more queued epochs, a near-cap replace waits until the queued epochs are settled and is checked again before signing; the 1 GiB object ceiling is unchanged. Sponsored commits keep their single-stage ref limit (§12.2.1).
Manifest depth 4 is possible within the object-count cap, and because manifest boundaries depend on both path and descriptor, replacing an object can split manifest nodes. For then-current-tier objects at widths up to 6 and D=64, a depth-4 replace with no DAG repartitioning costs 1,108 structural accesses before context surcharges, leaving 6 units and admitting no queued epoch. At that earlier revision, a depth-3 replace with two extra added DAG nodes cost 1,112 and fit that revision’s 1,114 capacity. A depth-3 replace with three extra added DAG nodes costs 1,122, and a depth-4 replace with one extra added DAG node costs 1,126. Both exceed 1,114 even with no queued epochs. The client refuses such a mutation before signing; the owner can delete and recreate the object in separate commits if the pre-signing cost check (preview) of each commit passes, and the object is absent in between.
Legacy mutation budgets
K=2,M=1 objects with 16 MiB chunks stay readable, but the atomic fit guarantee applies only to then-current-tier objects. Such a 1 GiB object has 64 chunks. Replacing it with 32 then-current-tier chunks and one rewritten manifest leaf gives Ac=33,Rc=65, with w=3,H=1,D=3,K=0,J=0, so F′=1,370 exceeds 1,114. The SDK keeps all old chunks in the removal delta and refuses before signing. A deletion with Ac=0,Rc=65 in the same H/D/K/J context costs 1,038 and fits, though deeper or repartitioned manifests or queued epochs can change that. An over-budget replacement can be done as a delete and a write in separate commits when both previews pass. Reading such an object needs no rewrite.
Before reading or uploading object data, the SDK and CLI check the current staged preview and the incoming creation lower bound with fresh queued-epoch and authority context (K, J in CIP-43 current staged-manifest funding), and check the actual delta again before signing.
Governance proposals may adjust these parameters.
15. Security considerations
Private volumes, CBSS account secrets and Cowchat room keys are protected against classical attackers only. Owner ECDSA-derived wrapping, HPKE X25519 and committee BLS IBE do not resist a cryptographically relevant quantum computer. Algorithm tags make the format extensible but do not upgrade existing ciphertext or protect ciphertext harvested today. No post-quantum suite is enabled; one would need its own implementation and key and data rotation policy. Committee envelopes keep CIP-24’s tag 2 (the current BLS12-381 IBE/HKDF/AES-GCM suite). Unknown tags must reject.15.1 CapToken forgery
A CapToken’s authority is the copy that the issuing system instruction writes into Storage Manager state (§7.1); a presented token is honored only if it matches that copy byte for byte. Forging one therefore requires writing Storage Manager state, which is equivalent to compromising consensus. Thenonce is derived at issuance as the first 8 bytes, read little-endian, of keccak256("cip9:cap-token:" ‖ "nonce" ‖ vrf_seed ‖ job_id ‖ runner_address ‖ u64_be(attachment_index) ‖ volume_id), so it cannot be predicted before the VRF seed is final. Replay protection is the byte-for-byte match against the stored copy plus valid_until, which is fixed in that copy, so a Runner cannot extend a token.
15.2 Runner compromise
A compromised Runner with a WRITE_ONLY token can write garbage, consume the account’s byte quota, and write shards outside its path prefix, which Relay Nodes cannot detect because shard IDs are opaque (§7.3.1). On a private volume it cannot exfiltrate data, but only because the Relay Node refusesGET_SHARD and GET_PLACEMENT: the mount already holds the DEK (§9.2), and the client-side refusal runs inside the container the Runner controls. A public volume has nothing to hide. The mount also learns the object namespace through its manifest refresh (§12.1.3), so “write-only” limits what the Runner may change and not what it may see. Mitigations:
- Byte quotas limit the damage per job.
- A prefix-scoped writer cannot commit. It stages, and a full-volume holder reviews and publishes the delta (§7.3.1).
- Shards that no committed manifest references are garbage-collected after
ORPHAN_SHARD_TTL_SECS. - The token has no delete access, so existing data cannot be destroyed.
tee_required=true) for sensitive workloads, and scope path_prefix to the sub-path the job needs. READ_WRITE is an explicit opt-in.
15.3 Relay Node compromise
Relay Nodes hold ciphertext shards (plaintext for public volumes) and cannot read private data without the volume DEK.15.4 Denial of service
15.5 Key management
- Each volume DEK is wrapped per its access class (§9.1). If the owner loses the wallet key, the owner-wrap path is gone, but a committee-wrapped volume stays readable through an authorized runner. The wraps live onchain in the
StorageCommitment, so only the ability to unwrap a given wrap can be lost. - The Dispatcher never holds a plaintext DEK (§9.2), so it cannot exfiltrate one. A compromised Dispatcher could mis-authorize a seal request, a risk bounded by the same consensus-level trust already required to issue CapTokens.
RewrapVolume(§9.4.1) and the transfer rewrap (§13.4) keep the same DEK and revoke nothing, and this CIP does not permit either to be described as a security boundary. DEK rotation does not exist (§9.4.2). After a compromised owner key, or a compromised grantee that already decrypted its package, recovery means creating a new volume and re-encrypting the data into it by hand.- Removing a grant (§7.7) deletes its
grantee_wrapped_dekand stops new CapTokens. That fully withdraws a grantee that never used its package, but cannot reach a key already decrypted, and the permanentpath_taghistory stays linkable under the DEK.
16. Implementation notes
16.1 FUSE mount implementation
The FUSE mount translates POSIX operations into CIP-9 object operations through three components:- The FUSE daemon handles
read,write,readdir,getattrand the other filesystem calls by delegating to the local cache. - The local cache is an in-process memory cache (stub, clean and dirty entries) with a shared size budget. Every read and write hits it first. It fills lazily from Relay Nodes on first access, and eagerly for locally modified files.
- The sync daemon runs the push/pull loop every
sync_interval, detecting local and remote changes and doing encryption, erasure coding and shard distribution.
16.2 Object API client library
Without a FUSE mount, the Runner SDK exposes the Object API directly:16.3 Relay Node implementation
A Relay Node runs these subsystems:- The blob store maps
(shard_id, shard_index) → shard_bytes. Ashard_idis an opaque BLAKE3 hash (§5.3), so the node never sees object paths (§9.3). - The placement store maps
shard_id → PlacementRecord, kept apart from shard data so repair can read placement without the encrypted manifest. - The RPC server accepts shard operations (
PUT_SHARD,GET_SHARD,PROVE_SHARD) and placement operations (PutPlacement,GetPlacement,ReplicatePlacement). TheAuthProviderauthenticates client-facing operations (§7.1.1); node-to-node operations (§7.1.2) are authorized by relay assertion. For public volumes,GET_SHARDandGetPlacementneed no CapToken (§7.6.3), but writes do. There is no listing operation; clients list objects from the manifest, and requests are keyed byshard_id, never by path. - The repair loop runs the two-phase repair cycle (§5.5) against a peer list seeded from config and updated as the node discovers peers.
- Placement sync replicates PlacementRecords to the other assignees and gossips peer lists via
SyncPeers, both as relay assertions (§7.1.2). - Registry upkeep: there is no heartbeat instruction. The relay’s registry status is its liveness (§5.2), and
UpdateRelaychanges its advertised profile. - The spot-check responder returns a random chunk of a named shard when challenged.
16.4 Performance expectations
Expected latencies for the direct Object API:
Expected latencies for the FUSE mount:
16.5 Runner ↔ validator wire compatibility
Runners submit transactions the validator must decode: heartbeats, job results and manifest commitments. A runner built against an older node can emit an encoding the newer validator cannot decode (failed to decode transaction submission); the runner then polls forever and completes nothing, with no visible failure. The current release uses version-3 commonware-codec transaction bytes (CURRENT_TX_VERSION = 3) and their shared verifier (CIP-11 §10.3); actor CBOR payload encoding does not define the envelope. CIP-9 requires:
- A runner’s protocol types MUST be compatible with the deployed validator. A runner outside the validator’s supported wire range is rejected with the typed error
TxUnsupportedVersion(HTTP 400) returned to the submitter, never a silent drop or a generic decode error. - Wire-format changes to runner↔chain transactions MUST be versioned with a discriminant the validator checks. The current coordinated fresh-devnet release accepts only its current codec version; all runners and validators MUST use the same release before producers resume. There is no rolling-upgrade compatibility window.
- Decode-reject reasons MUST be observable to the runner operator, so an operator whose runner cannot complete jobs because of a wire-format mismatch can find out why.
16.6 Client bootstrap and discovery
A conforming client needs only the node RPC URL and discovers everything else from chain; flags and environment variables only override discovered values.chain_id,chain_instance_idandnetworkcome from the closed/chain-inforesponse. Operational parameters come from/chain-parameters, used only after its nested identity matches. The basefee schedule comes fromGET /basefeeandsuggested_max_fee_*, never from hand-set flags; a wrongchain_idor fee fails certificate auth or is dropped by the mempool (fee_below_basefee).- Storage relays come from the relay registry (§5.2), and DEK delivery from chain (§9.2): a dispatched grant’s package from the commitment, and for committee-seal delivery, the committee from the CBSS registry (CIP-24) by epoch, with the sealed ciphertext held by the registered relays.
- The RPC URL MUST be resolved once at startup and passed to every component that needs it. A subcommand MUST NOT silently fall back to a hardcoded default (e.g.,
127.0.0.1:4000) or ignore an explicit--rpc-url. A missing or unreachable RPC is a clear error.
17. Scope and future work
17.1 In scope (this CIP)
- Private and public account-scoped volumes.
- READ_ONLY, WRITE_ONLY, and READ_WRITE CapToken access modes; PUBLIC volume visibility mode (§7.6).
- Concurrent CapTokens on the same volume (agent swarm pattern).
- Two client interfaces, the FUSE filesystem mount (for agentic/LLM workloads) and the Object API (for programmatic workloads).
- Hybrid sync strategy for FUSE mounts (local cache plus background push/pull).
- Relay Nodes as a dedicated storage layer with staking and incentives.
- Reed-Solomon erasure coding (default 4/6) for durability.
- AES-256-GCM encryption with HKDF-derived keys.
- BLAKE3 content hashing at object, ciphertext, and shard levels.
- Path-based addressing with Merkle manifest for integrity proofs.
- Per-epoch billing with grace period and garbage collection.
- Lazy and proactive shard repair.
- Private dual-wrapped volumes with access classes (
owner-key,committee-only;tee-gatedreserved), and owner-curated cross-owner mount allowlists that also authorize direct-client access under the grantee’s own delegation, with recipient-sealed DEK delivery (§7.7, §7.7.1, §9.2.1). - Control-plane writes as client-signed account transactions (§12.2.1), with client discovery of chain parameters, relays and CBSS endpoints from chain (§16.6).
17.2 Explicitly out of scope
- Request-level possession proof for owner tokens. Owner and grantee tokens are bearer tokens after issuance, bounded by the §12.2.1 TTLs. A request-signing extension binding each relay request to the delegated key would cover both token kinds (§7.7.1).
- Content-addressed retrieval. RAS addresses by path only; CID-based retrieval could be layered on top.
- A storage marketplace where Relay Nodes bid on deals. The protocol sets pricing.
- Storage backends other than CBFS (Filecoin, Arweave and others), which would need a follow-on standard.
- READ_UNCOMMITTED consistency, which needs a shard discovery mechanism this CIP does not define (§7.5).
- Rotation without re-encryption, which revokes nothing and would let a commitment assert a rotation that did not happen. Proxy re-encryption or any rewrite-free re-key would be a follow-on standard. DEK rotation itself is in scope and is an unfilled gap (§9.4.2–§9.4.3).
- Runner-initiated task dispatch. Only onchain Actors dispatch tasks; this is a CIP-2 extension.
- A full container runtime spec (images, registries, resource limits, GPU passthrough, network policy). CIP-9 defines the storage layer, FUSE mount and object API; a separate CIP defines the runtime.
17.3 Implementation status
- Deletion that protects newly published bytes (§12.2) is implemented. Relays apply publication holds, the current-set check and a publication guard on every relay write and delete.
- The canonical owner-authorization signing path (§12.2.1) is implemented. Owner-auth, owner-delegation and relay-auth signing bytes come from one canonical encoding shared by the validator and clients, and the owner-delegation verifier binds
chain_idto the running chain. wrapping_key_policy(§9.1) is pending. The access-class field does not exist anywhere in the node, so §9.1’sowner-key/committee-only/tee-gatedtable is a target, and no normative rule may be conditioned on reading it from state.owner_wrapped_dekexists onStorageCommitment, on the create request and on the open/mount response, alongside the committeewrapped_dek.- DEK rotation (§9.4.2) has no design;
RewrapVolumeis live as §9.4.1 describes. §9.4.3 lists the obstacles any rotation design must solve on the shipped chain. - The CapToken
signaturefield (§7.1) is reserved for a future client-verifiable token form and is zero today.
Appendix A: Worked examples
The examples use the actor SDK. An actor dispatches a container job from a@runner.continuation handler with runner.container(image, command, volumes=[...]). Each volume entry is a name (read-only) or a (name, mode) pair with mode "ro", "rw" or "wo", and the Runner FUSE-mounts each volume at /mnt/volumes/{name}/. The Runner runtime does the encryption, erasure coding and Relay Node traffic. The workload uses ordinary file operations, and a mount pushes and commits its changes when the container exits.
A.1 AI agent with persistent memory
A trading agent runs as an LLM loop inside a container with a read-write mount:A.2 Distributed scraping with map-reduce
Two WRITE tokens with an empty prefix cannot be active on one volume at once (§7.3), so each scraper writes to its own volume. The actor invokes this handler once per site, and each invocation dispatches one scraper with a write-only mount:/mnt/volumes/scrape-0/about-page.json. Its writes commit when the container exits.
After all scrapers finish, the actor dispatches a collator that reads all five volumes and writes to a results volume:
A.3 Agent swarm with a coordinator
Five sub-agents each write a report to their own volume with a write-only mount. A coordinator mounts all five read-only and its own volume read-write, and reads each report as it lands. Reads are READ_COMMITTED (§7.5), so the coordinator sees a sub-agent’s files only after a commit containing them lands. A sub-agent’s output can land across several commits while its container runs (§12.1.3), and the remaining staged files land when it exits; a sub-agent that needs its report to appear whole callsfsync() on every report file, writes a completion marker only after all of those succeed, and then calls fsync() on the marker. The coordinator waits for the marker. The actor invokes coordinate once and research once per agent, and each invocation dispatches its own job:
ls and Read.
A.4 Multi-stage pipeline with handoff
The two stages run one after the other on one volume, so only one WRITE token is active at a time. Stage 1 preprocesses data:Appendix B: Comparison with existing systems
Appendix C: CapToken wire format
cap_token_hash and cap_token_signing_digest) encode access mode as one byte using ReadOnly = 0, WriteOnly = 1, ReadWrite = 2, matching §8.1. The encode_cap_token transport is the runner-token tag followed by fixed-integer, little-endian bincode serialization of CapToken. Its access-mode enum discriminant is u32le with WriteOnly = 0, ReadOnly = 1, ReadWrite = 2. Do not use the hash-preimage order to encode that transport. The RAS MountGrant commonware wire, owner-action preimage and grant AAD use the one-byte RAS order specified in §9.2.1.
The signature field is reserved and zero. Token authority is the byte-for-byte match against the copy in Storage Manager state (§7.1).
Appendix D: “Why won’t my volume mount?”
A storage-attached job must clear six independent gates, in order, before a Runner executes it. A conforming implementation MUST make each gate emit a distinct, structured reason.
A job that fails one gate never reaches the next. Implementations MUST report which gate stopped the job instead of collapsing all six into a generic “job did not dispatch”.