Skip to main content

CIP-9: Runner-Backed Storage

Status: Draft Type: Standards Track Category: Core Created: 2026-03-04 Updated: 2026-11-05
Cowboy Cloud and Cowboy Chain. This CIP specifies the Cowboy protocol. Cowboy Cloud runs on this protocol today. Cowboy Chain, a public network on the same protocol, is under consideration for a future release. Where the two differ, this CIP says so.

1. Background and goal

CIP-2 Runners are stateless and short-lived. A job that needs secrets, large outputs, data carried between jobs, or a persistent container disk has nowhere durable to keep it (§2). This CIP specifies off-chain storage volumes that belong to a Cowboy account and that Runners attach during jobs. It extends CIP-2 (Verifiable Off-Chain Compute) with storage that outlives a single job. CBFS (the Cowboy File System) is the data plane: encrypted object writes, manifests, erasure coding, Relay Node RPCs, placement records, repair, and the FUSE and object clients. Cowboy adds the control plane: on-chain StorageCommitment records, CapToken issuance and revocation, Relay Registry state, billing, and task attachment. Together they are Runner Attached Storage (RAS).
  • A Cowboy account (EOA or Actor) owns and pays for every volume.
  • Volumes are private by default. A private volume is encrypted by the Runner before data leaves it, and only the owning account can decrypt it. A public volume (visibility = PUBLIC) is plaintext, and anyone can read it without a CapToken; it serves web assets (CIP-15) and public datasets.
  • A CapToken grants a Runner read-only, write-only, or read-write access to a volume. Several can be active on one volume at once. Public volumes need one only for writes.
  • Every object has a deterministic path, so the owner can read or delete it directly.
  • The client erasure-codes each object (Reed-Solomon) across several Relay Nodes.
  • Storage is billed in CBY per byte per epoch, including erasure overhead, at CIP-31 rates in nano-CBY. This extends CIP-3 Cells to off-chain blobs.
To learn how data is stored and read, take the sections in this order:
  1. What a volume, object, chunk, shard, stripe, manifest and root are: §3 and §3.1.
  2. How a write flows (upload shards, publish manifest nodes, commit the root on chain): §4.2, §5.3, §5.4, §5.4.1, §12.1.3, §12.2 and §12.2.1.
  3. How a read flows (resolve the root, fetch the manifest, fetch shards, decode): §5.3.2, §6.2 and §7.5.
  4. What a reader is guaranteed to see: §12.1.5.
  5. Failure handling and repair: §5.5 and §5.6.
  6. Rent and fees: §10 and CIP-31.

Genesis signing and material identity

CIP-40 §§7.6–7.7 define the normative signature slots, domains and tagged keys for this CIP. These keys use SigningPublicKey restricted to Ed25519: RasOwnerDelegation.delegated_pubkey, owner-authorization keys, DelegationRevocation keys, relay identity, mount-grant signing keys, and the CIP-10 workload release and commit recipient keys. Their authorization fields use SignatureSlot under the table’s policy. A storage key that contains a signing key uses storage_key_fragment() (algorithm ‖ key). This includes ras:delegation:, ras:revocation: and ras:svrn:. Decoding is structural. Consensus charges for per-key validation before it validates and stores the key. Material and volume-DEK hashes bind the real chain identity in the length-framed BLAKE3 construction (u16le domain length ‖ domain, then each field as u32le length ‖ field). The first two identity fields are u64le(chain_id) and then chain_instance_id[32]. These two fields take the place of a single material_chain_id field in the release-key-material and volume-DEK hashes, and of the pair chain_id, material_chain_id in the service-volume request ID. There is no protocol-owned material chain ID 1. The domain labels and the order and width of every other field are as specified; these codec APIs take the chain identity as input. Owner-key-wrap HKDF info and grant-DEK AAD are not delegations. They keep their specified chain_id ‖ network framing. The AAD encodes the grantee’s signing key in the canonical tagged-key encoding and the grantee’s encryption key as specified.

2. Motivation

CIP-2 Runners are stateless: each job runs in isolation and terminates. That leaves four gaps:
  1. Runners need secrets (API keys, certificates, credentials) that only authorized Runners can decrypt, optionally after TEE attestation.
  2. Jobs produce artifacts larger than the 64 KiB on-chain inline cap (whitepaper §7), such as images, datasets or model weights. These need off-chain storage with a hash of the content recorded on chain.
  3. Training loops, agents with history, pipelines and swarms need addressable storage that persists between jobs.
  4. CIP-10 containers need persistent storage across restarts.
Most Runners are short-lived containers without durable disk, so Runners are clients of a separate storage layer.

2.1 Why not existing systems?

Existing systems each fall short for Runners: RAS takes Storj’s Macaroon-style scoped tokens, Sia and Storj’s erasure coding, S3’s path addressing and Sia’s contract-expiry cleanup, and fits them to Cowboy’s accounts, Runners and fees.

3. Definitions

  • A Volume is a named, account-scoped namespace of objects. An account may own several.
  • An Object is one blob in a Volume, identified by a path key.
  • CBFS is the off-chain storage engine. CBFS nodes are the Relay Node data plane, and CBFS clients implement the object API, manifest handling and FUSE mount.
  • A Storage Commitment is the on-chain record of a Volume’s existence, owner, size, shard placement, creation epoch and billing state.
  • A Capability Token (CapToken) is a bearer token that carries a client’s permissions (read-only, write-only or read-write), scope (volume and path prefix), time bounds and size quota for one Volume. The client may be a Runner, the owner’s CLI or SDK, or another authorized caller.
  • A Relay Node is a CBFS node that stores erasure-coded shards. It is a separate network role from Runners and Validators, with its own staking and incentives.
  • A Shard is one erasure-coded fragment of an object chunk or manifest node; any K of its K+M shards reconstruct that chunk or node.
  • A Stripe is all w = K+M erasure shards of one object chunk or manifest node (leaf or interior), sharing one shard_id and containing each index in 0..w (0 through w−1) exactly once. An object can span several stripes. The chain’s own record of stored data (its native inventory) and its rent assignments both count in stripes (§11.1).
  • An ObjectDescriptor is the immutable identity of one object: path, optional content type, write ID, aggregate plaintext hash and length, and an ordered vector of ChunkDescriptor commitments. Each chunk carries its own plaintext and stored-byte hashes, nonce, stored length, erasure grid and shard commitments. The manifest holds it in ManifestEntry::File, encrypted for private volumes.
  • A PlacementRecord is the mutable assignment record for one stripe (one object chunk or one manifest DAG node). It holds the shard_id, assignees, duplicated erasure parameters, shard hashes and proof-of-retrievability roots (PoR, §5.6), the stored-byte length before erasure padding (ciphertext_size) and a compare-and-swap (CAS) version, which a writer must match to replace the record. Relays replicate it separately from the manifest, so repair needs no manifest decryption.
  • The Volume DEK is the per-volume AES-256 data-encryption key. Depending on the access class (§9.1), the client wraps it to the owner’s wallet-derived key, to the CBSS committee, or both.
  • CBSS (the Cowboy Secret Service) is the threshold committee that can unwrap a committee-wrapped Volume DEK (using IBE) for an authorized Runner (§9.2).
  • The Dispatcher is the CIP-2 Job Dispatcher system actor (0x02). It runs inside block processing, selects Runners (VRF plus committee filtering), issues and revokes CapTokens, and manages the job lifecycle. For CIP-9 it also applies the storage-capability filter (§5.1.1), scopes CapTokens to mount grants (§7.7), and picks the private-volume delivery path (§9.2), constructing and authorizing the CBSS SealRequest for a committee seal. It never handles a plaintext DEK.

3.1 Canonical manifest DAG

The manifest is a content-chunked Merkle DAG ordered by path. Every read verifies against the on-chain root (StorageCommitment.manifest_root, §11.1). For a public volume any party can recompute the root from the entries with the algorithm below. For a private volume each node’s locator is the hash of its encrypted envelope, so a reader verifies each fetched envelope against its locator. All integers below are big-endian. string(s) and bytes(b) are a u64 length followed by the raw bytes. opt(x) is u8 0 for absent, or u8 1 followed by x. The constants are:
  1. The manifest is the set of ManifestEntry values (File, Symlink, Directory), sorted by path in UTF-8 byte order, one entry per path. An empty manifest has the all-zero root and no nodes.
  2. Each entry is encoded as a leaf pair, and each child pointer as a child pair:
  3. A pair hits the boundary when h = BLAKE3(domain ‖ pair) gives (u16_be(h[0], h[1]) & 0x03FF) == 0. The test hashes each pair on its own; there is no rolling hash.
  4. Leaf chunking walks the sorted entries with a running byte count that starts at 14. For each leaf pair:
    • A pair longer than MANIFEST_NODE_MAX_BYTES − 14 fails the build.
    • If the current chunk is non-empty and already holds 2,048 entries, or adding the pair would push the byte count past 262,144, the chunk is cut before the pair and the count resets to 14.
    • The pair is appended and its length added to the count.
    • If the chunk now holds at least 64 entries and the pair hits the boundary under ENTRY_BOUNDARY_DOMAIN, the chunk is cut after the pair and the count resets to 14.
    A final non-empty chunk is kept.
  5. A merge pass then runs over the chunk list from the second chunk on. A chunk with fewer than 64 entries is merged into its predecessor when the merged node encodes to at most 262,144 bytes and holds at most 2,048 entries. After a merge, the chunk that moves into that position is tested against the grown predecessor. A chunk that cannot merge is left as it is. The first chunk is never merged.
  6. Each chunk becomes a node, encoded as:
    A node’s min_path is its first entry’s path or first child’s min_path. Its subtree_count is its entry count for a leaf, or the sum of its children’s counts for an interior node.
  7. While a level holds more than one node, the next level is built from that level’s child pairs with steps 3–5, using CHILD_BOUNDARY_DOMAIN. The single remaining node is the root.
  8. A node’s stored blob is its encoding for a public volume, or for a private volume the envelope "CBME" ‖ u8 version=1 ‖ nonce[12] ‖ AES-256-GCM(dek, nonce, node, aad). Its locator is BLAKE3(stored blob), and manifest_root is the root node’s locator.
MANIFEST_NODE_AAD_DOMAIN is b"cbfs.manifest-node-aad.v1"; its version is not part of the magic. An insert, update or delete usually rewrites only its leaf and the interior nodes above it, and every other node keeps its locator. A byte-bound cut or a merge can shift every later leaf, so one insert or delete can repack every following leaf. The builder takes the prior DAG snapshot and returns the new root plus added_nodes and removed_nodes, so a commit publishes and bills only changed nodes. A private node whose plaintext encoding is unchanged keeps its previous envelope and locator. The set of added and removed nodes and shards is the commit’s delta. The PoR shard-hash records (§5.6) and commit authority, which is checked against the delta (§7.2), depend on this. For private volumes the client encrypts each node with AES-256-GCM using the volume DEK directly as the key. The AAD is the literal domain bytes followed by the 32-byte volume_id, without delimiters or a terminating NUL, so manifest_root commits to ciphertext. Public nodes are plaintext, so an unauthenticated Gateway can verify the root (§5.3.2, §7.6.3).

4. Design overview

4.1 Architecture

Ephemeral Runners read and write over the network to CBFS Relay Nodes, which hold the data.

4.2 Lifecycle

  1. The owner creates a Volume through the Storage Manager with a name, optional size quota and replication parameters. The Storage Manager writes an on-chain Storage Commitment.
  2. The owner attaches volumes in a CIP-2 task definition, each with a name, access mode and optional path prefix.
  3. The Dispatcher (or a delegated Storage Manager) issues each selected Runner a CapToken scoped to the volume, access mode, path prefix, job duration and byte quota.
  4. The Runner writes by splitting the object into bounded chunks, encrypting private chunks, erasure-coding each chunk into K+M shards and sending each stripe to Relay Nodes. Other token holders can fetch the shards once relays store them.
  5. The Runner reads each chunk in order: it fetches K valid shards, reconstructs and verifies the stored bytes, decrypts a private chunk and checks its plaintext hash, then checks the whole-object hash.
  6. At job completion (or periodically), the Runner commits a manifest root over all objects written to the Storage Commitment.
  7. The owner can list, read and delete objects or whole Volumes at any time through the Storage Manager.

4.2.1 Committee composition for storage-attached jobs

A job that mounts a volume uses the normal CIP-2 verification committee, with one prerequisite: every runner considered for a storage-attached job MUST be storage-capable (§5.1.1), meaning storage_support = true with a valid x25519 encryption_pubkey. The Dispatcher filters before sizing the committee. If the verification committee size M exceeds the number of eligible storage-capable runners, dispatch fails with InsufficientStorageCapableRunners { required, available, reasons } (§8.2.1) and does not proceed. CIP-2 has not settled whether storage reads may use a smaller committee (for example M = 1) than verified compute; until it does, a storage-attached job uses the compute committee size. Implementations MUST expose the storage-capable count at dispatch, so an operator can tell when committee size is the limiting factor and storage availability is not.

4.2.2 Dependency on runner-backed continuations

Except for CLI and client-app mounts, a mount comes from a runner-backed continuation (an actor’s runner.agent or runner.http job), which CIP-2 and CIP-5 materialize on chain as a deferred job_submit transaction. If that transaction is dropped, for example by a speculative-execution cache eviction, the mount job never exists and storage never sees it. Appendix D lists every check a storage job must pass.

4.3 Canonical implementation boundary

Both the CBFS data plane and the Cowboy control plane are normative parts of a conforming implementation. The CBFS data plane is responsible for:
  • Object encryption/decryption for PRIVATE volumes and plaintext handling for PUBLIC volumes.
  • Reed-Solomon erasure coding, shard hashing, and shard placement records.
  • Relay Node RPCs (PUT_SHARD, GET_SHARD, placement replication, repair traffic).
  • Manifest storage, reconstruction, Merkle root computation, and verification against the authoritative root.
  • FUSE mount behavior, local cache, sync daemon, and direct object API.
  • Shard repair, garbage collection of orphan shards, and local usage reporting hooks.
The Cowboy control plane is responsible for:
  • StorageCommitment lifecycle, authoritative manifest_root, and volume ownership semantics.
  • CapToken issuance, revocation, and task-scoped attachment semantics.
  • Relay Registry membership, staking, health, and repair coordination triggers.
  • Billing, fee settlement, storage grace periods, and slashing policy.
  • Integration with CIP-2 task submission and the Runner execution flow.

4.4 Relationship to existing CIPs

  • CIP-9 adds volume attachments to the CIP-2 OffchainTask, and the CIP-2 Runner Submission Contract accepts storage manifests alongside result_data.
  • CIP-9 adds a CIP-3 fee dimension, storage billed per byte per epoch including erasure overhead. Relay Nodes meter it outside the VM, and the chain settles it through attestation-based billing.
  • Storage Commitments live in the CIP-4 STORAGE key space under the Storage Manager’s address. Volume data is not in the MPT trie.
  • CIP-10 runtimes mount the volumes, attachment semantics and object API defined here. Container images, cgroups, network policy and GPU passthrough belong to CIP-10.

5. Relay nodes

5.1 Role and responsibilities

Relay Nodes are CBFS storage nodes, a role separate from Runners and Validators. A Relay Node:
  • Stores erasure-coded shards of encrypted Volume data.
  • Stores PlacementRecord entries that map shard IDs to their assigned nodes (see §5.3.1).
  • Serves shards and placement records to authorized requesters (Runners with valid CapTokens, account owners).
  • Keeps its profile in the onchain Relay Registry current (§5.2).
  • Runs autonomous two-phase repair (self-heal and redundancy repair) without Runner involvement (see §5.5).
  • Replicates PlacementRecords to peer nodes via ReplicatePlacement RPCs.
Relay Nodes hold opaque ciphertext shards and never see plaintext. They check CapTokens on shard and placement operations (§7.1.1) and compute nothing on the data beyond repair.

5.1.1 Runner storage capability

The dispatcher assigns a storage-attached job only to a Runner that advertises a storage capability (§4.2.1, §8.1). The capability is two fields in the CIP-2 RunnerCapabilities record:
  • storage_support: bool: the Runner is willing and configured to mount volumes.
  • encryption_pubkey: EncryptionPublicKey: the Runner’s algorithm-tagged X25519 public key, used as the recipient key for sealed DEK delivery (§9.2).
A Runner MUST set storage_support = true and publish a valid encryption_pubkey to be eligible. The dispatcher’s storage filter rejects any runner missing either. Both default to off (storage_support is #[serde(default)] false). Auto-registration derives them from the Runner’s storage configuration (a CBSS/secrets endpoint and an x25519 key). Auto-registration MUST NOT set storage_support = true unless a valid encryption_pubkey is present in the same registration. Capabilities are fixed at registration. A second register_runner for an already-registered identity fails with RunnerAlreadyRegistered. To change storage capability, the operator MUST be able to deregister_runner (which refunds stake) and register again.

5.2 Relay registry

The Relay Registry is a system contract, like the CIP-2 Runner Registry. It manages Relay Node registration, staking, health and capacity.
Clients discover the active relay set from this registry and need no relay configuration. CBSS DEK delivery (§9.2) uses no relay-registry field. A relay’s lifecycle:
  • The Relay Node registers with a RegisterRelay instruction carrying owner_address, node_id, relay_identity_pubkey, the owner and relay authorizations, endpoint, capacity_bytes, an optional region and an optional statusz_url. Registration requires MIN_RELAY_STAKE.
  • UpdateRelay changes the endpoint, capacity, region or status URL and refreshes updated_epoch. There is no separate heartbeat instruction. The auto-drain scan treats every ACTIVE relay as live, so updated_epoch does not trigger a drain (§5.8).
  • The auto-drain policy (§5.8) moves an over-capacity, under-staked or governance-targeted relay out of ACTIVE to DRAINING or INACTIVE, and its shards are moved or flagged for repair (§5.5).
  • A Relay Node may unstake after a cooldown (RELAY_UNSTAKE_DELAY_EPOCHS) if it has no active shard assignments or has transferred them.

5.3 Shard assignment

Writes are index-aligned. Shard index i of every stripe goes to relay_nodes[i], the relay the volume’s StorageCommitment names for that index, and to no other relay. The client does no selection, rotation or fallback:
  1. The volume’s relay_nodes list is fixed at creation and changed only by epoch-sweep rotation (§5.6.1). It holds exactly K+M distinct relays.
  2. Each chunk’s assignment is recorded in its PlacementRecord as relay_nodes[i] for every index i, including an index whose shard the write could not place (§5.4.1). The record is replicated to Relay Nodes (§5.3.1). Readers and repair workers use it to find which nodes hold which shards.
Each stored object produces one immutable ObjectDescriptor, stored in the manifest (encrypted for private volumes), and one mutable PlacementRecord per chunk, replicated to Relay Nodes outside the manifest. The descriptor’s chunks: Vec<ChunkDescriptor> holds one entry per chunk, each naming its own stripe with its own shard_id, shard_hashes and shard_chunk_roots. The chunk ID is object_chunk_shard_id(volume_id, write_id, chunk_index) = BLAKE3("cbfs.object-chunk.v1" ‖ volume_id ‖ write_id ‖ u32_le(chunk_index)), with the zero-based chunk position encoded as a little-endian u32. It survives renames and differs for each chunk position within one write. Every chunk of an object MUST use the volume’s K and M. The per-chunk erasure_k and erasure_m fields duplicate the volume grid, and descriptor validation rejects mixed grids. The descriptor fields are:
Repair workers are Relay Nodes without the DEK, so they cannot read a private manifest. The PlacementRecord’s copies of the erasure params, shard hashes, PoR roots and ciphertext_size let them verify, reconstruct and reassign shards anyway. The CAS version makes reassignment atomic: a worker reads the current version, computes a new assignment, and writes with expected_version = current; if another node wrote first, the write fails and the worker retries. The write_id gives each overwrite of a path distinct shard addresses, so an overwrite never replaces the shards of the previous version. The old shards stay on Relay Nodes until the new manifest commits, which keeps rollback, concurrent writers and reads from old manifests working. After the commit they are orphans, and garbage collection removes them after ORPHAN_SHARD_TTL_SECS. A volume has two relay lists, and the protocol does not keep them in agreement. The volume’s relay_nodes is set at creation. The epoch sweep, the block phase after transactions that processes bounded storage settlement and cleanup work, replaces a departed relay in it (§5.6.1). It is the only record the chain has of which relay is responsible for each shard index. An object’s PlacementRecord.assignments records where its shards are. A write records relay_nodes[i] at every index, and repair Phase 2 (§5.5) can later move an index to another relay without telling the chain. §5.6.1 describes how the two diverge; this is a known defect. relay_nodes is a Vec<RelayEndpoint> on the StorageCommitment. An explicit list at create_volume MUST name exactly K + M relays with pairwise-distinct node_ids. Any other length, or a repeated node_id, fails the whole instruction as E_VOLUME_PARAM_INVALID (receipt code 2002) before any instruction state read; the chain never deduplicates the list. An empty list requests auto-assignment: the chain takes the first K + M relays in the registry’s active-relay list that are Active and have used_bytes < capacity_bytes, and refuses creation with E_VOLUME_RELAY_FLOOR (§12.3.1) when fewer exist. Explicit order is preserved: relay_nodes[i] is the relay the chain holds responsible for shard index i.

5.3.1 Placement persistence

Relay Nodes store PlacementRecords independently of the manifest. Three RPCs move them: The same AuthProvider that gates shard operations gates all three RPCs (§7.1.1). A PlacementRecord MUST carry its volume_id. When a Relay Node accepts a placement update or replication for a shard assigned to itself, it MUST reject a record whose volume conflicts with that shard’s existing local volume binding. At commit, the SDK publishes each PlacementRecord to all its assignees via PutPlacement. For manifest-node records, every assignee that holds its shard MUST accept the record before the SDK submits CommitManifest; the relay at an owed index (§5.4.1) is skipped, and repair hands it the record from a peer. Acceptance is Ok, or ErrConflict for a record the assignee already holds at that version or newer. A refusal or an unreachable assignee aborts the commit before the chain call, and the client’s local commit journal (§12.1.3) keeps the commit marked “not ready” so the next attempt republishes. Object-shard records are best-effort: the SDK logs failures, and placement sync and repair fix them later. At open, the SDK calls GetPlacement for each shard ID in the manifest. If no Relay Node returns a record for a shard (for example, on the first open of a moved volume), the reader asks every relay in the volume’s relay set for every index of that shard. The Relay Node stores PlacementRecords in a versioned envelope: magic: [u8; 2] = [0xCB, 0xD3] || schema_version: u8 || bincode(PlacementRecord). bincode is not self-describing, so a field added without a version change makes old records decode as truncated or garbage, and #[serde(default)] does not help. On a schema_version mismatch the node treats the record as unreadable and re-derives placement from peers. The schema_version MUST be incremented on any change to the PlacementRecord layout. Placement records name node_ids and volume_ids that are meaningful only within one chain instance. A Relay Node MUST record the genesis hash alongside its placement store and, on first boot against a different genesis, wipe the placement store. A relay that keeps placements from an old chain crash-loops on repairs it can never satisfy (§5.5 dead-placement backoff).

5.3.2 GET_MANIFEST RPC

A client can always read the manifest by traversing the DAG (§3.1) from the on-chain manifest_root, fetching each node via GET_SHARD at manifest_node_shard_id(volume_id, locator) = BLAKE3("cbfs.manifest-node.v2" || volume_id || locator). For public volumes, GET_MANIFEST returns the whole manifest in one round trip. It is the recommended path for Gateways, indexers and other public-volume consumers:
The response is the manifest’s entry map; the DAG encoding of §3.1 is never sent. The response carries no root. A root returned by the relay would prove nothing, because a dishonest relay could supply both the root and a manifest that matches it. The caller compares against the root it supplied, and a stale but genuine root of the same volume will verify cleanly and return an outdated manifest. The request carries erasure_k and erasure_m because handle_get_manifest reaches assembly holding only the AuthDecision ({allowed, max_bytes, metadata}), without the StorageCommitment that carries them (§11.1). An implementation that keeps the commitment in hand MAY take k/m from it and ignore the request’s values. This CIP states no preference, because a SHOULD either way would make two conforming relays answer one request differently. The reference implementation passes the request’s values through. A Relay Node MUST range-check erasure_k and erasure_m before any peer fetch, because the caller controls them on an unauthenticated RPC and k + m bounds the per-node fanout of assembly. This bound is per DAG node. The number of nodes an assembly walks depends on the manifest the caller named, and this section specifies no total budget per request (node count, response size or rate limit). A Relay Node that serves this RPC at volume will want one. A relay serves GET_MANIFEST without a CapToken for a volume whose StorageCommitment has visibility PUBLIC and status ACTIVE or GRACE_PERIOD. The decision goes through the same AuthProvider entry point that authorizes an anonymous GET_SHARD and GET_PLACEMENT (§7.6.3), so the three read paths cannot disagree. The decision never comes from stored shard metadata, although PUT_SHARD persists whatever metadata the AuthDecision carries. Metadata written at commit time cannot track later status changes: on disk, a DELETED or GARBAGE_COLLECTING volume looks like an active one until the volume-event poller removes its shards. The commitment path also lags, through a bounded-staleness cache, but by orders of magnitude less. GET_MANIFEST also names no shard, so there is no shard metadata to consult. Shard metadata therefore carries no visibility marker. It carries the shard’s volume binding, commit state, kind and tentative deadline (the time after which an uncommitted shard may be deleted). An anonymous GET_SHARD must match the shard’s stored volume binding against the volume named in the request, so naming a public volume cannot fetch a shard of a different, possibly private, volume. When a token is present, the handler takes the authorized volume from the shard’s recorded binding and uses the caller’s claim only for a shard that carries no binding. A relay does not serve GET_MANIFEST for private volumes. Without the DEK it cannot decrypt a node to learn its child locators (§3.1), and the privacy model forbids it from inspecting the manifest (§9.3). A DEK holder reads a private manifest node by node via GET_SHARD from the on-chain root. A relay also treats the manifest of a DELETED or GARBAGE_COLLECTING volume as absent. The access-mode check admits GET_MANIFEST for the volume owner’s READ_WRITE token under the owner-authority rules (§7.2, §12.2.1), and does not admit it for a grant-scoped token (§7.7.1). The handler performs no visibility check of its own, so an owner’s request for a private volume reaches assembly. It fails when a manifest node decodes as an encrypted envelope, after the placement lookup and peer shard fetches, and returns no private manifest. A Relay Node SHOULD therefore refuse GET_MANIFEST at the authorization layer for a volume whose commitment is not PUBLIC, on the token path as well as the anonymous one. Until relays do this, the access-mode check lets the request through and assembly is what fails. The relay is untrusted, and its entry map is bound to no root. The caller MUST rebuild the manifest from those entries, recompute its DAG root, and reject unless that root byte-matches the manifest_root it supplied. The caller MUST have taken that root from the on-chain StorageCommitment. It MUST NOT cache or serve content from a manifest that fails this check. On mismatch, the caller refetches from another Relay Node and marks the offending node suspect (§15.3 reputation). Clients treat a Relay Node that does not implement GET_MANIFEST as outdated and fall back to node-by-node GET_SHARD traversal. For the entry map, senders MUST encode each map key as its value’s entry.path(), with one entry per path. Receivers MUST bound bincode decoding by the received payload length, using the fixed-width integer encoding of bincode::serialize. Malformed or truncated payloads and DAG-construction errors MUST be rejected. Decoding permits trailing bytes. A receiver MUST construct the canonical lookup map from the entry values’ entry.path() fields and use that same map for both DAG-root verification and subsequent path lookup. A receiver MAY reject mismatched wire keys or duplicate entry paths. If it accepts them, it MUST normalize as the reference SDK does, before computing the root: it iterates the decoded BTreeMap in wire-key order and inserts each value under entry.path(), replacing any earlier value at that path. Only this normalized, root-verified map may be cached or used to serve content. Looking up content through the original wire keys is non-conforming. GET_MANIFEST is an optimization. Placement does not guarantee that any one Relay Node holds the whole manifest, because each DAG node is an erasure-coded object of its own. A relay answers only if it can assemble the public DAG, by reconstructing each node from K shards, from a cache it built at commit time, or both. Assembly is possible only for public volumes, because private nodes are opaque to the relay. A relay that cannot assemble the DAG does not implement the RPC, and clients fall back to node-by-node traversal. Because the caller always recomputes the root, a stale or partial cache fails verification and the caller refetches.

5.4 Erasure coding

RAS spreads each object chunk as a Reed-Solomon stripe across Relay Nodes. Governance can tune the defaults, and each volume can override them: The writing client performs the write path:
  1. Split the plaintext into the volume grid’s object chunks (§14); an empty object still has one chunk. Generate one write_id = CSPRNG(16), compute the aggregate plaintext content_hash, and derive each chunk’s shard_id (§5.3).
  2. Encrypt each private chunk independently with AES-256-GCM, a fresh nonce, and AAD equal to its shard_id (§9). Public chunks stay plaintext with a zero nonce. Record the chunk’s plaintext content_hash, stored-byte ciphertext_hash and pre-padding size_bytes (which includes the 16-byte tag for a private chunk).
  3. Reed-Solomon encode each chunk’s stored bytes into K data shards and M parity shards.
  4. Compute each shard’s BLAKE3 hash and small-chunk PoR Merkle root.
  5. For each stripe, PUT shard i to relay_nodes[i] for every index, with the bounded concurrency of §5.4.1, authenticated by the CapToken (§5.4.1 covers refusals, owed indexes and durability).
  6. Produce one ObjectDescriptor with the ordered ChunkDescriptor vector and the whole object’s plaintext hash and length for the manifest, and one PlacementRecord per chunk for publication at commit (§5.3.1).
The reading client performs the read path:
  1. Resolve the ObjectDescriptor from the verified manifest and validate its path, lengths, uniform chunk grid and position-bound chunk IDs before allocating or transferring data. Get a PlacementRecord for each chunk’s shard_id, from cache or via GetPlacement.
  2. For each chunk in order, request K valid shards from the assignees, verify each against that chunk’s manifest-committed shard hash, and fail over on unavailable or corrupt shards.
  3. Reconstruct the chunk’s stored bytes and strip padding using ChunkDescriptor.size_bytes (duplicated as PlacementRecord.ciphertext_size).
  4. Verify ciphertext_hash, then decrypt a private chunk with its recorded nonce and shard_id AAD.
  5. Verify the chunk’s plaintext content_hash and deliver chunks in order.
  6. After the full read, verify the aggregate plaintext against ObjectDescriptor.content_hash. A failed streaming read may already have written a prefix, and callers must discard it.
Other systems use these schemes: Accounts may choose higher redundancy at volume creation, for example 6/10 for 1.67x overhead or 10/16 for 1.6x.

5.4.1 Write durability

Placing a write’s shards, as this section describes, comes before the write commits. A write is committed, and readable by other mounts, only once the commit carrying its new manifest root lands on chain through the parent-root compare-and-set (§7.3.1), which is also when a default --fsync commit fsync() returns (§12.1.3). Writes are index-aligned. The client sends shard i of each stripe to relay_nodes[i] and to no other relay, authenticated by the CapToken. Uploads run concurrently up to max(1, floor(8 MiB / shard size)) at a time, capped at the stripe width, so an 8 MiB shard uploads one index at a time. Each answer has one of three outcomes:
  • Ok places the shard.
  • ErrCapacity, or a relay that cannot be reached, leaves index i owed. PutShard returns ErrCapacity for the node-wide cap, a per-authorization byte cap and a failed fsync.
  • Any other refusal (authorization, an invalid request, a conflict) fails the write whatever the other indexes did, so the real cause is reported. The client lets the attempts already started finish before it returns the refusal.
A stripe succeeds when at least K of its K + M shards are placed. With fewer than K placed shards the write fails. A stripe with owed indexes is a degraded write. Its PlacementRecord still names relay_nodes[i] at every index, so the relay at an owed index knows what it has to reconstruct. The commit carries the owed indexes as owed_shards, a list of OwedShardRef { shard_id, shard_index }. The chain requires each owed index to name an added shard, at most once, and at most M owed indexes per stripe, so every committed stripe has at least K placed shards. A degraded commit lands like any other commit. The manifest-node stripes that commit publishes (§5.3.1) follow the same rule, and their owed indexes join the same list. An owed index is credited to no relay (§5.6.1), earns no rent and is never challenged by PoR. Repair Phase 1 (§5.5) on the relay at that index reconstructs the shard from K peers. That relay then submits SettleOwedShards, signed by its identity key over the volume’s current relay_set_revision, its node_id and, for each shard, the stripe incarnation (added_at_event_seq) and the index’s chunk_root. The chain checks that the signer is relay_nodes[i], that the index is still owed and that the incarnation and root match the stripe row. It then credits the relay with the shard and admits the index to PoR. A landed write, including one that a successful fsync acknowledged (§12.1.5), therefore rests on at least K shards per stripe. A stripe with o owed indexes survives the loss of M − o more shards until repair re-creates the owed ones. A stripe committed with exactly K shards has no loss margin until then. Repair runs every REPAIR_CHECK_INTERVAL_SECS (300 s) and can re-create an owed shard only while the relay at that index is up and K peers serve their shards. The record must also reach a relay. Repair walks only records that relays hold, and object-shard record publication is best-effort (§5.3.1, §12.2). An owed index is repaired only after at least one relay holds the record. §5.4.2 has to work around three properties of the built system:
  • The client holds every shard, so it computes every shard_hash and PoR chunk_root and commits them in the ObjectDescriptor. PutShard carries only {volume_id, bytes}, so the receiving relay has no hash to check and stores what it is given. The descriptor, which the on-chain manifest root commits to, is the only unforgeable copy of those hashes. Repair, drain and GetPlacement verify against PlacementRecord.shard_hashes, an unsigned duplicate that relays serve.
  • Only a client creates a new assignment. Relays write records (the repair Phase 2 leader CAS-updates one, §5.5 step 5; the drain/rebalance worker builds a successor; ReplicatePlacement accepts a peer copy if the sender already appears in its assignments, §7.1.2), but no relay adds an index or a position it did not already hold. In the drain push (§7.1.2) the draining relay supplies the old_record, so the assignee check proves only self-consistency. Two rules contain the push: §7.1.2’s MUST that the sender’s registry status be Draining, and the destination writing the shard as bound to the volume but uncommitted. Only the volume’s own on-chain add event, from a commit that adds the shard, promotes it to committed.
  • Relays do not check record coverage. validate_placement_record bounds assignments.len() at K + M and each shard_index below K + M, but permits duplicate shard_index and node_id values. A writer’s record names each relay_nodes[i] once, and create-time validation makes those relays distinct (§5.3).

5.4.2 Server-side fan-out (proposed, not implemented)

The write path costs the client K + M shard uploads of ceil(ciphertext_size / K) bytes each, (K + M) / K of the padded ciphertext (1.5× at K=4/M=2, more for small objects). Relays already rebuild shards from peers during repair. The proposal reuses that on the write path: the client uploads once, and relays send the remaining shards to each other. Uploading only the K data shards makes every stripe a degraded write with M owed indexes (§5.4.1), at zero parity until repair re-creates each one from K peers. Any design in this space MUST produce a record that names all K + M assignees at publication time, whatever subset of them the client itself talked to. A design that fits the existing invariants keeps the client as author of the descriptor and the complete record, and moves only the bytes. Per object-chunk stripe:
  1. The client encrypts private chunks and erasure-codes each chunk locally, so it holds every shard’s bytes, hash and chunk-root without uploading them.
  2. It takes the K + M assignees relay_nodes[0..K+M] as today and designates exactly one as the ingest relay. Several ingest relays would either each hold fewer than K shards, too few to reconstruct, or cost the client more uplink than today.
  3. It uploads the K data shards to the ingest relay, together with the complete assignment list, the per-shard hashes and the erasure parameters.
  4. The ingest relay reconstructs the K + M − 1 shards it is not itself keeping, verifies each against the client-supplied shard_hash, and pushes it to the named assignee.
  5. The client publishes the PlacementRecord naming all K + M assignees, and the chunk descriptor with all K + M hashes and chunk-roots, exactly as today.
Step 4 needs the ingest relay’s re-encode to be byte-identical to the client’s, or every reconstructed shard fails its hash check. That is a cross-implementation contract over the padding rule (shard_size = ceil(ciphertext_size / K), zero-filled to K × shard_size), the empty-chunk case, and the Reed-Solomon generator matrix. §5.4 does not pin it, and fan-out cannot be specified until it is pinned. Today the write returns once at least K shards of each stripe are on their relays (§5.4.1). With fan-out that becomes two states. A stripe is accepted when the ingest relay holds its K data shards and owes delivery of the rest. The object may still be unreadable: readers fetch only from the record’s assignments, one address per index, so the ingest relay can serve one shard per stripe, and a readable object needs delivery to at least K assignees in every stripe. A stripe is placed when every named assignee holds its shard. A commit of an accepted but unplaced stripe asserts a placement that does not exist yet. A proposal must say whether that is permitted and what bounds the window.

5.4.2.1 Unsolved problems

A proposal must address each of these obstacles in the built system:
  1. PoR liability does not follow the bytes, even today. A challenge binds responsible_node_id to commitment.relay_nodes[shard_index] (including epoch-sweep replacements), not to the record’s assignee, and §5.6.1 lists the ways the two lists diverge. Every repair Phase-2 reassignment already aims liability at a relay that may not hold the shard. Fan-out widens this gap and must decide which list is authoritative.
  2. An assignee cannot reject a mis-delivered shard, because PutShard carries no hash. It stores the bytes and fails its next challenge. That lands on the fraud row of §5.6, an order of magnitude above the miss row, although the implementation has no fraud branch and folds an invalid proof into the miss counter. Either way, wrong bytes from the ingest relay leave the assignee accountable.
  3. There is no authorization for the push, and one existing MUST forbids it. §7.1.2 states that the sender of a PeerRelayPushShard MUST be Draining, which excludes an Active ingest relay; relaxing that gate would let any registered relay push. Reusing PutShard would hand the ingest relay the client’s CapToken, a bearer credential scoped to the whole volume (including deletes) whose path prefix relays do not enforce. No shard-scoped, single-use credential exists.
  4. An undelivered index self-heals at a cost. Phase 1 pulls K shards from peers for each undelivered index, on top of the fan-out traffic, and a client can trigger this with one upload to a slow ingest relay. PoR then avoids slashing only if self-heal wins the race, which the repair budget and a 300 s cycle bound by timing alone.
  5. Surplus copies on the ingest relay become undeletable and unpaid. It holds K data shards but is the named assignee for at most one. At commit its volume-event poller promotes any local copy matching (shard_id, shard_index) and the volume binding to committed without checking node id. GC never deletes a shard marked committed, so those bytes stay indefinitely. Rent flows only to named assignees.
  6. Today a relay receives at most one shard of each stripe (about 1/K of that chunk’s stored bytes), because each index goes to its own relay. An ingest relay with K distinct indexes can reconstruct the chunk, and with K for every stripe, the whole object, in plaintext for a PUBLIC volume. With (5) it may keep them permanently. The current threat model does not cover this.
  7. effective_size_delta is committed for all K + M shards, so the owner is billed from the commit for placement that may not exist yet. Relays earn rent only with a valid PoR response, so an assignee that never received its shard earns nothing.
  8. Nothing bounds the relay’s retries: how long an ingest relay retries a delivery, how many it may owe at once, or what it does with bytes for a delivery it can never complete. A capacity refusal during fan-out has no fallback, because the ingest relay cannot rewrite the assignment list.
  9. If Phase 2 reassigns an index while its delivery is in flight, the bytes land on a relay the record no longer names. By (5) the commit pins them there, and the new assignee self-heals by (4).
  10. The client has no usable evidence. write_object returns a receipt the client builds itself, and no relay signs anything on the write path, so an ingest relay can acknowledge and discard undetectably. The drain path has the needed shape, a destination-signed absorption receipt settled on chain; fan-out needs one at ingest and at delivery.
  11. With one ingest relay the client sends K shard-units and relays send K + M − 1, so the network moves 2K + M − 1: 9 units at K=4/M=2, against 6 today. §10.1 charges data transfer “at read/write time” and §10.4 routes it to the relay serving the shard; neither covers a relay-to-relay leg. The ingest relay also picks the delivery order, so it decides which assignees can serve reads, and earn fees, during the accepted window.

5.5 Autonomous shard repair

Relay Nodes run repair themselves, with no Runner and no onchain timer. Each Relay Node periodically walks the PlacementRecords it holds and runs two phases. Phase 1 (self-heal) checks local shards. For each PlacementRecord that names this node as a holder:
  1. Verify the local shard bytes against the expected shard_hash from the PlacementRecord.
  2. If the shard is missing or corrupt, fetch K healthy shards from peer Relay Nodes listed in the same PlacementRecord.
  3. Reconstruct the ciphertext using Reed-Solomon decoding and truncate it to ciphertext_size. Truncating to the padded shard length produces garbage.
  4. Re-encode and extract the needed shard. Verify its hash. Store it locally.
Self-heal writes only to local storage and never modifies the PlacementRecord, so any number of nodes can run it at once. Phase 2 (redundancy repair) replaces dead nodes:
  1. For each PlacementRecord, probe all assigned nodes. A node counts as dead if it is unreachable after the configured timeout.
  2. Only the live assigned node with the lowest shard_index drives reassignment for that PlacementRecord, so nodes do not race to repair the same shard.
  3. The leader selects replacement node(s) from the known peer list, excluding already-assigned nodes.
  4. The leader reconstructs the missing shard(s) from K surviving shards, as in Phase 1, and uploads them via PutShard to the replacement node(s).
  5. The leader writes an updated PlacementRecord with the new assignments and an incremented version through a CAS write: the PutPlacement RPC specifies expected_version = current_version. If another node already updated the record, the CAS fails and the leader retries from step 1 on the next cycle.
  6. The leader replicates the updated PlacementRecord to all assigned nodes via ReplicatePlacement.
With K=4, M=2, each stripe tolerates any 2 simultaneous Relay Node failures while four valid, distinct shard indexes remain available. A complete object read needs K valid shards for every chunk stripe, and failures in different stripes do not add up to one object-wide limit. Losing more than M distinct shards of one stripe makes it unrecoverable. A temporary outage makes the object unavailable until enough shards return. The default repair interval is REPAIR_CHECK_INTERVAL_SECS = 300 s. A PlacementRecord whose assigned nodes are all unreachable (0 of K+M live) cannot be repaired, because no shard survives to reconstruct from. The repair worker MUST NOT re-attempt such a record every cycle. It applies exponential backoff (2^n cycles, capped at MAX_REPAIR_BACKOFF). After LOST_PLACEMENT_CYCLES consecutive all-dead observations, it marks the placement LOST and stops until a new placement supersedes it. Without this rule, stale placements (for example, across a re-genesis, §5.3.1) make the worker log “0/N shards available” every cycle without bound. Repair takes everything it needs from PlacementRecords: erasure params, shard hashes, ciphertext_size and assignments. StorageCommitment.degraded_shards is a reporting counter and does not trigger repair. Repair Phase 1 also re-creates owed shards (§5.4.1).

5.6 Proof of retrievability (PoR) challenges

A Relay Node that stays Active can still have lost data, or withhold it (keep it and refuse to serve it). RAS issues random PoR challenges to check that Relay Nodes hold the shards they claim. A challenge runs in three steps:
  1. A periodic onchain timer (CIP-5, every POR_CHALLENGE_INTERVAL blocks) selects a random set of (shard_id, shard_index, byte_offset, byte_length, nonce) challenges targeting active Relay Nodes. Both the direct global sampler and the bounded volume-directory builder MUST exclude GARBAGE_COLLECTING volumes before drawing, even while their native inventory rows remain for cleanup (§13.3). The nonce is fresh per challenge. Consensus reads the stripe’s current row in the chain’s native RAS inventory, keyed by (volume_id, shard_id), and copies the selected index’s chunk_root, the stripe’s shard_size, and the sequence number of the event that published the stripe into the challenge (§11.1). The PoR weight page, the paged table that PoR draws its random targets from, carries w = K+M, so choosing targets at block start needs five reads and no extra commitment read.
  2. The challenged Relay Node must respond within POR_RESPONSE_WINDOW blocks with:
    • The requested byte range of the specified shard. The nonce seeds which range is asked, so a relay cannot precompute or replay a response.
    • A range-inclusion proof: a Merkle path from the fixed-size chunks (e.g. 4 KiB) covering the range to the shard’s chunk-root, the Merkle root over that shard’s chunks.
  3. The onchain verifier checks against the issue-time snapshot_chunk_root carried in the challenge:
    • the returned bytes hash into the chunk leaves of the range-inclusion proof, which resolves to a chunk-root;
    • that chunk-root equals the selected stripe index’s snapshotted chunk-root;
    • if either fails, the response is invalid.
The response window is POR_RESPONSE_WINDOW = 50 blocks, plus a congestion grace of POR_RESPONSE_GRACE = 20 additional blocks. Set deadline_block = saturating_add(issued_at_block, 50) and grace_deadline = saturating_add(deadline_block, 20) using u64 saturation. A still-open challenge accepts an otherwise valid response at block_height <= grace_deadline. Unanswered-challenge settlement is permitted only at block_height > grace_deadline: through issue height + 70 inclusive for responses, with settlement first possible at +71. At u64::MAX saturation no later settlement height exists. The grace is a fixed consensus value that no node-local estimate or per-challenge extension changes. A relay that discarded the shard cannot produce valid chunks against the snapshotted chunk-root, and cannot reuse an old response because the nonce moves the challenged range. The per-shard chunk-root is public and covers erasure-coded ciphertext, so the chain verifies private and public volumes the same way without the DEK. manifest_root cannot anchor PoR, because it commits to ciphertext manifest nodes (§3.1). Only the challenged range is transferred. If the volume commits between issuance and response, the relay still proves against the issue-time snapshot_chunk_root. If a commit removed the challenged shard, or the volume enters GARBAGE_COLLECTING before an unanswered challenge settles, settlement MUST void that challenge, even if its native row remains during bounded GC. A later re-addition does not rewrite the pending snapshot. DELETED remains challengeable, because it is reversible and the relay must still retain the shard. The slashing ladder runs only when governance has set cip31.cbfs.por_slashing_enabled, which defaults to unset. While it is unset, a settled miss emits a ras.por.miss_alarm event and records nothing else, since the consecutive-miss counter is written inside the slashing path. The table gives the behavior with slashing enabled; §5.6.2 lists what must hold first. PoR spot-checks random byte ranges and does not seal sectors or re-prove continuously as Filecoin’s Proof-of-Spacetime does. 1% of every per-epoch storage-fee batch funds the PoR challenge pool at 0x0B (STORAGE_FEE_CHALLENGE_POOL_BPS = 100; split in §10.4 and CIP-31 §4). Submitting IssuePorChallenge { volume_id, shard_id, shard_index, byte_offset, byte_length } requires the caller to escrow RELAY_CHALLENGE_BOND (default 66 CBY, Tier-0 tunable; CIP-31 §7) at 0x0B for the challenge window. The bond settles as follows:
  • After a valid response or unconfirmed miss, the bond is refunded minus POR_CHALLENGE_FEE (default 7 CBY, retained by 0x0B). A rejected invalid response does not resolve the challenge.
  • After a confirmed third consecutive miss with slashing enabled, the bond is refunded in full, and the challenger may receive CHALLENGER_BOUNTY (default 33 CBY) from the PoR challenge pool, bounded by the pool balance and its epoch cap. The RELAY_EVICTION_PENALTY slash is capped by actual stake and distributed under CIP-31 §8. This path does not charge POR_MISS_PENALTY or POR_FRAUD_PENALTY. The slashed-stake 10% share is burned; the storage-fee 10% share goes to the Platform Fee Account (0x18).
  • After a void, the full bond is refunded, with no POR_CHALLENGE_FEE and no bounty.
“Bond less fee” means the settlement rule released = min(challenge.bond, escrow_balance), fee = min(configured_fee, released) on fee-retaining outcomes, and refund = released - fee. Void and confirmed-slash outcomes retain zero fee. The escrow debit is released, and refund + fee == released, so a fee raised above an older challenge’s bond cannot underflow its refund. CIP-31 §7 defines one nano-CBY scale for balances, escrow, pool reserves and bounty caps, with checked conversion from whole-CBY parameters before arithmetic. The current handlers still use raw 66/7/33 base units and must be reconciled before slashing is enabled. The CBY defaults here are not already-scaled on-chain amounts. Before paying a bounty, settlement MUST keep effective_reserve = max(configured_pool_reserve, next_fire_max_cost) in the pool, with next_fire_max_cost computed from the scheduled timer’s resource limits and applicable fee prices, recomputed when either changes. When the pool balance is below that reserve the bounty is zero. The reserve does not protect against future price increases or an unfunded pool. The current reserve defaults to zero (CIP-31 §7). The genesis-seeded PoR timer names fee_payer = STORAGE_MANAGER (0x0A) as a marker that selects native funding; it does not debit the Storage Manager’s account. Before handler execution, the block timer phase reserves max_cost directly from 0x0B/CHALLENGE_POOL_BALANCE_KEY, and unused reservation returns to that pool, including after handler failure. The net pool debit is the consumed timer fee, with the ordinary burn/tip distribution. Challenger bond escrow cannot fund timer gas or bounties. max_cost includes all active timer resource meters, including the Access reservation in the Access-meter release. If the pool is insufficient or a relevant system actor is paused, the timer phase skips the handler, emits por.challenge_paused, and re-arms the same timer for the next interval. A successful fire is also re-armed natively (or continued next block for pending bounded work). These rules override the ordinary removal of a timer after it fires and its deletion for insufficient funds. Both re-arms preserve the timer ID and economics and reset skip/retry counters. Ordinary actors cannot select this funding source, because CIP-5 §4.2 permits a system payer only for its own actor.

5.6.1 Who a challenge holds liable

The chain names the liable relay from commitment.relay_nodes[shard_index] and binds it into the challenge at issue time as responsible_node_id, for both the manual and automatic issuers. An already-issued challenge MUST retain its issue-time relay: response authorization and miss settlement use that recorded identity, even if the epoch sweep later rotates the relay at that index. After settling a volume’s storage epoch against the pre-rotation commitment, the sweep decides relay replacement within its remaining bounded work budget. Work that does not fit waits for a later epoch.
  • Only an ACTIVE volume is eligible. An index needs replacement when its relay profile is missing or its registry status is not Active. An assigned relay that is Active is not replaced, even when full.
  • For each such index, in index order, the sweep takes the first entry in the registry’s active-relay list that exists, is Active, has used_bytes < capacity_bytes, and is not already held by this volume or selected earlier in the pass. The sweep reserves no capacity on the replacement for the shard. Without a replacement, the old assignment stays responsible and gets no new repair grace.
  • The sweep does not apply its decision. It stores it on the commitment as pending_rotation { decided_epoch, replacements }, where replacements lists (shard_index, RelayEndpoint) pairs. The volume’s next FinalizeVolumeCheckpoint applies it in the same transaction, after every update anchored at the old relay set is checkpointed. If none comes, the sweep applies it itself in a later epoch once current_epoch > decided_epoch.
  • The handoff (applying the rotation) replaces each index’s RelayEndpoint, moves the departing node’s per_relay_bytes entry (zero if absent) to the replacement, increments relay_set_revision and records the rotation block for each (volume_id, shard_index). A CommitManifest or SettleOwedShards that names an older relay_set_revision is refused. The handoff moves commitment attribution only. It does not copy shard bytes or update the PlacementRecord, volume sizes, or native shard inventory; repair still has to place the bytes.
Each successful handoff starts POR_RELAY_REPAIR_GRACE_BLOCKS = 7,200 blocks without new PoR challenges for that volume/index. While height < rotated_at.saturating_add(7_200), manual issuance rejects with ras.por_relay_repair_grace before debiting a bond, and automatic issuance skips the target. Both read the current rotation marker, including when automatic sampling used the parent state. The exclusion ends at the exact end height, and a later rotation resets the marker. The grace does not certify that repair completed or cancel an already-issued challenge. relay_nodes[shard_index] is the chain’s only per-index notion of responsibility. A commit carries no per-relay byte deltas. The chain derives them under the commit’s (prev_root, relay_set_revision) fence: an added shard at index i credits relay_nodes[i] with its shard_size unless the commit owes it (§5.4.1), and a removed shard debits the relay now at index i unless that index is still owed. Writes are index-aligned (§5.3), so a write’s PlacementRecord names the same relay at each index. This section is the definitive statement of how the two can still differ (see also §5.4.2.1(1)). The two lists diverge in two ways:
  1. Repair reassigns. Phase 2 selects replacements from the repairing relay’s own peer table, unrelated to the volume’s relay set, so a repaired index can leave relay_nodes entirely. The chain is not told.
  2. A relay leaves. An index naming a node_id the registry never held settles with only a miss alarm. A drained, self-disabled or evicted relay’s profile still loads. Until an eligible volume’s sweep finds a replacement within budget, new challenges bind that relay. During the replacement’s repair grace, issuance excludes that index. After the grace, new challenges bind the new relay. Existing challenges keep their issue-time relay. Rotation does not establish which relay physically holds the shard.
Where they diverge:
  • The named relay is challenged for a shard it does not hold, cannot answer, and builds up consecutive misses toward eviction.
  • The relay that holds the shard is never challenged for that index, so the shard is unaudited.
  • The chain-derived per_relay_bytes credit follows relay_nodes[i] and does not prove which relay holds a shard. Rent payout uses prior-epoch valid PoR and assignment evidence and ignores the byte ledger.
Cowboy-backed CBFS clients MUST require CBFS_MANIFEST_COMMIT_WIRE = 1 in the node’s fingerprinted activation.policy map. A missing or mismatched capability refuses new mutations, signatures, and submissions. A future incompatible manifest wire change MUST advance this policy value, which is not an activation height.

5.6.2 Before slashing is enabled

The slashing ladder in §5.6 (penalties that escalate with consecutive misses) has no effect until cip31.cbfs.por_slashing_enabled is set. These requirements apply:
  • How a relay consents to an assignment is not yet settled. Explicit relay lists require registered identities and use registry endpoints. Registration does not prove that the relay accepted this volume or received its shards, so an owner can name another registered relay and challenge bytes never delivered to it.
  • Relay-authorized responses are implemented and active from genesis. The submitter may be a relayer, but the answer MUST carry the responsible relay’s authorization. The verifier reconstructs the signed payload from chain state (chain_id, challenge_id, shard_id, shard_index, under the response operation domain) and checks the registered relay identity key, owner, node ID and epoch. It rejects an unbound challenge or invalid authorization. Anonymous access to public shards does not bypass this check. The range proof establishes retrievability and the signature establishes relay consent; neither establishes continuous dedicated storage.
  • CIP-31 §7’s unit-scale, timer-reserve, and chain-observable, relay-consented assignment requirements MUST be enforced before the slashing flag can take effect. Merely setting the governance parameter MUST NOT be sufficient in the release implementation. The current flag reader only checks for 1.
  • Per-index liability must correspond to something the chain can observe (§5.6.1).
The last requirement has at least four options:
  1. Make repair placement follow relay_nodes as writes already do, so the chain’s model is true by construction. This gives up repair’s free choice of replacement, which keeps one unreachable relay from stranding a repair.
  2. Add reassignment by an instruction the incoming relay signs, linking repair to the on-chain assignment. The epoch sweep already mutates relay_nodes (§5.6.1) without consent, and the chain already keeps a per-volume relay-assignment snapshot.
  3. Put relay identity in the commit’s shard references. The commit is owner-signed, so the relay must counter-sign, as the drain path’s destination-signed absorption receipt already does. Any such attestation MUST bind volume_id, shard_id, shard_index, chunk_root, the chain id and a freshness window, or one attestation is replayable into another volume.
  4. Drop per-index liability: challenge the volume and let any holder answer. Payout already requires a valid response. This gives up naming a specific relay as having lost specific bytes.

5.7 Relay node incentives

Relay Nodes earn fees from two sources:
  1. The relay share of the per-epoch storage fee paid by the account owner, weighted by valid PoR responses and assignment age (CIP-31 §5).
  2. Reader-authorized transfer fees for complete logical shards, counted per rounded MiB and paid from dedicated funded channels (§10.4.1). A relay’s own byte report does not set them.
The fees are computed as:
used_bytes counts stored, erasure-expanded bytes. The first unsettled epoch is billed at epoch_peak_used_bytes, the largest used_bytes since the last settlement, and each later unsettled epoch at the current used_bytes. Both use the storage fee in the fee snapshot the epoch sweep recorded when it queued the epoch. The rent-floor policy stamped on the volume sets only its minimum escrow, not the rent rate. shards_lost (§5.6) is a counter only. No assignment reads it: auto-assignment and rotation take the first Active relay with free capacity.

5.8 Relay drain governance

Draining a relay moves its shards to other relays and releases it. Three triggers start a drain:
  • The operator initiates a manual drain, posts a drain bond, and gets a window to move the shards.
  • The per-epoch storage-settlement scan auto-drains over-capacity or under-staked relays under the governance policy.
  • Governance forces a drain of a silent or misbehaving relay whose operator will not cooperate.
This section specifies the two governance-controlled paths. Both use CIP-12’s proposal pipeline (SubmitProposal=45 / CastVote=46 / ExecuteProposal=47) and share the Proposal storage schema at GOVERNANCE_SYSTEM_ACTOR=0x09, each with its own submit-proposal opcode. The payload discriminator ProposalPayloadKind (CIP-12 §5.4) includes DrainRelay (opcode 85) and UpdateAutoDrainPolicy (opcode 86) among its variants. The Proposal record carries two payload fields, populated by variant: payload_relay_node_id: Option<[u8; 32]> (DrainRelay) and payload_auto_drain_policy: Option<AutoDrainPolicyConfig> (UpdateAutoDrainPolicy). Both are None for every other variant. SubmitDrainRelayProposal (opcode 85) has the wire format { description_hash: [u8; 32], voting_blocks: u64, node_id: [u8; 32] }. It requires MIN_VOTING_BLOCKS ≤ voting_blocks ≤ MAX_VOTING_BLOCKS (else InvalidData) and standard base gas, and any account may submit it. The target node_id is validated only at execution, where an unknown node_id yields InvalidData. Submit writes an Active Proposal (payload_kind = DrainRelay, payload_relay_node_id = Some(node_id), deadline = block + voting_blocks) and emits governance.proposal.submitted. ExecuteProposal after a passing tally reads the relay’s RelayNodeProfile from RELAY_REGISTRY=0x0B and writes the node into its auto_drain_governance_scan slot; the next epoch’s storage-settlement loop performs the drain. A governance-initiated drain has no operator drain bond. It reuses the drain windowing, shard-absorption receipts, and stake-reservation and slashing logic, with the relay’s existing stake as the slashable collateral. SubmitAutoDrainPolicyProposal (opcode 86) has the wire format { description_hash, voting_blocks, enabled, high_water_mark_bps, over_capacity_epochs, stale_heartbeat_epochs, liveness_window_epochs, min_active_stake, drain_bond_base, drain_bond_per_gib, drain_window_epochs }. It writes the network-wide AutoDrainPolicyConfig: Submit and ExecuteProposal both validate the ranges above; enabled and the stake and bond fields may be zero. Execution writes the policy to 0x0B:AUTO_DRAIN_POLICY_KEY, and later epochs apply it in the auto-drain scan. Drain-relay and auto-drain-policy proposals are Tier-0 governance actions under CIP-12 §5.1, with no proposer-selected tier field. Opcodes 85/86 are pinned in the CIP-13 master opcode table. A draining relay’s outstanding CBFS rent obligations follow the moving shards to the absorbing relay; CIP-31 owns the per-shard rent accrual the drain window is sized against.

6. Storage addressing

6.1 Path-based namespace

Every object in RAS is addressed by a three-part key:
  • account_address is the owning account’s 20-byte protocol Address, hex-encoded. It is the same Address type used protocol-wide and carried by VolumeAttachment.owner (§8.1).
  • volume_name is a UTF-8 string of at most 64 bytes, restricted to [a-zA-Z0-9_\-.].
  • object_path is a UTF-8 string of at most 512 bytes that uses / as a logical separator. It has no leading or trailing /.
Examples:

6.2 Content integrity

The manifest commits a whole-object plaintext hash and separate hashes for every ordered object chunk j:
ChunkDescriptor.shard_hashes[i] is the whole-shard integrity hash that repair uses. shard_chunk_roots[i] is the Merkle root over shard i’s small fixed-size PoR chunks (e.g. 4 KiB). These small PoR chunks are distinct from the bounded object chunks. For public object chunks the stored bytes equal the plaintext, so the two per-chunk content hashes are equal. There is no single object-level ciphertext hash. A successful commit_manifest delta writes the chunk root into the shard index’s entry in the stripe’s row of the chain’s native RAS inventory, and PoR snapshots that row when it issues a challenge (§5.6). The corresponding ChunkDescriptor carries both shard commitments, and its PlacementRecord duplicates them. Every added and removed ShardRef in a commit delta MUST carry the shard’s real, nonzero chunk_root and shard_size. For a removal, the Storage Manager MUST require the existing stripe row and reject a zero or mismatched chunk_root at any index, or a mismatched shard_size. A removal cannot identify a shard by (shard_id, shard_index) alone. Validation MUST carry the loaded rows into mutation rather than reading them again. The worst-case Access reservation for inventory work is 3 reads and 4 writes (3R+4W) per removed stripe and 1 read and 2 writes (1R+2W) per added stripe, before rent-assignment work (§11.1; CIP-43 §5.5). Content hashing uses BLAKE3 throughout. The integrity checks run off chain, so onchain hash compatibility does not matter. The onchain manifest root (§3.1) lets any party verify each stage of a read. Shard hashes verify shards, per-chunk stored-byte hashes verify reconstruction, per-chunk plaintext hashes verify decryption, and the aggregate plaintext hash verifies the whole object. A path from a leaf node to the root is O(log N) in the number of objects. For a public volume this path proves object membership. For a private volume the nodes are ciphertext, so the chain cannot verify membership from manifest_root alone, and PoR instead checks shard-level retrievability against the public per-shard chunk roots in the native RAS inventory (§5.6).

6.3 Addressing for deletion

The account owner deletes objects by their full path:
delete_object marks the object’s shards for removal on the assigned Relay Nodes and updates the onchain manifest, and the relays garbage-collect the data. delete_volume is a soft delete: it sets the status to DELETED and starts the GC grace timer (§13.2), and the owner may call undelete_volume() until the window ends. The StorageCommitment and rent persist through the grace window. After it, the volume is garbage-collected and fees cease. Only the account owner, or an Actor acting for the account, may delete. Runners never receive delete permission, even in read-write mode.

7. Access control

7.1 Capability tokens

Access to a Volume goes through Capability Tokens (CapTokens), modeled on Storj’s Macaroons and UCANs:
A CapToken gets its authority from Storage Manager state. The system instruction that records volume attachments mints the token (§8.2) and writes the authoritative copy into Storage Manager actor state. Relays (through the AuthProvider) and the validator (through validate_token) honor a presented token only if it matches that stored copy byte for byte, so forging a token requires compromising consensus. The signature field is reserved for a future client-verifiable token form (§17.3). Ending a job (completion, cancellation or timeout failure) does not immediately revoke its CapTokens or release its write-prefix reservations. Issuance stores a bounded per-job token-hash index (job_cap_tokens:, at most MAX_ATTACHMENTS_PER_JOB = 64 hashes, each 32 bytes). The CIP-2 admission-owned retained-record reap, subject to its pause and damaged-state qualifications, deletes each indexed ras:token:{hash} row and then the index. A token stops being authoritative when the reap deletes it (including after cancellation during private key delivery) or when valid_until passes. Public M=1 retry deletes the outgoing tokens before replacing the runner handle. Native route-session revocation has its own immediate lifecycle, and the later reap’s deletion is idempotent. Until then, the runner of a terminated job may still use the token’s scope, and overlapping write prefixes stay reserved until a later issuance finds them expired and releases them. Immediate revocation and prefix release at job termination are not yet specified; adding them requires re-deriving the BlockStart timeout budget.

7.1.1 Auth enforcement on placement RPCs

The same AuthProvider that gates shard operations also gates GetPlacement and PutPlacement. A request without a valid token, or with an expired or revoked token, MUST be rejected before the Relay Node reads or writes any placement data. A leaked PlacementRecord would tell an attacker which relays to target with denial of service. For public volumes, placement reads follow the open-access rule for shard reads (§7.6.3).

7.1.2 Node-to-node operations are not CapToken-gated

Relays send each other ReplicatePlacement, SyncPeers, the drain operations PeerRelayPushShard and PeerPlacementUpdate, and the audit read AuditGetShard. The calling relay’s identity authorizes these operations, so they are not gated by AuthProvider and MUST NOT be. They carry an empty auth_token. Each carries a relay assertion instead. The calling relay signs a domain-tagged digest with the identity key registered in its on-chain RelayNodeProfile, and the receiver verifies it against relay_identity_pubkey from the relay registry. The receiver MUST check all of:
  1. the signing timestamp is within its accepted clock-skew window;
  2. the sender resolves to a registered relay whose on-chain status is Active or Draining. Disabled and Inactive MUST be refused, since revocation can only come from the registry and never from the signature;
  3. the signature verifies against that profile’s relay_identity_pubkey;
  4. the digest covers the payload being acted on. A signature that binds only the sender can be replayed with attacker-chosen content. ReplicatePlacement and SyncPeers are both writes, so such a replay would corrupt placement records.
Each operation adds its own rule on top of those four:
  • ReplicatePlacement: the sender MUST already appear in the record’s assignments. Without this rule any registered relay could write any record for any volume.
  • SyncPeers: the advertised sender.node_id MUST equal the signing identity, so a registered relay cannot advertise another node’s id at an address it controls.
  • PeerRelayPushShard / PeerPlacementUpdate: the sender MUST additionally be Draining, since only a draining relay sends a drain push.
A relay that has no view of the chain (standalone or development deployments) has nothing to verify against and MUST refuse these operations rather than accept them unverified. A receiver MUST NOT distinguish these failures on the wire beyond a single authorization error. The caller learns only that it was refused.

7.2 Access modes (CapToken scopes)

The following modes apply to CapToken-gated access on private volumes: WRITE_ONLY is the default for most Runner jobs: the Runner produces output and cannot inspect existing data. On a private volume the Relay Node refuses a WRITE_ONLY token for GET_SHARD, GET_PLACEMENT and listings. On a public volume anyone can read without a token (§7.6), so a WRITE_ONLY mount can still refresh the manifest and learn the object namespace (§12.1.3); a grant that must hide a volume’s contents needs a private volume. READ_ONLY suits a Runner that only consumes data, such as loading model weights for inference. READ_WRITE suits a Runner that reads and updates the same volume, such as an agent updating its memory. Writing shards (PUT_SHARD) and committing the manifest need separate authority, because a commit replaces the volume’s single manifest_root. §7.3.1 specifies commit authority.

7.3 Concurrent CapTokens

Multiple CapTokens may be active on the same volume at once. In the agent swarm pattern, a coordinator Runner holds a READ_WRITE token while sub-agent Runners hold WRITE_ONLY tokens scoped to disjoint path prefixes. These rules apply:
  • If two CapTokens grant WRITE access, their path_prefix values MUST NOT overlap. The Dispatcher enforces this at token issuance.
    • The Dispatcher canonicalizes a prefix to end in /, so agent-1 is stored as agent-1/. Prefix A overlaps prefix B if, after canonicalization, either is a prefix of the other: agent-1/ and agent-1/sub/ overlap, and agent-1/ and agent-10/ do not.
    • An empty prefix ("") means full volume access. If any WRITE token has an empty prefix, no other WRITE CapToken may be active on the volume at the same time. READ_ONLY tokens are not counted.
  • Reads never conflict. READ_ONLY tokens may coexist with any other tokens, and a READ_WRITE token may read paths that other tokens are writing.
  • Concurrent writes to different paths are independent. There is no global write ordering across CapTokens.
  • There is no token revocation list. A token stops being authoritative when the job’s reap deletes its stored copy, when valid_until passes, or when an ownership transfer advances last_owner_transfer_block past its valid_from (§7.1, §13.4).

7.3.1 Prefix enforcement boundaries

Prefix enforcement happens at three layers. Layer 1 is the onchain check at issuance. Before issuing a WRITE CapToken, the Dispatcher checks the requested path_prefix against all active WRITE CapTokens on the volume and rejects an overlap. Layer 2 is commit authority, also onchain. A sub-agent’s commit authority limits what it can damage:
  • Parent-root CAS applies to all volumes. Every commit names the prev_root it extends, and the chain rejects the commit unless prev_root equals the volume’s current manifest_root (enforced today). A stale or racing committer must re-apply its changes on top of the current root and retry, so no commit overwrites an intervening one.
  • CIP-33 §2.1.2 adds a commit guard for a Public volume registered as a Store serving artifact. Direct CommitManifest and staged CommitManifestFinalize MUST reject while that volume’s CIP-33 pin count is nonzero. A staged commit prepared before registration still checks the pin at finalization. CIP-33 defines the references and release rule, including the future Forked spawn case. CommitManifestStage does not change the canonical root.
  • Prefix-scoped writers stage, and full-volume holders publish. A commit carries the new root and its added and removed shards but not the changed paths, so the chain cannot tell whether it stays inside a prefix. The chain therefore rejects CommitManifest from a writer scoped to a non-empty path_prefix, in any access mode and on either visibility. Such a CapToken may only stage its delta (CommitManifestStage, attributed to staged_by), and the stage carries no native-route finalize authority. A prefix-scoped mount grant confers no authority to commit. Only a full-volume holder (the owner, an owner delegation, or an empty-prefix grant or CapToken) issues CommitManifestFinalize or a direct CommitManifest. Before finalizing, the publisher confirms the staged delta touches only paths under the stager’s prefix; on a PRIVATE volume that review needs the DEK. The publisher is usually the coordinator that issued the sub-agent’s CapToken.
Layer 3 is the write at the Relay Nodes. PUT_SHARD requests are keyed by opaque shard_id values (BLAKE3("cbfs.object-chunk.v1" ‖ volume_id ‖ write_id ‖ u32_le(chunk_index))). A Relay Node can verify that the CapToken is valid (signature, expiry, volume ID, write permission) but cannot see the object path, so it cannot enforce prefixes (§16.3). A Runner scoped to agent-1/ can therefore upload shards for agent-2/poison.dat. Those shards are referenced only if the full-volume publisher finalizes a stage that names them. The token’s max_bytes quota bounds the waste, and Relay Nodes garbage-collect orphan shards after ORPHAN_SHARD_TTL_SECS (§14).

7.5 Read consistency

All reads are READ_COMMITTED. An object becomes visible only after the writing client has committed onchain a manifest that includes the object’s ObjectDescriptor. When a client fetches a manifest from Relay Nodes, it MUST:
  1. Fetch the manifest DAG nodes from Relay Nodes, starting at the root locator, and reconstruct them.
  2. Recompute the root locator of the reconstructed DAG (§3.1).
  3. Compare the computed root to the onchain manifest_root in the volume’s StorageCommitment.
  4. Reject the manifest if the roots differ, since it is stale, partially published or corrupted.
The onchain manifest_root is the single source of truth. A manifest that a writer published but has not committed onchain fails step 3, so no reader sees uncommitted data. §12.1.5 specifies what a mount guarantees on top of this: read-your-writes, monotonic reads, the cross-mount visibility bound, and how failed and concurrent commits resolve. A READ_UNCOMMITTED mode, where an object is visible once relays store its shards, is deferred to a future CIP. A reader cannot predict an uncommitted shard_id without the writer’s random write_id (§5.3), and prefix confinement is enforced only at commit (§7.3.1).

7.6 Public volumes (PUBLIC)

7.6.1 Overview

Any party can read a volume with visibility = PUBLIC without a CapToken. Public volumes serve DNS-addressable actors (CIP-14), static web assets (CIP-15), public datasets and shared artifacts. The owner sets visibility at creation (§12.3), and it never changes afterward.

7.6.2 Properties

  • Relay Nodes store objects in PUBLIC volumes unencrypted. The volume has no DEK, and the wrapped_dek field in the StorageCommitment is empty.
  • Any party can fetch shards from Relay Nodes without presenting a CapToken. Relay Nodes serve GET_SHARD for a public volume’s shards without authenticating the caller, under two conditions: the volume must be ACTIVE or GRACE_PERIOD (§13.5), and the shard’s recorded volume must be the one the request names (§7.6.3).
  • Writes still need a CapToken. Before issuing a WRITE_ONLY or READ_WRITE attachment token or reserving a write prefix, the Dispatcher MUST verify that the job submitter owns the volume or has an owner-issued mount grant permitting that mode and prefix. This applies equally to PRIVATE volumes. A supplied volume-owner address only locates the volume and proves no authority.
  • The client still computes and stores content_hash (BLAKE3) for every object. Readers MUST verify the content hash after shard reconstruction to detect corruption or tampering.
  • Erasure coding and storage fees are the same as for private volumes; the erasure input is plaintext.
  • list_objects needs no CapToken. Anyone can read the plaintext manifest from the on-chain root (§3.1) or with GET_MANIFEST (§5.3.2).

7.6.3 Relay node behavior

A Relay Node decides whether a shard is publicly readable from the volume’s StorageCommitment: visibility must be PUBLIC and status ACTIVE or GRACE_PERIOD (§13.5). Anonymous GET_SHARD, GET_PLACEMENT and GET_MANIFEST (§5.3.2) all go through one AuthProvider entry point that applies this predicate. Relays do not take visibility from shard metadata, because a volume that has since become DELETED or GARBAGE_COLLECTING looks identical on disk (§5.3.2). The commitment lookup reads a bounded-staleness cache. On an anonymous GET_SHARD, the AuthProvider also requires the shard’s recorded volume (from the stored shard_metadata) to equal the volume named in the request. Otherwise a caller could name a public volume and receive a shard of a private one. LIST_SHARDS is not exposed; clients list objects from the manifest. PUT_SHARD requests require a valid CapToken with write access. The relay persists the shard’s volume binding, taken from the authenticated payload’s volume_id (the value the CapToken is checked against), together with any metadata the AuthDecision carries and the shard’s commit state, kind and tentative deadline. The relay writes the binding; the AuthProvider’s decision is what lets it trust the payload’s claim. For private volumes the Relay Node refuses unauthenticated GET_SHARD requests. A Relay Node MUST perform the same work (metadata read, placement lookup, authorization round-trip) whatever the outcome, and collapse every refusal to one indistinguishable response. Otherwise latency reveals whether a refused shard exists. A payer may choose the funded-channel operation (§10.4.1) for public or private data. Its payment requirements apply only to that operation and leave GET_SHARD, manifest and repair behavior unchanged. A selected paid strategy MUST NOT fall back to an unpaid operation on error. A saved channel ACK handoff delivers no new data. It checks its durable request/voucher binding and payer signature, and it does not check a current data token, relay registration or volume reauthorization. The same separation applies to recovery after a token expires or serving eligibility changes. A Gateway (CIP-15) is an ordinary unauthenticated public-volume reader. There is no Gateway storage role, key or capability.

7.6.4 Shard ID opacity

Public volumes use the same opaque shard IDs as private ones (§16.3). Their manifest is plaintext, so anyone can read the object paths.

7.6.5 Content-type metadata

A public volume may store a content-type map at _meta/content_types.json:
Consumers such as CIP-15 Gateways use it to set Content-Type. Without a map, they infer types from file extensions.

7.6.6 Cache headers

A public volume may store a cache configuration at _meta/cache_config.json:
Consumers use it to set Cache-Control. An object’s ETag is the quoted string "b3_" followed by the lowercase hex content_hash (CIP-15 §8.8), for If-None-Match requests. A change in the onchain manifest_root signals that the volume’s contents changed.

7.7 Cross-owner volume access (mount allowlist)

By default only a volume’s owner can attach or open it. The owner lets other principals in through an on-chain mount allowlist on the StorageCommitment (§11.1), changed only by an owner-signed UpdateMountAllowlist transaction (§12.3). The allowlist is the only cross-owner authority, for both the Dispatcher-issued CapToken (§8) and the grantee’s self-issued owner-auth token (§7.7.1). A DelegationCert (§12.2.1) never carries volume authority of its own. Each grant is a MountGrant:
A grant without grantee_signing_key only lets the grantee attach the volume from a job it submits (§8). A grant with grantee_signing_key also authorizes direct client access under §7.7.1 by exactly that key. On a PRIVATE volume, direct reads also require grantee_encryption_key and grantee_wrapped_dek. On a PUBLIC volume these fields are absent. At dispatch, when the task submitter is not the volume owner, the Dispatcher resolves the submitter to a principal and looks up a matching unexpired grant. It scopes the issued CapToken to the grant’s max_access_mode and canonical_path_prefix, intersected with the attachment request. Delete is never grantable (§6.3). The Dispatcher skips grants whose valid_until has passed. Removing a grant deletes the grant row, including its grantee_wrapped_dek. The Dispatcher stops issuing that principal CapTokens, and the direct-client path, which resolves the grant on every request, stops serving it. A party that already decrypted its package, completed a CBSS seal or holds the owner’s wrapped copy of the DEK keeps the unchanged DEK. It can still read any ciphertext or plaintext it kept. Because path_tag = HMAC(dek, seq, path) and every commit publishes its path tags on chain, it can also test candidate paths against the whole commit history offline. Only rotating the DEK and re-encrypting the volume (§9.4) withdraws the key. An implementation MUST NOT present grant removal as key revocation. A grantee may be an Account or a deployed Actor. The mount check derives the submitter’s type from chain state (Actor(addr) if it carries an ActorManifest, else Account(addr)), so a manifest-less actor granted as Actor(addr) would fail an exact-variant match. UpdateMountAllowlist therefore keys grants by bare address, and the mount check matches the canonical address regardless of variant. An implementation MAY also require deployed actors to carry a manifest; the address-keyed match remains the normative rule. A grant MUST NOT be silently dropped on a variant mismatch. The MountPrincipal wire encoding is unchanged. Direct-client grants have these admission rules. UpdateMountAllowlist MUST reject a grant whose grantee_signing_key is present and whose canonical_path_prefix is non-empty, for every access mode. Shard addressing is opaque (§7.3.1, Layer 3) and a private volume has one DEK (§9.1), so a prefix cannot bound a direct client, and the protocol MUST NOT present a scope it cannot enforce. grantee_encryption_key and grantee_wrapped_dek MUST be both present or both absent, and MUST be absent on a PUBLIC volume. The package MUST be a canonical tagged v1 package within MAX_GRANT_PACKAGE_BYTES, with the selected suite’s exact field lengths (§9.2.1, §14), and MUST name the commitment’s current dek_version. A WRITE_ONLY or READ_WRITE direct grant lets the grantee upload shards and placements. Chain commit authority comes from the grant row itself, with the grantee’s account as the transaction sender (§7.7.1). A READ_WRITE grantee may stage, commit directly and finalize, and a WRITE_ONLY grantee may only stage, when the grant is whole-volume, not tee_required and unexpired, and on a PRIVATE volume carries a grantee_wrapped_dek. Otherwise uploaded data becomes visible only when a full-volume holder commits a manifest that references it. On a PRIVATE volume the grantee holds the whole DEK, so a write grant limits only what relays allow. A grantee holding the package can still read the volume’s contents. TransferVolume clears mount_allowed_actors atomically with the owner change (§13.2), and a recreated volume starts with an empty allowlist, so old grants never revive.

7.7.1 Grant-scoped owner tokens

A grantee exercises a direct-client grant with its own delegation cert (§12.2.1). It self-issues an OwnerCapTokenV1 exactly as an owner does, except that owner_address names the volume’s owner instead of the grantee’s wallet. Relay validation (validate_owner_cap_token) accepts such a token iff every ordinary condition holds (cert validity; the cert registered on chain and unrevoked for the grantee’s wallet; token signature by the cert’s key; volume, TTL and status bounds) and a grant on the commitment’s allowlist matches on every one of:
  • grantee equals the cert’s wallet_address by bare address;
  • grantee_signing_key is present and equals the cert’s cbfs_public_key;
  • token.access_mode is permitted by max_access_mode as an operation set: READ_ONLY permits only READ_ONLY tokens, WRITE_ONLY only WRITE_ONLY tokens, READ_WRITE any of the three;
  • token.path_prefix is "", because a grant-scoped token is always whole-volume;
  • valid_until, if present, has not passed at the relay’s block-height view; a relay without a height view MUST refuse an expiring grant;
  • tee_required is false. A client holder is never a TEE, so a tee_required grant does not authorize an owner-auth token.
The owner rule (cert.wallet_address == commitment.owner) and the grant rule are alternatives, and the relay refuses a token that satisfies neither. An owner-admitted token keeps every owner operation (§7.2, §12.2.1). A grant-admitted token gets only these operation sets:
  • A READ_ONLY token permits the metadata needed to open the volume, committed-manifest discovery, shard and placement reads, and read-only proofs. Each is permitted only when the relay resolved the target volume from authoritative stored state, so it refuses grantees a shard that is present but not recorded as belonging to a volume, and a PROVE_SHARD not tied to a volume; the owner keeps its exceptions. The relay MUST deny a READ_ONLY token shard, placement and replication writes, staging, finalization, manifest commit, deletion and every administrative operation.
  • A WRITE_ONLY token permits shard and placement writes. It denies reads, deletion, replication and every commit or administrative operation.
  • READ_WRITE is the union of the two.
A grant-scoped owner token carries no chain authority. The chain admits manifest transactions from a grantee only through the grant row, with the grantee’s account as the sender: a whole-volume, non-tee_required, unexpired grant (with a grantee_wrapped_dek on a PRIVATE volume) lets a READ_WRITE grantee stage, commit directly and finalize, and a WRITE_ONLY grantee stage. Allowlist changes, deposits, transfer and delete require the owner’s authorization, which a grantee’s delegation cannot produce. Grant revocation (UpdateMountAllowlist remove), delegation revocation, expiry, and ownership transfer all end access within the relay’s chain-state refresh bound, which MUST be at most 60 seconds. When commitment, delegation or revocation state is stale or unavailable, the relay refuses access. Issued tokens are bearer credentials, as owner tokens are (§12.2.1), and the short owner-token TTLs bound a copied token. Request signing for both is future work (§17). GET /ras/volumes/{name}/open resolves names in the caller’s namespace. A grantee names the owner: GET /ras/volumes/{name}/open?owner={address}. The node serves the request iff the caller is the owner or its authenticated delegated key matches a grant under the rules above. The response carries the caller’s grantee_wrapped_dek in place of owner_wrapped_dek, never the owner wrap, and is otherwise identical. GET /ras/volumes lists only volumes the caller owns.

8. Volume attachment

8.1 Attachment at job dispatch

A CIP-2 task definition lists volume attachments. VolumeAttachment is the normative wire structure; the same 10 fields, in order, back both the dispatch request and OffchainTask.volume_attachments (§12.4): The RAS MountGrant enum uses a different tag order (§9.2.1). Implementations MUST map between the two by variant name, never by ordinal.
Fields 8–10 are filled by the Dispatcher at runtime; a client-submitted spec carries relay_endpoints empty. When the Dispatcher builds the runner-facing OffchainTask.volume_attachments, it resolves relays and confirms the erasure parameters against the volume’s PlacementRecord (§5.3). Per CIP-11 §12.6, job_spec_hash covers the attachment exactly as the client submitted it. Dispatcher-side resolution MUST NOT rewrite the stored spec or move the hash, so changes to the relay set between submission and dispatch never invalidates an assignment. A job may attach several volumes, each authorized independently. A dispatched grant is a mount-allowlist grant (§7.7) whose grantee is the selected Runner’s account and keys (§9.2). A matching dispatched grant issues no CapToken, and other attachments each get a CapToken (§9.2). Only storage-capable runners receive such a task (§4.2.1, §5.1.1).

8.1.1 Per-job attachment admission caps

The task codec’s validate() runs three stateless checks on volume_attachments, using the task bytes alone, before any volume, owner or grant lookup:
  • attachments.len() <= 64.
  • Each volume_name MUST satisfy the §6.1 volume-name rule.
  • The canonical_path_prefix MUST already be in canonical form: "" for whole-volume access, or a non-empty prefix ending in a single / with no leading /. validate() rejects a non-canonical prefix and does not normalize it.

8.2 Attachment process

Attachment moves no data to the Runner. It has three steps:
  1. The Dispatcher authorizes the attachment. It either selects a matching dispatched grant (§9.2), issuing no CapToken on that path, or issues a scoped CapToken for the attachment.
  2. For private volumes only, the Runner obtains the volume key. On the dispatched-grant path, the Runner opens the grant’s package locally with no CBSS involvement. Otherwise the volume must carry a committee wrap (a copy of the DEK encrypted to the CBSS committee, §9.1), and the Runner completes the CBSS SealRequest that the Dispatcher constructs and authorizes (§9.2). An owner-key-only volume is runner-mountable only through a matching grant. The Dispatcher never handles a plaintext DEK.
  3. The Runner fetches the manifest DAG from the on-chain root locator (§3.1) by node-addressed GET_SHARD, recomputes the root, and verifies that it matches the onchain manifest_root (§7.5). On a public volume it MAY use GET_MANIFEST (§5.3.2) instead and verify the same way. The Runner trusts the manifest only after verification.
During execution the Runner reads objects on demand, with no bulk prefetch. Expected latency:

8.3 Detachment

When a Runner job completes or times out:
  1. The Runner commits the final storage manifest onchain if it wrote any objects.
  2. The CapToken becomes invalid once valid_until passes.
  3. Shards written during the job persist on Relay Nodes, so the Runner may terminate immediately after commit.

8.2.1 Dispatch eligibility diagnostics

A healthy runner can still be excluded from a storage job by reputation, job type, storage_support or DNS constraints (Appendix D). The Dispatcher MUST emit a structured, per-gate exclusion summary when selecting for a storage-attached job:
When selected is below the required committee size, the dispatch error MUST be the structured InsufficientStorageCapableRunners { required, available, reasons: [...] }. A runner excluded for reputation (for example, freshly bootstrapped, or with a reputation that decayed below the threshold when it was last recomputed) MUST appear as its own exclusion in the summary.

9. Encryption and privacy

9.1 Encryption at rest

The client that writes to a private Volume encrypts the data before erasure coding and distribution to Relay Nodes. That client is a Runner during a job, or the owner through the CLI. Relay Nodes never see plaintext. Public volumes (visibility = PUBLIC) skip encryption. There is no DEK and no wrapping key, and readers verify integrity with BLAKE3 content hashes (§7.6). Private volumes use envelope encryption with a random Data Encryption Key (DEK) per volume:
  1. The client generates the volume DEK, a random 256-bit key, at volume creation. It is never derived from the account’s signing key.
    The commitment carries a dek_version: u32 alongside the wraps. It is an input to the CBSS volume-DEK identity volume_dek_identity(chain_identity, owner, volume_id, dek_version, committee_epoch, release_key_material_hash, committee_override_hash) (CIP-24 §9.3), hashed as 4 little-endian bytes, and to release_context. Every direct-client GrantDekPackage v1 (§7.7) carries it as 4 big-endian bytes after the version and algorithm tags. Three preimages and the package layout fix its width, so an implementation may not widen it locally. create_volume takes an owner-supplied initial_dek_version and applies only a lower bound of 1. The chain does not assign it or count rotations. No operation advances it today (§9.4). The client then wraps the DEK (encrypts it under another key) one or more ways according to the volume’s access class (wrapping_key_policy, stored on-chain in the StorageCommitment, §11.1). A volume may carry either wrap or both:
    • The owner wrap (owner-key, v2) wraps the DEK to a key derived from a deterministic wallet signature. The owner can then read and write directly from the CLI or browser, including from extension wallets that never expose their private key. Native wallets, CLI wallets, and extension signMessage implementations MUST use this same derivation:
      The chain ID is unsigned canonical decimal with no leading zeroes. The volume hex is exactly 64 lowercase digits without 0x. Lengths count UTF-8 bytes, and the two length-framed strings MUST fit in u16. Both signature scalars MUST satisfy 0 < r,s < secp256k1_order. The client normalizes high-s before HKDF and excludes the recovery byte v. Extension wallets sign H through window.cowboy.signMessage("0x" + lowercase_hex(H)). This API accepts exactly 32 bytes, shows the user the hex of H, and adds the Cowboy personal-message prefix once. Native wallets and the cbfs CLI sign digest directly. The shared personal_message_digest(bytes) stays generic (prefix || decimal(byte_length(bytes)) || bytes); the owner-wrap path always passes the 32-byte H. The caller MUST establish that the signing wallet is the volume owner. The signature is secret key material. The client never sends the signature, the wrapping key or the plaintext DEK to a server, a web origin, telemetry or a clipboard, never logs them, and wipes its buffers after use. It never offers this signature as a proof of wallet ownership, and it explains the key-derivation purpose before requesting approval. Shared cross-client test vectors use deterministic nonces for tests only; production wrapping samples a fresh nonce. RFC 6979 lets the owner recover the same key and gives no secrecy once the signature is disclosed. The stored owner envelope is this canonical type, separate from the CLI’s local-development WrappedDek type:
      Algorithm 1 names the RFC 6979 secp256k1, HKDF-SHA256 and AES-256-GCM derivation above. The envelope is exactly 62 bytes, and the ciphertext includes its 16-byte authentication tag. Create and transfer admission check the version, the algorithm, the exact size and the absence of trailing bytes before storing it. CLI and WASM share one decoder, which rejects unknown tags, truncated envelopes and untagged bincode blobs and does no format sniffing. Version and algorithm are fixed validated constants for this suite, and changing either cannot reinterpret the ciphertext. The secret derivation and its AAD remain exactly as specified above. The deterministic (r,s) output is a per-volume secret and is not a publishable authentication signature. This design assumes a trusted owner client and signing endpoint. The client keeps the result inside that trusted client, never returns it to an untrusted web origin, telemetry, server or shared clipboard, and never reuses this derivation request as a public proof of wallet ownership. A wallet displays the requested hash; that display does not publish the signature. The client must explain the key derivation purpose before requesting approval. RFC 6979 permits recovery of the same key and provides no secrecy if the signature itself is disclosed. If the signature or the DEK leaks, the volume is compromised, including recorded ciphertext. A fresh nonce, a rewrap or an owner transfer cannot revoke a leaked DEK, and DEK rotation is not implemented. For future confidentiality, the owner creates a new volume with a fresh random DEK and re-encrypts its content; this does not erase plaintext or ciphertext already exposed. Algorithm tags do not change this threat model. A client unable to keep this secret MUST NOT use owner-key wrapping. Owner-wrap v2 is active from block 0 of the next regenesis. There is no derivation from a raw wallet secret.
    • The committee wrap (committee-only) IBE-encrypts the DEK to the CBSS committee under domain CIP9_VOLUME_DEK_IBE_DOMAIN = "cbss/ibe/cip9-volume-dek/v1". An authorized Runner obtains the DEK at mount through a threshold seal (§9.2), and no wallet key material reaches the runner. Release-key discovery MUST supply release_key_suite = 1 (BLS12-381 G2). Clients MUST refuse a missing tag, 0, 2, or any other unsupported tag before using committee material, as specified in CIP-24 §6.4. At CreateVolume and RewrapVolume, consensus MUST decode this envelope with the 512-byte bound and the exact bincode 1 fixed-integer, little-endian vector layout specified in CIP-24 §3.1, and reject trailing bytes, any version other than 2, a ciphertext length other than 48 bytes, or an ephemeral_u length other than 96 bytes. G2 decompression and full AAD and identity binding happen at release time (CIP-24 §3.1, §9.3).
    The canonical hash domain CIP9_HASH_DOMAIN_VOLUME_DEK = "cowboy.cip-9.volume-dek.v1" (with AAD domain …volume-dek-aad.v1) binds each wrap to its volume_id. The wraps are stored in the on-chain StorageCommitment (§11.1). The access classes (wrapping_key_policy) are: A matching dispatched grant (§9.2) also lets a Runner read and write an owner-key or committee-only volume without committee-seal delivery. --owner-key at create stores the owner wrap in addition to any committee wrap, and the CLI can write any volume that carries an owner wrap.
  2. The client encrypts each private object chunk independently with AES-256-GCM, using a fresh random 12-byte nonce per chunk:
    Each nonce is stored in that chunk’s ChunkDescriptor.encryption_nonce, and chunks[].size_bytes records the ciphertext length including its tag, before erasure padding. A (volume_dek, nonce) pair MUST NOT be reused across chunks or writes. Deterministic nonces derived from paths MUST NOT be used. The AAD binds the volume, write ID and chunk index and does not change on rename.
  3. The client erasure-codes the ciphertext, so Relay Nodes hold only ciphertext shards.
  4. The client encrypts the manifest (object paths, sizes, content hashes, ObjectDescriptors) with the volume DEK. Relay Nodes store PlacementRecords separately (§5.3.1), outside the encrypted manifest.

9.2 Runner key access

A Runner receives the volume DEK at job attachment (§8.2); the chain never stores it in plaintext. A matching dispatched grant selects grant delivery. Without one, runner access requires the volume’s committee wrap (§9.1). The Dispatcher never holds the owner wrapping key or a plaintext DEK. On the committee-seal path, each CBSS committee member returns a partial decryption of the committee wrap and the Runner combines them into the DEK. The Runner holds an x25519 recipient keypair whose public key is the encryption_pubkey in its registry capabilities (§5.1.1). The Dispatcher constructs and authorizes the SealRequest. At mount:
  1. The Runner issues a SealRequest for the volume’s committee-wrapped DEK, presenting its CapToken (and, for tee-gated, its attestation).
  2. Each CBSS committee member verifies authorization and returns a partial decryption.
  3. The Runner combines the threshold partials and completes the IBE decryption (domain cbss/ibe/cip9-volume-dek/v1) to recover the DEK, sealed in transit to its encryption_pubkey.
  4. The Runner holds the DEK in memory for the job and zeroizes it on completion.
The Runner discovers both inputs from chain (§16.6). It resolves the seal committee from the CBSS committee registry (CIP-24), keyed by the volume’s recorded committee epoch, and fetches the sealed ciphertext as an ordinary CBFS object from the relays in the relay registry (§5.2). On the grant path, the Dispatcher matches the Runner’s registered keys to the grant, and the Runner opens the recipient-sealed package (§9.2.1) that the owner or an authorized DEK holder sealed off-chain. This lets a Runner mount an owner-key-only volume. For a dispatched grant, the volume MUST be PRIVATE and ACTIVE, the attachment’s path prefix MUST be empty, and exactly one grant row MUST name the Runner’s account. That grant MUST be READ_WRITE, whole-volume, not tee_required, unexpired, and carry a valid grantee_wrapped_dek. Its signing and encryption keys MUST equal the Runner’s registered keys (§9.2.1). This path issues no CapToken (token = None) and needs no CBSS release-key material. cbss_rewrap_required gates only committee-seal delivery. Ownership transfer clears all grants, so the new owner must authorize a new grant to use this path. A persistent CIP-10 workload’s host requests the committee-wrapped DEK for one attempt (CIP-10 §17.5) with no CapToken and no owner delegation. The committee releases it to the recipient tuple {owner: Address, delegated_ed25519_pubkey: SigningPublicKey, mount_policy_hash: [u8; 32], recipient_key_id: u64, recipient_x25519_pubkey: [u8; 32]}, encoded in that order: the volume’s current owner, the attempt’s release key, the hash of the owner’s current CIP-10 mount policy for that workload and volume, the attempt number, and the host’s X25519 key. The release key uses the tagged SigningPublicKey encoding u8 algorithm ‖ varint length ‖ key, 34 bytes for an Ed25519 key. Execution fills owner and mount_policy_hash from current state at admission and requires recipient_key_id to equal the attempt. Each committee member releases a partial only when proofs against one finalized state root show that the attempt records that release key and recipient, that the stored policy still carries mount_policy_hash and covers the workload, and that the workload’s reporter still holds its host-pool membership. A revoked or replaced policy has no row with the old hash, so its recipients are never served again. Each committee member submits its partial in its own KeyDelivery(Cip9ServiceVolumeDek) transaction, signed by the member’s operator. The Storage Manager keeps one release row per request, recording the accepted partials in arrival order. The row becomes Ready at the release key’s threshold. Its discovery queue index, which tells committee members the request still needs partials, stays until accepted partials reach min(committee size, 64) or the request expires. The delivery that completes that count removes the queue index, not the release row. An authenticated fetch of a Ready row stays live until the request expires or loses authority. Before it accepts a partial, a delivery authenticates the sender against the committee operator at the partial’s position and verifies the partial’s signature. It also re-checks the request’s full liveness: the attempt, slot, volume state, mount policy, owner policy and release key. For every partial already in the row, it re-checks the partial’s binding to the request and its committee position and index. While the row is Pending, it also re-checks that each accepted proxy is still active: registered, eligible, not suspended, not unbonding, with a public key that derives its operator. A failed check makes the row Invalidated. Once the row is Ready, an accepted partial stays fetchable after its proxy is suspended or unbonds: only its binding and signature are re-checked. Every new delivery still requires its own proxy to be active. From SERVICE_RELEASE_DELIVERY_COST_ACTIVATION_HEIGHT (a CIP-50 not-before gate), a delivery verifies the new partial’s signature once and no longer re-verifies the signatures of partials already in the row. Only the delivery itself writes partials into a release row, and only after verifying each one against its operator, and the request fixes the committee epoch whose operators sign. Every other check above still runs on each accepted partial. RPC polling, the CBSS host’s authorizer and the Runner recipient still verify every partial. From the same gate, the delivery’s fixed access row is the CBSS pause read, the release-row read and the two writes. Once the delivery has decoded the row, it reserves its remaining reads for the row’s real status and accepted count k: 18 + 2k for a Pending row and 26 + k for a Ready row. Before the gate, it reserved them for the 64-partial maximum (CIP-43). The owner of an owner-wrapped volume unwraps the owner wrap locally with its wallet-derived key, with no runner or committee involved. The DEK never appears in plaintext on-chain or in any persistent store. READ_ONLY CapTokens on private volumes still require DEK delivery, because the Runner must decrypt to serve reads. For tee-gated volumes, the committee releases partials only against a valid attestation, and the DEK is sealed to the enclave and never exposed to the host OS. Public volumes skip key delivery, and the job assignment payload has no encrypted_dek field.

9.2.1 Delegated client key access

Pre-release format changes do not advance version numbers. This grant package’s layout at genesis is version 1. Other tags keep their values, including the committee envelope’s version 2 (§9.1). A direct-client grant (§7.7) authorizes storage operations and gives the grantee no owner unwrap key. Direct access to a PRIVATE volume therefore requires a recipient-sealed copy of the DEK, carried in the grant as grantee_wrapped_dek. The grantee generates a separate X25519 encryption keypair, never derived from its Ed25519 signing key, and gives the owner the public half, recorded as grantee_encryption_key. The sealer is the owner or, for a committee-wrapped volume, an authorized DEK holder acting under the applicable CBSS release policy. It seals the current DEK with the CBSS HPKE suite: DHKEM(X25519, HKDF-SHA256), HKDF-SHA256, AES-256-GCM, single-shot base mode, info = "cbfs/grant-volume-dek/v1". EncryptionPublicKey has one encoding: the commonware codec u8 algorithm ‖ Vec<u8> key, with a commonware varint length. Algorithm 1 encodes as 01 20 ‖ key[32] (34 bytes). Its strict commonware decoder MUST reject an unknown algorithm before reading the length, check the 2,048-byte key bound before allocation, and require exactly 32 contributory X25519 bytes for algorithm 1. A standalone value MUST have no trailing bytes. On every JSON interface listed below, the key is one string: "0x" followed by lowercase hex of those codec bytes, so algorithm 1 is "0x0120" followed by 64 hex digits. The list is consensus state, node RPC, relayer HTTP, runner registration, runner-node grant-identity output, cowboy runner register --grant-identity-json input, cbfs CLI --json grant rows and the cbss-wasm grant_dek_aad input. Decode requires the 0x prefix and lowercase, even-length hex, then the strict commonware decoder, and accepts no object form, integer array or bare 32-byte key. Off-chain APIs outside this list keep their own encodings; the harness /v1/workspace/grantee-key response stays 64 hex characters. Signing and AAD preimages use a different form, u8 algorithm ‖ u64be byte_length ‖ key, following the §12.2.1 Encoder convention. MountGrantPayload preimages and the grant AAD use it, the AAD in the exact field order below. This form applies only to signing and AAD preimages; stored and transported keys keep the encoding above. Runner registration is signed over keccak256 of its exact JSON bytes, which carry the key as the hex-codec string above. Runner registration is JSON carried in RunnerRegister { registration }, with a 10 KiB cap on registration. The algorithm-1 key string is 70 characters. A 2,048-byte key would take about 4.1 KB as hex, so enabling such a suite must revisit that cap. At genesis only algorithm 1 is enabled: HPKE DHKEM(X25519, HKDF-SHA256), HKDF-SHA256, AES-256-GCM, base mode, with a key of exactly 32 bytes. Decoders reject low-order and non-contributory X25519 keys. The runner’s registered grant_recipient_pubkey uses this same type. The dispatcher MUST match the registry’s (algorithm, key) against the grant’s grantee_encryption_key exactly, and the package’s recipient MUST equal it. The runner opens the package according to the registered algorithm. There is no implicit X25519 conversion and no algorithm fallback. The package bytes (grantee_wrapped_dek, GrantDekPackage v1) are:
The exact suite-1 lengths are 32, 32 and 48 bytes, respectively (124 package bytes, including all tags and length prefixes). Global ceilings are public key 2,048, enc 2,048 and ciphertext 4,096 bytes; MAX_GRANT_PACKAGE_BYTES=8,204. Decoders check each prefix before allocation. They reject trailing bytes, unknown versions or algorithms, wrong suite lengths, non-contributory recipient or enc keys, a recipient algorithm or key that differs from the grant, and a dek_version that differs from the commitment. Consensus and clients both apply these rules to constructed objects and to decoded wire data. Only this package format is accepted. The ceilings fit an X-Wing public key (1,216 bytes) and KEM ciphertext (enc, 1,120 bytes), but no X-Wing suite is allocated. A new suite needs a suite registry entry, exact lengths and an implementation. A post-quantum package of roughly 2.4 KB would not let a 64-grant roster fit in the 64 KiB StorageCommitment row or in the per-job issuance census bounded by MAX_ACTOR_KV_VALUE_BYTES, so such a suite must also change roster storage. The owner-wrap, HPKE-grant and committee namespaces are separate types. A decoder cannot parse an owner algorithm 1 envelope as a grant algorithm 1 envelope. The seal’s authenticated data is built with the §12.2.1 Encoder (big-endian fixed-width integers, u64-BE byte and string lengths, checksum address strings, and the tags defined below), in this exact order:
max_access_mode is one u8 in the RAS MountGrant tag order: WriteOnly = 0, ReadOnly = 1, ReadWrite = 2. The VolumeAttachment enum of §8.1 uses ReadOnly = 0, WriteOnly = 1, ReadWrite = 2. The two enums are separate types and MUST be converted by variant name, never by ordinal. The RAS order applies wherever a MountGrant field is encoded: the grant wire, the MountGrantPayload owner-action preimage (§12.2.1) and this AAD. tee_required is one u8: 0 = false, 1 = true. valid_until is u8 0 for None, or u8 1 followed by the u64be height for Some. The package’s recipient algorithm and key must equal the grant’s before the AAD is built, and the rest of the context comes from the target commitment. The AAD excludes the HPKE package ciphertext. Clients and the node share one helper in protocol-codec. Substituting a tag or key therefore cannot downgrade the seal or move it to another grant. The grantee persists only its own encrypted private keys and unseals the package locally. Under a WRITE_ONLY or READ_WRITE direct grant, the grantee encrypts uploads with the same DEK. Revoking a grant or delegation stops later authorized requests once cached authorization decisions expire, within the bound in §7.7.1, and cannot recall a DEK or plaintext already delivered. A DEK rotation (dek_version bump) invalidates every package on the allowlist. The owner’s rewrap flow MUST re-seal for each direct-client grant, and a grantee holding a stale package reads nothing newer than the rotation.

9.3 Privacy guarantees

For private volumes (visibility = PRIVATE):
  • Relay Nodes see only ciphertext shards indexed by opaque shard IDs (§16.3). Object paths are encrypted inside the manifest.
  • Runners not assigned to the job cannot access the volume DEK.
  • On-chain observers see only the Storage Commitment (volume ID, wrapped DEK, encrypted manifest hash, total size, shard assignments).
  • The account owner can read all of its volume data by unwrapping the DEK with its wrapping key.
Public volumes (visibility = PUBLIC) have no confidentiality: anyone can read the unencrypted data. BLAKE3 content hashes detect tampering, and only the account owner or Runners authorized by a CapToken can write.

9.4 Rewrapping and rotation

Rewrapping changes who can obtain the key. Rotation changes what the key opens, and only rotation revokes access. The protocol has rewrapping and does not have rotation.

9.4.1 Rewrapping keeps the DEK and replaces the wrap (built)

RewrapVolume replaces a private volume’s committee wrap (wrapped_dek) with one the owner supplies. It first checks that the accompanying cbss_committee_epoch and cbss_release_key_material_hash equal the owner’s current release key material. It leaves dek_version, owner_wrapped_dek and every stored byte unchanged, and it clears cbss_rewrap_required. TransferVolume sets cbss_rewrap_required on any volume that has a committee wrap, and the Dispatcher refuses committee-seal delivery while it is set (§13.4). An ordinary committee reshare does not set it. A CBSS force-rekey of an account walks the owner’s volume index and sets it on every private volume of that owner, at most 64 volumes per call (CIP-24). The new owner must rebind the committee wrap to their own release key before using the committee-seal path, or authorize a new matching grant (§9.2). The chain verifies the epoch and the material hash, never the wrapped key, so an owner can substitute a different key with no dek_version change. Rewrapping also revokes nothing: every party that has held the DEK still holds a working key.

9.4.2 Rotation is not implemented

Rotation MUST re-encrypt. While the ciphertext is unchanged the old DEK still opens it, so bumping dek_version and republishing wraps alone would assert a rotation that did not occur (§17.2 rules this out). Until rotation exists, the available operations can withdraw permission (§7.7) and none can take back a key already delivered, so an owner facing a compromised grantee must create a new volume and copy into it. This CIP does not specify the rotation instruction. What a rotation MUST achieve, whatever shape it takes:
  • R1, total re-encryption: every object and every manifest node is re-encrypted under the new DEK. A partial rotation does not count.
  • R2, no split state: no commitment is reachable in which dek_version has advanced but the root has not, or the reverse.
  • R3, owner authority: a principal that can rotate can lock the owner out, so rotation is not delegable to any grant, CapToken or delegation scope that is otherwise sufficient to commit.
  • R4, end of pre-rotation authority: after a rotation, no principal authorized before it retains read or write access.

9.4.3 What any rotation design has to solve

Each obstacle below exists in the current chain.
  1. A full-volume rewrite does not fit one commit. apply_manifest_commit refuses when added + removed shard references exceed MAX_SHARDS_PER_COMMIT (4,096), and the check that a commit fits its Access-meter budget (CIP-43) can refuse smaller commits, while a volume may hold MAX_OBJECTS_PER_VOLUME = 1,000,000 objects. R2 then needs a rekey that is durable across a multi-commit rewrite or an explicit mid-rotation state that readers can see.
  2. CommitManifestFinalize is { volume_id, stage_id } and carries no authorization. The chain derives its authority from the stage header, which admits a full-volume ReadWrite mount grantee and a staged owner delegation as well as the owner. Meeting R3 with an existing commit instruction needs a field that marks the commit as a rekey and narrows who may submit it.
  3. Clearing mount_allowed_actors does not invalidate CapTokens. TransferVolume invalidates them by advancing last_owner_transfer_block, which the chain compares against token.valid_from; R4 needs an equivalent marker that a rotation advances. Without one, a runner attached before the rotation can commit old-DEK objects onto the post-rotation root. A dek_version bump does make direct-client GrantDekPackages unusable, but keyless grants and issued CapTokens survive.
  4. The party being rotated away can keep the rotation from ever finishing. A staged rewrite spans many transactions and expires, and the stage header fixes the root the rewrite started from (prev_root). Any intervening commit by a principal that still holds write authority fails the compare-and-swap, and the owner re-uploads the whole volume. Authority has to be withdrawn before or during the rewrite.
  5. The owner of a committee-only volume cannot read without a runner (§9.2), so it cannot always re-encrypt client-side. That class needs a runner-mediated rotation, which conflicts with R3, or must be declared not rotatable.
  6. dek_version is owner-chosen and 32 bits. volume_id is deterministic in (owner, volume_name), the identity carries no per-creation nonce, and initial_dek_version is caller-supplied, so deleting and recreating a volume can reproduce an identity already used, and a party that combined threshold partials for it keeps its key. Overflow at u32::MAX is undefined.
  7. wrapping_key_policy does not exist on chain. A validator cannot read §9.1’s access classes as state (§17.3), so no rotation rule may depend on them.
  8. A rekey changes the commit payloads in the shared protocol codec, with golden vectors and a fixed-size owner-operation table, so every signer and verifier changes together.

9.4.4 What rotation would and would not revoke

  • Objects written after a rotation are opaque to a holder of the old DEK, and earlier objects only because R1 re-encrypted them.
  • Plaintext already extracted stays extracted.
  • path_tag history (§7.7) is computed under the old DEK and stays on chain, linkable by the old DEK.
  • Relays that retain superseded ciphertext hold bytes the old DEK opens until shard lifecycle (§13) removes them.
  • A rotation re-encrypts, re-erasure-codes and re-uploads every object, and the byte delta counts twice. Rotation MUST NOT be presented as a background operation.

10. Billing and fees

10.1 Fee components

RAS charges two fees and holds one refundable bond, all in CBY: create_volume debits the creation bond from the owner in addition to any initial escrow, and holds it inside the volume’s escrow_balance as deposit_wei. The bond is not available for rent, withdrawals, gas or the rent floor. When the volume is garbage-collected, the bond returns with the remaining escrow to the current owner’s account storage reserve.

10.2 Effective size and erasure overhead

A volume’s effective size is its raw size times the erasure overhead:
For the default 4/6 scheme, effective_size = raw_size * 1.5. The account pays for the effective size, which is what Relay Nodes store.

10.3 Persistent storage billing

Storage costs accrue every epoch. Relay Nodes hold the bytes, so the chain settles storage from attestations and checks them with Proof of Retrievability challenges (§5.6).
  • Rent settles once per epoch, per volume, at that epoch’s fee snapshot (CIP-31).
  • Each settlement first funds the volume’s escrow toward fee × (elapsed + E) epochs of rent, where E is max(min_storage_balance_epochs, target) and target is the owner’s auto top-up target. Funding draws from the owner’s account storage reserve, then from the owner’s spendable balance. Settlement never draws vesting-locked balance. Settlement then charges the elapsed rent from escrow, then from the reserve.
  • Auto top-up is on for every wallet by default (Default: the governed target auto_top_up_default_target_epochs from that segment’s fee snapshot, genesis 7, and no cap). An owner may set Custom (own target and a total cap) or Off (E is min_storage_balance_epochs and settlement never draws the spendable balance).
  • Residual escrow at garbage collection returns to the owner’s account storage reserve. §10.3.1 gives the grace and expiry transitions.

10.3.1 Escrow and rent lifecycle

At create_volume() the owner MAY prepay storage with --initial-escrow.
  • A volume may be created at zero bytes with any escrow, including none; a zero-byte volume’s rent floor (the minimum escrow a volume of its size must hold, CIP-31 §3) is 0. If the requested initial escrow exceeds the owner’s spendable balance, creation MUST fail with a distinct, named error (§12.3.1). Create records on the volume the rent-floor policy in force in its epoch (CIP-31 §3).
  • A commit that grows used_bytes MUST leave escrow at or above the rent floor of the new size, under the policy recorded on the volume. If escrow is short, the Storage Manager tops it up from the owner’s account storage reserve only, never from the wallet. If the reserve cannot cover the shortfall, the commit fails with E_STORAGE_GROWTH_BELOW_FLOOR. A commit that shrinks the volume always passes.
  • When escrow plus reserve falls below the rent floor at settlement, the volume enters GRACE_PERIOD for STORAGE_GRACE_EPOCHS. Funding at a later settlement, by deposit, reserve or auto top-up, returns it to ACTIVE.
  • If funds are still short after the grace window, the Storage Manager marks the volume GARBAGE_COLLECTING (§13).
Any sender may submit DepositVolumeEscrow, which moves the sender’s spendable balance into escrow the owner controls. The owner may withdraw escrow above the rent floor (§12.3). Two instructions act on the sender’s own account with no delegated authorization, so only the owner’s wallet can issue them:
  • SetStorageAutoTopUp { setting } (RAS kind 31) replaces the auto top-up setting with Default, Off, or Custom { cap_remaining_wei, target_epochs } (tags 0, 1, 2; any other tag is invalid). Custom requires a non-zero cap and target_epochs ≥ 1. Each Custom draw reduces the cap, caps do not accumulate, and a spent cap becomes Off. An explicit Off persists until the owner changes it. No setting releases escrow or reserve.
  • WithdrawStorageFunds { volume_id?, amount_wei } (RAS kind 32) moves unused storage money to the sender’s spendable balance. With no volume it draws the account storage reserve. With a volume, the sender must own it, it must be live (not DELETED or GARBAGE_COLLECTING), and it settles first. An amount that would leave escrow below the rent floor is refused with E_STORAGE_WITHDRAW_BELOW_FLOOR, which names the maximum withdrawable amount. amount_wei is at least 1 and fits in u64.
RAS kind 22 is unassigned. CIP-31 specifies rent accrual, grace and eviction economics, and the Tier-0 values (set by CIP-12’s lowest governance tier). Each transition here reports a distinct, named error or status.

10.4 Fee distribution

CIP-31 §4 holds the canonical split of the per-epoch storage fee. STORAGE_FEE_PLATFORM_BPS (default 1000 = 10%) goes to the Platform Fee Account 0x18. STORAGE_FEE_CHALLENGE_POOL_BPS (default 100 = 1%) goes to the PoR challenge pool at 0x0B; it is the POR_CHALLENGE_FEE_SHARE row in §14 and funds the challenger bounty (CIP-31 §7). STORAGE_FEE_RELAY_BPS (default 8900 = 89%) goes to active Relay Nodes in proportion to a sum over their counted valid PoR responses, where each response adds how long that relay has been assigned that shard index, up to a cap (CIP-31 §5). The three must satisfy STORAGE_FEE_PLATFORM_BPS + STORAGE_FEE_CHALLENGE_POOL_BPS + STORAGE_FEE_RELAY_BPS == 10000, and Tier-0 governance MAY rebalance them under that sum. Transfer fees (TRANSFER_FEE_PER_MIB) go entirely to the serving Relay Node.

10.4.1 Funded cumulative read channels

The version-1 funded-channel mechanism is active from genesis. Nodes MUST publish activation_height: 0, active: true in the funded-channel pricing capability and 0 in the corresponding /chain-info activation-height field. Operator services are opt-in. The advertised authorization is funded_payer_cumulative_read_channel_v1. Support for kind-27 read transfers MUST NOT imply channel support. The kind-27 instruction, its domains and records are separate from channels, and none of its signatures or fee caps count as channel authority. A channel opens with RAS kind 28 (OpenReadChannel), which MUST be a direct, externally signed transaction from the configured payer. Actor authority, whether exercised through a deferred transaction or held implicitly, is insufficient. The terms fix version, chain ID, genesis beacon, payer, relay wallet, relay node ID, volume ID, a nonzero funding nonce, maximum rate R, maximum cumulative units, deposit B, read-stop height, and a later claim deadline. Identities, deposit and unit bound are nonzero. The deadline is strictly later than read-stop and less than u64::MAX. The channel ID binds network, payer and funding nonce. A separate terms hash binds the complete fixed terms. Reusing a nonce with changed terms MUST collide with the permanent channel or its tombstone (the permanent record kept after the channel closes) and create no new channel. Opening requires an Active volume and an Active registered fixed relay with the matching wallet. Opening debits exactly B from the payer into a dedicated per-channel ledger under 0x0A / ras:read-channel:v1:, plus transaction gas. It MUST NOT fund reads from volume rent escrow, rent rewards, challenge pools, or an attachment/job budget. To read, the payer sends a signed off-chain request that binds channel ID, terms hash, a permanent read ID, shard ID/index, the complete logical payload length and BLAKE3, the previous sequence, and a deadline no later than read-stop. One complete logical shard is at most 256 MiB. A request lets the relay try to serve bytes and carries no spending authority. Every new data request MUST pass current serving-registration, manifest/placement, pinned transport-identity, and data-access checks (including a capability where the private-data policy requires one). Payment grants no data access, and an Actor, job, CapToken or DEK grants no payment authority. Active or Draining fixed relays may serve valid new reads. The payer signs a cumulative ACK only after the complete logical payload matches the independently trusted manifest. Each ACK contains:
  • channel ID and terms hash;
  • strictly advancing sequence;
  • cumulative units U = sum(ceil(logical_shard_bytes / 2^20));
  • cumulative fee ceiling F, read count, payload-byte count and receipt root.
For a new acknowledged request the receipt root is keccak256(READ_CHANNEL_RECEIPT_DOMAIN || previous_root || canonical_request_bytes). The domain is the bytes cowboy.cbfs.read-channel-receipt.v1 followed by one NUL, and the initial root is zero. Sequence, units, read count and payload bytes advance together under the codec’s rules on how those counters may change. Fee ceilings are monotone and bounded by B. An object-level cap bounds incremental exposure only. Verifiers MUST check network/domain separation and canonical low-S payer signatures. Transport chunks MUST NOT be individually rounded or acknowledged as extra logical shards. The relay claims with RAS kind 29 (SettleReadChannels), which carries an epoch label and 1..32 vouchers for one fixed relay wallet/node, sorted by unique channel ID. The epoch label is scheduling metadata; it neither namespaces replay nor resets a budget. Each selected channel needs only its latest cumulative ACK. A checkpoint settles one channel at the block height where the claim executes, and charges:
It MUST satisfy the fixed rate bound r <= R, the signed cumulative ceiling delta <= F - paid_atomic, and the remaining dedicated escrow. It MUST advance the settled-units counter (settled_units) exactly once, even when r == 0. All arithmetic is checked. Only the fixed relay wallet receives delta, whatever the current registry owner or volume ownership and status. A kind-29 transaction MUST NOT be deferred, and the page’s relay wallet MUST be one of the transaction’s signers, so no other party can submit the relay’s claim. The escrow is released to that wallet. The Storage Manager validates the entire page, meaning the batch of vouchers in one claim transaction (signatures, channels, aggregate payout), before any write, and one invalid entry rejects the page. Settlement at the claim deadline is permitted. Existing ACKs and claims MUST NOT depend on current serving registration or reauthorization of the volume. Any gas-paying account may submit RAS kind 30 (RefundReadChannel) strictly after the claim deadline. The remaining principal goes only to the fixed original payer, and the closed channel’s tombstone is permanent. Every valid state satisfies:
Refunding or changing ownership MUST NOT erase a funding nonce or revive a closed channel. The version-1 codec defines fixed big-endian terms (217 bytes), unsigned voucher (137 bytes), unsigned request (186 bytes) and refund ID (32 bytes). A signed voucher or request appends the payer’s AccountSignature (a varint policy revision and 1..16 signature slots), so its length varies. With one slot and a policy revision of 0..127 (a one-byte varint), a voucher is 209 bytes and a request 258 bytes; each further varint byte adds one. A settlement page is 62 + Σ signed voucher lengths bytes, for example 6,750 bytes at 32 such vouchers. CBFS operations 16 and 17 carry channel shard delivery and ACK handoff; operations 0..15 and the existing payment domains are unchanged. The canonical codec and its golden vectors define the signed bytes; JSON and host-native serialization do not.

10.4.2 Bounded journals, economics and recovery

Payers MUST reserve a channel’s full B and worst-case opening gas under explicit lifetime budgets, saved durably across restarts, before signing any opening. All payment APIs that share an identity or journal (the durable local record of signed and pending payment operations) share that allowance; independent APIs MUST NOT each allocate a fresh copy. A relay MUST reserve, per channel, room for one logical shard delivered but not yet acknowledged before serving bytes, enforce global limits on bytes and value not yet covered by a signed ACK and lifetime limits on all delivery at R, and retain permanent records of read IDs and settled cursors. If a payer refuses to ACK, the relay’s loss is bounded by that window; there is no fair-exchange guarantee for the last chunk. A full journal MUST refuse new operations, without dropping existing authority or resetting budgets. Exact signed opening/page bytes and nonce ownership MUST be durable before submission. When admission or finality is unknown, the client may only replay those exact bytes, never repriced, repacked, re-signed or under a new nonce. Release requires trusted finalized evidence for the exact transaction and committed nonce consumption; an HTTP status, a missing transaction or elapsed time does not count. After a page finalizes as reverted, a new budgeted page may reuse still-unconsumed vouchers; after a page whose outcome is unknown, it may not. A lost signed ACK is resent as saved. A crash between payload verification and signing may resume only the exact record of what it intended to sign, saved durably before the crash. It cannot create an ACK for bytes it never verified. A relay may need several budgeted pages and may postpone claiming small remaining amounts. The relay’s settlement submitter runs under three explicit, nonzero operator thresholds: minimum_revenue_atomic (a settlement earning less is not submitted), maximum_gas_share_bps (the most of a settlement’s revenue it may reserve for gas, 1..9,999 bps) and maximum_gas_atomic (the most any one settlement may reserve for gas). The gas budget for a settlement earning revenue is min(revenue × maximum_gas_share_bps / 10,000, maximum_gas_atomic). Current native accounting debits the SUM of all three tariff components from the same CBY balance. The CBY/DBY component field labels name one asset and MUST NOT justify a discount. Lifecycle cost includes opening, checkpoints, charged failed attempts, refunds, locked capital and bytes delivered but not yet covered by a signed ACK. The protocol sets no production lock windows, deposit tiers, gas/revenue ratios, carry-over or loss budgets. Pricing requires measured 1/8/32-channel Cycle/Cell/Access costs and equal-total-unit checkpoint-partition comparisons from execution receipts. Operators may stop new reads without invalidating funded claims or expiry refunds, and reverting deployment code must not strand escrow. Enabling a new client implementation implies no operator enablement, free fallback or journal deletion. Pinned trusted-node JSON state/finalized archives MUST be labelled as observations; they are not cryptographic state proofs or relay delivery evidence.

10.5 Relationship to CIP-3

RAS adds no meter to CIP-3. Onchain operations (creating Storage Commitments, writing manifests, opening, claiming or refunding read channels) consume Cycles, Cells and Accesses as normal transactions, separately from escrow principal. The Storage Manager debits off-chain storage fees from a separate ledger each epoch.

11. Onchain state

11.1 Storage Manager system actor

The Storage Manager system actor at STORAGE_MANAGER = 0x0A keeps one StorageCommitment per volume:
The authoritative live shard set lives in native consensus state, outside the serialized StorageCommitment. One stripe row, keyed by (volume_id, shard_id), holds all w = K+M per-index chunk_root values, the common shard_size, the publication event sequence, and a dense stripe ordinal (the stripe’s position in a gap-free, zero-based numbering of the volume’s stripes). An ordinal row keyed by (volume_id, stripe_ordinal) identifies the stripe. live_shard_count counts shard refs, w per stripe, so the stripe count and next append ordinal are live_shard_count / w, and MAX_LIVE_SHARDS_PER_VOLUME = 1,048,576 caps refs. A manifest commit updates the delta’s stripe rows and the ref count atomically with manifest_root. Removal moves the tail stripe into the removed slot. The whole-stripe rule (below: a commit adds or removes every index of a stripe together) reads only TaggedShardRef fields (kind, path_tag, shard_id, shard_index, chunk_root, shard_size). MAX_SHARDS_PER_COMMIT = 4,096 is a consensus cap on added plus removed refs across a complete commit. A direct CommitManifest and each CommitManifestStage page MUST also fit the stricter COMMIT_EVENT_SHARD_REFS_PER_CHUNK = 512 combined-ref cap. Stage checks the aggregate before writing a new page; an identical replay is not recounted. Finalize checks each page and the running aggregate. The shared apply path checks it again. Violations reject with RasInvalidDataSite::QuotaExceeded. The check that a commit fits its Access-meter budget (CIP-43) is a separate limit. Every object chunk MUST use the volume’s K and M, duplicated in its descriptor (§5.3), and consensus derives w from it. For each shard_id in each of the added and removed sets, a commit MUST carry indexes 0..w (0 through K+M−1) exactly once, with equal shard_size, kind, and path_tag across those indexes. For staged commits the rule applies to the concatenation of all stages, so a stage boundary may split a stripe but finalization rejects a partial or duplicated one. Shape validation runs after the manifest-root check and before inventory reads. A violation rejects with RasInvalidDataSite::CommitWholeStripeInvalid (slug ras.commit_whole_stripe_invalid). Every removed stripe MUST already exist and match the supplied root at every index and the supplied size (§6.2). Every added stripe MUST have no existing (volume_id, shard_id) inventory row, so a reused write_id fails the commit and cannot overwrite another object’s stripe. Rent evidence uses one history row per stripe inventory incarnation. An incarnation is one stay of a stripe in the inventory, from the event that adds it until its removal. The row is keyed by ras:rent:v1:history:<volume><shard><incarnation_be>. incarnation_be is the big-endian u64 encoding of the stripe’s publication event sequence (RasStripeRow.added_at_event_seq), which the challenge carries as snapshot_added_at_event_seq. The row’s stripe-wide generation is that sequence, with (node_id, chunk_root, assigned_at_block, closed_at_block, origin) per index. Re-adding a removed stripe creates a new incarnation. Assignment time MUST never be backdated for any index. PoR reads the history row for the challenge’s snapshot_added_at_event_seq, and FirstValidPor bootstrap checks that sequence against inventory. Proof keys and counted keys remain per index (CIP-31 §5). There is no separate ras:rent:v1:assignment: row; other rent keys, including rrp:, rrw: and rrf:, keep their formats. Rent rows use a key-aware binary codec that omits identifiers implied by the key, stripe header or index position and reconstructs them on decode. Remaining fields use fixed-width big-endian integers, raw arrays, one-byte Option/bool/enum tags, and bounded vectors with u32 lengths. FeeAmount is raw u128. The decoder rejects malformed keys, unknown tags, invalid booleans, truncation, trailing bytes and oversized vectors. The encoder rejects values that disagree with implied fields. There is no JSON fallback. /fees/cbfs-rent/epochs/{epoch} returns JSON whose assignment_generation field carries the inventory incarnation event sequence. PoR weight-page JSON stores {weight,w} entries, and the raw-state RPC exposes value hex. Direct commits and Finalize charge an upper bound of 1 Cell per logical persisted key+value byte for all state growth, plus full rewrites of rows sized by the transaction inputs, as specified in CIP-43 §5.5 and mirrored in cbfs_registry_proto. Inventory, rent, event, snapshot, funding and deferred rows count full writes, including overwrites; only the in-place remainder of StorageCommitment, the global stage index and existing PoR rows is covered by the flat base charge, and their growth is charged. Direct and staged paths have the same application charge for the same delta and settlement context. Each Stage separately reserves its full body, header, uploaded ordinals and accumulated totals, plus global-index growth. Clients price direct/Finalize with fresh, durably saved information about the storage epochs queued for rent settlement, and reserve width 32 for empty deltas, whose refs reveal no width; native execution uses the stored width. Width 32 bounds Cells metering and row decoding; the creatable grid is 2 ≤ K ≤ 16, 1 ≤ M ≤ 8 (K+M ∈ 3..=24). Shared read-only settlement reserves queued rent Cells before writes for DepositVolumeEscrow, WithdrawStorageFunds(Some(volume)), manifest application, and escrow-paid write admission other than CommitManifestStage. Escrow-paid Stage admission charges zero queued-rent Cells even when it writes rent rows. A catch-up settles storage epochs that are queued and not yet settled. On the charged paths, the first catch-up in an epoch writes the queued rent rows and charges 334 × unsettled_rows; a later catch-up in the same epoch, in any transaction, charges 334 × 0 = 0. In the native application formula, K is unsettled_rows. Clients budget the conservative pending_rows. A withdrawal from the account reserve without a volume does not catch up rent. A tight signed Cells limit can fail before the post-settlement terminal-state, balance or withdrawal-floor checks (E_STORAGE_WITHDRAW_BELOW_FLOOR), though an already-terminal volume is refused before settlement. On OutOfCells the full Cells limit counts as used and is not refunded, and the node rolls back the attempted rent and manifest writes. Stage header, uploaded-ordinal and totals values use a strict big-endian codec under the same keys. The decoder checks lengths before allocating and rejects unknown tags, unsorted or duplicate IDs or ordinals, truncation and trailing bytes. Bodies are bincode, and the shared global stage index is JSON. Wire pins are CBFS_MANIFEST_COMMIT_WIRE = 1 and RULESET_REVISION = 1; the codec was introduced at access table 25, and the current access table is 43. Computed direct/Finalize signers refresh K after their last await and reserve K+1, adding 334 Cells of inclusion headroom. Manual gas overrides MUST include their own K+1 margin. Partial or Finalize-only batches recover missing committed pages, then refresh queued-epoch context to price the complete delta. SDK, CLI and hooks clients compute their own limits. Computed direct/Finalize CLI and runner signers read K immediately before and after /estimate_gas, and again at the final signing await. Any change discards the estimate and repeats the check, up to three attempts; if K keeps changing, the signer returns a retryable “queued epochs are changing” error without signing. A stable estimate supplies access authority and Cells at that K+1. The signed access limit is min(ceiling, max(suggested, used + reserved) + client_access_limit(12)), where the extra inclusion epoch converts 12 structural accesses to charged units at the maximum governed scalar. Optional padding is capped at the lane ceiling. The signer never refuses locally on epoch-growth arithmetic; the node decides whether the transaction fits. Runner retries follow these rules. An uncertain retry keeps the nonce and the exact signed bytes. A definitive rejection before inclusion (non-admission), including an initial nonce_too_low, releases the nonce reservation without recording a failure, and the next submit re-reads the account nonce. During reconciliation, a low-nonce replay of exact bytes stays uncertain until its receipt resolves, and a missing receipt plus an advanced account nonce never permits a new signature. A failed Cells budget survives a successor’s non-admission. An executed Cells shortfall requires a strictly larger Cells limit; any other executed failure permits a freshly priced limit. Immediate success records nothing. Only an uncertain attempt later reconciled as success is answered once without re-signing, and its matching read consumes the success record. Any other repeat of Stage, Finalize or direct CommitManifest is signed and the chain decides. The account keeps at most 16,384 FIFO operation outcomes, refreshed on re-recording; eviction permits another submission. Byte-identical manifest changes can be distinct operations after the root returns to an earlier value. Uncertain pending bytes remain separate. Current Stage funding uses the source-derived auth-v3 reservation, whose ordinary-System structural capacity is 1,116. The shards RPC expands stripe ordinal c and index i into the computed shard number c·w+i; its total is live_shard_count, and the inventory revision detects concurrent mutations during pagination. PoR selects a shard index from a stripe row and records its issue-time commitment (§5.6). The Storage Manager also keeps a relay-rotation side table: Expiry needs no row deletion. On chain the committee wrap is wrapped_dek and the owner wrap is owner_wrapped_dek; there is no field named committee_wrapped_dek. CreateVolume requires wrapped_dek and a nonzero cbss_committee_epoch on a PRIVATE volume, and accepts owner_wrapped_dek only on a PRIVATE volume. Each wrap is bound to volume_id through the canonical hash domain cowboy.cip-9.volume-dek.v1. dek_version, the two cbss_* binding fields, cbss_rewrap_required and last_owner_transfer_block exist on the chain StorageCommitment. wrapping_key_policy exists nowhere in the node. §9.1’s access classes are a target, so no rule may depend on reading them from state. The block above is a partial view: the chain commitment also carries cbss_committee_override, the MPK-setup fields, relay_set_revision, pending_rotation, deleted_at_epoch, deposit_wei and the whole rent/escrow group. See §17.3. It keeps one AccountStorageSummary per account:
An account owns at most MAX_VOLUMES_PER_OWNER = 1,024 volumes. CreateVolume, and a TransferVolume naming the account as new_owner, MUST reject a 1,025th volume with RasInvalidDataSite::QuotaExceeded (E1213) before any write. Both also reject while the account’s CBSS force-rekey cursor is live. The cap bounds the owner-scoped volume index, one bincode Vec<VolumeId> value (32,776 bytes at the cap, inside the 64 KiB actor value limit), and the CBSS force-rekey walk over it, which marks at most 64 volumes per call (16 calls at the cap; CIP-24).

11.2 Relay registry

The Relay Registry is the system actor at RELAY_REGISTRY = 0x0B. It manages:
  • RelayNodeProfile entries (see §5.2)
  • Active relay list, stored as two fixed pages ras:active-relays:0 and ras:active-relays:1 of raw 20-byte addresses (2,048 per page). Liveness is RelayStatus; there is no health decay.
  • Shard assignment index: volume_id → list[PlacementAssignment]
  • The network-wide AUTO_DRAIN_POLICY_KEY slot written by governance (§5.8)

11.3 Key space

The Storage Manager stores Storage Commitments in the CIP-4 STORAGE key space under its own address, with ASCII-prefixed keys and JSON values:
The volume name index is the canonical (owner, name) → volume_id lookup (§12.3), written at create_volume(). It is keyed by the owner’s chain address, the same owner_address used to derive volume_id. The derivation is owner_volume_name_key(owner, volume_name):
A name lookup that misses this index MUST be treated as “volume not found for this owner” and never routed to a different owner. Account summaries:
Relay profiles, under the Relay Registry (0x0B), indexed both ways:

12. Client interfaces

RAS gives Runner workloads two client interfaces:
  • The filesystem interface (§12.1) is a FUSE-mounted directory that presents the volume as an ordinary filesystem. It is the primary interface for agentic workloads, where an LLM works through tool calls, and the model’s filesystem tools work on it unchanged.
  • The Object API (§12.2) is a programmatic interface for orchestration code and non-agentic workloads. The FUSE layer calls it internally.

12.1 Filesystem interface (FUSE mount)

12.1.1 Design rationale

The main consumer of Runner storage is an AI model making tool calls such as Read, Write and Bash, which operate on filesystem paths. A custom put_object / get_object API would need custom tool definitions for every model and would cut storage off from standard Unix tools. A mounted volume lets the model’s existing tools work directly. Each tool call maps to FUSE operations as follows:

12.1.2 Mount point layout

The Runner mounts each attached volume at a deterministic path inside its execution environment:
If the attachment specifies a path_prefix, only that subtree is visible:

12.1.3 Sync strategy (hybrid)

The FUSE mount keeps an in-process memory cache (each file a stub, clean or dirty) with a shared size budget, which every filesystem operation reads and writes first. A background sync daemon moves data between the local layer and the Relay Nodes in two cycles:
  • The push cycle (local to Relay Nodes) detects files written or modified locally (via inotify/fswatch), encrypts them, erasure-codes them, and distributes the shards to Relay Nodes.
  • The pull cycle (Relay Nodes to local) re-fetches the volume’s committed manifest from Relay Nodes, verifies it against the onchain manifest_root (§7.5), and copies into the local cache the new objects that are not already there. Reads are READ_COMMITTED (§7.5): a reader sees only what a committed root contains, so another writer’s objects appear only after that writer calls commit_manifest().
The sync interval is configurable per mount, with default SYNC_INTERVAL_SECONDS = 5. The journal is a log on the Runner’s local disk of manifest changes not yet committed on chain. Manifest commits are batched: one commit carries every change waiting on the volume. A change is staged once its shards are uploaded and it is journaled, and it has landed once the writer has verified that the chain finalized the commit that contains it (§12.1.5 gives the exact definitions). A write reaches the chain as follows:
  1. The push half of a sync cycle stages every dirty file: the file’s Relay Nodes accept its shards (§5.4.1) and the Runner durably journals the manifest change.
  2. Each volume has one commit loop. The loop waits out the commit window (COMMIT_WINDOW_SECONDS = 5), then submits everything staged since the last landed root as a single commit_manifest() and waits for finality. If nothing is staged, the loop submits nothing.
  3. A change staged while a commit is in flight joins the next commit, one window later.
An operation that needs the root onchain (job completion, a published root, a CBSS release, an fsync) does not wait for the commit window. It submits the waiting batch immediately and waits for it to land. This document calls that an anchoring commit. fsync() on a file stages that file, then submits the batch as an anchoring commit and returns once the commit has landed. This is the default --fsync commit mode, which the CLI, standalone mounts and every Runner mount use. With the opt-in --fsync local, fsync() returns once the file is staged, without putting the root on chain. A bare sync does nothing for the mount: the mount has no syncfs handler. sync DIR becomes fsyncdir(DIR), which publishes only names created directly in DIR that are not yet in the manifest. It does not push overwrites of existing files or anything in subdirectories, and it still returns success. In commit mode it then submits and waits for an anchoring commit. Per-file fsync() is the only durability barrier for a file’s contents. Some refusals cannot be fixed by retrying: the volume’s escrow is too low, or the signer lacks commit authority. On such a refusal the commit loop stops submitting and keeps the batch staged until someone acts, and a later fsync reports it as EIO, or as EACCES while the batch waits for the volume owner to finalize it (a prefix-scoped writer stages a commit and the owner finalizes it, §7.3.1). The commit loop retries transient failures with backoff, and the Runner’s next fsync recovers a failed commit before staging anything new. An anchoring commit retries a failed commit once before reporting any failure other than such a refusal. If a Runner dies after a change is staged and before its batch lands, it replays the journaled batch when it reopens the volume with the same local journal. The journal stays on that Runner, so if a different Runner mounts the volume next, staged writes that had not landed are lost. Relay Nodes delete shards that no committed root references after ORPHAN_SHARD_TTL_SECS (24 hours). The Runner uploads again any upload older than half the TTL before a root references it. If relays have already deleted an upload and it is still missing after retries, its path is dropped from the batch; the batch publishes the rest and reports the lost paths, and a Runner that still holds those bytes uploads them again. Otherwise those writes are lost. On container shutdown, the sync daemon pushes all dirty files a final time, then submits the batch as an anchoring commit. If the container crashes before that push, data written since the last push cycle is lost, unless fsync acknowledged it. Reads behave as follows: Writes behave as follows: Every mount schedules the pull cycle. The push cycle runs only under a grant that permits writes. The refresh is deliberately not a MUST, because §7.2 gives WRITE_ONLY no read authority. Whether a write-only mount’s pull succeeds depends on visibility:
  • On a public volume, reads and listings are open (§7.6), so the refresh succeeds and the mounter sees the object namespace (paths, sizes and object descriptors). The mount still refuses every POSIX read of object contents (§12.1.4). Any party can already read that namespace.
  • On a private volume, the refresh is GET_PLACEMENT followed by per-node GET_SHARD (§5.3.2), and §7.2 grants WRITE_ONLY neither. The Relay Node refuses, and the client cannot tell the refusal from transient unavailability. The mount has no committed manifest to compute its changes against, so in practice it cannot commit. The parent-root compare-and-swap of §7.3.1 (the chain accepts a commit only if the root it extends is the volume’s current root) prevents a destructive commit.
This gap is open. Closing it needs an exception that admits GET_SHARD for manifest-node shards only, and a way for a Relay Node to recognise such a shard, which opaque shard IDs do not give (§7.3.1 Layer 3, §7.6.4). The write-time shard metadata of §7.6.3 is the nearest existing mechanism. The gap MUST NOT be closed by admitting object GET_SHARD for WRITE_ONLY tokens wholesale. A write-only mount already holds the volume DEK (§9.2), so relay-side refusal of object GET_SHARD is the only thing between a write-only Runner and full plaintext. Such a change would belong in §15.2 as a deliberate change of the threat model. Object materialization (fetching object bytes into the local cache) is an object GET_SHARD and is checked as one. GET_SHARD, PUT_SHARD and COMMIT_MANIFEST are checked where they are performed: at the shard fetch, the shard put and the commit. §7.2 separates commit authority from write authority, so a PUT_SHARD answer must not be taken as a COMMIT_MANIFEST answer. A long-lived mount can outlive its CapToken’s valid_until. The sync daemon MUST re-mint the CapToken (obtain a fresh one) before expiry (via the mount’s set_auth_token hook) and continue without interrupting in-flight reads or writes. A write sent with a token that expires before the relay checks it retries with the refreshed token and does not surface a 403 to the workload.

12.1.4 FUSE operation mapping

This section defines access modes (§7.2) over POSIX mode bits, so that two implementations refuse the same requests. Depending on its grant, a mount withholds some mode bits: it reports them as 0 and refuses to set them. A grant is partial when it does not permit both reads and writes. For a given inode, a mount reports, and may change, only the bits its grant does not withhold: Linux always reports a symlink’s mode as 0o777, so a mount does not withhold the read and write bits on a symlink. A partial grant still withholds setuid, setgid and sticky on every inode, symlinks included, so it can never write those bits into the volume. A READ_WRITE grant withholds nothing. Under a partial grant, a withheld bit reads back as 0. A tool echoing that view (cp -p, rsync -a, tar -p, chmod --reference) then sends the same bytes as a caller asking to clear the bit:
  • A request that leaves the withheld bits at 0 is ambiguous, and the implementation MUST preserve the stored value of those bits. Storing the zeros would write them into the manifest and clear those bits for every mount.
  • A request that sets a withheld bit is unambiguous, because no caller read a 1 there from this mount. The implementation MUST refuse it with EPERM, and MUST NOT apply any other field of the same setattr, since chmod(2) applies the requested S_IALLUGO bits or fails.
Under WRITE_ONLY the withheld set is 0o7444, so any request naming one of those bits fails, including cp -p, rsync -a, install -m 0644, chmod u+r and chmod 0600. The file contents are still written; only the mode change is refused. A full grant has no withheld bits, so neither rule applies to it. On a WRITE_ONLY mount, no request can clear the group and other read bits. chmod 0600 over a stored 0o644 sets the withheld bit 0o400 and is refused. chmod 0200 succeeds and preserves the stored 0o644, so it reports success while the durable state keeps the group and other read bits. Implementations MAY materialize explicit directory entries in the manifest, to carry POSIX mode, ownership, timestamps and xattrs for directories that contain no files. Where they do, those entries are manifest state covered by §6.3. A Runner MUST NOT be able to remove one, in any access mode, because removing it mutates the volume index and produces a new manifest_root committed under a Runner CapToken with no owner authorization. Such an extension MUST NOT create a deletion capability that §6.3 and §7.2 withhold. The account owner removes directory entries through the onchain API.

12.1.5 Consistency model

This section states what a mount guarantees to the workload and to every other reader of the volume. It applies to every FUSE mount (CLI and Runner). An Object API client (§12.2) holds the root it last refreshed and sees a newer one only when it refreshes. A change is staged when the push half of a sync cycle has uploaded its shards and journaled it on the writing host. It has landed when the writer has verified a finalized-state proof in which the volume’s manifest_root and head sequence are the ones its commit produced. The head sequence is the volume’s commit counter. Only landed changes are visible to other mounts (§7.5). The unit of ordering is the volume: each commit replaces the root and advances the head sequence by one through the parent-root compare-and-swap (the chain accepts a commit only if the root it extends is the current root), and commits on one volume are totally ordered. The unit of conflict is the manifest entry at one path (a file, an explicit directory or a symlink). Commits that change disjoint paths all land: a writer whose commit is refused re-applies its changes on top of the new root (rebases) and submits again. A commit is atomic; a mount’s batch is not, because a conflict removes only the conflicting paths from it. A file is replaced whole: writes from two mounts to one file never merge byte ranges, so O_APPEND from two mounts keeps one writer’s file. Nothing is ordered across volumes. The guarantees are: The visibility bound is as follows. A change written on mount A at time t is visible on every healthy mount B no later than t + V, where
With the defaults at 1 s blocks, V ≈ 28 s + R. After a successful --fsync commit fsync() returns at t_f, the change is visible on every healthy mount by t_f + A + S + R (15 s + R). A staged batch larger than 4 MiB is submitted without waiting for W. The bound holds while the writer’s publications and the reader’s refreshes succeed. A transient publication failure backs off as W · 2^min(f, 4) after f consecutive failures (at most 80 s), and the change becomes visible within V of the publication that succeeds. A refresh that cannot obtain a proof younger than A, or cannot fetch the manifest, leaves the mount on the root it has; it never serves an unverified root (§7.5). An external tester SHOULD treat a path missing from B less than V after the write as a stale read, and one still missing after V with both mounts healthy as a lost write. A mount MUST keep pulling while it has changes it has not published or cannot publish, overlaying them on each pulled root, so that a writer that is failing to publish still converges on everyone else’s writes. The pull runs every sync cycle, and the pulled root is merged under the mount’s unpublished changes. A conditional publication is a commit transaction that is valid only against one approved root. While one awaits explicit recovery, the mount neither publishes nor pulls, so that the mount’s root stays the approved one. When shards or manifest nodes cannot be published, nothing is submitted. The writer keeps its changes in cache and journal, keeps showing them, and retries on the next cycle or backoff; an fsync() waiting on the batch fails with EIO. On a refusal that retrying cannot fix (the volume’s escrow is below its minimum balance, the owner must finalize the commit, or the client’s relay list is out of date), the writer stops submitting the batch: the next fsync() reports the refusal, and the batch stays staged until someone acts. Other readers see the previous root throughout, and published nodes of a root that never commits are never accepted by a reader. On commit failure, the chain refuses a commit whose parent root or head sequence is no longer current. The writer reads the new root from a finalized proof and re-applies its batch on top of it. Changes to paths the other commit did not touch are carried over and published in the next attempt. A path the other commit changed, where the batch’s base for that path is no longer the committed entry, is a conflict, resolved as below. A submission whose outcome is unknown (the volume gas nonce was taken, or the response was lost) is resolved from the next proof: if the commit landed, the writer accepts it as its own; if it did not, the writer re-applies the batch on the current root and resubmits. Readers never see a refused root. For concurrent writes to one path, the first commit to land for the path wins. A later batch changes that path only if its base for the path is still the committed entry. Otherwise its change to that path is dropped from the batch: the losing mount shows the winner’s version, the conflict is logged, and the next fsync() of the affected file (or the one already waiting) and fsyncdir() of its directory fail with EIO once. The losing bytes are not committed. A write made after the change is dropped is based on the winner and lands normally, so the last writer to write after seeing the winner wins. No write() returns an error for a conflict, and a write that was never fsync()ed can be lost this way without any syscall reporting it. Where directories are explicit manifest entries (§12.1.4), two mounts that create the same directory merge: the committed directory stands, both succeed, and the children both mounts create under it land. When two mounts create one path as different kinds (a file on one, a directory or symlink on the other, including creation under a path the other made a file), the committed entry wins. The losing mount moves its uncommitted entry, with everything under it that it has not committed, to <name>.conflict-<mount id> in the same directory and publishes it there. The mount id is eight hex digits, random per mount session; a taken name gets -2, -3, and so on, and the name is shortened to stay within MAX_OBJECT_PATH_LENGTH. Every mount then shows the committed entry at the original path and the moved entry at the conflict path. The next fsync() of each moved inode and fsyncdir() of the directory fail with EIO once. The moved entry MUST be journaled like any staged change, so that a crash or remount before it lands replays it at the conflict path.
Current implementation: the move is held in the mount’s memory until the conflict entry lands. A crash in that interval, or a conflict met by a batch restored from the journal after a remount, loses the moved bytes.

12.2 Object API (programmatic)

The Object API is the low-level interface the FUSE layer uses internally. Scripts, lightweight jobs and orchestration code outside a container call it directly.
put_object splits the plaintext into chunks and encrypts private chunks. It erasure-codes each chunk and distributes it separately. The result is one ObjectDescriptor with ordered chunk commitments and one PlacementRecord per chunk, which the client publishes at commit time. get_object locates, reconstructs and verifies each chunk, decrypts private chunks, and checks the aggregate object hash at the end of the full read. Committing a volume takes two steps, one in the data plane and one in the control plane (§4.3):
  1. The CBFS commit (commit) publishes the delta’s added_nodes to Relay Nodes, each addressed by manifest_node_shard_id(volume_id, locator) (§3.1), and publishes PlacementRecords to all assigned Relay Nodes via PutPlacement (§5.3.1). It does not touch the chain. Manifest-node PlacementRecord publication MUST succeed on every assignee that holds its shard before step 2. An owed index’s relay is skipped; repair gives it the record (§5.3.1). Object-shard record publication is best-effort: the client logs failures, and they do not block the commit.
  2. The Cowboy commit (commit_manifest) submits the new manifest DAG root, the prev_root it extends, and the added_shards / removed_shards deltas to the Storage Manager in a transaction the client signs and submits (§12.2.1). The chain rejects the commit unless prev_root equals the volume’s current manifest_root (parent-root compare-and-swap). In the same state transition, the chain updates live_shard_count and its own inventory of the volume’s stripes (the native stripe rows and dense stripe-ordinal rows, §11.1) for the added and removed stripes. READ_COMMITTED reads (§7.5) read from the committed manifest_root. The live inventory rows supply PoR’s per-shard issue-time commitments (§5.6). Individual ObjectDescriptors stay off-chain in the manifest DAG. For a public volume, the plaintext root yields O(log N) object-inclusion proofs. For a private volume, object contents stay opaque, and PoR checks shard retrievability against the selected native row.
Manifest nodes are content-addressed and immutable (§3.1), so a commit never overwrites the previous root’s nodes. The client publishes the new nodes first and advances the onchain root second. If the onchain commit fails, the chain still points at the prior root, whose nodes are untouched. Bounded orphan garbage collection (§13) deletes nodes that a failed or superseded commit left unreferenced, subject to the publication protection below. A Relay Node MUST preserve both the current committed shard set and newly published bytes awaiting a commit. Each accepted publication or republication, including an identical-byte upload at an existing node address, MUST establish a bounded tentative hold: a promise not to delete those bytes before a deadline. An event for a commit that predates that publication MUST NOT delete the bytes or clear the hold. The hold ends when a commit that incorporates the publication puts the bytes into the committed shard set, or when its tentative deadline expires. Publication and deletion MUST be atomic with respect to each other at the local shard: a delete authorized against an older stored publication MUST NOT delete a concurrent replacement. A publisher MUST keep publication holds valid through commitment. After a hold lapses, the publisher MUST re-establish publication before it attempts to commit that data. After a hold expires, deletion on a volume that has not entered GARBAGE_COLLECTING still MUST preserve shards in the current authoritative committed set. A relay establishes that set from the chain’s native RAS inventory (the chain’s own per-stripe record of committed shards) at the same finalized state as the current authoritative manifest_root (§11.1). The cursor-paged /ras/chain/volumes/{id}/shards view expands each stripe into computed shard numbers c·w+i (stripe ordinal c, stripe width w, shard index i) and exposes a change token. A scan MUST restart if that token changes between pages. The inventory includes manifest-node shards and works for private volumes without decrypting the DAG. A stale removal event, or a lagging event-derived live set, is not by itself proof of absence. If the relay cannot establish that a shard is currently absent from the committed set, or that the local publication may be deleted, it MUST defer deletion. Once the finalized volume status is GARBAGE_COLLECTING, that terminal status authorizes the relay to delete the whole volume’s data (§13.3), even while native rows remain until the chain finishes removing them in bounded steps. A shard shared with another volume still needs protection against deletion while that volume references it. Relays apply publication holds, the current-set check and a publication guard on every relay write and delete. On every successful commit_manifest, the Storage Manager emits a chain event:
Gateways and indexers use the event to invalidate caches. Polling manifest_root every MANIFEST_POLL_INTERVAL blocks remains valid as a fallback.

12.2.1 Manifest commit transport

Control-plane writes (create_volume, commit_manifest, UpdateMountAllowlist, relay registration, escrow deposits) are account transactions. The client, or the relay for its own duties, signs the transaction with its own account and submits it to a node, and that account pays gas unless the payer rules below select workload funding. Owner-authorized instructions also carry the owner authorization below. From CBFS_WORKLOAD_COMMIT_PAYER_ACTIVATION_HEIGHT, eligible workload-bound owner-delegation commits use C3’s payer rule. In this section a floor is a minimum resource budget that the signed transaction must carry. Admission (the node’s check before it accepts the transaction for inclusion) and classification use constant work only: a supported kind, a signed binding-presence hint where available, and fixed cycles and Cells floors. Admission MUST NOT scan refs or derive a payload-sized access floor. The live selector (the payer selection that runs when the transaction executes) prepays payload_length * SHAPE_SCAN_CYCLES_PER_BYTE, with that rate equal to one, before it reads the payload and counts every added or removed ref as a separate stripe. For A added refs, R removed refs, H chunks, O owed refs and payload length L, the width w is one plus the largest shard index (or MAX_STRIPE_WIDTH for no refs), and D is zero for an empty delta and w otherwise. At K = 0, base Cells are 1,048 + 4L + manifest_cells(w,D,A,R,A,R,0) for Direct and 1,048 + 4L + stage_cells(H,O,A,R) for Stage. The scan-derived access row reserves PoR net-stripe change for any nonempty delta. It uses the maximum token, bound-workload and owner-nonce branches, without typed vectors or stripe sets. Only after that does the selector compare all three signed dimensions with the complete base plus the C1 and C3 allowances. An initial-floor failure performs no scan or C3 work. A later shortfall selects signer payment and keeps the scan charge. A Finalize commit carries only volume and stage ids. Its live selector first prepays one retained header point read, within C3’s five-read allowance, before the scan, the complete comparison or any other C3 work. An unbound header selects signer payment, with only that read charged. A bound header continues with the scan and complete comparison, and reserves only four later discovery reads. Other unbound commit kinds pay for no C3 work. The complete live Finalize access comparison then reuses the paid volume and header records and storage epoch to apply C3’s persisted volume/chunk bound. That bound covers commits accepted by apply_manifest_commit. Malformed retained stages still fail both below and above the gate, and selected funding pays for an eligible failure. The live bound check follows six discovery accesses. A valid non-hooks Finalize whose signed/live structural budget runs from its exact native need through native need plus five can therefore become signer-paid OutOfGas at the gate. Direct and Stage typed decoding and payload hashing stay inside the fixed discovery allowance, regardless of a larger signed budget. Deterministic C3 decode and authority failures select signer payment without waiving ordinary RAS validation. Local storage and PVM faults escape before a receipt. Off-chain estimates may compute the complete signing floor, while admission uses only the constant-work possible-candidate check. Estimates and final signing include each overhead once. Below the gate, existing payer selection applies, and explicit volume-escrow transactions keep their separate funding rules. Payer advice is optional for both hooks and production runner signing. On a transport, HTTP or parse failure, on missing or malformed required-limit or overhead fields, on success: false, and on over-cap advice, signing MUST retain all original signer budgets atomically. Valid suggestions are inclusive totals. Signing consumes each one once and retains the exact Cells floor that the planner computes for the storage epochs waiting to be settled. The production runner uses reporter-scoped submission behind C3. It shares the Reporter’s nonce journal and preserves an existing stage’s signer across activation. Unknown Finalize stages keep the node signer. On shutdown, the runner keeps the Busy notifications it has already consumed so it can reconcile them later, with bounded retry delays. These client rules leave the consensus payer predicate unchanged. Only the current workload.reporter can select owning-actor funding, and other signers keep signer payment. C3’s payer rule gives the complete bound, eligibility examples and signing contract. Signers have three further requirements:
  • The owner authorization MUST canonicalize addresses as EIP-55 in both the payload hash and the signing bytes. The owner-action payload is signed over an explicit, language-neutral, big-endian byte layout independent of Rust struct encoding. Every signer and verifier, native or WASM, MUST produce these exact bytes. Golden vectors fix the layout.
  • Owner delegations, owner and relay authorizations, acknowledgements and relay-evidence signatures use CIP-40 §7.6’s object_signing_preimage with the running ChainIdentity. Delegations have no network field; the instance commits to the genesis network. A verifier never trusts a chain ID or network supplied by the signed object as its own expected identity, and chain ID zero has no exemption. RasOwnerDelegation and workload DelegationCert have distinct domains. Owner-authorization nonces are consumed once. Relay self-disable timestamps obey the drain commitment freshness window.
  • Processes that sign with the same gas key MUST select nonces through one shared account journal under an exclusive lock. Processes that share a key without it race on nonce selection.
Batched writes (put_many) and other long operations MUST refresh the CapToken mid-stream. The writer MUST re-mint before expiry, through the same set_auth_token path the FUSE mount uses (§12.1.3), and continue without aborting the batch.

12.3 Account owner API (via Storage Manager actor)

12.4 CIP-2 task definition extension

The OffchainTask struct from CIP-2 gains an optional volume_attachments field:
§8.1 defines VolumeAttachment normatively: the 10-field wire-canonical structure, its RelayEndpoint shape, and the AccessMode tag values (ReadOnly = 0, WriteOnly = 1, ReadWrite = 2). This extension carries the same structure. There is no read_consistency field, because all reads are READ_COMMITTED (§7.5).

12.3.1 Volume creation and lookup errors

Volume-create and name-lookup failures MUST be distinct, named errors, replacing the opaque E1900 "invalid data". The errors are:
  • E_VOLUME_NAME_DUPLICATE: (owner, volume_name) is already present in the name index (§11.3).
  • E_VOLUME_ESCROW_INSUFFICIENT: initial_escrow exceeds the owner’s spendable balance (§10.3.1).
Storage funding controls (§10.3.1) use their own named errors:
  • E_STORAGE_WITHDRAW_BELOW_FLOOR: a volume withdraw would leave escrow below the rent floor. The detail names the maximum withdrawable.
  • E_STORAGE_WITHDRAW_EXCEEDS_RESERVE: an account-reserve withdraw exceeds the reserve.
  • E_STORAGE_AUTO_TOP_UP_INVALID: a Custom auto top-up with a zero cap or target_epochs 0.
  • E_STORAGE_GROWTH_BELOW_FLOOR: a commit that grows a volume left escrow below the rent floor of the new size after the reserve draw. The detail names the floor, the escrow and the shortfall.
  • E_VOLUME_PARAM_INVALID (receipt code 2002): erasure k outside 2..=16 or m outside 1..=8 (the retained k+m ≤ 32 check is a metering-bound parity guard), unknown visibility or wrapping_key_policy, a volume_name violating the [a-zA-Z0-9_\-.] / 64-byte rule (§6.1), or an explicitly supplied relay_nodes list whose length is not erasure_k + erasure_m or that repeats a node_id (§5.3).
  • E_VOLUME_NOT_FOUND_FOR_OWNER: a name lookup missed the index for this owner-domain (§11.3). This MUST NOT be reported as a bare 404 that hides whether the caller queried under the wrong owner.
  • E_VOLUME_RELAY_FLOOR (receipt code 2005): auto-assignment finds fewer than erasure_k + erasure_m distinct usable relays (§5.3). An explicit list of the wrong length, or with a repeated node_id, fails earlier as E_VOLUME_PARAM_INVALID (§5.3). Creation MUST be refused and never half-completed, because each shard index needs its own relay (§5.4.1).
  • E_VOLUME_RELAY_UNKNOWN: an explicitly supplied relay_nodes entry names a node_id the relay registry has never held. relay_nodes[shard_index] is the per-index responsibility that PoR challenges (§7.6) and rent assignment intervals (§5.6.1) bind to, and a challenge bound to an unknown node settles with only a miss alarm. Creation MUST reject unknown identities and not rely on a later epoch-sweep replacement (§5.6.1). The explicit branch requires registry membership only, without the Active-and-has-room filter used for automatic assignment. The check promises nothing about later status, capacity or possession. The epoch sweep later replaces missing or non-Active assignments on ACTIVE volumes with an Active, non-full relay and starts its PoR repair grace (§5.6.1); if none fits the eligibility and work-budget checks, the assignment remains. An assigned relay that is full does not by itself trigger this rotation. A full relay’s index is owed at write time (§5.4.1). An owner can still name a registered relay it does not own, without that relay’s consent. §5.6.2 addresses this defect.
Each error maps to a stable code and string, so the CLI and SDK can branch on it without string-matching a generic message.

13. Garbage collection

13.1 Triggers

The protocol garbage-collects volume data when:
  1. The account owner calls delete_volume() (soft-delete plus a grace window, §13.2) or delete_object().
  2. Escrow plus the account storage reserve stays below the volume’s rent floor (CIP-31 §3) through STORAGE_GRACE_EPOCHS, and settlement funding cannot restore it (§10.3).
There is no expiry_height; escrow and rent alone decide how long a volume lives.

13.2 Process

13.3 Deletion semantics

  • delete_object has no onchain instruction of its own. It submits a manifest commit and uses that commit’s CIP-43 access row. On a Public volume with a nonzero CIP-33 Store pin count, any object deletion that changes the committed manifest is rejected (§2.1.2 of CIP-33). Otherwise the commit removes the object from the onchain manifest, CapToken holders can no longer reach it, and Relay Nodes garbage-collect its shards asynchronously.
  • delete_volume on a Public volume with a nonzero CIP-33 Store pin count is rejected (§2.1.2 of CIP-33; the pinned Trading Post implementation cited in §7.3.1). Otherwise the volume enters DELETED (soft-delete): all active CapTokens are revoked, no new ones can be issued, and storage fees keep accruing for VOLUME_DELETE_GRACE_EPOCHS (default: same as STORAGE_GRACE_EPOCHS). During this window the owner may call undelete_volume() to return the volume to ACTIVE.
  • After the grace window the volume moves to GARBAGE_COLLECTING. Both PoR sampling builders exclude it before drawing, even while its native rows remain. The epoch sweep prunes acknowledged event history.
  • The event-prune gate is passed once the volume’s acknowledged event history has been pruned. After that, a separate garbage-collection pass over the chain’s native inventory (the native-inventory GC lane) advances across blocks, doing a bounded amount of work in each. Each item deletes at most one stripe row and its dense stripe ordinal and durably decrements live_shard_count by w. The lane’s cursor lives in consensus state, survives restart and rotates among volumes in GARBAGE_COLLECTING. With healthy queue state and no rows frozen by a fault, a volume near the 1,048,576 live-shard-ref cap drains across blocks without waiting one daily rent epoch per batch.
  • Rows frozen by a fault stay in the queue until a reviewed recovery. Corruption that cannot be traced to one volume blocks all native-inventory cleanup at the head of the queue until repair, as CIP-31 specifies. Healthy native GC MUST NOT block later rent epochs or settle one epoch twice.
  • Committed-data faults follow CIP-31 native inventory GC fault isolation. A fault that belongs to one item quarantines only that item’s native cleanup: its records are kept and automatic processing stops touching it. Undecodable queue rows and faults in shared control state stop processing with the original savepoint intact. Infrastructure failures propagate. If adding to the queue hits corrupt shared GC control state, the storage cursor is kept and storage settlement waits for repair while other kinds of sweep work continue.
  • The StorageCommitment stays in GARBAGE_COLLECTING while any native row remains. Control-plane cleanup removes the volume from the live index only after the count reaches zero and the event-prune gate is satisfied. Relay Nodes may purge the volume’s bytes without waiting for its native rows to drain (§12.2). Storage fees stop under the terminal status.
  • The StorageCommitment survives removal from the live index as a terminal volume_id tombstone. CreateVolume rejects an occupied ID regardless of status, so garbage collection never makes an ID reusable.
  • Rent exhaustion can still move a pinned Public Store volume to GARBAGE_COLLECTING and remove its bytes. The Trading Post pin count (in separate actor storage) and the RAS ID tombstone both survive, and neither preserves the bytes. CIP-33 §2.1.2 records the unresolved availability and OneShot serving rule.
  • Garbage collection is irreversible. Once a volume enters GARBAGE_COLLECTING, its data cannot be recovered.
Event pruning stops after MAX_PRUNE_PER_PASS = 1,024 events for one volume in each daily storage epoch. A volume with a large acknowledged event history can therefore wait many epochs before the native-inventory GC lane becomes eligible, and a live volume that adds more than 1,024 events per day can grow its backlog without bound. Closing the gap needs:
  • A durable, fair per-block event-prune queue. A queue alone does not bound the backlog: the current sweep handles at most 32 items per block, shared with other work, and valid manifest traffic can create more event-prune work than that.
  • Before any claim of indefinite production readiness, a separate CIP-10 follow-up that bounds outstanding prune work with deterministic commit admission and backpressure, reserves prune capacity when a staged commit is accepted so that a full prune queue alone never blocks that commit at finalize, and makes temporary admission refusals automatically retryable by clients.
  • A recovery path that safely removes the need for a signed ACK from a relay that was assigned when the event was emitted and is now permanently unavailable. The prune gate still requires that relay’s signed ACK, even after eviction, assignment rotation or terminal volume status.
The queue, admission bound, client retry and lost-relay ACK recovery are proposed work. The native-row lane deletes rows only after the volume’s events are pruned and does not remove this earlier gate.

13.4 Ownership transfer

The account owner may transfer a volume to another Cowboy account with transfer_volume(volume_name, new_owner). Transfer:
  • Revokes all active CapTokens. It advances last_owner_transfer_block, and a token whose valid_from is below it is refused. It clears mount_allowed_actors in the same transaction.
  • Updates StorageCommitment.owner atomically. The new owner takes on billing from the next epoch.
  • TransferVolume.new_owner_wrapped_dek MUST be present if and only if the commitment already carries an owner wrap. When present, it MUST decode as a valid OwnerWrappedDek (§9.1), with the required version, algorithm and exact size and no trailing bytes. It re-encrypts the owner wrap to the new owner’s wallet-derived key, so the old owner must cooperate to unwrap and re-wrap. A commitment without an owner wrap MUST NOT gain one through transfer.
  • Sets cbss_rewrap_required if a committee wrap is present, which blocks committee-seal delivery until the new owner calls RewrapVolume (§9.4.1). A new dispatched grant from the new owner enables runner access without a rewrap (§9.2).
  • Rewraps the DEK without rotating it (§9.4). An old owner who held the DEK can still decrypt everything written before and after the transfer, and the new owner has no protocol remedy (§9.4.2). Public volumes have no DEK to rewrap.
  • Is rejected for a volume in DELETED or GARBAGE_COLLECTING status, and for a Public volume with a CIP-33 Store artifact pin. The pin outlives artifact retirement while a pinned hire can execute; CIP-33 §2.1.2 defines its release. The hire-revocation path releases the hire-lifetime reference only at its terminal transition.

13.5 Gateway HTTP serving by status

A public-volume Gateway (CIP-15) maps StorageCommitment.status to HTTP behavior using the existing lifecycle states. There is no DELINQUENT status. The Gateway keeps serving in GRACE_PERIOD because the owner may top up at any moment. DELETED is recoverable during the deletion grace window but is not served. GARBAGE_COLLECTING is irreversible, so the Gateway returns 410 Gone.

14. Parameters

Object chunk sizes

object_chunk_bytes(k, m) is 32 MiB for K+M ≤ 3, 32 MiB for widths 4–6, 64 MiB for 7–12, and 128 MiB for widths ≥13. CBFS-supported grids have 2 ≤ K ≤ 16 and 1 ≤ M ≤ 8, so K+M ∈ 3..24. These object chunks are separate from the small fixed-size chunks PoR uses inside a shard. Shards stay below MAX_SHARD_TRANSFER_SIZE = 256 MiB.

Chunk-tier compatibility

SDK and CLI readers accept the current tiers and, for K+M=3 only, the older 16 MiB tier, and validate each object against its stored chunk sizes. Each object must match one complete allowed tier with uniform visibility overhead: zero for public data, or one 16-byte AEAD tag per private chunk. Ranged reads, including FUSE cache reads, map byte offsets to chunk indexes using the descriptor’s validated plaintext tier, excluding any AEAD tag. Writers always use the current tier, so a reader without 32 MiB support cannot read a new K=2,M=1 object over 16 MiB. Readers must be upgraded before new writers are enabled, including on standalone and dev stores (which have no wipe) and for relay manifest blobs and manifest-root records (which survive the chain-derived-state wipe).

Staged-manifest funding

Current signing funds the source-derived auth-v3 staged-manifest reservation, including pending-epoch and route context. Its structural capacity is 1,116.

Historical object-fit audit

The fit costs, headroom and examples below use the pre-auth-v3 rent-inclusive formula and a capacity of 1,114. “Then-current-tier” means the object tiers this audit measured. Every legal then-current-tier object up to 1 GiB, for either visibility and every CBFS-supported K/M grid (2 ≤ K ≤ 16, 1 ≤ M ≤ 8), fits create, delete and replace in one atomic manifest commit, as measured by the rent-inclusive stripe formula in CIP-43 §5.5. This holds for path-local updates under the single-path model: manifest depth 1–3 with no DAG repartitioning, one rewritten node per level, and sufficient pending-epoch headroom. An object’s shard set is atomic; staging can carry a large delta but never splits one object’s publication across commits. Multi-object batches still need a complete-delta check: in volumes of many multi-chunk objects, one insert or delete can repack every following manifest leaf and exceed one commit even at shallow depth. Before signing, clients refuse a change set whose cost one commit cannot cover. Take a path-local update to a then-current-tier 1 GiB object, with manifest depth 1–3 and no DAG repartitioning, relay-delta count D ∈ {w, 2w, 64}, and no queued storage epochs or native-route surcharge. The worst signed structural costs are 644 create, 772 delete, and 1,084 replace, below the 1,114 ceiling. Empty-volume creation costs at most 580. The worst replace leaves 30 units. At 12 units per queued storage epoch, it admits at most two queued epochs, including the native-route surcharge of two units where applicable. With three or more queued epochs, a near-cap replace waits until the queued epochs are settled and is checked again before signing; the 1 GiB object ceiling is unchanged. Sponsored commits keep their single-stage ref limit (§12.2.1). Manifest depth 4 is possible within the object-count cap, and because manifest boundaries depend on both path and descriptor, replacing an object can split manifest nodes. For then-current-tier objects at widths up to 6 and D=64, a depth-4 replace with no DAG repartitioning costs 1,108 structural accesses before context surcharges, leaving 6 units and admitting no queued epoch. At that earlier revision, a depth-3 replace with two extra added DAG nodes cost 1,112 and fit that revision’s 1,114 capacity. A depth-3 replace with three extra added DAG nodes costs 1,122, and a depth-4 replace with one extra added DAG node costs 1,126. Both exceed 1,114 even with no queued epochs. The client refuses such a mutation before signing; the owner can delete and recreate the object in separate commits if the pre-signing cost check (preview) of each commit passes, and the object is absent in between.

Legacy mutation budgets

K=2,M=1 objects with 16 MiB chunks stay readable, but the atomic fit guarantee applies only to then-current-tier objects. Such a 1 GiB object has 64 chunks. Replacing it with 32 then-current-tier chunks and one rewritten manifest leaf gives Ac=33,Rc=65, with w=3,H=1,D=3,K=0,J=0, so F′=1,370 exceeds 1,114. The SDK keeps all old chunks in the removal delta and refuses before signing. A deletion with Ac=0,Rc=65 in the same H/D/K/J context costs 1,038 and fits, though deeper or repartitioned manifests or queued epochs can change that. An over-budget replacement can be done as a delete and a write in separate commits when both previews pass. Reading such an object needs no rewrite. Before reading or uploading object data, the SDK and CLI check the current staged preview and the incoming creation lower bound with fresh queued-epoch and authority context (K, J in CIP-43 current staged-manifest funding), and check the actual delta again before signing. Governance proposals may adjust these parameters.

15. Security considerations

Private volumes, CBSS account secrets and Cowchat room keys are protected against classical attackers only. Owner ECDSA-derived wrapping, HPKE X25519 and committee BLS IBE do not resist a cryptographically relevant quantum computer. Algorithm tags make the format extensible but do not upgrade existing ciphertext or protect ciphertext harvested today. No post-quantum suite is enabled; one would need its own implementation and key and data rotation policy. Committee envelopes keep CIP-24’s tag 2 (the current BLS12-381 IBE/HKDF/AES-GCM suite). Unknown tags must reject.

15.1 CapToken forgery

A CapToken’s authority is the copy that the issuing system instruction writes into Storage Manager state (§7.1); a presented token is honored only if it matches that copy byte for byte. Forging one therefore requires writing Storage Manager state, which is equivalent to compromising consensus. The nonce is derived at issuance as the first 8 bytes, read little-endian, of keccak256("cip9:cap-token:" ‖ "nonce" ‖ vrf_seed ‖ job_id ‖ runner_address ‖ u64_be(attachment_index) ‖ volume_id), so it cannot be predicted before the VRF seed is final. Replay protection is the byte-for-byte match against the stored copy plus valid_until, which is fixed in that copy, so a Runner cannot extend a token.

15.2 Runner compromise

A compromised Runner with a WRITE_ONLY token can write garbage, consume the account’s byte quota, and write shards outside its path prefix, which Relay Nodes cannot detect because shard IDs are opaque (§7.3.1). On a private volume it cannot exfiltrate data, but only because the Relay Node refuses GET_SHARD and GET_PLACEMENT: the mount already holds the DEK (§9.2), and the client-side refusal runs inside the container the Runner controls. A public volume has nothing to hide. The mount also learns the object namespace through its manifest refresh (§12.1.3), so “write-only” limits what the Runner may change and not what it may see. Mitigations:
  • Byte quotas limit the damage per job.
  • A prefix-scoped writer cannot commit. It stages, and a full-volume holder reviews and publishes the delta (§7.3.1).
  • Shards that no committed manifest references are garbage-collected after ORPHAN_SHARD_TTL_SECS.
  • The token has no delete access, so existing data cannot be destroyed.
A compromised Runner with a READ_ONLY token can exfiltrate data within its scope; TEE attestation and path-prefix scoping are the only mitigations. READ_ONLY still beats READ_WRITE for jobs that only consume data, because it removes the garbage-write attack. A compromised Runner with a READ_WRITE token can do both. Mitigations: require TEE-attested Runners (CIP-2 tee_required=true) for sensitive workloads, and scope path_prefix to the sub-path the job needs. READ_WRITE is an explicit opt-in.

15.3 Relay Node compromise

Relay Nodes hold ciphertext shards (plaintext for public volumes) and cannot read private data without the volume DEK.

15.4 Denial of service

15.5 Key management

  • Each volume DEK is wrapped per its access class (§9.1). If the owner loses the wallet key, the owner-wrap path is gone, but a committee-wrapped volume stays readable through an authorized runner. The wraps live onchain in the StorageCommitment, so only the ability to unwrap a given wrap can be lost.
  • The Dispatcher never holds a plaintext DEK (§9.2), so it cannot exfiltrate one. A compromised Dispatcher could mis-authorize a seal request, a risk bounded by the same consensus-level trust already required to issue CapTokens.
  • RewrapVolume (§9.4.1) and the transfer rewrap (§13.4) keep the same DEK and revoke nothing, and this CIP does not permit either to be described as a security boundary. DEK rotation does not exist (§9.4.2). After a compromised owner key, or a compromised grantee that already decrypted its package, recovery means creating a new volume and re-encrypting the data into it by hand.
  • Removing a grant (§7.7) deletes its grantee_wrapped_dek and stops new CapTokens. That fully withdraws a grantee that never used its package, but cannot reach a key already decrypted, and the permanent path_tag history stays linkable under the DEK.

16. Implementation notes

16.1 FUSE mount implementation

The FUSE mount translates POSIX operations into CIP-9 object operations through three components:
  1. The FUSE daemon handles read, write, readdir, getattr and the other filesystem calls by delegating to the local cache.
  2. The local cache is an in-process memory cache (stub, clean and dirty entries) with a shared size budget. Every read and write hits it first. It fills lazily from Relay Nodes on first access, and eagerly for locally modified files.
  3. The sync daemon runs the push/pull loop every sync_interval, detecting local and remote changes and doing encryption, erasure coding and shard distribution.

16.2 Object API client library

Without a FUSE mount, the Runner SDK exposes the Object API directly:

16.3 Relay Node implementation

A Relay Node runs these subsystems:
  1. The blob store maps (shard_id, shard_index) → shard_bytes. A shard_id is an opaque BLAKE3 hash (§5.3), so the node never sees object paths (§9.3).
  2. The placement store maps shard_id → PlacementRecord, kept apart from shard data so repair can read placement without the encrypted manifest.
  3. The RPC server accepts shard operations (PUT_SHARD, GET_SHARD, PROVE_SHARD) and placement operations (PutPlacement, GetPlacement, ReplicatePlacement). The AuthProvider authenticates client-facing operations (§7.1.1); node-to-node operations (§7.1.2) are authorized by relay assertion. For public volumes, GET_SHARD and GetPlacement need no CapToken (§7.6.3), but writes do. There is no listing operation; clients list objects from the manifest, and requests are keyed by shard_id, never by path.
  4. The repair loop runs the two-phase repair cycle (§5.5) against a peer list seeded from config and updated as the node discovers peers.
  5. Placement sync replicates PlacementRecords to the other assignees and gossips peer lists via SyncPeers, both as relay assertions (§7.1.2).
  6. Registry upkeep: there is no heartbeat instruction. The relay’s registry status is its liveness (§5.2), and UpdateRelay changes its advertised profile.
  7. The spot-check responder returns a random chunk of a named shard when challenged.
A Relay Node needs stable uptime, connectivity and disk, and no GPU, TEE or high-compute hardware.

16.4 Performance expectations

Expected latencies for the direct Object API: Expected latencies for the FUSE mount:

16.5 Runner ↔ validator wire compatibility

Runners submit transactions the validator must decode: heartbeats, job results and manifest commitments. A runner built against an older node can emit an encoding the newer validator cannot decode (failed to decode transaction submission); the runner then polls forever and completes nothing, with no visible failure. The current release uses version-3 commonware-codec transaction bytes (CURRENT_TX_VERSION = 3) and their shared verifier (CIP-11 §10.3); actor CBOR payload encoding does not define the envelope. CIP-9 requires:
  • A runner’s protocol types MUST be compatible with the deployed validator. A runner outside the validator’s supported wire range is rejected with the typed error TxUnsupportedVersion (HTTP 400) returned to the submitter, never a silent drop or a generic decode error.
  • Wire-format changes to runner↔chain transactions MUST be versioned with a discriminant the validator checks. The current coordinated fresh-devnet release accepts only its current codec version; all runners and validators MUST use the same release before producers resume. There is no rolling-upgrade compatibility window.
  • Decode-reject reasons MUST be observable to the runner operator, so an operator whose runner cannot complete jobs because of a wire-format mismatch can find out why.

16.6 Client bootstrap and discovery

A conforming client needs only the node RPC URL and discovers everything else from chain; flags and environment variables only override discovered values.
  • chain_id, chain_instance_id and network come from the closed /chain-info response. Operational parameters come from /chain-parameters, used only after its nested identity matches. The basefee schedule comes from GET /basefee and suggested_max_fee_*, never from hand-set flags; a wrong chain_id or fee fails certificate auth or is dropped by the mempool (fee_below_basefee).
  • Storage relays come from the relay registry (§5.2), and DEK delivery from chain (§9.2): a dispatched grant’s package from the commitment, and for committee-seal delivery, the committee from the CBSS registry (CIP-24) by epoch, with the sealed ciphertext held by the registered relays.
  • The RPC URL MUST be resolved once at startup and passed to every component that needs it. A subcommand MUST NOT silently fall back to a hardcoded default (e.g., 127.0.0.1:4000) or ignore an explicit --rpc-url. A missing or unreachable RPC is a clear error.

17. Scope and future work

17.1 In scope (this CIP)

  • Private and public account-scoped volumes.
  • READ_ONLY, WRITE_ONLY, and READ_WRITE CapToken access modes; PUBLIC volume visibility mode (§7.6).
  • Concurrent CapTokens on the same volume (agent swarm pattern).
  • Two client interfaces, the FUSE filesystem mount (for agentic/LLM workloads) and the Object API (for programmatic workloads).
  • Hybrid sync strategy for FUSE mounts (local cache plus background push/pull).
  • Relay Nodes as a dedicated storage layer with staking and incentives.
  • Reed-Solomon erasure coding (default 4/6) for durability.
  • AES-256-GCM encryption with HKDF-derived keys.
  • BLAKE3 content hashing at object, ciphertext, and shard levels.
  • Path-based addressing with Merkle manifest for integrity proofs.
  • Per-epoch billing with grace period and garbage collection.
  • Lazy and proactive shard repair.
  • Private dual-wrapped volumes with access classes (owner-key, committee-only; tee-gated reserved), and owner-curated cross-owner mount allowlists that also authorize direct-client access under the grantee’s own delegation, with recipient-sealed DEK delivery (§7.7, §7.7.1, §9.2.1).
  • Control-plane writes as client-signed account transactions (§12.2.1), with client discovery of chain parameters, relays and CBSS endpoints from chain (§16.6).

17.2 Explicitly out of scope

  • Request-level possession proof for owner tokens. Owner and grantee tokens are bearer tokens after issuance, bounded by the §12.2.1 TTLs. A request-signing extension binding each relay request to the delegated key would cover both token kinds (§7.7.1).
  • Content-addressed retrieval. RAS addresses by path only; CID-based retrieval could be layered on top.
  • A storage marketplace where Relay Nodes bid on deals. The protocol sets pricing.
  • Storage backends other than CBFS (Filecoin, Arweave and others), which would need a follow-on standard.
  • READ_UNCOMMITTED consistency, which needs a shard discovery mechanism this CIP does not define (§7.5).
  • Rotation without re-encryption, which revokes nothing and would let a commitment assert a rotation that did not happen. Proxy re-encryption or any rewrite-free re-key would be a follow-on standard. DEK rotation itself is in scope and is an unfilled gap (§9.4.2–§9.4.3).
  • Runner-initiated task dispatch. Only onchain Actors dispatch tasks; this is a CIP-2 extension.
  • A full container runtime spec (images, registries, resource limits, GPU passthrough, network policy). CIP-9 defines the storage layer, FUSE mount and object API; a separate CIP defines the runtime.

17.3 Implementation status

  • Deletion that protects newly published bytes (§12.2) is implemented. Relays apply publication holds, the current-set check and a publication guard on every relay write and delete.
  • The canonical owner-authorization signing path (§12.2.1) is implemented. Owner-auth, owner-delegation and relay-auth signing bytes come from one canonical encoding shared by the validator and clients, and the owner-delegation verifier binds chain_id to the running chain.
  • wrapping_key_policy (§9.1) is pending. The access-class field does not exist anywhere in the node, so §9.1’s owner-key / committee-only / tee-gated table is a target, and no normative rule may be conditioned on reading it from state. owner_wrapped_dek exists on StorageCommitment, on the create request and on the open/mount response, alongside the committee wrapped_dek.
  • DEK rotation (§9.4.2) has no design; RewrapVolume is live as §9.4.1 describes. §9.4.3 lists the obstacles any rotation design must solve on the shipped chain.
  • The CapToken signature field (§7.1) is reserved for a future client-verifiable token form and is zero today.

Appendix A: Worked examples

The examples use the actor SDK. An actor dispatches a container job from a @runner.continuation handler with runner.container(image, command, volumes=[...]). Each volume entry is a name (read-only) or a (name, mode) pair with mode "ro", "rw" or "wo", and the Runner FUSE-mounts each volume at /mnt/volumes/{name}/. The Runner runtime does the encryption, erasure coding and Relay Node traffic. The workload uses ordinary file operations, and a mount pushes and commits its changes when the container exits.

A.1 AI agent with persistent memory

A trading agent runs as an LLM loop inside a container with a read-write mount:
Inside the container, the model uses its ordinary file tools and never sees CapTokens, shards or Relay Nodes:
The next run mounts the same volume and continues from the saved state.

A.2 Distributed scraping with map-reduce

Two WRITE tokens with an empty prefix cannot be active on one volume at once (§7.3), so each scraper writes to its own volume. The actor invokes this handler once per site, and each invocation dispatches one scraper with a write-only mount:
Each scraper is a plain script that writes one JSON file per page into its mount, for example /mnt/volumes/scrape-0/about-page.json. Its writes commit when the container exits. After all scrapers finish, the actor dispatches a collator that reads all five volumes and writes to a results volume:
The collator’s model explores the data with Unix tools:

A.3 Agent swarm with a coordinator

Five sub-agents each write a report to their own volume with a write-only mount. A coordinator mounts all five read-only and its own volume read-write, and reads each report as it lands. Reads are READ_COMMITTED (§7.5), so the coordinator sees a sub-agent’s files only after a commit containing them lands. A sub-agent’s output can land across several commits while its container runs (§12.1.3), and the remaining staged files land when it exits; a sub-agent that needs its report to appear whole calls fsync() on every report file, writes a completion marker only after all of those succeed, and then calls fsync() on the marker. The coordinator waits for the marker. The actor invokes coordinate once and research once per agent, and each invocation dispatches its own job:
A sub-agent’s tool calls:
The coordinator’s tool calls:
The coordinator needs no polling API beyond ls and Read.

A.4 Multi-stage pipeline with handoff

The two stages run one after the other on one volume, so only one WRITE token is active at a time. Stage 1 preprocesses data:
After the Stage 1 callback, Stage 2 runs inference on the same volume:

Appendix B: Comparison with existing systems


Appendix C: CapToken wire format

The diagram is a non-normative schematic and does not show the runner-token transport layout. The runner CapToken hash and signing preimages (cap_token_hash and cap_token_signing_digest) encode access mode as one byte using ReadOnly = 0, WriteOnly = 1, ReadWrite = 2, matching §8.1. The encode_cap_token transport is the runner-token tag followed by fixed-integer, little-endian bincode serialization of CapToken. Its access-mode enum discriminant is u32le with WriteOnly = 0, ReadOnly = 1, ReadWrite = 2. Do not use the hash-preimage order to encode that transport. The RAS MountGrant commonware wire, owner-action preimage and grant AAD use the one-byte RAS order specified in §9.2.1. The signature field is reserved and zero. Token authority is the byte-for-byte match against the copy in Storage Manager state (§7.1).

Appendix D: “Why won’t my volume mount?”

A storage-attached job must clear six independent gates, in order, before a Runner executes it. A conforming implementation MUST make each gate emit a distinct, structured reason. A job that fails one gate never reaches the next. Implementations MUST report which gate stopped the job instead of collapsing all six into a generic “job did not dispatch”.