​

CIP-9: Runner-Backed Storage

​

1. Abstract

• Account-scoped: All storage is owned by and billed to a Cowboy account (EOA or Actor).
• Private by default, public by choice: Private volumes are encrypted client-side before leaving the Runner; only the owning account can decrypt. Public volumes (visibility = PUBLIC) are stored unencrypted and readable by any party without a CapToken, enabling use cases like web asset hosting (CIP-15) and public datasets.
• Access-controlled: Runners receive scoped capability tokens granting read-only, write-only, or read-write access to a storage volume. Multiple concurrent CapTokens may be active on the same volume simultaneously. Public volumes require CapTokens only for writes, not reads.
• Addressable: Every stored object has a deterministic path within the account’s storage namespace, enabling targeted reads and deletes by the account owner.
• Durable: Objects are erasure-coded (Reed-Solomon) and distributed across multiple Relay Nodes, tolerating node failures without data loss.
• Metered: Storage is billed in CBY via a per-epoch, per-byte fee model (including erasure coding overhead) that extends the Cells concept from CIP-3 to persistent off-chain blobs. Rates are denominated in nano-CBY and set by CIP-31.

​

2. Motivation

1. Secret management. Runners need access to API keys, certificates, and credentials without exposing them on the public chain. These secrets must only be decryptable by authorized Runners, optionally gated by TEE attestation.
2. Large off-chain output. Computation frequently produces artifacts exceeding the 64 KiB onchain inline cap (whitepaper §7). LLM inference may generate images, audio, datasets, or model weights that are too large for result_data and inefficient to pass through onchain callbacks. These must be stored off-chain with a verifiable content commitment anchored onchain.
3. Multi-step data flow. Iterative computation (AI training loops accumulating weights across invocations), stateful agents (conversation history, tool outputs, learned preferences persisting across scheduling cycles), pipeline handoff (Runner A produces intermediate data consumed by Runner B), and agent swarms (coordinator + concurrent sub-agents sharing a volume) all require persistent, addressable storage between jobs.
4. Container persistence. CIP-10 container runtimes require persistent storage layers for stateful applications, databases, and caching across job restarts. Without attachable storage, containers are limited to ephemeral scratch space.

​

2.1 Why not existing systems?

​

3. Definitions

• Volume: A named, account-scoped storage namespace. An account may own multiple Volumes. Each Volume is an isolated container for stored objects.
• Object: A single blob of data stored within a Volume, identified by a path key.
• CBFS: The canonical off-chain storage engine used by this CIP. CBFS nodes implement the Relay Node data plane and CBFS clients implement the object API, manifest handling, and FUSE mount described here.
• Storage Commitment: An onchain record that tracks a Volume’s existence, owner, size, shard placement, creation epoch, and billing state.
• Capability Token (CapToken): A cryptographic bearer token encoding the permissions (read-only, write-only, or read-write), scope (volume + path prefix), time bounds, and size quota for a client’s access to a Volume. The client may be a Runner during a job, the owner via the CLI/SDK, or any other authorized caller.
• Relay Node: A CBFS storage node participating in the Cowboy network that persistently stores erasure-coded shards of Volume data. Relay Nodes are a distinct network role from Runners and Validators, with their own staking and incentive model.
• Shard: A fragment of an erasure-coded object. An object is split into K data shards and M parity shards (K+M total); any K shards are sufficient to reconstruct the original object.
• ObjectDescriptor: The immutable content-identity record for a stored object. Contains the path, content hash, ciphertext hash, encryption nonce, size, erasure parameters, and per-shard hashes. Stored inside the encrypted manifest via ManifestEntry::File.
• PlacementRecord: The mutable shard-to-node assignment record for a stored object. Contains a shard_id, a list of PlacementAssignment (shard_index → node_id), duplicated erasure params and shard hashes (so repair workers can operate without manifest access), a ciphertext_size (the pre-padding ciphertext length needed for correct erasure reconstruction), and a CAS version for atomic updates. Replicated to all participating Relay Nodes independently of the manifest.
• Volume DEK: The per-volume AES-256 data-encryption key. It is wrapped one or more ways according to the volume’s access class (§9.1) — to the owner’s wallet-derived key, to the CBSS committee, or both.
• CBSS: The Cowboy Secret Service — the threshold committee that holds the IBE capability to unwrap a committee-wrapped Volume DEK on behalf of an authorized Runner (§9.2).
• Write-Relayer: The cowboy-ras-write-relayer service, which submits owner-signed RAS control-plane transactions (volume create, manifest commit, allowlist updates) and pays their gas on a client’s behalf (§12.2.1).
• Dispatcher: The Job Dispatcher system actor (0x02), defined by CIP-2 — the in-consensus job-dispatch logic that selects Runners (VRF + committee filtering), issues and revokes CapTokens, and manages job lifecycle. It is not a separate off-chain service: its actions execute inside block processing at consensus trust level. In CIP-9 the Dispatcher additionally applies the storage-capability filter (§5.1.1), scopes a CapToken to a mount grant for cross-owner access (§7.7), and constructs and authorizes the CBSS SealRequest for private-volume key delivery (§9.2) — it never handles a plaintext DEK.

​

3.1 Canonical Manifest DAG

1. The manifest is the set of ManifestEntry (cbfs/types): File { descriptor, metadata }, Symlink { … }, Directory { … }, ordered lexicographically by path() (UTF-8 byte ordering).
2. Entries are grouped into leaf nodes by content-defined chunking: once a node holds at least MANIFEST_NODE_MIN_ENTRIES (64) entries, a boundary is cut at the first entry whose domain-separated hash hits the target — BLAKE3("cbfs.manifest-entry-boundary.v1" || encoded_entry) with its low MANIFEST_NODE_TARGET_BITS (10) bits all zero (≈1024-entry target) — and is forced at MANIFEST_NODE_MAX_ENTRIES (2048) or MANIFEST_NODE_MAX_BYTES (256 KiB). This is a per-entry hash test, not a rolling hash. A Leaf { entries } carries its slice of the sorted entries.
3. Leaf nodes are reduced into interior nodes the same way (boundary domain cbfs.manifest-child-boundary.v1); an Interior { children } carries ManifestChild { min_path, locator, subtree_count } pointers. Levels reduce bottom-up until one node remains.
4. Each node is content-addressed by its locator — the BLAKE3 hash of its canonical encoding (magic CBMN for plaintext nodes, CBME for encrypted envelopes). The root is the locator of the top node; manifest_root is that value.

​

4. Design Overview

​

4.1 Architecture

​

4.2 Lifecycle

1. Create: Account owner creates a Volume via the Storage Manager system actor, specifying a name, optional size quota, and replication parameters. An onchain Storage Commitment is written.
2. Attach: When submitting a CIP-2 task, the owner includes one or more volume attachments in the task definition, each specifying the volume name, access mode (read-only, write-only, or read-write), and an optional path prefix scope.
3. Authorize: The Dispatcher (or a delegated Storage Manager) issues a CapToken to each selected Runner. The token is scoped to the specified volume, access mode, path prefix, job duration, and byte quota.
4. Write: The Runner encrypts object data, erasure-codes it into K+M shards, and distributes shards to Relay Nodes. Shards are immediately available for retrieval by other CapToken holders on the same volume.
5. Read: The Runner fetches any K of K+M shards from Relay Nodes, reconstructs the object, decrypts, and verifies the content hash.
6. Commit: At job completion (or periodically), the Runner commits a storage manifest — a Merkle root of all objects written — to the onchain Storage Commitment.
7. Manage: The account owner can list, read, and delete individual objects or entire Volumes at any time via the Storage Manager.

​

4.2.1 Committee Composition for Storage-Attached Jobs

​

4.2.2 Dependency on Runner-Backed Continuations

​

4.3 Canonical Implementation Boundary

• Object encryption/decryption for PRIVATE volumes and plaintext handling for PUBLIC volumes.
• Reed-Solomon erasure coding, shard hashing, and shard placement records.
• Relay Node RPCs (PUT_SHARD, GET_SHARD, LIST_SHARDS, placement replication, repair traffic).
• Manifest storage, reconstruction, Merkle root computation, and verification against the authoritative root.
• FUSE mount behavior, local cache, sync daemon, and direct object API.
• Shard repair, garbage collection of orphan shards, and local usage reporting hooks.

• StorageCommitment lifecycle, authoritative manifest_root, and volume ownership semantics.
• CapToken issuance, revocation, and task-scoped attachment semantics.
• Relay Registry membership, staking, health, and repair coordination triggers.
• Billing, fee settlement, storage grace periods, and slashing policy.
• Integration with CIP-2 task submission and the Runner execution flow.

​

4.4 Relationship to Existing CIPs

• CIP-2 (Off-Chain Compute): RAS extends the OffchainTask definition to include volume attachments. The Runner Submission Contract is extended to accept storage manifests alongside result_data.
• CIP-3 (Fee Model): RAS introduces a new fee dimension — persistent storage fees — billed per byte per epoch (including erasure overhead). Unlike CIP-3 Cycles and Cells (which are metered by the VM during transaction execution), storage usage is metered externally by Relay Nodes and settled onchain via attestation-based billing.
• CIP-4 (State Storage): onchain Storage Commitments live in the existing STORAGE key space under the Storage Manager actor’s address. Volume data itself is NOT stored in the MPT trie.
• CIP-10 (Runner Container Runtime): CIP-10 consumes the storage primitive defined here. Container image handling, cgroups, network policy, and GPU passthrough are separate concerns; volume attachment, mount semantics, and the object API are defined by CIP-9 and then mounted into CIP-10 runtimes.

​

5. Relay Nodes

​

5.1 Role and Responsibilities

• Stores erasure-coded shards of encrypted Volume data.
• Stores PlacementRecord entries that map shard IDs to their assigned nodes (see §5.3.1).
• Serves shards and placement records to authorized requesters (Runners with valid CapTokens, account owners).
• Heartbeats to the onchain Relay Registry to prove liveness.
• Runs autonomous two-phase repair (self-heal + redundancy restoration) without Runner involvement (see §5.5).
• Replicates PlacementRecords to peer nodes via ReplicatePlacement RPCs.

​

5.1.1 Runner Storage Capability

• storage_support: bool — the Runner is willing and configured to mount volumes.
• encryption_pubkey: [u8; 32] — the Runner’s x25519 public key, used as the recipient key for sealed DEK delivery (§9.2).

​

5.2 Relay Registry

// Consensus RelayNodeProfile — see node/ras/src/types.rs. The per-block `health` decay
// model was never deployed; liveness is a RelayStatus enum + heartbeat freshness (§5.2 note).
RelayNodeProfile {
  address:              bytes20,
  owner_address:        bytes20,        // wallet that owns/controls the relay; receives settlement payouts
  node_id:              bytes32,        // routing identifier (distinct from address)
  relay_identity_pubkey:bytes32,        // Ed25519 pubkey for handshake / PoR signatures
  status:               RelayStatus,    // ACTIVE | DRAINING | DISABLED | INACTIVE (replaces per-block health)
  endpoint:             string,         // advertised relay endpoint
  capacity_bytes:       u64,            // advertised storage capacity
  used_bytes:           u64,            // consensus-derived from committed manifest assignment deltas, not heartbeat self-reporting
  registered_epoch:     u64,
  activated_epoch:      u64,
  updated_epoch:        u64,            // last profile mutation (heartbeat freshness); replaces last_heartbeat
  drain_epoch:          u64,
  stake_amount:         u128,           // CBY staked
  shards_held:          u32,            // number of active shards
  shards_lost:          u32,            // historical shard losses (reputation)
  metadata_version:     u64,            // CAS version for profile updates
  pending_unstake_epoch:Option<u64>,
  region:               Option<string>, // operator-declared canonical region code
  last_auto_drain_reason:Option<AutoDrainReason>,  // OverCapacity | StaleHeartbeat | InsufficientStake | Governance
  last_auto_drain_epoch: u64,
  auto_drain_blocked_epoch: u64,
  // System-level rebalance visibility (derived from absorption receipts + drain state):
  rebalance_shards_moved:     u64,
  rebalance_shards_remaining: u64,
  rebalance_started_epoch:    Option<u64>
}

• Register: Relay Node stakes MIN_RELAY_STAKE CBY and calls register_relay(capacity_bytes).
• Heartbeat: Relay Node calls heartbeat() periodically, refreshing its updated_epoch. Liveness is tracked by heartbeat freshness, not a per-block health counter (the health-decay model of earlier drafts was never deployed): a relay whose last heartbeat is older than the auto-drain policy’s stale_heartbeat_epochs becomes eligible for auto-drain (AutoDrainReason::StaleHeartbeat, §5.8).
• Removal: A stale-heartbeat or misbehaving relay is transitioned out of ACTIVE (status → DRAINING / INACTIVE) via the auto-drain policy (§5.8). Its shards are migrated / flagged for repair (see §5.5).
• Unstake: A Relay Node may unstake after a cooldown period (RELAY_UNSTAKE_DELAY), provided it has no active shard assignments or has transferred them.

​

5.3 Shard Assignment

1. Eligible set: All Relay Nodes in the active list with status = ACTIVE and sufficient free capacity.
2. Diversity: Selected nodes SHOULD have distinct region_hint values (best-effort, not enforced).
3. Determinism: The initial assignment is recorded in the object’s PlacementRecord (replicated to Relay Nodes, §5.3.1) so any future reader or repair worker knows which Relay Nodes hold which shards.

ObjectDescriptor {
  object_path:      string,
  write_id:         bytes16,          // CSPRNG(16) — fresh random per write, ensures version isolation
  content_hash:     bytes32,          // BLAKE3 hash of plaintext
  ciphertext_hash:  bytes32,          // BLAKE3 hash of ciphertext (pre-erasure-coding)
  encryption_nonce: bytes12,          // random nonce used for AES-256-GCM (unique per write)
  size_bytes:       u64,              // original object size
  ciphertext_size:  u64,              // ciphertext length before erasure padding (needed for correct reconstruction)
  shard_id:         bytes32,          // BLAKE3(volume_id || object_path || write_id) — opaque, version-unique
  erasure_k:        u8,
  erasure_m:        u8,
  shard_hashes:     [bytes32; K+M],   // BLAKE3 of each whole shard (integrity / repair)
  shard_chunk_roots:[bytes32; K+M],   // per-shard chunk Merkle root; the native RAS inventory anchors PoR (§5.6)
}

PlacementRecord {
  volume_id:        bytes32,          // volume binding checked against locally stored shard metadata
  shard_id:         bytes32,          // matches ObjectDescriptor.shard_id
  version:          u64,              // CAS version — monotonically incremented on each reassignment
  assignments: [
    { shard_index: u8, node_id: bytes32 },
    ...  // K+M entries
  ],
  // Duplicated from ObjectDescriptor so repair workers can operate
  // without manifest access (critical for private volumes where the
  // manifest is encrypted):
  erasure_k:        u8,
  erasure_m:        u8,
  ciphertext_size:  u64,
  shard_hashes:     [bytes32; K+M],
  shard_chunk_roots:[bytes32; K+M],   // duplicated so relays can build PoR proofs without the manifest (§5.6)
}

​

5.3.1 Placement Persistence

​

5.3.2 GET_MANIFEST RPC

GET_MANIFEST { volume_id: bytes32, manifest_root: bytes32, erasure_k: u8, erasure_m: u8 }
  → { manifest: bincode(map<path → ManifestEntry>) }

• Status is dynamic; metadata written at commit time cannot carry it. A volume that has since become DELETED or GARBAGE_COLLECTING is indistinguishable from an active one by the bytes on disk. A relay would serve it until its volume-event poller happened to tear the shards down, which is a teardown interval rather than a cache window. The commitment path is not instantaneous either — it reads a bounded-staleness cache — but the two differ by orders of magnitude, so this paragraph would be creating the lag, not closing it.
• GET_MANIFEST is volume-scoped. “The stored shard metadata” has no referent for a request that names no shard.

​

5.4 Erasure Coding

1. Generate write_id = CSPRNG(16) and compute shard_id = BLAKE3(volume_id || object_path || write_id).
2. Encrypt the object (AES-256-GCM, see §9). Record ciphertext_size (pre-padding length).
3. Erasure-code the ciphertext into K data shards and M parity shards using Reed-Solomon (reed-solomon-erasure crate).
4. Compute shard_hash = BLAKE3(shard_bytes) for each shard.
5. Select K+M Relay Nodes and PUT each shard to its assigned node, authenticated by the CapToken. The PUTs are issued concurrently, so the latency is one round trip plus the slowest relay, not K+M serial trips. See §5.4.1 for what happens when one refuses, and for when the write is durable.
6. Produce an ObjectDescriptor (stored in the manifest) and a PlacementRecord (published to Relay Nodes at commit time, §5.3.1).

1. Look up the ObjectDescriptor from the manifest and the PlacementRecord (from local cache or fetched from Relay Nodes via GetPlacement).
2. Request any K shards from available Relay Nodes listed in the PlacementRecord (prefer lowest-latency nodes; fall back if some are unavailable).
3. Reconstruct the ciphertext using Reed-Solomon decoding, truncating to ciphertext_size to remove erasure padding.
4. Verify ciphertext_hash.
5. Decrypt (AES-256-GCM).
6. Verify content_hash.

​

5.4.1 Write Durability

• Integrity is established at write time. The client holds every shard, so it computes every shard_hash and every PoR chunk_root itself and commits them in the ObjectDescriptor. Note two asymmetries. PutShard carries only {volume_id, bytes}, so the receiving relay has no hash to check the bytes against — it stores what it is given. And the manifest-anchored descriptor is the only unforgeable carrier of those numbers: repair, drain and the GetPlacement path all verify against PlacementRecord.shard_hashes, a duplicate copy that carries no client signature and is served by relays.
• Only a client authors an assignment that did not already exist. Relays do write records — repair’s Phase 2 leader CAS-updates one with an incremented version (§5.5 step 5), the drain/rebalance worker builds a successor, and ReplicatePlacement accepts a peer-authored copy under §7.1.2’s rule that the sender already appear in the record’s assignments. What no relay does is introduce an index, or put itself somewhere it was not. The drain push (§7.1.2) is where a relay moves another relay’s bytes, and its containment is worth stating exactly, because the obvious reading is wrong: the old_record a draining relay presents is supplied by that relay, so the “sender is the assignee at this index” check binds the sender to a record it chose — self-consistency, not authorization. What contains it is §7.1.2’s MUST that the sender’s registry status be Draining, together with the destination writing the shard as volume-bound but never committed, so promotion to committed still comes from the volume’s own on-chain add event rather than from the sender’s assertion.
• The distinctness and coverage in the first paragraph are protocol, not an enforced check. validate_placement_record bounds assignments.len() at K + M and each shard_index below K + M; it permits a duplicate shard_index and a duplicate node_id. The writer-side gate is a count, not per-index coverage. An implementation that checks only the count has not met this section’s condition — so the two MUSTs in the first paragraph are normative additions here, not descriptions of an enforced check, and closing the gap means tightening validate_placement_record or the writer.

​

5.4.2 Server-Side Fan-Out (proposed, not implemented)

1. The client encrypts and erasure-codes locally, exactly as today, so it holds every shard’s bytes, hash and chunk-root without uploading them.
2. It selects K + M assignees as today and designates exactly one of them as the ingest relay. (Splitting the K data shards across several ingest relays does not work: Reed-Solomon reconstruction needs K shards, and no such relay would hold K. Sending the full input to several is strictly worse than today for the client’s uplink, which is the thing the proposal exists to reduce.)
3. It uploads the K data shards to the ingest relay, together with the complete assignment list, the per-shard hashes and the erasure parameters.
4. The ingest relay reconstructs the K + M − 1 shards it is not itself keeping, verifies each against the client-supplied shard_hash, and pushes it to the named assignee.
5. The client publishes the PlacementRecord naming all K + M assignees, and the descriptor with all K + M hashes and chunk-roots, exactly as today.

• Accepted — the ingest relay holds the K data shards and has taken on the duty to deliver the rest. Note what this does not mean: the object is not readable. Readers build their fetch targets from the record’s assignments, one address per index, and have no discovery path to a relay holding a shard it is not assigned. An ingest relay assigned one index can serve exactly one shard, so an accepted object is unreadable until at least K assignees have been delivered to.
• Placed — every named assignee holds its shard. This is what §5.4.1 calls durable today.

​

5.4.2.1 Unsolved problems

1. PoR liability does not follow the bytes — and does not today either. A challenge binds responsible_node_id to commitment.relay_nodes[shard_index], the volume-level relay list fixed at create, not to the record’s assignee. §5.6.1 states the four ways the two diverge — there is no configuration in which they reliably agree — so the alternates path of §5.4.1 and every repair Phase-2 reassignment already point liability at a relay that may not hold the shard, and fan-out would widen an existing gap rather than open a new one. Any fan-out design has to resolve which list is authoritative. Tracked as COW-4124, because it is not a fan-out problem.
2. A mis-delivered shard cannot be rejected by the relay that stores it. Because PutShard carries no hash, an assignee cannot reject bytes that do not match the committed chunk_root; it stores them and answers the next challenge with a proof that fails verification, which lands on the fraud row of §5.6 rather than the miss row — an order of magnitude larger. An ingest relay that delivers wrong bytes therefore puts the assignee in the same position as one that delivered nothing: §5.6 records that there is no fraud branch in the implementation — an invalid proof reverts and the challenge folds into the same miss counter — so the two outcomes converge rather than diverge. The accountability problem is unchanged; only its size is.
3. There is no authorization for the push, and one existing MUST forbids it. Neither existing path fits. §7.1.2 states that the sender of a PeerRelayPushShard MUST be Draining, so an ingest relay — which is Active — is not merely unauthorized but structurally excluded; reusing the primitive means relaxing its only ground-truth gate, widening a self-consistency-checked path from “any draining relay” to the whole registry. Reusing ordinary PutShard means handing the ingest relay the client’s CapToken, a bearer credential scoped to the volume, not to a shard, whose path prefix is explicitly not enforced at the relay — i.e. delegating volume-wide write authority, including deletes, to a party the client does not control. A shard-scoped, single-use credential does not exist.
4. An undelivered index does not stay empty — it self-heals, expensively. The assignee holds the record, Phase 1 sees a missing shard and pulls K shards from peers to reconstruct it. So the redundancy recovers, but a single lazy or failed fan-out costs the network a K-shard pull per undelivered index on relay-to-relay links, on top of the fan-out itself. It is client-inducible: one upload to a slow ingest relay provokes it. It also means the reason PoR may not slash is that self-heal usually wins the race — bounded by the repair budget and a 300 s cycle, which is a timing argument, not a guarantee.
5. Surplus copies on the ingest relay become undeletable and unpaid. The ingest relay holds K data shards, but is the named assignee for at most one. When the client commits, the relay’s volume-event poller promotes any local copy matching (shard_id, shard_index) and the volume binding to committed — it does not check node id — and GC’s committed-marker veto then holds those bytes indefinitely. Rent flows only to named assignees, and there is no fan-out equivalent of the displaced-assignee reclaim. A bandwidth favour turns into an unbounded, client-inducible storage liability.
6. Fan-out collapses the one-relay-holds-a-fraction property. Today no relay holds more than 1/K of an object, because alternates are consumed globally so no two shards of one object land on the same relay. An ingest relay holding K data shards can reconstruct the whole object alone — and for a PUBLIC volume the ciphertext is plaintext, so a single relay holds the object in the clear. Combined with (5), it may hold it permanently. Nothing in the current threat model accounts for that.
7. Billing asserts owner storage before it exists. effective_size_delta is committed for all K + M shards, so the owner is billed for placement from the commit. (The relay side is already self-correcting: rent rewards are gated on a valid PoR response, not on holding an assignment, so an assignee that never received its shard earns nothing.)
8. The retry budget moves server-side and is unbounded there. The client’s deadline bounds its own fallback chain. Nothing bounds how long an ingest relay keeps retrying a delivery, how many deliveries it may owe at once, or what it does with bytes for a duty it can never discharge — and a capacity refusal during fan-out has no fallback at all, since the ingest relay cannot rewrite the assignment list.
9. Delivery races reassignment. If Phase 2 reassigns an index while a delivery for it is in flight, the bytes land on a relay the record no longer names — where, by (5), the commit then pins them — while the new assignee holds nothing and self-heals by (4).
10. The client has no evidence it can use. write_object returns a receipt the client itself constructs; there is no relay signature anywhere on the write path, so an ingest relay can acknowledge and discard and no party can prove it. The shape needed already exists on the drain path, where the destination signs an absorption receipt that is settled on chain. Fan-out needs its equivalent at ingest, and again at delivery.
11. The bandwidth accounting is not a wash. With one ingest relay the client sends K shard-units and the relay-to-relay leg is K + M − 1, so the network moves 2K + M − 1 — 9 units at K=4/M=2, against 6 today. Fan-out saves the client M units and costs the relays K + M − 1; it does not re-route a fixed total, it increases it by half. The fee model does not clearly pay for it either: §10.1 charges data transfer “at read/write time” while §10.4 routes it to the relay serving the shard, and neither contemplates a relay-to-relay leg on someone else’s behalf. Worse, the ingest relay chooses the delivery order, so it decides which assignees can answer reads during the accepted window, and therefore who earns those fees.

​

5.5 Autonomous Shard Repair

1. Verify the local shard bytes against the expected shard_hash from the PlacementRecord.
2. If the shard is missing or corrupt, fetch K healthy shards from peer Relay Nodes listed in the same PlacementRecord.
3. Reconstruct the ciphertext using Reed-Solomon decoding, truncating to ciphertext_size (not the padded shard length — using the wrong length produces garbage).
4. Re-encode and extract the needed shard. Verify its hash. Store locally.

1. For each PlacementRecord, probe all assigned nodes. A node is considered dead if it is unreachable after the configured timeout.
2. Leader election: Only the assigned node with the lowest shard_index among live nodes drives reassignment for that PlacementRecord. This prevents multiple nodes from racing to repair the same shard.
3. The leader selects replacement node(s) from the known peer list (excluding already-assigned nodes).
4. The leader reconstructs the missing shard(s) from K surviving shards (same erasure reconstruction as Phase 1) and uploads via PutShard to the replacement node(s).
5. The leader writes an updated PlacementRecord with the new assignments and an incremented version, using a CAS (Compare-and-Swap) write: the PutPlacement RPC specifies expected_version = current_version. If another node raced and already updated the record, the CAS fails and the leader retries from step 1 on the next cycle.
6. The updated PlacementRecord is replicated to all assigned nodes via ReplicatePlacement.

​

5.6 Proof of Retrievability (PoR) Challenges

1. A periodic onchain timer (CIP-5, every POR_CHALLENGE_INTERVAL blocks) selects a random set of (shard_id, shard_index, byte_offset, byte_length, nonce) challenges targeting active Relay Nodes. Both the direct global sampler and the bounded volume-directory builder MUST exclude GARBAGE_COLLECTING volumes before drawing, even while their native inventory rows remain for cleanup (§13.3). The nonce is fresh per challenge. Consensus point-reads the live native RAS inventory row for each selected (volume_id, shard_id, shard_index) and snapshots its chunk_root, shard_size, and publication event sequence in the challenge (§11.1).
2. The challenged Relay Node must respond within POR_RESPONSE_WINDOW blocks with:
   • The requested byte range of the specified shard. The nonce seeds which range is asked, so a relay cannot precompute or replay a prior response.
   • A range-inclusion proof: a Merkle path from the fixed-size chunks (e.g. 4 KiB) covering the requested range to the shard’s chunk-root — the Merkle root over that shard’s chunks. (The relay already builds this chunk tree to serve byte ranges.)
3. The onchain verifier checks against the issue-time snapshot_chunk_root carried in the challenge:
   • the returned bytes hash into the chunk leaves of the range-inclusion proof, which resolves to a chunk-root;
   • that chunk-root equals the selected inventory row’s snapshotted chunk-root;
   • if either fails, the response is invalid.

• Valid response or unconfirmed miss: bond is refunded minus POR_CHALLENGE_FEE (default 7 CBY, retained by 0x0B). A rejected invalid response does not resolve the challenge.
• Confirmed third consecutive miss with slashing enabled: bond is refunded in full, and the challenger may receive CHALLENGER_BOUNTY (default 33 CBY) from the PoR challenge pool, bounded by available pool balance and its epoch cap. The Relay’s RELAY_EVICTION_PENALTY slash is capped by actual stake and distributed under CIP-31 §8. POR_MISS_PENALTY and POR_FRAUD_PENALTY remain governed parameters but are not charged by this settlement path. The slashed-stake 10% share is burned, while the storage-fee 10% share is credited to the Platform Fee Account (0x18).
• Void after shard removal/replacement or terminal GC: the full bond is refunded, with no POR_CHALLENGE_FEE and no bounty.

STORAGE_MANAGER = bytes(19) + b"\x0a"          # 0x0A, left-padded to a 20-byte address

def rearm_challenge_timer(self):               # runs *as* the 0x0A system actor
    pvm_host.schedule_timer_ex(
        pvm_host.block_height() + self.CHALLENGE_INTERVAL,
        b"",                                   # payload: the challenge round is derived at fire time
        fee_payer=STORAGE_MANAGER,             # == actor_address (0x0A) → self-funding exemption
        gas_limit=self.CHALLENGE_GAS_PER_FIRE, # bounded per-fire budget
        # expires_at omitted → protocol-default TTL; the handler re-arms on each fire
    )

​

5.6.1 Who a challenge holds liable

1. The client’s selector reorders. The reference selector rotates a cursor per selection and then re-ranks the result by region novelty. The rotation is the identity only when the pool length equals K + M, and the region re-rank permutes even then whenever the volume’s relays are not all in one region — which the on-chain relay profile supplies, on exactly the chain-integrated path. Opening a volume consumes a selection before any write, so for a volume named with more than K + M relays the first write already starts at a rotated offset. There is no “first write agrees” case.
2. The write falls back. A capacity refusal moves a shard to an alternate and the record names the alternate (§5.4.1). The chain is not told.
3. Repair reassigns. Phase 2 selects replacements from the repairing relay’s own peer table, which is wider than, and unrelated to, the volume’s relay set. A repaired index can leave relay_nodes entirely, on any volume.
4. The list can be short, or point at a relay that has left. Auto-assignment returns however many eligible relays it found; an index past the end of the list has no relay, so the user challenge path refuses it and the timer path binds no responsible node and settles alarm-only forever. Separately, a named relay may since have drained, self-disabled or been evicted, at which point its profile no longer loads and the index is likewise permanently alarm-only.

• The named relay is challenged for a shard it does not hold, cannot answer, and accumulates a run toward eviction.
• The relay that holds the shard is never challenged for that index, so the shard is unaudited.
• The relay that holds it also cannot be credited: a commit may only move bytes for a relay already in relay_nodes or already in per_relay_bytes, so a repaired-to relay is outside the accounting entirely.

​

5.6.2 Before slashing is enabled

• A relay must not be assignable to a volume without its consent, or at least without existing. create_volume accepts an explicit relay_nodes list and validates only its length and endpoint-string length — not that the named relays are registered, Active, or have capacity. Since that list is what a slash binds to, an owner can name any relay’s public node_id, commit shard references for bytes it never delivered, and challenge the resulting index. The relay cannot answer by construction. Tracked as COW-4125.
• A response must prove possession by the relay being credited. The respond instruction takes no sender: any account may answer any open challenge, which resolves it and resets the named relay’s run. For a public volume the shards are anonymously readable, so a relay that discarded every byte can have its challenges answered by anyone who can fetch them. A pass is therefore no more evidence about the named relay than a miss is.
• Per-index liability must correspond to something the chain can observe — §5.6.1.

1. Make placement follow relay_nodes. The chain’s model becomes true by construction. It costs the write fallback and repair’s free choice of replacement — both of which exist because a single unreachable relay would otherwise fail a write or strand a repair — and makes the owner’s create-time list a veto over every future write to the volume.
2. Make relay_nodes mutable by a reassignment instruction the incoming relay signs. Repair and fallback then reconcile rather than diverge. The chain already maintains a per-volume relay-assignment snapshot, so the plumbing is partly there.
3. Put relay identity in the commit’s shard references. The commit payload is owner-signed, so this needs the relay to counter-sign, or an owner could name a relay that never held the shard. The drain path already has that shape — a destination-signed absorption receipt settled on chain — so it is a reuse rather than a new primitive. Any such attestation MUST bind volume_id, shard_id, shard_index, chunk_root, the chain id and a freshness window, or one attestation is replayable into another volume.
4. Drop per-index liability. Challenge the volume and let any relay that holds the shard answer. Payout already works this way in spirit, since a reward requires a valid response rather than an assignment. What this gives up is the ability to name a specific relay as having lost specific bytes.

​

5.7 Relay Node Incentives

1. Storage fees: A share of the per-epoch storage fee paid by the account owner, proportional to the number and size of shards held.
2. Transfer fees: Reader-authorized fees for complete logical shards, counted per rounded MiB and paid from dedicated funded channels (§10.4.1), not a relay’s unilateral byte report.

Per epoch, for each volume:
  total_storage_fee = ceil(volume_effective_size / 2^20) * STORAGE_FEE_PER_MIB_PER_EPOCH
  per_relay_share = total_storage_fee / num_relay_nodes_holding_shards

Per payer-acknowledged complete logical shard:
  acknowledged_units += ceil(logical_shard_bytes / 2^20)
Per funded-channel checkpoint:
  transfer_fee = (acknowledged_units - settled_units) * current_TRANSFER_FEE_PER_MIB
  (bounded by fixed terms and cumulative payer cap; paid to the fixed serving relay)

​

5.8 Relay Drain Governance

pub enum ProposalPayloadKind {
    UpdateBasefeeConfig,    // CIP-3
    DrainRelay,             // §5.8, opcode 85
    UpdateAutoDrainPolicy,  // §5.8, opcode 86
}

​

6. Storage Addressing

​

6.1 Path-Based Namespace

/{account_address}/{volume_name}/{object_path}

• account_address: The owning account’s 20-byte protocol Address (hex-encoded) — the same Address type used protocol-wide and carried by VolumeAttachment.owner (§8.1).
• volume_name: A UTF-8 string (max 64 bytes, restricted to [a-zA-Z0-9_\-.]).
• object_path: A UTF-8 string (max 512 bytes) using / as a logical separator. No leading or trailing /.

/0xaabb...ccdd/model-weights/v3/layer_0.bin
/0xaabb...ccdd/agent-memory/conversations/2026-03-04/session_1.cbor
/0xaabb...ccdd/pipeline-scratch/job_4821/intermediate.parquet

​

6.2 Content Integrity

content_hash     = BLAKE3(plaintext_bytes)       // verifies decrypted content
ciphertext_hash  = BLAKE3(ciphertext_bytes)      // verifies pre-erasure-coding ciphertext
shard_hash[i]    = BLAKE3(shard_bytes[i])        // verifies a whole shard (integrity / repair)
chunk_root[i]    = merkle_root(fixed-size chunks of shard_bytes[i])  // PoR spot-check anchor (§5.6)

• Integrity verification: Any party can verify the full object lifecycle — shard hashes verify individual shards, ciphertext hash verifies reconstruction, content hash verifies decryption.
• Efficient proofs: a path from a leaf node to the DAG root is O(log N) in the number of objects. For a public volume this proves object membership directly; for a private volume the nodes are ciphertext, so object membership is not chain-verifiable from manifest_root alone — shard-level retrievability is anchored separately by the public per-shard chunk-roots in native RAS inventory (§5.6).

​

6.3 Addressing for Deletion

delete_object(volume_name, object_path) -> bool
delete_volume(volume_name) -> bool

• delete_object marks all shards of the object for removal on the assigned Relay Nodes and updates the onchain manifest. Relay Nodes garbage-collect the shard data.
• delete_volume initiates a soft delete: it sets the volume’s status to DELETED and starts the GC grace timer (§13.2). The StorageCommitment and rent persist through the grace window — the owner may call undelete_volume() to restore the volume — after which it is garbage-collected and fees cease.
• Only the account owner (or an Actor acting on behalf of the account) may delete. Runners are never granted delete permission, even under read-write mode — this prevents malicious or compromised Runners from destroying data.

​

7. Access Control

​

7.1 Capability Tokens

CapToken {
  volume_id:      bytes32,          // keccak256(account_address || volume_name)
  access_mode:    ENUM { READ_ONLY, WRITE_ONLY, READ_WRITE },
  path_prefix:    string,           // scope to objects under this prefix (e.g., "agent-3/")
  max_bytes:      u64,              // maximum total bytes writable
  valid_from:     u64,              // block height
  valid_until:    u64,              // block height (job timeout)
  runner_address: bytes32,          // the specific Runner authorized
  nonce:          u64,              // prevents replay
  signature:      bytes64           // reserved; presently zero — authority is the stored-token match, see below
}

​

7.1.1 Auth Enforcement on Placement RPCs

​

7.1.2 Node-to-Node Operations Are Not CapToken-Gated

1. the signing timestamp is within its accepted clock-skew window;
2. the sender resolves to a registered relay whose on-chain status is Active or Draining — Disabled and Inactive MUST be refused, since revocation can only come from the registry and never from the signature;
3. the signature verifies against that profile’s relay_identity_pubkey;
4. the digest covers the payload being acted on. A signature that binds only the sender is replayable against attacker-chosen content, which for ReplicatePlacement and SyncPeers — both writes — is placement poisoning by a longer route.

• ReplicatePlacement: the sender MUST already appear in the record’s assignments. Relay identity alone would otherwise let any registered relay write any record for any volume.
• SyncPeers: the advertised sender.node_id MUST equal the signing identity, so a registered relay cannot advertise another node’s id at an address it controls.
• PeerRelayPushShard / PeerPlacementUpdate: the sender MUST additionally be Draining, since a drain push is only legitimate from a relay that is draining.

​

7.2 Access Modes (CapToken Scopes)

Scope of that guarantee. The table above governs CapToken-gated access on private volumes, and there it holds: a WRITE_ONLY token is refused GET_SHARD, GET_PLACEMENT and listings at the Relay Node. It does not extend to a WRITE_ONLY grant over a public volume, whose reads are open to any party without a token (§7.6) — such a mount can refresh the manifest and therefore learns the object namespace. See §12.1.3. A grant that must not reveal what a volume contains is not a write-only mount on a public volume; it is a private volume.

Note — Public volumes: Volumes with visibility = PUBLIC use a different access model. Reads and listings are open to any party without a CapToken. Writes still require a CapToken (WRITE_ONLY or READ_WRITE). See §7.6 for the full PUBLIC volume specification.

​

7.3 Concurrent CapTokens

• Non-overlapping write prefixes: If two CapTokens grant WRITE access, their path_prefix values MUST NOT overlap. The Dispatcher enforces this at token issuance.
  • Prefix canonicalization: Prefixes are canonicalized by ensuring a trailing / separator. A prefix agent-1 is stored as agent-1/. This prevents ambiguity: agent-1/ and agent-10/ are non-overlapping; agent-1/ and agent-1/sub/ DO overlap (the first is a parent of the second). Overlap is defined as: prefix A overlaps prefix B if A is a prefix of B or B is a prefix of A (after canonicalization).
  • Empty prefix ("") means full volume access. No other WRITE CapToken may be active on the volume simultaneously if any token has an empty prefix.
• Reads never conflict: READ_ONLY tokens may coexist with any number of other READ_ONLY or WRITE tokens. A READ_WRITE token may read paths being written by other tokens.
• No total ordering of writes: Concurrent writes to different paths are independent. There is no global write ordering across CapTokens.
• CapToken revocation: A CapToken can be revoked before its valid_until by the Dispatcher recording the token’s nonce in a revocation list. Relay Nodes check the revocation list on each request. Writes in-flight at revocation time may or may not land; the next manifest commit determines the canonical state. Revocation is best-effort and convergent — Relay Nodes may serve a revoked token briefly until the revocation propagates.

​

7.3.1 Prefix Enforcement Boundaries

• Parent-root CAS (all volumes). Every commit names the prev_root it extends; the chain rejects it unless prev_root equals the volume’s current manifest_root (enforced today). A stale or racing committer cannot silently overwrite an intervening commit — it must re-base on the current root and retry. This alone removes the lost-update / replace-from-stale-base class of clobber.
• Public Store artifact pin. CIP-33 §2.1.2 adds a commit guard for a Public volume registered as a Store serving artifact. Direct CommitManifest and staged CommitManifestFinalize MUST reject while that volume’s CIP-33 pin count is nonzero. A staged commit prepared before registration still checks the pin at finalization. CIP-33 defines the references and release rule, including the future Forked spawn case; CommitManifestStage does not change the canonical root. Node #1708 at 194f400f adds Shared-hire lifetime references to the existing pin.
• Prefix-confined commits. A WRITE_ONLY committer may change only manifest nodes under its own path_prefix. How this is enforced depends on visibility, because the chain can only inspect paths it can see:
  • PUBLIC volumes — manifest nodes are plaintext (§3.1), so the chain verifies prefix-confinement directly: a commit whose changed nodes (relative to prev_root) touch any path outside the committer’s prefix is rejected onchain. Prevention, not detection.
  • PRIVATE volumes — manifest nodes are encrypted (§3.1), so paths are not visible onchain; the chain cannot check them without breaking the privacy guarantee (§9.3), and sampling cannot prove the absence of an out-of-prefix path. Prevention therefore routes through a DEK holder: a WRITE_ONLY sub-agent stages its prefix-scoped delta (CommitManifestStage, attributed to staged_by) but cannot finalize the canonical root; the DEK-holding coordinator or owner decrypts the staged delta, verifies prefix-confinement, and issues CommitManifestFinalize. A sub-agent therefore cannot unilaterally set manifest_root, so it cannot clobber another prefix.

​

7.4 Caveats and Restrictions

• Path prefix narrowing: A CapToken scoped to job_4821/ can be further restricted to job_4821/checkpoints/ but never broadened to /.
• Byte quota reduction: A 1 GiB quota can be reduced to 512 MiB but never increased.
• Time window narrowing: The valid window can be shortened but never extended.

​

7.5 Read Consistency

1. Fetch the manifest DAG nodes from Relay Nodes (starting at the root locator) and reconstruct them.
2. Recompute the root locator of the reconstructed DAG (§3.1).
3. Compare the computed root to the onchain manifest_root in the volume’s StorageCommitment.
4. Reject on mismatch. A mismatched root means the fetched manifest is stale, partially published, or corrupted.

Future work — READ_UNCOMMITTED: A mode where objects are visible as soon as shards land on Relay Nodes (before commit_manifest()) is desirable for the real-time agent swarm pattern, where latency matters more than strict consistency. However, it is not implementable on the current design for two reasons:

1. Discovery: With versioned shard IDs (§5.3, write_id in the shard address), a reader cannot predict the shard_id for an uncommitted write — they don’t know the writer’s random write_id. Discovering uncommitted objects requires a separate metadata channel (pubsub or uncommitted manifest fragments) that this spec does not yet define.
2. Prefix safety: prefix-confinement is enforced at commit (§7.3.1); an uncommitted read would bypass that gate and could observe out-of-prefix shards from a rogue writer before the commit that would reject them.

READ_UNCOMMITTED is deferred to a future CIP that defines the discovery mechanism.

​

7.6 Public Volumes (PUBLIC)

​

7.6.1 Overview

​

7.6.2 Properties

• No encryption: Objects in PUBLIC volumes are stored unencrypted on Relay Nodes. No DEK is generated for the volume. The wrapped_dek field in the StorageCommitment is empty.
• No CapToken for reads: Any party can fetch shards from Relay Nodes without presenting a CapToken. Relay Nodes serve GET_SHARD for a public volume’s shards without authenticating the caller — but not unconditionally: the volume must be ACTIVE or GRACE_PERIOD (§13.5), and the shard’s recorded volume must be the one the request names (§7.6.3).
• CapToken still required for writes: Only the account owner (or authorized Runners via CapToken) can write to the volume. Write access uses the same CapToken mechanism as private volumes.
• Content integrity preserved: content_hash (BLAKE3) is still computed and stored for every object. Readers MUST verify the content hash after shard reconstruction to detect corruption or tampering.
• Erasure coding preserved: Reed-Solomon coding applies identically. The only change is that the input to erasure coding is plaintext (not ciphertext).
• Billing unchanged: The account owner pays the same per-epoch, per-byte storage fees as private volumes.
• Listing is public: list_objects for public volumes does not require a CapToken. The manifest is stored unencrypted and readable by anyone who traverses the manifest DAG from the on-chain root (§3.1) or calls GET_MANIFEST (§5.3.2).

​

7.6.3 Relay Node Behavior

• GET_SHARD requests are served without CapToken verification, authorized from the commitment as above and additionally bound to the shard’s recorded volume. Serving is status-gated: a volume that is not ACTIVE or GRACE_PERIOD is not served anonymously, matching §13.5.
• GET_MANIFEST (§5.3.2) is served without a CapToken — the same open-read rule.
• LIST_SHARDS is not exposed; listing is performed client-side from the manifest.
• PUT_SHARD requests still require a valid CapToken with write access. The Relay Node persists the shard’s volume binding alongside the shard, taking it from the authenticated PUT_SHARD payload’s volume_id — the same value the CapToken is checked against — together with whatever metadata the AuthDecision carries, and with the shard’s commit state, kind and tentative deadline. The binding is written by the relay, not minted by the AuthProvider; the provider’s decision is what makes the payload’s claim trustworthy enough to persist.

​

7.6.4 Shard ID Opacity

​

7.6.5 Content-Type Metadata

{
  "defaults": {
    ".html": "text/html; charset=utf-8",
    ".css": "text/css",
    ".js": "application/javascript",
    ".json": "application/json",
    ".png": "image/png",
    ".jpg": "image/jpeg",
    ".svg": "image/svg+xml",
    ".woff2": "font/woff2"
  },
  "overrides": {
    "data/feed.xml": "application/atom+xml"
  }
}

​

7.6.6 Cache Headers

{
  "default_max_age": 3600,
  "paths": {
    "assets/*": {"max_age": 86400, "immutable": true},
    "index.html": {"max_age": 0, "must_revalidate": true}
  }
}

​

7.7 Cross-Owner Volume Access (Mount Allowlist)

MountGrant {
  grantee:               MountPrincipal,                           // Account(addr) | Actor(addr); existing wire tags; matched by bare address
  max_access_mode:       ENUM { READ_ONLY, WRITE_ONLY, READ_WRITE },
  canonical_path_prefix: string,                                   // "" = whole volume; canonical form (§8.1.1)
  tee_required:          bool,
  valid_until:           u64?,                                     // block height; absent = no expiry
  grantee_signing_key:   bytes32?,                                 // the grantee's cbfs Ed25519 delegated key this grant is bound to (§7.7.1)
  grantee_encryption_key: bytes32?,                                // the grantee's separate X25519 key the DEK is sealed to (§9.2.1)
  grantee_wrapped_dek:   bytes?,                                   // HPKE package sealing the DEK to grantee_encryption_key; PRIVATE volumes only
}

• Any ciphertext it kept, and any plaintext it extracted, stay readable. Removal bounds future fetches, not past ones.
• The DEK is also the path-tag key: path_tag = HMAC(dek, seq, path) and every path_tag is published on-chain in every commit, permanently. A former DEK holder can therefore test candidate paths against the whole commit history offline — no CapToken, no relay, no network. Removing the grant does not close that, and nothing except rotating the DEK does.

​

7.7.1 Grant-Scoped Owner Tokens

• grantee equals the cert’s wallet_address by bare address;
• grantee_signing_key is present and equals the cert’s cbfs_public_key;
• token.access_mode is permitted by max_access_mode as an operation set: READ_ONLY permits only READ_ONLY tokens, WRITE_ONLY only WRITE_ONLY tokens, READ_WRITE any of the three;
• token.path_prefix is "" — a grant-scoped token is always whole-volume;
• valid_until, if present, has not passed at the relay’s block-height view; a relay without a height view MUST refuse an expiring grant;
• tee_required is false — a client holder is never a TEE, so a tee_required grant does not authorize an owner-auth token.

​

8. Volume Attachment

​

8.1 Attachment at Job Dispatch

RelayEndpoint {
  node_id:        bytes32,          // relay's on-chain node identity (RelayNodeProfile.address, §5.2)
  address:        string,           // network-routable address for shard RPCs (host:port or URL)
  relay_address:  Address?,         // optional: relay operator's on-chain account, if distinct from node_id
}

​

8.1.1 Per-Job Attachment Admission Caps

• Attachment count. attachments.len() <= 64. A task carrying more than 64 VolumeAttachment entries is rejected outright.
• volume_name charset/length. Each volume_name MUST satisfy the volume-name namespace rule of §6.1 (the single normative statement of the length cap and character set) — an attachment naming a volume that would itself be an invalid volume_name under §6.1 is rejected at admission, before any (owner, volume_name) lookup is attempted.
• canonical_path_prefix canonical form. The prefix MUST already be in canonical form: "" for whole-volume access, or a non-empty prefix ending in a single trailing / with no leading / (matching the separator convention of §6.1). validate() rejects a non-canonical prefix; it does not normalize one on the caller’s behalf — canonicalization is the caller’s responsibility before submission.

​

8.2 Attachment Process

1. CapToken issuance: The Dispatcher issues a scoped CapToken for each volume attachment.
2. Volume key delivery (private volumes only): the Runner obtains the DEK via a CBSS SealRequest it completes itself (§9.2); the Dispatcher constructs and authorizes the seal request but never unwraps or handles a plaintext DEK. A private volume must carry a committee wrap to be runner-mountable — an owner-key-only volume is read by its owner via the CLI (§9.2), not delivered to a Runner. Public volumes skip this step.
3. Manifest fetch: The Runner fetches the manifest DAG from the on-chain root locator (§3.1) by node-addressed GET_SHARD, recomputes the root, and verifies it matches the onchain manifest_root (§7.5). On a public volume it MAY use GET_MANIFEST (§5.3.2) instead and verify the same way; a private volume has no such option, since a relay cannot assemble a manifest it cannot decrypt. Only after verification does the Runner trust the manifest for reading or writing.

attachment_cost = BASE_ATTACHMENT_FEE

​

8.3 Detachment

1. The Runner commits the final storage manifest onchain (if it wrote any objects).
2. The CapToken is invalidated (past valid_until).
3. Shards written during the job persist on Relay Nodes, independent of the Runner’s lifecycle.

​

8.2.1 Dispatch Eligibility Diagnostics

{ total_candidates, excluded_by_job_type, excluded_by_health,
  excluded_by_reputation, excluded_by_storage_support, selected }

​

9. Encryption and Privacy

​

9.1 Encryption at Rest

1. Volume DEK (Data Encryption Key): A random 256-bit key generated at volume creation. The DEK is never derived from the account’s signing key — signing keys sign; they do not derive encryption keys.
   volume_dek = CSPRNG(32)  // generated once at create_volume()
   
   The commitment carries a dek_version: u32 alongside the wraps. It is an input to the CBSS volume-DEK identity volume_dek_identity(chain_id, owner, volume_id, dek_version, committee_epoch, release_key_material_hash, committee_override_hash) (CIP-24 §9.3), hashed as 4 little-endian bytes, and to release_context; it is also carried in every direct-client GrantDekPackage (§7.7), whose fixed 116-byte layout puts it in the first 4 bytes big-endian. Its width is part of three preimages and one fixed wire layout, so it is not a field an implementation may widen locally. It is not initialised to 1 and it is not chain-assigned: create_volume takes an owner-supplied initial_dek_version and applies only a floor of 1. A dek_version is therefore a label the owner chose, not a count of rotations, and nothing today advances it — see §9.4. The DEK is then wrapped one or more ways according to the volume’s access class (wrapping_key_policy, stored on-chain in the StorageCommitment, §11.1). The wraps are independent and additive — a volume may carry either or both:
   • Owner wrap (owner-key, v2): the DEK is wrapped to a key derived from a deterministic wallet signature, so the owner can read and write directly from the CLI or browser, including extension wallets that never expose their private key. Native wallets, CLI wallets, and extension signMessage implementations MUST use this same derivation:
     msg = UTF8("cbfs/owner-key-wrap/v2|" || decimal(chain_id) || "|"
                || network || "|" || lowercase_hex(volume_id))
     H = keccak256(msg)  // exactly 32 bytes
     digest = keccak256(0x19 || UTF8("Cowboy Signed Message:\n32") || H)
     signature = RFC6979_secp256k1_sign(digest)
     ikm = r_be32 || min(s, secp256k1_order - s)_be32
     salt = UTF8("cbfs/owner-key-volume-dek-wrap/salt/v2")
     info = UTF8("cbfs/owner-key-volume-dek-wrap/info/v2")
            || u64_be(chain_id)
            || u16_be(byte_length(UTF8(network))) || UTF8(network)
            || owner_address_raw20 || volume_id_raw32
            || u16_be(byte_length(UTF8(volume_name))) || UTF8(volume_name)
     wrapping_key = HKDF-SHA256(ikm=ikm, salt=salt, info=info, length=32)
     nonce = CSPRNG(12)
     owner_wrapped_dek = AES-256-GCM(wrapping_key, nonce, volume_dek,
                                    aad=volume_id_raw32)
     
     The chain ID is unsigned canonical decimal with no leading zeroes; volume hex is exactly 64 lowercase digits without 0x. Lengths count UTF-8 bytes, and the two length-framed strings MUST fit in u16. Both signature scalars MUST satisfy 0 < r,s < secp256k1_order; normalize high-s before HKDF and exclude the recovery byte v. Extension wallets sign the 32-byte hash H through window.cowboy.signMessage("0x" + lowercase_hex(H)); this API accepts exactly 32 bytes and shows the user the hex of H. It adds the Cowboy personal-message prefix exactly once. Native wallets and the cbfs CLI sign digest directly. The shared personal_message_digest(bytes) remains generic (prefix || decimal(byte_length(bytes)) || bytes), but the owner-wrap path always supplies H, so its prefixed length is always 32. The caller MUST establish that the signing wallet is the volume owner. The signature is secret key material for this derivation: retain it, the wrapping key, and the plaintext DEK only in the client; never send them to a server or log them. Wipe owned signature/IKM, key, and plaintext buffers after use. The shared implementation is cowboy-protocol-client-crypto, exported by cowboy-protocol-cbss-wasm; its tests/fixtures/owner_wrap_v2_vectors.json fixes the cross-client bytes, including deterministic nonces for tests only. Production wrapping always samples a fresh nonce. The existing stored owner-wrap envelope format is unchanged. Owner-wrap v2 is genesis-active from block 0 of the next regenesis. The raw-wallet-secret v1 derivation is removed; there is no migration path.
   • Committee wrap (committee-only): the DEK is IBE-encrypted to the CBSS committee under domain CIP9_VOLUME_DEK_IBE_DOMAIN = "cbss/ibe/cip9-volume-dek/v1", so an authorized Runner can obtain it at mount via a threshold seal (§9.2) without the owner ever placing wallet key material on the runner.
   A canonical hash domain CIP9_HASH_DOMAIN_VOLUME_DEK = "cowboy.cip-9.volume-dek.v1" (with AAD domain …volume-dek-aad.v1) binds each wrap to its volume_id. The wraps are stored in the on-chain StorageCommitment (§11.1). Access classes (wrapping_key_policy):

   Policy	Wraps present	Who can write	Who can read
   owner-key	owner (and, typically, committee)	owner via CLI; runners with an RW CapToken	owner directly; committee-served runners
   committee-only	committee	runners with an RW CapToken	committee-served runners (the owner cannot read without a runner)
   tee-gated (future)	committee, release gated on TEE attestation	attested runners with an RW CapToken	attested committee-served runners

   This resolves the long-standing “can’t write a private volume from the CLI” confusion: CLI writes are valid for any volume that carries an owner wrap, and --owner-key at create means also store the owner wrap — additive, not exclusive. The threat model is per-class and deliberate: committee-only keeps wallet key material off every client machine at the cost of the owner needing a runner to read; owner-key trades that for direct CLI access.
2. Object Encryption: Each object is encrypted with AES-256-GCM using a random nonce per write:
   nonce = CSPRNG(12)       // fresh random nonce for EVERY write, including overwrites
   ciphertext = AES-256-GCM(key=volume_dek, nonce=nonce, plaintext=object_bytes, aad=object_path)
   
   The nonce is stored alongside the ciphertext in the ObjectDescriptor. This is critical: deterministic nonces derived from the object path would cause nonce reuse on overwrites, which is catastrophic for AES-GCM (leaks XOR of plaintexts, breaks authentication). A fresh random nonce on every write eliminates this class of attack entirely.
3. Shard opacity: Erasure coding is applied to the ciphertext, not the plaintext. Relay Nodes hold shards of ciphertext — even if they reconstructed all shards, they would only have ciphertext.
4. Manifest encryption: The volume manifest (list of object paths, sizes, content hashes, ObjectDescriptors) is encrypted with the volume DEK before transmission to Relay Nodes. PlacementRecords are stored separately on Relay Nodes (§5.3.1) and are not part of the encrypted manifest.

​

9.2 Runner Key Access

1. The Runner issues a SealRequest for the volume’s committee-wrapped DEK, presenting its CapToken (and, for tee-gated, its attestation).
2. Each CBSS committee member verifies authorization and returns a partial decryption.
3. The Runner combines the threshold partials and completes the IBE decryption (domain cbss/ibe/cip9-volume-dek/v1) to recover the DEK, sealed in transit to its encryption_pubkey.
4. The DEK is held in memory for the job and zeroized on completion.

• The DEK never appears in plaintext on-chain or in any persistent store.
• READ_ONLY CapTokens on private volumes still require DEK delivery — the Runner must decrypt ciphertext shards to serve reads.
• For TEE-attested runners (tee-gated), the committee releases partials only against a valid attestation, and the DEK is sealed to the enclave, never exposed to the host OS.

​

9.2.1 Delegated Client Key Access

​

9.3 Privacy Guarantees

• Relay Nodes see only ciphertext shards indexed by opaque shard IDs (see §16.3). They cannot read object contents or inspect the manifest. Object paths are encrypted within the manifest and never exposed to Relay Nodes.
• Other Runners (not assigned to the job) cannot access the volume DEK.
• onchain observers see only the Storage Commitment (volume ID, wrapped DEK, encrypted manifest hash, total size, shard assignments). They cannot determine what is stored.
• The account owner has full access to all their volume data by unwrapping the DEK with their wrapping key.

• No confidentiality: Data is stored unencrypted. Relay Nodes, Runners, and any network participant can read object contents. This is by design — public volumes are intended for publicly readable data (web assets, public datasets, shared artifacts).
• Integrity preserved: Content hashes (BLAKE3) ensure that data has not been tampered with, even though it is unencrypted.
• Write access is still restricted: Only the account owner (or authorized Runners via CapToken) can write to a public volume. Public readability does not imply public writability.

​

9.4 Rewrapping and Rotation

​

9.4.1 Rewrapping — same DEK, new wrap (built)

• The chain cannot check that the re-wrapped plaintext is the same DEK. It verifies the epoch and the material hash, never the wrapped key. An owner can substitute a different key with no dek_version change, which produces the inverse hazard to the one §9.4.2 is about: the commitment asserts no rotation while the served key no longer opens the ciphertext.
• Rewrapping revokes nothing. Every party that has ever held the DEK still holds a working key afterwards. This CIP does not permit rewrapping to be described as a security boundary.

​

9.4.2 Rotation — not implemented, and not cheap

• R1 — Re-encryption is total. Every object and every manifest node is re-encrypted under the new DEK. A partial rotation leaves the remainder readable and is not a rotation.
• R2 — No split state. No commitment is reachable in which dek_version has advanced but the root has not, or the reverse. A reader that trusts the commitment must never be handed a key that does not open the data it names.
• R3 — Authority is the owner’s. A principal that can rotate can lock the owner out, so rotation is not delegable to any grant, CapToken or delegation scope that is otherwise sufficient to commit.
• R4 — Pre-rotation authority ends. After a rotation, no principal authorized before it retains read or write access. Permission and key must be withdrawn together, or the rotation is decorative.

​

9.4.3 What any rotation design has to solve

1. A full-volume rewrite does not fit one commit. apply_manifest_commit refuses when added + removed shard references exceed MAX_COMMIT_FINALIZE_SHARDS (65,536). At the default K=4/M=2 that is on the order of 5,400 objects, against a MAX_OBJECTS_PER_VOLUME of 1,000,000. R2’s atomicity and a bounded finalize are therefore in direct tension for exactly the volumes most worth rotating, and resolving it is the first job of any design — either by making the rekey durable across a multi-commit rewrite, or by giving the volume an explicit mid-rotation state that readers can see.
2. The finalize instruction carries no authorization. CommitManifestFinalize is { volume_id, stage_id }. Its authority is derived from the stage header and admits a full-volume ReadWrite mount grantee and a staged owner delegation, not only the owner. R3 cannot be met by riding an existing commit instruction without adding a discriminator that narrows authority when a rekey is present.
3. Clearing the allowlist does not end pre-rotation authority. TransferVolume invalidates CapTokens by advancing last_owner_transfer_block, which token.valid_from is compared against; clearing mount_allowed_actors alone does not. R4 needs a rotation analogue of that fence, or a runner attached before the rotation keeps write authority and can commit old-DEK objects onto the post-rotation root — reintroducing exactly the split state R2 forbids. (Direct-client grants are a different case: each carries a GrantDekPackage bound to dek_version, so those go inert on a bump by themselves. It is the keyless grants and the already-issued CapTokens that survive.)
4. The party being rotated away can starve the rotation. A staged rewrite spans many transactions and expires; the stage header pins prev_root, and apply_manifest_commit refuses on a prev_root mismatch. Any intervening commit by a principal that still holds write authority busts the CAS and the owner re-uploads the whole volume. Authority has to be withdrawn before or during the rewrite, not on its success.
5. An owner may not be able to decrypt its own volume. §9.2 states that for a committee-only volume the owner cannot read without a runner. Client-side re-encryption by the owner is therefore not universally available, and rotation for that class needs either a runner-mediated form — which collides with R3 — or an explicit statement that the class is not rotatable.
6. dek_version is owner-chosen and 32 bits. A monotonic +1 rule on the rotation path does not make an identity one-shot: volume_id is deterministic in (owner, volume_name), the identity carries no incarnation nonce, and initial_dek_version is caller-supplied, so delete-and-recreate can re-reach a spent identity — and a party that once combined threshold partials for it holds that decryption key permanently. Overflow at u32::MAX is also undefined.
7. wrapping_key_policy does not exist on chain. §9.1’s access classes are a target, not state a validator can read (§17.3), so no rotation rule may be conditioned on them today.
8. The rekey is a cross-repo wire change. The commit payloads are owned by the cowboy-protocol codec, pinned by revision into node and mirrored in cbfs, with golden vectors in three places and a fixed-size owner-operation table. Any new signed field is a coordinated pin bump, not a local schema addition.

​

9.4.4 What rotation would and would not revoke

• Objects written after a rotation are opaque to a holder of the old DEK. Objects that existed before it are opaque only because R1 re-encrypted them.
• Plaintext already extracted is gone; nothing on-chain recalls it.
• path_tag history (§7.7) is computed under the old DEK and stays on chain forever. Rotation stops new tags being linkable; it does not unlink old ones.
• Relays that retain superseded pre-rotation ciphertext hold bytes the old DEK still opens, until shard lifecycle (§13) removes them.
• A rotation rewrites the whole volume: every object re-encrypted, re-erasure-coded and re-uploaded, with the byte delta counted twice over. It is not a background operation and MUST NOT be presented as one.

​

10. Billing and Fees

​

10.1 Fee Components

​

10.2 Effective Size and Erasure Overhead

effective_size = raw_size * (K + M) / K

​

10.3 Persistent Storage Billing

• Each epoch, the protocol calculates the total effective storage used by each account across all volumes.
• The per-epoch storage fee is deducted from the account’s balance.
• If the account’s balance falls below MIN_STORAGE_BALANCE (sufficient to cover one epoch of fees), the protocol enters a grace period of STORAGE_GRACE_EPOCHS.
• After the grace period, if the balance is still insufficient, all volumes owned by the account are marked for garbage collection.

​

10.3.1 Escrow and Rent Lifecycle

• Create: with no escrow and insufficient balance to cover MIN_STORAGE_BALANCE, creation MUST fail with a distinct, named error (§12.3.1) — not a generic E1900 "invalid data", which today conflates the duplicate-name and escrow-shortfall cases. A no-escrow volume that is created bills from balance immediately and expires after one epoch if balance is insufficient.
• Active → grace: when escrow is exhausted and balance falls below MIN_STORAGE_BALANCE, the volume enters GRACE_PERIOD for STORAGE_GRACE_EPOCHS. A top-up of escrow or balance returns it to ACTIVE.
• Expiry: after the grace window with still-insufficient funds, the volume is marked GARBAGE_COLLECTING (§13).

​

10.4 Fee Distribution

Account Owner ──(per-epoch storage fee)──► Protocol ──► { platform (0x18) :: PoR challenge pool (0x0B) :: Relays pro-rata }
Explicit Payer ──(fixed deposit)──► Dedicated Read Channel ──(bounded cumulative ACK claim)──► Fixed Serving Relay

• STORAGE_FEE_PLATFORM_BPS (default 1000 = 10%) → credited to the Platform Fee Account 0x18 per CIP-31 §4.
• STORAGE_FEE_CHALLENGE_POOL_BPS (default 100 = 1%) → accrued to the PoR challenge pool at 0x0B (this is the explicit form of the POR_CHALLENGE_FEE_SHARE row in §14; it funds the challenger bounty defined in CIP-31 §7).
• STORAGE_FEE_RELAY_BPS (default 8900 = 89%) → distributed pro-rata across active Relay Nodes with weight (shard_count × shard_age_in_epochs) per CIP-31 §5.

​

10.4.1 Funded cumulative read channels

• channel ID and terms hash;
• strictly advancing sequence;
• cumulative units U = sum(ceil(logical_shard_bytes / 2^20));
• cumulative fee ceiling F, read count, payload-byte count and receipt root.

delta = (U - settled_units) * current_TRANSFER_FEE_PER_MIB

escrow_remaining + paid_atomic + refunded_atomic == B

​

10.4.2 Bounded journals, economics and recovery

​

10.5 Relationship to CIP-3

• onchain operations (creating Storage Commitments, writing manifests, opening/claiming/refunding read channels) consume Cycles, Cells and Accesses as normal CIP-3 transactions, separate from escrow principal.
• Off-chain storage fees are a separate ledger entry, debited from the account balance per epoch by the Storage Manager system actor.

​

11. onchain State

​

11.1 Storage Manager System Actor

StorageCommitment {
  volume_id:             bytes32,    // keccak256(account_address || volume_name)
  owner:                 address,
  volume_name:           string,
  visibility:            ENUM { PRIVATE, PUBLIC },  // PRIVATE = encrypted, CapToken-gated; PUBLIC = unencrypted, open reads
  wrapping_key_policy:   ENUM { OWNER_KEY, COMMITTEE_ONLY, TEE_GATED },  // access class (§9.1); ignored for PUBLIC
  created_at:            u64,        // block height
  owner_wrapped_dek:     bytes?,     // DEK wrapped to the owner wallet key (§9.1); absent if no owner wrap / PUBLIC
  wrapped_dek:           bytes?,     // DEK IBE-wrapped to the CBSS committee (§9.1); absent if no committee wrap / PUBLIC
  dek_version:           u32,        // §9.1. Owner-chosen at create (floored at 1); nothing advances it today.
  cbss_committee_epoch:  u64,        // committee epoch the committee wrap is bound to (§9.2)
  cbss_release_key_material_hash: bytes32,  // binds the wrap to the owner's release key material (CIP-24 §9.3)
  relay_nodes:           [RelayEndpoint],  // the volume's relay list (§5.3). Position is meaningful: index i is the relay the chain holds responsible for shard index i (§5.6.1). Written at create; no instruction updates it.
  cbss_rewrap_required:  bool,       // set by TransferVolume; blocks private attachment until rewrapped (§9.4.1)
  last_owner_transfer_block: u64,    // CapTokens with valid_from below this are revoked (§13.4)
  manifest_root:         bytes32,    // root locator of the manifest DAG (§3.1); over ciphertext nodes for PRIVATE, plaintext for PUBLIC
  live_shard_count:      u64,        // number of dense native RAS inventory ordinals for this volume (§5.6); at most MAX_LIVE_SHARDS_PER_VOLUME
  raw_size_bytes:        u64,        // sum of object sizes (pre-erasure)
  effective_size_bytes:  u64,        // raw_size * (K+M)/K
  erasure_k:             u8,         // data shards
  erasure_m:             u8,         // parity shards
  last_updated:          u64,        // block height of last manifest update
  degraded_shards:       u16,        // count of shards needing repair
  mount_allowed_actors:  [MountGrant],  // cross-owner allowlist (§7.7); matched by bare address; authorizes dispatch CapTokens and grantee owner-auth tokens (§7.7.1)
  status:                ENUM { ACTIVE, GRACE_PERIOD, DELETED, GARBAGE_COLLECTING }
}

AccountStorageSummary {
  total_volumes:          u32,
  total_effective_bytes:  u64,
  last_billed_epoch:      u64,
  balance_reserved:       u256       // CBY reserved for storage fees
}

​

11.2 Relay Registry

• RelayNodeProfile entries (see §5.2)
• Active relay list (ordered, health-decaying, analogous to CIP-2 Runner Registry)
• Shard assignment index: volume_id → list[PlacementAssignment]
• The network-wide AUTO_DRAIN_POLICY_KEY slot written by governance (§5.8)

​

11.3 Key Space

key = 0x1 || keccak256(storage_manager_address) || 0x00 || keccak256("commitment" || volume_id)
value = rlp(StorageCommitment)

key = "ras:owner-name:" || owner_address || keccak256(volume_name)   // OWNER_NAME_PREFIX = b"ras:owner-name:"
value = volume_id (bytes32)

key = 0x1 || keccak256(storage_manager_address) || 0x00 || keccak256("summary" || account_address)
value = rlp(AccountStorageSummary)

key = 0x1 || keccak256(relay_registry_address) || 0x00 || keccak256("relay" || relay_address)
value = rlp(RelayNodeProfile)

​

12. Client Interfaces

• Filesystem interface (§12.1): A FUSE-mounted directory that presents the volume as a standard filesystem. This is the primary interface for agentic workloads where an LLM (Claude, Kimi-K2, GPT, etc.) operates via tool calling. The model’s existing filesystem tools — Read, Write, Bash (ls, grep, find), Glob, Grep — work unchanged against the mounted volume. No custom tool definitions required.
• Object API (§12.2): A programmatic interface for orchestration code and non-agentic workloads. This is what the FUSE layer calls internally, and is also available directly for lightweight write-only jobs that don’t need filesystem semantics.

​

12.1 Filesystem Interface (FUSE Mount)

​

12.1.1 Design Rationale

• Injecting custom tool definitions into every model’s tool set
• Models are less fluent with unfamiliar, domain-specific tools
• Loss of composability with standard unix tools (grep, find, jq, wc, etc.)
• Every model provider’s runner integration needs custom work

Model tool call                        What happens under the hood
─────────────────                      ──────────────────────────────
Read("/mnt/memory/state.json")    →    FUSE read → fetch shards → reconstruct → decrypt
Write("/mnt/memory/state.json")   →    FUSE write → local buffer → background push
Bash("ls /mnt/memory/logs/")      →    FUSE readdir → list objects from manifest
Bash("grep -r 'error' /mnt/mem")  →    FUSE read (multiple) → local cache → grep
Bash("wc -l /mnt/memory/*.json")  →    FUSE read (multiple) → local cache → wc

​

12.1.2 Mount Point Layout

/mnt/volumes/{volume_name}/

# Full volume mount:
/mnt/volumes/agent-memory/
├── state/
│   ├── memory.json
│   └── portfolio.json
├── logs/
│   └── 2026-03-04/
│       └── analysis.json
└── config.json

# Prefix-scoped mount (path_prefix="state/"):
/mnt/volumes/agent-memory/
├── memory.json
└── portfolio.json

​

12.1.3 Sync Strategy (Hybrid)

• Pull cycle (Relay Nodes → local): Re-fetches the volume’s committed manifest from Relay Nodes, verifies it against the onchain manifest_root (§7.5), and materializes any new objects not already in the local tmpfs. Because reads are READ_COMMITTED (§7.5), the pull cycle only discovers objects that have been included in a committed manifest — objects from a sub-agent become visible only after that sub-agent calls commit_manifest(). In the agent swarm pattern, this means the coordinator sees a sub-agent’s files appear as a batch when the sub-agent commits, not individually as they are written.
• Push cycle (local → Relay Nodes): Detects locally written or modified files (via inotify/fswatch), encrypts them, erasure-codes, and distributes shards to Relay Nodes.

• Public volume. Reads and listings are open without a CapToken (§7.6), so a write-only mount’s manifest refresh succeeds. It follows that a WRITE_ONLY grant on a public volume exposes the object namespace — paths, sizes, and object descriptors — to the mounter, even though every POSIX read of an object’s contents is refused at the syscall surface (§12.1.4). That is not a weakening: a public volume’s namespace is open to any party already.
• Private volume. The refresh is GET_PLACEMENT followed by per-node GET_SHARD, both presenting the mount’s CapToken, and §7.2 grants WRITE_ONLY neither. It is refused at the Relay Node, and a client cannot tell that refusal from transient unavailability. A WRITE_ONLY mount on a private volume therefore has no committed base state to compute a delta against. The parent-root CAS of §7.3.1 is what keeps that from becoming a destructive commit; the practical consequence is that such a mount cannot commit rather than that it commits something wrong.

​

12.1.4 FUSE Operation Mapping

• A request that leaves the masked-out bits at 0 is ambiguous, and the implementation MUST preserve the stored value of those bits. Storing the masked value instead would carry the mask into PosixMetadata, replicate it through the manifest, and narrow the object for every mount — losing more bits on each cycle.
• A request that sets a masked-out bit is not ambiguous: the report showed 0 there, so no caller read a 1 from this mount. The implementation MUST refuse it with EPERM, and MUST NOT apply any other field of the same setattr. chmod(2) is specified to apply the requested S_IALLUGO bits or fail, and reporting success while the durable state keeps bits the caller asked to change is a false answer on the call used to tighten permissions.

​

12.2 Object API (Programmatic)

• The Runner is a script (not an LLM doing tool calling) that just needs to write output files.
• The workload is lightweight and a full FUSE mount is unnecessary overhead.
• The orchestration layer needs to interact with storage outside of a container context.

# Object operations (subject to CapToken permissions)
put_object(cap_token: bytes, object_path: str, data: bytes) -> ObjectReceipt
put_many(cap_token: bytes, items: list[(str, bytes)]) -> list[ObjectReceipt]  # batched; refreshes token mid-stream (§12.2.1)
get_object(cap_token: bytes, object_path: str) -> bytes          # READ_ONLY or READ_WRITE
list_objects(cap_token: bytes, prefix: str = "") -> list[str]    # READ_ONLY or READ_WRITE

# CBFS-level commit (data plane)
commit(cap_token: bytes) -> CommitReceipt

# Cowboy-level commit (control plane, onchain)
commit_manifest(cap_token: bytes, prev_root: bytes32, new_root: bytes32,
                added_shards: [ShardRef], removed_shards: [ShardRef]) -> bool

1. CBFS commit (commit): Publishes the manifest DAG’s new nodes — only the delta’s added_nodes, since unchanged nodes are structurally shared with the prior root (§3.1) — to Relay Nodes, each addressed by manifest_node_shard_id(volume_id, locator), and publishes PlacementRecords to all assigned Relay Nodes via PutPlacement (§5.3.1). This is a data-plane operation — it makes the data durable and discoverable by other CBFS clients, but does not touch the chain. Manifest-node PlacementRecord publication MUST succeed on every assignee before step 2 (§5.3.1, COW-3263); object-shard record publication is best-effort — failures are logged but do not block the commit.
2. Cowboy commit (commit_manifest): Submits the new manifest DAG root, the prev_root it extends, and the added_shards / removed_shards deltas to the Storage Manager via the write-relayer (§12.2.1). The chain enforces a parent-root compare-and-swap — it rejects the commit unless prev_root equals the volume’s current manifest_root — so concurrent or stale commits cannot silently lose updates. In the same state transition, it updates the delta’s native shard primary and dense ordinal rows and live_shard_count (§11.1). manifest_root anchors READ_COMMITTED consistency (§7.5); the live rows supply PoR’s per-shard issue-time commitments (§5.6). Individual ObjectDescriptors stay off-chain in the manifest DAG; for a public volume the plaintext root yields O(log N) object-inclusion proofs, while for a private volume object contents stay opaque and shard retrievability is checked against the selected native row instead.

event ManifestCommitted {
    volume_id:      bytes32,
    manifest_root:  bytes32,
    block_height:   u64,
    raw_size_delta: i64,     // signed delta from the previous manifest
    visibility:     u8       // 0 = PRIVATE, 1 = PUBLIC
}

​

12.2.1 Manifest Commit Transport (Write-Relayer)

• Routable bind + self-registration. A write-relayer MUST bind a routable address (not loopback) and, given a public endpoint, self-register on the on-chain /ras/write-relayers registry, so clients discover relayers from chain rather than hardcoding an address. A loopback-only, unregistered relayer breaks the “RPC-URL only” goal — it forces an out-of-band tunnel.
• Canonical owner-authorization signing domain. The owner authorization MUST canonicalize addresses as EIP-55 in both the payload hash and the signing bytes; a mismatch between the two is a class of signature-rejection bug and is non-conforming. The owner-action payload is signed over an explicit, language-neutral big-endian byte layout (not a Rust struct encoding), so a non-Rust or alternate-chain backend can produce an identical signing domain. This canonical layout is now the wired implementation, owned by cowboy-protocol-codec (cowboy_protocol_codec::ras) as the single owner of the RAS wire contract and its signing bytes; both the node validator and the cbfs client derive from it rather than carrying their own copy — one implementation, structurally preventing the encoder fork that conformance depends on (native == WASM, golden-vector-locked). See §17.3.
• Chain-binding of owner authorizations. The owner authorization and owner delegation carry a signed chain_id (and network). A verifier MUST bind chain_id to the running chain — it MUST reject any owner authorization or delegation whose chain_id does not equal the running chain’s chain_id — and it MUST NOT validate a delegation’s chain_id against the delegation’s own self-claimed value (a self-comparison is tautological and permits cross-chain replay). network binding is currently advisory / deferred: the transaction-replay model already rejects any transaction whose chain_id differs from the running chain (a foreign-chain tx is dropped at mempool/block validation), so a distinct chain_id per network already prevents cross-network replay — network verification is therefore defense-in-depth rather than independently load-bearing. On an unconfigured genesis (chain_id == 0, dev/test) the binding degrades to a no-op, matching the transaction model.
• Per-relayer gas key. Each relayer instance MUST use its own gas key; a key shared across instances races on nonce selection.

​

12.3 Account Owner API (via Storage Manager Actor)

# Volume lifecycle
create_volume(
    volume_name: str,
    max_size_bytes: int = 0,
    erasure_k: int = 4,                       # data shards (default 4)
    erasure_m: int = 2,                       # parity shards (default 2)
    visibility: str = "PRIVATE",              # "PRIVATE" (encrypted, default) or "PUBLIC" (unencrypted, open reads)
    wrapping_key_policy: str = "owner-key",   # access class (§9.1): "owner-key" | "committee-only" | "tee-gated"
    initial_escrow: int = 0,                  # CBY prepaid toward storage rent (§10.3.1)
) -> bytes32  # returns volume_id; writes StorageCommitment + name index atomically (§11.3)

delete_volume(volume_name: str) -> bool           # soft-delete: sets status=DELETED, starts GC timer
list_volumes() -> list[VolumeInfo]
transfer_volume(volume_name: str, new_owner: address) -> bool  # transfer ownership to another account

# Key management (§9.4). Rewrap is the only one that exists; see §9.4.2 for rotation.
rewrap_volume(                                    # same DEK, rebound to the caller's release key material
    volume_name: str,
    wrapped_dek: bytes,                           # DEK re-wrapped by the caller; the chain cannot check it is the same key
    cbss_committee_epoch: int,                    # MUST equal the owner's current release epoch
    cbss_release_key_material_hash: bytes32,
) -> bool

# Lookup & cross-owner access
get_volume_id_by_name(owner: address, volume_name: str) -> bytes32   # (owner, name) → volume_id (§11.3)
update_mount_allowlist(volume_name: str, grant: MountGrant, remove: bool = False) -> bool  # owner-signed (§7.7); a direct-client grant names the grantee's signing key and, for PRIVATE volumes, seals the DEK to the grantee's X25519 key (§9.2.1)
open_volume(volume_name: str, owner: address = SELF) -> OpenVolumeResponse           # owner or allowlisted grantee (§7.7.1)

# Object management
delete_object(volume_name: str, object_path: str) -> bool
list_objects(volume_name: str, prefix: str = "") -> list[ObjectInfo]
get_volume_info(volume_name: str) -> VolumeInfo

# Billing
get_storage_usage() -> AccountStorageSummary
reserve_storage_balance(amount: uint256) -> bool

​

12.4 CIP-2 Task Definition Extension

struct OffchainTask:
    ... (existing CIP-2 fields) ...
    volume_attachments: list[VolumeAttachment]  # NEW: optional

​

12.3.1 Volume Creation and Lookup Errors

• E_VOLUME_NAME_DUPLICATE — (owner, volume_name) already present in the name index (§11.3).
• E_VOLUME_ESCROW_INSUFFICIENT — initial_escrow (or account balance) below MIN_STORAGE_BALANCE (§10.3.1).
• E_VOLUME_PARAM_INVALID — erasure k/m out of bounds, unknown visibility or wrapping_key_policy, or a volume_name violating the [a-zA-Z0-9_\-.] / 64-byte rule (§6.1).
• E_VOLUME_NOT_FOUND_FOR_OWNER — a name lookup missed the index for this owner-domain (§11.3); this MUST NOT be reported as a bare 404 that hides whether the caller queried under the wrong owner.
• E_VOLUME_RELAY_FLOOR — the relay set, auto-assigned or explicitly supplied, holds fewer than erasure_k + erasure_m distinct node_ids (§5.4.1). Counted by node_id, not by list length: a list that repeats one relay would write every shard of an object to one machine, so the erasure coding protects nothing. Such a volume can never serve a write while its escrow is locked to it, so creation MUST be refused rather than half-completed.
• E_VOLUME_RELAY_UNKNOWN — an explicitly supplied relay_nodes entry names a node_id the relay registry has never held. relay_nodes[shard_index] is the only per-index responsibility the chain records — a PoR challenge binds responsible_node_id to it (§7.6) and a rent assignment interval is opened against it (§5.6.1) — and neither issue path validates it: both range-check the index and take the node_id verbatim, so a challenge against such an index is accepted and bonded, and only settlement discovers the miss, emits an alarm and slashes nothing. Such an index can never produce a slash and is unauditable for the volume’s life, while every challenge against it spends a bond cycle to reach that alarm. Auto-assignment has always drawn from the registry; the explicit branch MUST be held to the registry too — to membership, not to the stricter Active-and-has-room filter auto-assignment applies when it is choosing. Registration, and deliberately NOT status or capacity. A relay profile row is never deleted — a full unstake zeroes the stake and marks the relay disabled but keeps the row — so registration is monotone, and a create-time check of it stays true for the life of the list §5.6.1 freezes. status and used capacity both move every epoch while no instruction rewrites relay_nodes, so a create-time check of either would assert a property that expires one block later; a full relay is already answered with ErrCapacity and falls through to an alternate (§5.4.1). This narrows no adversary’s choice of victim: the registry is public, so an owner can still name a registered relay it does not own. That residual is a consent defect, addressed by §5.6.2, not by this rule.

​

13. Garbage Collection

​

13.1 Triggers

1. Explicit deletion: Account owner calls delete_volume() (soft-delete + grace window, §13.2) or delete_object().
2. Escrow/balance exhaustion: after escrow is depleted and the account balance stays below MIN_STORAGE_BALANCE through STORAGE_GRACE_EPOCHS (§10.3). There is no separate expiry_height — volume lifetime is governed by escrow/rent, not a fixed height.

​

13.2 Process

​

13.3 Deletion Semantics

• On delete_object: A Public volume with a nonzero CIP-33 Store pin count rejects any object deletion that changes the committed manifest (§2.1.2 of CIP-33). delete_object has no separate onchain instruction; it submits a manifest commit and uses that commit’s CIP-43 access row. Otherwise, all shards of the object are marked for removal on their respective Relay Nodes. The onchain manifest is updated. Relay Nodes garbage-collect shard data asynchronously, but the object is immediately inaccessible to CapToken holders.
• On delete_volume: A Public volume with a nonzero CIP-33 Store pin count rejects owner deletion (§2.1.2 of CIP-33; node PR #1708 at d60babed). Otherwise, the volume enters the DELETED state (soft-delete). All active CapTokens are revoked. New CapTokens cannot be issued. Storage fees continue accruing during a grace window of VOLUME_DELETE_GRACE_EPOCHS (default: same as STORAGE_GRACE_EPOCHS). During this window, the account owner may call undelete_volume() to restore the volume to ACTIVE status. After the grace window, the volume transitions to GARBAGE_COLLECTING; both PoR sampling builders exclude it before drawing, including while its native rows remain. The epoch sweep prunes acknowledged event history. An independent, bounded native-inventory GC lane advances across blocks after the event-prune gate, deleting at most one primary row and its matching dense ordinal per item and durably decrementing live_shard_count. Its consensus-state progress cursor survives restart and rotates fairly among terminal volumes; a volume with the maximum 1,048,576 live rows makes progress across blocks rather than waiting for one daily rent epoch per batch. Native GC MUST NOT block later rent epochs or settle one epoch twice. The StorageCommitment remains in GARBAGE_COLLECTING while any native row remains. Only after the count reaches zero and the event-prune gate is satisfied may control-plane cleanup remove the volume from the live index. Relay Nodes may purge that terminal volume’s bytes asynchronously without waiting for its native rows to drain (§12.2); storage fees cease under the terminal status. Live-index removal does not delete the StorageCommitment: it remains as a terminal volume_id tombstone. CreateVolume rejects an occupied ID regardless of status, so rent-driven garbage collection cannot make that ID reusable (COW-3813).
• Rent exhaustion can still move a pinned Public Store volume to GARBAGE_COLLECTING and remove its bytes. The Trading Post pin count survives in separate actor storage, and the RAS commitment stays as an ID tombstone, but neither preserves those bytes. CIP-33 §2.1.2 records the unresolved availability and OneShot serving rule; the owner-deletion guard does not resolve it.
• Garbage collection is irreversible. Once a volume enters GARBAGE_COLLECTING, data cannot be recovered.

​

13.4 Ownership Transfer

• All active CapTokens are revoked (they were issued under the old owner’s authority).
• The StorageCommitment.owner field is updated atomically.
• The new owner assumes billing responsibility starting from the next epoch.
• All active CapTokens are revoked by advancing last_owner_transfer_block; a token whose valid_from is below it is refused. mount_allowed_actors is cleared in the same transaction.
• For private volumes, the owner wrap is re-encrypted to the new owner’s wallet-derived key as part of the transfer transaction. This requires the old owner to unwrap and re-wrap the DEK, so transfer is an interactive operation requiring the old owner’s cooperation.
• A committee wrap is not unaffected, contrary to an earlier revision of this clause: the transfer sets cbss_rewrap_required, and the Dispatcher refuses to serve a private attachment while it is set. The new owner must call RewrapVolume (§9.4.1) to rebind the wrap to their own release key material before any runner can mount the volume.
• Transfer is a rewrap, not a rotation (§9.4). The DEK is unchanged, so an old owner who held it — which the owner-wrap path requires, though a committee-only volume’s owner may never have — can still decrypt everything the volume held at transfer time and everything written afterwards under the same key. A new owner who needs that not to be true has no protocol remedy today (§9.4.2).
• For public volumes, no key re-wrapping is needed (no DEK exists).
• Transfer of a volume in DELETED or GARBAGE_COLLECTING status is rejected.
• A Public volume with a CIP-33 Store artifact pin cannot be transferred. The pin remains after artifact retirement while a pinned hire can execute; CIP-33 §2.1.2 defines its terminal release. Node #1708 at 194f400f adds hire-lifetime references to this guard.

​

13.5 Gateway HTTP Serving by Status

​

14. Parameters

​

15. Security Considerations

​

15.1 CapToken Forgery

​

15.2 Runner Compromise

• Byte quotas limit the damage per job.
• Prefix-confined commits: A WRITE_ONLY commit may change only manifest nodes under the committer’s prefix — enforced onchain for public volumes (parent-root CAS + changed-node prefix check) and via staged-commit + DEK-holder finalize for private volumes (§7.3.1). Out-of-prefix commits are rejected (public) or never finalized (private), not merely detected after the fact.
• Orphan shard GC: Shards not referenced by any committed manifest are garbage collected after ORPHAN_SHARD_TTL_SECS.
• No delete access: A compromised Runner cannot destroy existing data.

• TEE attestation: For sensitive workloads, require TEE-attested Runners (CIP-2 tee_required=true).
• Minimal scope: Use path_prefix to restrict access to only the necessary sub-path.
• Account owner discretion: READ_WRITE is an explicit opt-in; the account owner accepts the elevated trust.

​

15.3 Relay Node Compromise

​

15.4 Denial of Service

​

15.5 Key Management

• A Volume DEK is wrapped per the volume’s access class (§9.1): to the owner’s wallet-derived key, to the CBSS committee, or both. Losing the owner’s wallet key forfeits the owner-wrap path, but a committee-wrapped volume remains readable through an authorized runner. The wraps live on-chain in the StorageCommitment, so the DEK itself is never at risk of loss — only the ability to unwrap a given wrap.
• Dispatcher trust boundary. The Dispatcher is never on the key path: for private volumes the Runner completes the CBSS SealRequest itself (§9.2) and the Dispatcher only constructs and authorizes it, so the Dispatcher never sees a plaintext DEK. A compromised Dispatcher could mis-authorize a seal request — bounded by the same consensus-level trust already required to issue CapTokens — but cannot exfiltrate volume DEKs, since it never holds them.
• Rewrapping is not rotation. RewrapVolume (§9.4.1) and the ownership-transfer rewrap (§13.4) change which wrap is published; the DEK is unchanged, so every prior holder keeps a working key. Neither revokes anything, and this CIP does not permit either to be described as a security boundary. DEK rotation does not exist (§9.4.2): there is today no operation that withdraws a volume key, only ones that withdraw permission. A compromised owner key, or a compromised grantee that already decrypted its package, requires creating a new volume and re-encrypting into it by hand.
• Grant removal is not key revocation (§7.7). It deletes the grant’s own grantee_wrapped_dek and stops new CapTokens, which is a complete withdrawal for a grantee that never used its package — but it cannot reach a key already decrypted, and it does not close the path_tag history oracle, which is computed under the DEK and is permanent public chain state.

​

16. Implementation Notes

​

16.1 Canonical Implementation Stack

​

16.2 FUSE Mount Implementation

1. FUSE daemon (fuser crate): Implements the Filesystem trait, handling read, write, readdir, getattr, etc. Delegates to the local cache layer.
2. Local cache (tmpfs-backed): An in-memory filesystem that serves as the working copy. All reads/writes hit the cache first. The cache is populated lazily on first access (fetch from Relay Nodes) and eagerly for files modified locally.
3. Sync daemon (background task): Runs a push/pull loop at the configured sync_interval. Uses notify for detecting local changes and polls Relay Nodes for remote changes. Handles encryption, erasure coding, and shard distribution.

┌─────────────────────────────────────────────────────────┐
│ Container / Runner process                              │
│                                                         │
│  Model (Claude, Kimi-K2, etc.)                          │
│    │                                                    │
│    ├─ Read("/mnt/volumes/mem/state.json")               │
│    ├─ Write("/mnt/volumes/mem/state.json", data)        │
│    └─ Bash("ls /mnt/volumes/mem/")                      │
│         │                                               │
│         ▼                                               │
│  ┌─────────────┐     ┌──────────────┐                   │
│  │ FUSE daemon │◄───►│ Local cache   │                  │
│  │ (fuser)     │     │ (tmpfs)       │                  │
│  └─────────────┘     └──────┬───────┘                   │
│                             │                           │
│                      ┌──────▼───────┐                   │
│                      │ Sync daemon  │                   │
│                      │  push/pull   │                   │
│                      └──────┬───────┘                   │
│                             │                           │
└─────────────────────────────┼───────────────────────────┘
                              │ encrypt / erasure code / QUIC
                              ▼
                     ┌─────────────────┐
                     │   Relay Nodes   │
                     └─────────────────┘

​

16.3 Object API Client Library

// Programmatic usage (non-agentic runners):
let data = volume.get("state/memory.cbor").await?;
volume.put("state/memory.cbor", &updated_data).await?;

// Under the hood:
// get: fetch PlacementRecord → identify assigned nodes → fetch K shards
//      → Reed-Solomon reconstruct (truncate to ciphertext_size)
//      → verify ciphertext hash → AES-GCM decrypt → verify content hash
// put: AES-GCM encrypt → Reed-Solomon encode → distribute K+M shards
//      → produce ObjectDescriptor (manifest) + PlacementRecord (published at commit)

​

16.4 Relay Node Implementation

1. Blob store (cbfs-store): A sled-backed key-value store mapping (shard_id, shard_index) → shard_bytes. The shard_id is an opaque BLAKE3 hash (see §5.3) — Relay Nodes never see object paths, only opaque identifiers. This ensures the privacy guarantee in §9.3.
2. Placement store (cbfs-placement): A sled-backed store mapping shard_id → PlacementRecord. Stores the mutable shard-to-node assignments separately from shard data, enabling repair workers to read placement information without accessing the encrypted manifest.
3. RPC server: Accepts shard operations (PUT_SHARD, GET_SHARD, PROVE_SHARD) and placement operations (PutPlacement, GetPlacement, ReplicatePlacement). Client-facing operations are authenticated by the AuthProvider (§7.1.1); the node-to-node operations of §7.1.2, ReplicatePlacement among them, are authorized by relay assertion instead. For public volumes (visibility = PUBLIC), GET_SHARD and GetPlacement are served without CapToken verification (§7.6.3); write operations still require a CapToken. Relay Nodes do NOT expose any listing operation — object listing is performed client-side by reading the manifest. Requests are keyed by shard_id, not object path.
4. Repair loop (repair.rs): Periodically runs the two-phase autonomous repair cycle (§5.5). Uses a mutable peer list (Arc<RwLock<Vec<NodeInfo>>>) bootstrapped from seed peers in the node config and updated as new nodes are discovered.
5. Placement sync (placement_sync.rs): Replicates PlacementRecords to peer nodes to ensure all assigned nodes have a consistent view of shard assignments, and gossips peer lists via SyncPeers. Both are relay assertions under §7.1.2.
6. Heartbeat loop: Periodically calls heartbeat() on the onchain Relay Registry.
7. Spot-check responder: When challenged, returns a random chunk of a specified shard for integrity verification.

​

16.5 Performance Expectations

​

16.6 Runner ↔ Validator Wire Compatibility

• A runner’s cowboy-ras protocol types MUST be pinned compatibly with the deployed validator. A runner outside the validator’s supported wire range is rejected with a clear, typed reason (“unsupported transaction wire version — upgrade runner”) surfaced back to the submitter, not a silent drop or a generic decode error.
• Wire-format changes to runner↔chain transactions MUST be versioned (a discriminant the validator checks), and the validator SHOULD accept a defined compatibility window rather than only the exact current version, so a rolling upgrade does not strand every runner at once.
• Decode-reject reasons MUST be observable to the runner operator — a runner that cannot complete jobs because of wire skew has to be able to find out why.

​

16.7 Client Bootstrap and Discovery

• chain_id / network from chain-info, plus consensus parameters and the basefee schedule (GET /basefee, suggested_max_fee_*) — not hand-set flags. A wrong or missing chain_id/fee fails certificate auth or is silently dropped by the mempool (fee_below_basefee), so guessing is worse than discovering.
• Storage relays from the relay registry (§5.2), write-relayers from /ras/write-relayers (§12.2.1), and CBSS DEK delivery from chain (§9.2) — the seal committee from the CBSS committee registry (CIP-24) by epoch, and sealed ciphertext as a CBFS object from the relays already listed in the registry. None of it is client-configured.
• The RPC URL MUST be resolved once at startup and threaded everywhere. A subcommand MUST NOT silently fall back to a hardcoded default (e.g., 127.0.0.1:4000) or ignore an explicit --indexer-url; a missing or unreachable RPC is a clear error, not a silent localhost attempt.

​

17. Scope and Future Work

​

17.1 In Scope (This CIP)

• Private and public account-scoped volumes.
• READ_ONLY, WRITE_ONLY, and READ_WRITE CapToken access modes; PUBLIC volume visibility mode (§7.6).
• Concurrent CapTokens on the same volume (agent swarm pattern).
• Two client interfaces: FUSE filesystem mount (for agentic/LLM workloads) and Object API (for programmatic workloads).
• Hybrid sync strategy for FUSE mounts (local cache + background push/pull).
• Relay Nodes as a dedicated storage layer with staking and incentives.
• Reed-Solomon erasure coding (default 4/6) for durability.
• AES-256-GCM encryption with HKDF-derived keys.
• BLAKE3 content hashing at object, ciphertext, and shard levels.
• Path-based addressing with Merkle manifest for integrity proofs.
• Per-epoch billing with grace period and garbage collection.
• Lazy and proactive shard repair.
• Private dual-wrapped volumes with first-class access classes (owner-key, committee-only; tee-gated reserved), and owner-curated cross-owner mount allowlists that also authorize direct-client access under the grantee’s own delegation, with recipient-sealed DEK delivery (§7.7, §7.7.1, §9.2.1).
• Control-plane writes via the cowboy-ras-write-relayer, with client discovery of chain parameters, relays, write-relayers, and CBSS endpoints from chain (§16.7).

​

17.2 Explicitly Out of Scope

• Request-level possession proof for owner tokens: owner and grantee tokens are bearer after issuance, bounded by the §12.2.1 TTLs. A request-signing extension that binds each relay request to the delegated key is a follow-up covering both token kinds together (§7.7.1).
• Content-addressed retrieval: RAS uses path-based addressing only. CID-based retrieval (IPFS-style) may be layered on top in the future.
• Storage marketplace: A competitive market where Relay Nodes bid on storage deals (Filecoin-style) is a future extension; pricing is protocol-set.
• Alternative storage backends: This CIP standardizes on CBFS as the canonical storage layer. Supporting Filecoin, Arweave, or other backends under the same CIP-9 surface is future work and would require a follow-on standard.
• READ_UNCOMMITTED consistency mode: Pre-commit read visibility for real-time agent swarms. Requires a shard discovery mechanism (pubsub or uncommitted manifest fragments) not defined in this CIP. See §7.5.
• Rotation without re-encryption: advancing a volume’s key while leaving stored ciphertext openable by the old one. It provides no revocation and would let a commitment assert a rotation that did not happen. Proxy re-encryption, or any scheme that re-keys ciphertext without a full rewrite, would be a follow-on standard. DEK rotation itself is not out of scope — it is an unfilled gap, with its requirements and unsolved problems in §9.4.2–§9.4.3.
• Runner-initiated task dispatch: Allowing a coordinator Runner to dynamically spawn sub-agent tasks. Currently, all tasks must be dispatched by onchain Actors. This is a CIP-2 extension.
• Full container runtime spec: Container image management, container registries, resource limits, GPU passthrough, and network policies. CIP-9 defines the CBFS-backed storage primitive (including the FUSE mount and object API); a separate CIP will define the full container runtime that consumes it.

​

17.3 Implementation Status

• Publication-safe manifest-node reclamation (§12.2) — pending (COW-4300). At CBFS devnet efd80995, node/src/volume_events.rs::apply_event_batch deletes volume-bound shards on the removal event alone, including a freshly republished node needed by a newer commit. promote_added_shards warns when an added shard is missing but does not restore its bytes. The path does not yet enforce the publication hold and current-set checks above. Source inspection confirms this gap; the full end-to-end sequence remains to be reproduced. This specification amendment records the required behavior and does not fix the relay implementation.
• Canonical owner-authorization signing path — LANDED (COW-2636). The canonical owner-auth / owner-delegation / relay-auth signing bytes are now wired via cowboy-protocol-codec (cowboy_protocol_codec::ras), the single owner of the RAS wire contract; the node validator and the cbfs client both derive from it (no copies), golden-vector-locked with native == WASM. This supersedes the prior bincode-serialized path (formerly registry-proto/src/canonical.rs); delegation and relay-auth were canonicalized off bincode in the same change. Node PR #1037 additionally fixed the owner-delegation verifier to bind chain_id to the running chain (§12.2.1) rather than comparing the delegation’s chain_id to itself (which was a tautology permitting cross-chain replay).
• wrapping_key_policy (§9.1) — pending. The access-class field does not exist anywhere in the node; §9.1’s owner-key / committee-only / tee-gated table is a target, and no normative rule may be conditioned on reading it from state. The owner-wrap field is no longer pending: owner_wrapped_dek exists on StorageCommitment, on the create request and on the open/mount response, alongside the committee wrapped_dek. (An earlier revision of this bullet said otherwise; it was stale.)
• DEK rotation (§9.4.2) — a gap, not a design. RewrapVolume is live and behaves as §9.4.1 describes. Nothing rotates a DEK, so there is no operation that withdraws a volume key. A first design was drafted and withdrawn: review found it unsatisfiable against the shipped chain — a full-volume rewrite exceeds MAX_COMMIT_FINALIZE_SHARDS, CommitManifestFinalize carries no authorization to make the operation owner-only, clearing the allowlist does not invalidate issued CapTokens, and the party being rotated away retains the write authority needed to starve the rewrite. §9.4.3 records those obstacles so the next attempt begins from them.
• CapToken client-verifiable signature field (§7.1) — reserved. The CapToken signature field is reserved for a future client-verifiable token form and is presently zero; authority today is the byte-for-byte match against the stored token in Storage Manager actor state (§7.1).

​

Appendix A: Worked Examples

​

A.1 AI Agent with Persistent Memory (Filesystem Mount)

submit_task(
    task_definition=encode_task({
        "model": "claude-sonnet",
        "system": "You are a trading analyst. Your memory and portfolio are in /mnt/volumes/agent-memory/state/.",
        "prompt": "Analyze today's market conditions. Update your memory and portfolio.",
    }),
    volume_attachments=[
        VolumeAttachment(
            volume_name="agent-memory",
            access_mode=READ_WRITE,
            mount=True,                    # FUSE mount at /mnt/volumes/agent-memory/
            max_bytes=500 * 1024 * 1024,
        ),
    ],
    num_runners=1,
    timeout_blocks=500,
    proof_type_requested=TEE,
    ...
)

── Claude's tool calls during execution ──────────────────────────

1. Read("/mnt/volumes/agent-memory/state/memory.json")
   → Returns the agent's memory from yesterday's run
   → (Under the hood: FUSE → fetch shards from Relay Nodes → reconstruct → decrypt)

2. Read("/mnt/volumes/agent-memory/state/portfolio.json")
   → Returns the current portfolio state

3. Bash("ls /mnt/volumes/agent-memory/logs/")
   → 2026-03-01/  2026-03-02/  2026-03-03/

4. Bash("grep -r 'NVDA' /mnt/volumes/agent-memory/logs/2026-03-03/")
   → Shows yesterday's NVDA-related log entries

5. [Claude performs analysis, makes decisions]

6. Write("/mnt/volumes/agent-memory/state/memory.json", updated_memory)
   → Writes to local tmpfs instantly
   → (Under the hood: sync daemon pushes to Relay Nodes within 5 seconds)

7. Write("/mnt/volumes/agent-memory/state/portfolio.json", updated_portfolio)

8. Write("/mnt/volumes/agent-memory/logs/2026-03-04/analysis.json", todays_analysis)

── Container shuts down ──────────────────────────────────────────
   → Final sync pushes any remaining dirty files
   → Manifest committed onchain
   → tmpfs deleted. Data persists on Relay Nodes.

​

A.2 Distributed Scraping with Map-Reduce (Direct Object API)

for i, site in enumerate(sites):
    submit_task(
        task_definition=encode_task({"action": "scrape", "url": site}),
        volume_attachments=[
            VolumeAttachment(
                volume_name="scrape-results",
                access_mode=WRITE_ONLY,
                path_prefix=f"scraper-{i}/",
                max_bytes=500_000_000,
                mount=False,                   # no FUSE mount, direct object writes
            ),
        ],
        ...
    )

# Simple Python script, not an LLM. Uses the Object API directly.
for page in crawl(site_url):
    put_object(cap_token, f"scraper-{my_id}/{page.slug}.json", json.dumps({
        "url": page.url,
        "title": page.title,
        "content": page.text,
        "links": page.links,
    }))
commit_manifest(cap_token, manifest_root)

submit_task(
    task_definition=encode_task({
        "model": "kimi-k2",
        "system": "You have scraped web data in /mnt/volumes/scrape-results/. Analyze and collate.",
        "prompt": "Find all mentions of product launches across the scraped sites. Write a summary.",
    }),
    volume_attachments=[
        VolumeAttachment(
            volume_name="scrape-results",
            access_mode=READ_WRITE,
            mount=True,                        # FUSE mount for filesystem access
            max_bytes=100_000_000,
        ),
    ],
    ...
)

1. Bash("find /mnt/volumes/scrape-results -name '*.json' | wc -l")
   → 47

2. Bash("ls /mnt/volumes/scrape-results/")
   → scraper-0/  scraper-1/  scraper-2/  scraper-3/  scraper-4/

3. Bash("cat /mnt/volumes/scrape-results/scraper-0/about-page.json | jq '.title'")
   → "About Us - Acme Corp"

4. Bash("grep -rl 'product launch' /mnt/volumes/scrape-results/")
   → scraper-0/news.json
   → scraper-2/blog-post-3.json
   → scraper-4/press-release.json

5. Read("/mnt/volumes/scrape-results/scraper-0/news.json")
   → [full content]

6. [... reads relevant files, analyzes ...]

7. Write("/mnt/volumes/scrape-results/collated/summary.md", summary)
8. Write("/mnt/volumes/scrape-results/collated/product-launches.json", structured_data)

​

A.3 Agent Swarm with Batch Coordination (Filesystem Mount + Concurrent Writers)

# Coordinator: READ_WRITE mount, sees the full volume
submit_task(
    task_definition=encode_task({
        "model": "claude-sonnet",
        "system": "You are coordinating 5 research agents. Their reports will appear in /mnt/volumes/swarm/agent-*/. Poll for new reports and synthesize findings. Reports appear in batches as each agent completes and commits.",
    }),
    volume_attachments=[
        VolumeAttachment(volume_name="swarm", access_mode=READ_WRITE, mount=True,
                         sync_interval=5, max_bytes=100_000_000),
    ],
    timeout_blocks=2000,
    ...
)

# Sub-agents: WRITE_ONLY mount, scoped to their prefix
for i in range(5):
    submit_task(
        task_definition=encode_task({
            "model": "claude-haiku",
            "system": f"You are research agent {i}. Write your findings to /mnt/volumes/swarm/.",
            "prompt": f"Research: {topics[i]}",
        }),
        volume_attachments=[
            VolumeAttachment(volume_name="swarm", access_mode=WRITE_ONLY, mount=True,
                             path_prefix=f"agent-{i}/", max_bytes=50_000_000),
        ],
        timeout_blocks=1000,
        ...
    )

1. [Performs research using web tools]

2. Write("/mnt/volumes/swarm/report.md", research_findings)
   → (Visible path: agent-2/report.md due to prefix scoping)

3. Write("/mnt/volumes/swarm/sources.json", source_list)
   → (Visible path: agent-2/sources.json)

── Container shuts down ──
── Sync daemon pushes all shards to Relay Nodes ──
── Runtime calls commit_manifest(), updating onchain manifest_root ──
── Agent-2's files are now committed and visible to other readers ──

1. Bash("ls /mnt/volumes/swarm/")
   → (empty — no sub-agents have committed yet)

   [Waits... sync daemon re-fetches manifest every 5 seconds,
    verifying against onchain manifest_root (§7.5)]

   [Agent-0 finishes and commits its manifest]

2. Bash("ls /mnt/volumes/swarm/")
   → agent-0/
   → (Agent-0 committed — all its files appear at once)

3. Read("/mnt/volumes/swarm/agent-0/report.md")
   → [agent-0's findings]

   [Agent-1 and agent-3 finish and commit around the same time]

4. Bash("ls /mnt/volumes/swarm/")
   → agent-0/  agent-1/  agent-3/
   → (Two more agents committed since last check)

5. Read("/mnt/volumes/swarm/agent-1/report.md")
   → [agent-1's findings]

6. Read("/mnt/volumes/swarm/agent-3/report.md")
   → [agent-3's findings]

   [... continues polling until all 5 agents have committed ...]

7. Write("/mnt/volumes/swarm/synthesis/final-report.md", synthesized_findings)

​

A.4 Multi-Stage Pipeline with Handoff

submit_task(
    task_definition=encode_task({"stage": "preprocess", "source": "https://data.example.com/feed"}),
    volume_attachments=[
        VolumeAttachment(volume_name="pipeline", access_mode=WRITE_ONLY,
                         path_prefix="stage1/", mount=False, max_bytes=2_000_000_000),
    ],
    ...
)

submit_task(
    task_definition=encode_task({
        "model": "kimi-k2",
        "system": "Input data is in /mnt/volumes/pipeline/stage1/. Run inference and write results to /mnt/volumes/pipeline/stage2/.",
    }),
    volume_attachments=[
        VolumeAttachment(volume_name="pipeline", access_mode=READ_WRITE,
                         mount=True, max_bytes=1_000_000_000),
    ],
    ...
)

​

Appendix B: Comparison with Existing Systems

​

Appendix C: CapToken Wire Format

CapToken (variable length)
┌──────────────────────────────────────────────────┐
│ version:         u8          (1 byte) [reserved]  │
│ volume_id:       bytes32     (32 bytes)          │
│ access_mode:     u8          (1 byte) [0=RO,1=WO,2=RW] │
│ path_prefix_len: u16         (2 bytes)           │
│ path_prefix:     bytes       (variable)          │
│ max_bytes:       u64         (8 bytes)           │
│ valid_from:      u64         (8 bytes)           │
│ valid_until:     u64         (8 bytes)           │
│ runner_address:  bytes32     (32 bytes)          │
│ nonce:           u64         (8 bytes)           │
│ caveats_hash:    bytes32     (32 bytes) [reserved]│
│ signature:       bytes64     (64 bytes)          │
└──────────────────────────────────────────────────┘

​

Appendix D: “Why Won’t My Volume Mount?”