​

CIP-9: Runner-Backed Storage

​

1. Abstract

• Account-scoped: All storage is owned by and billed to a Cowboy account (EOA or Actor).
• Private by default, public by choice: Private volumes are encrypted client-side before leaving the Runner; only the owning account can decrypt. Public volumes (visibility = PUBLIC) are stored unencrypted and readable by any party without a CapToken, enabling use cases like web asset hosting (CIP-15) and public datasets.
• Access-controlled: Runners receive scoped capability tokens granting read-only, write-only, or read-write access to a storage volume. Multiple concurrent CapTokens may be active on the same volume simultaneously. Public volumes require CapTokens only for writes, not reads.
• Addressable: Every stored object has a deterministic path within the account’s storage namespace, enabling targeted reads and deletes by the account owner.
• Durable: Objects are erasure-coded (Reed-Solomon) and distributed across multiple Relay Nodes, tolerating node failures without data loss.
• Metered: Storage is billed in CBY via a per-epoch, per-byte fee model (including erasure coding overhead) that extends the Cells concept from CIP-3 to persistent off-chain blobs.

​

2. Motivation

1. Secret management. Runners need access to API keys, certificates, and credentials without exposing them on the public chain. These secrets must only be decryptable by authorized Runners, optionally gated by TEE attestation.
2. Large off-chain output. Computation frequently produces artifacts exceeding the 64 KiB onchain inline cap (whitepaper §7). LLM inference may generate images, audio, datasets, or model weights that are too large for result_data and inefficient to pass through onchain callbacks. These must be stored off-chain with a verifiable content commitment anchored onchain.
3. Multi-step data flow. Iterative computation (AI training loops accumulating weights across invocations), stateful agents (conversation history, tool outputs, learned preferences persisting across scheduling cycles), pipeline handoff (Runner A produces intermediate data consumed by Runner B), and agent swarms (coordinator + concurrent sub-agents sharing a volume) all require persistent, addressable storage between jobs.
4. Container persistence. CIP-10 container runtimes require persistent storage layers for stateful applications, databases, and caching across job restarts. Without attachable storage, containers are limited to ephemeral scratch space.

​

2.1 Why not existing systems?

​

3. Definitions

• Volume: A named, account-scoped storage namespace. An account may own multiple Volumes. Each Volume is an isolated container for stored objects.
• Object: A single blob of data stored within a Volume, identified by a path key.
• CBFS: The canonical off-chain storage engine used by this CIP. CBFS nodes implement the Relay Node data plane and CBFS clients implement the object API, manifest handling, and FUSE mount described here.
• Storage Commitment: An onchain record that tracks a Volume’s existence, owner, size, shard placement, creation epoch, and billing state.
• Capability Token (CapToken): A cryptographic bearer token encoding the permissions (read-only, write-only, or read-write), scope (volume + path prefix), time bounds, and size quota for a client’s access to a Volume. The client may be a Runner during a job, the owner via the CLI/SDK, or any other authorized caller.
• Relay Node: A CBFS storage node participating in the Cowboy network that persistently stores erasure-coded shards of Volume data. Relay Nodes are a distinct network role from Runners and Validators, with their own staking and incentive model.
• Shard: A fragment of an erasure-coded object. An object is split into K data shards and M parity shards (K+M total); any K shards are sufficient to reconstruct the original object.
• ObjectDescriptor: The immutable content-identity record for a stored object. Contains the path, content hash, ciphertext hash, encryption nonce, size, erasure parameters, and per-shard hashes. Stored inside the encrypted manifest via ManifestEntry::File.
• PlacementRecord: The mutable shard-to-node assignment record for a stored object. Contains a shard_id, a list of PlacementAssignment (shard_index → node_id), duplicated erasure params and shard hashes (so repair workers can operate without manifest access), a ciphertext_size (the pre-padding ciphertext length needed for correct erasure reconstruction), and a CAS version for atomic updates. Replicated to all participating Relay Nodes independently of the manifest.
• Volume DEK: The per-volume AES-256 data-encryption key. It is wrapped one or more ways according to the volume’s access class (§9.1) — to the owner’s wallet-derived key, to the CBSS committee, or both.
• CBSS: The Cowboy Secret Service — the threshold committee that holds the IBE capability to unwrap a committee-wrapped Volume DEK on behalf of an authorized Runner (§9.2).
• Write-Relayer: The cowboy-ras-write-relayer service, which submits owner-signed RAS control-plane transactions (volume create, manifest commit, allowlist updates) and pays their gas on a client’s behalf (§12.2.1).
• Dispatcher: The Job Dispatcher system actor (0x02), defined by CIP-2 — the in-consensus job-dispatch logic that selects Runners (VRF + committee filtering), issues and revokes CapTokens, and manages job lifecycle. It is not a separate off-chain service: its actions execute inside block processing at consensus trust level. In CIP-9 the Dispatcher additionally applies the storage-capability filter (§5.1.1), scopes a CapToken to a mount grant for cross-owner access (§7.7), and constructs and authorizes the CBSS SealRequest for private-volume key delivery (§9.2) — it never handles a plaintext DEK.

​

3.1 Canonical Manifest DAG

1. The manifest is the set of ManifestEntry (cbfs/types): File { descriptor, metadata }, Symlink { … }, Directory { … }, ordered lexicographically by path() (UTF-8 byte ordering).
2. Entries are grouped into leaf nodes by content-defined chunking: once a node holds at least MANIFEST_NODE_MIN_ENTRIES (64) entries, a boundary is cut at the first entry whose domain-separated hash hits the target — BLAKE3("cbfs.manifest-entry-boundary.v1" || encoded_entry) with its low MANIFEST_NODE_TARGET_BITS (10) bits all zero (≈1024-entry target) — and is forced at MANIFEST_NODE_MAX_ENTRIES (2048) or MANIFEST_NODE_MAX_BYTES (256 KiB). This is a per-entry hash test, not a rolling hash. A Leaf { entries } carries its slice of the sorted entries.
3. Leaf nodes are reduced into interior nodes the same way (boundary domain cbfs.manifest-child-boundary.v1); an Interior { children } carries ManifestChild { min_path, locator, subtree_count } pointers. Levels reduce bottom-up until one node remains.
4. Each node is content-addressed by its locator — the BLAKE3 hash of its canonical encoding (magic CBMN/v1 for plaintext nodes, CBME/v1 for encrypted). The root is the locator of the top node; manifest_root is that value.

​

4. Design Overview

​

4.1 Architecture

​

4.2 Lifecycle

1. Create: Account owner creates a Volume via the Storage Manager system actor, specifying a name, optional size quota, and replication parameters. An onchain Storage Commitment is written.
2. Attach: When submitting a CIP-2 task, the owner includes one or more volume attachments in the task definition, each specifying the volume name, access mode (read-only, write-only, or read-write), and an optional path prefix scope.
3. Authorize: The Dispatcher (or a delegated Storage Manager) issues a CapToken to each selected Runner. The token is scoped to the specified volume, access mode, path prefix, job duration, and byte quota.
4. Write: The Runner encrypts object data, erasure-codes it into K+M shards, and distributes shards to Relay Nodes. Shards are immediately available for retrieval by other CapToken holders on the same volume.
5. Read: The Runner fetches any K of K+M shards from Relay Nodes, reconstructs the object, decrypts, and verifies the content hash.
6. Commit: At job completion (or periodically), the Runner commits a storage manifest — a Merkle root of all objects written — to the onchain Storage Commitment.
7. Manage: The account owner can list, read, and delete individual objects or entire Volumes at any time via the Storage Manager.

​

4.2.1 Committee Composition for Storage-Attached Jobs

​

4.2.2 Dependency on Runner-Backed Continuations

​

4.3 Canonical Implementation Boundary

• Object encryption/decryption for PRIVATE volumes and plaintext handling for PUBLIC volumes.
• Reed-Solomon erasure coding, shard hashing, and shard placement records.
• Relay Node RPCs (PUT_SHARD, GET_SHARD, LIST_SHARDS, placement replication, repair traffic).
• Manifest storage, reconstruction, Merkle root computation, and verification against the authoritative root.
• FUSE mount behavior, local cache, sync daemon, and direct object API.
• Shard repair, garbage collection of orphan shards, and local usage reporting hooks.

• StorageCommitment lifecycle, authoritative manifest_root, and volume ownership semantics.
• CapToken issuance, revocation, and task-scoped attachment semantics.
• Relay Registry membership, staking, health, and repair coordination triggers.
• Billing, fee settlement, storage grace periods, and slashing policy.
• Integration with CIP-2 task submission and the Runner execution flow.

​

4.4 Relationship to Existing CIPs

• CIP-2 (Off-Chain Compute): RAS extends the OffchainTask definition to include volume attachments. The Runner Submission Contract is extended to accept storage manifests alongside result_data.
• CIP-3 (Fee Model): RAS introduces a new fee dimension — persistent storage fees — billed per byte per epoch (including erasure overhead). Unlike CIP-3 Cycles and Cells (which are metered by the VM during transaction execution), storage usage is metered externally by Relay Nodes and settled onchain via attestation-based billing.
• CIP-4 (State Storage): onchain Storage Commitments live in the existing STORAGE key space under the Storage Manager actor’s address. Volume data itself is NOT stored in the MPT trie.
• CIP-10 (Runner Container Runtime): CIP-10 consumes the storage primitive defined here. Container image handling, cgroups, network policy, and GPU passthrough are separate concerns; volume attachment, mount semantics, and the object API are defined by CIP-9 and then mounted into CIP-10 runtimes.

​

5. Relay Nodes

​

5.1 Role and Responsibilities

• Stores erasure-coded shards of encrypted Volume data.
• Stores PlacementRecord entries that map shard IDs to their assigned nodes (see §5.3.1).
• Serves shards and placement records to authorized requesters (Runners with valid CapTokens, account owners).
• Heartbeats to the onchain Relay Registry to prove liveness.
• Runs autonomous two-phase repair (self-heal + redundancy restoration) without Runner involvement (see §5.5).
• Replicates PlacementRecords to peer nodes via ReplicatePlacement RPCs.

​

5.1.1 Runner Storage Capability

• storage_support: bool — the Runner is willing and configured to mount volumes.
• encryption_pubkey: [u8; 32] — the Runner’s x25519 public key, used as the recipient key for sealed DEK delivery (§9.2).

​

5.2 Relay Registry

RelayNodeProfile {
  address:           bytes32,
  stake_amount:      u256,           // CBY staked
  capacity_bytes:    u64,            // advertised storage capacity
  used_bytes:        u64,            // current usage
  last_heartbeat:    u64,            // block height
  health:            u8,             // decays per block, reset on heartbeat
  shards_held:       u32,            // number of active shards
  shards_lost:       u32,            // historical shard losses (reputation)
  region_hint:       bytes4          // optional: geographic hint for latency optimization
}

• Register: Relay Node stakes MIN_RELAY_STAKE CBY and calls register_relay(capacity_bytes).
• Heartbeat: Relay Node calls heartbeat() periodically. Health resets to MAX_RELAY_HEALTH (e.g., 100 blocks). Health decays by 1 per block.
• Removal: If health reaches 0, the Relay Node is removed from the active list. Shards assigned to it are flagged for repair (see §5.5).
• Unstake: A Relay Node may unstake after a cooldown period (RELAY_UNSTAKE_DELAY), provided it has no active shard assignments or has transferred them.

​

5.3 Shard Assignment

1. Eligible set: All Relay Nodes in the active list with health > MIN_HEALTH_FOR_ASSIGNMENT and sufficient free capacity.
2. Diversity: Selected nodes SHOULD have distinct region_hint values (best-effort, not enforced).
3. Determinism: The initial assignment is recorded in the object’s PlacementRecord (replicated to Relay Nodes, §5.3.1) so any future reader or repair worker knows which Relay Nodes hold which shards.

ObjectDescriptor {
  object_path:      string,
  write_id:         bytes16,          // CSPRNG(16) — fresh random per write, ensures version isolation
  content_hash:     bytes32,          // BLAKE3 hash of plaintext
  ciphertext_hash:  bytes32,          // BLAKE3 hash of ciphertext (pre-erasure-coding)
  encryption_nonce: bytes12,          // random nonce used for AES-256-GCM (unique per write)
  size_bytes:       u64,              // original object size
  ciphertext_size:  u64,              // ciphertext length before erasure padding (needed for correct reconstruction)
  shard_id:         bytes32,          // BLAKE3(volume_id || object_path || write_id) — opaque, version-unique
  erasure_k:        u8,
  erasure_m:        u8,
  shard_hashes:     [bytes32; K+M],   // BLAKE3 of each whole shard (integrity / repair)
  shard_chunk_roots:[bytes32; K+M],   // per-shard chunk Merkle root; PoR commits these via shard_root (§5.6)
}

PlacementRecord {
  shard_id:         bytes32,          // matches ObjectDescriptor.shard_id
  version:          u64,              // CAS version — monotonically incremented on each reassignment
  assignments: [
    { shard_index: u8, node_id: bytes32 },
    ...  // K+M entries
  ],
  // Duplicated from ObjectDescriptor so repair workers can operate
  // without manifest access (critical for private volumes where the
  // manifest is encrypted):
  erasure_k:        u8,
  erasure_m:        u8,
  ciphertext_size:  u64,
  shard_hashes:     [bytes32; K+M],
  shard_chunk_roots:[bytes32; K+M],   // duplicated so relays can build PoR proofs without the manifest (§5.6)
}

​

5.3.1 Placement Persistence

​

5.3.2 GET_MANIFEST RPC

GET_MANIFEST { volume_id: bytes32 }
  → { manifest: bytes, manifest_root: bytes32 }

​

5.4 Erasure Coding

1. Generate write_id = CSPRNG(16) and compute shard_id = BLAKE3(volume_id || object_path || write_id).
2. Encrypt the object (AES-256-GCM, see §9). Record ciphertext_size (pre-padding length).
3. Erasure-code the ciphertext into K data shards and M parity shards using Reed-Solomon (reed-solomon-erasure crate).
4. Compute shard_hash = BLAKE3(shard_bytes) for each shard.
5. Select K+M Relay Nodes and PUT each shard to its assigned node, authenticated by the CapToken.
6. Produce an ObjectDescriptor (stored in the manifest) and a PlacementRecord (published to Relay Nodes at commit time, §5.3.1).

1. Look up the ObjectDescriptor from the manifest and the PlacementRecord (from local cache or fetched from Relay Nodes via GetPlacement).
2. Request any K shards from available Relay Nodes listed in the PlacementRecord (prefer lowest-latency nodes; fall back if some are unavailable).
3. Reconstruct the ciphertext using Reed-Solomon decoding, truncating to ciphertext_size to remove erasure padding.
4. Verify ciphertext_hash.
5. Decrypt (AES-256-GCM).
6. Verify content_hash.

​

5.5 Autonomous Shard Repair

1. Verify the local shard bytes against the expected shard_hash from the PlacementRecord.
2. If the shard is missing or corrupt, fetch K healthy shards from peer Relay Nodes listed in the same PlacementRecord.
3. Reconstruct the ciphertext using Reed-Solomon decoding, truncating to ciphertext_size (not the padded shard length — using the wrong length produces garbage).
4. Re-encode and extract the needed shard. Verify its hash. Store locally.

1. For each PlacementRecord, probe all assigned nodes. A node is considered dead if it is unreachable after the configured timeout.
2. Leader election: Only the assigned node with the lowest shard_index among live nodes drives reassignment for that PlacementRecord. This prevents multiple nodes from racing to repair the same shard.
3. The leader selects replacement node(s) from the known peer list (excluding already-assigned nodes).
4. The leader reconstructs the missing shard(s) from K surviving shards (same erasure reconstruction as Phase 1) and uploads via PutShard to the replacement node(s).
5. The leader writes an updated PlacementRecord with the new assignments and an incremented version, using a CAS (Compare-and-Swap) write: the PutPlacement RPC specifies expected_version = current_version. If another node raced and already updated the record, the CAS fails and the leader retries from step 1 on the next cycle.
6. The updated PlacementRecord is replicated to all assigned nodes via ReplicatePlacement.

​

5.6 Proof of Retrievability (PoR) Challenges

1. A periodic onchain timer (CIP-5, every POR_CHALLENGE_INTERVAL blocks) selects a random set of (shard_id, shard_index, byte_offset, byte_length, nonce) challenges targeting active Relay Nodes. The nonce is fresh per challenge.
2. The challenged Relay Node must respond within POR_RESPONSE_WINDOW blocks with:
   • The requested byte range of the specified shard. The nonce seeds which range is asked, so a relay cannot precompute or replay a prior response.
   • A range-inclusion proof: a Merkle path from the fixed-size chunks (e.g. 4 KiB) covering the requested range to the shard’s chunk-root — the Merkle root over that shard’s chunks. (The relay already builds this chunk tree to serve byte ranges.)
   • A commitment proof: a path from that chunk-root to the volume’s onchain shard_root (§11.1) — the public commitment over the volume’s per-shard chunk-roots, maintained incrementally from the added_shards / removed_shards deltas every commit_manifest carries (§12.2).
3. The onchain verifier checks, against the shard_root snapshot carried in the challenge:
   • the returned bytes hash into the chunk leaves of the range-inclusion proof, which resolves to a chunk-root;
   • that chunk-root resolves to the onchain shard_root;
   • if either fails, the response is invalid.

• Valid response within POR_RESPONSE_WINDOW: bond is refunded minus POR_CHALLENGE_FEE (default 1 CBY, retained by 0x0B).
• Miss or fraudulent response: bond is refunded in full, and the challenger additionally receives CHALLENGER_BOUNTY (default 5 CBY) from the PoR challenge pool. The Relay’s slash (POR_MISS_PENALTY / POR_FRAUD_PENALTY / RELAY_EVICTION_PENALTY) follows the same 10/2/88 burn/challenge-pool/Relay-pro-rata split as storage fees, with the slashed Relay excluded from the 88% distribution that epoch (CIP-31 §8).

​

5.7 Relay Node Incentives

1. Storage fees: A share of the per-epoch storage fee paid by the account owner, proportional to the number and size of shards held.
2. Transfer fees: A per-byte fee for serving shards to Runners (read bandwidth).

Per epoch, for each volume:
  total_storage_fee = volume_effective_size * STORAGE_FEE_PER_BYTE_PER_EPOCH
  per_relay_share = total_storage_fee / num_relay_nodes_holding_shards

Per read operation:
  transfer_fee = bytes_served * TRANSFER_FEE_PER_BYTE
  (paid to the specific Relay Node serving the shard)

​

5.8 Relay Drain Governance

pub enum ProposalPayloadKind {
    UpdateBasefeeConfig,    // CIP-3
    DrainRelay,             // §5.8, opcode 85
    UpdateAutoDrainPolicy,  // §5.8, opcode 86
}

​

6. Storage Addressing

​

6.1 Path-Based Namespace

/{account_address}/{volume_name}/{object_path}

• account_address: The 32-byte Ed25519 public key of the owning account (hex-encoded).
• volume_name: A UTF-8 string (max 64 bytes, restricted to [a-zA-Z0-9_\-.]).
• object_path: A UTF-8 string (max 512 bytes) using / as a logical separator. No leading or trailing /.

/0xaabb...ccdd/model-weights/v3/layer_0.bin
/0xaabb...ccdd/agent-memory/conversations/2026-03-04/session_1.cbor
/0xaabb...ccdd/pipeline-scratch/job_4821/intermediate.parquet

​

6.2 Content Integrity

content_hash     = BLAKE3(plaintext_bytes)       // verifies decrypted content
ciphertext_hash  = BLAKE3(ciphertext_bytes)      // verifies pre-erasure-coding ciphertext
shard_hash[i]    = BLAKE3(shard_bytes[i])        // verifies a whole shard (integrity / repair)
chunk_root[i]    = merkle_root(fixed-size chunks of shard_bytes[i])  // PoR spot-check anchor (§5.6)

• Integrity verification: Any party can verify the full object lifecycle — shard hashes verify individual shards, ciphertext hash verifies reconstruction, content hash verifies decryption.
• Efficient proofs: a path from a leaf node to the DAG root is O(log N) in the number of objects. For a public volume this proves object membership directly; for a private volume the nodes are ciphertext, so object membership is not chain-verifiable from manifest_root alone — shard-level retrievability is anchored separately by the public shard-hash root (§5.6).

​

6.3 Addressing for Deletion

delete_object(volume_name, object_path) -> bool
delete_volume(volume_name) -> bool

• delete_object marks all shards of the object for removal on the assigned Relay Nodes and updates the onchain manifest. Relay Nodes garbage-collect the shard data.
• delete_volume initiates a soft delete: it sets the volume’s status to DELETED and starts the GC grace timer (§13.2). The StorageCommitment and rent persist through the grace window — the owner may call undelete_volume() to restore the volume — after which it is garbage-collected and fees cease.
• Only the account owner (or an Actor acting on behalf of the account) may delete. Runners are never granted delete permission, even under read-write mode — this prevents malicious or compromised Runners from destroying data.

​

7. Access Control

​

7.1 Capability Tokens

CapToken {
  volume_id:      bytes32,          // keccak256(account_address || volume_name)
  access_mode:    ENUM { READ_ONLY, WRITE_ONLY, READ_WRITE },
  path_prefix:    string,           // scope to objects under this prefix (e.g., "agent-3/")
  max_bytes:      u64,              // maximum total bytes writable
  valid_from:     u64,              // block height
  valid_until:    u64,              // block height (job timeout)
  runner_address: bytes32,          // the specific Runner authorized
  nonce:          u64,              // prevents replay
  signature:      bytes64           // reserved; presently zero — authority is the stored-token match, see below
}

​

7.1.1 Auth Enforcement on Placement RPCs

​

7.2 Access Modes (CapToken Scopes)

Note — Public volumes: Volumes with visibility = PUBLIC use a different access model. Reads and listings are open to any party without a CapToken. Writes still require a CapToken (WRITE_ONLY or READ_WRITE). See §7.6 for the full PUBLIC volume specification.

​

7.3 Concurrent CapTokens

• Non-overlapping write prefixes: If two CapTokens grant WRITE access, their path_prefix values MUST NOT overlap. The Dispatcher enforces this at token issuance.
  • Prefix canonicalization: Prefixes are canonicalized by ensuring a trailing / separator. A prefix agent-1 is stored as agent-1/. This prevents ambiguity: agent-1/ and agent-10/ are non-overlapping; agent-1/ and agent-1/sub/ DO overlap (the first is a parent of the second). Overlap is defined as: prefix A overlaps prefix B if A is a prefix of B or B is a prefix of A (after canonicalization).
  • Empty prefix ("") means full volume access. No other WRITE CapToken may be active on the volume simultaneously if any token has an empty prefix.
• Reads never conflict: READ_ONLY tokens may coexist with any number of other READ_ONLY or WRITE tokens. A READ_WRITE token may read paths being written by other tokens.
• No total ordering of writes: Concurrent writes to different paths are independent. There is no global write ordering across CapTokens.
• CapToken revocation: A CapToken can be revoked before its valid_until by the Dispatcher recording the token’s nonce in a revocation list. Relay Nodes check the revocation list on each request. Writes in-flight at revocation time may or may not land; the next manifest commit determines the canonical state. Revocation is best-effort and convergent — Relay Nodes may serve a revoked token briefly until the revocation propagates.

​

7.3.1 Prefix Enforcement Boundaries

• Parent-root CAS (all volumes). Every commit names the prev_root it extends; the chain rejects it unless prev_root equals the volume’s current manifest_root (enforced today). A stale or racing committer cannot silently overwrite an intervening commit — it must re-base on the current root and retry. This alone removes the lost-update / replace-from-stale-base class of clobber.
• Prefix-confined commits. A WRITE_ONLY committer may change only manifest nodes under its own path_prefix. How this is enforced depends on visibility, because the chain can only inspect paths it can see:
  • PUBLIC volumes — manifest nodes are plaintext (§3.1), so the chain verifies prefix-confinement directly: a commit whose changed nodes (relative to prev_root) touch any path outside the committer’s prefix is rejected onchain. Prevention, not detection.
  • PRIVATE volumes — manifest nodes are encrypted (§3.1), so paths are not visible onchain; the chain cannot check them without breaking the privacy guarantee (§9.3), and sampling cannot prove the absence of an out-of-prefix path. Prevention therefore routes through a DEK holder: a WRITE_ONLY sub-agent stages its prefix-scoped delta (CommitManifestStage, attributed to staged_by) but cannot finalize the canonical root; the DEK-holding coordinator or owner decrypts the staged delta, verifies prefix-confinement, and issues CommitManifestFinalize. A sub-agent therefore cannot unilaterally set manifest_root, so it cannot clobber another prefix.

​

7.4 Caveats and Restrictions

• Path prefix narrowing: A CapToken scoped to job_4821/ can be further restricted to job_4821/checkpoints/ but never broadened to /.
• Byte quota reduction: A 1 GiB quota can be reduced to 512 MiB but never increased.
• Time window narrowing: The valid window can be shortened but never extended.

​

7.5 Read Consistency

1. Fetch the manifest DAG nodes from Relay Nodes (starting at the root locator) and reconstruct them.
2. Recompute the root locator of the reconstructed DAG (§3.1).
3. Compare the computed root to the onchain manifest_root in the volume’s StorageCommitment.
4. Reject on mismatch. A mismatched root means the fetched manifest is stale, partially published, or corrupted.

Future work — READ_UNCOMMITTED: A mode where objects are visible as soon as shards land on Relay Nodes (before commit_manifest()) is desirable for the real-time agent swarm pattern, where latency matters more than strict consistency. However, it is not implementable on the current design for two reasons:

1. Discovery: With versioned shard IDs (§5.3, write_id in the shard address), a reader cannot predict the shard_id for an uncommitted write — they don’t know the writer’s random write_id. Discovering uncommitted objects requires a separate metadata channel (pubsub or uncommitted manifest fragments) that this spec does not yet define.
2. Prefix safety: prefix-confinement is enforced at commit (§7.3.1); an uncommitted read would bypass that gate and could observe out-of-prefix shards from a rogue writer before the commit that would reject them.

READ_UNCOMMITTED is deferred to a future CIP that defines the discovery mechanism.

​

7.6 Public Volumes (PUBLIC)

​

7.6.1 Overview

​

7.6.2 Properties

• No encryption: Objects in PUBLIC volumes are stored unencrypted on Relay Nodes. No DEK is generated for the volume. The wrapped_dek field in the StorageCommitment is empty.
• No CapToken for reads: Any party can fetch shards from Relay Nodes without presenting a CapToken. Relay Nodes serve GET_SHARD requests for public shards unconditionally.
• CapToken still required for writes: Only the account owner (or authorized Runners via CapToken) can write to the volume. Write access uses the same CapToken mechanism as private volumes.
• Content integrity preserved: content_hash (BLAKE3) is still computed and stored for every object. Readers MUST verify the content hash after shard reconstruction to detect corruption or tampering.
• Erasure coding preserved: Reed-Solomon coding applies identically. The only change is that the input to erasure coding is plaintext (not ciphertext).
• Billing unchanged: The account owner pays the same per-epoch, per-byte storage fees as private volumes.
• Listing is public: list_objects for public volumes does not require a CapToken. The manifest is stored unencrypted and readable by anyone who traverses the manifest DAG from the on-chain root (§3.1) or calls GET_MANIFEST (§5.3.2).

​

7.6.3 Relay Node Behavior

• GET_SHARD requests are served without CapToken verification (authorized via shard metadata).
• GET_MANIFEST (§5.3.2) is served without a CapToken — the same open-read rule.
• LIST_SHARDS is not exposed; listing is performed client-side from the manifest.
• PUT_SHARD requests still require a valid CapToken with write access. The AuthProvider attaches { "visibility": "PUBLIC" } metadata to the AuthDecision, which the Relay Node persists alongside the shard.

​

7.6.4 Shard ID Opacity

​

7.6.5 Content-Type Metadata

{
  "defaults": {
    ".html": "text/html; charset=utf-8",
    ".css": "text/css",
    ".js": "application/javascript",
    ".json": "application/json",
    ".png": "image/png",
    ".jpg": "image/jpeg",
    ".svg": "image/svg+xml",
    ".woff2": "font/woff2"
  },
  "overrides": {
    "data/feed.xml": "application/atom+xml"
  }
}

​

7.6.6 Cache Headers

{
  "default_max_age": 3600,
  "paths": {
    "assets/*": {"max_age": 86400, "immutable": true},
    "index.html": {"max_age": 0, "must_revalidate": true}
  }
}

​

7.7 Cross-Owner Volume Access (Mount Allowlist)

MountGrant {
  grantee:      bytes32,                                   // account or actor address
  access_mode:  ENUM { READ_ONLY, WRITE_ONLY, READ_WRITE },
  path_prefix:  string,                                    // "" = whole volume
  valid_until:  u64,                                       // block height; 0 = no expiry
}

​

8. Volume Attachment

​

8.1 Attachment at Job Dispatch

VolumeAttachment {
  volume_name:   string,
  access_mode:   ENUM { READ_ONLY, WRITE_ONLY, READ_WRITE },
  path_prefix:   string?,          // optional: restrict to sub-path
  max_bytes:     u64,              // byte quota for this job
}

​

8.2 Attachment Process

1. CapToken issuance: The Dispatcher issues a scoped CapToken for each volume attachment.
2. Volume key delivery (private volumes only): the Runner obtains the DEK via a CBSS SealRequest it completes itself (§9.2); the Dispatcher constructs and authorizes the seal request but never unwraps or handles a plaintext DEK. A private volume must carry a committee wrap to be runner-mountable — an owner-key-only volume is read by its owner via the CLI (§9.2), not delivered to a Runner. Public volumes skip this step.
3. Manifest fetch: The Runner fetches the manifest DAG from the on-chain root locator (§3.1) — by node-addressed GET_SHARD or via GET_MANIFEST — recomputes the root, and verifies it matches the onchain manifest_root (§7.5). Only after verification does the Runner trust the manifest for reading or writing.

attachment_cost = BASE_ATTACHMENT_FEE

​

8.3 Detachment

1. The Runner commits the final storage manifest onchain (if it wrote any objects).
2. The CapToken is invalidated (past valid_until).
3. Shards written during the job persist on Relay Nodes, independent of the Runner’s lifecycle.

​

8.2.1 Dispatch Eligibility Diagnostics

{ total_candidates, excluded_by_job_type, excluded_by_health,
  excluded_by_reputation, excluded_by_storage_support, selected }

​

9. Encryption and Privacy

​

9.1 Encryption at Rest

1. Volume DEK (Data Encryption Key): A random 256-bit key generated at volume creation. The DEK is never derived from the account’s signing key — signing keys sign; they do not derive encryption keys.
   volume_dek = CSPRNG(32)  // generated once at create_volume()
   
   The DEK is then wrapped one or more ways according to the volume’s access class (wrapping_key_policy, stored on-chain in the StorageCommitment, §11.1). The wraps are independent and additive — a volume may carry either or both:
   • Owner wrap (owner-key): the DEK is wrapped to a key derived from the owner’s wallet via HKDF, so the owner can read and write the volume directly from the CLI on any machine. Canonical domains: OWNER_KEY_WRAP_SALT = "cbfs/owner-key-volume-dek-wrap/salt/v1" and OWNER_KEY_WRAP_INFO = "cbfs/owner-key-volume-dek-wrap/info/v1".
   • Committee wrap (committee-only): the DEK is IBE-encrypted to the CBSS committee under domain CIP9_VOLUME_DEK_IBE_DOMAIN = "cbss/ibe/cip9-volume-dek/v1", so an authorized Runner can obtain it at mount via a threshold seal (§9.2) without the owner ever placing wallet key material on the runner.
   A canonical hash domain CIP9_HASH_DOMAIN_VOLUME_DEK = "cowboy.cip-9.volume-dek.v1" (with AAD domain …volume-dek-aad.v1) binds each wrap to its volume_id. The wraps are stored in the on-chain StorageCommitment (§11.1). Access classes (wrapping_key_policy):

   Policy	Wraps present	Who can write	Who can read
   owner-key	owner (and, typically, committee)	owner via CLI; runners with an RW CapToken	owner directly; committee-served runners
   committee-only	committee	runners with an RW CapToken	committee-served runners (the owner cannot read without a runner)
   tee-gated (future)	committee, release gated on TEE attestation	attested runners with an RW CapToken	attested committee-served runners

   This resolves the long-standing “can’t write a private volume from the CLI” confusion: CLI writes are valid for any volume that carries an owner wrap, and --owner-key at create means also store the owner wrap — additive, not exclusive. The threat model is per-class and deliberate: committee-only keeps wallet key material off every client machine at the cost of the owner needing a runner to read; owner-key trades that for direct CLI access.
2. Object Encryption: Each object is encrypted with AES-256-GCM using a random nonce per write:
   nonce = CSPRNG(12)       // fresh random nonce for EVERY write, including overwrites
   ciphertext = AES-256-GCM(key=volume_dek, nonce=nonce, plaintext=object_bytes, aad=object_path)
   
   The nonce is stored alongside the ciphertext in the ObjectDescriptor. This is critical: deterministic nonces derived from the object path would cause nonce reuse on overwrites, which is catastrophic for AES-GCM (leaks XOR of plaintexts, breaks authentication). A fresh random nonce on every write eliminates this class of attack entirely.
3. Shard opacity: Erasure coding is applied to the ciphertext, not the plaintext. Relay Nodes hold shards of ciphertext — even if they reconstructed all shards, they would only have ciphertext.
4. Manifest encryption: The volume manifest (list of object paths, sizes, content hashes, ObjectDescriptors) is encrypted with the volume DEK before transmission to Relay Nodes. PlacementRecords are stored separately on Relay Nodes (§5.3.1) and are not part of the encrypted manifest.

​

9.2 Runner Key Access

1. The Runner issues a SealRequest for the volume’s committee-wrapped DEK, presenting its CapToken (and, for tee-gated, its attestation).
2. Each CBSS committee member verifies authorization and returns a partial decryption.
3. The Runner combines the threshold partials and completes the IBE decryption (domain cbss/ibe/cip9-volume-dek/v1) to recover the DEK, sealed in transit to its encryption_pubkey.
4. The DEK is held in memory for the job and zeroized on completion.

• The DEK never appears in plaintext on-chain or in any persistent store.
• READ_ONLY CapTokens on private volumes still require DEK delivery — the Runner must decrypt ciphertext shards to serve reads.
• For TEE-attested runners (tee-gated), the committee releases partials only against a valid attestation, and the DEK is sealed to the enclave, never exposed to the host OS.

​

9.3 Privacy Guarantees

• Relay Nodes see only ciphertext shards indexed by opaque shard IDs (see §16.3). They cannot read object contents or inspect the manifest. Object paths are encrypted within the manifest and never exposed to Relay Nodes.
• Other Runners (not assigned to the job) cannot access the volume DEK.
• onchain observers see only the Storage Commitment (volume ID, wrapped DEK, encrypted manifest hash, total size, shard assignments). They cannot determine what is stored.
• The account owner has full access to all their volume data by unwrapping the DEK with their wrapping key.

• No confidentiality: Data is stored unencrypted. Relay Nodes, Runners, and any network participant can read object contents. This is by design — public volumes are intended for publicly readable data (web assets, public datasets, shared artifacts).
• Integrity preserved: Content hashes (BLAKE3) ensure that data has not been tampered with, even though it is unencrypted.
• Write access is still restricted: Only the account owner (or authorized Runners via CapToken) can write to a public volume. Public readability does not imply public writability.

​

10. Billing and Fees

​

10.1 Fee Components

​

10.2 Effective Size and Erasure Overhead

effective_size = raw_size * (K + M) / K

​

10.3 Persistent Storage Billing

• Each epoch, the protocol calculates the total effective storage used by each account across all volumes.
• The per-epoch storage fee is deducted from the account’s balance.
• If the account’s balance falls below MIN_STORAGE_BALANCE (sufficient to cover one epoch of fees), the protocol enters a grace period of STORAGE_GRACE_EPOCHS.
• After the grace period, if the balance is still insufficient, all volumes owned by the account are marked for garbage collection.

​

10.3.1 Escrow and Rent Lifecycle

• Create: with no escrow and insufficient balance to cover MIN_STORAGE_BALANCE, creation MUST fail with a distinct, named error (§12.3.1) — not a generic E1900 "invalid data", which today conflates the duplicate-name and escrow-shortfall cases. A no-escrow volume that is created bills from balance immediately and expires after one epoch if balance is insufficient.
• Active → grace: when escrow is exhausted and balance falls below MIN_STORAGE_BALANCE, the volume enters GRACE_PERIOD for STORAGE_GRACE_EPOCHS. A top-up of escrow or balance returns it to ACTIVE.
• Expiry: after the grace window with still-insufficient funds, the volume is marked GARBAGE_COLLECTING (§13).

​

10.4 Fee Distribution

Account Owner ──(per-epoch storage fee)──► Protocol ──► { burn (0x00) :: PoR challenge pool (0x0B) :: Relays pro-rata }
Runner        ──(per-byte transfer fee) ──► Relay Node (serving the shard)

• STORAGE_FEE_BURN_BPS (default 1000 = 10%) → burned to 0x00, consistent with CIP-3’s deflationary design.
• STORAGE_FEE_CHALLENGE_POOL_BPS (default 200 = 2%) → accrued to the PoR challenge pool at 0x0B (this is the explicit form of the POR_CHALLENGE_FEE_SHARE row in §14; it funds the challenger bounty defined in CIP-31 §7).
• STORAGE_FEE_RELAY_BPS (default 8800 = 88%) → distributed pro-rata across active Relay Nodes with weight (shard_count × shard_age_in_epochs) per CIP-31 §5.

​

10.5 Relationship to CIP-3

• onchain operations (creating Storage Commitments, writing manifests) consume Cycles and Cells as normal CIP-3 transactions.
• Off-chain storage fees are a separate ledger entry, debited from the account balance per epoch by the Storage Manager system actor.

​

11. onchain State

​

11.1 Storage Manager System Actor

StorageCommitment {
  volume_id:             bytes32,    // keccak256(account_address || volume_name)
  owner:                 address,
  volume_name:           string,
  visibility:            ENUM { PRIVATE, PUBLIC },  // PRIVATE = encrypted, CapToken-gated; PUBLIC = unencrypted, open reads
  wrapping_key_policy:   ENUM { OWNER_KEY, COMMITTEE_ONLY, TEE_GATED },  // access class (§9.1); ignored for PUBLIC
  created_at:            u64,        // block height
  owner_wrapped_dek:     bytes,      // DEK wrapped to the owner wallet key (§9.1); empty if no owner wrap / PUBLIC
  committee_wrapped_dek: bytes,      // DEK IBE-wrapped to the CBSS committee (§9.1); empty if no committee wrap / PUBLIC
  manifest_root:         bytes32,    // root locator of the manifest DAG (§3.1); over ciphertext nodes for PRIVATE, plaintext for PUBLIC
  shard_root:            bytes32,    // public commitment over the volume's per-shard chunk-roots (§5.6); DEK-independent, anchors PoR for PRIVATE volumes
  raw_size_bytes:        u64,        // sum of object sizes (pre-erasure)
  effective_size_bytes:  u64,        // raw_size * (K+M)/K
  erasure_k:             u8,         // data shards
  erasure_m:             u8,         // parity shards
  last_updated:          u64,        // block height of last manifest update
  degraded_shards:       u16,        // count of shards needing repair
  mount_allowed_actors:  [MountGrant],  // cross-owner mount allowlist (§7.7); matched by bare address
  status:                ENUM { ACTIVE, GRACE_PERIOD, DELETED, GARBAGE_COLLECTING }
}

AccountStorageSummary {
  total_volumes:          u32,
  total_effective_bytes:  u64,
  last_billed_epoch:      u64,
  balance_reserved:       u256       // CBY reserved for storage fees
}

​

11.2 Relay Registry

• RelayNodeProfile entries (see §5.2)
• Active relay list (ordered, health-decaying, analogous to CIP-2 Runner Registry)
• Shard assignment index: volume_id → list[PlacementAssignment]
• The network-wide AUTO_DRAIN_POLICY_KEY slot written by governance (§5.8)

​

11.3 Key Space

key = 0x1 || keccak256(storage_manager_address) || 0x00 || keccak256("commitment" || volume_id)
value = rlp(StorageCommitment)

key = "ras:owner-name:" || owner_address || keccak256(volume_name)   // OWNER_NAME_PREFIX = b"ras:owner-name:"
value = volume_id (bytes32)

key = 0x1 || keccak256(storage_manager_address) || 0x00 || keccak256("summary" || account_address)
value = rlp(AccountStorageSummary)

key = 0x1 || keccak256(relay_registry_address) || 0x00 || keccak256("relay" || relay_address)
value = rlp(RelayNodeProfile)

​

12. Client Interfaces

• Filesystem interface (§12.1): A FUSE-mounted directory that presents the volume as a standard filesystem. This is the primary interface for agentic workloads where an LLM (Claude, Kimi-K2, GPT, etc.) operates via tool calling. The model’s existing filesystem tools — Read, Write, Bash (ls, grep, find), Glob, Grep — work unchanged against the mounted volume. No custom tool definitions required.
• Object API (§12.2): A programmatic interface for orchestration code and non-agentic workloads. This is what the FUSE layer calls internally, and is also available directly for lightweight write-only jobs that don’t need filesystem semantics.

​

12.1 Filesystem Interface (FUSE Mount)

​

12.1.1 Design Rationale

• Injecting custom tool definitions into every model’s tool set
• Models are less fluent with unfamiliar, domain-specific tools
• Loss of composability with standard unix tools (grep, find, jq, wc, etc.)
• Every model provider’s runner integration needs custom work

Model tool call                        What happens under the hood
─────────────────                      ──────────────────────────────
Read("/mnt/memory/state.json")    →    FUSE read → fetch shards → reconstruct → decrypt
Write("/mnt/memory/state.json")   →    FUSE write → local buffer → background push
Bash("ls /mnt/memory/logs/")      →    FUSE readdir → list objects from manifest
Bash("grep -r 'error' /mnt/mem")  →    FUSE read (multiple) → local cache → grep
Bash("wc -l /mnt/memory/*.json")  →    FUSE read (multiple) → local cache → wc

​

12.1.2 Mount Point Layout

/mnt/volumes/{volume_name}/

# Full volume mount:
/mnt/volumes/agent-memory/
├── state/
│   ├── memory.json
│   └── portfolio.json
├── logs/
│   └── 2026-03-04/
│       └── analysis.json
└── config.json

# Prefix-scoped mount (path_prefix="state/"):
/mnt/volumes/agent-memory/
├── memory.json
└── portfolio.json

​

12.1.3 Sync Strategy (Hybrid)

• Pull cycle (Relay Nodes → local): Re-fetches the volume’s committed manifest from Relay Nodes, verifies it against the onchain manifest_root (§7.5), and materializes any new objects not already in the local tmpfs. Because reads are READ_COMMITTED (§7.5), the pull cycle only discovers objects that have been included in a committed manifest — objects from a sub-agent become visible only after that sub-agent calls commit_manifest(). In the agent swarm pattern, this means the coordinator sees a sub-agent’s files appear as a batch when the sub-agent commits, not individually as they are written.
• Push cycle (local → Relay Nodes): Detects locally written or modified files (via inotify/fswatch), encrypts them, erasure-codes, and distributes shards to Relay Nodes.

​

12.1.4 FUSE Operation Mapping

​

12.2 Object API (Programmatic)

• The Runner is a script (not an LLM doing tool calling) that just needs to write output files.
• The workload is lightweight and a full FUSE mount is unnecessary overhead.
• The orchestration layer needs to interact with storage outside of a container context.

# Object operations (subject to CapToken permissions)
put_object(cap_token: bytes, object_path: str, data: bytes) -> ObjectReceipt
put_many(cap_token: bytes, items: list[(str, bytes)]) -> list[ObjectReceipt]  # batched; refreshes token mid-stream (§12.2.1)
get_object(cap_token: bytes, object_path: str) -> bytes          # READ_ONLY or READ_WRITE
list_objects(cap_token: bytes, prefix: str = "") -> list[str]    # READ_ONLY or READ_WRITE

# CBFS-level commit (data plane)
commit(cap_token: bytes) -> CommitReceipt

# Cowboy-level commit (control plane, onchain)
commit_manifest(cap_token: bytes, prev_root: bytes32, new_root: bytes32,
                added_shards: [ShardRef], removed_shards: [ShardRef]) -> bool

1. CBFS commit (commit): Publishes the manifest DAG’s new nodes — only the delta’s added_nodes, since unchanged nodes are structurally shared with the prior root (§3.1) — to Relay Nodes, each addressed by manifest_node_shard_id(locator), and publishes PlacementRecords to all assigned Relay Nodes via PutPlacement (§5.3.1). This is a data-plane operation — it makes the data durable and discoverable by other CBFS clients, but does not touch the chain. PlacementRecord publication is best-effort; failures are logged but do not block the commit.
2. Cowboy commit (commit_manifest): Submits the new manifest DAG root, the prev_root it extends, and the added_shards / removed_shards deltas to the onchain StorageCommitment via the write-relayer (§12.2.1). The chain enforces a parent-root compare-and-swap — it rejects the commit unless prev_root equals the volume’s current manifest_root — so concurrent or stale commits cannot silently lose updates. From the shard deltas it updates the volume’s public shard_root (§5.6) incrementally in O(changed), with no full pass. This anchors the manifest for READ_COMMITTED consistency (§7.5) and PoR (§5.6). Individual ObjectDescriptors stay off-chain in the manifest DAG; for a public volume the plaintext root yields O(log N) object-inclusion proofs, while for a private volume object contents stay opaque and shard retrievability is proven against shard_root instead.

event ManifestCommitted {
    volume_id:      bytes32,
    manifest_root:  bytes32,
    block_height:   u64,
    raw_size_delta: i64,     // signed delta from the previous manifest
    visibility:     u8       // 0 = PRIVATE, 1 = PUBLIC
}

​

12.2.1 Manifest Commit Transport (Write-Relayer)

• Routable bind + self-registration. A write-relayer MUST bind a routable address (not loopback) and, given a public endpoint, self-register on the on-chain /ras/write-relayers registry, so clients discover relayers from chain rather than hardcoding an address. A loopback-only, unregistered relayer breaks the “RPC-URL only” goal — it forces an out-of-band tunnel.
• Canonical owner-authorization signing domain. The owner authorization MUST canonicalize addresses as EIP-55 in both the payload hash and the signing bytes; a mismatch between the two is a class of signature-rejection bug and is non-conforming. The owner-action payload is signed over an explicit, language-neutral big-endian byte layout (not a Rust struct encoding), so a non-Rust or alternate-chain backend can produce an identical signing domain. (This is the target encoding — registry-proto/src/canonical.rs — and is not yet wired into the signing path, which still bincode-serializes; see §17.3. Do not sign owner actions from a non-Rust client until the canonical path ships.)
• Per-relayer gas key. Each relayer instance MUST use its own gas key; a key shared across instances races on nonce selection.

​

12.3 Account Owner API (via Storage Manager Actor)

# Volume lifecycle
create_volume(
    volume_name: str,
    max_size_bytes: int = 0,
    erasure_k: int = 4,                       # data shards (default 4)
    erasure_m: int = 2,                       # parity shards (default 2)
    visibility: str = "PRIVATE",              # "PRIVATE" (encrypted, default) or "PUBLIC" (unencrypted, open reads)
    wrapping_key_policy: str = "owner-key",   # access class (§9.1): "owner-key" | "committee-only" | "tee-gated"
    initial_escrow: int = 0,                  # CBY prepaid toward storage rent (§10.3.1)
) -> bytes32  # returns volume_id; writes StorageCommitment + name index atomically (§11.3)

delete_volume(volume_name: str) -> bool           # soft-delete: sets status=DELETED, starts GC timer
list_volumes() -> list[VolumeInfo]
transfer_volume(volume_name: str, new_owner: address) -> bool  # transfer ownership to another account

# Lookup & cross-owner access
get_volume_id_by_name(owner: address, volume_name: str) -> bytes32   # (owner, name) → volume_id (§11.3)
update_mount_allowlist(volume_name: str, grant: MountGrant, remove: bool = False) -> bool  # owner-signed (§7.7)

# Object management
delete_object(volume_name: str, object_path: str) -> bool
list_objects(volume_name: str, prefix: str = "") -> list[ObjectInfo]
get_volume_info(volume_name: str) -> VolumeInfo

# Billing
get_storage_usage() -> AccountStorageSummary
reserve_storage_balance(amount: uint256) -> bool

​

12.4 CIP-2 Task Definition Extension

struct OffchainTask:
    ... (existing CIP-2 fields) ...
    volume_attachments: list[VolumeAttachment]  # NEW: optional

struct VolumeAttachment:
    volume_name:      string
    access_mode:      uint8          # 0 = READ_ONLY, 1 = WRITE_ONLY, 2 = READ_WRITE
    path_prefix:      string         # "" for full volume access (canonicalized with trailing /)
    max_bytes:        uint64         # byte quota for this job
    mount:            bool           # true = FUSE mount at /mnt/volumes/{name}, false = Object API only
    sync_interval:    uint32         # seconds between sync cycles (default 5, mount only)
    # no read_consistency field — all reads are READ_COMMITTED (see §7.5)

​

12.3.1 Volume Creation and Lookup Errors

• E_VOLUME_NAME_DUPLICATE — (owner, volume_name) already present in the name index (§11.3).
• E_VOLUME_ESCROW_INSUFFICIENT — initial_escrow (or account balance) below MIN_STORAGE_BALANCE (§10.3.1).
• E_VOLUME_PARAM_INVALID — erasure k/m out of bounds, unknown visibility or wrapping_key_policy, or a volume_name violating the [a-zA-Z0-9_\-.] / 64-byte rule (§6.1).
• E_VOLUME_NOT_FOUND_FOR_OWNER — a name lookup missed the index for this owner-domain (§11.3); this MUST NOT be reported as a bare 404 that hides whether the caller queried under the wrong owner.

​

13. Garbage Collection

​

13.1 Triggers

1. Explicit deletion: Account owner calls delete_volume() (soft-delete + grace window, §13.2) or delete_object().
2. Escrow/balance exhaustion: after escrow is depleted and the account balance stays below MIN_STORAGE_BALANCE through STORAGE_GRACE_EPOCHS (§10.3). There is no separate expiry_height — volume lifetime is governed by escrow/rent, not a fixed height.

​

13.2 Process

​

13.3 Deletion Semantics

• On delete_object: All shards of the object are marked for removal on their respective Relay Nodes. The onchain manifest is updated. Relay Nodes garbage-collect shard data asynchronously, but the object is immediately inaccessible to CapToken holders.
• On delete_volume: The volume enters the DELETED state (soft-delete). All active CapTokens are revoked. New CapTokens cannot be issued. Storage fees continue accruing during a grace window of VOLUME_DELETE_GRACE_EPOCHS (default: same as STORAGE_GRACE_EPOCHS). During this window, the account owner may call undelete_volume() to restore the volume to ACTIVE status. After the grace window, the volume transitions to GARBAGE_COLLECTING and all shards are purged. The onchain Storage Commitment is removed and storage fees cease.
• Garbage collection is irreversible. Once a volume enters GARBAGE_COLLECTING, data cannot be recovered.

​

13.4 Ownership Transfer

• All active CapTokens are revoked (they were issued under the old owner’s authority).
• The StorageCommitment.owner field is updated atomically.
• The new owner assumes billing responsibility starting from the next epoch.
• For private volumes, the owner wrap is re-encrypted to the new owner’s wallet-derived key as part of the transfer transaction (a committee wrap, if present, is unaffected — the committee can still serve the new owner’s runners). This requires the old owner to unwrap and re-wrap the DEK, so transfer is an interactive operation requiring the old owner’s cooperation.
• For public volumes, no key re-wrapping is needed (no DEK exists).
• Transfer of a volume in DELETED or GARBAGE_COLLECTING status is rejected.

​

13.5 Gateway HTTP Serving by Status

​

14. Parameters

​

15. Security Considerations

​

15.1 CapToken Forgery

​

15.2 Runner Compromise

• Byte quotas limit the damage per job.
• Prefix-confined commits: A WRITE_ONLY commit may change only manifest nodes under the committer’s prefix — enforced onchain for public volumes (parent-root CAS + changed-node prefix check) and via staged-commit + DEK-holder finalize for private volumes (§7.3.1). Out-of-prefix commits are rejected (public) or never finalized (private), not merely detected after the fact.
• Orphan shard GC: Shards not referenced by any committed manifest are garbage collected after ORPHAN_SHARD_TTL.
• No delete access: A compromised Runner cannot destroy existing data.

• TEE attestation: For sensitive workloads, require TEE-attested Runners (CIP-2 tee_required=true).
• Minimal scope: Use path_prefix to restrict access to only the necessary sub-path.
• Account owner discretion: READ_WRITE is an explicit opt-in; the account owner accepts the elevated trust.

​

15.3 Relay Node Compromise

​

15.4 Denial of Service

​

15.5 Key Management

• A Volume DEK is wrapped per the volume’s access class (§9.1): to the owner’s wallet-derived key, to the CBSS committee, or both. Losing the owner’s wallet key forfeits the owner-wrap path, but a committee-wrapped volume remains readable through an authorized runner. The wraps live on-chain in the StorageCommitment, so the DEK itself is never at risk of loss — only the ability to unwrap a given wrap.
• Dispatcher trust boundary. The Dispatcher is never on the key path: for private volumes the Runner completes the CBSS SealRequest itself (§9.2) and the Dispatcher only constructs and authorizes it, so the Dispatcher never sees a plaintext DEK. A compromised Dispatcher could mis-authorize a seal request — bounded by the same consensus-level trust already required to issue CapTokens — but cannot exfiltrate volume DEKs, since it never holds them.
• Key rotation — re-wrapping the DEK under a new key without re-encrypting data — is future work; today a compromised owner key requires creating new volumes and re-encrypting. During ownership transfer (§13.4) the DEK is re-wrapped to the new owner: the one case where a wrap changes without re-encrypting all data.

​

16. Implementation Notes

​

16.1 Canonical Implementation Stack

​

16.2 FUSE Mount Implementation

1. FUSE daemon (fuser crate): Implements the Filesystem trait, handling read, write, readdir, getattr, etc. Delegates to the local cache layer.
2. Local cache (tmpfs-backed): An in-memory filesystem that serves as the working copy. All reads/writes hit the cache first. The cache is populated lazily on first access (fetch from Relay Nodes) and eagerly for files modified locally.
3. Sync daemon (background task): Runs a push/pull loop at the configured sync_interval. Uses notify for detecting local changes and polls Relay Nodes for remote changes. Handles encryption, erasure coding, and shard distribution.

┌─────────────────────────────────────────────────────────┐
│ Container / Runner process                              │
│                                                         │
│  Model (Claude, Kimi-K2, etc.)                          │
│    │                                                    │
│    ├─ Read("/mnt/volumes/mem/state.json")               │
│    ├─ Write("/mnt/volumes/mem/state.json", data)        │
│    └─ Bash("ls /mnt/volumes/mem/")                      │
│         │                                               │
│         ▼                                               │
│  ┌─────────────┐     ┌──────────────┐                   │
│  │ FUSE daemon │◄───►│ Local cache   │                  │
│  │ (fuser)     │     │ (tmpfs)       │                  │
│  └─────────────┘     └──────┬───────┘                   │
│                             │                           │
│                      ┌──────▼───────┐                   │
│                      │ Sync daemon  │                   │
│                      │  push/pull   │                   │
│                      └──────┬───────┘                   │
│                             │                           │
└─────────────────────────────┼───────────────────────────┘
                              │ encrypt / erasure code / QUIC
                              ▼
                     ┌─────────────────┐
                     │   Relay Nodes   │
                     └─────────────────┘

​

16.3 Object API Client Library

// Programmatic usage (non-agentic runners):
let data = volume.get("state/memory.cbor").await?;
volume.put("state/memory.cbor", &updated_data).await?;

// Under the hood:
// get: fetch PlacementRecord → identify assigned nodes → fetch K shards
//      → Reed-Solomon reconstruct (truncate to ciphertext_size)
//      → verify ciphertext hash → AES-GCM decrypt → verify content hash
// put: AES-GCM encrypt → Reed-Solomon encode → distribute K+M shards
//      → produce ObjectDescriptor (manifest) + PlacementRecord (published at commit)

​

16.4 Relay Node Implementation

1. Blob store (cbfs-store): A sled-backed key-value store mapping (shard_id, shard_index) → shard_bytes. The shard_id is an opaque BLAKE3 hash (see §5.3) — Relay Nodes never see object paths, only opaque identifiers. This ensures the privacy guarantee in §9.3.
2. Placement store (cbfs-placement): A sled-backed store mapping shard_id → PlacementRecord. Stores the mutable shard-to-node assignments separately from shard data, enabling repair workers to read placement information without accessing the encrypted manifest.
3. RPC server: Accepts shard operations (PUT_SHARD, GET_SHARD, PROVE_SHARD) and placement operations (PutPlacement, GetPlacement, ReplicatePlacement). All operations are authenticated by the AuthProvider (§7.1.1). For public volumes (visibility = PUBLIC), GET_SHARD and GetPlacement are served without CapToken verification (§7.6.3); write operations still require a CapToken. Relay Nodes do NOT expose any listing operation — object listing is performed client-side by reading the manifest. Requests are keyed by shard_id, not object path.
4. Repair loop (repair.rs): Periodically runs the two-phase autonomous repair cycle (§5.5). Uses a mutable peer list (Arc<RwLock<Vec<NodeInfo>>>) bootstrapped from seed peers in the node config and updated as new nodes are discovered.
5. Placement sync (placement_sync.rs): Replicates PlacementRecords to peer nodes to ensure all assigned nodes have a consistent view of shard assignments.
6. Heartbeat loop: Periodically calls heartbeat() on the onchain Relay Registry.
7. Spot-check responder: When challenged, returns a random chunk of a specified shard for integrity verification.

​

16.5 Performance Expectations

​

16.6 Runner ↔ Validator Wire Compatibility

• A runner’s cowboy-ras protocol types MUST be pinned compatibly with the deployed validator. A runner outside the validator’s supported wire range is rejected with a clear, typed reason (“unsupported transaction wire version — upgrade runner”) surfaced back to the submitter, not a silent drop or a generic decode error.
• Wire-format changes to runner↔chain transactions MUST be versioned (a discriminant the validator checks), and the validator SHOULD accept a defined compatibility window rather than only the exact current version, so a rolling upgrade does not strand every runner at once.
• Decode-reject reasons MUST be observable to the runner operator — a runner that cannot complete jobs because of wire skew has to be able to find out why.

​

16.7 Client Bootstrap and Discovery

• chain_id / network from chain-info, plus consensus parameters and the basefee schedule (GET /basefee, suggested_max_fee_*) — not hand-set flags. A wrong or missing chain_id/fee fails certificate auth or is silently dropped by the mempool (fee_below_basefee), so guessing is worse than discovering.
• Storage relays from the relay registry (§5.2), write-relayers from /ras/write-relayers (§12.2.1), and CBSS DEK delivery from chain (§9.2) — the seal committee from the CBSS committee registry (CIP-24) by epoch, and sealed ciphertext as a CBFS object from the relays already listed in the registry. None of it is client-configured.
• The RPC URL MUST be resolved once at startup and threaded everywhere. A subcommand MUST NOT silently fall back to a hardcoded default (e.g., 127.0.0.1:4000) or ignore an explicit --indexer-url; a missing or unreachable RPC is a clear error, not a silent localhost attempt.

​

17. Scope and Future Work

​

17.1 In Scope (This CIP)

• Private and public account-scoped volumes.
• READ_ONLY, WRITE_ONLY, and READ_WRITE CapToken access modes; PUBLIC volume visibility mode (§7.6).
• Concurrent CapTokens on the same volume (agent swarm pattern).
• Two client interfaces: FUSE filesystem mount (for agentic/LLM workloads) and Object API (for programmatic workloads).
• Hybrid sync strategy for FUSE mounts (local cache + background push/pull).
• Relay Nodes as a dedicated storage layer with staking and incentives.
• Reed-Solomon erasure coding (default 4/6) for durability.
• AES-256-GCM encryption with HKDF-derived keys.
• BLAKE3 content hashing at object, ciphertext, and shard levels.
• Path-based addressing with Merkle manifest for integrity proofs.
• Per-epoch billing with grace period and garbage collection.
• Lazy and proactive shard repair.
• Private dual-wrapped volumes with first-class access classes (owner-key, committee-only; tee-gated reserved), and owner-curated cross-owner mount allowlists (§7.7).
• Control-plane writes via the cowboy-ras-write-relayer, with client discovery of chain parameters, relays, write-relayers, and CBSS endpoints from chain (§16.7).

​

17.2 Explicitly Out of Scope

• Content-addressed retrieval: RAS uses path-based addressing only. CID-based retrieval (IPFS-style) may be layered on top in the future.
• Storage marketplace: A competitive market where Relay Nodes bid on storage deals (Filecoin-style) is a future extension; pricing is protocol-set.
• Alternative storage backends: This CIP standardizes on CBFS as the canonical storage layer. Supporting Filecoin, Arweave, or other backends under the same CIP-9 surface is future work and would require a follow-on standard.
• READ_UNCOMMITTED consistency mode: Pre-commit read visibility for real-time agent swarms. Requires a shard discovery mechanism (pubsub or uncommitted manifest fragments) not defined in this CIP. See §7.5.
• Key rotation: Rotating volume encryption keys without re-encrypting all data.
• Runner-initiated task dispatch: Allowing a coordinator Runner to dynamically spawn sub-agent tasks. Currently, all tasks must be dispatched by onchain Actors. This is a CIP-2 extension.
• Full container runtime spec: Container image management, container registries, resource limits, GPU passthrough, and network policies. CIP-9 defines the CBFS-backed storage primitive (including the FUSE mount and object API); a separate CIP will define the full container runtime that consumes it.

​

Appendix A: Worked Examples

​

A.1 AI Agent with Persistent Memory (Filesystem Mount)

submit_task(
    task_definition=encode_task({
        "model": "claude-sonnet",
        "system": "You are a trading analyst. Your memory and portfolio are in /mnt/volumes/agent-memory/state/.",
        "prompt": "Analyze today's market conditions. Update your memory and portfolio.",
    }),
    volume_attachments=[
        VolumeAttachment(
            volume_name="agent-memory",
            access_mode=READ_WRITE,
            mount=True,                    # FUSE mount at /mnt/volumes/agent-memory/
            max_bytes=500 * 1024 * 1024,
        ),
    ],
    num_runners=1,
    timeout_blocks=500,
    proof_type_requested=TEE,
    ...
)

── Claude's tool calls during execution ──────────────────────────

1. Read("/mnt/volumes/agent-memory/state/memory.json")
   → Returns the agent's memory from yesterday's run
   → (Under the hood: FUSE → fetch shards from Relay Nodes → reconstruct → decrypt)

2. Read("/mnt/volumes/agent-memory/state/portfolio.json")
   → Returns the current portfolio state

3. Bash("ls /mnt/volumes/agent-memory/logs/")
   → 2026-03-01/  2026-03-02/  2026-03-03/

4. Bash("grep -r 'NVDA' /mnt/volumes/agent-memory/logs/2026-03-03/")
   → Shows yesterday's NVDA-related log entries

5. [Claude performs analysis, makes decisions]

6. Write("/mnt/volumes/agent-memory/state/memory.json", updated_memory)
   → Writes to local tmpfs instantly
   → (Under the hood: sync daemon pushes to Relay Nodes within 5 seconds)

7. Write("/mnt/volumes/agent-memory/state/portfolio.json", updated_portfolio)

8. Write("/mnt/volumes/agent-memory/logs/2026-03-04/analysis.json", todays_analysis)

── Container shuts down ──────────────────────────────────────────
   → Final sync pushes any remaining dirty files
   → Manifest committed onchain
   → tmpfs deleted. Data persists on Relay Nodes.

​

A.2 Distributed Scraping with Map-Reduce (Direct Object API)

for i, site in enumerate(sites):
    submit_task(
        task_definition=encode_task({"action": "scrape", "url": site}),
        volume_attachments=[
            VolumeAttachment(
                volume_name="scrape-results",
                access_mode=WRITE_ONLY,
                path_prefix=f"scraper-{i}/",
                max_bytes=500_000_000,
                mount=False,                   # no FUSE mount, direct object writes
            ),
        ],
        ...
    )

# Simple Python script, not an LLM. Uses the Object API directly.
for page in crawl(site_url):
    put_object(cap_token, f"scraper-{my_id}/{page.slug}.json", json.dumps({
        "url": page.url,
        "title": page.title,
        "content": page.text,
        "links": page.links,
    }))
commit_manifest(cap_token, manifest_root)

submit_task(
    task_definition=encode_task({
        "model": "kimi-k2",
        "system": "You have scraped web data in /mnt/volumes/scrape-results/. Analyze and collate.",
        "prompt": "Find all mentions of product launches across the scraped sites. Write a summary.",
    }),
    volume_attachments=[
        VolumeAttachment(
            volume_name="scrape-results",
            access_mode=READ_WRITE,
            mount=True,                        # FUSE mount for filesystem access
            max_bytes=100_000_000,
        ),
    ],
    ...
)

1. Bash("find /mnt/volumes/scrape-results -name '*.json' | wc -l")
   → 47

2. Bash("ls /mnt/volumes/scrape-results/")
   → scraper-0/  scraper-1/  scraper-2/  scraper-3/  scraper-4/

3. Bash("cat /mnt/volumes/scrape-results/scraper-0/about-page.json | jq '.title'")
   → "About Us - Acme Corp"

4. Bash("grep -rl 'product launch' /mnt/volumes/scrape-results/")
   → scraper-0/news.json
   → scraper-2/blog-post-3.json
   → scraper-4/press-release.json

5. Read("/mnt/volumes/scrape-results/scraper-0/news.json")
   → [full content]

6. [... reads relevant files, analyzes ...]

7. Write("/mnt/volumes/scrape-results/collated/summary.md", summary)
8. Write("/mnt/volumes/scrape-results/collated/product-launches.json", structured_data)

​

A.3 Agent Swarm with Batch Coordination (Filesystem Mount + Concurrent Writers)

# Coordinator: READ_WRITE mount, sees the full volume
submit_task(
    task_definition=encode_task({
        "model": "claude-sonnet",
        "system": "You are coordinating 5 research agents. Their reports will appear in /mnt/volumes/swarm/agent-*/. Poll for new reports and synthesize findings. Reports appear in batches as each agent completes and commits.",
    }),
    volume_attachments=[
        VolumeAttachment(volume_name="swarm", access_mode=READ_WRITE, mount=True,
                         sync_interval=5, max_bytes=100_000_000),
    ],
    timeout_blocks=2000,
    ...
)

# Sub-agents: WRITE_ONLY mount, scoped to their prefix
for i in range(5):
    submit_task(
        task_definition=encode_task({
            "model": "claude-haiku",
            "system": f"You are research agent {i}. Write your findings to /mnt/volumes/swarm/.",
            "prompt": f"Research: {topics[i]}",
        }),
        volume_attachments=[
            VolumeAttachment(volume_name="swarm", access_mode=WRITE_ONLY, mount=True,
                             path_prefix=f"agent-{i}/", max_bytes=50_000_000),
        ],
        timeout_blocks=1000,
        ...
    )

1. [Performs research using web tools]

2. Write("/mnt/volumes/swarm/report.md", research_findings)
   → (Visible path: agent-2/report.md due to prefix scoping)

3. Write("/mnt/volumes/swarm/sources.json", source_list)
   → (Visible path: agent-2/sources.json)

── Container shuts down ──
── Sync daemon pushes all shards to Relay Nodes ──
── Runtime calls commit_manifest(), updating onchain manifest_root ──
── Agent-2's files are now committed and visible to other readers ──

1. Bash("ls /mnt/volumes/swarm/")
   → (empty — no sub-agents have committed yet)

   [Waits... sync daemon re-fetches manifest every 5 seconds,
    verifying against onchain manifest_root (§7.5)]

   [Agent-0 finishes and commits its manifest]

2. Bash("ls /mnt/volumes/swarm/")
   → agent-0/
   → (Agent-0 committed — all its files appear at once)

3. Read("/mnt/volumes/swarm/agent-0/report.md")
   → [agent-0's findings]

   [Agent-1 and agent-3 finish and commit around the same time]

4. Bash("ls /mnt/volumes/swarm/")
   → agent-0/  agent-1/  agent-3/
   → (Two more agents committed since last check)

5. Read("/mnt/volumes/swarm/agent-1/report.md")
   → [agent-1's findings]

6. Read("/mnt/volumes/swarm/agent-3/report.md")
   → [agent-3's findings]

   [... continues polling until all 5 agents have committed ...]

7. Write("/mnt/volumes/swarm/synthesis/final-report.md", synthesized_findings)

​

A.4 Multi-Stage Pipeline with Handoff

submit_task(
    task_definition=encode_task({"stage": "preprocess", "source": "https://data.example.com/feed"}),
    volume_attachments=[
        VolumeAttachment(volume_name="pipeline", access_mode=WRITE_ONLY,
                         path_prefix="stage1/", mount=False, max_bytes=2_000_000_000),
    ],
    ...
)

submit_task(
    task_definition=encode_task({
        "model": "kimi-k2",
        "system": "Input data is in /mnt/volumes/pipeline/stage1/. Run inference and write results to /mnt/volumes/pipeline/stage2/.",
    }),
    volume_attachments=[
        VolumeAttachment(volume_name="pipeline", access_mode=READ_WRITE,
                         mount=True, max_bytes=1_000_000_000),
    ],
    ...
)

​

Appendix B: Comparison with Existing Systems

​

Appendix C: CapToken Wire Format

CapToken (variable length)
┌──────────────────────────────────────────────────┐
│ version:         u8          (1 byte)            │
│ volume_id:       bytes32     (32 bytes)          │
│ access_mode:     u8          (1 byte) [0=RO,1=WO,2=RW] │
│ path_prefix_len: u16         (2 bytes)           │
│ path_prefix:     bytes       (variable)          │
│ max_bytes:       u64         (8 bytes)           │
│ valid_from:      u64         (8 bytes)           │
│ valid_until:     u64         (8 bytes)           │
│ runner_address:  bytes32     (32 bytes)          │
│ nonce:           u64         (8 bytes)           │
│ caveats_hash:    bytes32     (32 bytes)          │
│ signature:       bytes64     (64 bytes)          │
└──────────────────────────────────────────────────┘

​

Appendix D: “Why Won’t My Volume Mount?”