CIP-9: CBFS-Backed Runner Storage

​

1. Abstract

• Account-scoped: All storage is owned by and billed to a Cowboy account (EOA or Actor).
• Private by default, public by choice: Private volumes are encrypted client-side before leaving the Runner; only the owning account can decrypt. Public volumes (visibility = PUBLIC) are stored unencrypted and readable by any party without a CapToken, enabling use cases like web asset hosting (CIP-15) and public datasets.
• Access-controlled: Runners receive scoped capability tokens granting read-only, write-only, or read-write access to a storage volume. Multiple concurrent CapTokens may be active on the same volume simultaneously. Public volumes require CapTokens only for writes, not reads.
• Addressable: Every stored object has a deterministic path within the account’s storage namespace, enabling targeted reads and deletes by the account owner.
• Durable: Objects are erasure-coded (Reed-Solomon) and distributed across multiple Relay Nodes, tolerating node failures without data loss.
• Metered: Storage is billed in CBY via a per-epoch, per-byte fee model (including erasure coding overhead) that extends the Cells concept from CIP-3 to persistent off-chain blobs.

​

2. Motivation

1. Secret management. Runners need access to API keys, certificates, and credentials without exposing them on the public chain. These secrets must only be decryptable by authorized Runners, optionally gated by TEE attestation.
2. Large off-chain output. Computation frequently produces artifacts exceeding the 64 KiB onchain inline cap (whitepaper §7). LLM inference may generate images, audio, datasets, or model weights that are too large for result_data and inefficient to pass through onchain callbacks. These must be stored off-chain with a verifiable content commitment anchored onchain.
3. Multi-step data flow. Iterative computation (AI training loops accumulating weights across invocations), stateful agents (conversation history, tool outputs, learned preferences persisting across scheduling cycles), pipeline handoff (Runner A produces intermediate data consumed by Runner B), and agent swarms (coordinator + concurrent sub-agents sharing a volume) all require persistent, addressable storage between jobs.
4. Container persistence. CIP-10 container runtimes require persistent storage layers for stateful applications, databases, and caching across job restarts. Without attachable storage, containers are limited to ephemeral scratch space.

​

2.1 Why not existing systems?

​

3. Definitions

• Volume: A named, account-scoped storage namespace. An account may own multiple Volumes. Each Volume is an isolated container for stored objects.
• Object: A single blob of data stored within a Volume, identified by a path key.
• CBFS: The canonical off-chain storage engine used by this CIP. CBFS nodes implement the Relay Node data plane and CBFS clients implement the object API, manifest handling, and FUSE mount described here.
• Storage Commitment: An onchain record that tracks a Volume’s existence, owner, size, shard placement, creation epoch, and billing state.
• Capability Token (CapToken): A cryptographic bearer token encoding the permissions (read-only, write-only, or read-write), scope (volume + path prefix), time bounds, and size quota for a Runner’s access to a Volume.
• Relay Node: A CBFS storage node participating in the Cowboy network that persistently stores erasure-coded shards of Volume data. Relay Nodes are a distinct network role from Runners and Validators, with their own staking and incentive model.
• Shard: A fragment of an erasure-coded object. An object is split into K data shards and M parity shards (K+M total); any K shards are sufficient to reconstruct the original object.
• ObjectDescriptor: The immutable content-identity record for a stored object. Contains the path, content hash, ciphertext hash, encryption nonce, size, erasure parameters, and per-shard hashes. Stored inside the encrypted manifest via ManifestEntry::File.
• PlacementRecord: The mutable shard-to-node assignment record for a stored object. Contains a shard_id, a list of PlacementAssignment (shard_index → node_id), duplicated erasure params and shard hashes (so repair workers can operate without manifest access), a ciphertext_size (the pre-padding ciphertext length needed for correct erasure reconstruction), and a CAS version for atomic updates. Replicated to all participating Relay Nodes independently of the manifest.

​

4. Design Overview

​

4.1 Architecture

​

4.2 Lifecycle

1. Create: Account owner creates a Volume via the Storage Manager system actor, specifying a name, optional size quota, and replication parameters. An onchain Storage Commitment is written.
2. Attach: When submitting a CIP-2 task, the owner includes one or more volume attachments in the task definition, each specifying the volume name, access mode (read-only, write-only, or read-write), and an optional path prefix scope.
3. Authorize: The Dispatcher (or a delegated Storage Manager) issues a CapToken to each selected Runner. The token is scoped to the specified volume, access mode, path prefix, job duration, and byte quota.
4. Write: The Runner encrypts object data, erasure-codes it into K+M shards, and distributes shards to Relay Nodes. Shards are immediately available for retrieval by other CapToken holders on the same volume.
5. Read: The Runner fetches any K of K+M shards from Relay Nodes, reconstructs the object, decrypts, and verifies the content hash.
6. Commit: At job completion (or periodically), the Runner commits a storage manifest — a Merkle root of all objects written — to the onchain Storage Commitment.
7. Manage: The account owner can list, read, and delete individual objects or entire Volumes at any time via the Storage Manager.

​

4.3 Canonical Implementation Boundary

• Object encryption/decryption for PRIVATE volumes and plaintext handling for PUBLIC volumes.
• Reed-Solomon erasure coding, shard hashing, and shard placement records.
• Relay Node RPCs (PUT_SHARD, GET_SHARD, LIST_SHARDS, placement replication, repair traffic).
• Manifest storage, reconstruction, Merkle root computation, and verification against the authoritative root.
• FUSE mount behavior, local cache, sync daemon, and direct object API.
• Shard repair, garbage collection of orphan shards, and local usage reporting hooks.

• StorageCommitment lifecycle, authoritative manifest_root, and volume ownership semantics.
• CapToken issuance, revocation, and task-scoped attachment semantics.
• Relay Registry membership, staking, health, and repair coordination triggers.
• Billing, fee settlement, storage grace periods, and slashing policy.
• Integration with CIP-2 task submission and the Runner execution flow.

​

4.4 Relationship to Existing CIPs

• CIP-2 (Off-Chain Compute): RAS extends the OffchainTask definition to include volume attachments. The Runner Submission Contract is extended to accept storage manifests alongside result_data.
• CIP-3 (Fee Model): RAS introduces a new fee dimension — persistent storage fees — billed per byte per epoch (including erasure overhead). Unlike CIP-3 Cycles and Cells (which are metered by the VM during transaction execution), storage usage is metered externally by Relay Nodes and settled onchain via attestation-based billing.
• CIP-4 (State Storage): onchain Storage Commitments live in the existing STORAGE key space under the Storage Manager actor’s address. Volume data itself is NOT stored in the MPT trie.
• CIP-10 (Runner Container Runtime): CIP-10 consumes the storage primitive defined here. Container image handling, cgroups, network policy, and GPU passthrough are separate concerns; volume attachment, mount semantics, and the object API are defined by CIP-9 and then mounted into CIP-10 runtimes.

​

5. Relay Nodes

​

5.1 Role and Responsibilities

• Stores erasure-coded shards of encrypted Volume data.
• Stores PlacementRecord entries that map shard IDs to their assigned nodes (see §5.3.1).
• Serves shards and placement records to authorized requesters (Runners with valid CapTokens, account owners).
• Heartbeats to the onchain Relay Registry to prove liveness.
• Runs autonomous two-phase repair (self-heal + redundancy restoration) without Runner involvement (see §5.5).
• Replicates PlacementRecords to peer nodes via ReplicatePlacement RPCs.

​

5.2 Relay Registry

RelayNodeProfile {
  address:           bytes32,
  stake_amount:      u256,           // CBY staked
  capacity_bytes:    u64,            // advertised storage capacity
  used_bytes:        u64,            // current usage
  last_heartbeat:    u64,            // block height
  health:            u8,             // decays per block, reset on heartbeat
  shards_held:       u32,            // number of active shards
  shards_lost:       u32,            // historical shard losses (reputation)
  region_hint:       bytes4          // optional: geographic hint for latency optimization
}

• Register: Relay Node stakes MIN_RELAY_STAKE CBY and calls register_relay(capacity_bytes).
• Heartbeat: Relay Node calls heartbeat() periodically. Health resets to MAX_RELAY_HEALTH (e.g., 100 blocks). Health decays by 1 per block.
• Removal: If health reaches 0, the Relay Node is removed from the active list. Shards assigned to it are flagged for repair (see §5.5).
• Unstake: A Relay Node may unstake after a cooldown period (RELAY_UNSTAKE_DELAY), provided it has no active shard assignments or has transferred them.

​

5.3 Shard Assignment

1. Eligible set: All Relay Nodes in the active list with health > MIN_HEALTH_FOR_ASSIGNMENT and sufficient free capacity.
2. Diversity: Selected nodes SHOULD have distinct region_hint values (best-effort, not enforced in v1).
3. Determinism: The initial assignment is recorded in the object’s PlacementRecord (replicated to Relay Nodes, §5.3.1) so any future reader or repair worker knows which Relay Nodes hold which shards.

ObjectDescriptor {
  object_path:      string,
  write_id:         bytes16,          // CSPRNG(16) — fresh random per write, ensures version isolation
  content_hash:     bytes32,          // BLAKE3 hash of plaintext
  ciphertext_hash:  bytes32,          // BLAKE3 hash of ciphertext (pre-erasure-coding)
  encryption_nonce: bytes12,          // random nonce used for AES-256-GCM (unique per write)
  size_bytes:       u64,              // original object size
  ciphertext_size:  u64,              // ciphertext length before erasure padding (needed for correct reconstruction)
  shard_id:         bytes32,          // BLAKE3(volume_id || object_path || write_id) — opaque, version-unique
  erasure_k:        u8,
  erasure_m:        u8,
  shard_hashes:     [bytes32; K+M],   // BLAKE3 hash of each shard
}

PlacementRecord {
  shard_id:         bytes32,          // matches ObjectDescriptor.shard_id
  version:          u64,              // CAS version — monotonically incremented on each reassignment
  assignments: [
    { shard_index: u8, node_id: bytes32 },
    ...  // K+M entries
  ],
  // Duplicated from ObjectDescriptor so repair workers can operate
  // without manifest access (critical for private volumes where the
  // manifest is encrypted):
  erasure_k:        u8,
  erasure_m:        u8,
  ciphertext_size:  u64,
  shard_hashes:     [bytes32; K+M],
}

​

5.3.1 Placement Persistence

​

5.4 Erasure Coding

1. Generate write_id = CSPRNG(16) and compute shard_id = BLAKE3(volume_id || object_path || write_id).
2. Encrypt the object (AES-256-GCM, see §9). Record ciphertext_size (pre-padding length).
3. Erasure-code the ciphertext into K data shards and M parity shards using Reed-Solomon (reed-solomon-erasure crate).
4. Compute shard_hash = BLAKE3(shard_bytes) for each shard.
5. Select K+M Relay Nodes and PUT each shard to its assigned node, authenticated by the CapToken.
6. Produce an ObjectDescriptor (stored in the manifest) and a PlacementRecord (published to Relay Nodes at commit time, §5.3.1).

1. Look up the ObjectDescriptor from the manifest and the PlacementRecord (from local cache or fetched from Relay Nodes via GetPlacement).
2. Request any K shards from available Relay Nodes listed in the PlacementRecord (prefer lowest-latency nodes; fall back if some are unavailable).
3. Reconstruct the ciphertext using Reed-Solomon decoding, truncating to ciphertext_size to remove erasure padding.
4. Verify ciphertext_hash.
5. Decrypt (AES-256-GCM).
6. Verify content_hash.

​

5.5 Autonomous Shard Repair

1. Verify the local shard bytes against the expected shard_hash from the PlacementRecord.
2. If the shard is missing or corrupt, fetch K healthy shards from peer Relay Nodes listed in the same PlacementRecord.
3. Reconstruct the ciphertext using Reed-Solomon decoding, truncating to ciphertext_size (not the padded shard length — using the wrong length produces garbage).
4. Re-encode and extract the needed shard. Verify its hash. Store locally.

1. For each PlacementRecord, probe all assigned nodes. A node is considered dead if it is unreachable after the configured timeout.
2. Leader election: Only the assigned node with the lowest shard_index among live nodes drives reassignment for that PlacementRecord. This prevents multiple nodes from racing to repair the same shard.
3. The leader selects replacement node(s) from the known peer list (excluding already-assigned nodes).
4. The leader reconstructs the missing shard(s) from K surviving shards (same erasure reconstruction as Phase 1) and uploads via PutShard to the replacement node(s).
5. The leader writes an updated PlacementRecord with the new assignments and an incremented version, using a CAS (Compare-and-Swap) write: the PutPlacement RPC specifies expected_version = current_version. If another node raced and already updated the record, the CAS fails and the leader retries from step 1 on the next cycle.
6. The updated PlacementRecord is replicated to all assigned nodes via ReplicatePlacement.

​

5.6 Proof of Retrievability (PoR) Challenges

1. A periodic onchain timer (CIP-5, every POR_CHALLENGE_INTERVAL blocks) selects a random set of (shard_id, shard_index, byte_offset, byte_length) tuples targeting active Relay Nodes.
2. The challenged Relay Node must respond within POR_RESPONSE_WINDOW blocks with:
   • The requested byte range of the specified shard.
   • A Merkle proof chain linking the byte range to the onchain manifest_root:
     1. byte_range_proof: Merkle proof from the byte range to the shard_hash.
     2. shard_inclusion_proof: Merkle proof from the shard_hash to the manifest_root stored in the volume’s onchain StorageCommitment.
3. The onchain verifier checks the full proof chain:
   • Verify BLAKE3(byte_range_data) matches the leaf in byte_range_proof.
   • Verify byte_range_proof resolves to shard_hash.
   • Verify shard_inclusion_proof resolves to the onchain manifest_root.
   • If any step fails, the response is invalid.

​

5.7 Relay Node Incentives

1. Storage fees: A share of the per-epoch storage fee paid by the account owner, proportional to the number and size of shards held.
2. Transfer fees: A per-byte fee for serving shards to Runners (read bandwidth).

Per epoch, for each volume:
  total_storage_fee = volume_effective_size * STORAGE_FEE_PER_BYTE_PER_EPOCH
  per_relay_share = total_storage_fee / num_relay_nodes_holding_shards

Per read operation:
  transfer_fee = bytes_served * TRANSFER_FEE_PER_BYTE
  (paid to the specific Relay Node serving the shard)

​

6. Storage Addressing

​

6.1 Path-Based Namespace

/{account_address}/{volume_name}/{object_path}

• account_address: The 32-byte Ed25519 public key of the owning account (hex-encoded).
• volume_name: A UTF-8 string (max 64 bytes, restricted to [a-zA-Z0-9_\-.]).
• object_path: A UTF-8 string (max 512 bytes) using / as a logical separator. No leading or trailing /.

/0xaabb...ccdd/model-weights/v3/layer_0.bin
/0xaabb...ccdd/agent-memory/conversations/2026-03-04/session_1.cbor
/0xaabb...ccdd/pipeline-scratch/job_4821/intermediate.parquet

​

6.2 Content Integrity

content_hash    = BLAKE3(plaintext_bytes)       // verifies decrypted content
ciphertext_hash = BLAKE3(ciphertext_bytes)      // verifies pre-erasure-coding ciphertext
shard_hash[i]   = BLAKE3(shard_bytes[i])        // verifies individual shards

• Integrity verification: Any party can verify the full object lifecycle — shard hashes verify individual shards, ciphertext hash verifies reconstruction, content hash verifies decryption.
• Efficient proofs: A Merkle proof for a single object is O(log N) in the number of objects.

​

6.3 Addressing for Deletion

delete_object(volume_name, object_path) -> bool
delete_volume(volume_name) -> bool

• delete_object marks all shards of the object for removal on the assigned Relay Nodes and updates the onchain manifest. Relay Nodes garbage-collect the shard data.
• delete_volume marks all objects for deletion and removes the onchain Storage Commitment. Storage fees cease immediately.
• Only the account owner (or an Actor acting on behalf of the account) may delete. Runners are never granted delete permission, even under read-write mode — this prevents malicious or compromised Runners from destroying data.

​

7. Access Control

​

7.1 Capability Tokens

CapToken {
  volume_id:      bytes32,          // keccak256(account_address || volume_name)
  access_mode:    ENUM { READ_ONLY, WRITE_ONLY, READ_WRITE },
  path_prefix:    string,           // scope to objects under this prefix (e.g., "agent-3/")
  max_bytes:      u64,              // maximum total bytes writable
  valid_from:     u64,              // block height
  valid_until:    u64,              // block height (job timeout)
  runner_address: bytes32,          // the specific Runner authorized
  nonce:          u64,              // prevents replay
  signature:      bytes64           // Ed25519 signature by account owner (or delegated Storage Manager)
}

​

7.1.1 Auth Enforcement on Placement RPCs

​

7.2 Access Modes (CapToken Scopes)

Note — Public volumes: Volumes with visibility = PUBLIC use a different access model. Reads and listings are open to any party without a CapToken. Writes still require a CapToken (WRITE_ONLY or READ_WRITE). See §7.6 for the full PUBLIC volume specification.

​

7.3 Concurrent CapTokens

• Non-overlapping write prefixes: If two CapTokens grant WRITE access, their path_prefix values MUST NOT overlap. The Dispatcher enforces this at token issuance.
  • Prefix canonicalization: Prefixes are canonicalized by ensuring a trailing / separator. A prefix agent-1 is stored as agent-1/. This prevents ambiguity: agent-1/ and agent-10/ are non-overlapping; agent-1/ and agent-1/sub/ DO overlap (the first is a parent of the second). Overlap is defined as: prefix A overlaps prefix B if A is a prefix of B or B is a prefix of A (after canonicalization).
  • Empty prefix ("") means full volume access. No other WRITE CapToken may be active on the volume simultaneously if any token has an empty prefix.
• Reads never conflict: READ_ONLY tokens may coexist with any number of other READ_ONLY or WRITE tokens. A READ_WRITE token may read paths being written by other tokens.
• No total ordering of writes: Concurrent writes to different paths are independent. There is no global write ordering across CapTokens.
• CapToken revocation: A CapToken can be revoked before its valid_until by the Dispatcher recording the token’s nonce in a revocation list. Relay Nodes check the revocation list on each request. Writes in-flight at revocation time may or may not land; the next manifest commit determines the canonical state. Revocation is best-effort and convergent — Relay Nodes may serve a revoked token briefly until the revocation propagates.

​

7.3.1 Prefix Enforcement Boundaries

1. Revoke the offending sub-agent’s CapToken (preventing further writes).
2. Re-commit a corrected manifest that excludes the unauthorized objects.
3. Dispute the Runner via the job settlement mechanism (CIP-2), potentially slashing the Runner’s stake.

1. Privacy conflict: For private volumes, the manifest is encrypted — object paths are not visible onchain. Requiring the chain to verify paths would reveal them, contradicting the privacy guarantees in §9.3.
2. Sampling is insufficient: Even for public volumes, randomly sampling paths from the manifest and checking they fall within the prefix cannot prove the absence of unauthorized paths. A rogue Runner could include 1,000 valid paths and 1 unauthorized path; sampling would likely miss it.

​

7.4 Caveats and Restrictions

• Path prefix narrowing: A CapToken scoped to job_4821/ can be further restricted to job_4821/checkpoints/ but never broadened to /.
• Byte quota reduction: A 1 GiB quota can be reduced to 512 MiB but never increased.
• Time window narrowing: The valid window can be shortened but never extended.

​

7.5 Read Consistency

1. Fetch K shards of the manifest from Relay Nodes and reconstruct it.
2. Compute the Merkle root of the reconstructed manifest.
3. Compare the computed root to the onchain manifest_root in the volume’s StorageCommitment.
4. Reject on mismatch. A mismatched root means the fetched manifest is stale, partially published, or corrupted.

Future work — READ_UNCOMMITTED: A mode where objects are visible as soon as shards land on Relay Nodes (before commit_manifest()) is desirable for the real-time agent swarm pattern, where latency matters more than strict consistency. However, it is not implementable on the current design for two reasons:

1. Discovery: With versioned shard IDs (§5.3, write_id in the shard address), a reader cannot predict the shard_id for an uncommitted write — they don’t know the writer’s random write_id. Discovering uncommitted objects requires a separate metadata channel (pubsub or uncommitted manifest fragments) that this spec does not yet define.
2. Prefix safety: Without the coordinator verifying the committed manifest, a reader could observe out-of-prefix garbage from a rogue writer.

READ_UNCOMMITTED is deferred to a future CIP that defines the discovery mechanism.

​

7.6 Public Volumes (PUBLIC)

​

7.6.1 Overview

​

7.6.2 Properties

• No encryption: Objects in PUBLIC volumes are stored unencrypted on Relay Nodes. No DEK is generated for the volume. The wrapped_dek field in the StorageCommitment is empty.
• No CapToken for reads: Any party can fetch shards from Relay Nodes without presenting a CapToken. Relay Nodes serve GET_SHARD requests for public shards unconditionally.
• CapToken still required for writes: Only the account owner (or authorized Runners via CapToken) can write to the volume. Write access uses the same CapToken mechanism as private volumes.
• Content integrity preserved: content_hash (BLAKE3) is still computed and stored for every object. Readers MUST verify the content hash after shard reconstruction to detect corruption or tampering.
• Erasure coding preserved: Reed-Solomon coding applies identically. The only change is that the input to erasure coding is plaintext (not ciphertext).
• Billing unchanged: The account owner pays the same per-epoch, per-byte storage fees as private volumes.
• Listing is public: list_objects for public volumes does not require a CapToken. The manifest is stored unencrypted and readable by anyone who fetches it from Relay Nodes at the well-known manifest shard address (BLAKE3(volume_id || "__manifest__")).

​

7.6.3 Relay Node Behavior

• GET_SHARD requests are served without CapToken verification (authorized via shard metadata).
• PUT_SHARD requests still require a valid CapToken with write access. The AuthProvider attaches { "visibility": "PUBLIC" } metadata to the AuthDecision, which the Relay Node persists alongside the shard.

​

7.6.4 Shard ID Opacity

​

7.6.5 Content-Type Metadata

{
  "defaults": {
    ".html": "text/html; charset=utf-8",
    ".css": "text/css",
    ".js": "application/javascript",
    ".json": "application/json",
    ".png": "image/png",
    ".jpg": "image/jpeg",
    ".svg": "image/svg+xml",
    ".woff2": "font/woff2"
  },
  "overrides": {
    "data/feed.xml": "application/atom+xml"
  }
}

​

7.6.6 Cache Headers

{
  "default_max_age": 3600,
  "paths": {
    "assets/*": {"max_age": 86400, "immutable": true},
    "index.html": {"max_age": 0, "must_revalidate": true}
  }
}

​

8. Volume Attachment

​

8.1 Attachment at Job Dispatch

VolumeAttachment {
  volume_name:   string,
  access_mode:   ENUM { READ_ONLY, WRITE_ONLY, READ_WRITE },
  path_prefix:   string?,          // optional: restrict to sub-path
  max_bytes:     u64,              // byte quota for this job
}

​

8.2 Attachment Process

1. CapToken issuance: The Dispatcher issues a scoped CapToken for each volume attachment.
2. Volume key delivery (private volumes only): The Dispatcher unwraps the volume DEK, re-encrypts it to the assigned Runner’s ephemeral public key, and includes the encrypted_dek in the job assignment payload (see §9.2). Public volumes skip this step.
3. Manifest fetch: The Runner fetches the current volume manifest from Relay Nodes at the well-known manifest shard address (BLAKE3(volume_id || "__manifest__")), reconstructs it, and verifies the Merkle root matches the onchain manifest_root (§7.5). Only after verification does the Runner trust the manifest for reading or writing.

attachment_cost = BASE_ATTACHMENT_FEE

​

8.3 Detachment

1. The Runner commits the final storage manifest onchain (if it wrote any objects).
2. The CapToken is invalidated (past valid_until).
3. Shards written during the job persist on Relay Nodes, independent of the Runner’s lifecycle.

​

9. Encryption and Privacy

​

9.1 Encryption at Rest

1. Volume DEK (Data Encryption Key): A random 256-bit key generated at volume creation time. The DEK is never derived from the account’s signing key — this maintains Ethereum-style key hygiene where signing keys are only used for signing, not key derivation.
   volume_dek = CSPRNG(32)  // generated once at create_volume()
   
   The DEK is envelope-encrypted for storage:
   wrapped_dek = AES-256-GCM(
     key = account_wrapping_key,   // derived from account owner's secret via HKDF
     nonce = random(12),
     plaintext = volume_dek,
     aad = volume_id
   )
   
   The account_wrapping_key is derived via HKDF from the owner’s secret material (for EOAs: a dedicated encryption seed separate from the signing key; for Actor contracts: a key managed by the controlling EOA or a threshold scheme among authorized signers). The wrapped DEK is stored in the onchain Storage Commitment. Only the account owner can unwrap it.
2. Object Encryption: Each object is encrypted with AES-256-GCM using a random nonce per write:
   nonce = CSPRNG(12)       // fresh random nonce for EVERY write, including overwrites
   ciphertext = AES-256-GCM(key=volume_dek, nonce=nonce, plaintext=object_bytes, aad=object_path)
   
   The nonce is stored alongside the ciphertext in the ObjectDescriptor. This is critical: deterministic nonces derived from the object path would cause nonce reuse on overwrites, which is catastrophic for AES-GCM (leaks XOR of plaintexts, breaks authentication). A fresh random nonce on every write eliminates this class of attack entirely.
3. Shard opacity: Erasure coding is applied to the ciphertext, not the plaintext. Relay Nodes hold shards of ciphertext — even if they reconstructed all shards, they would only have ciphertext.
4. Manifest encryption: The volume manifest (list of object paths, sizes, content hashes, ObjectDescriptors) is encrypted with the volume DEK before transmission to Relay Nodes. PlacementRecords are stored separately on Relay Nodes (§5.3.1) and are not part of the encrypted manifest.

​

9.2 Runner Key Access

1. The Dispatcher reads the wrapped_dek from the Storage Manager for each attached volume.
2. The Dispatcher unwraps the DEK using the account owner’s wrapping key. The wrapping key is derived deterministically from the owner’s account keypair: wrapping_key = HKDF-SHA256(account_secret, "cbfs-volume-" || volume_id). The Storage Manager stores a wrapping_key_hash to verify correctness.
3. The Dispatcher re-encrypts the plaintext DEK to the assigned Runner’s ephemeral public key (from the Runner’s CIP-2 registration) using X25519 + AES-256-GCM (ECIES).
4. The encrypted_dek is included in the job assignment payload alongside the CapToken.

• The DEK never appears in plaintext on-chain or in any persistent store.
• The Dispatcher sees the plaintext DEK transiently during re-encryption. This is acceptable because the Dispatcher is a system actor with the same trust level as consensus.
• If the Runner is TEE-attested (CIP-2 tee_required=true), the DEK is sealed to the enclave and never exposed to the host OS.
• The wrapping_key_hash in the StorageCommitment allows the Storage Manager to verify that the correct wrapping key is used without storing the key itself.
• READ_ONLY CapTokens on private volumes still require DEK delivery — the Runner must decrypt ciphertext shards to serve reads.

​

9.3 Privacy Guarantees

• Relay Nodes see only ciphertext shards indexed by opaque shard IDs (see §16.3). They cannot read object contents or inspect the manifest. Object paths are encrypted within the manifest and never exposed to Relay Nodes.
• Other Runners (not assigned to the job) cannot access the volume DEK.
• onchain observers see only the Storage Commitment (volume ID, wrapped DEK, encrypted manifest hash, total size, shard assignments). They cannot determine what is stored.
• The account owner has full access to all their volume data by unwrapping the DEK with their wrapping key.

• No confidentiality: Data is stored unencrypted. Relay Nodes, Runners, and any network participant can read object contents. This is by design — public volumes are intended for publicly readable data (web assets, public datasets, shared artifacts).
• Integrity preserved: Content hashes (BLAKE3) ensure that data has not been tampered with, even though it is unencrypted.
• Write access is still restricted: Only the account owner (or authorized Runners via CapToken) can write to a public volume. Public readability does not imply public writability.

​

10. Billing and Fees

​

10.1 Fee Components

​

10.2 Effective Size and Erasure Overhead

effective_size = raw_size * (K + M) / K

​

10.3 Persistent Storage Billing

• Each epoch, the protocol calculates the total effective storage used by each account across all volumes.
• The per-epoch storage fee is deducted from the account’s balance.
• If the account’s balance falls below MIN_STORAGE_BALANCE (sufficient to cover one epoch of fees), the protocol enters a grace period of STORAGE_GRACE_EPOCHS.
• After the grace period, if the balance is still insufficient, all volumes owned by the account are marked for garbage collection.

​

10.4 Fee Distribution

Account Owner ──(per-epoch storage fee)──► Protocol ──► Relay Nodes (pro-rata by shards held)
Runner ──(per-byte transfer fee)──► Relay Node (serving the shard)

​

10.5 Relationship to CIP-3

• onchain operations (creating Storage Commitments, writing manifests) consume Cycles and Cells as normal CIP-3 transactions.
• Off-chain storage fees are a separate ledger entry, debited from the account balance per epoch by the Storage Manager system actor.

​

11. onchain State

​

11.1 Storage Manager System Actor

StorageCommitment {
  volume_id:            bytes32,    // keccak256(account_address || volume_name)
  owner:                address,
  volume_name:          string,
  visibility:           ENUM { PRIVATE, PUBLIC },  // PRIVATE = encrypted, CapToken-gated; PUBLIC = unencrypted, open reads
  created_at:           u64,        // block height
  wrapped_dek:          bytes,      // envelope-encrypted volume DEK (see §9.1); empty for PUBLIC volumes
  wrapping_key_hash:    bytes32,    // BLAKE3(wrapping_key) for verification without storing the key
  manifest_root:        bytes32,    // Merkle root of manifest (encrypted for PRIVATE, plaintext for PUBLIC)
  raw_size_bytes:       u64,        // sum of object sizes (pre-erasure)
  effective_size_bytes: u64,        // raw_size * (K+M)/K
  erasure_k:            u8,         // data shards
  erasure_m:            u8,         // parity shards
  last_updated:         u64,        // block height of last manifest update
  degraded_shards:      u16,        // count of shards needing repair
  status:               ENUM { ACTIVE, GRACE_PERIOD, DELETED, GARBAGE_COLLECTING }
}

AccountStorageSummary {
  total_volumes:          u32,
  total_effective_bytes:  u64,
  last_billed_epoch:      u64,
  balance_reserved:       u256       // CBY reserved for storage fees
}

​

11.2 Relay Registry

• RelayNodeProfile entries (see §5.2)
• Active relay list (ordered, health-decaying, analogous to CIP-2 Runner Registry)
• Shard assignment index: volume_id → list[PlacementAssignment]

​

11.3 Key Space

key = 0x1 || keccak256(storage_manager_address) || 0x00 || keccak256("commitment" || volume_id)
value = rlp(StorageCommitment)

key = 0x1 || keccak256(storage_manager_address) || 0x00 || keccak256("summary" || account_address)
value = rlp(AccountStorageSummary)

key = 0x1 || keccak256(relay_registry_address) || 0x00 || keccak256("relay" || relay_address)
value = rlp(RelayNodeProfile)

​

12. Client Interfaces

• Filesystem interface (§12.1): A FUSE-mounted directory that presents the volume as a standard filesystem. This is the primary interface for agentic workloads where an LLM (Claude, Kimi-K2, GPT, etc.) operates via tool calling. The model’s existing filesystem tools — Read, Write, Bash (ls, grep, find), Glob, Grep — work unchanged against the mounted volume. No custom tool definitions required.
• Object API (§12.2): A programmatic interface for orchestration code and non-agentic workloads. This is what the FUSE layer calls internally, and is also available directly for lightweight write-only jobs that don’t need filesystem semantics.

​

12.1 Filesystem Interface (FUSE Mount)

​

12.1.1 Design Rationale

• Injecting custom tool definitions into every model’s tool set
• Models are less fluent with unfamiliar, domain-specific tools
• Loss of composability with standard unix tools (grep, find, jq, wc, etc.)
• Every model provider’s runner integration needs custom work

Model tool call                        What happens under the hood
─────────────────                      ──────────────────────────────
Read("/mnt/memory/state.json")    →    FUSE read → fetch shards → reconstruct → decrypt
Write("/mnt/memory/state.json")   →    FUSE write → local buffer → background push
Bash("ls /mnt/memory/logs/")      →    FUSE readdir → list objects from manifest
Bash("grep -r 'error' /mnt/mem")  →    FUSE read (multiple) → local cache → grep
Bash("wc -l /mnt/memory/*.json")  →    FUSE read (multiple) → local cache → wc

​

12.1.2 Mount Point Layout

/mnt/volumes/{volume_name}/

# Full volume mount:
/mnt/volumes/agent-memory/
├── state/
│   ├── memory.json
│   └── portfolio.json
├── logs/
│   └── 2026-03-04/
│       └── analysis.json
└── config.json

# Prefix-scoped mount (path_prefix="state/"):
/mnt/volumes/agent-memory/
├── memory.json
└── portfolio.json

​

12.1.3 Sync Strategy (Hybrid)

• Pull cycle (Relay Nodes → local): Re-fetches the volume’s committed manifest from Relay Nodes, verifies it against the onchain manifest_root (§7.5), and materializes any new objects not already in the local tmpfs. Because reads are READ_COMMITTED (§7.5), the pull cycle only discovers objects that have been included in a committed manifest — objects from a sub-agent become visible only after that sub-agent calls commit_manifest(). In the agent swarm pattern, this means the coordinator sees a sub-agent’s files appear as a batch when the sub-agent commits, not individually as they are written.
• Push cycle (local → Relay Nodes): Detects locally written or modified files (via inotify/fswatch), encrypts them, erasure-codes, and distributes shards to Relay Nodes.

​

12.1.4 FUSE Operation Mapping

​

12.2 Object API (Programmatic)

• The Runner is a script (not an LLM doing tool calling) that just needs to write output files.
• The workload is lightweight and a full FUSE mount is unnecessary overhead.
• The orchestration layer needs to interact with storage outside of a container context.

# Object operations (subject to CapToken permissions)
put_object(cap_token: bytes, object_path: str, data: bytes) -> ObjectReceipt
get_object(cap_token: bytes, object_path: str) -> bytes          # READ_ONLY or READ_WRITE
list_objects(cap_token: bytes, prefix: str = "") -> list[str]    # READ_ONLY or READ_WRITE

# CBFS-level commit (data plane)
commit(cap_token: bytes) -> CommitReceipt

# Cowboy-level commit (control plane, onchain)
commit_manifest(cap_token: bytes, manifest_root: bytes32) -> bool

1. CBFS commit (commit): Publishes the encrypted manifest to Relay Nodes at the well-known manifest shard address (BLAKE3(volume_id || "__manifest__")), and publishes PlacementRecords to all assigned Relay Nodes via PutPlacement (§5.3.1). This is a data-plane operation — it makes the data durable and discoverable by other CBFS clients, but does not touch the chain. PlacementRecord publication is best-effort; failures are logged but do not block the commit.
2. Cowboy commit (commit_manifest): Submits only the Merkle root (32 bytes) to the onchain StorageCommitment via the Runner Submission Contract. This anchors the manifest for READ_COMMITTED consistency (§7.5) and PoR challenge verification (§5.6). Individual ObjectDescriptors are stored off-chain in the encrypted manifest — the onchain Merkle root enables O(log N) inclusion proofs without requiring the full manifest onchain.

​

12.3 Account Owner API (via Storage Manager Actor)

# Volume lifecycle
create_volume(
    volume_name: str,
    max_size_bytes: int = 0,
    erasure_k: int = 4,            # data shards (default 4)
    erasure_m: int = 2,            # parity shards (default 2)
    visibility: str = "PRIVATE",   # "PRIVATE" (encrypted, default) or "PUBLIC" (unencrypted, open reads)
) -> bytes32  # returns volume_id

delete_volume(volume_name: str) -> bool           # soft-delete: sets status=DELETED, starts GC timer
list_volumes() -> list[VolumeInfo]
transfer_volume(volume_name: str, new_owner: address) -> bool  # transfer ownership to another account

# Object management
delete_object(volume_name: str, object_path: str) -> bool
list_objects(volume_name: str, prefix: str = "") -> list[ObjectInfo]
get_volume_info(volume_name: str) -> VolumeInfo

# Billing
get_storage_usage() -> AccountStorageSummary
reserve_storage_balance(amount: uint256) -> bool

​

12.4 CIP-2 Task Definition Extension

struct OffchainTask:
    ... (existing CIP-2 fields) ...
    volume_attachments: list[VolumeAttachment]  # NEW: optional

struct VolumeAttachment:
    volume_name:      string
    access_mode:      uint8          # 0 = READ_ONLY, 1 = WRITE_ONLY, 2 = READ_WRITE
    path_prefix:      string         # "" for full volume access (canonicalized with trailing /)
    max_bytes:        uint64         # byte quota for this job
    mount:            bool           # true = FUSE mount at /mnt/volumes/{name}, false = Object API only
    sync_interval:    uint32         # seconds between sync cycles (default 5, mount only)
    # read_consistency removed in v1 — all reads are READ_COMMITTED (see §7.5)

​

13. Garbage Collection

​

13.1 Triggers

1. Explicit deletion: Account owner calls delete_volume() or delete_object().
2. Balance exhaustion: Account balance insufficient after STORAGE_GRACE_EPOCHS.
3. Expiry: If a volume has an optional expiry_height set at creation, data is collected after that height.

​

13.2 Process

​

13.3 Deletion Semantics

• On delete_object: All shards of the object are marked for removal on their respective Relay Nodes. The onchain manifest is updated. Relay Nodes garbage-collect shard data asynchronously, but the object is immediately inaccessible to CapToken holders.
• On delete_volume: The volume enters the DELETED state (soft-delete). All active CapTokens are revoked. New CapTokens cannot be issued. Storage fees continue accruing during a grace window of VOLUME_DELETE_GRACE_EPOCHS (default: same as STORAGE_GRACE_EPOCHS). During this window, the account owner may call undelete_volume() to restore the volume to ACTIVE status. After the grace window, the volume transitions to GARBAGE_COLLECTING and all shards are purged. The onchain Storage Commitment is removed and storage fees cease.
• Garbage collection is irreversible. Once a volume enters GARBAGE_COLLECTING, data cannot be recovered.

​

13.4 Ownership Transfer

• All active CapTokens are revoked (they were issued under the old owner’s authority).
• The StorageCommitment.owner field is updated atomically.
• The new owner assumes billing responsibility starting from the next epoch.
• For private volumes, the wrapped_dek is re-encrypted to the new owner’s wrapping key as part of the transfer transaction. This requires the old owner to unwrap and re-wrap the DEK — the transfer is therefore an interactive operation requiring the old owner’s cooperation.
• For public volumes, no key re-wrapping is needed (no DEK exists).
• Transfer of a volume in DELETED or GARBAGE_COLLECTING status is rejected.

​

14. Parameters

​

15. Security Considerations

​

15.1 CapToken Forgery

​

15.2 Runner Compromise

• Byte quotas limit the damage per job.
• Coordinator prefix verification: The coordinator (READ_WRITE holder) verifies all committed paths fall within the writer’s prefix after each manifest commit. Out-of-prefix writes are detected and the CapToken is revoked (§7.3.1).
• Orphan shard GC: Shards not referenced by any committed manifest are garbage collected after ORPHAN_SHARD_TTL.
• No delete access: A compromised Runner cannot destroy existing data.

• TEE attestation: For sensitive workloads, require TEE-attested Runners (CIP-2 tee_required=true).
• Minimal scope: Use path_prefix to restrict access to only the necessary sub-path.
• Account owner discretion: READ_WRITE is an explicit opt-in; the account owner accepts the elevated trust.

​

15.3 Relay Node Compromise

​

15.4 Denial of Service

​

15.5 Key Management

• Volume DEKs are envelope-encrypted with the account owner’s wrapping key, which is derived deterministically from the account keypair via HKDF (§9.2). Loss of the account keypair means loss of access to all private volume data. The wrapped_dek is stored onchain in the StorageCommitment, so there is no risk of losing the DEK itself — only the ability to unwrap it.
• Dispatcher trust boundary: In v1, the Dispatcher transiently sees plaintext DEKs during key delivery (§9.2). This is equivalent to the trust already placed in the Dispatcher for job assignment and CapToken issuance. A compromised Dispatcher could exfiltrate DEKs for all private volumes it handles. The v2 Secrets Manager path (§9.2) eliminates this by moving key delivery to a sealed Runner↔SecretsMgr channel.
• Key rotation is not supported in v1. If an account’s keypair is compromised, the owner must create new volumes, generate new DEKs, and re-encrypt data. Key rotation (re-wrapping DEKs with a new wrapping key without re-encrypting all data) is planned for a future CIP.
• During ownership transfer (§13.4), the DEK is re-wrapped to the new owner’s wrapping key. This is the only case where a DEK changes its wrapping key without re-encrypting all data.

​

16. Implementation Notes

​

16.1 Canonical Implementation Stack

​

16.2 FUSE Mount Implementation

1. FUSE daemon (fuser crate): Implements the Filesystem trait, handling read, write, readdir, getattr, etc. Delegates to the local cache layer.
2. Local cache (tmpfs-backed): An in-memory filesystem that serves as the working copy. All reads/writes hit the cache first. The cache is populated lazily on first access (fetch from Relay Nodes) and eagerly for files modified locally.
3. Sync daemon (background task): Runs a push/pull loop at the configured sync_interval. Uses notify for detecting local changes and polls Relay Nodes for remote changes. Handles encryption, erasure coding, and shard distribution.

┌─────────────────────────────────────────────────────────┐
│ Container / Runner process                              │
│                                                         │
│  Model (Claude, Kimi-K2, etc.)                          │
│    │                                                    │
│    ├─ Read("/mnt/volumes/mem/state.json")               │
│    ├─ Write("/mnt/volumes/mem/state.json", data)        │
│    └─ Bash("ls /mnt/volumes/mem/")                      │
│         │                                               │
│         ▼                                               │
│  ┌─────────────┐     ┌──────────────┐                   │
│  │ FUSE daemon │◄───►│ Local cache   │                  │
│  │ (fuser)     │     │ (tmpfs)       │                  │
│  └─────────────┘     └──────┬───────┘                   │
│                             │                           │
│                      ┌──────▼───────┐                   │
│                      │ Sync daemon  │                   │
│                      │  push/pull   │                   │
│                      └──────┬───────┘                   │
│                             │                           │
└─────────────────────────────┼───────────────────────────┘
                              │ encrypt / erasure code / QUIC
                              ▼
                     ┌─────────────────┐
                     │   Relay Nodes   │
                     └─────────────────┘

​

16.3 Object API Client Library

// Programmatic usage (non-agentic runners):
let data = volume.get("state/memory.cbor").await?;
volume.put("state/memory.cbor", &updated_data).await?;

// Under the hood:
// get: fetch PlacementRecord → identify assigned nodes → fetch K shards
//      → Reed-Solomon reconstruct (truncate to ciphertext_size)
//      → verify ciphertext hash → AES-GCM decrypt → verify content hash
// put: AES-GCM encrypt → Reed-Solomon encode → distribute K+M shards
//      → produce ObjectDescriptor (manifest) + PlacementRecord (published at commit)

​

16.4 Relay Node Implementation

1. Blob store (cbfs-store): A sled-backed key-value store mapping (shard_id, shard_index) → shard_bytes. The shard_id is an opaque BLAKE3 hash (see §5.3) — Relay Nodes never see object paths, only opaque identifiers. This ensures the privacy guarantee in §9.3.
2. Placement store (cbfs-placement): A sled-backed store mapping shard_id → PlacementRecord. Stores the mutable shard-to-node assignments separately from shard data, enabling repair workers to read placement information without accessing the encrypted manifest.
3. RPC server: Accepts shard operations (PUT_SHARD, GET_SHARD, PROVE_SHARD) and placement operations (PutPlacement, GetPlacement, ReplicatePlacement). All operations are authenticated by the AuthProvider (§7.1.1). For public volumes (visibility = PUBLIC), GET_SHARD and GetPlacement are served without CapToken verification (§7.6.3); write operations still require a CapToken. Relay Nodes do NOT expose any listing operation — object listing is performed client-side by reading the manifest. Requests are keyed by shard_id, not object path.
4. Repair loop (repair.rs): Periodically runs the two-phase autonomous repair cycle (§5.5). Uses a mutable peer list (Arc<RwLock<Vec<NodeInfo>>>) bootstrapped from seed peers in the node config and updated as new nodes are discovered.
5. Placement sync (placement_sync.rs): Replicates PlacementRecords to peer nodes to ensure all assigned nodes have a consistent view of shard assignments.
6. Heartbeat loop: Periodically calls heartbeat() on the onchain Relay Registry.
7. Spot-check responder: When challenged, returns a random chunk of a specified shard for integrity verification.

​

16.5 Performance Expectations

​

17. Scope and Future Work

​

17.1 v1 Scope (This CIP)

• Private and public account-scoped volumes.
• READ_ONLY, WRITE_ONLY, and READ_WRITE CapToken access modes; PUBLIC volume visibility mode (§7.6).
• Concurrent CapTokens on the same volume (agent swarm pattern).
• Two client interfaces: FUSE filesystem mount (for agentic/LLM workloads) and Object API (for programmatic workloads).
• Hybrid sync strategy for FUSE mounts (local cache + background push/pull).
• Relay Nodes as a dedicated storage layer with staking and incentives.
• Reed-Solomon erasure coding (default 4/6) for durability.
• AES-256-GCM encryption with HKDF-derived keys.
• BLAKE3 content hashing at object, ciphertext, and shard levels.
• Path-based addressing with Merkle manifest for integrity proofs.
• Per-epoch billing with grace period and garbage collection.
• Lazy and proactive shard repair.

​

17.2 Explicitly Out of Scope

• Content-addressed retrieval: v1 uses path-based addressing only. CID-based retrieval (IPFS-style) may be layered on top in the future.
• Storage marketplace: A competitive market where Relay Nodes bid on storage deals (Filecoin-style) is a future extension. In v1, pricing is protocol-set.
• Alternative storage backends: v1 standardizes on CBFS as the canonical storage layer. Supporting Filecoin, Arweave, or other backends under the same CIP-9 surface is future work and would require a follow-on standard.
• READ_UNCOMMITTED consistency mode: Pre-commit read visibility for real-time agent swarms. Requires a shard discovery mechanism (pubsub or uncommitted manifest fragments) not defined in this CIP. See §7.5.
• Cross-account sharing: Delegating CapTokens across accounts for collaborative storage.
• Key rotation: Rotating volume encryption keys without re-encrypting all data.
• Runner-initiated task dispatch: Allowing a coordinator Runner to dynamically spawn sub-agent tasks. Currently, all tasks must be dispatched by onchain Actors. This is a CIP-2 extension.
• Full container runtime spec: Container image management, container registries, resource limits, GPU passthrough, and network policies. CIP-9 defines the CBFS-backed storage primitive (including the FUSE mount and object API); a separate CIP will define the full container runtime that consumes it.

​

Appendix A: Worked Examples

​

A.1 AI Agent with Persistent Memory (Filesystem Mount)

submit_task(
    task_definition=encode_task({
        "model": "claude-sonnet",
        "system": "You are a trading analyst. Your memory and portfolio are in /mnt/volumes/agent-memory/state/.",
        "prompt": "Analyze today's market conditions. Update your memory and portfolio.",
    }),
    volume_attachments=[
        VolumeAttachment(
            volume_name="agent-memory",
            access_mode=READ_WRITE,
            mount=True,                    # FUSE mount at /mnt/volumes/agent-memory/
            max_bytes=500 * 1024 * 1024,
        ),
    ],
    num_runners=1,
    timeout_blocks=500,
    proof_type_requested=TEE,
    ...
)

── Claude's tool calls during execution ──────────────────────────

1. Read("/mnt/volumes/agent-memory/state/memory.json")
   → Returns the agent's memory from yesterday's run
   → (Under the hood: FUSE → fetch shards from Relay Nodes → reconstruct → decrypt)

2. Read("/mnt/volumes/agent-memory/state/portfolio.json")
   → Returns the current portfolio state

3. Bash("ls /mnt/volumes/agent-memory/logs/")
   → 2026-03-01/  2026-03-02/  2026-03-03/

4. Bash("grep -r 'NVDA' /mnt/volumes/agent-memory/logs/2026-03-03/")
   → Shows yesterday's NVDA-related log entries

5. [Claude performs analysis, makes decisions]

6. Write("/mnt/volumes/agent-memory/state/memory.json", updated_memory)
   → Writes to local tmpfs instantly
   → (Under the hood: sync daemon pushes to Relay Nodes within 5 seconds)

7. Write("/mnt/volumes/agent-memory/state/portfolio.json", updated_portfolio)

8. Write("/mnt/volumes/agent-memory/logs/2026-03-04/analysis.json", todays_analysis)

── Container shuts down ──────────────────────────────────────────
   → Final sync pushes any remaining dirty files
   → Manifest committed onchain
   → tmpfs deleted. Data persists on Relay Nodes.

​

A.2 Distributed Scraping with Map-Reduce (Direct Object API)

for i, site in enumerate(sites):
    submit_task(
        task_definition=encode_task({"action": "scrape", "url": site}),
        volume_attachments=[
            VolumeAttachment(
                volume_name="scrape-results",
                access_mode=WRITE_ONLY,
                path_prefix=f"scraper-{i}/",
                max_bytes=500_000_000,
                mount=False,                   # no FUSE mount, direct object writes
            ),
        ],
        ...
    )

# Simple Python script, not an LLM. Uses the Object API directly.
for page in crawl(site_url):
    put_object(cap_token, f"scraper-{my_id}/{page.slug}.json", json.dumps({
        "url": page.url,
        "title": page.title,
        "content": page.text,
        "links": page.links,
    }))
commit_manifest(cap_token, manifest_root)

submit_task(
    task_definition=encode_task({
        "model": "kimi-k2",
        "system": "You have scraped web data in /mnt/volumes/scrape-results/. Analyze and collate.",
        "prompt": "Find all mentions of product launches across the scraped sites. Write a summary.",
    }),
    volume_attachments=[
        VolumeAttachment(
            volume_name="scrape-results",
            access_mode=READ_WRITE,
            mount=True,                        # FUSE mount for filesystem access
            max_bytes=100_000_000,
        ),
    ],
    ...
)

1. Bash("find /mnt/volumes/scrape-results -name '*.json' | wc -l")
   → 47

2. Bash("ls /mnt/volumes/scrape-results/")
   → scraper-0/  scraper-1/  scraper-2/  scraper-3/  scraper-4/

3. Bash("cat /mnt/volumes/scrape-results/scraper-0/about-page.json | jq '.title'")
   → "About Us - Acme Corp"

4. Bash("grep -rl 'product launch' /mnt/volumes/scrape-results/")
   → scraper-0/news.json
   → scraper-2/blog-post-3.json
   → scraper-4/press-release.json

5. Read("/mnt/volumes/scrape-results/scraper-0/news.json")
   → [full content]

6. [... reads relevant files, analyzes ...]

7. Write("/mnt/volumes/scrape-results/collated/summary.md", summary)
8. Write("/mnt/volumes/scrape-results/collated/product-launches.json", structured_data)

​

A.3 Agent Swarm with Batch Coordination (Filesystem Mount + Concurrent Writers)

# Coordinator: READ_WRITE mount, sees the full volume
submit_task(
    task_definition=encode_task({
        "model": "claude-sonnet",
        "system": "You are coordinating 5 research agents. Their reports will appear in /mnt/volumes/swarm/agent-*/. Poll for new reports and synthesize findings. Reports appear in batches as each agent completes and commits.",
    }),
    volume_attachments=[
        VolumeAttachment(volume_name="swarm", access_mode=READ_WRITE, mount=True,
                         sync_interval=5, max_bytes=100_000_000),
    ],
    timeout_blocks=2000,
    ...
)

# Sub-agents: WRITE_ONLY mount, scoped to their prefix
for i in range(5):
    submit_task(
        task_definition=encode_task({
            "model": "claude-haiku",
            "system": f"You are research agent {i}. Write your findings to /mnt/volumes/swarm/.",
            "prompt": f"Research: {topics[i]}",
        }),
        volume_attachments=[
            VolumeAttachment(volume_name="swarm", access_mode=WRITE_ONLY, mount=True,
                             path_prefix=f"agent-{i}/", max_bytes=50_000_000),
        ],
        timeout_blocks=1000,
        ...
    )

1. [Performs research using web tools]

2. Write("/mnt/volumes/swarm/report.md", research_findings)
   → (Visible path: agent-2/report.md due to prefix scoping)

3. Write("/mnt/volumes/swarm/sources.json", source_list)
   → (Visible path: agent-2/sources.json)

── Container shuts down ──
── Sync daemon pushes all shards to Relay Nodes ──
── Runtime calls commit_manifest(), updating onchain manifest_root ──
── Agent-2's files are now committed and visible to other readers ──

1. Bash("ls /mnt/volumes/swarm/")
   → (empty — no sub-agents have committed yet)

   [Waits... sync daemon re-fetches manifest every 5 seconds,
    verifying against onchain manifest_root (§7.5)]

   [Agent-0 finishes and commits its manifest]

2. Bash("ls /mnt/volumes/swarm/")
   → agent-0/
   → (Agent-0 committed — all its files appear at once)

3. Read("/mnt/volumes/swarm/agent-0/report.md")
   → [agent-0's findings]

   [Agent-1 and agent-3 finish and commit around the same time]

4. Bash("ls /mnt/volumes/swarm/")
   → agent-0/  agent-1/  agent-3/
   → (Two more agents committed since last check)

5. Read("/mnt/volumes/swarm/agent-1/report.md")
   → [agent-1's findings]

6. Read("/mnt/volumes/swarm/agent-3/report.md")
   → [agent-3's findings]

   [... continues polling until all 5 agents have committed ...]

7. Write("/mnt/volumes/swarm/synthesis/final-report.md", synthesized_findings)

​

A.4 Multi-Stage Pipeline with Handoff

submit_task(
    task_definition=encode_task({"stage": "preprocess", "source": "https://data.example.com/feed"}),
    volume_attachments=[
        VolumeAttachment(volume_name="pipeline", access_mode=WRITE_ONLY,
                         path_prefix="stage1/", mount=False, max_bytes=2_000_000_000),
    ],
    ...
)

submit_task(
    task_definition=encode_task({
        "model": "kimi-k2",
        "system": "Input data is in /mnt/volumes/pipeline/stage1/. Run inference and write results to /mnt/volumes/pipeline/stage2/.",
    }),
    volume_attachments=[
        VolumeAttachment(volume_name="pipeline", access_mode=READ_WRITE,
                         mount=True, max_bytes=1_000_000_000),
    ],
    ...
)

​

Appendix B: Comparison with Existing Systems

​

Appendix C: CapToken Wire Format

CapToken (variable length)
┌──────────────────────────────────────────────────┐
│ version:         u8          (1 byte)            │
│ volume_id:       bytes32     (32 bytes)          │
│ access_mode:     u8          (1 byte) [0=RO,1=WO,2=RW] │
│ path_prefix_len: u16         (2 bytes)           │
│ path_prefix:     bytes       (variable)          │
│ max_bytes:       u64         (8 bytes)           │
│ valid_from:      u64         (8 bytes)           │
│ valid_until:     u64         (8 bytes)           │
│ runner_address:  bytes32     (32 bytes)          │
│ nonce:           u64         (8 bytes)           │
│ caveats_hash:    bytes32     (32 bytes)          │
│ signature:       bytes64     (64 bytes)          │
└──────────────────────────────────────────────────┘