​

Abstract

​

Introduction

• State — small, consensus‑critical key/value data that participates directly in the state transition function. Billed per cell, subject to rent, stored in QMDB, and bounded by per‑actor quotas.
• Storage — large, potentially private data, durably kept across a permissionless Relay Node network and referenced on‑chain only by compact commitments. Billed per byte per epoch in a separate market, mediated by capability tokens rather than transactions.

​

Architectural Overview

​

Terminology

• Volume: A named, account‑scoped storage namespace. Each volume is a private (encrypted) or public (plaintext) container of objects identified by path keys.
• Object: A single blob within a volume, identified by its path. Encrypted client‑side for private volumes; plaintext for public ones.
• Relay Node: A network participant that stores erasure‑coded shards of volume data, answers reads, participates in repair, and serves Proof‑of‑Retrievability challenges. A distinct role from Validators and Runners.
• Shard: A fragment of an erasure‑coded object. An object is split into K data + M parity shards; any K reconstruct the ciphertext.
• Manifest: The authoritative, Merkle‑committed index of a volume’s contents. Encrypted for private volumes, plaintext for public ones, itself stored as erasure‑coded shards on Relay Nodes.
• StorageCommitment: The on‑chain record of a volume’s existence, owner, manifest root, erasure parameters, size, billing state, and status.
• PlacementRecord: The off‑chain, mutable record binding a shard ID to its assigned Relay Nodes. Replicated across assigned nodes; versioned with compare‑and‑swap semantics.
• CapToken: A signed bearer token granting a specific runner or owner scoped (volume, path prefix, access mode, byte quota, time window) access to a volume. Two variants: runner CapTokens (issued by the chain at job dispatch) and owner CapTokens (client‑signed by a delegated key).
• DelegationCert: A wallet‑signed certificate that binds an Ed25519 CBFS key to a Cowboy account, scoped to storage use only, with a short (30‑day) expiry.
• Storage Manager (0x0A): The Cowboy system actor that maintains StorageCommitments, mints runner CapTokens, handles per‑epoch billing, and governs volume status transitions.
• Relay Registry (0x0B): The Cowboy system actor that maintains Relay Node profiles, staking, health, and repair/challenge coordination.

​

Key Features

• Off‑chain data plane. Reads and writes are Relay‑to‑client RPCs authenticated by CapTokens; no on‑chain transaction is submitted on the data path. The chain is touched only at volume lifecycle boundaries (create, delete, commit manifest root, register relay, settle billing, resolve challenge).
• Client‑side encryption and erasure coding. Private volumes encrypt every object with AES‑256‑GCM using a per‑volume Data Encryption Key (DEK), and split the ciphertext into K+M Reed‑Solomon shards before distribution. Relay Nodes hold only ciphertext shards; they never see plaintext.
• Delegation‑based owner authorization. A one‑time wallet signature produces a short‑lived DelegationCert binding an Ed25519 CBFS key to the account. The owner then constructs client‑signed CapTokens locally for each data operation, with no further chain writes and no centralized token minter on the data path.
• On‑chain anchoring with off‑chain serving. A volume’s manifest is served by Relay Nodes, but its 32‑byte Merkle root is committed on‑chain in the StorageCommitment. This yields READ_COMMITTED consistency across runners, supports Proof‑of‑Retrievability verification, and bounds the trust placed in any individual Relay Node.
• Autonomous repair. Relay Nodes continuously self‑heal (verify local shards against expected hashes, reconstruct from peers) and coordinate dead‑node replacement (leader‑elected per shard via lowest‑index rule, CAS‑versioned PlacementRecords) without Runner involvement or per‑event on‑chain traffic.

​

Differences vs. On‑chain State

​

Related Work

​

Content‑Addressed Systems (IPFS, Swarm)

• Working state churns constantly. Every minor write produces a new CID, forcing callers to rewrite pointers and invalidating caches.
• No natural access scoping. Privacy depends on not disclosing the CID; anyone who learns the CID can read forever.
• Retrieval is best‑effort. Without pinning or gateways, there is no guarantee anyone is still serving the content.

​

Storage Deal Markets (Filecoin, Sia)

• Duration mismatch. A runner job lasts minutes to hours. A Filecoin deal lasts months. An agent that wanted job‑scoped storage on Filecoin would either lock up capital for irrelevant timescales or constantly renegotiate.
• Per‑deal chain overhead. Every new volume, extension, or modification requires a deal transaction. Cowboy actors create volumes in a single call and let the Relay Registry assign nodes automatically.
• Retrieval is a separate market. Filecoin retrieval is asynchronous and can take minutes. Agent workloads need sub‑second access.

​

Permaweb (Arweave)

• Agent data should be deletable. GDPR, cost control, retractions, and simple obsolescence all require explicit eviction. Arweave offers no mechanism to stop paying.
• Overwrite cost is linear in writes. Every update is a fresh endowment. Caches and working state would pay permanent storage prices for data that lives for hours.
• No access control. Arweave assumes public data. Agent workloads routinely process private data — user prompts, API keys, personal datasets.

​

Client‑Encrypted S3 Alternatives (Storj)

• Central Satellite coordination. Every Storj operation is brokered by a Satellite — a trusted service that holds metadata and issues access grants. CBFS validates CapTokens directly at Relay Nodes against cached chain state, with no central broker on the data path.
• Erasure overhead. Storj’s 29/80 scheme costs ~2.76× in storage to achieve high durability. CBFS defaults to 4/6 (1.5×) tolerating any 2 simultaneous node failures per object, and owners can opt into higher schemes up to 16/24 when they need more durability.
• Standalone product. Storj is not integrated with a smart contract platform. Payments, identity, and access live in its own stack. CBFS/RAS is native to Cowboy: volumes bill from the same account that runs actors, CapTokens integrate with job dispatch, relays register via the same PoS stake primitives, and storage usage flows into the same economic accounting as compute.

​

Blob Storage with DA (Walrus, Celestia, EigenDA)

​

Tabular Summary

​

Why CBFS/RAS for Cowboy

1. Off‑chain data plane with zero per‑access chain overhead. Reads and writes travel as QUIC RPCs directly between runner and Relay Node, authenticated by a CapToken and verified locally against cached chain state. The only on‑chain event on the data path is the periodic manifest root commit, at a cadence chosen by the actor. IPFS has no economic layer; Filecoin and Sia tie every significant action to a deal or contract; Arweave pays upfront but reads via gateways. CBFS gives agents the latency of a private storage system and the durability of a staked public one.
2. Runner‑scoped capability tokens. Cowboy’s Storage Manager mints CapTokens with exact scope — volume, path prefix, mode, byte quota, TTL — bound to a specific runner address at job dispatch. Swarms of ephemeral agents can share a volume with non‑overlapping write prefixes and automatic expiry. No other decentralized storage system models per‑job access this way; all assume account‑level ownership and require the owner to be online to re‑sign every operation.
3. Mutable path‑addressed storage with deterministic consistency. Working state, artifact caches, and runner intermediate results are fundamentally mutable. CBFS’s manifest‑and‑root model provides mutability with READ_COMMITTED consistency via the on‑chain manifest_root, without forcing the re‑writing and indirection overhead of content‑addressed systems.
4. POSIX‑native FUSE mount as a first‑class surface. Modern LLM agents have filesystem tools — Read, Write, Bash, grep, jq, find — as first‑class primitives in their operating vocabulary. A FUSE mount at /mnt/workspace lets an agent use those tools directly against persistent storage with no bespoke storage SDK. CBFS is the only decentralized storage system in this comparison that treats a POSIX mount as primary rather than as an afterthought.

​

Volumes and the Account Model

• PRIVATE: Objects are AES‑256‑GCM encrypted under a per‑volume DEK. The manifest is encrypted. Reads and writes require CapTokens.
• PUBLIC: Objects are plaintext. The manifest is plaintext. Reads are open to any party without a CapToken; writes still require one.

​

The CBFS Data Plane

​

Object Representation

​

Placement

• PutPlacement(shard_id, record, expected_version) — CAS store/update.
• GetPlacement(shard_id) — fetch.
• ReplicatePlacement(shard_id, record) — peer‑to‑peer replication, signed by the sending relay’s identity key.

​

Erasure Coding

​

Transport

​

FUSE Mount

​

The CapToken Authorization Model

• Sessions on‑chain — putting every token in consensus state scales poorly and creates rogue‑node minting risk.
• Transaction per access — wrapping every read or write in a chain transaction destroys the latency and throughput guarantees a storage system needs.
• Centralized token broker — a single online signer is a single point of failure and a censorship vector.

​

Runner CapTokens (chain‑issued)

​

Owner CapTokens (client‑signed, v1)

payload = bincode(OwnerCapTokenSigningPayload {
    version: 1,
    volume_id, volume_name, owner_address, access_mode, path_prefix,
    valid_from_ms, valid_until_ms, mount, sync_interval, delegation_cert
})
owner_signature = Ed25519Sign(cbfs_private_key, payload)

​

DelegationCert

payload = bincode(DelegationCertSigningPayload {
    wallet_address, cbfs_public_key, scope, chain_id, network, aud, expires_at_ms
})
wallet_signature = Secp256k1Sign(wallet_key, keccak256(payload))

​

Access Modes

​

Scoping and Concurrency

1. Issuance time (strong). Storage Manager rejects new write tokens whose prefix overlaps an existing write token on the same volume.
2. Commit time (detection). A coordinator token holder reads the committed manifest after sub‑agent commits and verifies all paths fall within each delegated prefix. Rogue commits are detected and revoked, then corrected.
3. Write time (weak). Relay Nodes cannot verify prefix compliance: shard IDs are opaque hashes by design, so per‑shard path‑check is impossible. Relay Nodes verify signature, mode, and byte quota; they do not verify that a write targets a permitted path.

​

Volume Lifecycle

​

Create (control plane)

1. Derives volume_id = keccak256(owner_address || volume_name). If the volume already exists, revert with VolumeAlreadyExists.
2. Charges VOLUME_CREATION_FEE = 1,000 CBY from the owner’s account.
3. For PRIVATE volumes: the owner/client generates the 32‑byte DEK off-chain (cbfs-crypto::generate_dek) and submits it already-wrapped; the on-chain create_volume handler does not generate the DEK — it only receives and validates the supplied wrapped_dek (length + cbss_committee_epoch/cbss_release_key_material_hash against the CBSS committee; node/execution/src/ras.rs) and records it. (This corrects the earlier “the actor generates … HKDF-SHA256(account_secret, …)” language — see the §“Create” errata; wrapping is CBSS threshold-IBE, and generation is client-side, consistent with §10.1 “the DEK never leaves the owner”.)
4. Writes a StorageCommitment record with status ACTIVE, empty manifest_root, effective_size_bytes = 0.
5. Assigns initial Relay Nodes via the Relay Registry (§7).

​

Attach (control plane)

​

Write (data plane, no on‑chain traffic)

1. Generates a fresh 16‑byte write_id and computes shard_id = BLAKE3(volume_id || object_path || write_id).
2. Encrypts the object: ciphertext = AES-256-GCM(DEK, nonce = CSPRNG(12), plaintext, aad = canonical_metadata).
3. Reed‑Solomon encodes the ciphertext to K+M shards of ceil(ciphertext_size / K) bytes each, with padding.
4. Selects K+M Relay Nodes from the cached Relay Registry and issues PutShard(shard_id, shard_index, shard_bytes, metadata) RPCs in parallel, each authenticated by the CapToken.
5. Records the new ObjectDescriptor in the local manifest and constructs the PlacementRecord.

​

Read (data plane, no on‑chain traffic)

1. Fetch the manifest root from the authoritative Storage Manager state (or a cached, freshness‑bounded value).
2. Fetch the manifest blob from Relay Nodes via GetShard on BLAKE3(volume_id || "__manifest__"); for private volumes, decrypt with the DEK.
3. Resolve object_path to an ObjectDescriptor; fetch its PlacementRecord via GetPlacement.
4. Request any K of K+M shards in parallel (prefer low‑latency, fail over on timeout or corruption).
5. Reconstruct the ciphertext via Reed‑Solomon, truncate to ciphertext_size, verify ciphertext_hash.
6. Decrypt to plaintext, verify content_hash.

​

Commit (two‑level)

1. CBFS commit (commit()): Publish the (encrypted) manifest as a well‑known shard (BLAKE3(volume_id || "__manifest__")) and the PlacementRecord for each changed shard to its assigned Relay Nodes via PutPlacement. Best‑effort; failures are logged but do not block.
2. Cowboy commit (commit_manifest(manifest_root: bytes32)): Submit only the 32‑byte Merkle root to the on‑chain StorageCommitment.manifest_root. This is the sole on‑chain write on the data path, and it is triggered by the owner or runner at a natural boundary (job end, user command), not per object.

​

Delete, Grace, Restore

• All active CapTokens are revoked.
• New CapTokens cannot be issued.
• Storage rent continues to accrue.
• The owner MAY call undelete_volume to restore status to ACTIVE.

​

Relay Nodes

​

Registration and Stake

​

Repair (autonomous, two‑phase)

1. Fetch K healthy shards from peers listed in the PlacementRecord.
2. Reconstruct the ciphertext via Reed‑Solomon, truncating to ciphertext_size (this step is why ciphertext_size is duplicated into the PlacementRecord — without it, padding would corrupt the reconstruction).
3. Re‑encode, extract the expected shard, verify its hash, persist locally.

1. Leader election. Among the live assignees, the node with the lowest shard_index is the unique leader for this PlacementRecord. Ties cannot occur (indices are distinct).
2. The leader selects a replacement Relay Node from the peer list (excluding current assignees) using the NodeSelector.
3. The leader reconstructs the missing shard(s) from surviving shards and pushes them to the replacement via PutShard or the signed PeerRelayPushShard RPC.
4. The leader writes a new PlacementRecord with the updated assignment and version + 1 via a CAS PutPlacement(shard_id, record, expected_version). If CAS fails (because another node replaced a different dead node in the same record concurrently), the leader retries in the next cycle.
5. Updated PlacementRecord is replicated to all current assignees via ReplicatePlacement.

​

Drain and Peer‑Push

• PeerRelayPushShard(sender_node_id, timestamp, shard_id, shard_index, old_record_hash, shard_bytes_hash, shard_bytes, signature)
• PeerPlacementUpdate(sender_node_id, timestamp, old_record_hash, new_record_hash, new_record, signature)

​

Incentives

1. Per‑epoch storage rent. The total per‑epoch rent on a volume is distributed pro rata across the Relay Nodes holding its shards:
   total_storage_fee = ceil(effective_size / 2^20) × STORAGE_FEE_PER_MIB_PER_EPOCH
   per_relay_share   = total_storage_fee / num_relay_nodes_holding_shards
   
2. Metered transfer fees. Each serving relay receives ceil(bytes_served / 2^20) × TRANSFER_FEE_PER_MIB from the reader’s account at read time. The exact rates are governance‑tunable.

⚠️ Fee-split errata (2026-07). The deployed three-way split is 10% platform / 1% challenge pool / 89% relay (STORAGE_FEE_PLATFORM_BPS = 1_000, STORAGE_FEE_CHALLENGE_POOL_BPS = 100, STORAGE_FEE_RELAY_BPS = 8_900; node/ras/src/lib.rs), changed to 89/1 on 2026-07-10 (PR #231). The “90% to relays / 2% PoR” figures throughout this WP are stale; relays get 89% and the PoR challenge pool is 1%, not 2%. This is a governance-tunable economic value — CIP-31 §4 has since been reconciled to 10/1/89 to match this code; treat the deployed BPS as authoritative pending governance. The 10% share is credited to the Platform Fee Account (0x18, CIP-31 §4), not burned.

​

Proof of Retrievability

​

Challenge Protocol

• The requested byte range (ciphertext bytes) from the named shard.
• A two‑level Merkle proof chain:
  1. byte_range_proof — folds the challenged chunk of ciphertext up to the shard’s chunk_root (cbfs_erasure::chunk_root).
  2. shard_inclusion_proof — folds the shard leaf BLAKE3(0x00 ‖ shard_id ‖ shard_index ‖ chunk_root) up to the StorageCommitment’s shard_root accumulator (a keyed sparse Merkle root over all ciphertext shard commitments; cowboy_ras::shard_root).

1. Recomputes the chunk leaves from the submitted ciphertext bytes and verifies byte_range_proof resolves to the shard’s chunk_root.
2. Verifies shard_inclusion_proof resolves to challenge.snapshot_shard_root, snapshotted from StorageCommitment.shard_root at challenge issuance.

Note (2026-07): the PoR anchor is shard_root over ciphertext commitments, not manifest_root. Every input to verification — snapshot_shard_root, shard_id/shard_index, chunk_root, and the relay’s ciphertext bytes — is a public commitment. Because data is encrypt → erasure‑code → hash (relays hold only ciphertext shards, §“Object Representation”), shard commitments are DEK‑independent: PoR for a PRIVATE volume is fully verifiable on‑chain without the DEK. Earlier drafts of this section described a shard_hash → manifest_root chain; that never matched the shipped code, which anchors on the distinct shard_root field.

​

Outcomes

​

Billing, Rent, and Eviction

​

Fee Components

​

Settlement

1. Compute epoch_fee = ceil(effective_size_bytes / 2^20) × STORAGE_FEE_PER_MIB_PER_EPOCH.
2. If balance_reserved ≥ epoch_fee: charge, split 10% burn / 90% to relays (pro rata by shards held), increment last_billed_epoch.
3. If balance_reserved < epoch_fee: mark the volume as entering GRACE_PERIOD, with reads permitted and writes denied.
4. If the volume has been in GRACE_PERIOD for STORAGE_GRACE_EPOCHS = 1 epoch (cowboy-protocol-types; 1 epoch = STORAGE_EPOCH_BLOCKS = 86,400 blocks ≈ 24 h at 1 s/block — the unit is epochs, not blocks, and the value is 1, not 7,200) without top‑up, transition to GARBAGE_COLLECTING and broadcast a purge signal to holding relays.

​

Comparison with State Rent

​

On‑Chain Control Plane

​

Storage Manager (0x0A)

volume_id:            bytes32     // keccak256(owner || volume_name)
owner:                Address
volume_name:          String      // UTF-8, ≤64 bytes
visibility:           PRIVATE | PUBLIC
created_at:           u64         // block height
wrapped_dek:          bytes       // envelope-encrypted volume DEK (empty for PUBLIC)
wrapping_key_hash:    bytes32
manifest_root:        bytes32     // Merkle root of manifest
raw_size_bytes:       u64
effective_size_bytes: u64         // raw × (K+M)/K
erasure_k:            u8
erasure_m:            u8
last_updated:         u64         // block height of last manifest commit
degraded_shards:      u16
status:               ACTIVE | GRACE_PERIOD | DELETED | GARBAGE_COLLECTING

​

Relay Registry (0x0B)

​

Cache Freshness

​

State Transition Function (Storage Subset)

1. Process RAS system instructions. Volume creation, deletion, undelete, commit manifest, revoke delegation, and settlement apply to Storage Manager state; relay registration, heartbeat, drain, and settlement apply to Relay Registry state.
2. Issue runner CapTokens. For each runner job assigned in the block with volume attachments, derive and persist the runner CapToken and sealed DEK copies.
3. Resolve PoR challenges. For challenges whose response window closes this block, verify submitted proofs and apply slashing and reputation updates.
4. Epoch boundary (every rent_epoch_length). Iterate all active volumes, compute storage rent, transition statuses (ACTIVE → GRACE_PERIOD → GARBAGE_COLLECTING, with restoration to ACTIVE permitted from GRACE_PERIOD), and distribute relay earnings.

​

Normative Conventions

​

1. Volume Model

​

1.1. Volume identity

​

1.2. Visibility

​

1.3. Object identity and descriptor

​

1.4. Quotas

​

2. CapTokens

​

2.1. Encoding

​

2.2. Runner CapTokens

​

2.3. Owner CapTokens (v1)

​

2.4. TTL caps

​

2.5. Delegation certs

​

2.6. Prefix overlap (writes)

​

3. Encryption

​

3.1. DEK generation

⚠️ Superseded (see the §“Create” errata, 2026-07): the shipped mechanism envelope‑encrypts the DEK under CBSS (CIP-24) threshold-IBE committee key material (cbss_committee_epoch + cbss_release_key_material_hash, validated against cbss::AccountReleaseKey at CBSS_SYSTEM_ACTOR (0x04)) — not local HKDF-SHA256(account_secret, …). account_secret is undefined and does not exist in the deployed create_volume path (node/execution/src/ras.rs); a private volume is rejected unless it supplies wrapped_dek and a non-zero cbss_committee_epoch. The sentence below is retained only for historical context pending a normative rewrite.

​

3.2. Object encryption

​

3.3. Content and ciphertext hashes

​

3.4. Nonce uniqueness

​

4. Manifest and Placement

​

4.1. Manifest

​

4.2. Manifest shard address

​

4.3. PlacementRecord

​

4.4. PlacementRecord fields

​

5. Erasure Coding

​

5.1. Scheme

​

5.2. Parameters

​

5.3. Padding

​

5.4. Integrity

​

6. Relay Nodes

​

6.1. Registration

​

6.2. Heartbeat and health

​

6.3. Unstaking

​

6.4. Peer‑relay RPC auth

​

6.5. Repair cadence

​

7. Proof of Retrievability

​

7.1. Challenge cadence

​

7.2. Response window

​

7.3. Proof structure

​

7.4. Slashing

​

8. Billing and Fees

​

8.1. Fee components

​

8.2. Fee split

​

8.3. Grace period

​

8.4. Soft‑delete grace

​

8.5. No third meter

​

9. RPC and Wire Format

​

9.1. Transport

​

9.2. Frame format

​

9.3. RPC operations

​

9.4. Relay handshake

​

10. Security Considerations

​

10.1. Client‑side encryption

​

10.2. Shard ID opacity

​

10.3. Erasure threshold

​

10.4. Rogue runner with write token

​

10.5. Relay eclipse and withholding

​

10.6. Sybil resistance

​

10.7. Key compromise

​

10.8. Dispatcher trust (v1)

​

10.9. Replay and nonce reuse

​

11. Parameters (Genesis Defaults)

⚠️ Block-time & erasure errata (2026-07). (1) This chain runs at 1 s/block (BLOCKS_PER_YEAR = 31,536,000). Every “≈24 h” annotation on a 7,200 blocks window here (RELAY_UNSTAKE_DELAY, ORPHAN_SHARD_TTL, VOLUME_DELETE_GRACE_EPOCHS) is a leftover 12 s/block figure: 7,200 blocks = ~2 h at 1 s/block; a genuine 24 h window is STORAGE_EPOCH_BLOCKS = 86,400. (2) The erasure bounds 2 ≤ K ≤ 16, 1 ≤ M ≤ 8 are enforced only by the cbfs client library (cbfs/types/src/lib.rs); the on-chain consensus gate (validate_create_volume, node/execution/src/ras.rs) is weaker — it rejects only k == 0 || m == 0 || k + m > MAX_SHARDS_PER_VOLUME (32), so on-chain a volume may be created with K = 1 or K + M up to 32.

​

12. Relationship to the Technical Whitepaper

• Actors, messages, timers, reentrancy, and per‑handler resource limits follow Technical Whitepaper §3.
• Fee markets (Cycles, Cells) and per‑block basefee adjustment follow Technical Whitepaper §4 and §17. Storage operations that touch the chain consume these meters normally.
• State rent for on‑chain actor storage follows Technical Whitepaper §4.4 and §17.5; storage rent in this document applies only to off‑chain CBFS volumes.
• The Runner Marketplace (Technical Whitepaper §5) is the consumer of runner CapTokens. Job dispatch integrates with the Storage Manager via issue_attachments() at job submission time.
• Consensus, randomness, and networking follow Technical Whitepaper §6. Relay Nodes rely on the same QC‑derived randomness beacon for PoR challenge targeting.
• Governance, upgrades, and system actor hot‑code replacement follow Technical Whitepaper §11.
• Entitlements (Technical Whitepaper §15) MAY gate storage capabilities at the runner and actor manifest level (for example, whether an actor is permitted to open a volume at all); RAS CapTokens are the second, fine‑grained layer within an entitlement’s allowed scope.