Skip to main content

CIP-10 v2

Versioning. This is v2 of CIP-10. v1 is preserved verbatim as Part I. v2 combines that base specification, the code-alignment revision, and the persistent-workload class. Conflict rule: Part II is canonical wherever it contradicts Part I. The Persistent Workloads section is canonical for persistent workloads and supersedes the Part I §16.2 deferral of long-running services. The key words MUST, MUST NOT, REQUIRED, SHOULD, SHOULD NOT, and MAY are to be interpreted as described in RFC 2119 and RFC 8174. Summary of v2 changes
  • Container Registry actor allocated at 0x13 (0x11 is the CIP-11 VALIDATOR_SET genesis snapshot and 0x12 is the CIP-18 PaymentGate reservation, so the registry takes the next free slot).
  • Fee distribution routes through a SettlementConfig at GOVERNANCE_SYSTEM_ACTOR=0x09 via a dedicated UpdateContainerSettlementConfig opcode (164) — matching the existing per-pool UpdateRegistrySettlementConfig pattern rather than a single opcode with a target_pool discriminant (the live codec never adopted the discriminant form).
  • Three-flow off-chain billing model clarified: runner job (CIP-2), storage rent (CIP-9), container compute (CIP-10) are independent escrows that share only the SettlementConfig shape, each with its own update opcode.
  • BillingAttestation.tee_signature will upgrade to CompositeAttestation per CIP-23 v2 §3.12; the shipped Phase-B type carries Option<String> pending that TEE wiring.
  • System instruction opcodes 160–164 allocated for the Container Registry per the live codec (CIP-13 v2 §1 authoritative wire table; the next free block after system-DKG took 158–159). Image/class registration is system_deployers-gated (operational, like the CIP-23 TEE trusted-key registry), while UpdateContainerSettlementConfig stays governance/0x09-gated (economic, like UpdateRegistrySettlementConfig).
  • Billing meters real cgroup accounting at millisecond granularity with a 100 ms floor; the enforced-cgroup Standard sandbox runs rootless runc via systemd-cgroup delegation (§6–7 below).

Part I — v1 Specification (verbatim from cip-10-runner-containers.md)

Status: Draft
Type: Standards Track
Category: Core
Created: 2026-03-07
Requires: CIP-2, CIP-9

1. Abstract

This proposal defines the Runner Container Runtime — a standardized, OCI-compatible container execution environment for off-chain Runner jobs. It specifies how container images are addressed, fetched, and cached; how containers are provisioned with CIP-9 storage mounts, resource limits, network policies, and GPU access; and how the container lifecycle integrates with CIP-2’s task dispatch and result submission flow. Key properties:
  • OCI-compatible: Containers use standard OCI images. Any image that runs on Docker or Podman runs on a Cowboy Runner.
  • Storage-integrated: CIP-9 volumes are mounted as FUSE filesystems inside the container at deterministic paths. The model or script sees a standard filesystem.
  • Resource-bounded: Every container declares CPU, memory, disk, and optional GPU limits. Runners enforce these limits and refuse tasks that exceed their capacity.
  • Network-isolated: Containers run with no ingress and allowlisted egress by default. The task definition specifies which external endpoints are reachable.
  • Ephemeral: Containers are destroyed after job completion. All persistent state lives in CIP-9 volumes; the container’s own filesystem is scratch space.
  • Model-agnostic: The runtime supports LLM tool-calling workloads (Llama, Kimi-K2, Claude, GPT, etc.), Python scripts, compiled binaries, and arbitrary OCI entrypoints with the same container primitive.

2. Motivation

CIP-2 defines how Actors dispatch tasks to Runners and receive results. CIP-9 defines how Runners access durable, encrypted storage volumes. However, neither CIP specifies the execution environment in which Runner code actually runs. This leaves critical questions unanswered:
  1. What environment does the code run in? A bare Python process? A Docker container? A VM? Without a standard, Runners cannot guarantee reproducible execution.
  2. How are dependencies managed? An ML inference job needs specific library versions, model weights, and system packages. The Runner node’s host environment should not bleed into the job.
  3. How is the job sandboxed? A malicious task definition could attempt to access the Runner’s host filesystem, exfiltrate secrets, or abuse network access.
  4. How are hardware accelerators exposed? GPU inference requires device passthrough with controlled access.
  5. What does an LLM see? When Claude or Kimi-K2 runs as a Cowboy Runner doing tool calling, it needs a shell environment with its standard tools (Read, Write, Bash, etc.) operating against mounted volumes. The container must provide this environment transparently.
Existing container orchestration systems (Kubernetes, Nomad, Fly.io) solve pieces of this, but none integrate with onchain task dispatch, VRF-based runner selection, or decentralized billing. CIP-10 defines the minimal container runtime spec needed for Cowboy Runners, built on OCI standards so that existing tooling and images work out of the box.

3. Definitions

  • Container: An isolated, ephemeral execution environment created from an OCI image. A container has its own filesystem root, process namespace, network namespace, and resource limits. It is destroyed after the job completes.
  • OCI Image: A container image conforming to the Open Container Initiative Image Specification. Consists of an ordered set of filesystem layers, a configuration (entrypoint, env vars, working directory), and a manifest.
  • Image Manifest: The OCI descriptor that identifies an image by its content-addressed digest (sha256:...). Pinning a digest ensures reproducible builds.
  • Runtime Config: A structured definition within a CIP-2 task that specifies the container image, resource limits, network policy, GPU requirements, volume mounts, and environment variables for the job.
  • Scratch Filesystem: The container’s own writable filesystem layer (overlayfs). This is ephemeral and destroyed on container teardown. It is NOT backed by CIP-9 storage.
  • Base Image: A pre-built, protocol-maintained OCI image optimized for common Runner workloads (LLM tool-calling, Python data science, etc.). Base images may be cached on Runner nodes for fast startup.

4. Design Overview

4.1 Architecture

A Runner node is a machine (physical or virtual) that runs an OCI-compatible container runtime. When a Runner is selected for a CIP-2 task, it:
  1. Pulls the container image (if not cached).
  2. Creates a container with the specified resource limits and network policy.
  3. Mounts CIP-9 volumes as FUSE filesystems inside the container.
  4. Starts the container entrypoint.
  5. Monitors execution until completion, timeout, or crash.
  6. Commits storage manifests and submits results onchain.
  7. Destroys the container.

4.2 Relationship to Existing CIPs

  • CIP-2 (Off-Chain Compute): CIP-10 extends the OffchainTask struct with a runtime_config field that specifies the container environment. The existing VRF selection, result submission, and deferred callback mechanisms are unchanged.
  • CIP-9 (Runner Attached Storage): CIP-9 volumes specified in volume_attachments are mounted inside the container as FUSE filesystems at /mnt/volumes/{name}/. The FUSE daemon and sync daemon (CIP-9 §12.1) run as sidecar processes alongside the container.
  • CIP-3 (Fee Model): Container resource usage (CPU-seconds, memory-seconds, GPU-seconds) uses attestation-based billing — metered externally by Runner cgroup counters, settled onchain via BillingAttestation (§12.3). This is distinct from CIP-3 Cycles/Cells, which are metered directly by the VM during transaction execution.

5. Container Images

5.1 Image Format

All images MUST conform to the OCI Image Specification v1.1+. This ensures compatibility with Docker, Podman, containerd, and other standard tools. Supported image media types:
  • application/vnd.oci.image.manifest.v1+json
  • application/vnd.docker.distribution.manifest.v2+json (Docker v2, backward-compatible)
Multi-architecture manifests (application/vnd.oci.image.index.v1+json) are supported. The Runner selects the appropriate platform variant (linux/amd64 or linux/arm64) based on its host architecture.

5.2 Image Addressing

Images are addressed by digest for reproducibility:
Digest pinning is mandatory in task definitions. Tags are informational — the runtime always pulls by digest. This prevents supply chain attacks where a tag is re-pointed to a different image after a task is submitted.

5.3 Image Registries

Runner nodes pull images from OCI-compliant registries. Three registry tiers are supported:
  1. Protocol registry (registry.cowboylabs.org): Maintained by the Cowboy protocol. Hosts base images and community-vetted images. Images are replicated across multiple mirrors for availability. No authentication required for pulls.
  2. Public registries (ghcr.io, docker.io, etc.): Standard public registries. The task definition specifies the full image reference. Runner nodes must have network access to the registry.
  3. Private registries: Authenticated registries where the account owner provides pull credentials in the task definition (encrypted with the Runner’s TEE attestation key). Credentials are scoped to the job duration and never persisted by the Runner.

5.4 Image Caching

Runner nodes maintain a local image cache (LRU, configurable max size). Cache behavior:
  • Cache hit: Image layers already present locally. Container creation starts immediately.
  • Cache miss: Layers are pulled from the registry. Pull time depends on image size and network.
  • Base images: Protocol base images (§5.5) are pre-pulled and pinned in the cache. They are never evicted.
Expected pull times: Task submitters SHOULD prefer thin images that layer on top of cached base images to minimize startup latency.

5.5 Base Images

The protocol maintains a set of base images optimized for common workloads: The runner-agent image is the recommended base for LLM tool-calling workloads. It provides the standard Unix tools that models expect when using filesystem-based tool sets (Read, Write, Bash, Glob, Grep). Base image digests are published onchain in the Container Registry actor (§11.2), allowing task submitters to reference them by well-known name and have the digest resolved deterministically.

6. Runtime Environment

6.1 Container Filesystem Layout

Every container starts with the following filesystem structure, regardless of image:
Key directories:

6.2 Environment Variables

The runtime injects the following environment variables into every container:
Additional environment variables from the task definition’s runtime_config.env are merged in (task-defined vars take precedence for non-COWBOY_ prefixed keys). The COWBOY_ prefix is reserved and cannot be overridden.

6.3 Entrypoint and Command

The container entrypoint is resolved in this order:
  1. runtime_config.command (if specified in the task definition) — overrides the image’s ENTRYPOINT and CMD.
  2. Image ENTRYPOINT + CMD — the default from the OCI image config.
For LLM tool-calling workloads, the entrypoint is typically a model harness process that:
  1. Connects to the model API (Claude, Kimi-K2, etc.) or runs a local model.
  2. Provides the model with a system prompt and tools (Read, Write, Bash, etc.).
  3. Executes tool calls against the container’s filesystem (including FUSE-mounted volumes).
  4. Returns the final result to the Runner engine for onchain submission.
The protocol does NOT prescribe how the model harness works. This is the domain of Runner operator software. CIP-10 only specifies the container environment in which it runs.

6.4 User and Permissions

Containers run as a non-root user (uid=1000, gid=1000) by default. This can be overridden in the image but is constrained:
  • Root (uid=0) is prohibited unless runtime_config.allow_root = true and the Runner supports it. Runners MAY reject tasks requesting root.
  • FUSE mount points (/mnt/volumes/*) are owned by the container user.
  • The container process has no access to the host filesystem, network namespace, or other containers.

7. Resource Limits

7.1 Resource Declaration

Every task MUST declare resource limits in its runtime_config. Runners use these limits to determine if they can accept the task and to enforce isolation during execution.

7.2 Enforcement

CIP-9 volume storage limits are enforced by CapToken max_bytes (CIP-9 §7.1), not by container-level disk quotas.

7.3 Resource Classes

To simplify task definition, the protocol defines standard resource classes. Task submitters can specify a class name instead of individual limits: Resource classes are defined onchain in the Container Registry actor and are governance-tunable. Custom limits override class defaults.

7.4 Runner Capability Advertising

Runners advertise their available resources in the Runner Registry (CIP-2). The RunnerProfile struct is extended:
When evaluating whether to accept a task, a Runner checks that the requested resources fit within its available capacity (total minus currently allocated to running containers).

8. GPU Passthrough

8.1 GPU Request

Tasks requiring GPU access specify a GpuRequest:

8.2 Device Exposure

GPU devices are exposed to the container via the OCI runtime’s device mapping:
  • NVIDIA GPUs: Exposed via nvidia-container-runtime (CDI). The container sees /dev/nvidia* devices and CUDA libraries.
  • AMD GPUs: Exposed via ROCm device mapping. The container sees /dev/kfd and /dev/dri/render*.
Only the requested number of GPUs are visible to the container. The Runner engine manages GPU allocation across concurrent containers.

8.3 GPU Capability in Runner Registry

Runners with GPUs advertise them:

8.4 Capability-Aware Runner Prefiltering

The naive approach — VRF selects from all active Runners, then incapable Runners call skip_task() — creates a latency griefing problem for resource-constrained tasks. If only 5% of Runners have GPUs, a GPU task could bounce through 20+ skip rounds before landing on a capable Runner, each round adding ~12 seconds of onchain latency. CIP-10 introduces capability prefiltering as an extension to the CIP-2 VRF selection:
  1. The Dispatcher maintains capability indices — filtered sublists of the active runner list grouped by advertised capabilities (GPU vendor/model, platform architecture, memory tier, cached base images).
  2. When a task specifies resource requirements (e.g., gpu.count > 0, memory_mib > 16384), the VRF selection runs against the filtered sublist of capable Runners, not the full active list.
  3. The start_index calculation from CIP-2 §6 is applied to the filtered list: start_index = hash(vrf_seed + (submission_block - vrf_generation_block)) (mod filtered_list_size).
  4. If the filtered list is empty (no capable Runners registered), the task fails immediately at submission with NO_CAPABLE_RUNNERS.
Verification: The capability index is deterministic — it is derived from onchain RunnerProfile data. Any party can reconstruct the filtered list and verify the VRF selection. skip_task() remains as a fallback for edge cases (e.g., a Runner’s advertised capacity is currently fully allocated to other containers). For tasks with no special requirements (no GPU, standard resource class), the VRF selection operates on the full active list as in CIP-2, with no behavioral change.

9. Network Policy

9.1 Default: Isolated

By default, containers have no network access. This is the safest posture and sufficient for pure computation tasks that read from CIP-9 volumes and write results.

9.2 Egress Allowlist

Tasks that need external network access (API calls, web scraping, model API endpoints) declare an egress allowlist:
Rules:
  • No wildcards: Each allowed host must be explicitly listed. *.example.com is not valid.
  • DNS resolution and IP pinning: DNS resolution is performed by a host-side DNS proxy (not inside the container) that enforces the allowlist. The proxy resolves each allowlisted hostname at container startup, pins the resolved IP(s), and configures iptables rules to permit traffic only to those pinned IPs on the specified ports. This prevents DNS rebinding attacks (where an attacker changes a DNS record mid-session to redirect traffic to an internal IP). The container’s /etc/resolv.conf points to the host proxy, which rejects queries for non-allowlisted domains.
    • TLS SNI verification: For TLS connections (port 443), the Runner’s network filter verifies that the TLS ClientHello SNI matches the allowlisted hostname. This prevents an attacker from using an allowlisted IP to tunnel traffic to a different hostname.
    • DNS TTL refresh: Pinned IPs are refreshed at DNS TTL expiry (minimum 60s, maximum 300s) to handle legitimate IP rotations (CDNs, load balancers). New IPs are verified against the allowlist hostname before being permitted.
  • No ingress: Containers cannot listen on ports or accept incoming connections. There are no inbound requests to the container.
  • No inter-container networking: Containers from different tasks cannot communicate directly, even if they run on the same Runner node. Communication between tasks happens through CIP-9 shared volumes.

9.3 Model API Access

For LLM tool-calling workloads, the model API endpoint must be in the egress allowlist. The Runner operator’s model harness handles authentication with the model provider.
For Runners that host models locally (on-device inference), no egress is needed — the model runs inside the container.

10. Container Lifecycle

10.1 Full Lifecycle

10.2 Phase Details

Phase 1: Setup (~1-30s depending on image cache state)
  1. Image pull: If the image is not cached, pull layers from the registry. If pull fails (registry unavailable, digest mismatch), the Runner calls skip_task().
  2. Container creation: Create the container with resource limits (cgroups v2), namespace isolation (PID, mount, network, user), and filesystem layers (overlayfs for scratch).
  3. Volume mounts: For each VolumeAttachment in the task definition:
    • Obtain CapToken from the Dispatcher.
    • Obtain volume encryption key (TEE-sealed or threshold-shared, per CIP-9 §9.2).
    • Start the FUSE daemon mounting the volume at /mnt/volumes/{name}/.
    • Start the sync daemon for background push/pull.
    • Fetch the current manifest from Relay Nodes.
  4. Environment injection: Set COWBOY_* env vars and task-defined env vars.
  5. Start entrypoint: Execute the container’s entrypoint process.
Phase 2: Execution (bounded by max_duration_sec)
  • The entrypoint process runs. For LLM workloads, this is the model harness executing tool calls against the filesystem.
  • FUSE-mounted volumes handle reads/writes transparently (CIP-9 §12.1).
  • The sync daemon pushes and pulls in the background at the configured interval.
  • The Runner engine monitors resource usage and enforces limits.
Phase 3: Teardown (~5-30s)
  1. Entrypoint exit: The entrypoint exits with code 0 (success) or non-zero (failure).
  2. Final sync: The sync daemon performs a final push of all dirty files to Relay Nodes. This blocks until complete or until a teardown timeout (TEARDOWN_TIMEOUT_SEC, default 30s) is reached.
  3. Manifest commit: The Runner commits storage manifests onchain for each attached volume.
  4. Result submission: The Runner calls submit_result() on the CIP-2 Runner Submission Contract with the job output.
  5. Container destruction: The container, its scratch filesystem, and all in-memory state are destroyed. Volume data persists on Relay Nodes.

10.3 Failure Modes

10.4 Exit Codes

The Runner engine maps container exit codes to CIP-2 task result statuses:

11. onchain State

11.1 CIP-2 Task Definition Extension

The OffchainTask struct from CIP-2 is extended with a runtime_config field:
Where:

11.2 Container Registry Actor

A new system actor at 0x0...cowboy.containers maintains: BaseImageEntry (per base image):
ResourceClassEntry (per resource class):
Base image digests and resource classes are updated via governance proposals.

11.3 Key Space

Container Registry entries use the CIP-4 STORAGE key space:

12. Billing and Fees

12.1 Compute Resource Billing

Container resource usage is billed alongside the CIP-2 payment_per_runner. The task submitter locks funds at submit_task() time covering the maximum possible resource cost:
At job completion, the actual usage is metered and the difference is refunded:

12.2 Image Pull Fees

Image pulls from the protocol registry are free. Pulls from external registries incur bandwidth costs:
This is waived for cached images (size = 0 bytes transferred).

12.3 Billing Attestations and Dispute Window

Off-chain billing requires trust that the Runner honestly reports resource usage. Without verification, a malicious Runner could over-report to extract more payment or under-report to undercharge. RAS mitigates this with billing attestations and a dispute window. Billing attestation: When submitting results, the Runner includes a BillingAttestation:
For TEE Runners: The attestation is signed by the enclave. The TEE monitors cgroup counters and produces a tamper-proof usage report. This is the strongest guarantee — the Runner cannot inflate measurements without breaking the TEE. For non-TEE Runners: The attestation is self-reported and signed by the Runner. This is weaker but is constrained by the dispute window. Dispute window: After result submission, there is a BILLING_DISPUTE_WINDOW (e.g., 300 blocks, ~1 hour) during which the task submitter can challenge the billing attestation. Dispute resolution: The dispute mechanism is intentionally simple for v1 — it does not try to adjudicate the “true” resource usage (which is unknowable onchain for non-TEE Runners). Instead, it uses economic incentives: Runners who over-report get disputed and lose reputation; the task submitter’s worst case is the pre-locked max cost (which they already accepted at submission). This is similar to optimistic rollup dispute games — honest behavior is the dominant strategy because disputes are expensive for both parties.

12.4 Relationship to CIP-3

Like CIP-9 storage fees, container compute fees use attestation-based billing — the metering happens externally (cgroup counters on the Runner), but the settlement is entirely onchain (fund locking, attestation submission, dispute resolution, refunds). This is distinct from CIP-3 Cycles and Cells, which are metered directly by the VM during transaction execution. The chain cannot run the container itself, so it relies on the Runner’s BillingAttestation (verified by TEE signature or bounded by the dispute window) to determine actual usage. onchain operations (task submission, result submission, manifest commits) consume Cycles and Cells per CIP-3 as normal.

13. Parameters


14. Security Considerations

14.1 Container Escape

A container escape (breaking out of namespaces/cgroups into the host) is the most critical threat. Mitigations:
  • Namespace isolation: PID, mount, network, user, UTS, IPC namespaces are all isolated.
  • Seccomp profile: A restrictive seccomp profile blocks dangerous syscalls (mount, reboot, kexec_load, etc.).
  • Capabilities dropped: All Linux capabilities are dropped except a minimal set (CAP_NET_BIND_SERVICE for egress, CAP_FOWNER for FUSE mounts).
  • Read-only root: The image filesystem is mounted read-only. Only /tmp, /workspace, and CIP-9 mounts are writable.
  • No privileged mode: --privileged is never allowed. Even allow_root=true does not grant host capabilities.
  • gVisor/Kata (optional): Runner operators MAY use gVisor (application kernel) or Kata Containers (lightweight VM) for additional isolation. This is an operator choice, not a protocol requirement.

14.2 Image Supply Chain

  • Digest pinning: Images are always pulled by digest, preventing tag-based supply chain attacks.
  • Base image governance: Protocol base images are updated only via governance proposals. Digests are recorded onchain.
  • No implicit pulls: The Runner never pulls an image not explicitly specified in the task definition.
  • Layer verification: Each layer’s digest is verified on pull per the OCI distribution spec.

14.3 Network Exfiltration

  • Default deny: No network access unless explicitly allowlisted.
  • No wildcards: Allowlist entries must be specific hostnames.
  • No DNS exfiltration: DNS resolution happens inside the container but is restricted to resolving allowlisted hosts (Runners SHOULD use a DNS proxy that blocks queries for non-allowlisted domains).
  • Bandwidth limits: MAX_EGRESS_BANDWIDTH prevents a container from saturating the Runner’s network.

14.4 Resource Exhaustion

  • Mandatory limits: Tasks without resource limits are rejected at the Dispatcher.
  • cgroups enforcement: CPU throttling and memory OOM-kill prevent runaway containers.
  • Disk quotas: Scratch disk is bounded by overlayfs/tmpfs limits.
  • CIP-9 quotas: Volume write quotas are enforced by CapToken max_bytes.

14.5 Secret Leakage

  • No host env inheritance: Container environment is clean — only COWBOY_* vars and task-defined vars.
  • No host filesystem: The container has no access to the Runner’s host filesystem, Docker socket, or metadata services.
  • TEE attestation: For sensitive workloads, Runners must attest via TEE (CIP-2 tee_required=true). The volume key (CIP-9) and any task secrets are sealed to the enclave.
  • Scratch destruction: Container scratch filesystem is destroyed immediately after teardown.

14.6 GPU Side Channels

  • MIG isolation (NVIDIA): For multi-tenant GPU sharing, Runners SHOULD use Multi-Instance GPU (MIG) to provide hardware-level isolation between containers.
  • Memory clearing: GPU memory is cleared between container executions to prevent cross-job data leakage.
  • Single-tenant default: In v1, a GPU device is assigned to at most one container at a time (no sharing).

15. Implementation Notes

15.2 Rootless Operation

Runner operators are RECOMMENDED to run the container runtime in rootless mode (user namespaces, rootless containerd). This provides defense-in-depth — even if a container escape occurs, the attacker has only unprivileged host access.

15.3 Container Creation Time Budget

Target: container ready to execute within 5 seconds of task acceptance (assuming cached image):

15.4 Logging

Container stdout/stderr is captured by the Runner engine. Log handling:
  • Logs are buffered in memory (max LOG_BUFFER_SIZE, default 10 MiB).
  • Logs are available to the task submitter as part of the result payload (if the result_schema requests them).
  • Logs are NOT persisted by the Runner after container destruction.
  • If the task attaches a CIP-9 volume, the entrypoint can write logs to the volume for durable storage.

16. Scope and Future Work

16.1 v1 Scope (This CIP)

  • OCI-compatible container images with digest pinning.
  • Container filesystem isolation with read-only root and writable scratch.
  • CIP-9 FUSE volume mounts inside containers.
  • Mandatory resource limits (CPU, memory, scratch disk, wall-clock time).
  • GPU passthrough (NVIDIA CUDA, AMD ROCm).
  • Network isolation with explicit egress allowlists.
  • Standard base images for common workloads (agent, Python, ML).
  • Container lifecycle integrated with CIP-2 task dispatch.
  • Compute resource billing (CPU-sec, memory-sec, GPU-sec).
  • onchain Container Registry for base image digests and resource classes.

16.2 Explicitly Out of Scope

  • Long-running services: v1 containers are ephemeral (bounded by max_duration_sec). Persistent services (web servers, databases) that run indefinitely are a future CIP. CIP-5 timers can be used to re-dispatch periodic jobs.
  • Container-to-container networking: Direct communication between containers from different tasks. In v1, coordination happens via CIP-9 shared volumes.
  • Custom container runtimes: v1 requires an OCI-compatible runtime. Support for WASM, Firecracker micro-VMs, or other execution models is a future extension.
  • Image building onchain: v1 images are built externally and pushed to registries. A decentralized image build service is a future CIP.
  • Trusted builder attestation: Verifying that an image was built from a specific source repository (e.g., via Sigstore). This is a future supply chain security enhancement.
  • Spot/preemptible execution: v1 has no concept of interruptible, lower-cost execution tiers. This is a future pricing extension.
  • Multi-container pods: v1 runs a single container per task. Sidecar patterns (e.g., running a local model alongside a tool-calling harness) require the entrypoint to manage sub-processes internally or be combined into a single image.

Appendix A: Examples

A.1 LLM Tool-Calling Agent (Claude)

An autonomous research agent runs as Claude with tool calling, reading/writing to a persistent memory volume. Task submission:
What happens on the Runner node:
Claude’s tool calls inside the container:
Teardown:

A.2 GPU ML Inference

A model inference job runs PyTorch on a GPU, reading input from one volume and writing predictions to another. Task submission:
Inside the container:
No network needed. GPU exposed via CUDA. Volumes look like normal directories.

A.3 Agent Swarm with Custom Image

A custom image bundles specialized tools for a financial analysis swarm. The coordinator uses the runner-agent base image; sub-agents use a custom image with financial data libraries. Sub-agent task:
Kimi-K2’s tool calls:
The custom image brings domain-specific tools (fin-scrape, fin-analyze) that the model uses via Bash. The FUSE mount handles persistence. The model doesn’t know about containers, shards, or Relay Nodes.

Appendix B: Container Security Profile

The default seccomp profile for CIP-10 containers. Runners MUST apply at least this restrictive a profile: Allowed syscall categories:
  • Process management: clone, fork, execve, exit, wait4, kill, getpid, getppid
  • File I/O: open, read, write, close, stat, fstat, lstat, readdir, mkdir, unlink, rename
  • Memory: mmap, munmap, mprotect, brk, madvise
  • Network (if allowlisted): socket, connect, sendto, recvfrom, bind (loopback only)
  • Time: clock_gettime, nanosleep, gettimeofday
  • Misc: ioctl (limited), fcntl, pipe, poll, select, epoll_*, futex
Blocked syscall categories:
  • Mount operations: mount, umount2, pivot_root (FUSE mounts are set up by the host before container start)
  • Module loading: init_module, finit_module, delete_module
  • System: reboot, sethostname, setdomainname, syslog
  • Dangerous: ptrace, process_vm_readv, process_vm_writev, kexec_load
  • Raw I/O: iopl, ioperm
The FUSE filesystem is mounted by the Runner engine (host-side) before the container starts. The container process interacts with it through normal file I/O syscalls — no mount privileges required inside the container.

Part II — v2 Revision (canonical; concrete addresses + fee plumbing)

0. What this revision does

CIP-10 v1 §11.2 placeholder-addresses the Container Registry actor as 0x0...cowboy.containers. CIP-10 v1 §12 specifies billing fee constants but does not say where collected fees route or how they integrate with CIP-3’s SettlementConfig pattern. CIP-23 (TEE Execution) §3.12 amends CIP-10 §12.3 BillingAttestation to use CompositeAttestation. v2 pins all three.

1. Container Registry actor address: 0x13

The Container Registry actor is allocated at 0x13 (CONTAINER_REGISTRY, node/runner/src/system_actors.rs; code-deployed, Council-pausable). 0x11 is the CIP-11 VALIDATOR_SET genesis snapshot and 0x12 is the CIP-18 PaymentGate reservation, so the registry sits at the next free slot. All v1 §11.2 references to 0x0...cowboy.containers resolve to 0x13. WP §9.1 is the canonical cross-CIP allocation table.

2. Container fee distribution via SettlementConfig

CIP-10 v1 §12 specifies fee constants (CPU_FEE_PER_CORE_SEC, MEMORY_FEE_PER_GIB_SEC, GPU_FEE_PER_SEC, EGRESS_FEE_PER_BYTE) but does not define how collected fees are split between runner / treasury / burn. v2 routes container compute fees through the existing SettlementConfig pattern at GOVERNANCE_SYSTEM_ACTOR=0x09:
Stored at 0x09 under key system:container_settlement_config. Updated via a dedicated UpdateContainerSettlementConfig opcode (164), matching the per-pool UpdateRegistrySettlementConfig pattern the live codec already uses (the single-opcode-with-target_pool-discriminant form was never adopted). Defaults match the CIP-2 runner-job split for consistency. Governance-gated (0x09, proposal-driven) and MAY be tuned.

3. Three independent off-chain billing flows (clarification)

ext_cip-2-9-10-runner-fee-chain.md §3 identifies three independent billing flows. v2 confirms they share no escrow, no settlement actor, and no update opcode — only the SettlementConfig shape (each pool has its own Update…SettlementConfig opcode and storage key): All three reserve funds at request time and refund the unused portion at settlement. There is no cross-pool escrow; a deficit in one flow does not draw from another.

4. BillingAttestation reuses CIP-23 CompositeAttestation

Per CIP-23 v2 §3.12, the BillingAttestation.tee_signature: Option<bytes64> field defined in CIP-10 v1 §12.3 is replaced with:
Shipped state (trust gate). Until CIP-23 cryptographic verification lands (COW-2504), the shipped field is tee_signature: Option<String> and its presence grants no authority — an unverified marker must not authenticate a meter or bypass the dispute window. Settlement trusts the meter only when metered = true (real cgroup accounting, auditable via the digested snapshot, §7); an unauthenticated self-report charges the full escrow the submitter agreed to at submit. When CIP-23 CompositeAttestation verification exists, a VERIFIED attestation (signature + domain separation + binding to job_id/runner/meters via 0x05::VerifyCae, result_hash = keccak(billing_fields_rlp) per CIP-23 §3.12) may settle immediately per the v1 §12.3 table. Every other settlement defers behind the dispute window (§10).

5. Container Registry instructions (opcodes 160–164)

CIP-10 v1 implies but does not enumerate the system instructions for image / class management. v2 allocates concrete opcodes in the live codec (CIP-13 v2 §1 authoritative wire table). Two authorization tiers apply, each matching its nearest existing analog: Authorization split. Image and resource-class registration (160–163) is operational — a genesis system_deployer key may seed the registry directly (SystemEntitlementUnauthorized otherwise), exactly like the CIP-23 TEE trusted-key registry. Gating these on 0x09 would make them reachable only through an ExecuteProposal governance vote, which cannot be seeded on a fresh devnet and would leave container jobs unpriceable. The settlement split (164) is economic and stays proposal-gated at 0x09, matching UpdateRegistrySettlementConfig; it has a safe default (89/1/10) so nothing needs seeding. Earlier drafts proposed 60–63 / 61–64; the block moved to 160–164 because 158–159 were taken by system-DKG. The cowboy container register-image / register-class CLI submits 160–163; 164 goes through governance; cowboy container dispute submits 165 (see §10 for its optimistic-fallback semantics).

6. Millisecond billing granularity

The v1 §12 fee formulas express duration in whole seconds (*_duration_sec). The shipped billing meters wall-clock in milliseconds and floors the billed duration to MIN_BILLABLE_DURATION_MS = 100:
max_compute_cost (the submit-time escrow, v1 §12.1) still uses the resource class’s max_duration_sec as the upper bound; settlement caps actual at that escrow and refunds max_compute_cost − actual. The floor exists because a sub-second job would otherwise meter zero cpu-time and the runner — whose only container income is the compute split (max_price defaults to 0 for job_request_v1) — would work for free.

7. Enforced-cgroup Standard sandbox + real metering

The Standard tier is hardened rootless runc. Rootless runc cannot mkdir a limit cgroup directly under the root-owned user-<uid>.slice, so the enforced path routes the container’s cgroup through systemd (--systemd-cgroup; OCI cgroupsPath = user.slice:cowboy:<id>; XDG_RUNTIME_DIR defaulted to /run/user/<uid> so rootless runc reaches the user manager). This applies real memory / cpu / pids caps under a delegated user slice (systemd Delegate=yes). Because systemd garbage-collects the transient scope the instant the container exits, the runner samples the cgroup while the container runs (cpu.stat usage_usec, memory.peak) and reports the last readings as the honest meter: cpu_used_millicores = cpu_usec / duration_ms, peak_memory = cgroup high-watermark, metered = true, cgroup_digest = blake3(cpu.stat ‖ NUL ‖ memory.peak) — the raw snapshot the meters were parsed from, bound atomically with the billed values so a dispute (§10) has evidence to audit. Sampling is undercount-bounded: a reading can only miss usage accrued after the final sample, never overstate it. The RUNNER_CONTAINER_UNSAFE_NO_CGROUPS=1 escape hatch (no delegation available) keeps namespace/seccomp/cap isolation but drops resource caps and self-reports the reserved tier limits (metered = false) — a loudly-named, trusted-devnet-only fallback.

9. Governance-tunable fee rates + escrow-time snapshot

Strictly additive: a new record family and four operational opcodes at an existing actor. Task-scoped containers, their billing, and every Part I / Part II rule are untouched. No state migration; a chain with no workload records behaves identically to today. Rates are read once, at escrow time, and snapshotted into the JobComputeLock alongside amount and submitter. Settlement meters with the snapshot — a SetGovParam between submit and settle can never reprice an in-flight job in either direction. Locks written before rate snapshotting deserialize to the genesis constants (they settle at the rates they were escrowed under).

10. Deferred settlement + billing disputes (optimistic fallback)

Every container settlement defers: at the verified result the node computes actual (metered gate, §4) and records a PendingContainerSettlement { escrowed, actual, settle_at_block = result_block + dispute_window_blocks, disputed } at 0x13 — no funds move. A finalize pass at each block pays out settlements whose window has closed: the §2 split on actual, refund escrowed − actual, lock cleared, 0x13 nets to zero per job. Within the window the job’s submitter may submit DisputeContainerBilling { job_id } (opcode 165). The shipped resolution is the v1 §12.3 optimistic fallback, not adjudication: the disputed settlement charges the full escrow (refund = 0 — the amount the submitter agreed to at submit), and the runner takes a reputation hit. Nobody profits from the dispute itself; it exists to make over-metering unprofitable and reputation-costly. TEE-verified attestations (COW-2504) will bypass the window per §4.

11. Registry-backed digest images (supersedes v1 §5.2 / §14.2 addressing)

v1 §5.2’s OCI-registry ImageRef { registry, repository, digest: "sha256:..." } and §14.2’s pull-time layer verification assumed network image pulls. The shipped model has no network pull — images are operator-provisioned on the runner host (pull fees remain future work, v1 §12.2):
  • ImageRef is either a protocol BaseImage enum key or a bare 32-byte digest. A digest ref is admissible iff registered at 0x13 (RegisterBaseImage writes a digest→name index; unregistered digests reject at submit — there is no trust path from a bare digest to a provisioned image).
  • The image digest is blake3 over a canonical rootfs walk — sorted directory entries, each contributing relative-path ‖ NUL ‖ file-mode ‖ NUL ‖ (content | symlink-target) with directories contributing path+mode only. runner-node image-digest <rootfs> prints it; cowboy container register-image --digest records it on-chain.
  • The runner serves a digest image from <image_root>/<digest-hex>/rootfs only after re-hashing the tree and matching the claimed digest — a tampered or mis-provisioned rootfs never runs.
  • Runners advertise serves_registered_images in their container capability; the dispatcher’s Filter 3.5 matches digest jobs only to such runners.
  • Clients resolve image names to digests via the exact-key storage read RPC (GET /actors/{addr}/storage/value?key= — registry keys exceed 32 bytes and are hash-stored, invisible to prefix scans).

12. Strong isolation tier (gVisor)

The Strong tier is gVisor runsc behind the same parameterized OCI driver as runc — same bundle, same timeout/reap hardening. Tier facts as shipped:
  • Selection: the executor picks the lowest tier satisfying the job’s min_isolation_tier, so Standard jobs never consume gVisor capacity; the dispatcher’s Filter 3.5 prefilters on advertised tiers, and a runner advertises Strong only when the runsc binary is actually usable (fail-closed by construction).
  • Rootless runsc invocation: --network=none (the egress-deny model as a flag), --ignore-cgroups (runsc otherwise creates a default sandbox cgroup rootless cannot), --rootless when unprivileged, and no OCI user namespace in the bundle — the sentry is the isolation boundary; a spec-supplied userns kills the gofer re-exec.
  • Billing is unmetered under rootless runsc: it cannot program host cgroups, so the attestation self-reports metered = false and the §4 trust gate charges the full escrow — conservative by construction. gVisor resource caps + metering arrive with rootful deployments.

13. Fail-closed capability advertising

A runner running with RUNNER_CONTAINER_UNSAFE_NO_CGROUPS=1 (§7) cannot enforce resource caps, so it must not advertise capacity it cannot bound: registration fail-closes the container capability rather than advertising tiers the host cannot enforce. Enforced-cgroup hosts advertise normally.

14. Backwards compatibility

Strictly additive. The actor at 0x13 did not previously exist; allocating it does not change any existing state. Container fees were already specified in CIP-10 §12; v2 only pins the routing path.

Persistent Workloads

1. Scope

Task containers are dispatched by a job, bounded by max_duration_sec, and destroyed at exit. Serving inference requires a process that loads weights once, stays warm, holds streaming connections, and outlives any individual request. This section defines the bounded persistent-workload class: its registry record, lifecycle, restart policy, volume mounts, health reporting, and inbound serving endpoint. Initial deployments are first-party services on operator-managed hosts. General third-party scheduling, isolation, migration, and billing remain deferred. A persistent workload differs from an unmanaged sidecar because it has a mandatory on-chain registry record that controls identity, ownership, lifecycle, and route eligibility.

2. Workload model

A persistent workload is a container whose lifetime is governed by its registry record rather than a dispatched job:
  • It has no max_duration_sec bound. Its resource class still limits CPU, memory, scratch storage, and GPU resources.
  • It starts and stops according to desired_state, not per-request dispatch.
  • It exposes one serving endpoint for CIP-15 gateway routes and authorized first-party callers.
  • It MUST load required weights and mounts before reporting Running.
  • Its egress behavior follows the network policy in Part I §9.
Operator-managed workloads MAY be provisioned manually, but the registry record remains mandatory.

3. Registry record

The Container Registry actor at 0x13 stores persistent workloads under:
The suffix is the raw 32-byte workload identifier. The stored JSON record has this schema:
workload_id and owning_actor are immutable. generation begins at 1 and increments on every successful update. observed_at is the consensus-written freshness anchor: registration, a process-changing update, and an accepted status report set it to the executing block height. The mutable update envelope is:
The wire codec MUST bound resource_class to 64 bytes, volume_mounts to 32 entries, each mount path to 256 bytes, and serving_endpoint to 256 bytes. On registration and every update that changes either field, resource_class MUST be non-empty UTF-8 naming an existing Container Registry resource class, and a digest-form image MUST resolve through the registry’s digest index. Protocol base-image enum values are valid by construction. A registered digest or resource class MUST NOT be removed while any workload record references it. The Container Registry stores u64 JSON-decimal reverse reference counts at system:container_registry:workload_ref:image:<digest> and system:container_registry:workload_ref:class:<name>; the image suffix is the raw 32-byte digest and the class suffix is the raw validated UTF-8 name bytes. Registration increments the new references, deregistration decrements the old references, and a process-changing update atomically decrements each changed old reference before incrementing its replacement. A zero count deletes the key. The transaction rejects count overflow, underflow, or removal of an image or class while its count is nonzero. The stored JSON encoding is consensus state and MUST be covered by a golden-vector test. Struct fields are emitted in the order shown above without insignificant whitespace. Fixed and variable byte sequences are JSON arrays of decimal integers in 0..=255; addresses use the canonical Cowboy address string; enum values use the exact variant names in §3.1; ImageRef uses Serde’s externally tagged form ({"Base":"RunnerBase"} or {"Digest":[...]}); options use either null or the encoded value; and lists are JSON arrays. Readers MUST reject records whose decoded workload_id differs from the storage-key suffix.

3.1 Enum tags

The commonware wire tags are: Unknown tags MUST be rejected.

3.2 Instructions

This section allocates system opcodes 196 through 199: The protocol opcode-uniqueness test MUST cover all four values.

3.3 Registration and updates

RegisterWorkload is create-only. It MUST reject an existing workload_id and an owning_actor that does not exist. Registration MUST ignore submitted readiness values and store:
  • generation = 1
  • observed_status = Provisioning
  • observed_at = current_block
UpdateWorkload MUST compare expected_generation with the stored generation and reject a mismatch. Every successful update increments the generation exactly once. An update at generation u64::MAX MUST reject rather than wrap the generation counter. Changes to image, resource_class, volume_mounts, or serving_endpoint describe a different process. They require desired_state = Stopped and a workload that is not live under §4.3. A successful process-changing update resets observed_status = Provisioning and observed_at = current_block. Changes to desired_state, restart_policy, or reporter MAY occur while the workload is live. The new generation prevents a supervisor observing the previous configuration from renewing the new record. Reference, endpoint, and mount validation runs only when the corresponding field is registered or changed. In particular, an expired or revoked mount grant MUST NOT prevent an authorized update that stops the workload, changes its restart policy or reporter, or removes all mounts after the workload is stopped.

3.4 Endpoint validation

serving_endpoint is a control-plane trust boundary because the Gateway dials it. Registration and every update touching the endpoint MUST enforce:
  • UTF-8 encoding;
  • an absolute http or https URL;
  • an explicit host and port;
  • no userinfo, query, or fragment; and
  • a maximum encoded length of 256 bytes.
Operator deployments MAY use private-network endpoints or mutually authenticated transport. The Gateway MUST resolve the endpoint from the registry record and MUST NOT accept an endpoint from route configuration or request data.

4. Lifecycle and health

desired_state is written by an authorized deployer. observed_status is written by the record’s reporter except for the consensus-controlled Provisioning initialization and reset in §3.3. observed_at is written only by those consensus paths and accepted reporter updates; neither sender supplies its stored value directly. ReportWorkloadStatus MUST reject the wrong reporter or a generation mismatch. Every accepted report sets observed_at to the current block.

4.1 Route eligibility

A Gateway MAY establish a new workload connection only when all three conditions hold:
  • desired_state = Running;
  • observed_status = Running; and
  • saturating_sub(current_block, observed_at) <= WORKLOAD_LEASE_TTL_BLOCKS.
WORKLOAD_LEASE_TTL_BLOCKS = 60. Eligibility is evaluated from a fresh proof-checked registry read when the connection is established. A later record change does not mutate an established connection.

4.2 Supervisor behavior

The workload MUST expose GET /_cowboy/health at the serving endpoint’s origin and return 200 only when it is ready to serve. The probe URL keeps the endpoint’s scheme, host, and explicit port, replaces any configured base path with /_cowboy/health, and carries no query or fragment. Three consecutive successful probes report Running; three consecutive failed probes report Degraded. The reporter MUST submit a status-change report and MUST renew an unchanged status at least once every 60 blocks. The supervisor MUST fail closed when it cannot read the current record or renew the lease: before its last accepted report becomes stale, it stops admitting work, terminates the process after the drain window, and revokes mount material. A network partition therefore cannot leave a registry-ineligible process serving indefinitely with retained volume access. While desired state is Running, Never makes every exit terminal, OnFailure restarts every exit except an explicit process exit code of zero, and Always restarts every exit. Signal termination, OOM kill, loss of the process without an exit status, and nonzero exit codes are failures; an intentional supervisor termination after desired state becomes Stopped is not. A terminal exit follows the Stopped reporting order in §4.3. A restartable exit reports Degraded before waiting, then retries indefinitely. Consecutive failure number n, starting at zero, waits min(2^n, 60) seconds with no jitter. The retry counter resets only after 300 seconds of continuous healthy operation; changing desired state to Stopped cancels any pending retry.

4.3 Stop and deregistration

A transition to Stopped stops admission of new connections. Existing connections MAY drain for up to 30 seconds before termination. The supervisor MUST revoke mount material after the process exits. After an intentional stop has drained, the process has exited, and mount material has been revoked, the reporter MUST report observed_status = Stopped. A terminal exit that the restart policy will not restart follows the same ordering. If the reporter cannot submit that final report, the record becomes removable only through the stale-cleanup path. DeregisterWorkload requires desired_state = Stopped and either:
  • observed_status = Stopped; or
  • current_block >= saturating_add(observed_at, WORKLOAD_LEASE_TTL_BLOCKS + WORKLOAD_STALE_CLEANUP_GRACE_BLOCKS).
WORKLOAD_STALE_CLEANUP_GRACE_BLOCKS = 30. The stale cleanup comparison is inclusive at the exact boundary. A fresh Provisioning or Degraded record is not deregisterable.

5. Volume mounts

On registration and every update touching mounts:
  • the volume MUST exist;
  • mount_path MUST be valid UTF-8, absolute, contain no NUL byte, . segment, .. segment, repeated /, or trailing / except for the root path, and be no longer than 256 bytes;
  • mount paths MUST NOT overlap after trailing-slash normalization; and
  • principal MUST be the volume owner or hold a grant covering the requested access mode.
Immutable assets such as model weights SHOULD be mounted read-only. The registry has no asset-purpose field, so consensus enforces the declared mode and its grant but does not infer a volume’s contents. Mount material is scoped to the workload lifetime and MUST NOT reuse a per-job CapToken as an unbounded credential. It MUST NOT outlive the underlying volume grant. The supervisor performs a proof-checked grant validation before mounting, before every lease report, and at least once per health interval. After observing expiry or revocation it stops request admission and revokes mount material before the next health interval elapses; it MUST NOT report Running while any mount is unauthorized. Private volumes use the existing CIP-9 and CBSS key-release mechanisms; this section does not create a second key-custody path.

6. Billing

Persistent workloads on operator-managed hosts do not use the task-container escrow, compute-metering, or dispute path. Request-level payment is defined outside CIP-10. Billing for third-party persistent-workload hosts is deferred.

7. Deferred capabilities

  • hibernation and summon-on-message;
  • migration and rescheduling across hosts;
  • autoscaling;
  • third-party scheduling and isolation;
  • persistent-workload host billing and disputes;
  • container-to-container networking and multi-container pods; and
  • connectors, secret capabilities, and other fields beyond the registry record in §3.

8. Activation

Activation is coordinated in this order:
  1. merge the canonical protocol types and opcodes;
  2. advance the node protocol pin and activate the 0x13 handlers;
  3. advance the runner protocol pin and deploy the supervisor and signed reporter;
  4. deploy proof-checked Gateway workload resolution and streaming routes; and
  5. configure and register the operator-managed workload.
After each merge, consumers MUST use an exact protocol revision containing opcodes 196 through 199. The registry is not active until the reporter is deployed and this end-to-end sequence passes:
Tests MUST prove:
  • an old-generation status report rejects;
  • a process-changing update rejects while desired state is Running;
  • a successful process-changing update clears inherited readiness;
  • a new Provisioning record is not stale at its registration block;
  • stale cleanup rejects one block before its boundary and succeeds at the boundary; and
  • a fresh Degraded workload with desired state Stopped is not deregisterable.

9. Backwards compatibility

The registry adds a record family and four operational opcodes to the existing Container Registry actor. A chain with no workload records behaves as before and requires no state migration.