CIP-11: Runner Connectivity and Push Job Delivery

​

1. Abstract

1. A runtime connectivity layer: each runner holds long-lived QUIC streams to a fixed-size, deterministically assigned subset of validators.
2. A vote-piggybacked attestation: each validator’s vote on block H carries a bitmap describing the runners it is locally connected to at the moment of voting.
3. A canonical presence bitmap derivable from the finalization quorum: the bitmap for block H is a deterministic function of the validator votes that finalize H. Selection at block H+1 filters the runner candidate set by canonical_bitmap[H] and applies a weight multiplier to the most-recently-successful runner per (submitter, job_kind).

• No polling on the hot path. Job dispatch latency drops from ~poll_interval (default 5 s) to one network round trip.
• Bounded fan-out. Each runner holds k connections (default 3–8 depending on validator-set size); each validator holds O(N_runners · k / N_validators) connections.
• Consensus-safe determinism. The presence bitmap is a function of the finalization quorum’s signed votes, not of any validator’s local off-chain view.
• Reuses on-chain identity. Runner identity, stake, capabilities, and entitlements remain in the Runner Registry (0x0000…0001); CIP-11 adds no new on-chain registration step.
• Reuses existing transport. Inter-validator messages ride on the existing commonware-p2p vote channel; no new p2p stack is introduced.

​

2. Motivation

1. The Job Dispatcher (0x0000…0002) selects M runners on-chain via stake-weighted Fisher-Yates over the registered candidate set and writes each assignment to runner_jobs_key(addr).
2. Each runner separately polls GET /runner/{addr}/jobs over plain HTTP every job_poll_interval_seconds (default 5 s).
3. When the runner has a result, it POSTs to /runner/{addr}/job_result. The validator builds a JobResultSubmit transaction and forwards it to the mempool.

1. Polling latency is the floor on job start time. A 5 s poll interval is already a large fraction of the desired end-to-end budget for short jobs (LLM completions, single MCP calls). Driving it lower wastes bandwidth across thousands of runners.
2. Liveness is not actually consulted. The Registry’s last_heartbeat field exists but is not a hard gate on candidate selection. Selection happily assigns jobs to runners whose process died ten minutes ago, which then time out and re-select. The user-visible effect is a multi-block latency spike on every silent runner failure.
3. Heartbeats are expensive. With ~1000 runners each posting a POST /heartbeat transaction every few blocks, the chain pays for liveness signal bandwidth that an attestation layer could carry off-chain at near-zero cost.
4. There is no path to runner stickiness. “Most-recently-successful runner for actor A” is a natural cache locality optimization (CBFS volume warm caches, MCP server connection reuse, model warm starts) but cannot be expressed cleanly when delivery is pull-based and dispatcher state has no MRU index.

​

3. Definitions

• Connectivity Subset (Sub(R)): The deterministic, fixed-size set of validators that runner R is required to maintain QUIC connections to. Computed from the current validator set (§5).
• Vote Presence Bitmap: A bitmap embedded in a validator’s vote on block H, indexed by Runner Registry order, indicating which runners that validator currently has a healthy control stream to.
• Canonical Presence Bitmap (P(H)): A deterministic bitmap derived from the vote presence bitmaps of the finalization quorum for block H. Indexed by Runner Registry order. Bit i is set iff at least presence_threshold of the finalizing votes have bit i set.
• Push Dispatch: The act of a validator opening a new QUIC stream to a runner and writing a JobAssignment frame, replacing the runner’s GET /runner/{addr}/jobs poll.
• MRU Weight Multiplier: A deterministic weight adjustment applied to the most-recently-successful runner for a given (submitter, job_kind) pair during the first Fisher-Yates draw (§9.3).
• Subset Slot: An integer in [0, k) identifying the runner’s position in the deterministic ordering relative to a single validator. Used to load-balance reconnects.
• Dispatch Outcome: One of Success, Duplicate, SoftFailure, or HardFailure — a validator’s classification of a single push dispatch attempt (§10.6).

​

4. Design Overview

​

4.1 Architecture

​

4.2 Lifecycle

1. Bootstrap. Runner reads the current validator set and computes its connectivity subset Sub(R) (§5). It opens a long-lived QUIC connection to each validator in Sub(R).
2. Handshake. On each connection, runner and validator complete a pubkey-bound handshake (§6.2). Validator verifies the runner is registered, has sufficient stake, and is correctly assigned to it via Sub(R).
3. Heartbeat. On the control stream, the runner emits a HeartbeatPing once per block. The validator records the runner as locally present.
4. Vote with attestation. When a validator votes on block H, its vote payload includes a vote_presence_bitmap summarizing which runners it currently has healthy control streams to (§7).
5. Finalization derives canonical bitmap. Once H is finalized by a 2f+1 quorum of votes, every validator computes P(H) deterministically from the finalizing votes (§8). No off-chain message timing is consulted.
6. Selection. When a job is submitted at block H+1, the Job Dispatcher runs the existing Fisher-Yates VRF selection over the candidate set, additionally gated by P(H) and with the first draw weight-multiplied for the MRU runner (§9).
7. Push dispatch. Validators that hold an active connection to an assigned runner open a job stream and send JobAssignment. The runner deduplicates by job_id.
8. Result. The runner streams the result back on the same job stream and signs it. The first validator to receive a valid result builds the JobResultSubmit transaction (existing CIP-2 path).
9. MRU update. When the Result Verifier records a successful JobResult, it updates mru_key(submitter, job_kind) in the Job Dispatcher actor (§9.4).
10. Re-selection. If no validator reports successful delivery within dispatch_timeout_blocks, the Dispatcher applies the existing CIP-2 §6 timeout-based re-selection.

​

4.3 Relationship to Existing CIPs

• CIP-2 (Off-Chain Compute). CIP-11 strictly extends CIP-2. The candidate list (§4), Fisher-Yates VRF selection (§5), commit-reveal verification, and timeout-based re-selection (§6) are all preserved. Push dispatch over QUIC replaces polling on the delivery hot path; the polling endpoint GET /runner/{addr}/jobs is retained as a delivery-only fallback during and after migration (§14). The last_heartbeat field on RunnerRegistration is retained as a slow-path signal but is no longer load-bearing.
• CIP-9 (Runner Storage). Push delivery reduces the cold-start window before a runner can begin reading from CBFS volumes. MRU bias (§9.3) directly improves CBFS read-cache hit rates by routing jobs for the same actor to the same runner where possible.
• CIP-10 (Runner Containers). Lower job-start latency reduces idle container cost. CIP-10 base images SHOULD ship with the CIP-11 client transport.
• CIP-13 (Runner Delegation). Delegated stake is opaque to CIP-11; a runner’s identity for connectivity purposes is still its registered address.

​

5. Connectivity Subset Assignment

​

5.1 Subset Function

Sub(R, t) = first k validators in
            sort_by(v ∈ V_t, key = keccak256(R.pubkey ‖ v.pubkey))

k = clamp(ceil(log2(|V_t|)) + 1, MIN_SUBSET, MAX_SUBSET)

​

5.2 Properties

• Stable under runner churn. A new runner’s subset is independent of existing runners’ subsets. Adding or removing a runner does not change any other runner’s subset.
• Stable-ish under validator churn. When a single validator joins or leaves, the expected fraction of runners whose subset changes is bounded by O(k / |V_t|). Runners affected by a validator leaving simply open a connection to the next validator in the sorted list.
• Load-balanced in expectation. Each validator handles |R| · k / |V_t| runner connections in expectation, with concentration of O(sqrt(|R|)) by Chernoff bounds.
• Validator-side DoS gate. A validator MUST reject connections from a runner not in the runner’s expected Sub(R, t) for the validator’s own identity. This bounds Sybil connection storms to k connections per registered runner identity, which costs a full Runner Registry registration plus MIN_STAKE_CBY_WEI.

​

5.3 Subset Rotation

​

6. The QUIC Connection Layer

​

6.1 Transport

• One control stream (bidirectional, stream id 0) for the lifetime of the connection.
• Zero or more job streams (validator-initiated, bidirectional), one per active job assignment.

​

6.2 Handshake and Authentication

Hello {
  version:           u16,                  // CIP-11 wire version
  party_pubkey:      [u8; 33],             // secp256k1 compressed
  party_role:        Role,                 // Runner | Validator
  block_height:      u64,                  // sender's view of head
  challenge_nonce:   [u8; 32],             // recipient must echo+sign
}

HelloAck {
  version:           u16,
  party_pubkey:      [u8; 33],
  party_role:        Role,
  signed_challenge:  [u8; 65],             // ecdsa over peer's challenge_nonce
  block_height:      u64,
}

​

6.3 Heartbeats

HeartbeatPing {
  block_height:      u64,                  // runner's view
  nonce:             u64,                  // monotonically increasing
  signed:            [u8; 65],             // ecdsa over (block_height ‖ nonce)
}

HeartbeatPong {
  block_height:      u64,                  // validator's view
  validator_signed:  [u8; 65],             // ecdsa over (block_height ‖ runner_pubkey)
}

​

6.4 Backpressure

BackpressureSignal {
  active_jobs:        u32,
  max_concurrent:     u32,                 // from RunnerCapabilities
  accepting_new:      bool,                // false ⇒ omit me from next vote bitmap
  reason:             Option<String>,      // human-readable, e.g. "gpu_oom"
}

​

6.5 Capability and Entitlement Updates

CapabilityDelta {
  block_height:       u64,
  added_capabilities: Vec<Capability>,
  removed_capabilities: Vec<Capability>,
  added_entitlements: Vec<EntitlementId>,
  removed_entitlements: Vec<EntitlementId>,
  signed:             [u8; 65],
}

​

7. Vote-Piggybacked Presence Attestations

​

7.1 Vote Payload Extension

vote_presence_bitmap: BitVec               // length = |RunnerRegistry as of block H|

​

7.2 Bitmap Construction (Per Validator)

1. Runner i has an authenticated, open QUIC control stream to this validator.
2. The validator has received a valid HeartbeatPing from runner i within the last PRESENCE_TIMEOUT_BLOCKS blocks.
3. The runner has not most recently signaled BackpressureSignal { accepting_new: false }.
4. The validator’s identity is in Sub(runner_i.pubkey, current_subset_epoch).

​

7.3 Optional Observability Channel (Non-Normative)

​

8. Canonical Presence Bitmap

​

8.1 Derivation

present_count[i] = | { v ∈ Votes(H) : b_v[i] = 1 } |

P(H)[i]          = 1  iff  present_count[i] ≥ presence_threshold(|Votes(H)|)
                   0  otherwise

​

8.2 Threshold

presence_threshold(n) = floor((n - 1) / 3) + 1            // == f + 1, where n = 3f + 1

1. Fabrication resistance. A Byzantine fault budget of f adversarial validators cannot reach f+1 attestations on their own. To set a bit in P(H), at least one honest validator must have a real, authenticated control stream to that runner. A bit set in P(H) therefore implies the runner is reachable from the validator set.
2. Suppression resistance. With n = 3f+1 minimum finalization quorum, at most f honest validators may legitimately attest “not present” for any given runner that has been actually online. Even if every adversarial validator also attests “not present,” there are still f+1 honest “present” votes — sufficient to set the bit. A proposer cannot suppress a present runner without the cooperation of f+1 actual offline-observation events.

​

8.3 Storage Cost

​

8.4 Indexing Across Registry Mutations

​

9. Updated Selection Algorithm

• The candidate filter list (CIP-2 §4 / select_runner_committee_with_seed filter chain): one new filter is inserted.
• The first weighted Fisher-Yates draw (CIP-2 §5): the MRU runner’s stake weight is multiplied by MRU_WEIGHT_MULTIPLIER.

​

9.1 Candidate Filter Diff

Existing dispatcher filters (unchanged numbering):
  0. Exclusion list (re-selection)
  1. Health:                  HealthStatus::Healthy
  1.5. Presence (NEW):        P(H-1)[runner_idx] = 1
  2. Reputation:              reputation ≥ 50
  3. Capability:              supports JobType
  4. TEE:                     tee_required ⇒ tee_support.is_some()
  5. Price:                   compute_second ≤ max_price
  6. Concurrency:             active_jobs < max_concurrent_jobs
  7. Entitlement:             holds required_runner_pool
  8. Stake sufficiency:       stake ≥ 1.5 × max_price

​

9.2 Selection Algorithm (Unchanged)

seed = keccak256(block_hash_proxy ‖ "cowboy-runner-select-v2:" ‖ job_id ‖ submitted_at_le8)

​

9.3 MRU Weight Multiplier

mru = lookup_mru(submitter, job_kind, current_block_height)        // §9.4
for i in candidates:
    base[i] = stake_to_weight(candidates[i].stake, MIN_STAKE_CBY_WEI)
    if Some((addr, set_at)) = mru
       and candidates[i].address == addr
       and (current_block_height - set_at) ≤ MRU_TTL_BLOCKS:
        weights_iter0[i] = base[i].saturating_mul(MRU_WEIGHT_MULTIPLIER)
    else:
        weights_iter0[i] = base[i]

​

9.4 New Dispatcher State: mru_key

mru_key(submitter: Address, job_kind: u8) → MruRecord {
  runner_address: Address,
  set_at_block:   u64,
}

​

9.5 MRU Scope (Open for Future Refinement)

​

10. Push Job Delivery

​

10.1 Dispatch

JobAssignment {
  job_id:           [u8; 32],
  job_spec:         JobSpec,                    // canonical encoding from CIP-2
  block_height:     u64,                        // H
  deadline_block:   u64,                        // H + timeout_blocks
  validator_signed: [u8; 65],                   // ecdsa over (job_id ‖ deadline)
}

​

10.2 Runner Acknowledgment

JobAck {
  job_id:           [u8; 32],
  status:           AckStatus,                  // see below
  reason:           Option<String>,
  signed:           [u8; 65],
}

enum AckStatus {
  Accepted,                // first delivery for this job_id, runner is executing
  Duplicate,               // job_id already in flight from another validator (NORMAL)
  Reject(RejectReason),    // runner cannot serve, e.g. capacity, missing entitlement
}

​

10.3 Result

JobResult {
  job_id:           [u8; 32],
  result_data:      Vec<u8>,                    // CIP-2 commit-reveal payload
  runner_signed:    [u8; 65],
}

​

10.4 Streaming Progress (Optional)

JobProgress {
  job_id:           [u8; 32],
  fraction:         u8,                         // 0–100
  detail:           Option<String>,
}

​

10.5 Cancellation

JobCancel {
  job_id:           [u8; 32],
  reason:           CancelReason,               // Reassigned | Expired | Failed
  validator_signed: [u8; 65],
}

​

10.6 Dispatch Outcome Classification

​

11. Failure Handling and Re-selection

​

11.1 Per-Dispatch Timeout

​

11.2 Per-Job Timeout

​

11.3 Connection Loss

• The validator clears the runner from its local presence set immediately.
• The runner attempts to reconnect with exponential backoff (100 ms × 2^n, capped at 30 s, jittered ±25%).
• If the runner has not reconnected by the next vote, the validator’s vote bitmap will not include the runner; downstream this may push present_count below presence_threshold and clear P(H)[runner_idx].

​

11.4 Validator Set Changes Mid-Job

​

11.5 Reputation and Slashing

​

12. Wire Format

​

12.1 Frame Encoding

Frame {
  length:    u32 (big-endian),     // bytes following
  type:      u8,                   // see frame type table
  payload:   bytes,                // serde-cbor encoded
}

​

12.2 Frame Type Table

​

12.3 Vote Payload Extension (Inter-Validator)

VotePresenceBitmap {
  bit_length:        u32,                       // == |RunnerRegistry at parent block|
  bytes:             Vec<u8>,                   // ceil(bit_length / 8) bytes, little-endian
}

​

12.4 Versioning

​

13. System Constants

​

14. Migration and Coexistence

​

Phase 1 — Shadow

• New transport ships in validator and runner binaries.
• Runners open QUIC connections; validators include vote_presence_bitmap in their votes.
• The canonical bitmap P(H) is computed but ignored by the dispatcher (presence_filter_active = false).
• Polling endpoint GET /runner/{addr}/jobs remains the load-bearing dispatch path.
• Goal: validate vote-bitmap convergence and observe P(H) accuracy under production load without affecting selection.

​

Phase 2 — Hot Path

• presence_filter_active = true. The presence filter (§9.1) is enforced.
• Push dispatch over QUIC is the primary delivery path.
• The polling endpoint GET /runner/{addr}/jobs is retained as a delivery-only fallback: a runner that has been selected via P(H-1) but whose QUIC stream broke between selection and dispatch can still recover the assignment via polling. Polling does not contribute to presence. A runner that has never established QUIC connectivity to any validator in Sub(R) will not appear in P(H) and is therefore not eligible for selection. Operators on the legacy binary will need to upgrade.
• Goal: shift the job-start latency floor from poll_interval to one network RTT.

​

Phase 3 — Sunset

• Polling endpoint returns 410 Gone.
• On-chain POST /heartbeat transactions are accepted but no longer required for liveness; last_heartbeat is purely informational.
• Goal: reclaim the on-chain heartbeat tx bandwidth.

​

15. Security Considerations

​

15.1 Sybil Connection Storms

​

15.2 Presence Lying and Censorship

• Fabrication. To set a bit in P(H), at least f+1 votes in the finalization quorum must claim that runner is connected. Adversary controls at most f votes, so independent fabrication is impossible. To set a bit, at least one honest validator must have a real, authenticated control stream to the runner — which by definition means the runner is reachable from the validator set. Fabrication therefore reduces to “the runner is online with at least one honest validator,” which is exactly what we want the bit to mean.
• Suppression by adversarial proposer. A proposer must include ≥ 2f+1 votes to finalize a block. Even if every adversarial validator votes “not present” for runner R, the remaining f+1 honest votes determine whether R’s bit is set. If ≥ f+1 honest validators legitimately observe R as present and vote that way, R’s bit is set in P(H) — the proposer cannot exclude any of them without dropping below the finalization quorum.
• Suppression by adversarial honest-vote omission. A proposer could in principle include only 2f+1 votes selected to minimize “present” counts (i.e., prefer “not present” votes when they exist). For this to clear an honest runner’s bit, at least f+1 honest validators would have to have legitimately voted “not present” — meaning they did not have a control stream to R at vote time. This is an honest disagreement about presence; the runner is genuinely partially unreachable, and clearing the bit is correct behavior.

​

15.3 Connection Hijack

​

15.4 MEV / Job Front-Running

​

15.5 Bitmap Index Replay

​

15.6 Plaintext Wire (Removed)

​

15.7 Vote-Size Amplification

​

16. Open Questions

1. Optional observability gossip channel. §7.3 mentions a non-normative monitoring-only gossip layer. Is it worth specifying as part of CIP-11 (so dashboards across implementations agree), or left as an implementation detail?
2. MRU scope refinement. Default scope (submitter, job_kind) is coarse; more useful would be (submitter, model_id) for LLM and (submitter, primary_volume_id) for storage-attached jobs. Tracked for v1.1 (§9.5).
3. Multi-region validator sets. If validators are split across geographies, the deterministic subset may give a runner all-foreign validators. Worth considering a region-aware subset function in a follow-up CIP.
4. Aggregate vote signatures. With BLS or similar aggregate signatures, the vote bitmap evidence could be compressed into a single per-bit aggregate signature attesting “these f+1 validators agree.” Out of scope until BLS lands in the consensus stack.
5. Standalone runner crate vs in-tree types crate. Implementation will need to land canonical wire types in a single crate (proposed: node/runner types are canonical, runner/crates/* re-exports). Out of scope for the spec; tracked for the implementation PR.

​

17. Reference Implementation Notes

• node/validator/src/main.rs — extend the consensus vote payload with vote_presence_bitmap; ensure the signature covers it.
• node/runner/src/types.rs — add wire-frame types (JobAssignment, Hello, HeartbeatPing, etc.) and the MruRecord storage record.
• node/execution/src/runner/dispatcher.rs — insert the §9.1 presence filter; add lookup_mru() and apply the §9.3 multiplier in iteration 0 of select_runner_committee_with_seed.
• node/execution/src/runner/verifier.rs — write mru_key(submitter, job_kind) on verified result.
• node/runner/src/storage_keys.rs — add the mru_key family.
• node/types/src/execution.rs — add a derived presence_bitmap_at(H) accessor (computed from finalizing votes, not stored as a block field).
• node/rpc/src/handlers/runner.rs — gate GET /runner/{addr}/jobs behind a feature flag for Phase 3 sunset.
• runner/crates/runner-node/src/node.rs — add QUIC client, replace polling loop with JobAssignment consumption.
• runner/crates/chain-client/src/client.rs — keep submit_result_via_rest as fallback; add stream-based result path.