Gateway Implementation Guide

​

0. Mental model

                                          ┌─ per-volume metadata cache (CIP-15 §8.2)
                          ┌─ caches  ◄────┤  keyed: volume_id
                          │                └─ object cache (LRU, bounded)
       HTTP request ──►   GW ──┐
                          │    └─► state-read RPC (full nodes)
                          │        manifest+shard RPCs (Relay Nodes)
                          │
                          └─ per-actor routes cache (CIP-15 §8.2)
                             keyed: actor_address

1. Resolves the actor from the host header (CIP-14 Route Registry — already implemented).
2. Looks up the cached Routes for that actor.
3. Picks a (verb, path) winner (CIP-15 §6.6).
4. Dispatches: serve from a CBFS volume (static), or invoke a named handler via existing CIP-14 RPC (dynamic). Optionally gates on payment per CIP-18.

​

1. Build order

​

2. Phase 1 — Routes fetch + method dispatch

​

2.1 New types

RoutesValue   = CBOR-encoded `Routes` (CIP-15 §6.2)
RoutesCache   = LRU<actor_address, ActorRoutesCache>
ActorRoutesCache {
  routes:               Routes,
  routes_bytes:         bytes,          // verified RoutesValue, for change detection
  last_verified_block:  BlockHeight,
}

​

2.2 New RPC: GET_STATE

• /actor/{address} — committed actor summary and manifest with at most 1,000 enumerable storage rows; storage_truncated indicates additional enumerable rows (no Merkle proof)
• /actors/{address}/storage — paginated listing of verbatim keys of at most 64 bytes (no Merkle proof)
• /actors/{address}/storage/value — exact-key read, including longer keys that cannot be enumerated (no Merkle proof)
• read-only handler invocation: POST /actor/read per CIP-14 §8.3 (implemented; Gateway budget propagation and conformance coverage remain to be verified)

GET_STATE(actor_address, key) -> {
    value: bytes,
    proof: MerkleProof,   // QMDB/MMR operation-inclusion proof against the unified state_root
    state_root: bytes32,
    block_height: u64,
}

​

2.3 Routes fetch loop

for each actor with active traffic:
  every MANIFEST_POLL_INTERVAL blocks:
    resp = node.GET_STATE(actor, "__cowboy/routes")   # a full node; Runners do not serve GET_STATE yet (CIP-17 §1)
    trusted_root = trusted_header(resp.block_height).state_root
    if not verify_state_read(resp, trusted_root):      # CIP-17 §5.3
      try another full node; if all fail, log warning, keep stale cache
      continue
    if resp.value == cache[actor].routes_bytes:
      continue                       # nothing changed
    routes = cbor.decode(resp.value)
    if not validate(routes):         # CIP-15 §6.8
      log warning; fall back to all-CIP-14 dispatch
      continue
    cache[actor] = ActorRoutesCache(routes, resp.value, resp.block_height)

• The poll interval is shared with the volume manifest_root poll — one polling task can refresh both anchors per actor.
• Empty value (actor declares no routes) → not an error. Cache an empty Routes; dispatch falls back to CIP-14.
• Failures are sticky in the worst sense: if validation fails, the previously-cached table stays in use. That’s the safe behavior.

​

2.4 The resolver

def resolve(actor, verb, path) -> Resolution:
    # 1. Reserved
    if path.startswith("/_cowboy/"):
        return Resolution.cowboy_intercepted()

    # 2. Cached routes (or empty)
    routes = cache[actor].routes if actor in cache else None
    if routes is None or len(routes.routes) == 0:
        return Resolution.cip14_fallback()  # dispatch to http.request handler

    # 3. Filter enabled, match (verb, path)
    candidates = [r for r in routes.routes
                  if r.enabled
                  and (r.verb == verb or r.verb == "ANY")
                  and path_matches(r.path, path)]

    if not candidates:
        return Resolution.not_found_404()

    # 4. Tie-break: priority desc, longest concrete prefix, non-ANY verb, Method > Volume
    winner = max(candidates, key=lambda r: (
        r.priority,
        concrete_prefix_length(r.path),
        0 if r.verb == "ANY" else 1,
        0 if r.target.kind == "volume" else 1,
    ))

    return Resolution.route(winner, extract_path_params(winner.path, path))

​

2.5 Dispatch — Method target, pays = actor

def dispatch_method(actor, route, request, path_params):
    envelope = HttpRequestEnvelope(
        method=request.method,
        path=request.path,
        path_params=path_params,           # populated by the Gateway resolver
        query=request.query,
        headers=request.headers,
        body=request.body,
        host=request.host,
        request_id=request.request_id,
    )

    mode = route.mode
    if mode is None:
        mode = "read" if request.method in ("GET", "HEAD") else "submit"

    if mode == "read":
        return runner.read_handler(actor, route.target.name, envelope)   # unsigned query
    if mode == "submit":
        return runner.send_command(actor, route.target.name, envelope)   # signed, wait for result
    if mode == "submit_no_wait":
        return runner.send_command_no_wait(actor, route.target.name, envelope)  # signed, no wait
    raise ValueError("invalid route mode")

​

2.6 Tests for P1

• Resolver: every branch in CIP-15 §6.6 — priority ties, longest-prefix ties, ANY vs concrete verb, Method-vs-Volume tie, empty routes table, reserved paths.
• Validation: malformed CBOR, oversized table, unknown volume, missing price for caller, invalid status codes, reserved path prefix.
• Path matching: literals, {id}, *rest, edge cases (trailing slashes, empty segments, encoded chars).

• Deploy a sample actor with two routes: GET /a → method_a, POST /b → method_b.
• curl GET /a → handler method_a runs.
• curl POST /b → handler method_b runs.
• curl GET /unknown → 404.
• Upgrade the actor with a new routes table, wait one poll interval, verify dispatch follows the new table.
• Have the actor return zero routes; verify CIP-14 fallback (everything hits http.request).

• One real actor, one Gateway, one Runner. Run the resolver under load (e.g., wrk -t4 -c100 -d30s http://<0xactor>.canyon.cowboylabs.net/api/foo). Watch for cache miss thrashing if the poll interval is wrong.

​

3. Phase 2 — Static volume serving (Volume targets)

​

3.1 Components

• Per-volume metadata cache (CIP-15 §8.2 Layer 1a): manifest, content_types, cache_config, cors_config.
• Object cache (Layer 2): bounded LRU, keyed by (volume_id, object_path).
• manifest_root polling for invalidation (CIP-15 §8.3).
• GET_MANIFEST + GET_SHARD RPCs (CIP-15 §8.4).
• Reed-Solomon reconstruction + BLAKE3 integrity chain (CIP-15 §8.5–§8.6).
• Hedged parallel fetch (CIP-15 §8.7).

​

3.2 Volume-target dispatch

def dispatch_volume(actor, route, request):
    if request.method not in ("GET", "HEAD"):
        return Response(status=405, body="Method Not Allowed")

    volume_path = resolve_volume_path(route.target, request.path)  # CIP-15 §6.7
    obj = get_object(route.target.volume_name, volume_path)        # cache or fetch
    if obj is None:
        if route.target.fallback:
            obj = get_object(route.target.volume_name, route.target.fallback)
            if obj is None:
                return Response(status=404)
            return serve(obj, status=route.target.fallback_status)
        return Response(status=404)

    return serve(obj, status=200)

​

3.3 Tests for P2

• Volume mount serves from the right volume_name and applies strip_prefix / volume_path_prefix correctly.
• Fallback path (SPA case): GET /about with no about object → serves index.html with status 200.
• 404 fallback: GET /missing.png with no fallback configured → 404.
• 405 on POST /assets/foo.png (volume route, non-GET).
• Integrity chain: corrupted shard → fetch from another node and verify; if all corrupt → 502.

​

4. Phase 3 — Route updates

• Decode with the protocol codec’s Routes type (cowboy-protocol-codec, cip15-routes), the same type the node validates with, including CIP-11 runner_route entries (which the Gateway does not dispatch). gateway_types::Routes is replaced by it; the Gateway keeps no second copy of the schema.
• A table that fails Gateway validation means the Gateway is older than the node’s schema: log a warning, keep the previously cached table, and alert the operator.

​

5. Phase 4 — pays = caller (CIP-18)

def dispatch_method(actor, route, request, path_params):
    if route.pays == "caller":
        proof = extract_payment_proof(request.headers)        # CIP-18
        if proof is None or not verify_payment(proof, route.price):
            return cip18_402_response(route.price)            # CIP-18
    # ... existing dispatch

​

6. Phase 5 — CORS, conditional requests, compression

​

7. Phase 6 — Optimization

• Hedged parallel shard fetch (CIP-15 §8.7).
• Compressed variants in the cache.
• Cache warming on actor registration.
• Coalescing concurrent requests for the same object.

​

8. Phase 7 — Workload streaming routes (WS/SSE)

• Actor route dispatch: accept Target::Workload from the normal actor route cache. For operator fallback entries, use the StaticWorkloadRoute schema keyed (actor, verb, path); collision with actor routes fails config load, and fallback dispatch rechecks overlap against the current actor-route snapshot so a later actor route prevents the conflicting static entry from serving.
• Endpoint resolution: workload record id → serving_endpoint via the workload registry record. Actor routes require owning_actor == URL actor with no override. Operator fallback routes require owning_actor == entry.actor or the explicit expected_workload_owner. Never accept an endpoint from route or request data.
• SSE relay: streamed responses flush unbuffered; idle_timeout_ms on response-chunk silence; WORKLOAD_CONNECT_TIMEOUT_MS on establishment; cancellation and permit release when the client drops the body.
• WS relay: auth / pays gating and backend handshake before the client 101; propagate backend rejection and selected subprotocol; disable extensions; then relay data/close semantics in both directions, handling ping/pong hop-locally while counting them as activity, with backend death → client close 1011.
• Caps: route max_concurrent_streams (default DEFAULT_MAX_CONCURRENT_STREAMS), acquired before registry/dial work and held through body/WS teardown; 503 beyond, no queueing.
• Payment gate: resolve an enabled CIP-18 price_table before the route’s pays fallback; policy read or decode failures return 503. The current Gateway returns 503 for an observed exhausted actor budget. CIP-18’s configured 402 alternative and a cross-replica pre-serve budget reservation remain integration work: asynchronous deductions can otherwise race when two requests read the same positive balance.
• Tests: actor-declared route dispatch with no operator entry; wrong-owner and stale-report rejection; SSE event flush latency (no whole-body buffering), idle teardown, client-drop cancellation/permit release, concurrent-cap rejection at bound + zero/ceiling validation, connection-lifetime cap including handshake time, mid-stream backend kill emits close 1011, backend handshake 401 propagation, subprotocol selection and extension stripping, WS Origin passthrough, 402-gated establishment, owner/actor binding enforcement (expected_workload_owner limited to operator routes; record-swap negative), semantic static-vs-actor collision rejected at load and a conflicting operator fallback rejected with a diagnostic when attempted under a refreshed actor-route snapshot (wildcard, param, and ANY overlap vectors, not just equal keys), cross-target auth replay (Runner-signed request rejected on a Workload route and vice versa — distinct domain tags) plus golden vectors for the pinned "cowboy/cip15/workload-route-auth/v1" domain, workload-id encoding validation (length + noncanonical hex rejected), path/query/header forwarding and hop-by-hop stripping.

​

9. Operational surface

• Per-Gateway: cache hit ratio (object, metadata, routes), cache size, evictions.
• Per-actor: requests/s, dispatch breakdown (volume vs method, by verb), routes-table change rate.
• Per-route: hit count, p50/p95/p99 latency.
• Per workload route: open connections vs cap, connection duration histogram, idle vs backend-death vs client-close teardown counts.
• Errors: validation failures (per actor), proof failures, 502s from integrity failures.

​

10. Open questions (confirm before merging the PR)

1. GET_STATE RPC. Resolved: specified by CIP-17 and implemented in core — see §2.2.
2. Path-param delivery. This CIP specifies the Gateway extracts path_params and passes them in the HttpRequestEnvelope. Confirm with the runtime team that the envelope schema can carry them, or extend it.
3. Named-handler dispatch. Confirm the runner-side dispatcher accepts arbitrary handler names on the CIP-14 query and command paths (it should — that’s how non-HTTP messages work — but verify).
4. CIP-18 readiness. r2 elevates CIP-18 from Companion to Requires for Phase 4 — track CIP-18 PaymentGate 0x12 deployment. Ship Phase 4 only after CIP-18 settles.
5. Rollout. Is there an existing actor that’s a good first canary? 17-hn-feed? A toy actor in a devnet?

​

11. What success looks like

• Existing actors keep working with zero changes.
• Actors that declare routes get verb-aware dispatch, named handlers and per-route payment policy.
• Static asset serving meets CDN-class latency for cache hits (sub-10ms p99 from a warm Gateway).
• Routes updates land in Gateway caches within 6 seconds of commit.
• A FastAPI/axum/Hono developer can read a CIP-15 actor and immediately understand what it does.