Multi-Runner Setup

This is a self-hosting guide. The hosted product at https://www.orcapods.ai needs none of this: sign in and go.

Why Multiple Runners?

Running multiple runners lets you:

Scale horizontally: more runners means more concurrent sessions
Specialize by runtime: dedicate runners to specific LLM providers
Isolate workloads: separate runners for different tenants or use cases (future)
Improve resilience: conductor routes away from unhealthy runners

How Session Routing Works

Each runner has a hash derived from its RUNNER_BASE_URL:

hash = SHA256(RUNNER_BASE_URL)[:8]

Session IDs embed this hash: sess-<hash>-<8hex> When any conductor receives a request targeting a session, it extracts the hash, looks it up in its remote.Pool, and routes directly to the owning runner. Routing is stateless by session ID, but membership changes are currently held in each conductor process; horizontally replicated conductors must therefore reconcile their registries independently.

Basic Two-Runner Setup

# Terminal 1: Runner A (claude + vercel)
RUNNER_PORT=7070 \
RUNNER_BASE_URL=http://localhost:7070 \
RUNNER_CAPABILITIES=claude,vercel \
CLAUDE_SIDECAR_URL=http://localhost:7071 \
VERCEL_SIDECAR_URL=http://localhost:7073 \
ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY} \
  go run ./agent-runtime/cmd/runner

# Terminal 2: Runner B (codex + vercel)
RUNNER_PORT=7075 \
RUNNER_BASE_URL=http://localhost:7075 \
RUNNER_CAPABILITIES=codex,vercel \
CODEX_SIDECAR_URL=http://localhost:7072 \
VERCEL_SIDECAR_URL=http://localhost:7073 \
OPENAI_API_KEY=${OPENAI_API_KEY} \
  go run ./agent-runtime/cmd/runner

# Terminal 3: Conductor pointing at both
CONDUCTOR_PORT=8080 \
RUNNER_URLS=http://localhost:7070,http://localhost:7075 \
  go run ./agent-runtime/cmd/conductor

Capability-Based Routing

Runners advertise their capabilities. The conductor only routes sessions to runners that support the requested runtime: Set capabilities via RUNNER_CAPABILITIES (comma-separated):

RUNNER_CAPABILITIES=claude,vercel

Sidecar Configuration per Runner

Each runner can have different sidecars for different runtimes:

Env Var	Purpose
`SIDECAR_URL`	Single poly-sidecar for all runtimes
`PI_SIDECAR_URL`	Pi-specific sidecar (overrides `SIDECAR_URL`)
`CLAUDE_SIDECAR_URL`	Claude-specific sidecar (overrides `SIDECAR_URL`)
`CODEX_SIDECAR_URL`	Codex-specific sidecar (overrides `SIDECAR_URL`)
`VERCEL_SIDECAR_URL`	General-mode sidecar (overrides `SIDECAR_URL`)

Per-runtime sidecars (recommended for production):

# Runner with all three specialized sidecars
CLAUDE_SIDECAR_URL=http://<claude-sidecar>:7071
CODEX_SIDECAR_URL=http://<codex-sidecar>:7072
VERCEL_SIDECAR_URL=http://<vercel-sidecar>:7073

Single poly-sidecar (simpler for development):

# MODE=all dispatches by profile.runtime
SIDECAR_URL=http://<all-runtimes-sidecar>:7070

Checking Topology

The GET /api/topology endpoint shows all runners, their capabilities, health, membership state, process footprint, session breakdown, and the sidecar worker instances observed beneath each runner:

curl http://localhost:8080/api/topology | jq

[
  {
    "hash": "a1b2c3d4",
    "capabilities": ["claude", "vercel"],
    "capabilitiesKnown": true,
    "state": "live",
    "healthy": true,
    "latencyMs": 8,
    "activeSessions": 3
  },
  {
    "hash": "e5f6a7b8",
    "capabilities": ["codex", "vercel"],
    "healthy": true,
    "latencyMs": 10,
    "activeSessions": 1
  }
]

Runner entries are keyed by hash rather than by URL: the runner’s base URL is an internal infrastructure identifier and is stripped before this response is built, so it never reaches a tenant. process, sessionsByRuntime, and sidecars can be null when a runner does not answer its topology probe. sidecars[].observedInstances is a lower bound on worker replicas, not a count, and cold means an idle or scaled-to-zero sidecar rather than a fault. There is currently no dashboard view of this data; use the curl example above, or GET /api/stats/summary for an aggregate healthy/total runner count.

Runner Health & Failover

The conductor probes each runner at startup and during reconcile, and records a membership state. An individual runner that is unreachable during startup no longer prevents the conductor from booting when at least one runner answers; it remains visible as unreachable and is excluded from new session routing until a successful reconcile. A runner that is deliberately draining is also excluded from new sessions, while sessions it already owns remain routed by their session ID. Boot still fails when no runner answers at all.

Sessions are never migrated automatically. Sessions already assigned to an unreachable runner remain pinned to it and can error until it recovers; sessions on a draining runner remain routable so the runner can finish its in-flight work. Removing a runner outright can orphan those sessions.

Multi-Host Deployment

When runners run on different hosts, RUNNER_BASE_URL must be a routable URL that all conductors can reach:

# Runner on host-a
RUNNER_BASE_URL=http://<runner-a-host>:7070

# Runner on host-b
RUNNER_BASE_URL=http://<runner-b-host>:7070

# Conductor (anywhere)
RUNNER_URLS=http://<runner-a-host>:7070,http://<runner-b-host>:7070

Never use localhost for RUNNER_BASE_URL in multi-host setups. It needs to be a URL that resolves from the conductor’s network.

Runner-Specific Tool Configuration

Different runners can have different tools available. For example:

Runner A has TAVILY_API_KEY set, so web_search and web_extract work
Runner B does not, so those tools return errors

Sessions are assigned to runners based on capability, not tool availability. If you rely on specific tools, ensure all runners in the pod have the necessary env vars set.

Get Started

Core Concepts

Guides

Runbooks

Multi-Runner Setup

Why Multiple Runners?

How Session Routing Works

Basic Two-Runner Setup

Capability-Based Routing

Sidecar Configuration per Runner

Checking Topology

Runner Health & Failover

Multi-Host Deployment

Runner-Specific Tool Configuration

​Why Multiple Runners?

​How Session Routing Works

​Basic Two-Runner Setup

​Capability-Based Routing

​Sidecar Configuration per Runner

​Checking Topology

​Runner Health & Failover

​Multi-Host Deployment

​Runner-Specific Tool Configuration

Why Multiple Runners?

How Session Routing Works

Basic Two-Runner Setup

Capability-Based Routing

Sidecar Configuration per Runner

Checking Topology

Runner Health & Failover

Multi-Host Deployment

Runner-Specific Tool Configuration