> ## Documentation Index
> Fetch the complete documentation index at: https://docs.orcapods.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Kubernetes Deployment

> Scale Orca to production with Kubernetes StatefulSets for runners and Deployments for conductors.

<Note>This is a self-hosting guide. The hosted product at [https://www.orcapods.ai](https://www.orcapods.ai) needs none of this — sign in and go.</Note>

## Architecture on Kubernetes

```mermaid theme={null}
flowchart TB
  ingress["Ingress<br/>nginx"]
  conductor["Conductor HPA<br/>Deployment, N replicas<br/>conductor-0<br/>conductor-1<br/>..."]
  runners["Runners<br/>StatefulSet<br/>runner-0.runners.svc:7070<br/>runner-1.runners.svc:7070<br/>..."]
  sidecar["agent-worker<br/>Sidecar container<br/>MODE=claude"]

  ingress --> conductor
  conductor -->|"RUNNER_URLS<br/>stable pod DNS"| runners
  runners -->|"additional container<br/>inside each pod"| sidecar
```

Key points:

* **Conductors** → `Deployment` + `HPA` (stateless, scale freely)
* **Runners** → `StatefulSet` + `headless Service` (stable pod DNS = stable `RUNNER_BASE_URL`)
* **Sidecars** → additional containers inside each runner pod
* **Ingress** → nginx with `proxy_buffering off` for SSE

***

## Conductor Deployment

```yaml theme={null}
apiVersion: apps/v1
kind: Deployment
metadata:
  name: conductor
  namespace: orca
spec:
  replicas: 3
  selector:
    matchLabels:
      app: conductor
  template:
    metadata:
      labels:
        app: conductor
    spec:
      containers:
        - name: conductor
          image: orca/conductor:latest
          ports:
            - containerPort: 8080
          env:
            - name: CONDUCTOR_PORT
              value: "8080"
            - name: RUNNER_URLS
              value: >-
                http://runner-0.runners.orca.svc:7070,
                http://runner-1.runners.orca.svc:7070,
                http://runner-2.runners.orca.svc:7070
            - name: AGENT_ORC_RUNS_DIR
              value: /data/runs
            - name: AGENT_ORC_PLANS_DIR
              value: /data/plans.jsonl
            - name: S3_BUCKET
              value: agent-artifacts
            - name: AWS_REGION
              value: us-east-1
            - name: AWS_ENDPOINT_URL_S3
              value: "http://minio.orca.svc:9000"
            - name: AWS_ACCESS_KEY_ID
              valueFrom:
                secretKeyRef:
                  name: orca-secrets
                  key: s3-access-key-id
            - name: AWS_SECRET_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: orca-secrets
                  key: s3-secret-access-key
            - name: S3_FORCE_PATH_STYLE
              value: "true"
          volumeMounts:
            - name: runs-storage
              mountPath: /data/runs
          readinessProbe:
            httpGet:
              path: /healthz
              port: 8080
            initialDelaySeconds: 5
            periodSeconds: 10
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: 1
              memory: 512Mi
      volumes:
        - name: runs-storage
          persistentVolumeClaim:
            claimName: conductor-runs-pvc
---
apiVersion: v1
kind: Service
metadata:
  name: conductor
  namespace: orca
spec:
  selector:
    app: conductor
  ports:
    - port: 8080
      targetPort: 8080
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: conductor-hpa
  namespace: orca
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: conductor
  minReplicas: 2
  maxReplicas: 10
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60
```

***

## Runner StatefulSet

Runners must have **stable DNS names** so conductors can route sessions to them.

```yaml theme={null}
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: runner
  namespace: orca
spec:
  serviceName: runners          # headless service name
  replicas: 3
  selector:
    matchLabels:
      app: runner
  template:
    metadata:
      labels:
        app: runner
    spec:
      containers:
        # --- Go Runner ---
        - name: runner
          image: orca/runner:latest
          ports:
            - containerPort: 7070
          env:
            - name: RUNNER_PORT
              value: "7070"
            - name: RUNNER_BASE_URL
              value: "http://$(HOSTNAME).runners.orca.svc:7070"
            - name: MCP_BASE_URL
              value: "http://localhost:7070"
            - name: RUNNER_CAPABILITIES
              value: "claude,general"
            - name: CLAUDE_SIDECAR_URL
              value: "http://localhost:7071"
            - name: GENERAL_SIDECAR_URL
              value: "http://localhost:7073"
            - name: ANTHROPIC_API_KEY
              valueFrom:
                secretKeyRef:
                  name: orca-secrets
                  key: anthropic-api-key
            - name: TAVILY_API_KEY
              valueFrom:
                secretKeyRef:
                  name: orca-secrets
                  key: tavily-api-key
            - name: S3_BUCKET
              value: agent-artifacts
            - name: AWS_REGION
              value: us-east-1
            - name: AWS_ENDPOINT_URL_S3
              value: "http://minio.orca.svc:9000"
            - name: AWS_ACCESS_KEY_ID
              valueFrom:
                secretKeyRef:
                  name: orca-secrets
                  key: s3-access-key-id
            - name: AWS_SECRET_ACCESS_KEY
              valueFrom:
                secretKeyRef:
                  name: orca-secrets
                  key: s3-secret-access-key
            - name: S3_FORCE_PATH_STYLE
              value: "true"
          readinessProbe:
            httpGet:
              path: /healthz
              port: 7070
            initialDelaySeconds: 10
            periodSeconds: 15
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
            limits:
              cpu: 2
              memory: 2Gi

        # --- Claude Sidecar ---
        - name: sidecar-claude
          image: orca/agent-worker:latest
          env:
            - name: MODE
              value: claude
            - name: PORT
              value: "7071"
            - name: ANTHROPIC_API_KEY
              valueFrom:
                secretKeyRef:
                  name: orca-secrets
                  key: anthropic-api-key
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: 1
              memory: 512Mi

        # --- General Sidecar ---
        - name: sidecar-general
          image: orca/agent-worker:latest
          env:
            - name: MODE
              value: general
            - name: PORT
              value: "7073"
            - name: ANTHROPIC_API_KEY
              valueFrom:
                secretKeyRef:
                  name: orca-secrets
                  key: anthropic-api-key
            - name: OPENAI_API_KEY
              valueFrom:
                secretKeyRef:
                  name: orca-secrets
                  key: openai-api-key
          resources:
            requests:
              cpu: 250m
              memory: 256Mi
            limits:
              cpu: 1
              memory: 512Mi
---
# Headless service for stable DNS
apiVersion: v1
kind: Service
metadata:
  name: runners
  namespace: orca
spec:
  clusterIP: None
  selector:
    app: runner
  ports:
    - port: 7070
      targetPort: 7070
```

With this `StatefulSet` and headless service, pods get stable DNS names:

* `runner-0.runners.orca.svc:7070`
* `runner-1.runners.orca.svc:7070`
* `runner-2.runners.orca.svc:7070`

Set these in `RUNNER_URLS` on the conductor.

***

## Secrets

```yaml theme={null}
apiVersion: v1
kind: Secret
metadata:
  name: orca-secrets
  namespace: orca
type: Opaque
stringData:
  anthropic-api-key: "sk-ant-..."
  openai-api-key: "sk-..."
  tavily-api-key: "tvly-..."
  s3-access-key-id: "agent-orc"
  s3-secret-access-key: "replace-me"
```

***

## VirtualFS Strict Mount Configuration

When deploying the standalone VirtualFS service (`orca/vfs`) to Kubernetes, the strict mount model requires explicit backend configuration. Do NOT use `--profile dev` in production — the dev profile uses in-process RAM and cannot persist data across pod restarts.

Required environment variables for the prod profile:

| Variable                | Description                                                                     |
| ----------------------- | ------------------------------------------------------------------------------- |
| `VFS_S3_BUCKET`         | R2/S3 bucket name for the production root mount                                 |
| `VFS_S3_ENDPOINT`       | S3-compatible endpoint URL (e.g., `https://<account>.r2.cloudflarestorage.com`) |
| `AWS_ACCESS_KEY_ID`     | S3/R2 access key                                                                |
| `AWS_SECRET_ACCESS_KEY` | S3/R2 secret key                                                                |
| `VFS_AUTH_TOKEN`        | Bearer token for VirtualFS API authentication                                   |

If these are missing or the bucket is unreachable at pod startup, the affected mounts surface as `unavailable` in `GET /vfs/mounts`. The server still passes `/healthz` — liveness checks are independent of mount status. Use the VirtualFS smoke script (`scripts/vfs-smoke.sh`) in a post-deploy readiness check to assert all expected mounts are `ready` before traffic is sent.

```yaml theme={null}
# VirtualFS Deployment excerpt — add to your manifest
env:
  - name: VFS_S3_BUCKET
    valueFrom:
      secretKeyRef:
        name: orca-secrets
        key: vfs-s3-bucket
  - name: VFS_S3_ENDPOINT
    value: "https://<account>.r2.cloudflarestorage.com"
  - name: AWS_ACCESS_KEY_ID
    valueFrom:
      secretKeyRef:
        name: orca-secrets
        key: s3-access-key-id
  - name: AWS_SECRET_ACCESS_KEY
    valueFrom:
      secretKeyRef:
        name: orca-secrets
        key: s3-secret-access-key
  - name: VFS_AUTH_TOKEN
    valueFrom:
      secretKeyRef:
        name: orca-secrets
        key: vfs-auth-token
```

***

## Ingress with SSE Support

<Warning>
  Without `proxy-buffering: "false"`, Server-Sent Events will not stream correctly. nginx-ingress will buffer the entire response.
</Warning>

```yaml theme={null}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: orca
  namespace: orca
  annotations:
    nginx.ingress.kubernetes.io/proxy-buffering: "off"
    nginx.ingress.kubernetes.io/proxy-read-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-send-timeout: "3600"
    nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
spec:
  ingressClassName: nginx
  rules:
    - host: orca.example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: conductor
                port:
                  number: 8080
```

***

## Updating Runner Count

When you scale the StatefulSet, update the conductor's `RUNNER_URLS` to include the new pod DNS names. The safest approach:

1. Scale the StatefulSet: `kubectl scale statefulset runner --replicas=4 -n orca`
2. Wait for the new pod to be ready: `kubectl rollout status statefulset runner -n orca`
3. Update the conductor `Deployment` env to include `runner-3.runners.orca.svc:7070`
4. Roll out the conductor: `kubectl rollout restart deployment conductor -n orca`

New sessions will round-robin to include the new runner. Existing sessions remain pinned to their original runner.

***

## Namespace and RBAC

```yaml theme={null}
apiVersion: v1
kind: Namespace
metadata:
  name: orca
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: orca
  namespace: orca
```

***

## Monitoring Integration

Both conductor and runner expose `/metrics` in Prometheus text format. Create a `ServiceMonitor` if you're using the Prometheus Operator:

```yaml theme={null}
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: orca
  namespace: monitoring
spec:
  selector:
    matchLabels:
      app: conductor
  namespaceSelector:
    matchNames: [orca]
  endpoints:
    - port: http
      path: /metrics
      interval: 30s
```