Base URL
:8090; inside the container it listens on :8080.
GET /healthz and GET /metrics are unauthenticated probes. Every other /vfs/... endpoint goes through the configured authenticator (see Authentication). Errors use a structured JSON envelope:
Authentication
vfs serve selects an authenticator at boot from VFS_AUTH_TOKEN:
Set
VFS_AUTH_TOKEN on every public deployment. The boot log line vfs.ready ... auth=bearer (VFS_AUTH_TOKEN) confirms which path is hot.
When the conductor proxies VirtualFS at /api/vfs/*, it strips client-supplied Authorization headers, injects its own Bearer ${VFS_AUTH_TOKEN} server-side, and forwards the verified tenant as X-Tenant-ID. See the Conductor API VirtualFS Proxy section.
When the Vercel-hosted dashboard rewrites /vfs/* directly to a Railway VirtualFS service, dashboard/middleware.ts performs the same header swap at the edge using the VFS_TOKEN Vercel environment variable. The dashboard SPA never sees the real bearer token.
Agent Tool Bundle
The runner wires a VirtualFS dispatcher from environment at startup. Profiles can opt in to the@vfs capability bundle, while the default @fs, @pool, and @sandbox-transfer tools use the same dispatcher-backed substrate when it is available. The @vfs bundle is intentionally minimal — agents work with the VirtualFS as if it were a real filesystem:
For agents, other file operations — read, list, find, write (via redirect), delete (via
rm), stat, cd, etc. — are reached through vfs_execute. The two-tool surface mirrors how agents work with a real shell. Non-agent callers can use the typed JSON endpoints below when they need structured responses or binary-safe reads and writes.
vfs_execute requires cmd; cwd defaults to /. Use ls or ls / as the first call to discover visible top-level directories such as /s3, /data, /kb, and /agents. In the prod, dev, and disk profiles those paths are directories inside the single root mount at /; runtime user mounts may also claim non-root prefixes. Creates and writes must target a path inside a registered mount; read ops (find, grep, ls, stat) work at / and honor the active session allowlist before returning entries or matches. stat returns directory metadata for mount roots and directories represented by _dir placeholders or child entries. Text ls output appends / to directory paths and leaves file paths bare; the dashboard Files page and CLI adapters use that suffix to recover directory-vs-file state. Empty ls results render as (empty), and empty find results render as (no matches), so agents can distinguish successful empty output from failed execution; the dashboard Files page filters the bare (empty) ls sentinel into its empty-state UI instead of showing it as a file row. The dashboard Files page disables new file and new folder actions at / and prompts users to open a directory first. The smoke in-memory driver treats _dir placeholder files created by mkdir as internal markers and hides them from listing output.
vfs_grep requires path and pattern. By default, pattern is treated as a literal string; set regex=true to use Go regexp syntax. Results are capped at maxMatches or 100 when omitted.
Shell Parser and Compiler
virtualfs/bash.Parse scaffolds the sealed shell adapter by parsing command strings with mvdan.cc/sh/v3/syntax only. It does not use the interp package and does not fall back to a host shell.
The parser returns a flat Pipeline of stages with literal argv values, redirects, and connectors for pipes, &&, ||, and ;. Unsupported constructs such as variable expansion, command substitution, backgrounding, and control flow are recorded on Stage.Unsupported so the compiler can return UNSUPPORTED_SHELL_FEATURE.
virtualfs/bash.Compile resolves each stage through a sealed built-in registry. Unknown commands return COMMAND_NOT_FOUND; there is no host shell or PATH lookup. CompileWithGlobber can expand globbed path-position arguments through an injected Globber; without one, glob patterns remain literal. Predicate and option values, such as find -name "*.txt" and head -n 2, stay literal. Non-recursive globs such as /agents/foo/*.md match only direct children of the directory; use **, such as /agents/foo/**/*.md, to include nested descendants. Empty path-position glob matches compile as no-op input stages. The registered v1.5 built-ins are:
The compiler does not perform ACL checks; callers dispatch the compiled Op Registry requests through the VirtualFS dispatcher policy. Output redirects (
> and >>) compile to synthetic Write ops, with append redirects carrying Append=true; runtime plumbing supplies the redirected content.
Workspace Execute
virtualfs.New(dispatcher, policy) constructs the in-process shell executor for VirtualFS. The returned *virtualfs.Workspace parses a command with virtualfs/bash.Parse, compiles each stage left-to-right with virtualfs/bash.Compile, and dispatches each compiled file operation through the configured dispatcher.Dispatcher. Successful cd stages update the working directory for later relative paths in the same Execute call.
Execute supports the parser/compiler’s sealed shell subset only; there is no host-shell fallback, no PATH lookup, and unknown commands return a compile error with ExitCode: 1. ExecOpts.Cwd defaults to / when omitted. The returned Output contains Stdout, Stderr, ExitCode, and DurationMs.
Pipeline stages honor |, &&, ||, and ;. Pipe-connected stdout feeds the next stage’s content transform, while non-pipe stage stdout is flushed into the final output. Dispatcher errors are written to stderr and set the stage exit code to 1; context cancellation before a stage returns partial output with the context error.
ACL checks remain the dispatcher’s responsibility. Workspace.Execute does not call Policy.Allows directly; the workspace stores the bound policy for downstream features such as file-prompt generation and error formatting. A nil policy is accepted, and any policy configured on the dispatcher still applies to dispatched operations.
The workspace carries bash.Options for adapter configuration. Its zero value is permissive: an empty BuiltinAllowList allows all registered built-ins, and MaxOutputBytes == 0 means no buffered output limit.
File Byte Cache
VirtualFS has a separate byte-level content cache undervirtualfs/cache/bytes. This cache stores raw file bytes by mount, tenant, path, and content hash, so stale bytes from an older file version are not reused after the catalog hash changes.
When a dispatcher is configured with Deps.BytesCache, whole-file Cat calls check the byte cache before reading from the mount driver. Range reads bypass the byte cache, and cache misses are stored only when the returned content is at or below MaxObjBytes, which defaults to 4 MiB when unset. Write and Delete invalidate the old byte-cache key before calling the driver, resolving the old content hash from the index cache or a direct Stat.
The production backend is Redis via virtualfs/cache/bytes/redis. Redis entries use keys shaped as vfs:bytes:{mount}:{tenant}:{path}:{hash}. Callers choose the TTL for each write; the dispatcher default is one hour. Redis eviction policy, such as allkeys-lru, is configured on the Redis server rather than in the package.
Set VFS_TEST_REDIS_URL to a Redis URL such as redis://localhost:6379/0 to run the Redis byte-cache integration tests.
Development Tests
The VirtualFS integration test exercises write, catalog,Ls, and Cat with an in-process HTTP server, memory cache, and stub mount driver on every test run. Set VFS_TEST_POSTGRES_DSN to run the same flow against the Postgres catalog cache.
The Postgres integration path still uses the stub mount driver, so it does not require VFS_TEST_R2_BUCKET or live R2 credentials. Test and server code that need a ready-to-use catalog cache can use cache.NewPostgresFromDSN(ctx, dsn), which opens a pgxpool and applies the embedded VirtualFS migrations before returning the cache.
The CLI smoke gate runs the v1.5 north-star UX samples against an in-process VirtualFS, so it does not require Postgres, Redis, R2, or a running server:
runtime.WithVFSFromEnv() registers dispatcher-backed @fs and @vfs tools in an agent session:
/healthz, mount discovery, file write/read/list/stat/find/delete, recursive grep, and sealed /vfs/exec operations. Set VFS_TENANT to isolate the temporary paths; it defaults to railway-smoke.
For a reusable local HTTP server, use serve:
serve listens on :8080 by default, accepts the smokeAuth dev-token shown in the boot banner unless VFS_AUTH_TOKEN is set, emits structured startup and request logs, exposes /healthz and /metrics, and seeds /s3/log.jsonl plus /s3/report.parquet unless --seed=false is passed. Override the bind address with --addr HOST:PORT or VFS_SERVE_ADDR.
Backend selection is per component. --postgres-dsn uses Postgres for the index cache, falling back to memory if the connection fails. --redis-url enables the Redis byte cache, falling back to no byte cache if Redis is unavailable. --s3-bucket uses the R2/S3-compatible mount driver, with --s3-endpoint for local S3-compatible services; when no bucket is configured, serve uses the smoke in-memory driver. The default Postgres, Redis, and generic S3 flag values are blank for local smoke runs, while VFS_S3_BUCKET and VFS_S3_ENDPOINT are always honored as explicit S3/R2 settings. Set VFS_BACKEND=production to let the other backend defaults read VFS_POSTGRES_DSN, POSTGRES_DSN, POSTGRES_HOST parts, VFS_REDIS_URL, REDIS_URL, CF_R2_BUCKET, CF_R2_ENDPOINT, S3_BUCKET, and AWS_ENDPOINT_URL_S3.
Bucket selection is --s3-bucket, VFS_S3_BUCKET, then production-only CF_R2_BUCKET and S3_BUCKET; endpoint selection is --s3-endpoint, VFS_S3_ENDPOINT, then production-only CF_R2_ENDPOINT and AWS_ENDPOINT_URL_S3. Credential selection follows the endpoint shape: endpoints matching .r2.cloudflarestorage.com use CF_R2_ACCESS_KEY_ID and CF_R2_SECRET_ACCESS_KEY, while other endpoints use AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. R2 endpoints use region auto unless AWS_REGION is set.
API-created R2 mounts use secret:// credential refs instead of process environment variables. vfs serve resolves those refs through the conductor’s internal-listener route POST /internal/secrets/resolve when CONDUCTOR_INTERNAL_URL is set. Requests are signed with chatsig HMAC using VFS_INTERNAL_SIGNING_KEY_CURRENT (base64); the authenticated tenant travels in the X-Chat-Gateway-Tenant header and is stamped onto the request context by the conductor. The legacy AGENT_ORC_INTERNAL_BEARER / AGENT_ORC_CONDUCTOR_URL / CONDUCTOR_BASE_URL variables are deprecated; resolver returns ErrNoResolver if the new variables are unset, so mounts that rely on process-env credentials still work.
Production Deployment
The repo ships a Railway-ready Dockerfile atdocker/Dockerfile.vfs. It builds ./virtualfs/cmd/vfs from the Go workspace, runs as nonroot on a distroless base, and entrypoints into vfs serve. The shared root railway.toml already wires /healthz as the deploy gate for Go monorepo services with a 300s timeout and ON_FAILURE restart policy.
The reference service config lives at docker/railway-vfs.toml. Required environment for a Railway deploy:
Backend wiring uses the production-mode env contract from the serve flags:
- Postgres index cache — link a Railway Postgres plugin and set
VFS_POSTGRES_DSN=${{Postgres.DATABASE_URL}}(or rely on thePOSTGRES_HOST/USER/PASSWORD/DBparts). - Redis byte cache — link a Railway Redis plugin and set
VFS_REDIS_URL=${{Redis.REDIS_URL}}. - Cloudflare R2 — set
CF_R2_BUCKET,CF_R2_ENDPOINT,CF_R2_ACCESS_KEY_ID,CF_R2_SECRET_ACCESS_KEY. The driver auto-detects R2 by the.r2.cloudflarestorage.comendpoint suffix and uses regionauto. - AWS S3 — set
VFS_S3_BUCKET,AWS_REGION,AWS_ACCESS_KEY_ID,AWS_SECRET_ACCESS_KEY.
- Conductor — set
VFS_BASE_URLandVFS_AUTH_TOKENon the conductor process to mount/api/vfs/*as a reverse proxy. ClientAuthorizationheaders are stripped and the conductor injects the bearer token server-side. - Vercel dashboard —
dashboard/vercel.jsonrewrites/vfs/*to the Railway VirtualFS URL, anddashboard/middleware.tsinjectsBearer ${VFS_TOKEN}from the Vercel project environment.VFS_TOKENon Vercel must matchVFS_AUTH_TOKENon Railway; if it is missing the middleware returns500 VFS_TOKEN_UNSET.
Health
Unauthenticated liveness probe. A healthy server returns200 OK with an empty body.
Metrics
Unauthenticated Prometheus scrape endpoint served byvfs serve on the same HTTP listener as the API. It includes VirtualFS dispatcher, cache, driver, shell execution, and HTTP request metrics with the vfs_ prefix.
Mount Table
Returns the registered mount table in dispatcher registration order. The response is tenant-agnostic and read-only; the dashboard Files page uses it to render the live mount table instead of inferring it fromls /. The prod, dev, and disk profiles register one system mount named root at /; well-known paths such as /agents, /data, /s3, and /kb are directories inside that mount. Each entry includes source, a display label such as Cloudflare R2, AWS S3, Local, or RAM, and origin, which is system for boot-time mounts and user for mounts created through POST /vfs/mounts.
Strict mount semantics
Each mount declares its concrete data source via thebackend field. status reflects whether the backend came up successfully at boot:
ready— the backend driver initialized and this mount accepts operations.unavailable— the backend failed to initialize at boot (e.g., unreachable R2 bucket, missing credentials). Operations on this mount returnMOUNT_UNAVAILABLE. The server continues to boot; all other mounts are unaffected. Thereasonfield carries the human-readable boot-time error.
unavailable, not silently backed by RAM.
500 NOT_CONFIGURED.
Runtime Mounts
Create Mount
Registers a mount at runtime. The response shape matches one entry fromGET /vfs/mounts; backend initialization failures still return 200 OK with status: "unavailable" and a reason.
409 MOUNT_EXISTS means either the mount name or path prefix is already registered.
Delete Mount
Removes a runtime mount and releases its bound driver when no remaining mount references the same backend. Returns204 No Content on success and 404 NOT_FOUND when the mount is not registered.
Typed File Operations
These endpoints expose the dispatcher’s typed file operations as JSON-in/JSON-out HTTP calls. They are intended for SDKs, dashboard services, scripts, and other non-agent callers that should not go through the sealed shell parser. They share the same authenticator and structured error envelope asPOST /vfs/exec.
cat, ls, stat, find, write, and delete require path. grep requires both path and pattern; when maxMatches is omitted or zero, the server defaults it to 100.
Cross-mount traversal
Whenpath resolves to a single mount, the op runs against that mount only. In the default prod/dev/disk profiles, / resolves to the root mount. When path is a directory above one or more non-root mounts and does not resolve to a broader mount, the op fans out to every nested mount and unions the results:
Mounts whose
status="unavailable" are silently skipped from the union;
their contents do not appear and the op does not fail. Direct ops that
resolve to an unavailable mount still return MOUNT_UNAVAILABLE.
encoding on writes must be text, base64, or omitted. Omitted encoding is treated as text.
Execute Command
Runs a sealed VirtualFS shell command through the server’s configuredWorkspaceExecutor. The server must be configured with ServerConfig.Executor; otherwise the endpoint returns 500 NOT_CONFIGURED.
200 OK; callers should inspect exitCode. The dashboard /api/vfs client treats a non-zero exitCode as a VfsError, using a CODE: message prefix from stderr when present so failed ls, cat, mkdir, and write calls surface as structured UI errors. Dispatcher and policy errors use the structured error envelope and the corresponding HTTP status. If a non-OK response has no structured error body, the dashboard client falls back to UPSTREAM_ERROR for 5xx responses or HTTP_<status> for other statuses, with the HTTP status text or a status-derived message.
The endpoint caps each output stream to 8 MiB by default and sets truncated: true when either stdout or stderr is capped. Override the per-stream cap with VFS_EXEC_MAX_OUTPUT_BYTES.
Stream Command
Runs the same request body asPOST /vfs/exec, but returns Server-Sent Events with Content-Type: text/event-stream.
The current workspace executor buffers command output before returning, so streaming is coarse-grained rather than live per-byte output: the handler emits at most one stdout event, at most one stderr event, then a terminal exit event.
Multi-line stdout, stderr, and error payloads are encoded as standard SSE multi-line events, with each payload line emitted as its own data: field.
error event whose data is the same structured error envelope used by JSON endpoints. Bad JSON and missing cmd are still returned as 400 INVALID_REQUEST before the SSE stream is opened.
Invalidate Cache
Marks a cached directory listing as stale by settingfully_listed=false in the VirtualFS catalog. Use this after out-of-band writes, such as direct object uploads, so the next tree or list operation refreshes from the mount driver.
Tree
Returns a catalog-only directory listing. The handler reads from the index cache and does not fall through to the mount driver. If the catalog has no matching entries, it returns an emptyentries array.
Query params: