Configuration files

CapyCTL reads four kinds of YAML file: deployments, and the files of the server, a host and standalone. Check any of them with:

capyctl validate config --file <file>

The examples below are the files in docs/examples/, shown as they are; a test checks that CapyCTL accepts every one. They show the format; they are not tuned settings for any model. Every setting of the server, host and standalone files can also be given as a flag or an environment variable, and capyctl config show prints where each value came from: see Settings.

The smallest deployment

Three fields; CapyCTL fills in the rest from the checkpoint and the machine.

# The smallest capyctl deployment (`capyctl deploy model --file deployment-minimal.yaml`).
# Parsed by the product: `capyctl validate config --file docs/examples/deployment-minimal.yaml`
# shows the document with every default filled; `--host docs/examples/host.yaml`
# also shows what that host fills in.
#
# Everything else is chosen by capyctl: the route is the name, the recipe is
# `standard`, the host's `vllm` engine at its published revision, its GPU, the
# park tier its hardware supports, and memory sized from the checkpoint and the
# GPU. Any field of `deployment-single.yaml` may be added to override a default.
name: coding-small
# The engine installation (runtime profile) on the host. `vllm` or `sglang`
# also names a host's one installation of that engine.
engine: vllm
# A directory under the host's models directory (`~/models` by default), an
# absolute path, `~/...` (expanded by the CLI), or a pinned Hugging Face
# repository: `model: {hf: Qwen/Qwen3-4B-Instruct-2507}` (the CLI pins it to
# the commit it names now; `@<commit>` pins it yourself).
model: coding-small-r1

Deployment for standalone, every field

This one names local, the engine standalone takes from --vllm-bin or CAPYCTL_VLLM_BIN. With an engine added by capyctl engine add, name that profile instead.

# A deployment for `capyctl start standalone` with the engine in CAPYCTL_VLLM_BIN.
# Deploy it with `capyctl deploy model --file deployment-standalone.yaml --activate`.
schema_version: 1
kind: deployment
name: my-model
# The `model` name clients send in their OpenAI requests.
routes: ["my-model"]
# `local` is the engine standalone found in CAPYCTL_VLLM_BIN or CAPYCTL_SGLANG_BIN.
runtime_profile: local
runtime_profile_revision: 1
recipe: standard
# `deep` frees the model's GPU memory while it is parked.
residency: deep
recovery: reconcile
model:
  # A checkpoint directory under the models directory (~/models by default).
  path: Qwen3-4B
  # Labels for this copy of the weights; change them when the files change.
  # A `sha256:` followed by 64 hex digits is not a label: it is CapyCTL's
  # checkpoint digest (over every file under the model directory, not one
  # file's hash), which the start then requires. Leave it out to have CapyCTL
  # measure it; `capyctl status deployment <name> --format json` shows it.
  content_fingerprint: "qwen3-4b-1"
  revision: "1"
devices:
  - id: gpu0
    sharing: shared
engine_config:
  memory:
    # GPU memory the model may use while it runs: weights plus KV cache.
    request: "12GiB"

Deployment on one host

# capyctl deployment on one host (`capyctl deploy model --file deployment-single.yaml`).
# Parsed by the product: `capyctl validate config --file docs/examples/deployment-single.yaml
# --host docs/examples/host.yaml` also resolves it against that host.
# An illustration of the schema, not a tested engine recipe.
schema_version: 1
kind: deployment
name: coding-small
# Shorthand for `placement: {hosts: [gpu-box]}`.
host: gpu-box
routes: ["coding-small"]
runtime_profile: vllm
runtime_profile_revision: 1
recipe: standard
# `deep` releases weights and KV cache when parked; `restart_only` never parks.
residency: deep
recovery: reconcile
model:
  # Relative to the host's model store.
  path: coding-small-r1
  content_fingerprint: "sha256:coding-small-r1"
  revision: r1
devices:
  - id: gpu0
    sharing: shared
# Typed engine settings. Memory phases are derived from `memory` and the
# checkpoint's weights when `resources` is omitted.
engine_config:
  dtype: bfloat16
  context_length: 65536
  max_concurrent_requests: 16
  memory:
    request: "40GiB"
    kv_cache: "12GiB"
  vllm:
    block_size_tokens: 16
    # auto (default): chosen by model family; none; or the engine's parser name.
    tool_call_parser: auto
    reasoning_parser: auto
request_deadline: "900s"
# Derived from the checkpoint's size when omitted; never beyond the request deadline.
timeouts:
  initialize: "600s"
  wake: "300s"

Deployment on several hosts

# capyctl deployment with two instances spread over two hosts. Each instance is a
# whole engine on one host. To run one engine across hosts, see
# deployment-multinode.yaml (a group, docs/guide/several-machines.md).
# Parsed by the product: `capyctl validate config --file docs/examples/deployment-spread.yaml
# --host docs/examples/host.yaml` also resolves it against that host.
# An illustration of the schema, not a tested engine recipe.
schema_version: 1
kind: deployment
name: coding-large
instances: 2
placement:
  hosts: ["gpu-box", "workstation"]
  # Only hosts whose labels match are candidates.
  selector:
    accelerator: example-gpu
  strategy: spread
  max_per_host: 1
routes: ["coding-large"]
runtime_profile: sglang
runtime_profile_revision: 1
recipe: standard
residency: deep
recovery: reconcile
model:
  source:
    type: local
    path: /srv/models/coding-large-r1
  content_fingerprint: "sha256:coding-large-r1"
  revision: r1
# Devices are not named (`id`): a named device requires exactly one allowed
# host. Each instance takes a shared device from its own host.
devices:
  - sharing: shared
engine_config:
  dtype: bfloat16
  context_length: 32768
  memory:
    request: "72GiB"
    kv_cache: "16GiB"
  sglang:
    chunked_prefill_size: 8192
    # auto (default): chosen by model family; none; or the engine's parser name.
    tool_call_parser: auto
    reasoning_parser: none
request_deadline: "1800s"
timeouts:
  initialize: "1200s"
  wake: "600s"

One model across two hosts

# One model across two hosts: a group of two tensor-parallel ranks, one per
# host (docs/guide/several-machines.md, "One model across machines"). The first
# host in placement.hosts is the head: it runs rank 0 and serves the API. The
# second runs rank 1. Each named host must declare its peer address
# (resource_policy.groups.peer_address in docs/examples/host.yaml and
# docs/examples/host-b.yaml) and publish the `vllm` profile with the same build.
# Parsed by the product: `capyctl validate config --file docs/examples/deployment-multinode.yaml
# --host docs/examples/host.yaml --host docs/examples/host-b.yaml` resolves it on both hosts.
# An illustration of the schema, not a tested engine recipe.
name: qwen3-30b-tp2
engine: vllm
# The same files on both hosts; vLLM accepts a different path on each. Only a
# SGLang group with `residency: deep` needs the same path on every host.
model: qwen3-30b-a3b
topology:
  tensor_parallel: 2
  pipeline_parallel: 1
placement:
  hosts: ["gpu-box", "host-b"]
residency: deep
# Per member: each host charges its own rank this much.
engine_config:
  memory:
    request: "40GiB"
timeouts:
  initialize: "1800s"

Host with unified memory

# capyctl host role document (`capyctl start host --config host.yaml`) for a
# unified-memory host (one memory pool shared by device and system).
# Parsed by the product: `capyctl validate config --file docs/examples/host.yaml`.
# An illustration of the schema, not a tested engine recipe: executables,
# fingerprints and limits are placeholders for the host's own values.
schema_version: 1
kind: host
name: gpu-box
state_dir: /var/lib/capyctl/host
# Must be `<state_dir>/identity`.
identity_dir: /var/lib/capyctl/host/identity
# capyctl's runtime helpers are compiled into the binary and written, owner-only,
# to <state_dir>/runtime at every start. Name runtime_dir only to run from a
# directory you maintain yourself (development); capyctl never writes to it.
# runtime_dir: /home/operator/capyctl/runtime
# The host's own installation may also be named here instead of (or besides)
# runtime_profiles; it is published as the profile `local` (both engines:
# `local-vllm` and `local-sglang`). --vllm-bin or CAPYCTL_VLLM_BIN name it too
# (docs/operations/configuration.md).
# local_engine:
#   vllm: /opt/vllm/bin/vllm
#   cuda_home: /usr/local/cuda   # or --cuda-home, CAPYCTL_CUDA_HOME
hardware_fingerprint: example-hw-1
environment_fingerprint: example-env-1
# Relative model paths in deployments resolve under this directory.
model_store:
  path: /srv/models
# The private ingress the server forwards inference to (loopback or a
# 100.64.0.0/10 overlay-network address, such as a Tailscale tailnet).
ingress:
  transport: trusted_private_link
  address: "http://100.64.0.21:8444"
  bind: "100.64.0.21:8444"
load_report_interval: "1s"
shutdown:
  drain_timeout: "30s"
resource_policy:
  domains:
    unified:
      memory: unified
      managed_limit: "96GiB"
      free_reserve: "12GiB"
      parked_limit: "16GiB"
      host_kv_limit: "8GiB"
  devices:
    gpu0:
      domain: unified
      sharing: shared
  device_sharing: shared
  max_parked: 3
  observation_ttl: "2s"
  endpoint_port_range:
    start: 8100
    end: 8199
  planner_max_states: 4096
  queue:
    admission_window: "2s"
    max_pending_per_deployment: 64
    max_pending_total: 256
    max_buffered_bytes_total: "64MiB"
    request_deadline: "1800s"
    stream_idle_timeout: "120s"
  # Deployments select hosts by these labels (`placement.selector`).
  labels:
    site: lab
    accelerator: example-gpu
  # One model across machines (docs/guide/several-machines.md): the address
  # this host's group peers reach it on, over the direct link. A host without
  # it is never named in a group. Also `--peer-address` or
  # CAPYCTL_PEER_ADDRESS. docs/examples/host-b.yaml is the second host.
  groups:
    peer_address: 192.0.2.10
    # The head's rendezvous ports (default 25000-25099), and whether a missing
    # memlock or InfiniBand setting refuses a group (default false: warn).
    # rendezvous_port_range: {start: 25000, end: 25099}
    # require_rdma: false
# Engine installations. Engine tuning belongs to each deployment's
# `engine_config`; a profile keeps host-fixed settings only.
runtime_profiles:
  vllm:
    engine: vllm
    revision: 1
    executable: /opt/vllm/bin/vllm
    build_fingerprint: "vllm 0.31.0"
    args: []
    # Optional: the CUDA toolkit whose bin/ joins the engine PATH (vLLM needs
    # nvcc there to use FlashInfer). Omit it to keep the minimal PATH.
    cuda_home: /usr/local/cuda
    # Optional overrides of the JIT build limits capyctl computes at launch
    # (MAX_JOBS = free memory, less the deployment's Ready charge on unified
    # memory, / 8 GiB, at most the CPU count, at least 1).
    env: {}
    log_policy:
      max_file_bytes: "16MiB"
      retained_files: 3
    security:
      # Deep parking is on unless a host opts out with `disabled`.
      deep_park: enabled
      trust_remote_code: false
      credential_ref: "secret://engine-key"
      # `warn` (default) or `refuse` when the installation's files change
      # after the agent registered them.
      installation_drift: warn
  sglang:
    engine: sglang
    revision: 1
    executable: /opt/sglang/bin/python3
    build_fingerprint: "sglang 0.5.21"
    args: []
    env: {}
    log_policy:
      max_file_bytes: "16MiB"
      retained_files: 3
    security:
      deep_park: enabled
      trust_remote_code: false
      credential_ref: "secret://engine-key"
      admin_credential_ref: "secret://admin-key"
      installation_drift: refuse

Second host of a group

# The second host of the group in docs/examples/deployment-multinode.yaml
# (`capyctl start host --config host-b.yaml`): the same engines, under the same
# names and builds, as docs/examples/host.yaml, and its own peer address on the
# direct link between the two hosts. See docs/guide/several-machines.md, "One
# model across machines".
# Parsed by the product: `capyctl validate config --file docs/examples/host-b.yaml`.
# An illustration of the schema, not a tested engine recipe: executables,
# fingerprints and limits are placeholders for the host's own values.
schema_version: 1
kind: host
name: host-b
state_dir: /var/lib/capyctl/host
identity_dir: /var/lib/capyctl/host/identity
hardware_fingerprint: example-hw-1
environment_fingerprint: example-env-1
model_store:
  path: /srv/models
ingress:
  transport: trusted_private_link
  address: "http://100.64.0.22:8444"
  bind: "100.64.0.22:8444"
resource_policy:
  domains:
    unified:
      memory: unified
      managed_limit: "96GiB"
      free_reserve: "12GiB"
      parked_limit: "16GiB"
      host_kv_limit: "8GiB"
  devices:
    gpu0:
      domain: unified
      sharing: shared
  device_sharing: shared
  max_parked: 3
  observation_ttl: "2s"
  endpoint_port_range:
    start: 8100
    end: 8199
  planner_max_states: 4096
  queue:
    admission_window: "2s"
    max_pending_per_deployment: 64
    max_pending_total: 256
    max_buffered_bytes_total: "64MiB"
    request_deadline: "1800s"
    stream_idle_timeout: "120s"
  labels:
    site: lab
    accelerator: example-gpu
  # The address host A's ranks reach this host on (also `--peer-address` or
  # CAPYCTL_PEER_ADDRESS).
  groups:
    peer_address: 192.0.2.11
runtime_profiles:
  vllm:
    engine: vllm
    revision: 1
    executable: /opt/vllm/bin/vllm
    build_fingerprint: "vllm 0.31.0"
    args: []
    cuda_home: /usr/local/cuda
    env: {}
    log_policy:
      max_file_bytes: "16MiB"
      retained_files: 3
    security:
      deep_park: enabled
      trust_remote_code: false
      credential_ref: "secret://engine-key"
      installation_drift: warn
  sglang:
    engine: sglang
    revision: 1
    executable: /opt/sglang/bin/python3
    build_fingerprint: "sglang 0.5.21"
    args: []
    env: {}
    log_policy:
      max_file_bytes: "16MiB"
      retained_files: 3
    security:
      deep_park: enabled
      trust_remote_code: false
      credential_ref: "secret://engine-key"
      admin_credential_ref: "secret://admin-key"
      installation_drift: refuse

Host with a discrete GPU

# capyctl host role document (`capyctl start host --config host.yaml`) for a
# discrete-GPU host (one 24 GB NVIDIA card; device memory is its own domain).
# Host RAM is the `system` domain and the card's memory is the `gpu0` domain;
# every deployment is charged on both.
# Parsed by the product: `capyctl validate config --file docs/examples/host-discrete.yaml`.
# An illustration of the schema, not a tested engine recipe: executables,
# fingerprints and limits are placeholders for the host's own values.
schema_version: 1
kind: host
name: desk-gpu
state_dir: /var/lib/capyctl/host
# Must be `<state_dir>/identity`.
identity_dir: /var/lib/capyctl/host/identity
# capyctl's runtime helpers are compiled into the binary and written, owner-only,
# to <state_dir>/runtime at every start. Name runtime_dir only to run from a
# directory you maintain yourself (development); capyctl never writes to it.
# runtime_dir: /home/operator/capyctl/runtime
# The host's own installation may also be named here instead of (or besides)
# runtime_profiles; it is published as the profile `local` (both engines:
# `local-vllm` and `local-sglang`). --vllm-bin or CAPYCTL_VLLM_BIN name it too
# (docs/operations/configuration.md).
# local_engine:
#   vllm: /opt/vllm/bin/vllm
#   cuda_home: /usr/local/cuda   # or --cuda-home, CAPYCTL_CUDA_HOME
hardware_fingerprint: example-hw-1
environment_fingerprint: example-env-1
# Relative model paths in deployments resolve under this directory.
model_store:
  path: /srv/models
# The private ingress the server forwards inference to (loopback or a
# 100.64.0.0/10 overlay-network address, such as a Tailscale tailnet).
ingress:
  transport: trusted_private_link
  address: "http://100.64.0.21:8444"
  bind: "100.64.0.21:8444"
load_report_interval: "1s"
shutdown:
  drain_timeout: "30s"
resource_policy:
  domains:
    system:
      memory: distinct
      managed_limit: "32GiB"
      free_reserve: "12GiB"
      parked_limit: "16GiB"
      host_kv_limit: "8GiB"
    gpu0:
      memory: device
      device: gpu0
      managed_limit: "22GiB"
      free_reserve: "2GiB"
      parked_limit: "2GiB"
  # host_backed parks keep weights here: size parked_limit on `system` for them.
  devices:
    gpu0:
      domain: gpu0
      sharing: shared
  device_sharing: shared
  max_parked: 3
  observation_ttl: "2s"
  endpoint_port_range:
    start: 8100
    end: 8199
  planner_max_states: 4096
  queue:
    admission_window: "2s"
    max_pending_per_deployment: 64
    max_pending_total: 256
    max_buffered_bytes_total: "64MiB"
    request_deadline: "1800s"
  # Deployments select hosts by these labels (`placement.selector`).
  labels:
    site: lab
    accelerator: example-gpu
# Engine installations. Engine tuning belongs to each deployment's
# `engine_config`; a profile keeps host-fixed settings only.
runtime_profiles:
  vllm:
    engine: vllm
    revision: 1
    executable: /opt/vllm/bin/vllm
    build_fingerprint: "vllm 0.31.0"
    args: []
    # Optional: the CUDA toolkit whose bin/ joins the engine PATH (vLLM needs
    # nvcc there to use FlashInfer). Omit it to keep the minimal PATH.
    cuda_home: /usr/local/cuda
    # Optional overrides of the JIT build limits capyctl computes at launch
    # (MAX_JOBS = free memory / 8 GiB, at most the CPU count, at least 1).
    env: {}
    log_policy:
      max_file_bytes: "16MiB"
      retained_files: 3
    security:
      # Deep parking is on unless a host opts out with `disabled`.
      deep_park: enabled
      trust_remote_code: false
      credential_ref: "secret://engine-key"
      # `warn` (default) or `refuse` when the installation's files change
      # after the agent registered them.
      installation_drift: warn
  sglang:
    engine: sglang
    revision: 1
    executable: /opt/sglang/bin/python3
    build_fingerprint: "sglang 0.5.21"
    args: []
    env: {}
    log_policy:
      max_file_bytes: "16MiB"
      retained_files: 3
    security:
      deep_park: enabled
      trust_remote_code: false
      credential_ref: "secret://engine-key"
      admin_credential_ref: "secret://admin-key"
      installation_drift: refuse

Server

Unset, a timer in lifecycle_defaults is off: no idle model is stopped or parked.

# capyctl server role document (`capyctl start server --config server.yaml`).
# Parsed by the product: `capyctl validate config --file docs/examples/server.yaml`.
# An illustration of the schema, not a tested deployment of any engine.
schema_version: 1
kind: server
name: lab
state_dir: /var/lib/capyctl/server
# Must be `<state_dir>/identity`.
identity_dir: /var/lib/capyctl/server/identity
listeners:
  # Management stays on loopback. Inference listens on all interfaces and
  # requires the API key; `--listen 127.0.0.1:8443` or this bind narrows it
  # (docs/operations/network-access.md).
  management:
    bind: "127.0.0.1:7443"
    authentication: token
  inference:
    bind: "0.0.0.0:8443"
    authentication: api_key
  # Host agents enroll through bootstrap and hold their control session here.
  bootstrap:
    bind: "100.64.0.10:7444"
    authentication: server_tls
  control:
    bind: "100.64.0.10:7445"
    authentication: mutual_tls
enrollment:
  # One certificate name for both addresses.
  bootstrap_address: "https://capyctl.lab:7444"
  control_address: "https://capyctl.lab:7445"
lifecycle_defaults:
  ready_idle_timeout: "5m"
  parked_idle_timeout: "30m"
control:
  heartbeat_suspend_after: "5s"
  heartbeat_lost_after: "30s"
shutdown:
  drain_timeout: "30s"

Standalone

capyctl start standalone writes this file itself on first start, and reads the GPU to size memory. You only need your own to change its defaults.

# capyctl standalone role document: an embedded server and host on one machine,
# with authenticated listeners. `capyctl start standalone` generates this shape
# under its state directory when none exists. The engine installation is
# named here (`host.local_engine`), by a flag (--vllm-bin) or by a variable
# (CAPYCTL_VLLM_BIN); a flag wins over a variable, which wins over this file
# (docs/operations/configuration.md lists every setting). Inference listens on
# all interfaces and requires the API key; management stays on loopback
# (docs/operations/network-access.md). The listeners serve plain HTTP, so
# there is no `tls` block: use a private network or a TLS reverse proxy.
# Models live in ~/models unless `host.model_store.path` (or --models-root,
# CAPYCTL_MODELS_ROOT) names another directory; Hugging Face and HTTP downloads
# are allowed with a 500 GiB cap unless `host.model_sources` says otherwise.
# Parsed by the product: `capyctl validate config --file docs/examples/standalone.yaml`.
schema_version: 1
kind: standalone
name: local
server:
  name: local
  state_dir: /home/operator/.local/state/capyctl/server
  listeners:
    management:
      bind: "127.0.0.1:7443"
      authentication: admin_token
    inference:
      bind: "0.0.0.0:8443"
      authentication: api_key
host:
  name: local
  state_dir: /home/operator/.local/state/capyctl/host
  connection: embedded
  resource_policy:
    allowed_devices: auto
    memory:
      accounting: auto
      system:
        managed_limit: auto   # 50 % of memory; or a size ("90GiB") or a share ("75%")
        free_reserve: auto    # 20 % of memory; or a size or a share
        parked_limit: auto    # 25 % of memory, what parked models may hold; a size or a share, at most the managed limit
    parked_growth_limit: auto # how far one launch's parked charge may grow past its first park before
                              # its next park is a stop: 100 % of that charge; a share ("50%"), a size ("8GiB") or off
    # Optional request queue bounds, as on a host (defaults shown):
    # queue:
    #   request_deadline: "1800s"     # until the first output, prefill included
    #   stream_idle_timeout: "120s"   # longest silence after the first output
  runtime_profiles: {}
  # Optional (defaults shown in comments):
  # model_store:
  #   path: /home/operator/models
  # model_sources:
  #   huggingface: allowed      # or disabled
  #   http: allowed             # or disabled
  #   max_bytes: 500GiB
  # local_engine:
  #   vllm: /opt/vllm/bin/vllm  # or sglang: /opt/sglang/bin/python3
  #   deep_park: "on"           # or "off"
  #   installation_drift: warn  # or refuse
  # runtime_dir: /home/operator/capyctl/runtime   # development only
shutdown:
  drain_timeout: "30s"