Control what runs next.
Run more models than your GPU can hold. Deploy, park, wake and switch models on vLLM, SGLang, TensorFold, llama.cpp or a server of your own, behind one OpenAI-compatible endpoint.
More models than memory
Most home GPUs hold one or two models at a time. CapyCTL parks the idle ones, releases their GPU memory, and wakes the one a request asks for, behind the same endpoint. Your apps keep one base URL and pick a model by name; CapyCTL decides what has to move to make room.
$ capyctl list deployments
NAME STATE READY REVISION HOSTS
coding-small ready 1/1 3 gpu-box
chat-large ready 1/1 1 workstation
vision-small parked 0/1 2 gpu-box
How it works
What can you keep ready?
| Memory | Typical setup |
|---|---|
| Gaming card 24 GB | One 8B model at 16-bit or one 14B model at 4-bit stays live. Everything else is parked and wakes on request. |
| High-end card 32 GB | One 8B model at 16-bit or one 32B model at 4-bit stays live. Everything else is parked and wakes on request. |
| Workstation card 48 GB | One 14B model at 16-bit or one 32B model at 4-bit stays live, or 2 8B models at once. Everything else is parked and wakes on request. |
| Workstation card 96 GB | One 32B model at 16-bit or one 70B model at 4-bit stays live, or 4 8B models at once. Everything else is parked and wakes on request. |
| Unified-memory box 128 GB | One 32B model at 16-bit or one 70B model at 4-bit stays live, or 5 8B models at once. Everything else is parked and wakes on request. |
| Several machines | Each machine as above. The server places each model on a machine with room and routes requests there. A model too large for one machine can run across two, one engine rank on each (several machines). |
Illustrative, from model weight sizes. KV cache and engine overhead vary with context length and settings. On one machine a model runs on one GPU. Not a list of supported cards.
Engines and platforms
| Engines | vLLM, SGLang, TensorFold, llama.cpp, or any other server as a custom engine |
|---|---|
| Hosts | Linux on x86-64 and ARM64, NVIDIA GPUs, directly or in a container |
| Models | Anything your engine can load |
| API | OpenAI-compatible, streaming, tool calls, decision models, and a request ID on every answer |
| Privacy | Engine logs can stay in memory, so no prompt or answer reaches the disk (data retention) |
| Automation | A management API your own program can drive (controller guide) |
Bring your own engine installation. CapyCTL manages the runtime around it.
Install
$ curl -fsSL https://edurdias.github.io/capyctl/install.sh | sh
$ capyctl engine add ~/venvs/vllm
$ capyctl start standalone
$ capyctl deploy model --file my-model.yaml --activate --waitOne static binary. Bring your own engine; weights come from a directory or Hugging Face. Run on one machine