|
| 1 | +--- |
| 2 | +sidebar_label: QVAC |
| 3 | +description: Run local-first, peer-to-peer AI models with QVAC and connect them to Roo Code through its OpenAI-compatible server. |
| 4 | +keywords: |
| 5 | + - QVAC |
| 6 | + - local models |
| 7 | + - Roo Code |
| 8 | + - OpenAI compatible |
| 9 | + - peer-to-peer AI |
| 10 | + - local-first |
| 11 | + - offline AI |
| 12 | + - gpt-oss |
| 13 | + - tool calling |
| 14 | +--- |
| 15 | + |
| 16 | +# Using QVAC With Roo Code |
| 17 | + |
| 18 | +[QVAC](https://qvac.com) is an open-source runtime for local-first, peer-to-peer AI. It can expose your local models through an OpenAI-compatible HTTP server, letting you connect them to Roo Code using the **OpenAI Compatible** provider. |
| 19 | + |
| 20 | +**Website:** [https://qvac.com](https://qvac.com) |
| 21 | + |
| 22 | +--- |
| 23 | + |
| 24 | +## Setting Up QVAC |
| 25 | + |
| 26 | +1. **Install the QVAC CLI:** |
| 27 | + |
| 28 | + ```bash |
| 29 | + npm i -g @qvac/cli |
| 30 | + ``` |
| 31 | + |
| 32 | +2. **Define a model alias:** Create a `qvac.config.json` that maps a serve alias to a model. The alias you choose here is the model id you will enter in Roo Code. |
| 33 | + |
| 34 | + ```json |
| 35 | + { |
| 36 | + "serve": { |
| 37 | + "models": { |
| 38 | + "gpt-oss-20b": { |
| 39 | + "model": "GPT_OSS_20B_INST_Q4_K_M", |
| 40 | + "preload": true, |
| 41 | + "config": { |
| 42 | + "ctx_size": 32768, |
| 43 | + "tools": true |
| 44 | + } |
| 45 | + } |
| 46 | + } |
| 47 | + } |
| 48 | + } |
| 49 | + ``` |
| 50 | + |
| 51 | + Two settings matter when using QVAC as a coding agent: |
| 52 | + * **`ctx_size`** defaults to `1024`, which is far too small for agent prompts. Set it explicitly (e.g. `32768`). |
| 53 | + * **`tools: true`** enables function/tool calling. Roo Code relies on native tool calling, so without this the model returns plain text instead of tool calls. |
| 54 | + |
| 55 | +3. **Start the server:** |
| 56 | + |
| 57 | + ```bash |
| 58 | + qvac serve openai |
| 59 | + ``` |
| 60 | + |
| 61 | + This starts an OpenAI-compatible REST API on port `11434` by default (use `--port` to change it). Your base URL is `http://127.0.0.1:11434/v1`. |
| 62 | + |
| 63 | +--- |
| 64 | + |
| 65 | +## Configuration in Roo Code |
| 66 | + |
| 67 | +1. **Open Roo Code Settings:** Click the gear icon (<Codicon name="gear" />) in the Roo Code panel. |
| 68 | +2. **Select Provider:** Choose "OpenAI Compatible" from the "API Provider" dropdown. |
| 69 | +3. **Enter Base URL:** Use `http://127.0.0.1:11434/v1` (or the port you set with `--port`). |
| 70 | +4. **Enter API Key:** QVAC's server does not validate the key, but the field is required—enter any non-empty string (e.g. `qvac`). |
| 71 | +5. **Enter Model ID:** Use the serve alias from your `qvac.config.json` (e.g. `gpt-oss-20b`). |
| 72 | +
|
| 73 | +--- |
| 74 | +
|
| 75 | +## Tips and Notes |
| 76 | +
|
| 77 | +* **Use a capable, agent-tuned model.** Tool-calling quality is bounded by the model you run. Small models often fail to invoke tools reliably; a larger agent-tuned model such as `gpt-oss-20b` is a good local default. |
| 78 | +* **Set the context window explicitly.** The QVAC LLM `ctx_size` default of `1024` is too small for Roo Code's prompts. Set it to something like `32768` in `qvac.config.json`. |
| 79 | +* **Enable tools.** Roo Code uses native tool calling exclusively. Set `"tools": true` in the model config or the model will respond with text instead of tool calls. |
| 80 | +* **Reasoning models.** For reasoning-tuned models such as Qwen3, set `"reasoning_budget": 0` in the model config unless you specifically want extended reasoning. |
| 81 | +* **Preload for a faster first response.** Setting `"preload": true` loads the model when the server starts, avoiding a cold start on your first request. |
| 82 | +* **Resource requirements.** Running large language models locally is resource-intensive. Make sure your machine can handle the model and context size you choose. |
| 83 | + |
| 84 | +--- |
| 85 | + |
| 86 | +## Troubleshooting |
| 87 | + |
| 88 | +* **"Model Not Found":** The model id in Roo Code must exactly match a serve alias defined in `qvac.config.json`. |
| 89 | +* **Model replies with text instead of using tools:** Add `"tools": true` to the model's `config` in `qvac.config.json` and restart the server. |
| 90 | +* **Context overflow or truncated prompts:** Increase `ctx_size` (the default `1024` is too small for agent prompts). |
| 91 | +* **Connection errors:** Confirm `qvac serve openai` is running and that the Base URL and port match (`http://127.0.0.1:11434/v1` by default). |
0 commit comments