Skip to content

Commit 406242e

Browse files
docs: add QVAC provider setup guide
Add a setup page for QVAC, a local-first, peer-to-peer AI runtime that exposes an OpenAI-compatible server via `qvac serve openai`, and connects to Roo Code through the OpenAI Compatible provider. Register it in the providers index so it appears in the sidebar and provider table.
1 parent a676c41 commit 406242e

2 files changed

Lines changed: 97 additions & 0 deletions

File tree

docs/providers/index.json

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,6 +84,12 @@
8484
"extension": true,
8585
"cloud": true
8686
},
87+
{
88+
"id": "providers/qvac",
89+
"title": "QVAC",
90+
"extension": true,
91+
"cloud": false
92+
},
8793
{
8894
"id": "providers/qwen-code",
8995
"title": "Qwen Code CLI",

docs/providers/qvac.md

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
---
2+
sidebar_label: QVAC
3+
description: Run local-first, peer-to-peer AI models with QVAC and connect them to Roo Code through its OpenAI-compatible server.
4+
keywords:
5+
- QVAC
6+
- local models
7+
- Roo Code
8+
- OpenAI compatible
9+
- peer-to-peer AI
10+
- local-first
11+
- offline AI
12+
- gpt-oss
13+
- tool calling
14+
---
15+
16+
# Using QVAC With Roo Code
17+
18+
[QVAC](https://qvac.com) is an open-source runtime for local-first, peer-to-peer AI. It can expose your local models through an OpenAI-compatible HTTP server, letting you connect them to Roo Code using the **OpenAI Compatible** provider.
19+
20+
**Website:** [https://qvac.com](https://qvac.com)
21+
22+
---
23+
24+
## Setting Up QVAC
25+
26+
1. **Install the QVAC CLI:**
27+
28+
```bash
29+
npm i -g @qvac/cli
30+
```
31+
32+
2. **Define a model alias:** Create a `qvac.config.json` that maps a serve alias to a model. The alias you choose here is the model id you will enter in Roo Code.
33+
34+
```json
35+
{
36+
"serve": {
37+
"models": {
38+
"gpt-oss-20b": {
39+
"model": "GPT_OSS_20B_INST_Q4_K_M",
40+
"preload": true,
41+
"config": {
42+
"ctx_size": 32768,
43+
"tools": true
44+
}
45+
}
46+
}
47+
}
48+
}
49+
```
50+
51+
Two settings matter when using QVAC as a coding agent:
52+
* **`ctx_size`** defaults to `1024`, which is far too small for agent prompts. Set it explicitly (e.g. `32768`).
53+
* **`tools: true`** enables function/tool calling. Roo Code relies on native tool calling, so without this the model returns plain text instead of tool calls.
54+
55+
3. **Start the server:**
56+
57+
```bash
58+
qvac serve openai
59+
```
60+
61+
This starts an OpenAI-compatible REST API on port `11434` by default (use `--port` to change it). Your base URL is `http://127.0.0.1:11434/v1`.
62+
63+
---
64+
65+
## Configuration in Roo Code
66+
67+
1. **Open Roo Code Settings:** Click the gear icon (<Codicon name="gear" />) in the Roo Code panel.
68+
2. **Select Provider:** Choose "OpenAI Compatible" from the "API Provider" dropdown.
69+
3. **Enter Base URL:** Use `http://127.0.0.1:11434/v1` (or the port you set with `--port`).
70+
4. **Enter API Key:** QVAC's server does not validate the key, but the field is required—enter any non-empty string (e.g. `qvac`).
71+
5. **Enter Model ID:** Use the serve alias from your `qvac.config.json` (e.g. `gpt-oss-20b`).
72+
73+
---
74+
75+
## Tips and Notes
76+
77+
* **Use a capable, agent-tuned model.** Tool-calling quality is bounded by the model you run. Small models often fail to invoke tools reliably; a larger agent-tuned model such as `gpt-oss-20b` is a good local default.
78+
* **Set the context window explicitly.** The QVAC LLM `ctx_size` default of `1024` is too small for Roo Code's prompts. Set it to something like `32768` in `qvac.config.json`.
79+
* **Enable tools.** Roo Code uses native tool calling exclusively. Set `"tools": true` in the model config or the model will respond with text instead of tool calls.
80+
* **Reasoning models.** For reasoning-tuned models such as Qwen3, set `"reasoning_budget": 0` in the model config unless you specifically want extended reasoning.
81+
* **Preload for a faster first response.** Setting `"preload": true` loads the model when the server starts, avoiding a cold start on your first request.
82+
* **Resource requirements.** Running large language models locally is resource-intensive. Make sure your machine can handle the model and context size you choose.
83+
84+
---
85+
86+
## Troubleshooting
87+
88+
* **"Model Not Found":** The model id in Roo Code must exactly match a serve alias defined in `qvac.config.json`.
89+
* **Model replies with text instead of using tools:** Add `"tools": true` to the model's `config` in `qvac.config.json` and restart the server.
90+
* **Context overflow or truncated prompts:** Increase `ctx_size` (the default `1024` is too small for agent prompts).
91+
* **Connection errors:** Confirm `qvac serve openai` is running and that the Base URL and port match (`http://127.0.0.1:11434/v1` by default).

0 commit comments

Comments
 (0)