Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ jobs:
.venv/bin/python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
.venv/bin/python -m pip install ".[dev]" "transformers==${{ matrix.transformers }}"
.venv/bin/python -m pip check
.venv/bin/python -c 'import rcc, sys; from pathlib import Path; assert Path(rcc.__file__).is_relative_to(Path(sys.prefix))'
.venv/bin/python -c 'import rclc, sys; from pathlib import Path; assert Path(rclc.__file__).is_relative_to(Path(sys.prefix))'
scripts/install-hooks.sh
- name: Harness
run: .venv/bin/python scripts/check.py
Expand Down
4 changes: 2 additions & 2 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,9 @@ the same quality.

Write a callable that follows the
[selector contract](docs/selectors.md#custom-selection) and pass it to
`rcc.transfer(..., selector=my_selector)`.
`rclc.transfer(..., selector=my_selector)`.

For a contribution to the package, put the function in `src/rcc/selectors/`.
For a contribution to the package, put the function in `src/rclc/selectors/`.
Keep it small and leave sender state unchanged. Extend an existing end-to-end
scenario to cover selection, delivery and receiver continuation; avoid a test
per helper. Follow the [development harness](docs/development.md) and run:
Expand Down
82 changes: 42 additions & 40 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# RCLC
# CacheBack

**Receiver-conditioned latent communication for agents.**

Code for *Receiver-Conditioned Latent Communication gives 94% CacheBack*.
**[Website](https://maxr0ssi.github.io/rclc/)** · **[Paper](https://arxiv.org/abs/2609.32046)** · [Python API](https://github.com/maxr0ssi/rclc/blob/main/docs/api.md) · [Paper replication](https://github.com/maxr0ssi/rclc/blob/main/docs/replication.md)
**[Website](https://agentcacheback.github.io/)** · **[Paper](https://arxiv.org/abs/2609.32046)** · [Python API](https://github.com/agentcacheback/cacheback/blob/main/docs/api.md) · [Paper replication](https://github.com/agentcacheback/cacheback/blob/main/docs/replication.md)

Agents distribute large contexts across senders and receivers. Text messages
require decoding and can omit evidence. Full KV-cache messages accumulate every
Expand All @@ -12,17 +12,17 @@ sender's context at the receiver, increasing memory use and context length.
in a request. **CacheBack** is a training-free selector that uses attention to
that request to choose which sender positions enter the handoff.

![FanOutQA accuracy and latency across four model families, alongside receiver-conditioned selection of sender state.](https://raw.githubusercontent.com/maxr0ssi/rclc/main/docs/assets/cacheback-overview.png)
![FanOutQA accuracy and latency across four model families, alongside receiver-conditioned selection of sender state.](https://raw.githubusercontent.com/agentcacheback/cacheback/main/docs/assets/cacheback-overview.png)

*Highest-accuracy CacheBack setting per family versus same-size text on FanOutQA,
with 50 concurrent tasks on eight H100 GPUs. Operating points differ by family.
Right: the receiver query guides which sender positions enter the handoff.*

## Watch the demo

**[▶ Watch the 36-second coding demo](https://maxr0ssi.github.io/rclc/#demo)**
**[▶ Watch the 36-second coding demo](https://agentcacheback.github.io/#demo)**

[![CacheBack finishes a Django fix while text workers are still writing reports.](demo/coding/poster.jpg)](https://maxr0ssi.github.io/rclc/#demo)
[![CacheBack finishes a Django fix while text workers are still writing reports.](demo/coding/poster.jpg)](https://agentcacheback.github.io/#demo)

Seven Qwen3-8B workers help a coordinator fix a Django bug. CacheBack completes
this recorded case in **25.66 s**, versus **113.21 s** for text: **4.41× faster**.
Expand All @@ -31,13 +31,13 @@ test grading are excluded from the clocks. The silent video reconstructs the
interface from separate runs and varies playback speed. This is one case,
not an aggregate benchmark. [Recorded results](demo/coding/evidence.json).

Also explore the [interactive booking relay](https://maxr0ssi.github.io/rclc/demo/)
Also explore the [interactive booking relay](https://agentcacheback.github.io/demo/)
to inspect selected positions and text messages at each handoff. Its
[demo guide](demo/README.md) covers recording your own.

## Install

The distribution is named `rclc`; the Python import is `rcc`. Until the PyPI
Install and import the package as `rclc`. Until the PyPI
release, install from a checkout:

```bash
Expand All @@ -54,21 +54,21 @@ python -m pip install .
```

The engine needs the capture connector; see the
[vLLM setup](https://github.com/maxr0ssi/rclc/blob/main/docs/api.md#vllm-default) and
[integration example](https://github.com/maxr0ssi/rclc/blob/main/examples/vllm_transfer.py).
[vLLM setup](https://github.com/agentcacheback/cacheback/blob/main/docs/api.md#vllm-default) and
[integration example](https://github.com/agentcacheback/cacheback/blob/main/examples/vllm_transfer.py).
vLLM 0.26.0 is an explicit opt-in. Both pins passed a Qwen3-8B GPU journey on an
A100 ([results](https://github.com/maxr0ssi/rclc/blob/main/docs/development.md#gpu-validation)).
A100 ([results](https://github.com/agentcacheback/cacheback/blob/main/docs/development.md#gpu-validation)).

## Use it

Bind existing Hugging Face Qwen3 agents by their loaded model and tokenizer:

```python
import rcc
import rclc

agent_x = rcc.bind(model, tokenizer, messages=history_x, backend="hf")
agent_y = rcc.bind(model, tokenizer, messages=history_y, backend="hf")
await rcc.transfer(agent_x, agent_y, "Who owns Cedar?")
agent_x = rclc.bind(model, tokenizer, messages=history_x, backend="hf")
agent_y = rclc.bind(model, tokenizer, messages=history_y, backend="hf")
await rclc.transfer(agent_x, agent_y, "Who owns Cedar?")

inputs = agent_y.pop()
output = model.generate(**inputs, return_dict_in_generate=True)
Expand All @@ -82,53 +82,53 @@ supported.

| Supply state with | Use when |
| --- | --- |
| `messages=history` | You have chat messages; RCC applies the chat template. |
| `messages=history` | You have chat messages; RCLC applies the chat template. |
| `prompt=text_or_ids` | You have rendered text or exact token IDs. |
| `prompt=ids, past_key_values=cache` | You already have an HF cache. |
| `generation=output` | You have a native HF generation result. |
| `prompt=ids, request_id=capture_id` | You already captured the vLLM prompt. |
| Neither | Receive first, or supply sender state later with `update`. |

- Outside an async function or notebook, use `rcc.transfer_sync(...)` with the
- Outside an async function or notebook, use `rclc.transfer_sync(...)` with the
same options. Native generation and `update` block; finish each operation
before reusing the model
([concurrency](https://github.com/maxr0ssi/rclc/blob/main/docs/api.md#concurrency)).
- vLLM: `rcc.bind(llm, backend="vllm", messages=history)` with the
[engine setup](https://github.com/maxr0ssi/rclc/blob/main/docs/api.md#vllm-default).
([concurrency](https://github.com/agentcacheback/cacheback/blob/main/docs/api.md#concurrency)).
- vLLM: `rclc.bind(llm, backend="vllm", messages=history)` with the
[engine setup](https://github.com/agentcacheback/cacheback/blob/main/docs/api.md#vllm-default).
Local dense Qwen3 weights are required; hosted chat APIs are not supported.
- `await agent_y.append(new_text_or_ids)` adds a tool result without losing
received state; `await agent_y.inputs()` continues locally; `save`/`load`
snapshot retained state
([continuation and snapshots](https://github.com/maxr0ssi/rclc/blob/main/docs/api.md#continue-append-and-resume)).
([continuation and snapshots](https://github.com/agentcacheback/cacheback/blob/main/docs/api.md#continue-append-and-resume)).
- `agent_y.inspect()` shows queued handoffs; `record_selection=True` plus
`agent_y.selection_html()` shows what survived compression
([selection inspection](https://github.com/maxr0ssi/rclc/blob/main/docs/api.md#inspect-check-and-view-selection)).
- `python -m rcc doctor` (`--backend vllm` for CUDA) or `rcc.check(agent_y)`
([selection inspection](https://github.com/agentcacheback/cacheback/blob/main/docs/api.md#inspect-check-and-view-selection)).
- `python -m rclc doctor` (`--backend vllm` for CUDA) or `rclc.check(agent_y)`
checks setup without running a model or downloading weights.

Examples: [quickstart](https://github.com/maxr0ssi/rclc/blob/main/examples/quickstart.py),
[cached conversation](https://github.com/maxr0ssi/rclc/blob/main/examples/existing_agents.py),
[Colab GPU notebook](https://github.com/maxr0ssi/rclc/blob/main/examples/colab.ipynb).
Examples: [quickstart](https://github.com/agentcacheback/cacheback/blob/main/examples/quickstart.py),
[cached conversation](https://github.com/agentcacheback/cacheback/blob/main/examples/existing_agents.py),
[Colab GPU notebook](https://github.com/agentcacheback/cacheback/blob/main/examples/colab.ipynb).

### Options

```python
from functools import partial

await rcc.transfer(senders, receivers, requests) # Relative r4 (default)
await rcc.transfer(senders, receivers, requests, ratio=16) # Keep 1/16 of new positions
await rcc.transfer(senders, receivers, requests, budget=1024) # Bounded message size
await rcc.transfer(senders, receivers, requests, selector=rcc.selectors.qsnap)
await rcc.transfer(senders, receivers, requests,
reasoning=partial(rcc.latent_mass, steps=50), budget=128)
await rclc.transfer(senders, receivers, requests) # Relative r4 (default)
await rclc.transfer(senders, receivers, requests, ratio=16) # Keep 1/16 of new positions
await rclc.transfer(senders, receivers, requests, budget=1024) # Bounded message size
await rclc.transfer(senders, receivers, requests, selector=rclc.selectors.qsnap)
await rclc.transfer(senders, receivers, requests,
reasoning=partial(rclc.latent_mass, steps=50), budget=128)
```

Each argument is one item or a list: two senders, two receivers and two requests
produce four deliveries of two messages each. See
[budgets](https://github.com/maxr0ssi/rclc/blob/main/docs/api.md#relative-and-bounded-budgets),
[selectors](https://github.com/maxr0ssi/rclc/blob/main/docs/selectors.md),
[reasoning](https://github.com/maxr0ssi/rclc/blob/main/docs/api.md#reasoning) and
[representations](https://github.com/maxr0ssi/rclc/blob/main/docs/api.md#representations).
[budgets](https://github.com/agentcacheback/cacheback/blob/main/docs/api.md#relative-and-bounded-budgets),
[selectors](https://github.com/agentcacheback/cacheback/blob/main/docs/selectors.md),
[reasoning](https://github.com/agentcacheback/cacheback/blob/main/docs/api.md#reasoning) and
[representations](https://github.com/agentcacheback/cacheback/blob/main/docs/api.md#representations).

## Results in the paper

Expand All @@ -142,10 +142,10 @@ the paper's benchmark results, not performance claims for this package.
## Bring your own selector or benchmark

Pass any callable as `selector=` and compare it with CacheBack using the
[small evaluation script](https://github.com/maxr0ssi/rclc/blob/main/CONTRIBUTING.md#2-run-a-small-comparison).
[small evaluation script](https://github.com/agentcacheback/cacheback/blob/main/CONTRIBUTING.md#2-run-a-small-comparison).
The paper's **FanOutQA** and **LongBench v2** panels need separate integration.
New benchmarks and negative results are welcome; see
[Contributing](https://github.com/maxr0ssi/rclc/blob/main/CONTRIBUTING.md).
[Contributing](https://github.com/agentcacheback/cacheback/blob/main/CONTRIBUTING.md).

## Branches

Expand All @@ -155,9 +155,9 @@ New benchmarks and negative results are welcome; see
| `paper` | Four-family runtimes, both benchmarks, twelve experiment configs, pinned serving stacks, data bundles |

The `paper-v1` tag fixes the replication snapshot; see the
[replication guide](https://github.com/maxr0ssi/rclc/blob/main/docs/replication.md).
[replication guide](https://github.com/agentcacheback/cacheback/blob/main/docs/replication.md).
For checks, tests and GPU validation, see the
[development guide](https://github.com/maxr0ssi/rclc/blob/main/docs/development.md).
[development guide](https://github.com/agentcacheback/cacheback/blob/main/docs/development.md).

## License and citation

Expand All @@ -169,6 +169,8 @@ Apache-2.0. Data licenses and replication requirements are documented on the
title = {Receiver-Conditioned Latent Communication gives 94\% CacheBack},
author = {Rossi, Maximillian and Raghunath, Prajwal and Xuan, Haoqing and Zhang, Yusen and Wu, Eugene},
year = {2026},
note = {Preprint}
eprint = {2609.32046},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2609.32046}
}
```
10 changes: 5 additions & 5 deletions demo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Harbour Hub, whose passport requirement overrides the original desk's ID advice.
The [booking section](../index.html#booking-demo), below the main coding video, presents the recorded final answers
and completion times before the replay. Selected positions use teal highlights.

Open <https://maxr0ssi.github.io/rclc/demo/> and press **Run**. To serve it
Open <https://agentcacheback.github.io/demo/> and press **Run**. To serve it
locally, run this from the repository root and open <http://127.0.0.1:8765/demo/>:

```bash
Expand Down Expand Up @@ -50,7 +50,7 @@ animation.

## Record

Install RCC as described in the [repository README](../README.md#install), then
Install RCLC as described in the [repository README](../README.md#install), then
run:

```bash
Expand All @@ -70,12 +70,12 @@ GPU with a high-RAM host. Prepare its source bundle from a committed checkout:

```bash
mkdir -p output/colab
git archive --format=zip --output=output/colab/rcc-demo-colab-source.zip HEAD
git archive --format=zip --output=output/colab/rclc-demo-colab-source.zip HEAD
```

Upload that ZIP when prompted, mount Drive and run the cells in order. Each run
saves its source, configuration, environment, complete logs and traces to a new
`MyDrive/rcc-demo/<run-id>/` folder. The notebook retains its width comparison
`MyDrive/rclc-demo/<run-id>/` folder. The notebook retains its width comparison
for analysis; the viewer shows W=4 only. The last cell downloads a portable
replay; serve its extracted directory with the command above. Replay needs no
model.
Expand All @@ -88,7 +88,7 @@ handoff alone. Later agents cannot reread discarded evidence.

Text generates new notes; each text column is the actual message generated at
that step, not a summary written for the viewer. RCLC sends selected input
embeddings through `rcc.transfer_sync`, with no extra latent steps. Highlights
embeddings through `rclc.transfer_sync`, with no extra latent steps. Highlights
follow exact tokenizer offsets, including word fragments. As another document
arrives, selection can drop evidence retained at the previous handoff.

Expand Down
18 changes: 9 additions & 9 deletions demo/colab.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"id": "cell-0",
"metadata": {},
"source": [
"# Record the RCC relay with Qwen3-8B\n",
"# Record the RCLC relay with Qwen3-8B\n",
"\n",
"This notebook runs the existing demo recorder at **W=16 and W=4**, with the same\n",
"r4 budget, questions and source documents. It preserves unedited answers,\n",
Expand All @@ -18,16 +18,16 @@
" **Runtime > Change runtime type**. An L4 or A100 is the intended target.\n",
" This is an unquantized BF16/FP16 run. The notebook checks GPU memory before\n",
" downloading weights; a 16 GB T4 is too small for this configuration.\n",
"2. Upload the separate `rcc-demo-colab-source.zip` when prompted, then allow\n",
"2. Upload the separate `rclc-demo-colab-source.zip` when prompted, then allow\n",
" Drive mounting. This does not require a public repository or a GitHub token.\n",
"3. Run the remaining cells in order. Results go to a unique directory under\n",
" `MyDrive/rcc-demo/`. The final cell downloads a portable replay ZIP.\n",
" `MyDrive/rclc-demo/`. The final cell downloads a portable replay ZIP.\n",
"\n",
"Build the source archive from the RCC repository after committing your changes:\n",
"Build the source archive from the RCLC repository after committing your changes:\n",
"\n",
"```bash\n",
"mkdir -p output/colab\n",
"git archive --format=zip --output=output/colab/rcc-demo-colab-source.zip HEAD\n",
"git archive --format=zip --output=output/colab/rclc-demo-colab-source.zip HEAD\n",
"```\n",
"\n",
"Colab hardware availability and session lifetime vary; see the\n",
Expand Down Expand Up @@ -86,22 +86,22 @@
" )\n",
" uploaded = files.upload()\n",
" if len(uploaded) != 1:\n",
" raise ValueError(\"Upload exactly one RCC source ZIP.\")\n",
" raise ValueError(\"Upload exactly one RCLC source ZIP.\")\n",
" SOURCE_ZIP = Path(next(iter(uploaded)))\n",
" drive.mount(\"/content/drive\")\n",
" OUTPUT_ROOT = Path(\"/content/drive/MyDrive/rcc-demo\")\n",
" OUTPUT_ROOT = Path(\"/content/drive/MyDrive/rclc-demo\")\n",
" DEVICE = \"cuda\"\n",
"\n",
"run_id = datetime.now(timezone.utc).strftime(\"%Y%m%dT%H%M%SZ\") + \"-\" + uuid.uuid4().hex[:8]\n",
"RUN = OUTPUT_ROOT / run_id\n",
"RUN.mkdir(parents=True)\n",
"SOURCE = Path(tempfile.mkdtemp(prefix=\"rcc-demo-source-\"))\n",
"SOURCE = Path(tempfile.mkdtemp(prefix=\"rclc-demo-source-\"))\n",
"with zipfile.ZipFile(SOURCE_ZIP) as archive:\n",
" if any(Path(name).is_absolute() or \"..\" in Path(name).parts for name in archive.namelist()):\n",
" raise ValueError(\"Source ZIP contains an unsafe path.\")\n",
" archive.extractall(SOURCE)\n",
"if not (SOURCE / \"demo/record.py\").is_file() or not (SOURCE / \"pyproject.toml\").is_file():\n",
" raise ValueError(\"Expected a git archive of the RCC repository root.\")\n",
" raise ValueError(\"Expected a git archive of the RCLC repository root.\")\n",
"shutil.copyfile(SOURCE_ZIP, RUN / \"source.zip\")\n",
"(RUN / \"config.json\").write_text(\n",
" json.dumps(\n",
Expand Down
2 changes: 1 addition & 1 deletion demo/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ <h1 id="question">Where should Maya collect her pass on Friday, and what ID shou
<p class="sr-only" id="status" role="status" aria-live="polite"></p>
<footer>
<p><span id="run-info">Recorded replay</span> · <span id="timing-note">Estimated streaming, measured completion times.</span></p>
<a href="https://github.com/maxr0ssi/rclc/tree/main/demo">Code &amp; recording</a>
<a href="https://github.com/agentcacheback/cacheback/tree/main/demo">Code &amp; recording</a>
</footer>
</main>
<script>
Expand Down
4 changes: 2 additions & 2 deletions demo/record.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,8 @@
import transformers
from transformers import AutoModelForCausalLM, AutoTokenizer

from rcc import Delivery, SenderState, transfer_sync
from rcc.selectors import cacheback
from rclc import Delivery, SenderState, transfer_sync
from rclc.selectors import cacheback

DOCUMENTS = [
{
Expand Down
Loading
Loading