🚀 NVIDIA Inception Program Member — ReDevOps is a member of the NVIDIA Inception Program, supporting startups advancing AI and accelerated computing. Membership provides access to NVIDIA technology, technical resources, and the startup ecosystem. It does not imply product endorsement by NVIDIA.
What it does. A "query planner for LLM context": the application states a goal and a budget; the runtime
decides what the model sees (what to retrieve, compress, route and verify), emits an inspectable, replayable
plan (explain, simulate), and learns from the outcome. It is the Context part of ReDevOps Runtime: context
selection and optimization. Thesis and positioning: POSITIONING.md.
Who it's for. Engineers whose application makes recurring choices about which context, config or sources to use (top-k, rerank, which tools, which model tier) and has a measurable outcome to learn from.
Maturity: reference implementation (alpha), simulation-backed. Python is the source of truth. It runs offline with stub models. No production deployment is claimed here, nothing is independently validated, and the Go port lives in a separate private repository that cannot be reviewed from here. Detail: STATUS.md.
git clone --depth 1 https://github.com/redevops-io/context-runtime.git && cd context-runtime
python -m venv .venv && . .venv/bin/activate
pip install -e . # offline stub path; no provider key, no network at run time
PYTHONPATH=. python examples/sidekick_learning.py # learned policy vs fixed baseline on a seeded simulationExpected (seeded; this is what the 2026-10-15 re-run printed): Context Runtime (learning): 67% against
naive baseline (fixed): 33%. This is a simulated acceptance signal, not a measurement on real users.
The API tour is in 30-second tour. This package is not on PyPI (no ReDevOps package is);
install from a clone, or pip install "context-runtime @ git+https://github.com/redevops-io/context-runtime.git".
Every number in this README is the output of a script in examples/, on seeded synthetic data.
Cards with commit, protocol, metrics and limitations: docs/evidence/. Labels used:
Illustrative/synthetic | Internally reproduced | Externally reproduced | Field validated. Nothing in
this repository is externally reproduced or field validated. The tenant table below was re-run on
2026-10-15 and matched; that makes it Illustrative/synthetic, because the environments are simulators.
Simulated rewards, not production outcomes (one sidekick example "simulates the acceptance signal"); single seeded runs unless a table says otherwise; Go parity is not verifiable here; the heterogeneous-shard benchmark needs a corpus that is not in the repo; version identifiers are inconsistent (see STATUS.md).
One public product, ReDevOps Runtime: Mission = governed execution, Context = context selection and optimization (this repo), Discovery = finding and prioritizing opportunities. Agentic Apps are applications using the runtime; Agentic OS is the integrated control-plane/deployment surface, not a fourth runtime. Glossary: agentic-os/docs/GLOSSARY.md.
STATUS · ARCHITECTURE · SPEC · BENCHMARKS · ROADMAP · POSITIONING · SECURITY · CONTRIBUTING · CHANGELOG
It optimizes any app with (a) a decision point about what context/config to use and
(b) a measurable outcome. Eleven example tenants run offline (each number is the
learned-vs-baseline reward its example in examples/ prints on a seeded simulator; Illustrative/synthetic,
cards in docs/evidence/):
| Tenant | Context Runtime tunes | Result |
|---|---|---|
| sidekick | which skills to recall · budget | drop-in for SkillStore; 67% vs 33% naive baseline acceptance |
| redevops-rag | pool · limit · threshold · rerank … per query |
ContextRuntimeRetrieverTuner; 0.780 vs 0.428 vs fixed default |
| edge-sentinel (SOC) | which sources to pull per alert (CrowdSec · threat-intel · EDR) | tool-using + approval-gated; 0.900 vs 0.800 always-full baseline |
| growth-engine | which attribution window + source bundle per lead-source query | 7.851 vs 5.282 vs fixed window |
| control-tower | which Metabase query set per "ask anything" question | 5.326 vs 1.643 vs core query set |
| agentic-billing | which usage/invoice/dunning signals to pull per account | 4.122 vs 2.442 vs full-stack |
| social-autopilot | which channel/timing/content strategy per goal | 3.875 vs 0.773 vs fixed strategy |
| agentic-support | which KB/tickets/account context to retrieve per ticket | 3.679 vs 2.394 vs full-context |
| agentic-books | which ledgers/reports to pull per books question | 3.632 vs 2.430 vs full-books |
| market-radar | which competitor watches to sweep per intel question | 3.611 vs 0.403 vs full-sweep |
| agentic-compliance | which rule-family evidence to pull per finding | 3.562 vs 2.463 vs full-evidence |
Label: Illustrative/synthetic. Reward numbers are single-run outputs of each tenant's
examples/script on simulated environments (the sidekick example simulates the acceptance signal), not production results. Re-run: all eleven rows matched on 2026-10-15 at commit6e8aa19(Python 3.12). They can drift with model or data changes. Units differ per row (acceptance rate, retrieval reward, cumulative reward), so rows are not comparable with each other.
PYTHONPATH=. python examples/sidekick_learning.py # discrete-strategy bandit
PYTHONPATH=. python examples/rag_tuning.py # numeric-knob tuning
PYTHONPATH=. python examples/soc_triage.py # tool-using cybersecurity tenantPlus the ToolPlugin seam (context_runtime/tools/ — how plans reach external systems,
with an approval-gated audit trail) and trace exporters (context_runtime/observability/ exporters.py — JSONL offline, or Langfuse / OpenLLMetry-OTel when the extras are
installed).
Status: runnable reference implementation (Python). Runs fully offline with stub models; real bindings are wired and lazy-imported: LiteLLM models, and redevops-rag / DuckDB / Postgres retrieval (BM25 · dense · hybrid · graph-PPR · community · sharded routing · cross-modal). Beyond the initial slice it ships score calibration, a learned quality-aware planner, and EXPLAIN for the retrieval decision (see Also shipped). See SPEC.md §10 for the conformance checklist these tests assert against. The earlier phrase "Python and Go at feature parity" is withdrawn: the Go port is in a private repository and was not reviewed.
The reward table above is the simulated slice. The fleet below is the set of services in the internal
deployment mirrored at demo.redevops.io; the "15 services" count is from this README's
list and was not independently counted (other repos give different counts: see
agentic-os ARCHITECTURE). Each
service wraps a mature OSS core and exposes /health, a Material-3 dashboard, and its agent endpoints —
the Context Runtime is the decision layer inside each (which context / tools / config per request),
while the core does the domain work:
| App | OSS core | App | OSS core |
|---|---|---|---|
| agentic-billing | Lago | market-radar | changedetection |
| agentic-books | ERPNext | edge-sentinel (SOC) | CrowdSec |
| agentic-support | Chatwoot | agentic-compliance | OpenSCAP |
| agentic-crm | ERPNext (CRM) | control-tower | Metabase |
| social-autopilot | Postiz | growth-engine | Umami |
| lifecycle | Listmonk | guide | redevops-rag |
Plus outreach-engine (Twenty CRM), and growth-assistant + agentic-privacy (both on ERPNext — leads / contacts, not books), and the control plane
(agentic_os: deploy / observe / approve, /m/<app> proxy + a module catalog). Most have the offline
reward examples shown above (examples/<app>.py); the rest are native realizations without a
standalone tuner.
The enterprise line (a separate repository that is not public) adds a commercial layer that composes with this optimizer rather than replacing it: Policy-Constrained Planning — provider / budget / PII / data-residency / mandatory-verification / human-approval predicates define the feasible plan set before the cost model ranks, so every execution can be proven within governance — and Trust-Aware Execution, a trust ledger built from operator acceptance / overrides / regenerations / grounded abstention, so the planner optimizes for the strategy operators will actually rely on. (The mission-runtime line has since shipped — the v6 mission cockpit is live at demo.redevops.io.)
pip install -e . # core (offline stub path, zero heavy deps)
pip install -e ".[litellm]" # real models across 100+ providers
pip install -e ".[rag]" # redevops-rag — single-hop hybrid retrieval
pip install -e ".[hipporag]" # HippoRAG — multi-hop graph retrieval (the planner picks per query)Single-hop vs multi-hop is a per-query decision. The planner classifies intent and
routes: BM25/hybrid (redevops-rag) when the answer is in one chunk, graph (HippoRAG)
when it lives in the connections between documents — and the cost model only pays the
graph premium when it's warranted. python examples/hop_routing.py shows single-hop
missing the bridge document that multi-hop surfaces.
from context_runtime import ContextRuntime, SourceRef
rt = ContextRuntime.default(docs) # offline: stub model + in-memory store
# RUN — the core abstraction (plan → build_context → execute → verify)
res = rt.run("Explain why deployment X failed",
sources=[SourceRef("docs", "docs")],
constraints={"max_cost_usd": 2.0, "require_citations": True})
print(res.answer, res.cost_usd, res.trace)
# EXPLAIN — debug the plan like SQL (add analyze=True for EXPLAIN ANALYZE)
ex = rt.explain("Explain why deployment X failed")
print(ex.intent.bucket, len(ex.candidates), ex.chosen.score.total)
# SIMULATE — forecast cost/latency/tokens with confidence intervals, no execution
sim = rt.simulate("Explain why deployment X failed")
print(sim.expected_cost_usd, sim.expected_models, sim.based_on_samples)Or from the CLI / config:
PYTHONPATH=. python examples/incident_review.py
context-runtime --corpus ./docs run "what's our incident process?"
context-runtime --config context_runtime.yaml explain --analyze "why did deploy X fail?"The six plugin seams and their initial slice; several "real bindings" below are now shipped (see Also shipped).
| Seam (SPEC) | initial slice | Real binding |
|---|---|---|
| Planner trio (intent/candidate/optimizer) | rule-table intent → candidate gen → heuristic cost model | — (the genuinely new core) |
| Cost model + statistics | PlanScore weighted utility + pg_statistic-style calibration |
learned/neural (v0.3+) |
| Optimizer | knapsack / greedy-by-utility over the feasible set | OR-Tools CP-SAT (v0.2) |
| Execution Graph IR | linear graph carrying branch/loop/rollback kinds | full shapes (v0.4) |
| Scheduler | topo-sort waves | Dagster / cost-aware (v2) |
| Reasoner | SingleShotReasoner (one model) |
mixtures: plan-worker-critic (v0.3+) |
| Model plugin | offline StubModel |
LiteLLM + native cost-tiered routing |
| Retriever/Store | InMemoryStore (keyword) |
redevops-rag (DuckDB+BM25+RRF+rerank) |
| Compression | sidekick clip structural pack |
LLMLingua-2 semantic (v0.1 optional) |
| Verifier | citation/grounding check | RAGAS / Instructor |
| Observability | in-process Trace + JSON |
OpenLLMetry → Langfuse |
| Plan Cache | null/always-miss stub | semantic cache (v0.2) |
Beyond the initial slice, in both the Python source-of-truth (a Go port exists in a separate private repository):
- Retrieval as a routable capability — BM25 · dense (fastembed ONNX) · hybrid (RRF) · graph (Personalized-PageRank multi-hop) · community · sharded coverage routing for heterogeneous corpora · cross-modal (CLIP image, ColPali multi-vector, video segments — Python) · quantized ANN (TurboVec) · two-stage cost-gated fusion.
- DSpark-inspired planning — score calibration (isotonic →
P(relevant)), grounded abstention, and a load-aware sizer that prunes the expensive stage. Measured v1→v2 lift in BENCHMARKS.md. - Quality-aware routing — a quality ledger that tracks learned quality apart from cost per intent, so a genuinely better arm (or provider) wins at equal cost. Opt-in.
- EXPLAIN (
context_runtime/explain.py,POST /librechat/explain,examples/explain.py) — the DB EXPLAIN-ANALYZE analogue for the retrieval decision: every candidate arm ranked with its quality/cost decomposition, the per-method trace with calibratedP(relevant), served/abstain, and reward provenance. Read-only. Visualized at redevops.io/planner. - LiteLLM model binding + native cost-tiered routing; DuckDB and Postgres stores.
Runnable results in BENCHMARKS.md (all Illustrative/synthetic: seeded simulations).
Headline: the v1 → v2 table in both runtimes (Python + Go; the Go column is produced by the merged
benchmark script, the Go code itself is not in this repo) — learned-policy precision +14.6 / +11.3 pts, abstention
0 → 100%, expensive-stage depth −62%. Plus retrieval over a heterogeneous financial ×
medical corpus (coverage routing cuts cross-domain pollution 22→0 with recall intact), the
3-index chat memory tenant (+2.93 learned vs read-all-three), and parallel sharded fusion
(5.8× fan-out). Every number is produced by an example in examples/; cards in docs/evidence/.
The pollution result needs a FinanceBench corpus that is not in the repository (the example exits with
"corpus not found"), so it is not reproducible from a clean clone.
The decision layer is thin; the substrate is reused. See:
- ARCHITECTURE.md — the layered design and the cost-based optimizer loop
- SPEC.md — the normative interface contracts (six plugin seams, IR, trace, plan-cache key)
- ROADMAP.md — v0.1 → v2 phasing with per-phase exit benchmarks
pip install -e ".[dev]" numpy httpx && pytest # 349 collected: 338 passed, 11 skipped (2026-10-15); test_conformance.py == SPEC §10