Supplementary code for the paper:
"A Domain-Agnostic Agentic Architecture for Structured Extraction of Engineering Knowledge" Oscar Ikechukwu, Mehdi Tarkian, Sanjay Nambiar, Marie Jonsson, Christoffer Brax Division of Product Realization, IEI, Linköping University Funded by Vinnova DART project (grant 2024-01420)
This repository contains the implementation of the RCP (Relational Control Plane) framework, a SQL-backed agentic architecture that persists orchestration state as queryable relational records and enforces a six-stage verify-then-summarise control loop. Synthesis is permitted only after retrieved evidence satisfies validation constraints, transforming potential hallucinations into explicit, auditable failures. Two case studies are included: Case I (Hydroscand hydraulic product catalog, n=100 queries) and Case II (Company B aerospace connector/cable catalog, n=100 queries).
| Layer | Role | Key Components |
|---|---|---|
| Layer 1 | Extraction pipeline | PDF rendering, VLM-based parsing, SQLite schema |
| Layer 2 | Reasoning engine | LangGraph workflow, 6-stage control loop, function library |
| Layer 3 | User interface | Flask web app, CLI entry point |
├── Layer_1_Extraction/
│ └── Case_I/ # Extraction pipeline for Case I (Hydroscand)
│ ├── Layer_1a/ # Legacy extraction (reference)
│ └── Layer_1b/ # Production extraction pipeline
├── Layer_2_Agentic_Reasoning/ # Core RCP reasoning framework
│ ├── config/ # Configuration: constants, prompts, domain settings
│ ├── db/ # Database connections and strategy templates
│ └── logic/ # State graph, workflow nodes, function library
├── Layer_3_User_Interface/ # Web interface and APIs
├── RCP_notebook/ # Runnable Jupyter walk-throughs (Layers 1–3, Case I demo)
├── Experiments/
│ ├── Case_I/ # Case I evaluation (Hydroscand, n=100 queries)
│ │ ├── Baseline_RAG/ # B1: Naive RAG baseline
│ │ ├── Baseline_SQL_Retrieval/ # B2: SQL retrieval baseline
│ │ ├── RCP_Framework/ # B3: RCP framework evaluation
│ │ ├── compute_mcnemar.py # McNemar's test for statistical significance
│ │ └── test_questions.json # Annotated query set (100 questions)
│ ├── Case_II/ # Case II evaluation (Company B, n=100 queries)
│ │ └── test_questions_company_b.json # Annotated query set (100 questions)
│ └── questions/ # Shared test question sets
├── database/ # SQLite databases and schema
│ ├── harvested.db # Product database (Case I)
│ ├── agentic.db # Workflow state database
│ ├── harvested_schema.sql # Schema definition
│ └── db_utils.py # Database utilities
├── docs/ # Architecture and design documentation
├── tests/ # Unit, integration, and end-to-end tests
├── main.py # CLI entry point
└── run_web.py # Web server launcher
- Python 3.12
- Ollama running locally — provides every model used by the framework: a vision-language model for Layer 1 extraction,
llama3.2for Layer 2 reasoning, andqwen3-embedding:8bfor embeddings. All generation and embedding run on-device; no commercial API keys are required.
git clone https://github.com/oscik559/RCP-Framework.git
cd RCP-Framework
pip install -r requirements.txt
# or install in editable mode
pip install -e .
# install test tooling (required to run pytest commands below)
pip install -e ".[test]"Create a .env file at the project root:
SECRET_KEY=<generate-with: python -c "import secrets; print(secrets.token_urlsafe(48))">
FLASK_DEBUG=false# Production pipeline (Layer 1b)
python Layer_1_Extraction/Case_I/Layer_1b/0_extract_knowledge.py
python Layer_1_Extraction/Case_I/Layer_1b/2b_extract_categories.py
python Layer_1_Extraction/Case_I/Layer_1b/3a_extract_families.py
python Layer_1_Extraction/Case_I/Layer_1b/3b_extract_products_vlm.pyRequires Ollama running locally (ollama serve) with a vision model.
See Layer_1_Extraction/Case_I/Layer_1b/README.md for the full pipeline.
CLI:
python main.pyEdit the query directly in main.py. Debug verbosity is controlled by debug_level (0 = silent, 4 = verbose).
Web interface:
python run_web.py
# Open http://localhost:5001Three runnable Jupyter notebooks walk through the framework layer by layer on Case I (Hydroscand) data, inlining the production code so you can step through and edit it live (they ship with the pre-populated harvested.db, so most cells run read-only).
01_layer1_extraction.ipynb— the six-stage extraction pipeline (PDF catalog →harvested.db). Read-only by default; re-running extraction is opt-in behind aRERUN_EXTRACTIONflag and needs Ollama + a vision model (ollama pull qwen2-vl).02_agentic_reasoning.ipynb— the RCP engine: buildsagentic.db, seeds 6 strategies + 10 functions, wires the 7-node LangGraph state machine, and runs queries end-to-end with the verify-then-summarise judge gate. Needs Ollama onlocalhost:11434withllama3.2:latest+nomic-embed-text:latest(a §1.5 cell can install/pull these).03_layer3_user_interface.ipynb— starts the Flask + SSE web app over Layer 2 and exercises its/query,/progress/<id>, and/result/<id>endpoints. Same Ollama setup as Notebook 2.
Scope: these are an illustrative single-domain demo on Case I only (and deliberately lighter than the repo) — not the full 200-query evaluation (Case I + Case II, n=100 each), which lives in
Experiments/.
The experiment folders contain the three baselines (B1: Naive RAG, B2: SQL Retrieval, B3: RCP) and the annotated query sets used in the paper.
# Run Case I evaluation
cd Experiments/Case_I
python run_evaluation.py
# Run Case II evaluation
cd Experiments/Case_II
python run_evaluation_company_b.py
# Statistical significance (McNemar's test)
python Experiments/Case_I/compute_mcnemar.pyResults are written to results/ within each experiment folder.
Install test dependencies first if you used only -r requirements.txt or pip install -e .:
python -m pip install -e ".[test]"# All tests
python -m pytest tests/
# Unit tests only
python -m pytest tests/unit/
# With coverage
python -m pytest tests/ --cov=Layer_2_Agentic_ReasoningThe pre-populated product database for Case I is included at database/harvested.db. The schema is documented in database/harvested_schema.sql and database/README.md.
# Inspect or reinitialize the database
python database/db_utils.py --help- SETUP.md — detailed setup instructions (Windows, macOS, Linux)
- docs/graph.png — workflow state graph
- docs/Case_I/STRATEGY_ARCHITECTURE.md — RCP strategy design (Case I)
- docs/Case_I/GENERIC_FUNCTIONS_SUMMARY.md — function library reference
- docs/Case_I/SPEC_GLOSSARY.md — product database attribute glossary
See CONTRIBUTING.md for setup instructions, code conventions, and how to submit changes.
Creative Commons Attribution 4.0 International (CC-BY 4.0) — see LICENSE.