OSII API overview

Purpose

This document provides a practical overview of the current AI-Ready-Ingest backend API.

It is intended for developers and downstream consumers who need to:

discover backend capabilities
inspect canonical corpus resources
manage curated collections
access grounded text spans
retrieve derived enrichments
perform search over extracted content
run ingestion and embedding workflows

This overview is descriptive and implementation-aligned. Exact request and response schemas should be defined in openapi.yaml.

API direction

The backend API is resource-oriented and centered on core backend capabilities.

User-facing dashboard rendering does not live in this repository.

Downstream consumers should use:

scope resources
collection resources
object resources
text-span resources
enrichment resources
search resources

Compatibility read routes under /api/osii/... still exist, but new integrations should prefer the newer resource-oriented route families.

Base assumptions

By default the application commonly runs on:

http://localhost:8511

The OSII store is typically located at:

.\osii-data\.osii

The shared data root is typically:

.\osii-data\source

Canonical collection metadata is stored inside the OSII store, typically at:

.\osii-data\.osii\collections\<collection-id>\collection.toml

The disposable .osii\state\catalog.sqlite3 database accelerates API reads and can always be rebuilt from canonical files.

Collection metadata must survive:

extraction reruns
synthesis reruns
embedding rebuilds

Collection membership must reference stable object identifiers such as file_id, not only transient filesystem paths.

Core endpoint families

Scopes

GET /api/scopes/root
GET /api/scopes/folders
GET /api/scopes/collections
POST /api/scopes/describe

Collections

GET /api/collections
POST /api/collections
GET /api/collections/{collection_id}
PATCH /api/collections/{collection_id}
DELETE /api/collections/{collection_id}
GET /api/collections/{collection_id}/members
POST /api/collections/{collection_id}/members
DELETE /api/collections/{collection_id}/members/{file_id}
GET /api/collections/{collection_id}/export
POST /api/packages/import

Objects

GET /api/objects/{file_id}
GET /api/objects/{file_id}/manifest
GET /api/objects/{file_id}/texts
GET /api/objects/{file_id}/texts/preferred
GET /api/objects/{file_id}/syntheses
GET /api/objects/{file_id}/governance
PUT /api/objects/{file_id}/governance
POST /api/objects/{file_id}/deletion-preview
DELETE /api/objects/{file_id}

Text spans

GET /api/text/objects/{file_id}/span
GET /api/text/objects/{file_id}/span/context

Enrichments

POST /api/enrichments/list
GET /api/enrichments/objects/{file_id}/{filename}

Search

GET /api/search

Ingestion and derived-data workflows

GET /api/intake/readiness
GET /api/browse
POST /api/resolve
GET /api/extractors
GET /api/synthesizers
GET /api/folder-synthesizers
POST /api/runs
GET /api/runs/{run_id}
GET /api/runs/{run_id}/logs
POST /api/embeddings/build
GET /api/embeddings/build/{job_id}
GET /api/embeddings/meta

Setup and local administration

GET /api/admin/setup returns the human-facing readiness summary, selected methods, provider presence, and capability-service state.
PUT /api/admin/model-providers/{provider_id}/credential accepts an API key as a write-only value when local .env writes are enabled.
DELETE /api/admin/model-providers/{provider_id}/credential forgets a key previously saved by OSII.
GET /api/admin/services lists allowlisted capability services.
POST /api/admin/services/{service_id}/{start|stop|restart} controls only a service owned by the local launcher.
GET /api/admin/services/{service_id}/logs returns a bounded recent log tail.

Credential responses report presence and source but never return the value. The service-control endpoints return 503 in deployments without the local loopback supervisor.

Compatibility and artifact serving

compatibility read routes under /api/osii/...
GET /artifact/{file_path:path}

General conventions

Stable identifiers

The API prefers stable identifiers in canonical read paths:

file_id
folder_id
collection_id
run_id
job_id

Relative path conventions

Where relevant, source-relative paths are exposed as canonical relpaths rooted at the shared data root, for example:

reports/FY26/Q2_Status_Report.pdf

Canonical versus derived data

Downstream consumers should distinguish between:

canonical stored artifacts, such as object text and manifest spans
derived artifacts, such as enrichments and syntheses
retrieval/index artifacts, such as embedding chunks and FAISS identifiers

The resource model and search semantics documents define these distinctions more precisely.

Scope resources

Scope resources provide a uniform way to describe root, folder, collection, and object scopes.

Root scope

GET /api/scopes/root

Example response:

{
  "scope": {
    "scope_type": "root",
    "scope_id": "root",
    "label": "root"
  },
  "member_file_ids": [
    "sha256-test123"
  ]
}

Folder scope catalog

GET /api/scopes/folders

This returns a flat catalog of folder scope descriptors.

Example response:

{
  "scopes": [
    {
      "scope_type": "folder",
      "scope_id": "folder-root",
      "folder_id": "folder-root",
      "path": "",
      "label": "root-folder"
    }
  ]
}

Collection scope catalog

GET /api/scopes/collections

This returns collection scope descriptors, not full collection resources.

Example response:

{
  "scopes": [
    {
      "scope_type": "collection",
      "scope_id": "col-abc123def456",
      "collection_id": "col-abc123def456",
      "label": "sensor-calibration-review",
      "kind": "file-list",
      "description": "Documents related to calibration methods and drift analysis.",
      "document_count": 2
    }
  ]
}

Describe a scope

POST /api/scopes/describe
Content-Type: application/json

Supported request variants:

root
folder by folder_id
collection by collection_id
object by file_id

Example request:

{
  "scope_type": "collection",
  "collection_id": "col-abc123def456"
}

Example response:

{
  "scope": {
    "scope_type": "collection",
    "scope_id": "col-abc123def456",
    "collection_id": "col-abc123def456",
    "label": "sensor-calibration-review"
  },
  "member_file_ids": [
    "sha256-test123"
  ]
}

Folder scope behavior is subtree-based for filtering and search semantics.

Collection resources

Collections are curated groupings of documents. They are not canonical OSII hierarchy nodes.

List collections

GET /api/collections

Example response:

{
  "collections": [
    {
      "id": "col-abc123def456",
      "name": "sensor-calibration-review",
      "description": "Documents related to calibration methods and drift analysis.",
      "kind": "file-list",
      "color": "#3366ff",
      "document_count": 2,
      "created_utc": "2026-06-18T12:00:00Z",
      "updated_utc": "2026-06-18T12:05:00Z"
    }
  ]
}

Create a collection

POST /api/collections
Content-Type: application/json

Required fields:

name

Optional fields:

description
kind
color

Example request:

{
  "name": "sensor-calibration-review",
  "description": "Documents related to calibration methods and drift analysis.",
  "kind": "file-list",
  "color": "#3366ff"
}

Example response:

{
  "collection": {
    "id": "col-abc123def456",
    "name": "sensor-calibration-review",
    "description": "Documents related to calibration methods and drift analysis.",
    "kind": "file-list",
    "color": "#3366ff",
    "document_count": 0,
    "created_utc": "2026-06-18T12:00:00Z",
    "updated_utc": "2026-06-18T12:00:00Z"
  }
}

Update a collection

PATCH /api/collections/{collection_id}
Content-Type: application/json

Writable fields:

name
description
kind
color

Example request:

{
  "description": "Updated description",
  "kind": "manual"
}

Read one collection

GET /api/collections/{collection_id}

Delete one collection

DELETE /api/collections/{collection_id}

Read collection membership

GET /api/collections/{collection_id}/members

Example response:

{
  "collection": {
    "id": "col-abc123def456",
    "name": "sensor-calibration-review",
    "description": "Documents related to calibration methods and drift analysis.",
    "kind": "file-list",
    "color": "#3366ff",
    "document_count": 2,
    "created_utc": "2026-06-18T12:00:00Z",
    "updated_utc": "2026-06-18T12:05:00Z"
  },
  "file_ids": [
    "sha256-test123"
  ]
}

Add members to a collection

POST /api/collections/{collection_id}/members
Content-Type: application/json

Membership addition is batch-oriented and effectively idempotent.

Remove a member from a collection

DELETE /api/collections/{collection_id}/members/{file_id}

Removal is success/no-op style. If the collection exists but the document is not currently a member, the response reports removed: false rather than treating that case as an error.

Export and import an OSII package

GET /api/collections/{collection_id}/export
POST /api/packages/import

Export returns a ZIP containing collection and object sidecars plus a versioned osii-package.json checksum manifest; it never includes source files. Import uses multipart field package, validates every member before writing, merges objects by immutable file_id, unions governance labels, and invalidates corpus-wide derived indexes. See sensitive data and transfer.

Object resources

Object resources provide the preferred read surface for canonical object metadata, text access, syntheses, and related summaries.

Read or update governance metadata

GET /api/objects/{file_id}/governance
PUT /api/objects/{file_id}/governance

The JSON fields are sensitivity_labels, tags, and handling_notes. They are stored canonically in objects/<file_id>/governance.toml and communicate awareness; they do not enforce access control.

Preview and delete one object

POST /api/objects/{file_id}/deletion-preview
DELETE /api/objects/{file_id}

Preview accepts mode sidecar_only or source_and_sidecar. Delete requires the same mode, the returned preview_token, and confirmation exactly equal to the file ID. Active Intake jobs or a stale preview block deletion.

Read object aggregate

GET /api/objects/{file_id}

Example response:

{
  "file_id": "sha256-test123",
  "meta": {
    "file": {
      "source_relpath": "reports/example.pdf",
      "filename": "example.pdf",
      "mime": "application/pdf",
      "size_bytes": 1234,
      "mtime_utc": "2026-05-21T00:00:00Z"
    },
    "hash": {
      "sha256": "test123"
    }
  },
  "overview": {
    "file_id": "sha256-test123",
    "meta": {
      "file": {
        "source_relpath": "reports/example.pdf",
        "filename": "example.pdf",
        "mime": "application/pdf",
        "size_bytes": 1234,
        "mtime_utc": "2026-05-21T00:00:00Z"
      },
      "hash": {
        "sha256": "test123"
      }
    },
    "text_count": 1,
    "image_count": 0,
    "has_synth": true,
    "text_items": [
      {
        "kind": "text",
        "id": "seg-000001",
        "path": "text.txt",
        "type": "page",
        "span": {
          "char_start": 0,
          "char_end": 38
        },
        "source_origin": {
          "source_type": "pdf",
          "unit_type": "page",
          "page": 1
        },
        "related_ids": []
      }
    ],
    "image_items": []
  },
  "collections": [
    {
      "id": "col-abc123def456",
      "name": "sensor-calibration-review",
      "kind": "file-list"
    }
  ],
  "processing": {
    "extractor": {
      "name": null,
      "display_name": null
    },
    "synthesizer": {
      "name": null,
      "display_name": null
    },
    "canonical_text_path": "objects/sha256-test123/text.txt",
    "editable_text_path": null,
    "has_editable_text": false,
    "capabilities": {
      "supports_markdown_render": false
    }
  },
  "enrichments": [
    {
      "name": "keywords--stats_keywords.json",
      "kind": "file",
      "relpath": "objects/sha256-test123/enrichments/keywords--stats_keywords.json"
    }
  ]
}

Read object manifest

GET /api/objects/{file_id}/manifest

Example response:

{
  "file_id": "sha256-test123",
  "records": [
    {
      "kind": "text",
      "id": "seg-000001",
      "path": "text.txt",
      "type": "page",
      "span": {
        "char_start": 0,
        "char_end": 38
      },
      "source_origin": {
        "source_type": "pdf",
        "unit_type": "page",
        "page": 1
      },
      "related_ids": []
    }
  ]
}

List object text representations and canonical segments

GET /api/objects/{file_id}/texts

Example response:

{
  "file_id": "sha256-test123",
  "representations": [
    {
      "name": "canonical",
      "kind": "canonical_extracted_text",
      "path": "objects/sha256-test123/text.txt",
      "exists": true,
      "preferred": true
    },
    {
      "name": "editable",
      "kind": "editable_text",
      "path": "objects/sha256-test123/editable_text.txt",
      "exists": false,
      "preferred": false
    }
  ],
  "segments": [
    {
      "kind": "text",
      "id": "seg-000001",
      "path": "text.txt",
      "type": "page",
      "span": {
        "char_start": 0,
        "char_end": 38
      },
      "source_origin": {
        "source_type": "pdf",
        "unit_type": "page",
        "page": 1
      },
      "related_ids": []
    }
  ]
}

Read preferred text representation

GET /api/objects/{file_id}/texts/preferred

Example response:

{
  "file_id": "sha256-test123",
  "representation": "canonical",
  "kind": "canonical_extracted_text",
  "text": "Thermal calibration drift was reduced.",
  "path": "objects/sha256-test123/text.txt"
}

If editable text exists, it may become the preferred representation.

Read available syntheses

GET /api/objects/{file_id}/syntheses

Example response:

{
  "file_id": "sha256-test123",
  "current_text": "This appears to be a technical report about thermal calibration drift.",
  "current_toml": {
    "path": {
      "source_relpath": "reports/example.pdf"
    },
    "synthesis": {
      "synthesis": "Technical report about thermal calibration drift.",
      "doc_type": "technical report",
      "quality": "default"
    },
    "details": {
      "description": "This appears to be a technical report about thermal calibration drift."
    }
  },
  "syntheses": [
    {
      "name": "current",
      "text_path": "C:\\...\\.osii\\objects\\sha256-test123\\synth.txt",
      "toml_path": "C:\\...\\.osii\\objects\\sha256-test123\\synth.toml",
      "scope": "object"
    }
  ]
}

Text-span resources

Text-span routes provide canonical grounding over object text stored in:

objects/<file_id>/text.txt

Read a canonical text span

GET /api/text/objects/{file_id}/span?char_start=0&char_end=10

Example response:

{
  "file_id": "sha256-test123",
  "char_start": 0,
  "char_end": 10,
  "text": "Thermal ca"
}

Notes:

char_start and char_end are required query parameters
char_end is effectively exclusive
spans are validated against canonical object text
current invalid-range behavior returns a JSON error body with HTTP 200 rather than strict 400 or 404

Read span context

GET /api/text/objects/{file_id}/span/context?char_start=0&char_end=10&context_chars=5

Example response:

{
  "file_id": "sha256-test123",
  "char_start": 0,
  "char_end": 10,
  "match_text": "Thermal ca",
  "before_text": "",
  "after_text": "librat",
  "window_start": 0,
  "window_end": 15
}

context_chars is optional and currently defaults to 200.

Enrichment resources

Enrichments are optional, derived, durable-on-disk outputs associated with scopes or objects.

They are not canonical extraction artifacts.

List enrichments for a scope

POST /api/enrichments/list
Content-Type: application/json

Supported request variants:

root
folder
collection
object

Example request:

{
  "scope_type": "object",
  "file_id": "sha256-test123"
}

Example response:

{
  "scope": {
    "scope_type": "object",
    "file_id": "sha256-test123"
  },
  "enrichments": [
    {
      "name": "keywords--stats_keywords.json",
      "kind": "file",
      "relpath": "objects/sha256-test123/enrichments/keywords--stats_keywords.json"
    }
  ]
}

Read an object enrichment payload

GET /api/enrichments/objects/{file_id}/{filename}

This currently returns JSON-wrapped object enrichment payloads rather than arbitrary raw bytes.

Example response:

{
  "file_id": "sha256-test123",
  "filename": "keywords--stats_keywords.json",
  "relpath": "objects/sha256-test123/enrichments/keywords--stats_keywords.json",
  "data": {
    "keywords": [
      {
        "term": "thermal",
        "count": 1
      }
    ],
    "input_object_count": 1
  }
}

Search

Public search route

GET /api/search?q=thermal calibration drift&top_k=5

Example response:

{
  "query": "thermal calibration drift",
  "top_k": 5,
  "results": [
    {
      "faiss_id": 0,
      "chunk_id": "chunk-sha256-test123-000001",
      "file_id": "sha256-test123",
      "source_relpath": "reports/example.pdf",
      "chunk_method": "paragraph",
      "chunk_index": 1,
      "char_start": 0,
      "char_end": 38,
      "source_text_representation": "canonical",
      "source_text_kind": "canonical_extracted_text",
      "truncated": false,
      "score": 0.8123
    }
  ]
}

The current public route is a simple semantic search endpoint.

Richer scope-aware search behavior exists in backend service logic, but should not be treated as public route behavior unless and until it is exposed as a formal route.

Search result semantics

Search results may be ranked using derived embedding chunks.

They should be interpreted as retrieval results with canonical grounding information, not as canonical extraction segment identifiers.

Downstream consumers should use returned object identity and text-span grounding fields for navigation and text interaction.

In particular:

faiss_id is an internal retrieval identifier
chunk_id identifies a derived retrieval chunk, not a canonical manifest segment
file_id, char_start, and char_end are the grounding fields relevant for canonical text interaction

Discovery and workflow endpoints

The backend also exposes discovery and orchestration routes for extraction, synthesis, run control, and embeddings.

These include:

GET /api/intake/readiness
GET /api/browse
POST /api/resolve
GET /api/extractors
GET /api/synthesizers
GET /api/folder-synthesizers
POST /api/runs
GET /api/runs/{run_id}
GET /api/runs/{run_id}/logs
POST /api/embeddings/build
GET /api/embeddings/build/{job_id}
GET /api/embeddings/meta

The existing implementation-aligned examples for these routes may be retained, with one update:

embeddings metadata should mention chunking method and chunk manifest as part of current implementation detail

Artifact serving

Serve an OSII-relative artifact

GET /artifact/{file_path:path}

Example:

/artifact/objects/sha256-.../artifacts/artifact-000001.png

This route serves OSII-relative artifact paths only.

It is not arbitrary filesystem serving, and path traversal protections are enforced.

Compatibility

Compatibility read routes under /api/osii/... are still implemented.

New consumers should prefer:

/api/scopes/...
/api/collections/...
/api/objects/...
/api/text/...
/api/enrichments/...

Removed route families include:

/api/dashboard/...
dashboard/workbench HTML routes

See compatibility.md for migration-oriented notes.

Next step

Use this overview together with:

resource-model.md
search-semantics.md
compatibility.md
openapi.yaml

The Markdown documents explain behavior and conventions. The OpenAPI document should define exact schemas, parameters, and response contracts.

Uh oh!

FilesExpand file tree

index.md

Latest commit

History

index.md

File metadata and controls

OSII API overview

Purpose

API direction

Base assumptions

Core endpoint families

Scopes

Collections

Objects

Text spans

Enrichments

Search

Ingestion and derived-data workflows

Setup and local administration

Compatibility and artifact serving

General conventions

Stable identifiers

Relative path conventions

Canonical versus derived data

Scope resources

Root scope

Folder scope catalog

Collection scope catalog

Describe a scope

Collection resources

List collections

Create a collection

Update a collection

Read one collection

Delete one collection

Read collection membership

Add members to a collection

Remove a member from a collection

Export and import an OSII package

Object resources

Read or update governance metadata

Preview and delete one object

Read object aggregate

Read object manifest

List object text representations and canonical segments

Read preferred text representation

Read available syntheses

Text-span resources

Read a canonical text span

Read span context

Enrichment resources

List enrichments for a scope

Read an object enrichment payload

Search

Public search route

Search result semantics

Discovery and workflow endpoints

Artifact serving

Serve an OSII-relative artifact

Compatibility

Next step