Skip to content

[Feature] Docs RAG MCP — index any URL into a single-source-of-truth via S3 + Lambda + OpenSearch #8034

Description

@salawhaaat

Kiro Product

IDE

Feature Description

A built-in workflow that lets users point Kiro at any documentation URL, crawl and index it on AWS-native infrastructure, and expose the result as an MCP search_docs tool — creating a single source of truth for AI-assisted queries inside Kiro.

Proposed architecture:

  1. URL input — user provides one or more docs URLs (e.g. https://docs.anthropic.com) via a Kiro command or settings panel.
  2. Crawler Lambda — scrapes the site (respecting sitemap.xml and robots.txt), downloads raw HTML/Markdown, and writes it to an S3 bucket partitioned by source domain + crawl timestamp.
  3. Indexer Lambda — triggered by S3 ObjectCreated events; chunks documents, generates embeddings via Amazon Bedrock Titan, and upserts vectors into Amazon OpenSearch Serverless.
  4. MCP service — a Lambda Function URL exposing a search_docs(query, source?) tool that performs semantic RAG retrieval against OpenSearch and returns ranked passages with metadata (URL, section title, score).
  5. Kiro integration — the MCP server is auto-registered in Kiro so agents and the assistant can call search_docs directly during coding/Q&A.

Source steering files:

Each indexed source can include a <source>.mcp.md steering file — following the same pattern as Kiro's existing steering and skills system — that configures RAG behavior per-domain:

# docs.anthropic.com.mcp.md

## Source metadata
name: Anthropic Docs
base_url: https://docs.anthropic.com
priority: high

## Chunking strategy
chunk_size: 512
overlap: 64
split_on: [h2, h3, code_block]

## Query hints
prefer_sections: [API Reference, Guides]
boost_keywords: [claude, model, token, tool_use]
exclude_paths: [/changelog, /legacy]

## MCP tool behavior
tool_description: "Search Anthropic's official API and SDK documentation"
response_format: passage_with_url
max_results: 5

This gives the search_docs tool source-aware context — better relevance, tighter chunking, and no generic one-size-fits-all retrieval. Since it follows the steering/skills convention already familiar to Kiro users, adding or tuning a source feels native rather than a separate config system.

Use Case

When working in Kiro with an unfamiliar SDK or internal docs site, I want to ask questions like "how do I configure auth in X library?" and have the assistant retrieve the actual docs rather than hallucinating. With a single URL registration, all agents in Kiro share the same indexed knowledge base — no per-agent duplication, no stale context.

Additional Context

  • All infra is AWS-native (Lambda, S3, OpenSearch Serverless, Bedrock) — fits naturally with existing AWS integrations.
  • Re-indexing should be triggerable on demand or on a schedule (e.g. nightly cron via EventBridge).
  • OpenSearch Serverless handles scaling; no EC2 or persistent cluster needed.
  • The MCP endpoint should support filtering by source domain so multi-source indexes don't bleed across projects.
  • I'd love to participate in implementing this feature. My GitHub is salawhaaat — feel free to add me as a collaborator or assign me to the issue. I can contribute a reference CDK/SAM template, the Lambda indexer, or the MCP service layer — happy to start wherever is most useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentationidemcp

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions