diff --git a/CHANGELOG.md b/CHANGELOG.md index efeb4580d..e8f76eb30 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,6 +17,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - **One parser, built in.** The native engine now reads every language on its own; the WebAssembly fallback is gone. Installs are smaller, the MCP server starts lighter, and Node.js 25 is no longer refused. Release bundles cover macOS, Linux and Windows. - **`codegraph ui` — your graph in a browser.** A local, read-only viewer: your code with its callers and callees in the margin, a map of the whole repository, a Screens tab for Expo Router, React Router, Next.js, TanStack Router, Vue Router / Nuxt and SvelteKit apps, and a Steps tab that draws what one action sets in motion, in the order the code runs, with the condition on each arrow. Not in an upstream release yet: set `CODEGRAPH_UI=1` to run it. - **Your earlier agent sessions are searchable.** `codegraph sessions` and the `codegraph_sessions` tool find what a past Claude Code, Codex, Cursor, OpenCode or AGY session said about this project. +- **Explore starts compact.** `codegraph_explore` returns a map by default — matching symbols with their files and lines, the call path and blast radius, without source bodies — and lists `codegraph_node` for the one body you need. `mode: "snippets"` adds each returned excerpt's first lines, and `mode: "source"` restores the previous output. A three-turn A/B on 2026-11-01 cut retrieval residual 29–37% against source mode on gin and n8n. - **Documentation is in the graph.** Markdown headings, sections and links are indexed, and a documentation question gets the section, not a code guess. - **Fewer wrong links, more right ones.** Name resolution now reads what each file actually binds — declarations, parameters, imports — instead of pattern-matching its source, for JavaScript, TypeScript, Python, Go, Java, Kotlin, PHP and C/C++. A name imported from a package no longer lands on a project symbol that shares it, a parameter shadows only inside its own function, and Kotlin imports are followed for the first time. - **Files with syntax errors are read, not skipped.** Macro-heavy C and C++ headers and other files the parser cannot fully accept still contribute their symbols. @@ -39,6 +40,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). - Installers can verify daemon readiness and safely hand writer ownership across build promotion and rollback through a supported runtime-control API. +- `codegraph_explore` takes `mode: "map" | "snippets" | "source"` (CLI: `--mode`), with `codegraph.json` `explore.mode` and `CODEGRAPH_EXPLORE_MODE` as persistent overrides. `map` is the default; `source` is byte-for-byte the previous output, and treating `codegraph_node`'s source as already Read still applies. - `codegraph_explore` now finds quoted prose in script strings and template text through a capped source scan, ignoring case and punctuation without requiring a re-index. - After a complete scan of indexed source without skipped files, `codegraph_explore` lists query names it cannot find, so an agent can catch a guessed name. diff --git a/README.md b/README.md index c96121653..087a7933c 100644 --- a/README.md +++ b/README.md @@ -84,6 +84,7 @@ Compared with upstream `main` at `cd04b21a` (after v1.6.2), re-checked against t | Feature | Upstream | Fork | What it does | |---|:-:|:-:|---| +| Compact explore (`codegraph_explore` map default) | — | ✓ | Explore returns a compact map by default — matching symbols, `file:line`, the flow and blast radius, without source bodies — and lists `codegraph_node` as the body follow-up; `mode: "snippets"` adds each returned excerpt's first lines, and `mode: "source"` restores the previous verbatim output. Per call, `CODEGRAPH_EXPLORE_MODE`, or `codegraph.json` `explore.mode` overrides it. A 3-turn A/B on 2026-11-01 cut retrieval residual 29–37% against source mode on gin and n8n. | | Session search (`codegraph sessions`, `codegraph_sessions`) | — | ✓ | Searches this project's earlier Claude Code, Codex, Cursor/T3, OpenCode, AGY, Devin and Grok transcripts and its git commit messages, so an agent can find what a past session decided. `codegraph_explore` also names the sessions that mentioned the symbols it returns. `"sessions": false` in `codegraph.json` turns it off. Experimental: `projects` / `--project` searches up to five projects' histories in one call. | | Markdown indexing | — | ✓ | Headings, sections and local links become graph nodes, with tables, commands and list items kept in the section's searchable text; a documentation question gets the matching section. Explicit code-file pins remain eligible in mixed documentation/code requests. | | Missing names reported | Reports them | Reports them, and proves absence | Upstream's note says a requested name is not in the index and the defining file may be unindexed. The fork keeps that note and additionally lists a name only when a bounded (300 ms) scan of every hand-written source file — plus file stems and bindings — completes without finding it as any word, so a listed name is absent from the project outright, not just unindexed. An incomplete scan lists nothing rather than guess. | @@ -713,7 +714,7 @@ npm install -g @colbymchenry/codegraph } ``` -`alwaysLoad` loads all tools exposed by this server at session start, avoiding a tool-search step. In this fork that includes `codegraph_explore` and `codegraph_sessions`. See [Claude Code's MCP documentation](https://code.claude.com/docs/en/mcp#exempt-a-server-from-deferral). +`alwaysLoad` loads all tools exposed by this server at session start, avoiding a tool-search step. In this fork that includes `codegraph_explore` and `codegraph_sessions`, plus `codegraph_node` when explore serves the compact map. See [Claude Code's MCP documentation](https://code.claude.com/docs/en/mcp#exempt-a-server-from-deferral). **Add to `~/.claude/settings.json` (optional, for auto-allow):** ```json @@ -726,7 +727,7 @@ npm install -g @colbymchenry/codegraph } ``` -One wildcard auto-approves every CodeGraph tool — `codegraph_explore` is the only one listed by default, but if you re-enable others via `CODEGRAPH_MCP_TOOLS` they're already permitted, no prompt. +One wildcard auto-approves every CodeGraph tool — `codegraph_explore` and `codegraph_sessions` are always listed, and map mode adds `codegraph_node`; if you re-enable others via `CODEGRAPH_MCP_TOOLS` they're already permitted, no prompt. @@ -736,7 +737,7 @@ npm install -g @colbymchenry/codegraph CodeGraph's MCP server delivers its usage guidance to your agent **automatically**, in the MCP `initialize` response. In short, it tells the agent to: - **Answer structural questions directly with CodeGraph** — it *is* the pre-built index, so a grep/read loop just repeats work it already did. Treat the returned source as already read. -- **Reach for `codegraph_explore` for almost anything** — "how does X work", a flow/"how does X reach Y", or surveying an area. One call returns the relevant symbols' verbatim source grouped by file, the call paths between them (dynamic-dispatch hops included), and a blast-radius summary. Name a file or symbol in the query to read its current line-numbered source. +- **Reach for `codegraph_explore` for almost anything** — "how does X work", a flow/"how does X reach Y", or surveying an area. By default it returns a compact map: the matching symbols with their files and lines, the call paths between them (dynamic-dispatch hops included), and a blast-radius summary, without source bodies. Fetch one symbol's body with `codegraph_node` (`includeCode: true`), or pass `mode: "source"` for the full verbatim source. - **Trust the results — don't re-verify with grep**, and check the staleness banner after edits. - Works **per project**: query any project that has a `.codegraph/` index by passing `projectPath` — so a monorepo where only some services are indexed, or a second repo, works in one session. A path with no index returns clean guidance to use built-in tools; indexing stays your decision. @@ -793,7 +794,7 @@ codegraph status [path] # Show statistics codegraph ui [path] # Browser viewer (alias: web; --port, --no-open); not released yet, needs CODEGRAPH_UI=1 codegraph unlock [path] # Remove a stale lock file that's blocking indexing codegraph query # Search symbols (--kind, --limit, --json) -codegraph explore # Relevant symbols' source + call paths in one shot (same output as the codegraph_explore MCP tool) +codegraph explore # Compact map by default (--mode map|snippets|source; same output as codegraph_explore) codegraph context # Context for a task: relevant symbols, relationships and code (--format markdown|json, --max-nodes, --no-code) codegraph sessions # Search Claude Code, Codex, Cursor/T3, OpenCode, AGY, Devin, and Grok transcripts and commit messages for this project (--role, --since , --until , --session, --any, --full, --context , --project (experimental), --json; same output as codegraph_sessions) codegraph node # One symbol's source + callers, or read a file with line numbers (same output as codegraph_node) @@ -841,14 +842,15 @@ fi ## MCP Tools -When running as an MCP server, CodeGraph exposes **one tool for code** — `codegraph_explore` — and one for the project's own history — `codegraph_sessions`. Measured agent behavior showed that one strong code tool steers agents better than a menu of narrower ones — fewer mis-picks, and it saves context every session: +When running as an MCP server, CodeGraph exposes **one code entry point** — `codegraph_explore` — plus `codegraph_node` as its body follow-up in map mode, and `codegraph_sessions` for the project's own history. Measured agent behavior showed that one strong entry point steers agents better than a menu of narrower tools — fewer mis-picks, and it saves context every session: | Tool | Purpose | |------|---------| -| `codegraph_explore` | Answer almost any question in one call — "how does X work", a flow ("how does X reach Y"), or surveying an area — returning the relevant symbols' verbatim source grouped by file, plus the call paths between them and a blast-radius summary. Surfaces dynamic-dispatch hops (callbacks, React re-render, interface→impl) grep can't follow. Name a file or symbol in the query to read its current line-numbered source, the same shape the Read tool gives you. | +| `codegraph_explore` | Answer almost any question from a compact map by default — "how does X work", a flow ("how does X reach Y"), or surveying an area — returning the matching symbols with `file:line`, the call paths between them, and a blast-radius summary. Surfaces dynamic-dispatch hops (callbacks, React re-render, interface→impl) grep can't follow. `mode: "snippets"` adds each returned excerpt's first lines; `mode: "source"` returns the verbatim, line-numbered source in Read's shape. | +| `codegraph_node` | One symbol's definition, signature and source (`includeCode: true`; a container returns its member outline, so use `mode: "source"` for its full body), or a whole file in Read's shape; listed beside explore when it serves the compact map. | | `codegraph_sessions` | Search this project's Claude Code, Codex, Cursor/T3, OpenCode, AGY, Devin, and Grok transcripts and its last 2000 git commit messages, including active sessions: prompts, replies, and compaction summaries, excluding tool traffic. Uses stemmed, BM25-ranked full-text search, stored locally in `.codegraph/sessions-v2.db` and refreshed for changed files on each call. Each hit includes its session (`claude:`, `codex:`, `cursor:`, `opencode:`, `agy:`, `devin:`, `grok:`, or `git:`), role, time, transcript path, and matching passage; `full` returns each hit's whole stored passage instead of the snippet, up to 16,000 bytes in all, and later hits keep their snippet; `context` (0-5, implies `full`) adds that many neighbouring passages either side of each hit from the same transcript, spending the same budget after the hits themselves; `projects` (experimental, up to five paths) searches those projects' session histories in one call, interleaving hits by each project's own ranking and naming each hit's project. Passages come back as stored, credentials included. Common words are dropped, and when too few passages hold every remaining word, passages holding some of them follow, marked. Harness-injected text (skill bodies, system reminders) is not indexed. Set `"sessions": false` in `codegraph.json` to opt out; `CODEGRAPH_SESSIONS_DIR` selects another Claude-format transcript directory exclusively. | -The other tools (`codegraph_node`, `codegraph_search`, `codegraph_callers`, `codegraph_callees`, `codegraph_impact`, `codegraph_files`, `codegraph_status`) stay fully functional but **unlisted by default** — everything they return already arrives inline on `codegraph_explore` (its blast-radius section, the relationship map, a symbol's body as its callee list). Re-enable any of them for the MCP surface with the `CODEGRAPH_MCP_TOOLS` environment variable (e.g. `CODEGRAPH_MCP_TOOLS=explore,node,search,callers`), or use their CLI equivalents (`codegraph node` / `query` / `callers` / `callees` / `impact` / `files` / `status`). +The other tools (`codegraph_search`, `codegraph_callers`, `codegraph_callees`, `codegraph_impact`, `codegraph_files`, `codegraph_status`) stay fully functional but **unlisted by default** — everything they return already arrives inline on `codegraph_explore` (its blast-radius section, the relationship map, a symbol's body as its callee list). `codegraph_node` is the exception: map mode lists it as the body follow-up. Re-enable any of them for the MCP surface with the `CODEGRAPH_MCP_TOOLS` environment variable (e.g. `CODEGRAPH_MCP_TOOLS=explore,node,search,callers`), or use their CLI equivalents (`codegraph node` / `query` / `callers` / `callees` / `impact` / `files` / `status`). Even when the server's own root has no `.codegraph/` index, the tools stay available: pass `projectPath` to query any indexed project — a sub-service in a monorepo, or a second repo — in the same session. A path that has no index returns clean guidance to use built-in tools instead, so nothing fails loudly, and indexing stays your decision. A project opened this way is watched and kept in sync while the session uses it, and released after 10 minutes without a query (`CODEGRAPH_PROJECT_IDLE_TIMEOUT_MS`; `0` keeps it open). diff --git a/__tests__/daemon-client-liveness.test.ts b/__tests__/daemon-client-liveness.test.ts index 63e8c2cc9..d1f8e09fe 100644 --- a/__tests__/daemon-client-liveness.test.ts +++ b/__tests__/daemon-client-liveness.test.ts @@ -15,6 +15,15 @@ describe('parseClientHelloLine', () => { .toEqual({ pid: 1234, hostPid: 56 }); }); + it('carries the launching session explore mode and ignores blank values', () => { + expect(parseClientHelloLine('{"codegraph_client":1,"pid":1234,"exploreMode":"source"}')) + .toEqual({ pid: 1234, hostPid: null, exploreMode: 'source' }); + expect(parseClientHelloLine('{"codegraph_client":1,"pid":1234,"exploreMode":" "}')) + .toEqual({ pid: 1234, hostPid: null }); + expect(parseClientHelloLine('{"codegraph_client":1,"pid":1234,"exploreMode":null,"advertisedMode":"map"}')) + .toEqual({ pid: 1234, hostPid: null, exploreMode: null, advertisedMode: 'map' }); + }); + it('accepts a null host pid and a missing host pid', () => { expect(parseClientHelloLine('{"codegraph_client":1,"pid":1234,"hostPid":null}')) .toEqual({ pid: 1234, hostPid: null }); diff --git a/__tests__/explore-compact.test.ts b/__tests__/explore-compact.test.ts new file mode 100644 index 000000000..a6d213723 --- /dev/null +++ b/__tests__/explore-compact.test.ts @@ -0,0 +1,371 @@ +/** + * Compact explore output: a map is the default so a session does not keep + * ~10k tokens of source per call resident for every later turn; snippets keep + * the first lines of each file, and source is unchanged. The map is for + * deciding where to look — codegraph_node (or mode "source") returns a body. + */ +import { describe, it, expect, beforeEach, afterEach } from 'vitest'; +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import type { Node } from '../src/types'; +import { compactExploreText, compactSymbolRows, getStaticTools, injectCompactSymbolTables, resolveExploreMode, ToolHandler } from '../src/mcp/tools'; + +const RENDERED = [ + '**Exploration: Widget**', + '', + 'Found 1 symbol across 2 files.', + '', + '**Blast radius — what depends on these (update/verify before editing)**', + '', + '- `A` (a.ts:1) — 1 caller', + '- `B` (b.ts:2) — 1 caller', + '- `C` (c.ts:3) — 1 caller', + '- `D` (d.ts:4) — 1 caller', + '- `E` (e.ts:5) — 1 caller', + '- `F` (f.ts:6) — 1 caller', + '- `G` (g.ts:7) — 1 caller', + '', + '**Relationships**', + '', + '**calls:**', + '- A → B', + '- A → C', + '- A → D', + '- A → E', + '- A → F', + '', + '**extends:**', + '- A → Base', + '', + '**imports:**', + '- A → util', + '', + '**references:**', + '- A → type', + '', + '**Source Code**', + '', + '> The code below is the verbatim source.', + '', + '**`a.ts`** — A(calls)', + '', + '```typescript', + '1\tfunction a() {', + '2\t return 1;', + '3\t}', + '4\tconst late = 2;', + '5\tconst later = 3;', + '6\tconst latest = 4;', + '7\tconst tail = 5;', + '8\tconst end = 6;', + '9\tconst more = 7;', + '10\tconst extra = 8;', + '11\tconst beyond = 9;', + '12\tconst twelve = 10;', + '13\tconst thirteen = 11;', + '```', + '', + '**`b.ts`** — B', + '', + '```typescript', + '1\tfunction b() { return 2; }', + '```', + '', + '---', + '> **Verbatim source for 2 files is included above.**', +].join('\n'); + +let tmpDir: string; +beforeEach(() => { + tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'cg-explore-mode-')); +}); +afterEach(() => { + fs.rmSync(tmpDir, { recursive: true, force: true }); +}); + +describe('compactExploreText', () => { + it('map keeps the summary, bounded context and file headers, and drops source', () => { + const out = compactExploreText(RENDERED, 'map'); + expect(out).toContain('Found 1 symbol across 2 files.'); + expect(out).toContain('**Blast radius'); + expect(out).toContain('- `F`'); + expect(out).not.toContain('- `G`'); + expect(out).toContain('**calls:**'); + expect(out).toContain('- A → E'); + expect(out).not.toContain('- A → F'); + // Only three relationship kinds fit, and the fourth kind's entries must + // not be relabelled under the third. + expect(out).toContain('**imports:**'); + expect(out).toContain('- A → util'); + expect(out).not.toContain('**references:**'); + expect(out).not.toContain('- A → type'); + expect(out).toContain('**`a.ts`**'); + expect(out).toContain('**`b.ts`**'); + expect(out).not.toContain('function a()'); + expect(out).toContain('call codegraph_node'); + expect(out).toMatch(/Source omitted in compact map mode/); + }); + + it('snippets keeps the first source lines of each file', () => { + const out = compactExploreText(RENDERED, 'snippets'); + expect(out).toContain('function a()'); + expect(out).toContain('const twelve'); + expect(out).not.toContain('const thirteen'); + expect(out).toContain('...'); + expect(out).toMatch(/Only the first 12 lines of each returned excerpt/); + }); + + it('stays under the map budget on a large response', () => { + const headers = Array.from({ length: 200 }, (_, i) => `**\`f${i}.ts\`** — symbol`).join('\n'); + const big = RENDERED.replace('**Source Code**', `**Source Code**\n\n${headers}`); + const out = compactExploreText(big, 'map'); + expect(out.length).toBeLessThanOrEqual(4000 + 200); + expect(out).toContain('compact response truncated'); + }); +}); + +describe('compactExploreText promises', () => { + const TEXT = [ + '**Flow**', + '', + '> These sites choose their call target at runtime (registry) — run codegraph_explore or codegraph_node on a candidate; source for the sites above is included below.', + '', + '> Source for these symbols follows; gap markers identify any ranges omitted to fit the output budget.', + '', + '**Source Code**', + '', + '```typescript', + '1\tconst x = 1;', + '```', + ].join('\n'); + + it('does not promise source a map does not contain', () => { + const out = compactExploreText(TEXT, 'map'); + expect(out).not.toContain('included below'); + expect(out).not.toContain('Source for these symbols follows'); + expect(out).toContain('fetch a candidate\'s body with codegraph_node includeCode: true'); + expect(out).toContain('**Source Code** — omitted in map mode'); + expect(out).not.toContain('const x = 1;'); + }); + + it('leaves snippets promises alone because the source does follow', () => { + const out = compactExploreText(TEXT, 'snippets'); + expect(out).toContain('excerpted below'); + expect(out).toContain('Source for these symbols follows'); + expect(out).toContain('**Source Code**\n'); + expect(out).toContain('const x = 1;'); + }); +}); + +describe('compactExploreText sections', () => { + it('preserves graph sections that follow documentation source', () => { + const text = [ + '**Exploration: README runNode**', + '', + 'Found 2 symbols across 2 files.', + '', + '**Source Code**', + '', + '**`README.md`** — sections', + '', + '```markdown', + '1\t# Title', + '```', + '', + '**`a.ts`** — runNode(function)', + '', + '```typescript', + '1\tfunction runNode() {}', + '```', + '', + '**Blast radius — what depends on these (update/verify before editing)**', + '', + '- `runNode` (a.ts:1) — 1 caller', + '', + '**Relationships**', + '', + '**calls:**', + '- A → B', + '', + ].join('\n'); + const out = compactExploreText(text, 'map'); + expect(out).toContain('**Blast radius'); + expect(out).toContain('- `runNode` (a.ts:1) — 1 caller'); + expect(out).toContain('**Relationships**'); + expect(out).toContain('- A → B'); + expect(out).toContain('**`a.ts`**'); + }); + + it('ends the blast limit before the session-history section', () => { + const text = [ + '**Blast radius — what depends on these (update/verify before editing)**', + '', + ...Array.from({ length: 7 }, (_, i) => `- b${i}`), + '', + '**Discussed in earlier sessions** (codegraph_sessions with the name reads them)', + '', + '- `runNode`: claude (2026-11-01)', + '', + '**Relationships**', + '', + '**calls:**', + '- A → B', + ].join('\n'); + const out = compactExploreText(text, 'map'); + expect(out).toContain('- b5'); + expect(out).not.toContain('- b6'); + expect(out).toContain('1 more blast-radius entry'); + expect(out).toContain('- `runNode`: claude (2026-11-01)'); + }); + + it('reports relationship entries and kinds the compact limits omit', () => { + const out = compactExploreText(RENDERED, 'map'); + expect(out).toContain('2 more relationship entries, 1 more relationship kind'); + }); + + it('bounds the echoed query before the map budget is applied', () => { + const query = 'x'.repeat(500); + const out = compactExploreText(`**Exploration: ${query}**\n\nFound 1 symbol across 1 file.`, 'map'); + expect(out.split('\n')[0]!.length).toBeLessThanOrEqual(175); + expect(out).toContain('…'); + // A multiline query heading must not bypass the bound either. + const multiline = compactExploreText(`**Exploration: src/a.ts\n${query}**\n\nFound 1 symbol.`, 'map'); + expect(multiline.split('\n')[0]!.length).toBeLessThanOrEqual(175); + expect(multiline).toContain('Found 1 symbol.'); + }); + + it('does not let a nested Markdown fence close the source wrapper', () => { + const text = [ + '**Source Code**', + '', + '**`doc.md`** — section', + '', + '````markdown', + '# Title', + '```yaml', + '- a', + '```', + '````', + ].join('\n'); + const out = compactExploreText(text, 'map'); + expect(out).not.toContain('- a'); + }); + + it('removes source-completeness claims from compact output', () => { + const text = [ + '**Not shown above — explore these names for their source**', + '', + '- file.ts: run', + '', + '> **Complete source for 1 file is included above — do NOT re-read them.**', + ].join('\n'); + const out = compactExploreText(text, 'map'); + expect(out).not.toContain('Complete source'); + expect(out).toContain('- file.ts: run'); + }); + + it('only names codegraph_node when the client can call it', () => { + const served = compactExploreText(RENDERED, 'map', true); + expect(served).toContain('codegraph_node'); + expect(served).toContain('container returns its member outline'); + const unserved = compactExploreText(RENDERED, 'map', false); + expect(unserved).not.toContain('codegraph_node'); + expect(unserved).toContain('mode: "source"'); + // Snippets keeps the source, so it keeps the "included below" promise but + // must still not name a tool the client cannot call. + const snippets = compactExploreText( + '> These sites choose their call target at runtime; source for the sites above is included below.', + 'snippets', false, + ); + expect(snippets).not.toContain('codegraph_node'); + expect(snippets).toContain('excerpted below'); + }); +}); + +describe('compact symbol rows', () => { + const node = { + id: 'n1', name: 'runNode', kind: 'method', startLine: 1349, language: 'typescript', + filePath: 'a.ts', signature: 'async runNode(workflow: Workflow)', + } as unknown as Node; + + it('names the location and signature before the body is dropped', () => { + expect(compactSymbolRows([node], 6).rows).toEqual([ + '- `runNode` (method) :1349 — async runNode(workflow: Workflow)', + ]); + }); + + it('keeps explicitly requested symbols ahead of the row limit', () => { + const mk = (id: string, startLine: number) => ({ + id, name: id, kind: 'function', startLine, language: 'typescript', filePath: 'a.ts', + }) as unknown as Node; + const result = compactSymbolRows([mk('a', 1), mk('b', 2), mk('c', 3)], 1, new Set(['c'])); + expect(result.rows).toEqual(['- `c` (function) :3']); + expect(result.omitted).toBe(2); + }); + + it('inserts the rows under the matching file header', () => { + const text = '**`a.ts`** — runNode(function)\n```typescript\n1\tfunction runNode() {}\n```'; + expect(injectCompactSymbolTables(text, new Map([['a.ts', ['- `runNode` (method) :1349']]]))) + .toBe('**`a.ts`** — runNode(function)\n- `runNode` (method) :1349\n```typescript\n1\tfunction runNode() {}\n```'); + }); +}); + +describe('resolveExploreMode', () => { + it('falls back to the product default and reads codegraph.json', () => { + expect(resolveExploreMode(undefined, tmpDir, {})).toBe('map'); + fs.writeFileSync(path.join(tmpDir, 'codegraph.json'), JSON.stringify({ explore: { mode: 'snippets' } })); + expect(resolveExploreMode(undefined, tmpDir, {})).toBe('snippets'); + }); + + it('prefers the per-call value, then the environment, then the project file', () => { + fs.writeFileSync(path.join(tmpDir, 'codegraph.json'), JSON.stringify({ explore: { mode: 'source' } })); + expect(resolveExploreMode(undefined, tmpDir, { CODEGRAPH_EXPLORE_MODE: 'snippets' })).toBe('snippets'); + expect(resolveExploreMode('map', tmpDir, { CODEGRAPH_EXPLORE_MODE: 'snippets' })).toBe('map'); + expect(resolveExploreMode('bogus', tmpDir, {})).toBe('source'); + // The launching session's override beats the daemon's inherited env. + expect(resolveExploreMode(undefined, tmpDir, { CODEGRAPH_EXPLORE_MODE: 'source' }, 'map')).toBe('map'); + expect(resolveExploreMode('source', tmpDir, { CODEGRAPH_EXPLORE_MODE: 'map' }, 'source')).toBe('source'); + // An explicit null says the session had no override: skip the daemon env + // and use the project file. + expect(resolveExploreMode(undefined, tmpDir, { CODEGRAPH_EXPLORE_MODE: 'snippets' }, null)).toBe('source'); + }); +}); + +describe('the default surface follows the explore mode', () => { + const ENV = 'CODEGRAPH_EXPLORE_MODE'; + const original = process.env[ENV]; + afterEach(() => { + if (original === undefined) delete process.env[ENV]; + else process.env[ENV] = original; + }); + + const listed = () => new ToolHandler(null).getTools().map((t) => t.name).sort(); + + it('serves node beside explore on the product default', () => { + delete process.env[ENV]; + expect(listed()).toEqual(['codegraph_explore', 'codegraph_node', 'codegraph_sessions']); + delete process.env.CODEGRAPH_MCP_TOOLS; + // The pre-project proxy surface must agree, or the agent sees node listed + // and then cannot find it after the project opens. + expect(getStaticTools().map((t) => t.name)).toContain('codegraph_node'); + }); + + it('keeps node off the source-mode surface', () => { + process.env[ENV] = 'source'; + expect(listed()).toEqual(['codegraph_explore', 'codegraph_sessions']); + expect(getStaticTools().map((t) => t.name)).not.toContain('codegraph_node'); + process.env[ENV] = 'snippets'; + expect(listed()).toContain('codegraph_node'); + }); + + it('lets an explicit allowlist replace the mode-derived surface', () => { + process.env[ENV] = 'map'; + process.env.CODEGRAPH_MCP_TOOLS = 'explore'; + try { + expect(listed()).toEqual(['codegraph_explore']); + } finally { + delete process.env.CODEGRAPH_MCP_TOOLS; + } + }); +}); diff --git a/__tests__/fork-tooling-scripts.test.ts b/__tests__/fork-tooling-scripts.test.ts index 8c9da4008..2056d6b56 100644 --- a/__tests__/fork-tooling-scripts.test.ts +++ b/__tests__/fork-tooling-scripts.test.ts @@ -101,9 +101,18 @@ describe('probe-explore rows', () => { const row = probeRow('t1', text, 12, ['src/b\\.ts', 'absent']); expect(row).toEqual({ id: 't1', ms: 12, chars: text.length, sourceAt: 6, - files: ['src/a.ts', 'src/b.ts'], mustMatchAt: [text.indexOf('src/b.ts'), -1], + files: ['src/a.ts', 'src/b.ts'], mapFiles: [], compact: false, + mustMatchAt: [text.indexOf('src/b.ts'), -1], }); expect(probeRow('t2', 'no source here', 1).sourceAt).toBe(-1); + // A compact map names files but renders no source: the source-loss gate + // must not mistake its file headers for bodies. + const compactText = '**Source Code** — omitted in map mode; fetch a body.\n**`src/a.ts`**\n'; + const compactRow = probeRow('t3', compactText, 1); + expect(compactRow.sourceAt).toBe(-1); + expect(compactRow.files).toEqual([]); + expect(compactRow.mapFiles).toEqual(['src/a.ts']); + expect(compactRow.compact).toBe(true); }); it('selects tasks by id globs', () => { diff --git a/__tests__/mcp-tool-annotations.test.ts b/__tests__/mcp-tool-annotations.test.ts index 89c51a1d5..dfa492194 100644 --- a/__tests__/mcp-tool-annotations.test.ts +++ b/__tests__/mcp-tool-annotations.test.ts @@ -55,6 +55,9 @@ describe('Read-only annotations on the codegraph MCP tools (#1018)', () => { expect(tools.length).toBeGreaterThan(0); for (const tool of tools) expectReadOnly(tool); expectExploreAlwaysLoad(tools); + // `codegraph_node` is explore's body-on-demand follow-up in map mode; a + // deferred tool would need a ToolSearch step before the agent could use it. + expect(tools.find((t) => t.name === 'codegraph_node')!._meta).toEqual({ 'anthropic/alwaysLoad': true }); }); it('the static proxy surface carries annotations on every exposed tool', () => { diff --git a/scripts/agent-eval/probe-explore.mjs b/scripts/agent-eval/probe-explore.mjs index c757b2263..eaa5a1d62 100644 --- a/scripts/agent-eval/probe-explore.mjs +++ b/scripts/agent-eval/probe-explore.mjs @@ -25,6 +25,8 @@ // one running this script, so `bun probe-explore.mjs --mcp` // serves on Bun) // --timeout-ms per-request limit with --mcp (default 30000) +// --mode explore mode (default: source, so the source-loss gate +// compares bodies; use map for compact coverage) import { spawn } from 'node:child_process'; import { mkdirSync, readFileSync, writeFileSync } from 'node:fs'; import { join, resolve } from 'node:path'; @@ -32,12 +34,17 @@ import { fileURLToPath, pathToFileURL } from 'node:url'; const SOURCE_HEADER = '**Source Code**'; const FILE_HEADER = /^\*\*[^`\n]*`([^`]+)`\*\*/; +const MAP_HEADER = 'omitted in map mode'; -/** The source files an explore answer renders, in order, from its Source Code section on. */ +/** The source files an explore answer actually renders, in order. A map renders none. */ export function renderedFiles(text) { const files = []; let inSource = false; for (const line of text.split('\n')) { + if (line.startsWith(SOURCE_HEADER) && line.includes(MAP_HEADER)) { + inSource = false; + continue; + } if (line.startsWith(SOURCE_HEADER)) inSource = true; const m = inSource ? FILE_HEADER.exec(line) : null; if (m) files.push(m[1]); @@ -45,14 +52,24 @@ export function renderedFiles(text) { return files; } +/** The files a compact map names, for coverage checks when it carries no bodies. */ +export function mapFiles(text) { + if (!/^\*\*Source Code\*\* — omitted in map mode/m.test(text)) return []; + const at = text.indexOf(SOURCE_HEADER); + if (at < 0) return []; + return [...text.slice(at).matchAll(new RegExp(FILE_HEADER.source, 'gm'))].map((m) => m[1]); +} + /** One probe row: where the answer's source starts and where each expected pattern first lands. */ export function probeRow(id, text, ms, mustMatch = []) { return { id, ms, chars: text.length, - sourceAt: text.indexOf(SOURCE_HEADER), + sourceAt: /^\*\*Source Code\*\* — omitted in map mode/m.test(text) ? -1 : text.indexOf(SOURCE_HEADER), files: renderedFiles(text), + mapFiles: mapFiles(text), + compact: /^\*\*Source Code\*\* — omitted in map mode/m.test(text), mustMatchAt: mustMatch.map((re) => new RegExp(re).exec(text)?.index ?? -1), }; } @@ -83,7 +100,7 @@ function parseArgs(argv) { const a = argv[i]; if (a === '--mcp') opts.mcp = true; else if (a === '--expect') opts.expect.push(argv[++i]); - else if (['--tasks', '--ids', '--out', '--build', '--runtime', '--timeout-ms'].includes(a)) opts[a.slice(2)] = argv[++i]; + else if (['--tasks', '--ids', '--out', '--build', '--runtime', '--timeout-ms', '--mode'].includes(a)) opts[a.slice(2)] = argv[++i]; else opts.positional.push(a); } return opts; @@ -104,8 +121,10 @@ async function inProcess(build, repo) { const handler = new ToolHandler(cg); return { pid: process.pid, - async explore(query, maxFiles) { - const res = await handler.execute('codegraph_explore', maxFiles === undefined ? { query } : { query, maxFiles }); + async explore(query, maxFiles, mode) { + const res = await handler.execute('codegraph_explore', { + query, ...(maxFiles === undefined ? {} : { maxFiles }), ...(mode ? { mode } : {}), + }); if (res.isError) throw new Error(`codegraph_explore: ${JSON.stringify(res.content)}`); return (res.content ?? []).map((c) => c.text ?? '').join('\n'); }, @@ -172,8 +191,10 @@ async function overMcp(build, repo, runtime, timeoutMs) { proc.stdin.write(JSON.stringify({ jsonrpc: '2.0', method: 'notifications/initialized', params: {} }) + '\n'); return { pid: proc.pid, - async explore(query, maxFiles) { - const args = maxFiles === undefined ? { query, projectPath: repo } : { query, projectPath: repo, maxFiles }; + async explore(query, maxFiles, mode) { + const args = { + query, projectPath: repo, ...(maxFiles === undefined ? {} : { maxFiles }), ...(mode ? { mode } : {}), + }; const res = await rpc('tools/call', { name: 'codegraph_explore', arguments: args }); return (res.result?.content ?? []).map((c) => c.text ?? '').join('\n'); }, @@ -208,7 +229,7 @@ async function main() { const idle = memMB(session.pid); for (const task of tasks) { const t0 = Date.now(); - const text = await session.explore(task.prompt, task.maxFiles); + const text = await session.explore(task.prompt, task.maxFiles, opts.mode ?? 'source'); const row = probeRow(task.id, text, Date.now() - t0, task.mustMatch); missing += row.mustMatchAt.filter((at) => at < 0).length; if (opts.out) writeFileSync(join(opts.out, `${task.id}.md`), text); diff --git a/site/src/content/docs/reference/cli.md b/site/src/content/docs/reference/cli.md index ca5118715..34af6625c 100644 --- a/site/src/content/docs/reference/cli.md +++ b/site/src/content/docs/reference/cli.md @@ -15,7 +15,7 @@ codegraph status [path] # Show statistics (--json) codegraph ui [path] # Open the browser viewer (not released yet — see below) codegraph unlock [path] # Remove a stale lock file that's blocking indexing codegraph query # Search symbols (--kind, --limit, --json) -codegraph explore # Relevant symbols' source + call paths in one shot (same output as the codegraph_explore MCP tool) +codegraph explore # Compact map by default; --mode map|snippets|source (same output as codegraph_explore) codegraph context # Context for a task: relevant symbols, relationships and code (--format markdown|json, --max-nodes, --no-code) codegraph sessions # Search this project's earlier agent sessions (--role, --since , --until , --session, --any, --full, --context , --project (experimental), --json; same output as codegraph_sessions) codegraph node # One symbol's source + callers, or read a file with line numbers (same output as codegraph_node) diff --git a/site/src/content/docs/reference/mcp-server.md b/site/src/content/docs/reference/mcp-server.md index 1528ae2ae..fe0fc7066 100644 --- a/site/src/content/docs/reference/mcp-server.md +++ b/site/src/content/docs/reference/mcp-server.md @@ -11,11 +11,11 @@ codegraph serve --mcp The tools below are listed in every workspace. Where the project has no `.codegraph/` index, a call returns guidance (for example, to pass `projectPath` for a sub-project that is indexed) instead of an error, and indexing stays your decision. A project opened by `projectPath` is watched and kept in sync while the session uses it, and released after 10 minutes without a query; set `CODEGRAPH_PROJECT_IDLE_TIMEOUT_MS` to change that (`0` keeps it open until the server exits). A session that can't reach the shared background server serves itself and keeps retrying the server, first after 5 seconds and backing off to every 5 minutes; `CODEGRAPH_DAEMON_RETRY_MS` and `CODEGRAPH_DAEMON_RETRY_MAX_MS` set those delays, and `CODEGRAPH_DAEMON_RETRY_MS=0` turns retrying off. -## Two tools by default: `codegraph_explore` and `codegraph_sessions` +## The default code surface: `codegraph_explore`, `codegraph_node` and `codegraph_sessions` -By default the server exposes `codegraph_explore` for code and `codegraph_sessions` for project history. +By default the server exposes `codegraph_explore` for code and `codegraph_sessions` for project history; when explore serves its compact map it also lists `codegraph_node` as the body follow-up. -`codegraph_explore` It's Read-equivalent: give it a natural-language question or a bag of symbol and file names, and it returns the **verbatim, line-numbered source** of the relevant symbols grouped by file — the same shape the `Read` tool gives you — plus the call paths between them (including dynamic-dispatch hops like callbacks, React re-render, and JSX children that grep can't follow) and a blast-radius summary of what depends on them. One call usually answers the whole question. When a complete scan of indexed source finishes without skipped files, the summary lists requested names it could not find, helping catch a guessed name. +`codegraph_explore` returns a compact **map** by default: the matching symbols with their files and lines, the call paths between them (including dynamic-dispatch hops like callbacks, React re-render, and JSX children that grep can't follow), and a blast-radius summary of what depends on them. It does not include source bodies. Fetch one symbol's body with `codegraph_node` (`includeCode: true`), pass `mode: "snippets"` for each returned excerpt's first lines, or pass `mode: "source"` for the **verbatim, line-numbered source** grouped by file — the same shape the `Read` tool gives you. When a complete scan of indexed source finishes without skipped files, the summary lists requested names it could not find, helping catch a guessed name. `codegraph_sessions` searches this project's earlier agent sessions (Claude Code, Codex, Cursor/T3, OpenCode, AGY, Devin and Grok transcripts, plus git commit messages) for what a previous session asked, decided or tried. It answers "why is X like this" questions, which the code graph cannot. A hit shows a short snippet; pass `full` (CLI: `--full`) to get each hit's whole stored passage instead, up to 16,000 bytes in all, with later hits keeping their snippet. Add `context` (CLI: `--context `, 0 to 5, implies `full`) for the passages either side of each hit in the same transcript; they spend the same budget after the hits themselves. Experimental: `projects` (CLI: `--project `) takes up to five project paths and searches their session histories in one call. Each project answers from its own index, the hits are interleaved by each project's own ranking because scores cannot be compared across indexes, and each hit names its project. One limit and one passage budget cover the list; a project with no transcripts, with sessions turned off, or without an index is listed as skipped. Searching refreshes each listed project's own session index. Passages are returned as stored, so a credential a transcript holds is returned too. Set `"sessions": false` in `codegraph.json` to turn it off. @@ -23,7 +23,7 @@ T3-hosted provider transcripts can appear in session search; CodeGraph does not Session history lives in `.codegraph/sessions-v2.db`, separate from the code graph. The first session search imports remembered roots from `sessions.db` once and reindexes available transcripts. The older file remains untouched for older executables, and roots an older executable records after that stay in the older file. Session mentions in code answers resume after the new store is populated. Cached passages from transcripts that were unavailable when the new store was first built remain only in the older store. Retaining both files uses extra disk space. Indexes that predate remembered roots start with none. A damaged newer roots table stops initialization. -Exposing one strong code tool is deliberate. Measured agent behavior showed that one well-aimed tool steers agents to a direct answer better than a menu of narrower ones — fewer mis-picks — and agents reach for it both when answering questions and while editing code. +Exposing one strong code entry point is deliberate. Measured agent behavior showed that one well-aimed tool steers agents to a direct answer better than a menu of narrower ones — fewer mis-picks — and agents reach for it both when answering questions and while editing code. `codegraph_node` is the exception because a map has no bodies: it is listed only while explore is in a compact mode, so the agent can fetch exactly the body it needs. Explore reports requested files that did not receive pin priority within the file limit. Those files may still appear through other matches. References beyond the bounded path scan are marked unexamined. When a capped file gather leaves indexed source uncovered, the summary offers continuation ranges for another explore call. Existing file and output limits still apply. diff --git a/src/bin/codegraph.ts b/src/bin/codegraph.ts index e6796a999..eef4578ff 100644 --- a/src/bin/codegraph.ts +++ b/src/bin/codegraph.ts @@ -1482,7 +1482,8 @@ program .description('Explore an area: relevant symbols\' source + call paths in one shot (same output as the codegraph_explore MCP tool)') .option('-p, --path ', 'Project path') .option('--max-files ', 'Maximum number of files to include source from') - .action(async (queryParts: string[], options: { path?: string; maxFiles?: string }) => { + .option('--mode ', 'Output shape: map (default), snippets or source; codegraph.json explore.mode and CODEGRAPH_EXPLORE_MODE also set it') + .action(async (queryParts: string[], options: { path?: string; maxFiles?: string; mode?: string }) => { const projectPath = resolveProjectPath(options.path); try { @@ -1498,6 +1499,7 @@ program const args: Record = { query: queryParts.join(' ') }; if (options.maxFiles) args.maxFiles = parseInt(options.maxFiles, 10); + if (options.mode) args.mode = options.mode; const result = await handler.execute('codegraph_explore', args); console.log(result.content[0]?.text ?? ''); diff --git a/src/installer/instructions-template.ts b/src/installer/instructions-template.ts index 5cff91193..4cc3fdee1 100644 --- a/src/installer/instructions-template.ts +++ b/src/installer/instructions-template.ts @@ -57,8 +57,8 @@ export const CODEGRAPH_INSTRUCTIONS_BLOCK = `${CODEGRAPH_SECTION_START} In repositories indexed by CodeGraph (a \`.codegraph/\` directory exists at the repo root), reach for it before grep/find or reading files when you need to understand or locate code: -- **MCP tool** (when available): \`codegraph_explore\` answers most code questions in one call — the relevant symbols' verbatim source plus the call paths between them, including dynamic-dispatch hops grep can't follow. Name a file or symbol in the query to read its current line-numbered source. If it's listed but deferred, load it by name via tool search. -- **Shell** (always works): \`codegraph explore ""\` prints the same output. +- **MCP tool** (when available): \`codegraph_explore\` answers most code questions with a compact map — the matching symbols with their files and lines, the call paths between them (including dynamic-dispatch hops grep can't follow) and a blast-radius summary. Fetch one symbol's body with \`codegraph_node\` (\`includeCode: true\`), or pass \`mode: "source"\` for the verbatim, line-numbered source. If a tool is listed but deferred, load it by name via tool search. +- **Shell** (always works): \`codegraph explore "" [--mode source]\` prints the same output. ${CODEGRAPH_QUERY_FIRST_SENTENCE} diff --git a/src/mcp/daemon.ts b/src/mcp/daemon.ts index 0474a4fad..a3019cc5c 100644 --- a/src/mcp/daemon.ts +++ b/src/mcp/daemon.ts @@ -194,6 +194,10 @@ export interface DaemonClientHello { codegraph_client: 1; pid: number; // the proxy process's own pid hostPid: number | null; // the MCP host pid (past any launcher shim), if known + /** The launching session's CODEGRAPH_EXPLORE_MODE; null means it was unset. */ + exploreMode?: string | null; + /** The mode the proxy's tools/list advertised to this session. */ + advertisedMode?: string; } export interface DaemonStartResult { @@ -220,7 +224,7 @@ export class Daemon { // and so is not yet in `clients`. `server.close()` waits for all of them. private sockets = new Set(); /** Per-client peer pids from the optional client-hello, for the liveness sweep. */ - private clientPeers = new Map(); + private clientPeers = new Map(); private idleTimer: NodeJS.Timeout | null = null; private idleTimeoutMs: number; private maxIdleMs: number; @@ -519,6 +523,8 @@ export class Daemon { const session = new MCPSession(transport, this.engine, { explicitProjectPath: this.projectRoot, refreshWatcher: (params) => this.refreshWatcher(params), + exploreMode: peers.exploreMode, + advertisedMode: peers.advertisedMode, }); transport.onClose(() => this.dropClient(session)); this.clients.add(session); @@ -1038,13 +1044,26 @@ function resolveInstallCheckMs(): number { */ export function parseClientHelloLine( line: string, -): { pid: number; hostPid: number | null } | null { +): { pid: number; hostPid: number | null; exploreMode?: string | null; advertisedMode?: string } | null { let parsed: unknown; try { parsed = JSON.parse(line); } catch { return null; } if (!parsed || typeof parsed !== 'object') return null; const o = parsed as Record; if (o.codegraph_client !== 1 || typeof o.pid !== 'number') return null; - return { pid: o.pid, hostPid: typeof o.hostPid === 'number' ? o.hostPid : null }; + const exploreMode = o.exploreMode === null + ? null + : typeof o.exploreMode === 'string' && o.exploreMode.trim() + ? o.exploreMode.trim() + : undefined; + const advertisedMode = typeof o.advertisedMode === 'string' && o.advertisedMode.trim() + ? o.advertisedMode.trim() + : undefined; + return { + pid: o.pid, + hostPid: typeof o.hostPid === 'number' ? o.hostPid : null, + ...(exploreMode !== undefined ? { exploreMode } : {}), + ...(advertisedMode ? { advertisedMode } : {}), + }; } /** @@ -1073,13 +1092,13 @@ export function peerIsDead( */ function readClientHello( socket: net.Socket, -): Promise<{ pid: number | null; hostPid: number | null }> { +): Promise<{ pid: number | null; hostPid: number | null; exploreMode?: string | null; advertisedMode?: string }> { return new Promise((resolve) => { let chunks: Buffer[] = []; let total = 0; let settled = false; const finish = ( - peers: { pid: number | null; hostPid: number | null }, + peers: { pid: number | null; hostPid: number | null; exploreMode?: string | null; advertisedMode?: string }, putBack?: Buffer, ) => { if (settled) return; diff --git a/src/mcp/proxy.ts b/src/mcp/proxy.ts index 188738bcc..086d549dc 100644 --- a/src/mcp/proxy.ts +++ b/src/mcp/proxy.ts @@ -22,6 +22,7 @@ import * as fs from 'fs'; import * as net from 'net'; import { HOST_PPID_ENV } from '../extraction/node-runtime-flags'; import { DaemonClientHello, DaemonHello, MAX_HELLO_LINE_BYTES } from './daemon'; +import { DEFAULT_EXPLORE_MODE, exploreModeOption } from './tools'; import { EARLY_PPID } from './early-ppid'; import { supervisionLostReason } from './ppid-watchdog'; import { armStartupHandshakeTimeout } from './startup-handshake'; @@ -214,10 +215,16 @@ export async function connectWithHello( * host pid if set, else our own parent (the host, on a no-relaunch bundle). */ function sendClientHello(socket: net.Socket): void { + const envMode = process.env.CODEGRAPH_EXPLORE_MODE?.trim(); + // `null` says "this session has no env override", so the daemon does not + // inherit an older session's override. The advertised mode mirrors the + // proxy's tools/list choice (env or the product default). const clientHello: DaemonClientHello = { codegraph_client: 1, pid: process.pid, hostPid: parseHostPpid(process.env[HOST_PPID_ENV]) ?? EARLY_PPID, + exploreMode: envMode ? envMode : null, + advertisedMode: exploreModeOption(envMode) ?? DEFAULT_EXPLORE_MODE, }; try { socket.write(JSON.stringify(clientHello) + '\n'); } catch { /* best-effort */ } } diff --git a/src/mcp/server-instructions.ts b/src/mcp/server-instructions.ts index 83ce1dbbc..a38ec232e 100644 --- a/src/mcp/server-instructions.ts +++ b/src/mcp/server-instructions.ts @@ -33,29 +33,33 @@ fewer tokens and round-trips than reading files yourself. ## The code tool: codegraph_explore — use it instead of reading files -For code there is one tool, \`codegraph_explore\`, and it is Read-equivalent. It -takes either a natural-language question or a bag of symbol/file names and -returns the **verbatim, line-numbered source** of the relevant symbols -grouped by file — the same \`\\t\` shape \`Read\` gives you, safe to -\`Edit\` from — PLUS the call path among them (including dynamic-dispatch hops -like callbacks, React re-render, and JSX children that grep can't follow) and -a blast-radius summary of what depends on them. +For code there is one tool, \`codegraph_explore\`. By default it returns a +compact **map** of the relevant symbols: their names, kinds, \`file:line\`, +signatures, the call path among them (including dynamic-dispatch hops like +callbacks, React re-render and JSX children that grep can't follow), and a +blast-radius summary of what depends on them. It does **not** include source +bodies. The map is for deciding where to look; before you edit a symbol, call +\`codegraph_node\` with its name and \`includeCode: true\` (a container returns its +member outline; use \`codegraph_explore\` with \`mode: "source"\` for its full body) +for the verbatim, line-numbered source — the same +\`\\t\` shape \`Read\` gives you, safe to \`Edit\` from. Whether you're answering "how does X work" or implementing a change (fixing a bug, adding a feature), call \`codegraph_explore\` before you Read; one call -usually answers the whole question. Codegraph is a pre-built search index, so -your own grep + read loop, or a file-reading sub-agent, repeats work it already -did and costs more for the same answer: typically one to a few codegraph calls -against dozens of greps and reads. +usually answers the whole question or names the exact symbols to fetch. +Codegraph is a pre-built search index, so your own grep + read loop, or a +file-reading sub-agent, repeats work it already did and costs more for the same +answer: typically one to a few codegraph calls against dozens of greps and +reads. ## How to query -- **Almost any code question — "how does X work", architecture, a bug, "what/where is X", or surveying an area** → \`codegraph_explore\` with a natural-language question or the relevant names. It returns the verbatim source grouped by file, capped by project size. -- **"How does X reach/become Y? / the flow / the path from X to Y"** → \`codegraph_explore\`, naming the symbols that span the flow (e.g. \`mutateElement renderScene\`) — it surfaces the call path among them, riding dynamic-dispatch hops, and returns their source. -- **Reading or editing a file/symbol you can name** → put its name or file path in the \`codegraph_explore\` query — it returns that current line-numbered source (safe to \`Edit\` from) with the call path and blast radius attached, so you don't Read it separately. For an overloaded name it returns every matching definition's body in one call. -- Name what you need as precisely as you can — a receiver type with its method, a constant or variable, a callable, or a file path. Named items are funded first within the output cap; an oversized body comes back as a bounded excerpt whose gap markers name what was left out. Treat returned ranges as already Read; for trimmed parts, query the names in the gap marker. +- **Almost any code question — "how does X work", architecture, a bug, "what/where is X", or surveying an area** → \`codegraph_explore\` with a natural-language question or the relevant names. It returns the map (symbols, \`file:line\`, flow and blast radius), capped at about 4K characters. +- **"How does X reach/become Y? / the flow / the path from X to Y"** → \`codegraph_explore\`, naming the symbols that span the flow (e.g. \`mutateElement renderScene\`) — it surfaces the call path among them, riding dynamic-dispatch hops. +- **Reading or editing a file/symbol you can name** → call \`codegraph_node\` with its name and \`includeCode: true\` (or put the name in \`codegraph_explore\` with \`mode: "source"\`) — it returns that current line-numbered source, safe to \`Edit\` from, with the call path and blast radius attached. For an overloaded name it returns every matching definition's body in one call. +- Name what you need as precisely as you can — a receiver type with its method, a constant or variable, a callable, or a file path. Named items are funded first within the output cap; in \`mode: "source"\` an oversized body comes back as a bounded excerpt whose gap markers name what was left out. Treat source returned by \`mode: "source"\` (or \`codegraph_node\` with \`includeCode\`) as already Read; for trimmed parts, query the names in the gap marker. - **"Why is this like this? What did the last session decide / try / get told about X?"** → \`codegraph_sessions\` with a few words. It searches the prose of this project's earlier Claude Code, Codex, Cursor/T3, OpenCode, AGY, and Devin sessions (prompts, replies, compaction summaries — stemmed, ranked) and names the session each hit came from (\`claude:\`, \`codex:\`, \`cursor:\`, \`opencode:\`, \`agy:\`, \`devin:\`). History and rationale live there, not in the code. A hit is a short snippet; pass \`full: true\` to read each hit's whole passage in the same call (up to 16,000 bytes in all), and \`context: N\` (0-5, implies \`full\`) to also get the N passages either side of each hit from the same transcript. \`sinceDays: N\` keeps only the last N days and \`until: "2026-10-20"\` only docs before that date, so a search can reach history older than the newest 100 hits. Experimental: \`projects: [paths]\` (up to 5) searches those projects' session histories in one call instead of this one, each hit naming its project. Passages are returned as stored, including any credentials a transcript holds. -- **Need more?** Call \`codegraph_explore\` again with more specific names — treat the source it returns as already Read. Suggested call counts are advisory only, NOT a quota; extra calls are never rejected or rate-limited. +- **Need more?** Call \`codegraph_explore\` again with more specific names; for source bodies use \`codegraph_node\` with \`includeCode: true\` or \`codegraph_explore\` with \`mode: "source"\`. Suggested call counts are advisory only, NOT a quota; extra calls are never rejected or rate-limited. - Qualified symbol names accept dots, \`::\`, or slashes, including containers whose names contain dots (for example, \`AppWeb.Format.group\`). - Named-symbol call paths require exact matches; partial or mistyped names are never silently substituted as flow endpoints. If a graph query reports a missing symbol with did-you-mean suggestions, query the suggested name explicitly. - Explore matches names and indexed code words lexically, not by meaning; an empty result reports word matches and may suggest indexed candidate names to retry with \`codegraph_explore\`. @@ -69,7 +73,7 @@ against dozens of greps and reads. - Questions about tests include source from the nearest test callers of named symbols, within three caller hops and the file/output budgets. These are static caller links, not measured runtime coverage. Naming a JavaScript or TypeScript test file prioritizes matching \`it\`/\`test\` callbacks, including \`.each\` parameterized tests. - **The source codegraph returns is the file's current text** (files that changed since the last sync are flagged), so re-checking it with grep costs time and context without adding accuracy. Call edges from the parse are reliable; a hop marked as a name match (see Limitations) is the one to check. -- **Read/Grep are for what the index lacks**: a detail a codegraph answer didn't cover, or files codegraph doesn't index (such as configs). Markdown is indexed — every \`.md\` file's headings, links and section text (including tables and commands) — so a documentation question (a rule, a runbook, a plan row, a research finding) goes to \`codegraph_explore\` too; it returns the section body, not just its heading. +- **Read/Grep are for what the index lacks**: a detail a codegraph answer didn't cover, or files codegraph doesn't index (such as configs). Markdown is indexed — every \`.md\` file's headings, links and section text (including tables and commands) — so a documentation question (a rule, a runbook, a plan row, a research finding) goes to \`codegraph_explore\` too; pass \`mode: "source"\` for the section body, not just its heading. - **A question about the working changes** ("what do my changes affect", "this branch", \`main..HEAD\`) goes to \`codegraph_explore\` as asked: it reads the diff itself (merge base with the default branch plus uncommitted edits, or the named range) and leads with the changed symbols, their callers and their tests. - **Near-duplicates (update together)** in a blast radius or trail names other function bodies nearly identical to the one you are reading. A bug fixed in one copy is usually still in the others: check them, and tell the user if you leave them unchanged. - **Requested-source limit notices** distinguish files not pinned within the file limit from unexamined path-like references. A file without pin priority may still appear through other matches. If the summary names uncovered indexed source ranges, explore those ranges to continue; the file and output limits still apply. diff --git a/src/mcp/session.ts b/src/mcp/session.ts index 50fbcafe1..6b04e533b 100644 --- a/src/mcp/session.ts +++ b/src/mcp/session.ts @@ -122,6 +122,10 @@ export interface MCPSessionOptions { explicitProjectPath?: string | null; /** Only a shared daemon can acknowledge ownership-bound reconciliation. */ refreshWatcher?: (params: unknown) => Promise; + /** The launching proxy's CODEGRAPH_EXPLORE_MODE; null means it was unset. */ + exploreMode?: string | null; + /** The mode the proxy advertised in tools/list. */ + advertisedMode?: string; } /** @@ -136,6 +140,8 @@ export class MCPSession { private resolvePromise: Promise | null = null; private explicitProjectPath: string | null; private refreshWatcher?: MCPSessionOptions['refreshWatcher']; + private readonly exploreMode?: string | null; + private readonly advertisedMode?: string; /** * What `codegraph_explore` has already returned to THIS client, per project * (CG-17). Owned by the session, not the engine: the daemon shares one engine @@ -153,6 +159,8 @@ export class MCPSession { ) { this.explicitProjectPath = opts.explicitProjectPath ?? null; this.refreshWatcher = opts.refreshWatcher; + this.exploreMode = opts.exploreMode; + this.advertisedMode = opts.advertisedMode; } /** @@ -367,7 +375,9 @@ export class MCPSession { await this.retryInitIfNeeded(); if (process.env.CODEGRAPH_MCP_DEBUG) process.stderr.write(`[mcp-debug] toolsCall ${toolName} id=${String(request.id)} dispatch\n`); - const result = await this.engine.getToolHandler().execute(toolName, toolArgs, this.exploreSession); + const result = await this.engine.getToolHandler().execute( + toolName, toolArgs, this.exploreSession, this.exploreMode, this.advertisedMode, + ); if (process.env.CODEGRAPH_MCP_DEBUG) process.stderr.write(`[mcp-debug] toolsCall ${toolName} id=${String(request.id)} done\n`); this.transport.sendResult(request.id, result); // After the reply is on the wire — telemetry must never delay a tool diff --git a/src/mcp/tools.ts b/src/mcp/tools.ts index f0bd5150f..e446e04ef 100644 --- a/src/mcp/tools.ts +++ b/src/mcp/tools.ts @@ -469,6 +469,406 @@ export function getExploreOutputBudget(fileCount: number): ExploreOutputBudget { }; } +/** The output shape of `codegraph_explore`. */ +export type ExploreMode = 'map' | 'snippets' | 'source'; +export const EXPLORE_MODES: readonly ExploreMode[] = ['map', 'snippets', 'source']; +/** + * The product default (one flip point). The 2026-11-01 three-turn A/B measured + * map below source on both corpora (gin 8,369 vs 13,327 residual tokens; n8n + * 20,430 vs 28,806), so a session starts with the map and fetches bodies with + * codegraph_node or mode: "source" only when it needs them. + */ +export const DEFAULT_EXPLORE_MODE: ExploreMode = 'map'; +const EXPLORE_MAP_MAX_CHARS = 4000; +const EXPLORE_SNIPPETS_MAX_CHARS = 8000; +const EXPLORE_SNIPPET_LINES = 12; + +export function exploreModeOption(value: unknown): ExploreMode | undefined { + return typeof value === 'string' && (EXPLORE_MODES as readonly string[]).includes(value) + ? (value as ExploreMode) + : undefined; +} + +function projectExploreMode(projectRoot: string): ExploreMode | undefined { + try { + const parsed = JSON.parse(readFileSync(resolvePath(projectRoot, 'codegraph.json'), 'utf8')) as { + explore?: { mode?: unknown }; + }; + return exploreModeOption(parsed?.explore?.mode); + } catch { + return undefined; + } +} + +/** + * Per-call `mode`, then `CODEGRAPH_EXPLORE_MODE`, then the project's + * `codegraph.json` `explore.mode`, then `map`. A compact map is the default: + * the source bodies are what make explore's answer stay resident in the + * agent's context for every later turn, and `codegraph_node` serves one body + * far more cheaply when an edit actually needs it. `source` restores the + * previous verbatim output exactly. + */ +export function resolveExploreMode( + arg: unknown, + projectRoot: string, + env: NodeJS.ProcessEnv = process.env, + sessionMode?: unknown, +): ExploreMode { + const call = exploreModeOption(arg); + if (call) return call; + if (sessionMode !== undefined) { + const session = exploreModeOption(sessionMode); + if (session) return session; + // A proxy hello with no env override means the daemon's inherited env is + // not this session's env; fall through to the project layer instead. + return projectExploreMode(projectRoot) ?? DEFAULT_EXPLORE_MODE; + } + return exploreModeWithoutProject(env) + ?? projectExploreMode(projectRoot) + ?? DEFAULT_EXPLORE_MODE; +} + +/** The layers that do not need an open project: per-call is handled by the caller. */ +export function exploreModeWithoutProject(env: NodeJS.ProcessEnv = process.env): ExploreMode | undefined { + return exploreModeOption(env.CODEGRAPH_EXPLORE_MODE); +} + +/** + * One compact map row per selected symbol. Explicitly requested nodes (query + * names, line anchors, the flow spine) go first, so a row limit cannot drop the + * symbol the caller actually named. Returns how many eligible rows were left out. + */ +export function compactSymbolRows( + nodes: readonly Node[], + max: number, + priority: ReadonlySet = new Set(), +): { rows: string[]; omitted: number } { + const eligible = nodes + .filter((node) => node.kind !== 'export' && node.kind !== 'import' + && node.startLine != null && !isConfigLeafNode(node)) + .map((node, index) => ({ node, index })) + .sort((a, b) => + (Number(priority.has(b.node.id)) - Number(priority.has(a.node.id))) || (a.index - b.index)) + .map(({ node }) => node); + const rows: string[] = []; + const seen = new Set(); + for (const node of eligible) { + if (rows.length >= max) break; + const key = `${node.name}\u0000${node.startLine}`; + if (seen.has(key)) continue; + seen.add(key); + const signature = node.signature?.replace(/\s+/g, ' ').trim(); + const suffix = signature + ? ` — ${signature.length > 120 ? `${signature.slice(0, 117)}…` : signature}` + : ''; + rows.push(`- \`${node.name}\` (${node.kind}) :${node.startLine}${suffix}`); + } + return { rows, omitted: eligible.length - rows.length }; +} + +/** + * Put the compact symbol rows under their file headers before the source is + * dropped. The header alone names symbols without lines; the map's contract is + * location and signature, so the rows carry both. + */ +export function injectCompactSymbolTables( + text: string, + tables: ReadonlyMap, +): string { + if (tables.size === 0) return text; + const out: string[] = []; + for (const line of text.split('\n')) { + out.push(line); + const header = /^\*\*`(.+?)`\*\*/.exec(line); + if (!header) continue; + const rows = tables.get(header[1]!); + if (rows) out.push(...rows); + } + return out.join('\n'); +} + +/** + * A fence longer than any backtick run in the body, so a Markdown excerpt's + * own ``` block cannot close the wrapper (visible with line numbers off). + */ +function sourceFence(body: string): string { + let longest = 3; + for (const run of body.match(/`{3,}/g) ?? []) longest = Math.max(longest, run.length + 1); + return '`'.repeat(longest); +} + +/** + * Reduce a rendered explore response to a map or snippets. The renderer above + * has already chosen and ordered the files; this keeps its summary, flow, + * blast radius, a bounded Relationships list and the per-file symbol headers, + * and drops the source fences (map) or keeps the first `EXPLORE_SNIPPET_LINES` + * lines of each returned excerpt (snippets). + * + * Sections are tracked explicitly rather than by "have we reached Source + * Code": a documentation answer puts the graph sections AFTER the source, so + * an `inSource` flag that never clears silently deletes them. + */ +export function compactExploreText( + text: string, + mode: Exclude, + nodeServed = true, +): string { + type Section = 'preamble' | 'blast' | 'discussed' | 'relationships' | 'source' | 'pointers'; + let section: Section = 'preamble'; + const out: string[] = []; + let inFence = false; + let fence = ''; + let snippet = 0; + let queryEcho: string | null = null; + let blastEntries = 0; + let blastOmitted = 0; + let relKinds = 0; + let relEntries = 0; + let relOmitted = 0; + let relKindsOmitted = 0; + let skipRelKind = false; + let pointerEntries = 0; + let pointerOmitted = 0; + + const plural = (n: number, one: string, many: string) => `${n} ${n === 1 ? one : many}`; + const flushOmissions = () => { + if (section === 'blast' && blastOmitted > 0) { + out.push(`- (+${plural(blastOmitted, 'more blast-radius entry', 'more blast-radius entries')} omitted by the compact view)`); + blastOmitted = 0; + } + if (section === 'relationships' && (relOmitted > 0 || relKindsOmitted > 0)) { + const bits: string[] = []; + if (relOmitted > 0) bits.push(plural(relOmitted, 'more relationship entry', 'more relationship entries')); + if (relKindsOmitted > 0) bits.push(plural(relKindsOmitted, 'more relationship kind', 'more relationship kinds')); + out.push(`- (+${bits.join(', ')} omitted by the compact view)`); + relOmitted = 0; + relKindsOmitted = 0; + } + if (section === 'pointers' && pointerOmitted > 0) { + out.push(`- (+${plural(pointerOmitted, 'more file', 'more files')} omitted by the compact view)`); + pointerOmitted = 0; + } + }; + + const compactQuery = (query: string) => { + const normalized = query.replace(/\s+/g, ' ').trim(); + return normalized.length > 160 ? `${normalized.slice(0, 157)}…` : normalized; + }; + for (const line of text.split('\n')) { + if (queryEcho !== null) { + queryEcho += ` ${line}`; + if (line.endsWith('**')) { + out.push(`**Exploration: ${compactQuery(queryEcho.slice('**Exploration: '.length, -2))}**`); + queryEcho = null; + } + continue; + } + const relKindLine = /^\*\*[^*\n]+:\*\*$/.test(line); + if (!inFence) { + if (line.startsWith('**Exploration: ')) { + if (line.endsWith('**')) { + out.push(`**Exploration: ${compactQuery(line.slice('**Exploration: '.length, -2))}**`); + } else { + queryEcho = line; + } + continue; + } + if (line === '**Source Code**') { + flushOmissions(); + section = 'source'; + out.push(mode === 'map' + ? `**Source Code** — omitted in map mode; ${nodeServed ? 'fetch a body with codegraph_node includeCode: true.' : 'pass mode: "source" for the verbatim source.'}` + : line); + continue; + } + if (line.startsWith('**Blast radius')) { + flushOmissions(); + section = 'blast'; + blastEntries = 0; + out.push(line); + continue; + } + if (line.startsWith('**Discussed in earlier sessions')) { + flushOmissions(); + section = 'discussed'; + out.push(line); + continue; + } + if (line.startsWith('**Relationships**')) { + flushOmissions(); + section = 'relationships'; + relKinds = 0; + relEntries = 0; + skipRelKind = false; + out.push(line); + continue; + } + if (line.startsWith('**Not shown above')) { + flushOmissions(); + section = 'pointers'; + out.push(line); + continue; + } + if (/^\*\*`.*`\*\*/.test(line)) { + if (section === 'source') { + out.push(line); + continue; + } + } else if (line.startsWith('**') && !(section === 'relationships' && relKindLine)) { + // A new top-level section; after source this is how the doc-tier graph + // sections are preserved instead of dropped. + flushOmissions(); + section = 'preamble'; + out.push(line); + continue; + } + } + + if (inFence) { + if (/^`{3,}\s*$/.test(line) && (line.match(/^`+/)![0].length >= fence.length)) { + inFence = false; + if (mode === 'snippets') { + if (snippet >= EXPLORE_SNIPPET_LINES) out.push('...'); + out.push(line); + } + continue; + } + if (mode === 'snippets' && snippet < EXPLORE_SNIPPET_LINES) { + out.push(line); + snippet++; + } + continue; + } + + if (/^> \*\*Complete source for .*included above|source above is verbatim|do NOT re-read/i.test(line)) { + continue; + } + + switch (section) { + case 'source': { + const openFence = /^(`{3,})/.exec(line); + if (openFence) { + inFence = true; + fence = openFence[1]!; + snippet = 0; + if (mode === 'snippets') out.push(line); + continue; + } + // Injected symbol rows sit outside the fence, one per selected symbol. + if (line.startsWith('- ')) { + out.push(line); + continue; + } + if (line === '---') out.push(line); + continue; + } + case 'pointers': { + if (line.startsWith('- ')) { + if (pointerEntries < 10) { + out.push(line); + pointerEntries++; + } else { + pointerOmitted++; + } + continue; + } + if (line.trim()) out.push(line); + continue; + } + case 'blast': { + if (line.startsWith('- ')) { + if (blastEntries < 6) { + out.push(line); + blastEntries++; + } else { + blastOmitted++; + } + continue; + } + if (line.trim()) out.push(line); + continue; + } + case 'discussed': { + out.push(line); + continue; + } + case 'relationships': { + if (relKindLine) { + if (relKinds < 3) { + out.push(line); + relKinds++; + relEntries = 0; + skipRelKind = false; + } else { + relKindsOmitted++; + skipRelKind = true; + } + continue; + } + if (line.startsWith('- ')) { + const more = /^- \.\.\. and (\d+) more$/.exec(line); + if (more) { + if (!skipRelKind) out.push(line); + else relOmitted += Number(more[1]); + continue; + } + if (!skipRelKind && relEntries < 4) { + out.push(line); + relEntries++; + } else { + relOmitted++; + } + continue; + } + if (line.trim() && !skipRelKind) out.push(line); + continue; + } + default: { + if (line.startsWith('> These sites choose their call target at runtime')) { + const candidate = nodeServed ? line : line.replace(' or codegraph_node', ''); + out.push(mode === 'map' + ? candidate.replace( + /; source for the sites above is included below\.$/, + nodeServed + ? '; fetch a candidate\'s body with codegraph_node includeCode: true.' + : '; pass mode: "source" for the body.', + ) + : candidate.replace( + 'source for the sites above is included below.', + 'source for the sites above is excerpted below; for a site past the excerpt, use codegraph_explore mode: "source".', + )); + continue; + } + if (mode === 'map' && line.startsWith('> Source for these symbols follows')) { + out.push(nodeServed + ? '> Source is omitted in compact mode; the symbols and file headers above are the map. Fetch a body with codegraph_node includeCode: true.' + : '> Source is omitted in compact mode; the symbols and file headers above are the map. Pass mode: "source" for the body.'); + continue; + } + out.push(line); + continue; + } + } + } + flushOmissions(); + + let body = out.join('\n').replace(/\n{3,}/g, '\n\n').trimEnd(); + const maxChars = mode === 'map' ? EXPLORE_MAP_MAX_CHARS : EXPLORE_SNIPPETS_MAX_CHARS; + if (body.length > maxChars) { + const cut = body.slice(0, maxChars - 160); + const at = cut.lastIndexOf('\n'); + body = (at > 0 ? cut.slice(0, at) : cut) + + '\n\n... (compact response truncated; narrow the query or name the symbols you need)'; + } + const followUp = nodeServed + ? 'For one symbol\'s body, call codegraph_node with its name and `includeCode: true`; a container returns its member outline, so pass `mode: "source"` for its full body. For the full source, call codegraph_explore with mode: "source".' + : 'For the full source, call codegraph_explore with mode: "source".'; + const hint = mode === 'map' + ? `> Source omitted in compact map mode. ${followUp}` + : `> Only the first ${EXPLORE_SNIPPET_LINES} lines of each returned excerpt are shown. ${followUp}`; + return body + '\n\n' + hint + '\n'; +} + // ── Explore relevance scoring (CG-10 / #1500) ────────────────────────────── // // A file earns its slice of the explore envelope from the symbols in it that the @@ -2071,7 +2471,7 @@ export const tools: ToolDefinition[] = [ }, { name: 'codegraph_node', - description: 'Two modes. (1) Read a file: pass `file` (a path or basename) with no `symbol` to get that file\'s current on-disk source with line numbers in Read\'s shape (`\\t`, safe to Edit from), narrowable with `offset`/`limit`, plus a one-line note of which files depend on it. (2) One symbol you can name: its location, signature, verbatim source (includeCode=true) and caller/callee trail in one call, so before changing it you see what calls it and what your edit would break. For an ambiguous name it returns every matching definition\'s body; pass `file`/`line` to pin one. Use codegraph_explore for several related symbols or the full flow.', + description: 'Two modes. (1) Read a file: pass `file` (a path or basename) with no `symbol` to get that file\'s current on-disk source with line numbers in Read\'s shape (`\\t`, safe to Edit from), narrowable with `offset`/`limit`, plus a one-line note of which files depend on it. (2) One symbol you can name: its location, signature, verbatim source (includeCode=true; a container returns its member outline) and caller/callee trail in one call, so before changing it you see what calls it and what your edit would break. For an ambiguous name it returns every matching definition\'s body; pass `file`/`line` to pin one. Use codegraph_explore for several related symbols or the full flow.', inputSchema: { type: 'object', properties: { @@ -2081,7 +2481,7 @@ export const tools: ToolDefinition[] = [ }, includeCode: { type: 'boolean', - description: 'Symbol mode: include the symbol\'s full body (default: false). Ignored in file mode, which always returns source unless `symbolsOnly` is set.', + description: 'Symbol mode: include the symbol\'s source (default: false). A container returns its member outline; use codegraph_explore mode: "source" for its full body. Ignored in file mode, which always returns source unless `symbolsOnly` is set.', default: false, }, file: { @@ -2110,10 +2510,13 @@ export const tools: ToolDefinition[] = [ required: [], }, annotations: READ_ONLY_ANNOTATIONS, + // In map mode node is explore's body-on-demand follow-up, so it must be + // callable without a ToolSearch step; it is not listed in source mode. + _meta: { 'anthropic/alwaysLoad': true }, }, { name: 'codegraph_explore', - description: 'Returns the verbatim, line-numbered source of the code relevant to a query, grouped by file (the same `\\t` shape as Read, safe to Edit from), plus the call path among the named symbols — including dynamic-dispatch hops grep cannot follow — and, when it fits, a summary of what depends on them. Use it first for questions about indexed code (how X works, where X is, a flow, a bug, a documentation section) and to load the symbols you are about to change. The query is a natural-language question or a bag of symbol names and file paths; named items are prioritised, and a question about the working changes ("my changes", "this branch", `main..HEAD`) is answered from the git diff. Output is capped by project size (about 13K–24K characters); files that did not fit may be listed by name for a follow-up call. Treat returned ranges as already Read, except files the response flags as pending sync or changed on disk. It does not return files codegraph does not index, such as most configs. One call usually covers a question.', + description: 'Returns a compact map of the code relevant to a query by default: the matching symbols with file:line and signatures, the call path among the named symbols — including dynamic-dispatch hops grep cannot follow — and, when it fits, a summary of what depends on them. It does not include source bodies. It is for deciding where to look; before editing a symbol, call codegraph_node with its name and includeCode: true (or pass mode: "source") for the verbatim, line-numbered source. Use it first for questions about indexed code (how X works, where X is, a flow, a bug, a documentation section). The query is a natural-language question or a bag of symbol names and file paths; named items are prioritised, and a question about the working changes ("my changes", "this branch", `main..HEAD`) is answered from the git diff. Output is capped at about 4K characters in map mode and 8K in snippets; source mode uses the project-sized 13K–24K cap. Treat source returned by mode "source" as already Read, except files the response flags as pending sync or changed on disk. It does not return files codegraph does not index, such as most configs. One call usually covers a question.', inputSchema: { type: 'object', properties: { @@ -2123,7 +2526,12 @@ export const tools: ToolDefinition[] = [ }, maxFiles: { type: 'number', - description: 'Maximum number of files to return source from, 1–20. Omit it to use a default sized to the project (4 files below 150 indexed files, 5 below 500, 8 otherwise); when omitted and the query quotes a literal, the cap rises to the number of files holding it, up to 12. Total output is also capped by characters (about 13K–24K by project size), so a higher maxFiles spreads that budget over more files rather than returning more text.', + description: 'Maximum number of files to include, 1–20. Omit it to use a default sized to the project (4 files below 150 indexed files, 5 below 500, 8 otherwise); when omitted and the query quotes a literal, the cap rises to the number of files holding it, up to 12. In source mode the total is also capped by characters (about 13K–24K by project size), so a higher maxFiles spreads that budget over more files rather than returning more text.', + }, + mode: { + type: 'string', + enum: ['map', 'snippets', 'source'], + description: 'Output shape. "map" (the default unless codegraph.json explore.mode or CODEGRAPH_EXPLORE_MODE overrides it) returns symbols, file:line, signatures, the flow and blast radius without source bodies. "snippets" adds the first 12 lines of each returned excerpt. "source" returns the full verbatim, line-numbered source (the previous behavior).', }, projectPath: projectPathProperty, }, @@ -2340,7 +2748,8 @@ function withRequiredProjectPath(defs: ToolDefinition[]): ToolDefinition[] { export function getStaticTools(): ToolDefinition[] { const raw = process.env.CODEGRAPH_MCP_TOOLS; if (!raw || !raw.trim()) { - return tools.filter(t => DEFAULT_MCP_TOOLS.has(t.name.replace(/^codegraph_/, ''))); + const surface = defaultSurfaceFor(exploreModeWithoutProject() ?? DEFAULT_EXPLORE_MODE); + return tools.filter(t => surface.has(t.name.replace(/^codegraph_/, ''))); } const allow = new Set(raw.split(',').map(s => s.trim().replace(/^codegraph_/, '')).filter(Boolean)); return allow.size ? tools.filter(t => allow.has(t.name.replace(/^codegraph_/, ''))) : tools; @@ -2362,6 +2771,20 @@ export function getStaticTools(): ToolDefinition[] { */ const DEFAULT_MCP_TOOLS = new Set(['explore', 'sessions']); +/** + * The default surface for one output mode. A compact response carries no source + * body, so `codegraph_node` is listed beside `codegraph_explore` as the way to + * fetch one symbol's body — without it the agent has no on-demand source path + * and asks explore for `mode: "source"` instead, which defeats the point. In + * `source` mode explore already returns the body, so listing node again only + * adds a mis-pick candidate (#2486). + */ +function defaultSurfaceFor(mode: ExploreMode): ReadonlySet { + return mode === 'source' + ? DEFAULT_MCP_TOOLS + : new Set([...DEFAULT_MCP_TOOLS, 'node']); +} + /** Call-site lines codegraph_callers shows per caller (#2909), so one busy caller stays one line. */ const MCP_CALL_SITES_SHOWN = 10; @@ -2622,6 +3045,18 @@ export class ToolHandler { * tool is then truly absent from ListTools rather than merely denied on call. * Matching is on the short form, so "node" and "codegraph_node" both work. */ + /** The output mode explore will use for the open project (or env-only with none). */ + private servedExploreMode(): ExploreMode { + return this.cg + ? resolveExploreMode(undefined, this.cg.getProjectRoot()) + : exploreModeWithoutProject() ?? DEFAULT_EXPLORE_MODE; + } + + /** The default surface for the mode this handler will serve. */ + private defaultSurface(): ReadonlySet { + return defaultSurfaceFor(this.servedExploreMode()); + } + private toolAllowlist(): Set | null { const raw = process.env.CODEGRAPH_MCP_TOOLS; if (!raw || !raw.trim()) return null; @@ -2646,7 +3081,7 @@ export class ToolHandler { private isToolServed(name: string): boolean { const short = name.replace(/^codegraph_/, ''); const allow = this.toolAllowlist(); - return allow ? allow.has(short) : DEFAULT_MCP_TOOLS.has(short); + return allow ? allow.has(short) : this.defaultSurface().has(short); } /** @@ -2660,9 +3095,10 @@ export class ToolHandler { // No explicit allowlist → the default surface (see // DEFAULT_MCP_TOOLS for the evidence). An allowlist replaces the // default entirely, so any defined tool can be re-enabled. + const surface = this.defaultSurface(); let visible = allow ? tools.filter(t => allow.has(t.name.replace(/^codegraph_/, ''))) - : tools.filter(t => DEFAULT_MCP_TOOLS.has(t.name.replace(/^codegraph_/, ''))); + : tools.filter(t => surface.has(t.name.replace(/^codegraph_/, ''))); // No default project loaded → no-root-index case (#993): a gateway server // started outside any repo, or a monorepo root whose indexes live in // sub-projects. With nothing to fall back to, EVERY call needs an explicit @@ -3314,6 +3750,8 @@ export class ToolHandler { toolName: string, args: Record, sessionState?: ExploreSessionState, + exploreMode?: unknown, + advertisedMode?: unknown, ): Promise { // A cross-project session search (experimental) names its projects itself and // needs no default one: anchor the call on the first listed project that has an @@ -3421,6 +3859,10 @@ export class ToolHandler { // structured-clone boundary into a worker, where a closure or a handler // field could not follow. const dispatchArgs = this.withSessionView(toolName, args, sessionState); + if (toolName === 'codegraph_explore') { + if (exploreMode !== undefined) dispatchArgs.__exploreMode = exploreMode; + if (advertisedMode !== undefined) dispatchArgs.__advertisedMode = advertisedMode; + } // The default may have appeared after the workers started. Pass the // main thread's current root explicitly; with no root or projectPath, // keep the main handler's workspace-specific not-indexed guidance. @@ -4811,7 +5253,11 @@ export class ToolHandler { } } - private async handleExplore(args: Record): Promise { + private async handleExplore( + args: Record, + sessionMode?: unknown, + advertisedMode?: unknown, + ): Promise { const rawQuery = this.validateString(args.query, 'query'); if (typeof rawQuery !== 'string') return rawQuery; // One normalization point so the flow-builder, relevance search, and @@ -4820,6 +5266,18 @@ export class ToolHandler { const cg = this.getCodeGraph(args.projectPath as string | undefined); const projectRoot = cg.getProjectRoot(); + // The proxy's session override travels with the call; the daemon's own env + // is only the fallback when the launching session had none. + const sessionModeArg = '__exploreMode' in args ? args.__exploreMode : sessionMode; + const mode = resolveExploreMode(args.mode, projectRoot, process.env, sessionModeArg); + // The follow-up must match the surface the client was actually offered, not + // the per-call mode: a map requested from a source-mode session cannot call + // node. The proxy's hello carries that advertised mode. + const advertised = exploreModeOption( + '__advertisedMode' in args ? args.__advertisedMode : advertisedMode, + ) ?? this.servedExploreMode(); + const allow = this.toolAllowlist(); + const nodeServed = allow ? allow.has('node') : defaultSurfaceFor(advertised).has('node'); // Resolve adaptive output budget from project size. Falls back to the // largest-tier defaults if stats aren't available, which preserves @@ -6369,8 +6827,9 @@ export class ToolHandler { ); // Step 3: Build relationship map + const headingQuery = query.replace(/\s+/g, ' ').trim(); const lines: string[] = [ - `**Exploration: ${query}**`, + `**Exploration: ${headingQuery.length > 160 ? `${headingQuery.slice(0, 157)}…` : headingQuery}**`, '', // Curated summary — filled in after the source loop (see below). We do NOT // report `subgraph.nodes.size` / `fileGroups.size` here: that's the raw @@ -7175,7 +7634,8 @@ export class ToolHandler { backReferencedFiles.push(filePath); } if (body.length > 0) { - lines.push('```' + lang, body, '```', ''); + const fence = sourceFence(body); + lines.push(fence + lang, body, fence, ''); // ```lang \n body \n ``` \n '' \n — exact, same as the header above. totalChars += body.length + lang.length + 11; sourceSpent += body.length; @@ -7199,11 +7659,12 @@ export class ToolHandler { noteEmitted(filePath, opts.covered, 0, fingerprint); renderedFilePaths.push(filePath); if (!suppressedFallback && opts.fullBody.length > 0) { + const fence = sourceFence(opts.fullBody); suppressedFallback = { filePath, at, replacing: lines.length - at, - section: [opts.header, '', '```' + lang, opts.fullBody, '```', ''], + section: [opts.header, '', fence + lang, opts.fullBody, fence, ''], sourceChars: opts.fullBody.length, overhead: opts.overhead, ranges: opts.fullRanges, @@ -7489,8 +7950,9 @@ export class ToolHandler { if (parts.length > 0) { const suffix = `sections: ${shown.join(' · ')}${elided ? ' (long sections show only the rows that match the question)' : ''} — the rest of the file is not shown`; const header = fileSectionHeader(filePath, suffix); - lines.push(header, '', '```' + lang, parts.join('\n\n'), '```', ''); - docRenderedLines.set(filePath, { marker: header + '\n\n```' + lang + '\n', lines: renderedLines }); + const fence = sourceFence(parts.join('\n\n')); + lines.push(header, '', fence + lang, parts.join('\n\n'), fence, ''); + docRenderedLines.set(filePath, { marker: header + '\n\n' + fence + lang + '\n', lines: renderedLines }); totalChars += used + 120; noteEmitted(filePath, mergeRanges([...picked.values()].flat() .map(ln => ({ start: ln, end: ln }))), used, fingerprint); @@ -9044,7 +9506,7 @@ export class ToolHandler { ? exploreCompletenessNotes(filesIncluded, trimmedShown, [...renderedFilePaths, ...fileGroups.keys()]) .map((note) => ['', '---', note]) : anyFileTrimmed || trimmedShown.length > 0 - ? [['', `> Some file sections were trimmed for size. Elided symbols are named inside gap markers as \`name (file:line)\` and preferred in the file header — run another \`codegraph_explore\`${this.isToolServed('node') ? ' (or `codegraph_node`)' : ''} with those exact names for their source.`]] + ? [['', `> Some file sections were trimmed for size. Elided symbols are named inside gap markers as \`name (file:line)\` and preferred in the file header — run another \`codegraph_explore\`${nodeServed ? ' (or `codegraph_node`)' : ''} with those exact names for their source.`]] : []; /** Whether a trimmed section survives in `text` — picks a fallback note's wording after a cut. */ const trimmedIn = (text: string): boolean => @@ -9214,6 +9676,30 @@ export class ToolHandler { gatherNotice(uncoveredGather), ); finalText = finalText.replace(summaryPlaceholder, summaryLine); + if (mode !== 'source') { + const priority = new Set([ + ...subgraph.roots, ...exactNodeIds, ...flow.namedNodeIds, ...flow.pathNodeIds, + ]); + const tables = new Map(); + for (const fp of survivors) { + const group = fileGroups.get(fp); + // Doc headings are navigation, not code to locate; a smaller list keeps + // the doc-tier graph tail inside the map budget. + const symbols = compactSymbolRows(group?.nodes ?? [], docTierFiles.has(fp) ? 3 : 6, priority); + const rows = [...symbols.rows]; + if (symbols.omitted > 0) { + rows.push(`- (+${symbols.omitted} more symbol${symbols.omitted === 1 ? '' : 's'} in this file)`); + } + // A quoted-prose or path anchor can land in a file with no symbol rows; + // its line is the whole answer, so keep it even when the source is gone. + for (const anchor of lineAnchors.filter((a) => a.file === fp).slice(0, 3)) { + rows.push(`- line :${anchor.start}${anchor.end > anchor.start ? `-${anchor.end}` : ''}`); + } + if (rows.length > 0) tables.set(fp, rows); + } + finalText = injectCompactSymbolTables(finalText, tables); + finalText = compactExploreText(finalText, mode, nodeServed); + } // Emit the allocation diagnostic from the FINAL text, so per-file bytes and // shares account for the hard-ceiling truncation above (CG-4). @@ -9229,16 +9715,20 @@ export class ToolHandler { // be re-served for nothing. const emittedFiles: ExploreFileEmission[] = []; let sourceBytes = 0; - for (const fp of survivors) { - const emitted = emittedByFile.get(fp); - if (!emitted || emitted.ranges.length === 0) continue; - emittedFiles.push({ - path: fp, - ranges: emitted.ranges, - bytes: emitted.bytes, - fingerprint: emitted.fingerprint, - }); - sourceBytes += emitted.bytes; + // A compact response carries no source body, so it records no emitted + // source; a later call must not back-reference bytes the agent never saw. + if (mode === 'source') { + for (const fp of survivors) { + const emitted = emittedByFile.get(fp); + if (!emitted || emitted.ranges.length === 0) continue; + emittedFiles.push({ + path: fp, + ranges: emitted.ranges, + bytes: emitted.bytes, + fingerprint: emitted.fingerprint, + }); + sourceBytes += emitted.bytes; + } } // Include graph-only and omitted/deleted files, not just emitted source. const answerPaths = new Set(fileGroups.keys()); diff --git a/vitest.config.mts b/vitest.config.mts index 31a128ebd..8b2c3c5b1 100644 --- a/vitest.config.mts +++ b/vitest.config.mts @@ -39,6 +39,12 @@ export default defineConfig({ * they inject their own `env` via the Telemetry constructor. */ CODEGRAPH_TELEMETRY: '0', + /** + * The product default is the compact map. Most suites were written + * against the source rendering, so pin them to it; the map default has + * its own tests in __tests__/explore-compact.test.ts. + */ + CODEGRAPH_EXPLORE_MODE: 'source', /** * Every spawned MCP server, daemons included, would otherwise ask GitHub * for the latest release, because the sandboxed home has no cached