Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

- **Your earlier agent sessions are searchable: `codegraph sessions` and the `codegraph_sessions` tool.** A code graph answers "how does X work"; it cannot answer "why is X like this" or "what did the last session decide about Y" — that history lives in the transcripts the agent already wrote, hundreds of megabytes of JSONL nobody greps. CodeGraph now indexes the prose of a project's Claude Code sessions (`~/.claude/projects/<slug>/`: prompts, replies and compaction summaries; tool calls, results and thinking stay out) into an FTS5 table with porter stemming and BM25 rank, in its own `.codegraph/sessions.db` beside the graph. The index refreshes on each call for files whose size or mtime moved, so a query after one live session costs tens of milliseconds and the first index of a few hundred transcripts about a second. A hit names its session id, title, role, time and the matching passage; `role`, `sinceDays`, `session` (an id prefix) and `any` (OR the words) narrow or widen it. The tool joins `codegraph_explore` in the default MCP surface — a different question over a different corpus, so it cannot steer a mis-pick against explore — and the CLI command prints the same text for subagents without MCP. `"sessions": false` in `codegraph.json` opts a project out; `CODEGRAPH_SESSIONS_DIR` points at another transcript directory. Readers are one module per agent host, Claude Code first.

- **Markdown is indexed, and a documentation question gets the section, not the graph.** Every `.md` file's headings, sections, tables and links are nodes (the extractor from #361), and a doc-shaped `codegraph_explore` query that names a markdown file now renders that file's best sections first and whole — the top three by idf-weighted line hits, a heading the query covers word for word counted as named, 8k characters per file — with the blast-radius, relationships and "additional files" blocks held back unless a code file rendered too. Measured on a 109-file docs corpus under headless Claude Code, 36 cells over three rounds: the right file and section in every call, median 1 tool call against 4 for Grep-then-Read, 36 of 36 correct. Code answers keep their shape: markdown nodes leave a subgraph the doc tier did not seed, a markdown body is never mistaken for a generated-file header, and the explore budget tiers count code files only, so a README-heavy repo does not cross a breakpoint. The server instructions say markdown is indexed, which the branch's own text still denied. (#361, #1439)
- **Markdown is indexed, and a documentation question gets the section, not the graph.** Every `.md` file's headings, sections and links are nodes, with tables, commands and list items kept in the section's searchable text (the extractor from #361), and a doc-shaped `codegraph_explore` query that names a markdown file now renders that file's best sections first and whole — the top three by idf-weighted line hits, a heading the query covers word for word counted as named, 8k characters per file — with the blast-radius, relationships and "additional files" blocks held back unless a code file rendered too. Measured on a 109-file docs corpus under headless Claude Code, 36 cells over three rounds: the right file and section in every call, median 1 tool call against 4 for Grep-then-Read, 36 of 36 correct. Code answers keep their shape: markdown nodes leave a subgraph the doc tier did not seed, a markdown body is never mistaken for a generated-file header, and the explore budget tiers count code files only, so a README-heavy repo does not cross a breakpoint. The server instructions say markdown is indexed, which the branch's own text still denied. (#361, #1439)

- Installers can start or reuse a checkout's active watcher without replacing existing writers, and MCP launchers can preserve existing daemons across reconnects with `--preserve-existing`.

Expand All @@ -102,6 +102,8 @@ and adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

- **A documentation link to a file that is not there no longer gets its own graph node.** A Markdown link or path mention whose target basename no indexed file has minted an import node and a `contains` edge, even though the resolver's file-path arm could not match it. On supabase a full index now skips 10,394 of 14,038 Markdown import nodes and their `contains` edges. The reference is always kept, so the resolver's route, module and table arms still match one (a relative `api/users.json` link included), `codegraph sync` can resolve a target added later from the parked failed reference, and a skipped target stays in a separate `links:` part of the file digest so it remains searchable past the 12-item display digest. Resolved Markdown edges are byte-identical on supabase (3,058), n8n, pretix and discourse, and the failed Markdown reference count is unchanged (11,368 on supabase). A full supabase index is a median 16.176 s to 15.953 s (Hyperfine 2.0.0, 8 runs per build, two ABBA rounds, 0.223 s/1.4%, interquartile ranges not overlapping). Re-index Markdown-heavy projects after upgrading.

- **A Markdown table row, list item or fenced command is section text, not its own node.** Each row created a `constant` or `function` node plus a `contains` edge even though the text already lived in the section's docstring and searchable text. A full index now keeps those rows in the section, attaches their references to it (a script a command runs keeps its `calls` edge), and appends a short `commands:` summary so a command past the docstring's 600-character cap stays searchable. An unlabeled fence no longer turns output, trees or JSON fragments into commands; a labeled shell fence keeps every line. Text the 600-character section preview drops — late rows, table headers, commands past the 8-command summary, and unrecognized fence lines — is kept on the owning node's new `search_text` column (indexed by the full-text table, not displayed), so a search hit still lands on the section that owns the line. An unlabeled fence recognizes every interpreter `extractScriptPath` supports (`node`, `powershell`, ...). On supabase that removes 6,222 of 27,380 Markdown nodes and 6,247 edges; index time shows no clear change (15.969 s against 16.047 s median, Hyperfine 2.0.0, 8 runs per build, two ABBA rounds, ranges overlapping). n8n drops 3,719 nodes and discourse 869. References that had resolved to a removed row/command node now fail instead of pointing at it (25 on supabase, 3 on n8n, 1 on discourse); references from a row or command move to its section. Doc search still returns the section for a command or table query, and the README, the languages reference and the MCP instructions now describe the section-text model. Re-index Markdown-heavy projects after upgrading.

- **`codegraph sessions` and `codegraph_sessions` can search history before a date.** A search could only look back a number of days (`--since`, `sinceDays`), so its best 100 hits always came from the newest matching sessions and older context could not be reached. `--until <date>` and `until` keep only docs before a date (read as UTC) or a timestamp that names its zone; together with `--since` they bound a window.

- A background sync after editing a widely imported file no longer leaves the MCP daemon holding about 19 GB (n8n, one new export in a barrel imported from 335 files). Re-checking the importers built a native resolver per binding and the process kept about 2.5 MB of each; they now share one. That check went from 38.7 s to 0.5 s, and idle memory after the edit from 19.4 GB to 1.4 GB.
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ Compared with upstream `main` at `cd04b21a` (after v1.6.2), re-checked against t
| Feature | Upstream | Fork | What it does |
|---|:-:|:-:|---|
| Session search (`codegraph sessions`, `codegraph_sessions`) | — | ✓ | Searches this project's earlier Claude Code, Codex, Cursor/T3, OpenCode, AGY, Devin and Grok transcripts and its git commit messages, so an agent can find what a past session decided. `codegraph_explore` also names the sessions that mentioned the symbols it returns. `"sessions": false` in `codegraph.json` turns it off. Experimental: `projects` / `--project` searches up to five projects' histories in one call. |
| Markdown indexing | — | ✓ | Headings, sections, tables and links become graph nodes; a documentation question gets the matching section. Explicit code-file pins remain eligible in mixed documentation/code requests. |
| Markdown indexing | — | ✓ | Headings, sections and local links become graph nodes, with tables, commands and list items kept in the section's searchable text; a documentation question gets the matching section. Explicit code-file pins remain eligible in mixed documentation/code requests. |
| Missing names reported | Reports them | Reports them, and proves absence | Upstream's note says a requested name is not in the index and the defining file may be unindexed. The fork keeps that note and additionally lists a name only when a bounded (300 ms) scan of every hand-written source file — plus file stems and bindings — completes without finding it as any word, so a listed name is absent from the project outright, not just unindexed. An incomplete scan lists nothing rather than guess. |
| Requested-source limits reported | — | ✓ | Explore reports requested files that did not receive pin priority within the file limit, unexamined path-like references beyond its bounded scan, and uncovered indexed continuation ranges after a capped file gather. Existing output limits still apply. |
| Local callee source with file pins | — | ✓ | When a query names functions in pinned files, `codegraph_explore` prioritizes local callee bodies through two call or callback hops, within the file and output budgets. |
Expand Down Expand Up @@ -1134,7 +1134,7 @@ command:
| Solidity | `.sol` | Full support (contracts, libraries, interfaces, structs, enums, modifiers, events, errors, state variables, `import`/`using` directives, `emit`/`revert` calls) |
| Terraform / OpenTofu | `.tf`, `.tfvars`, `.tofu` | Full support (resources, data sources, modules, variables, outputs, providers incl. aliases, `locals`; `var.`/`local.`/`module.`/resource references with Terraform's per-directory scoping enforced; module calls bridged across the boundary — inputs to the child module's variables, `module.M.out` to the child's output, `source` to the module's files; cloudposse/atmos `remote-state` cross-component wiring when the component is statically named; `provider = aws.east` selections resolved up the module tree; `moved`/`import`/`removed`/`check` block references; `.tfvars` assignments linked to the variables they set) |
| Nix | `.nix` | Full support (functions with simple/destructured/curried params, `let`/attrset bindings, `inherit`, `import ./path` file edges — `./dir` resolving through `default.nix` — plus NixOS module `imports = [ ./x.nix ]` lists and `callPackage ./pkg.nix` file edges; call edges; module-system option wiring — a config write like `launchd.user.agents.x = { ... }` links to the module declaring `options.launchd.user.agents`, so option flows trace across modules) |
| Markdown | `.md`, `.mdx`, `.markdown` | Documentation structure (headings, sections, local links, selected table rows and list items, shell command references); this fork only |
| Markdown | `.md`, `.mdx`, `.markdown` | Documentation structure (headings, sections and local links as nodes; tables, commands and list items as searchable section text); this fork only |
| Razor / Blazor | `.cshtml`, `.razor` | Markup linked to the C# it names (`@model`, `@inherits`, components, `@inject`) |
| XML | `.xml` | MyBatis mapper statements linked to their Java mapper methods |
| YAML | `.yml`, `.yaml` | File tracking; Drupal `*.routing.yml` routes and Spring config keys |
Expand Down
39 changes: 18 additions & 21 deletions __tests__/extraction.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -360,8 +360,11 @@ node scripts/release.mjs
expect(headings.map((n) => n.name)).toContain('Project Guide');
expect(headings.map((n) => n.name)).toContain('Release');

const commandNode = result.nodes.find((n) => n.kind === 'function' && n.signature === 'node scripts/release.mjs');
expect(commandNode).toBeDefined();
// The command lives in the Release section's searchable summary, not as
// its own node.
const release = headings.find((n) => n.name === 'Release');
expect(release?.docstring).toContain('commands:');
expect(release?.docstring).toContain('node scripts/release.mjs');

expect(result.unresolvedReferences).toEqual(
expect.arrayContaining([
Expand Down Expand Up @@ -394,19 +397,13 @@ node scripts/release.mjs

const result = extractFromSource('phases/phase4.md', markdown);

const tableRows = result.nodes.filter((n) => n.kind === 'constant' && n.qualifiedName.includes('table-row'));
expect(tableRows.map((n) => n.name)).toEqual(expect.arrayContaining(['P4-S1', 'P4-S2']));

const p4s1 = tableRows.find((n) => n.name === 'P4-S1');
expect(p4s1?.signature).toContain('Template: P4-S1');
expect(p4s1?.signature).toContain('Dispatcher: scripts/csv_search.py::run_p4');

const commandNode = result.nodes.find((n) =>
n.kind === 'function' &&
n.language === 'markdown' &&
n.signature?.includes('python "{script_path}" p4')
);
expect(commandNode).toBeDefined();
// Rows and commands are section text now: no constant/function nodes.
expect(result.nodes.filter((n) => n.kind === 'constant' || n.kind === 'function')).toEqual([]);
const phase4 = result.nodes.find((n) => n.kind === 'module' && n.name === 'Phase 4');
expect(phase4?.docstring).toContain('P4-S1');
expect(phase4?.docstring).toContain('scripts/csv_search.py::run_p4');
expect(phase4?.docstring).toContain('commands:');
expect(phase4?.docstring).toContain('python "{script_path}" p4');

expect(result.unresolvedReferences).toEqual(
expect.arrayContaining([
Expand Down Expand Up @@ -439,12 +436,12 @@ node scripts/release.mjs
`;

const result = extractFromSource('docs/runbook.md', markdown);
const constants = result.nodes.filter((n) => n.kind === 'constant');

expect(constants).toEqual(expect.arrayContaining([
expect.objectContaining({ docstring: 'POST-FENCE references src/auth.ts::login.' }),
expect.objectContaining({ name: 'POST-TABLE' }),
]));
// The list item and the table row are section text, not nodes.
expect(result.nodes.filter((n) => n.kind === 'constant' || n.kind === 'function')).toEqual([]);
const runbook = result.nodes.find((n) => n.kind === 'module' && n.name === 'Runbook');
expect(runbook?.docstring).toContain('POST-FENCE references src/auth.ts::login.');
expect(runbook?.docstring).toContain('POST-TABLE');
expect(result.unresolvedReferences.filter((ref) => ref.referenceName === 'docs/src/auth.ts::login').length).toBeGreaterThanOrEqual(2);
});

it('indexes Setext (underline) headings and skips frontmatter / code fences', () => {
Expand Down
Loading
Loading