fix: bound large-repo memory and traversal work by danusha2345 · Pull Request #1583 · colbymchenry/codegraph

danusha2345 · 2026-08-21T10:16:40Z

Addresses #1553.

Rebased onto current main (6421644). The pieces that landed upstream in the meantime are dropped from this PR:

dropped from this PR	now on main as
size guard in the resolver's `readFileCached`, shared size constant	#2006
dropping watches for a removed directory	#2025
WAL backpressure	#1751
#1606 / #1607 fix	#1614
line lookups in `callback-synthesizer.ts`	left to #1985, which does the same
tag-based CFML files releasing their parse trees	#2296

What remains

The resolver's file and line caches are bounded by bytes held, not only by entry count (lru-cache.ts).
C function-pointer synthesis holds less at once. Its caches are bounded by bytes. Macro captures are copied out so they don't pin whole headers. Only macros used in conditionals are kept, and working data is freed between stages.
Synthesis passes are merged as each one finishes, and results from worker threads still merge in registry order. This is compatible with fix(sync): refresh synthesized edges after incremental changes (#1988) #2033, which only changes what happens after the merge.
Idle query workers are recycled after 60 s, and one fresh worker stays warm. A retired worker is asked to close: it closes its projects, runs a full collection and exits by itself, the path worker-teardown.ts measured as safe on Windows. Only a worker that does not exit within 5 s is terminated, through the pool's own start-aware terminateStarted().
Shortest-path search uses a parent map and a head index instead of copying paths. It keeps fix(graph): enqueue path-search targets once (#1359) #2016's enqueue-once set. fix(graph): enqueue path-search targets once (#1359) #2016's "dense fan-in" test now counts lookups per id instead of queue items; its expectation is unchanged and still fails on the old algorithm.
The impact walk has a node/edge budget (10k/50k by default, 1k/5k from MCP), merged into fix(graph): don't lose dependents within the depth limit to edge order (#1974) #2000's depth-aware walk. Re-expanding a node from a nearer depth is still allowed; only new nodes and new edges spend the budget. The MCP answer says when it was truncated and suggests file or a smaller depth.

Tests

The PR's own tests pass, as do graph, lru-cache, query-pool (including a real worker that retires with exit code 0 and is never terminated), mcp-daemon, worker-teardown, the resolution suites, the C function-pointer and synthesis suites and CFML. Full suite on Linux (Node 22, native kernel): 6,205 passed, 34 skipped, 0 failed. The per-file memos #2362 and #2344 added are untouched: on betaflight src/ and Django the graph is byte-identical with and without this branch. Not run on Windows.

🤖 Generated with Claude Code

danusha2345 · 2026-09-08T14:36:04Z

Merged current main (3adf067, post-#1770) into this branch: head e681dcd. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T14:43:24Z

Merged current main (43271f3) into this branch: head 1b81c76. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T14:57:45Z

Merged current main (85550eb) into this branch: head c48e644. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T15:12:23Z

Merged current main (8733c28) into this branch: head ef854d5. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T15:55:07Z

Merged current main (8c04734) into this branch: head 68cc98f. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T16:33:03Z

Merged current main (ee83636) into this branch: head 8856408. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T16:54:10Z

Merged current main (e720f6c) into this branch: head a3c66ea. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T19:09:13Z

Merged current main (de5adba, post-#1790) into this branch: head 5bbaec8. The only conflict was the js-builtins import line beside memory-budget in resolution/index.ts (plus the CHANGELOG) — both imports kept, no behaviour change. tsc clean; the branch's tests and resolution.test.ts pass.

danusha2345 · 2026-09-08T19:28:23Z

Merged current main (cece072) into this branch: head c47857c. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T20:14:32Z

Merged current main (040ba38) into this branch: head 1cf0ceb. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T20:52:12Z

Merged current main (374b3b4) into this branch: head b6749d7. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T21:56:47Z

Merged current main (71d049c) into this branch: head 8b17110. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T22:36:25Z

Merged current main (9181dd1) into this branch: head 5c6f6a3. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T22:48:35Z

Merged current main (64bd45c) into this branch: head 9fbbb8b. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

danusha2345 · 2026-09-08T23:39:57Z

Merged current main (799f8b2, post-#1806) into this branch: head 810ce53. The only conflict was the import lines at the top of extraction.test.ts (#1806's scanDirectoryAsync / ScanSkipStats beside this branch's getParser); all kept. tsc clean; the branch's tests pass (758).

danusha2345 · 2026-09-09T00:08:33Z

Merged current main (748311f) into this branch: head c0c6354. The only conflict was the CHANGELOG entry; no source conflict. tsc clean, the branch's tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Recycling an idle pool terminated every used worker. On Windows, a worker thread that ends while V8's concurrent marker is still marking its heap can take the whole process down (worker-teardown.ts), and the idle shrink runs in the long-lived daemon after every burst of queries. The pool now retires an idle worker the way the resolver pool closes its workers: it posts 'close', and the worker closes its project connections, runs collectBeforeExit() and calls process.exit(0). A worker that has not exited after 5 s is terminated as before, through terminateStarted(). destroy() is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…mchenry#2876; 11 landed PRs yield to main; EXTRACTION_VERSION 55) Main landed eleven of our PRs as our own commits with the maintainer's additions (colbymchenry#2847, colbymchenry#2848, colbymchenry#2850, colbymchenry#2851, colbymchenry#2853, colbymchenry#2854, colbymchenry#2855, colbymchenry#2856, colbymchenry#2868, colbymchenry#2874, colbymchenry#2875), the traversal half of colbymchenry#2839 (colbymchenry#2860) and the trail hardening of colbymchenry#1667 (colbymchenry#2869). Where the integration carried our original version, main's wins: play.ts, routes.ts, the tsjs kernel extractor, import-resolver's PHP alias lookup, isDecisiveSameFileMatch, matchNewReceiverCall and the tests of those PRs. Kept beside main: colbymchenry#1952's enclosing-scope member pick, colbymchenry#1936's closest-copy confidence, colbymchenry#2258's Kotlin imported-type block, colbymchenry#1875's commitHdlProfile on indexFiles and hdlAccess, colbymchenry#1583's impact budget (the type-hierarchy walk itself is main's), colbymchenry#1736's C #if tests. 22 changelog bullets of closed PRs removed where main has its own wording. EXTRACTION_VERSION 55: Rust receiver chains (colbymchenry#2846), `new C().m` shapes (colbymchenry#2854), Java records (colbymchenry#2868), Go method exportedness (colbymchenry#2857), generic supertypes (colbymchenry#2856). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

…mchenry#2972; 10 landed PRs yield to main; EXTRACTION_VERSION 56) Main landed ten more of our PRs as our own commits with the maintainer's additions (colbymchenry#2923 ← colbymchenry#1936, colbymchenry#2924 ← colbymchenry#1951, colbymchenry#2925 ← colbymchenry#1939, colbymchenry#2930 ← colbymchenry#2921, colbymchenry#2940 ← colbymchenry#1736, colbymchenry#2942 ← colbymchenry#1947, colbymchenry#2950 ← colbymchenry#1952, colbymchenry#2961 ← colbymchenry#1954, colbymchenry#2965 ← colbymchenry#1942, colbymchenry#2972 ← colbymchenry#1933) and the module-path half of colbymchenry#2308 (colbymchenry#2955). Where the integration carried our version, main's wins: the enclosing-scope member pick and its tests, the Go and Java receiver typers, the delegating-call self-loop rule, the duplicated-module copy confidence, the C #if collapse, the module-path resolution of Rust paths, the trail/path hardening in utils.ts. Kept beside main: colbymchenry#1875's Verilog hooks and status fields, colbymchenry#2839's implementations command, colbymchenry#1923, colbymchenry#2922, colbymchenry#2258, colbymchenry#1761, colbymchenry#1583, the bare-call half of colbymchenry#2308 (rust-use-bindings.ts). CHANGELOG is main's plus the open PRs' bullets. EXTRACTION_VERSION 56: the C #if group handling (colbymchenry#2940) and the kernel change. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

This was referenced Aug 22, 2026

Daemon OOMs (~4 GB heap) on small project during startup catch-up sync; stale daemon.pid + OS PID reuse then blocks all daemon restarts (unlock does not help) #1553

Closed

fix: avoid quadratic synthesis line lookups #1580

Closed

maxmilian mentioned this pull request Aug 27, 2026

v1.5.0: codegraph.db-wal grew to 64 GiB and filled disk on a Linux-kernel-sized repo #1539

Closed

colbymchenry mentioned this pull request Sep 8, 2026

fix(db): fail closed when WAL valve cannot checkpoint past caps (#1539) #1751

Merged

3 tasks done

danusha2345 mentioned this pull request Sep 22, 2026

fix(extraction): a file over the size limit is never read (#1910) #1915

Closed

inth3shadows mentioned this pull request Sep 26, 2026

impact --depth N misses dependents within N hops (depth-limited DFS marks nodes visited at the limit) #1974

Closed

danusha2345 mentioned this pull request Sep 27, 2026

fix(graph): don't lose dependents within the depth limit to edge order (#1974) #1991

Closed

colbymchenry mentioned this pull request Sep 27, 2026

fix(resolution): reject oversized targets before decoding (#1553) #2006

Merged

danusha2345 force-pushed the fix/resolution-memory-and-graph-bounds branch 5 times, most recently from 61c3a3a to f249ca4 Compare October 3, 2026 09:03

danusha2345 force-pushed the fix/resolution-memory-and-graph-bounds branch from f249ca4 to 23884a4 Compare October 6, 2026 08:14

danusha2345 force-pushed the fix/resolution-memory-and-graph-bounds branch 3 times, most recently from 61514c4 to 62fc866 Compare October 10, 2026 11:26

danusha2345 force-pushed the fix/resolution-memory-and-graph-bounds branch 6 times, most recently from a8695ca to 0820f4e Compare October 22, 2026 10:55

danusha2345 and others added 6 commits October 22, 2026 14:00

fix(resolution): bound synthesis memory

fe79434

fix(mcp): recycle idle query workers

7420a3c

fix(graph): make shortest-path search linear

e64a5c6

fix(graph): bound impact traversal

540410c

docs(changelog): note the resolution memory and graph bound fixes

896841c

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

danusha2345 force-pushed the fix/resolution-memory-and-graph-bounds branch from 0820f4e to af4aba1 Compare October 22, 2026 11:12

Conversation

danusha2345 commented Aug 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What remains

Tests

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 8, 2026

Uh oh!

danusha2345 commented Sep 9, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

danusha2345 commented Aug 21, 2026 •

edited

Loading