[colbymchenry]

Summary

A program that uses CodeGraph as a library and runs indexAll() again on a project it has already indexed no longer empties the name-word vocabulary: the words in symbol names that the prompt hook and codegraph_explore use to match a plain-language question ("how does the order state machine work") to a symbol (OrderStateMachine).

Reimplements #2252 by @rudycelekli, who found the bug.

Root cause

CodeGraph.indexAll cleared name_segment_vocab before extraction on every run. Over an existing graph the store skips files whose content hash is unchanged, so their names were never written back:

• a second run with no edits left the table empty;
• a run after an edit kept only the edited files' names, and since sync and healSegmentVocabIfEmpty() repair only a completely empty table, a partly filled one stayed that way;
• the clear ran before the abort check, so a cancelled run emptied the table too.

codegraph index was not affected: it recreates the database before indexing, so every run there is a fresh one.

Fix

• The table is cleared up front only when the database is fresh (freshDb), as before for init and codegraph index.
• Over an existing graph the table is kept. Once a run completes without being cancelled, one DELETE drops the rows of names that no symbol carries any more (file and import nodes don't count, as on the write path). That keeps the orphan cleanup the clear used to do. It runs after resolution, so a name a framework pass renamed is covered too.
• The prune also resets the session's set of names already written (segmentedNames), so a pruned name that comes back later in the same session gets its rows again instead of being skipped as already seen.
• It doesn't reuse rebuildNameSegmentVocab(), which pages with LIMIT/OFFSET.

Validation

• Kit repro with the PR's fixture: on main a second indexAll() took the table from 11 rows to 0, an edit plus a third run left 4, and a cancelled run emptied it. With the fix it stays at 11 rows, the renamed class's 3 rows give way to the new name's 3, and a cancel keeps them all.

• 5 new tests in __tests__/segment-vocab.test.ts, which compare the table itself with the segments of every symbol name in the graph (getSegmentMatches hides orphan rows, so a prune shows only in the table). 4 fail on main's source; the fresh-database guard passes on both. Mutations caught: no prune, no reset of the written-names set, no clear on a fresh database.

• CLI: codegraph init then codegraph index write the same 11 rows on main and with the fix.

• Real repo, vscode src/vs/base + src/vs/editor (1,388 files, 56,406 nodes, 66,250 vocabulary rows; vscode 69e9e92f), a library indexAll() over the existing index, 3 interleaved runs per arm:

  	vocabulary rows after	run time (median)	prune
  main	66,250 → 0	8.37 s	–
  fix	66,250 → 66,250	8.13 s	40 ms

• Prune cost at scale: 288 ms on a 10× replica of that index (564,060 nodes), about 1.2 s on a synthetic graph with 505k vocabulary rows (170k distinct names). A full re-index of that size re-parses every file.

• Rebased onto main 4955eac (fix(rust): use bindings follow Rust's scopes, and module paths walk the crate's modules #3014, Rust resolution only); the diff is unchanged (same patch-id), and the measurements above were taken on the first base, e7c9ffd.

• Full suite on this exact commit (009cc45, on main 4955eac), native kernel built: 784 files, 9,286 tests passed, 0 failed, 85 skipped (--maxWorkers=2, no reruns).

Left out

• A vocabulary that an earlier version already left partly empty is not refilled by the next library indexAll(), which only prunes. codegraph index rebuilds it, and sync and the prompt hook still refill a table that is completely empty. Refilling on every full index would cost about another 0.6 s per 400k symbols for a one-time repair.

🤖 Generated with Claude Code