fix(llm): hold semantic source_file to the dispatched set, not to existence (#2217)

[hassaanch23]

Closes #2217.

The defect

The merged-graph filter and the semantic cache disagree about what counts as
valid provenance, and graph.json takes the lenient side.

# graphify/llm.py — _out_of_scope (#1895): drops only an EXISTING file
return p.is_file() and p not in _dispatched_resolved

# graphify/cache.py — save_semantic_cache's write loop refuses both halves:
#   a path that is not a real file, and (#1757) a real file outside the batch
if p.is_file():
    if allowed_paths is not None and p not in allowed_paths:
        ...skip...          # #1757 out-of-scope guard
else:
    skipped_not_file += 1   # ...and a non-file group is never written at all

A source_file the model invented — a plausible repo path with nothing behind
it — has p.is_file() == False, so the graph admits it and the cache refuses
it. The old comment described this as intentional ("non-file source_files
(concepts, model-invented anchors) pass through untouched").

That divergence is why #2217 reports non-convergence: the ghost is in
graph.json but never in the cache, so every --update re-extracts the source
and mints a different ghost.

Same script against base and this branch — one dispatched notes.md, a model
returning a node attributed to src/auth/session.py:

base 0b2bd93            after
  ghost in graph : True   ghost in graph : False
  out_of_scope   : 0      out_of_scope   : 1
  cache(ghost)   : 0      cache(ghost)   : 0     <- cache refused it all along
  cache(notes.md): 1      cache(notes.md): 1     <- probe does distinguish accept/refuse

The fix

The existence test is kept and a claim test added alongside it:

real FILE outside the dispatched set  -> misattributed  (#1895, unchanged)
real DIRECTORY                        -> kept
nothing on disk, and claims a file    -> fabricated     (#2217, new)
anything else (prose, concept marker) -> kept

Replacing the existence arm rather than extending it looks equivalent and is
not: it re-admits every real undispatched file whose name carries no corpus
extension (Makefile, LICENSE, pyproject.toml, data.csv), which the cache
still refuses — reopening the exact divergence this closes, for that class.
Pinned by test_filter_is_a_superset_of_the_1895_existence_check.

Recognising a "claim" is deliberately conservative, because a false positive
deletes a node and cascades through its edges. A separator alone is not
evidence — models put N/A, CI/CD, TCP/IP, and/or, I/O,
client/server architecture in this field. A corpus suffix alone is not either
— Node.js, Vue.js, D3.js. So a claim requires both: a separator, and a
final segment that is a single token ending in a known corpus extension.
Directory-shaped values are excluded on purpose — cache.py records that
subagent fragments really do carry a directory here, and the base filter kept
them.

Accepted gap, stated plainly: a fabricated bare basename (session.py,
no directory) is not recognised, because nothing syntactic separates it from the
technology names above. When it names a real file the existence arm still drops
it; when it names nothing it survives. Precision-first, matching
_bind_node_evidence, which likewise prefers a missed fabrication to a
destroyed node.

Both halves must canonicalise identically

_claims_a_file flips backslashes and strips segments. If the dispatched-set
lookup does not, the two disagree in the destructive direction: on POSIX a node
attributed to a genuinely dispatched file as pkg\B.md misses the dispatched
set, misses is_file()/is_dir(), and is then deleted as a fabrication by the
one half that did normalise it — taking its edges with it.

And it would not even stay deleted. save_semantic_cache already applies
_normalize_source_file_value (#2197 — same field, same problem) to every item
before grouping, so the cache stores that node under the real dispatched path
while the graph deletes it. Since the merged-result filter is bypassed on cache
replay, the next --update puts it straight back. That is this PR's own
invariant inverted.

So the filter reuses the repo's existing normaliser rather than a second, subtly
different one. Pinned by
test_alternate_spellings_of_a_dispatched_path_are_tolerated, parametrised over
pkg/B.md, pkg/ B.md, pkg\B.md and pkg\B.md.

Further defects found while testing the above

All become reachable once dropping stops being rare:

• a node id appearing twice (chunks are concatenated, dedup is downstream) could
  be dropped in one copy and kept in another; the cascade then stripped every
  edge from the surviving, correctly-attributed copy. cache.py fixed this same
  hazard in its own prune (fix(cache): prune edges/hyperedges referencing never-written node groups #1916, skipped_ids -= written_ids).
• a non-scalar id or edge endpoint — extract: AttributeError ('list' object has no attribute 'get') at merge when a semantic chunk fails — partial results discarded, semantic cache write also fails #1631's malformed-result class, a list
  where a scalar belongs — raised TypeError and aborted the run after every
  chunk had been paid for. cache.py guards the same spot.
• an edge or hyperedge carrying its own bad source_file was filtered only when
  some node happened to be dropped, so a clean-nodes/dirty-edge response went
  unexamined — while save_semantic_cache groups edges by their own
  source_file and refuses them. Now filtered unconditionally, and counted.
• _endpoint_dropped judged both endpoints under one try/except, so an
  unhashable source masked a genuinely dropped target and left the edge
  dangling.
• a URL in source_file was read as a claimed corpus path and deleted. A URL is
  a source_url; base kept these.
• pre-existing on v8, not introduced here (verified on the base commit): a
  non-string source_file reached Path() in the extract: LLM-omitted documents vanish silently — no dispatch/return reconciliation, no anti-regression guard (53% of docs lost even with gpt-5) #1890 reconciliation and
  aborted the run. Guarded only because the filter above now guards the identical
  class two blocks up, and a crash between them would be incoherent. Happy to
  split this out if you'd rather it landed separately.

Verdicts are memoised per source_file value — one corpus repeats the same path
across thousands of items, and each miss costs two stat syscalls.

Commit 2: the prompt

The skill path has carried a source_file RULE since #1366 ("copy the FILE_LIST
entry character-for-character"). The native/API prompt had no rule at all —
it showed "source_file":"relative/path" in the schema and left the rest to the
model. That is the upstream half of this bug, and it matches the report (a
DeepSeek run reconstructing a path out of a node id quoted inside a document).

The test asserts against the rule paragraph, not the whole prompt — matching
the whole prompt proves nothing here, since the base prompt already contains
<untrusted_source> (SECURITY), "node id" (Node ID format: ...) and
"citation" (EXTRACTED: ... import, call, citation, ...). Verified by mutation:
an inverted rule ("pick whatever source_file seems most plausible, basename is
fine") fails the test.

It is split out because it costs money. It changes the prompt fingerprint
(f33081f95084 → 850648cc0d13), so under #1939 every existing semantic cache
entry falls out of its namespace and the next run re-extracts the corpus for
real. Plus +490 chars (~122 tokens) per extraction request. If that trade isn't right for a
release, drop commit 2 and keep commit 1 — the graph is correct either way, it
just keeps filtering fabrications instead of asking for fewer. Commit 1 passes
the suite standalone.

Measurement

worked/ is the only production-shaped semantic output in the tree, so commit 3
turns the check into a standing test.

fixture	nodes	ground truth	attributions checked	fabricated
rsl-siege-manager	1,886 (1,268 AST + 618 LLM rationale)	293-file manifest	1,880	0
karpathy-repos	177	AST layer	0 → skipped	—
httpx, mixed-corpus	166	pure AST, no independent truth	0 → skipped	—

Only one fixture carries real signal, and the test now says so rather than
reporting green on the other three. karpathy-repos's 33 non-code nodes are
bare id+community stubs with no source_file at all — not LLM output — so
they are skipped, not counted as audited.

Mutation-tested: injecting a fabricated rationale node with a full path fails
the audit, and the claim floor (≥1,800) is asserted before the skip so a total
collapse of _claims_a_file fails rather than quietly skipping the only fixture
that checks anything — verified by forcing it False.

An earlier revision of this audit passed the injection outright, because the
reference set was derived from the audited nodes themselves and a fabricated node
validated itself. A second revision used an endswith fallback that accepted any
trailing segment, so it would have accepted source_file: "env.py" against a
manifest containing ...\alembic\env.py; the reference set is now reduced to
corpus-relative form and compared exactly.

Diagnostics

out_of_scope_dropped keeps its meaning as the node total. New counters split
failure modes that have different causes and different fixes:

• provenance_dropped_misattributed — a real corpus file you can go read (deep mode: nodes with out-of-scope source_file are warned about but still written to graph.json #1895)
• provenance_dropped_fabricated — a file-claiming value with nothing on disk (Semantic extraction: nonexistent model-invented source_file bypasses #1895 graph filter #2217)
• provenance_dropped_edges / provenance_dropped_hyperedges — the edge side

Tests

tests/test_provenance_invariant.py covers the defect, the edge/hyperedge
cascade, the counter split, the superset property, duplicate ids, unhashable
ids, whitespace, and a table of values that must and must not read as file
claims. Plus the property whose violation caused the non-convergence: for any
source_file that claims a file, graph admission and cache admission are the same
decision — asserted in both directions so it cannot pass vacuously.

Full suite: 3,659 passed. The 14 failures are identical to the 14 on v8
before this branch (verified by diffing the failure sets): 12 are missing
optional/dev deps in my environment — tree-sitter-hcl (7), openai (4),
build (1) — and 2 are unrelated pre-existing failures
(test_labeling list-ordering, test_manifest_ingest KeyError: 'version').
python -m tools.skillgen --check passes (134 artifacts).

[graphify-labs]

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

Graphify reviewed this change.

Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).

---

Graphify review panel

Reviewed by claude, gpt.

This pull request modifies the provenance filtering logic in graphify/llm.py that decides which extracted nodes/edges are kept in the merged graph based on their source_file attribute. It adds a new _claims_a_file helper (with a lazily-loaded, memoized set of corpus file extensions) and reworks the out-of-scope filter into a _scope_verdict function that distinguishes "misattributed" real files (existing #1895 behavior) from newly-handled "fabricated" file-claiming paths that resolve to nothing (#2217), while reusing the cache module's source-file normalizer and adding resolution/verdict caching. The change touches the extraction spec prompt text (adding an explicit source_file rule) and expands the test suite across LLM backend, provenance-invariant, and worked-graph provenance test files. Reviewers should focus on the new provenance classification helpers, the normalization consistency between the filter and the cache path, and the associated new tests.

No blocking issues surfaced. 6 lower-confidence candidates did not survive cross-model review.

Analysis details — impact, health, verification

Impact & health

graphify review

Impact — 671 function(s) in the blast radius of 307 changed node(s).

Health — regressions introduced by this change:

• worsened (Ca·Ce 115→150): extract_corpus_parallel() (Ca=25 Ce=6) — fans out to 6 callees (efferent coupling); 25 callers depend on it (afferent coupling)
• new offender: _run() (Ca=15 Ce=1) — 15 callers depend on it (afferent coupling)

Verification — 671 function(s) need re-proving (callee-first):
tests_test_worked_graph_provenance_norm → tests_test_worked_graph_provenance_test_norm_preserves_dot_directories → tests_test_worked_graph_provenance_strip_common_prefix → tests_test_worked_graph_provenance_ground_truth → graphify_llm_corpus_suffixes → graphify_llm_claims_a_file → tests_test_worked_graph_provenance_test_no_node_claims_a_file_the_run_never_saw → tests_test_worked_graph_provenance_rationale_77 → tests_test_worked_graph_provenance_rationale_55 → tests_test_worked_graph_provenance_rationale_45 …

Gate & verification

graphify gate

PASS — objectively clean (no health regressions, tests not run — proofs not enforced this run (re-run with --prove to verify the blast radius)). Grounded, not self-assessed.

Advisory (not blocking):

• verification_scope: 481 function(s) in blast radius — re-run with --prove to formally verify them

· 1 grounded finding(s) anchored inline below; 1 more finding(s) on lines outside this diff (see the check run).

[graphify-labs]

⚠️ Health regression — _run()

15 callers depend on it (afferent coupling).

Grounded coupling-delta finding (deterministic), not an LLM guess.