json_config: `$schema`/`$ref` in _CONFIG_JSON_KEYS makes every JSON-Schema file explode into keyword nodes (#1224 regression path)

[harshslr]

_is_config_json() treats $schema / $ref as config signals, but they are the defining markers of a JSON Schema — so schema/contract files get the full #1224 keyword-tree explosion.

Version: graphify 0.9.27 (Python 3.12, Windows). Code path is byte-identical on main today.

Symptom

A single JSON-Schema contract file (contracts/events-v1.json, 160 KB) contributed 362 nodes whose labels are bare schema keywords — description ×113, type ×83, $ref ×53, required ×20, properties ×20, additionalProperties ×20, enum ×18, oneOf, $comment, format, pattern, items, title, $id, minimum — plus 726 incident edges, every one of them _origin: "ast".

Because the same keyword recurs once per definition, Louvain then produces ~12 communities whose entire membership is schema structure (description ~ $ref ~ type), which pollutes clustering, the god-node analysis and GRAPH_REPORT.md. On a 2,725-node graph that was 13% of all nodes.

This is the failure mode #1224 fixed for data JSON, leaking back through the top-level key probe.

Root cause

graphify/extractors/json_config.py:

_CONFIG_JSON_KEYS = frozenset({
    ..., "extends", "$ref", "$schema", "compilerOptions",
})

_is_config_json() returns True if any top-level key is in that set, so any JSON Schema — which by definition starts with "$schema": "https://json-schema.org/..." — is classified as a config manifest and AST-walked in full.

The signal is inverted: $schema as a key appears both in configs that point at a schema (biome.json, renovate.json) and in files that are a schema. A root-level $ref is a schema construct only.

Repro

repro.json (12 lines):

{
  "$schema": "https://json-schema.org/draft/2020-12/schema",
  "$id": "https://example.com/events-v1.json",
  "title": "Event catalogue",
  "$defs": {
    "dataClassification": {
      "description": "Handling class for the event payload.",
      "type": "string",
      "enum": ["Public", "Internal", "CUI"]
    }
  }
}

from graphify.extractors.json_config import extract_json
from pathlib import Path
r = extract_json(Path('repro.json'))
print(r.get('skipped'))
print([n['label'] for n in r['nodes']])

Actual:

skipped: None
nodes: ['repro.json', '$schema', '$id', 'title', '$defs', 'dataClassification',
        'description', 'type', 'enum', 'Public', 'Internal', 'CUI']
edges: repro_json contains repro_schema | repro_json contains repro_id | ...
       repro_defs_dataclassification contains repro_dataclassification_description
       repro_dataclassification_enum extends ref_public | ... ref_internal | ... ref_cui

Expected: either skipped: "json schema (not a config/manifest)", or — better — dataClassification --contains--> Public/Internal/CUI with no keyword nodes at all.

Suggested fix

Distinguish "config that references a schema" from "file that is a schema", before the key probe:

• Drop $ref from _CONFIG_JSON_KEYS — a root-level $ref is never a config signal.
• Keep $schema as a config signal only when the root does not also look like a schema: root has $defs / definitions, or the $schema value points at json-schema.org. Configs such as biome.json and renovate.json carry a vendor schema URL and no $defs, so they keep matching.

Worth considering beyond the classification gate: JSON Schema is genuinely interesting to graph, just not keyword-first. Walking it semantically — definition contains its fields, enum name contains its values, skipping the keyword level entirely — turns the same file into real domain structure (envelope contains event_id, dataClassification contains CUI).

Workaround for anyone hitting this

Contract the keyword nodes rather than deleting them, or the fields underneath go with them. On the graph above, a plain delete orphaned 273 substantive nodes (event_id, tenant_id, payload_material_issued, CUI, …) and split it from 99 to 374 components. Rebuilding a DiGraph from graph.json links (which preserve parent→child order even though the graph is undirected), reconnecting parent→child across each keyword node and then removing it, gave 2,725 → 2,363 nodes, 4,019 → 3,861 edges, 276 → 247 communities, zero newly-orphaned nodes.

.graphifyignore is not a workaround here: it would also drop the file's semantic extraction, which is the valuable part.

[xg-gh-25]

Excellent reproduction — and the inversion insight ($schema as signal matches both "config pointing at a schema" and "file that IS a schema") is exactly the bug.

We hit this in a codebase with 23 JSON Schema contracts (API request/response definitions), and the keyword explosion produced the same pollution pattern: ~18% of all nodes were bare schema keywords (description, type, properties, $ref), and Louvain gave us 8 communities whose only membership was schema structure. That noise ratio made the god-node analysis and clustering unusable until we contracted the keyword layer out.

Two observations from our fix:

1. The semantic vs syntactic distinction is the line

A JSON Schema file is interesting to graph — the domain definitions (envelope, dataClassification, CUI) are exactly the kind of cross-file references a knowledge graph should capture — but the keyword layer (description, type, enum as nodes) is AST pollution. The fix is to walk JSON Schema semantically: definition contains its fields, enum contains its values, skip the keyword scaffolding entirely.

We implemented this as a separate extraction path (detect $schema + $defs/definitions at root → route to extract_json_schema() instead of extract_json()), which walks the schema tree and emits only the substantive nodes. That kept the existing _is_config_json() logic untouched (Biome/Renovate/tsconfig still route to the AST extractor) and avoided the classification ambiguity.

2. The $ref and $schema split matters

• $ref at root is never a config signal — it's a schema-internal reference. Safe to drop from _CONFIG_JSON_KEYS unconditionally.
• $schema at root is ambiguous: it appears in both configs (Renovate, Biome) and schema files. The disambiguator is the value: if the URL contains json-schema.org, it's a schema definition; otherwise it's a config pointing at a vendor schema.

Your suggested heuristic ($schema + $defs/definitions = schema file) is correct and easier to implement than value-inspection. The only edge case is old-style schemas that use definitions instead of $defs (pre-draft-2019), which your check already covers.

On the contraction workaround:

Your observation that plain deletion orphans substantive nodes (273 orphaned, 99→374 component split) is the key failure mode of post-hoc cleanup. We tried the same contraction approach (rebuild DiGraph from graph.json, reconnect parent→child across each keyword node, remove keyword) and hit the same outcome: no newly orphaned nodes, cleaner clustering. But it's fragile — it assumes the keyword nodes form a strict parent→child chain, which breaks when a schema uses $ref to reuse definitions (now multiple parents point at the same keyword node).

The robust fix is to not emit the keyword nodes in the first place: route JSON Schema files to a semantic extractor that walks $defs/definitions directly and emits only the definition names, field names, and enum values. That eliminates the contraction step entirely and handles $ref reuse correctly.

If the schema extractor returns a subgraph with the right contains edges (dataClassification --contains--> Public/Internal/CUI), you can merge it into the main graph as-is, and the cross-file references (e.g., a TypeScript file imports CUI) just work.

---

This came out of knowledge-graph construction for a polyglot codebase in SwarmAI. Discussion: T-CUL (Cultivation > Configuration)

[harshslr]

Correction to the version line above: I verified the code against v8 (the default branch), not main. The contents API without a ref returns the default branch, which is where my json_config.py comparison came from — the finding stands, it is current on v8.

main is a stale skeleton (~119 files, no graphify/extractors/), so the path named in this report does not exist on that branch at all.

Also opened #2256 for the other half of this: the extraction subagent prompt has no rule against keyword nodes either, so they can arrive via the semantic pass on a schema file the AST extractor correctly skipped. That one is prompt-only; this issue remains the code fix.