Repository navigation
fix(markdown): fold commands and table rows into their section - #613
Merged
Merged
Conversation
![bompus-ai-review[bot]](https://avatars.githubusercontent.com/u/253635?s=60&v=4){kind=small}
A Markdown table row, list item or fenced command minted a constant or function node plus a contains edge even though the text already lived in the section docstring. On supabase that was 6,222 of 27,380 Markdown nodes. Rows/items now contribute their references to the owning section and commands keep their script calls reference from the section. Text the 600-character preview drops stays searchable on the owning node's new search_text column, indexed in nodes_fts but not displayed: late rows, table headers, every recognized command past the 8-command summary, and unrecognized fence lines. Token-boundary dedupe keeps a later CG-713 from being treated as a substring of XCG-713. Unlabeled fences recognize every interpreter extractScriptPath supports (node, powershell, ...). Supabase vs the merged #612 base: 141,880 -> 135,658 nodes, 317,743 -> 311,496 edges; 25 references that pointed at a removed row/command node now fail and 112 re-source to their section. Index time shows no clear change (15.969 vs 16.047 s median, two ABBA rounds, ranges overlapping). Schema v24 adds nodes.search_text and indexes it in nodes_fts; the golden dumps re-baseline only for nodes that carry the field. Full suite 960 files / 11,359 passed.
bompus
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
A Markdown table row, list item or fenced command minted a
constantorfunctionnode plus acontainsedge even though its text already lived in the section's docstring and searchable text. On supabase that was 6,222 of 27,380 Markdown nodes.Change
callsreference from the section.commands:summary (8 per owner, 240 characters) for display. Text the 600-character section preview drops stays searchable on the owning node's newsearch_textcolumn (indexed innodes_fts, not displayed, no file-wide cap): late rows, table headers, every recognized command even past the 8-command summary, and unrecognized unlabeled fence lines.extractScriptPathsupports is recognized (node,powershell, ...), so output, trees and JSON fragments stop becoming commands while a Node.js example keeps its script edge; a labeled shell fence keeps every line.__tests__/markdown-section-model.test.ts; extraction and full-pipeline tests updated to the section model.Results
Supabase, base
422b1f81vs this branch:The 25 lost resolved edges pointed at a removed row/command node; those references now fail instead of pointing at it. Another 112 resolved edges re-source from a row/command to its section with the same target. Everything else (non-Markdown nodes, Markdown file/module/import ids) is unchanged.
Index time shows no clear change: supabase median 15.969 s against 16.047 s (Hyperfine 2.0.0, 8 runs per build across two ABBA rounds; ranges overlap). The value is the 6,222-node graph reduction with the search text retained. n8n drops 3,719 nodes and discourse 869 from the same command/table removal.
Checks
payroll-go.dumploses 20 command/table nodes and their contains edges; the section docstrings gain the command summary.Re-index Markdown-heavy projects after upgrading.