1.2.0: stabilization, and SQL completion driven by the parser - #25
Merged
Conversation
The repository had no frontend test runner and no way to run tsc as a check, so type errors first surfaced during the release build. Adds vitest plus `typecheck`, `test` and `test:watch` scripts.
The packed wire format replaced any 0x1F/0x1E byte occurring inside cell data with a space and encoded SQL NULL as the literal text "null". Both lost information, and both corrupted primary-key values on the way to the frontend, so generated UPDATE and DELETE statements could match the wrong row or no row at all. Separators inside data are now escaped with 0x1D and NULL has its own two-byte marker, so a real NULL, an empty string and the text "null" are three distinguishable values end to end. Decoding takes a fast path when a page contains no escape byte, which is the common case. Result rows are typed (string | null)[][] throughout, and each consumer now makes an explicit choice: the grid draws NULL muted and labelled so it cannot be mistaken for an empty cell, CSV writes an empty field, JSON writes null, SQL writes the NULL keyword, and search skips NULL cells. Two places that had reimplemented the format locally — the export helpers and the diff view — use the shared codec, which also fixes the diff view treating NULL and "" as equal. Adds round-trip tests on both sides covering NULL, the text "null", empty strings, embedded separators, escape sequences and multibyte content.
Inline row editing could silently do nothing, and in one case wrote to the wrong table. Four separate causes: Commit applied only cell edits and Delete applied only row deletions, and each cleared the whole edit state afterwards, so pressing Commit with rows marked for deletion discarded those deletions without a word. There is now a single Apply that sends updates and deletions together. The edit state lived in the results panel component rather than on a tab, so switching tabs kept the previous table, key columns and marked rows while the rows underneath changed — a delete marked on one table could run against another. The session now lives on the tab and Apply refuses to run when the editor no longer targets the table the session was opened on. Pending changes were keyed by grid row index, which means nothing once a result is refreshed and could index past the end of a shorter result and throw outside the error handler. They are now keyed by the primary-key tuple captured when the change was marked, and rows that cannot be identified are rejected with a message instead of guessed at. Statements were built as SQL text in the frontend and their result was never checked, so a DELETE matching zero rows reported success. Statements are now built in Rust from validated column names, bound as parameters cast to the column's catalog type, and run inside a real transaction that requires every statement to affect exactly one row. Zero or several rows aborts the batch with an error naming the row key. Column names are checked against pg_attribute rather than merely quoted, and the statement timeout is set with SET LOCAL so it cannot leak into a pooled session.
Every tab-store mutation took a position, and callers captured that position before awaiting a query. Closing or reordering a tab while a query ran therefore wrote the result into whichever tab now sat at that index, or threw on an index past the end. The throw happened inside the try block whose catch wrote to the same index and threw again, leaving an unhandled rejection and a tab stuck on "Executing query..." forever. Mutating actions now take a tab id and no-op when that tab is gone, and the query lifecycle captures the id before its first await and clears the executing flag in a finally. Selection still works by index; only addressing changed. Also routes two remaining raw separator splits in the query lifecycle through the wire decoder, so streamed and virtual results decode NULL and escaped separators like every other path.
Cancel tokens were stored one per project and overwritten by every query, so with two queries in flight on the same connection cancelling hit whichever started last. The token recorded at connect time belonged to a client that had already gone back to the pool, so it could cancel nothing at all. Tokens are now keyed by an exec id generated per run and carried on the tab, registered before execution and removed when it finishes. Cancelling an exec id that has already completed reports false instead of raising. The mutation and snapshot paths no longer register tokens: both are short and transactional, and no UI offers to cancel them.
The target/ rule was anchored to the repo root, so a build tree outside src-tauri stayed untracked-but-visible; crates/ alone held 3 GB of build artifacts that a `git add -A` would have committed. Local database rules had the same problem, and one of them was inert because a comment was glued onto the pattern. target/ and **/gen/schemas/ now match at any depth, the sled database rules are directory-anchored, and docs/superpowers/ stays out of the repository.
Virtual query execution buffered the whole result as owned strings and then built the packed pages beside it, so both full copies were live at once and nothing bounded either. Rows are now packed into page-sized strings as they arrive over simple_query_raw, so the unpacked copy never exists, and accumulation stops at 1M rows or 512 MB with the result marked truncated. The cap bounds this process only — the server still finishes sending what it was asked for; bounding that needs a cursor holding a connection open, which is a separate change. The snapshot tables went with it. Pages were written to the local database on every scroll, but only for pages the user happened to visit, so the stored set was full of holes by construction and restoring always had to re-run the original SQL anyway. Nothing could even reach those rows after a restart, because the frontend does not persist which query a page belongs to. They are dropped at startup and the file is vacuumed once. Pool acquisition now times out after ten seconds instead of waiting forever, which is what an exhausted pool looked like from the outside: a window that had stopped responding. Sizes drop from 16/8 to 8/4 per project, since several projects can be connected at the same time.
The repository ran no frontend checks at all, so a type error first surfaced during a release build on four platforms at once. Clippy ran with continue-on-error, and cargo test ran against a suite that did not exist. Adds a frontend workflow running biome, tsc and vitest. Clippy now fails the build: its fourteen existing warnings were all trivial and are fixed here. Formatting is checked too. Row mutations get integration tests against a real PostgreSQL service container, covering what unit tests cannot: that the text "null" deletes its own row rather than the NULL one, that a zero-row or multi-row match aborts the batch, that composite keys and control characters in key values resolve correctly, and that the $n::text::<type> cast recipe holds for numeric, timestamptz, arrays, jsonb, enums, bytea, boolean and uuid. The toolchain is pinned in rust-toolchain.toml instead of tracking nightly, which nothing in the codebase required. A version-consistency script replaces the manual three-file bump that the history shows drifting more than once, and it gates both CI and the release job. Version bumped to 1.2.0.
Setting a wait timeout without telling deadpool which runtime drives its timers makes the builder reject the pool, so connecting failed with "Timeouts require a runtime" for every project. The timeout was added in this release; the runtime was not. Pool construction only happens on connect, which no test reached, so this surfaced as a runtime failure rather than a build or test one. Both pool variants now have a test that builds them.
Three defects found auditing the paths this release had not touched. Deleting a project failed outright. It still deleted from the virtual snapshot tables, which this release drops at startup, and the error propagated rather than being ignored. The cleanup around it was wrong in three further ways: cancel tokens were looked up by project id after being rekeyed by exec id so none were freed, LISTEN tasks kept polling a connection that no longer existed because their handles are keyed "<project>:<channel>", and the SSH tunnel kept holding its local port. Importing a CSV into any table with a numeric, date or boolean column failed with "error serializing parameter 0". CSV fields are text, and binding them straight at a typed column makes tokio-postgres refuse the parameter, so only all-text tables could ever be imported. Each parameter is now cast to the column's catalog type, the same recipe the row-mutation builder uses, and column names are validated against pg_attribute rather than interpolated. An absent or empty field becomes NULL instead of an empty string. The import also ran on a bare BEGIN and returned without rolling back when a row failed to parse, handing a connection back to the pool with a transaction still open; it uses a real transaction now. Every Postgres error reached the user as the word "db error". tokio-postgres displays only the error kind at the top level and puts the server's message — the missing relation, the syntax position, the constraint name — in the source. A shared error_chain walks that chain and query_failed applies it at the hundred-plus sites that convert a Postgres error, so a failed query now says what actually went wrong.
Completion asked the server per table and per schema while the user typed, putting an IPC round trip on the keystroke path — typing FROM in a database with fifty schemas started fifty round trips per keystroke. pgsql_load_schema_index fetches a schema in one shot instead: relations with their kind and comment, columns with type, nullability, default, primary-key flag and single-column foreign-key target, and functions with their signature. It runs on the meta pool, and a very wide schema is detected with LIMIT cap+1 rather than a counting query, returning relations without columns so the frontend can load those per table. The catalog SQL lives in files shared by the implementation and the integration tests, so the tests exercise the query text that actually ships. Verified against PostgreSQL 16: kinds are right and indexes and sequences are excluded, view columns are indexed, and foreign keys resolve to schema, table and column. Also upgrades monaco-editor to 0.56 and monaco-sql-languages to 1.2, and pins what the SQL parser reports at the caret in a characterization test. That test settles the design's open question: entities do carry aliases, a caret after an alias dot reports a column context with the qualifier chain, and the parser gives the exact word range to replace — so the regex-based completion can be replaced rather than patched.
The editor was unpleasant for reasons that had names. Enter accepted the highlighted suggestion instead of breaking the line. Space was a trigger character, so the widget opened on every word and each opening started metadata loads. Typing FROM asked the server once per schema, per keystroke. The document's own words competed with schema suggestions. No item carried a range, so Monaco guessed the span to replace and items labelled `public.users u` overwrote the wrong text. Nothing carried sortText, so the columns you wanted ranked below 143 keywords. And the widget opened inside string literals. Underneath all of it, setup.ts disabled the library's context-aware completion and matched context with regexes that could not see aliases, CTEs or scopes. The parser already reports what the grammar expects at the caret, which keywords are valid there, the exact word range to replace, and the statement's relations with their aliases. Completion now reads that. The decisions live in a pure buildCompletions, so what appears, in what order, and what it replaces is tested without Monaco, a store or a database. A second suite runs the real parser into that core, which is where an adapter that misread the parser would show up. Columns rank above relations above functions above keywords; an unrecognised qualifier yields nothing rather than unrelated columns; a CTE is recognised as something the catalog cannot answer for. Column suggestions now carry type, primary key, nullability and foreign-key target, and relations carry their comment, all from the schema index. monaco-editor goes to 0.55.0 rather than 0.56.0, pinned: 0.56 narrowed its exports map and monaco-sql-languages' worker still imports the old path, so 0.56 fails to build inside that package. The reason is recorded next to the import.
No suggestions appeared at all. Routing completion through the library's completionService put three things between a keystroke and a suggestion: a registration that happens lazily via onLanguage, a worker that has to resolve under the right label, and the library importing monaco-editor/esm/vs/editor/editor.api while the app imports monaco-editor. Any of the three failing is silent — the promise rejects and the widget stays empty. Completion is now registered directly with registerCompletionItemProvider, which is how it worked before this branch and is known to reach the editor, and the parser runs inline. That makes the code path at runtime the one pipeline.test.ts already exercises. A statement of a few hundred characters parses in about 2ms and a 7000-character script in 10ms, measured, which is affordable for a user-triggered request; beyond 200k characters it declines to parse. Also fixes a second cause of emptiness: with no project attached to the tab the service returned nothing at all, where it should still offer the keywords the grammar allows and the snippets, neither of which needs a catalog.
Monaco swallows an exception thrown by a completion provider and renders an empty list, which is indistinguishable from "there is nothing to suggest". A provider that fails for any reason — a parse error, a catalog lookup, a store shape change — therefore looks exactly like a working one with no ideas. The provider now logs what went wrong and falls back to the snippets, so a failure is both visible on the console and survivable.
Two problems once suggestions actually appeared. Every suggestion showed up twice, and once more after each save: the provider had no idempotency guard, and vite re-executes the module on hot reload, so registrations stacked. Re-registering now disposes the previous provider, which also means a reload replaces the stale one rather than leaving it in charge. A test with a stub Monaco pins it. The suggest widget was unreadable because the theme set four of its colour keys and left the rest to the base theme, which was tolerable when the list was short and stopped being so once items carried icons, a type, key flags and comments. Both themes now set the label, selected, match, icon and description colours explicitly. Every foreground-on-background pair was measured: the weakest is 5.91:1, above the 4.5:1 needed for body text. Adds the statement-start keywords the grammar cannot supply for a wholly empty document — which is why a fresh tab offered no SELECT — and falls back to the editor's own word range when the parser reports none, so accepting SELECT after typing SEL no longer leaves SELSELECT.
Enter breaking the line unconditionally traded one annoyance for another: it fixed newlines inserting stray keywords, but meant reaching for Tab even when a suggestion was obviously the thing wanted. "smart" accepts when the suggestion would change the text and breaks the line when it would not. Tab still always accepts.
Accepting a table after typing a schema qualifier produced `noexapp.noexapp.agent_memory_preferences`. The replacement range covers only what follows the qualifier, so the schema is still in the buffer, but the insert text was schema-qualified as well. Relation items now always insert the bare name, and the parameter that allowed qualifying them is gone rather than merely unused, so the same mistake cannot come back. Columns and functions were already bare.
The status bar sampled resource usage once a second, and each sample enumerated every process on the machine to find the app's own tree, then rescanned that whole list once per level of the tree until no new child turned up. It also locked both connection pools, contending with running queries. Discovery now happens on one sample in five, since the tree only changes when the app spawns something such as a terminal; in between, only the known pids are refreshed. Finding descendants builds a children map once and walks it breadth-first, so it is linear in the process count rather than quadratic, and a parent cycle can no longer hang it. The poll drops to two seconds, skips while a sample is still outstanding so a slow one cannot queue up more, and stops entirely while the window is hidden — there is nothing to read then. It resumes on the way back.
Until the catalog arrived, completion answered without tables and looked identical to a schema that has none. Monaco already shows "Loading..." while a provider is pending, so the first request that needs a schema now waits for its snapshot and lets the editor say so. Every later request reads memory, which is the point of the index. The wait is capped at three seconds so a slow catalog answers with whatever is available rather than wedging the editor, and while a snapshot is still outstanding the list is marked incomplete so Monaco asks again on the next keystroke instead of filtering a partial list forever. Which schema a request depends on is worked out separately from the provider and tested: a schema qualifier needs that schema, an alias follows its relation, and a keyword-only position needs nothing loaded at all.
The loading state never appeared because ensureIndex returned immediately when a fetch was already in flight. Connecting starts one, so by the time anyone typed there was usually a fetch running, the provider's await resolved at once, and completion answered with neither a wait nor an index. Concurrent callers now share the running promise. A failed fetch was also swallowed, which left the schema pending forever: every keystroke then waited the full three seconds and showed nothing. The failure is now logged, and a schema whose last fetch failed is still retried in the background but never waited on again.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
# Conflicts: # package.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Inline row deletion sometimes silently did nothing, and in one case wrote to the wrong table. Auditing that turned up a wider set of defects in the paths that mutate data, address tabs, and report errors. The editor's completion was rebuilt on top of that work after it turned out to be fighting the parser it already ships with.
Version bumped to 1.2.0.
The reported bug had six causes
"null"were indistinguishable on the wire, so a row whose key is the stringnullproducedWHERE pk IS NULL.0x1F/0x1Eoccurring in data with a space, corrupting key values on the way to the frontend.Also fixed
Lifecycle. Tabs were addressed by index across await points, so closing one during a query routed the result elsewhere or left the tab spinning forever. Cancel tokens were stored one per project and overwritten by every query, so cancelling hit the wrong one.
Project deletion failed outright, still deleting from tables this release drops. Its cleanup also missed LISTEN tasks and the SSH tunnel.
CSV import failed for any table with a numeric, date or boolean column — text parameters bound at a typed column make tokio-postgres refuse to serialize them. Empty fields became empty strings rather than NULL, and a parse error returned without rolling back.
Every Postgres error reached the user as the word "db error". tokio-postgres puts the server's message in the error's source, and the hundred-plus conversion sites read only the top level.
Memory. Virtual query execution held the whole result twice — as owned strings and again as packed pages — with nothing bounding either. Rows are packed as they arrive and accumulation stops at 1M rows or 512 MB. The snapshot tables went with it: pages were written on every scroll but nothing could reach them after a restart. Pool acquisition now times out instead of waiting forever.
The footer's resource readout enumerated every process on the machine once a second and rescanned that list once per level of the process tree. Discovery happens on one sample in five, the walk is linear, and polling stops while the window is hidden.
SQL completion
setup.tsdisabled the library's context-aware completion and matched context with regexes that could not see aliases, CTEs or scopes. The parser already reports what the grammar expects at the caret, which keywords are valid there, the exact range to replace, and the statement's relations with their aliases.Completion reads that now, backed by a new
pgsql_load_schema_indexthat fetches a schema in one shot — previously typingFROMin a database with fifty schemas started fifty round trips per keystroke. Suggestions carry type, primary key, nullability and foreign-key target.The editor's own settings accounted for much of the pain: Enter accepted the highlighted suggestion instead of breaking the line, space was a trigger character, the document's own words competed with schema suggestions, and no item carried a range or sort order.
Testing
Was zero tests. Now 46 Rust unit, 18 integration against a real PostgreSQL, and 110 frontend, with CI running biome, tsc, vitest,
cargo fmt, blocking clippy, and the integration suite against a service container. The toolchain is pinned rather than tracking nightly, which nothing needed, and a script checks the three files that carry the version agree.Before merging
The completion work was verified interactively and several defects only appeared that way; the mutation path has integration coverage but has not been exercised by hand against a production-shaped database. Worth checking: applying a mixed update and delete, a delete of a row another session already removed, a tab close during a running query, and auto-update from an installed 1.1.5.
Known issues carried forward
null.monaco-editoris pinned to 0.55.0: 0.56 narrowed its exports map and monaco-sql-languages' worker still imports the old path.🤖 Generated with Claude Code