Skip to content

fix: upgrade SQL parser to 0.1.22 and align editor completion and highlighting - #630

Merged
emrberk merged 14 commits into
mainfrom
fix/editor-lexical-states
Nov 13, 2026
Merged

emrberk merged 14 commits into
mainfrom
fix/editor-lexical-states

Conversation

@emrberk

@emrberk emrberk commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Upgrade @questdb/sql-parser from 0.1.20 to the published 0.1.22 release and update the lockfile. Combine the parser improvements from questdb/sql-parser#42 with the console changes needed for completion insertion and highlighting.

Parser upgrade

  • Accept more server SQL, including grouping sets, DEDUP SKIP, implicit SELECT bodies, additional literals and expressions, and Enterprise statements.
  • Offer catalog functions immediately after pg_catalog., with table-valued functions in FROM/JOIN and scalar functions in expressions.
  • Restrict information_schema.* suggestions to prefix/suffix matches or an explicit namespace. Typing SELECT no longer exposes loose fuzzy matches.
  • Improve DEDUP/retention column scope and formatting of passwords, quoted values, and bind/slice expressions. Preserve the CST shape used by ordinary notebook DECLARE/global merging.

Editor changes

  • Complete backtick-quoted names with the matching closing quote, replacing an existing closing quote when present.
  • Respect nested comments and ignore comment markers inside quoted names when deciding whether to show completion.
  • Replace the full qualified function name when accepting a catalog completion, without duplicating its namespace. Alias-qualified column completion still preserves the alias.
  • Give backtick contents normal identifier styling, so names such as order no longer receive keyword styling. Identifier styling uses the theme's default text color.
  • Highlight numeric literals, E-prefixed strings, bind variables, and expressions inside arrays. Distinguish binds in CASE/nested parentheses from slice separators: arr[:1] contains a bind, while arr[: 5] and arr[1:n] contain slice operators. Leading signs and :: retain operator styling.

The QuestDB test submodule is also updated. The parser dependency is an exact npm version; the PR contains no local file link.

Validation

  • Published npm package @questdb/sql-parser@0.1.22 installed with yarn install --immutable.
  • Full unit, component, and integration suite: 200 files / 3,404 tests passed.
  • Typecheck, full lint, and production build passed.
  • Checks against the installed package confirmed catalog completion after the dot, information_schema filtering/suffix matches, and backtick parsing.

… editor

- Accepting a suggestion inside a backtick-quoted name (SELECT * FROM `tra)
  inserted "trades " and left the backtick open. Backtick names are now
  handled like double-quoted ones and closed with the quote that opened them.
- "--" or "/*" inside a backtick name (FROM `my--table` WHERE) was taken as
  the start of a comment, so no suggestions were shown.
- Block comments nest in QuestDB: after the inner "*/" of
  "/* outer /* inner */" the cursor is still in the comment, so no
  suggestions are shown there.
…ntents

- Block comments nest: text after an inner "*/" stays a comment.
- A backtick-quoted name is colored as an identifier (`order` no longer
  looks like a keyword).
- Number literals follow the @questdb/sql-parser lexer, so 1.5f, 1e36d, 10L,
  1.5e3m, 100ns, 0.0 and 1_000 are each one number. A leading sign is an
  operator, as in the parser (previously "v+10" colored "+10" as a number).
- E'...' is a string; $1 and :name are variables. "::" stays the cast
  operator, and ":" inside [...] stays a slice separator.
- Inside ARRAY[...] and other square brackets, functions, CASE words, names
  and operators are colored like everywhere else.
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown

Web Console deploy preview

Preview Commit Logs
https://pr-630--web-console.netlify.app 94c9754 build log

@emrberk emrberk changed the title fix: editor autocomplete and highlighting for backtick names, nested comments and literals fix: upgrade SQL parser to 0.1.22 and align editor completion and highlighting Nov 12, 2026

emrberk commented Nov 13, 2026

Copy link
Copy Markdown
Member Author

PR #630 review — level 3

PR: fix: upgrade SQL parser to 0.1.22 and align editor completion and highlighting

Base: b6415926fb19b4328f04f9b932246af367df0422
Head: 42b43436c651c5348eea0111c3c8168ff04c9ef6

Issues

Issue ID Issue name Category Severity Location Net impact Evidence Description Steps to reproduce Suggested fix
#1 Saved layouts stop restoring Cross-context caller impact Moderate out-of-diff — src/scenes/Editor/Notebook/queryKey.ts:8, consumed by result-table/ResultGridPanel.tsx:81–84 and cellVirtualization/GridShimmer.tsx:212–227. Violated contract: unchanged notebook SQL must retain its persisted layout identity. Attributed to the parser upgrade in package.json:48. Affected notebook users lose their saved column arrangement after upgrade. Rung 2: base/head roundtrip restores the saved layout at b6415926 and returns null at 42b43436. Rung 5: Cypress observation at 42b43436 shows default order [a,b]; changing only the stored key restores [b,a]. Full evidence. The upgraded formatter changes the query hash used to retrieve saved notebook column layouts. For the reproduced array query, the key changes from q1kmnpp6 to qar5hy2. The stored layout remains intact, but the grid loads its default arrangement. Exact-key lookup has no migration or fallback. This is a bounded layout-restoration failure; users can reconfigure the layout.
  • On base, run SELECT ARRAY[1.0,2.0][1: 2] AS a, 7 AS b in a saved notebook.
  • Move column b before a.
  • Upgrade to head, reopen the notebook, and run the unchanged SQL.
  • Observe default order a,b instead of saved order b,a.
Preserve backward-compatible persisted layout keys, or migrate existing entries before switching normalization. Add an upgrade regression test using a layout saved with the previous parser.
#2 Quoted table completions disappear Accessibility & UX Moderate in-diff — src/scenes/Editor/Monaco/questdb-sql/createSchemaCompletionProvider.ts:234–250. Users of catalog-prefixed table names lose autocomplete during quoted-name editing. Rung 2: identical-input provider comparison offers the table at b6415926 and omits it at 42b43436. Rung 5: Cypress observation confirms server acceptance, schema availability, head widget omission, and base-provider insertion. Full evidence. The new namespace branch interprets a quoted table name beginning with pg_catalog. as a function namespace. It returns catalog functions instead of the matching table. Monaco filters those functions out for the reproduced prefix, leaving no matching suggestion. Retrying completion does not recover it. Manual typing or deleting back before the dot still works.
  • Create a table named pg_catalog.foo using a quoted identifier.
  • Type SELECT * FROM "pg_catalog.f".
  • Place the cursor before the closing quote and request completion.
  • Observe that the existing table is missing; base offers it.
Retain matching quoted-table suggestions alongside catalog functions. Add a regression test for a real table whose literal name starts with a recognized namespace.

Adjacent findings (not blocking — file as issues)

None admitted.

Summary

  • Verdict: approve with comments. Address Moderate issues wip test for error range #1 and init react components package #2; neither blocks approval.
  • Correctness gate: pass. No admitted Critical findings remain open.
  • Test gate: pass. 0 admitted coverage gaps.
  • Quality checks: yarn typecheck, yarn build, yarn lint, and yarn test passed at 42b43436. The test suite passed 201 files / 3,473 tests.
  • Browser validation: The committed editorParserIntegration.spec.js suite passed 11 tests, with one retry. Both additional targeted browser probes passed, as did their base/head Vitest comparisons.
  • Regressions and tradeoffs: Certain existing notebook layouts stop restoring after upgrade. Catalog-function completion also displaces quoted-table completion for namespace-prefixed names. Both effects have bounded UI impact; the stored layout remains intact, and manual table-name entry remains available.
  • Location split: 1 in-diff, 1 out-of-diff.
  • Severity distribution: 0 Critical, 2 Moderate, 0 Minor.
  • Validation limits: Browser checks used the existing local database server, whose revision was unverified. Base comparisons executed the exact base completion adapter and the unchanged layout-key/store consumers with parser 0.1.20; they did not use a separate full base deployment.
  • Workspace: Temporary review source files and the test table were removed. Existing workspace changes were preserved.

@emrberk
emrberk merged commit 83d0b85 into main Nov 13, 2026
7 checks passed
@emrberk
emrberk deleted the fix/editor-lexical-states branch November 13, 2026 13:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants