Repository navigation
feat(sql): quoted arguments and implicit comparisons read as DuckDB does (SCHEMA-5b) - #460
Merged
Merged
Conversation
…oes (SCHEMA-5b) Story SCHEMA-5b. Reviewed: r0:must-fix, r1:must-fix, r2:must-fix, r3:must-fix, r4:clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cript entry (SCHEMA-5b) Story SCHEMA-5b. Reviewed: r5:clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…script (SCHEMA-5b) A conditional operand (NULLIF, COALESCE, CASE) has no column for `exists` to test, so Elasticsearch rejected the query. Such an operand now takes the script path, in both bridges, and is tested as one value. The #382 integration test now asserts what a query does with a text against a number: accepted, and failing when a text that spells no number is read. The computed-column refusal is unchanged. Story SCHEMA-5b. Reviewed: r6:clean, r7:clean. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… once per scope (SCHEMA-5b) The implicit-cast table made every comparison re-derive its target type at each of its renderers, and NULLIF re-ask, on every read of an argument's type, whether a literal-first COALESCE stands in for that argument. - Expression.implicitComparisonTarget is memoised per scope (PerScope), as literalReading is; InExpr and BetweenExpr override the computation. - ComparisonRule.literalInPlaceOf asks whether the operand is a COALESCE before it walks both chains for aggregates. - NullIf.first / second are memoised per scope. No verdict and no generated script changes. Two specs pin that one node answers each scope (query, transform, processor, computed-column validation) as a fresh node does, in any order. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Cost of this PR, measured on the pushed head (also added to the description). Main (
|
fupelaqu
marked this pull request as ready for review
November 10, 2026 12:33
This was referenced Nov 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Quoted arguments and implicit comparisons read as DuckDB does (SCHEMA-5b, after #455 and #456). A quoted argument of
COALESCE,GREATEST,LEASTor aCASEresult, beside a number, a date, a timestamp or a boolean, is read as that type:GREATEST(n, '1.5')over anINTanswers2(main refused it), andCOALESCE(n, '3')is anINT(main answered the text'3'on every row). A query also compares a text with a number or a boolean, and a boolean with a number, the way DuckDB casts them. That covers 17 of the 25 scalar cells DuckDB answers and core refused. Along the way, fourCOALESCE/ string-function rendering defects present on main are fixed: they produced wrong values or scripts that did not compile.Closes no issue.
What changes
BIGINT. A text that spells no value of that type leaves the expression as it was (COALESCE(n, 'abc')answers text,GREATEST(n, 'abc')is refused).BIGINT, againstDOUBLE/REAL/NUMERICasDOUBLE, againstBOOLEANasBOOLEAN; aBOOLEANagainst a whole type as1/0. These apply to=,<>,!=,IN,NULLIFandCASE x WHEN, and the boolean-against-whole row also to ordering andBETWEEN. A stored text that spells no value fails the query, as in DuckDB. Ordering a text against a number stays refused. Outside a query (computed columns, pipelines, transforms) a text side keeps 0.24.0's verdict, so a pipeline never silently drops a document; the boolean rows apply everywhere.BOOLEANbeside whole numbers answers1/0inCOALESCE, and inGREATEST/LEAST, in a query and in a computed column.COALESCE(COALESCE(n, '1.5'), k)answers'2'), which also answers statements that were refused because a nested expression mixed text with a number or a date.COALESCE('2024-01-15', d)compared for equality with a text compares the literal's text (=,<>,!=,NULLIF,CASE … WHEN), andCOALESCE(NULLIF(F, k), k)answersCOALESCE(F, k), both in a query only. Main answered these with the same values (the literal-first forms) or the same dates in another text (COALESCE(NULLIF(F, k), k)over aCASE); refusing them would have broken working statements.COALESCEfixes, in every scope (query,CREATE TABLE/ALTER TABLE,UPDATE … SET, transforms):CASEargument before the last one is tested whole forNULL: main tested theELSEalone, soCOALESCE(CASE WHEN c THEN a ELSE b END, k)answered NULL wherecheld andawas NULL, and a NULLELSEdid not compile;GREATEST/LEASTargument holding a literal, which is never NULL:COALESCE(GREATEST(n, 3), l)and its twins failed to compile on main (Cannot cast from [long] to [java.lang.Object]).UPPER(n)stored no value, andLOWER(CASE … END)lowered only theELSE.CAST(b AS DATE | TIME | TIMESTAMP)over aBOOLEANis refused in every scope, as DuckDB refuses it. Main answered the boolean itself in a date column.BOOLEANwith a number compares it as1/0, as a query does. Main compared the boolean itself and stored a wrong value or none.functions_conditional.md,operators.md("Comparing values of different types") andtype_conversion.mdlist every moved statement by name.Release notes
GREATEST(n, '1.5')=2(was refused);COALESCE(n, '3')is anINT(was text);COALESCE(d, '2024-01-15')is aDATE(was text).DESCRIBE, JDBC / Arrow types and CTAS / MV mappings move with it (keyword→integer/date), and so does every consumer:ORDER BYandGROUP BYkeys follow the new type, a comparison orCAST(… AS VARCHAR)reads the number ('1.5'→'2'), and arithmetic answers (COALESCE(n, '1.5') + 1).'on'/'off'/' true 'beside a boolean answers, where DuckDB fails.1/0: inCOALESCEin a query (wastrue/falsein an integer column), and inGREATEST/LEASTin a query (was refused) and in a computed column (accepted on main, but no value was stored).ALTER TABLEre-creates (new bytes, values corrected): computed columns holding a quoted argument read as a value (COALESCE(n, '3')stores3;'1.5'beside an integer stores2, where Elasticsearch truncated it to1);GREATEST/LEASTover aBOOLEANbeside whole numbers;CASEcomparing aBOOLEANwith a number (the shapes are listed infunctions_conditional.md); string functions over a number, a boolean or a conditional;COALESCEwith aCASEargument before its last;COALESCEover aGREATEST/LEASTholding a literal.COALESCE(GREATEST(n, '2'), k),COALESCE(GREATEST(d, '2024-01-15'), k)and the other nested shapes listed infunctions_conditional.md;NULLIF(b, v)/NULLIF(v, b)over aBOOLEANand a number in a computed column;NULLIF(COALESCE(d, '2024-01-15'), k) - d2andCOALESCE(NULLIF(COALESCE(d, '2024-01-15'), k), d2) - d2in a computed column.ISNULL/ISNOTNULLover a conditional in aWHEREnow run: an operand with no column (NULLIF,COALESCE,CASE, ...) is evaluated by a script and tested as one value. Elasticsearch rejected these queries before (field name is null or empty).COALESCE(NULLIF(F, k), k)over a dateCASEanswers'2024-01-15'where main answered'2024-01-15T00:00Z'; over a timestampCASEthe literal answers'2024-01-15T10:30Z'where main answered'2024-01-15 10:30:00'.Answered before (wrong or failing), refused now
KEYWORD, dDATE, tsTIMESTAMP, nINT, bBOOLEAN)NULLIF(F, k),NULLIF(k, F),CASE F WHEN k,F = kand expressions over them, forFa conditional holding a quoted date or timestamp (COALESCE(d, '2024-01-15'),CASE WHEN … THEN d ELSE '2024-01-15' END, …; full list infunctions_conditional.md)CASEoperands and the literal-first forms with a time part or a timestamp; a runtime failure over the others (measured on ES 8.18.3)COALESCE(d, '2024-01-15') = k,COALESCE(n, '3') > k,COALESCE(n, '3') LIKE '1%',CAST(COALESCE(n, '3') AS DATE)COALESCE('2024-01-15', d) < k/>= kNULLIF(COALESCE(n, '3'), k),NULLIF(k, COALESCE(n, '3')),NULLIF(COALESCE(n, '3'), '2024-01-15'),COALESCE(NULLIF(COALESCE(n, '3'), k), k | g)CREATE TABLEandALTER TABLE, including anyALTER TABLEof a table that already holds oneCAST(b AS DATE | TIME | TIMESTAMP)Acceptance evidence
QuotedArgumentConditionalSpec(testkit trait, wired for es6 jest / rest, es7 rest, es8 java, es9 java) andJavaClientSchemaConsumerSpec;QuotedArmReadingSpec,ImplicitComparisonCastSpecand the updated sql specsCASEoperands wrong)CREATE TABLE,ALTER TABLE,UPDATE, transform; every moved verdict or byte is listed in the docsDateArithmeticSpec,DateFunctionExecutionSpec,ScrollCompletenessSpec,GroupByCompletenessSpec,ComputedProjectionSpec,PredicateFunctionResultSpec,JavaClientSpec)sbt "+ compile"OK (2.12.20 and 2.13.16); on the first commit: bridge 353/0, es6 bridge 360/0, core 1654/0COALESCEcomparison entered directly in a transform or pipeline context renders as before (the rule is query-only)ISNULL/ISNOTNULLover an operand with no columnexistson an empty field; aCASEoperand measured answering the wrong rows until it was enclosed). On the fix:QuotedArgumentConditionalSpec28/0 on both, the #382 test passing on both, and on ES 8.18.3GatewayApiIntegrationSpec,PredicateFunctionResultSpec,DateFunctionExecutionSpec,GroupByCompletenessSpec,GroupedAggregateSemanticsSpecall passing; sql 2345/0, es8 bridge 353/0, es6 bridge 360/0sbt "+ compile"OK; ES 8.18.3 leg 434/0 (DateArithmeticSpec,DateFunctionExecutionSpec,ScrollCompletenessSpec,GroupByCompletenessSpec,ComputedProjectionSpec,PredicateFunctionResultSpec,JavaClientSpec,QuotedArgumentConditionalSpec,JavaClientSchemaConsumerSpec,GatewayApiIntegrationSpec,TypesPredicatesFunctionsSpec,GreatestLeastSpec)headerCheck scalafmtSbtCheck scalafmtCheck test:scalafmtCheck)Apart from the ES 6.8 runs of the
ISNULL/ISNOTNULLrow, not run locally: ES 6.8, 7.17 and 9.0, which this PR's CI runs, including the trait rows. Transforms were compared at the rendering level only, with no run. The new rules decide at render time, memoised per scope, and add no per-row work.Cost
Main (
bc88e45c) against this branch, interleaved, warm JIT, JVMs started directly, 5 rounds with the order rotated, machine load below 8. Planning is what a request pays before Elasticsearch sees it: parse, schema attach, type validation and Painless emission, over 116 statements (110 with the same verdict on both trees). Medians of the per-round sums:96b5b858)WHERE id IN (1, 2, 3)over a KEYWORDidemits more because it now emits the conversion; main compared the text with the numbers and matched no row.CREATE TABLEwith a NULLIF shape at depth 6 takes 3.7 s on main and 2.7 ms here; a dateCASEat depth 6 takes over 20 s on main and 52 ms here.Recorded, not fixed
Measured the same on main and on this branch:
CASE WHEN COALESCE('2024-01-15', d) BETWEEN k AND g THEN …(two text bounds) is accepted and fails when the query runs.k = '2024-01-15T00:00:00Z'), or no date ('3'), compares as text with a literal-firstCOALESCE('2024-01-15', d): not equal. Comparing it as a date needs a runtime date parse ofk.COALESCE(COALESCE(GREATEST(n, 3), l), x)still null-tests the inner number and fails to compile.COALESCE(GREATEST(MAX(n), MAX(l)), 0)rendersMath.max(...) != null; it is refused before it runs.CASE WHEN n IS NULL THEN …renders a condition that is always false, so aSCRIPT AScolumn takes theELSEwherenis missing.COALESCE(<arm>, 7)declaresBIGINT, but a row that takes the literal holds anIntegeruntil the typed result path converts it.GREATEST(d, '…')compared in a pipeline fails to compile; aGREATESTholding a NULL literal in a pipeline rendersMath.max(null, ...).NULLIFinside aCASE WHENcondition is not judged by theNULLIFtype rule.HAVING MAX(keyword | text) > '2024-01-01'keeps main's refusal, and comparisons over atextcolumn are left to Elasticsearch: 8 of the 25 cells, on the 0.25.0 gate list.Measured on this branch (the rendering is unchanged from main, not run on main):
NULLIFthat answers NULL (UPPER(NULLIF(k, '1'))) calls the method on NULL and fails the query.Introduced by this branch, value-safe, recorded:
COALESCEstops at a never-NULL argument, the script may declare a parameter it never reads (def param1 = ctx.l). Values are unaffected.Release preconditions (0.25.0)
🤖 Generated with Claude Code