From 67242136f2db52714c8af849fa74ab4190aef381 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?St=C3=A9phane=20Manciot?= Date: Sun, 15 Nov 2026 08:06:54 +0100 Subject: [PATCH] docs: materialized view column types and limitations (engine 0.25.0) A view now declares its columns with the types a query of its definition reports (KEYWORD/TEXT and computed columns keep theirs), and reads its WHERE, HAVING and calculations with the source types. Document that, the real DESCRIBE MATERIALIZED VIEW output, the SUM column type the web page already carried, and three limitations measured on Elasticsearch 8.x; point the boolean rows of operators.md at the view exception. Co-Authored-By: Claude Opus 5.5 --- documentation/sql/materialized_views.md | 60 ++++++++++++++++++++----- documentation/sql/operators.md | 4 +- 2 files changed, 53 insertions(+), 11 deletions(-) diff --git a/documentation/sql/materialized_views.md b/documentation/sql/materialized_views.md index aef93672..2efaa940 100644 --- a/documentation/sql/materialized_views.md +++ b/documentation/sql/materialized_views.md @@ -249,9 +249,35 @@ YEAR(o.created_at)`, or `GROUP BY y`, or `GROUP BY 1` — stores one row per gro a `keyword` sub-field for `UPPER` and the other string functions. A view grouped by an aliased column (`SELECT c.city AS town, COUNT(*) … GROUP BY town`, `GROUP BY c.city` or the ordinal) stores the key under its alias, over one table or a JOIN. A group whose key is `NULL` is not stored. +A `SUM` column is declared `BIGINT` over an integer column and `DOUBLE` over a +`REAL` or a `DOUBLE` column (*since engine `0.25.0`*; see [Aggregate functions](functions_aggregate.md)). A `HAVING` in a view is more constrained than a `HAVING` in a `SELECT` — most importantly, every aggregate it reads must be published in the `SELECT` list **with an alias**, as `SUM(o.amount) AS total_amount` is above. See [HAVING in a materialized view](#having-in-a-materialized-view). +### Column types + +*Since engine `0.25.0`*, a view declares each column of its `SELECT` list with the type a query of +its definition reports — the type `DESCRIBE` gives that column of the view's `SELECT` — over a +single table, and table by table over a JOIN, with two exceptions: + +- A column keeps the type of its source mapping where that type says more: a `KEYWORD` or `TEXT` + column, which a query reports as `VARCHAR`, is stored as `KEYWORD` or `TEXT`. +- A computed column (`UPPER(c.name) AS customer_name_upper`) keeps the type its script writes. + +A per-group calculation over aggregates is declared with the type the query reports: a `DATEDIFF` +is a `BIGINT` (a view created before `0.25.0` declared it `DOUBLE`; the values are the same). + +`DESCRIBE MATERIALIZED VIEW` reports the declared types. A query over the view reports a query's +types, as over any table: `VARCHAR` for a `KEYWORD` or `TEXT` column. + +The view's `WHERE`, its `HAVING` and its per-group calculations read the definition with the source +tables' types, as a query does: a `BOOLEAN` column compared for equality with a number column +(`b = n`, `b <> n`) is answered, where it was refused (HTTP 400) before `0.25.0`. The transform +still applies some conditions differently from a query: see [Limitations](#limitations). + +A view created before `0.25.0` keeps the types it was created with: a `CREATE OR REPLACE` with the +same definition leaves it as it is, and `DROP MATERIALIZED VIEW` followed by `CREATE` re-types it. + --- ## DROP MATERIALIZED VIEW @@ -310,7 +336,12 @@ REFRESH MATERIALIZED VIEW orders_with_customers_mv WITH SCHEDULE NOW; DESCRIBE MATERIALIZED VIEW view_name; ``` -Returns the schema of the materialized view index, showing columns, types, nullability, and other metadata. +Returns the schema of the materialized view index, showing columns, types, nullability, and other +metadata. The types are the ones the view declares (see [Column types](#column-types)). + +Besides the `SELECT` items, the view index holds the `keyword` sub-field of each text column copied +from a source table (here `customer_name` and `email`), the columns its stages read (here the join key `customer_id`, the `WHERE` column `status` and the +`createdAt` the computed `effective_date` reads), and `_last_updated`, the time of the last write. **Example:** @@ -318,15 +349,21 @@ Returns the schema of the materialized view index, showing columns, types, nulla DESCRIBE MATERIALIZED VIEW orders_with_customers_mv; ``` -| Field | Type | Null | Key | Default | Comment | Script | Extra | -|----------------------|-----------|------|-----|---------|---------|--------|-------| -| id | INT | yes | | NULL | | | () | -| amount | DOUBLE | yes | | NULL | | | () | -| customer_name | VARCHAR | yes | | NULL | | | () | -| email | VARCHAR | yes | | NULL | | | () | -| customer_zip | VARCHAR | yes | | NULL | | | () | -| customer_name_upper | KEYWORD | yes | | NULL | | | () | -| effective_date | TIMESTAMP | yes | | NULL | | | () | +| Field | Type | Null | Key | Default | Comment | Script | Extra | +|-------------------------|-----------|------|-----|---------------------|----------------------------------------------|---------------------------|-------| +| `_last_updated` | TIMESTAMP | no | | `_ingest.timestamp` | Last update timestamp | | () | +| `amount` | DOUBLE | yes | | NULL | | | () | +| `createdAt` | DATE | yes | | NULL | | | () | +| `customer_id` | INT | yes | | NULL | | | () | +| `customer_name` | VARCHAR | yes | | NULL | Enriched from customers.name | | () | +| `customer_name.keyword` | KEYWORD | yes | | NULL | Keyword multi-field for aggregations | | () | +| `customer_name_upper` | VARCHAR | yes | | NULL | Computed: UPPER(customer_name) | `UPPER(customer_name)` | () | +| `customer_zip` | KEYWORD | yes | | NULL | Enriched from customers.department.zip_code | | () | +| `effective_date` | DATE | yes | | NULL | Computed: COALESCE(NULLIF(createdAt, …), …) | `COALESCE(NULLIF(createdAt, DATE_PARSE('2025-09-11', '%Y-%m-%d') - INTERVAL 2 DAY), CURRENT_DATE)` | () | +| `email` | VARCHAR | yes | | NULL | Enriched from customers.email | | () | +| `email.keyword` | KEYWORD | yes | | NULL | Keyword multi-field for aggregations | | () | +| `id` | INT | yes | PRI | NULL | | | () | +| `status` | KEYWORD | yes | | NULL | | | () | --- @@ -548,6 +585,9 @@ DROP MATERIALIZED VIEW IF EXISTS orders_with_customers_mv; | **`LIMIT` / `OFFSET`** | A `LIMIT` in the definition is only recorded: the view stores every row or group. An `OFFSET` of groups is refused by name. | | **A `GROUP BY` key the view cannot compute** | A key whose function the engine does not script — `DATE_TRUNC` with an interval, `DATE_FORMAT`, … — is refused at creation, naming the key and its expression. So is a view whose aggregation would never write one of its key columns. | | **The `NULL` group** | A group whose key is `NULL` is not stored, as a query's `GROUP BY` forms no `NULL` group either. | +| **A comparison only a query converts** | *Since engine `0.25.0`* a query compares a text column with a number by converting the text (see [Comparing values of different types](operators.md#comparing-values-of-different-types)). A view keeps the earlier refusal where the comparison runs as a script — a computed column or `GROUP BY` key (`NULLIF(k, 0)`, `CASE WHEN k = 0 …` over a `KEYWORD` or `TEXT` column `k`), or a `WHERE` that compares the text column with a number column (`WHERE k = n`): the view is refused at creation with HTTP 400 and the engine's type message, for example *"NULLIF(k, 0) compares KEYWORD with BIGINT: NULLIF requires two arguments of comparable types, so cast one of them"*, before anything is created. Without the refusal, a view grouped by `CASE WHEN k = 0 …` stored a wrong group, without an error. A text column compared with a literal (`WHERE k = 0`) runs as before. | +| **`YEAR(…)` of a `DATE` column in `WHERE`** | Measured on Elasticsearch `8.x`: a view whose `WHERE` reads `YEAR(created_at) = 2024` over a `DATE` column is created, but stores **no** rows, where the same query returns the matching rows: the transform's source query does not apply the function. Filter when querying the view. | +| **`COALESCE` over a `DATE`, or a `BOOLEAN` ordered against a number, in `WHERE`** | Measured on Elasticsearch `8.x`: `WHERE COALESCE(created_at, '2024-01-15') = '2024-01-15'` (or `>`), and `WHERE b > 0` over a `BOOLEAN` column `b`, fail at creation with HTTP 500 — *"Failed to deploy materialized view artifacts: Deployment failed and rolled back: …"* — and the deployment is rolled back. The query answers. Filter when querying the view. | | **A nested column in `WHERE`** | A condition over a nested column written without `NESTED(…)` matches nothing in a view's `WHERE` (a query tests the column's elements). Write the `NESTED(…)` condition. | | **Every table needs its schema** | A view one of whose tables has no schema under its index name is refused: *"Materialized view v cannot be typed: no schema was loaded for table t."* | | **UNNEST JOIN** | Not supported in materialized views | diff --git a/documentation/sql/operators.md b/documentation/sql/operators.md index c8d91a73..f8796736 100644 --- a/documentation/sql/operators.md +++ b/documentation/sql/operators.md @@ -726,7 +726,9 @@ SELECT * FROM t WHERE b = n; -- b BOOLEAN read as 1 / 0 - Outside a query — a computed column (`SCRIPT AS`, including the ones a `CREATE TABLE … AS SELECT` copies), an ingest pipeline, a materialized view's transform — a text side keeps `0.24.0`'s verdict: `NULLIF(k, 0)` is refused there, because a pipeline would silently drop a - document whose text converts to nothing. The boolean rows apply everywhere. A conditional that + document whose text converts to nothing. The boolean rows apply everywhere, except that a + materialized view's `WHERE` that orders a `BOOLEAN` against a number (`b > 0`) fails at creation + (see [Materialized view limitations](materialized_views.md#limitations)). A conditional that reads a quoted argument as a number is that number there too (see [Conditional functions](functions_conditional.md)), so `NULLIF(COALESCE(n, '3'), k)`, `NULLIF(k, COALESCE(n, '3'))` and `NULLIF(COALESCE(n, '3'), '2024-01-15')` are refused in a