diff --git a/CHANGELOG.md b/CHANGELOG.md index e99ffa70..b9a562f9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,109 @@ +## 1.34.0 + +`null` and `[]` mean the same in a condition: an empty set for `=~`/`!~`, a +`null` for `==`/`!=`. Behavior changes, on every DB adapter, for any field +(a number, a text, a relationship, a path through one): + +| Condition | Now | Was | +|---|---|---| +| `x =~ null`, `x =~ ?` with `null` | none (as `x =~ []`) | `IS NULL` | +| `x !~ null`, `x !~ ?` with `null` | all (as `x !~ []`) | `IS NOT NULL` | +| `x == []`, `x == ?` with `[]` | `IS NULL` (as `x == null`) | memory: none | +| `x != []`, `x != ?` with `[]` | `IS NOT NULL` (as `x != null`) | memory: all | + +(SQL already encoded `== []` / `!= []` as `IS NULL` / `IS NOT NULL`.) + +An explicit `null` element is kept: `number =~ [null]` / `=~ [3, null]` +(also a parameter `[null]`) still match a `null` (`IS NULL`), except for a +relationship field: + +- A relationship field (a to-one: an entity or an `EntityReference`; a + to-many: a `List` of entities or an `EntityReferenceList`) is compared by + the IDs of its related entities, which are never `null`: a `null` element + is ignored. `store =~ [null]` matches none and `store =~ [1, null]` is + `store =~ [1]` (it was `IS NULL`, "without a related entity"). Use + `store == null` / `store != null` (or `== []` / `!= []`) for "without" / + "with" a related entity. (`KeyCondition.isRelationshipKey`.) + A relationship field is resolved by its declared type, on memory and SQL + alike (`KeyCondition.isRelationshipType`): an `EntityReference`, an + `EntityReferenceList`, an entity type, or a `List` of one (not a `List` + of enums or of other classes); on SQL not by the DB's foreign keys (a + to-one may have none, e.g. `User.photo`, a `String` ID). +- `==`/`!=` with a single-element list is with its element, as SQL + already did: `store == [7]` is `store == 7` (memory matched none). Not + for a list or binary field (a `List`, a `Uint8List`): it's + compared with the list as a whole (`tags == ["a"]` doesn't match + `["a", "b"]`). +- An `EntityReference` (or an `EntityReferenceList`) as a parameter is + compared by its ID(s), with `=~`/`!~` (also of a path ending in a + reference: `owner.userInfo =~ ?`) and with `==`/`!=` (a reference, or a + reference list with up to 1 ID: as `== [1, 3]`, more IDs aren't a + single value). They matched none (SQL: `IS NULL`). +- `== []` is `IS NULL` also for a list or binary field (an empty + `List`, an empty `Uint8List`, is not `== []`). A missing named + parameter (`x =~ ?:a` without `a`) is a `null`: an empty set. +- Memory: a path through a to-one whose row is missing (only its ID) is + its ID for the ID field and a `null` for other fields, as a SQL outer + join; it failed. +- `==`/`!=` with a parameter list of 2 or more values isn't a single value: + SQL fails (unchanged), and memory matches none (it compared any element). +- Fix (SQL): an enum or an entity in a one-element list (`roles.type == ?` + with `[RoleType.admin]`, `photo == ?` with `[photo]`) was compared as its + text (`toString`), not its name / ID. +- Fix (memory): a list field with a `null` element (`['a', null]`) was + `== null` (any element `null`); a list is a value, not `null`. +- Fix (SQL, 1.33.0 regression): a query through a to-many relationship, + without an explicit order (`orderByID: null`), lost the ID order its + `JOIN` had: its IDs subquery (`id IN ( SELECT ... )`) may not keep the + table order (PostgreSQL's hash semi-join). It's ordered by ID again (as + memory). +- Fix: `store == ?` with `[]` (a to-one entity) failed resolving `[]` as an + entity's ID (memory). A list as a single entity's value: `[]` is `null`, + `[x]` is `x`. +- Fix (SQL): an entity as a parameter of a `String`/`int` ID column (a + to-one without a foreign key, `photo == ?` with a `Photo`) was compared + as its text (`toString`), not its ID. +- Fix (memory): a to-one whose entity is stored by another adapter (a + `Photo` in an object storage) was resolved as `null`: `photo == null` + matched every user, and `photo =~ [id]` none. Its ID is kept, as a SQL + column. And a `String` ID for an entity-typed key (`photo =~ [""]`) + failed resolving it as an entity. +- `KeyConditionINBase.valuesAsList(null)` is `[]` (it was `[null]`). + +Tests: + +- 'Relationship operators: by ID, on every adapter', on all the DB + adapters, with the expected results computed from a model (each entity's + set of related IDs), each also checked through `!(...)`: `=~`/`!~` with + `[]`, `null`, `[null]`, an ID, `[id]`, `[id1, id2]`, `[id, null]` (as + literals, and as `?` and `?:name` parameters), and entities as parameters; + `==`/`!=` with `null`, `[]` (literals and parameters) and IDs; `Set` + parameters, `EntityReference`/`EntityReferenceList` parameters, `== [id]`, + and `&&`/`||` combinations. Over a to-many `EntityReferenceList` + (`Address.closedStores`), a to-many `List` (`Address.stores`), a to-one + entity (`Store.owner`), a to-one `EntityReference` (`User.userInfo`), and + a to-one with a `String` ID without a foreign key (`User.photo`); a path + ending in a reference (`owner.userInfo`) with references as parameters; + and a to-many's `countByQuery` and `limit`/`offset` pages. +- 'Operators: the same results on every adapter': `=~ null`, `!~ null`, + `=~ [null]`, `== []`, `!= []`, `== [v]`, `=~ [v]` and their parameters, + over text, numbers, a to-one's and a to-many's fields. +- Unit: `==`/`!=` with a list on a list or binary field (as a whole). + +Known differences (unchanged): + +- 2 to-many relationships in one condition (`closedStores =~ [1] && + stores =~ [2]`) fail on SQL (one relationship table is joined): see + 1.33.0. +- A `String` ID for an integer-ID relationship (`owner == ?` with `'1'`) + matches on SQL (the DB converts it), and not in memory (`'1'` isn't + `1`). +- Matching entity objects (not a DB adapter: `Condition.matchesEntity`) + doesn't unpack an `EntityReferenceList` field to its IDs. +- A path through a to-one stored by another adapter (`photo.data`, a + `Photo` in an object storage) isn't supported (an error, on every + adapter). + ## 1.33.0 The condition operators give the same results on every DB adapter (memory, diff --git a/benchmark/db_users/HISTORY.md b/benchmark/db_users/HISTORY.md index a883f8de..4d6f6f7e 100644 --- a/benchmark/db_users/HISTORY.md +++ b/benchmark/db_users/HISTORY.md @@ -23,45 +23,92 @@ its date's column here too. ### `memory` -| Operation | 2026-10-03 | 2026-10-13 | 2026-10-14 | -|---|--:|--:|--:| -| `user/byId` | 25303 | 30312 (+19.8%) | 30937 (+2.1%) | -| `user/byEmail` | 1407 | 1449 (+3.0%) | 1437 (-0.8%) | -| `user/byState (join, limit 20)` | 1409 | 1594 (+13.1%) | 1601 (+0.4%) | -| `user/list (page of 20)` | 682 | 720 (+5.6%) | 719 (-0.1%) | -| `user/count` | 193802 | 320214 (+65.2%) | 320629 (+0.1%) | -| `authenticate (login)` | 1393 | 1459 (+4.7%) | 1491 (+2.2%) | -| `user/register` | 2623 | 3514 † | 3525 (+0.3%) | -| `user/update` | 1881 | 5215 † | 5190 (-0.5%) | -| `user/remove` | 19969 | 12506 † | 12476 (-0.2%) | +| Operation | 2026-10-03 | 2026-10-13 | 2026-10-14 | 2026-11-08 | +|---|--:|--:|--:|--:| +| `user/byId` | 25303 | 30312 (+19.8%) | 30937 (+2.1%) | 36725 (+18.7%) | +| `user/byEmail` | 1407 | 1449 (+3.0%) | 1437 (-0.8%) | 1428 (-0.6%) | +| `user/byState (join, limit 20)` | 1409 | 1594 (+13.1%) | 1601 (+0.4%) | 1790 (+11.8%) | +| `user/list (page of 20)` | 682 | 720 (+5.6%) | 719 (-0.1%) | 766 (+6.5%) | +| `user/count` | 193802 | 320214 (+65.2%) | 320629 (+0.1%) | 244367 (-23.8%) | +| `authenticate (login)` | 1393 | 1459 (+4.7%) | 1491 (+2.2%) | 1435 (-3.8%) | +| `user/register` | 2623 | 3514 † | 3525 (+0.3%) | 3553 (+0.8%) | +| `user/update` | 1881 | 5215 † | 5190 (-0.5%) | 5444 (+4.9%) | +| `user/remove` | 19969 | 12506 † | 12476 (-0.2%) | 2909 (-76.7%) ‡ | ### `sqlite` -| Operation | 2026-10-03 | 2026-10-13 | 2026-10-14 | -|---|--:|--:|--:| -| `user/byId` | 17138 | 21220 (+23.8%) | 24721 (+16.5%) | -| `user/byEmail` | 16580 | 20528 (+23.8%) | 24067 (+17.2%) | -| `user/byState (join, limit 20)` | 2326 | 3184 (+36.9%) | 3283 (+3.1%) | -| `user/list (page of 20)` | 2405 | 3352 (+39.4%) | 3433 (+2.4%) | -| `user/count` | 121903 | 162445 (+33.3%) | 195377 (+20.3%) | -| `authenticate (login)` | 14764 | 18140 (+22.9%) | 20985 (+15.7%) | -| `user/register` | 7727 | 8284 (+7.2%) | 9719 (+17.3%) | -| `user/update` | 10302 | 11665 (+13.2%) | 13774 (+18.1%) | -| `user/remove` | 13036 | 14247 (+9.3%) | 16953 (+19.0%) | +| Operation | 2026-10-03 | 2026-10-13 | 2026-10-14 | 2026-11-08 | +|---|--:|--:|--:|--:| +| `user/byId` | 17138 | 21220 (+23.8%) | 24721 (+16.5%) | 29556 (+19.6%) | +| `user/byEmail` | 16580 | 20528 (+23.8%) | 24067 (+17.2%) | 28649 (+19.0%) | +| `user/byState (join, limit 20)` | 2326 | 3184 (+36.9%) | 3283 (+3.1%) | 4382 (+33.5%) | +| `user/list (page of 20)` | 2405 | 3352 (+39.4%) | 3433 (+2.4%) | 4731 (+37.8%) | +| `user/count` | 121903 | 162445 (+33.3%) | 195377 (+20.3%) | 190389 (-2.6%) | +| `authenticate (login)` | 14764 | 18140 (+22.9%) | 20985 (+15.7%) | 24460 (+16.6%) | +| `user/register` | 7727 | 8284 (+7.2%) | 9719 (+17.3%) | 9520 (-2.0%) | +| `user/update` | 10302 | 11665 (+13.2%) | 13774 (+18.1%) | 15050 (+9.3%) | +| `user/remove` | 13036 | 14247 (+9.3%) | 16953 (+19.0%) | 17013 (+0.4%) | ### `postgres` -| Operation | 2026-10-03 | 2026-10-13 | 2026-10-14 | -|---|--:|--:|--:| -| `user/byId` | 1453 | 1436 (-1.2%) | 1795 (+25.0%) | -| `user/byEmail` | 1518 | 1414 (-6.9%) | 1823 (+28.9%) | -| `user/byState (join, limit 20)` | 537 | 733 (+36.5%) | 887 (+21.0%) | -| `user/list (page of 20)` | 490 | 810 (+65.3%) | 888 (+9.6%) | -| `user/count` | 6167 | 5424 (-12.0%) | 6693 (+23.4%) | -| `authenticate (login)` | 1474 | 1350 (-8.4%) | 1724 (+27.7%) | -| `user/register` | 611 | 629 (+2.9%) | 718 (+14.1%) | -| `user/update` | 713 | 684 (-4.1%) | 850 (+24.3%) | -| `user/remove` | 803 | 762 (-5.1%) | 890 (+16.8%) | +| Operation | 2026-10-03 | 2026-10-13 | 2026-10-14 | 2026-11-08 | +|---|--:|--:|--:|--:| +| `user/byId` | 1453 | 1436 (-1.2%) | 1795 (+25.0%) | 1629 (-9.2%) | +| `user/byEmail` | 1518 | 1414 (-6.9%) | 1823 (+28.9%) | 1669 (-8.4%) | +| `user/byState (join, limit 20)` | 537 | 733 (+36.5%) | 887 (+21.0%) | 1069 (+20.5%) | +| `user/list (page of 20)` | 490 | 810 (+65.3%) | 888 (+9.6%) | 1148 (+29.3%) | +| `user/count` | 6167 | 5424 (-12.0%) | 6693 (+23.4%) | 6033 (-9.9%) | +| `authenticate (login)` | 1474 | 1350 (-8.4%) | 1724 (+27.7%) | 1668 (-3.2%) | +| `user/register` | 611 | 629 (+2.9%) | 718 (+14.1%) | 927 (+29.1%) | +| `user/update` | 713 | 684 (-4.1%) | 850 (+24.3%) | 875 (+2.9%) | +| `user/remove` | 803 | 762 (-5.1%) | 890 (+16.8%) | 1187 (+33.4%) | + +2026-11-08: 1.34.0 at `024551e` (PR #205, the condition operators' fixes +of 1.32.1–1.34.0), the median of 5 rounds (`--repeat 5`). The change from +2026-10-14 spans 1.22.0 → 1.34.0, not only these fixes: to measure them, +1.32.1, 1.33.0 and this PR were run one after another in the same session +(below). PostgreSQL ran on one existing container for all three +(`--pg-host`, `postgres:latest` in Docker Desktop), not `--docker`. + +These fixes, same machine and session (ops/sec; PR vs 1.32.1): + +| Operation | memory 1.32.1 | 1.33.0 | PR | | sqlite 1.32.1 | 1.33.0 | PR | | postgres 1.32.1 | 1.33.0 | PR | | +|---|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:|--:| +| `user/byId` | 37318 | 36960 | 36725 | -1.6% | 29640 | 29269 | 29556 | -0.3% | 1797 | 1642 | 1629 | -9.3% | +| `user/byEmail` | 1437 | 1434 | 1428 | -0.6% | 28641 | 28204 | 28649 | 0.0% | 1774 | 1261 | 1669 | -5.9% | +| `user/byState (join, limit 20)` | 1808 | 1814 | 1790 | -1.0% | 4397 | 4226 | 4382 | -0.3% | 1130 | 1019 | 1069 | -5.4% | +| `user/list (page of 20)` | 771 | 778 | 766 | -0.6% | 4746 | 4562 | 4731 | -0.3% | 1180 | 952 | 1148 | -2.7% | +| `user/count` | 243991 | 245087 | 244367 | +0.2% | 189944 | 169970 | 190389 | +0.2% | 6512 | 6143 | 6033 | -7.4% | +| `authenticate (login)` | 1460 | 1475 | 1435 | -1.7% | 24511 | 24295 | 24460 | -0.2% | 1747 | 1628 | 1668 | -4.5% | +| `user/register` | 3599 | 3457 | 3553 | -1.3% | 8955 | 9251 | 9520 | +6.3% | 929 | 913 | 927 | -0.2% | +| `user/update` | 5394 | 5435 | 5444 | +0.9% | 15006 | 14858 | 15050 | +0.3% | 912 | 879 | 875 | -4.1% | +| `user/remove` | 2626 | 2782 | 2909 | +10.8% | 17706 | 16001 | 17013 | -3.9% | 1204 | 1209 | 1187 | -1.4% | + +- Memory and SQLite: the PR is within noise of 1.32.1 (±2%, `remove` and + `register` vary the most between rounds). These operations filter by a + field, an ID or a to-one's field (`byState`), not through a to-many + relationship, so the to-many subquery and its ID order don't run here; + the conditions' new checks (a relationship key, `null` and `[]` values) + cost nothing measurable. 1.33.0's SQLite dip (`byState`/`list` -4%, + `count` -10%) isn't in the PR. +- PostgreSQL: 1-9% lower in this table, but `count` (no condition: none + of the changed code runs) dropped as much (-7.4%), and the versions ran + in order on one container. An A/B alternating 1.32.1 and the PR 3 times + each on one container (`--repeat 3`, medians of the 3 runs) measured no + loss: `byId` 1741 vs 1733 (-0.5%), `byEmail` 1762 vs 1735 (-1.5%), + `byState` 1122 vs 1120 (-0.2%), `list` 1185 vs 1179 (-0.5%), `count` + 6505 vs 6565 (+0.9%), within the runs' spread (`byEmail` 1710–1794, + `count` 5712–7006). The table's drop is the container's drift during the + session (see the 2026-10-13 notes), not these fixes. + +‡ Memory `remove` is ~2.7k ops/sec on all three versions (1.32.1 too): the +drop from 12476 (2026-10-14, 1.22.0) is older than these fixes (between +1.22.0 and 1.32.1), as `count`'s -24%. Not investigated here. + +The machine's label here is the one detected (`Apple M5`; the earlier +entries were labeled `Mac17,3 (Apple M5)`): the same machine. Two idle +`dart` processes of other projects (a dev server and `webdev serve`, ~0% +CPU) were running; no tests or builds. 2026-10-14: 1.22.0 at `a508d8f`, the median of 5 rounds (`--repeat 5`), on AC power with the machine idle. Since 2026-10-13: the IDs of ID @@ -91,6 +138,86 @@ scans tables, so its numbers depend the most on the table size. --- +### 2026-11-08 — `postgres` — Apple M5 + +- **bones_api:** 1.34.0 (`024551e`) +- **Machine:** Apple M5, 10 cores, 16 GB RAM +- **OS:** macos (Version 26.6.2 (Build 25G83)) +- **Dart:** 3.13.5 +- **DB:** postgres (PostgreSQL 18.6 (Debian 18.6-1.pgdg13+2), localhost:55432) +- **Users:** 1000; **duration:** 3s (warm-up 1s) × 5 rounds, median shown per operation + +| Operation | ops/sec | mean (us) | p50 (us) | p95 (us) | p99 (us) | rounds (ops/sec) | +|---|--:|--:|--:|--:|--:|--:| +| `user/byId` | 1629 | 613.7 | 603 | 671 | 779 | 1448–1699 | +| `user/byEmail` | 1669 | 599.0 | 596 | 639 | 687 | 1663–1680 | +| `user/byState (join, limit 20)` | 1069 | 935.8 | 932 | 998 | 1067 | 1046–1071 | +| `user/list (page of 20)` | 1148 | 870.8 | 865 | 932 | 1015 | 1140–1151 | +| `user/count` | 6033 | 165.7 | 164 | 178 | 194 | 5989–6079 | +| `authenticate (login)` | 1668 | 599.6 | 596 | 637 | 709 | 1622–1703 | +| `user/register` | 927 | 1078.6 | 1052 | 1355 | 1389 | 914–960 | +| `user/update` | 875 | 1143.5 | 1141 | 1202 | 1265 | 853–891 | +| `user/remove` | 1187 | 842.5 | 837 | 889 | 947 | 1184–1206 | + +1.34.0, PR #205: the condition operators' fixes (1.32.1–1.34.0). On an +existing `postgres:latest` container in Docker Desktop, shared by the runs +of 1.32.1 and 1.33.0 just before it (see the progression's table). Lower +than 1.32.1 by 1-9%, as `count` (none of the changed code) is: the +container's drift. Alternating 1.32.1 and this code on one container +measured them within 1.5% (the progression's note). + +### 2026-11-08 — `sqlite` — Apple M5 + +- **bones_api:** 1.34.0 (`024551e`) +- **Machine:** Apple M5, 10 cores, 16 GB RAM +- **OS:** macos (Version 26.6.2 (Build 25G83)) +- **Dart:** 3.13.5 +- **DB:** sqlite (temporary file) +- **Users:** 1000; **duration:** 3s (warm-up 1s) × 5 rounds, median shown per operation + +| Operation | ops/sec | mean (us) | p50 (us) | p95 (us) | p99 (us) | rounds (ops/sec) | +|---|--:|--:|--:|--:|--:|--:| +| `user/byId` | 29556 | 33.8 | 32 | 38 | 76 | 29487–29660 | +| `user/byEmail` | 28649 | 34.9 | 34 | 39 | 76 | 28577–28727 | +| `user/byState (join, limit 20)` | 4382 | 228.2 | 221 | 279 | 301 | 4375–4398 | +| `user/list (page of 20)` | 4731 | 211.4 | 206 | 257 | 272 | 4719–4746 | +| `user/count` | 190389 | 5.3 | 5 | 6 | 7 | 189785–191802 | +| `authenticate (login)` | 24460 | 40.9 | 39 | 44 | 103 | 24359–24488 | +| `user/register` | 9520 | 105.0 | 86 | 108 | 1351 | 9481–9631 | +| `user/update` | 15050 | 66.4 | 63 | 71 | 89 | 15007–15082 | +| `user/remove` | 17013 | 58.8 | 48 | 56 | 121 | 15537–17421 | + +Same code as the `postgres` entry above. Within ±0.3% of 1.32.1 on the +reads, `count` and `update` (run just before it); 1.33.0's dip on +`byState`/`list` (-4%) and `count` (-10%) is gone. + +### 2026-11-08 — `memory` — Apple M5 + +- **bones_api:** 1.34.0 (`024551e`) +- **Machine:** Apple M5, 10 cores, 16 GB RAM +- **OS:** macos (Version 26.6.2 (Build 25G83)) +- **Dart:** 3.13.5 +- **DB:** memory (DBSQLMemoryAdapter) +- **Users:** 1000; **duration:** 3s (warm-up 1s) × 5 rounds, median shown per operation + +| Operation | ops/sec | mean (us) | p50 (us) | p95 (us) | p99 (us) | rounds (ops/sec) | +|---|--:|--:|--:|--:|--:|--:| +| `user/byId` | 36725 | 27.2 | 26 | 30 | 44 | 36621–37253 | +| `user/byEmail` | 1428 | 700.0 | 724 | 1277 | 1357 | 1406–1534 | +| `user/byState (join, limit 20)` | 1790 | 558.8 | 548 | 625 | 640 | 1753–1792 | +| `user/list (page of 20)` | 766 | 1305.5 | 1299 | 1371 | 1393 | 761–766 | +| `user/count` | 244367 | 4.1 | 4 | 4 | 5 | 243801–245117 | +| `authenticate (login)` | 1435 | 696.7 | 701 | 1284 | 1355 | 1416–1540 | +| `user/register` | 3553 | 281.4 | 279 | 408 | 440 | 3517–3709 | +| `user/update` | 5444 | 183.7 | 182 | 219 | 245 | 5421–5462 | +| `user/remove` | 2909 | 343.8 | 269 | 484 | 496 | 2417–4934 | + +Same code as the `postgres` entry above. Within ±2% of 1.32.1 (`remove` +varies between rounds: 2417–4934). `remove` and `count` are much lower +than on 2026-10-14 on all three versions (‡ in the progression). + +--- + ### 2026-10-14 — `postgres` — Mac17,3 (Apple M5) - **bones_api:** 1.22.0 (`a508d8f`) diff --git a/lib/src/bones_api_base.dart b/lib/src/bones_api_base.dart index c86907c0..92c6ec26 100644 --- a/lib/src/bones_api_base.dart +++ b/lib/src/bones_api_base.dart @@ -52,7 +52,7 @@ typedef APILogger = void Function( /// Bones API Library class. class BonesAPI { // ignore: constant_identifier_names - static const String VERSION = '1.33.0'; + static const String VERSION = '1.34.0'; static bool _boot = false; diff --git a/lib/src/bones_api_condition.dart b/lib/src/bones_api_condition.dart index 014767ee..b012697c 100644 --- a/lib/src/bones_api_condition.dart +++ b/lib/src/bones_api_condition.dart @@ -1546,7 +1546,7 @@ abstract class KeyCondition extends Condition { var type = handler.getFieldType(null, key.name, resolveFiledName: true); if (type == null) break; - if (type.isListEntityOrReference) { + if (isToManyType(type, handler)) { toMany = true; break; } @@ -1559,6 +1559,171 @@ abstract class KeyCondition extends Condition { return toMany; } + /// A cache of [isRelationshipKey]: the [EntityHandler] whose result is + /// cached in [_isRelationshipKeyCacheResult]. + /// + /// Why: as [hasToManyInKeyPath] (see [_hasToManyInKeyPathCacheHandler]), + /// it's called for every entity matched, and resolved by the entity + /// handler's field types: computed once per handler. + EntityHandler? _isRelationshipKeyCacheHandler; + + /// The cached [isRelationshipKey] of [_isRelationshipKeyCacheHandler] + /// (`null`: not computed yet). + bool? _isRelationshipKeyCacheResult; + + /// If the [keys] are a single relationship field of [entityHandler]'s + /// entity: a to-one (an entity, `User? owner`, or an [EntityReference]) or + /// a to-many (a `List` of entities or an [EntityReferenceList]). Not a + /// path through one (`owner.email`), nor a field of another type. + /// + /// A relationship is compared by the IDs of its related entities, and it + /// never has a `null` ID: so a `null` element of a `=~`/`!~` list + /// (`=~ [1, null]`, `=~ [null]`) is ignored (for another field, it matches + /// a `null`, as `IS NULL`). As any field, `=~ null` (or a `null` + /// parameter) is an empty set (`=~` matches none, `!~` all, as with `[]`), + /// and `==`/`!=` with `[]` are with `null` ("without related entities" / + /// "with some"). + /// + /// Cached per [entityHandler] (see [_isRelationshipKeyCacheHandler]). + bool isRelationshipKey(EntityHandler? entityHandler) { + if (identical(entityHandler, _isRelationshipKeyCacheHandler)) { + return _isRelationshipKeyCacheResult ?? false; + } + + var relationship = false; + var key = keys.length == 1 ? keys.first : null; + if (entityHandler != null && key is ConditionKeyField) { + var type = entityHandler.getFieldType( + null, + key.name, + resolveFiledName: true, + ); + if (type != null) { + relationship = isRelationshipType(type, entityHandler); + } + } + + _isRelationshipKeyCacheHandler = entityHandler; + _isRelationshipKeyCacheResult = relationship; + return relationship; + } + + /// The right side of a `=~`/`!~` ([value], a list) with its parameters + /// resolved (a parameter's list flattened), without `null`s: the IDs (or + /// entities) of a relationship (see [isRelationshipKey]) to match. Empty + /// for `[]`, `null`, `[null]` or a `null` parameter. + List _relationshipRightIDs( + Object? parameters, + List? positionalParameters, + Map? namedParameters, + ) { + var ids = []; + + void add(Object? v) { + if (v == null) return; + if (v is Iterable) { + for (var e in v) { + add(e); + } + } else if (v is EntityReference) { + // A reference as a parameter: its ID (`null` if empty). + add(v.id); + } else if (v is EntityReferenceList) { + add(v.ids); + } else { + ids.add(v); + } + } + + var values = value; + for (var v in values is Iterable ? values : [values]) { + add( + v is ConditionParameter + ? v.getValue( + parameters: parameters, + positionalParameters: positionalParameters, + namedParameters: namedParameters, + ) + : v, + ); + } + + return ids; + } + + /// If the right side of a `=~`/`!~` ([value], a list) is a single + /// parameter whose value is `null` (`x =~ ?` with `null`): an empty set, + /// as `x =~ null` and `x =~ []` (`=~` matches none, `!~` all, any field). + bool _isNullParameterSet( + Object? parameters, + List? positionalParameters, + Map? namedParameters, + ) { + var values = value; + if (values is! List || values.length != 1) return false; + var p = values.first; + return p is ConditionParameter && + p.getValue( + parameters: parameters, + positionalParameters: positionalParameters, + namedParameters: namedParameters, + ) == + null; + } + + /// The right side of a `==`/`!=` ([value]) as SQL compares it: + /// - `[]` (also a parameter's) is `null`: `== []` is `== null` + /// (`IS NULL`), any field. + /// - A list as a single value, `[x]` is `x` (`== [7]` is `== 7`), unless + /// the left side ([keyValue], the entity's) is itself a list field (a + /// `List`, a `Uint8List`; not a to-many relationship's values, + /// see [hasToManyInKeyPath]): then the lists are compared as a whole. + /// - An [EntityReference] (also a parameter's, or a list's element) is + /// its ID. + /// + /// Otherwise returns [value] unchanged (a [ConditionParameter] is then + /// resolved while compared, as before). + Object? _equalsRightValue( + Object? keyValue, + EntityHandler? entityHandler, + Object? parameters, + List? positionalParameters, + Map? namedParameters, + ) { + Object? v = value; + if (v is ConditionParameter) { + var paramValue = v.getValue( + parameters: parameters, + positionalParameters: positionalParameters, + namedParameters: namedParameters, + ); + if (paramValue is! Iterable && paramValue is! EntityReference) { + return value; + } + v = paramValue; + } + + if (v is EntityReference) return v.id; + + if (v is Iterable) { + var list = v.toList(); + if (list.isEmpty) return null; + + // A list field (not a to-many's values): compared as a whole, by + // [value] (a parameter is compared with [ConditionParameter.matches]). + var listField = + keyValue is Iterable && !hasToManyInKeyPath(entityHandler); + if (listField) return value; + + if (list.length == 1) { + var e = list.first; + return e is EntityReference ? e.id : e; + } + } + + return v; + } + /// Matches the left side of a comparison with [compare], one value at a /// time through a to-many relationship (as a SQL join of its rows). /// @@ -1596,6 +1761,27 @@ abstract class KeyCondition extends Condition { return compare(keyValue); } + /// If a left side's value ([v], one compared by [_matchesToManyOrValue]) is + /// `null`, for `== null` / `!= null` (`IS NULL` / `IS NOT NULL`). + /// + /// A list (a list field's value: a to-many's values are compared one at a + /// time) is a value, never `null`, also with a `null` element + /// (`['a', null]`): [equalsConditionValue] would compare its elements + /// ("any is `null`"). + bool _isNullLeftValue( + Object? v, + TypeInfo? keyType, + EntityHandler? keyEntityHandler, + ) { + if (v is Iterable) return false; + return equalsConditionValue( + v, + null, + keyType: keyType, + keyEntityHandler: keyEntityHandler, + ); + } + @override void resolve({ Condition? parent, @@ -1704,6 +1890,12 @@ abstract class KeyCondition extends Condition { } } }).toList(); + } else if (valueEntityHandler != null && + (value as Object).isPrimitiveValue) { + // A related entity not resolved (only its ID: one stored by another + // adapter, or a missing row): its ID field is the ID; any other + // field is unknown, a `null` (as a SQL outer join's `NULL` row). + keyValue = keyName == valueEntityHandler.idFieldName() ? value : null; } else { throw StateError( "Can't access key[$keyName] for type: ${(value as Object?).runtimeTypeNameUnsafe}", @@ -1781,7 +1973,29 @@ abstract class KeyCondition extends Condition { return parse(value)!; } - EntityHandler? _resolveValueEntityHandler( + /// If [type] is a to-many relationship's: an [EntityReferenceList], or a + /// `List` of an entity type (one with an [EntityHandler], resolved from + /// [entityHandler]). Not a `List` of enums or of other classes. + static bool isToManyType(TypeInfo type, EntityHandler? entityHandler) { + if (type.isEntityReferenceListType) return true; + if (!type.isListEntity) return false; + var elementType = type.arguments0; + return elementType != null && + _resolveValueEntityHandler(elementType, entityHandler) != null; + } + + /// If [type] is a relationship field's (see [isRelationshipKey]): a + /// to-one, an [EntityReference] or an entity type (one with an + /// [EntityHandler], resolved from [entityHandler]), or a to-many (see + /// [isToManyType]). + static bool isRelationshipType(TypeInfo type, EntityHandler? entityHandler) { + if (type.isEntityReferenceBaseType) return true; + if (type.isListEntity) return isToManyType(type, entityHandler); + if (type.isPrimitiveType || type.isIterable || type.isMap) return false; + return _resolveValueEntityHandler(type, entityHandler) != null; + } + + static EntityHandler? _resolveValueEntityHandler( TypeInfo? objType, EntityHandler? objEntityHandler, ) { @@ -1871,18 +2085,29 @@ class KeyConditionEQ extends KeyCondition { }) { var keyValue = getEntityKeyValue(o, entityHandler); + // `== []` is `== null`, `== [x]` is `== x` (see [_equalsRightValue]): + var right = _equalsRightValue( + keyValue, + entityHandler, + parameters, + positionalParameters, + namedParameters, + ); + // Through a to-many relationship, each related entity's value (or a // `null` without any): return _matchesToManyOrValue( keyValue, entityHandler, - (v) => equalsConditionValue( - v, - value, - parameters: parameters, - positionalParameters: positionalParameters, - namedParameters: namedParameters, - ), + (v) => right == null + ? _isNullLeftValue(v, null, null) + : equalsConditionValue( + v, + right, + parameters: parameters, + positionalParameters: positionalParameters, + namedParameters: namedParameters, + ), ); } @@ -1896,20 +2121,31 @@ class KeyConditionEQ extends KeyCondition { }) { var keyValue = getEntityMapKeyValue(o, entityHandler); + // `== []` is `== null`, `== [x]` is `== x` (see [_equalsRightValue]): + var right = _equalsRightValue( + keyValue?.value, + entityHandler, + parameters, + positionalParameters, + namedParameters, + ); + // Through a to-many relationship, each related entity's value (or a // `null` without any): return _matchesToManyOrValue( keyValue?.value, entityHandler, - (v) => equalsConditionValue( - v, - value, - parameters: parameters, - positionalParameters: positionalParameters, - namedParameters: namedParameters, - keyType: keyValue?.type, - keyEntityHandler: keyValue?.entityHandler, - ), + (v) => right == null + ? _isNullLeftValue(v, keyValue?.type, keyValue?.entityHandler) + : equalsConditionValue( + v, + right, + parameters: parameters, + positionalParameters: positionalParameters, + namedParameters: namedParameters, + keyType: keyValue?.type, + keyEntityHandler: keyValue?.entityHandler, + ), ); } @@ -1985,9 +2221,32 @@ class KeyConditionNotEQ extends KeyCondition { List? positionalParameters, Map? namedParameters, ) { + // `!= []` is `!= null`, `!= [x]` is `!= x` (see [_equalsRightValue]): + var right = _equalsRightValue( + keyValue, + entityHandler, + parameters, + positionalParameters, + namedParameters, + ); + + // `!= null` (`IS NOT NULL`): with a value; a relationship, with some + // related entity. + if (right == null) { + return _matchesToManyOrValue( + keyValue, + entityHandler, + (v) => !_isNullLeftValue( + v, + keyConditionValue?.type, + keyConditionValue?.entityHandler, + ), + ); + } + bool notEquals(Object? v) => !equalsConditionValue( v, - value, + right, parameters: parameters, positionalParameters: positionalParameters, namedParameters: namedParameters, @@ -1995,7 +2254,7 @@ class KeyConditionNotEQ extends KeyCondition { keyEntityHandler: keyConditionValue?.entityHandler, ); - if (keyValue is Iterable && value is! Iterable) { + if (keyValue is Iterable && right is! Iterable) { if (_isEmptyToMany(keyValue, entityHandler)) return notEquals(null); return keyValue.isEmpty || keyValue.any(notEquals); } @@ -2234,8 +2493,15 @@ class KeyConditionLessThanOrEqual extends KeyCondition { } abstract class KeyConditionINBase extends KeyCondition> { + /// The values of an `IN` list. + /// + /// `x =~ null` is an empty set of values, as `x =~ []`: `=~` matches none + /// and `!~` all (any field). An explicit `null` element (`x =~ [null]`, + /// `x =~ [3, null]`) is kept: for a field (not a relationship, see + /// [KeyCondition.isRelationshipKey]), it also matches a `null` (as + /// `IS NULL`). static List valuesAsList(dynamic value) { - if (value == null) return [null]; + if (value == null) return []; if (value is List) return value; @@ -2273,6 +2539,7 @@ abstract class KeyConditionINBase extends KeyCondition> { return _matchesIn( keyValue, _isEmptyToMany(keyValue, entityHandler), + isRelationshipKey(entityHandler), parameters, positionalParameters, namedParameters, @@ -2293,6 +2560,7 @@ abstract class KeyConditionINBase extends KeyCondition> { return _matchesIn( keyValue?.value, _isEmptyToMany(keyValue?.value, entityHandler), + isRelationshipKey(entityHandler), parameters, positionalParameters, namedParameters, @@ -2304,17 +2572,53 @@ abstract class KeyConditionINBase extends KeyCondition> { /// join of its rows), with [not] (`NOT IN`): any of them not in. Without /// any ([emptyToMany], see [_isEmptyToMany]), as a `NULL` row (in a list /// with a `null`; also `NOT IN` a list without one). + /// + /// With a [relationshipKey] (see [isRelationshipKey]), the related + /// entities' IDs are matched with a set of IDs, without `null`s (see + /// [_matchesRelationshipIn]). bool _matchesIn( Object? keyValue, bool emptyToMany, + bool relationshipKey, Object? parameters, List? positionalParameters, Map? namedParameters, EntityHandler? entityHandler, ) { + if (relationshipKey) { + return _matchesRelationshipIn( + keyValue, + emptyToMany, + parameters, + positionalParameters, + namedParameters, + entityHandler, + ); + } + + // `x =~ ?` with `null`: an empty set, as `x =~ null` and `x =~ []` + // (`=~` matches none, `!~` all). + if (_isNullParameterSet( + parameters, + positionalParameters, + namedParameters, + )) { + return not; + } + + // References on the right side (a path ending in a reference, + // `owner.userInfo =~ ?`): compared by their IDs, as SQL. + var rightValues = + _rightValuesWithReferencesAsIDs( + parameters, + positionalParameters, + namedParameters, + ) ?? + value; + bool inValues(Object? v) => inConditionValues( v, - value, + rightValues, parameters: parameters, positionalParameters: positionalParameters, namedParameters: namedParameters, @@ -2330,6 +2634,93 @@ abstract class KeyConditionINBase extends KeyCondition> { return inValues(keyValue) != not; } + /// The right side of a `=~`/`!~` ([value]) with its parameters resolved, + /// and its [EntityReference]s (also an [EntityReferenceList]'s) as their + /// IDs (an empty reference: a `null` element). Returns `null` if there's + /// no reference: [value] is compared as before (its parameters resolved + /// while compared). + List? _rightValuesWithReferencesAsIDs( + Object? parameters, + List? positionalParameters, + Map? namedParameters, + ) { + var hasReference = false; + var resolved = []; + + void add(Object? v) { + if (v is EntityReference) { + hasReference = true; + resolved.add(v.id); + } else if (v is EntityReferenceList) { + hasReference = true; + resolved.addAll(v.ids ?? const []); + } else if (v is Iterable) { + for (var e in v) { + add(e); + } + } else { + resolved.add(v); + } + } + + for (var v in value) { + add( + v is ConditionParameter + ? v.getValue( + parameters: parameters, + positionalParameters: positionalParameters, + namedParameters: namedParameters, + ) + : v, + ); + } + + return hasReference ? resolved : null; + } + + /// A relationship's `=~`/`!~` (see [isRelationshipKey]): the left side + /// ([keyValue]) is the related entities' IDs (a to-one's, or a to-many's + /// list); the right side is the set of IDs (or entities) to match + /// ([_relationshipRightIDs]: [value] with its parameters resolved, and + /// without `null`s). + /// + /// - An empty set (`[]`, `null`, `[null]`, a `null` parameter): `=~` + /// matches none and `!~` all (a relationship never has a `null` ID). + /// - Without related entities (a `null` to-one, an empty to-many): in no + /// set (`=~` doesn't match, `!~` does). + /// - A to-many: `=~` if any related ID is in the set, `!~` if any isn't + /// (as a SQL join of its rows). + bool _matchesRelationshipIn( + Object? keyValue, + bool emptyToMany, + Object? parameters, + List? positionalParameters, + Map? namedParameters, + EntityHandler? entityHandler, + ) { + var ids = _relationshipRightIDs( + parameters, + positionalParameters, + namedParameters, + ); + if (ids.isEmpty) return not; + + if (keyValue == null || + emptyToMany || + (keyValue is Iterable && keyValue.isEmpty)) { + return not; + } + + bool inIDs(Object? v) => + inConditionValues(v, ids, entityHandler: entityHandler); + + if (keyValue is Iterable) { + return not ? keyValue.any((v) => !inIDs(v)) : keyValue.any(inIDs); + } + + return inIDs(keyValue) != not; + } + String get encodeValue => Condition.encodeConditionValue(value); @override diff --git a/lib/src/bones_api_condition_encoder.dart b/lib/src/bones_api_condition_encoder.dart index 22d90a9f..89030052 100644 --- a/lib/src/bones_api_condition_encoder.dart +++ b/lib/src/bones_api_condition_encoder.dart @@ -1,4 +1,5 @@ import 'dart:math' as math; +import 'dart:typed_data'; import 'package:async_extension/async_extension.dart'; import 'package:collection/collection.dart'; @@ -8,6 +9,7 @@ import 'package:statistics/statistics.dart'; import 'bones_api_condition.dart'; import 'bones_api_entity.dart'; +import 'bones_api_entity_reference.dart'; import 'bones_api_entity_sql.dart'; import 'bones_api_logging.dart'; import 'bones_api_mixin.dart'; @@ -1757,10 +1759,19 @@ abstract class ConditionEncoder { Type valueType, { bool valueAsList = false, }) { + // A reference (as a parameter) is compared by its ID(s): + if (value is EntityReference) { + value = value.id; + } else if (value is EntityReferenceList) { + value = value.ids ?? const []; + } + if (valueAsList) { if (value == null) { - // A `null` as a list is `[null]` (`x =~ null` is `IS NULL`), not an - // empty list (`x =~ []` matches nothing). + // A `null` element of a list is `[null]` (`x =~ [null]` is `IS NULL`). + // `x =~ null` and a single `null` parameter don't get here: they're + // an empty set (see `ConditionSQLEncoder._isNullParameterSet` and + // `KeyConditionINBase.valuesAsList`). return [null]; } else if (value is Iterable) { var list = value @@ -1794,6 +1805,21 @@ abstract class ConditionEncoder { } FutureOr _resolveValueToTypeImpl(Object? value, Type valueType) { + // A list as a single value (`== [x]` is `== x`), unless the field is a + // list itself (a `List`, a `Uint8List`: compared as a whole): + // first, so its element is resolved as a value (an enum by its name, an + // entity by its ID), not the list as text. + if (value is Iterable && + value is! TypedData && + !TypeInfo.from(valueType).isIterable) { + var list = value.toList(); + if (list.isEmpty) return null; + if (list.length == 1) value = list.first; + } + + // A reference (an element of a parameter's list) is its ID: + if (value is EntityReference) value = value.id; + if (value == null) { return null; } @@ -1826,6 +1852,26 @@ abstract class ConditionEncoder { } } + var schemeProvider = this.schemeProvider; + + // An entity compared with an ID column (a `String`/`int` relationship + // column, e.g. a to-one without a foreign key): its ID, not the + // entity's text (parsed below as a `String`). + if (schemeProvider != null && + TypeParser.isPrimitiveType(valueType) && + !TypeParser.isPrimitiveValue(value) && + value is! Iterable && + value is! Map && + value is! Enum && + value is! DateTime && + value is! DynamicNumber) { + var entityID = schemeProvider.getEntityID( + value, + entityType: value.runtimeType, + ); + if (entityID != null) value = entityID; + } + var valueTypeInfo = TypeInfo.from(valueType); var valueParser = TypeParser.parserForTypeInfo(valueTypeInfo); @@ -1853,10 +1899,15 @@ abstract class ConditionEncoder { } } - var schemeProvider = this.schemeProvider; - if (schemeProvider != null) { if (!TypeParser.isPrimitiveType(valueType)) { + // (A list as a single entity, `[]` or `[x]`, was unwrapped at the + // top.) + + // An ID (a primitive value, e.g. a `String` ID) for an entity type: + // already the entity's ID. + if (TypeParser.isPrimitiveValue(value)) return value; + var entityID = schemeProvider.getEntityID(value, entityType: valueType); return entityID; } else { diff --git a/lib/src/bones_api_condition_sql.dart b/lib/src/bones_api_condition_sql.dart index a7b1709d..1bbc614a 100644 --- a/lib/src/bones_api_condition_sql.dart +++ b/lib/src/bones_api_condition_sql.dart @@ -1,9 +1,11 @@ import 'package:async_extension/async_extension.dart'; import 'package:logging/logging.dart' as logging; +import 'package:reflection_factory/reflection_factory.dart'; import 'package:statistics/statistics.dart'; import 'bones_api_condition.dart'; import 'bones_api_condition_encoder.dart'; +import 'bones_api_extension.dart'; final _log = logging.Logger('ConditionSQLEncoder'); @@ -449,25 +451,91 @@ class ConditionSQLEncoder extends ConditionEncoder { String operator, { bool valueAsList = false, }) { - var retKeySQL = keyToSQL(c, context); + // Only an `IN` list of a relationship ignores its `null`s: + var retRelationship = valueAsList ? _isRelationshipKey(c, context) : false; + + return retRelationship.resolveMapped((relationship) { + var retKeySQL = keyToSQL(c, context); + + return retKeySQL.resolveMapped((keySQL) { + var keyType = keySQL.key; + var tableKey = keySQL.value; + + return encodeConditionValuesWithOperator( + context, + keyType, + null, + tableKey, + operator, + valueAsList && _isNullParameterSet(c, context) ? const [] : c.value, + valueAsList, + negated: c.isNegated, + nullable: !_isMainTableIDKey(c, context), + condition: c, + nullsAsAbsent: relationship, + ); + }); + }); + } - return retKeySQL.resolveMapped((keySQL) { - var keyType = keySQL.key; - var tableKey = keySQL.value; + /// If [c]'s `IN` list is a single parameter whose value is `null` + /// (`x =~ ?` with `null`): an empty set, as `x =~ null` and `x =~ []` + /// (`IN` matches none, `NOT IN` all; any field). As memory's + /// `KeyConditionINBase` (`_isNullParameterSet`). + bool _isNullParameterSet(KeyCondition c, EncodingContext context) { + var values = c.value; + if (values is! List || values.length != 1) return false; + var p = values.first; + return p is ConditionParameter && + p.getValue( + parameters: context.parameters, + positionalParameters: context.positionalParameters, + namedParameters: context.namedParameters, + encodingParameters: context.encodingParameters, + ) == + null; + } - return encodeConditionValuesWithOperator( - context, - keyType, - null, - tableKey, - operator, - c.value, - valueAsList, - negated: c.isNegated, - nullable: !_isMainTableIDKey(c, context), - condition: c, - ); - }); + /// If [c]'s key is a single relationship field of the main entity: its `IN` + /// list is a set of related IDs, without `null`s (a relationship never has + /// a `null` ID; see [KeyCondition.isRelationshipKey]). + /// + /// Resolved as memory's [KeyCondition.isRelationshipKey], by the field's + /// declared type (not by the DB scheme: a to-one may have no foreign key, + /// e.g. a `String` ID): an [EntityReference] / [EntityReferenceList], or + /// an entity type (or a `List` of one) with a table. + FutureOr _isRelationshipKey(KeyCondition c, EncodingContext context) { + var keys = c.keys; + if (keys.length != 1) return false; + + var key = keys.first; + if (key is! ConditionKeyField) return false; + + var schemeProvider = this.schemeProvider; + if (schemeProvider == null || _isMainTableIDKey(c, context)) return false; + + return schemeProvider + .getFieldType( + key.name, + entityName: context.entityName, + tableName: context.tableName, + ) + .resolveMapped((type) { + if (type == null) return false; + if (type.isEntityReferenceBaseType) return true; + + TypeInfo? entityType; + if (type.isListEntity) { + entityType = type.arguments0; + } else if (!type.isPrimitiveType && !type.isIterable && !type.isMap) { + entityType = type; + } + if (entityType == null) return false; + + return schemeProvider + .getTableForType(entityType) + .resolveMapped((table) => table != null); + }); } /// If [c]'s key is the ID of the main table (never `NULL`). @@ -505,6 +573,7 @@ class ConditionSQLEncoder extends ConditionEncoder { bool negated = false, bool nullable = true, Condition? condition, + bool nullsAsAbsent = false, }) { // The encoding value rather than only its text: a null supplied as a // parameter encodes to a placeholder, so the text alone cannot say whether @@ -529,7 +598,8 @@ class ConditionSQLEncoder extends ConditionEncoder { // An empty list (`x =~ []`, or a parameter's `[]`): `IN` matches // nothing and `NOT IN` everything, also a `NULL` (as in memory). It // is not a `null` (`x =~ null` and `[null]` are `IS NULL`, and arrive - // as a list with a `null`). + // as a list with a `null`), except for a relationship + // ([nullsAsAbsent], below). if (elements.isEmpty) { context.write(operator == 'IN' ? ' FALSE ' : ' TRUE '); return context; @@ -546,7 +616,20 @@ class ConditionSQLEncoder extends ConditionEncoder { } } - if (nonNulls.isEmpty) { + if (nullsAsAbsent) { + // A relationship's set of IDs ([_isRelationshipKey]): its `null`s + // aren't IDs. Without any other, an empty set (as `[]`). + if (nonNulls.isEmpty) { + context.write(operator == 'IN' ? ' FALSE ' : ' TRUE '); + return context; + } + value = EncodingValueList( + value.key, + value.type, + nonNulls, + encodeEncodingValueList, + ); + } else if (nonNulls.isEmpty) { value = EncodingValueNull( value.key, value.type, diff --git a/lib/src/bones_api_entity_db_memory.dart b/lib/src/bones_api_entity_db_memory.dart index 5fe3be90..22f9cd06 100644 --- a/lib/src/bones_api_entity_db_memory.dart +++ b/lib/src/bones_api_entity_db_memory.dart @@ -1331,13 +1331,17 @@ class DBSQLMemoryAdapter extends DBSQLAdapter var fieldObj = _getByID(refField.targetTable, id); - if (fieldObj != null) { - var tableScheme2 = getTableScheme(refField.targetTable); + // Not in this adapter (e.g. an entity stored by another adapter, as a + // `Photo` in an object storage), or a missing row: keep the ID, as a SQL + // column does. Replacing it with `null` lost the reference: `photo == + // null` matched it, and `photo =~ [id]` didn't. + if (fieldObj == null) return id; - if (tableScheme2 != null && tableScheme2.hasTableReferences) { - var entityHandler2 = getEntityHandler(tableName: tableScheme2.name); - fieldObj = _resolveEntityMap(fieldObj, entityHandler2, tableScheme2); - } + var tableScheme2 = getTableScheme(refField.targetTable); + + if (tableScheme2 != null && tableScheme2.hasTableReferences) { + var entityHandler2 = getEntityHandler(tableName: tableScheme2.name); + fieldObj = _resolveEntityMap(fieldObj, entityHandler2, tableScheme2); } return fieldObj; diff --git a/lib/src/bones_api_entity_db_sql.dart b/lib/src/bones_api_entity_db_sql.dart index 934c115c..d2f54cf7 100644 --- a/lib/src/bones_api_entity_db_sql.dart +++ b/lib/src/bones_api_entity_db_sql.dart @@ -3263,7 +3263,16 @@ abstract class DBSQLAdapter extends DBRelationalAdapter // A `limit` of `0` is no `LIMIT` clause here: it doesn't need an order. var effectiveLimit = limit != null && limit > 0 ? limit : null; - if (OrderDirection.resolveOrderByID(orderByID, offset, effectiveLimit)) { + // A query through a to-many relationship selects the matching IDs in a + // subquery (see [_joinsSubqueryConditionSQL]): without an explicit order + // (`orderByID: null`), ordered by ID, as its plain `JOIN` (and memory) + // returned them. An `IN ( subquery )` doesn't keep the table order (e.g. + // PostgreSQL's hash semi-join). + var toManySubquery = + orderByID == null && context.relationshipTables.isNotEmpty; + + if (toManySubquery || + OrderDirection.resolveOrderByID(orderByID, offset, effectiveLimit)) { var tableFieldID = context.tableFieldID ?? 'id'; tail += dialect.orderBySQL( tableAlias, diff --git a/pubspec.yaml b/pubspec.yaml index 8bd06b7d..7c369e9e 100644 --- a/pubspec.yaml +++ b/pubspec.yaml @@ -1,6 +1,6 @@ name: bones_api description: Bones_API - A powerful API backend framework for Dart. It comes with a built-in HTTP Server, route handler, entity handler, SQL translator, and DB adapters. -version: 1.33.0 +version: 1.34.0 homepage: https://github.com/Colossus-Services/bones_api environment: diff --git a/test/bones_api_condition_not_test.dart b/test/bones_api_condition_not_test.dart index a9156a2b..434960bc 100644 --- a/test/bones_api_condition_not_test.dart +++ b/test/bones_api_condition_not_test.dart @@ -474,16 +474,29 @@ void main() { test('an empty `IN` list is not a `null` (SQL as memory)', () async { // Memory: `=~ []` matches nothing, `!~ []` everything (also a `null`); - // `=~ null` is "is null". + // `=~ null` (also a `null` parameter) is as `=~ []` (since 1.34.0); + // `=~ [null]` is "is null". for (var a in [null, 5]) { var empty = {'a': []}; expect(matches('a =~ ?', {'a': a}, parameters: empty), isFalse); expect(matches('a !~ ?', {'a': a}, parameters: empty), isTrue); expect(matches('!(a =~ ?)', {'a': a}, parameters: empty), isTrue); + + var aNull = {'a': null}; + expect(matches('a =~ ?', {'a': a}, parameters: aNull), isFalse); + expect(matches('a !~ ?', {'a': a}, parameters: aNull), isTrue); + expect(matches('a =~ null', {'a': a}), isFalse); + expect(matches('a !~ null', {'a': a}), isTrue); } - var aNull = {'a': null}; - expect(matches('a =~ ?', {'a': null}, parameters: aNull), isTrue); - expect(matches('a =~ ?', {'a': 5}, parameters: aNull), isFalse); + expect(matches('a =~ [null]', {'a': null}), isTrue); + expect(matches('a =~ [null]', {'a': 5}), isFalse); + expect(matches('a =~ [5, null]', {'a': null}), isTrue); + + // `== []` is `== null` (since 1.34.0, as SQL): + expect(matches('a == []', {'a': null}), isTrue); + expect(matches('a == []', {'a': 5}), isFalse); + expect(matches('a != []', {'a': null}), isFalse); + expect(matches('a != []', {'a': 5}), isTrue); // SQL (1.32.0 wrote `[]` as `IS NULL`/`IS NOT NULL`): Future sql(String q, Object? p) async => @@ -508,17 +521,19 @@ void main() { '( "ac"."email" = @email OR FALSE )', ); - // Not an `IN`: `==`/`!=` with an empty list are still compared to - // `null` (unchanged; in memory they match none/all). + // Not an `IN`: `==`/`!=` with an empty list are compared to `null` + // (as memory, since 1.34.0): expect(await sql('pass == []', null), '( "ac"."pass" IS NULL )'); expect(await sql('pass != []', null), '( "ac"."pass" IS NOT NULL )'); - // A `null` (as a list) is still `IS NULL`: - expect(await sql('pass =~ ?', {'pass': null}), '( "ac"."pass" IS NULL )'); - expect( - await sql('pass !~ ?', {'pass': null}), - '( "ac"."pass" IS NOT NULL )', - ); + // `=~ null` and a `null` parameter are as `=~ []` (since 1.34.0): + expect(await sql('pass =~ null', null), '( FALSE )'); + expect(await sql('pass !~ null', null), '( TRUE )'); + expect(await sql('pass =~ ?', {'pass': null}), '( FALSE )'); + expect(await sql('pass !~ ?', {'pass': null}), '( TRUE )'); + + // An explicit `null` element is still `IS NULL`: + expect(await sql('pass =~ [null]', null), '( "ac"."pass" IS NULL )'); expect( await sql('pass =~ ?', { 'pass': [null], @@ -638,6 +653,77 @@ void main() { expect(matches('tags != "a"', ['a']), isFalse); expect(matches('tags != "a"', ['a', 'b']), isTrue); }); + + test('`==` / `!=` a list: compared as a whole (not `[x]` as `x`)', () { + // A literal: + expect(matches('tags == ["a"]', ['a']), isTrue); + expect(matches('tags == ["a"]', ['a', 'b']), isFalse); + expect(matches('tags != ["a"]', ['a', 'b']), isTrue); + expect(matches('tags == ["a", "b"]', ['a', 'b']), isTrue); + + // A parameter: + expect( + matches( + 'tags == ?', + ['a', 'b'], + [ + ['a'], + ], + ), + isFalse, + ); + expect( + matches( + 'tags == ?', + ['a', 'b'], + [ + ['a', 'b'], + ], + ), + isTrue, + ); + expect( + matches( + 'tags != ?', + ['a', 'b'], + [ + ['a', 'b'], + ], + ), + isFalse, + ); + + // A binary field: + var data = Uint8List.fromList([7, 8, 9]); + expect( + matches('tags == ?', data, [ + Uint8List.fromList([7, 8, 9]), + ]), + isTrue, + ); + expect( + matches('tags == ?', data, [ + Uint8List.fromList([7]), + ]), + isFalse, + ); + expect( + matches('tags != ?', data, [ + Uint8List.fromList([7, 8, 9]), + ]), + isFalse, + ); + + // `[]` is still `null` (`IS NULL`): an empty list is not `null`. + expect(matches('tags == []', []), isFalse); + expect(matches('tags == []', null), isTrue); + + // A list with a `null` element is a value, not `null`: + expect(matches('tags == null', ['a', null]), isFalse); + expect(matches('tags != null', ['a', null]), isTrue); + expect(matches('tags != []', ['a', null]), isTrue); + expect(matches('tags == []', ['a', null]), isFalse); + }); }); } diff --git a/test/bones_api_cov_condition_test.dart b/test/bones_api_cov_condition_test.dart index daabb889..834f8ca0 100644 --- a/test/bones_api_cov_condition_test.dart +++ b/test/bones_api_cov_condition_test.dart @@ -710,7 +710,8 @@ void main() { }); test('KeyConditionINBase.valuesAsList', () { - expect(KeyConditionINBase.valuesAsList(null), equals([null])); + // `x =~ null` is an empty set, as `x =~ []` (since 1.34.0): + expect(KeyConditionINBase.valuesAsList(null), equals([])); expect(KeyConditionINBase.valuesAsList([1, 2]), equals([1, 2])); expect(KeyConditionINBase.valuesAsList({1, 2}), equals([1, 2])); expect(KeyConditionINBase.valuesAsList({'a': 1, 'b': 2}), equals([1, 2])); diff --git a/test/bones_api_entity_db_tests_base.dart b/test/bones_api_entity_db_tests_base.dart index be2ac9a3..c9c6d9f8 100644 --- a/test/bones_api_entity_db_tests_base.dart +++ b/test/bones_api_entity_db_tests_base.dart @@ -4944,10 +4944,10 @@ Future runAdapterTests( ), equals({'s2'}), ); - // ... and a `null` parameter is `IS NULL`: + // ... and a `null` parameter is as `[]` (since 1.34.0): expect( await stores('number =~ ?', parameters: {'number': null}), - equals({'s2'}), + isEmpty, ); // A to-one relationship (a subquery): a missing owner is included. @@ -5221,6 +5221,15 @@ Future runAdapterTests( await s('name !~ null', allStores); await s('name =~ []', {}); await s('name !~ []', allStores); + // `== []` is `== null`: + await s('name == []', {}); + await s('name != []', allStores); + // `== [v]` is `== v`: + await s('name == ["ox-apple"]', {'s1'}); + await s('name != ["ox-apple"]', {'s2', 's3', 's4', 's5'}); + await s('name =~ ["ox-apple"]', {'s1'}); + await s('name == ?', {}, [[]]); + await s('name != ?:n', allStores, {'n': []}); await s('name == ?', {'s3'}, ['ox-cherry']); await s('name != ?:n', {'s1', 's2', 's4', 's5'}, {'n': 'ox-cherry'}); await s( @@ -5265,14 +5274,26 @@ Future runAdapterTests( await s('number != null', {'s1', 's3', 's4', 's5'}); await s('number =~ [1, 3]', {'s1', 's3', 's5'}); await s('number !~ [1, 3]', {'s2', 's4'}); - await s('number =~ null', {'s2'}); - await s('number !~ null', {'s1', 's3', 's4', 's5'}); + // `=~ null` is as `=~ []` (an empty set); an explicit `null` element + // matches a `null`: + await s('number =~ null', {}); + await s('number !~ null', allStores); await s('number =~ [null]', {'s2'}); await s('number !~ [null]', {'s1', 's3', 's4', 's5'}); await s('number =~ [3, null]', {'s2', 's3'}); await s('number !~ [3, null]', {'s1', 's4', 's5'}); await s('number =~ []', {}); await s('number !~ []', allStores); + // `== []` is `== null`: + await s('number == []', {'s2'}); + await s('number != []', {'s1', 's3', 's4', 's5'}); + // `== [v]` is `== v`: + await s('number == [3]', {'s3'}); + await s('number != [3]', {'s1', 's2', 's4', 's5'}); + await s('number =~ [3]', {'s3'}); + await s('number !~ [3]', {'s1', 's2', 's4', 's5'}); + await s('number == ?', {'s2'}, [[]]); + await s('number != ?:n', {'s1', 's3', 's4', 's5'}, {'n': []}); await s('number =~ /^3/', {'s3', 's4'}); await s('number !~ /^3/', {'s1', 's2', 's5'}); await s(r'number =~ /0$/', {'s4'}); @@ -5298,8 +5319,26 @@ Future runAdapterTests( 'n': [1, 30], }, ); - await s('number =~ ?', {'s2'}, [null]); - await s('number !~ ?', {'s1', 's3', 's4', 's5'}, [null]); + // A `null` parameter is as `[]`; a `[null]` parameter matches a + // `null`: + await s('number =~ ?', {}, [null]); + await s('number !~ ?', allStores, [null]); + await s('number =~ ?:n', {}, {'n': null}); + await s('number !~ ?:n', allStores, {'n': null}); + await s( + 'number =~ ?', + {'s2'}, + [ + [null], + ], + ); + await s( + 'number !~ ?', + {'s1', 's3', 's4', 's5'}, + [ + [null], + ], + ); await s( 'number =~ ?', {'s2', 's3'}, @@ -5326,8 +5365,10 @@ Future runAdapterTests( await s('owner != ?', {'s2', 's3', 's4', 's5'}, [ia]); await s('owner == null', {'s2'}); await s('owner != null', {'s1', 's3', 's4', 's5'}); - await s('owner =~ null', {'s2'}); - await s('owner !~ null', {'s1', 's3', 's4', 's5'}); + // A relationship's `=~`/`!~` is with a set of IDs, never `null`: + // `null` is an empty set (see 'Relationship operators'). + await s('owner =~ null', {}); + await s('owner !~ null', allStores); await s( 'owner =~ ?', {'s1', 's4'}, @@ -5373,7 +5414,10 @@ Future runAdapterTests( // A missing owner (`s2`) is a `null` field: `== null` matches it (as // `!=` does): await nullSplit(selectStores, allStores, 'owner.email', {'s2'}); - await s('owner.email =~ null', {'s2'}); + await s('owner.email =~ null', {}); + await s('owner.email !~ null', allStores); + await s('owner.email =~ [null]', {'s2'}); + await s('owner.email == []', {'s2'}); await s( 'owner.email =~ ?', {'s1', 's2'}, @@ -5400,7 +5444,10 @@ Future runAdapterTests( 's3', 's5', }); - await s('owner.level =~ null', {'s2', 's3', 's5'}); + await s('owner.level =~ null', {}); + await s('owner.level =~ [null]', {'s2', 's3', 's5'}); + await s('owner.level == []', {'s2', 's3', 's5'}); + await s('owner.level =~ ?', {}, [null]); await s('owner.level =~ [5, null]', {'s2', 's3', 's4', 's5'}); await s('owner.level !~ [5, null]', {'s1'}); await s('owner.level == ?:l', {'s2', 's3', 's5'}, {'l': null}); @@ -5435,7 +5482,8 @@ Future runAdapterTests( ); await s('owner.address.city =~ /?/', {'s3', 's5'}, ['Beta\$']); await nullSplit(selectStores, allStores, 'owner.address.city', {'s2'}); - await s('owner.address.city =~ null', {'s2'}); + await s('owner.address.city =~ null', {}); + await s('owner.address.city =~ [null]', {'s2'}); // A to-many relationship through a to-one one (`owner.roles`): `s2` // has no owner, `s4`'s owner (`uC`) no roles. @@ -5461,7 +5509,8 @@ Future runAdapterTests( // Each entity's own `null`s: await nullSplit(selectStores, allStores, 'name', {}); await nullSplit(selectStores, allStores, 'number', {'s2'}); - await nullSplit(selectStores, allStores, 'owner', {'s2'}); + // (Not `owner`: a relationship's `=~ [null]` is an empty set of IDs, + // see above.) // A single value (not a list) with `=~`/`!~`: await s('name =~ "ox-apple"', {'s1'}); @@ -5551,7 +5600,10 @@ Future runAdapterTests( ], ); await nullSplit(selectUsers, allUsers, 'roles.type', {'c'}); - await u('roles.type =~ null', {'c'}); + await u('roles.type =~ null', {}); + await u('roles.type !~ null', allUsers); + await u('roles.type =~ [null]', {'c'}); + await u('roles.type == []', {'c'}); await u('roles.type =~ ["admin", null]', {'a', 'c'}); await u('roles.type !~ ["admin", null]', {'b'}); await u( @@ -5573,6 +5625,25 @@ Future runAdapterTests( ], ); await u('roles.type != ?:t', {'b', 'c'}, {'t': RoleType.admin}); + // An enum in a one-element list (`== [x]` is `== x`): + await u( + 'roles.type == ?', + {'a'}, + [ + [RoleType.admin], + ], + ); + await u( + 'roles.type != ?', + {'b', 'c'}, + [ + [RoleType.admin], + ], + ); + // A to-many path's `== []` and `=~ null`: + await u('roles.id == []', {'c'}); + await u('roles.id != []', {'a', 'b'}); + await u('roles.type =~ ?', {}, [null]); // A bool: await u('roles.enabled =~ [false]', {'b'}); await u('roles.enabled != true', {'b', 'c'}); @@ -5643,8 +5714,12 @@ Future runAdapterTests( // The user's own fields: await nullSplit(selectUsers, allUsers, 'level', {'b'}); await u('level == null', {'b'}); - await u('level =~ null', {'b'}); - await u('level !~ null', {'a', 'c'}); + await u('level =~ null', {}); + await u('level !~ null', allUsers); + await u('level =~ [null]', {'b'}); + await u('level !~ [null]', {'a', 'c'}); + await u('level == []', {'b'}); + await u('level != []', {'a', 'c'}); await u('level !~ [10]', {'b', 'c'}); await u('level =~ [5, null]', {'b', 'c'}); await u('address.city =~ /alpha/i', {'a', 'c'}); @@ -5760,6 +5835,533 @@ Future runAdapterTests( expect(failures, isEmpty, reason: failures.join('\n')); }); + test('Relationship operators: by ID, on every adapter', () async { + final userRepo = entityRepositoryProvider.userAPIRepository; + final storeRepo = entityRepositoryProvider.storeAPIRepository; + final addressRepo = entityRepositoryProvider.addressAPIRepository; + + var failures = []; + + // The expected results come from a model: each entity's set of + // related IDs. A relationship is compared by its related IDs, and + // never has a `null` ID: + // - `=~ S`: any related ID in `S` (without `null`s: `[]`, `null`, + // `[null]` are an empty set, which matches none). + // - `!~ S`: an empty `S` matches all; otherwise without related + // entities, or any related ID not in `S` (as a SQL join of its + // rows). + // - `== null` / `== []`: without related entities; `!=`: with some. + // - `== v`: any related ID is `v`; `!= v`: without related + // entities, or any related ID isn't `v`. + Future relationship({ + required String field, + required Map> model, + required Future> Function(String q, Object? params) + select, + required Object id1, + required Object id2, + required List entityParams, + List? referenceParams, + Future Function(String q, Object? params)? count, + Future Function(String q, Object? params, int limit, int offset)? + pageLength, + }) async { + var all = model.keys.toSet(); + + // An ID as a literal (a `String` ID quoted): + String lit(Object id) => id is String ? '"$id"' : '$id'; + + Set where(bool Function(Set ids) f) => { + for (var e in model.entries) + if (f(e.value)) e.key, + }; + + Set inSet(Iterable s) { + var ids = s.nonNulls.toSet(); + return where((r) => r.any(ids.contains)); + } + + Set notInSet(Iterable s) { + var ids = s.nonNulls.toSet(); + if (ids.isEmpty) return all; + return where((r) => r.isEmpty || r.any((v) => !ids.contains(v))); + } + + var none = where((r) => r.isEmpty); + var some = where((r) => r.isNotEmpty); + + Future check(String q, Object? params, Set exp) async { + var p = params == null ? '' : ' $params'; + for (var (query, expected) in [ + (q, exp), + ('!($q)', all.difference(exp)), + ]) { + var expList = expected.toList()..sort(); + try { + var got = await select(query, params); + if (got.join(',') != expList.join(',')) { + failures.add( + '`$query`$p: [${got.join(',')}], expected $expList', + ); + } + } catch (e) { + failures.add('`$query`$p: ERROR $e'); + } + } + } + + // The sets: literals, and parameters (positional and named). + var sets = <(String, List)>[ + ('[]', []), + ('null', [null]), + ('[null]', [null]), + (lit(id1), [id1]), + ('[${lit(id1)}]', [id1]), + ('[${lit(id1)}, ${lit(id2)}]', [id1, id2]), + ('[${lit(id2)}, null]', [id2, null]), + ]; + for (var (literal, s) in sets) { + await check('$field =~ $literal', null, inSet(s)); + await check('$field !~ $literal', null, notInSet(s)); + } + + var params = <(Object?, List)>[ + ([], []), + (null, [null]), + ([null], [null]), + (id1, [id1]), + ([id1], [id1]), + ([id1, id2], [id1, id2]), + ([id2, null], [id2, null]), + ]; + for (var (param, s) in params) { + await check('$field =~ ?', [param], inSet(s)); + await check('$field !~ ?', [param], notInSet(s)); + await check('$field =~ ?:ids', {'ids': param}, inSet(s)); + await check('$field !~ ?:ids', {'ids': param}, notInSet(s)); + } + + // Entities as parameters (compared by their IDs): + await check('$field =~ ?', [entityParams.first], inSet([id1])); + await check('$field =~ ?', [entityParams], inSet([id1, id2])); + await check('$field !~ ?', [entityParams], notInSet([id1, id2])); + + // References as parameters (compared by their IDs): + if (referenceParams != null) { + await check('$field =~ ?', [referenceParams.first], inSet([id1])); + await check('$field =~ ?', [referenceParams], inSet([id1, id2])); + await check('$field !~ ?', [referenceParams], notInSet([id1, id2])); + await check('$field == ?', [ + referenceParams.first, + ], where((r) => r.contains(id1))); + } + + // A `Set` parameter: + await check('$field =~ ?', [ + {id1}, + ], inSet([id1])); + await check('$field !~ ?', [ + {id1, null}, + ], notInSet([id1])); + + // `== [id]` (a single element) is `== id`; an entity as a `==` + // parameter is its ID: + await check('$field == [${lit(id1)}]', null, inSet([id1])); + await check('$field == ?', [ + [id1], + ], inSet([id1])); + await check('$field != ?', [ + [id1], + ], where((r) => r.isEmpty || r.any((v) => v != id1))); + await check('$field == ?', [entityParams.first], inSet([id1])); + await check('$field != ?', [ + entityParams.first, + ], where((r) => r.isEmpty || r.any((v) => v != id1))); + // An entity in a one-element list (`== [x]` is `== x`): + await check('$field == ?', [ + [entityParams.first], + ], inSet([id1])); + await check('$field != ?', [ + [entityParams.first], + ], where((r) => r.isEmpty || r.any((v) => v != id1))); + + // Counted and paginated once each (a to-many's join): + var inBoth = inSet([id1, id2]); + if (count != null) { + var n = await count('$field =~ ?', [ + [id1, id2], + ]); + if (n != inBoth.length) { + failures.add( + 'count `$field =~ [id1, id2]`: $n, expected ' + '${inBoth.length}', + ); + } + var page = await select('$field =~ ?', [ + [id1, id2], + ]); + if (page.length != inBoth.length) { + failures.add('select `$field =~ [id1, id2]`: $page'); + } + } + if (pageLength != null) { + // Each page has as many entities as its `limit` (none repeated): + for (var (limit, offset, exp) in [ + (1, 0, 1), + (1, 1, 1), + (2, 0, inBoth.length), + (2, 1, inBoth.length - 1), + ]) { + var n = await pageLength( + '$field =~ ?', + [ + [id1, id2], + ], + limit, + offset, + ); + if (n != exp) { + failures.add( + 'page `$field =~ [id1, id2]` limit $limit offset $offset: ' + '$n, expected $exp', + ); + } + } + } + + // With another condition (`&&`, `||`): + await check('$field =~ ? && $field !~ ?', [ + [id1, id2], + [id1], + ], inSet([id1, id2]).intersection(notInSet([id1]))); + await check('$field =~ ?:a || $field == null', { + 'a': [id2], + }, inSet([id2]).union(none)); + await check('$field =~ ?:a || $field =~ ?:b', { + 'a': null, + 'b': [], + }, {}); + + // `==` / `!=`: `null` and `[]` are "without related entities": + for (var literal in ['null', '[]']) { + await check('$field == $literal', null, none); + await check('$field != $literal', null, some); + } + for (var param in [null, []]) { + await check('$field == ?', [param], none); + await check('$field != ?', [param], some); + await check('$field == ?:v', {'v': param}, none); + await check('$field != ?:v', {'v': param}, some); + } + for (var id in [id1, id2]) { + await check( + '$field == ${lit(id)}', + null, + where((r) => r.contains(id)), + ); + await check( + '$field != ${lit(id)}', + null, + where((r) => r.isEmpty || r.any((v) => v != id)), + ); + await check('$field == ?', [id], where((r) => r.contains(id))); + } + } + + // A to-many `EntityReferenceList` (`Address.closedStores`) and a + // to-many `List` (`Address.stores`): + var sc1 = Store('rk-c1', 1); + var sc2 = Store('rk-c2', 2); + var sc3 = Store('rk-c3', 3); + for (var s in [sc1, sc2, sc3]) { + await storeRepo.store(s); + } + + var addresses = { + 'A': Address( + 'RK', + 'RK A', + 'RK Street', + 1, + stores: [Store('rk-a1', 1)], + closedStores: [sc1], + ), + 'B': Address( + 'RK', + 'RK B', + 'RK Street', + 2, + stores: [Store('rk-b1', 2), Store('rk-b2', 3)], + closedStores: [sc1, sc2], + ), + 'C': Address('RK', 'RK C', 'RK Street', 3), + }; + var addressNames = {}; + for (var e in addresses.entries) { + addressNames[(await addressRepo.store(e.value))!] = e.key; + } + + Future> selectAddresses(String q, Object? params) async => + (await addressRepo.selectByQuery( + q, + parameters: params, + )).map((e) => addressNames[e.id]).nonNulls.toList()..sort(); + + // (The IDs of this test's stores: only its addresses are counted.) + Future countAddresses(String q, Object? params) async => + await addressRepo.countByQuery(q, parameters: params); + + Future pageAddresses( + String q, + Object? params, + int limit, + int offset, + ) async => (await addressRepo.selectByQuery( + q, + parameters: params, + limit: limit, + offset: offset, + orderByID: true, + )).length; + + Set ids(Iterable stores) => + stores.map((s) => s.id!).toSet(); + + await relationship( + field: 'closedStores', + model: { + 'A': {sc1.id!}, + 'B': {sc1.id!, sc2.id!}, + 'C': {}, + }, + select: selectAddresses, + id1: sc1.id!, + id2: sc2.id!, + entityParams: [sc1, sc2], + count: countAddresses, + pageLength: pageAddresses, + ); + + // A to-many relationship's query, without an explicit order: in ID + // order (as its `JOIN` was, before its IDs subquery). + { + var ids = (await addressRepo.selectByQuery( + 'closedStores =~ ?', + parameters: [ + [sc1.id, sc2.id], + ], + )).map((e) => e.id as int).toList(); + var sorted = [...ids]..sort(); + if (ids.join(',') != sorted.join(',')) { + failures.add('`closedStores =~ ?` order: $ids, expected $sorted'); + } + } + + // A to-many path (`stores.number`: `C` has no stores, a `null` row): + // `=~ null` is an empty set, `=~ [null]` matches a `null`. + for (var (q, exp) in [ + ('stores.number =~ null', []), + ('stores.number !~ null', ['A', 'B', 'C']), + ('stores.number =~ [null]', ['C']), + ('stores.number == []', ['C']), + ('stores.number == 1', ['A']), + ]) { + var got = await selectAddresses(q, null); + if (got.join(',') != exp.join(',')) { + failures.add('`$q`: $got, expected $exp'); + } + } + + // An `EntityReferenceList` as a parameter (its IDs): + var bClosed = addresses['B']!.closedStores; + for (var (q, exp) in [ + ('closedStores =~ ?', ['A', 'B']), + ('closedStores !~ ?', ['C']), + ]) { + var got = await selectAddresses(q, [bClosed]); + if (got.join(',') != exp.join(',')) { + failures.add('`$q` [$bClosed]: $got, expected $exp'); + } + } + + var aStores = addresses['A']!.stores; + var bStores = addresses['B']!.stores; + await relationship( + field: 'stores', + model: {'A': ids(aStores), 'B': ids(bStores), 'C': {}}, + select: selectAddresses, + id1: aStores.first.id!, + id2: bStores.first.id!, + entityParams: [aStores.first, bStores.first], + count: countAddresses, + pageLength: pageAddresses, + ); + + // A to-one entity (`Store.owner`) and a to-one `EntityReference` + // (`User.userInfo`): + var alpha = Address('RK', 'RK User', 'RK Street', 9); + var users = { + 'X': User( + 'rk-x@$testDomain', + 'p', + alpha, + [], + userInfo: UserInfo('rk-x'), + photo: Uint8List.fromList([82, 75, 1]), + creationTime: DateTime.utc(2020), + ), + 'Y': User( + 'rk-y@$testDomain', + 'p', + alpha, + [], + userInfo: UserInfo('rk-y'), + photo: Uint8List.fromList([82, 75, 2]), + creationTime: DateTime.utc(2020), + ), + 'Z': User( + 'rk-z@$testDomain', + 'p', + alpha, + [], + creationTime: DateTime.utc(2020), + ), + }; + var userNames = {}; + for (var e in users.entries) { + userNames[(await userRepo.store(e.value))!] = e.key; + } + + Future> selectUsers(String q, Object? params) async => + (await userRepo.selectByQuery( + q, + parameters: params, + )).map((e) => userNames[e.id]).nonNulls.toList()..sort(); + + var (ux, uy) = (users['X']!, users['Y']!); + var infoX = ux.userInfo.id!; + var infoY = uy.userInfo.id!; + await relationship( + field: 'userInfo', + model: { + 'X': {infoX}, + 'Y': {infoY}, + 'Z': {}, + }, + select: selectUsers, + id1: infoX, + id2: infoY, + entityParams: [ux.userInfo.entity!, uy.userInfo.entity!], + referenceParams: [ux.userInfo, uy.userInfo], + ); + + // A to-one with a `String` ID, without a foreign key (`User.photo`): + var (photoX, photoY) = (ux.photo!, uy.photo!); + await relationship( + field: 'photo', + model: { + 'X': {photoX.id}, + 'Y': {photoY.id}, + 'Z': {}, + }, + select: selectUsers, + id1: photoX.id, + id2: photoY.id, + entityParams: [photoX, photoY], + ); + + var stores = { + 'X': Store('rk-sx', 1, owner: ux), + 'Y': Store('rk-sy', 2, owner: uy), + 'Z': Store('rk-sz', 3), + }; + var storeNames = {}; + for (var e in stores.entries) { + storeNames[(await storeRepo.store(e.value))!] = e.key; + } + + Future> selectStores(String q, Object? params) async => + (await storeRepo.selectByQuery( + q, + parameters: params, + )).map((e) => storeNames[e.id]).nonNulls.toList()..sort(); + + await relationship( + field: 'owner', + model: { + 'X': {ux.id!}, + 'Y': {uy.id!}, + 'Z': {}, + }, + select: selectStores, + id1: ux.id!, + id2: uy.id!, + entityParams: [ux, uy], + ); + + // A path ending in a reference (`owner.userInfo`, not a + // relationship key): references as parameters are compared by + // their IDs too. + for (var (q, params, exp) in <(String, Object?, List)>[ + ( + 'owner.userInfo =~ ?', + [ + [ux.userInfo], + ], + ['X'], + ), + ( + 'owner.userInfo =~ ?', + [ + [ux.userInfo, uy.userInfo], + ], + ['X', 'Y'], + ), + ('owner.userInfo == ?', [ux.userInfo], ['X']), + ('owner.userInfo != ?', [ux.userInfo], ['Y', 'Z']), + ( + 'owner.userInfo !~ ?', + [ + [ux.userInfo], + ], + ['Y', 'Z'], + ), + // An `EntityReferenceList` as the parameter (its IDs): + ( + 'owner.userInfo =~ ?', + [ + EntityReferenceList.fromIDs([infoX, infoY]), + ], + ['X', 'Y'], + ), + ]) { + var got = await selectStores(q, params); + if (got.join(',') != exp.join(',')) { + failures.add('`$q` $params: $got, expected $exp'); + } + } + + // A path through a to-one whose row isn't resolved (only its ID: a + // missing row, or one stored by another adapter): its ID field is + // the ID, any other field a `null` (as a SQL outer join's row). + var storeHandler = entityRepositoryProvider.storeEntityHandler; + var unresolvedOwner = {'id': 1, 'name': 'rk-u', 'owner': 987654}; + for (var (q, exp) in [ + ('owner.id == 987654', true), + ('owner.email == null', true), + ('owner.email != null', false), + ('owner.email == "rk-x@$testDomain"', false), + ]) { + var got = Condition.parse(q) + .matchesEntityMap(unresolvedOwner, entityHandler: storeHandler); + if (got != exp) { + failures.add('unresolved owner `$q`: $got, expected $exp'); + } + } + + expect(failures, isEmpty, reason: failures.join('\n')); + }); + test('populate', () async { var result = await entityRepositoryProvider.storeAllFromJson({ 'user': [