Skip to content

Commit fb2db78

Browse files
authored
fix(cli): finish meta deps sync and check on ports' own metadata (#497)
* feat(deps): check and publish corpora for cross-repo model sharing; sync finishes on port-native layouts (FR-023) Add an expectCheck arm to fixtures/dependency-conformance (current, drifted, unresolved) run in TypeScript, and the fixtures/shared-model-conformance publish corpus (exact artifact and manifest bytes plus the refusals), run in TypeScript with a serializer arm in Python. meta deps sync and check now prove the locked dependencies load together when the project's own metadata is in a build-tool directory or uses another port's vocabulary, instead of failing after writing. Record the Maven and NuGet packaged transports as a design note. Fix the stale CONFORMANCE.md cell. * fix(cli): keep the dependency-only load proof narrow and its failures labelled Fall back only for ERR_UNKNOWN_TYPE/SUBTYPE and only in a project with a config; label resolve failures in deps check; cite the tracking issue, not a plan. * refactor(cli): type the dependency-load proof as a union and reuse the verified snapshot * no-mistakes(review): apply reviewer inline fixes * no-mistakes(review): Fall back only when every load error is own port-only vocabulary * no-mistakes(document): Updated fallback-trigger prose in metadata-dependencies.md; other docs already current. * chore(site): regenerate site payload for 32 corpora
1 parent f2736e6 commit fb2db78

32 files changed

Lines changed: 1970 additions & 98 deletions

‎AGENTS.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -71,7 +71,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken.
7171
- **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres.
7272

7373
**Cross-port conformance corpora** (every port runs the shared corpus):
74-
- Metamodel: `fixtures/conformance/` (413 fixtures; 31 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
74+
- Metamodel: `fixtures/conformance/` (413 fixtures; 32 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green.
7575
- Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical.
7676
- Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015; the FR-044 `report-*` scenarios also run on SQLite, MySQL and D1 where a port has the engine, see [docs/CONFORMANCE.md](docs/CONFORMANCE.md)). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".)
7777
- API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. Three sub-corpora run the **generated lane only, on all five ports** — `write-through/`, `projection/` (F22: a view-only `object.projection` serves GET list + GET by id and answers every write verb with `405 {"error": "method_not_allowed"}`) and `report/` (FR-044: a view-backed `object.report` serves GET list, answers `POST` with that same `405`, and mounts no `/{id}` route). That is deliberate, not a gap: what is under test is whether a port's GENERATOR emits those routes, and a hand-rolled reference server would answer every scenario by construction. The `m2m/` sub-corpus also gates **TPH x M:N together** (base-declared, subtype-declared, abstract-mid-declared, a non-subtype source onto a subtype TARGET, and the cross-subtype source id answering `200 []`) — the two corpora were originally built disjoint (`tph/` had no relationships, `m2m/` no discriminators), which is precisely how that defect class survived.

‎CHANGELOG.md‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,17 @@ here.**
2424
port yet, so no command's behavior changes. A pattern with an empty segment, a trailing `::` or an
2525
odd run of `:` is refused with `ERR_SCOPE_PATTERN_INVALID`.
2626

27+
- **Two cross-port corpora for sharing a model across repositories (FR-023).**
28+
`fixtures/dependency-conformance/` gains a check arm (`expectCheck`, 11 cases): for each declared
29+
dependency, `current`, `drifted`, or `unresolved` (a missing path, a bad manifest, a hash
30+
mismatch, or not yet in the lock), run in TypeScript. The new `fixtures/shared-model-conformance/`
31+
(11 cases) pins what a publisher writes: the exact artifact and manifest bytes for a source tree and
32+
`include` / `exclude` / `files`, and the refusals (an empty selection, a closure failure naming every
33+
`referrer → target` pair, vocabulary a consumer's toolchain does not ship). TypeScript runs all of
34+
it and Python runs the serializer arm. These are the reference the Java, Kotlin and C# slices of
35+
Phase 2 ([#482](https://github.com/metaobjectsdev/metaobjects/issues/482)) are written against.
36+
The feature doc records the Maven and NuGet packaged transports as a design for a later slice.
37+
2738
- **C# reads its generated-code directory from `.metaobjects/config.json`, so a plain
2839
`dotnet meta verify` runs the codegen gate.** Set `"csharp": { "outDir": "Generated" }`
2940
(relative to the directory holding `.metaobjects/`). `dotnet meta gen` writes there and
@@ -78,6 +89,15 @@ here.**
7889

7990
### Fixed
8091

92+
- **`meta deps sync` and `meta deps check` finish cleanly on a project whose own metadata the
93+
Node toolchain cannot read.** A Maven project (`src/main/resources/metadata`), a C# project, or a
94+
model using vocabulary only another port registers declares no neutral `sources`, so the closing
95+
"does the dependency set load?" proof failed, and `sync` exited 1 for a layout nothing was wrong
96+
with. In those two cases only, the proof now loads the locked snapshots together, prints that it
97+
did, and says that the port's own `verify` is the full check. Any other load failure still fails
98+
the command and rolls the sync back. A stale `docs/CONFORMANCE.md` cell that claimed Python passed
99+
only 2 of the dependency corpus's cases is corrected.
100+
81101
- **A failed `meta deps sync` no longer leaves a lock and snapshot that cannot load.** The
82102
command wrote `deps.lock.json` and the `.metaobjects/deps/` snapshot before loading the
83103
combined model, and on a load failure it exited 1 but kept the new files. Because the lock

‎docs/CONFORMANCE.md‎

Lines changed: 42 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# Conformance coverage
22

3-
The MetaObjects standard ships **31 shared conformance corpora** under
3+
The MetaObjects standard ships **32 shared conformance corpora** under
44
[`fixtures/`](../fixtures/). Every port runs every corpus that is *applicable to
55
it* and asserts the same expected behaviour against the same fixtures. **This page
66
is the inverse index**: fixture → feature doc + per-port pass status, and it is the
@@ -46,7 +46,8 @@ regenerate with `ls -d fixtures/<corpus>/*/ | wc -l` for directory-shaped corpor
4646
| [`fixtures/provider-composition-conformance/`](../fixtures/provider-composition-conformance/) | 9 (5 error-shape + 4 compose-load) | ✓ | ✓ | — (JVM registry via Java) | ✓ | ✓ |
4747
| [`fixtures/source-resolution-conformance/`](../fixtures/source-resolution-conformance/) | 25 cases | ✓ (reference implementation) | ✓ | inherits via Java | ✓ | ✓ |
4848
| [`fixtures/scope-conformance/`](../fixtures/scope-conformance/) | 10 cases | ✓ (reference implementation) | ✓ | inherits via Java | ✓ | ✓ |
49-
| [`fixtures/dependency-conformance/`](../fixtures/dependency-conformance/) | 23 cases | ✓ (reference implementation) | — (Phase 2) | — (Phase 2) | — (Phase 2) | ✓ (2 of 23 assert a documented Python-only vocabulary gap instead of the corpus's full contract — see below) |
49+
| [`fixtures/dependency-conformance/`](../fixtures/dependency-conformance/) | 35 cases (24 resolution + 11 check) | ✓ (reference implementation; both arms) | — (Phase 2) | — (Phase 2) | — (Phase 2) | ✓ (resolution arm, all 24 identically; no check yet, so the 11 check cases are not run) |
50+
| [`fixtures/shared-model-conformance/`](../fixtures/shared-model-conformance/) | 11 cases (7 publish + 4 refusal) | ✓ (reference implementation) | — (Phase 2) | — (Phase 2) | — (Phase 2) | ✓ (serializer arm: the 7 publish cases' document bytes; no publisher) |
5051
| [`fixtures/agent-context-conformance/`](../fixtures/agent-context-conformance/) | 4 | ✓ (the emitter is TS-owned) | — | — | — | — |
5152
| [`fixtures/metamodel-docs/`](../fixtures/metamodel-docs/) | 1 | ✓ (docs emit is TS-owned) | — | — | — | — |
5253
| [`fixtures/fmt-conformance/`](../fixtures/fmt-conformance/) (#304 — `meta fmt`) | 12 | ✓ (reference) | ✓ | inherits via Java | ✓ | ✓ |
@@ -73,6 +74,7 @@ the corpora above do two different jobs. Only the first is a promise to adopters
7374
- **Core — the contract.** `conformance/`, `yaml-conformance/`, `registry-conformance/`,
7475
`validation-conformance/`, `provider-composition-conformance/`,
7576
`source-resolution-conformance/`, `scope-conformance/`, `dependency-conformance/`,
77+
`shared-model-conformance/` (the artifact and manifest bytes a consumer verifies),
7678
`object-model-conformance/`, `render-conformance/`, `template-codegen-conformance/` (the
7779
Mustache engine), `template-output-render-conformance/`, `output-prompt-conformance/`,
7880
`extract-conformance/`, `verify-conformance/`, `verify-strict-conformance/`,
@@ -438,23 +440,47 @@ also matches before a trailing newline. Kotlin has no runner of its own: it inhe
438440
the Java loader. Each of those two guards is pinned by a test beside the corpus, since
439441
the corpus asserts matches only and carries no invalid-pattern case.
440442

441-
### `fixtures/dependency-conformance/` (23 cases)
443+
### `fixtures/dependency-conformance/` (35 cases)
442444

443-
All 23 cases → [features/metadata-dependencies.md](features/metadata-dependencies.md)
445+
All 35 cases → [features/metadata-dependencies.md](features/metadata-dependencies.md)
444446
(declaring a dependency, `meta deps sync`, the committed snapshot + lock, default
445-
exclusion of imported metadata, overlay/extends across the boundary, and every load-
446-
and resolution-time failure). File-shaped like `scope-conformance/` above: one
447-
committed `cases.json`, no per-port fixture, no ledger.
448-
449-
**TypeScript (the reference implementation) and Python both run all 23 cases,
450-
identically** — every case's `expectFiles`/`expectImported`/`expectSelected`/
451-
`expectMigrateGoverned`/`expectLoadError` assertions apply to both runners with no
452-
exemption. The two overlay cases that need a view child as incidental content use
453-
`view.currency` (the one concrete `view.*` subtype registered cross-port), not
454-
`view.text` — a prior revision used `view.text`, which Python did not register then, and
455-
carried a since-discharged allowlist for it. Java, Kotlin and C# have no runner —
447+
exclusion of imported metadata, overlay/extends across the boundary, every load-
448+
and resolution-time failure, and the upstream check). File-shaped like `scope-conformance/`
449+
above: one committed `cases.json`, no per-port fixture, no ledger. The corpus has two arms:
450+
451+
- **Resolution (24 cases).** Every case's `expectFiles`/`expectImported`/`expectSelected`/
452+
`expectMigrateGoverned`/`expectLoadError` assertions. **TypeScript (the reference
453+
implementation) and Python both run all 24, identically**, with no exemption. The two
454+
overlay cases that need a view child as incidental content use `view.currency` (the one
455+
concrete `view.*` subtype registered cross-port), not `view.text` — a prior revision used
456+
`view.text`, which Python did not register then, and carried a since-discharged allowlist
457+
for it.
458+
- **Check (11 cases, `expectCheck`).** For each declared dependency, `current`, `drifted` or
459+
`unresolved` (a missing path, a bad manifest, a hash mismatch, or not yet in the lock). Run
460+
by TypeScript only, in `cli/test/dependency-check-conformance.test.ts`; Python has no
461+
`deps check` yet and filters these cases out. The arm exists so that each port's check is
462+
written against the same edge cases (Phase 2, #482).
463+
464+
Java, Kotlin and C# have no runner —
456465
Phase 1a is TypeScript + Python only; those three ports arrive in Phase 2.
457466

467+
### `fixtures/shared-model-conformance/` (11 cases)
468+
469+
→ [features/metadata-dependencies.md](features/metadata-dependencies.md) (the publisher's
470+
`sharedModelFile()`). A source tree under `source/`, plus `files` / `include` / `exclude` /
471+
`version`, must produce **exactly** the pinned artifact and manifest bytes (seven cases), or be
472+
refused for the pinned reason (four: an empty selection, a reference or `extends` outside the
473+
selection with the `referrer → target` pairs named, and vocabulary only a consumer provider
474+
registers). `requirement.*` and `template.*` nodes are always dropped, and the order of `files`
475+
does not change the bytes. The manifest goldens carry the token `@@METAMODEL_VERSION@@` where
476+
the manifest records the toolchain's `metamodelVersion`, so they do not move with it.
477+
478+
**TypeScript runs all 11**, in `codegen-ts/test/shared-model-conformance.test.ts`. **Python runs
479+
the serializer arm**: for each publish case it loads `files`, takes the nodes the pinned manifest
480+
lists, and `serialize_shared_document` must reproduce the artifact byte for byte (and the sha256
481+
the manifest records). Python has no publisher, so selection and the refusals are TypeScript's.
482+
Java, Kotlin and C# have no runner (Phase 2).
483+
458484
> **ADR-0055 (deferred overlay application) — SHIPPED in every port.** The nine cases added
459485
> for it (`overlay-mixed-file-base-in-later-file`, `overlay-mixed-file-base-in-earlier-file`,
460486
> `overlay-same-file-before-base`, `overlay-nested-under-plain-parent-base-later`,
@@ -561,7 +587,7 @@ files into a real Cube.
561587

562588
The fixtures in the nine corpora mapped above (metamodel 413 + yaml 16 + verify 37
563589
+ render 16 + persistence 59 + api-contract 117 + source-resolution 25 + scope 10 +
564-
dependency 23) each map to a feature doc, and so do the two requirement corpora and the
590+
dependency 35) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance` and the
565591
`cube-model` corpus, whose case lists live in their own READMEs. None are orphaned today. The remaining
566592
corpora in the totals table gate tooling contracts (registry manifests, provider
567593
composition, agent context, docs emit) rather than user-facing metamodel behaviour,

‎docs/features/cli.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ command surface splits in two:
3333
| **Template/prompt drift** (`verify --templates`) | **Node `meta`** | `meta verify --templates` | TS reference (ADR-0021 D2) — `{{field}}`↔payload; part of the bare-`verify` default |
3434
| **Vocabulary upgrade** (`upgrade`) | **Node `meta`** | `meta upgrade [--to <version>] [--apply]` | **any backend** — rewrites RETIRED metadata vocabulary (`@violation` → `@counterexample`, `@readOnly` → `@mutability`, dropping `@verifiedBy`) and resolves ATTRIBUTE CONTRADICTIONS (`@fields` beside `@expr` on an index key). Node-only because it edits the metadata documents themselves, which every port shares; a non-TS project runs `npx meta upgrade` against its own `metaobjects/`. **Canonical JSON and YAML alike.** Previews by default. Retirements needing a human decision are refused and the run exits non-zero, so CI cannot record a partial migration as finished |
3535
| **Vocabulary search** (`types`) | **Node `meta`** | `meta types [query]` | **any backend** — apropos/`kubectl explain` over the live metamodel registry (names + descriptions + when-to-use); the vocabulary is cross-port identical (registry-conformance) |
36-
| **Dependency sync** (`deps`) | **Node `meta`** | `meta deps sync \| check \| list` | **any backend** — resolves a declared metadata dependency's `path` transport into a committed snapshot + lock (FR-023), so `sync`/`check`/`list` themselves stay Node-only regardless of your server language. TypeScript and Python then LOAD that snapshot at every rung of their own source ladder — the two ports that implement dependency resolution in Phase 1a; Java, Kotlin, and C# don't read `dependencies` yet (Phase 2). See [`metadata-dependencies.md`](metadata-dependencies.md) |
36+
| **Dependency sync** (`deps`) | **Node `meta`** | `meta deps sync \| check \| list` | **any backend** — resolves a declared metadata dependency's `path` transport into a committed snapshot + lock (FR-023), so `sync`/`check`/`list` themselves stay Node-only regardless of your server language. TypeScript and Python then LOAD that snapshot at every rung of their own source ladder — the two ports that implement dependency resolution in Phase 1a; Java, Kotlin, and C# don't read `dependencies` yet (Phase 2). On a project whose own metadata Node `meta` cannot read (a Maven `src/main/resources/metadata` directory, a C# metadata directory, or another port's vocabulary), `sync` and `check` prove the locked dependencies load together and say that the port's own `verify` is the full check. See [`metadata-dependencies.md`](metadata-dependencies.md) |
3737
| **Canonical format** (`fmt`) | every port | `meta fmt` / `dotnet meta fmt` / `mvn metaobjects:fmt` / `metaobjects fmt` | (#304) ships in **every** port — the canonical serializer already exists in all five, so formatting is local, not Node-only. `--check` lists non-canonical files and exits non-zero without changing anything. Formats each file STANDALONE — own-mode, declared-here layer only (ADR-0039); an `extends` onto another file's base is preserved as-is, never resolved or erred on; a file declaring `overlay: true` anywhere — whether or not a same-file base exists — is reported as skipped rather than merged (merging would drop the overlay marker, a structural change fmt never makes). A YAML file is always skipped — no canonical YAML emitter exists (ADR-0006). Before writing, every port reloads the whole project with the candidate swapped in and refuses the write unless the reloaded model's canonical form is byte-identical to the untouched baseline — fmt never changes meaning. See [Fixture corpus](#meta-fmt--canonical-formatting-304) below and `fixtures/fmt-conformance/README.md` |
3838
| TS codegen | Node `meta` | `meta gen` | TS projects. **No `--template-spec` flag, deliberately** — `metaobjects.config.ts` already takes generator VALUES, so a declarative template generator is declared there (`templateGenerator()`, or `templateSpecToGenerators(parseTemplateSpec(spec))` to reuse a C#/Python spec file). Keeping it in the config is what lets `meta verify --codegen` regenerate WITH it, since that gate re-runs the config's generator list; see [declarative template scopes](codegen-concepts.md#declarative-template-scopes) |
3939
| C# codegen | `dotnet meta` | `dotnet meta init` / `gen` / `verify --templates` / `verify --codegen` | a .NET tool (`ToolCommandName=dotnet-meta`); invoked `dotnet meta` so it never shadows the Node `meta`; ships the ADR-0021 D2 subverbs (`--db` rejected, exit 2; bare `verify` = `--templates` plus `--codegen` when an output directory (`csharp.outDir` in `.metaobjects/config.json`, or `--out`) and `codegen/Generators.cs` are present). **The generator list is `codegen/Generators.cs`** (ADR-0034 Amendment 9): `dotnet meta init` writes it with `codegen/Program.cs` (two lines that run it) and `codegen/Codegen.csproj`, and `gen` / `verify --codegen` always run it through `dotnet run --project codegen`. With no `codegen/Generators.cs` they exit 2 and write nothing; `--generators` only narrows the list to declared entries. Every entry may set its own output directory (`ApiDocsGenerator { OutDir = ... }`), and `verify --codegen` compares each one. A declarative Mustache template-spec (the cross-port JSON contract shared with Python) is one more entry, `TemplateSpecGenerator { Spec = "<json>", TemplateRoot = "<dir>" }`; `gen` no longer reads `--template-spec` or a project-root `template-spec.json`, and a run that finds either with no such entry fails before writing anything; see [declarative template scopes](codegen-concepts.md#declarative-template-scopes). `dotnet meta docs` was removed in 1.1.0: it prints the line that points at [the migration section](migrations/docs-are-generators.md#c) and exits 2 |

0 commit comments

Comments
 (0)