Skip to content

CONFORMANCE.md

Latest commit

 

History

History
584 lines (507 loc) · 45.3 KB

CONFORMANCE.md

File metadata and controls

584 lines (507 loc) · 45.3 KB
Raw

Conformance coverage

The MetaObjects standard ships 30 shared conformance corpora under fixtures/. Every port runs every corpus that is applicable to it and asserts the same expected behaviour against the same fixtures. This page is the inverse index: fixture → feature doc + per-port pass status, and it is the single place per-corpus counts are maintained (the port READMEs deliberately point here instead of restating them).

Not every corpus is a five-port corpus, and that asymmetry is deliberate rather than a gap — schema migration is TypeScript-owned (ADR-0015), Kotlin runs on the JVM and inherits the Java loader/render/extract engines instead of re-implementing them, and a few corpora gate tooling that only one port ships. The dark cells below say which is which.

If you are coming from a feature doc's ## Verified by section, you are in the right place. If you are wondering whether a particular fixture has a human-readable explanation somewhere, look it up in the "Fixture-to-doc mapping" table below.

Per-corpus totals

Counts are fixture directories (or scenario files, where a corpus is file-shaped); regenerate with ls -d fixtures/<corpus>/*/ | wc -l for directory-shaped corpora, find fixtures/<corpus> -name "*.yaml" | wc -l for file-shaped ones.

Corpus Fixtures TS Java Kotlin C# Python
fixtures/conformance/ (metamodel) 410 ✓ ✓ inherits via metadata-ktx ✓ ✓
fixtures/yaml-conformance/ 16 16 / 16 15 / 16 (1 ledgered: yaml-quoted-leading-zero — Java pipeline strips quotes off "007") inherits via Java 15 / 16 (1 ledgered: error-yaml-coerced-hex-in-string — YamlDotNet doesn't coerce 0xFF) 16 / 16
fixtures/verify-conformance/ 37 ✓ ✓ inherits via Java ✓ ✓
fixtures/verify-strict-conformance/ 1 ✓ — — — ✓
fixtures/render-conformance/ 16 ✓ ✓ inherits via Java ✓ ✓
fixtures/extract-conformance/ 48 ✓ ✓ inherits the shared JVM engine ✓ ✓
fixtures/output-prompt-conformance/ 17 ✓ ✓ ✓ ✓ ✓
fixtures/persistence-conformance/ 59 (53 query + 6 migration) all 59; the 24 report-* and the -15 decimal scenarios also on SQLite, MySQL and D1, decimal-exact also on MySQL (a scenario's engines: key is exact) 53 query (migrations TS-only, ADR-0015); the report-* and engines: scenarios also on SQLite and MySQL 53 query (via Exposed); the report-* and engines: scenarios also on SQLite and MySQL 53 query; the report-* and engines: scenarios also on SQLite and MySQL 53 query (pg8000); the report-* and engines: scenarios also on SQLite (sqlite3) and MySQL (PyMySQL)
fixtures/api-contract-conformance/ 106 (31 core + 10 tph + 9 m2m + 4 jsonb + 2 write-through + 12 projection + 27 report + 11 decimal) ✓ (Fastify reference + generated lane); report/ and decimal/ also on SQLite, MySQL and D1 ✓ (embedded HTTP + JDBC); report/ and decimal/ generated controllers over OMDB on Postgres, SQLite and MySQL ✓ (embedded HTTP + Exposed); report/ and decimal/ also on SQLite and MySQL ✓ (Kestrel + Npgsql); report/ and decimal/ also on SQLite and MySQL ✓ (FastAPI + pg8000); report/ and decimal/ also on SQLite and MySQL
fixtures/decimal-wire/ 1 vector file (format, parse, filter operand, 15-digit read) ✓ (runtime-ts and the generated helper) ✓ (CanonicalDecimal) ✓ (the generated wire file) ✓ (DecimalWire) ✓ (decimal_wire)
fixtures/validation-conformance/ 42 cases ✓ (generated Zod + run-time runValidators) ✓ ✓ ✓ ✓ (generated Pydantic + run-time run_validators)
fixtures/registry-conformance/ 1 canonical manifest ✓ (reference emitter) ✓ ✓ ✓ ✓
fixtures/object-model-conformance/ 1 shared metadata fixture (per-port scenarios) ✓ ✓ ✓ ✓ ✓
fixtures/codegen-conformance/ 6 shared inputs; shared-provided-enum carries the enum-type decision table and cross-package-binding the reference-binding decision table (ADR-0059) every port answers ✓ ✓ ✓ ✓ ✓
fixtures/template-codegen-conformance/ 3 ✓ ✓ ✓ ✓ ✓
fixtures/template-output-render-conformance/ 5 ✓ ✓ ✓ ✓ ✓
fixtures/generator-registry-conformance/ 1 canonical manifest ✓ ✓ ✓ ✓ ✓
fixtures/provider-composition-conformance/ 9 (5 error-shape + 4 compose-load) ✓ ✓ — (JVM registry via Java) ✓ ✓
fixtures/source-resolution-conformance/ 25 cases ✓ (reference implementation) ✓ inherits via Java ✓ ✓
fixtures/scope-conformance/ 10 cases ✓ (reference implementation) — — — ✓
fixtures/dependency-conformance/ 23 cases ✓ (reference implementation) — (Phase 2) — (Phase 2) — (Phase 2) ✓ (2 of 23 assert a documented Python-only vocabulary gap instead of the corpus's full contract — see below)
fixtures/agent-context-conformance/ 4 ✓ (the emitter is TS-owned) — — — —
fixtures/metamodel-docs/ 1 ✓ (docs emit is TS-owned) — — — —
fixtures/fmt-conformance/ (#304 — meta fmt) 12 ✓ (reference) ✓ inherits via Java ✓ ✓
fixtures/field-lint-conformance/ (the verify field authoring lint) 15 ✓ (reference) ✓ inherits via Java ✓ ✓
fixtures/requirement-check-conformance/ (the verify requirement gate, ADR-0057) 43 cases ✓ (reference) ✓ inherits via Java (one Maven metaobjects:verify goal) ✓ ✓
fixtures/requirement-test-identity-conformance/ (the identity, skip state and digest of each generated requirement test, and the filter seam, ADR-0057) 26 cases ✓ (reference) ✓ identity function inherits via Java; emitted names asserted by its own generator test ✓ ✓
fixtures/naming-conformance/ 8 cases ✓ ✓ inherits via Java (RouteNaming.pluralize) ✓ ✓
fixtures/codegen-noop/ (FR-044 — reporting vocabulary: what is lowered, what stays inert) 1 model pair (reporting/with vs reporting/without) ✓ (codegen + migrate) ✓ ✓ ✓ ✓
fixtures/cube-model/ (FR-044 Plan 4 — the cube-model generator's golden Cube files, plus the canonical model's golden) 46 cases (36 trees + 10 errors) + 1 canonical golden ✓ (the exporter is TypeScript-only, spec R6; its cube lane also loads the files into a real Cube) — — — —

A ✓ means the port runs the corpus green; an explicit n / m is used where a port carries a ledgered divergence. The two ledgered YAML fixtures are documented library-vs-pipeline divergences (see the _comment block in each port's yaml-conformance-expected-failures.json for the full reconciliation note). They are tracked as known-gaps rather than silently patched — the runner treats listed fixtures as passing, but a future port-level reconciliation pass would close them.

Core corpora and template quality checks

Since ADR-0034 Amendment 3, the corpora above do two different jobs. Only the first is a promise to adopters.

  • Core — the contract. conformance/, yaml-conformance/, registry-conformance/, validation-conformance/, provider-composition-conformance/, source-resolution-conformance/, scope-conformance/, dependency-conformance/, object-model-conformance/, render-conformance/, template-codegen-conformance/ (the Mustache engine), template-output-render-conformance/, output-prompt-conformance/, extract-conformance/, verify-conformance/, verify-strict-conformance/, persistence-conformance/ (runtime reads and writes, and the TS-owned migration scenarios), agent-context-conformance/, metamodel-docs/, fmt-conformance/ (the canonical serializer, surfaced per-file — ADR-0034's "canonical format" is core), field-lint-conformance/ (the advisory field lint every port's verify prints) and requirement-check-conformance/ (the requirement gate every port's verify runs; a drift gate is core, and it is not ejectable in any port). A red cell here is a MetaObjects bug.
  • Template quality checks — not a promise. The generated lane of api-contract-conformance/, generator-registry-conformance/ (stable generator names), codegen-noop/ (vocabulary with no lowering yet emits nothing), cube-model/ (the Cube exporter's output, a TypeScript reference helper), requirement-test-identity-conformance/ (which tests the requirement-tests reference generator yields from a ledger and what each is called; the text it writes is not pinned), the codegen-compile gate and the eject contract (below). They check that the reference generators are correct starting points; an adopter's ejected copy is theirs and is not gated. The hand-rolled reference-server lane of api-contract-conformance/ still pins the wire contract that the runtimes and the client speak, which is core.

Split coverage: where a feature is gated, and where it deliberately is not

The matrix is corpus-shaped, so a feature whose coverage splits — across several corpora, or between its vocabulary and its behaviour — never gets a row of its own, and its boundary ends up discoverable only from the feature doc. The split ones are recorded here instead.

The eject contract (ADR-0034 Amendment 8). A gate, not a corpus: it has no fixtures of its own, and reads generator-registry-conformance/registry.json.

  • Gated in all five ports. Each port's eject-contract test asserts that every generator it ships is in the registry (census), that every registered generator ships eject source unless the shared file exempts it (completeness), and that each generator ejected alone compiles or loads, references nothing in the codegen packages outside the engine surface and emits what the packaged generator emits (depth). TypeScript: cli/test/eject-contract.test.ts. C#: EjectContractTests. Java: EjectContractTest in maven-plugin. Kotlin: EjectContractTest in codegen-kotlin. Python: tests/conformance/test_eject_contract.py.
  • A gap is listed, never silent. eject.pending lists every generator that still fails an assertion; it is empty ([]) for 1.1.0, so every reference passes every assertion. A port's test fails when an unlisted gap appears and when a listed one closes without its entry being removed, and a final release is refused while the list has entries (ADR-0034 Amendment 8).
  • Not gated: an adopter's owned copy. The contract is about what the reference hands over.

Capability requirements (features/requirements.md):

  • Vocabulary — gated in all five ports. The types, their attributes and their closed enums travel in registry-conformance's byte-matched manifest, which every port reproduces exactly, and accept/reject behaviour is pinned by requirement-* fixtures in fixtures/conformance/. A port that drifts on what it will load fails.
  • Checks — gated in all five ports. Every port's verify runs the requirement gate (ADR-0057, which reversed a TypeScript-only split recorded here). requirement-check-conformance pins each diagnostic's code, severity, path and message text and the summary counts against the TypeScript reference. Kotlin has no CLI of its own and runs the gate through the Java Maven goal. verify-conformance still holds no requirement cases: that corpus is template drift, and the gate has its own.
  • Generated tests — identity gated in all five ports, text in none. requirement-test-identity-conformance pins which tests a ledger yields, each one's id, witness key, skip state and digest, and what a project's filter is shown. The file a port writes is its reference helper's output and is not a cross-port contract.
  • Authoring lint — TypeScript only, by decision. The seven WARN_REQUIREMENT_* authoring advisories are printed by the Node meta verify and by no other port (ADR-0057, Consequences). They never fail a build, and no corpus holds them. The structured --format json|toon payload is TypeScript-only too.

Stated as mechanisms rather than as a list of attribute names on purpose — the requirement vocabulary has a breaking change scheduled (FR-038), which moves what the manifest contains without moving these boundaries.

The Cube exporter (features/cube-export.md):

  • Mapping — gated in TypeScript only, by decision. fixtures/cube-model/ holds 46 cases, 36 with the exact tree the generator writes and 10 with the exact error message, plus the canonical model's golden. codegen-ts/test/cube/cube-model-corpus.test.ts and cube-model-canonical.test.ts run them. No other port has an exporter (spec R6), so no other port makes a claim the corpus would have to check.
  • Cube accepts the output — queried on Postgres and MySQL, in development and production mode. The cube lane (scripts/ci-local.sh --only cube, which needs Docker) loads the canonical golden and the 36 cases that hold a tree into a pinned Cube (cubejs/cube:v1.7.43) and requires each to compile, the three MySQL cases included: compiling a model runs no SQL, so the lane's Postgres data source serves them too. For the canonical reports it compares the Cube query with the report view (a @spine report through its Cube view, its empty spine rows included), and for the escaping and measure-default cases it reads the SQL back from Cube's /v1/sql. A second live file (cube-model-mysql.live.ts) runs the MySQL path: the canonical golden generated with dialect: "mysql" is loaded into the same pinned Cube over a real MySQL 8.4 holding the report views buildReportViews lowers (fixtures/cube-model/canonical/seed.mysql.sql), and each of the twenty-three served reports' Cube query is compared with SELECT * from its view, with the pre-aggregation used asserted. A third file (cube-model-mysql-cases.live.ts) executes the three MySQL cases (dimension-time-mysql, measure-mysql, measure-boolean-mysql) on a real MySQL 8.4 with seeded tables, comparing 14 queries with rows worked out by hand from the seed (the cases have no report view to compare with). A fourth (cube-model-production.live.ts) runs Cube in production mode (CUBEJS_DEV_MODE=false, a signed token, a separate cubejs/cubestore:v1.7.43 container), on Postgres and on MySQL: it builds every rollup through the pre-aggregations jobs API, requires each served report with a rollup to be answered from Cube Store with the view's rows, and requires every rollup the model declares to answer at least one query, so a rollup no query can match fails the lane. A fifth (cube-model-shapes.live.ts) queries the shapes the canonical model does not carry (alias cubes, a TPH subtype, a one-to-one join, the int-backed enum's CASE, a two-hop @spine report) from their fixtures/cube-model/ trees and compares the rows with plain SQL.
  • The lane is a gate, not a second corpus. Like the codegen-compile gate, it reuses fixtures/persistence-conformance/canonical/meta.fitness.json and the committed schema.postgres.sql rather than adding a model beside them; only the seed rows are its own (fixtures/cube-model/canonical/seed.sql).

Does the generated code compile (codegen-compile-conformance, all five ports):

  • Not a corpus, so it has no row. It reuses fixtures/persistence-conformance/canonical/meta.fitness.json rather than adding a kitchen sink beside the one that already exists — 18 entities, two view-backed projections (one restating an entity's enum through extends), M:N junctions including a self-join, a 5-entity TPH hierarchy whose subtypes are an FK and M:N target and declare an M:N of their own, a @required + @default subtype field, jsonb, isArray, currency, decimal, and AllTypes carrying every persistable field subtype. A second such model would drift from the first, and this one is maintained by the other corpora already. Shapes an adopter's build tripped over are folded INTO it, not into a second model.
  • Why it exists. Every corpus above gates BEHAVIOUR. None asks whether the emitted code builds — so four "generated code does not compile" defects shipped in 1.0.4 with the whole matrix green, because gen exits 0 in all four cases and the adopter's build is the first thing that disagrees. The gate found five more across the ports on its first runs.
  • Each port uses its own compiler, so it is the same question rather than the same code: ts.createProgram, Roslyn, javac via ToolProvider, KotlinCompilation, and — Python having no static compiler — importing the generated package (which is what resolves the imports BETWEEN modules) plus ruff F821 for the branches importing never executes.
  • Every port excludes its framework-bound route tier — TS routesFile, C# RoutesGenerator, Java SpringControllerGenerator, Kotlin KotlinSpringControllerGenerator. Those imports are not on an in-memory compile's classpath and stubbing them drowns the signal; that tier is compiled and booted over real HTTP in the api-contract integration lane instead. The boundary is one cross-port rule, not four local concessions.
  • The prompt tier is out of scope here because the fitness corpus declares no template.* nodes. Including those generators would emit nothing and read as coverage that is not there.

Do features work in combination, judged independently (TypeScript only, the integration-tests package, real Postgres):

  • Why it exists. Every corpus above exercises each feature on its own, and every one was green while the first real application found nine defects in an afternoon — each where two features meet (a reference onto a TPH subtype, a required default on one, an M:N declared on one). Two of them were invisible for a structural reason as well: meta verify --db diffs a database against the same buildExpectedSchema that wrote its migration, so a dropped FK looked in sync; and a dropped .references() compiles.
  • The independent oracle (src/independent-oracle.ts) restates, from the metadata alone, which table stores each object's rows, which columns and FKs that table has, and which routes the API serves — and reads the answer from pg_catalog, from Drizzle's own getTableConfig, and from the booted Fastify app, never through migrate-ts or codegen. independent-oracle-pg.test.ts asks it of a hand-written TPH model, including an M:N traversal against real rows, and of the committed canonical/schema.postgres.sql every port's query runner executes.
  • The feature-combination gate (feature-combinations-pg.test.ts) enumerates feature axes — hierarchy shape, where a link is declared, what it points at, link kind, link-table key, a required or defaulted field — and runs a pairwise covering set of models (23 today) through generate, tsc with routes included, migrate, and boot, judged by the oracle. It found three defects on its first run that no other gate could see.
  • Known defects are a ledger, not a skip. A check that fails for a recorded defect passes only if every failure line matches that defect's signature, and a recorded defect that stops reproducing fails its case. The ledger can only shrink.

How to tell a deliberate split from a real parity gap, since the two look identical in the matrix — both show one port covered and four blank. Ask what the uncovered ports claim. For the requirement authoring lint they claim nothing: no other port's verify prints one of those advisories, exactly as the feature doc says. (The requirement checks were a split of this kind until ADR-0057 moved them into every port, and they stopped being one the moment a second port made the claim.) Contrast {{#hasField}} in 0.23.1, where the JVM emitted has<Field>() onto generated payload records and verify accepted the section, while no render engine in any port implemented the other half — two ports shipping halves of one promise, with no fixture that could see it. A split is deliberate when no port makes a claim the corpus would have to check; it is a gap when some port already made one.

A hazard for whoever removes an attribute — and it has now fired once. Contract prose lives inside attr descriptions in expected-registry.json, where five ports byte-match it and nothing else gates it. @trackedBy still carries "NOT resolved by verify, which does not reach the network". @verifiedBy carried "verify checks each name EXISTS and is not skipped; it never runs them" — and that attribute was retired in 0.24.0 (FR-038), which deleted the only gated statement of that guarantee along with it.

That removal was done to the rule: the sentence was rehomed in the same change, to spec/capability-ledger.md (§ "verifiedBy — RETIRED") and to the migration guide, before the description was deleted. Do the same for the next one. Sweep every description being removed for contract prose and rehome it in the same change, not after — the same class as the ADR-0047 renumbering trap, where a string that reads like prose is actually a gated artifact.

Per-port runners + commands:

Port Metamodel + YAML + render + verify Persistence API contract
TypeScript cd server/typescript && bun test (per-package, ~3s) scripts/integration-test.sh ts (needs Docker) cd server/typescript/packages/integration-tests && bun test test/api-contract.test.ts (needs Docker)
Java cd server/java && mvn -pl metadata test (and per-tier -pl render, etc.) scripts/integration-test.sh java (needs Docker) mvn -f server/java/integration-tests/pom.xml test -Dtest=ApiContractConformanceTest (needs Docker)
Kotlin cd server/java && mvn -pl codegen-kotlin test (snapshot suite) mvn -f server/java/integration-tests-kotlin/pom.xml test (needs Docker) mvn -f server/java/integration-tests-kotlin/pom.xml test -Dtest=ApiContractConformanceTest (needs Docker)
C# dotnet test (per project) scripts/integration-test.sh csharp (needs Docker) dotnet test server/csharp/MetaObjects.IntegrationTests/MetaObjects.IntegrationTests.csproj --filter "FullyQualifiedName~ApiContractConformanceTest" (needs Docker)
Python pytest (per package) scripts/integration-test.sh python (needs Docker) cd server/python && uv run --extra integration pytest tests/integration/test_api_contract.py (needs Docker)

The persistence-conformance corpus is intentionally on-demand — none of the unit-test runners (bun test, dotnet test, pytest, mvn test) pull Docker. scripts/integration-test.sh is the entry point and is wired into docs/RELEASING.md §2b as the pre-latest gate.

Fixture-to-doc mapping

fixtures/conformance/ — metamodel loader + canonical serializer (410)

Fixture prefix Feature doc
loader-basic-*, error-parse-* features/loaders.md
extends-*, error-extends-* features/entities.md (inheritance)
ref-*, error-ref-*, package-default-per-file, serialize-multi-package-document features/abstracts-and-inheritance.md (how a reference resolves: bare, ::, ..::, per-file package; ADR-0059)
identity-*, subtype-entity-*, subtype-value-* features/entities.md (identity / entity vs value)
attr-*, error-attr-*, error-reserved-word-as-attr features/entities.md (attributes)
doc-common-attrs-* features/entities.md (documentation common attrs)
auto-set-on-* features/entities.md (auto-set timestamps)
attr-filter-*, loader-filterable-*, warning-filterable-*, layout-data-grid-*, error-data-grid-* features/entities.md (filter / sort / grid)
overlay-* features/entities.md (overlay / merge)
merge-three-way-no-conflict, error-merge-conflict-attr, merge-conflict-unmarked-attr-redeclaration, warning-duplicate-declaration features/loaders.md (multi-file merge attribution, FR5c)
field-string-*, field-decimal-*, field-object-storage-*, error-field-object-storage-* features/field-types.md
currency-* features/field-types.md (currency)
enum-*, error-enum-*, warning-enum-* features/field-types.md (enum)
source-rdb-*, source-db-table-*, source-db-view-*, source-multi-source-*, error-source-* features/source-kinds.md
relationship-*, error-unknown-relationship-* features/relationships.md
template-*, error-template-* features/templates-and-payloads.md
origin-*, error-origin-* features/templates-and-payloads.md (payload origins)
projection-*, error-projection-*, field-readonly-on-view-projection features/source-kinds.md (projections + the object taxonomy, ADR-0028)
requirement-*, error-unknown-attr-requirement features/requirements.md (vocabulary only — the gate has its own corpus, requirement-check-conformance/; see "Split coverage" above)
reporting-*, error-dimension-*, error-measure-*, error-ratio-*, error-segment-*, error-report-*, error-relative-date-* features/reporting.md (the dimension / measure / segment / object.report vocabulary and its load-time rules; design in the FR-044 spec)
smoke-empty-metadata features/entities.md

fixtures/yaml-conformance/ (16)

All 16 fixtures → features/yaml-authoring.md. The corpus splits into 9 happy-path fixtures (sigil-free attrs, array suffix, anchor/alias, block scalars, mixed bare-and-prefixed, quoted leading zero, an overlay declared before its base in one file, etc.) and 7 error-yaml-* fixtures that pin the YAML 1.1 coercion guards (bool / null / hex in string contexts; numeric in enum contexts; reserved-as-attr).

fixtures/render-conformance/ (16)

All 16 fixtures → features/templates-and-payloads.md (render engine output section). 4 are end-to-end shape examples (prompt / email / spreadsheet / CSV-injection escape); 11 pin Mustache-engine semantics — dotted-path lookup, parent-context fallthrough, falsy/empty-array section behavior, inverted sections, nested partials, standalone-tag whitespace stripping, raw-HTML bypass, trailing-newline preservation, unicode multibyte handling, and the -last list-separator accessor.

fixtures/verify-conformance/ (37)

All 37 fixtures → features/migrations-and-drift.md (template drift section — Renderer.verify). A node of a fixture's payload.json field tree may carry "isArray": true; with fields it is a list of objects, without it a list of scalars. The six verify-positional-accessor-* cases pin where -last is available.

fixtures/persistence-conformance/ (59 — 53 query + 6 migration)

Engines beyond Postgres (FR-044 reports, decimals). The twenty-four queries/report-*.yaml scenarios also run on SQLite and MySQL 8.4 in every port, and on D1's local runtime in TypeScript (no cloud account). A scenario that declares engines: runs on exactly those engines (decimal-exact-15 on all four, decimal-exact on Postgres and MySQL). Java reads SQLite through OMDB's SQLiteDriver, and Python through SqliteDriver (sqlite3) and MySqlDriver (PyMySQL). Each engine executes an artifact TypeScript produced (canonical/schema.sqlite.sql from meta migrate --dialect sqlite; canonical/schema.mysql.sql from the adopter-written tables plus buildReportViews), held equal to its generator by a test, so no port writes view SQL. The expect blocks stay the Postgres wire ones; engine spelling is normalised on the actual side only (a SQLite ratio is a REAL, a MySQL DECIMAL keeps its scale, a MySQL DATETIME is the UTC wall clock), and report-relative-date carries a per-engine seed (seed-data-engine). op: relate and the write ops stay Postgres-only. See features/reporting.md.

fixtures/api-contract-conformance/ (106)

All 106 scenarios → features/api-contract.md (cross-port REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD routes answer identically over HTTP — list / get / create / patch+put / delete, plus pagination (limit/offset), sort (sort=field:dir), the withCount=1 envelope, the not_found / invalid_sort error envelopes, and the 201 / 204 status codes.

The corpus also covers the 9 cross-port filter operators (eq, ne, gt, gte, lt, lte, in, like, isNull) plus the implicit-AND combinator and 2 error shapes (invalid_filter_field / invalid_filter_op) under the URL grammar ?filter[<field>][<op>]=<value> (FR-009). On top of the 31 core scenarios the corpus carries seven sub-corpora — tph/ (10, single-table inheritance; a subtype decimal filters and refuses extra digits), m2m/ (9 — 3 plain, 5 gating TPH x M:N together, the combination each corpus alone could not reach, and 1 pinning the collection-URL spelling), jsonb/ (4, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string), write-through/ (2, table-write + view-read entities), projection/ (12, a read-only view answers reads and refuses writes with 405; a projection with no declared identity has no item route; a projection keyed on a field not named id is addressed by it, whether its identity names @fields or derives them; decimal and float fields filter), decimal/ (11, ADR-0019 Amendment 1: a decimal, array element and undeclared-precision column included, is one canonical string, read exactly, filtered exactly and refused past its declared digits) and report/ (27, FR-044: a view-backed object.report is listed, filtered, sorted and paged on its derived fields, answers POST with 405 and mounts no /{id}; a @spine report keeps a row with no facts, and a measure with @default filters and sorts as its default). All 5 ports — TS, Java, Kotlin, C#, Python — run it in BOTH lanes: a hand-rolled reference server and the port's own GENERATED API artifact booted over HTTP. write-through/, projection/, decimal/ and report/ run the generated lane only, on all five ports: what they test is whether a port's generator emits the routes. The report/ sub-corpus is the one that reads real views, so it runs full stack on Postgres, SQLite and MySQL (report/schema.sqlite.sql, report/schema.mysql.sql, both TypeScript-produced) in every port, and on D1's local runtime in TypeScript. The decimal/ sub-corpus does the same over decimal columns.

fixtures/source-resolution-conformance/ (25 cases)

All 25 cases → features/metadata-sources.md (how a declared sources set resolves to a file list). Companion to scope-conformance/ below — sources decides which files are read, scope filters what is emitted from them. The corpus is file-shaped: one committed cases.json, read directly by every port's runner, with no per-port fixture and no ledger.

It pins the resolved file SET for a declared sources list — the default directory, replacement-not-merge, the relative-path base (the directory HOLDING .metaobjects/, never the process cwd), recursive directory walking, case-insensitive extension matching, union-with-de-duplication, and every error condition (an unresolvable path, an unsupported resource/package kind, and a malformed config — "expectError": true pins only that resolution RAISES, since which error code it raises with is deliberately NOT a cross-port contract; see the corpus README). All four CLI surfaces run it — TypeScript (the reference implementation), C#, Python, and Java (Kotlin inherits it, since Kotlin has no CLI entry point of its own and runs through the same Maven plugin as Java): server/typescript/packages/sdk/test/source-resolution-conformance.test.ts, server/csharp/MetaObjects.Conformance.Tests/SourceResolutionConformanceTests.cs, server/python/tests/conformance/test_source_resolution_conformance.py, and server/java/metadata/src/test/java/com/metaobjects/config/SourceResolutionConformanceTest.java.

fixtures/scope-conformance/ (10 cases)

All 10 cases → features/metadata-sources.md (the scope pattern grammar). The corpus is file-shaped: one committed cases.json, read directly by every port's runner, with no per-port fixture and no ledger.

It pins the semantics of a consumer's include/exclude scope over fully-qualified names — * matches within one :: segment and never crosses it; a segment that is exactly ** matches one or more whole segments (so acme::** does not match the bare acme); every other character is literal, regex metacharacters included; an absent or empty include means everything; multiple include patterns are a union and exclude is applied after it; and matching is case-sensitive. These are exactly the rules four independent implementations would otherwise each get slightly wrong — the failure mode that produced the cross-port LIKE/ILIKE divergence fixed in 0.21.6.

TypeScript and Python run it today. The reference implementation is server/typescript/packages/metadata/src/scope.ts (compilePattern / compileScope / matchesScope; @metaobjectsdev/sdk re-exports it unchanged, since it moved there from sdk when FR-023 needed it from codegen-ts without a sdk dependency), and the corpus was authored against it. Python's port (server/python/src/metaobjects/scope.py, matches_scope using re.fullmatch) runs the same corpus. Java, Kotlin and C# have no runner yet; when each gains one, this corpus is what it implements against — it exists now precisely so those three land on one grammar rather than three.

fixtures/dependency-conformance/ (23 cases)

All 23 cases → features/metadata-dependencies.md (declaring a dependency, meta deps sync, the committed snapshot + lock, default exclusion of imported metadata, overlay/extends across the boundary, and every load- and resolution-time failure). File-shaped like scope-conformance/ above: one committed cases.json, no per-port fixture, no ledger.

TypeScript (the reference implementation) and Python both run all 23 cases, identically — every case's expectFiles/expectImported/expectSelected/ expectMigrateGoverned/expectLoadError assertions apply to both runners with no exemption. The two overlay cases that need a view child as incidental content use view.currency (the one concrete view.* subtype registered cross-port), not view.text — a prior revision used view.text, which Python did not register then, and carried a since-discharged allowlist for it. Java, Kotlin and C# have no runner — Phase 1a is TypeScript + Python only; those three ports arrive in Phase 2.

ADR-0055 (deferred overlay application) — SHIPPED in every port. The nine cases added for it (overlay-mixed-file-base-in-later-file, overlay-mixed-file-base-in-earlier-file, overlay-same-file-before-base, overlay-nested-under-plain-parent-base-later, overlay-applies-after-all-plain-declarations, overlay-two-overlays-source-order, error-overlay-no-target, error-overlay-no-target-nested, and the YAML twin yaml-overlay-before-base-same-file) are green on TypeScript, C#, Java, Kotlin (via the JVM loader) and Python, with no entry in any expected-failures ledger — they ran rather than being excused. The fixtures were committed while three ports were still red, and the red was recorded here rather than in a ledger, because ledgering a fixture the design intends to pass is how a port quietly stays behind.

fixtures/naming-conformance/ (8 cases)

All 8 cases → the already-plural pluralize fix (CHANGELOG ## [Unreleased]): an entity whose name is already a plural noun (Stats, Settings, Series) used to get double-pluralized (ProgramPurchaseStats → ProgramPurchaseStatses) in every port's REST collection path, generated hook/query/finder/list names, and DbSet/collection names. File-shaped like scope-conformance/ above: one committed already-plural-pluralize.json, read directly by every port's naming unit-test file, no per-port fixture, no ledger.

Each case carries BOTH axes the fix touches: apiPlural (the API/code-surface pluralizer every generator-facing use now gets) and legacyPlural (the FROZEN default-physical-table-name pluralizer, deliberately NOT fixed, so an adopter's live database never sees a proposed rename). Three already-plural cases (Stats, Settings, Series) prove the two axes now DIVERGE; three look-alike cases (Status, Address, Alias — each ends in "...s" preceded by one of the excluded letters s/u/i/a) prove both axes stay IDENTICAL to the pre-fix behavior; two baseline cases (Category, Box) don't end in "s" at all before pluralizing and are untouched by either axis.

No HTTP server, no database — this corpus is a pure function check. The route-spelling half of the general pluralization rule is additionally gated end-to-end over HTTP by fixtures/api-contract-conformance/m2m/scenarios/route-spelling-multiword-collection.yaml (the PostCategory multi-word/consonant-y scenario, predating this fix); the default-physical-table-name axis has no HTTP-observable surface at all, so it can only be proven at the pluralizer-function level, which is what this corpus does for both axes in one data file.

All five ports run it (Kotlin inherits, since KotlinNaming.collectionSegment delegates to the same RouteNaming.pluralize Java runs): server/typescript/packages/metadata/test/naming.test.ts, server/csharp/MetaObjects.Codegen.Tests/CSharpNamingTests.cs, server/java/codegen-base/src/test/java/com/metaobjects/generator/util/RouteNamingTest.java

  • server/java/metadata/src/test/java/com/metaobjects/source/Fr016SourcePhysicalNameTest.java (API-surface and frozen-legacy halves split across the two modules that already own those two functions), and server/python/tests/codegen/test_route_path_naming.py + server/python/tests/unit/test_fr016_source_name_and_kind_aliases.py (same split).

fixtures/requirement-check-conformance/ (43 cases)

All 43 cases → features/requirements.md ("The gate in every port"). Each case is a directory: input/ metadata, an optional options.json (libraries, requireImplementers) and an expected.json holding the gate's diagnostics (severity, code, requirement path, message text) and the ledger summary, or null for a model with no requirement. The expectations are written from the TypeScript reference and a port that disagrees with one is wrong. The case list, and what each pins, is in the corpus README.

Every port's verify is held to it: server/typescript/packages/cli/test/requirement-check-conformance.test.ts, server/python/tests/conformance/test_requirement_check_conformance.py, server/java/metadata/src/test/java/com/metaobjects/conformance/RequirementCheckConformanceTest.java (Kotlin inherits it, through the shared Maven goal) and server/csharp/MetaObjects.Conformance.Tests/RequirementCheckConformanceTests.cs.

fixtures/requirement-test-identity-conformance/ (26 cases)

All 26 cases → features/requirements.md ("Generated requirement tests and witnesses"). It pins the identity of every test the requirement-tests generator yields from a ledger (package, path, unit, id, witness key, status, skip state and digest), witness-key collisions, and the view a project's filter receives, through eight named filters each port's runner implements in its own language. It does not pin the text of a generated file, the uncovered warning, or a requirement name outside ASCII. The case list is in the corpus README.

All five ports run it: server/typescript/packages/codegen-ts/test/requirement-test-identity-conformance.test.ts, server/java/metadata/src/test/java/com/metaobjects/conformance/RequirementTestIdentityConformanceTest.java, server/python/tests/conformance/test_requirement_test_identity_conformance.py and server/csharp/MetaObjects.Conformance.Tests/RequirementTestIdentityConformanceTests.cs. Kotlin's identity function is the Java one; the names its generator emits are asserted by server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinRequirementTestsGeneratorTest.kt.

fixtures/cube-model/ (46 cases)

All 46 cases and the canonical golden → features/cube-export.md. Each case is the smallest model for one mapping rule (an entity's cube, a join, an alias cube, a dimension type, a measure, a measure's @default, a segment, a relative date, a rollup, a @spine report's Cube view, the escaping rules) or for one ERR_CUBE_* refusal, and holds exactly one of an expected file tree and an expected error message. The case list, with the rule each pins, is the corpus README.

One port runs it: TypeScript, in server/typescript/packages/codegen-ts/test/cube/cube-model-corpus.test.ts. The cube lane (server/typescript/packages/integration-tests/cube-live/cube-model.live.ts) loads the same files into a real Cube.

Orphaned fixtures (tested but not yet documented)

The fixtures in the nine corpora mapped above (metamodel 410 + yaml 16 + verify 37

  • render 16 + persistence 59 + api-contract 106 + source-resolution 25 + scope 10 + dependency 23) each map to a feature doc, and so do the two requirement corpora and the cube-model corpus, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider composition, agent context, docs emit) rather than user-facing metamodel behaviour, so they have no feature-doc row.

If you add a new fixture and don't see a clear home for it, either:

  1. Add a section to the closest matching feature doc and reference it from its ## Verified by block, or
  2. Open an issue if the fixture exercises behavior the feature docs haven't yet described.

Orphaned docs (documented but no fixture coverage)

None of the 8 feature docs lack fixture coverage. The closest case is the high-level overview material in each doc (e.g., the "Anatomy of an entity" explainer in entities.md) — these are pedagogical, not behavioral, and don't require pinning.

See also