Repository navigation
feat(metadata): resolve references once after the tree - #444
Merged
Merged
Conversation
…erator handEdits - gen deletes a recorded, untouched file no generator emits any more (TS, Python, C#, Java, Kotlin) - Java/Kotlin decide overwrites from the committed hash manifest (ADR-0059) - handEdits: keep|overwrite per generator in all five ports
…t (ADR-0059) A reference was interpreted in two places: the YAML desugar rewrote reference values while lowering, and canonical JSON took them as written, refused `::` and `..::`, and resolved them later in each port's loader. The two paths disagreed, so the same model loaded differently from YAML and from JSON, and depended on file order. Now every source keeps references exactly as written. One resolver per port binds them after the whole tree has loaded and validated, records the bound target and the authored spelling, and rewrites each value to its canonical spelling before freeze. - One grammar in every source: `::Name` (root-absolute), `..::Name` (parent-relative, over-drop is ERR_BAD_ATTR_VALUE), `pkg::Name`, and a bare name (own file's package, then root). ERR_RELATIVE_REF_IN_CANONICAL is retired. - The referrer package is the package of the top-level node that declares the attribute, so an inherited reference means what it meant where it was written. `@of` after `@via` uses the package the walk reaches, and `@spine` uses the report's `@from` package. - The default package is per file, nested nodes included. - The serializer writes canonical values plus a package rule that lets a multi-package export load back to itself. `meta fmt` keeps authored spellings. - Consumers in codegen and runtime read the bound target (TS `refTarget`), never a short-name match. `findObject` accepts canonical values. Human-facing text keeps quoting the authored spelling. - Two new gates in every port: the YAML twin (each fixture loads to the same model from JSON and from YAML) and the round trip (each expected.json serializes back to itself), with zero exclusions. - YAML-twin defects fixed: C# YAML overlays now defer, and Java accepts a non-string field `default:` and list-valued `aliases`/`seeAlso`. - Shared goldens regenerated from the TypeScript oracle. Ten new conformance fixtures; one retired.
…ions at build time
…d for sweep and verify
…onfigured directory writers
…uild Combines the rc.6 stale-file work (orphan sweep in every port, the JVM hash manifest with (generator, outputDir) owners and the one-run-per-pair build-time check, the per-generator handEdits setting) with the resolve-once reference redesign, for one validation run. Both branches had numbered their new ADR 0059. The resolve-once ADR keeps ADR-0059 and the JVM hash-manifest ADR becomes ADR-0060; the CHANGELOG, docs and test comments that cite it move with it. CHANGELOG conflicts were additive and keep both sides.
…ors compile A review round renamed KotlinGenUtil.resolveObjectByShortOrFqn and changed RelationshipReferences' target parameter from a String to the bound node. Generators an adopter ejected before 1.1.0 call the old forms, so EjectedAtBaseGeneratorsCompatTest no longer compiled them. Restore the old signatures as deprecated delegates that do the exact canonical lookup (ADR-0059 R7), never a short-name match.
…n three docs and one stale audit-axis reference; ADRs, CHANGELOG and JVM docs already matched the change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR carries two lines of 1.1.0 rc.6 work, merged into one branch and validated in one run:
loaded, in every port.
no longer emits. Java and Kotlin get a committed hash manifest. Every generator entry gets a
handEditssetting.The rc.6 cleanup branch was merged in as one merge commit. Both branches had used the number
ADR-0059, so the JVM hash-manifest ADR is now ADR-0060, and every reference to it is updated.
Part 1: resolve-once references (ADR-0059)
References now resolve once, after the whole tree has loaded, in every port (ADR-0059). Every source
keeps reference strings exactly as written. One resolver per port binds them after the last
validation pass, records the bound target and the authored spelling, and rewrites each value to its
canonical spelling before freeze. A model now loads to the same thing from YAML and from JSON,
whatever order its files load in, and an exported model loads back to itself.
This is the build of the resolve-once audit's section 5 (C1 = A, C2 = A), plus the six extra defects.
It folds into 1.1.0 rc.6.
What changed
::Nameis root-absolute,..::Nameis parent-relative (anover-drop is
ERR_BAD_ATTR_VALUE), andpkg::Nameis exact. A bare name looks in its own file'spackage, then at the root. Canonical JSON accepts every form, and
ERR_RELATIVE_REF_IN_CANONICALis retired in all ports and in
ERROR-CODES.json.so an inherited reference keeps its meaning.
@ofafter@viauses the package the walk reaches.@spineuses the report's@frompackage.extendsuses the declaring node's own file package.loads back to itself.
meta fmtuses an authored-refs mode, so it does not rewrite adopters'references.
refTarget)instead of matching by short name.
findObjectaccepts canonical values. Docs, describe text andrefusal messages still quote the authored spelling. Generated type names use the target's own
name.
generators ejected before 1.1.0 call (
KotlinGenUtil.resolveObjectByShortOrFqn, theStringforms of
RelationshipReferences.resolveRelationshipReferenceandreferenceCandidatesFor)stay as deprecated delegates. They now do the exact canonical lookup, never a short-name match.
model from JSON and from YAML. The round-trip gate checks that each
expected.jsonserializes backto itself. Shared exclusion files hold zero divergences: the round trip excludes nothing, and the
twin excludes only nine fixtures that are format-native by design.
default:and list-valuedaliases/seeAlso.fixtures were added (
ref-*,error-ref-parent-over-drop,extends-bare-declaring-file-package,error-extends-bare-cross-package,package-default-per-file,serialize-multi-package-document,and two nested-overlay fixtures).
error-relative-ref-in-canonicalis removed. The corpus now has394 fixtures.
CONFORMANCE.mdcounts and mapping, and the
CHANGELOG1.1.0 entries.Open points from the audit (section 7), resolved
@implementedByis excluded from the reference set. Its grammar names members as well asobjects, and the requirement gate resolves it separately.
targetRefis canonical (shared goldenOrder.txt:rel product -> shop::Product).expected.json.::Namefrom a packaged referrerand
Namefrom a root-level one. The spelling uses the declaring node's package when thereferrer package is empty.
expected.jsonare spelled
::Name.ERR_RELATIVE_REF_IN_CANONICALlived only inERROR-CODES.jsonand each port's enum or constants, not in
expected-registry.json. It is removed everywhere.core-springandspring-boot-starterhave no further matchers.metadata-ktxandcodegen-kotlin.PendingOverlaysreach the loader on the YAML path. The twin gate is green with zerodivergences.
Deviations and judgement calls
fmtsafety check. It compares canonical output (meaning), whilefmtwrites authored output.The audit's T8 text compared authored output.
package. When theroot has none and every node names
Pitself, the root package stays absent. Documented inADR-0059 Decision 8.
extendslookup stays type-scoped, as before. TypeScript's is not. No fixtureseparates the two; noted for a future parity pass.
ResolutionKeyfalls back to an ancestor's package only when the node hasneither its own package nor a recorded file default.
Part 2: rc.6 generated-file cleanup (ADR-0060)
What changed
gendeletes a file it wrote earlier andno longer emits when the file still matches its recorded hash. It refuses a hand-edited one
by name.
verify --codegenuses the same rule, so its remedy for anextra:file is correct.Before, Python never deleted such a file and
verifytold the user to regenerate, which couldnot remove it.
GeneratedFileWriterrecords each file in the committed.metaobjects/.gen-state/.hashes.json, like TypeScript, Python and C#. An edited file is nowrefused instead of overwritten. The first run on an 8.0.x project adopts every generated file
and deletes nothing. Migration notes:
docs/features/migrations/jvm-codegen-hash-manifest.md.handEdits: keep | overwriteon every generator entry, in every port.keepis thedefault.
same
<outputDir>failgenerateandverifybefore anything is written, naming both.Decisions already taken on this work
handEditsis captain-requested. The captain's words: "The overwrite of a file should beconfigurable. Someone may have accidentally edited the codegen file, but we actually still want
it to be overridden. Can we do it per codegen config? Default would be not to overwrite." A
review finding that the setting exceeds the intent was declined for that reason.
share one orphan rule.
execution id, so
default-cli, a renamed execution and the lifecycle execution share owners.A sweep deletes only its own pairs' orphans and the orphans of pairs the module no longer
configures.
each profile of the module and its parent chain activated alone, and the active profile set.
A profile's files are never treated as stranded by a build without that profile.
refused at build time with an error naming both executions (and their profiles). The earlier
shared-pair sweep machinery is removed.
owners, so on-demand output is never swept by a lifecycle build. Invoking a disabled execution
explicitly when an enabled one owns the same pair is refused with the same error.
Tests
On the merged head:
scripts/ci-local.sh --quick, then the fullscripts/ci-local.sh. The full run found two redlanes, both fixed: the showcase site payload's fixture count, and the Kotlin ejected-generator
compatibility test (see "Ejected generators keep compiling"). After the fixes,
scripts/ci-local.sh --only java --only gatesis green: Java and Kotlin conformance, the Javareactor, the Java and Kotlin integration tests, and every repository gate. The TypeScript,
Python, C#, integration and Cube lanes were green in the full run, and nothing they cover
changed after it.
Validation runs
01M55DAEDRG1RP9H7164C885T1) was cancelledat review after three fix rounds, when the work paused; this combined run is run 2.
01M54ZH6AVE8G8ND0QD32HDPBX) failed; run 2(
01M559GKRX3SJC3HNGB4CWW61P) was aborted during its test step; this combined run is run 3.01M5E2PWAGJRTTWX9ZJ475V143.What Changed
resolve-references.ts,resolve_references.py,ResolveReferences.cs,References.java), which runs after validation and stores each resolved target. Every reference form (::Name,..::Name,pkg::Name, bare) now resolves the same way in YAML and JSON, and canonical JSON accepts all of them. The serializer writes canonical spellings with per-object packages, andmeta fmtkeeps authored references as written. ADR-0059 records the decision andERR_RELATIVE_REF_IN_CANONICALis retired..hashes.jsonmanifest (ADR-0060), and two runs of one generator into the same output folder are refused at build time.extendsacross packages, inherited bare references, and per-file default packages. CHANGELOG 1.1.0 entries and migration notes are updated.Validation
Risk Assessment
Testing
Drove the worktree TS
metaCLI and PythonmetaobjectsCLI live on disposable projects under /tmp. Resolve-once checks: JSON and YAML twin of the same model export identically, ::Apple binds the root object, bare Apple binds acme::Apple, a file with no package does not inherit a package, export round-trips, fmt keeps authored spellings, and an unresolved ref fails by name. Orphan sweep: TS and Python both delete an untouched file after a rename and keep a hand-edited one with a named refusal; Python handEdits: overwrite replaces the edited file while default refuses. Targeted and per-package suites pass (metadata, cli, codegen-ts, migrate-ts, runtime-ts, python orphan and overwrite tests). The single full server/typescript bun test run did not finish within the 1500s limit, so a full-suite green is unconfirmed. C#, Java and Kotlin ports were not driven live. Overall: every live scenario passed.Evidence: TS and YAML export parity transcript (JSON export)
Evidence: YAML twin export (identical to JSON)
Evidence: Unresolved ref error
Evidence: TS orphan refused for hand edit
Pipeline
Updates from git push no-mistakes
✅ **Intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
CHANGELOG.md:1433- The 1.1.0 CHANGELOG said JavareferenceCandidatesFor/resolveRelationshipReferencelose their name overloads. Commit 2af9d5f restored those overloads as deprecated delegates, and also kept KotlinKotlinGenUtil.resolveObjectByShortOrFqnas a deprecated delegate, so the claim was wrong. The changelog also never said thatresolveObjectByShortOrFqnstopped matching short names. I reworded the bullet: C# loses its string overloads; Java and Kotlin keep theirs, deprecated, for generators ejected before 1.1.0; those overloads now do an exact canonical lookup, so a short name no longer matches a packaged object; callers should pass the target object or uselookupCanonicalObject.server/typescript- Full server/typescriptbun testdid not finish: killed at the 1500s background limit with no output (bun buffers until exit). Per-package runs of metadata, cli, codegen-ts, migrate-ts and runtime-ts all pass, but one full-suite run is not confirmed green.scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchainsmeta export on JSON project and YAML twin project, diffed (identical)meta export -> reload -> export round-trip (identical)meta fmt on authored refs (::Apple and Apple kept)meta export with unresolved ::Missing (exit 1, named error)meta export no-package file does not inherit first-loaded packagemeta init + eject entity + gen rename Telemetry->TelemetryReading + verify --codegen (TS orphan swept, verify green)TS gen with hand-edited orphan (refused by name, file kept)python3 -m metaobjects.cli gen rename sweep + verify --codegen (orphan removed, verify green)python gen hand-edited orphan refused by namepython per-generator handEdits: overwrite overwrites hand-edited filebun test packages/metadata (3694 pass)bun test packages/cli (1717 pass)bun test packages/codegen-ts (2927 pass)bun test packages/migrate-ts (1062 pass)bun test packages/runtime-ts (746 pass)bun test test/codegen-drift-orphan-policy.test.ts (4 pass)pytest tests/codegen/test_orphan_sweep.py tests/codegen/test_overwrite_policy_hash.py (19 pass)full bun test in server/typescript: killed at 1500s, not completed✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.