Skip to content

Generate the regular method families of the temporal classes from the catalog - #89

Merged
estebanzimanyi merged 2 commits into
masterfrom
refactor/oo-codegen-meos-idl
Oct 31, 2026
Merged

estebanzimanyi merged 2 commits into
masterfrom
refactor/oo-codegen-meos-idl

Conversation

@estebanzimanyi

@estebanzimanyi estebanzimanyi commented May 19, 2026 •

Copy link
Copy Markdown
Member

tools/oo_codegen/codegen.py reads the meos-idl.json catalog of the MEOS
the package builds against, given by --idl or MEOS_IDL, and writes the
regular method families of TCbuffer, TPose, TNpoint and TRgeometry, the
comparisons, the spatial relationships, the distances and the
restrictions, as the mixins pymeos/main/_generated/_methods.py:
each method dispatches by the argument's type to the pymeos_cffi
function of its overload, and each temporal class takes its mixin as its
first base, so the generated method is the one the class answers with
and its fallback reaches Temporal. The constructors, the conversions, the
accessors, the transformations and the input and output stay in the
hand-written classes.

The generator writes a mixin through ruff check --fix and ruff format, so
the committed file is its output and passes the ruff workflow; a catalog
argument token the family model does not map, or that has no place in
the dispatch order, stops it with the token's name. --check accounts for
every catalogued function, emitted or counted under a named exclusion.
The rigid geometry family reads the trgeometry token of the catalog and
restricts through the public trgeometry_at_* and trgeometry_minus_*
functions, to a geometry, a pose, a set of poses and the time types; a
temporal pose restricts to an elevation span through tpose_at_elevation
and tpose_minus_elevation.

The test workflow sets MEOS_IDL to the catalog it derives;
tests/test_oo_codegen_coverage.py checks the accounting, that every
committed mixin is the generator's output on that catalog and that every
function a mixin calls is a function of the catalog.

Witness: against PyMEOS master 59191dd,
TPoseSeq('[Pose(Point(1 1 1),0.0,0.0,0.0,1.0)@2001-01-01, Pose(Point(1 1
3),0.0,0.0,0.0,1.0)@2001-01-03]').at(FloatSpan('[1.5, 2]')) raises
TypeError and a TRgeometry has no within_distance; against this branch
the first answers {[Pose(POINT Z (1 1 1.5),0,0,0,1)@2001-01-01
12:00:00+00, Pose(POINT Z (1 1 2),0,0,0,1)@2001-01-02 00:00:00+00]} and
the within_distance of a moving triangle with itself at 1.0 answers
{[t@2001-01-01 00:00:00+00, t@2001-01-03 00:00:00+00]}.

Measured: on the catalog of MobilityDB 5bf840dca5, 5446 functions
accounted for, 123 regular methods across the six families, 77 of them
wired into the four classes, 76 in place of the hand-written methods of
the same name and TRgeometry's within_distance; a second generator run rewrites the
four mixins byte for byte; 4851 tests, 6 of them the new restrictions and
within_distance; ruff check and ruff format report no finding.

Why: the bindings are projections of the MEOS catalog, so a method
family that the catalog states is generated from it and follows it as
MEOS gains overloads, where a hand-written family falls behind.

@estebanzimanyi
estebanzimanyi force-pushed the feat/extended-temporal-types branch from 98f5065 to a5a558f Compare May 19, 2026 07:24
@estebanzimanyi
estebanzimanyi force-pushed the refactor/oo-codegen-meos-idl branch from 1e7d6ea to 85deb9b Compare May 19, 2026 07:25
estebanzimanyi added a commit that referenced this pull request May 19, 2026
Implements the PyMEOS-side follow-up of RFC #94 §5: a consumer that
emits faithful geo/temporal mixins from the canonical
oo.<family>.<member>.dispatch blocks the parallel MEOS-API session is
adding to meta/meos-meta.json (merged into meos-idl.json).

Additive only -- the 4 shipped families' FAMILY_MODEL path
(emit_faithful_mixin) is untouched; #90/#91/#92/#93 mixins regenerate
byte-identical (regression-guarded) and #89 coverage gate stays green.

Proven correct WITHOUT waiting on the not-yet-published metadata:
_serialize_family_dispatch() expresses each of the 4 A/B-proven families
(cbuffer/pose/npoint/rgeo) in the RFC #94 §3 schema; feeding those back
through emit_from_oo_dispatch() reproduces their committed mixins
BYTE-IDENTICALLY for all 4 (the new --verify-oo-roundtrip gate +
OoDispatchConsumer test, permanently enforced). geo/temporal then plug
into this same proven consumer the instant MEOS-API emits their
oo.dispatch (via --mixin-from-dispatch), closing codegen 6/6 -- no
PyMEOS-local transcription, equivalence by construction at the catalog
level.

This is the non-wasted parallel prep: when the keystone enrichment
lands, PyMEOS converges immediately. Stacked on #94 (the RFC).
estebanzimanyi added a commit that referenced this pull request May 19, 2026
Implements the PyMEOS-side follow-up of RFC #94 §5: a consumer that
emits faithful geo/temporal mixins from the canonical
oo.<family>.<member>.dispatch blocks the parallel MEOS-API session is
adding to meta/meos-meta.json (merged into meos-idl.json).

Additive only -- the 4 shipped families' FAMILY_MODEL path
(emit_faithful_mixin) is untouched; #90/#91/#92/#93 mixins regenerate
byte-identical (regression-guarded) and #89 coverage gate stays green.

Proven correct WITHOUT waiting on the not-yet-published metadata:
_serialize_family_dispatch() expresses each of the 4 A/B-proven families
(cbuffer/pose/npoint/rgeo) in the RFC #94 §3 schema; feeding those back
through emit_from_oo_dispatch() reproduces their committed mixins
BYTE-IDENTICALLY for all 4 (the new --verify-oo-roundtrip gate +
OoDispatchConsumer test, permanently enforced). geo/temporal then plug
into this same proven consumer the instant MEOS-API emits their
oo.dispatch (via --mixin-from-dispatch), closing codegen 6/6 -- no
PyMEOS-local transcription, equivalence by construction at the catalog
level.

This is the non-wasted parallel prep: when the keystone enrichment
lands, PyMEOS converges immediately. Stacked on #94 (the RFC).
estebanzimanyi added a commit that referenced this pull request May 19, 2026
Implements the PyMEOS side of RFC #94 §5/§7: a consumer that emits
faithful geo/temporal mixins from the canonical objectModel.dispatch
metadata (MEOS-API #10 feat/object-model, merged into meos-idl.json),
closing the OO codegen to all 6 temporal type families with
equivalence by construction at the catalog level.

Additive only — the 4 derivable families' FAMILY_MODEL path
(emit_faithful_mixin) is untouched; #90/#91/#92/#93 mixins regenerate
byte-identical (--verify-oo-roundtrip + OoDispatchConsumer
.test_roundtrip_byte_identical, permanently enforced) and the #89
coverage gate stays green.

geo + the 4 temporal concretes (TFloat/TInt/TBool/TText) are not
derivable from the signature catalog (RFC #94 §1), so they are driven
by the verbatim objectModel.dispatch metadata and proven equivalent to
the hand-written oracle by dispatch-skeleton equality:

  * tools/oo_codegen/_d1-dispatch-extended-fixture.json — the verbatim
    §7 dispatch metadata (== MEOS-API #10 objectModel.dispatch).
  * tools/oo_codegen/_oracle_extended_methods.py — the byte-for-byte
    hand-written editorial methods (AST-extracted) the proof checks
    against; never executed, never imported.
  * --verify-oo-dispatch-extended + OoDispatchConsumer
    .test_extended_dispatch_matches_oracle: each of the 32 editorial
    methods (geo at/minus/distance/nearest_approach_distance + each
    concrete's always/ever/temporal eq·ne, at, minus) reduces to an
    identical ordered dispatch skeleton — isinstance type-set, backing
    symbol, arg transform, super-coerce, fallback, result wrap. 0
    divergence: same native calls, same transforms, never reimplemented.

The closed argTransform vocabulary maps to PyMEOS idioms
(scalarValue, scalarCast-per-base, geoToGserialized with
geodeticFromSelf, stboxToGeo, innerPtr, textsetMake, list[str],
coerce + via:super).

Stacked on #94 (the RFC, including the verbatim §7 SoT).
estebanzimanyi added a commit that referenced this pull request May 19, 2026
Implements the PyMEOS side of RFC #94 §5/§7: a consumer that emits
faithful geo/temporal mixins from the canonical objectModel.dispatch
metadata (MEOS-API #10 feat/object-model, merged into meos-idl.json),
closing the OO codegen to all 6 temporal type families with
equivalence by construction at the catalog level.

Additive only — the 4 derivable families' FAMILY_MODEL path
(emit_faithful_mixin) is untouched; #90/#91/#92/#93 mixins regenerate
byte-identical (--verify-oo-roundtrip + OoDispatchConsumer
.test_roundtrip_byte_identical, permanently enforced) and the #89
coverage gate stays green.

geo + the 4 temporal concretes (TFloat/TInt/TBool/TText) are not
derivable from the signature catalog (RFC #94 §1), so they are driven
by the verbatim objectModel.dispatch metadata and proven equivalent to
the hand-written oracle by dispatch-skeleton equality:

  * tools/oo_codegen/_d1-dispatch-extended-fixture.json — the verbatim
    §7 dispatch metadata (== MEOS-API #10 objectModel.dispatch).
  * tools/oo_codegen/_oracle_extended_methods.py — the byte-for-byte
    hand-written editorial methods (AST-extracted) the proof checks
    against; never executed, never imported.
  * --verify-oo-dispatch-extended + OoDispatchConsumer
    .test_extended_dispatch_matches_oracle: each of the 32 editorial
    methods (geo at/minus/distance/nearest_approach_distance + each
    concrete's always/ever/temporal eq·ne, at, minus) reduces to an
    identical ordered dispatch skeleton — isinstance type-set, backing
    symbol, arg transform, super-coerce, fallback, result wrap. 0
    divergence: same native calls, same transforms, never reimplemented.

The closed argTransform vocabulary maps to PyMEOS idioms
(scalarValue, scalarCast-per-base, geoToGserialized with
geodeticFromSelf, stboxToGeo, innerPtr, textsetMake, list[str],
coerce + via:super).

Stacked on #94 (the RFC, including the verbatim §7 SoT).
estebanzimanyi added a commit to estebanzimanyi/PyMEOS that referenced this pull request Jun 25, 2026
master predates the meos-idl.json OO codegen, which lives in the open stack.
Add tools/pin/compose-order.txt (the canonical fold manifest with the
disposition of every open PR; MobilityDB#95 is the verified OO-codegen frontier
containing MobilityDB#88/MobilityDB#89 and the family-switch stack) and GENERATION.md (the
per-binding generator policy: PyMEOS is two layers — PyMEOS-CFFI low-level +
the PyMEOS OO layer generated by tools/oo_codegen/codegen.py; the regular
families switch to generated mixins generate-then-retire).
@estebanzimanyi
estebanzimanyi force-pushed the feat/extended-temporal-types branch from a5a558f to 201a0c1 Compare October 22, 2026 11:54
@estebanzimanyi
estebanzimanyi force-pushed the refactor/oo-codegen-meos-idl branch from 85deb9b to 6deeb71 Compare October 22, 2026 12:00
estebanzimanyi added a commit that referenced this pull request Oct 22, 2026
Promotes the meos-idl.json-driven OO codegen from Draft preview to a
wired-in solution for the TCbuffer vertical slice.

- Generator (tools/oo_codegen/codegen.py): adds a behaviourally-faithful
  emitter (--mixin) that reproduces the hand-written isinstance ladder,
  argument transforms, result post-processing and super()/raise fallback
  exactly; only modelled families are wired (one family per PR). The
  Draft _preview/ path and the #89 coverage gate are unchanged.
- pymeos/main/_generated/tcbuffer_methods.py: generated TCbufferRegularMixin
  (30 regular methods: comparison, spatial relationship, distance,
  restriction), each dispatching to the exact pymeos_cffi backing the
  hand-written method used.
- pymeos/main/tcbuffer.py: inherits TCbufferRegularMixin; the 30
  now-generated regular methods are removed (never_* and the irregular
  hand-written core are kept).
- Provenance: regen-from-meos-api.sh + README document that the catalog
  is byte-schema-identical to MEOS-API run.py output and the generator
  runs unchanged on it.

Behavioural equivalence is proven by the existing suite: the full
tests/main/tcbuffer_test.py is byte-identical before and after the
switch -- 26 passed, 1 failed, where the single failure is a pre-existing
unrelated pymeos_cffi cbuffer_as_hexwkb out-parameter bug in the base
Cbuffer class (not temporal, not codegen). Static check: 30/30 generated
methods are backing-and-fallback identical to the hand-written oracle.

Stacked on #89; relies on #88's TCbuffer being instantiable.
estebanzimanyi added a commit that referenced this pull request Oct 22, 2026
Fans the meos-idl.json faithful codegen out to TPose (second family
after TCbuffer #90).

- codegen.py: add the `pose` FAMILY_MODEL entry (+ the `point` arg
  token — tpose distance spells its geometry overload
  `tdistance_tpose_point`). The Draft _preview path and #89 coverage
  gate are unchanged.
- pymeos/main/_generated/tpose_methods.py: generated TPoseRegularMixin
  (12 regular methods — comparison, temporal comparison, restriction,
  distance; pose has no spatial-relationship API). Each dispatches to the
  exact pymeos_cffi backing the hand-written method used.
- pymeos/main/tpose.py: inherits TPoseRegularMixin; the 12 now-generated
  regular methods removed. never_*/the irregular core and the
  already-real MEOS-backed value accessors (start_value/end_value/
  value_set/value_at_timestamp/value_n) are kept. Adds documented
  NotImplementedError stubs for from_base_time (MobilityDB#1084) and
  from_mfjson (MobilityDB#1085) so TPose/Inst/Seq/SeqSet are concrete.

Proof: tests/main/tpose_test.py is byte-identical hand-written vs wired
(A/B with the 2 stubs held constant) — 31 passed, 4 failed in BOTH; the
4 failures are pre-existing #88 defects OUTSIDE the codegen-switch domain
(TestTPoseOutputs/Pose test_str, TestTPoseAccessors test_value_at_timestamp,
TestPose test_from_as_hexwkb — hand-written accessors / base Pose, not
the regular families). Static check: 12/12 generated methods are
backing-and-fallback identical to the hand-written oracle. All
switch-domain tests (comparison/distance/restriction) pass. Stacked on
estebanzimanyi added a commit that referenced this pull request Oct 22, 2026
Third family in the codegen-switch fan-out (after TCbuffer #90, TPose

- codegen.py: add the `npoint` FAMILY_MODEL (+ `npointset` and `point`
  arg tokens; tnpoint distance spells `tdistance_tnpoint_point`). Make
  shortest_line's precision per-family (npoint: `precision: int = 15`;
  tcbuffer/tpose hardcode 10 with no param and omit the key, so #90/#91
  regenerate byte-identical -- verified). Draft _preview path and #89
  gate untouched.
- pymeos/main/_generated/tnpoint_methods.py: generated TNpointRegularMixin
  (12 regular methods -- comparison, temporal comparison, restriction,
  distance; npoint has no spatial-relationship API). Static check: 12/12
  backing-and-fallback identical to the hand-written oracle.
- pymeos/main/tnpoint.py: inherits the mixin; 12 now-generated regular
  methods removed; never_*/irregular core kept. Adds documented
  NotImplementedError stubs (value accessors -> MobilityDB#1082,
  from_base_time -> #1084, from_mfjson -> #1086) so TNpoint*/Inst/Seq/
  SeqSet are concrete.
- pymeos/collections/npoint/npointset.py: NpointSet shipped abstract in
  #88 (pre-existing base-collection defect, unrelated to the switch);
  add the 5 missing Set ops as thin super() delegations to the generic
  base Set (the established GeoSet.contains pattern), making NpointSet
  concrete.

Proof: tests/main/tnpoint_test.py is byte-identical hand-written vs
wired (A/B, stubs+NpointSet-fix held constant) -- both error IDENTICALLY
at the same pre-existing point (`tnpoint_test.py:211` TestNpointSet,
MeosTextInputError 'Missing delimiter'), so the switch delta is provably
zero. That collection error is a pre-existing #88/MEOS npointset
test-fixture/parser issue OUTSIDE the codegen-switch domain (base
NpointSet, not the temporal regular families), unrelated to and
unaffected by this change -- a separate follow-up. Static 12/12
identical is the positive proof. Stacked on #91.
@estebanzimanyi
estebanzimanyi force-pushed the feat/extended-temporal-types branch 5 times, most recently from e3597a1 to 407ec84 Compare October 31, 2026 19:56
@estebanzimanyi

Copy link
Copy Markdown
Member Author

The generated method families land as one pull request on master, generated from the catalog CI derives at its MobilityDB commit; this branch stays as a reference.

Every class has the readers and writers of the portable functions, each
calling the MEOS function the SQL function of the same name reaches, as
the catalog's typeEncodings states them: as_text and from_text for every
class; as_wkb and as_hexwkb writing plain WKB, as the SQL asBinary and
asHexWKB do, in the byte order an endian argument names ("NDR", "XDR",
or "" for the machine's own) through wkb_variant_from_endian; and, for a
type carrying an SRID, as_ewkt, as_ewkb and as_hexewkb with from_ewkt,
from_ewkb and from_hexewkb. str is as_text for every class: each class's
text writer is its as_text and its __str__ returns as_text().

The spatial temporal classes, TPoint, TCbuffer, TPose, TNpoint and
TRgeometry, take the E forms from the mixin TSpatialIO
(tspatial_as_ewkt, tspatial_as_ewkb, tspatial_as_hexewkb,
temporal_from_wkb, temporal_from_hexwkb), TRgeometry keeping its
trgeometry_as_ewkt. The spatial sets, GeoSet, PoseSet, CbufferSet and
NpointSet, write spatialset_as_ewkb and spatialset_as_hexewkb and read
set_from_wkb and set_from_hexwkb; STBox writes stbox_as_ewkb and
stbox_as_hexewkb, its text carrying its SRID; Pose, Cbuffer, Npoint and
Nsegment read and write through their own *_as_ewkb, *_as_hexewkb,
*_from_wkb and *_from_hexwkb, Nsegment gaining its binary and E forms.
Nsegment's text is nsegment_as_text and TNpoint's tspatial_as_text, the
writers the SQL asText of nsegment and tnpoint reach.

tests/io_forms_test.py checks, over a value of each of 29 classes, the
text str and as_text write, its reading back through from_text, the
byte-order marker of NDR and XDR WKB and their reading back, the EWKT of
the 14 values carrying an SRID and the EWKB and hex EWKB round trips of
those and STBox.

Witness: for TGeomPointSeq('SRID=4326;[Point(1 1)@2001-01-01, Point(2
2)@2001-01-02]'), against PyMEOS master cfed5e3 as_hexwkb() writes
012E004EE6100000..., the extended form carrying SRID 4326 under the plain
name; against this branch as_hexwkb() writes 012E000E02..., plain WKB
reading back with SRID 0, as_hexewkb() writes 012E004EE6100000...,
reading back with SRID 4326, and as_hexwkb("XDR") starts 00.

Measured: 4841 tests, 117 of them in tests/io_forms_test.py; ruff check
and ruff format report no finding.

Why: a binding's object layer holds every value as its MEOS type, and
its input and output are the portable functions with the MEOS function
the SQL function reaches, so a value a binding writes reads back in
PostgreSQL and every other binding, and the plain and E forms mean the
same everywhere.
@estebanzimanyi
estebanzimanyi changed the base branch from feat/extended-temporal-types to master October 31, 2026 21:39
… catalog

tools/oo_codegen/codegen.py reads the meos-idl.json catalog of the MEOS
the package builds against, given by --idl or MEOS_IDL, and writes the
regular method families of TCbuffer, TPose, TNpoint and TRgeometry, the
comparisons, the spatial relationships, the distances and the
restrictions, as the mixins pymeos/main/_generated/<module>_methods.py:
each method dispatches by the argument's type to the pymeos_cffi
function of its overload, and each temporal class takes its mixin as its
first base, so the generated method is the one the class answers with
and its fallback reaches Temporal. The constructors, the conversions, the
accessors, the transformations and the input and output stay in the
hand-written classes.

The generator writes a mixin through ruff check --fix and ruff format, so
the committed file is its output and passes the ruff workflow; a catalog
argument token the family model does not map, or that has no place in
the dispatch order, stops it with the token's name. --check accounts for
every catalogued function, emitted or counted under a named exclusion.
The rigid geometry family reads the trgeometry token of the catalog and
restricts through the public trgeometry_at_* and trgeometry_minus_*
functions, to a geometry, a pose, a set of poses and the time types; a
temporal pose restricts to an elevation span through tpose_at_elevation
and tpose_minus_elevation.

The test workflow sets MEOS_IDL to the catalog it derives;
tests/test_oo_codegen_coverage.py checks the accounting, that every
committed mixin is the generator's output on that catalog and that every
function a mixin calls is a function of the catalog.

Witness: against PyMEOS master 59191dd,
TPoseSeq('[Pose(Point(1 1 1),0.0,0.0,0.0,1.0)@2001-01-01, Pose(Point(1 1
3),0.0,0.0,0.0,1.0)@2001-01-03]').at(FloatSpan('[1.5, 2]')) raises
TypeError and a TRgeometry has no within_distance; against this branch
the first answers {[Pose(POINT Z (1 1 1.5),0,0,0,1)@2001-01-01
12:00:00+00, Pose(POINT Z (1 1 2),0,0,0,1)@2001-01-02 00:00:00+00]} and
the within_distance of a moving triangle with itself at 1.0 answers
{[t@2001-01-01 00:00:00+00, t@2001-01-03 00:00:00+00]}.

Measured: on the catalog of MobilityDB 5bf840dca5, 5446 functions
accounted for, 123 regular methods across the six families, 77 of them
wired into the four classes, 76 in place of the hand-written methods of
the same name and TRgeometry's within_distance; a second generator run rewrites the
four mixins byte for byte; 4851 tests, 6 of them the new restrictions and
within_distance; ruff check and ruff format report no finding.

Why: the bindings are projections of the MEOS catalog, so a method
family that the catalog states is generated from it and follows it as
MEOS gains overloads, where a hand-written family falls behind.
@estebanzimanyi
estebanzimanyi force-pushed the refactor/oo-codegen-meos-idl branch from 6deeb71 to 0afc75e Compare October 31, 2026 22:11
@estebanzimanyi estebanzimanyi changed the title feat: meos-idl.json-driven OO method-family codegen (Draft preview) Generate the regular method families of the temporal classes from the catalog Oct 31, 2026
@estebanzimanyi
estebanzimanyi merged commit 84dd465 into master Oct 31, 2026
8 checks passed
@estebanzimanyi
estebanzimanyi deleted the refactor/oo-codegen-meos-idl branch October 31, 2026 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant