Repository navigation
Conversation
Adversarial audit findings (PIV converter, audited at stack tip incl. the hardening PR)Two substantive findings, both empirically demonstrated with pyshacl end-to-end probes and cross-checked against LinkML's reference rule semantics ( A1 — real bug: greedy dispatch drops extra pre/postcondition operators → false positives. A2 — edge case: boolean-guard shadows PIV for Verified clean (attacked, held up): absent-target semantics (violation via Test gaps (cosmetic): These findings equally apply to the consolidated branch of #18. Suggest addressing A1/A2 as a follow-up commit on this stack before upstreaming. |
|
Status (head 1e506c2, mirrored as linkml#3989): both findings are fixed in this PR's single commit. #22 was closed unmerged, and its fixes were folded into that commit.
|
2e56a36 to
8999ad6
Compare
8999ad6 to
e161ce7
Compare
jdsika
changed the title
|
Mirrored upstream as linkml#3989. |
1bf1dc9 to
66276ea
Compare
The rules-to-SHACL-SPARQL converter recognised two named patterns. This
adds the presence-implies-value pattern: a precondition asserting
`value_presence: PRESENT` on one slot, and a postcondition constraining
another slot with `equals_string` or `equals_string_in`. It reads as
"if the guard slot is present, the target slot must be present and hold
one of the allowed values", and generalises the boolean guard to
arbitrary enum values. The boolean guard becomes the case
`equals_string: "true"` of this pattern on an xsd:boolean-typed flag.
`equals_string` / `equals_string_in` compare strings. The metamodel
defines them for slots of range string; enums and types with datatype
xsd:string are treated alike, and a slot without a range takes the
schema's default_range. On an xsd:boolean-typed slot each value must be
a lexical form of xsd:boolean (XML Schema 1.1 Part 2 section 3.3.2.2:
true, false, 1, 0) and denotes that boolean. A rule applying them to
any other range is skipped with a warning: its RDF values are typed
literals or IRIs that equal no string. Built-in type names resolve
without importing linkml:types, as in the main slot loop; the built-in
name of `curie` was misspelt there and is corrected.
Values are compared with `=`, spelled out as the disjunction SPARQL 1.1
section 17.4.1.9 defines `IN` to be, so the RDF 1.1-identical plain and
xsd:string literal forms both match and booleans compare by value;
rdflib evaluates `IN` and triple-pattern constants by term identity.
COALESCE turns the type error that RDFterm-equal raises for
incomparable literals into false, so a spec-conformant engine reports
such a value instead of dropping the row. Queries are DISTINCT and
project ?path (and ?value where there is one), so each result names the
property and the offending value. A value containing a backslash
followed by "u" or "U" is split with CONCAT, since codepoint escapes are
replaced before a query is parsed.
The exclusive-value pattern accepts the precondition
`has_member: {equals_string: V}`, which states "V is one of the values".
A bare `equals_string: V` is read the same way, as before, now with a
warning on a multivalued slot: the specification applies a slot
constraint to all members of a collection. The pattern gets the same
range check and value comparison, and its maximum_cardinality > 1 form
tests HAVING on the COUNT aggregate: expressions projected by SELECT are
not visible to HAVING (SPARQL 1.1 section 18.2.4.2), so
`HAVING (?count > N)` never held and the constraint never fired.
A rule applies to all members of its class, so a class shape now
carries the rules of its ancestors and mixins, as the JSON Schema
generator applies them. Each rule is translated in the inheriting
class's context, where slot_usage may refine the slots it references.
Classes sharing a class_uri share one shape, which carries each rule
once. With `open_world: true` "the postconditions may be omitted in
instance data", so an absent target is no violation.
A rule whose conditions carry any operator beyond what a pattern
translates, or name an unknown or identifier slot, is skipped rather
than partially translated. Each problem with a rule is logged once as a
warning naming the declaring class, the rule's position and the class
shapes it affects. The metadata fields the operator accounting ignores
are derived from the metamodel. Slot resolution goes through induced
slots, so slot_usage overrides, slot_uri overrides and alias-form keys
resolve to the IRI sh:path emits.
Behaviour changes for existing schemas: the boolean guard rejects a
string "true" on an xsd:boolean flag and skips a flag whose type has
another datatype IRI (main compared with str()); exclusive-value rules
with maximum_cardinality > 1 now fire; inherited rules are now enforced
on subclass shapes; problems with rules are logged at WARNING instead of
DEBUG; rule results carry sh:resultPath and sh:value; sh:message follows
the rule's in_language or default_language.
Co-authored-by: jdsika <carlo.van-driesten@vdl.digital>
66276ea to
1e506c2
Compare
2 participants
Mirror of linkml#3989, same head. Review there; this PR tracks the fork stack.
Summary
Translates the rule pattern "if slot A is present, slot B must hold one of these values" into a SHACL-SPARQL constraint (SHACL §5).
With this rule, a
Documentthat has asignatureviolates the constraint when itsstatusis missing or notPublished. The existing boolean guard is the caseequals_string: "true"on a boolean slot.Semantics
equals_string/equals_string_inare translated on string-valued slots (enums,xsd:stringtypes) and onxsd:booleanslots. Booleans accept the lexical formstrue/false/1/0and compare by value. Rules on other ranges are skipped with a warning, because their RDF values are typed literals or IRIs that equal no string.=, which is how SPARQL definesIN(§17.4.1.9). So"x"also matches"x"^^xsd:string, the same term in RDF 1.1; rdflib'sINuses term identity and misses it. The comparison is wrapped inCOALESCE, so strict engines report incomparable values instead of dropping them (§17.4.1.7).open_world: true: the target may be omitted.sh:resultPathandsh:value(SHACL §5.3.2).Changes to the existing patterns (linkml#3451)
maximum_cardinality > 1never fired, becauseHAVINGcan't see SELECT aliases (§18.2.4.2). Fixed.has_member: {equals_string: V}. A bareequals_stringon a multivalued slot keeps its "one of the values" reading, now with a warning (question 3 in the comments).xsd:booleanranges, compares by value, and honoursopen_world.Testing
test_shaclgen.pyandtest_shacl_validation_plugin.pycover each pattern end to end with pyshacl. Cross-generator tests validate the same instances with the generated JSON Schema and, through the generated JSON-LD context, with the generated SHACL; the verdicts must agree. This coversxsd:string-coerced data, subclasses andopen_world.Limitations
equals_stringon IRI-valued ranges (uriorcurie, class references) is skipped (gen-shacl rules: translate equals_string on IRI- and typed-literal-valued ranges (follow-up to linkml/linkml#3989) #37).sh:instill misses"x"^^xsd:stringunder pyshacl. This comes from rdflib's literal identity (Literal equality fails for datatypesNoneandXSD.stringRDFLib/rdflib#2123) and is unchanged here.Upstream stack: linkml#3989 → linkml#3990 → linkml#3991 (docs).
Fork stack: #19 → #20 → #23 (docs) and #35 (tests).