Skip to content

Load the published reserved_tags.json and apply all of its rules (#1462) - #1467

Merged
VisLab merged 3 commits into
hed-standard:mainfrom
VisLab:reserved_tags
Oct 17, 2026
Merged

VisLab merged 3 commits into
hed-standard:mainfrom
VisLab:reserved_tags

Conversation

@VisLab

@VisLab VisLab commented Oct 17, 2026

Copy link
Copy Markdown
Member

hed/validator/data/reserved_tags.json is a verbatim copy of the file the HED specification publishes (section 3.2.8, Appendix A.6); the new reserved_tags.yaml workflow fails when it drifts from hed-specification main. The old reservedTags.json is removed. Every row applies in every standard schema version from 8.0.0: the file's requireValue, tagGroup and topLevelTagGroup supply what the 8.0.0 to 8.2.0 schemas lack on Delay and Duration.

Annotation validation:

  • one governing row per group, Delay secondary beside Onset, Offset, Inset or Duration; a bare (Delay/3 s) needs its inner group;
  • anchor and inner-group errors carry the row's code, as do a reserved tag outside a group or a top-level group; a tag not allowed beside a reserved tag stays TAG_GROUP_ERROR (specification rule 6);
  • a reserved tag cannot be extended (Onset/Blah, Onset/#: TAG_EXTENSION_INVALID); # as an extension of any other tag is PLACEHOLDER_INVALID; Definition/MyDef/3 is DEFINITION_INVALID;
  • requireValue, requiresTimeline and allowSpliceInGroup are read from the file; a {column} inside an Event-context group is TAG_GROUP_ERROR;
  • HED 8.0.0, whose time units carry no conversionFactor: a Delay or Duration value in the default unit (s, second, seconds, or no unit) is taken as is in a timeline table; ms or minute is still TEMPORAL_TAG_ERROR.

Schema compliance:

  • SCHEMA_LIBRARY_INVALID l covers every library schema (no library element may carry reserved), with new reasons m (a library tag named like a reserved tag) and n (a library node under a reserved tag);
  • new SCHEMA_RESERVED_INVALID: a standard schema must define the tags the file names with reserved (from 8.2.0) and topLevelTagGroup (a), and a reserved tag has no child other than # (b). Released 8.0.0 to 8.5.0 pass.

Tests: tests/validator/test_reserved_checker.py (new); spec_tests/hed-tests moves to e2753fb (the reserved-tag cases and schema cases), 1215 of 1215 pass; spec_tests/hed-schemas moves to 8fd70f3 (the 8.5.0 release).

…-standard#1462)

hed/validator/data/reserved_tags.json is a verbatim copy of the file the
HED specification publishes (section 3.2.8, Appendix A.6); the new
reserved_tags.yaml workflow fails when it drifts from hed-specification
main. The old reservedTags.json is removed. Every row applies in every
standard schema version from 8.0.0: the file's requireValue, tagGroup and
topLevelTagGroup supply what the 8.0.0 to 8.2.0 schemas lack on Delay and
Duration.

Annotation validation:
- one governing row per group, Delay secondary beside Onset, Offset, Inset
  or Duration; a bare (Delay/3 s) needs its inner group;
- anchor and inner-group errors carry the row's code, as do a reserved tag
  outside a group or a top-level group; a tag not allowed beside a reserved
  tag stays TAG_GROUP_ERROR (specification rule 6);
- a reserved tag cannot be extended (Onset/Blah, Onset/#:
  TAG_EXTENSION_INVALID); # as an extension of any other tag is
  PLACEHOLDER_INVALID; Definition/MyDef/3 is DEFINITION_INVALID;
- requireValue, requiresTimeline and allowSpliceInGroup are read from the
  file; a {column} inside an Event-context group is TAG_GROUP_ERROR;
- HED 8.0.0, whose time units carry no conversionFactor: a Delay or
  Duration value in the default unit (s, second, seconds, or no unit) is
  taken as is in a timeline table; ms or minute is still TEMPORAL_TAG_ERROR.

Schema compliance:
- SCHEMA_LIBRARY_INVALID l covers every library schema (no library element
  may carry reserved), with new reasons m (a library tag named like a
  reserved tag) and n (a library node under a reserved tag);
- new SCHEMA_RESERVED_INVALID: a standard schema must define the tags the
  file names with reserved (from 8.2.0) and topLevelTagGroup (a), and a
  reserved tag has no child other than # (b). Released 8.0.0 to 8.5.0 pass.

Tests: tests/validator/test_reserved_checker.py (new); spec_tests/hed-tests
moves to e2753fb (the reserved-tag cases and schema cases), 1215 of 1215
pass; spec_tests/hed-schemas moves to 8fd70f3 (the 8.5.0 release).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Case-sensitive reserved-name comparisons allow invalid library and older standard schemas to bypass compliance checks.

2 open issues
What changed in this PR

Aligns HED annotation validation and schema compliance with the specification's published reserved-tag rules.

Changes:

  • Replaces the legacy rules file and adds an automated drift check.
  • Applies reserved-tag grouping, value, splice, and timeline rules.
  • Expands schema compliance checks and regression coverage.
File Description
tests/​validator/​test_value_text.py Excludes unrelated fixture compliance errors.
tests/​validator/​test_tag_validator_library.py Updates placement-message expectations.
tests/​validator/​test_staged_validation.py Tests rejection of extension placeholders.
tests/​validator/​test_reserved_checker.py Adds reserved-rule regression tests.
tests/​validator/​test_onset_validator.py Updates placement-message expectations.
tests/​validator/​test_hed_validator.py Updates expected placeholder errors.
tests/​schema/​test_schema_wiki_fatal_errors.py Filters unrelated fixture errors.
tests/​schema/​test_schema_compliance.py Tests reserved-tag schema rules.
tests/​schema/​test_hed_schema_io.py Filters unrelated fixture errors.
tests/​models/​test_definition_dict.py Updates definition-error expectations.
hed/​validator/​util/​tag_util.py Enforces extension and required-value rules.
hed/​validator/​util/​group_util.py Applies file-driven placement rules.
hed/​validator/​spreadsheet_validator.py Uses timeline rules and handles generated placeholders.
hed/​validator/​sidecar_validator.py Rejects forbidden column splices.
hed/​validator/​reserved_checker.py Loads published rules and selects governing tags.
hed/​validator/​onset_validator.py Derives timeline restrictions from rules.
hed/​validator/​hed_validator.py Validates two-part Definition values.
hed/​validator/​data/​reservedTags.json Removes legacy rules.
hed/​validator/​data/​reserved_tags.json Adds published rules.
hed/​schema/​schema_validation/​compliance.py Adds standard and library reserved-tag checks.
hed/​schema/​schema_validation/​compliance_summary.py Removes an outdated unchecked-item entry.
hed/​models/​hed_tag.py Supports older schemas' default time units.
hed/​errors/​schema_error_messages.py Adds schema compliance messages.
hed/​errors/​known_error_codes.py Registers the new compliance code.
hed/​errors/​error_types.py Defines reserved-rule error identifiers.
hed/​errors/​error_messages.py Adds forbidden-splice diagnostics.
docs/​user_guide.md Documents expanded compliance checks.
.rules/​ci_cd.md Documents the drift-check workflow.
.github/​workflows/​reserved_tags.yaml Checks the bundled rules against upstream.

Review effort: Balanced

Comment thread hed/schema/schema_validation/compliance.py Outdated
Comment thread hed/schema/schema_validation/compliance.py Outdated
VisLab and others added 2 commits October 17, 2026 08:16
…names without regard to case

Tag lookups casefold their key, so a library 'task' or 'ONSET' is the
reserved tag in every other respect yet escaped SCHEMA_LIBRARY_INVALID m,
'rooted=onset' escaped reason n, and in 8.0.0 and 8.1.0, which carry no
reserved attribute, a child under 'ONSET' escaped SCHEMA_RESERVED_INVALID b.
ReservedChecker exposes a casefolded copy of the reserved-name set and the
three comparisons use it. Regression tests for each.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
spec_tests/hed-schemas moves 8fd70f3 -> 67cda64 (hed-schemas PR hed-standard#474, the
score 2.2.0 release). hed/schema/schema_data/HED_score_2.2.0.xml is the
released file, byte-identical to the submodule's copy; it is listed by
get_hed_versions, loads with partner 8.5.0 and passes compliance, including
the new reserved-tag checks. library_data.json already matched.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@VisLab
VisLab merged commit ec739f7 into hed-standard:main Oct 17, 2026
19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants