Skip to content

feat(core): add build(validate=True) to catch state keys that nothing sets - #1038

Open
YongGoose wants to merge 2 commits into
apache:mainfrom
YongGoose:feat/graph-validation
Open

YongGoose wants to merge 2 commits into
apache:mainfrom
YongGoose:feat/graph-validation

Conversation

@YongGoose

Copy link
Copy Markdown
Member

Adds an opt-in check that fails fast when an action or a transition condition reads a state key that no action writes and the initial state does not provide. Today that only shows up as a KeyError once the action runs, often after earlier (LLM) steps have already run. This is the first check described in #67 and the "build-time graph validation" item in the planned-capabilities list.

app = (
    ApplicationBuilder()
    .with_actions(...)
    .with_transitions(...)
    .with_state(...)
    .with_entrypoint(...)
    .build(validate=True)
)
# ValueError: ... Graph validation failed:
#   - Action `win` reads ['challenges'], but no action writes them and they are not in the initial state.

Changes

  • Graph.validate(initial_state_keys=()) returns a list of problems (empty when there are none). It reports action reads and condition keys that are not in the initial state and not in any action's writes. Keys starting with __ are skipped.
  • ApplicationBuilder.build(validate=False) / abuild(validate=False): when True, raise a ValueError listing every problem. The initial keys are those from with_state plus initialize_from(default_state=...). The check runs before any persisted state is loaded, so the result does not depend on what has been saved.
  • Docs: a "Validation" section in docs/concepts/state-machine.rst.
  • Fixed the two examples the check flags (separate commit):
    • examples/llm-adventure-game: win declared reads=["challenges"], which is a module-level list, not state.
    • examples/simulation (and its copy in docs/examples/data-science/simulation.md): forecast writes current_timestep without declaring it.

How I tested this

  • New unit tests in tests/core/test_graph.py and tests/core/test_application.py, covering a valid graph, an unset action read, an unset condition key, initial state keys, __ keys, abuild, initialize_from default state, and validation being off by default.
  • Full tests/ suite in a Linux container (Python 3.12, with graphviz), excluding the persister integration and Bedrock tests: 773 passed, 1 skipped.
  • black 23.11.0, isort 5.12.0, flake8 6.1.0 and scripts/check_asf_headers.py on the changed files.
  • Ran the check over the example apps. 18 of the 50 example files that build an app could be built locally (the rest need dependencies I did not install). Before the example fixes, 2 apps were flagged, both real declaration errors (above). After the fixes, 0 are flagged.

Notes

  • Why opt-in: Burr does not enforce declarations at runtime. State.subset skips missing keys, and single-step actions can write keys they do not declare. So an app with a stale declaration (like the two examples) still runs today, and turning this on by default would break it. Whether it should later become a default warning is the maintainers' call.

  • Not included from Add graph validation/compilation #67:

    • A path-sensitive check (a key written on some paths but not on others). I prototyped it, and on the examples it produced only false positives (3 of 3). The actions guard those reads with state.get(...) or an earlier branch, and actions have no way to declare an optional read (conditions have optional_keys).
    • Type checks.
    • halt_before/halt_after reachability.

    Happy to follow up on any of these.

  • This does not touch _validate_transitions, so it should not conflict with fix(core): reject transitions after defaults #906.

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future TODOs are captured in comments
  • Project documentation has been updated if adding/changing functionality.

🤖 Generated with Claude Code

YongGoose and others added 2 commits November 11, 2026 15:49
`build(validate=True)` and `abuild(validate=True)` raise a ValueError when an
action or a transition condition reads a state key that no action writes and
the initial state does not provide. `Graph.validate()` returns the same
problems without raising. The check runs before persisted state is loaded and
is off by default, since Burr does not enforce reads/writes at runtime and
apps with stale declarations still run today.

Part of apache#67.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…idation

- llm-adventure-game: `win` declared `reads=["challenges"]`, which is a
  module-level list, not state.
- simulation: `forecast` writes `current_timestep` without declaring it
  (same fix in the docs copy).

Both examples run today only because declarations are not enforced at runtime.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions github-actions Bot added area/core Application, State, Graph, Actions area/website burr.apache.org website area/examples Relates to /examples labels Nov 11, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Application, State, Graph, Actions area/examples Relates to /examples area/website burr.apache.org website

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant