Repository navigation
Conversation
…Python 3.13+ A function with postponed annotations (`from __future__ import annotations`) decorated by a `functools.wraps` wrapper that another module defines could not become a node on Python 3.13 and newer: building the graph raised NameError. Pandera's `check_input` is such a wrapper. The node constructor passed `callabl.__globals__` to `typing.get_type_hints` on 3.13+. For a wrapped function that is the wrapper's module, where the names the user's annotations use do not exist. Without `globalns`, `get_type_hints` follows `__wrapped__` to the namespace the annotations were written in, as it already did on 3.11 and 3.12, and as `Node.from_fn` already did for the return type on every version. The 3.13 branch came in with apache#1411 ("Drops support for 3.9, adds support for 3.13") with no stated reason in the commit or its review. The suite passes without it on 3.11, 3.13 and 3.15. The tests cover one and two `functools.wraps` layers, wrappers that hide or add a parameter, a wrapper without `functools.wraps`, a bound method of a callable object, a `functools.partial`, an undecorated function, a Driver run, and Pandera's `check_input` on a Polars LazyFrame and a pandas DataFrame. (cherry picked from commit 23b9c21c3fc353887c99fd321520ca57938b9c7f)
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed and why
On Python 3.13 and newer, a function with postponed annotations (
from __future__ import annotations) decorated by afunctools.wrapswrapper from another module could not become a node: building the graph raisedNameError. Pandera'scheck_inputis such a wrapper.The node constructor in
hamilton/node.pypassedcallabl.__globals__totyping.get_type_hintson 3.13+. For a wrapped function that is the wrapper's module, where the names in the user's annotations do not exist. The fix stops passing a namespace, soget_type_hintsfollows__wrapped__to the module the annotations were written in. That is what 3.11 and 3.12 already did, and whatNode.from_fnalready did for the return type on every version. The change tohamilton/is four lines.Before and after, from the user's side
driver.Builder().with_modules(my_module).build()on Python 3.13+:NameError: name 'PALazyFrame' is not defineddoubled_pricesruns behind Pandera's check.How it was shown working
All runs are local, because core CI does not yet run for pull requests into
2.0(Ticket 01 adds that). Machine: Linux x86_64; CPython 3.11.15, 3.12.14, 3.13.12, 3.14.3, 3.15.0; Pandera 0.36.0, Polars 2.1.0 with its compatibility runtime.The new tests, selected with
pytest tests/test_node.py tests/test_end_to_end.py tests/integrations/pandera/test_pandera_check_input.py -k "wrapper or wraps or callable or partial or check_input":Before the fix (
hamilton/node.pyas onupstream/mainatef88b1e9), Python 3.13.12 (3.15.0 fails the same nine):Before the fix, Python 3.11.15, where the failure does not exist:
14 passed, 59 deselected.After the fix, on this branch:
14 passed, 59 deselectedon each of 3.11, 3.12, 3.13, 3.14 and 3.15.Project checks on this branch:
uv run prek run --all-files --hook-stage pre-merge-commituv run pytest tests/ --ignore tests/integrations -quv run pytest tests/integrationsNote on 3.11:
tests/plugins/test_huggingface_extensions.pywas left out of that run. On this machineimport lancedbdies with an illegal-instruction signal (the CPU lacks AVX2), with or without this change, solancedbwas removed from the local environment. CI on a normal runner has to cover that file.No type checker is configured on
main, and2.0does not carry the pyrefly configuration yet, so no type check was run.Decisions
inspect.unwrap(callabl).__globals__. Both make the two failing cases pass. Dropping it leaves one code path for every supported Python, and it does not raiseAttributeErrorfor a callable that has no__globals__(afunctools.partial, an instance with__call__).apache/hamiltonpull request 1411, "Drops support for 3.9, adds support for 3.13". The commit message, the description and the review thread give no reason for it.typing.get_type_hintspicks the namespace the same way in 3.12 and 3.13, and the core suite passes without the branch on 3.11, 3.13 and 3.15 (on 3.11 and 3.13 minus the one file in the note above).check_inputnode whose annotation isPALazyFrame[PriceSchema], withPALazyFramethe user module's alias forpandera.typing.polars.LazyFrame, and a second one on a pandas DataFrame. Both live intests/integrations/pandera/, which CI's "Test integrations" step runs.2.0, so the fix itself comes along; Ticket 08 keeps the tests.main, anddocs/migration-to-2.0.mddoes not exist yet (Ticket 04). One error message does change on 3.13+:Node(...)built directly from afunctools.partialor a callable instance withoutinput_typesfailed withAttributeErrorabout__globals__; it now fails with the standard library'sTypeError(3.13) or Hamilton's "Missing type hint"ValueError(3.14+). It failed before and fails after.What the tests cover beyond the two cases
One and two
functools.wrapslayers; a wrapper that hides a parameter; a wrapper that adds a parameter and its annotation; a wrapper withoutfunctools.wrapswhose own annotations resolve in its own module; an unannotated wrapper (still rejected with "Missing type hint"); a bound method of a callable object; afunctools.partialwith explicitinput_types; an undecorated function; and one Driver run over the whole module with hand-computed results.Checklist