Skip to content

Fix type hints of a function behind a wrapper from another module - #1811

Open
Dev-iL wants to merge 1 commit into
apache:mainfrom
SummitSG-LLC:2611/fix-hints
Open

Dev-iL wants to merge 1 commit into
apache:mainfrom
SummitSG-LLC:2611/fix-hints

Conversation

@Dev-iL

@Dev-iL Dev-iL commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator

What changed and why

On Python 3.13 and newer, a function with postponed annotations (from __future__ import annotations) decorated by a functools.wraps wrapper from another module could not become a node: building the graph raised NameError. Pandera's check_input is such a wrapper.

The node constructor in hamilton/node.py passed callabl.__globals__ to typing.get_type_hints on 3.13+. For a wrapped function that is the wrapper's module, where the names in the user's annotations do not exist. The fix stops passing a namespace, so get_type_hints follows __wrapped__ to the module the annotations were written in. That is what 3.11 and 3.12 already did, and what Node.from_fn already did for the return type on every version. The change to hamilton/ is four lines.

Before and after, from the user's side

# my_module.py
from __future__ import annotations
import pandera.polars as pa
import polars as pl
from pandera.typing.polars import LazyFrame as PALazyFrame

class PriceSchema(pa.DataFrameModel):
    price: float

@pa.check_input(PriceSchema.to_schema())
def doubled_prices(raw_prices: PALazyFrame[PriceSchema]) -> PALazyFrame[PriceSchema]:
    return raw_prices.with_columns(pl.col("price") * 2)

driver.Builder().with_modules(my_module).build() on Python 3.13+:

  • Before: NameError: name 'PALazyFrame' is not defined
  • After: the Driver builds, and doubled_prices runs behind Pandera's check.

How it was shown working

All runs are local, because core CI does not yet run for pull requests into 2.0 (Ticket 01 adds that). Machine: Linux x86_64; CPython 3.11.15, 3.12.14, 3.13.12, 3.14.3, 3.15.0; Pandera 0.36.0, Polars 2.1.0 with its compatibility runtime.

The new tests, selected with pytest tests/test_node.py tests/test_end_to_end.py tests/integrations/pandera/test_pandera_check_input.py -k "wrapper or wraps or callable or partial or check_input":

Before the fix (hamilton/node.py as on upstream/main at ef88b1e9), Python 3.13.12 (3.15.0 fails the same nine):

.FFFF....FFFFF                                                           [100%]
E   NameError: name 'Decimal' is not defined
E   NameError: name 'Decimal' is not defined
E   NameError: name 'Decimal' is not defined
E   NameError: name 'Decimal' is not defined
E   AttributeError: 'functools.partial' object has no attribute '__globals__'
E   AttributeError: 'Converter' object has no attribute '__globals__'
E   NameError: name 'Decimal' is not defined
E   NameError: name 'PALazyFrame' is not defined
E   NameError: name 'PADataFrame' is not defined
FAILED tests/test_node.py::test_node_from_future_annotation_behind_wrapper_from_another_module[functools.wraps]
FAILED tests/test_node.py::test_node_from_future_annotation_behind_wrapper_from_another_module[two-wrapper-layers]
FAILED tests/test_node.py::test_node_from_future_annotation_behind_wrapper_from_another_module[wrapper-hides-a-parameter]
FAILED tests/test_node.py::test_node_from_future_annotation_behind_wrapper_from_another_module[wrapper-adds-a-parameter-and-its-annotation]
FAILED tests/test_node.py::test_node_from_callable_without_annotations_reports_the_missing_hints[functools.partial]
FAILED tests/test_node.py::test_node_from_callable_without_annotations_reports_the_missing_hints[callable-object]
FAILED tests/test_end_to_end.py::test_driver_runs_future_annotation_functions_behind_wrappers_from_another_module
FAILED tests/integrations/pandera/test_pandera_check_input.py::test_check_input_on_polars_lazyframe_with_future_annotations
FAILED tests/integrations/pandera/test_pandera_check_input.py::test_check_input_on_pandas_dataframe_with_future_annotations
9 failed, 5 passed, 59 deselected in 19.23s

Before the fix, Python 3.11.15, where the failure does not exist: 14 passed, 59 deselected.

After the fix, on this branch: 14 passed, 59 deselected on each of 3.11, 3.12, 3.13, 3.14 and 3.15.

Project checks on this branch:

Command Python 3.11 Python 3.15
uv run prek run --all-files --hook-stage pre-merge-commit exit 0 exit 0
uv run pytest tests/ --ignore tests/integrations -q 1721 passed, 7 skipped (see the note) 1691 passed, 8 skipped
uv run pytest tests/integrations 15 passed, 1 xpassed 15 passed, 1 xpassed

Note on 3.11: tests/plugins/test_huggingface_extensions.py was left out of that run. On this machine import lancedb dies with an illegal-instruction signal (the CPU lacks AVX2), with or without this change, so lancedb was removed from the local environment. CI on a normal runner has to cover that file.

No type checker is configured on main, and 2.0 does not carry the pyrefly configuration yet, so no type check was run.

Decisions

  • Drop the namespace argument instead of passing inspect.unwrap(callabl).__globals__. Both make the two failing cases pass. Dropping it leaves one code path for every supported Python, and it does not raise AttributeError for a callable that has no __globals__ (a functools.partial, an instance with __call__).
  • Why the 3.13 branch could go. It arrived with apache/hamilton pull request 1411, "Drops support for 3.9, adds support for 3.13". The commit message, the description and the review thread give no reason for it. typing.get_type_hints picks the namespace the same way in 3.12 and 3.13, and the core suite passes without the branch on 3.11, 3.13 and 3.15 (on 3.11 and 3.13 minus the one file in the note above).
  • Which Pandera example. The outline does not show its code. The test builds a check_input node whose annotation is PALazyFrame[PriceSchema], with PALazyFrame the user module's alias for pandera.typing.polars.LazyFrame, and a second one on a pandas DataFrame. Both live in tests/integrations/pandera/, which CI's "Test integrations" step runs.
  • Cherry-pick, not tests only. Ticket 08 is not merged into 2.0, so the fix itself comes along; Ticket 08 keeps the tests.
  • No migration entry. The change is a fix that is also proposed for main, and docs/migration-to-2.0.md does not exist yet (Ticket 04). One error message does change on 3.13+: Node(...) built directly from a functools.partial or a callable instance without input_types failed with AttributeError about __globals__; it now fails with the standard library's TypeError (3.13) or Hamilton's "Missing type hint" ValueError (3.14+). It failed before and fails after.

What the tests cover beyond the two cases

One and two functools.wraps layers; a wrapper that hides a parameter; a wrapper that adds a parameter and its annotation; a wrapper without functools.wraps whose own annotations resolve in its own module; an unannotated wrapper (still rejected with "Missing type hint"); a bound method of a callable object; a functools.partial with explicit input_types; an undecorated function; and one Driver run over the whole module with hand-computed results.

Checklist

  • PR has an informative and human-readable title (this will be pulled into the release notes)
  • Changes are limited to a single goal (no scope creep)
  • Code passed the pre-commit check & code is left cleaner/nicer than when first encountered.
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future TODOs are captured in comments
  • Project documentation has been updated if adding/changing functionality.

…Python 3.13+

A function with postponed annotations (`from __future__ import annotations`)
decorated by a `functools.wraps` wrapper that another module defines could not
become a node on Python 3.13 and newer: building the graph raised NameError.
Pandera's `check_input` is such a wrapper.

The node constructor passed `callabl.__globals__` to `typing.get_type_hints`
on 3.13+. For a wrapped function that is the wrapper's module, where the names
the user's annotations use do not exist. Without `globalns`, `get_type_hints`
follows `__wrapped__` to the namespace the annotations were written in, as it
already did on 3.11 and 3.12, and as `Node.from_fn` already did for the return
type on every version.

The 3.13 branch came in with apache#1411 ("Drops support for 3.9, adds support for
3.13") with no stated reason in the commit or its review. The suite passes
without it on 3.11, 3.13 and 3.15.

The tests cover one and two `functools.wraps` layers, wrappers that hide or add
a parameter, a wrapper without `functools.wraps`, a bound method of a callable
object, a `functools.partial`, an undecorated function, a Driver run, and
Pandera's `check_input` on a Polars LazyFrame and a pandas DataFrame.

(cherry picked from commit 23b9c21c3fc353887c99fd321520ca57938b9c7f)

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant