Skip to content

fix(python): keep builtin-named members of absolutely imported project modules - #2632

Open
BGamboa13 wants to merge 1 commit into
colbymchenry:mainfrom
BGamboa13:fix/python-src-layout-module-members
Open

BGamboa13 wants to merge 1 commit into
colbymchenry:mainfrom
BGamboa13:fix/python-src-layout-module-members

Conversation

@BGamboa13

Copy link
Copy Markdown

fix(python): keep builtin-named members of absolutely imported project modules

Problem

A Python project module's function named like a built-in method — get, write, update, items, find, read… — was dropped as a dict/list/file method whenever the module was imported by its absolute package path, which is how src-layout projects import their own code. Only relative imports were recognised as project modules.

# src/app/db.py
def get(key): ...
# tests/test_x.py
from app import db
db.get("k")            # before: no calls edge (treated as dict.get)   after: -> app/db.py::get
import app.store as store
store.write(rec)       # same

What changes

  • isPythonProjectModule falls back to the dotted-module file lookup (findPythonModuleFile), as resolvePythonModuleMember already did.
  • To keep that from linking .get() / .write() on an unrelated object, a receiver that the call's own function rebinds is no longer treated as the module: parameter, with … as / for / except … as target, plain/augmented/annotated assignment, walrus, del, a nested def/class of that name, a comprehension or lambda variable on the call's own statement, a function-local import of a different module under the same name, a match capture. Keyword arguments, comments, strings and sibling or nested function bodies never count as rebinding. An enclosing function's locals apply to a nested def.
  • Function scopes are built once per file (memoized per context, cleared with the other import memos), so indexing time is unchanged.

Resolution only: no extraction change, EXTRACTION_VERSION untouched (main is already at the unreleased 28).

Precision evidence

  • A 1,544-file src-layout Python project: 2,261 new edges (2,247 calls, 14 references). Population check: 2,261/2,261 have a receiver that is the imported project module (checked with the AST, independent of the resolver); 0 receivers rebound in scope. 54/54 hand-checked edges true (100%, 93.4–100% at 95%).
  • store.write incoming edges 41 → 1,675 with 0 false positives; file/socket/buffer .write() calls stay unresolved. Other targets (store.get, doctrine.get, registry.find, …) show the same pattern; edges counted there that were not added by this change already existed on main (e.g. 129 dict.get-style edges to doctrine.get).
  • Coverage of the class: of 2,557 builtin-named alias.name uses where alias is a project module, 2,556 now resolve.
  • Deliberate unresolved-over-wrong (14 edges). Against the first version of this change the shadow guard removes 14 edges on that project, all cases where the receiver is a function-local re-import of a different module under the same name or a match capture; 0 of the removed edges were false. A guard that over-matched on keyword args / comprehensions / nested-def parameters / comments (it dropped 185 correct edges) was tightened with regression tests before this PR.
  • Known limits: a function-local import other as db of a different module is only caught when the file binds that name to more than one module; case db: captures and exotic rebindings are treated conservatively (unresolved). Remaining unresolved uses on that project are class/annotation/value references (pytest.raises(store.Err), except store.Err, annotations), which are unchanged pre-existing behaviour.

Benchmarks

Index of the 1,544-file project: 45.1 s [42.6–65.4] → 41.4 s [37.6–45.4], peak RSS 3.4 → 3.0 GB, DB 137.7 → 137.9 MB (noise; no regression). An earlier version of the scope check re-blanked the whole file per function and doubled index time (≈51 s → ≈115 s); the memoized per-file scope tree fixed that (final: 41.3 s vs 40.5–41.5 s baseline).

Tests

__tests__/python-module-member-builtin-name.test.ts (8 tests): absolute from pkg import mod; import pkg.mod as m; shadowing by param / with as / assignment / for / except as / lambda; receiver text that only looks rebound (kwarg, comprehension, nested-def param, comment, local import of the same module, lambda elsewhere); rebindings that must stay unresolved (comprehension target equal to receiver on the call's statement, closure over an outer param, annotated and augmented assignment, local import of a different module, match capture). Against plain main, 6 of 8 fail. Python-named suites, frameworks-integration, pr19-improvements: 15 files pass. Full suite: 6,898 passed, 2 failed (bundle-launcher, host lacks zip; same on main).

Relation to existing issues/PRs

Not included

Package-root discovery for src layouts (#1921), aliased from-imports (#1453), class/annotation/value references through a module (mod.Class), exceptions in except.

…t modules

`from app import db; db.get(...)` and `import app.db as store;
store.write(...)` were dropped as dict/list/file method calls whenever
the module was imported by its absolute package path (how src-layout
projects import their own code), because only relative imports were
recognised as project modules. Fall back to the dotted-module file
lookup, as the module-member resolver already does.

To keep that from linking `.get()`/`.write()` on an unrelated object, a
receiver the call's own function rebinds is no longer treated as the
module: a parameter, `with ... as` / `for` / `except ... as` target,
assignment (plain, augmented, annotated), walrus, `del`, a nested
def/class of that name, a comprehension or lambda variable on the
call's statement, a local re-import of another module under the same
name, or a `match` capture. Kwargs, comments, strings and sibling or
nested function bodies never count. Function scopes are built once per
file (memoized per context) so indexing time is unchanged.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant