Repository navigation
fix(python): a parameterized base links its class, like class Leaf(Base[str]) - #2998
JosefAschauer wants to merge 1 commit into
Conversation
…se[str]) The base-list reader took only bare identifiers and attributes, so a subscripted base (`Base[str]`, `models.Base[int]`) produced no `extends` reference and the class had no supertype edge. Both arms (wasm TreeSitterExtractor and the kernel's python.rs) now name the subscript's value. Generic[T] and other bases outside the project stay unresolved, as their plain forms do. The resolver's own text reader of a class's base list (pythonBases, used by module-global and self. dispatch) split `Pair[str, int]` at its inner comma and looked up `Base[str]` verbatim; it now drops bracketed type arguments first. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018hJjBUPFiLSFBssshFwHzq
|
Thanks for this, and for testing both extraction paths! On current main a parameterized base like |
|
Thanks for checking both paths. It's finished from my side; marking it ready for the kernel arm and the A/B. |
…se[str]) A Python class with a parameterized base had no extends edge. The grammar parses Base[str] in a base list as a subscript, and the base-list reader took only bare identifiers and attributes, so the base was skipped, on the wasm path and in the kernel alike. Everything that follows the class hierarchy missed such a class. The resolver's own text reader of a base list, pythonBases, also looked up Base[str] verbatim and split Pair[str, int] at its inner comma, so current = Leaf() then current.run() did not reach Base.run. A subscript base is now unwrapped to its value before the identifier or attribute check, in src/extraction/tree-sitter.ts and in the kernel's python.rs. pythonBases drops bracketed type arguments before splitting the list. Generic[T] and Protocol[T] stay outside the project like their plain forms; a computed base, a subscript of a subscript and keyword arguments link nothing. Validation: - The test file runs on both arms: red on main's source and kernel (12 of 16 fail, 6 per arm), green here (16 passed). - Kernel parity with this kernel on 3,155 Python files of flask, requests, mypy, litestar, sqlalchemy, polar and django: 0 diffs, and the same 5 files defer to wasm as on main. The kernel and wasm edge dumps of the repos run both ways are identical. - A/B against main on 7 Python repos, 457,948 edges: 1,492 added (901 extends, 555 calls, 36 references), 0 removed, 607 call sites moved, 223 kept their target with higher confidence. polar: 491 self.get_one_or_none() style calls move from the protocol stub RepositoryProtocol at 0.4 to RepositoryBase and its mixin at 0.9. litestar: 12 calls in GenericSyncMockRepository move from AbstractAsyncRepository to AbstractSyncRepository. sqlalchemy: 435 extends and 554 calls added, like t.insert() reaching TableClause.insert. We read 44 sampled changes and every small bucket. They read right, except 12 self.op() sites that move to the typing-only overload SQLCoreOperations.op and one Enum.copy that moves to TypeEngine.copy. - 3 of the 901 extends are wrong: sqlalchemy's Sequence[_T], Tuple[int, ...] and MutableSet[_T] from typing link to its own classes of those names. The plain form does this on main (28 class X(Enum) there), and none of the 3 brought a call edge. - Controls express and gin: full dumps byte-identical. - Index cost with PR #3001 on top, sqlalchemy, median of 3 interleaved: 23.7 s on main, 25.2 s here, with 1,143 more edges. Original PR #2998 by @JosefAschauer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Thanks @JosefAschauer! This landed as #3101 with you as the commit author, and your fix went in unchanged. The kernel arm and the A/B on real Python repos both came out well: on polar, 491 calls now reach |
Both of the resolver's readers of a Python base list stopped early. pythonBases, behind module-global and self. dispatch, read only the class line. declaredSupertypes, behind class-name calls, read three lines and needed the closing parenthesis on them. black and ruff wrap a long list one base per line, so class Wrapped(Mixin, Base) written that way seemed to have no bases: current = Wrapped() then current.run(), and Wrapped.run(x), reached nothing a base declares. Extraction already read the whole list. A new pythonBaseList reads from the class line to the list's closing parenthesis, counting nested parentheses, within the class's extent, and leaves out comments, so a ) in one doesn't end the list. Both readers use it. declaredSupertypes still drops type arguments first, the composition the author posted on PR #3001, and keeps its metaclass= and factory rewrites. Validation: - The test file: 7 of 9 fail without this change (the two one-line controls pass), 9 pass here. Putting the type-argument drop out of the merged case 'python' fails the new composition case and the two of PR #3083. - Wrapped.run(None) on a wrapped base list reaches Base::run with the kernel and with wasm, as do a list over five lines, a decorated class, and a wrapped Base[Optional[User]]. - A/B on top of PR #2998, 7 Python repos, 459,440 edges: 118 added (102 calls, 16 references), 0 removed, 959 call sites moved, 113 kept their target with higher confidence. requests, flask, litestar, mypy and django do not change. polar: 942 XRepository.from_session(session) calls move from a sibling repository's from_session at 0.65 to RepositoryBase.from_session at 0.85. sqlalchemy: wrapped heads like class Table( and class ColumnElement( now reach what they inherit: t.select() reaches FromClause.select. Its 17 moved sites read right; 15 of them land on the typing-only SQLCoreOperations overload of the same ColumnOperators method. - Controls express and gin: full dumps byte-identical. - No kernel change. Index cost with PR #2998, sqlalchemy, median of 3 interleaved: 23.7 s on main, 25.2 s here. Original PR #3001 by @JosefAschauer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t class of that name (#3109) A Python class whose base the file imports from a package, like class Color(Enum) after from enum import Enum, linked to any project class named Enum: the import found no project file, and the name match or a framework resolver then took the first class of that name. sqlalchemy's enums all extended its own sql/sqltypes.py Enum, and litestar's pydantic models extended a BaseModel in a docs example. PR #2998 extended this to parameterized bases, like Sequence[_T] from typing. isBoundToOutOfRepoImport is the gate that drops such an edge for JS, TS and Rust. It now answers for Python too: the name is outside when every import binding it is absolute and resolveImportPath, which searches for a module the way Python does, finds none. A class in a file the import path names stays, because that search can miss a module on a path the program sets up itself, and so does a class of the referring file, which may rebind the name. A name no named import binds, like one from a star import, is left as it was. Validation: - New python-outside-bases.test.ts: the 7 outside bases fail on main, plain and parameterized, and every case that must keep its edge passes both ways. Green here (7 passed). Taking out each of the 4 guards fails exactly its own test. - A/B on 7 Python repos, 469,927 edges with the controls: 67 extends removed, 0 added, 0 moved. All 67 read against their imports and all were wrong: sqlalchemy 31 (28 from enum, 3 from typing), litestar 21 (pydantic BaseModel), polar 14 (11 fastapi APIRoute, 2 enum IntEnum, 1 opentelemetry Meter), django 1 (ctypes Union). - Controls express and gin unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Problem
A Python class with a parameterized base,
class Leaf(Base[str]), has noextendsedge. The grammar parsesBase[str]in the base list as asubscript, and the base-list reader only took bare identifiers and attributes, so the base was skipped. Everything that follows the class hierarchy misses such a class. This is the Python counterpart of #2856 (Java/TSStore<String>), and the "generic bases" item left open in #2076.The resolver also reads a class's base list as text (
pythonBasesinname-matcher.ts, behind module-global andself.dispatch). It looked upBase[str]verbatim and splitPair[str, int]at its inner comma, socurrent = Leaf()thencurrent.run()did not reachBase.run.Fix
subscriptbase is unwrapped to itsvaluefield before the existingidentifier | attributecheck, insrc/extraction/tree-sitter.tsand in the kernel'scodegraph-kernel/src/python.rs. The reference sits at the value's start, the position the plain form had.pythonBasesdrops bracketed type arguments (nested ones too) before splitting the list.Generic[T]/Protocol[T](outside the project, like their plain forms), a computed base likemake()[0], a subscript of a subscript likeBase[int][str], and keyword arguments likemetaclass=Meta.Validation
New
__tests__/python-generic-bases.test.ts, run on both arms: wasm (CODEGRAPH_KERNEL=0) and kernel (CODEGRAPH_KERNEL_LANGS=python; skipped when no kernel is staged). 16 tests:Base[str],Base[list[int]],Pair[str, dict[str, int]], a base written over several lines, a mixed list(Mixin, Base[str]), andmodels.Base[int]all link like their plain forms;Base[int][str]link nothing;Generic[T]links nothing;Leaf/TworesolvesruntoBase::run/Pair::run, with a same-named decoy class in the file.Each fix was mutation-checked on its own: disabling the wasm unwrap fails only wasm tests, disabling the kernel unwrap fails only kernel tests, and disabling the
pythonBaseschange fails the module-global test on both arms.npx tsc --noEmit: clean. Kernel built withnpm run build:kernel.Full suite on Linux with the kernel staged: 9281 passed, 4 failed. The failures are environmental and also fail on
main:bundle-launcher×2: nozipbinary.cli-sync×2: the host runs Node 26, which prints the unsupported-Node banner.Together with feat(resolution): link Python base methods to their overrides (python-override) #2076,
Base.rungets its override link toLeaf.runon both arms.Follow-ups
pythonBasesreads only theclassline, so a base list written over several lines is invisible to it (plain bases too). Fixed in fix(python): a base list wrapped over several lines is read whole #3001, which is stacked on this PR.declaredSupertypes' Python branch counts type arguments (str,User) as supertypes, soRepo.save()onclass Repo(Base[User])reachedUser.save. Fixed in fix(python): a base's type argument is no supertype, so Repo.save() is not User.save #3000, which is independent.Independent of #2076.
🤖 Generated with Claude Code
https://claude.ai/code/session_018hJjBUPFiLSFBssshFwHzq