fix(python): resolve imports from the package roots the project has by inth3shadows · Pull Request #1921 · colbymchenry/codegraph

inth3shadows · 2026-09-23T18:28:26Z

Split from #1704, as the second of its three focused pieces. It is independent of the reworked #1704 and based directly on main. Whichever lands second needs a small rebase.

Problem: Python imports were resolved in two places. Relative specifiers went through the import resolver. Absolute ones (pkg.sub.mod) fell back to a filename-suffix match, which could bind import json to a project serializers/json.py, and import openai to llm/openai.py. src/ layouts, monorepo services, __init__.py re-exports and parenthesised imports were missed or guessed.

Change: a single resolver now serves every Python import.

Package roots come from evidence only. Candidates are the repo root, a service directory with its own pyproject.toml/setup.cfg/setup.py, its src/, the package dirs its config declares, and a script's own folder. A root inside the importer wins, nearest first. Otherwise exactly one project root must provide the module; if two do, no edge is drawn.
Test, fixture, example and vendor trees: directories with those names keep a nested project's packages from answering imports from outside that tree. So a fixture named requests/ can't capture the app's import requests.
Precedence follows CPython.
- A regular package (pkg/__init__.py) beats pkg.py, which beats a namespace directory.
- For from pkg import N, a name that pkg/__init__.py defines or re-exports beats a submodule pkg/N.py.
- Star re-exports honour __all__.
Import parsing now reads parenthesised multi-line lists, backslash continuations and import a, b, and ignores docstrings and comments.
Sync matches a fresh index. Adding or removing a package, editing an __init__.py's bindings, or editing the build config (via codegraph sync; the file watcher doesn't track build configs yet) now converges to the same graph a fresh index would build. Only affected importers are re-resolved, so a no-op sync re-opens nothing.

Overlap: this also fixes the case in #1518 (thanks @JacquesBLR): from mod import fn as alias followed by a bare alias() call now gets its calls edge. It also fixes #1517's case (thanks @JacquesBLR): a relative import list wrapped after its first name, from .services import (a,\n b). #1517's own test fails on current main and passes here. #1453's case (aliased module imports) was already fixed on main by #1785, and still passes here.

It also fixes #2566 (thanks @kogriv): from scipy import stats (or from os import path, or from scipy import stats as st) no longer binds the name by exact match to a same-named variable or function in another project file, so the import cycle that edge reported is gone. The issue's cases are pinned in python-module-resolution.test.ts; they fail on main and pass here.

Validation:

50 tests in python-module-resolution.test.ts, many of them sync-vs-fresh differential tests. Full engine suite: 4563 passed.
A differential harness compared sync with a fresh index across 32 add/remove/rename/edit scenarios, and all match.
django:
- Python edges went from 132,923 to 144,661, and unresolved refs from 103,959 to 88,840.
- About 3.4k calls moved off wrong same-name targets. Fabrications were removed, e.g. import json → serializers/json.py and stdlib html.escape → django/utils/html.py.
- Index time is unchanged.
- Adding a subpackage syncs in 1.3 s (upstream 1.0 s), and editing db/models/__init__.py in 2.2 s (upstream 1.2 s, but upstream's result doesn't match a fresh index).
openevolve: 36 wrong-file matches in examples/ fixed, and the fake openai link dropped.
Four independent review rounds, with probes compared against real python3 runtime behaviour.

Trade-offs:

A library vendored as third_party/foo (with its own pyproject, installed via pip install -e) and imported as import foo is left unresolved. That's a missed edge, chosen over letting fixtures capture the app's imports.
A name bound only under if TYPE_CHECKING: in __init__.py is treated as bound. That follows mypy/pyright rather than the runtime, where the submodule would win.

Not verified: native Windows, and sys.path / .pth manipulation.

…bymchenry#1704) Resolve against colbymchenry#1921: keep both sync resurrect paths, route the dotted module descent through resolveImportPath (package roots replace the removed findPythonModuleFile fallback), and keep findExportedSymbol for the single-member lookup.

Tree = upstream main + the open PRs that still merge cleanly: colbymchenry#1925, colbymchenry#1957, colbymchenry#1962, colbymchenry#1955, colbymchenry#1965 and the current heads of colbymchenry#1842 and colbymchenry#1913. Upstream has already carried colbymchenry#1969-colbymchenry#1971, colbymchenry#1082 and colbymchenry#1791. Dropped until they are rebased onto upstream's resolver rewrite: colbymchenry#1704, colbymchenry#1921, colbymchenry#1926 and colbymchenry#1935.

…olbymchenry#1704) Python absolute specifiers never went through resolveImportPath. Every caller bolted on a suffix match against any `…/a/b/c.py`, which bound `import json` to a project's `app/utils/json.py` (and `yaml`, `requests`, `openai` to any same-named nested file) and picked an arbitrary twin when two services both carry `core/models.py`. Python named imports couldn't resolve through the import at all: Python nodes are never isExported, and re-exports were JS-only. One resolver now answers every Python specifier, relative and absolute, from two kinds of evidence-derived root: - project roots: the repo root, and each service with its own pyproject.toml/setup.cfg/setup.py, plus its `src/` and the package dirs its config declares (read from the right tables, nested configs included). These may answer any importer, but only unambiguously. A service inside a package, or under a directory named as non-project code (tests, fixtures, examples, vendor, ...), only answers within that directory. - local roots: the parent of each top-level regular package, a script's own directory, and a namespace package seen from inside it. These answer only importers inside them, so a fixture's `tests/fixtures/proj/requests/` never becomes the application's `requests`. A containing root answers first. Among other roots, a regular package or module shadows a namespace portion. In one directory `pkg/__init__.py` beats `pkg.py`, and a dotted path is walked level by level, so a plain module ends it. A relative import can't climb above its top-level package. `from P import N` looks N up on the package first, as CPython's `_handle_fromlist` does: a def or re-export of N in `P/__init__.py` beats a submodule `P/N`, which answers only when the package doesn't bind N. Module-level defs are importable without an export flag, and a module's `from X import …` lines are its re-exports, chased with the existing cycle guard and depth cap. `from m import *` honours a literal `__all__`, and otherwise binds only public names. Import parsing reads parenthesised lists, continuations, `import a, b` and indented imports over comment- and docstring-blanked source. An attribute of an imported function (`task.delay()`) keeps the unresolved answer it had before, rather than landing on the function. Sync: the Python root set (each root with its tier and reach) is fingerprinted in project_metadata. When it changes, every Python resolution edge is re-opened (reusing the CG-33 resurrect helper) and failed Python refs are requeued. When only modules or packages below the roots appear or vanish, sync re-opens just the edges into those module names, the edges out of a changed package, and the parked failures naming them. When a package's `__init__.py` is edited, it re-opens its importers' references to the names whose binding moved (a def or re-export added, removed or repointed). The narrow sets are resolved directly; either way sync converges to what a fresh index produces. Trade-off: a package nested under a directory named as non-project code (third_party/, vendor/, examples/, tests/, ...) doesn't answer imports from outside that directory, so a `third_party/foo` used through an editable install is unresolved: a missed edge rather than a fixture capturing the application's imports.

…inds to a nested copy of it `from requests import get; get("x")` still linked the call and the import to `get` in a copy of `requests` nested in a test fixture. The bare-call gate decided whether the project provides `requests` by path suffix, so any file ending in `requests/__init__.py` counted. It now asks the import resolver, which finds modules from the package roots, through a new optional `resolveModuleFile` on the resolution context. A python `imports` reference for a name from a module the project does not provide is not name-matched either.

…oots (colbymchenry#1704 + colbymchenry#1921)

…ever binds a same-named project symbol `from scipy import stats` bound `stats` by exact name to a top-level `stats` in another project file, and reported an import cycle when that file imports this one. The stdlib form and the aliased form (`from scipy import stats as st`) did the same. Package-root resolution already removes the edge; these cases fail on main and pass here.

…Go field/interface types, workspace globs, baseUrl imports) Conflicts: name-matcher's receiver typing takes main's TS array/indexed-property block, then our Java branch (colbymchenry#1942) and Go outside-type check (colbymchenry#1954); `tsFieldCall` (main's new home of the this-field tie-break) keeps colbymchenry#1983's code-unit ordering; types.ts / index.ts keep colbymchenry#1921's `resolveModuleFile` beside main's `isDeclaredPackage` / `isIndexedFile`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Keep both CHANGELOG bullets. Both sides added resolver-context methods: keep resolveModuleFile next to main's isDeclaredPackage and isIndexedFile (types and coordinator), with main's newer wording of the path-aliases doc, and import the union of both lists from the import resolver.

Keep both CHANGELOG bullets.

Two pieces of main reached Python modules the old way, which this branch replaces with package roots: - A function-local `import a.b as X` (PR 2607) found its module with the suffix match and its member with pythonModuleSymbol, both gone here. It now resolves the module like any other import and reads the member with findExportedSymbol, as the module-level path below it. - The outside-module receiver check (PR 2606) split out isPythonProjectModule without the importer, which the root-aware lookup needs to key its memo and to ask resolveModuleFile; it now takes the importing file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

The import mappings apply to every function of the file, so reading `import X as Y` at any indentation bound a function's own alias in the others: a parameter named like it, or a module-level import of the same name in another function, took that function's module (the two scoping tests from PR 2607). Plain imports are read at module level again, keeping this branch's comma lists and backslash continuations; a function-local `import a.b as X` binds X in its own function only, through pythonFunctionModuleAlias. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

… indexing (#2734) python-module-resolution.test.ts removed a test's temp dir only when the test never opened a CodeGraph (`if (cg) cg.destroy(); else if ...`), and destroy() closes the database without deleting anything. So every test that indexed left its codegraph-pymod-* dir behind, 47 per full run, passing or not, since PR #1921 added the file. The dev box's %TEMP% held 1,739 of them by 2026-10-21. The dir is now removed after destroy(), as the other test files do. Checked with a private TEMP: main's file leaves 47 codegraph-pymod-* dirs, this one none, and its 50 tests pass both ways. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ymchenry#1921, colbymchenry#1935, colbymchenry#1955, colbymchenry#1957, colbymchenry#1965, colbymchenry#2575, colbymchenry#2595, colbymchenry#2597, colbymchenry#2603–colbymchenry#2608, colbymchenry#2611 landed upstream) Conflicts: files owned by the foreign PRs now in main (colbymchenry#1921's Python import resolver and tests, colbymchenry#1935's class-strategy method) take main's version; our colbymchenry#1954/colbymchenry#1942/colbymchenry#1945/colbymchenry#1939/colbymchenry#1715/colbymchenry#1951/colbymchenry#2308 hunks sit beside main's new Rust and Python blocks as in the rebased PR branches; java.rs keeps the integration's `constant_declaration` fields under main's initializer walk; three CHANGELOG lines the merged PRs had already contributed are deduplicated, and the method blocks the merge duplicated in queries.ts and extraction/index.ts (the resolution-edge queries and the sync resurrection helpers, present once on main) are kept once. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

… again (#3141) PR #1921 made Python imports resolve from the package roots a project has, and stopped binding by any file whose path ends in the module's name. A folder that is neither a root nor the importer's own is skipped on purpose, so this common script layout lost its links: redis's modules/vector-sets/tests/*.py do from test import TestCase, and modules/vector-sets/test.py is the runner whose folder is first on sys.path. No root says so. On redis, main lost 137 true edges (69 imports, 68 calls) that 1.6.2 had. As a last step, resolvePythonModule now walks the importer's ancestors for the module, nearest first and passing over packages. A name like test or utils may just as well be the stdlib's, so what proves the module is what the importer takes from it: every name must be one the file defines at module level. A plain import or a star import takes no name, so it proves nothing and stays unresolved, as on main. A module a root provides still answers first. Reading the importer's own from-imports keeps this inside the resolver, so every caller gets one answer: context.resolveModuleFile finds the module too, and the outside-the-project check from PR #3109 does not count it as outside. A sync follows it. Such a module binds only while it defines those names, so an edit can link or unlink importers though no file appeared or vanished. An edited module in a folder that is neither a package nor a root, with Python folders below it, now re-opens the edges into it and the failures naming it. Any other edit re-opens nothing. Validation: - 9 new tests in python-module-resolution.test.ts, 2 of them on sync. Without the fallback the 4 that expect a link fail, and the 5 that expect none pass, as they should. Here all 9 pass. Taking out each guard (every name, plain imports, packages passed over, the nearest module with the names, the re-open on edit) fails exactly its own test. - A/B against main on 14 Python repos plus express and gin, 741,560 edges: 173 added, 0 removed, 0 moved, 44 extends upgraded from exact-match to import, all in redis. All 137 edges 1.6.2 had and main lost are back. The other 36 added are the file-level imports of test.py, as a from-import from the same folder gets. The controls' full dumps are byte-identical. - A trace of the fallback found every candidate it could see on those repos and on whole django, polar's server, ansible and cpython: redis's test.py from its 36 test files, all proven, and nothing else, so no wrong edge and no rejected candidate. No plain or star import had a candidate, so leaving those unresolved costs nothing on these repos. - Index cost on redis, median of 3 interleaved: 12.0 s on main, 10.7 s here, which is noise. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…der links again (#3155) PR #1921 made Python imports resolve from the package roots a project has. A file's own folder counts as a root only when it holds no __init__.py, because in a real package a bare `from utils import x` must not take a sibling over the true top-level module. That also lost this layout: rocksdb's tools/block_cache_analyzer/block_cache_pysim_test.py does `from block_cache_pysim import (...)` beside block_cache_pysim.py, in a folder that holds an __init__.py. A script run from there has that folder first on sys.path all the same. The fallback from PR #3141 now starts at the importer's own folder, still after every root has answered, so a top-level module a root provides keeps winning. A package above the importer stays passed over. Proving the names is not enough beside a file. A trace of every candidate found that most bare names with a module beside them are the stdlib's, and the proof passes on django/utils/text.py, whose `from html import escape` would have taken django/utils/html.py. So two checks now come with it. A name in sys.stdlib_module_names stays the stdlib's, above a script as well as beside it; CPython's `test` package isn't on that list, so redis's `from test import TestCase` is unaffected. And a module that imports its own name wraps the real one, which is what that name then means. That covers a wrapper's neighbours and the wrapper itself, so a file is never its own answer. A sync follows it. An edit re-opens a module's edges only if an import node spells that module from a folder that can reach it: a file in a package folder, for the path from that folder, or a file anywhere below a plain folder that isn't a root. Those spellings are looked up by name, one per folder above the module, so an edit nobody imports that way re-opens nothing, even for a hub like mypy/types.py, and costs nothing measurable. It also closes the gap PR #3141 left open, a dotted module above a script, and a module that is a package. The [Unreleased] bullet from PR #1921 gains a clause. No new bullet, because the regression never shipped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

inth3shadows force-pushed the fix/python-module-roots branch 2 times, most recently from 09a7988 to 9324b0f Compare October 3, 2026 16:23

maxmilian mentioned this pull request Oct 8, 2026

wrong import relationship in table edges #2409

Closed

This was referenced Oct 11, 2026

fix(sync): a Python import alias links its module when the module appears later #2501

Merged

fix(sync): a Python import of a name a package re-exports links when the package appears later #2507

Merged

inth3shadows force-pushed the fix/python-module-roots branch from 9324b0f to b9840f6 Compare October 14, 2026 18:05

This was referenced Oct 14, 2026

docs(readme): Python attribute fixes are back on the fork inth3shadows/codegraph#9

Merged

Re-merge #1921 (Python package roots) onto fork main alongside #1704/#1926 inth3shadows/codegraph#10

Closed

kogriv mentioned this pull request Oct 15, 2026

Python: from <external> import name binds to an unrelated project symbol of the same name (exact-match) — false import cycles #2566

Closed

inth3shadows added 2 commits October 15, 2026 10:16

inth3shadows added a commit to inth3shadows/codegraph that referenced this pull request Oct 15, 2026

fix(python): resolve attribute-receiver modules through the package r…

b8ce930

…oots (colbymchenry#1704 + colbymchenry#1921)

inth3shadows force-pushed the fix/python-module-roots branch from b9840f6 to 8a63b2a Compare October 15, 2026 14:50

inth3shadows mentioned this pull request Oct 15, 2026

Python: from app import x links to a project variable named app, not the app package #2584

Closed

BGamboa13 mentioned this pull request Oct 17, 2026

fix(python): keep builtin-named members of absolutely imported project modules #2632

Closed

colbymchenry and others added 5 commits October 20, 2026 01:27

Merge origin/main into fix/python-module-roots

b42d251

Keep both CHANGELOG bullets.

Merge remote-tracking branch 'origin/main' into review/1921

1fc5fb3

colbymchenry merged commit 6fe9dc1 into colbymchenry:main Oct 20, 2026

colbymchenry mentioned this pull request Oct 20, 2026

fix(rust): a trait's associated type named <T as Trait>::X or bound as Trait<X = …> links only to what it names #2712

Merged

This was referenced Oct 20, 2026

fix(rust): a call on a local reaches the method of the type it is declared with #2715

Merged

test(python): the module-resolution tests remove their temp dir after indexing #2734

Merged

vio-lin mentioned this pull request Oct 29, 2026

When is the next release (1.6.3) planned? #3002

Open

This was referenced Oct 30, 2026

fix(python): a parameter or local named like an imported module is not that module #3046

Merged

fix(python): a script's import of a module in a folder above it links again #3141

Merged

colbymchenry mentioned this pull request Oct 31, 2026

fix(python): a script's import of a module beside it in a package folder links again #3155

Merged

Navigation Menu

Repository navigation

fix(python): resolve imports from the package roots the project has - #1921

fix(python): resolve imports from the package roots the project has#1921
colbymchenry merged 8 commits into
colbymchenry:mainfrom
inth3shadows:fix/python-module-roots

inth3shadows commented Sep 23, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

inth3shadows commented Sep 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

inth3shadows commented Sep 23, 2026 •

edited

Loading