902 Commits

Author SHA1 Message Date
Gregory P. Smith 9a57179d74 GH-83065: Fix import deadlock by implementing hierarchical module locking (GH-137196)
Make _find_and_load() acquire the module locks for the full
dotted-name chain (parent before child) when loading a nested module, so
both threads contend on the same first lock and serialise instead of
deadlocking.

When acquiring a parent's lock would itself deadlock with another thread
that is loading that parent (cross-package circular imports), the parent's
lock is skipped and the partially-initialised parent is accepted -- the
same policy _lock_unlock_module() already applies on the existing code
path -- so concurrent circular imports that worked before continue to work.
2026-04-28 01:06:23 -07:00
Jason R. Coombs 480edc1aae gh-121190: Emit a better error message from importlib.resources.files() when module spec is None" (#148460)
Also merges incidental changes from importlib_resources 7.1.

Co-authored by: Yuichiro Tachibana (Tsuchiya) <t.yic.yt@gmail.com>
2026-04-12 18:15:01 -04:00
Jason R. Coombs daa2578dc0 gh-127012: Traversable.read_text now allows/solicits an errors parameter. (#148401)
Applies changes from importlib_resources 6.5.2.
2026-04-11 22:25:20 +00:00
Jason R. Coombs f5d47fceb0 gh-143387: Raise an exception instead of returning None when metadata file is missing. (#146234) 2026-03-23 09:12:36 -04:00
Jason R. Coombs b5e4c46be2 gh-146228: Better fork support in cached FastPath (#146231)
* Apply changes from importlib_metadata 8.9.0
* Suppress deprecation warning in fork.
2026-03-20 20:10:50 +00:00
Steve Dower a51b1b512d gh-145506: Fixes CVE-2026-2297 by ensuring SourcelessFileLoader uses io.open_code (GH-145507) 2026-03-04 19:55:52 +00:00
Philipp A. ae7fc4a4f6 gh-110937: Document full public importlib.metadata.Distribution API (#143480) 2026-02-23 18:51:03 -05:00
Filipe Laíns 157f271de3 gh-139899: Introduce MetaPathFinder.discover and PathEntryFinder.discover (#139900)
* gh-139899: Introduce MetaPathFinder.discover and PathEntryFinder.discover

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Fix doc reference

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Remove specific doc references

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Fix docstrings

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Revert "Remove specific doc references"

This reverts commit 31d1a8f551.

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Fix news references

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Add docs warning

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Raise ValueError on invalid parent

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Dedupe __path__ in PathFinder.discover

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Use context manager and add error handling to os.scandir

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Raise ValueError on invalid parent

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Dedupe when package exists with multiple suffixes

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Apply suggestions from code review

Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>

* Add tests

Signed-off-by: Filipe Laíns <lains@riseup.net>

---------

Signed-off-by: Filipe Laíns <lains@riseup.net>
Co-authored-by: Alyssa Coghlan <ncoghlan@gmail.com>
Co-authored-by: Brett Cannon <brett@python.org>
2026-02-19 15:25:50 +00:00
Pablo Galindo Salgado 46d5106cfa gh-142349: Implement PEP 810 - Explicit lazy imports (#142351)
Co-authored-by: T. Wouters <twouters@meta.com >
Co-authored-by: Brittany Reynoso <breynoso@meta.com>
Co-authored-by: Dino Viehland <dinoviehland@meta.com>
2026-02-12 00:15:33 +00:00
Gregory P. Smith ac8b5b6890 gh-143650: Fix importlib race condition on import failure (GH-143651)
Fix a race condition where a thread could receive a partially-initialized
module when another thread's import fails. The race occurs when:

1. Thread 1 starts importing, adds module to sys.modules
2. Thread 2 sees the module in sys.modules via the fast path
3. Thread 1's import fails, removes module from sys.modules
4. Thread 2 returns a stale module reference not in sys.modules

The fix adds verification after the "skip lock" optimization in both Python
and C code paths to check if the module is still in sys.modules. If the
module was removed (due to import failure), we retry the import so the
caller receives the actual exception from the import failure rather than
a stale module reference.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-02-10 14:08:33 +01:00
Hugo van Kemenade 28fb13cb33 gh-143658: Use str.lower and replace to further improve performance of importlib.metadata.Prepared.normalized (#144083)
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
2026-02-06 19:38:58 +02:00
Hugo van Kemenade cbf9b8cc08 gh-143658: importlib.metadata: Use str.translate to improve performance of importlib.metadata.Prepared.normalized (#143660)
Co-authored-by: Henry Schreiner <henryschreineriii@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Bartosz Sławecki <bartosz@ilikepython.com>
2026-01-13 08:54:15 +02:00
Brett Cannon dfeefbe8ea GH-139686: Revert "gh-139686: Make reloading a lazy module no-op (GH-139857)" (#143584)
This reverts commits 57db12514a and 0a97941245.
2026-01-08 22:08:03 +00:00
Hugo van Kemenade b3b644b6b8 gh-141081: Revert "Add a .gitignore file to __pycache__ folders (#141162)" (#142758) 2025-12-15 21:27:06 +02:00
Stan Ulbrych 15c9d9027e gh-141081: Add a .gitignore file to __pycache__ folders (#141162)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
2025-12-15 12:16:56 +02:00
Amer Esmail Elsheikh 57db12514a gh-139686: Make reloading a lazy module no-op (GH-139857)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
2025-12-12 20:26:50 +00:00
Brett Cannon af185727b2 GH-65961: Stop setting __cached__ on modules (GH-142165) 2025-12-11 11:44:46 -08:00
Brett Cannon dac4589726 GH-142203: Remove the debug_override parameter from packaging.util.cache_from_source() (GH-142204) 2025-12-11 09:55:47 -08:00
Brett Cannon 880a7905ca GH-97850: Remove all uses and definitions of load_module() from importlib (#142205) 2025-12-10 15:35:51 -08:00
Stefano Rivera 656a64b37f gh-141930: Use the regular IO stack to write .pyc files for a better error message on failure (GH-141931)
* Use open() to write the bytecode
* Convert to unittest style asserts
* Tweak news, thanks @vstinner
* Tidy
* reword NEWS, avoid word "retried"
2025-11-27 19:17:59 +00:00
Guo Ci 453d886f85 GH-90344: replace single-call io.IncrementalNewlineDecoder usage with non-incremental newline decoders (GH-30276)
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
2025-11-15 00:13:37 +00:00
Serhiy Storchaka d8e6bdc0d0 gh-135801: Add the module parameter to compile() etc (GH-139652)
Many functions related to compiling or parsing Python code, such as
compile(), ast.parse(), symtable.symtable(),
and importlib.abc.InspectLoader.source_to_code() now allow to pass
the module name used when filtering syntax warnings.
2025-11-13 13:21:32 +02:00
Filipe Laíns ede5693be1 GH-119668: expose importlib.machinery.NamespacePath (#119669)
* GH-119668: expose importlib.NamespacePath

Signed-off-by: Filipe Laíns <lains@riseup.net>

* add news

Signed-off-by: Filipe Laíns <lains@riseup.net>

* add to docs

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Apply suggestions from code review

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

* Fix news (importlib.NamespacePath > importlib.machinery.NamespacePath)

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Link to module.__path__ in NamespacePath docs

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Mention the path argument in the documentation

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Simplify docs text

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Highlight argument names in docs text

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Update Lib/importlib/_bootstrap_external.py

Co-authored-by: Brett Cannon <brett@python.org>

* Rewrite NamespacePath's doc

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Specify path_finder's type in the NamespacePath docstring

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Fix doc tests

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Apply suggestions from code review

Co-authored-by: Barry Warsaw <barry@python.org>

* Fix doc lint

Signed-off-by: Filipe Laíns <lains@riseup.net>

* Update Doc/library/importlib.rst

Co-authored-by: Brett Cannon <brett@python.org>

---------

Signed-off-by: Filipe Laíns <lains@riseup.net>
Co-authored-by: Brett Cannon <brett@python.org>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Barry Warsaw <barry@python.org>
2025-11-01 00:39:48 +00:00
Petr Viktorin 3416e7c8dc gh-140633: AppleFrameworkLoader: Ignore AttributeError when setting __file__ (GH-140635) 2025-10-27 10:09:22 +01:00
Bartosz Sławecki bcced02604 gh-140141: Properly break exception chain in importlib.metadata.Distribution.from_name (#140142) 2025-10-15 12:49:14 -04:00
Semyon Moroz 8d17d79299 gh-138044: Remove deprecated parameter alias for importlib.resources.files (#138059) 2025-10-04 14:53:43 +01:00
Lisa Roach 2fd43a1ffe gh-138310: Adds sys.audit event for import_module (#138311)
* Updates sys.audit calls for imports to include import_module
* Adds unit tests for existing and new functionality
2025-09-19 06:21:42 -07:00
Jacob Walls 92b2a8a04d gh-93334: Fix homonym edge case in PathFinder.find_spec() (GH-98100) 2025-09-05 15:41:01 -07:00
Jason R. Coombs 9b38c6698a Reapply "gh-132947: Apply changes from importlib_metadata 8.7 (#137885)" (#137924) (#137930)
* Reapply "gh-132947: Apply changes from importlib_metadata 8.7 (#137885)" (#137924)

This reverts commit 3706ef66ef.

* Skip the triggering test on buildbots only.
2025-09-01 11:27:01 -04:00
Jason R. Coombs 3706ef66ef Revert "gh-132947: Apply changes from importlib_metadata 8.7 (#137885)" (#137924)
This reverts commit 5292fc00f2.
2025-08-18 20:57:36 +00:00
Jason R. Coombs 5292fc00f2 gh-132947: Apply changes from importlib_metadata 8.7 (#137885)
* Copied files from python/importlib_metadata@b67ac80c49.
2025-08-18 10:54:10 -07:00
Petr Viktorin d8a9466e29 bpo-38735: Don't fail when importing from / with sys.pycache_prefix set (GH-30456)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2025-08-18 13:53:01 +02:00
Jason R. Coombs ec4021c6d7 gh-120492: Sync importlib_metadata 8.2.0 (#124033)
* Sync with importlib_metadata 8.2.0

Removes deprecated behaviors, including support for `PackageMetadata.__getitem__` returning None for missing keys and Distribution subclasses not implementing abstract methods.
Prioritizes valid dists to invalid dists when retrieving by name (python/cpython/#120492). Adds SimplePath to `importlib.metadata.__all__`.

* Add blurb
2025-08-15 14:19:23 -07:00
Brett Cannon 7140b99b0d GH-137426: Remove code deprecation of importlib.abc.ResourceLoader (GH-137567)
Enough other classes in `importlib.abc` inherit from the class and the deprecation was to redirect people to `TraversableResources`. The documentation now makes it clear the class only exists for backwards compatibility.

---------

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-08-11 16:18:54 -07:00
sobolevn f7305a06c7 gh-115942: Add locked to several multiprocessing locks (#115944)
Co-authored-by: mpage <mpage@cs.stanford.edu>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-04-08 11:14:12 +03:00
Sam Gross 857bdba0ac gh-130094: Fix race conditions in importlib (gh-130101)
Entries may be added or removed from `sys.meta_path` concurrently. For
example, setuptools temporarily adds and removes the `distutils` finder from
the beginning of the list. The local copy ensures that we don't skip over any
entries.

Some packages modify `sys.modules` during import. For example, `collections`
inserts the entry for `collections.abc`  into `sys.modules` during import. We
need to ensure that we re-check `sys.modules` *after* the parent module is
fully initialized.
2025-02-18 18:02:42 -05:00
Tomas R. aa81a6f6e4 gh-97850: Update the deprecation warning of importlib.abc.Loader.load_module (GH-129855) 2025-02-11 11:04:16 -08:00
Alex Willmer a8dc6d6d44 gh-115911: Ignore PermissionError during import from cwd (#116131)
Ignore PermissionError when checking cwd during import

On macOS `getcwd(3)` can return EACCES if a path component isn't readable,
resulting in PermissionError. `PathFinder.find_spec()` now catches these and
ignores them - the same treatment as a missing/deleted cwd.

Introduces `test.support.os_helper.save_mode(path, ...)`, a context manager
that restores the mode of a path on exit.

This is allows finer control of exception handling and robust environment
restoration across platforms in `FinderTests.test_permission_error_cwd()`.

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Co-authored-by: Brett Cannon <brett@python.org>
2025-01-26 19:00:28 +00:00
Jason R. Coombs b543b32eff gh-123987: Fix NotADirectoryError in NamespaceReader when sentinel present (#124018) 2025-01-26 16:23:54 +00:00
Thomas Grainger d05140f9f7 gh-121604: fix ResourceLoader deprecation warning message (GH-128859) 2025-01-15 21:13:59 +00:00
Tomas R. bd3baa8b1a gh-121604: Make sure all deprecated items in importlib raise DeprecationWarning (#128007)
Co-authored-by: rashansmith <smith.rashan@gmail.com>
Co-authored-by: Stan Ulbrych <89152624+StanFromIreland@users.noreply.github.com>
Co-authored-by: Brett Cannon <brett@python.org>
2025-01-14 16:48:46 -08:00
CF Bolz-Tereick c695e37a3f GH-126606: don't write incomplete pyc files (GH-126627)
Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
Co-authored-by: Brett Cannon <brett@python.org>
2024-11-13 21:39:10 +00:00
Alex Waygood c5c21fee7a gh-125519: Improve traceback if importlib.reload() is called with a non-module object (#125520) 2024-10-21 07:53:21 +01:00
Jason R. Coombs a53812df12 gh-123085: Fix issue in inferred caller when resources package has no source (#123102)
gh-123085: Fix issue in inferred caller when resources package has no source.

From importlib_resources 6.4.3 (python/importlib_resources#314).
2024-09-12 19:08:06 +00:00
Jason R. Coombs ba687d9481 gh-121735: Fix module-adjacent references in zip files (#123037)
* gh-116608: Apply style and compatibility changes from importlib_metadata.

* gh-121735: Ensure module-adjacent resources are loadable from a zipfile.

* gh-121735: Allow all modules to be processed by the ZipReader.

* Add blurb

* Remove update-zips script, unneeded.

* Remove unnecessary references to removed static fixtures.

* Remove zipdata fixtures, unused.
2024-09-11 22:33:07 -04:00
abstractee 1f4a49ea53 Fix typos in warnings, docstrings, comments and text files (#123597) 2024-09-03 02:20:40 +02:00
Jason R. Coombs e913d2c87f gh-116608: Apply style and compatibility changes from importlib_metadata. (#123028) 2024-08-15 15:32:05 -04:00
Eric Snow ee1b8ce26e gh-122907: Fix Builds Without HAVE_DYNAMIC_LOADING Set (gh-122952)
As of 529a160 (gh-118204), building with HAVE_DYNAMIC_LOADING stopped working.  This is a minimal fix just to get builds working again.  There are actually a number of long-standing deficiencies with HAVE_DYNAMIC_LOADING builds that need to be resolved separately.
2024-08-13 14:44:57 -06:00
Serhiy Storchaka 03b88522f5 gh-122188: Remove _imp.pyc_magic_number (GH-122503)
_imp.pyc_magic_number_token should be enough.
2024-08-02 13:12:19 +03:00
Bénédikt Tran 88030861e2 gh-122555: Remove removed functions from Doc/data/refcounts.dat (#122556) 2024-08-01 12:26:09 +02:00