Commit Graph

8839 Commits

Author SHA1 Message Date
Mark Shannon 39981fd07a GH-118095: Make sure that progress is made if there are pending calls being handled. (GH-118484) 2024-05-01 22:18:31 +01:00
Irit Katriel 6763bfcc0f gh-118272: set stacktop to 0 before freeing contents, to avoid access to invalid objects during GC (#118478) 2024-05-01 21:51:40 +01:00
Hugo van Kemenade 3b3f8dea57 gh-117225: Move colorize functionality to own internal module (#118283) 2024-05-01 12:27:06 -06:00
Victor Stinner b52c753e0f gh-110850: Add PyTime_TimeRaw() function (#118394)
Add "Raw" variant of PyTime functions:

* PyTime_MonotonicRaw()
* PyTime_PerfCounterRaw()
* PyTime_TimeRaw()

Changes:

* Add documentation and tests. Tests release the GIL while calling
  raw clock functions.
* py_get_system_clock() and py_get_monotonic_clock() now check that
  the GIL is hold by the caller if raise_exc is non-zero.
* Reimplement "Unchecked" functions with raw clock functions.

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-05-01 18:05:01 +00:00
Brandt Bucher 49baa656cb GH-115802: Use the GHC calling convention in JIT code (GH-118287) 2024-05-01 08:05:53 -07:00
Anthony Shaw beb653cc24 gh-117958: Expose JIT code via method in UOpExecutor (#117959) 2024-05-01 07:11:14 -07:00
Irit Katriel c1bf4874c1 gh-116767: fix crash on 'async with' with many context managers (GH-118348)
Account for `add_stopiteration_handler` pushing a block for `async with`.
To allow generator functions that previously almost hit the `CO_MAXBLOCKS`
limit by nesting non-async blocks, the limit is increased by 1.
This increase allows one more block in non-generator functions.
2024-05-01 13:01:16 +02:00
Mark Shannon f6fab21721 GH-118095: Make invalidating and clearing executors memory safe (GH-118459) 2024-05-01 11:34:50 +01:00
Guido van Rossum 7d83f7bcc4 gh-118335: Configure Tier 2 interpreter at build time (#118339)
The code for Tier 2 is now only compiled when configured
with `--enable-experimental-jit[=yes|interpreter]`.

We drop support for `PYTHON_UOPS` and -`Xuops`,
but you can disable the interpreter or JIT
at runtime by setting `PYTHON_JIT=0`.
You can also build it without enabling it by default
using `--enable-experimental-jit=yes-off`;
enable with `PYTHON_JIT=1`.

On Windows, the `build.bat` script supports
`--experimental-jit`, `--experimental-jit-off`,
`--experimental-interpreter`.

In the C code, `_Py_JIT` is defined as before
when the JIT is enabled; the new variable
`_Py_TIER2` is defined when the JIT *or* the
interpreter is enabled. It is actually a bitmask:
1: JIT; 2: default-off; 4: interpreter.
2024-04-30 18:26:34 -07:00
Ken Jin 7fabcc727d gh-117657: Don't specialize RESUME_CHECK when specialization is disabled (GH-118349) 2024-04-30 21:51:59 +00:00
Victor Stinner e93c39b47e gh-118422: Fix run_fileexflags() test (#118429)
Don't test the undefined behavior of fileno()
on a closed file, but use fstat() as a reliable
test if the file was closed or not.
2024-04-30 22:32:55 +02:00
Sam Gross b2c3b70c71 gh-118332: Fix deadlock involving stop the world (#118412)
Avoid detaching thread state when stopping the world. When re-attaching
the thread state, the thread would attempt to resume the top-most
critical section, which might now be held by a thread paused for our
stop-the-world request.
2024-04-30 15:01:28 -04:00
Dino Viehland 4a1cf66c5c gh-117657: Fix small issues with instrumentation and TSAN (#118064)
Small TSAN fixups for instrumentation
2024-04-30 11:38:05 -07:00
Irit Katriel 1f16b4ce56 gh-118272: Clear generator frame's locals when the generator is closed (#118277)
Co-authored-by: Thomas Grainger <tagrain@gmail.com>
2024-04-30 19:32:25 +01:00
Nice Zombies 9a75d56d5d gh-118379: Use PyTuple_Pack instead of Py_BuildValue if possible (GH-118381) 2024-04-30 17:55:15 +03:00
Malcolm Smith 3b268f4edc gh-116622: Redirect stdout and stderr to system log when embedded in an Android app (#118063) 2024-04-30 16:00:31 +02:00
Mark Shannon 5b05d452cd GH-118095: Add tier 2 support for YIELD_VALUE (GH-118380) 2024-04-30 11:33:13 +01:00
Savannah Ostrowski 8b56d82c59 GH-118306: Update JIT to use LLVM 18 (GH-118307) 2024-04-29 21:09:16 +00:00
Alex Turner 2ba1aed596 gh-117657: TSAN fix race on gstate->young.count (#118313) 2024-04-29 20:26:26 +00:00
Eric Snow 529a160be6 gh-117953: Share More Machinery Code Between Builtin and Dynamic Extensions (gh-118204)
This change will make some later changes simpler. It also brings more consistent behavior and lower maintenance costs.
2024-04-29 12:53:04 -06:00
Sam Gross 7ccacb220d gh-117783: Immortalize objects that use deferred reference counting (#118112)
Deferred reference counting is not fully implemented yet. As a temporary
measure, we immortalize objects that would use deferred reference
counting to avoid multi-threaded scaling bottlenecks.

This is only performed in the free-threaded build once the first
non-main thread is started. Additionally, some tests, including refleak
tests, suppress this behavior.
2024-04-29 14:36:02 -04:00
Eric Snow 44f57a952e gh-117953: Split Up _PyImport_LoadDynamicModuleWithSpec() (gh-118203)
Basically, I've turned most of _PyImport_LoadDynamicModuleWithSpec() into two new functions (_PyImport_GetModInitFunc() and _PyImport_RunModInitFunc()) and moved the rest of it out into _imp_create_dynamic_impl().  There shouldn't be any changes in behavior.

This change makes some future changes simpler.  This is particularly relevant to potentially calling each module init function in the main interpreter first.  Thus the critical part of the PR is the addition of _PyImport_RunModInitFunc(), which is strictly focused on running the init func and validating the result.  A later PR will take it a step farther by capturing error information rather than raising exceptions.

FWIW, this change also helps readers by clarifying a bit more about what happens when an extension/builtin module is imported.
2024-04-29 09:29:07 -06:00
Tian Gao 375c94c75d gh-107674: Lazy load line number to improve performance of tracing (GH-118127) 2024-04-29 09:54:52 +01:00
Mark Shannon ab6eda0ee5 GH-118095: Allow a variant of RESUME_CHECK in tier 2 (GH-118286) 2024-04-29 07:54:05 +01:00
Jelle Zijlstra 2326d6c868 gh-109118: Make comprehensions work within annotation scopes, but without inlining (#118160)
Co-authored-by: Carl Meyer <carl@oddbird.net>
2024-04-28 06:21:28 -07:00
Sam Gross 5a90de0d4c gh-116749: Disable GIL by default in free-threaded build (#118295)
Switch GIL to disabled by default in free-threaded build so that the
free-threaded CIs catch thread-safety issues.
2024-04-26 14:22:29 -04:00
Mark Shannon 3e06c7f719 GH-118095: Add dynamic exit support and FOR_ITER_GEN support to tier 2 (GH-118279) 2024-04-26 18:08:50 +01:00
neonene 63add11704 gh-117680: Fix msvc warning in instruction_sequence.c (#118326) 2024-04-26 17:20:30 +01:00
Tian Gao 0f99861332 gh-117385: Remove unhooked events on sys.settrace (GH-117386) 2024-04-26 17:01:44 +01:00
Eric Snow 09c2947581 gh-110693: Pending Calls Machinery Cleanups (gh-118296)
This does some cleanup in preparation for later changes.
2024-04-26 01:05:51 +00:00
Mark Shannon f180b31e76 GH-118095: Handle RETURN_GENERATOR in tier 2 (GH-118180) 2024-04-25 11:32:47 +01:00
Nice Zombies 10bb90ed49 gh-102511: Speed up os.path.splitroot() with native helpers (GH-118089) 2024-04-25 10:07:38 +01:00
Eric Snow 85ec1c2dc6 gh-117953: Fix Refleaks Introduced by gh-118194 (gh-118250)
A couple of refleaks slipped through in gh-118194. This takes care of them.

(AKA _Py_ext_module_loader_info_init() does not steal references.)
2024-04-24 21:23:45 +00:00
Eric Snow 5865fa5f9b gh-117953: Add Internal struct _Py_ext_module_loader_info (gh-118194)
This helps with a later change that splits up _PyImport_LoadDynamicModuleWithSpec().
2024-04-24 17:42:01 +00:00
Eric Snow 1acd249798 gh-117953: Let update_global_state_for_extension() Caller Decide If Singlephase or Not (gh-118193)
This change makes other upcoming changes simpler.
2024-04-24 16:28:35 +00:00
Eric Snow 03e3e31723 gh-76785: Rename _xxsubinterpreters to _interpreters (gh-117791)
See https://discuss.python.org/t/pep-734-multiple-interpreters-in-the-stdlib/41147/26.
2024-04-24 16:18:24 +00:00
Eric Snow af3c1d817d gh-117953: Cleanups For fix_up_extension() in import.c (gh-118192)
These are cleanups I've pulled out of gh-118116.  Mostly, this change moves code around to align with some future changes and to improve clarity a little.  There is one very small change in behavior: we now add the module to the per-interpreter caches after updating the global state, rather than before.
2024-04-24 09:55:48 -06:00
Mark Shannon 83235f7791 GH-115419: Move setting the instruction pointer to error exit stubs (GH-118088) 2024-04-24 14:41:30 +01:00
Mark Shannon 77cd0428b6 GH-118095: Convert DEOPT_IFs on likely side exits to EXIT_IFs (GH-118106)
Covert DEOPT_IFs on likely side exits to EXIT_IFs
2024-04-24 14:37:55 +01:00
Irit Katriel 0aa0fc3d3c gh-117901: Add option for compiler's codegen to save nested instruction sequences for introspection (#118007) 2024-04-24 09:46:17 +00:00
Guido van Rossum 7e87d30f1f gh-118074: Immortal executors are not GC-able (#118182)
Better version of gh-118117.
Just check for immortality instead of an address range check.
2024-04-23 13:38:23 -07:00
mpage 2e7771a03d gh-117657: Quiet TSAN warnings about remaining non-atomic accesses of tstate->state (#118165)
Quiet TSAN warnings about remaining non-atomic accesses of `tstate->state`
2024-04-23 10:20:14 -07:00
Eric Snow 23950beff8 gh-117953: Small Cleanup of Extensions-Related Machinery Code (gh-118167)
This is a collection of very basic cleanups I've pulled out of gh-118116.  It is mostly renaming variables and moving a couple bits of code in functionally equivalent ways.
2024-04-23 08:25:50 -06:00
Joe Jevnik eb927e9fc8 gh-68114: Fix handling for removed PyArg_ParseTuple 'w' formatters (GH-8204)
Co-authored-by: Joe Jevnik <joe@quantopian.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
2024-04-23 13:15:15 +02:00
Guido van Rossum 1b85b3424c GH-118074: Executors in the COLD_EXITS array are not GC'able (#118117) 2024-04-22 16:20:39 -07:00
Jelle Zijlstra 85f727c5fb gh-109118: Allow lambdas in annotation scopes in classes (#118019) 2024-04-22 12:50:26 -07:00
Mark Shannon a6647d16ab GH-115480: Reduce guard strength for binary ops when type of one operand is known already (GH-118050) 2024-04-22 13:34:06 +01:00
Dino Viehland 8b541c017e gh-112075: Make instance attributes stored in inline "dict" thread safe (#114742)
Make instance attributes stored in inline "dict" thread safe on free-threaded builds
2024-04-21 22:57:05 -07:00
Dino Viehland 07525c9a85 gh-116818: Make sys.settrace, sys.setprofile, and monitoring thread-safe (#116775)
Makes sys.settrace, sys.setprofile, and monitoring generally thread-safe.

Mostly uses a stop-the-world approach and synchronization around the code object's _co_instrumentation_version.  There may be a little bit of extra synchronization around the monitoring data that's required to be TSAN clean.
2024-04-19 14:47:42 -07:00
Russell Keith-Magee ab99438900 gh-114099: Modify preprocessor symbol usage to support older macOS SDKs (GH-118073)
Co-authored-by: Joshua Root jmr@macports.org
2024-04-19 14:56:33 -04:00