Commit Graph

9464 Commits

Author SHA1 Message Date
Bénédikt Tran f693f84227 gh-129173: simplify PyCodec_XMLCharRefReplaceErrors logic (#129894)
Writing the decimal representation of a Unicode codepoint only requires to know the number of digits.

---------

Co-authored-by: Petr Viktorin <encukou@gmail.com>
2025-03-03 11:43:22 +00:00
Ned Batchelder efbc5929ca docs: internal notes have moved, correct references (#130762) 2025-03-03 06:38:49 -05:00
Brandt Bucher 7afa476874 GH-130415: Use boolean guards to narrow types to values in the JIT (GH-130659) 2025-03-02 13:21:34 -08:00
Hugo Beauzée-Luyssen 214562ed4d gh-130740: Move some stdbool.h includes after Python.h (#130738)
Move some `#include <stdbool.h>` after `#include "Python.h"` when `pyconfig.h` is not
included first and when we are in a platform-agnostic context. This is to avoid having
features defined by `stdbool.h` before those decided by `Python.h`.
2025-03-02 09:56:49 +00:00
Mark Shannon 54965f3fb2 GH-130296: Avoid stack transients in four instructions. (GH-130310)
* Combine _GUARD_GLOBALS_VERSION_PUSH_KEYS and _LOAD_GLOBAL_MODULE_FROM_KEYS into _LOAD_GLOBAL_MODULE

* Combine _GUARD_BUILTINS_VERSION_PUSH_KEYS and _LOAD_GLOBAL_BUILTINS_FROM_KEYS into _LOAD_GLOBAL_BUILTINS

* Combine _CHECK_ATTR_MODULE_PUSH_KEYS and _LOAD_ATTR_MODULE_FROM_KEYS into _LOAD_ATTR_MODULE

* Remove stack transient in LOAD_ATTR_WITH_HINT
2025-02-28 18:00:38 +00:00
Sam Gross 038e4d606b gh-130605: Use relaxed atomics to set the GIL switch interval (gh-130654)
The interval may be concurrently read by a thread attempting to acquire
the GIL.
2025-02-28 09:27:18 -05:00
Hugo Beauzée-Luyssen 830f04b505 Postpone <stdbool.h> inclusion after Python.h (#130641)
Remove inclusions prior to Python.h.

<stdbool.h> will cause <features.h> to be included before Python.h can
define some macros to enable some additional features, causing multiple
types not to be defined down the line.
2025-02-28 10:09:27 +01:00
Petr Viktorin fecf8bc8f2 gh-130595: Fix leak in WITH_EXCEPT_START error case (GH-130626)
Co-authored-by: Ken Jin <kenjin@python.org>
2025-02-28 08:58:50 +00:00
Sam Gross 7aeaa5af2c gh-130091: Reorder _PyThreadState_Attach to avoid data race (gh-130092)
This moves `tstate_activate()` down to avoid a data race in the free
threading build on the `_PyRuntime`'s thread-local `autoTSSkey`. This
key is deleted during runtime finalization, which may happen
concurrently with a call to `_PyThreadState_Attach`.

The earlier `tstate_try/wait_attach` ensures that the thread is blocked
before it attempts to access the deleted `autoTSSkey`.

This fixes a TSAN reported data race in
`test_threading.test_import_from_another_thread`.
2025-02-27 13:57:19 -05:00
Sam Gross d027787c8d gh-130421: Fix data race on timebase initialization (gh-130592)
Windows and macOS require precomputing a "timebase" in order to convert
OS timestamps into nanoseconds. Retrieve and compute this value during
runtime initialization to avoid data races when accessing the time.
2025-02-27 13:27:54 +00:00
Mark Shannon 2a18e80695 GH-128534: Instrument branches for async for loops. (GH-130569) 2025-02-27 09:36:41 +00:00
Victor Stinner 05aba4e799 gh-111178: Fix function signatures in instruction_sequence.c (#130591) 2025-02-26 21:36:26 +01:00
Victor Stinner 1b635d86cd gh-111178: Fix function signatures in symtable.c (#130589) 2025-02-26 21:35:24 +01:00
Dino Viehland 5c8e8704c3 gh-130595: Keep traceback alive for WITH_EXCEPT_START (#130562)
Keep traceback alive for WITH_EXCEPT_START
2025-02-26 10:41:26 -08:00
Serhiy Storchaka 0ef4ffeefd gh-130163: Fix crashes related to PySys_GetObject() (GH-130503)
The use of PySys_GetObject() and _PySys_GetAttr(), which return a borrowed
reference, has been replaced by using one of the following functions, which
return a strong reference and distinguish a missing attribute from an error:
_PySys_GetOptionalAttr(), _PySys_GetOptionalAttrString(),
_PySys_GetRequiredAttr(), and _PySys_GetRequiredAttrString().
2025-02-25 23:04:27 +02:00
Tian Gao c5f925c8c9 gh-122029: Move monitoring after method expand for CALL_KW (GH-130488) 2025-02-25 15:43:49 +00:00
Sergey B Kirpichev f39a07be47 gh-87790: support thousands separators for formatting fractional part of floats (#125304)
```pycon
>>> f"{123_456.123_456:_._f}"  # Whole and fractional
'123_456.123_456'
>>> f"{123_456.123_456:_f}"    # Integer component only
'123_456.123456'
>>> f"{123_456.123_456:._f}"   # Fractional component only
'123456.123_456'
>>> f"{123_456.123_456:.4_f}"  # with precision
'123456.1_235'
```
2025-02-25 16:27:07 +01:00
Bénédikt Tran fa6a8140dd gh-129173: refactor PyCodec_ReplaceErrors into separate functions (#129893)
The logic of `PyCodec_ReplaceErrors` is now split into separate functions,
each of which handling a specific exception type.
2025-02-25 14:24:46 +01:00
Mark Shannon 014223649c GH-130396: Use computed stack limits on linux (GH-130398)
* Implement C recursion protection with limit pointers for Linux, MacOS and Windows

* Remove calls to PyOS_CheckStack

* Add stack protection to parser

* Make tests more robust to low stacks

* Improve error messages for stack overflow
2025-02-25 09:24:48 +00:00
Petr Viktorin ef29104f7d GH-91079: Revert "GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)" for now (GH130413)
Revert "GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)" for now

Unfortunatlely, the change broke some buildbots.

This reverts commit 2498c22fa0.
2025-02-24 11:16:08 +01:00
Łukasz Langa 72ea3c030a gh-128627: Skip wasm-gc on iOS Safari where it's broken (#130418)
As of iOS 18.3.1, enabling wasm-gc breaks the interpreter. This disables the wasm-gc
trampoline on iOS.

Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com>
2025-02-24 07:26:04 +08:00
Yan Yanchii 38642bff13 gh-126835: Move constant unaryop & binop folding to CFG (#129550) 2025-02-21 17:54:22 +00:00
Sam Gross ca22147547 gh-111924: Fix data races when swapping allocators (gh-130287)
CPython current temporarily changes `PYMEM_DOMAIN_RAW` to the default
allocator during initialization and shutdown. The motivation is to
ensure that core runtime structures are allocated and freed using the
same allocator. However, modifying the current allocator changes global
state and is not thread-safe even with the GIL. Other threads may be
allocating or freeing objects use PYMEM_DOMAIN_RAW; they are not
required to hold the GIL to call PyMem_RawMalloc/PyMem_RawFree.

This adds new internal-only functions like `_PyMem_DefaultRawMalloc`
that aren't affected by calls to `PyMem_SetAllocator()`, so they're
appropriate for Python runtime initialization and finalization. Use
these calls in places where we previously swapped to the default raw
allocator.
2025-02-20 11:31:15 -05:00
Bénédikt Tran e24a1ac17c gh-129173: Use _PyUnicodeError_GetParams in PyCodec_SurrogateEscapeErrors (GH-129175) 2025-02-20 13:18:47 +00:00
Mark Shannon 2498c22fa0 GH-91079: Implement C stack limits using addresses, not counters. (GH-130007)
* Implement C recursion protection with limit pointers

* Remove calls to PyOS_CheckStack

* Add stack protection to parser

* Make tests more robust to low stacks

* Improve error messages for stack overflow
2025-02-19 11:44:57 +00:00
Pablo Galindo Salgado 51d4bf1e0e bpo-45325: Add a new 'p' parameter to Py_BuildValue to convert an integer into a Python bool (#28634) 2025-02-18 17:14:11 +00:00
Ken Jin 46ac85e4d9 gh-129989: Change Py_TAIL_CALL_INTERP ifndef to ! (#130269)
Change Py_TAIL_CALL_INTERP ifndef to !
2025-02-18 15:48:49 +00:00
Irit Katriel c9b1bf302c gh-130139: always check ast node type in ast.parse() with ast input (#130140) 2025-02-16 13:32:39 +00:00
Ken Jin 359c7dde3b gh-129989: Properly disable tailcall interp in configure (GH-129991)
Co-authored-by: Zanie Blue <contact@zanie.dev>
2025-02-16 03:01:24 +08:00
Bénédikt Tran 1775091dc1 gh-129173: Use _PyUnicodeError_GetParams in PyCodec_SurrogatePassErrors (GH-129134) 2025-02-14 18:34:32 +01:00
Yan Yanchii 334589f619 gh-126835: Set location for noped out instructions after constant folding in CFG. (#130109) 2025-02-14 14:15:08 +00:00
Sam Gross 451f291baa gh-128130: Fix unhandled keyboard interrupt data race (gh-129975)
Use an atomic operation when setting
`_PyRuntime.signals.unhandled_keyboard_interrupt`. We now only clear the
variable at the start of `_PyRun_Main`, which is the same function where
we check it.

This avoids race conditions where previously another thread might call
`run_eval_code_obj()` and erroneously clear the unhandled keyboard
interrupt.
2025-02-13 12:29:03 -05:00
Sam Gross 0559339ccd gh-130019: Fix data race in _PyType_AllocNoTrack (gh-130058)
The reference count fields, such as `ob_tid` and `ob_ref_shared`, may be
accessed concurrently in the free threading build by a `_Py_TryXGetRef`
or similar operation. The PyObject header fields will be initialized by
`_PyObject_Init`, so only call `memset()` to zero-initialize the remainder
of the allocation.
2025-02-13 11:50:45 -05:00
Yan Yanchii 140e69c4a8 gh-126835: Move const folding of lists & sets from ast_opt.c to flowgraph.c (#130032) 2025-02-13 12:11:07 +00:00
Sam Gross e09442089e gh-130030: Fix crash on 32-bit Linux with free threading (gh-130043)
The `gc_get_refs` assertion needs to be after we check the alive and
unreachable bits. Otherwise, `ob_tid` may store the actual thread id
instead of the computed `gc_refs`, which may trigger the assertion if
the `ob_tid` looks like a negative value.

Also fix a few type warnings on 32-bit systems.
2025-02-12 18:09:15 -05:00
Ken Jin 1b27f36eb0 gh-129819: Allow tier2/JIT and tailcall (GH-129820) 2025-02-13 02:18:36 +08:00
Brandt Bucher 11bb08e4ec GH-129715: Don't project traces that return to an unknown caller (GH-130024) 2025-02-12 10:16:43 -08:00
Mark Shannon 72f56654d0 GH-128682: Account for escapes in DECREF_INPUTS (GH-129953)
* Handle escapes in DECREF_INPUTS

* Mark a few more functions as escaping

* Replace DECREF_INPUTS with PyStackRef_CLOSE where possible
2025-02-12 17:44:59 +00:00
Andrew Svetlov 469d2e416c gh-129889: Support context manager protocol by contextvars.Token (#129888) 2025-02-12 12:32:58 +01:00
Stefano Rivera e1b38ea82e Update manpage environment variables and command line arguments (#129623)
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
2025-02-12 13:00:35 +02:00
Michael Droettboom 00ec781877 gh-129244: Only remove the workaround when MSVC has the bugfix (#130011) 2025-02-11 14:49:42 -05:00
Ken Jin 5cdd6e5e75 gh-130004: Disable PGO for ceval.c on MSVC for default build (GH-130009)
Disable PGO for ceval.c on MSVC on default build
2025-02-12 01:13:05 +08:00
Ken Jin 247b50dec8 gh-130004: Revert commit 9e52e55 (GH-130005)
Revert commit 9e52e55
2025-02-11 23:23:58 +08:00
Yan Yanchii 91d9544112 gh-126835: Make CFG optimizer skip over NOP's when looking for const sequence construction (#129703)
Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2025-02-09 18:00:41 +00:00
Bénédikt Tran a56ead089c gh-129173: Use _PyUnicodeError_GetParams in PyCodec_NameReplaceErrors (GH-129135) 2025-02-08 16:01:57 +01:00
Irit Katriel a1417b211f gh-100239: replace BINARY_SUBSCR & family by BINARY_OP with oparg NB_SUBSCR (#129700) 2025-02-07 22:39:54 +00:00
Brandt Bucher 5fa7e1b7fd GH-129715: Remove _DYNAMIC_EXIT (GH-129716) 2025-02-07 11:41:17 -08:00
Brandt Bucher 70e387c990 GH-129709: Clean up tier two (GH-129710) 2025-02-07 09:52:49 -08:00
Brandt Bucher fbaa6c8ff0 GH-129763: Remove the LLTRACE macro (GH-129764) 2025-02-07 08:49:51 -08:00
Ken Jin 175844713a gh-128563: Move assignment of opcode into ifdef (GH-129803) 2025-02-07 21:11:57 +08:00