* Add FOR_ITER_VIRTUAL to specialize FOR_ITER for virtual iterators
* Add GET_ITER_SELF to specialize GET_ITER for iterators (including generators)
* Add GET_ITER_VIRTUAL to specialize GET_ITER for iterables as virtual iterators
* Add new (internal) _tp_iteritem function slot to PyTypeObject
* Put limited RESUME at start of genexpr for free-threading. Fix up exception handling in genexpr
There are newly documented restrictions on tp_traverse:
The traversal function must not have any side effects.
It must not modify the reference counts of any Python
objects nor create or destroy any Python objects.
* Add several functions that are guaranteed side-effect-free,
with a _DuringGC suffix.
* Use these in ctypes
* Consolidate tp_traverse docs in gcsupport.rst, moving unique
content from typeobj.rst there
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
When compiling for abi3t, define Py_GIL_DISABLED, so that users who
check it to enable additional locking aren't broken.
But also avoid using Py_GIL_DISABLED in Python headers themselves
-- abi3 and abi3t ought to be the same except
the _Py_OPAQUE_PYOBJECT differences.
A check for this is coming in a later PR.
It will require rewriting some preprocessor conditions, some of these
changes are included in this PR.
For _Py_IsOwnedByCurrentThread & supporting functions
I opted to move them to a cpython/ header, as they're rather self-contained.
* Make Py_{SIZE,IS_TYPE,SET_SIZE} regular functions in stable ABI
Group them together with Py_TYPE & Py_SET_TYPE to cut down
on repetitive preprocessor macros.
Format repetitive definitions in object.c more concisely.
Py_SET_TYPE is still left out of the Limited API.
We already use an anonymous union for PyObject. This makes the workarounds available in all public headers:
- MSVC: `__pragma(warning(disable: 4201))` (with push/pop). Warning 4201 is specifically for anonymous unions, so let's disable for all of `<Python.h>`
- GCC/clang, pedantic old C standards: define `_Py_ANONYMOUS` as `__extension__`
- otherwise, define `_Py_ANONYMOUS` as nothing
(Note that this is only for public headers -- CPython internals use C11, which has anonymous structs/unions.)
C API WG vote: https://github.com/capi-workgroup/decisions/issues/74
Allow Py_LIMITED_API for (Py_GIL_DISABLED && _Py_OPAQUE_PYOBJECT)
API that's removed when _Py_OPAQUE_PYOBJECT is defined:
- PyObject_HEAD
- _PyObject_EXTRA_INIT
- PyObject_HEAD_INIT
- PyObject_VAR_HEAD
- struct _object (i.e. PyObject) (opaque)
- struct PyVarObject (opaque)
- Py_SIZE
- Py_SET_TYPE
- Py_SET_SIZE
- PyModuleDef_Base (opaque)
- PyModuleDef_HEAD_INIT
- PyModuleDef (opaque)
- _Py_IsImmortal
- _Py_IsStaticImmortal
Note that the `_Py_IsImmortal` removal (and a few other issues)
means _Py_OPAQUE_PYOBJECT only works with limited
API 3.14+ now.
Co-authored-by: Victor Stinner <vstinner@python.org>
* Replace _Py_ALIGN_AS(V) by _Py_ALIGNED_DEF(N, T)
This is now a common façade for the various `_Alignas` alternatives,
which behave in interesting ways -- see the source comment.
The new macro (and MSVC's `__declspec(align)`) should not be used
on a variable/member declaration that includes a struct declaraton.
A workaround is to separate the struct definition.
Do that for `PyASCIIObject.state`.
* Specify minimum PyGC_Head and PyObject alignment
As documented in InternalDocs/garbage_collector.md, the garbage collector
stores flags in the least significant two bits of the _gc_prev pointer
in struct PyGC_Head. Consequently, this pointer is only capable of storing
a location that's aligned to a 4-byte boundary.
Encode this requirement using _Py_ALIGNED_DEF.
This patch fixes a segfault in m68k, which was previously investigated
by Adrian Glaubitz here:
https://lists.debian.org/debian-68k/2024/11/msg00020.htmlhttps://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1087600
Original patch (using the GCC-only Py_ALIGNED) by Finn Thain.
Co-authored-by: Finn Thain <fthain@linux-m68k.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: John Paul Adrian Glaubitz <glaubitz@physik.fu-berlin.de>
Fix Py_RETURN_NONE, Py_RETURN_TRUE and Py_RETURN_FALSE macros in the
limited C API 3.11 and older:
Don't treat Py_None, Py_True and Py_False as immortal.
In the free-threaded build, avoid data races caused by updating type
slots or type flags after the type was initially created. For those
(typically rare) cases, use the stop-the-world mechanism. Remove the
use of atomics when reading or writing type flags.
In the free-threaded build, avoid data races caused by updating type slots
or type flags after the type was initially created. For those (typically
rare) cases, use the stop-the-world mechanism. Remove the use of atomics
when reading or writing type flags. The use of atomics is not sufficient to
avoid races (since flags are sometimes read without a lock and without
atomics) and are no longer required.
Disable pedantic check for c++03 (unlimited API)
Also add a check for c++03 *limited* API, which passes in pedantic mode
after removing a comma in the `PySendResult` declaration, and allowing
`long long`.
On Arm v5 it is not possible to get the thread ID via c13 register
hence the illegal instruction. The c13 register started to provide
thread ID since Arm v6K architecture variant. Other variants of
Arm v6 (T2, Z and base) don’t provide the thread ID via c13.
For the sake of simplicity we group v5 and v6 together and
consider that instructions for Arm v7 only.
Add Py_GetConstant() and Py_GetConstantBorrowed() functions.
In the limited C API version 3.13, getting Py_None, Py_False,
Py_True, Py_Ellipsis and Py_NotImplemented singletons is now
implemented as function calls at the stable ABI level to hide
implementation details. Getting these constants still return borrowed
references.
Add _testlimitedcapi/object.c and test_capi/test_object.py to test
Py_GetConstant() and Py_GetConstantBorrowed() functions.
* gh-112529: Remove PyGC_Head from object pre-header in free-threaded build
This avoids allocating space for PyGC_Head in the free-threaded build.
The GC implementation for free-threaded CPython does not use the
PyGC_Head structure.
* The trashcan mechanism uses the `ob_tid` field instead of `_gc_prev`
in the free-threaded build.
* The GDB libpython.py file now determines the offset of the managed
dict field based on whether the running process is a free-threaded
build. Those are identified by the `ob_ref_local` field in PyObject.
* Fixes `_PySys_GetSizeOf()` which incorrectly incorrectly included the
size of `PyGC_Head` in the size of static `PyTypeObject`.
* gh-112529: Implement GC for free-threaded builds
This implements a mark and sweep GC for the free-threaded builds of
CPython. The implementation relies on mimalloc to find GC tracked
objects (i.e., "containers").
Petr Viktorin