When someone adds a new type but doesn't increment
`_Py_MAX_MANAGED_STATIC_BUILTIN_TYPES` or
`_Py_MAX_MANAGED_STATIC_EXT_TYPES`, JIT tests fail,
because JIT builds define an extra type.
But the JIT tests don't necessarily run for the commit
that causes the failure.
As a workaround, use the same size for the array for all
builds, potentially with an empty spot.
Avoid racing with the owning thread's refcount operations when
immortalizing an interned string: if we don't own it and its refcount
isn't merged, intern a copy we own instead. Use atomic stores in
_Py_SetImmortalUntracked so concurrent atomic reads are race-free.
Fix free-threading scaling bottleneck in sys.intern and `PyObject_SetAttr` by
avoiding the interpreter-wide lock when the string is already interned and
immortalized.
Add special cases for classmethod and staticmethod descriptors in
_PyObject_GetMethodStackRef() to avoid calling tp_descr_get, which
avoids reference count contention on the bound method and underlying
callable. This improves scaling when calling classmethods and
staticmethods from multiple threads.
Also refactor method_vectorcall in classobject.c into a new _PyObject_VectorcallPrepend() helper so that it can be used by
PyObject_VectorcallMethod as well.
Add TYPE_FROZENDICT to the marshal module.
Add C API functions:
* PyAnyDict_Check()
* PyAnyDict_CheckExact()
* PyFrozenDict_Check()
* PyFrozenDict_CheckExact()
* PyFrozenDict_New()
Add PyFrozenDict_Type C type.
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Adam Johnson <me@adamj.eu>
Co-authored-by: Benedikt Johannes <benedikt.johannes.hofer@gmail.com>
Add `_Py_type_getattro_stackref`, a variant of type attribute lookup
that returns `_PyStackRef` instead of `PyObject*`. This allows returning
deferred references in the free-threaded build, reducing reference count
contention when accessing type attributes.
This significantly improves scaling of namedtuple instantiation across
multiple threads.
* Add blurb
* Rename PyObject_GetAttrStackRef to _PyObject_GetAttrStackRef
* Apply suggestion from @vstinner
Co-authored-by: Victor Stinner <vstinner@python.org>
* Apply suggestion from @vstinner
Co-authored-by: Victor Stinner <vstinner@python.org>
* format
* Update Include/internal/pycore_function.h
Co-authored-by: Victor Stinner <vstinner@python.org>
---------
Co-authored-by: Victor Stinner <vstinner@python.org>
Move classmethod and staticmethod initialization from __init__() to
__new__().
PyClassMethod_New() and PyStaticMethod_New() now copy attributes of
the wrapped functions: __module__, __name__, __qualname__ and
__doc__.
Change static type initialization: initialize PyStaticMethod_Type and
PyCFunction_Type earlier.
Remove test_refleaks_in_classmethod___init__() and
test_refleaks_in_staticmethod___init__() tests from test_descr since
classmethod and staticmethod have no __init__() method anymore.
* Make Py_{SIZE,IS_TYPE,SET_SIZE} regular functions in stable ABI
Group them together with Py_TYPE & Py_SET_TYPE to cut down
on repetitive preprocessor macros.
Format repetitive definitions in object.c more concisely.
Py_SET_TYPE is still left out of the Limited API.
* Promote _PyObject_Dump() as a public function.
* Keep _PyObject_Dump() alias to PyUnstable_Object_Dump()
for backward compatibility.
* Replace _PyObject_Dump() with PyUnstable_Object_Dump().
Co-authored-by: Peter Bierma <zintensitydev@gmail.com>
Co-authored-by: Kumar Aditya <kumaraditya@python.org>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
For several builtin functions, we now fall back to __main__.__dict__ for the globals
when there is no current frame and _PyInterpreterState_IsRunningMain() returns
true. This allows those functions to be run with Interpreter.call().
The affected builtins:
* exec()
* eval()
* globals()
* locals()
* vars()
* dir()
We take a similar approach with "stateless" functions, which don't use any
global variables.
Adds `_PyObject_GetMethodStackRef` which uses stackrefs and takes advantage of deferred reference counting in free-threading while calling method objects in vectorcall.
After gh-130704, the interpreter replaces some uses of `LOAD_FAST` with
`LOAD_FAST_BORROW` which avoid incref/decrefs by "borrowing" references
on the interpreter stack when the bytecode compiler can determine that
it's safe.
This change broke some checks in C API extensions that relied on
`Py_REFCNT()` of `1` to determine if it's safe to modify an object
in-place. Objects may have a reference count of one, but still be
referenced further up the interpreter stack due to borrowing of
references.
This provides a replacement function for those checks.
`PyUnstable_Object_IsUniqueReferencedTemporary` is more conservative:
it checks that the object has a reference count of one and that it exists as a
unique strong reference in the interpreter's stack of temporary
variables in the top most frame.
See also:
* https://github.com/numpy/numpy/issues/28681
Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>
Co-authored-by: T. Wouters <thomas@python.org>
Co-authored-by: mpage <mpage@cs.stanford.edu>
Co-authored-by: Mark Shannon <mark@hotpy.org>
Co-authored-by: Victor Stinner <vstinner@python.org>
In the free threaded build, the `_PyObject_LookupSpecial()` call can lead to
reference count contention on the returned function object becuase it
doesn't use stackrefs. Refactor some of the callers to use
`_PyObject_MaybeCallSpecialNoArgs`, which uses stackrefs internally.
This fixes the scaling bottleneck in the "lookup_special" microbenchmark
in `ftscalingbench.py`. However, the are still some uses of
`_PyObject_LookupSpecial()` that need to be addressed in future PRs.
* Add location information when accessing already closed stackref
* Add #def option to track closed stackrefs to provide precise information for use after free and double frees.
Victor Stinner