Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds.
This PR comes fresh from a pile of work done in our private PSRT security response team repo.
This backports https://github.com/python/cpython/pull/96499 aka 511ca94520
Signed-off-by: Christian Heimes [Red Hat] <christian@python.org>
Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org>
Reviews via the private PSRT repo via many others (see the NEWS entry in the PR).
<!-- gh-issue-number: gh-95778 -->
* Issue: gh-95778
<!-- /gh-issue-number -->
I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#).
They support now splitting escape sequences between input chunks.
Add the third parameter "final" in codecs.raw_unicode_escape_decode().
It is True by default to match the former behavior.
(cherry picked from commit 39aa98346d)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
They support now splitting escape sequences between input chunks.
Add the third parameter "final" in codecs.unicode_escape_decode().
It is True by default to match the former behavior.
(cherry picked from commit c96d1546b1)
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Fix PyAiter_Check to only check for the `__anext__` presense (not for
`__aiter__`). Rename `PyAiter_Check()` to `PyAIter_Check()`,
`PyObject_GetAiter()` -> `PyObject_GetAIter()`.
I tried to be relatively thorough and give lots of links.
One reason is that this wasn't deprecated very long; also it seems people running into this tend to not be familiar with similar APIs.
(cherry picked from commit 29987f7265)
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
* Remove struct _node from the stable ABI list
This struct was removed along with the old parser in Python 3.9 (PEP 617)
* Stable ABI list: Use the public name "PyFrameObject" rather than "_frame"
* Ensure limited API doesn't contain private names
Names prefixed by an underscore are private by definition.
* Add a blurb
(cherry picked from commit 7cad9cb51b)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Py_FrozenMain was added to the Limited C API in [bpo-42591]() (3.10.0a4);
but to fix that issue it would be enough to add it to the regular C API.
The function is undocumented, tests were added very recently ([bpo-44131]()),
and most importantly, it is not present in all builds of Python, as
the linker sometimes omits it as unused.
It should be added back when these issues are fixed.
Note that this does not affect Python's regular C API.
(cherry picked from commit d16856960e)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
- Reformat the C API and ABI Versioning page (and extend/clarify a bit)
- Rewrite the stable ABI docs into a general text on C API Compatibility
- Add a list of Limited API contents, and notes for the individual items.
- Replace `Include/README.rst` with a link to a devguide page with the same info
(cherry picked from commit b05955d6f5)
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Co-authored-by: Petr Viktorin <encukou@gmail.com>
Add new C-API functions to control the state of the garbage collector:
PyGC_Enable(), PyGC_Disable(), PyGC_IsEnabled(),
corresponding to the functions in the gc module.
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Victor Stinner <vstinner@python.org>
- `_Py_EncodeLocaleRaw`, which is private by name, undocumented,
and wasn't exported in `python3.dll`, is moved to a private header.
- `_Py_HashSecret_Initialized`, again private by name, undocumented,
and not exported in `python3.dll`, is excluded with `Py_LIMITED_API`.
- `PyMarshal_*` and `PyMember_*One` functions, declared in private headers and
not exported in `python3.dll`, are removed from `Doc/data/stable_abi.dat`.
- `PyMem_Calloc` which *was* exported in `python3dll.c`, is moved to public
headers where it joins its other `PyMem_*` friends.
Only the last change is documented in the blurb; others are not user-visible.
(Nothing uses `Doc/data/stable_abi.dat` yet.)
https://bugs.python.org/issue43795
Add the Py_Is(x, y) function to test if the 'x' object is the 'y'
object, the same as "x is y" in Python. Add also the Py_IsNone(),
Py_IsTrue(), Py_IsFalse() functions to test if an object is,
respectively, the None singleton, the True singleton or the False
singleton.
The stable_abi.py script no longer parse macros. Macro targets can be
static inline functions which are not part of the stable ABI, only
part of the limited C API.
Run "make regen-limited-abi" to exclude PyType_HasFeature from
Doc/data/stable_abi.dat.
Rename Include/symtable.h to to Include/internal/pycore_symtable.h,
don't export symbols anymore (replace PyAPI_FUNC and PyAPI_DATA with
extern) and rename functions:
* PyST_GetScope() to _PyST_GetScope()
* PySymtable_BuildObject() to _PySymtable_Build()
* PySymtable_Free() to _PySymtable_Free()
Remove PySymtable_Build(), Py_SymtableString() and
Py_SymtableStringObject() functions.
The Py_SymtableString() function was part the stable ABI by mistake
but it could not be used, since the symtable.h header file was
excluded from the limited C API.
The Python symtable module remains available and is unchanged.
The new API allows to efficiently send values into native generators
and coroutines avoiding use of StopIteration exceptions to signal
returns.
ceval loop now uses this method instead of the old "private"
_PyGen_Send C API. This translates to 1.6x increased performance
of 'await' calls in micro-benchmarks.
Aside from CPython core improvements, this new API will also allow
Cython to generate more efficient code, benefiting high-performance
IO libraries like uvloop.
new_interpreter() now calls _PyBuiltin_Init() to create the builtins
module and calls _PyImport_FixupBuiltin(), rather than using
_PyImport_FindBuiltin(tstate, "builtins").
pycore_init_builtins() is now responsible to initialize
intepr->builtins_copy: inline _PyImport_Init() and remove this
function.
* bpo-36540: Documentation for PEP570 - Python positional only arguments
* fixup! bpo-36540: Documentation for PEP570 - Python positional only arguments
* Update reference for compound statements
* Apply suggestions from Carol
Co-Authored-By: Carol Willing <carolcode@willingconsulting.com>
* Update Doc/tutorial/controlflow.rst
Co-Authored-By: Carol Willing <carolcode@willingconsulting.com>
* Add extra bullet point and minor edits
Fixed some errors in refcounts.dat, remove functions removed in
Python 3, and add more entries for documented functions. This will
add several automatically generated notes about return values.
Makes the documentation more comprehensive in terms of indicating
whether or not a function returns a new reference.
Also fixes some errors and adds missing functions.
* Set the limited API version for PyImport_GetModule and PyOS_*Fork
functions.
* Add PyImport_GetModule and Py_UTF8Mode in PC/python3.def.
* Add several functions in Doc/data/refcounts.dat.
* Add timezone to datetime C API
* Add documentation for timezone C API macros
* Add dedicated tests for datetime type check macros
* Remove superfluous C API test
* Drop support for TimeZoneType in datetime C API
* Expose UTC singleton to the datetime C API
* Update datetime C-API documentation to include links
* Add reference count information for timezone constructors
Miss Islington (bot)