Commit Graph

8446 Commits

Author SHA1 Message Date
Sam Gross 628f6eb003 gh-111924: Use PyMutex for runtime global locks.
This replaces some usages of PyThread_type_lock with PyMutex, which
does not require memory allocation to initialize.
2023-11-16 17:39:49 -05:00
Sam Gross 446f18a911 gh-111956: Add thread-safe one-time initialization. (gh-111960) 2023-11-16 12:19:54 -07:00
Mark Shannon 4bbb367ba6 GH-111848: Set the IP when de-optimizing (GH-112065)
* Replace jumps with deopts in tier 2

* Fewer special cases of uop names

* Add target field to uop IR

* Remove more redundant SET_IP and _CHECK_VALIDITY micro-ops

* Extend whitelist of non-escaping API functions.
2023-11-15 15:48:58 +00:00
Mark Shannon a519b87958 GH-111848: Convert remaining jumps to deopts into tier 2 code. (GH-112045) 2023-11-14 15:30:33 +00:00
Serhiy Storchaka b11c443bb2 gh-111789: Simplify bytecodes.c by using PyDict_GetItemRef() (GH-111978) 2023-11-14 15:38:49 +02:00
Victor Stinner 4f04172c92 gh-111262: Add PyDict_Pop() function (#112028)
_PyDict_Pop_KnownHash(): remove the default value and the return type
becomes an int.

Co-authored-by: Stefan Behnel <stefan_ml@behnel.de>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-11-14 12:51:00 +00:00
Serhiy Storchaka 16055c1604 gh-111789: Simplify ceval.c by using PyDict_GetItemRef() (GH-111980) 2023-11-14 11:29:49 +02:00
Serhiy Storchaka 95365625f4 gh-111789: Simplify import.c by using PyDict_GetItemRef() (GH-111979) 2023-11-14 11:29:20 +02:00
Brandt Bucher 31ad7e061e GH-111520: Add back the operand local (GH-111813) 2023-11-13 17:27:19 -08:00
Irit Katriel 36aab34fab gh-107149: make new opcode util functions private rather than public and unstable (#112042) 2023-11-14 00:31:02 +00:00
AN Long 29af7369db gh-111856: Fix os.fstat on windows with FAT32 and exFAT filesystem (GH-112038) 2023-11-13 16:10:06 +00:00
Markus Mohrhard 1447af7970 gh-106905: avoid incorrect SystemError about recursion depth mismatch (#106906)
* gh-106905: avoid incorrect SystemError about recursion depth mismatch

* Update Misc/NEWS.d/next/Core and Builtins/2023-07-20-11-41-16.gh-issue-106905.AyZpuB.rst

---------

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-11-13 13:05:17 +00:00
Serhiy Storchaka 1d75ef6b61 gh-111999: Add signatures and improve docstrings for builtins (GH-112000) 2023-11-13 09:13:49 +02:00
Sergey B Kirpichev 12a30bc1aa gh-111933: fix broken link to A.Neumaier article (gh-111937) 2023-11-11 18:53:41 -06:00
Serhiy Storchaka 06c47a305d Remove dead code left after gh-110721 (#111905) 2023-11-10 22:30:01 +00:00
Samuel Thibault 0802fd6c8e gh-81925: Implement native thread ids for kFreeBSD (#111761)
---------

Co-authored-by: Antoine Pitrou <antoine@python.org>
2023-11-09 21:02:30 +01:00
Michael Droettboom bc12f79112 gh-111786: Optimize for space for _PyEval_EvalFrameDefault on MSVC for PGO (#111794)
In PGO mode, this function caused a compiler error in MSVC.
It turns out that optimizing for space only save the day, and is even faster.
However, without PGO, this is neither necessary nor slower.
2023-11-09 18:41:40 +00:00
Mark Shannon 34a03e951b GH-111843: Tier 2 exponential backoff (GH-111850) 2023-11-09 13:49:51 +00:00
Mark Shannon 25c4956488 GH-109369: Exit tier 2 if executor is invalid (GH-111657) 2023-11-09 11:19:51 +00:00
Sam Gross 31c90d5838 gh-111569: Implement Python critical section API (gh-111571)
Critical sections are helpers to replace the global interpreter lock
with finer grained locking.  They provide similar guarantees to the GIL
and avoid the deadlock risk that plain locking involves.  Critical
sections are implicitly ended whenever the GIL would be released.  They
are resumed when the GIL would be acquired.  Nested critical sections
behave as if the sections were interleaved.
2023-11-08 15:39:29 -07:00
Mark Shannon 06efb60264 GH-111848: Tidy up tier 2 handling of FOR_ITER specialization by using DEOPT_IF instead of jumps. (GH-111849) 2023-11-08 13:31:55 +00:00
Victor Stinner 11e83488c5 gh-111089: Revert PyUnicode_AsUTF8() changes (#111833)
* Revert "gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)"

This reverts commit d9b606b3d0.

* Revert "gh-111089: Use PyUnicode_AsUTF8() in getargs.c (#111620)"

This reverts commit cde1071b2a.

* Revert "gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL (#111091)"

This reverts commit d731579bfb.

* Revert "gh-111089: Add PyUnicode_AsUTF8() to the limited C API (#111121)"

This reverts commit d8f32be5b6.

* Revert "gh-111089: Use PyUnicode_AsUTF8() in sqlite3 (#111122)"

This reverts commit 37e4e20eaa.
2023-11-07 22:36:13 +00:00
Anthony Shaw 178861b193 gh-111623: Add Support for Cross-interpreter tuples (gh-111628) 2023-11-07 10:58:29 -07:00
Serhiy Storchaka f55cb44359 gh-106672: C API: Report indiscriminately ignored errors (GH-106674)
Functions which indiscriminately ignore all errors now report them as
unraisable errors.
2023-11-07 15:58:04 +02:00
Mark Shannon 931f4438c9 GH-111485: Allow arbitrary annotations on instructions and micro-ops. (GH-111697) 2023-11-07 09:42:39 +00:00
Brandt Bucher 3e99c9cbf6 GH-111485: Make BEFORE_WITH a uop (GH-111812) 2023-11-06 16:42:49 -08:00
Eric Snow d4426e8d00 gh-76785: Move _Py_excinfo Functions Out of the Internal C-API (gh-111715)
I added _Py_excinfo to the internal API (and added its functions in Python/errors.c) in gh-111530 (9322ce9).  Since then I've had a nagging sense that I should have added the type and functions in its own PR.  While I do plan on using _Py_excinfo outside crossinterp.c very soon (see gh-111572/gh-111573), I'd still feel more comfortable if the _Py_excinfo stuff went in as its own PR.  Hence, here we are.

(FWIW, I may combine that with gh-111572, which I may, in turn, combine with gh-111573.  We'll see.)
2023-11-06 11:09:22 -07:00
Mark Shannon a7b0f63cdb GH-111772: Specialize slot loads and stores for _Py_T_OBJECT (GH-111773) 2023-11-06 13:55:04 +00:00
Mark Shannon d78c872e0d GH-111646: Simplify optimizer, by compacting uops when making executor. (GH-111647) 2023-11-06 11:28:52 +00:00
Antoine Pitrou 0e9c364f4a GH-110829: Ensure Thread.join() joins the OS thread (#110848)
Joining a thread now ensures the underlying OS thread has exited. This is required for safer fork() in multi-threaded processes.

---------

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2023-11-04 13:59:24 +00:00
Eric Snow df9815eb11 gh-111684: Avoid a Compiler Warning (gh-111706) 2023-11-03 13:37:10 -06:00
Tian Gao e0afed7e27 gh-103615: Use local events for opcode tracing (GH-109472)
* Use local monitoring for opcode trace

* Remove f_opcode_trace_set

* Add test for setting f_trace_opcodes after settrace
2023-11-03 16:39:50 +00:00
Irit Katriel d49aba5a7a gh-111354: Simplify _PyGen_yf by moving some of its work to the compiler and frame state (#111648) 2023-11-03 10:01:36 +00:00
Serhiy Storchaka 26c0e5e03a gh-108082: Remove _PyErr_WriteUnraisableMsg() (GH-111643)
Replace the remaining calls with PyErr_FormatUnraisable().
2023-11-03 09:45:53 +02:00
AN Long 3a1b09e6d0 gh-111654: remove redundant decref in LOAD_FROM_DICT_OR_DEREF (#111655) 2023-11-02 21:06:51 -07:00
Eric Snow 93206d19a3 gh-76785: Minor Fixes in crossinterp.c (gh-111671)
There were a few corner cases I didn't handle properly in gh-111530, which I've noticed while working on a follow-up PR.  This fixes those cases.
2023-11-03 00:45:42 +00:00
Michael Droettboom 25937e3188 gh-111663: Restore the Tier 2 uop count pystats (#111664) 2023-11-02 15:24:52 -07:00
Tian Gao f4b5588bde gh-110892: Return NULL for PyTrace_RETURN events caused by an exception (GH-110909) 2023-11-02 16:38:08 +00:00
Irit Katriel 52cc4af6ae gh-111354: simplify detection of RESUME after YIELD_VALUE at except-depth 1 (#111459) 2023-11-02 10:18:43 +00:00
Serhiy Storchaka 970e719a7a gh-108082: Use PyErr_FormatUnraisable() (GH-111580)
Replace most of calls of _PyErr_WriteUnraisableMsg() and some
calls of PyErr_WriteUnraisable(NULL) with PyErr_FormatUnraisable().

Co-authored-by: Victor Stinner <vstinner@python.org>
2023-11-02 09:16:34 +00:00
Serhiy Storchaka a12f624a9d Remove unnecessary includes (GH-111633) 2023-11-02 10:42:58 +02:00
Anthony Shaw 230e8e924d GH-111435: Add Support for Sharing True and False Between Interpreters (gh-111436)
This only affects users of the APIs in pycore_crossinterp.h (AKA _xxsubinterpretersmodule.c and _xxinterpchannels.c).
2023-11-02 00:09:01 +00:00
Eric Snow 9322ce90ac gh-76785: Crossinterp utils additions (gh-111530)
This moves several general internal APIs out of _xxsubinterpretersmodule.c and into the new Python/crossinterp.c (and the corresponding internal headers).

Specifically:

* _Py_excinfo, etc.:  the initial implementation for non-object exception snapshots (in pycore_pyerrors.h and Python/errors.c)
* _PyXI_exception_info, etc.:  helpers for passing an exception beween interpreters (wraps _Py_excinfo)
* _PyXI_namespace, etc.:  helpers for copying a dict of attrs between interpreters
* _PyXI_Enter(), _PyXI_Exit():  functions that abstract out the transitions between one interpreter and a second that will do some work temporarily

Again, these were all abstracted out of _xxsubinterpretersmodule.c as generalizations.  I plan on proposing these as public API at some point.
2023-11-01 17:36:40 -06:00
Victor Stinner cde1071b2a gh-111089: Use PyUnicode_AsUTF8() in getargs.c (#111620)
Replace PyUnicode_AsUTF8AndSize() with PyUnicode_AsUTF8() to remove
the explicit check for embedded null characters.
2023-11-02 00:13:55 +01:00
Guido van Rossum ff3b0a6938 High level docs for the VM state (#111621) 2023-11-01 22:55:10 +00:00
Guido van Rossum e4b37835ef GH-111485: Silence warnings in Python/executor_cases.c.h (#111619) 2023-11-01 14:24:52 -07:00
Yilei Yang 45a36d5f56 gh-111374: Add a new PYTHON_FROZEN_MODULES env var, equivalent of -X frozen_modules. (#111411)
Adds a new PYTHON_FROZEN_MODULES env var to correspond with -X frozen_modules.

Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
2023-11-01 20:39:31 +00:00
Guido van Rossum 7e135a48d6 gh-111520: Integrate the Tier 2 interpreter in the Tier 1 interpreter (#111428)
- There is no longer a separate Python/executor.c file.
- Conventions in Python/bytecodes.c are slightly different -- don't use `goto error`,
  you must use `GOTO_ERROR(error)` (same for others like `unused_local_error`).
- The `TIER_ONE` and `TIER_TWO` symbols are only valid in the generated (.c.h) files.
- In Lib/test/support/__init__.py, `Py_C_RECURSION_LIMIT` is imported from `_testcapi`.
- On Windows, in debug mode, stack allocation grows from 8MiB to 12MiB.
- **Beware!** This changes the env vars to enable uops and their debugging
  to `PYTHON_UOPS` and `PYTHON_LLTRACE`.
2023-11-01 13:13:02 -07:00
Victor Stinner d9b606b3d0 gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)
Replace PyUnicode_AsUTF8AndSize() with PyUnicode_AsUTF8() to remove
the explicit check for embedded null characters.

The change avoids to have to include explicitly <string.h> to get the
strlen() function when using a recent version of the limited C API.
2023-11-01 16:34:42 +01:00
Mark Shannon 5697fc2d4b GH-111537: Avoid using this_instr in asserts. (GH-111600) 2023-11-01 12:59:08 +00:00