Commit Graph

8487 Commits

Author SHA1 Message Date
Mark Shannon 0c55f27060 GH-111485: Factor out tier 2 code generation from the rest of the interpreter code generator (GH-112968) 2023-12-12 12:12:17 +00:00
Sam James c454e934d3 gh-112970: Detect and use closefrom() when available (#112969)
glibc-2.34 implements closefrom(3) using the same semantics as on BSD.
Check for closefrom() in configure and use the check result in
fileutils.c, rather than hardcoding a FreeBSD check.

Some implementations of closefrom() return an int. Explicitly discard 
the return value by casting it to void, to avoid future compiler
warnings.

Signed-off-by: Sam James <sam@gentoo.org>
2023-12-12 11:25:27 +01:00
Guido van Rossum 5b86644338 A smattering of cleanups in uop debug output and lltrace (#112980)
* Include destination T1 opcode in Error debug message
* Include destination T1 opcode in DEOPT debug message
* Remove obsolete comment from remove_unneeded_uops
* Change lltrace_instruction() to print caller's opcode/oparg
2023-12-11 16:42:30 -08:00
Yan Yanchii fed294c645 gh-112978: Remove redundant condition inside take_gil (gh-112979) 2023-12-12 08:23:41 +09:00
colorfulappl 0066ab5bc5 gh-90350: Optimize builtin functions min() and max() (GH-30286)
Builtin functions min() and max() now use METH_FASTCALL
2023-12-11 21:27:06 +02:00
Pablo Galindo Salgado a135a6d2c6 gh-112943: Correctly compute end offsets for multiline tokens in the tokenize module (#112949) 2023-12-11 11:44:22 +00:00
Sam Gross cf6110ba13 gh-111924: Use PyMutex for Runtime-global Locks. (gh-112207)
This replaces some usages of PyThread_type_lock with PyMutex, which does not require memory allocation to initialize.

This simplifies some of the runtime initialization and is also one step towards avoiding changing the default raw memory allocator during initialize/finalization, which can be non-thread-safe in some circumstances.
2023-12-07 12:33:40 -07:00
Sam Gross db460735af gh-112538: Add internal-only _PyThreadStateImpl "wrapper" for PyThreadState (gh-112560)
Every PyThreadState instance is now actually a _PyThreadStateImpl.
It is safe to cast from `PyThreadState*` to `_PyThreadStateImpl*` and back.
The _PyThreadStateImpl will contain fields that we do not want to expose
in the public C API.
2023-12-07 12:11:45 -07:00
Sam Gross 2d76be251d gh-111962: Make dtoa thread-safe in --disable-gil builds. (#112049)
This updates `dtoa.c` to avoid using the Bigint free-list in --disable-gil builds and
to pre-computes the needed powers of 5 during interpreter initialization.

* gh-111962: Make dtoa thread-safe in `--disable-gil` builds.

This avoids using the Bigint free-list in `--disable-gil` builds
and pre-computes the needed powers of 5 during interpreter initialization.

* Fix size of cached powers of 5 array.

We need the powers of 5 up to 5**512 because we only jump straight to
underflow when the exponent is less than -512 (or larger than 308).

* Rename Py_NOGIL to Py_GIL_DISABLED

* Changes from review

* Fix assertion placement
2023-12-07 13:47:55 +00:00
Mark Shannon b449415b2f GH-111485: Separate out parsing, analysis and code-gen phases of tier 1 code generator (GH-112299) 2023-12-07 12:49:40 +00:00
Kushal Das 4ba15de191 gh-74616: Raise ValueError in case of null character in input prompt (GH-1738)
If the input prompt to the builtin input function on terminal has any null
character, then raise ValueError instead of silently truncating it.

Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
2023-12-07 10:22:52 +00:00
Serhiy Storchaka 8660fb7fd7 gh-112660: Do not clear arbitrary errors on import (GH-112661)
Previously arbitrary errors could be cleared during formatting error
messages for ImportError or AttributeError for modules. Now all
unexpected errors are reported.
2023-12-07 12:19:43 +02:00
Pablo Galindo Salgado 16448cab44 gh-112730: Use color to highlight error locations (gh-112732)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
2023-12-06 23:29:54 +01:00
Victor Stinner 828451dfde gh-111545: Add Py_HashPointer() function (#112096)
* Implement _Py_HashPointerRaw() as a static inline function.
* Add Py_HashPointer() tests to test_capi.test_hash.
* Keep _Py_HashPointer() function as an alias to Py_HashPointer().
2023-12-06 15:09:22 +01:00
Matt Prodani a2a46f9f1e gh-112606: Use sem_clockwait with monotonic time when supported in parking_lot.c (gh-112733) 2023-12-06 15:54:57 +09:00
Serhiy Storchaka da6760bdf5 gh-65210: Add const qualifiers in PyArg_VaParseTupleAndKeywords() (GH-105958)
Change the declaration of the keywords parameter in functions
PyArg_ParseTupleAndKeywords() and PyArg_VaParseTupleAndKeywords() from `char **`
to `char * const *` in C and `const char * const *` in C++.

It makes these functions compatible with argument of type `const char * const *`,
`const char **` or `char * const *` in C++ and `char * const *` in C
without explicit type cast.

Co-authored-by: C.A.M. Gerlach <CAM.Gerlach@Gerlach.CAM>
2023-12-04 13:14:56 +02:00
Serhiy Storchaka dee7beeb4f bpo-34392: Add sys. _is_interned() (GH-8755) 2023-12-04 11:09:06 +02:00
Victor Stinner d9e444dbb8 gh-106560: Fix redundant declarations in Python/frozen.c (#112612)
Avoid duplicated declarations of "extern" functions in
Python/frozen.c.

Compiler warnings seen by building Python with gcc -Wredundant-decls.
2023-12-03 12:18:24 +01:00
Victor Stinner 5c5022b862 gh-112567: Add _PyTimeFraction C API (#112568)
Use a fraction internally in the _PyTime API to reduce the risk of
integer overflow: simplify the fraction using Greatest Common
Divisor (GCD). The fraction API is used by time functions:
perf_counter(), monotonic() and process_time().

For example, QueryPerformanceFrequency() usually returns 10 MHz on
Windows 10 and newer. The fraction SEC_TO_NS / frequency =
1_000_000_000 / 10_000_000 can be simplified to 100 / 1.

* Add _PyTimeFraction type.
* Add functions:

  * _PyTimeFraction_Set()
  * _PyTimeFraction_Mul()
  * _PyTimeFraction_Resolution()

* No longer check "numer * denom <= _PyTime_MAX" in
  _PyTimeFraction_Set(). _PyTimeFraction_Mul() uses _PyTime_Mul()
  which handles integer overflow.
2023-12-01 19:50:10 +01:00
Victor Stinner 05a370abd6 gh-112567: Add _Py_GetTicksPerSecond() function (#112587)
* Move _PyRuntimeState.time to _posixstate.ticks_per_second and
  time_module_state.ticks_per_second.
* Add time_module_state.clocks_per_second.
* Rename _PyTime_GetClockWithInfo() to py_clock().
* Rename _PyTime_GetProcessTimeWithInfo() to py_process_time().
* Add process_time_times() helper function, called by
  py_process_time().
* os.times() is now always built: no longer rely on HAVE_TIMES.
2023-12-01 17:05:56 +01:00
Pablo Galindo Salgado a73aa48e6b gh-112367: Only free perf trampoline arenas at shutdown (#112368)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
2023-12-01 13:20:51 +00:00
Irit Katriel 07ebd46f9e gh-112519: Make it possible to specify instruction flags for pseudo instructions in bytecodes.c (#112520) 2023-11-30 11:03:30 +00:00
Kirill Podoprigora 0785c68559 gh-111972: Make Unicode name C APIcapsule initialization thread-safe (#112249) 2023-11-30 11:12:49 +01:00
Guido van Rossum e723700190 Rename ...Uop... to ...UOp... (uppercase O) for consistency (#112327)
* Rename _PyUopExecute to _PyUOpExecute (uppercase O) for consistency
* Also rename _PyUopName and _PyUOp_Replacements, and some output strings
2023-11-28 17:10:11 -08:00
apaz 8f71b349de gh-112217: Add check to call result for do_raise() where cause is a type. (#112216) 2023-11-27 21:13:27 +00:00
Serhiy Storchaka 936c503a44 gh-111789: Use PyDict_GetItemRef() in Python/_warnings.c (gh-112080) 2023-11-27 18:58:43 +01:00
Serhiy Storchaka befbad3663 gh-111789: Use PyDict_GetItemRef() in Python/symtable.c (gh-112084) 2023-11-27 18:55:30 +01:00
Serhiy Storchaka aa438bdd6d gh-111789: Use PyDict_GetItemRef() in Python/codecs.c (gh-112082) 2023-11-27 18:53:43 +01:00
Serhiy Storchaka 395fd9c180 gh-111789: Use PyDict_GetItemRef() in Python/bltinmodule.c (gh-112081) 2023-11-27 18:52:54 +01:00
Serhiy Storchaka 4eea1e8236 gh-112438: Fix support of format units with the "e" prefix in nested tuples in PyArg_Parse (gh-112439) 2023-11-27 18:32:55 +01:00
Irit Katriel fb202af447 gh-99606: Make code generated for an empty f-string identical to that of a normal empty string (#112407) 2023-11-26 17:13:57 +00:00
Eric Snow 9e56eedd01 gh-76785: Return an "excinfo" Object From Interpreter.run() (gh-111573) 2023-11-23 00:55:00 +00:00
Mark Shannon 1619f4350e GH-111485: Sort cases in the case generator output (GH-112315) 2023-11-22 15:19:50 +00:00
Michael Droettboom 6a00a58f60 gh-111786: Use separate opcode vars for Tier 1 and Tier 2 (#112289)
This makes Windows about 3% faster on pyperformance benchmarks.
2023-11-20 15:13:44 -08:00
Guido van Rossum 8deb8bc2e5 gh-112287: Speed up Tier 2 (uop) interpreter a little (#112286)
This makes the Tier 2 interpreter a little faster.
I calculated by about 3%,
though I hesitate to claim an exact number.

This starts by doubling the trace size limit (to 512),
making it more likely that loops fit in a trace.

The rest of the approach is to only load
`oparg` and `operand` in cases that use them.
The code generator know when these are used.

For `oparg`, it will conditionally emit
```
oparg = CURRENT_OPARG();
```
at the top of the case block.
(The `oparg` variable may be referenced multiple times
by the instructions code block, so it must be in a variable.)

For `operand`, it will use `CURRENT_OPERAND()` directly
instead of referencing the `operand` variable,
which no longer exists.
(There is only one place where this will be used.)
2023-11-20 11:25:32 -08:00
Guido van Rossum c4c63211e8 gh-111848: Clean up RESERVE() macro (#112274)
Also avoid compiler warnings about unused 'reserved' variable.
2023-11-20 10:45:42 -08:00
Guido van Rossum 1995955173 gh-106529: Make FOR_ITER a viable uop (#112134)
This uses the new mechanism whereby certain uops
are replaced by others during translation,
using the `_PyUop_Replacements` table.
We further special-case the `_FOR_ITER_TIER_TWO` uop
to update the deoptimization target to point
just past the corresponding `END_FOR` opcode.

Two tiny code cleanups are also part of this PR.
2023-11-20 10:08:53 -08:00
Hugo van Kemenade 3b3ec0d77f gh-111863: Rename Py_NOGIL to Py_GIL_DISABLED (#111864)
Rename Py_NOGIL to Py_GIL_DISABLED
2023-11-20 15:52:00 +02:00
Serhiy Storchaka 91d1730541 gh-112234: Remove the toplevel parameter in converttuple() (GH-112235)
It is and always was 0.
2023-11-18 13:47:34 +02:00
Guido van Rossum eb3c94ea66 gh-110319: Assert type_version != 0 before using it (#112226)
- Ensure that `assert(type_version != 0);` always comes *before* using `type_version`

Also:
- In cases_generator, rename `-v` to from `--verbose` to `--viable`
2023-11-17 20:58:13 -08:00
Guido van Rossum 7405745817 Various small improvements to uop debug output (#112218)
- Show uop name in Error/DEOPT messages
- Add target to some messages
- Expose uop_name() as _PyUopName()
2023-11-17 22:25:57 +00:00
Guido van Rossum be0bd54c6b gh-106529: Cleanups split off gh-112134 (#112214)
- Double max trace size to 256
- Add a dependency on executor_cases.c.h for ceval.o
- Mark `_SPECIALIZE_UNPACK_SEQUENCE` as `TIER_ONE_ONLY`
- Add debug output back showing the optimized trace
- Bunch of cleanups to Tools/cases_generator/
2023-11-17 11:49:42 -08:00
Sam Gross 446f18a911 gh-111956: Add thread-safe one-time initialization. (gh-111960) 2023-11-16 12:19:54 -07:00
Mark Shannon 4bbb367ba6 GH-111848: Set the IP when de-optimizing (GH-112065)
* Replace jumps with deopts in tier 2

* Fewer special cases of uop names

* Add target field to uop IR

* Remove more redundant SET_IP and _CHECK_VALIDITY micro-ops

* Extend whitelist of non-escaping API functions.
2023-11-15 15:48:58 +00:00
Mark Shannon a519b87958 GH-111848: Convert remaining jumps to deopts into tier 2 code. (GH-112045) 2023-11-14 15:30:33 +00:00
Serhiy Storchaka b11c443bb2 gh-111789: Simplify bytecodes.c by using PyDict_GetItemRef() (GH-111978) 2023-11-14 15:38:49 +02:00
Victor Stinner 4f04172c92 gh-111262: Add PyDict_Pop() function (#112028)
_PyDict_Pop_KnownHash(): remove the default value and the return type
becomes an int.

Co-authored-by: Stefan Behnel <stefan_ml@behnel.de>
Co-authored-by: Antoine Pitrou <pitrou@free.fr>
2023-11-14 12:51:00 +00:00
Serhiy Storchaka 16055c1604 gh-111789: Simplify ceval.c by using PyDict_GetItemRef() (GH-111980) 2023-11-14 11:29:49 +02:00
Serhiy Storchaka 95365625f4 gh-111789: Simplify import.c by using PyDict_GetItemRef() (GH-111979) 2023-11-14 11:29:20 +02:00
Brandt Bucher 31ad7e061e GH-111520: Add back the operand local (GH-111813) 2023-11-13 17:27:19 -08:00