Commit Graph

214 Commits

Author SHA1 Message Date
Michael Droettboom b05afdd5ec gh-115168: Add pystats counter for invalidated executors (GH-115169) 2024-02-26 17:51:47 +00:00
Guido van Rossum 142502ea8d Tier 2 cleanups and tweaks (#115534)
* Rename `_testinternalcapi.get_{uop,counter}_optimizer` to `new_*_optimizer`
* Use `_PyUOpName()` instead of` _PyOpcode_uop_name[]`
* Add `target` to executor iterator items -- `list(ex)` now returns `(opcode, oparg, target, operand)` quadruples
* Add executor methods `get_opcode()` and `get_oparg()` to get `vmdata.opcode`, `vmdata.oparg`
* Define a helper for printing uops, and unify various places where they are printed
* Add a hack to summarize_stats.py to fix legacy uop names (e.g. `POP_TOP` -> `_POP_TOP`)
* Define helpers in `test_opt.py` for accessing the set or list of opnames of an executor
2024-02-20 20:24:35 +00:00
Ken Jin 7cce857622 gh-114058: Foundations of the Tier2 redundancy eliminator (GH-115085)
---------

Co-authored-by: Mark Shannon <9448417+markshannon@users.noreply.github.com>
Co-authored-by: Jules <57632293+JuliaPoo@users.noreply.github.com>
Co-authored-by: Guido van Rossum <gvanrossum@users.noreply.github.com>
2024-02-13 21:24:48 +08:00
Mark Shannon 8144661017 GH-113710: Fix updating of dict version tag and add watched dict stats (GH-115221) 2024-02-12 16:07:38 +00:00
Mark Shannon e66d0399cc GH-114806. Don't specialize calls to classes with metaclasses. (GH-114870) 2024-02-01 19:39:32 +00:00
Michael Droettboom ea3cd0498c gh-114312: Collect stats for unlikely events (GH-114493) 2024-01-25 11:10:51 +00:00
Peter Lazorchak f653caa5a8 gh-89811: Check for valid tp_version_tag in specializer (GH-113558) 2024-01-11 13:33:05 +08:00
Mark Shannon 0ae60b66de GH-113486: Do not emit spurious PY_UNWIND events for optimized calls to classes. (GH-113680) 2024-01-05 09:45:22 +00:00
Mark Shannon e96f26083b GH-111485: Generate instruction and uop metadata (GH-113287) 2023-12-20 14:27:25 +00:00
Guido van Rossum 7316dfb0eb gh-112320: Implement on-trace confidence tracking for branches (#112321)
We track the confidence as a scaled int.
2023-12-12 21:43:08 +00:00
Mark Shannon a7b0f63cdb GH-111772: Specialize slot loads and stores for _Py_T_OBJECT (GH-111773) 2023-11-06 13:55:04 +00:00
Michael Droettboom 84b4533e84 gh-109329: Count tier2 opcode misses (#110561)
This keeps a separate 'miss' counter for each micro-opcode, incremented whenever a guard uop takes a deoptimization side exit.
2023-10-30 17:02:45 -07:00
Sam Gross 6dfb8fe023 gh-110481: Implement biased reference counting (gh-110764) 2023-10-30 16:06:09 +00:00
Irit Katriel a0c414c35d gh-111354: define names for RESUME oparg values (#111365) 2023-10-26 16:30:18 +01:00
Irit Katriel 67a91f78e4 gh-109094: replace frame->prev_instr by frame->instr_ptr (#109095) 2023-10-26 13:43:10 +00:00
Mark Shannon b0699aa544 GH-111213: Fix a few broken stats (GH-111216) 2023-10-26 11:33:12 +01:00
Michael Droettboom 9eb2489266 gh-109329: Add stat for "trace too short" (GH-110402) 2023-10-05 16:12:06 +01:00
Michael Droettboom e561e98058 GH-109329: Add tier 2 stats (GH-109913) 2023-10-04 14:52:28 -07:00
Brandt Bucher 22e65eecaa GH-105848: Replace KW_NAMES + CALL with LOAD_CONST + CALL_KW (GH-109300) 2023-09-13 10:25:45 -07:00
Michael Droettboom 5dcbbd8861 GH-109330: Dump and compare stats using opcode names, not numbers (GH-109335) 2023-09-12 14:12:57 -07:00
Guido van Rossum bcce5e2718 gh-109039: Branch prediction for Tier 2 interpreter (#109038)
This adds a 16-bit inline cache entry to the conditional branch instructions POP_JUMP_IF_{FALSE,TRUE,NONE,NOT_NONE} and their instrumented variants, which is used to keep track of the branch direction.

Each time we encounter these instructions we shift the cache entry left by one and set the bottom bit to whether we jumped.

Then when it's time to translate such a branch to Tier 2 uops, we use the bit count from the cache entry to decided whether to continue translating the "didn't jump" branch or the "jumped" branch.

The counter is initialized to a pattern of alternating ones and zeros to avoid bias.

The .pyc file magic number is updated. There's a new test, some fixes for existing tests, and a few miscellaneous cleanups.
2023-09-11 18:20:24 +00:00
Victor Stinner fd5989bda1 gh-108753: _Py_PrintSpecializationStats() uses Py_hexdigits (#109040) 2023-09-07 04:47:57 +02:00
Victor Stinner a0773b89df gh-108753: Enhance pystats (#108754)
Statistics gathering is now off by default. Use the "-X pystats"
command line option or set the new PYTHONSTATS environment variable
to 1 to turn statistics gathering on at Python startup.

Statistics are no longer dumped at exit if statistics gathering was
off or statistics have been cleared.

Changes:

* Add PYTHONSTATS environment variable.
* sys._stats_dump() now returns False if statistics are not dumped
  because they are all equal to zero.
* Add PyConfig._pystats member.
* Add tests on sys functions and on setting PyConfig._pystats to 1.
* Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h
  header files.
* Rename '_py_stats' variable to '_Py_stats'.
* Exclude Include/cpython/pystats.h from the Py_LIMITED_API.
* Move pystats.h include from object.h to Python.h.
* Add _Py_StatsOn() and _Py_StatsOff() functions. Remove
  '_py_stats_struct' variable from the API: make it static in
  specialize.c.
* Document API in Include/pystats.h and Include/cpython/pystats.h.
* Complete pystats documentation in Doc/using/configure.rst.
* Don't write "all zeros" stats: if _stats_off() and _stats_clear()
  or _stats_dump() were called.
* _PyEval_Fini() now always call _Py_PrintSpecializationStats() which
  does nothing if stats are all zeros.

Co-authored-by: Michael Droettboom <mdboom@gmail.com>
2023-09-06 15:54:59 +00:00
Victor Stinner b298b395e8 gh-108765: Cleanup #include in Python/*.c files (#108977)
Mention one symbol imported by each #include.
2023-09-06 15:56:08 +02:00
Guido van Rossum 59e46932c8 gh-108488: Initialize JUMP_BACKWARD cache to 0, not 17 (#108591)
This mis-initialization caused the executor optimization to kick in sooner than intended. It also set the lower 4 bits of the counter to `1` -- those bits are supposed to be reserved (the actual counter is in the upper 12 bits).
2023-08-29 18:14:56 +00:00
Dong-hee Na 66b4d9c9f0 gh-107265: Revert "Ensure _PyCode_Quicken does not handle ENTER_EXECUTOR" (#108485)
This reverts commit d6ac5c7b10.

Reason: the assert we just added could be triggered (see issue).
2023-08-25 15:23:39 +00:00
Dong-hee Na d6ac5c7b10 gh-107265: Ensure _PyCode_Quicken does not handle ENTER_EXECUTOR (gh-108460) 2023-08-24 23:03:26 +00:00
Irit Katriel 72119d16a5 gh-105481: remove regen-opcode. Generated _PyOpcode_Caches in regen-cases. (#108367) 2023-08-23 18:39:00 +01:00
Brandt Bucher 326f0ba1c5 GH-106485: Dematerialize instance dictionaries when possible (GH-106539) 2023-08-09 19:14:50 +00:00
Brandt Bucher a9caf9cf90 GH-105848: Simplify the arrangement of CALL's stack (GH-107788) 2023-08-09 18:19:39 +00:00
Brandt Bucher ea72c6fe3b GH-107596: Specialize str[int] (GH-107597) 2023-08-08 13:42:43 -07:00
Mark Shannon 2ba7c7f7b1 Add some GC stats to Py_STATS (GH-107581) 2023-08-04 10:34:23 +01:00
Victor Stinner 1a3faba9f1 gh-106869: Use new PyMemberDef constant names (#106871)
* Remove '#include "structmember.h"'.
* If needed, add <stddef.h> to get offsetof() function.
* Update Parser/asdl_c.py to regenerate Python/Python-ast.c.
* Replace:

  * T_SHORT => Py_T_SHORT
  * T_INT => Py_T_INT
  * T_LONG => Py_T_LONG
  * T_FLOAT => Py_T_FLOAT
  * T_DOUBLE => Py_T_DOUBLE
  * T_STRING => Py_T_STRING
  * T_OBJECT => _Py_T_OBJECT
  * T_CHAR => Py_T_CHAR
  * T_BYTE => Py_T_BYTE
  * T_UBYTE => Py_T_UBYTE
  * T_USHORT => Py_T_USHORT
  * T_UINT => Py_T_UINT
  * T_ULONG => Py_T_ULONG
  * T_STRING_INPLACE => Py_T_STRING_INPLACE
  * T_BOOL => Py_T_BOOL
  * T_OBJECT_EX => Py_T_OBJECT_EX
  * T_LONGLONG => Py_T_LONGLONG
  * T_ULONGLONG => Py_T_ULONGLONG
  * T_PYSSIZET => Py_T_PYSSIZET
  * T_NONE => _Py_T_NONE
  * READONLY => Py_READONLY
  * PY_AUDIT_READ => Py_AUDIT_READ
  * READ_RESTRICTED => Py_AUDIT_READ
  * PY_WRITE_RESTRICTED => _Py_WRITE_RESTRICTED
  * RESTRICTED => (READ_RESTRICTED | _Py_WRITE_RESTRICTED)
2023-07-25 15:28:30 +02:00
Victor Stinner ee15844db8 gh-106320: Move _PyMethodWrapper_Type to internal C API (#107064) 2023-07-22 20:57:59 +00:00
Mark Shannon 0c90e75610 GH-100288: Specialize LOAD_ATTR for simple class attributes. (#105990)
* Add two more specializations of LOAD_ATTR.
2023-07-10 11:40:35 +01:00
Guido van Rossum 17af98227f gh-106320: Fix specialize.c compilation by including pycore_pylifecycle.h (#106434)
Compilation of Python/specialize.c was broken on macOS for me by gh-106400.
2023-07-04 21:42:12 +00:00
Brandt Bucher 7b2d94d875 GH-106008: Make implicit boolean conversions explicit (GH-106003) 2023-06-29 13:49:54 -07:00
Guido van Rossum 51fc725117 gh-104584: Baby steps towards generating and executing traces (#105924)
Added a new, experimental, tracing optimizer and interpreter (a.k.a. "tier 2"). This currently pessimizes, so don't use yet -- this is infrastructure so we can experiment with optimizing passes. To enable it, pass ``-Xuops`` or set ``PYTHONUOPS=1``. To get debug output, set ``PYTHONUOPSDEBUG=N`` where ``N`` is a debug level (0-4, where 0 is no debug output and 4 is excessively verbose).

All of this code is likely to change dramatically before the 3.13 feature freeze. But this is a first step.
2023-06-26 19:02:57 -07:00
Mark Shannon 04492cbc9a GH-91095: Specialize calls to normal Python classes. (GH-99331) 2023-06-22 09:48:19 +01:00
Brandt Bucher 2beab5bdef GH-105840: Fix assertion failures when specializing calls with too many __defaults__ (GH-105847) 2023-06-16 11:01:15 -07:00
Mark Shannon e830289c52 GH-105229: Remove remaining two-codeunit superinstructions (GH-105326)
* Remove LOAD_CONST__LOAD_FAST and LOAD_FAST__LOAD_CONST superinstructions.
2023-06-08 12:35:34 +01:00
Mark Shannon 0689340366 GH-105229: Replace some superinstructions with single instruction equivalent. (GH-105230) 2023-06-05 11:07:04 +01:00
Brandt Bucher 1eb950ca55 GH-104405: Add missing PEP 523 checks (GH-104406) 2023-05-12 22:23:13 +00:00
Carl Meyer 77262458fe gh-87729: improve hit rate of LOAD_SUPER_ATTR specialization (#104270) 2023-05-11 08:08:13 -06:00
penguin_wwy 373bca0cc5 GH-102181: Improve specialization stats for SEND (GH-102182) 2023-05-10 22:40:59 +00:00
Carl Meyer afe7703744 gh-104184: fix building --with-pydebug --enable-pystats (#104217) 2023-05-09 08:53:19 -06:00
Mark Shannon 738c226786 GH-103082: Code cleanup in instrumentation code (#103474) 2023-04-29 04:51:55 +00:00
Carl Meyer ebf97c50f2 gh-103978: avoid using 'class' as an identifier (#103979) 2023-04-28 19:20:50 +00:00
Jelle Zijlstra 6c4124d11a gh-103879: Fix refleak in super specialization (#103882) 2023-04-26 08:50:19 -07:00
Carl Meyer ef25febcf2 gh-87729: specialize LOAD_SUPER_ATTR_METHOD (#103809) 2023-04-25 17:45:51 +00:00