Commit Graph

52 Commits

Author SHA1 Message Date
Diego Russo 9633c5239d GH-126910: Build/link the JIT shim in the Python interpreter (#148872) 2026-04-23 12:23:18 +01:00
Mark Shannon cecf564073 GH-146128: Fix AArch64 multi-instruction constants and relocations (GH-148598)
Fix AArch64 multi-instruction constants and relocations

* Elimates rendundant orr xN, xN, 0xffff after 16 or 32 bit loads
* Merges adrp (21rx) and ldr (12) relocations into single 33rx relocation, when safe to do so.
2026-04-16 15:33:09 +01:00
Brandt Bucher 6bb7b33e8f GH-146128: Remove the buggy AArch64 "33rx" relocation (#146263) 2026-04-07 07:52:59 +03:00
Petr Viktorin 9b08f8c56f GH-126910: Revert "Make _Py_get_machine_stack_pointer return the stack pointer (#147945)" (GH-147994)
Revert "GH-126910: Make `_Py_get_machine_stack_pointer` return the stack pointer (#147945)"

This reverts commit 255026d9ee,
which broke a tier-1 buildbot.
2026-04-02 16:53:09 +02:00
Mark Shannon 255026d9ee GH-126910: Make _Py_get_machine_stack_pointer return the stack pointer (#147945)
* Make _Py_get_machine_stack_pointer return the stack pointer (or close to it), not the frame pointer

* Make ``_Py_ReachedRecursionLimit`` inline again
* Remove ``_Py_MakeRecCheck`` relacing its use with ``_Py_ReachedRecursionLimit``
* Move stack swtiching check into ``_Py_CheckRecursiveCall``
2026-04-01 17:15:13 +01:00
Hai Zhu 81ef1b7317 gh-144888: Replace bloom filter linked lists with continuous arrays to optimize executor invalidating performance (GH-145873) 2026-03-16 15:58:18 +00:00
Chris Eibl caac966b00 fix warnings in jit builds (GH-144817) 2026-02-14 17:39:10 +00:00
Pablo Galindo Salgado 46d5106cfa gh-142349: Implement PEP 810 - Explicit lazy imports (#142351)
Co-authored-by: T. Wouters <twouters@meta.com >
Co-authored-by: Brittany Reynoso <breynoso@meta.com>
Co-authored-by: Dino Viehland <dinoviehland@meta.com>
2026-02-12 00:15:33 +00:00
Diego Russo 66055d0650 GH-126910: add test for manual frame unwinding (#144137) 2026-01-27 13:17:40 +00:00
Diego Russo 450e836aef JIT: don't leak shim memory when shutting down the interpreter (#142984) 2025-12-23 17:50:00 +00:00
Diego Russo 685272eb8a JIT: Rename trampoline.c to shim.c (#142974) 2025-12-19 14:39:41 +00:00
Donghee Na 14f0b5191a gh-142419: Add mmap.set_name method for user custom annotation (gh-142480) 2025-12-18 23:33:49 +09:00
Mark Shannon c7dcb26520 GH-142621: JIT: Avoid memory load for symbols within 4GB on AArch64 (GH-142820) 2025-12-17 12:07:07 +00:00
Mark Shannon 469f191a85 GH-135379: Top of stack caching for the JIT. (GH-135465)
Uses three registers to cache values at the top of the evaluation stack
This significantly reduces memory traffic for smaller, more common uops.
2025-12-11 10:32:52 +00:00
Diego Russo 46295677a1 GH-142305: JIT: Deduplicating GOT symbols in the trace (#142316) 2025-12-10 16:04:04 +00:00
Mark Shannon e0451ceef8 GH-139757: JIT: Remove redundant branches to jumps in the assembly optimizer (GH-140800)
JIT: Remove redundant branches to jump in the assembly optimizer

* Refactor JIT assembly optimizer making instructions instances not just strings
* Remove redundant jumps and branches where legal to do so
* Modifies _BINARY_OP_SUBSCR_STR_INT to avoid excessive inlining depth
2025-12-08 17:57:11 +00:00
Donghee Na c4ccaf4b10 gh-141770: Annotate anonymous mmap usage if "-X dev" is used (gh-142079) 2025-12-08 14:47:19 +00:00
Mark Shannon 62423c9c36 GH-141794: Limit size of generated machine code. (GH-142228)
* Factor out bodies of the largest uops, to reduce jit code size.
* Factor out common assert, also reducing jit code size.
* Limit size of jitted code for a single executor to 1MB.
2025-12-03 17:43:35 +00:00
Ken Jin 4fa80ce74c gh-139109: A new tracing JIT compiler frontend for CPython (GH-140310)
This PR changes the current JIT model from trace projection to trace recording. Benchmarking: better pyperformance (about 1.7% overall) geomean versus current https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251108-3.15.0a1%2B-7e2bc1d-JIT/bm-20251108-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-7e2bc1d-vs-base.svg, 100% faster Richards on the most improved benchmark versus the current JIT. Slowdown of about 10-15% on the worst benchmark versus the current JIT. **Note: the fastest version isn't the one merged, as it relies on fixing bugs in the specializing interpreter, which is left to another PR**. The speedup in the merged version is about 1.1%. https://raw.githubusercontent.com/facebookexperimental/free-threading-benchmarking/refs/heads/main/results/bm-20251112-3.15.0a1%2B-f8a764a-JIT/bm-20251112-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-f8a764a-vs-base.svg

Stats: 50% more uops executed, 30% more traces entered the last time we ran them. It also suggests our trace lengths for a real trace recording JIT are too short, as a lot of trace too long aborts https://github.com/facebookexperimental/free-threading-benchmarking/blob/main/results/bm-20251023-3.15.0a1%2B-eb73378-CLANG%2CJIT/bm-20251023-vultr-x86_64-Fidget%252dSpinner-tracing_jit-3.15.0a1%2B-eb73378-pystats-vs-base.md .

This new JIT frontend is already able to record/execute significantly more instructions than the previous JIT frontend. In this PR, we are now able to record through custom dunders, simple object creation, generators, etc. None of these were done by the old JIT frontend. Some custom dunders uops were discovered to be broken as part of this work gh-140277

The optimizer stack space check is disabled, as it's no longer valid to deal with underflow.

Pros:
* Ignoring the generated tracer code as it's automatically created, this is only additional 1k lines of code. The maintenance burden is handled by the DSL and code generator.
* `optimizer.c` is now significantly simpler, as we don't have to do strange things to recover the bytecode from a trace.
* The new JIT frontend is able to handle a lot more control-flow than the old one.
* Tracing is very low overhead. We use the tail calling interpreter/computed goto interpreter to switch between tracing mode and non-tracing mode. I call this mechanism dual dispatch, as we have two dispatch tables dispatching to each other. Specialization is still enabled while tracing.
* Better handling of polymorphism. We leverage the specializing interpreter for this.

Cons:
* (For now) requires tail calling interpreter or computed gotos. This means no Windows JIT for now :(. Not to fret, tail calling is coming soon to Windows though https://github.com/python/cpython/pull/139962

Design:
* After each instruction, the `record_previous_inst` function/label is executed. This does as the name suggests.
* The tracing interpreter lowers bytecode to uops directly so that it can obtain "fresh" values at the point of lowering.
* The tracing version behaves nearly identical to the normal interpreter, in fact it even has specialization! This allows it to run without much of a slowdown when tracing. The actual cost of tracing is only a function call and writes to memory.
* The tracing interpreter uses the specializing interpreter's deopt to naturally form the side exit chains. This allows it to side exit chain effectively, without repeating much code. We force a re-specializing when tracing a deopt.
* The tracing interpreter can even handle goto errors/exceptions, but I chose to disable them for now as it's not tested.
* Because we do not share interpreter dispatch, there is should be no significant slowdown to the original specializing interpreter on tailcall and computed got with JIT disabled. With JIT enabled, there might be a slowdown in the form of the JIT trying to trace.
* Things that could have dynamic instruction pointer effects are guarded on. The guard deopts to a new instruction --- `_DYNAMIC_EXIT`.
2025-11-13 18:08:32 +00:00
Savannah Ostrowski 4e2ff4ac4c GH-136895: Update JIT builds to use LLVM 20 (#140329)
Co-authored-by: Emma Harper Smith <emma@emmatyping.dev>
2025-11-03 10:01:44 -08:00
Shamil bedaea0598 gh-139269: Fix unaligned memory access in JIT code patching functions (GH-139271)
* Use memcpy for patching values instead of direct assignment


Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
2025-10-19 00:20:04 +01:00
Mark Shannon f1883852ed GH-135904: Implement assembler optimization for AArch64. (GH-139855) 2025-10-17 11:26:17 +01:00
Mark Shannon a8d9d94784 GH-137959: Replace shim code in jitted code with a single trampoline function. (GH-137961) 2025-08-21 10:40:53 +01:00
Mark Shannon e7b55f564d GH-136410: Faster side exits by using a cold exit stub (GH-136411) 2025-08-01 16:26:07 +01:00
Brandt Bucher d591b5effb GH-134291: Support older macOS deployment targets for JIT builds (GH-137211) 2025-07-30 15:48:18 -07:00
Brandt Bucher 3d8c38f6db GH-135904: Improve the JIT's performance on macOS (GH-136528) 2025-07-14 10:14:20 -07:00
Lysandros Nikolaou 60202609a2 gh-132661: Implement PEP 750 (#132662)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Wingy <git@wingysam.xyz>
Co-authored-by: Koudai Aono <koxudaxi@gmail.com>
Co-authored-by: Dave Peck <davepeck@gmail.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Paul Everitt <pauleveritt@me.com>
Co-authored-by: sobolevn <mail@sobolevn.me>
2025-04-30 11:46:41 +02:00
Victor Stinner 49fb75c676 gh-131238: Add missing pycore_function.h includes for JIT compiler (#131571) 2025-03-21 23:37:49 +00:00
Mark Shannon 7ebd71ee14 GH-131498: Remove conditional stack effects (GH-131499)
* Adds some missing #includes
2025-03-20 15:39:38 +00:00
Diego Russo 8a33034d82 GH-130956: Only emit AArch64 trampolines for long jumps (GH-131041) 2025-03-19 12:19:21 -07:00
Diego Russo a29a9c0f38 GH-129231: Group executable JIT code in memory (GH-129232) 2025-02-02 15:19:55 -08:00
Diego Russo 567394517a GH-128842: Collect JIT memory stats (GH-128941) 2025-02-02 15:17:53 -08:00
Victor Stinner 95504f429e gh-129354: Fix grammar in PyErr_FormatUnraisable() (#129475)
Replace "on verb+ing" with "while verb+ing".
2025-01-31 09:45:35 +01:00
Victor Stinner 4e47e05045 gh-129354: Use PyErr_FormatUnraisable() function (#129435)
Replace PyErr_WriteUnraisable() with PyErr_FormatUnraisable().
2025-01-30 16:09:38 +01:00
Diego Russo c45be8aa71 GH-126195: Use M1 JIT memory protection APIs (GH-126196) 2024-11-11 17:20:10 -08:00
Savannah Ostrowski c3a12ae13e GH-125911: Rename big trampoline to "shim" (GH-126339) 2024-11-05 15:26:46 -08:00
Nadeshiko Manju 7f6e884f3a GH-125911: Call combine_symbol_mask on the initial trampoline of a trace (GH-125973) 2024-10-25 15:43:02 +00:00
Michael Droettboom c8fd4b12e3 gh-125207: Fix MSVC 1935 build with JIT (#125209)
* gh-125207: Use {0} array initializers

* Simplify, as suggested in PR

* Revert change to explicitly specify length
2024-10-18 15:51:29 -04:00
Diego Russo b85923a0fe GH-119726: Deduplicate AArch64 trampolines within a trace (GH-123872) 2024-10-02 12:07:20 -07:00
Brandt Bucher 33903c53db GH-116017: Get rid of _COLD_EXITs (GH-120960) 2024-07-01 13:17:40 -07:00
Savannah Ostrowski 5b941e57c7 GH-118844: Fix build failures when combining --disable-gil with --enable-experimental-jit (GH-118935) 2024-05-11 16:19:31 -07:00
Brandt Bucher 1b7e5e6e60 GH-113464: Generate a more efficient JIT (GH-118512) 2024-05-03 16:41:07 -07:00
Brandt Bucher 49baa656cb GH-115802: Use the GHC calling convention in JIT code (GH-118287) 2024-05-01 08:05:53 -07:00
Savannah Ostrowski 8b56d82c59 GH-118306: Update JIT to use LLVM 18 (GH-118307) 2024-04-29 21:09:16 +00:00
Brandt Bucher 62aeb0ee69 GH-117512: Allow 64-bit JIT operands on 32-bit platforms (GH-117527) 2024-04-06 08:26:43 -07:00
Mark Shannon bf82f77957 GH-116422: Tier2 hot/cold splitting (GH-116813)
Splits the "cold" path, deopts and exits, from the "hot" path, reducing the size of most jitted instructions, at the cost of slower exits.
2024-03-26 09:35:11 +00:00
Brandt Bucher 2c82592ab4 GH-116017: Put JIT code and data on the same page (GH-116845) 2024-03-19 08:47:28 -07:00
Brandt Bucher ffed8d985b GH-116134: JIT aarch64-pc-windows-msvc (GH-116130) 2024-03-04 10:16:56 -08:00
Brandt Bucher f0df35eeca GH-115802: JIT "small" code for Windows (GH-115964) 2024-02-29 08:11:28 -08:00
Brandt Bucher 7259480957 GH-115802: JIT "small" code for macOS and Linux (GH-115826) 2024-02-26 08:32:44 -08:00