64 Commits

Author SHA1 Message Date
Mark Shannon 70bd1c2dd2 GH-143732: SEND specialization (GH-148963)
* SEND specialization. Adds 2 new specialized instructions:

* SEND_VIRTUAL: for sends to virtual iterators e.g lists and tuples
* SEND_ASYNC_GEN: for sends to async generators

Tweak FOR_ITER_VIRTUAL so that SEND_VIRTUAL and FOR_ITER_VIRTUAL use equivalent guards
2026-05-05 15:19:16 +01:00
Petr Viktorin 4599335a83 gh-149122: Fix refleak in codegen (GH-149179) 2026-04-30 10:05:48 +00:00
Irit Katriel 16f292ef4e gh-149122: Fix segfault in compiler when certain builtin functions are passed a coroutine as arg (#149138) 2026-04-29 13:35:51 +00:00
Mark Shannon 600f4dbd54 GH-145668: Add FOR_ITER specialization for virtual iterators. Specialize GET_ITER. (GH-147967)
* Add FOR_ITER_VIRTUAL to specialize FOR_ITER for virtual iterators
* Add GET_ITER_SELF to specialize GET_ITER for iterators (including generators)
* Add GET_ITER_VIRTUAL to specialize GET_ITER for iterables as virtual iterators
* Add new (internal) _tp_iteritem function slot to PyTypeObject
* Put limited RESUME at start of genexpr for free-threading. Fix up exception handling in genexpr
2026-04-16 15:22:22 +01:00
Carey Metcalfe cb339d3c9e gh-143886: Ensure function annotations are returned in order of definition (#143888)
Ensure function annotations are returned in order of definition

Previously, when getting type annotations of a function, normal
arguments were returned before positional-only ones in the dictionary.
Since `functools.singledispatch` relies on this ordering being correct
to dispatch based on the type of the first argument, this issue was
causing incorrect registrations for functions with positional-only
arguments.

This commit updates how annotations are generated so that
positional-only arguments are generated and added to the dictionary
before normal arguments.
2026-04-15 06:24:28 -07:00
Brandon ca95e979d6 gh-146442: Fix various bugs in compiler pipeline (#146443)
Fix null derefs, missing decrefs, and unchecked returns from bug report.
2026-03-30 22:04:04 +02:00
Mark Shannon 879c85f6e4 GH-145667: Merge GET_ITER and GET_YIELD_FROM_ITER (GH-146120)
* Merge GET_ITER and GET_YIELD_FROM_ITER. Modify SEND to make it a bit more like FOR_ITER
2026-03-21 10:48:13 +00:00
Stan Ulbrych 5e61a16c10 gh-145187: Fix crash on invalid type parameter bound expression in conditional block (GH-145188)
Fix parsing crash found by oss-fuzz
2026-02-24 12:44:57 -08:00
Benedikt Johannes 645f5c4a73 gh-144822: remove redundant decref in codegen.c (#144823) 2026-02-14 19:20:33 +00:00
Pablo Galindo Salgado 46d5106cfa gh-142349: Implement PEP 810 - Explicit lazy imports (#142351)
Co-authored-by: T. Wouters <twouters@meta.com >
Co-authored-by: Brittany Reynoso <breynoso@meta.com>
Co-authored-by: Dino Viehland <dinoviehland@meta.com>
2026-02-12 00:15:33 +00:00
adam j hartz ccbe41e27c gh-143055: Implementation of PEP 798 (#143056)
Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
2026-01-30 20:37:52 -08:00
Mark Shannon ae53da5758 GH-143493: Conform to spec for generator expressions while supporting virtual iterators (GH-143569)
* Moves the `GET_ITER` instruction into the generator function preamble.
  This means the the iterable is converted into an iterator during generator
  creation, as documented, but keeps it in the same code object allowing
  optimization.
2026-01-16 09:11:58 +00:00
Kumar Aditya 74bb3ca1f8 gh-142863: optimize list and set calls with generator expressions (#142864) 2026-01-06 23:23:28 +05:30
Jelle Zijlstra 7a6fd4a45d gh-138349: Fix crash when combining module-level annotation and listcomp (#138363) 2025-09-10 15:18:39 +02:00
Dino Viehland 04c7f36205 gh-138230: Remove dead code in code gen - codegen_check_annotation is only calle… (#138228)
Remove dead code in code gen - codegen_check_annotation is only called if future annotations are enabled, and if future annotations are enabled it does nothing.
2025-09-09 18:17:00 -07:00
Brian Schubert 5493b46462 gh-138372: Fix SyntaxWarning for erroneous t-string subscription (#138375) 2025-09-02 18:13:12 +01:00
Dave Peck c5e77af131 gh-132661: Disallow Template/str concatenation after PEP 750 spec update (#135996)
Co-authored-by: sobolevn <mail@sobolevn.me>
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2025-07-21 08:44:26 +02:00
Petr Viktorin 1b1ae82fab gh-135755: Move SPECIAL_ constants to a private header (GH-135922)
Macros without a `Py`/`_Py` prefix should not be defined in public headers.
2025-06-25 13:03:05 +02:00
Mark Shannon f6f4e8a662 GH-132554: "Virtual" iterators (GH-132555)
* FOR_ITER now pushes either the iterator and NULL or leaves the iterable and pushes tagged zero

* NEXT_ITER uses the tagged int as the index into the sequence or, if TOS is NULL, iterates as before.
2025-05-27 15:59:45 +01:00
Nybblista 20f8ed595d gh-133279: Assert with HAS_TARGET in the codegen_addop_j function (#133280) 2025-05-02 13:52:48 +01:00
Lysandros Nikolaou 60202609a2 gh-132661: Implement PEP 750 (#132662)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Wingy <git@wingysam.xyz>
Co-authored-by: Koudai Aono <koxudaxi@gmail.com>
Co-authored-by: Dave Peck <davepeck@gmail.com>
Co-authored-by: Terry Jan Reedy <tjreedy@udel.edu>
Co-authored-by: Paul Everitt <pauleveritt@me.com>
Co-authored-by: sobolevn <mail@sobolevn.me>
2025-04-30 11:46:41 +02:00
Jelle Zijlstra 922049b613 gh-130907: Treat all module-level annotations as conditional (#131550) 2025-04-28 06:10:28 -07:00
Mark Shannon d87e7f3529 GH-127682: Only call __iter__ once in generator expressions. (GH-132351) 2025-04-11 09:37:22 +01:00
Jelle Zijlstra 07b8d3117f gh-132261: Store annotations at hidden internal keys in the class dict (#132345) 2025-04-10 21:13:26 -07:00
Irit Katriel 2c8f329dc6 gh-131738: optimize builtin any/all/tuple calls with a generator expression arg (#131737) 2025-03-28 10:35:20 +00:00
Jelle Zijlstra 898e6b395e gh-130881: Handle conditionally defined annotations (#130935) 2025-03-26 03:48:19 +00:00
Yan Yanchii 75103d975c gh-126835: Move constant tuple folding from ast_opt to CFG (#130769) 2025-03-19 20:59:55 +00:00
Mark Shannon a45f25361d GH-131238: More refactoring of core header files (GH-131351)
Adds new pycore_stats.h header file to help break dependencies involving the pycore_code.h header.
2025-03-17 14:41:05 +00:00
Yan Yanchii 55815a6474 gh-130480: Move duplicate LOAD_SMALL_INT optimization from codegen to CFG (#130481) 2025-03-14 21:23:27 +00:00
Irit Katriel 4242c2b8d0 gh-130080: move _Py_EnsureArrayLargeEnough to a separate header so it can be used outside of the compiler (#130930) 2025-03-13 16:02:58 +00:00
Yan Yanchii 3618240624 gh-126835: Avoid creating unnecessary tuple when looking for constant sequence during constant folding (#131054) 2025-03-12 21:45:54 +00:00
Mark Shannon be046ee6e0 GH-123044: Give the POP_TOP after a case test a location in the body, not the pattern. (GH-130627) 2025-03-10 17:31:16 +00:00
Mark Shannon 89df62c120 GH-128534: Fix behavior of branch monitoring for async for (GH-130847)
* Both branches in a pair now have a common source and are included in co_branches
2025-03-07 14:30:31 +00:00
Hugo Beauzée-Luyssen 214562ed4d gh-130740: Move some stdbool.h includes after Python.h (#130738)
Move some `#include <stdbool.h>` after `#include "Python.h"` when `pyconfig.h` is not
included first and when we are in a platform-agnostic context. This is to avoid having
features defined by `stdbool.h` before those decided by `Python.h`.
2025-03-02 09:56:49 +00:00
Mark Shannon 2a18e80695 GH-128534: Instrument branches for async for loops. (GH-130569) 2025-02-27 09:36:41 +00:00
Mark Shannon 014223649c GH-130396: Use computed stack limits on linux (GH-130398)
* Implement C recursion protection with limit pointers for Linux, MacOS and Windows

* Remove calls to PyOS_CheckStack

* Add stack protection to parser

* Make tests more robust to low stacks

* Improve error messages for stack overflow
2025-02-25 09:24:48 +00:00
Irit Katriel a1417b211f gh-100239: replace BINARY_SUBSCR & family by BINARY_OP with oparg NB_SUBSCR (#129700) 2025-02-07 22:39:54 +00:00
Yan Yanchii 0664c1af9b gh-126835: Move constant subscript folding to CFG (#129568)
Move folding of constant subscription from AST optimizer to CFG.

Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
2025-02-04 10:10:55 +02:00
Kirill Podoprigora 7d0521d5fc gh-126835: Move optimization of constant sequence creation from codegen to CFG (#129426)
Codegen phase has an optimization that transforms
```
LOAD_CONST x
LOAD_CONST y
LOAD_CONXT z
BUILD_LIST/BUILD_SET (3)
```
->
```
BUILD_LIST/BUILD_SET (0)
LOAD_CONST (x, y, z)
LIST_EXTEND/SET_UPDATE 1
```
This optimization has now been moved to CFG phase to make #128802 work.


Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
Co-authored-by: Yan Yanchii <yyanchiy@gmail.com>
2025-02-01 11:39:44 +00:00
Mark Shannon 75b4962157 GH-128914: Remove all but one conditional stack effects (GH-129226)
* Remove all 'if (0)' and 'if (1)' conditional stack effects

* Use array instead of conditional for BUILD_SLICE args

* Refactor LOAD_GLOBAL to use a common conditional uop

* Remove conditional stack effects from LOAD_ATTR specializations

* Replace conditional stack effects in LOAD_ATTR with a 0 or 1 sized array.

* Remove conditional stack effects from CALL_FUNCTION_EX
2025-01-27 16:24:48 +00:00
Sam Gross a10f99375e Revert "GH-128914: Remove conditional stack effects from bytecodes.c and the code generators (GH-128918)" (GH-129202)
The commit introduced a ~2.5-3% regression in the free threading build.

This reverts commit ab61d3f430.
2025-01-23 09:26:25 +00:00
Mark Shannon ab61d3f430 GH-128914: Remove conditional stack effects from bytecodes.c and the code generators (GH-128918) 2025-01-20 17:09:23 +00:00
Mark Shannon 2434fd2d50 GH-128533: Add NOT_TAKEN instruction after bytecode optimization. (GH-128554) 2025-01-06 22:01:07 +00:00
Mark Shannon f826beca0c GH-128375: Better instrument for FOR_ITER (GH-128445) 2025-01-06 17:54:47 +00:00
Jelle Zijlstra 3480124321 gh-119180: Set the name of the param to __annotate__ to "format" (#124730) 2024-12-30 08:19:38 -08:00
Mark Shannon d2f1d917e8 GH-122548: Implement branch taken and not taken events for sys.monitoring (GH-122564) 2024-12-19 16:59:51 +00:00
Jelle Zijlstra dcf629213b gh-119180: Add VALUE_WITH_FAKE_GLOBALS format to annotationlib (#124415) 2024-11-26 15:40:13 +00:00
Marc Mueller 4fd9eb2aca Fix typo: Use AsyncFor element access in codegen (#127278)
Use AsyncFor element access in codegen
2024-11-26 01:00:46 +00:00
Xuanteng Huang 6ec886531f gh-126072: Set docstring attribute for module and class (#126231) 2024-11-08 15:13:18 +00:00
Bénédikt Tran c222441fa7 gh-120017: use 'do-while(0)' in some {codegen,compile}.c multi-line macros (#120018) 2024-11-07 23:03:11 +00:00