119 Commits

Author SHA1 Message Date
Pablo Galindo Salgado 6b49cc64a4 [3.10] gh-102416: Do not memoize incorrectly loop rules in the parser (GH-102467). (#102474) 2023-03-06 16:59:16 +00:00
Pablo Galindo Salgado 9345eea256 [3.10] gh-101046: Fix a potential memory leak in the parser when raising MemoryError (GH-101051). (#101086)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2023-01-16 23:48:38 +00:00
Shantanu 9fbc81760e [3.10] gh-94996: Disallow lambda pos only params with feature_version < (3, 8) (GH-95934) (GH-95938)
(cherry picked from commit a965db37f2)

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>

Automerge-Triggered-By: GH:lysnikolaou
2022-08-12 12:47:07 -07:00
Shantanu a92c2d6eb5 [3.10] gh-94996: Disallow parsing pos only params with feature_version < (3, 8) (GH-95935)
(cherry picked from commit https://github.com/python/cpython/commit/b5e3ea286289fcad12be78480daf3756e350f69f)

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>

Automerge-Triggered-By: GH:lysnikolaou
2022-08-12 12:03:44 -07:00
Shantanu 906b345ddd [3.10] gh-94949: Disallow parsing parenthesised ctx mgr with old feature_version (GH-94950) (#94990)
(cherry picked from commit 0daba82221)

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2022-07-19 07:24:48 +01:00
Shantanu dda9198021 [3.10] gh-94947: Disallow parsing walrus with feature_version < (3, 8) (GH-94948) (#94969)
* gh-94947: Disallow parsing walrus with feature_version < (3, 8)

* oops, commit the parser

* 📜🤖 Added by blurb_it.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>.
(cherry picked from commit ae0be5a53b)

Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
2022-07-18 20:43:23 +01:00
wookie184 0ae7284a23 [3.10] gh-94192: Fix error for dictionary literals with invalid expression as value. (GH-94304) (#94344)
Co-authored-by: wookie184 <wookie1840@gmail.com>
2022-06-27 19:21:42 +01:00
Pablo Galindo Salgado 05cae26572 [3.10] gh-92858: Improve error message for some suites with syntax error before ':' (GH-92894). (#94183)
(cherry picked from commit 2fc83ac3af)

Co-authored-by: wookie184 <wookie1840@gmail.com>

Co-authored-by: wookie184 <wookie1840@gmail.com>
2022-06-23 18:37:53 +01:00
Pablo Galindo Salgado 8f36c735b2 [3.10] gh-93671: Avoid exponential backtracking in deeply nested sequence patterns in match statements (GH-93680) (#93690)
Co-authored-by: Łukasz Langa <lukasz@langa.pl>.
(cherry picked from commit 53a8b17895)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2022-06-10 19:34:15 +01:00
Matthieu Dartiailh 94609e3192 [3.10] Backport bpo-47212 (GH-32302) to Python 3.10 (GH-32334)
(cherry picked from commit aa0f056a00)

# Conflicts:
#	Grammar/python.gram
#	Parser/action_helpers.c

Automerge-Triggered-By: GH:pablogsal
2022-04-05 09:21:49 -07:00
Pablo Galindo Salgado 14284b0e71 [3.10] Allow the parser to avoid nested processing of invalid rules (GH-31252). (GH-31257)
(cherry picked from commit 390459de6d)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2022-02-10 14:38:31 +00:00
Pablo Galindo Salgado 9b23f8f78f [3.10] bpo-46707: Avoid potential exponential backtracking in some syntax errors (GH-31241). (GH-31242)
(cherry picked from commit b71dc71905)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2022-02-10 03:54:47 +00:00
Pablo Galindo Salgado dc73199a21 [3.10] bpo-46110: Add a recursion check to avoid stack overflow in the PEG parser (GH-30177) (GH-30214)
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>.
(cherry picked from commit e9898bf153)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-12-20 16:23:37 +00:00
Victor Stinner b6defde2af bpo-45866: pegen strips directory of "generated from" header (GH-29777) (GH-29792)
"make regen-all" now produces the same output when run from a
directory other than the source tree: when building Python out of the
source tree.

(cherry picked from commit 253b7a0a9f)
2021-11-26 16:38:50 +01:00
Pablo Galindo Salgado c72311d917 [3.10] bpo-45727: Only trigger the 'did you forgot a comma' error suggestion if inside parentheses. (GH-29767)
Backport of GH-29757

Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-11-25 01:01:40 +00:00
Pablo Galindo Salgado 511ee1c0fa [3.10] bpo-45727: Make the syntax error for missing comma more consistent (GH-29427) (GH-29647)
(cherry picked from commit 546cefcda7)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-11-20 17:39:17 +00:00
Pablo Galindo Salgado 5618c81e13 [3.10] bpo-45716: Improve the error message when using True/False/None as keywords in a call (GH-29413). (GH-29428)
(cherry picked from commit e2d65630f3)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-11-16 22:31:16 +00:00
Pablo Galindo Salgado 0e34a5918c [3.10] bpo-44257: fix "assigment_expr" typo + regenerate the grammar, and remove unused imports (GH-29393) (GH-29395)
Co-authored-by: Wim Glenn <wglenn@jumptrading.com>.
(cherry picked from commit 762173c670)

Co-authored-by: wim glenn <wim.glenn@gmail.com>
2021-11-05 00:19:27 +01:00
Łukasz Langa 5c9cab595e [3.10] bpo-45494: Fix parser crash when reporting errors involving invalid continuation characters (GH-28993) (GH-29070)
There are two errors that this commit fixes:

* The parser was not correctly computing the offset and the string
  source for E_LINECONT errors due to the incorrect usage of strtok().
* The parser was not correctly unwinding the call stack when a tokenizer
  exception happened in rules involving optionals ('?', [...]) as we
  always make them return valid results by using the comma operator. We
  need to check first if we don't have an error before continuing..
(cherry picked from commit a106343f63)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-10-19 22:31:18 +02:00
Miss Islington (bot) 9e209d48ca bpo-43914: Correctly highlight SyntaxError exceptions for invalid generator expression in function calls (GH-28576)
(cherry picked from commit e5f13ce5b4)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-09-27 07:05:20 -07:00
Miss Islington (bot) 846a10fc45 bpo-44947: Refine the syntax error for trailing commas in import statements (GH-27814)
(cherry picked from commit b2f68b1900)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-08-18 13:32:01 -07:00
Pablo Galindo Salgado b1bd16c252 [3.10] bpo-44838: Refine the custom syntax errors for invalid 'if' expressions (GH-27615). (GH-27616)
…

(cherry picked from commit f5cbea6b1b)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>

Automerge-Triggered-By: GH:lysnikolaou
2021-08-05 11:00:19 -07:00
Miss Islington (bot) 567176249e bpo-44792: Improve syntax errors for if expressions (GH-27506) (GH-27565)
(cherry picked from commit 28b6dc9dd5)

Co-authored-by: Miguel Brito <5544985+miguendes@users.noreply.github.com>
2021-08-02 20:05:33 +01:00
Miss Islington (bot) 35035bc35a bpo-34013: Don't consider a grouped expression when reporting legacy print syntax errors (GH-27521)
(cherry picked from commit 208a7e957b)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-07-31 18:31:44 -07:00
Miss Islington (bot) 68e3dca068 bpo-34013: Move the Python 2 hints from the exception constructor to the parser (GH-27392)
(cherry picked from commit ecc3c8e421)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-07-27 14:19:18 -07:00
Pablo Galindo Salgado b977f8510e [3.10] bpo-34013: Generalize the invalid legacy statement error message (GH-27389). (GH-27391)
(cherry picked from commit 6948964ecf)

Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
2021-07-27 18:52:32 +01:00
Miss Islington (bot) 11f1a30cdb bpo-44456: Improve the syntax error when mixing keyword and positional patterns (GH-26793)
(cherry picked from commit 0acc258fe6)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-06-24 08:34:28 -07:00
Pablo Galindo a8c418d5ed [3.10] bpo-44368: Improve syntax errors with invalid as pattern targets (GH-26632) (GH-26792)
(cherry picked from commit 05073036dc)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-06-18 22:15:57 +01:00
Miss Islington (bot) 3e137426de bpo-44385: Remove unused grammar rules (GH-26655)
Automerge-Triggered-By: GH:lysnikolaou
(cherry picked from commit e7b4644607)

Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
2021-06-10 15:31:09 -07:00
Pablo Galindo e53f72a1b4 [3.10] bpo-44305: Improve syntax error for try blocks without except or finally (GH-26523) (GH-26524)
(cherry picked from commit b250f89bb7)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-06-04 00:11:43 +01:00
Miss Islington (bot) ae1732d461 bpo-44180: Fix edge cases in invalid assigment rules in the parser (GH-26283)
The invalid assignment rules are very delicate since the parser can
easily raise an invalid assignment when a keyword argument is provided.
As they are very deep into the grammar tree, is very difficult to
specify in which contexts these rules can be used and in which don't.
For that, we need to use a different version of the rule that doesn't do
error checking in those situations where we don't want the rule to raise
(keyword arguments and generator expressions).

We also need to check if we are in left-recursive rule, as those can try
to eagerly advance the parser even if the parse will fail at the end of
the expression. Failing to do this allows the parser to start parsing a
call as a tuple and incorrectly identify a keyword argument as an
invalid assignment, before it realizes that it was not a tuple after all.
(cherry picked from commit c878a97968)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-05-21 11:20:43 -07:00
Miss Islington (bot) ec0699c044 bpo-44168: Fix error message in the parser for keyword arguments for invalid expressions (GH-26210) (GH-26247)
(cherry picked from commit 33c0c90dea)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-05-19 19:28:31 +01:00
Miss Islington (bot) 9a0e65c8e0 bpo-43149: Correct the syntax error message for multiple exception types (GH-25996) GH-25997)
Automerge-Triggered-By: GH:pablogsal
(cherry picked from commit 6692dc1ca9)

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-05-09 22:13:50 +01:00
Brandt Bucher dbe60ee09d bpo-43892: Validate the first term of complex literal value patterns (GH-25735) 2021-04-29 17:19:28 -07:00
Nick Coghlan 1e7b858575 bpo-43892: Make match patterns explicit in the AST (GH-25585)
Co-authored-by: Brandt Bucher <brandtbucher@gmail.com>
2021-04-28 22:58:44 -07:00
Pablo Galindo a77aac4fca bpo-43914: Highlight invalid ranges in SyntaxErrors (#25525)
To improve the user experience understanding what part of the error messages associated with SyntaxErrors is wrong, we can highlight the whole error range and not only place the caret at the first character. In this way:

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^
SyntaxError: Generator expression must be parenthesized

becomes

>>> foo(x, z for z in range(10), t, w)
  File "<stdin>", line 1
    foo(x, z for z in range(10), t, w)
           ^^^^^^^^^^^^^^^^^^^^
SyntaxError: Generator expression must be parenthesized
2021-04-23 14:27:05 +01:00
Pablo Galindo 56c95dfe27 bpo-43859: Improve the error message for IndentationError exceptions (GH-25431) 2021-04-21 15:28:21 +01:00
Pablo Galindo b5b98bd8f8 bpo-43823: Fix location of one of the errors for invalid dictionary literals (GH-25427) 2021-04-16 00:45:42 +01:00
Pablo Galindo b280248be8 bpo-43822: Improve syntax errors for missing commas (GH-25377) 2021-04-15 21:38:45 +01:00
Pablo Galindo da74350174 bpo-43823: Improve syntax errors for invalid dictionary literals (GH-25378) 2021-04-15 14:06:39 +01:00
Pablo Galindo 30ed93bfec bpo-43797: Handle correctly invalid assignments inside function calls and generators (GH-25390) 2021-04-13 17:51:21 +01:00
Pablo Galindo d9151cb453 Ensure that early = are not matched by the parser as invalid comparisons (GH-25375) 2021-04-13 02:32:33 +01:00
Pablo Galindo b86ed8e3bb bpo-43797: Improve syntax error for invalid comparisons (#25317)
* bpo-43797: Improve syntax error for invalid comparisons

* Update Lib/test/test_fstring.py

Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>

* Apply review comments

* can't -> cannot

Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
2021-04-12 16:59:30 +01:00
Matthew Suozzo 75a06f067b bpo-43798: Add source location attributes to alias (GH-25324)
* Add source location attributes to alias.
* Move alias star construction to pegen helper.

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
2021-04-10 22:56:28 +02:00
Victor Stinner d27f8d2e07 bpo-43244: Rename pycore_ast.h functions to _PyAST_xxx() (GH-25252)
Rename AST functions of pycore_ast.h to use the "_PyAST_" prefix.
Remove macros creating aliases without prefix. For example, Module()
becomes _PyAST_Module(). Update Grammar/python.gram to use
_PyAST_xxx() functions.
2021-04-07 21:34:22 +02:00
Pablo Galindo 8efad61963 bpo-41064: Improve syntax error for invalid usage of '**' in f-strings (GH-25006) 2021-03-24 19:34:17 +00:00
Victor Stinner 6af528b4ab bpo-43244: Fix test_peg_generators on Windows (GH-24913)
Don't redefine Py_DebugFlag, it's already defined in pydebug.h which
is included by Python.h
2021-03-18 09:54:13 +01:00
Pablo Galindo 08fb8ac99a bpo-42128: Add 'missing :' syntax error message to match statements (GH-24733) 2021-03-18 01:03:11 +00:00
Jozef Grajciar c994ffe695 bpo-11717: fix ssize_t redefinition error when targeting 32bit Windows app (GH-24479) 2021-03-01 11:18:33 +00:00
Brandt Bucher 145bf269df bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917)
Co-authored-by: Guido van Rossum <guido@python.org>
Co-authored-by: Talin <viridia@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
2021-02-26 14:51:55 -08:00