801 Commits

Author SHA1 Message Date
Miss Islington (bot) de239beaec [3.12] gh-116576: Fix Tools/scripts/sortperf.py sorting the same list (GH-116577) (#116582)
gh-116576: Fix `Tools/scripts/sortperf.py` sorting the same list (GH-116577)
(cherry picked from commit 4704e55a71)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2024-03-11 07:15:51 +00:00
Miss Islington (bot) dd961d8329 [3.12] gh-108303: Move Lib/test/sortperf.py to Tools/scripts (GH-114687) (#115625)
gh-108303: Move `Lib/test/sortperf.py` to `Tools/scripts` (GH-114687)
(cherry picked from commit f9154f8f23)

Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
2024-02-18 07:46:23 +00:00
Carl Meyer 77262458fe gh-87729: improve hit rate of LOAD_SUPER_ATTR specialization (#104270) 2023-05-11 08:08:13 -06:00
penguin_wwy 373bca0cc5 GH-102181: Improve specialization stats for SEND (GH-102182) 2023-05-10 22:40:59 +00:00
Missoupro 01a49d1745 gh-102110: Add all tools description missed (GH-102625)
Added missing descriptions of some tools.

Automerge-Triggered-By: GH:hugovk
2023-03-30 13:49:07 -07:00
Brandt Bucher 0444ae2487 GH-100982: Break up COMPARE_AND_BRANCH (GH-102801) 2023-03-23 15:25:09 -07:00
Mark Shannon 7b14c2ef19 GH-100982: Add COMPARE_AND_BRANCH instruction (GH-100983) 2023-01-16 12:35:21 +00:00
Mark Shannon c00eb1eae6 Don't double count misses. (GH-100984) 2023-01-13 15:57:39 +00:00
penguin_wwy a02161286a GH-99770: Make the correct call specialization fail kind show up in the stats (GH-99771) 2022-12-22 12:44:59 +00:00
Mark Shannon 5693f45b19 Assorted minor fixes for specialization stats. (GH-100219) 2022-12-14 15:50:02 +00:00
Michael Droettboom 1583c6e326 GH-100143: Improve collecting pystats for parts of runs (GH-100144)
* pystats off by default

* Add -Xpystats flag

* Always dump pystats, even if turned off
2022-12-12 14:50:43 +00:00
Michael Droettboom 9dc787ea96 GH-100026: Include the number of raw input files in summarize_stats.py (GH-100027) 2022-12-06 11:14:47 +00:00
Michael Droettboom bf94c653f4 GH-99905: Fix output of misses in summarize_stats.py execution counts (GH-99906)
This was an indentation error introduced in 2844aa6a
2022-11-30 19:17:08 +00:00
Michael Droettboom 2844aa6a8e Support comparing two sets of pystats (GH-98816)
This adds support for comparing pystats collected from two different builds.

- The `--json-output` can be used to load in a set of raw stats and output a
  JSON file.
- Two of these JSON files can be provided on the next run, and then comparative
  results between the two are output.
2022-11-04 10:15:54 +00:00
Neil Schemenauer de6981680b gh-90716: add _pylong.py module (#96673)
Add Python implementations of certain longobject.c functions. These use
asymptotically faster algorithms that can be used for operations on
integers with many digits. In those cases, the performance overhead of
the Python implementation is not significant since the asymptotic
behavior is what dominates runtime. Functions provided by this module
should be considered private and not part of any public API.

Co-author: Tim Peters <tim.peters@gmail.com>
Co-author: Mark Dickinson <dickinsm@gmail.com>
Co-author: Bjorn Martinsson
2022-10-25 22:00:50 -07:00
Victor Stinner 1863302d61 gh-97669: Create Tools/build/ directory (#97963)
Create Tools/build/ directory. Move the following scripts from
Tools/scripts/ to Tools/build/:

* check_extension_modules.py
* deepfreeze.py
* freeze_modules.py
* generate_global_objects.py
* generate_levenshtein_examples.py
* generate_opcode_h.py
* generate_re_casefix.py
* generate_sre_constants.py
* generate_stdlib_module_names.py
* generate_token.py
* parse_html5_entities.py
* smelly.py
* stable_abi.py
* umarshal.py
* update_file.py
* verify_ensurepip_wheels.py

Update references to these scripts.
2022-10-17 12:01:00 +02:00
Victor Stinner d09d2c7c91 gh-97669: Remove Tools/scripts/startuptime.py (#98214)
The "pyperf command" tool be used instead. Example:

    $ python3 -m pyperf command -- python3 -c pass
    .....................
    command: Mean +- std dev: 17.8 ms +- 0.4 ms

pyperf also computes the standard deviation which gives an idea of
the benchmark looks reliable or not.
2022-10-12 17:53:46 +02:00
Victor Stinner 0895c2a066 gh-97669: Create Tools/patchcheck/ directory (#98186)
Move patchcheck.py, reindent.py and untabify.py scripts to a new
Tools/patchcheck/ directory.
2022-10-12 10:09:21 +02:00
Victor Stinner 454a6d61bc gh-97669: Remove abitype.py and pep384_macrocheck.py (#98165)
Remove abitype.py and pep384_macrocheck.py scripts of Tools/scripts/.
2022-10-11 13:18:53 +02:00
Victor Stinner e0ae9ddffe gh-97669: Remove outdated example scripts (#97675) (#98167)
Remove outdated example scripts of the Tools/scripts/ directory:

* gprof2html.py
* md5sum.py
* nm2def.py
* pathfix.py
* win_add2path.py

Remove test_gprof2html, test_md5sum and test_pathfix of test_tools.
2022-10-11 10:07:57 +02:00
Ken Jin b399115ef1 gh-95756: Lazily created cached co_* attrs (GH-97791) 2022-10-11 11:26:08 +08:00
Joannah Nanjekye 4ed00be98f gh-68686: Retire eptag ptag scripts (#98064)
* Retire eptag ptag scripts

* 📜🤖 Added by blurb_it.

* fix news entry error

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
2022-10-08 07:57:47 -07:00
Victor Stinner 002252c4ad gh-97669: Move difflib examples to Doc/includes/ (#97964)
Remove diff.py and ndiff.py scripts of Tools/scripts/: move them to
Doc/includes/.

* diff.py and ndiff.py files are no longer executable. Remove also
  their shebang ("#!/usr/bin/env python3").
* Remove the -profile command from ndiff.py to simply the code.
* Remove ndiff.py copyright and history command. The Python
  documentation examples are distributed under the "Zero Clause BSD
  License".
2022-10-07 10:57:48 -07:00
Łukasz Langa bbc7cd649a gh-97008: Add a Python implementation of AttributeError and NameError suggestions (#97022)
Relevant tests moved from test_exceptions to test_traceback to be able to
compare both implementations.

Co-authored-by: Carl Friedrich Bolz-Tereick <cfbolz@gmx.de>
2022-10-04 15:31:16 -07:00
Victor Stinner 116fa62c6e gh-97670: Remove sys.getdxp() and analyze_dxp.py script (#97671)
Remove the sys.getdxp() function and the Tools/scripts/analyze_dxp.py
script. DXP stands for "dynamic execution pairs". They were related
to DYNAMIC_EXECUTION_PROFILE and DXPAIRS macros which have been
removed in Python 3.11. Python can now be built with "./configure
--enable-pystats" to gather statistics on Python opcodes.
2022-10-04 15:28:57 +02:00
Victor Stinner 6cbbc26a73 gh-97669: Remove outdated example scripts (#97675)
Remove outdated example scripts of the Tools/scripts/ directory. A
copy can be found in the old-demos project:
https://github.com/gvanrossum/old-demos

Removed scripts (39):

* byext.py
* byteyears.py
* cleanfuture.py
* copytime.py
* crlf.py
* db2pickle.py
* dutree.doc
* dutree.py
* find-uname.py
* find_recursionlimit.py
* finddiv.py
* findlinksto.py
* findnocoding.py
* fixcid.py
* fixdiv.py
* fixheader.py
* fixnotice.py
* fixps.py
* get-remote-certificate.py
* google.py
* highlight.py
* ifdef.py
* import_diagnostics.py
* lfcr.py
* linktree.py
* lll.py
* mailerdaemon.py
* make_ctype.py
* mkreal.py
* objgraph.py
* pdeps.py
* pickle2db.py
* pindent.py
* pysource.py
* reindent-rst.py
* rgrep.py
* suff.py
* texi2html.py
* which.py

Changes:

* Remove test_fixcid, test_lll, test_pdeps and test_pindent
  of test.test_tools.
* Remove get-remote-certificate.py changelog entry, since the script
  was removed.

Note: there is a copy of crlf.py in Lib/test/test_lib2to3/data/.
2022-10-04 10:49:00 +02:00
Victor Stinner 83a0f44ffd gh-97612: Fix shell injection in get-remote-certificate.py (#97613)
Fix a shell code injection vulnerability in the
get-remote-certificate.py example script. The script no longer uses a
shell to run "openssl" commands. Issue reported and initial fix by
Caleb Shortt.

Remove the Windows code path to send "quit" on stdin to the "openssl
s_client" command: use DEVNULL on all platforms instead.

Co-authored-by: Caleb Shortt <caleb@rgauge.com>
2022-09-29 01:17:27 +02:00
Kumar Aditya 4e4bfffe2d GH-90699: use statically allocated interned strings in typeobject's slotdefs (GH-94706) 2022-09-07 15:02:08 -07:00
Kumar Aditya 6dab8c95bd GH-96458: Statically initialize utf8 representation of static strings (#96481) 2022-09-02 23:43:08 -07:00
Kumar Aditya 73ee5a6b86 GH-94851: check refcnt of immortal objects after finalization (GH-95001) 2022-07-25 10:43:59 -07:00
Brandt Bucher e402b26b7f GH-95113: Don't use EXTENDED_ARG_QUICK in unquickened code (GH-95121) 2022-07-22 11:04:20 -07:00
Christian Heimes 81dca70d70 gh-93939: Build C extensions without setup.py (GH-94474)
Combines GH-93940, GH-94452, and GH-94433
2022-07-14 00:51:49 -07:00
Christian Heimes 0c66074e9f gh-94773: deepfreeze: support frozensets with unsortable types (GH-94775) 2022-07-12 18:09:47 +02:00
Kumar Aditya 9dff9f4814 GH-90699: Intern statically allocated strings (GH-93597)
This is similar to how strings are interned for deepfreeze.
2022-07-08 10:47:37 -07:00
Kumar Aditya 71697664d7 GH-90699: Move generated static initializer to pycore_runtime_generated.h (GH-94051) 2022-07-07 13:04:05 -07:00
Christian Heimes 7bd67d1d88 gh-93939: Add script to check extension modules (#94545)
Add script ``Tools/scripts/check_modules.py`` to check and validate builtin
and shared extension modules. The script also handles ``Modules/Setup`` and
will eventually replace ``setup.py``.

Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Erlend Egeberg Aasland <erlend.aasland@protonmail.com>
2022-07-05 09:25:45 +02:00
Irit Katriel c57aad777a gh-94216: add pseudo instructions to the dis/opcodes modules (GH-94241) 2022-07-01 15:33:35 +01:00
Christian Heimes 44fa03d748 gh-93939: Create and install scripts in Makefile (GH-94324)
Scripts for 2to3, idle, and pydoc are now created and installed by make.
2022-06-28 14:56:06 +02:00
Mark Shannon c0453a40fa GH-94163: Add BINARY_SLICE and STORE_SLICE instructions. (GH-94168) 2022-06-27 12:24:23 +01:00
Ken Jin 50a5ab2c0b gh-93382: Sync up co_code changes with 3.11 (GH-94227)
Sync up co_code changes with 3.11 commit 852b4d4bcd.
2022-06-25 01:55:18 +08:00
Guido van Rossum 4e796f5646 gh-93771: Clarify how deepfreeze.py is run (#94150) 2022-06-23 09:31:39 -07:00
Victor Stinner 47e35625ff gh-84623: Remove unused imports (#94132) 2022-06-22 19:14:27 +02:00
Illia Volochii d36954b7ea gh-91172: Create a workflow for verifying bundled pip and setuptools (GH-31885)
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
2022-06-22 15:32:22 +02:00
Mark Shannon f805d37641 Stats: Add summary of top instructions for misses and deferred specialization. (GH-94072) 2022-06-22 08:50:02 +01:00
Ezio Melotti f28ec34c5c gh-82927: Update files related to HTML entities. (GH-92504) 2022-06-21 22:03:12 +02:00
Victor Stinner 616fa3465d gh-54781: Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/ (#94049)
* Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/.
* Remove Lib/test/test_lib2to3.py.
* Update imports.
* all_project_files(): use different paths and sort files
  to make the tests more reproducible.
* Update references to tests.
2022-06-21 15:21:22 +02:00
Mark Shannon ab0e601016 GH-93516: Speedup line number checks when tracing. (GH-93763)
* Use a lookup table to reduce overhead of getting line numbers during tracing.
2022-06-20 13:00:42 +01:00
Mark Shannon 45e62a2bc1 GH-93897: Store frame size in code object and de-opt if insufficient space on thread frame stack. (GH-93908) 2022-06-20 12:59:25 +01:00
Mark Shannon cdf7097612 Include freelists in allocation total. (GH-93799) 2022-06-14 14:44:26 +01:00
Mark Shannon 3cd1a5d3ec GH-93516: Store offset of first traceable instruction in code object (GH-93769) 2022-06-14 11:09:30 +01:00