Files
cpython/Tools
Petr Viktorin 6f1d448bc1 gh-113993: Allow interned strings to be mortal, and fix related issues (GH-120520)
* Add an InternalDocs file describing how interning should work and how to use it.

* Add internal functions to *explicitly* request what kind of interning is done:
  - `_PyUnicode_InternMortal`
  - `_PyUnicode_InternImmortal`
  - `_PyUnicode_InternStatic`

* Switch uses of `PyUnicode_InternInPlace` to those.

* Disallow using `_Py_SetImmortal` on strings directly.
  You should use `_PyUnicode_InternImmortal` instead:
  - Strings should be interned before immortalization, otherwise you're possibly
    interning a immortalizing copy.
  - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to
    `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in
    backports, as they are now part of public API and version-specific ABI.

* Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery.

* Make sure the statically allocated string singletons are unique. This means these sets are now disjoint:
  - `_Py_ID`
  - `_Py_STR` (including the empty string)
  - one-character latin-1 singletons

  Now, when you intern a singleton, that exact singleton will be interned.

* Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic).

* Intern `_Py_STR` singletons at startup.

* For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup.

* Beef up the tests. Cover internal details (marked with `@cpython_only`).

* Add lots of assertions

Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
2024-06-21 17:19:31 +02:00
..

This directory contains a number of Python programs that are useful
while building or extending Python.

build           Automatically generated directory by the build system
                contain build artifacts and intermediate files.

buildbot        Batchfiles for running on Windows buildbot workers.

c-analyzer      Tools to check no new global variables have been added.

cases_generator Tooling to generate interpreters.

clinic          A preprocessor for CPython C files in order to automate
                the boilerplate involved with writing argument parsing
                code for "builtins".

freeze          Create a stand-alone executable from a Python program.

gdb             Python code to be run inside gdb, to make it easier to
                debug Python itself (by David Malcolm).

i18n            Tools for internationalization. pygettext.py
                parses Python source code and generates .pot files,
                and msgfmt.py generates a binary message catalog
                from a catalog in text format.

importbench     A set of micro-benchmarks for various import scenarios.

msi             Support for packaging Python as an MSI package on Windows.

nuget           Files for the NuGet package manager for .NET.

patchcheck      Tools for checking and applying patches to the Python source code
                and verifying the integrity of patch files.

peg_generator   PEG-based parser generator (pegen) used for new parser.

scripts         A number of useful single-file programs, e.g. run_tests.py
                which runs the Python test suite.

ssl             Scripts to generate ssl_data.h from OpenSSL sources, and run
                tests against multiple installations of OpenSSL and LibreSSL.

tz              A script to dump timezone from /usr/share/zoneinfo.

unicode         Tools for generating unicodedata and codecs from unicode.org
                and other mapping files (by Fredrik Lundh, Marc-Andre Lemburg
                and Martin von Loewis).

unittestgui     A Tkinter based GUI test runner for unittest, with test
                discovery.

wasm            Config and helpers to facilitate cross compilation of CPython
                to WebAssembly (WASM).

Note: The pynche color editor has moved to https://gitlab.com/warsaw/pynche