58 Commits

Author SHA1 Message Date
Sergey Miryanov 4d0ae4cba9 [3.14] GH-148726: Add heap_size to _gc_runtime_state (#149025) 2026-04-29 13:04:11 +03:00
Sergey Miryanov 9a7e205e46 [3.14] GH-148726: Forward-port generational GC (#148720)
Co-authored-by: Neil Schemenauer <nas@arctrix.com>
Co-authored-by: Sergey Miryanov <sergey.miryanov@gmail.com>
Co-authored-by: Zanie Blue <contact@zanie.dev>
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
2026-04-26 21:12:52 +03:00
Sergey Miryanov 705e3ea9d1 [3.14] GH-91636: Clear weakrefs created by finalizers. (GH-136401) (#144444)
Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
2026-03-12 14:10:29 +02:00
Hugo van Kemenade 7642070807 [3.14] Revert "GH-91636: Clear weakrefs created by finalizers. (GH-136401) (#141993)" (#142152) 2025-12-01 20:34:37 +02:00
Miss Islington (bot) fed2af6f9e [3.14] GH-91636: Clear weakrefs created by finalizers. (GH-136401) (#141993)
GH-91636: Clear weakrefs created by finalizers. (GH-136401)

Weakrefs to unreachable garbage that are created during running of
finalizers need to be cleared.  This avoids exposing objects that
have `tp_clear` called on them to Python-level code.
(cherry picked from commit b6b99bf7f1)

Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
2025-11-28 23:34:36 -08:00
Miss Islington (bot) c1bfd4cb9c [3.14] gh-140358: Bring back elapsed time and unreachable count to gc debug output (GH-140359) (#140405)
Co-authored-by: Pål Grønås Drange <paal.drange+pgdr@gmail.com>
2025-10-27 15:29:43 +02:00
Mark Shannon d1a434f7b2 [3.14] GH-139951: Fix major GC performance regression. Backport of GH-140262 (GH-140447)
* Count number of actually tracked objects, instead of trackable objects. This ensures that untracking tuples has the desired effect of reducing GC overhead
* Do not track most untrackable tuples during creation. This prevents large numbers of small tuples causing execessive GCs.
2025-10-23 15:26:58 +01:00
Miss Islington (bot) e03db7317f [3.14] gh-136517: Print uncollectable objects if DEBUG_UNCOLLECTABLE mode was set (GH-136518) (#136522)
gh-136517: Print uncollectable objects if DEBUG_UNCOLLECTABLE mode was set (GH-136518)
(cherry picked from commit c560df9658)

Co-authored-by: Sergey Miryanov <sergey.miryanov@gmail.com>
2025-07-11 15:20:05 +01:00
Mark Shannon f554237b8e GH-133261: Make sure that the GC doesn't untrack objects in trashcan (GH-133431) 2025-05-05 13:44:50 +01:00
Mark Shannon 44e4c479fb GH-124715: Move trashcan mechanism into Py_Dealloc (GH-132280) 2025-04-30 11:37:53 +01:00
Mark Shannon ccf1b0b1c1 GH-132508: Use tagged integers on the evaluation stack for the last instruction offset (GH-132545) 2025-04-29 18:00:35 +01:00
mpage 053c285f6b gh-130704: Strength reduce LOAD_FAST{_LOAD_FAST} (#130708)
Optimize `LOAD_FAST` opcodes into faster versions that load borrowed references onto the operand stack when we can prove that the lifetime of the local outlives the lifetime of the temporary that is loaded onto the stack.
2025-04-01 10:18:42 -07:00
Martin DeMello 9c1e85fd64 gh-131740: minor readability fix in PyUnstable_GC_VisitObjects (gh-131786)
Minor readability fix in PyUnstable_GC_VisitObjects

Replaces `if (visit_generation())` with `if (visit_generation() < 0)`,
since we are checking for the failure case, and it's confusing to have
that be implicitly `true`.

Also fixes a misspelt variable name.
2025-03-29 08:58:17 +09:00
Donghee Na 7bb41aef4b gh-131740: Update PyUnstable_GC_VisitObjects to traverse perm gen (gh-131744) 2025-03-26 09:45:29 +09:00
Victor Stinner b69da006a4 gh-131238: Remove includes from pycore_interp.h (#131495)
Remove also now unused includes in C files.
2025-03-20 11:35:23 +00:00
Victor Stinner 22706843e0 gh-131238: Remove many includes from pycore_interp.h (#131472) 2025-03-19 17:46:24 +00:00
Mark Shannon 2bef8ea8ea GH-127705: Use _PyStackRefs in the default build. (GH-127875) 2025-03-10 14:06:56 +00:00
Sam Gross 0559339ccd gh-130019: Fix data race in _PyType_AllocNoTrack (gh-130058)
The reference count fields, such as `ob_tid` and `ob_ref_shared`, may be
accessed concurrently in the free threading build by a `_Py_TryXGetRef`
or similar operation. The PyObject header fields will be initialized by
`_PyObject_Init`, so only call `memset()` to zero-initialize the remainder
of the allocation.
2025-02-13 11:50:45 -05:00
Mark Shannon 72f56654d0 GH-128682: Account for escapes in DECREF_INPUTS (GH-129953)
* Handle escapes in DECREF_INPUTS

* Mark a few more functions as escaping

* Replace DECREF_INPUTS with PyStackRef_CLOSE where possible
2025-02-12 17:44:59 +00:00
Victor Stinner 3447f4a56a gh-129354: Use PyErr_FormatUnraisable() function (#129514)
Replace PyErr_WriteUnraisable() with PyErr_FormatUnraisable().
2025-01-31 14:20:35 +01:00
Victor Stinner 95504f429e gh-129354: Fix grammar in PyErr_FormatUnraisable() (#129475)
Replace "on verb+ing" with "while verb+ing".
2025-01-31 09:45:35 +01:00
Mark Shannon f5b6356a11 GH-128563: Add new frame owner type for interpreter entry frames (GH-129078)
Add new frame owner type for interpreter entry frames
2025-01-21 10:15:02 +00:00
Petr Viktorin 690fe077f6 gh-126491: Revert "GH-126491: Lower heap size limit with faster marking (GH-127519)" (GH-127770)
Revert "GH-126491: Lower heap size limit with faster marking (GH-127519)"

This reverts commit 023b7d2141, which introduced
a refleak.
2024-12-10 11:53:56 +01:00
Mark Shannon 023b7d2141 GH-126491: Lower heap size limit with faster marking (GH-127519)
* Faster marking of reachable objects

* Changes calculation of work to do and work done.

* Merges transitive closure calculations
2024-12-06 10:46:59 +00:00
Mark Shannon a8dd821d5b GH-126491: GC: Mark objects reachable from roots before doing cycle collection (GH-127110)
* Mark almost all reachable objects before doing collection phase

* Add stats for objects marked

* Visit new frames before each increment

* Update docs

* Clearer calculation of work to do.
2024-12-02 10:12:17 +00:00
Mark Shannon aea0c586d1 GH-127010: Don't lazily track and untrack dicts (GH-127027) 2024-11-20 16:41:20 +00:00
Mark Shannon a99dd23c1f GH-124567: Replace quadratic assert with linear one (GH-127009) 2024-11-19 13:38:59 +00:00
Hugo van Kemenade 899fdb213d Revert "GH-126491: GC: Mark objects reachable from roots before doing cycle collection (GH-126502)" (#126983) 2024-11-19 11:25:09 +02:00
Mark Shannon b0fcc2c47a GH-126491: GC: Mark objects reachable from roots before doing cycle collection (GH-126502)
* Mark almost all reachable objects before doing collection phase

* Add stats for objects marked

* Visit new frames before each increment

* Remove lazy dict tracking

* Update docs

* Clearer calculation of work to do.
2024-11-18 14:31:26 +00:00
Mark Shannon c0f045f7fd GH-124567: Reduce overhead of debug build for GC. Should help CI performance (GH-126777) 2024-11-15 08:59:01 +00:00
Neil Schemenauer 6f9525dd3f gh-116510: Fix crash during sub-interpreter shutdown (gh-124645)
Fix a bug that can cause a crash when sub-interpreters use "basic"
single-phase extension modules.  Shared objects could refer to PyGC_Head
nodes that had been freed as part of interpreter shutdown.
2024-09-26 19:33:07 -07:00
Sam Gross b2afe2aae4 gh-123923: Defer refcounting for f_executable in _PyInterpreterFrame (#123924)
Use a `_PyStackRef` and defer the reference to `f_executable` when
possible. This avoids some reference count contention in the common case
of executing the same code object from multiple threads concurrently in
the free-threaded build.
2024-09-12 12:37:06 -04:00
Mark Shannon f49a91648a GH-117759: Document incremental GC (GH-123266)
* Update what's new

* Update gc module docs and fix inconsistency in gc.get_objects
2024-08-27 15:23:39 +01:00
Mark Shannon 7cd3aa42f0 GH-122298: Restore printing of GC stats (GH-123261) 2024-08-23 16:39:08 +01:00
Mark Shannon a4fd7aa4a6 GH-115776: Allow any fixed sized object to have inline values (GH-123192) 2024-08-21 15:52:04 +01:00
Sam Gross e001027188 gh-117139: Garbage collector support for deferred refcounting (#122956)
The free-threaded GC now visits interpreter stacks to keep objects
that use deferred reference counting alive.

Interpreter frames are zero initialized in the free-threaded GC so
that the GC doesn't see garbage data. This is a temporary measure
until stack spilling around escaping calls is implemented.

Co-authored-by: Ken Jin <kenjin@python.org>
2024-08-15 16:09:11 +00:00
Victor Stinner c2d5df5787 gh-83754: Use the Py_TYPE() macro (#120599)
Don't access directly PyObject.ob_type, but use the Py_TYPE() macro
instead.
2024-06-17 10:34:29 +02:00
Xie Yanbo 9e052619a6 Fix typos in documentation and comments (#119763) 2024-06-04 10:22:22 +00:00
Victor Stinner aa61f8bfcf gh-110850: Remove _PyTime_TimeUnchecked() function (#118552)
Use the new public Raw functions:

* _PyTime_PerfCounterUnchecked() with PyTime_PerfCounterRaw()
* _PyTime_TimeUnchecked() with PyTime_TimeRaw()
* _PyTime_MonotonicUnchecked() with PyTime_MonotonicRaw()

Remove internal functions:

* _PyTime_PerfCounterUnchecked()
* _PyTime_TimeUnchecked()
* _PyTime_MonotonicUnchecked()
2024-05-05 12:15:19 +02:00
Mark Shannon c32dc47aca GH-115776: Embed the values array into the object, for "normal" Python objects. (GH-116115) 2024-04-02 11:59:21 +01:00
Guido van Rossum ddf814db74 Silence compiler warnings in gc.c (#117422) 2024-04-01 16:13:38 +00:00
Mark Shannon 8bef34f625 GH-117108: Set the "old space bit" to "visited" for all young objects (#117213)
Change old space bit of young objects from 0 to gcstate->visited_space.
This ensures that any object created *and* collected during cycle GC has the bit set correctly.
2024-03-26 11:11:42 +00:00
Mark Shannon 23e4f80ce2 A few minor tweaks to get stats working and compiling cleanly. (#117219)
Fixes a compilation error when configured with `--enable-pystats`,
an array size issue, and an unused variable.
2024-03-25 13:43:51 -07:00
Mark Shannon e28477f214 GH-117108: Change the size of the GC increment to about 1% of the total heap size. (GH-117120) 2024-03-22 18:43:25 +00:00
Mark Shannon 15309329b6 GH-108362: Incremental Cycle GC (GH-116206) 2024-03-20 08:54:42 +00:00
Pablo Galindo Salgado 02918aa961 gh-116604: Correctly honor the gc status when calling _Py_RunGC (#116628) 2024-03-12 12:00:49 +00:00
Victor Stinner 145bc2d638 gh-110850: Use public PyTime functions (#115746)
Replace private _PyTime functions with public PyTime functions.

random_seed_time_pid() now reports errors to its caller.
2024-02-20 23:31:30 +00:00
Victor Stinner 52d1477566 gh-110850: Rename internal PyTime C API functions (#115734)
Rename functions:

* _PyTime_GetSystemClock() => _PyTime_TimeUnchecked()
* _PyTime_GetPerfCounter() => _PyTime_PerfCounterUnchecked()
* _PyTime_GetMonotonicClock() => _PyTime_MonotonicUnchecked()
* _PyTime_GetSystemClockWithInfo() => _PyTime_TimeWithInfo()
* _PyTime_GetMonotonicClockWithInfo() => _PyTime_MonotonicWithInfo()
* _PyTime_GetMonotonicClockWithInfo() => _PyTime_MonotonicWithInfo()

Changes:

* Remove "typedef PyTime_t PyTime_t;" which was
  "typedef PyTime_t _PyTime_t;" before a previous rename.
* Update comments of "Unchecked" functions.
* Remove invalid PyTime_Time() comment.
2024-02-20 22:16:37 +00:00
Victor Stinner d207c7cd5a gh-110850: Cleanup pycore_time.h includes (#115724)
<pycore_time.h> include is no longer needed to get the PyTime_t type
in internal header files. This type is now provided by <Python.h>
include. Add <pycore_time.h> includes to C files instead.
2024-02-20 16:50:43 +00:00
Victor Stinner 9af80ec83d gh-110850: Replace _PyTime_t with PyTime_t (#115719)
Run command:

sed -i -e 's!\<_PyTime_t\>!PyTime_t!g' $(find -name "*.c" -o -name "*.h")
2024-02-20 15:02:27 +00:00