02c1abfc54
gh-69528: Distinguish between file modes "wb+" and "rb+" (GH-137834)
...
Co-authored-by: Xiang Zhang <angwerzx@126.com >
2025-10-21 20:33:30 +03:00
d301587369
gh-140135: Use PyBytesWriter in io.RawIOBase.readall(); 4x faster ( #140139 )
2025-10-15 14:04:17 +00:00
9b35f7cdfe
gh-129813, PEP 782: Use PyBytesWriter in bufferedio.c ( #139121 )
...
Replace PyBytes_FromStringAndSize(NULL, size) and _PyBytes_Resize()
with the new public PyBytesWriter API.
2025-09-18 17:00:10 +02:00
db68bfc771
gh-138720: Make Buffered closed check match flush (GH-138724)
...
In `_io__Buffered_flush_impl` the macro `CHECK_CLOSED` is used to check
the `buffered*` is in a good state to be flushed. That differs slightly
from `buffered_closed`.
In some cases, that difference would result in `close()` thinking the
file needed to be flushed and closed while `flush()` thought the file
was already closed.
This could happen during GC and would result in an unraisable exception.
2025-09-18 12:02:29 +02:00
4554486f15
gh-129813, PEP 782: Use PyBytesWriter in bufferedio.c ( #138954 )
...
Replace PyBytes_FromStringAndSize(NULL, size) and _PyBytes_Resize()
with the new public PyBytesWriter API.
2025-09-15 21:50:09 +00:00
e814e6b38b
gh-129813, PEP 782: Use PyBytesWriter in FileIO.read() ( #138955 )
...
Replace PyBytes_FromStringAndSize(NULL, size) and _PyBytes_Resize()
with the new public PyBytesWriter API.
2025-09-15 23:49:28 +02:00
71defb6943
gh-129813, PEP 782: Use PyBytesWriter in _io.FileIO.readall ( #138901 )
...
Performance about the same, using the benchmark from gh-120754:
```
pyperf compare_to main.json pep782.json
read_file_small: Mean +- std dev: [main] 5.71 us +- 0.05 us -> [pep782] 5.68 us +- 0.05 us: 1.01x faster
read_file_large: Mean +- std dev: [main] 89.7 us +- 0.9 us -> [pep782] 86.9 us +- 0.8 us: 1.03x faster
read_all_rst_bytes: Mean +- std dev: [main] 926 us +- 8 us -> [pep782] 920 us +- 12 us: 1.01x faster
read_all_rst_text: Mean +- std dev: [main] 2.24 ms +- 0.02 ms -> [pep782] 2.17 ms +- 0.04 ms: 1.03x faster
Benchmark hidden because not significant (1): read_all_py_bytes
Geometric mean: 1.01x faster
```
2025-09-15 22:38:01 +02:00
918e3ba6c0
GH-137623: Use an AC decorator for docstring line length enforcement ( #137690 )
2025-08-18 18:29:00 +01:00
d8fa40b08d
gh-132551: add missing critical sections on BytesIO methods ( #137073 )
2025-07-24 11:57:48 +00:00
b1056c2a44
gh-135607: remove null checking of weakref list in dealloc of extension modules and objects ( #135614 )
...
Co-authored-by: Kumar Aditya <kumaraditya@python.org >
Co-authored-by: Victor Stinner <vstinner@python.org >
2025-06-30 11:14:31 +00:00
73431356d3
Revert "gh-135410: use a critical section around StringIO.__next__ ( #135412 )" ( #135439 )
...
This reverts commit e6c3039cb3
2025-06-13 07:32:27 +09:00
e6c3039cb3
gh-135410: use a critical section around StringIO.__next__ ( #135412 )
2025-06-12 16:41:05 +05:30
44fb7c361c
gh-134908: Protect textiowrapper_iternext with critical section (gh-134910)
...
The `textiowrapper_iternext` function called `_textiowrapper_writeflush`, but did not
use a critical section, making it racy in free-threaded builds.
2025-06-02 17:22:41 +00:00
c45e661226
gh-131505: Move len boundary assertions before using len. ( #131536 )
...
Move len boundary assertions before using len.
2025-05-19 08:10:23 -07:00
2cd24ebfe9
fix thread safety of io.StringIO.truncate ( #133732 )
2025-05-09 07:59:17 +00:00
5dd3a3a58c
gh-132551: make io.BytesIO thread safe ( #132616 )
...
Co-authored-by: Kumar Aditya <kumaraditya@python.org >
2025-05-08 18:51:36 +00:00
e9253ebf74
gh-122559: Synchronize C and Python implementation of the io module about pickling (GH-122628)
...
In the C implementation, remove __reduce__ and __reduce_ex__ methods
that always raise TypeError and restore __getstate__ methods that always
raise TypeErrori.
This restores fine details of the pre-3.12 behavior and unifies
both implementations.
2025-05-04 14:06:49 +00:00
4e294f6feb
gh-133036: Deprecate codecs.open ( #133038 )
...
Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com >
Co-authored-by: Victor Stinner <vstinner@python.org >
2025-04-30 10:11:09 +09:00
58567cc18c
gh-132952: Speed up startup by importing _io instead of io ( #132957 )
2025-04-28 08:38:56 -07:00
edbf7fb129
gh-111178: remove redundant casts for functions with correct signatures ( #131673 )
2025-04-01 17:18:11 +02:00
8614f86b71
gh-131525: Cache the result of tuple_hash ( #131529 )
...
* gh-131525: Cache the result of tuple_hash
* Fix debug builds
* Add blurb
* Fix formatting
* Pre-compute empty tuple singleton
* Mostly set the cache within tuple_alloc
* Fixes for TSAN
* Pre-compute empty tuple singleton
* Fix for 32-bit platforms
* Assert that op != NULL in _PyTuple_RESET_HASH_CACHE
* Use FT_ATOMIC_STORE_SSIZE_RELAXED macro
* Update Include/internal/pycore_tuple.h
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com >
* Fix alignment
* atomic load
* Update Objects/tupleobject.c
Co-authored-by: Chris Eibl <138194463+chris-eibl@users.noreply.github.com >
---------
Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com >
Co-authored-by: Chris Eibl <138194463+chris-eibl@users.noreply.github.com >
2025-03-27 09:57:06 -04:00
486d537065
Fix Windows build warnings ( #131487 )
...
Fix the following warnings:
* Modules\_io\fileio.c(1296,13): unused variable 'self'
* Modules\_io\winconsoleio.c(334,9): unused variable 'fd_is_own'
* Modules\faulthandler.c(409,11): unused variable 'flags'
* Modules\posixmodule.c(5699,9): unused variable 'pathError'
* PC\winreg.c(2077,5): suggest braces around initialization of
subobject
* PC\winreg.c(34,13): unused variable 'errNotAHandle'
* Python\fileutils.c(132,12): result of comparison of constant
1114111 with expression of type 'wchar_t' (aka 'unsigned short') is
always false
* Python\fileutils.c(58,21): unused variable 'INCOMPLETE_CHARACTER'
* Python\sysmodule.c(2534,21): unused variable 'perf_map_state'
2025-03-20 12:03:20 +01:00
20c5f969dd
gh-131238: Remove more includes from pycore_interp.h ( #131480 )
2025-03-19 23:01:32 +01:00
f5e4c2955b
gh-111178: Fix function signatures for test_iter ( #131456 )
2025-03-19 14:42:51 +01:00
a45f25361d
GH-131238: More refactoring of core header files (GH-131351)
...
Adds new pycore_stats.h header file to help break dependencies involving the pycore_code.h header.
2025-03-17 14:41:05 +00:00
978e37bb5f
gh-131238: Add explicit includes to pycore headers ( #131257 )
2025-03-17 12:32:43 +01:00
9a63138e09
gh-111178: Fix function signatures in misc files ( #131180 )
2025-03-13 16:55:08 +01:00
9d759b63d8
gh-111178: Change Argument Clinic signature for METH_O ( #130682 )
...
Use "PyObject*" for METH_O functions to fix an undefined behavior.
2025-03-11 16:33:36 +01:00
de8818ae23
gh-131082: Add missing guards for WIN32_LEAN_AND_MEAN ( #131044 )
...
* Add missing guards for WIN32_LEAN_AND_MEAN
* add missing whitespaces
2025-03-11 12:33:01 +01:00
b1b4f9625c
gh-117151: IO performance improvement, increase io.DEFAULT_BUFFER_SIZE to 128k (GH-118144)
...
Co-authored-by: rmorotti <romain.morotti@man.com >
2025-03-07 11:36:12 -08:00
886a4d74ee
gh-129011: Update comments in FileIO to match current code ( #129012 )
2025-03-06 17:18:22 -08:00
3a7f17c7e2
gh-130790: Remove references about unicode's readiness from comments ( #130801 )
2025-03-03 19:18:09 +00:00
1988003625
gh-111178: fix UBSan failures in Modules/_io/*.c (GH-129083)
...
* fix UBSan failures for `buffered`, `rwpair`, `bytesio`, `bytesiobuf`, `iobase`, `stringio`, `nldecoder_object`, `textio`, `winconsoleio`
* arg names: use 'dummy' for NOARGS method and 'args' for others
2025-02-08 14:21:32 +01:00
14489c1bb4
gh-129354: Use PyErr_FormatUnraisable() function ( #129656 )
...
Replace PyErr_WriteUnraisable() with PyErr_FormatUnraisable().
2025-02-04 16:21:10 +01:00
732670d93b
gh-119182: Use public PyUnicodeWriter in stringio.c ( #129243 )
...
Replace the private _PyUnicodeWriter API with the public
PyUnicodeWriter API.
2025-01-23 23:31:49 +00:00
537296cdcd
gh-111178: Generate correct signature for most self converters ( #128447 )
2025-01-20 12:40:18 +01:00
b5d1e4552f
gh-128083: Fix macro redefinition warning in clinic. (GH-127950)
2024-12-19 15:00:30 +01:00
31f16e427b
gh-109523: Raise a BlockingIOError if reading text from a non-blocking stream cannot immediately return bytes. (GH-122933)
2024-12-02 14:18:30 +01:00
99490913a0
gh-127341: Argument Clinic: fix compiler warnings for getters with docstrings ( #127310 )
...
Co-authored-by: Erlend E. Aasland <erlend@python.org >
2024-11-29 11:12:13 +01:00
3cf83d91a5
gh-124008: Fix calculation of the number of written bytes for the Windows console (GH-124059)
...
Since MultiByteToWideChar()/WideCharToMultiByte() is not reversible if
the data contains invalid UTF-8 sequences, use binary search to
calculate the number of written bytes from the number of written
characters.
Also fix writing incomplete UTF-8 sequences.
Also fix handling of memory allocation failures.
2024-11-27 13:38:12 +02:00
a2ee899682
gh-127182: Fix io.StringIO.__setstate__ crash when None is the first value ( #127219 )
...
Co-authored-by: Victor Stinner <vstinner@python.org >
2024-11-25 20:32:02 +03:00
061e50f196
gh-122943: Add the varpos parameter in _PyArg_UnpackKeywords (GH-126564)
...
Remove _PyArg_UnpackKeywordsWithVararg.
Add comments for integer arguments of _PyArg_UnpackKeywords.
2024-11-08 14:23:50 +02:00
72dd4714f9
gh-120754: _io Ensure stat cache is cleared on fd change ( #125166 )
...
Performed an audit of `fileio.c` and `_pyio` and made sure anytime the
fd changes the stat result, if set, is also cleared/changed.
There's one case where it's not cleared, if code would clear it in
__init__, keep the memory allocated and just do another fstat with the
existing memory.
2024-11-01 22:50:49 +01:00
ebcc578dff
gh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR) ( #125583 )
...
Replace PyUnicode_FromStringAndSize(NULL, 0)
with Py_GetConstant(Py_CONSTANT_EMPTY_STR).
2024-10-25 11:14:52 +02:00
b9a8ca0a6a
gh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_STR) ( #125194 )
...
Replace PyUnicode_New(0, 0), PyUnicode_FromString("")
and PyUnicode_FromStringAndSize("", 0)
with Py_GetConstant(Py_CONSTANT_EMPTY_STR).
2024-10-09 17:15:23 +02:00
6a39e96ab8
gh-115754: Use Py_GetConstant(Py_CONSTANT_EMPTY_BYTES) ( #125195 )
...
Replace PyBytes_FromString("") and PyBytes_FromStringAndSize("", 0)
with Py_GetConstant(Py_CONSTANT_EMPTY_BYTES).
2024-10-09 17:12:11 +02:00
cc9b9bebb2
gh-90102: Remove isatty call during regular open ( #124922 )
...
Co-authored-by: Victor Stinner <vstinner@python.org >
2024-10-08 08:50:42 +02:00
d8f707420b
gh-111178: Fix function signatures in fileio.c ( #125043 )
...
* Add "fileio_" prefix to getter functions.
* Small refactoring.
2024-10-07 15:27:36 +02:00
43cd7aa8cd
gh-120754: Fix memory leak in FileIO.__init__() ( #124225 )
...
Free 'self->stat_atopen' before assigning it, since
io.FileIO.__init__() can be called multiple times manually
(especially by test_io).
2024-09-19 00:11:50 +02:00
8b6c7c7877
gh-120754: Refactor I/O modules to stash whole stat result rather than individual members ( #123412 )
...
Multiple places in the I/O stack optimize common cases by using the
information from stat. Currently individual members are extracted from
the stat and stored into the fileio struct. Refactor the code to store
the whole stat struct instead.
Parallels the changes to _io. The `stat` Python object doesn't allow
changing members, so rather than modifying estimated_size, just clear
the value.
2024-09-18 17:47:57 +02:00
Stan Ulbrych