No public API change. Lift the per-iteration select/read/write loop out of
Popen._communicate (POSIX) into a module-level _communicate_io_posix(), with
small _flush_stdin / _make_input_view / _translate_newlines helpers alongside
it. Popen._communicate calls the helper and persists the returned input
offset for resume-after-timeout.
Retire the private Popen._remaining_time method in favor of module-level
_deadline_remaining; all call sites (POSIX and Windows) updated.
Defensive behavioural deltas: the stdin and stdout/stderr .close() calls in
the I/O loop now swallow BrokenPipeError / OSError, matching __exit__ and the
no-input path; previously these were bare.
Adds test_communicate_timeout_resume_partial_write to cover _input_offset
bookkeeping across TimeoutExpired/resume.
Changed condition to evaluate if timeout is less than or equals to 0. This is needed for simulated time environments such as Shadow where the time will match exactly on the boundary.
---------
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
On Windows, Popen._communicate() previously wrote to stdin synchronously, which could block indefinitely if the subprocess didn't consume input= quickly and the pipe buffer filled up. The timeout= parameter was only checked when joining the reader threads, not during the stdin write.
This change moves the Windows stdin writing to a background thread (similar to how stdout/stderr are read in threads), allowing the timeout to be properly enforced. If timeout expires, TimeoutExpired is raised promptly and the writer thread continues in the background. Subsequent calls to communicate() will join the existing writer thread.
Adds test_communicate_timeout_large_input to verify that TimeoutExpired is raised promptly when communicate() is called with large input and a timeout, even when the subprocess doesn't consume stdin quickly.
This test already passed on POSIX (where select() is used) but failed on Windows where the stdin write blocks without checking the timeout.
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Fix inconsistent subprocess.Popen.communicate() behavior between Windows
and POSIX when using memoryview objects with non-byte elements as input.
On POSIX systems, the code was incorrectly comparing bytes written against
element count instead of byte count, causing data truncation for large
inputs with non-byte element types.
Changes:
- Cast memoryview inputs to byte view when input is already a memoryview
- Fix progress tracking to use len(input_view) instead of len(self._input)
- Add comprehensive test coverage for memoryview inputs
🤖 Generated with [Claude Code](https://claude.ai/code)
Co-Authored-By: Claude <noreply@anthropic.com>
* old-man-yells-at-ReST
* Update 2025-05-30-18-37-44.gh-issue-134453.kxkA-o.rst
* assertIsNone review feedback
* fix memoryview_nonbytes test to fail without our fix on main, and have a nicer error.
Thanks to Peter Bierma @ZeroIntensity for the code review.
* gh-141473: Fix subprocess.Popen.communicate to send input to stdin
* Docs: Clarify that `input` is one time only on `communicate()`
* NEWS entry
* Add a regression test.
---------
Co-authored-by: Gregory P. Smith <greg@krypto.org>
* Add _PYTHON_SUBPROCESS_USE_POSIX_SPAWN environment knob
Add support for disabling the use of `posix_spawn` via a variable in
the process environment.
While it was previously possible to toggle this by modifying the value
of `subprocess._USE_POSIX_SPAWN`, this required either patching CPython
or modifying it within the interpreter instance which is not always
possible, such as when running applications or scripts not under a
user's control.
Signed-off-by: Vincent Fazio <vfazio@gmail.com>
* fixup NEWS entry
---------
Signed-off-by: Vincent Fazio <vfazio@gmail.com>
This flag was added as an escape hatch in gh-91401 and backported to
Python 3.10. The flag broke at some point between its addition and now.
As there is currently no publicly known environments that require this,
remove it rather than work on fixing it.
This leaves the flag in the subprocess module to not break code which
may have used / checked the flag itself.
discussion: https://discuss.python.org/t/subprocess-use-vfork-escape-hatch-broken-fix-or-remove/56915/2
Add __all__ to the following modules:
importlib.machinery, importlib.util and xml.sax.
Add also "# noqa: F401" in collections.abc,
subprocess and xml.sax.
* Sort __all__; remove collections.abc.__all__; remove private names
* Add tests
Explicitly handle the case where stdout=STDOUT
as otherwise the existing error handling gets
confused and reports hard to understand errors.
Signed-off-by: Paulo Neves <ptsneves@gmail.com>
In free-threaded builds, running with `PYTHON_GIL=0` will now disable the
GIL. Follow-up issues track work to re-enable the GIL when loading an
incompatible extension, and to disable the GIL by default.
In order to support re-enabling the GIL at runtime, all GIL-related data
structures are initialized as usual, and disabling the GIL simply sets a flag
that causes `take_gil()` and `drop_gil()` to return early.
Only set filename to cwd if it was caused by failed chdir(cwd).
_fork_exec() now returns "noexec:chdir" for failed chdir(cwd).
Co-authored-by: Robert O'Shea <PurityLake@users.noreply.github.com>
Add support for `os.POSIX_SPAWN_CLOSEFROM` and
`posix_spawn_file_actions_addclosefrom_np` and have the `subprocess` module use
them when available. This means `posix_spawn` can now be used in the default
`close_fds=True` situation on many platforms.
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
* Allow posix_spawn to inherit environment form parent environ variable.
With this change, posix_spawn call can behave similarly to execv with regards to environments when used in subprocess functions.
Refactor delete-safe symbol handling in subprocess.
Only module globals are force-cleared during interpreter finalization, using a class reference instead of individually listing the constants everywhere is simpler.
This fixes several ways file descriptors could be leaked from `subprocess.Popen` constructor during error conditions by opening them later and using a context manager "fds to close" registration scheme to ensure they get closed before returning.
---------
Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
Add the -P command line option and the PYTHONSAFEPATH environment
variable to not prepend a potentially unsafe path to sys.path.
* Add sys.flags.safe_path flag.
* Add PyConfig.safe_path member.
* Programs/_bootstrap_python.c uses config.safe_path=0.
* Update subprocess._optim_args_from_interpreter_flags() to handle
the -P command line option.
* Modules/getpath.py sets safe_path to 1 if a "._pth" file is
present.
One more thing that can help prevent people from using `preexec_fn`.
Also adds conditional skips to two tests exposing ASAN flakiness on the Ubuntu 20.04 Address Sanitizer Github CI system. When that build is run on more modern systems the "problem" does not show up. It seems ASAN implementation related.
Co-authored-by: Zackery Spytz <zspytz@gmail.com>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
Just in case there is ever an issue with _posixsubprocess's use of
vfork() due to the complexity of using it properly and potential
directions that Linux platforms where it defaults to on could take, this
adds a failsafe so that users can disable its use entirely by setting
a global flag.
No known reason to disable it exists. But it'd be a shame to encounter
one and not be able to use CPython without patching and rebuilding it.
See the linked issue for some discussion on reasoning.
Also documents the existing way to disable posix_spawn.
Removes the `list` call in the Popen `repr`.
Current implementation:
For cmd = `python --version`, with `shell=True`.
```bash
<Popen: returncode: None args: ['p', 'y', 't', 'h', 'o', 'n', ' ', '-', '-',...>
```
For `shell=False` and args=`['python', '--version']`, the output is correct:
```bash
<Popen: returncode: None args: ['python', '--version']>
```
With the new changes the `repr` yields:
For cmd = `python --version`, with `shell=True`:
```bash
<Popen: returncode: None args: 'python --version'>
```
For `shell=False` and args=`['python', '--version']`, the output:
```bash
<Popen: returncode: None args: ['python', '--version']>
```
Automerge-Triggered-By: GH:gpshead
See [PEP 597](https://www.python.org/dev/peps/pep-0597/).
* Add `-X warn_default_encoding` and `PYTHONWARNDEFAULTENCODING`.
* Add EncodingWarning
* Add io.text_encoding()
* open(), TextIOWrapper() emits EncodingWarning when encoding is omitted and warn_default_encoding is enabled.
* _pyio.TextIOWrapper() uses UTF-8 as fallback default encoding used when failed to import locale module. (used during building Python)
* bz2, configparser, gzip, lzma, pathlib, tempfile modules use io.text_encoding().
* What's new entry
Check to make sure stdout and stderr are not empty before selecting an item from them in Windows subprocess._communicate.
Co-authored-by: Gregory P. Smith <greg@krypto.org>
Gregory P. Smith