gh-150743: Limit trailer lines and interim responses read by http.client (GH-150741)
http.client read chunked-response trailer lines and skipped interim (1xx)
responses in unbounded loops, so a server streaming either forever would
hang the client even with a socket timeout set (data keeps arriving, so
the timeout never fires).
Trailer lines are now limited to max_response_headers (100 by default)
and interim responses to 100; HTTPException is raised past either limit.
Follow-up to gh-88188 for CVE-2021-3737, which bounded header lines
within an interim response but not these two sibling loops.
(cherry picked from commit 41cc78a7a4)
Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
---
This issue was reported to us via [GHSA-w4q2-g22w-6fr4](https://github.com/python/cpython/security/advisories/GHSA-w4q2-g22w-6fr4) and was determined not to be high enough severity to handle privately.
Co-authored-by: Gregory P. Smith <68491+gpshead@users.noreply.github.com>
gh-149144: Use decodeURIComponent() for UTF-8 support in js_output() (GH-149157)
(cherry picked from commit 461b1d9631)
Co-authored-by: Seth Larson <seth@python.org>
Reject control characters in `http.cookies.Morsel.update()` and `http.cookies.BaseCookie.js_output`.
Co-authored-by: Victor Stinner <vstinner@python.org>
Co-authored-by: Victor Stinner <victor.stinner@gmail.com>
Reading the whole body of the HTTP response could cause OOM if
the Content-Length value is too large even if the server does not send
a large amount of data. Now the HTTP client reads large data by chunks,
therefore the amount of consumed memory is proportional to the amount
of sent data.
Update selected RFC 2822 references to RFC 5322
RFC 2822 was obsoleted by RFC 5322 in 2008. This updates references
to use the current standard in documentation, docstrings, and comments.
It preserves RFC 2822 references in legacy API components to maintain their
historical context.
RFC 822 → RFC 2822 → RFC 5322 progression is explained where relevant.
In some places specific sections of RFC are referenced where it seems helpful.
Scout rule was applied in some places and RFC mentions format was
normalized in doc strings and comments.
Remove 'response_class' from getresponse docstring.
This variable is not documented as part of the API in the standard
library documentation; it should be considered as an implementation
detail and as such should not be included in the doc string.
Closes#57665.
Prior to issue #120485 these servers did not allow port reuse, which
makes sense as the behavior of port reuse is surprising if you're not
expecting it. It's unclear to me why these services were switched to
allow port reuse, but I believe the desired behavior (unless subclasses
opt in) is to not allow port reuse.
See also: https://bugzilla.redhat.com/show_bug.cgi?id=2323170
* Always quote strings with non-ASCII characters.
* Allow some non-separator and non-control characters (like "." or "-")
be unquoted.
* Always quote strings that end with "\n".
* Use the fullmatch() method for clarity and optimization.
The `http.server` module now supports serving over HTTPS using the `http.server.HTTPSServer` class.
This functionality is also exposed by the command-line interface (`python -m http.server`) through the
`--tls-cert`, `--tls-key` and `--tls-password-file` options.
* Revert "gh-128982: Substitute regular expression in `http.cookiejar.join_header_words` for an efficient alternative (GH-128983)"
This reverts commit 56e1900681.
* Add tests
The parameter `amt` of `HTTPResponse.read()`, which could be a negative integer,
has not been handled before and led to waiting for the connection to close
for `keep-alive connections`. Now, this has been fixed, and passing negative values
to `HTTPResponse().read()` works the same as passing `None` value.
* Add support for 'partitioned' attribute in http.cookies
Co-authored-by: Giles Copp <gilesc@dropbox.com>
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
As reported in #117847 and #115366, an unpaired backtick in a docstring
tends to confuse e.g. Sphinx running on subclasses of standard library
objects, and the typographic style of using a backtick as an opening
quote is no longer in favor. Convert almost all uses of the form
The variable `foo' should do xyz
to
The variable 'foo' should do xyz
and also fix up miscellaneous other unpaired backticks (extraneous /
missing characters).
No functional change is intended here other than in human-readable
docstrings.
rfc9110 obsoletes the earlier rfc 7231. This document also includes some
status codes that were previously only used for WebDAV and assigns more
generic names to these status codes.
ref: https://www.rfc-editor.org/rfc/rfc9110.html#name-changes-from-rfc-7231
- http.HTTPStatus.CONTENT_TOO_LARGE (413, previously
REQUEST_ENTITY_TOO_LARGE)
- http.HTTPStatus.URI_TOO_LONG (414, previously REQUEST_URI_TOO_LONG)
- http.HTTPStatus.RANGE_NOT_SATISFYABLE (416, previously
REQUEST_RANGE_NOT_SATISFYABLE)
- http.HTTPStatus.UNPROCESSABLE_CONTENT (422, previously
UNPROCESSABLE_ENTITY)
The new constants are added to http.HTTPStatus and the old constant names are
preserved for backwards compatibility.
References in documentation to the obsoleted rfc 7231 are updated
Omit the `@interface_scope` from an IPv6 address when used as Host header by `http.client`.
---------
Co-authored-by: Gregory P. Smith <greg@krypto.org> [Google LLC]
Miss Islington (bot)