Commit Graph

11349 Commits

Author SHA1 Message Date
Victor Stinner a5447735c3 bpo-38392: Only declare visit_validate() if Py_DEBUG is defined (GH-16689)
bpo-38392, bpo-38426: Fix a compiler warning in gcmodule.c.

Fix also a typo in PYMEM_DEADBYTE macro comment.
2019-10-10 09:32:13 +02:00
Tim Peters ecbf35f933 bpo-38379: don't claim objects are collected when they aren't (#16658)
* bpo-38379:  when a finalizer resurrects an object,
nothing is actually collected in this run of gc.
Change the stats to relect that truth.
2019-10-09 12:37:30 -05:00
Antonio Gutierrez 594e2edfb5 closes bpo-36161: Use thread-safe ttyname_r instead of ttyname. (GH-14868)
Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
2019-10-08 19:19:48 -07:00
Vinay Sajip e8bedbddad bpo-38368: Added fix for ctypes crash when handling arrays in structs… (GH-16589) 2019-10-08 21:59:06 +01:00
Serhiy Storchaka d05b000c6b bpo-38371: Tkinter: deprecate the split() method. (GH-16584) 2019-10-08 14:31:35 +03:00
Antonio Gutierrez 0d3fe8ae49 closes bpo-38402: Check error of primitive crypt/crypt_r. (GH-16599)
Checks also for encryption algorithms methods not supported in different
OSs.

Signed-off-by: Antonio Gutierrez <chibby0ne@gmail.com>
2019-10-07 21:22:17 -07:00
Victor Stinner 4d5f94b8cd bpo-38070: Enhance visit_decref() debug trace (GH-16631)
subtract_refs() now pass the parent object to visit_decref() which
pass it to _PyObject_ASSERT(). So if the "is freed" assertion fails,
the parent is used in debug trace, rather than the freed object. The
parent object is more likely to contain useful information. Freed
objects cannot be inspected are are displayed as "<object at xxx is
freed>" with no other detail.
2019-10-08 02:37:38 +02:00
Victor Stinner 1b18455695 bpo-38392: PyObject_GC_Track() validates object in debug mode (GH-16615)
In debug mode, PyObject_GC_Track() now calls tp_traverse() of the
object type to ensure that the object is valid: test that objects
visited by tp_traverse() are valid.

Fix pyexpat.c: only track the parser in the GC once the parser is
fully initialized.
2019-10-08 00:09:31 +02:00
Victor Stinner 6876257eaa bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612)
bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is
now also available in release mode. For example, it can be used to
debug a crash in the visit_decref() function of the GC.

Modify the following functions to also work in release mode:

* _PyDict_CheckConsistency()
* _PyObject_CheckConsistency()
* _PyType_CheckConsistency()
* _PyUnicode_CheckConsistency()

Other changes:

* _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL
  (equals to 0).
* _PyBytesWriter_CheckConsistency() now returns 1 and is only used
  with assert().
* Reorder _PyObject_Dump() to write safe fields first, and only
  attempt to render repr() at the end.
2019-10-07 18:42:01 +02:00
Ben Harper 321def805a bpo-36356: Fix memory leak in _asynciomodule.c (GH-16598) 2019-10-07 12:19:58 -04:00
Xiang Zhang 303475e873 Fix a compile warning in selectmodule.c. (GH-16617) 2019-10-07 20:01:54 +08:00
Victor Stinner b96145a6b5 bpo-38353: Simplify calculate_pybuilddir() (GH-16614)
Calling _Py_wfopen() is enough to check if filename is an existing
file or not. There is no need to check first isfile().
2019-10-07 12:56:59 +02:00
Victor Stinner c02b41b1fb bpo-38353: getpath.c: allocates strings on the heap (GH-16585)
* _Py_FindEnvConfigValue() now returns a string allocated
  by PyMem_RawMalloc().
* calculate_init() now decodes VPATH macro.
* Add calculate_open_pyenv() function.
* Add substring() and joinpath2() functions.

* Fix add_exe_suffix()

And a few cleanup changes.
2019-10-04 19:53:43 +02:00
Victor Stinner abd7cd856b bpo-38353: getpath.c uses dynamically allocated strings (GH-16582)
* PyCalculatePath: add "_macro" to the 3 fields storing macro values
* Add dynamically allocated strings to PyCalculatePath
2019-10-04 15:25:50 +02:00
Serhiy Storchaka 06cb94bc84 bpo-13153: Use OS native encoding for converting between Python and Tcl. (GH-16545)
On Windows use UTF-16 (or UTF-32 for 32-bit Tcl_UniChar) with the
"surrogatepass" error handler for converting to/from Tcl Unicode objects.

On Linux use UTF-8 with the "surrogateescape" error handler for converting
to/from Tcl String objects.

Converting strings from Tcl to Python and back now never fails
(except MemoryError).
2019-10-04 13:09:52 +03:00
Victor Stinner f0c85793d2 bpo-38353: Rework ismodule() in getpath.c (GH-16574)
* ismodule() no longer modify the input string.
* copy_absolute(), joinpath(): rename parameters and local variables.
2019-10-04 03:53:26 +02:00
Victor Stinner 03a8a56fac bpo-38353: Add subfunctions to getpath.c (GH-16572)
Following symbolic links is now limited to 40 attempts, just to
prevent loops.

Add subfunctions:

* Add resolve_symlinks()
* Add calculate_argv0_path_framework()
* Add calculate_which()
* Add calculate_program_macos()

Fix also _Py_wreadlink(): readlink() result type is Py_ssize_t, not
int.
2019-10-04 02:22:39 +02:00
Victor Stinner 86ec5c65fe bpo-38353: Fix calculate_argv0_path() for symlinks (GH-16549)
calculate_argv0_path() must read argv0_path link, not read
program_full_path link.
2019-10-03 23:17:00 +02:00
Victor Stinner 61691d8336 bpo-38353: Cleanup includes in the internal C API (GH-16548)
Use forward declaration of types to avoid includes in the internal C
API. Add also comment to justify other includes.
2019-10-02 23:51:20 +02:00
Victor Stinner 5e0ea7540f bpo-37474: Don't call fedisableexcept() on FreeBSD (GH-16515)
On FreeBSD, Python no longer calls fedisableexcept() at startup to
control the floating point control mode. The call became useless
since FreeBSD 6: it became the default mode.
2019-10-01 13:12:29 +02:00
Victor Stinner 2f90261280 bpo-38321: Fix compiler warning in _randommodule.c (GH-16512)
Fix the GCC warning: "initialization discards ‘const’ qualifier from
pointer target type".
2019-10-01 12:45:52 +02:00
Victor Stinner 8462a4936b bpo-38304: PyConfig_InitPythonConfig() cannot fail anymore (GH-16509)
PyConfig_InitPythonConfig() and PyConfig_InitIsolatedConfig() no
longer return PyStatus: they cannot fail anymore.
2019-10-01 12:06:16 +02:00
Victor Stinner 3c30a76f3d bpo-38304: Remove PyConfig.struct_size (GH-16500) (GH-16508)
For now, we'll rely on the fact that the config structures aren't covered by the stable ABI.

We may revisit this in the future if we further explore the idea of offering a stable embedding API.

(cherry picked from commit bdace21b76)
2019-10-01 10:56:37 +02:00
Victor Stinner aca8c406ad bpo-38321: Fix _testcapimodule.c warning (GH-16494)
Fix the following warning:

    modules\_testcapimodule.c(6409):
    warning C4146: unary minus operator applied to unsigned type,
    result still unsigned
2019-09-30 21:14:26 +02:00
Neil Schemenauer bcda460baf Clear weakrefs in garbage found by the GC (#16495)
Fix a bug due to the interaction of weakrefs and the cyclic garbage
collector. We must clear any weakrefs in garbage in order to prevent
their callbacks from executing and causing a crash.
2019-09-30 19:06:45 +02:00
Victor Stinner c9a413ede4 bpo-38321: Fix PyCStructUnionType_update_stgdict() warning (GH-16492)
bpo-22273, bpo-38321: Fix following warning:

    modules\_ctypes\stgdict.c(704):
    warning C4244: 'initializing': conversion from 'Py_ssize_t' to 'int', possible loss of data
2019-09-30 16:49:47 +01:00
Victor Stinner efe74b6369 bpo-38321: Fix _asynciomodule.c compiler warning (GH-16493)
bpo-38248, bpo-38321: Fix warning:

    modules\_asynciomodule.c(2667):
    warning C4102: 'set_exception': unreferenced label

The related goto has been removed by
commit edad4d89e3.
2019-09-30 16:41:34 +02:00
Victor Stinner 441b10cf28 bpo-38304: Add PyConfig.struct_size (GH-16451)
Add a new struct_size field to PyPreConfig and PyConfig structures to
allow to modify these structures in the future without breaking the
backward compatibility.

* Replace private _config_version field with public struct_size field
  in PyPreConfig and PyConfig.
* Public PyPreConfig_InitIsolatedConfig() and
  PyPreConfig_InitPythonConfig()
  return type becomes PyStatus, instead of void.
* Internal _PyConfig_InitCompatConfig(),
  _PyPreConfig_InitCompatConfig(), _PyPreConfig_InitFromConfig(),
  _PyPreConfig_InitFromPreConfig() return type becomes PyStatus,
  instead of void.
* Remove _Py_CONFIG_VERSION
* Update the Initialization Configuration documentation.
2019-09-28 04:28:35 +02:00
Christian Heimes 9055815809 bpo-38270: More fixes for strict crypto policy (GH-16418)
test_hmac and test_hashlib test built-in hashing implementations and
OpenSSL-based hashing implementations. Add more checks to skip OpenSSL
implementations when a strict crypto policy is active.

Use EVP_DigestInit_ex() instead of EVP_DigestInit() to initialize the
EVP context. The EVP_DigestInit() function clears alls flags and breaks
usedforsecurity flag again.

Signed-off-by: Christian Heimes <christian@python.org>



https://bugs.python.org/issue38270
2019-09-27 06:03:53 -07:00
Benjamin Peterson 52b9408038 closes bpo-38174: Update vendored expat library to 2.2.8. (GH-16346)
Fixes CVE-2019-15903. See full changelog at https://github.com/libexpat/libexpat/blob/R_2_2_8/expat/Changes.
2019-09-25 21:33:58 -07:00
Christian Heimes df69e75edc bpo-38142: Updated _hashopenssl.c to be PEP 384 compliant (#16071)
* Updated _hashopenssl.c to be PEP 384 compliant
* Remove refleak test from test_hashlib. The updated type no longer accepts random arguments to __init__.
2019-09-25 23:03:30 +02:00
Yury Selivanov edad4d89e3 bpo-38248: Fix inconsistent immediate asyncio.Task cancellation (GH-16330) 2019-09-25 03:32:08 -07:00
Vinay Sajip c64af8fad3 Changed conditions for ctypes array-in-struct handling. (GH-16381) 2019-09-25 11:11:57 +01:00
Vinay Sajip 57dc7d5ae8 bpo-22273: Disabled tests while investigating buildbot failures on ARM7L/PPC64. (GH-16377) 2019-09-25 07:58:32 +01:00
Dong-hee Na ad7736faf5 bpo-38265: Update os.pread to accept the length type as Py_ssize_t. (GH-16359) 2019-09-25 08:47:04 +03:00
Vinay Sajip 12f209eccb bpo-22273: Update ctypes to correctly handle arrays in small structur… (GH-15839) 2019-09-25 04:38:44 +01:00
Victor Stinner 221fd84703 bpo-38234: Cleanup getpath.c (GH-16367)
* search_for_prefix() directly calls reduce() if found is greater
  than 0.
* Add calculate_pybuilddir() subfunction.
* search_for_prefix(): add path string buffer for readability.
* Fix some error handling code paths: release resources on error.
* calculate_read_pyenv(): rename tmpbuffer to filename.
* test.pythoninfo now also logs windows.dll_path
2019-09-25 02:54:25 +02:00
Victor Stinner bb6bf7d342 bpo-38234: Add tests for Python init path config (GH-16358) 2019-09-24 18:21:02 +02:00
Victor Stinner 85ce0a7178 bpo-38234: read_pth_file() now returns PyStatus (GH-16338)
Refactor path configuration code:

* read_pth_file() now returns PyStatus to report errors, rather than
  calling Py_FatalError().
* Move argv0_path and zip_path buffers out of PyCalculatePath
  structures.
* On Windows, _PyPathConfig.home is now preferred over PyConfig.home.
2019-09-24 00:55:48 +02:00
Victor Stinner 9c42f8cda5 bpo-38234: Fix _PyConfig_InitPathConfig() (GH-16335)
* _PyConfig_InitPathConfig() now starts by copying the global path
  configuration, and then override values set in PyConfig.
* _PyPathConfig_Calculate() implementations no longer override
  _PyPathConfig fields which are already computed. For example,
  if _PyPathConfig.prefix is not NULL, leave it unchanged.
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() doesn't
  call _PyPathConfig_Calculate() anymore.
* _PyPathConfig_Calculate() no longer uses PyConfig,
  except to initialize PyCalculatePath structure.
* pathconfig_calculate(): remove useless temporary
  "_PyPathConfig new_config" variable.
* calculate_module_search_path(): remove hack to workaround memory
  allocation failure, call Py_FatalError() instead.
* Fix get_program_full_path(): handle memory allocation failure.
2019-09-23 18:47:29 +02:00
Victor Stinner e267793aa4 bpo-38234: Fix PyConfig_Read() when Py_SetPath() was called (GH-16298)
* If Py_SetPath() has been called, _PyConfig_InitPathConfig() now
  uses its value.
* Py_Initialize() now longer copies path configuration from PyConfig
  to the global path configuration (_Py_path_config).
2019-09-21 01:50:16 +02:00
Orivej Desh 77abf23c67 bpo-6559: Update _posixsubprocess.fork_exec doc (GH-16283)
It did not list the argument added in d4cc7bf993.


https://bugs.python.org/issue6559



Automerge-Triggered-By: @gpshead
2019-09-20 10:01:09 -07:00
Eddie Elizondo 3368f3c6ae bpo-38140: Make dict and weakref offsets opaque for C heap types (#16076)
* Make dict and weakref offsets opaque for C heap types

* Add news
2019-09-19 17:29:05 +01:00
Victor Stinner 0a963fbc9c bpo-38203: faulthandler.dump_traceback_later() is always available (GH-16249)
dump_traceback_later() and cancel_dump_traceback_later() functions of
the faulthandler module are always available since Python 3.7.
2019-09-18 14:15:10 +02:00
Serhiy Storchaka f669581a95 bpo-38185: Fixed case-insensitive string comparison in sqlite3.Row indexing. (GH-16190) 2019-09-17 09:20:56 +03:00
Christian Heimes 724f1a5723 bpo-33936: Don't call obsolete init methods with OpenSSL 1.1.0+ (GH-16140)
``OPENSSL_VERSION_1_1`` was never defined in ``_hashopenssl.c``. 


https://bugs.python.org/issue33936
2019-09-16 12:10:05 -07:00
Serhiy Storchaka 8debfa5040 bpo-38175: Fix a memory leak in comparison of sqlite3.Row objects. (GH-16155) 2019-09-16 20:15:18 +03:00
Hai Shi 56a45142e7 Fix a possbile refleak in setint() of mmapmodule.c (GH-16136) 2019-09-16 13:56:57 +08:00
Raymond Hettinger 6e27a0d775 bpo-37798: Prevent undefined behavior in direct calls to the C helper function. (#16149) 2019-09-15 09:36:38 -07:00
Dino Viehland 09dc2c672f Fix missing dec ref (#16158) 2019-09-15 15:51:44 +01:00