mirror of
https://github.com/python/cpython.git
synced 2026-05-06 12:49:07 -04:00
gh-149202: Fix frame pointer unwinding on s390x and ARM (GH-149362)
-fno-omit-frame-pointer is not enough to make every target walkable by the simple manual frame pointer unwinder. The helper used by test_frame_pointer_unwind used to assume the frame pointer named a two-word record where fp[0] was the previous frame pointer and fp[1] was the return address. That is only the generic layout used by some targets. This patch keeps that default, but moves the slots behind named offsets so architecture-specific layouts can describe where the backchain and return address really live. On s390x, GCC and Clang do not emit a usable backchain unless -mbackchain is enabled. Without it, the unwinder stops at the current C frame and the test reports no Python frames. Once backchains are present, the helper must also stop at the current thread's known C stack bounds; otherwise it can follow the final backchain far enough to dereference an invalid frame and segfault. For Linux s390x backchain frames, the documented z/Architecture stack-frame layout saves r14, the return-address register, at byte offset 112 from the frame pointer, so read the return address from that named slot instead of fp[1]. The 112-byte offset comes from Linux's s390 debugging documentation: its Stack Frame Layout table shows z/Architecture backchain frames with the backchain at offset 0 and saved r14 of the caller function at offset 112: https://www.kernel.org/doc/html/v5.3/s390/debugging390.html#stack-frame-layout This helper remains scoped to Linux s390x backchain frames. GNU SFrame's s390x notes state that the s390x ELF ABI does not generally mandate where RA and FP are saved, or whether they are saved at all: https://sourceware.org/binutils/docs/sframe-spec.html#s390x As Jens Remus noted, -fno-omit-frame-pointer is not needed when -mbackchain is present. On 32-bit ARM, GCC defaults to Thumb mode on common armhf toolchains. The Thumb prologue keeps the saved frame pointer and link register at offsets that depend on the generated frame, which breaks the fp[0]/fp[1] walk used by the helper. Use -marm when it is supported for frame-pointer builds, and teach the helper the GCC ARM-mode slots where the previous frame pointer is at fp[-1] and the saved LR return address is at fp[0]. Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Victor Stinner <vstinner@python.org>
This commit is contained in:
committed by
GitHub
parent
646853df13
commit
4ed40146f1
@@ -218,8 +218,8 @@ How to obtain the best results
|
||||
------------------------------
|
||||
|
||||
For best results, keep frame pointers enabled. On supported GCC-compatible
|
||||
toolchains, CPython builds itself with ``-fno-omit-frame-pointer`` and, when
|
||||
available, ``-mno-omit-leaf-frame-pointer`` by default. These flags allow
|
||||
toolchains, CPython builds itself with ``-fno-omit-frame-pointer`` and similar
|
||||
flags (see :option:`--without-frame-pointers` for details). These flags allow
|
||||
profilers to unwind using only the frame pointer and not on DWARF debug
|
||||
information. This is because as the code that is interposed to allow ``perf``
|
||||
support is dynamically generated it doesn't have any DWARF debugging information
|
||||
|
||||
+13
-5
@@ -784,11 +784,19 @@ also be used to improve performance.
|
||||
|
||||
Disable frame pointers, which are enabled by default (see :pep:`831`).
|
||||
|
||||
By default, the build appends ``-fno-omit-frame-pointer`` (and
|
||||
``-mno-omit-leaf-frame-pointer`` when the compiler supports it) to
|
||||
``BASECFLAGS`` so profilers, debuggers, and system tracing tools
|
||||
(``perf``, ``eBPF``, ``dtrace``, ``gdb``) can walk the C call stack
|
||||
without DWARF metadata. The flags propagate to third-party C
|
||||
By default, the build appends flags to generate frame or backchain
|
||||
pointers to ``BASECFLAGS``:
|
||||
|
||||
- ``-fno-omit-frame-pointer`` and/or ``-mno-omit-leaf-frame-pointer``
|
||||
are added when the compiler supports them.
|
||||
- ``-marm`` is added on 32-bit ARM when supported,
|
||||
- on s390x platforms, when supported, ``-mbackchain`` is added *instead*.
|
||||
of the above frame pointer flags.
|
||||
|
||||
Frame pointers enable profilers, debuggers, and system tracing tools
|
||||
(``perf``, ``eBPF``, ``dtrace``, ``gdb``) to walk the C call stack
|
||||
without DWARF metadata.
|
||||
The flags propagate to third-party C
|
||||
extensions through :mod:`sysconfig`. On compilers that do not
|
||||
understand them, the build silently skips them.
|
||||
|
||||
|
||||
@@ -2552,6 +2552,19 @@ Build changes
|
||||
and :option:`-X dev <-X>` is passed to the Python or Python is built in :ref:`debug mode <debug-build>`.
|
||||
(Contributed by Donghee Na in :gh:`141770`.)
|
||||
|
||||
.. _whatsnew315-frame-pointers:
|
||||
|
||||
* CPython is now built with frame pointers enabled by default
|
||||
(:pep:`831`). Pass :option:`--without-frame-pointers` to opt out.
|
||||
|
||||
Authors of C extensions and native libraries built with custom build
|
||||
systems should ensure the unwind chain is intact.
|
||||
This is usually done by adding ``-fno-omit-frame-pointer`` and
|
||||
similar flags to ``CFLAGS``. See :option:`--without-frame-pointers`
|
||||
documentation for the specific flags Python uses.
|
||||
|
||||
(Contributed by Pablo Galindo Salgado and Savannah Ostrowski in :gh:`149201`.)
|
||||
|
||||
.. _whatsnew315-windows-tail-calling-interpreter:
|
||||
|
||||
* 64-bit builds using Visual Studio 2026 (MSVC 18) may now use the new
|
||||
|
||||
@@ -21,6 +21,16 @@ STACK_DEPTH = 10
|
||||
|
||||
|
||||
def _frame_pointers_expected(machine):
|
||||
_Py_WITH_FRAME_POINTERS = getattr(
|
||||
_testinternalcapi,
|
||||
"_Py_WITH_FRAME_POINTERS",
|
||||
-1,
|
||||
)
|
||||
if _Py_WITH_FRAME_POINTERS > 0:
|
||||
return True
|
||||
if _Py_WITH_FRAME_POINTERS == 0:
|
||||
return False
|
||||
|
||||
cflags = " ".join(
|
||||
value for value in (
|
||||
sysconfig.get_config_var("PY_CORE_CFLAGS"),
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
Enable frame pointers by default for GCC-compatible CPython builds, including
|
||||
``-mno-omit-leaf-frame-pointer`` when the compiler supports it, so profilers
|
||||
and debuggers can unwind native interpreter frames more reliably. Users can pass
|
||||
``--without-frame-pointers`` to opt out.
|
||||
``-mno-omit-leaf-frame-pointer``, ``-marm`` on 32-bit ARM, and/or ``-mbackchain``
|
||||
on s390x platforms when the compiler supports them, so profilers and debuggers
|
||||
can unwind native interpreter frames more reliably. Users can pass
|
||||
:option:`--without-frame-pointers` to ``./configure`` to opt out.
|
||||
|
||||
+139
-16
@@ -63,6 +63,40 @@
|
||||
static const uintptr_t min_frame_pointer_addr = 0x1000;
|
||||
#define MAX_UNWIND_FRAMES 200
|
||||
|
||||
#ifdef __s390x__
|
||||
// Linux's s390 "Stack Frame Layout" table documents that z/Architecture
|
||||
// backchain frames start with the backchain at offset 0 and store "saved r14
|
||||
// of caller function" at offset 112. The same document's register table
|
||||
// identifies r14 as the return-address register, so this backchain unwinder
|
||||
// reads the return address from fp + 112.
|
||||
// https://www.kernel.org/doc/html/v5.3/s390/debugging390.html#stack-frame-layout
|
||||
//
|
||||
// This is only for Linux s390x backchain frames. The s390x ELF ABI does not
|
||||
// generally mandate where RA and FP are saved, or whether they are saved at all.
|
||||
// https://sourceware.org/binutils/docs/sframe-spec.html#s390x
|
||||
# define S390X_FRAME_RETURN_ADDRESS_OFFSET 112
|
||||
#endif
|
||||
|
||||
// The generic manual unwinder treats the frame pointer as a two-word record:
|
||||
// fp[0] is the previous frame pointer and fp[1] is the return address. That is
|
||||
// not true for every architecture, even with frame pointers enabled, so these
|
||||
// offsets describe the actual slots used by each supported frame layout.
|
||||
#if defined(__arm__) && !defined(__thumb__) && !defined(__clang__)
|
||||
// GCC ARM mode keeps the caller's fp one word below fp and the saved LR at
|
||||
// fp[0], so the return address is not in the generic fp[1] slot.
|
||||
# define FRAME_POINTER_NEXT_OFFSET (-1)
|
||||
# define FRAME_POINTER_RETURN_OFFSET 0
|
||||
#elif defined(__s390x__)
|
||||
// s390x backchain frames keep the previous frame pointer at fp[0], but save the
|
||||
// return-address register in the ABI register save area rather than fp[1].
|
||||
# define FRAME_POINTER_NEXT_OFFSET 0
|
||||
# define FRAME_POINTER_RETURN_OFFSET \
|
||||
(S390X_FRAME_RETURN_ADDRESS_OFFSET / (Py_ssize_t)sizeof(uintptr_t))
|
||||
#else
|
||||
# define FRAME_POINTER_NEXT_OFFSET 0
|
||||
# define FRAME_POINTER_RETURN_OFFSET 1
|
||||
#endif
|
||||
|
||||
|
||||
static PyObject *
|
||||
_get_current_module(void)
|
||||
@@ -329,15 +363,96 @@ get_jit_backend(PyObject *self, PyObject *Py_UNUSED(args))
|
||||
#endif
|
||||
}
|
||||
|
||||
static int
|
||||
stack_address_is_valid(uintptr_t addr, uintptr_t stack_min, uintptr_t stack_max)
|
||||
{
|
||||
if (addr < min_frame_pointer_addr) {
|
||||
return 0;
|
||||
}
|
||||
if (stack_min != 0 && (addr < stack_min || addr >= stack_max)) {
|
||||
return 0;
|
||||
}
|
||||
return 1;
|
||||
}
|
||||
|
||||
static int
|
||||
frame_pointer_slot_is_valid(uintptr_t *frame_pointer, Py_ssize_t offset,
|
||||
uintptr_t stack_min, uintptr_t stack_max)
|
||||
{
|
||||
uintptr_t fp_addr = (uintptr_t)frame_pointer;
|
||||
uintptr_t slot_addr;
|
||||
uintptr_t delta = (uintptr_t)Py_ABS(offset) * sizeof(uintptr_t);
|
||||
if (offset < 0) {
|
||||
if (fp_addr < delta) {
|
||||
return 0;
|
||||
}
|
||||
slot_addr = fp_addr - delta;
|
||||
}
|
||||
else {
|
||||
if (fp_addr > UINTPTR_MAX - delta) {
|
||||
return 0;
|
||||
}
|
||||
slot_addr = fp_addr + delta;
|
||||
}
|
||||
if (!stack_address_is_valid(slot_addr, stack_min, stack_max)) {
|
||||
return 0;
|
||||
}
|
||||
if (stack_max != 0) {
|
||||
if (slot_addr > UINTPTR_MAX - sizeof(uintptr_t)) {
|
||||
return 0;
|
||||
}
|
||||
if (slot_addr + sizeof(uintptr_t) > stack_max) {
|
||||
return 0;
|
||||
}
|
||||
}
|
||||
return 1;
|
||||
}
|
||||
|
||||
static int
|
||||
next_frame_pointer_is_valid(uintptr_t *frame_pointer, uintptr_t *next_fp,
|
||||
uintptr_t stack_min, uintptr_t stack_max)
|
||||
{
|
||||
uintptr_t fp_addr = (uintptr_t)frame_pointer;
|
||||
uintptr_t next_addr = (uintptr_t)next_fp;
|
||||
if (!stack_address_is_valid(next_addr, stack_min, stack_max)) {
|
||||
return 0;
|
||||
}
|
||||
if ((next_addr % sizeof(uintptr_t)) != 0) {
|
||||
return 0;
|
||||
}
|
||||
#if _Py_STACK_GROWS_DOWN
|
||||
return next_addr > fp_addr;
|
||||
#else
|
||||
return next_addr < fp_addr;
|
||||
#endif
|
||||
}
|
||||
|
||||
static PyObject *
|
||||
manual_unwind_from_fp(uintptr_t *frame_pointer)
|
||||
{
|
||||
int stack_grows_down = _Py_STACK_GROWS_DOWN;
|
||||
uintptr_t stack_min = 0;
|
||||
uintptr_t stack_max = 0;
|
||||
|
||||
#ifdef __s390x__
|
||||
Py_BUILD_ASSERT(S390X_FRAME_RETURN_ADDRESS_OFFSET % sizeof(uintptr_t) == 0);
|
||||
#endif
|
||||
|
||||
if (frame_pointer == NULL) {
|
||||
return PyList_New(0);
|
||||
}
|
||||
|
||||
PyThreadState *tstate = _PyThreadState_GET();
|
||||
if (tstate != NULL) {
|
||||
_PyThreadStateImpl *tstate_impl = (_PyThreadStateImpl *)tstate;
|
||||
#if _Py_STACK_GROWS_DOWN
|
||||
stack_min = tstate_impl->c_stack_hard_limit;
|
||||
stack_max = tstate_impl->c_stack_top;
|
||||
#else
|
||||
stack_min = tstate_impl->c_stack_top;
|
||||
stack_max = tstate_impl->c_stack_hard_limit;
|
||||
#endif
|
||||
}
|
||||
|
||||
PyObject *result = PyList_New(0);
|
||||
if (result == NULL) {
|
||||
return NULL;
|
||||
@@ -357,7 +472,21 @@ manual_unwind_from_fp(uintptr_t *frame_pointer)
|
||||
MAX_UNWIND_FRAMES);
|
||||
return NULL;
|
||||
}
|
||||
uintptr_t return_addr = frame_pointer[1];
|
||||
if (!stack_address_is_valid(fp_addr, stack_min, stack_max)) {
|
||||
break;
|
||||
}
|
||||
if (!frame_pointer_slot_is_valid(frame_pointer,
|
||||
FRAME_POINTER_NEXT_OFFSET,
|
||||
stack_min, stack_max)) {
|
||||
break;
|
||||
}
|
||||
if (!frame_pointer_slot_is_valid(frame_pointer,
|
||||
FRAME_POINTER_RETURN_OFFSET,
|
||||
stack_min, stack_max)) {
|
||||
break;
|
||||
}
|
||||
uintptr_t *next_fp = (uintptr_t *)frame_pointer[FRAME_POINTER_NEXT_OFFSET];
|
||||
uintptr_t return_addr = frame_pointer[FRAME_POINTER_RETURN_OFFSET];
|
||||
|
||||
PyObject *addr_obj = PyLong_FromUnsignedLongLong(return_addr);
|
||||
if (addr_obj == NULL) {
|
||||
@@ -372,22 +501,10 @@ manual_unwind_from_fp(uintptr_t *frame_pointer)
|
||||
Py_DECREF(addr_obj);
|
||||
depth++;
|
||||
|
||||
uintptr_t *next_fp = (uintptr_t *)frame_pointer[0];
|
||||
// Stop if the frame pointer is extremely low.
|
||||
if ((uintptr_t)next_fp < min_frame_pointer_addr) {
|
||||
if (!next_frame_pointer_is_valid(frame_pointer, next_fp,
|
||||
stack_min, stack_max)) {
|
||||
break;
|
||||
}
|
||||
uintptr_t next_addr = (uintptr_t)next_fp;
|
||||
if (stack_grows_down) {
|
||||
if (next_addr <= fp_addr) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
else {
|
||||
if (next_addr >= fp_addr) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
frame_pointer = next_fp;
|
||||
}
|
||||
|
||||
@@ -3170,6 +3287,12 @@ module_exec(PyObject *module)
|
||||
return 1;
|
||||
}
|
||||
|
||||
#ifdef _Py_WITH_FRAME_POINTERS
|
||||
if (PyModule_AddIntMacro(module, _Py_WITH_FRAME_POINTERS) < 0) {
|
||||
return 1;
|
||||
}
|
||||
#endif
|
||||
|
||||
return 0;
|
||||
}
|
||||
|
||||
|
||||
@@ -10343,6 +10343,106 @@ else case e in #(
|
||||
esac
|
||||
fi
|
||||
|
||||
case $host_cpu in #(
|
||||
arm|armv*) :
|
||||
|
||||
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking whether C compiler accepts -marm" >&5
|
||||
printf %s "checking whether C compiler accepts -marm... " >&6; }
|
||||
if test ${ax_cv_check_cflags__Werror__marm+y}
|
||||
then :
|
||||
printf %s "(cached) " >&6
|
||||
else case e in #(
|
||||
e)
|
||||
ax_check_save_flags=$CFLAGS
|
||||
CFLAGS="$CFLAGS -Werror -marm"
|
||||
cat confdefs.h - <<_ACEOF >conftest.$ac_ext
|
||||
/* end confdefs.h. */
|
||||
|
||||
int
|
||||
main (void)
|
||||
{
|
||||
|
||||
;
|
||||
return 0;
|
||||
}
|
||||
_ACEOF
|
||||
if ac_fn_c_try_compile "$LINENO"
|
||||
then :
|
||||
ax_cv_check_cflags__Werror__marm=yes
|
||||
else case e in #(
|
||||
e) ax_cv_check_cflags__Werror__marm=no ;;
|
||||
esac
|
||||
fi
|
||||
rm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext
|
||||
CFLAGS=$ax_check_save_flags ;;
|
||||
esac
|
||||
fi
|
||||
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: $ax_cv_check_cflags__Werror__marm" >&5
|
||||
printf "%s\n" "$ax_cv_check_cflags__Werror__marm" >&6; }
|
||||
if test "x$ax_cv_check_cflags__Werror__marm" = xyes
|
||||
then :
|
||||
|
||||
frame_pointer_cflags="$frame_pointer_cflags -marm"
|
||||
|
||||
else case e in #(
|
||||
e) : ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
;; #(
|
||||
*) :
|
||||
;;
|
||||
esac
|
||||
case $host_cpu in #(
|
||||
s390*) :
|
||||
|
||||
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: checking whether C compiler accepts -mbackchain" >&5
|
||||
printf %s "checking whether C compiler accepts -mbackchain... " >&6; }
|
||||
if test ${ax_cv_check_cflags__Werror__mbackchain+y}
|
||||
then :
|
||||
printf %s "(cached) " >&6
|
||||
else case e in #(
|
||||
e)
|
||||
ax_check_save_flags=$CFLAGS
|
||||
CFLAGS="$CFLAGS -Werror -mbackchain"
|
||||
cat confdefs.h - <<_ACEOF >conftest.$ac_ext
|
||||
/* end confdefs.h. */
|
||||
|
||||
int
|
||||
main (void)
|
||||
{
|
||||
|
||||
;
|
||||
return 0;
|
||||
}
|
||||
_ACEOF
|
||||
if ac_fn_c_try_compile "$LINENO"
|
||||
then :
|
||||
ax_cv_check_cflags__Werror__mbackchain=yes
|
||||
else case e in #(
|
||||
e) ax_cv_check_cflags__Werror__mbackchain=no ;;
|
||||
esac
|
||||
fi
|
||||
rm -f core conftest.err conftest.$ac_objext conftest.beam conftest.$ac_ext
|
||||
CFLAGS=$ax_check_save_flags ;;
|
||||
esac
|
||||
fi
|
||||
{ printf "%s\n" "$as_me:${as_lineno-$LINENO}: result: $ax_cv_check_cflags__Werror__mbackchain" >&5
|
||||
printf "%s\n" "$ax_cv_check_cflags__Werror__mbackchain" >&6; }
|
||||
if test "x$ax_cv_check_cflags__Werror__mbackchain" = xyes
|
||||
then :
|
||||
|
||||
frame_pointer_cflags="-mbackchain"
|
||||
|
||||
else case e in #(
|
||||
e) : ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
;; #(
|
||||
*) :
|
||||
;;
|
||||
esac
|
||||
|
||||
else case e in #(
|
||||
e) : ;;
|
||||
@@ -10351,6 +10451,9 @@ fi
|
||||
|
||||
if test -n "$frame_pointer_cflags" && test "x$with_frame_pointers" != xno; then
|
||||
BASECFLAGS="$frame_pointer_cflags $BASECFLAGS"
|
||||
|
||||
printf "%s\n" "#define _Py_WITH_FRAME_POINTERS 1" >>confdefs.h
|
||||
|
||||
fi
|
||||
|
||||
CFLAGS_NODIST="$CFLAGS_NODIST -std=c11"
|
||||
|
||||
@@ -2548,9 +2548,23 @@ AS_VAR_IF([ac_cv_gcc_compat], [yes], [
|
||||
AX_CHECK_COMPILE_FLAG([-mno-omit-leaf-frame-pointer], [
|
||||
frame_pointer_cflags="$frame_pointer_cflags -mno-omit-leaf-frame-pointer"
|
||||
], [], [-Werror])
|
||||
AS_CASE([$host_cpu], [arm|armv*], [
|
||||
AX_CHECK_COMPILE_FLAG([-marm], [
|
||||
frame_pointer_cflags="$frame_pointer_cflags -marm"
|
||||
], [], [-Werror])
|
||||
])
|
||||
AS_CASE([$host_cpu], [s390*], [
|
||||
AX_CHECK_COMPILE_FLAG([-mbackchain], [
|
||||
dnl Do not use no-omit-frame-pointer; see gh-149362
|
||||
frame_pointer_cflags="-mbackchain"
|
||||
], [], [-Werror])
|
||||
])
|
||||
], [], [-Werror])
|
||||
if test -n "$frame_pointer_cflags" && test "x$with_frame_pointers" != xno; then
|
||||
BASECFLAGS="$frame_pointer_cflags $BASECFLAGS"
|
||||
AC_DEFINE([_Py_WITH_FRAME_POINTERS], [1],
|
||||
[Define to 1 if frame unwinding via pointers is expected
|
||||
to work, 0 if not. Leave undefined if unknown.])
|
||||
fi
|
||||
|
||||
CFLAGS_NODIST="$CFLAGS_NODIST -std=c11"
|
||||
|
||||
@@ -2073,6 +2073,10 @@
|
||||
/* Define if you want to use tail-calling interpreters in CPython. */
|
||||
#undef _Py_TAIL_CALL_INTERP
|
||||
|
||||
/* Define to 1 if frame unwinding via pointers is expected to work, 0 if not.
|
||||
Leave undefined if unknown. */
|
||||
#undef _Py_WITH_FRAME_POINTERS
|
||||
|
||||
/* Define to force use of thread-safe errno, h_errno, and other functions */
|
||||
#undef _REENTRANT
|
||||
|
||||
|
||||
Reference in New Issue
Block a user