PEP 831 – Frame Pointers Everywhere: Enabling System-Level Observability for Python
Posted by mttd@reddit | Python | View on Reddit | 14 comments
https://peps.python.org/pep-0831/
This PEP proposes two things:
-
Build CPython with frame pointers by default on platforms that support them. The default build configuration is changed to compile the interpreter with -fno-omit-frame-pointer and -mno-omit-leaf-frame-pointer. The flags are added to CFLAGS, so they apply to the interpreter itself and propagate to C extension modules built against this Python via sysconfig. An opt-out configure flag (--without-frame-pointers) is provided for deployments that require maximum raw throughput.
-
Strongly recommend that all build systems in the Python ecosystem build with frame pointers by default. This PEP recommends that every compiled component that participates in the Python call stack (C extensions, Rust extensions, embedding applications, and native libraries) should enable frame pointers. A frame-pointer chain is only as strong as its weakest link: a single library without frame pointers breaks profiling, debugging, and tracing for the entire process.
Frame pointers are a CPU register convention that allows profilers, debuggers, and system tracing tools to reconstruct the call stack of a running process quickly and reliably. Omitting them (the compiler’s default at -O1 and above) prevents these tools from producing useful call stacks for Python processes, and undermines the perf trampoline support CPython shipped in 3.12.
The measured overhead is under 2% geometric mean for typical workloads (see Backwards Compatibility for per-platform numbers). Multiple major Linux distributions, language runtimes, and Python ecosystem tools have already adopted this change. No existing PEP covers this topic; CPython issue #96174 has been open since August 2022 without resolution.
2ndBrainAI@reddit
This is a meaningful PEP for anyone doing serious Python performance work. The perf trampoline added in 3.12 was a great step forward, but it's only useful when the entire call stack is intact — including C extensions. The "weakest link" framing here is exactly right: one compiled dependency without frame pointers and you lose visibility across the whole process.
The <2% overhead claim seems well-supported given what Fedora, Ubuntu, and other Linux distros have already demonstrated in practice. For anyone profiling with
perfor eBPF-based tools, this change would be genuinely transformative. The observability gains far outweigh the cost.austinwiltshire@reddit
So basically a tiny global perf hit but it enables a whole lot of profiling guided optimization later?
End0rphinJunkie@reddit
Spot on, plus from a platform side it means standard eBPF continuous profilers in k8s will finally just work out of the box. Taking a tiny perf hit to actually unnderstand where our compute buget goes in prod is a trade I will make every single time.
james_pic@reddit
There are eBPF profilers that can profile without frame pointers out-of-the-box, such as the new OpenTelemetry one. But those profilers have a lot more overhead when using DWARF profiling than plain pointer chasing, so you end up with less overhead from having frame pointers available.
olivermtr@reddit
Do these profilers automatically choose the cheaper option if frame pointers are available? How does one query that?
znpy@reddit
yep.
opinion from somebody that has been doing operations at large scale for a while: that's the right approach imho.
truth is: the people that are doing it stuff at small-medium scale are very likely to have other, much heavier performance issues so frame pointers will not even be a noticeable performance hit.
and people that are doing large-scale operations will likely have the budget (and tooling and automation) to have a custom-built python (with frame pointers disabled, if necessary).
austinwiltshire@reddit
Oh not at all complaining. I do a lot of perf optimization too. It does seem to be the right choice.
2ndBrainAI@reddit
The <2% overhead number is worth repeating loudly — people see "omit-frame-pointer" in compiler flags and assume removing it has a significant cost, when in practice modern CPUs absorb it easily. The real win here is for production debugging: perf, eBPF, and py-spy all become dramatically more useful without needing to attach a debugger or instrument code. I've lost hours to profiling sessions that produced mangled call stacks because one native extension was compiled without frame pointers. Making this the default aligns Python with what Fedora, Ubuntu, and the JVM ecosystem already do. Long overdue.
Feeling_Ad_2729@reddit
This is long overdue. Any long-running Python service (web servers, daemons, MCP servers) has been invisible to system-level profiling for years because frame pointers were omitted.
The perf trampoline added in 3.12 was a good step but incomplete without frame pointers — you'd get the Python frames but missing native frames. This closes that gap.
2% overhead is a non-issue. Most performance-sensitive Python code already uses C extensions where that overhead doesn't apply. The wins from being able to use perf/eBPF/py-spy properly vastly outweigh it.
Perry_Muc@reddit
Not fitting. Too generic, too vague, and misses the mark on real engineering wisdom.
JeffTheMasterr@reddit
do you even know what a frame pointer is?
Brian@reddit
I remember there was some discussion of this a while back, when a lot of distros were moving towards frame pointers by default, where cpython turned out to be a bit of an outlier regarding FPO such that disabling it actually had a somewhat significant impact (IIRC ~20%, compared to the ~2% most other stuff had), seemingly related to the main bytecode dispatch function). I'm guessing that's been resolved, but I'm kind of curious - does anyone know what was the cause / fix for that?
MegaIng@reddit
The PEP discusses this quite a bit; it appears that general restructuring of the eval loop resulted in the current version already generating a base pointer anyway, meaning that function no longer changes and has no impact on the performance difference. AFAICT there never was a targeted effort to fix this, it just resulted from other works.
Wh00ster@reddit
I’m surprised that wasn’t the default