Tom Lane
9bcb8a694b
PL/Python set-returning functions can crash with a use-after-free when CREATE OR REPLACE FUNCTION is executed while the SRF is mid-iteration. The crash occurs because srfstate->savedargs is allocated in proc->mcxt, which gets deleted when the procedure is invalidated, leaving a dangling pointer that PLy_function_restore_args() then dereferences. The best fix is to use reference counting to prevent destroying the function state while it's still in use, similar to what PL/pgSQL has done. Rather than inventing a new wheel, this commit converts PL/Python to use the funccache.c infrastructure. The main challenge is that PL/Python uses SFRM_ValuePerCall for SRFs, where the handler is called multiple times. A naive implementation would allow the refcount to return to zero between calls, but we need to hang onto the original state and function body. SQL-language functions face the same challenge, so this commit follows the same approach used in functions.c: maintain a per-call-site cache struct (PLyProcedureCache) in fn_extra that holds both the pointer to the long-lived PLyProcedure and the SRF execution state. The use_count is incremented when we first obtain the procedure and is decremented via a MemoryContextCallback registered on fn_mcxt, which runs even during error aborts. Cleaning up the per-call SRF state needs more care: an ExprContextCallback handles the in-query cases, since the iterator is not guaranteed to run to completion (for example a LIMIT or a rescan can abandon it early). But unlike SQL functions, whose resources are released by transaction abort, PL/Python holds Python reference counts on the iterator and saved arguments that abort will not release, and ExprContextCallbacks are not invoked during an error abort. The MemoryContextCallback on fn_mcxt therefore doubles as the backstop that releases those references when a query errors out mid-iteration. Since fn_extra is now used for PLyProcedureCache, this commit removes use of the funcapi.h SRF infrastructure (SRF_IS_FIRSTCALL, SRF_RETURN_NEXT, etc.) and switches to direct isDone signaling via ReturnSetInfo, matching how SQL functions handle ValuePerCall mode. This fixes a longstanding bug, so ideally we'd back-patch it. But it'd be impractical to back-patch further than v18 where funccache.c came in. The patch is somewhat invasive, and the bug only arises in very uncommon usages (which is why it evaded detection for so long). On the whole, the risk/reward ratio for putting this into v18 doesn't seem good, so commit to master only. Bug: #19480 Reported-by: Andrzej Doros <adoros@starfishstorage.com> Author: Matheus Alcantara <matheusssilv97@gmail.com> Reviewed-by: Tom Lane <tgl@sss.pgh.pa.us> Discussion: https://postgr.es/m/19480-f1f9fdce30462fc4@postgresql.org
PostgreSQL Database Management System
This directory contains the source code distribution of the PostgreSQL database management system.
PostgreSQL is an advanced object-relational database management system that supports an extended subset of the SQL standard, including transactions, foreign keys, subqueries, triggers, user-defined types and functions. This distribution also contains C language bindings.
Copyright and license information can be found in the file COPYRIGHT.
General documentation about this version of PostgreSQL can be found at https://www.postgresql.org/docs/devel/. In particular, information about building PostgreSQL from the source code can be found at https://www.postgresql.org/docs/devel/installation.html.
The latest version of this software, and related software, may be obtained at https://www.postgresql.org/download/. For more information look at our web site located at https://www.postgresql.org/.