Commit Graph

9258 Commits

Author SHA1 Message Date
Mike Bayer 54a2991b50 experiment with alternate ways of detecting correct GC condtiions
Change-Id: If56ec107c0fd34333a48281486ac6e6fbc80cdde
2025-10-28 09:27:08 -04:00
Mike Bayer 293af0439d more cache_key coverage
Change-Id: I68ed84c1617b4d15b0103cf4a3debd1da05c6c97
2025-10-27 12:18:56 -04:00
Mike Bayer 3db9c6be2d try to improve coverage a bit
use pyproject.toml so the options are safe from "rm .coverage*"
types of commands

omit some files

add test coverage for a big section of cache-key

Change-Id: Ia540c632d91ec09284e187e3edeb8ccf0214e1a6
2025-10-27 10:56:49 -04:00
Mike Bayer a5ccd8e9ef repair / modernize binary literal round trip test
update this test to use modern literal round trip
now that execute_compiled is gone

Fixes: #12940
Change-Id: I587dc6845fa8ff078baf3f6e08e54ce6cec84630
2025-10-26 09:43:21 -04:00
Michael Bayer 3eb9308def Merge "Support for Create Table As" into main 2025-10-25 20:18:32 +00:00
Greg Jarzab 7f5e9e733c Support for Create Table As
Added support for the SQL ``CREATE TABLE ... AS SELECT`` construct via the
new :class:`_sql.CreateTableAs` DDL construct and the
:meth:`_sql.SelectBase.into` method. The new construct allows creating a
table directly from the results of a SELECT statement, with support for
options such as ``TEMPORARY`` and ``IF NOT EXISTS`` where supported by the
target database.  Pull request courtesy Greg Jarzab.

Fixes: #4950
Closes: #12860
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12860
Pull-request-sha: 7de8a109b8

Change-Id: Id9c8e4a3c520ffc61de1e48e331b6220e3d52fc9
2025-10-25 14:24:46 -04:00
Federico Caselli 413a87093c postfix of I5f0fe800e31aac052926cebe9206763eef9a804c
Change-Id: Ic007a5bd842207a2c5a56374953e48c6acc7b254
2025-10-24 23:54:48 +02:00
Michael Bayer 70041d4c9c Merge "proposal: remove _execute_compiled()" into main 2025-10-24 21:49:22 +00:00
Michael Bayer 33f93745bc Merge "improve sqlite reflection regex for unusual names/formats" into main 2025-10-24 14:57:33 +00:00
Mike Bayer 189907be93 improve sqlite reflection regex for unusual names/formats
A series of improvements have been made for reflection of CHECK constraints
on SQLite. The reflection logic now correctly handles table names
containing the strings "CHECK" or "CONSTRAINT", properly supports all four
SQLite identifier quoting styles (double quotes, single quotes, brackets,
and backticks) for constraint names, and accurately parses CHECK constraint
expressions containing parentheses within string literals using balanced
parenthesis matching with string context tracking.    Big thanks to
GruzdevAV for new test cases and implementation ideas.

Fixes: #12924
Change-Id: I0390ac334c98e934c7e0353f47c9f43204791af5
2025-10-24 09:54:44 -04:00
Mike Bayer 87e27693ad proposal: remove _execute_compiled()
this isn't used internally and only allows execute(stmt.compile())
to work, which is not something I want to support.

Since people are definitely using this [1] we might want to think about
either deprectation or having a recipe for people who really want
to still do this

[1] https://github.com/sqlalchemy/sqlalchemy/discussions/11108?converting=1

Change-Id: I5f0fe800e31aac052926cebe9206763eef9a804c
2025-10-23 21:38:15 +02:00
Mike Bayer 6023557539 ensure explicit left side considered for ON clause in join_from
Fixed an issue in :meth:`_sql.Select.join_from` where the join condition
between the left and right tables specified in the method call could be
incorrectly determined based on an intermediate table already present in
the FROM clause, rather than matching the foreign keys between the
immediate left and right arguments. The join condition is now determined by
matching primary keys between the two tables explicitly passed to
:meth:`_sql.Select.join_from`, ensuring consistent and predictable join
behavior regardless of the order of join operations or other tables present
in the query.  The fix is applied to both the Core and ORM implementations
of :meth:`_sql.Select.join_from`.

Fixes: #12931
Change-Id: Id457d441ee8a1bd64a002e57a9dcfd6fc56ff15e
2025-10-21 09:21:29 -04:00
Rebecca Chen 0ab0d0c980 Change typing tests to use assert_type instead of reveal_type
Closes: #12922
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12922
Pull-request-sha: 580f663816

Change-Id: I9f3bdb4c105971f53fa10ed8a934356203ddb080
2025-10-18 11:23:20 -04:00
Mike Bayer b8906e62ce re-document 12915 as not fully fixed
The ORM side of this issue can't be fixed without rewriting
.params() which will be 2.1 only, so clarify changelog

References: #12915
Change-Id: I2e524d5390241aa289786f524b6a51f39bc09876
2025-10-17 08:57:10 -04:00
Michael Bayer caf2b25c38 Merge "fully copy_internals for AnnotatedFromClause for straight cloned traverse" into main 2025-10-15 20:56:13 +00:00
Mike Bayer fc1fef7a5c fixes for 3.14t on github actions
Change-Id: Ibc99e3dfdfd2a516c336090570df64875d97b653
2025-10-15 16:52:54 -04:00
Mike Bayer 03116b8cc9 fully copy_internals for AnnotatedFromClause for straight cloned traverse
Fixed issue where using :meth:`_sql.Select.params` to replace bound
parameters in a query could fail for some cases where the parameters
were embedded in subqueries or CTEs when ORM classes were involved,
due to issues with internal query traversal for these cases.

Fixes: #12915
Change-Id: Ib63bca786a541682f6b2144fd5dd43350411ae9d
2025-10-15 15:21:08 -04:00
Michael Bayer 79969acaed Merge "Support VIRTUAL computed columns on PostgreSQL." into main 2025-10-14 23:07:05 +00:00
Federico Caselli 8d560a0aa5 Support VIRTUAL computed columns on PostgreSQL.
Support for ``VIRTUAL`` computed columns on PostgreSQL 18 and later has
been added. The default behavior when :paramref:`.Computed.persisted` is
not specified has been changed to align with PostgreSQL 18's default of
``VIRTUAL``. When :paramref:`.Computed.persisted` is not specified, no
keyword is rendered on PostgreSQL 18 and later; on older versions a
warning is emitted and ``STORED`` is used as the default. To explicitly
request ``STORED`` behavior on all PostgreSQL versions, specify
``persisted=True``.

Fixes: #12866
Change-Id: Ic2ebdbe79e230a88370cf2b3503d2d1815f72a39
2025-10-14 21:21:30 +00:00
Federico Caselli 1dd96d274d postgresql dialect table options
Support for storage parameters in ``CREATE TABLE`` using the ``WITH``
clause has been added. The ``postgresql_with`` dialect option of
:class:`_schema.Table` accepts a mapping of key/value options.

The PostgreSQL dialect now support reflection of table options, including
the storage parameters, table access method and table spaces. These options
are automatically reflected when autoloading a table, and are also
available via the :meth:`_engine.Inspector.get_table_options` and
:meth:`_engine.Inspector.get_multi_table_optionsmethod` methods.

Fixes: #10909
Closes: #12584
Closes: #12684
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12584
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12684
Pull-request-sha: a660459de9

Change-Id: I33e3d8b8eb0c02530933cb124e3d375ca4af7db2
2025-10-14 19:47:26 +02:00
Federico Caselli 5b49b59ab7 Improve postgresql reflection
Some improvements to the reflection of PostgreSQL to avoid
unnecessary queries when not needed.

Fixes: #12908
Change-Id: Ic3da50ee43670f26d3159f5ec9a235a9a1963b8a
2025-10-14 13:27:18 +00:00
Denis Laxalde 78af7c4f04 Reflect collation in types on PostgreSQL
Added support for reflection of collation in types for PostgreSQL.
The ``collation`` will be set only if different from the default
one for the type.

References #6511
Closes: #12510
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12510
Pull-request-sha: c0a390314e

Change-Id: I269a194d526a0689a4b38f10456d28539c73cffb
2025-10-14 09:26:38 -04:00
Mike Bayer 833953138f the gc_intensive blocks will continue until builds improve
Change-Id: Ie62a2371eb1512ecde3a886f11a4b3de6d6cde3d
2025-10-10 08:22:22 -04:00
Mike Bayer 0486e6eaf8 mark more GC sensitive tests as not open for github
Change-Id: Ib73ec54e4ed7ef2d8911bfe8ee7a719022c6cddd
2025-10-09 15:56:25 -04:00
Allen Chen 22ede6eedc Fix mssql index column order
Fixed issue where the index reflection for SQL Server would
not correctly return the order of the column inside an index
when the order of the columns in the index did not match the
order of the columns in the table.
Pull request courtesy of Allen Chen.

Fixes: #12894
Closes: #12895
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12895
Pull-request-sha: bd9bd43219

Change-Id: I45ed30bbd0fcfd4f67cb2b682ecb3a18029be2b7
2025-10-09 20:33:46 +02:00
Michael Bayer 5b8aed514d Merge "correct for FK reflecting on PK / index col at the same time" into main 2025-10-08 16:22:07 +00:00
Michael Bayer a38cd7925d Merge "Add Executable traverse internals to Executable subclasses and turn tests on" into main 2025-10-08 15:11:24 +00:00
Mike Bayer 652f610a35 correct for FK reflecting on PK / index col at the same time
Fixed issue in the MSSQL dialect's foreign key reflection query where
duplicate rows could be returned when a foreign key column and its
referenced primary key column have the same name, and both the referencing
and referenced tables have indexes with the same name. This resulted in an
"ForeignKeyConstraint with duplicate source column references are not
supported" error when attempting to reflect such tables. The query has been
corrected to exclude indexes on the child table when looking for unique
indexes referenced by foreign keys.

Fixes: #12907
Change-Id: I435d4cf3bfa9e861cbb5e1e5c8c81bd59c9a9d58
2025-10-08 11:11:01 -04:00
Federico Caselli 115e941a56 Merge "Add type annotations to indexable extension code" into main 2025-10-08 12:57:47 +00:00
Mike Bayer 148059ced6 Add Executable traverse internals to Executable subclasses and turn tests on
Fixed a caching issue where :func:`_orm.with_loader_criteria` would
incorrectly reuse cached bound parameter values when used with
:class:`_sql.CompoundSelect` constructs such as :func:`_sql.union`. The
issue was caused by the cache key for compound selects not including the
execution options that are part of the :class:`_sql.Executable` base class,
which :func:`_orm.with_loader_criteria` uses to apply its criteria
dynamically. The fix ensures that compound selects and other executable
constructs properly include execution options in their cache key traversal.

Fixes: #12905
Change-Id: I24bbd96233cddabe42eb716f078eea4c84b1af05
2025-10-08 08:47:24 -04:00
Mike Bayer 3815469635 trim down GH actions jobs
- remove 3.13t, it's not really viable compared to 3.14t
- fully block test_memusage

Change-Id: I78ffcde329a78a9d720569a4b46a20c071e94a5c
2025-10-08 08:05:21 -04:00
Federico Caselli 313eb9dd91 update oracle profiles
Change-Id: Ie1d5a8a6e655d7fc883f07348ee56a39fdc598da
2025-10-07 21:49:51 +02:00
Mike Bayer 1ad77a925c add teardown to "run_n_times"
Change-Id: Ibeeb90f7ea3bbd32ffd27b4eb2e2988018e87a5c
2025-10-07 14:05:49 -04:00
Michael Bayer a8a6972420 Merge "Add explicit multi-threaded tests and support free-threaded build" into main 2025-10-07 13:17:08 +00:00
Denis Laxalde b2aa0e2575 Add type annotations to indexable extension code
A typing test case (plain_files/ext/indexable.py) is also added.

In order to make the methods of index_property conform with type
definitions of `fget`, `fset` and `fdel` arguments of hybrid_property,
we need to make the signature of protocols
(e.g. `_HybridGetterType`) `__call__`) method positional only.

Related to #6810.

Closes: #12763
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12763
Pull-request-sha: 896b964262

Change-Id: I42980ccf20ff9c992ebd616f38545e06ede57d36
2025-10-04 21:10:38 +02:00
Lysandros Nikolaou 456727df50 Add explicit multi-threaded tests and support free-threaded build
Implemented initial support for free-threaded Python by adding new tests
and reworking the test harness and GitHub Actions to include Python 3.13t
and Python 3.14t in test runs. Two concurrency issues have been identified
and fixed: the first involves initialization of the ``.c`` collection on a
``FromClause``, a continuation of 🎫`12302`, where an optional mutex
under free-threading is added; the second involves synchronization of the
pool "first_connect" event, which first received thread synchronization in
🎫`2964`, however under free-threading the creation of the mutex
itself runs under the same free-threading mutex. Initial pull request and
test suite courtesy Lysandros Nikolaou.

py313t: yes
py314t: yes
Fixes: #12881
Closes: #12882
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12882
Pull-request-sha: 53d65d96b9

Co-authored-by: Mike Bayer <mike_mp@zzzcomputng.com>
Change-Id: I2e4f2e9ac974ab6382cb0520cc446b396d9680a6
2025-10-02 12:14:28 -04:00
Federico Caselli e67636fa53 Merge "deprecate inherit_schema argument" into main 2025-10-01 19:21:09 +00:00
Michael Bayer 1d80e68d74 Merge "fix: pass dialect-specific kwargs to MetaData.reflect" into main 2025-09-30 19:46:42 +00:00
Lukáš Kožušník 8a77dc640b fix: pass dialect-specific kwargs to MetaData.reflect
Fixed issue where :meth:`_schema.MetaData.reflect` did not forward
dialect-specific keyword arguments to the :class:`_engine.Inspector`
methods, causing options like ``oracle_resolve_synonyms`` to be ignored
during reflection. The method now ensures that all extra kwargs passed to
:meth:`_schema.MetaData.reflect` are forwarded to
:meth:`_engine.Inspector.get_table_names` and related reflection methods.
Pull request courtesy Lukáš Kožušník.

Fixes: #12884
Closes: #12885
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12885
Pull-request-sha: 9436812223

Change-Id: Idb1ca0bc624df8e175cbfa979a8f31e62de185fe
2025-09-30 13:27:20 -04:00
Mike Bayer 3f6dd98ae4 skip windows/macosx for GC sensitive tests
can't get these to pass on GH actions anymore,
something has changed.

Change-Id: Iecd78b7f959def69b0a016f8341fef035a0d6d39
2025-09-29 22:17:30 -04:00
Federico Caselli e688617c92 deprecate inherit_schema argument
Named types such as :class:`_postgresql.ENUM` and
:class:`_postgresql.DOMAIN` (as well as the dialect-agnostic
:class:`_types.Enum` version) are now more strongly associated with the
:class:`_schema.MetaData` at the top of the table hierarchy and are
de-associated with any particular :class:`_schema.Table` they may be a part
of. This better represents how PostgreSQL named types exist independently
of any particular table, and that they may be used across many tables
simultaneously.  The change impacts the behavior of the "default schema"
for a named type, as well as the CREATE/DROP behavior in relationship to
the :class:`.MetaData` and :class:`.Table` construct.  The change also
includes a new :class:`.CheckFirst` enumeration which allows fine grained
control over "check" queries during DDL operations, as well as that the
:paramref:`_types.SchemaType.inherit_schema` parameter is deprecated and
will emit a deprecation warning when used.  See the migration notes for
full details.

Fixes: #12690
Change-Id: I9752a9d52774ae760ec3448f62fc5046c58d68f5
2025-09-24 16:33:58 -04:00
Michael Bayer de92e8d3e9 Merge "use standard path for asyncio create w/ exception handler" into main 2025-09-23 19:55:54 +00:00
Michael Bayer a11e98a67e Merge "add create_type to Enum" into main 2025-09-23 14:51:45 +00:00
Mike Bayer 3d7022a9c8 use standard path for asyncio create w/ exception handler
Refactored all asyncio dialects so that exceptions which occur on failed
connection attempts are appropriately wrapped with SQLAlchemy exception
objects, allowing for consistent error handling.

Fixes: #11956
Change-Id: Ic3fdbf334f059f92b03896b6429efa50968ca8a8
2025-09-23 10:45:22 -04:00
Mike Bayer 89a4174a8d Improve asyncpg exception hierarchy and asyncio hierarchies overall
The "emulated" exception hierarchies for the asyncio
drivers such as asyncpg, aiomysql, aioodbc, etc. have been standardized
on a common base :class:`.EmulatedDBAPIException`, which is now what's
available from the :attr:`.StatementException.orig` attribute on a
SQLAlchemy :class:`.DBAPIException` object.   Within :class:`.EmulatedDBAPIException`
and the subclasses in its hiearchy, the original driver-level exception is
also now avaliable via the :attr:`.EmulatedDBAPIException.orig` attribute,
and is also available from :class:`.DBAPIException` directly using the
:attr:`.DBAPIException.driver_exception` attribute.

Added additional emulated error classes for the subclasses of
``asyncpg.exception.IntegrityError`` including ``RestrictViolationError``,
``NotNullViolationError``, ``ForeignKeyViolationError``,
``UniqueViolationError`` ``CheckViolationError``,
``ExclusionViolationError``.  These exceptions are not directly thrown by
SQLAlchemy's asyncio emulation, however are available from the
newly added :attr:`.DBAPIException.driver_exception` attribute when a
:class:`.IntegrityError` is caught.

Fixes: #8047
Change-Id: I6a34e85b055265c087b0615f7c573be8582b3486
2025-09-23 09:49:19 -04:00
Mike Bayer fbb623d9ce add create_type to Enum
Added new parameter :paramref:`.Enum.create_type` to the Core
:class:`.Enum` class. This parameter is automatically passed to the
corresponding :class:`_postgresql.ENUM` native type during DDL operations,
allowing control over whether the PostgreSQL ENUM type is implicitly
created or dropped within DDL operations that are otherwise targeting
tables only. This provides control over the
:paramref:`_postgresql.ENUM.create_type` behavior without requiring
explicit creation of a :class:`_postgresql.ENUM` object.

Fixes: #10604
Change-Id: I450003ec2a2a65c119fe7ca8ff201392ce6b91e1
2025-09-21 12:15:49 -04:00
Michael Bayer f4f5b1931a Merge "Apply nested for joined eager m2o w/ GROUP BY, DISTINCT" into main 2025-09-21 16:14:46 +00:00
Mike Bayer ba8efc5944 Apply nested for joined eager m2o w/ GROUP BY, DISTINCT
Fixed issue where joined eager loading would fail to use the "nested" form
of the query when GROUP BY or DISTINCT were present if the eager joins
being added were many-to-ones, leading to additional columns in the columns
clause which would then cause errors.  The check for "nested" is tuned to
be enabled for these queries even for many-to-one joined eager loaders, and
the "only do nested if it's one to many" aspect is now localized to when
the query only has LIMIT or OFFSET added.

Fixes: #11226
Change-Id: I0b4b71cacfe1c6a25d0924e0954ceced1e399ca1
2025-09-21 11:01:46 -04:00
Mike Bayer fb3243475e remove _py3k suffixes from test files
on the theme of 2.0/2.1 are relatively similar right now, do
some more cleanup removing the py3k suffixes that we dont need anymore.

I'm not sure that the logic that would search for these suffixes is
even present anymore and I would guess we removed it when we removed
py2k support.  However, I am planning on possibly bringing it back
for some of the py314 stuff, so I still want to be able to do
suffix stuff.  that will be for another patch.

Change-Id: I929e6edd922f8d5f943acce77191fb1e3035b42c
2025-09-20 18:31:25 -04:00
Michael Bayer 1239df3597 Merge "deprecate ARRAY.any(), ARRAY.all(), postgresql.Any(), postgresql.All()" into main 2025-09-20 21:29:29 +00:00