Fixed issue where DDL compilation options were registered to the hard-coded
dialect name ``mysql``. This made it awkward for MySQL-derived dialects
like MariaDB, StarRocks, etc. to work with such options when different sets
of options exist for different platforms. Options are now registered under
the actual dialect name, and a fallback was added to help avoid errors when
an option does not exist for that dialect. Pull request courtesy Tiansu Yu.
Fixes: #13134Closes: #13138
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13138
Pull-request-sha: 1bc953a2a1
Change-Id: Ifa700a4e34da4d1923e9473dd8f0d2417dcfded4
(cherry picked from commit 8c26205124)
Fixed issue in the :ref:`engine_insertmanyvalues` feature where using
PostgreSQL's ``ON CONFLICT`` clause with
:paramref:`_dml.Insert.returning.sort_by_parameter_order` enabled would
generate invalid SQL when the insert used an implicit sentinel (server-side
autoincrement primary key). The generated SQL would incorrectly declare a
sentinel counter column in the ``imp_sen`` table alias without providing
corresponding values in the ``VALUES`` clause, leading to a
``ProgrammingError`` indicating column count mismatch. The fix allows batch
execution mode when ``embed_values_counter`` is active, as the embedded
counter provides the ordering capability needed even with upsert behaviors,
rather than unnecessarily downgrading to row-at-a-time execution.
Fixes: #13107
Change-Id: I382472b2cf2991b520344adea5783584e27425d0
(cherry picked from commit 574facaaf4)
Fixed issue where PostgreSQL dialect options such as ``postgresql_include``
on :class:`.PrimaryKeyConstraint` and :class:`.UniqueConstraint` were
rendered in the wrong position when combined with constraint deferrability
options like ``deferrable=True``. Pull request courtesy G Allajmi.
Fixes: #12867Closes: #13003
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13003
Pull-request-sha: 1a9216062f
Change-Id: I8c55d8faae25d56ff63c9126d569c01d8ee6c7dd
(cherry picked from commit 9fe3c3cd30)
this DB is extremely erratic in being able to connect. Add
a brute force connection retrier to all engines everywhere
(which for oracledb we can fortunately use their built-in feature
that also works).
This actually works and I can see it pausing under load, reconnecting,
and succeeding. the problem is that absolutely every engine everywhere
needs this routine otherwise an engine without a retrier in it will
crash. That then necessitates digging into testing_engine(),
making sure testing_engine() is used everywhere an engine that's going
to connect is used, then dealing with the fallout from that.
We also simplify some older workarounds for cx_oracle and
hack into config/provision to make oracledb seem like the primary
DBAPI for most tests.
testing_engine has been completely overhauled, making use of a new
post_configure_testing_engine() hook which moves and refines
the SQLite pool sharing and savepoint logic all into sqlite/provision.py
and also allows for cx_oracle to apply a retry event handler.
Change-Id: I4ea4c523effb878290d28b94d8925eb32fc5ae3b
(cherry picked from commit af768d0c3c33fc2dfc6dfc7ce847dd88bd31bc06)
one particular vector test wont run on oracle 23c free, so
just disable it.
added better skips for the rest of the vector tests and
fixed a deprecation issue.
this will be the first run on the new oracle23 on CI so we'll have to
see how this goes.
Also adjust for mariabdb12 being overly helpful with regards
to stale row updates.
as we are having trouble getting 23c to pass throug transaction
tests, i noted we have an explosion of tests due to the multiple
drivers, so this patch introduces __sparse_driver_backend__
for all tests where we want variety of
database server but there's no need to test every driver.
This should dramatically reduce the size of the test suite run
Change-Id: Ic8d3eb0a60e76b4c54c6bb4a721f90c81ede782b
(cherry picked from commit 8ce47663c238b230400d3603fa403eb5fed227dc)
Fixed issue where using the :meth:`.ColumnOperators.in_` operator with a
nested :class:`.CompoundSelect` statement (e.g. an ``INTERSECT`` of
``UNION`` queries) would raise a :class:`NotImplementedError` when the
nested compound select was the first argument to the outer compound select.
The ``_scalar_type()`` internal method now properly handles nested compound
selects.
Fixes: #12987
Change-Id: I6aa1b38863588d371bbac74b3531b99ccd5fcaec
(cherry picked from commit 42710c9220)
The ORM side of this issue can't be fixed without rewriting
.params() which will be 2.1 only, so clarify changelog
References: #12915
Change-Id: I2e524d5390241aa289786f524b6a51f39bc09876
(cherry picked from commit b8906e62ce)
Fixed issue where using :meth:`_sql.Select.params` to replace bound
parameters in a query could fail for some cases where the parameters
were embedded in subqueries or CTEs when ORM classes were involved,
due to issues with internal query traversal for these cases.
Fixes: #12915
Change-Id: Ib63bca786a541682f6b2144fd5dd43350411ae9d
(cherry picked from commit 03116b8cc9)
Fixed a caching issue where :func:`_orm.with_loader_criteria` would
incorrectly reuse cached bound parameter values when used with
:class:`_sql.CompoundSelect` constructs such as :func:`_sql.union`. The
issue was caused by the cache key for compound selects not including the
execution options that are part of the :class:`_sql.Executable` base class,
which :func:`_orm.with_loader_criteria` uses to apply its criteria
dynamically. The fix ensures that compound selects and other executable
constructs properly include execution options in their cache key traversal.
Fixes: #12905
Change-Id: I24bbd96233cddabe42eb716f078eea4c84b1af05
(cherry picked from commit 148059ced6)
Implemented initial support for free-threaded Python by adding new tests
and reworking the test harness and GitHub Actions to include Python 3.13t
and Python 3.14t in test runs. Two concurrency issues have been identified
and fixed: the first involves initialization of the ``.c`` collection on a
``FromClause``, a continuation of 🎫`12302`, where an optional mutex
under free-threading is added; the second involves synchronization of the
pool "first_connect" event, which first received thread synchronization in
🎫`2964`, however under free-threading the creation of the mutex
itself runs under the same free-threading mutex. Initial pull request and
test suite courtesy Lysandros Nikolaou.
py313t: yes
py314t: yes
Fixes: #12881Closes: #12882
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12882
Pull-request-sha: 53d65d96b9
Co-authored-by: Mike Bayer <mike_mp@zzzcomputng.com>
Change-Id: I2e4f2e9ac974ab6382cb0520cc446b396d9680a6
(cherry picked from commit 456727df50)
Fixed issue where the :func:`_sql.any_` and :func:`_sql.all_` aggregation
operators would not correctly coerce the datatype of the compared value, in
those cases where the compared value were not a simple int/str etc., such
as a Python ``Enum`` or other custom value. This would lead to execution
time errors for these values. This issue is essentially the same as
🎫`6515` which was for the now-legacy :meth:`.ARRAY.any` and
:meth:`.ARRAY.all` methods.
Fixes: #12874
Change-Id: I980894c23b9974bc84d584a1a4c5fae72dded6d3
(cherry picked from commit aaa85f707e)
Improved the implementation of :meth:`.UpdateBase.returning` to use more
robust logic in setting up the ``.c`` collection of a derived statement
such as a CTE. This fixes issues related to RETURNING clauses that feature
expressions based on returned columns with or without qualifying labels.
Co-authored-by: Juhyeong Ko <dury.ko@gmail.com>
Fixes: #12271
Change-Id: Id0d486d4304002f1affdec2e7662ac2965936f2a
(cherry picked from commit 4c4011b50b)
The :func:`_sql.values` construct gains a new method :meth:`_sql.Values.cte`,
which allows creation of a named, explicit-columns :class:`.CTE` against an
unnamed ``VALUES`` expression, producing a syntax that allows column-oriented
selection from a ``VALUES`` construct on modern versions of PostgreSQL, SQLite,
and MariaDB.
Fixes: #12734
Change-Id: I4a0146418420cce3cbbda4e50f5eb32206dc696b
(cherry picked from commit 209d1913cc)
Hardening of the compiler's actions for UPDATE statements that access
multiple tables to report more specifically when tables or aliases are
referenced in the SET clause; on cases where the backend does not support
secondary tables in the SET clause, an explicit error is raised, and on the
MySQL or similar backends that support such a SET clause, more specific
checking for not-properly-included tables is performed. Overall the change
is preventing these erroneous forms of UPDATE statements from being
compiled, whereas previously it was relied on the database to raise an
error, which was not always guaranteed to happen, or to be non-ambiguous,
due to cases where the parent table included the same column name as the
secondary table column being updated.
Fixed bug where the ORM would pull in the wrong column into an UPDATE when
a key name inside of the :meth:`.ValuesBase.values` method could be located
from an ORM entity mentioned in the statement, but where that ORM entity
was not the actual table that the statement was inserting or updating. An
extra check for this edge case is added to avoid this problem.
Fixes: #12692
Change-Id: I342832b09dda7ed494caaad0cbb81b93fc10fe18
(cherry picked from commit dc0d081762)
Fixed issue where :func:`.select` of a free-standing, unnamed scalar expression that
has a unary operator applied, such as negation, would not apply result
processors to the selected column even though the correct type remains in
place for the unary expression.
This change opened up a typing rabbithole where we were led to also
improve and harden the typing for the Exists element, in particular
in that the Exists now always refers to a ScalarSelect object, and
no longer a SelectStatementGrouping within the _regroup() cases; there
did not seem to be any reason for this inconsistency.
Fixes: #12681
Change-Id: If9131807941030c627ab31ede4ccbd86e44e707f
(cherry picked from commit c96805a43a)
Since I want to get rid of util.portable_instancemethod, first
make sure we are testing pickle extensively including going through
all protocols for all metadata-oriented tests.
Change-Id: I0064bc16033939780e50c7a8a4ede60ef5835b38
(cherry picked from commit 239f629b9a)
Type the `create` and `drop` functions for `NamedType`s
Also partially type the SchemaType create/drop functions more generally
One change to this is that the default parameter of `None` is removed. It doesn't work and will fail with a `AttributeError` at runtime since it immediately tries to access a property of `None` which doesn't exist.
Fixes#12557
This pull request is:
- [X] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [X] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Closes: #12558
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12558
Pull-request-sha: 75c8d81bfb
Change-Id: I173771d365f34f54ab474b9661e1cdc70cc4de84
(cherry picked from commit b4d7bf7a2f)
Added new datatype :class:`_oracle.VECTOR` and accompanying DDL and DQL
support to fully support this type for Oracle Database. This change
includes the base :class:`_oracle.VECTOR` type that adds new type-specific
methods ``l2_distance``, ``cosine_distance``, ``inner_product`` as well as
new parameters ``oracle_vector`` for the :class:`.Index` construct,
allowing vector indexes to be configured, and ``oracle_fetch_approximate``
for the :meth:`.Select.fetch` clause. Pull request courtesy Suraj Shaw.
Fixes: #12317Closes: #12321
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12321
Pull-request-sha: a72a18a45c
Change-Id: I6f3af4623ce439d0820c14582cd129df293f0ba8
(cherry picked from commit 1b780ce3d3)
Implemented support for the GROUPS frame specification in window functions
by adding :paramref:`_sql.over.groups` option to :func:`_sql.over`
and :meth:`.FunctionElement.over`. Pull request courtesy Kaan Dikmen.
Fixes: #12450Closes: #12445
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12445
Pull-request-sha: c0808e135f
Change-Id: I9ff504a9c9650485830c4a0eaf44162898a3a2ad
(cherry picked from commit 0bbc515f904446d3f0beede54321b628f32fbdad)
When building a PostgreSQL ``ARRAY`` literal using
:class:`_postgresql.array` with an empty ``clauses`` argument, the
:paramref:`_postgresql.array.type_` parameter is now significant in that it
will be used to render the resulting ``ARRAY[]`` SQL expression with a
cast, such as ``ARRAY[]::INTEGER``. Pull request courtesy Denis Laxalde.
Fixes: #12432Closes: #12435
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12435
Pull-request-sha: 9633d3c15d
Change-Id: I29ed7bd0562b82351d22de0658fb46c31cfe44f6
(cherry picked from commit 588cc6ed8e)
Fixed regression caused by 🎫`7471` leading to a SQL compilation
issue where name disambiguation for two same-named FROM clauses with table
aliasing in use at the same time would produce invalid SQL in the FROM
clause with two "AS" clauses for the aliased table, due to double aliasing.
Fixes: #12451
Change-Id: I981823f8f2cdf3992d65ace93a21fc20d1d74cda
(cherry picked from commit 9ea3be0681)
Added support for specifying a list of columns for ``SET NULL`` and ``SET
DEFAULT`` actions of ``ON DELETE`` clause of foreign key definition on
PostgreSQL. Pull request courtesy Denis Laxalde.
Fixes: #11595Closes: #12421
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12421
Pull-request-sha: d0394db706
Change-Id: I036a559ae4a8efafe9ba64d776a840bd785a7397
(cherry picked from commit 39bb17442c)
Fixed issue in :class:`.CTE` constructs involving multiple DDL
:class:`.Insert` statements with multiple VALUES parameter sets where the
bound parameter names generated for these parameter sets would conflict,
generating a compile time error.
Fixes: #12363
Change-Id: If8344ff725d4e0ec58d3ff61f38a0edcfc5bdebd
(cherry picked from commit ec20f346a6)
improve test error logging
remove obsolete emulated pipeline
fix test in python 3.8
add order to test
Change-Id: I2003f256a2690ee5673c72e2f1cb1340af750f83
Added new parameters :paramref:`.AddConstraint.isolate_from_table` and
:paramref:`.DropConstraint.isolate_from_table`, defaulting to True, which
both document and allow to be controllable the long-standing behavior of
these two constructs blocking the given constraint from being included
inline within the "CREATE TABLE" sequence, under the assumption that
separate add/drop directives were to be used.
Fixes: #12382
Change-Id: I53c4170ccb5803f69945ba7aa3d3a143131508eb
(cherry picked from commit d6f11d9030)
Fixed issue where using :func:`_orm.aliased` around a :class:`.CTE`
construct could cause inappropriate "duplicate CTE" errors in cases where
that aliased construct appeared multiple times in a single statement.
Fixes: #12364
Change-Id: I9625cd83e9baf5312cdc644b38951353708d3b86
(cherry picked from commit 42ddb1fd5f)
Fixed SQL composition bug which impacted caching where using a ``None``
value inside of an ``in_()`` expression would bypass the usual "expanded
bind parameter" logic used by the IN construct, which allows proper caching
to take place.
Fixes: #12314
References: #12312
Change-Id: I0d2fc4e15c73407379ba368dd4ee32660fc66259
(cherry picked from commit 79505b03b6)
Reorganized the internals by which the `.c` collection on a
:class:`.FromClause` gets generated so that it is resilient against the
collection being accessed in concurrent fashion. An example is creating a
:class:`.Alias` or :class:`.Subquery` and accessing it as a module level
variable. This impacts the Oracle dialect which uses such module-level
global alias objects but is of general use as well.
Fixes: #12302
Change-Id: I30cb07c286affce24e2d85e49f9df5b787438d86
(cherry picked from commit 3cd9a5b42f)
Fixed issue in "lambda SQL" feature where the tracking of bound parameters
could be corrupted if the same lambda were evaluated across multiple
compile phases, including when using the same lambda across multiple engine
instances or with statement caching disabled.
Fixes: #12084
Change-Id: I327aa93ce7feb2326a22113164bd834b96b6b889
(cherry picked from commit 5bbefc41b7)
Fixed regression caused by an internal code change in response to recent
Mypy releases that caused the very unusual case of a list of ORM-mapped
attribute expressions passed to :meth:`.ColumnOperators.in_` to no longer
be accepted.
in this commit we had to revisit d8dd28c42e where mypy typing
didn't accept ColumnOperartors. the type here is the _HasClauseElement[_T]
protocol which means we need to use a duck type for a runtime check.
Fixes: #12019
Change-Id: Ib378e9cb8defb49d5ac4d726ec93d6bdc581b6a9
(cherry picked from commit aaddd7c840)
Fixed regression from 1.4 where some datatypes such as those derived from
:class:`.TypeDecorator` could not be pickled when they were part of a
larger SQL expression composition due to internal supporting structures
themselves not being pickleable.
Fixes: #12002
Change-Id: I016e37b0c62071413f24c9aac35f6ecf475becaa
(cherry picked from commit fa56821578)
Datatypes that are binary based such as :class:`.VARBINARY` will resolve to
:class:`.LargeBinary` when the :meth:`.TypeEngine.as_generic()` method is
called.
Fixes: #11978
Change-Id: I2e0586324fb0f1c367da61f0074b35c96fbe2fd0
(cherry picked from commit 858eba6156)
Fixed bug where the ``metadata`` element of an ``Enum`` datatype would not
be transferred to the new :class:`.MetaData` object when the type had been
copied via a :meth:`.Table.to_metadata` operation, leading to inconsistent
behaviors within create/drop sequences.
Fixes: #11802
Change-Id: Ibbc93aa31bdfde0d67a9530f41a08e826c17d58e
(cherry picked from commit 22cbc7dcb4)
Fixed regression in :meth:`_sql.Select.with_statement_hint` and others
where the generative behavior of the method stopped producing a copy of the
object.
Fixes: #11703
Change-Id: Ia4482f91f76fae9982dc6b075bf5cfec7042ffa6
(cherry picked from commit a9747467a8)
Follow up of 🎫`11471` to fix caching issue where using the
:meth:`.CompoundSelectState.add_cte` method of the
:class:`.CompoundSelectState` construct would not set a correct cache key
which distinguished between different CTE expressions. Also added tests
that would detect issues similar to the one fixed in 🎫`11544`.
Fixes: #11471
Change-Id: Iae6a91077c987d83cd70ea826daff42855491330
(cherry picked from commit 881be0a216)
Fixed bug where the :meth:`.Operators.nulls_first()` and
:meth:`.Operators.nulls_last()` modifiers would not be treated the same way
as :meth:`.Operators.desc()` and :meth:`.Operators.asc()` when determining
if an ORDER BY should be against a label name already in the statement. All
four modifiers are now treated the same within ORDER BY.
Fixes: #11592
Change-Id: I1de1aff679c56af1abfdfd07f9bcbc45ecc5a8cc
(cherry picked from commit 96f1172812)
Fixed issue where the :func:`_sql.collate` construct, which explicitly sets
a collation for a given expression, would maintain collation settings for
the underlying type object from the expression, causing SQL expressions to
have both collations stated at once when used in further expressions for
specific dialects that render explicit type casts, such as that of asyncpg.
The :func:`_sql.collate` construct now assigns its own type to explicitly
include the new collation, assuming it's a string type.
Fixes: #11576
Change-Id: I6fc8904d2bcbc21f11bbca57e4a451ed0edbd879
(cherry picked from commit 35c178c405)
The :paramref:`_orm.aliased.name` parameter to :func:`_orm.aliased` may now
be combined with the :paramref:`_orm.aliased.flat` parameter, producing
per-table names based on a name-prefixed naming convention. Pull request
courtesy Eric Atkin.
Fixes: #11575Closes: #11531
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11531
Pull-request-sha: f85535464b
Change-Id: If79679c7a9598fffe99c033894b7dffecef13939
(cherry picked from commit 0e40962bf3)
Fixed issue in "insertmanyvalues" feature where a particular call to
``cursor.fetchall()`` were not wrapped in SQLAlchemy's exception wrapper,
which apparently can raise a database exception during fetch when using
pyodbc.
Fixes: #11532
Change-Id: Ic07d3e79dd597e18d87a56b45ddffa25e762beb9
(cherry picked from commit fb47dbbc74)
Fixed caching issue where the
:paramref:`_sql.Select.with_for_update.key_share` element of
:meth:`_sql.Select.with_for_update` was not considered as part of the cache
key, leading to incorrect caching if different variations of this parameter
were used with an otherwise identical statement.
Also repairs a traversal issue where the ``of`` element of
``ForUpdateArg`` when set to ``None`` cannot be compared against a
non-None element because the traversal defines it as a clauselist.
Traversal in this case is adjusted to accommodate for this case so that
we dont need to create a risky-to-backport change to ``ForUpdateArg``
itself.
Fixes: #11544
Change-Id: Ie8a50716df06977af58b0c22a8c10e1b64d972b9
(cherry picked from commit 6d2f43e14f)
Fixed additional issues in the event system triggered by unpickling of a
:class:`.Enum` datatype, continuing from 🎫`11365` and
🎫`11360`, where dynamically generated elements of the event
structure would not be present when unpickling in a new process.
Fixes: #11530
Change-Id: Ie1f2b3453d4891051f8719f6d3f6703302d5a86e
(cherry picked from commit dffd96e754)
Fixed caching issue where using the :meth:`.TextualSelect.add_cte` method
of the :class:`.TextualSelect` construct would not set a correct cache key
which distinguished between different CTE expressions.
Fixes: #11471
Change-Id: Ia9ce2c8cfd128f0f130aa9b26448dc23d994c324
(cherry picked from commit faecebc9df)
Fixed bug in :meth:`_sql.FunctionFilter.filter` that would mutate
the existing function in-place. It now behaves like the rest of the
SQLAlchemy API, returning a new instance instead of mutating the
original one.
Fixes: #11426
Change-Id: I46ffebaed82426cfb1623db066686cfb911055a1
(cherry picked from commit fe2ced9e79b9640f3ca135f8d3782dd41ca16782)
Tiansu Yu