Fixed issue where PostgreSQL dialect options such as ``postgresql_include``
on :class:`.PrimaryKeyConstraint` and :class:`.UniqueConstraint` were
rendered in the wrong position when combined with constraint deferrability
options like ``deferrable=True``. Pull request courtesy G Allajmi.
Fixes: #12867Closes: #13003
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13003
Pull-request-sha: 1a9216062f
Change-Id: I8c55d8faae25d56ff63c9126d569c01d8ee6c7dd
The :meth:`_sql.Select.filter_by`, :meth:`_sql.Update.filter_by` and
:meth:`_sql.Delete.filter_by` methods now search across all entities
present in the statement, rather than limiting their search to only the
last joined entity or the first FROM entity. This allows these methods
to locate attributes unambiguously across multiple joined tables,
resolving issues where changing the order of operations such as
:meth:`_sql.Select.with_only_columns` would cause the method to fail.
If an attribute name exists in more than one FROM clause entity, an
:class:`_exc.AmbiguousColumnError` is now raised, indicating that
:meth:`_sql.Select.filter` (or :meth:`_sql.Select.where`) should be used
instead with explicit table-qualified column references.
Fixes: #8601
Change-Id: I6a46b8f4784801f95f7980ca8ef92f1947653572
Added support for Python 3.14+ template strings (t-strings) via the new
:func:`_sql.tstring` construct, as defined in :pep:`750`. This feature
allows for ergonomic SQL statement construction by automatically
interpolating Python values and SQLAlchemy expressions within template
strings.
Part of the challenge here is the syntax only works on py314, so we have
to exclude the test file at many levels when py314 is not used. not
sure yet how i want to adjust pep8 tests and rules for this.
Fixes: #12548
Change-Id: Ia060d1387ff452fe4f5d924f683529a22a8e1f72
this DB is extremely erratic in being able to connect. Add
a brute force connection retrier to all engines everywhere
(which for oracledb we can fortunately use their built-in feature
that also works).
This actually works and I can see it pausing under load, reconnecting,
and succeeding. the problem is that absolutely every engine everywhere
needs this routine otherwise an engine without a retrier in it will
crash. That then necessitates digging into testing_engine(),
making sure testing_engine() is used everywhere an engine that's going
to connect is used, then dealing with the fallout from that.
We also simplify some older workarounds for cx_oracle and
hack into config/provision to make oracledb seem like the primary
DBAPI for most tests.
testing_engine has been completely overhauled, making use of a new
post_configure_testing_engine() hook which moves and refines
the SQLite pool sharing and savepoint logic all into sqlite/provision.py
and also allows for cx_oracle to apply a retry event handler.
Change-Id: I4ea4c523effb878290d28b94d8925eb32fc5ae3b
one particular vector test wont run on oracle 23c free, so
just disable it.
added better skips for the rest of the vector tests and
fixed a deprecation issue.
this will be the first run on the new oracle23 on CI so we'll have to
see how this goes.
Also adjust for mariabdb12 being overly helpful with regards
to stale row updates.
as we are having trouble getting 23c to pass throug transaction
tests, i noted we have an explosion of tests due to the multiple
drivers, so this patch introduces __sparse_driver_backend__
for all tests where we want variety of
database server but there's no need to test every driver.
This should dramatically reduce the size of the test suite run
Change-Id: Ic8d3eb0a60e76b4c54c6bb4a721f90c81ede782b
Fixed issue where using the :meth:`.ColumnOperators.in_` operator with a
nested :class:`.CompoundSelect` statement (e.g. an ``INTERSECT`` of
``UNION`` queries) would raise a :class:`NotImplementedError` when the
nested compound select was the first argument to the outer compound select.
The ``_scalar_type()`` internal method now properly handles nested compound
selects.
Fixes: #12987
Change-Id: I6aa1b38863588d371bbac74b3531b99ccd5fcaec
Added new implementation for the :meth:`.Select.params` method and that of
similar statements, via a new statement-only
:meth:`.ExecutableStatement.params` method which works more efficiently and
correctly than the previous implementations available from
:class:`.ClauseElement`, by assocating the given parameter dictionary with
the statement overall rather than cloning the statement and rewriting its
bound parameters. The :meth:`_sql.ClauseElement.params` and
:meth:`_sql.ClauseElement.unique_params` methods, when called on an object
that does not implement :class:`.ExecutableStatement`, will continue to
work the old way of cloning the object, and will emit a deprecation
warning. This issue both resolves the architectural / performance
concerns of 🎫`7066` and also provides correct ORM compatibility for
functions like :func:`_orm.aliased`, reported by 🎫`12915`.
Fixes: #7066
Change-Id: I6543c7d0f4da3232b3641fb172c24c446f02f52a
Added support for the SQL ``CREATE VIEW`` statement via the new
:class:`.CreateView` DDL class. The new class allows creating database
views from SELECT statements, with support for options such as
``TEMPORARY``, ``IF NOT EXISTS``, and ``MATERIALIZED`` where supported by
the target database. Views defined with :class:`.CreateView` integrate with
:class:`.MetaData` for automated DDL generation and provide a
:class:`.Table` object for querying.
this alters the CreateTableAs commit:
* adds the ability for Table to be associated with Create and Drop
DDL constructs
* Adds CreateView variant of CreateTableAs
* Both associate themselves with Table so they take place in
create_all/create/drop_all/drop
Fixes: #181
Change-Id: If3e568d3d6a6ce19e3d15198c3fbbe06bd847c83
Added support for PostgreSQL 14+ HSTORE subscripting syntax. When connected
to PostgreSQL 14 or later, HSTORE columns now automatically use the native
subscript notation ``hstore_col['key']`` instead of the arrow operator
``hstore_col -> 'key'`` for both read and write operations. This provides
better compatibility with PostgreSQL's native HSTORE subscripting feature
while maintaining backward compatibility with older PostgreSQL versions.
as part of this change we add a new parameter to custom_op "visit_name"
which allows a custom op to refer to a specific visit method in a
dialect's compiler.
Fixes: #12948
Change-Id: Id98d333fe78e31d9c7679cb2902f1c7e458d6e11
use pyproject.toml so the options are safe from "rm .coverage*"
types of commands
omit some files
add test coverage for a big section of cache-key
Change-Id: Ia540c632d91ec09284e187e3edeb8ccf0214e1a6
Added support for the SQL ``CREATE TABLE ... AS SELECT`` construct via the
new :class:`_sql.CreateTableAs` DDL construct and the
:meth:`_sql.SelectBase.into` method. The new construct allows creating a
table directly from the results of a SELECT statement, with support for
options such as ``TEMPORARY`` and ``IF NOT EXISTS`` where supported by the
target database. Pull request courtesy Greg Jarzab.
Fixes: #4950Closes: #12860
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12860
Pull-request-sha: 7de8a109b8
Change-Id: Id9c8e4a3c520ffc61de1e48e331b6220e3d52fc9
Updated the :func:`_sql.over` clause to allow non-integer values in
:paramref:`_sql.over.range_` clause. Previously, only integer values
were allowed and any other values would lead to a failure.
To specify a non-integer value, use the new :class:`_sql.FrameClause`
construct along with the new :class:`_sql.FrameClauseType` enum to specify
the frame boundaries.
Fixes#12596.
Closes: #12695
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12695
Pull-request-sha: 8063cec65d
Change-Id: I248a938f6502d72555c005d86791c992822117d4
this isn't used internally and only allows execute(stmt.compile())
to work, which is not something I want to support.
Since people are definitely using this [1] we might want to think about
either deprectation or having a recipe for people who really want
to still do this
[1] https://github.com/sqlalchemy/sqlalchemy/discussions/11108?converting=1
Change-Id: I5f0fe800e31aac052926cebe9206763eef9a804c
Fixed an issue in :meth:`_sql.Select.join_from` where the join condition
between the left and right tables specified in the method call could be
incorrectly determined based on an intermediate table already present in
the FROM clause, rather than matching the foreign keys between the
immediate left and right arguments. The join condition is now determined by
matching primary keys between the two tables explicitly passed to
:meth:`_sql.Select.join_from`, ensuring consistent and predictable join
behavior regardless of the order of join operations or other tables present
in the query. The fix is applied to both the Core and ORM implementations
of :meth:`_sql.Select.join_from`.
Fixes: #12931
Change-Id: Id457d441ee8a1bd64a002e57a9dcfd6fc56ff15e
The ORM side of this issue can't be fixed without rewriting
.params() which will be 2.1 only, so clarify changelog
References: #12915
Change-Id: I2e524d5390241aa289786f524b6a51f39bc09876
Fixed issue where using :meth:`_sql.Select.params` to replace bound
parameters in a query could fail for some cases where the parameters
were embedded in subqueries or CTEs when ORM classes were involved,
due to issues with internal query traversal for these cases.
Fixes: #12915
Change-Id: Ib63bca786a541682f6b2144fd5dd43350411ae9d
Fixed a caching issue where :func:`_orm.with_loader_criteria` would
incorrectly reuse cached bound parameter values when used with
:class:`_sql.CompoundSelect` constructs such as :func:`_sql.union`. The
issue was caused by the cache key for compound selects not including the
execution options that are part of the :class:`_sql.Executable` base class,
which :func:`_orm.with_loader_criteria` uses to apply its criteria
dynamically. The fix ensures that compound selects and other executable
constructs properly include execution options in their cache key traversal.
Fixes: #12905
Change-Id: I24bbd96233cddabe42eb716f078eea4c84b1af05
Implemented initial support for free-threaded Python by adding new tests
and reworking the test harness and GitHub Actions to include Python 3.13t
and Python 3.14t in test runs. Two concurrency issues have been identified
and fixed: the first involves initialization of the ``.c`` collection on a
``FromClause``, a continuation of 🎫`12302`, where an optional mutex
under free-threading is added; the second involves synchronization of the
pool "first_connect" event, which first received thread synchronization in
🎫`2964`, however under free-threading the creation of the mutex
itself runs under the same free-threading mutex. Initial pull request and
test suite courtesy Lysandros Nikolaou.
py313t: yes
py314t: yes
Fixes: #12881Closes: #12882
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12882
Pull-request-sha: 53d65d96b9
Co-authored-by: Mike Bayer <mike_mp@zzzcomputng.com>
Change-Id: I2e4f2e9ac974ab6382cb0520cc446b396d9680a6
Named types such as :class:`_postgresql.ENUM` and
:class:`_postgresql.DOMAIN` (as well as the dialect-agnostic
:class:`_types.Enum` version) are now more strongly associated with the
:class:`_schema.MetaData` at the top of the table hierarchy and are
de-associated with any particular :class:`_schema.Table` they may be a part
of. This better represents how PostgreSQL named types exist independently
of any particular table, and that they may be used across many tables
simultaneously. The change impacts the behavior of the "default schema"
for a named type, as well as the CREATE/DROP behavior in relationship to
the :class:`.MetaData` and :class:`.Table` construct. The change also
includes a new :class:`.CheckFirst` enumeration which allows fine grained
control over "check" queries during DDL operations, as well as that the
:paramref:`_types.SchemaType.inherit_schema` parameter is deprecated and
will emit a deprecation warning when used. See the migration notes for
full details.
Fixes: #12690
Change-Id: I9752a9d52774ae760ec3448f62fc5046c58d68f5
The :meth:`_types.ARRAY.Comparator.any` and
:meth:`_types.ARRAY.Comparator.all` methods for the :class:`_types.ARRAY`
type are now deprecated for removal; these two methods along with
:func:`_postgresql.Any` and :func:`_postgresql.All` have been legacy for
some time as they are superseded by the :func:`_sql.any_` and
:func:`_sql.all_` functions, which feature more intutive use.
Fixes: #10821
Change-Id: I8eb3bbcb98af4ee60a21767dc3bdac771cbc0b4c
Fixed issue where the :func:`_sql.any_` and :func:`_sql.all_` aggregation
operators would not correctly coerce the datatype of the compared value, in
those cases where the compared value were not a simple int/str etc., such
as a Python ``Enum`` or other custom value. This would lead to execution
time errors for these values. This issue is essentially the same as
🎫`6515` which was for the now-legacy :meth:`.ARRAY.any` and
:meth:`.ARRAY.all` methods.
Fixes: #12874
Change-Id: I980894c23b9974bc84d584a1a4c5fae72dded6d3
Added new generalized aggregate function ordering to functions via the
:func:`_functions.FunctionElement.aggregate_order_by` method, which
receives an expression and generates the appropriate embedded "ORDER BY" or
"WITHIN GROUP (ORDER BY)" phrase depending on backend database. This new
function supersedes the use of the PostgreSQL
:func:`_postgresql.aggregate_order_by` function, which remains present for
backward compatibility. To complement the new parameter, the
:paramref:`_functions.aggregate_strings.order_by` which adds ORDER BY
capability to the :class:`_functions.aggregate_strings` dialect-agnostic
function which works for all included backends. Thanks much to Reuven
Starodubski with help on this patch.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Fixes: #12853Closes: #12856
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12856
Pull-request-sha: d93fb591751227eb1f96052ea3ad449f511f70b3
Change-Id: I8eb41ff2d57695963a358b5f0017ca9372f15f70
Improved the implementation of :meth:`.UpdateBase.returning` to use more
robust logic in setting up the ``.c`` collection of a derived statement
such as a CTE. This fixes issues related to RETURNING clauses that feature
expressions based on returned columns with or without qualifying labels.
Co-authored-by: Juhyeong Ko <dury.ko@gmail.com>
Fixes: #12271
Change-Id: Id0d486d4304002f1affdec2e7662ac2965936f2a
Added a new concept of "operator classes" to the SQL operators supported by
SQLAlchemy, represented within the enum :class:`.OperatorClass`. The
purpose of this structure is to provide an extra layer of validation when a
particular kind of SQL operation is used with a particular datatype, to
catch early the use of an operator that does not have any relevance to the
datatype in use; a simple example is an integer or numeric column used with
a "string match" operator.
Fixes: #12736
Change-Id: I44f46d7326aef6847dbf0cf7a325833f8e347da6
Improved the behavior of standalone "operators" like :func:`_sql.desc`,
:func:`_sql.asc`, :func:`_sql.all_`, etc. so that they consult the given
expression object for an overriding method for that operator, even if the
object is not itself a ``ClauseElement``, such as if it's an ORM attribute.
This allows custom comparators for things like :func:`_orm.composite` to
provide custom implementations of methods like ``desc()``, ``asc()``, etc.
Added default implementations of :meth:`.ColumnOperators.desc`,
:meth:`.ColumnOperators.asc`, :meth:`.ColumnOperators.nulls_first`,
:meth:`.ColumnOperators.nulls_last` to :func:`_orm.composite` attributes,
by default applying the modifier to all contained columns. Can be
overridden using a custom comparator.
Fixes: #12769
Change-Id: I055ce79bf7ac31fb61d48bc3ab34799d42fb6336
The :func:`_sql.values` construct gains a new method :meth:`_sql.Values.cte`,
which allows creation of a named, explicit-columns :class:`.CTE` against an
unnamed ``VALUES`` expression, producing a syntax that allows column-oriented
selection from a ``VALUES`` construct on modern versions of PostgreSQL, SQLite,
and MariaDB.
Fixes: #12734
Change-Id: I4a0146418420cce3cbbda4e50f5eb32206dc696b
Fixed issue in "insertmanyvalues" feature where an INSERT..RETURNING
that also made use of a sentinel column to track results would fail to
filter out the additional column when :meth:`.Result.unique` were used
to uniquify the result set.
Fixes: #10802
Change-Id: Ie4f9dab96193099002088c5219cc41a543a00f62
Added new Core feature :func:`_sql.from_dml_column` that may be used in
expressions inside of :meth:`.UpdateBase.values` for INSERT or UPDATE; this
construct will copy whatever SQL expression is used for the given target
column in the statement to be used with additional columns. The construct
is mostly intended to be a helper with ORM :class:`.hybrid_property` within
DML hooks.
This is the Core side of the feature being added to the ORM
for #12496
Change-Id: Ic568638a8ce3607deea44af988b6451b30cde36c
Added method :meth:`.TableClause.insert_column` to complement
:meth:`.TableClause.append_column`, which inserts the given column at a
specific index. This can be helpful for prepending primary key columns to
tables, etc.
Fixes: #7910
Change-Id: Ife1eb8ad90aa61d38c457a86312cfe5d0d471106
Hardening of the compiler's actions for UPDATE statements that access
multiple tables to report more specifically when tables or aliases are
referenced in the SET clause; on cases where the backend does not support
secondary tables in the SET clause, an explicit error is raised, and on the
MySQL or similar backends that support such a SET clause, more specific
checking for not-properly-included tables is performed. Overall the change
is preventing these erroneous forms of UPDATE statements from being
compiled, whereas previously it was relied on the database to raise an
error, which was not always guaranteed to happen, or to be non-ambiguous,
due to cases where the parent table included the same column name as the
secondary table column being updated.
Fixed bug where the ORM would pull in the wrong column into an UPDATE when
a key name inside of the :meth:`.ValuesBase.values` method could be located
from an ORM entity mentioned in the statement, but where that ORM entity
was not the actual table that the statement was inserting or updating. An
extra check for this edge case is added to avoid this problem.
Fixes: #12692
Change-Id: I342832b09dda7ed494caaad0cbb81b93fc10fe18
Fixed issue where :func:`.select` of a free-standing, unnamed scalar expression that
has a unary operator applied, such as negation, would not apply result
processors to the selected column even though the correct type remains in
place for the unary expression.
This change opened up a typing rabbithole where we were led to also
improve and harden the typing for the Exists element, in particular
in that the Exists now always refers to a ScalarSelect object, and
no longer a SelectStatementGrouping within the _regroup() cases; there
did not seem to be any reason for this inconsistency.
Fixes: #12681
Change-Id: If9131807941030c627ab31ede4ccbd86e44e707f
Since I want to get rid of util.portable_instancemethod, first
make sure we are testing pickle extensively including going through
all protocols for all metadata-oriented tests.
Change-Id: I0064bc16033939780e50c7a8a4ede60ef5835b38
Changed the default value of :paramref:`_types.Enum.inherit_schema` to
``True`` when :paramref:`_types.Enum.schema` and
:paramref:`_types.Enum.metadata` parameters are not provided.
The same behavior has been applied also to PostgreSQL
:class:`_postgresql.DOMAIN` type.
Fixes: #10594
Change-Id: Id3d819e3608974353e365cd063d9c5e40a071e73
Allow custom operator systems to use the @ Python operator (#12479).
### Description
Add a dummy implementation for the `__matmul__` operator rasing `NotImplementedError` by default.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [X] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Closes: #12583
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12583
Pull-request-sha: 7e69d23610
Change-Id: Ia0d565decd437b940efd3b97478c16d7a0377bc6
Type the `create` and `drop` functions for `NamedType`s
Also partially type the SchemaType create/drop functions more generally
One change to this is that the default parameter of `None` is removed. It doesn't work and will fail with a `AttributeError` at runtime since it immediately tries to access a property of `None` which doesn't exist.
Fixes#12557
This pull request is:
- [X] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [X] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Closes: #12558
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12558
Pull-request-sha: 75c8d81bfb
Change-Id: I173771d365f34f54ab474b9661e1cdc70cc4de84
Added new datatype :class:`_oracle.VECTOR` and accompanying DDL and DQL
support to fully support this type for Oracle Database. This change
includes the base :class:`_oracle.VECTOR` type that adds new type-specific
methods ``l2_distance``, ``cosine_distance``, ``inner_product`` as well as
new parameters ``oracle_vector`` for the :class:`.Index` construct,
allowing vector indexes to be configured, and ``oracle_fetch_approximate``
for the :meth:`.Select.fetch` clause. Pull request courtesy Suraj Shaw.
Fixes: #12317Closes: #12321
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12321
Pull-request-sha: a72a18a45c
Change-Id: I6f3af4623ce439d0820c14582cd129df293f0ba8
G Allajmi