Fixed issue where PostgreSQL dialect options such as ``postgresql_include``
on :class:`.PrimaryKeyConstraint` and :class:`.UniqueConstraint` were
rendered in the wrong position when combined with constraint deferrability
options like ``deferrable=True``. Pull request courtesy G Allajmi.
Fixes: #12867Closes: #13003
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13003
Pull-request-sha: 1a9216062f
Change-Id: I8c55d8faae25d56ff63c9126d569c01d8ee6c7dd
Fixed issue where anonymous label generation for :class:`.CTE` constructs
could produce name collisions when Python's garbage collector reused memory
addresses during complex query compilation. The anonymous name generation
for :class:`.CTE` and other aliased constructs like :class:`.Alias`,
:class:`.Subquery` and others now use :func:`os.urandom` to generate unique
identifiers instead of relying on object ``id()``, ensuring uniqueness even
in cases of aggressive garbage collection and memory reuse.
Fixes: #12990
Change-Id: If56a53840684bc7d2b7637f1e154dfed1cac5f32
Added support for monotonic server-side functions such as PostgreSQL 18's
``uuidv7()`` to work with the :ref:`engine_insertmanyvalues` feature.
By passing ``monotonic=True`` to any :class:`.Function`, the function can
be used as a sentinel for tracking row order in batched INSERT operations
with RETURNING, allowing the ORM and Core to efficiently batch INSERT
statements while maintaining deterministic row ordering.
Fixes: #13014
Change-Id: I2fabf96c8fbdb6c1d255fd4781cbd31fed17e1e9
The :meth:`_sql.Select.filter_by`, :meth:`_sql.Update.filter_by` and
:meth:`_sql.Delete.filter_by` methods now search across all entities
present in the statement, rather than limiting their search to only the
last joined entity or the first FROM entity. This allows these methods
to locate attributes unambiguously across multiple joined tables,
resolving issues where changing the order of operations such as
:meth:`_sql.Select.with_only_columns` would cause the method to fail.
If an attribute name exists in more than one FROM clause entity, an
:class:`_exc.AmbiguousColumnError` is now raised, indicating that
:meth:`_sql.Select.filter` (or :meth:`_sql.Select.where`) should be used
instead with explicit table-qualified column references.
Fixes: #8601
Change-Id: I6a46b8f4784801f95f7980ca8ef92f1947653572
Confirmed the upstream fix for [1] given at [2] solves the issue
illustrated here, this patch adds a test for this case as our
existing tests did not catch this error in python 3.14.1.
Fixes: #13021
Change-Id: Ie6827279ccf2b2cb2e0fe6029aafdcfefc790f1f
Fixed issue in the :meth:`.ConnectionEvents.after_cursor_execute` method
where the SQL statement and parameter list for an "insertmanyvalues"
operation sent to the event would not be the actual SQL / parameters just
emitted on the cursor, instead being the non-batched form of the statement
that's used as a template to generate the batched statements.
Fixes: #13018
Change-Id: Ib5b6c576f2c7a9ed275de30290d619cc651b4816
Fixed the structure of the SQL string used for the
:ref:`engine_insertmanyvalues` feature when an explicit sequence with
``nextval()`` is used. The SQL function invocation for the sequence has
been moved from being rendered inline within each tuple inside of VALUES to
being rendered once in the SELECT that reads from VALUES. This change
ensures the function is invoked in the correct order as rows are processed,
rather than assuming PostgreSQL will execute inline function calls within
VALUES in a particular order. While current PostgreSQL versions appear to
handle the previous approach correctly, the database does not guarantee
this behavior for future versions.
Fixes: #13015
Change-Id: Ia0a2a4e8f89e21852d7cb550dfa5d9ea9447b590
This adjusts the _DMLOnlyColumnArgument type to be a more
focused _OnlyColumnArgument type where we also add a more tightly
focused coercion, while still allowing ORM attributes to be used
as arguments.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Closes: #13012
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13012
Pull-request-sha: 5ebb402c68
Change-Id: I8bbccaf556ec5ecb2f5cfdd2030bcfa4eb5ce125
Added support for native BOOLEAN support in Oracle Database 23c and above.
The Oracle dialect now renders ``BOOLEAN`` automatically when
:class:`.Boolean` is used in DDL, and also now supports direct use of the
:class:`.BOOLEAN` datatype, when 23c and above is in use. For Oracle
versions prior to 23c, boolean values continue to be emulated using
SMALLINT as before. Special case handling is also present to ensure a
SMALLINT that's interpreted with the :class:`.Boolean` datatype on Oracle
Database 23c and above continues to return bool values. Pull request
courtesy Yeongbae Jeon.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Fixes: #11633Closes: #13000
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13000
Pull-request-sha: 65b8a2c92f
Change-Id: I13be32f6bad5d918f6c7c9622f0deae78f5c6551
Added support for Python 3.14+ template strings (t-strings) via the new
:func:`_sql.tstring` construct, as defined in :pep:`750`. This feature
allows for ergonomic SQL statement construction by automatically
interpolating Python values and SQLAlchemy expressions within template
strings.
Part of the challenge here is the syntax only works on py314, so we have
to exclude the test file at many levels when py314 is not used. not
sure yet how i want to adjust pep8 tests and rules for this.
Fixes: #12548
Change-Id: Ia060d1387ff452fe4f5d924f683529a22a8e1f72
fix a bunch of side effects that were not tested in the gerrit
phase because we run with --backendonly
Change-Id: Iebcedb962e6e11dd247b0da5f309a71db711694c
this DB is extremely erratic in being able to connect. Add
a brute force connection retrier to all engines everywhere
(which for oracledb we can fortunately use their built-in feature
that also works).
This actually works and I can see it pausing under load, reconnecting,
and succeeding. the problem is that absolutely every engine everywhere
needs this routine otherwise an engine without a retrier in it will
crash. That then necessitates digging into testing_engine(),
making sure testing_engine() is used everywhere an engine that's going
to connect is used, then dealing with the fallout from that.
We also simplify some older workarounds for cx_oracle and
hack into config/provision to make oracledb seem like the primary
DBAPI for most tests.
testing_engine has been completely overhauled, making use of a new
post_configure_testing_engine() hook which moves and refines
the SQLite pool sharing and savepoint logic all into sqlite/provision.py
and also allows for cx_oracle to apply a retry event handler.
Change-Id: I4ea4c523effb878290d28b94d8925eb32fc5ae3b
getting some fails on mariadb12 and likely 11 which appear to
be related to calling in MyISAM, which is not used in
modern mysql/mariadb. see if we can just remove this whole
thing and rely on default engines for mariadb/mysql.
this change also removes the "ignore errors" part of the
run deletes for the TablesTest fixture, which was resulting
in compound failures, and apparently a lot of tests were relying
on it skipping nonexistent tables. rather than check for that
we should just improve the tests and probably increase use of
pytest style fixtures overall.
this change also identifies and fixes that memusage_w_backend
tests were running for all backends with a tag like
py314_mysql_backendonly; the memusage tests should basically
never be run as part of the whole suite since they are entirely
unreliable within a full scale test run.
dialect suite tests are also further broken out into those where
every driver should be exercised (i.e. __backend__, for tests that
test datatypes going out and coming back from the database as well
as identity/autoincrement kinds of tests) vs. those where only
one driver per backend is needed (i.e. __sparse_driver_backend__,
for tests like reflection, DDL, CTEs, etc.).
we are also trying to get a --low-connections option that actually
works. changed this so that the testing reaper aggressively disposes
the "global" engines (one per backend / driver) after test classes
are done and before any testing_engine() call. This definitely
works, however some monitoring with PG shows the number of connections
still has brief bursts for some reason. it should be much more
effective than before though as oracle 23/26 really does not handle
more than a few connections.
this change reverts oracle to oracle18c for now in setup.cfg;
further work will be needed to determine if oracle23c can be
run with this test suite
Change-Id: Id87d0ea15155c452615a7edeb9d434c8e55151e7
one particular vector test wont run on oracle 23c free, so
just disable it.
added better skips for the rest of the vector tests and
fixed a deprecation issue.
this will be the first run on the new oracle23 on CI so we'll have to
see how this goes.
Also adjust for mariabdb12 being overly helpful with regards
to stale row updates.
as we are having trouble getting 23c to pass throug transaction
tests, i noted we have an explosion of tests due to the multiple
drivers, so this patch introduces __sparse_driver_backend__
for all tests where we want variety of
database server but there's no need to test every driver.
This should dramatically reduce the size of the test suite run
Change-Id: Ic8d3eb0a60e76b4c54c6bb4a721f90c81ede782b
The :meth:`_events.SessionEvents.do_orm_execute` event now allows direct
mutation or replacement of the :attr:`.ORMExecuteState.parameters`
dictionary or list, which will take effect when the the statement is
executed. Previously, changes to this collection were not accommodated by
the event hook. Pull request courtesy Shamil.
Fixes: #12921Closes: #12989
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12989
Pull-request-sha: 86b64e0617
Change-Id: I04874b6ca720eb2be1470067ced4afd79896e267
Fixed typing issue where :meth:`.Select.with_for_update` would not support
lists of ORM entities in the :paramref:`.Select.with_for_update.of`
parameter. Pull request courtesy Shamil.
Fixes: #12730Closes: #12988
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12988
Pull-request-sha: 41a38bfe38
Change-Id: I61d60a4f4d2b16037da8d5f30e33f5d74fa47374
Fixed issue where using the :meth:`.ColumnOperators.in_` operator with a
nested :class:`.CompoundSelect` statement (e.g. an ``INTERSECT`` of
``UNION`` queries) would raise a :class:`NotImplementedError` when the
nested compound select was the first argument to the outer compound select.
The ``_scalar_type()`` internal method now properly handles nested compound
selects.
Fixes: #12987
Change-Id: I6aa1b38863588d371bbac74b3531b99ccd5fcaec
Added :class:`_orm.DictBundle` as a subclass of :class:`_orm.Bundle`
that returns ``dict`` objects.
Fixes: #12960
Change-Id: I798fb917779eb95bda575f2809e58c2f6d3c4706
adds the ORM profiling tests to backend so that they run
on gerrits that use backendonly for nocext
fix CI only failures
Change-Id: Id36a9bcef1250af58f968a62edb3e7b7cb84f673
Fixed typing issue where :class:`.coalesce` would not return the correct
return type when a nullable form of that argument were passed, even though
this function is meant to select the non-null entry among possibly null
arguments. Pull request courtesy Yannick PÉROUX.
Closes: #12963
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12963
Pull-request-sha: 05d0d9784d
Change-Id: Ife83a384ea57faf446c1fdb542df14627348f40f
Added support for MySQL 8.0.1 + ``FOR SHARE`` to be emitted for the
:meth:`.Select.with_for_uddate` method, which offers compatibility with
``NOWAIT`` and ``SKIP LOCKED``. The new syntax is used only for MySQL when
version 8.0.1 or higher is detected. Pull request courtesy JetDrag.
Fixes: #10134Closes: #12964
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12964
Pull-request-sha: 03d5e37cfd
Change-Id: Iafb7a24363284edcfeead94a348f50a470a88403
Added new implementation for the :meth:`.Select.params` method and that of
similar statements, via a new statement-only
:meth:`.ExecutableStatement.params` method which works more efficiently and
correctly than the previous implementations available from
:class:`.ClauseElement`, by assocating the given parameter dictionary with
the statement overall rather than cloning the statement and rewriting its
bound parameters. The :meth:`_sql.ClauseElement.params` and
:meth:`_sql.ClauseElement.unique_params` methods, when called on an object
that does not implement :class:`.ExecutableStatement`, will continue to
work the old way of cloning the object, and will emit a deprecation
warning. This issue both resolves the architectural / performance
concerns of 🎫`7066` and also provides correct ORM compatibility for
functions like :func:`_orm.aliased`, reported by 🎫`12915`.
Fixes: #7066
Change-Id: I6543c7d0f4da3232b3641fb172c24c446f02f52a
Changed the query style for ORM queries emitted by :meth:`.Session.get` as
well as many-to-one lazy load queries to use the default labeling style,
:attr:`_sql.SelectLabelStyle.LABEL_STYLE_DISAMBIGUATE_ONLY`, which normally
does not apply labels to columns in a SELECT statement. Previously, the
older style :attr:`_sql.SelectLabelStyle.LABEL_STYLE_TABLENAME_PLUS_COL`
that labels columns as `<tablename>_<columname>` was used for
:meth:`.Session.get` to maintain compatibility with :class:`_orm.Query`.
The change allows the string representation of ORM queries to be less
verbose in all cases outside of legacy :class:`_orm.Query` use. Pull
request courtesy Inada Naoki.
Fixes: #12932Closes: #12926
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12926
Pull-request-sha: 6738a73f63
Change-Id: I044a54226a4fcade07adc1a3f5f60b4b3e451a1e
python3.14t is showing connections being referenced and not finalized.
explicit closes/disposes and such dont seem to be working but
ensuring globs are cleared out is working better
Change-Id: I0c53f7702027e37ad11bd6e15e828900c8f52213
Added support for the SQL ``CREATE VIEW`` statement via the new
:class:`.CreateView` DDL class. The new class allows creating database
views from SELECT statements, with support for options such as
``TEMPORARY``, ``IF NOT EXISTS``, and ``MATERIALIZED`` where supported by
the target database. Views defined with :class:`.CreateView` integrate with
:class:`.MetaData` for automated DDL generation and provide a
:class:`.Table` object for querying.
this alters the CreateTableAs commit:
* adds the ability for Table to be associated with Create and Drop
DDL constructs
* Adds CreateView variant of CreateTableAs
* Both associate themselves with Table so they take place in
create_all/create/drop_all/drop
Fixes: #181
Change-Id: If3e568d3d6a6ce19e3d15198c3fbbe06bd847c83
Added support for PostgreSQL 14+ HSTORE subscripting syntax. When connected
to PostgreSQL 14 or later, HSTORE columns now automatically use the native
subscript notation ``hstore_col['key']`` instead of the arrow operator
``hstore_col -> 'key'`` for both read and write operations. This provides
better compatibility with PostgreSQL's native HSTORE subscripting feature
while maintaining backward compatibility with older PostgreSQL versions.
as part of this change we add a new parameter to custom_op "visit_name"
which allows a custom op to refer to a specific visit method in a
dialect's compiler.
Fixes: #12948
Change-Id: Id98d333fe78e31d9c7679cb2902f1c7e458d6e11
Fixed issue in Python 3.14 where dataclass transformation would fail when
a mapped class using :class:`.MappedAsDataclass` included a
:func:`.relationship` referencing a class that was not available at
runtime (e.g., within a ``TYPE_CHECKING`` block). This occurred when using
Python 3.14's :pep:`649` deferred annotations feature, which is the
default behavior without a ``from __future__ import annotations``
directive.
Fixes: #12952
Change-Id: I32f0adba00c32f5bf98fe2880dda1b96a7774d07
GC seems to be working in fewer cases across different interpreters,
so add a new step to force an InstanceState to act as though
it got dereferenced, for those cases where we have del'ed the object,
done a gc collect, and we know it should not be reachable.
Change-Id: Ie97af1d5739f8d5a24aabe1296f14ce99ec1b0e5
the timing here is too sensitive to play out reliably
on CI machines particularly free threaded, so mock the time()
callable instead and ensure with subsecond clock intervals
we do the right math
Change-Id: Icc203ae2298eb4b64e3b45f063811e9527278d0c
G Allajmi