Implemented support for the GROUPS frame specification in window functions
by adding :paramref:`_sql.over.groups` option to :func:`_sql.over`
and :meth:`.FunctionElement.over`. Pull request courtesy Kaan Dikmen.
Fixes: #12450Closes: #12445
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12445
Pull-request-sha: c0808e135f
Change-Id: I9ff504a9c9650485830c4a0eaf44162898a3a2ad
(cherry picked from commit 0bbc515f904446d3f0beede54321b628f32fbdad)
When building a PostgreSQL ``ARRAY`` literal using
:class:`_postgresql.array` with an empty ``clauses`` argument, the
:paramref:`_postgresql.array.type_` parameter is now significant in that it
will be used to render the resulting ``ARRAY[]`` SQL expression with a
cast, such as ``ARRAY[]::INTEGER``. Pull request courtesy Denis Laxalde.
Fixes: #12432Closes: #12435
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12435
Pull-request-sha: 9633d3c15d
Change-Id: I29ed7bd0562b82351d22de0658fb46c31cfe44f6
(cherry picked from commit 588cc6ed8e)
Fixed regression caused by 🎫`7471` leading to a SQL compilation
issue where name disambiguation for two same-named FROM clauses with table
aliasing in use at the same time would produce invalid SQL in the FROM
clause with two "AS" clauses for the aliased table, due to double aliasing.
Fixes: #12451
Change-Id: I981823f8f2cdf3992d65ace93a21fc20d1d74cda
(cherry picked from commit 9ea3be0681)
Added support for specifying a list of columns for ``SET NULL`` and ``SET
DEFAULT`` actions of ``ON DELETE`` clause of foreign key definition on
PostgreSQL. Pull request courtesy Denis Laxalde.
Fixes: #11595Closes: #12421
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12421
Pull-request-sha: d0394db706
Change-Id: I036a559ae4a8efafe9ba64d776a840bd785a7397
(cherry picked from commit 39bb17442c)
Fixed issue in :class:`.CTE` constructs involving multiple DDL
:class:`.Insert` statements with multiple VALUES parameter sets where the
bound parameter names generated for these parameter sets would conflict,
generating a compile time error.
Fixes: #12363
Change-Id: If8344ff725d4e0ec58d3ff61f38a0edcfc5bdebd
(cherry picked from commit ec20f346a6)
improve test error logging
remove obsolete emulated pipeline
fix test in python 3.8
add order to test
Change-Id: I2003f256a2690ee5673c72e2f1cb1340af750f83
Added new parameters :paramref:`.AddConstraint.isolate_from_table` and
:paramref:`.DropConstraint.isolate_from_table`, defaulting to True, which
both document and allow to be controllable the long-standing behavior of
these two constructs blocking the given constraint from being included
inline within the "CREATE TABLE" sequence, under the assumption that
separate add/drop directives were to be used.
Fixes: #12382
Change-Id: I53c4170ccb5803f69945ba7aa3d3a143131508eb
(cherry picked from commit d6f11d9030)
Fixed issue where using :func:`_orm.aliased` around a :class:`.CTE`
construct could cause inappropriate "duplicate CTE" errors in cases where
that aliased construct appeared multiple times in a single statement.
Fixes: #12364
Change-Id: I9625cd83e9baf5312cdc644b38951353708d3b86
(cherry picked from commit 42ddb1fd5f)
Fixed SQL composition bug which impacted caching where using a ``None``
value inside of an ``in_()`` expression would bypass the usual "expanded
bind parameter" logic used by the IN construct, which allows proper caching
to take place.
Fixes: #12314
References: #12312
Change-Id: I0d2fc4e15c73407379ba368dd4ee32660fc66259
(cherry picked from commit 79505b03b6)
Reorganized the internals by which the `.c` collection on a
:class:`.FromClause` gets generated so that it is resilient against the
collection being accessed in concurrent fashion. An example is creating a
:class:`.Alias` or :class:`.Subquery` and accessing it as a module level
variable. This impacts the Oracle dialect which uses such module-level
global alias objects but is of general use as well.
Fixes: #12302
Change-Id: I30cb07c286affce24e2d85e49f9df5b787438d86
(cherry picked from commit 3cd9a5b42f)
Fixed issue in "lambda SQL" feature where the tracking of bound parameters
could be corrupted if the same lambda were evaluated across multiple
compile phases, including when using the same lambda across multiple engine
instances or with statement caching disabled.
Fixes: #12084
Change-Id: I327aa93ce7feb2326a22113164bd834b96b6b889
(cherry picked from commit 5bbefc41b7)
Fixed regression caused by an internal code change in response to recent
Mypy releases that caused the very unusual case of a list of ORM-mapped
attribute expressions passed to :meth:`.ColumnOperators.in_` to no longer
be accepted.
in this commit we had to revisit d8dd28c42e where mypy typing
didn't accept ColumnOperartors. the type here is the _HasClauseElement[_T]
protocol which means we need to use a duck type for a runtime check.
Fixes: #12019
Change-Id: Ib378e9cb8defb49d5ac4d726ec93d6bdc581b6a9
(cherry picked from commit aaddd7c840)
Fixed regression from 1.4 where some datatypes such as those derived from
:class:`.TypeDecorator` could not be pickled when they were part of a
larger SQL expression composition due to internal supporting structures
themselves not being pickleable.
Fixes: #12002
Change-Id: I016e37b0c62071413f24c9aac35f6ecf475becaa
(cherry picked from commit fa56821578)
Datatypes that are binary based such as :class:`.VARBINARY` will resolve to
:class:`.LargeBinary` when the :meth:`.TypeEngine.as_generic()` method is
called.
Fixes: #11978
Change-Id: I2e0586324fb0f1c367da61f0074b35c96fbe2fd0
(cherry picked from commit 858eba6156)
Fixed bug where the ``metadata`` element of an ``Enum`` datatype would not
be transferred to the new :class:`.MetaData` object when the type had been
copied via a :meth:`.Table.to_metadata` operation, leading to inconsistent
behaviors within create/drop sequences.
Fixes: #11802
Change-Id: Ibbc93aa31bdfde0d67a9530f41a08e826c17d58e
(cherry picked from commit 22cbc7dcb4)
Fixed regression in :meth:`_sql.Select.with_statement_hint` and others
where the generative behavior of the method stopped producing a copy of the
object.
Fixes: #11703
Change-Id: Ia4482f91f76fae9982dc6b075bf5cfec7042ffa6
(cherry picked from commit a9747467a8)
Follow up of 🎫`11471` to fix caching issue where using the
:meth:`.CompoundSelectState.add_cte` method of the
:class:`.CompoundSelectState` construct would not set a correct cache key
which distinguished between different CTE expressions. Also added tests
that would detect issues similar to the one fixed in 🎫`11544`.
Fixes: #11471
Change-Id: Iae6a91077c987d83cd70ea826daff42855491330
(cherry picked from commit 881be0a216)
Fixed bug where the :meth:`.Operators.nulls_first()` and
:meth:`.Operators.nulls_last()` modifiers would not be treated the same way
as :meth:`.Operators.desc()` and :meth:`.Operators.asc()` when determining
if an ORDER BY should be against a label name already in the statement. All
four modifiers are now treated the same within ORDER BY.
Fixes: #11592
Change-Id: I1de1aff679c56af1abfdfd07f9bcbc45ecc5a8cc
(cherry picked from commit 96f1172812)
Fixed issue where the :func:`_sql.collate` construct, which explicitly sets
a collation for a given expression, would maintain collation settings for
the underlying type object from the expression, causing SQL expressions to
have both collations stated at once when used in further expressions for
specific dialects that render explicit type casts, such as that of asyncpg.
The :func:`_sql.collate` construct now assigns its own type to explicitly
include the new collation, assuming it's a string type.
Fixes: #11576
Change-Id: I6fc8904d2bcbc21f11bbca57e4a451ed0edbd879
(cherry picked from commit 35c178c405)
The :paramref:`_orm.aliased.name` parameter to :func:`_orm.aliased` may now
be combined with the :paramref:`_orm.aliased.flat` parameter, producing
per-table names based on a name-prefixed naming convention. Pull request
courtesy Eric Atkin.
Fixes: #11575Closes: #11531
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11531
Pull-request-sha: f85535464b
Change-Id: If79679c7a9598fffe99c033894b7dffecef13939
(cherry picked from commit 0e40962bf3)
Fixed issue in "insertmanyvalues" feature where a particular call to
``cursor.fetchall()`` were not wrapped in SQLAlchemy's exception wrapper,
which apparently can raise a database exception during fetch when using
pyodbc.
Fixes: #11532
Change-Id: Ic07d3e79dd597e18d87a56b45ddffa25e762beb9
(cherry picked from commit fb47dbbc74)
Fixed caching issue where the
:paramref:`_sql.Select.with_for_update.key_share` element of
:meth:`_sql.Select.with_for_update` was not considered as part of the cache
key, leading to incorrect caching if different variations of this parameter
were used with an otherwise identical statement.
Also repairs a traversal issue where the ``of`` element of
``ForUpdateArg`` when set to ``None`` cannot be compared against a
non-None element because the traversal defines it as a clauselist.
Traversal in this case is adjusted to accommodate for this case so that
we dont need to create a risky-to-backport change to ``ForUpdateArg``
itself.
Fixes: #11544
Change-Id: Ie8a50716df06977af58b0c22a8c10e1b64d972b9
(cherry picked from commit 6d2f43e14f)
Fixed additional issues in the event system triggered by unpickling of a
:class:`.Enum` datatype, continuing from 🎫`11365` and
🎫`11360`, where dynamically generated elements of the event
structure would not be present when unpickling in a new process.
Fixes: #11530
Change-Id: Ie1f2b3453d4891051f8719f6d3f6703302d5a86e
(cherry picked from commit dffd96e754)
Fixed caching issue where using the :meth:`.TextualSelect.add_cte` method
of the :class:`.TextualSelect` construct would not set a correct cache key
which distinguished between different CTE expressions.
Fixes: #11471
Change-Id: Ia9ce2c8cfd128f0f130aa9b26448dc23d994c324
(cherry picked from commit faecebc9df)
Fixed bug in :meth:`_sql.FunctionFilter.filter` that would mutate
the existing function in-place. It now behaves like the rest of the
SQLAlchemy API, returning a new instance instead of mutating the
original one.
Fixes: #11426
Change-Id: I46ffebaed82426cfb1623db066686cfb911055a1
(cherry picked from commit fe2ced9e79b9640f3ca135f8d3782dd41ca16782)
Added :paramref:`_schema.Column.insert_default` as an alias of
:paramref:`_schema.Column.default` for compatibility with
func:`_orm.mapped_column`.
Fixes: #11374
Change-Id: I5509b6cbac7b37ac37430a88442b1319cc9c1024
(cherry picked from commit c87572b60c)
Fixed issue where a :class:`.MetaData` collection would not be
serializable, if an :class:`.Enum` or :class:`.Boolean` datatype were
present which had been adapted. This specific scenario in turn could occur
when using the :class:`.Enum` or :class:`.Boolean` within ORM Annotated
Declarative form where type objects frequently get copied.
Fixes: #11365
Change-Id: Iaaa64baad79c41075d37cf53dade744d79e600a3
(cherry picked from commit 93cfb49572)
### Description
Fixes#11328 by adding an overload to ColumnCollection when a non-None default is provided.
### Checklist
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Closes: #11329
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11329
Pull-request-sha: 32db849e0d
Change-Id: I8bef91c423fb7048ec8d4a7c99f70f0b1588c37a
(cherry picked from commit ab6df37dad)
Fixed issue in cursor handling which affected handling of duplicate
:class:`_sql.Column` or similar objcts in the columns clause of
:func:`_sql.select`, both in combination with arbitary :func:`_sql.text()`
clauses in the SELECT list, as well as when attempting to retrieve
:meth:`_engine.Result.mappings` for the object, which would lead to an
internal error.
Fixes: #11306
Change-Id: I418073b2fdba86b2121b6d00eaa40b1805b69bb8
(cherry picked from commit fbb7172c69)
Ensure the ``PYTHONPATH`` variable is properly initialized when
using ``subprocess.run`` in the tests.
Fixes: #11268
Change-Id: Ie2db656364931b3be9033dcaaf7a7c56b383ecca
(cherry picked from commit b5cf61c504)
Fixed regression from the 1.4 series where the refactor of the
:meth:`_types.TypeEngine.with_variant` method introduced at
:ref:`change_6980` failed to accommodate for the ``.copy()`` method, which
will lose the variant mappings that are set up. This becomes an issue for
the very specific case of a "schema" type, which includes types such as
:class:`.Enum` and :class:`.ARRAY`, when they are then used in the context
of an ORM Declarative mapping with mixins where copying of types comes into
play. The variant mapping is now copied as well.
Fixes: #11176
Change-Id: Icf1a2752f60fce863c87ead8b0fe298b0f3d3766
(cherry picked from commit 29a428955a904e235e1b85e928cbe89155aeca82)
Made a change to the adjustment made in version 2.0.10 for 🎫`9618`,
which added the behavior of reconciling RETURNING rows from a bulk INSERT
to the parameters that were passed to it. This behavior included a
comparison of already-DB-converted bound parameter values against returned
row values that was not always "symmetrical" for SQL column types such as
UUIDs, depending on specifics of how different DBAPIs receive such values
versus how they return them, necessitating the need for additional
"sentinel value resolver" methods on these column types. Unfortunately
this broke third party column types such as UUID/GUID types in libraries
like SQLModel which did not implement this special method, raising an error
"Can't match sentinel values in result set to parameter sets". Rather than
attempt to further explain and document this implementation detail of the
"insertmanyvalues" feature including a public version of the new
method, the approach is intead revised to no longer need this extra
conversion step, and the logic that does the comparison now works on the
pre-converted bound parameter value compared to the post-result-processed
value, which should always be of a matching datatype. In the unusual case
that a custom SQL column type that also happens to be used in a "sentinel"
column for bulk INSERT is not receiving and returning the same value type,
the "Can't match" error will be raised, however the mitigation is
straightforward in that the same Python datatype should be passed as that
returned.
Fixes: #11160
Change-Id: Ica62571e923ad9545eb90502e6732b11875b164a
(cherry picked from commit 4c0af9e93d)
Fixed issue in :ref:`engine_insertmanyvalues` feature where using a primary
key column with an "inline execute" default generator such as an explicit
:class:`.Sequence` with an explcit schema name, while at the same time
using the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
feature would fail to render the sequence or the parameters properly,
leading to errors.
Fixes: #11157
Change-Id: I35666af46d40996aff35d3d39f48c150d838e6e4
(cherry picked from commit e3f7bc683a)
Fixed issue in ORM annotated declarative where using
:func:`_orm.mapped_column()` with an :paramref:`_orm.mapped_column.index`
or :paramref:`_orm.mapped_column.unique` setting of False would be
overridden by an incoming ``Annotated`` element that featured that
parameter set to ``True``, even though the immediate
:func:`_orm.mapped_column()` element is more specific and should take
precedence. The logic to reconcile the booleans has been enhanced to
accommodate a local value of ``False`` as still taking precedence over an
incoming ``True`` value from the annotated element.
Fixes: #11091
Change-Id: I15cda4a0a07a289015c0a09bbe3ca2849956604e
(cherry picked from commit e4c4bd03ab)
Adjusted the fix made in 🎫`10570`, released in 2.0.23, where new
logic was added to reconcile possibly changing bound parameter values
across cache key generations used within the :func:`_orm.with_expression`
construct. The new logic changes the approach by which the new bound
parameter values are associated with the statement, avoiding the need to
deep-copy the statement which can result in a significant performance
penalty for very deep / complex SQL constructs. The new approach no longer
requires this deep-copy step.
Fixes: #11085
Change-Id: Ia51eb4e949c8f37af135399925a9916b9ed4ad2f
(cherry picked from commit 06be748b47)
Added new core execution option
paramref:`_engine.Connection.execution_options.preserve_rowcount`
to unconditionally save the ``rowcount`` attribute from the cursor in the
class:`_engine.Result` returned from an execution, regardless of the
statement being executed.
When this option is provided the correct value is also set when
an INSERT makes use of the "insertmanyvalues" mode, that may use
more than one actualy cursor execution.
Fixes: #10974
Change-Id: Icecef6b7539be9f0a1a02b9539864f5f163dcfbc
(cherry picked from commit f0537442eb)
manually update the files to remove literal string concat on the same line,
since black does not seem to be making progress in handling these
Change-Id: I3c651374c5f3db5b8bc0c700328d67ca03743b7b
(cherry picked from commit 3fbbe8d67b)
Fixed an issue regarding the use of the :class:`.Uuid` datatype with the
:paramref:`.Uuid.as_uuid` parameter set to False, when using the pymssql
dialect. ORM-optimized INSERT statements (e.g. the "insertmanyvalues"
feature) would not correctly align primary key UUID values for bulk INSERT
statements, resulting in errors.
This change also adds a small degree of generalization to the
Uuid datatype by adding the native/non-native compilation conditional
to the base compiler.
Patch is originally part of Ib920871102b9b64f2cba9697f5cb72b6263e4ed8
which is implementing native UUID for mariadb in 2.1 only.
Change-Id: I96cbec5c0ece312b345206aa5a5db2ffcf732d41
(cherry picked from commit 9b1c9d5d2e)
Fixed issues in :func:`_sql.case` where the logic for determining the
type of the expression could result in :class:`.NullType` if the last
element in the "whens" had no type, or in other cases where the type
could resolve to ``None``. The logic has been updated to scan all
given expressions so that the first non-null type is used, as well as
to always ensure a type is present. Pull request courtesy David Evans.
updates to test suite to use modern fixture patterns by Mike
Fixes: #10843Closes: #10847
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10847
Pull-request-sha: 4fd5c39ab5
Change-Id: I40f905ac336a8a42b617ff9473dbd9c22ac57505
(cherry picked from commit 8f4ac0c0f0)
Improved compilation of :func:`_sql.any_` / :func:`_sql.all_` in the
context of a negation of boolean comparison, will now render ``NOT (expr)``
rather than reversing the equality operator to not equals, allowing
finer-grained control of negations for these non-typical operators.
Fixes: #10817
Change-Id: If0b324b1220ad3c7f053af91e8a61c81015f312a
(cherry picked from commit f3ca2350a5)
Added support for :ref:`oracledb` in async mode.
The current implementation has some limitation, preventing
the support for :meth:`_asyncio.AsyncConnection.stream`.
Improved support if planned for the 2.1 release of SQLAlchemy.
Fixes: #10679
Change-Id: Iff123cf6241bcfa0fbac57529b80f933951be0a7
(cherry picked from commit dca7673fb6)
Kaan