Follow up of 🎫`11471` to fix caching issue where using the
:meth:`.CompoundSelectState.add_cte` method of the
:class:`.CompoundSelectState` construct would not set a correct cache key
which distinguished between different CTE expressions. Also added tests
that would detect issues similar to the one fixed in 🎫`11544`.
Fixes: #11471
Change-Id: Iae6a91077c987d83cd70ea826daff42855491330
(cherry picked from commit 881be0a216)
Fixed bug where the :meth:`.Operators.nulls_first()` and
:meth:`.Operators.nulls_last()` modifiers would not be treated the same way
as :meth:`.Operators.desc()` and :meth:`.Operators.asc()` when determining
if an ORDER BY should be against a label name already in the statement. All
four modifiers are now treated the same within ORDER BY.
Fixes: #11592
Change-Id: I1de1aff679c56af1abfdfd07f9bcbc45ecc5a8cc
(cherry picked from commit 96f1172812)
Fixed issue where the :func:`_sql.collate` construct, which explicitly sets
a collation for a given expression, would maintain collation settings for
the underlying type object from the expression, causing SQL expressions to
have both collations stated at once when used in further expressions for
specific dialects that render explicit type casts, such as that of asyncpg.
The :func:`_sql.collate` construct now assigns its own type to explicitly
include the new collation, assuming it's a string type.
Fixes: #11576
Change-Id: I6fc8904d2bcbc21f11bbca57e4a451ed0edbd879
(cherry picked from commit 35c178c405)
The :paramref:`_orm.aliased.name` parameter to :func:`_orm.aliased` may now
be combined with the :paramref:`_orm.aliased.flat` parameter, producing
per-table names based on a name-prefixed naming convention. Pull request
courtesy Eric Atkin.
Fixes: #11575Closes: #11531
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11531
Pull-request-sha: f85535464b
Change-Id: If79679c7a9598fffe99c033894b7dffecef13939
(cherry picked from commit 0e40962bf3)
Fixed issue in "insertmanyvalues" feature where a particular call to
``cursor.fetchall()`` were not wrapped in SQLAlchemy's exception wrapper,
which apparently can raise a database exception during fetch when using
pyodbc.
Fixes: #11532
Change-Id: Ic07d3e79dd597e18d87a56b45ddffa25e762beb9
(cherry picked from commit fb47dbbc74)
Fixed caching issue where the
:paramref:`_sql.Select.with_for_update.key_share` element of
:meth:`_sql.Select.with_for_update` was not considered as part of the cache
key, leading to incorrect caching if different variations of this parameter
were used with an otherwise identical statement.
Also repairs a traversal issue where the ``of`` element of
``ForUpdateArg`` when set to ``None`` cannot be compared against a
non-None element because the traversal defines it as a clauselist.
Traversal in this case is adjusted to accommodate for this case so that
we dont need to create a risky-to-backport change to ``ForUpdateArg``
itself.
Fixes: #11544
Change-Id: Ie8a50716df06977af58b0c22a8c10e1b64d972b9
(cherry picked from commit 6d2f43e14f)
Fixed additional issues in the event system triggered by unpickling of a
:class:`.Enum` datatype, continuing from 🎫`11365` and
🎫`11360`, where dynamically generated elements of the event
structure would not be present when unpickling in a new process.
Fixes: #11530
Change-Id: Ie1f2b3453d4891051f8719f6d3f6703302d5a86e
(cherry picked from commit dffd96e754)
Fixed caching issue where using the :meth:`.TextualSelect.add_cte` method
of the :class:`.TextualSelect` construct would not set a correct cache key
which distinguished between different CTE expressions.
Fixes: #11471
Change-Id: Ia9ce2c8cfd128f0f130aa9b26448dc23d994c324
(cherry picked from commit faecebc9df)
Fixed bug in :meth:`_sql.FunctionFilter.filter` that would mutate
the existing function in-place. It now behaves like the rest of the
SQLAlchemy API, returning a new instance instead of mutating the
original one.
Fixes: #11426
Change-Id: I46ffebaed82426cfb1623db066686cfb911055a1
(cherry picked from commit fe2ced9e79b9640f3ca135f8d3782dd41ca16782)
Added :paramref:`_schema.Column.insert_default` as an alias of
:paramref:`_schema.Column.default` for compatibility with
func:`_orm.mapped_column`.
Fixes: #11374
Change-Id: I5509b6cbac7b37ac37430a88442b1319cc9c1024
(cherry picked from commit c87572b60c)
Fixed issue where a :class:`.MetaData` collection would not be
serializable, if an :class:`.Enum` or :class:`.Boolean` datatype were
present which had been adapted. This specific scenario in turn could occur
when using the :class:`.Enum` or :class:`.Boolean` within ORM Annotated
Declarative form where type objects frequently get copied.
Fixes: #11365
Change-Id: Iaaa64baad79c41075d37cf53dade744d79e600a3
(cherry picked from commit 93cfb49572)
### Description
Fixes#11328 by adding an overload to ColumnCollection when a non-None default is provided.
### Checklist
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Closes: #11329
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11329
Pull-request-sha: 32db849e0d
Change-Id: I8bef91c423fb7048ec8d4a7c99f70f0b1588c37a
(cherry picked from commit ab6df37dad)
Fixed issue in cursor handling which affected handling of duplicate
:class:`_sql.Column` or similar objcts in the columns clause of
:func:`_sql.select`, both in combination with arbitary :func:`_sql.text()`
clauses in the SELECT list, as well as when attempting to retrieve
:meth:`_engine.Result.mappings` for the object, which would lead to an
internal error.
Fixes: #11306
Change-Id: I418073b2fdba86b2121b6d00eaa40b1805b69bb8
(cherry picked from commit fbb7172c69)
Ensure the ``PYTHONPATH`` variable is properly initialized when
using ``subprocess.run`` in the tests.
Fixes: #11268
Change-Id: Ie2db656364931b3be9033dcaaf7a7c56b383ecca
(cherry picked from commit b5cf61c504)
Fixed regression from the 1.4 series where the refactor of the
:meth:`_types.TypeEngine.with_variant` method introduced at
:ref:`change_6980` failed to accommodate for the ``.copy()`` method, which
will lose the variant mappings that are set up. This becomes an issue for
the very specific case of a "schema" type, which includes types such as
:class:`.Enum` and :class:`.ARRAY`, when they are then used in the context
of an ORM Declarative mapping with mixins where copying of types comes into
play. The variant mapping is now copied as well.
Fixes: #11176
Change-Id: Icf1a2752f60fce863c87ead8b0fe298b0f3d3766
(cherry picked from commit 29a428955a904e235e1b85e928cbe89155aeca82)
Made a change to the adjustment made in version 2.0.10 for 🎫`9618`,
which added the behavior of reconciling RETURNING rows from a bulk INSERT
to the parameters that were passed to it. This behavior included a
comparison of already-DB-converted bound parameter values against returned
row values that was not always "symmetrical" for SQL column types such as
UUIDs, depending on specifics of how different DBAPIs receive such values
versus how they return them, necessitating the need for additional
"sentinel value resolver" methods on these column types. Unfortunately
this broke third party column types such as UUID/GUID types in libraries
like SQLModel which did not implement this special method, raising an error
"Can't match sentinel values in result set to parameter sets". Rather than
attempt to further explain and document this implementation detail of the
"insertmanyvalues" feature including a public version of the new
method, the approach is intead revised to no longer need this extra
conversion step, and the logic that does the comparison now works on the
pre-converted bound parameter value compared to the post-result-processed
value, which should always be of a matching datatype. In the unusual case
that a custom SQL column type that also happens to be used in a "sentinel"
column for bulk INSERT is not receiving and returning the same value type,
the "Can't match" error will be raised, however the mitigation is
straightforward in that the same Python datatype should be passed as that
returned.
Fixes: #11160
Change-Id: Ica62571e923ad9545eb90502e6732b11875b164a
(cherry picked from commit 4c0af9e93d)
Fixed issue in :ref:`engine_insertmanyvalues` feature where using a primary
key column with an "inline execute" default generator such as an explicit
:class:`.Sequence` with an explcit schema name, while at the same time
using the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
feature would fail to render the sequence or the parameters properly,
leading to errors.
Fixes: #11157
Change-Id: I35666af46d40996aff35d3d39f48c150d838e6e4
(cherry picked from commit e3f7bc683a)
Fixed issue in ORM annotated declarative where using
:func:`_orm.mapped_column()` with an :paramref:`_orm.mapped_column.index`
or :paramref:`_orm.mapped_column.unique` setting of False would be
overridden by an incoming ``Annotated`` element that featured that
parameter set to ``True``, even though the immediate
:func:`_orm.mapped_column()` element is more specific and should take
precedence. The logic to reconcile the booleans has been enhanced to
accommodate a local value of ``False`` as still taking precedence over an
incoming ``True`` value from the annotated element.
Fixes: #11091
Change-Id: I15cda4a0a07a289015c0a09bbe3ca2849956604e
(cherry picked from commit e4c4bd03ab)
Adjusted the fix made in 🎫`10570`, released in 2.0.23, where new
logic was added to reconcile possibly changing bound parameter values
across cache key generations used within the :func:`_orm.with_expression`
construct. The new logic changes the approach by which the new bound
parameter values are associated with the statement, avoiding the need to
deep-copy the statement which can result in a significant performance
penalty for very deep / complex SQL constructs. The new approach no longer
requires this deep-copy step.
Fixes: #11085
Change-Id: Ia51eb4e949c8f37af135399925a9916b9ed4ad2f
(cherry picked from commit 06be748b47)
Added new core execution option
paramref:`_engine.Connection.execution_options.preserve_rowcount`
to unconditionally save the ``rowcount`` attribute from the cursor in the
class:`_engine.Result` returned from an execution, regardless of the
statement being executed.
When this option is provided the correct value is also set when
an INSERT makes use of the "insertmanyvalues" mode, that may use
more than one actualy cursor execution.
Fixes: #10974
Change-Id: Icecef6b7539be9f0a1a02b9539864f5f163dcfbc
(cherry picked from commit f0537442eb)
manually update the files to remove literal string concat on the same line,
since black does not seem to be making progress in handling these
Change-Id: I3c651374c5f3db5b8bc0c700328d67ca03743b7b
(cherry picked from commit 3fbbe8d67b)
Fixed an issue regarding the use of the :class:`.Uuid` datatype with the
:paramref:`.Uuid.as_uuid` parameter set to False, when using the pymssql
dialect. ORM-optimized INSERT statements (e.g. the "insertmanyvalues"
feature) would not correctly align primary key UUID values for bulk INSERT
statements, resulting in errors.
This change also adds a small degree of generalization to the
Uuid datatype by adding the native/non-native compilation conditional
to the base compiler.
Patch is originally part of Ib920871102b9b64f2cba9697f5cb72b6263e4ed8
which is implementing native UUID for mariadb in 2.1 only.
Change-Id: I96cbec5c0ece312b345206aa5a5db2ffcf732d41
(cherry picked from commit 9b1c9d5d2e)
Fixed issues in :func:`_sql.case` where the logic for determining the
type of the expression could result in :class:`.NullType` if the last
element in the "whens" had no type, or in other cases where the type
could resolve to ``None``. The logic has been updated to scan all
given expressions so that the first non-null type is used, as well as
to always ensure a type is present. Pull request courtesy David Evans.
updates to test suite to use modern fixture patterns by Mike
Fixes: #10843Closes: #10847
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10847
Pull-request-sha: 4fd5c39ab5
Change-Id: I40f905ac336a8a42b617ff9473dbd9c22ac57505
(cherry picked from commit 8f4ac0c0f0)
Improved compilation of :func:`_sql.any_` / :func:`_sql.all_` in the
context of a negation of boolean comparison, will now render ``NOT (expr)``
rather than reversing the equality operator to not equals, allowing
finer-grained control of negations for these non-typical operators.
Fixes: #10817
Change-Id: If0b324b1220ad3c7f053af91e8a61c81015f312a
(cherry picked from commit f3ca2350a5)
Added support for :ref:`oracledb` in async mode.
The current implementation has some limitation, preventing
the support for :meth:`_asyncio.AsyncConnection.stream`.
Improved support if planned for the 2.1 release of SQLAlchemy.
Fixes: #10679
Change-Id: Iff123cf6241bcfa0fbac57529b80f933951be0a7
(cherry picked from commit dca7673fb6)
Fixed issue in stringify for SQL elements, where a specific dialect is not
passed, where a dialect-specific element such as the PostgreSQL "on
conflict do update" construct is encountered and then fails to provide for
a stringify dialect with the appropriate state to render the construct,
leading to internal errors.
Fixed issue where stringifying or compiling a :class:`.CTE` that was
against a DML construct such as an :func:`_sql.insert` construct would fail
to stringify, due to a mis-detection that the statement overall is an
INSERT, leading to internal errors.
Fixes: #10753
Change-Id: I783eca3fc7bbc1794fedd325d58181dbcc7e0b75
(cherry picked from commit 0248efb761)
I updated flake8 in pre-commit a few days ago but forgot to do it in
tox.
this flake seems to be picking inside of f-strings, which black does
not fix, so fix these manually.
Change-Id: I9a641a99e280fbba9d893a6f1f051b5039d5d4eb
(cherry picked from commit 7bc33ca551)
Fixed issue where error reporting for unexpected schema item when creating
objects like :class:`_schema.Table` would incorrectly handle an argument
that was itself passed as a tuple, leading to a formatting error. The
error message has been modernized to use f-strings.
this change necessitated an update to flake8 as version 5 was
mis-interpreting f-strings that had semicolons in them.
Black is also unable to format some of these f-strings which had
to be broken out, unclear if there is a newer Black available.
Fixes: #10654
Change-Id: I703e94282c27ccf06f4aa315e8a11bd97b719170
(cherry picked from commit 2458ceee94)
Fixed issue where use of :func:`_orm.foreign` annotation on a
non-initialized :func:`_orm.mapped_column` construct would produce an
expression without a type, which was then not updated at initialization
time of the actual column, leading to issues such as relationships not
determining ``use_get`` appropriately.
Fixes: #10597
Change-Id: I8339ba715ec6bd1f50888f8a424c3ac156e2364f
(cherry picked from commit 432eb350a4)
Fixed issue where using the same bound parameter more than once with
``literal_execute=True`` in some combinations with other literal rendering
parameters would cause the wrong values to render due to an iteration
issue.
Fixes: #10142
Change-Id: Idde314006568e3445558f0104aed9d2f4af72b56
the expression clauselist feature added in #7744 failed to accommodate
this parameter that is used only by the PostgreSQL JSON
operators.
Fixed 2.0 regression caused by 🎫`7744` where chains of expressions
involving PostgreSQL JSON operators combined with other operators such as
string concatenation would lose correct parenthesization, due to an
implementation detail specific to the PostgreSQL dialect.
Fixes: #10479
Change-Id: Ic168bf6afd8bf1cfa648f2bad22fdd7254feaa34
Fixed regression in recently revised "insertmanyvalues" feature (likely
issue 🎫`9618`) where the ORM would inadvertently attempt to
interpret a non-RETURNING result as one with RETURNING, in the case where
the ``implicit_returning=False`` parameter were applied to the mapped
:class:`.Table`, indicating that "insertmanyvalues" cannot be used if the
primary key values are not provided.
This includes a refinement to insertmanyvalues where we consider
return_defaults() with supplemental_cols to be the same as an explicit
returning(), since that's the purpose of this. This saves us some
extra exceptions that could be thrown per-dialect if implicit_returning
were set to False in some cases.
Fixed issue within some dialects where the dialect could incorrectly return
an empty result set for an INSERT statement that does not actually return
rows at all, due to artfacts from pre- or post-fetching the primary key of
the row or rows still being present. Affected dialects included asyncpg,
all mssql dialects.
Fixes: #10453
Change-Id: Ie2e7e4f4cd9180558f9da315d21895347ec6d4f7
Fixed issue where referring to a FROM entry in the SET clause of an UPDATE
statement would not include it in the FROM clause of the UPDATE statement,
if that entry were nowhere else in the statement; this occurs currently for
CTEs that were added using :meth:`.Update.add_cte` to provide the desired
CTE at the top of the statement.
Fixes: #10408
Change-Id: I6e3c6ca7a00cc884bda7e0f24c62c34c75134e5b
Fixed a wide range of :func:`_orm.mapped_column` parameters that were not
being transferred when using the :func:`_orm.mapped_column` object inside
of a pep-593 ``Annotated`` object, including
:paramref:`_orm.mapped_column.sort_order`,
:paramref:`_orm.mapped_column.deferred`,
:paramref:`_orm.mapped_column.autoincrement`,
:paramref:`_orm.mapped_column.system`, :paramref:`_orm.mapped_column.info`
etc.
Additionally, it remains not supported to have dataclass arguments, such as
:paramref:`_orm.mapped_column.kw_only`,
:paramref:`_orm.mapped_column.default_factory` etc. indicated within the
:func:`_orm.mapped_column` received by ``Annotated``, as this is not
supported with pep-681 Dataclass Transforms. A warning is now emitted when
these parameters are used within ``Annotated`` in this way (and they
continue to be ignored).
Fixes: #10369Fixes: #10046
Change-Id: Ibcfb287cba0e764db0ae15fab8049bbb9f94dd1b
Fixed regression introduced in 2.0.20 via 🎫`9600` fix which
attempted to add more formal typing to
:paramref:`_schema.MetaData.naming_convention`. This change prevented basic
naming convention dictionaries from passing typing and has been adjusted so
that a plain dictionary of strings for keys as well as dictionaries that
use constraint types as keys or a mix of both, are again accepted.
As part of this change, lesser used forms of the naming convention
dictionary are also typed, including that it currently allows for
``Constraint`` type objects as keys as well.
Fixes: #9284Fixes: #10264
Change-Id: Ic6561dd65058e4de3a7a393295b9863fc065db13
The :class:`.Values` construct will now automatically create a proxy (i.e.
a copy) of a :class:`_sql.column` if the column were already associated
with an existing FROM clause. This allows that an expression like
``values_obj.c.colname`` will produce the correct FROM clause even in the
case that ``colname`` was passed as a :class:`_sql.column` that was already
used with a previous :class:`.Values` or other table construct.
Originally this was considered to be a candidate for an error condition,
however it's likely this pattern is already in widespread use so it's
now added to support.
* adjust unrelated dml test recently added for update..returning *
case to not rely upon ordering
Fixes: #10280
Change-Id: I6e60e5b7cb7abd6a7bbd4722970ebf025596ab9c
Adjusted the :class:`_types.Enum` datatype to accept an argument of
``None`` for the :paramref:`_types.Enum.length` parameter, resulting in a
VARCHAR or other textual type with no length in the resulting DDL. This
allows for new elements of any length to be added to the type after it
exists in the schema. Pull request courtesy Eugene Toder.
Fixes: #10269Closes: #10274
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10274
Pull-request-sha: 651afaaea7
Change-Id: I374ae9e5fa63da21a4cc87e323ce1a54acd6e39b
Adjusted the operator precedence for the string concatenation operator to
be equal to that of string matching operators, such as
:meth:`.ColumnElement.like`, :meth:`.ColumnElement.regexp_match`,
:meth:`.ColumnElement.match`, etc., as well as plain ``==`` which has the
same precedence as string comparison operators, so that parenthesis will be
applied to a string concatenation expression that follows a string match
operator. This provides for backends such as PostgreSQL where the "regexp
match" operator is apparently of higher precedence than the string
concatenation operator.
Fixes: #9610
Change-Id: I73640e40e445375177340e1ed8f45b5da98d6dfb
Fixes the rendering of the Oracle only ``order`` attribute in
Sequence and Identity that was passed also when rendering
the DDL in PostgreSQL.
Fixes: #10207
Change-Id: I5b918eab38ba68fa10a213a79e2bd0cc48401a02
Fixed issue where unpickling of a :class:`_schema.Column` or other
:class:`_sql.ColumnElement` would fail to restore the correct "comparator"
object, which is used to generate SQL expressions specific to the type
object.
Fixes: #10213
Change-Id: I74e805024bcc0d93d549bd94757c2865b3117d72
Fixed issue where internal cloning used by the ORM for expressions like
:meth:`_orm.relationship.Comparator.any` to produce correlated EXISTS
constructs would interfere with the "cartesian product warning" feature of
the SQL compiler, leading the SQL compiler to warn when all elements of the
statement were correctly joined.
Fixes: #10124
Change-Id: I31c1ba538e2b943278e8cc0b7fddc107968a0826
Renamed :attr:`_result.Row.t` and :meth:`_result.Row.tuple` to
:attr:`_result.Row._t` and :meth:`_result.Row._tuple`; this is to suit the
policy that all methods and pre-defined attributes on :class:`.Row` should
be in the style of Python standard library ``namedtuple`` where all fixed
names have a leading underscore, to avoid name conflicts with existing
column names. The previous method/attribute is now deprecated and will
emit a deprecation warning.
Fixes: #10093
Change-Id: Ibb0e9423b55590c9dda8ee554187abd9a9122590
Federico Caselli