Fixed bug where the ``metadata`` element of an ``Enum`` datatype would not
be transferred to the new :class:`.MetaData` object when the type had been
copied via a :meth:`.Table.to_metadata` operation, leading to inconsistent
behaviors within create/drop sequences.
Fixes: #11802
Change-Id: Ibbc93aa31bdfde0d67a9530f41a08e826c17d58e
Fixed regression in :meth:`_sql.Select.with_statement_hint` and others
where the generative behavior of the method stopped producing a copy of the
object.
Fixes: #11703
Change-Id: Ia4482f91f76fae9982dc6b075bf5cfec7042ffa6
Follow up of 🎫`11471` to fix caching issue where using the
:meth:`.CompoundSelectState.add_cte` method of the
:class:`.CompoundSelectState` construct would not set a correct cache key
which distinguished between different CTE expressions. Also added tests
that would detect issues similar to the one fixed in 🎫`11544`.
Fixes: #11471
Change-Id: Iae6a91077c987d83cd70ea826daff42855491330
Fixed bug where the :meth:`.Operators.nulls_first()` and
:meth:`.Operators.nulls_last()` modifiers would not be treated the same way
as :meth:`.Operators.desc()` and :meth:`.Operators.asc()` when determining
if an ORDER BY should be against a label name already in the statement. All
four modifiers are now treated the same within ORDER BY.
Fixes: #11592
Change-Id: I1de1aff679c56af1abfdfd07f9bcbc45ecc5a8cc
Fixed issue where the :func:`_sql.collate` construct, which explicitly sets
a collation for a given expression, would maintain collation settings for
the underlying type object from the expression, causing SQL expressions to
have both collations stated at once when used in further expressions for
specific dialects that render explicit type casts, such as that of asyncpg.
The :func:`_sql.collate` construct now assigns its own type to explicitly
include the new collation, assuming it's a string type.
Fixes: #11576
Change-Id: I6fc8904d2bcbc21f11bbca57e4a451ed0edbd879
The :paramref:`_orm.aliased.name` parameter to :func:`_orm.aliased` may now
be combined with the :paramref:`_orm.aliased.flat` parameter, producing
per-table names based on a name-prefixed naming convention. Pull request
courtesy Eric Atkin.
Fixes: #11575Closes: #11531
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11531
Pull-request-sha: f85535464b
Change-Id: If79679c7a9598fffe99c033894b7dffecef13939
Added new execution option
:paramref:`_engine.Connection.execution_options.driver_column_names`. This
option disables the "name normalize" step that takes place against the
DBAPI ``cursor.description`` for uppercase-default backends like Oracle,
and will cause the keys of a result set (e.g. named tuple names, dictionary
keys in :attr:`.Row._mapping`, etc.) to be exactly what was delivered in
cursor.description. This is mostly useful for plain textual statements
using :func:`_sql.text` or :meth:`_engine.Connection.exec_driver_sql`.
Fixes: #10789
Change-Id: Ib647b25bb53492fa839af04dd032d9f061e630af
Fixed issue in "insertmanyvalues" feature where a particular call to
``cursor.fetchall()`` were not wrapped in SQLAlchemy's exception wrapper,
which apparently can raise a database exception during fetch when using
pyodbc.
Fixes: #11532
Change-Id: Ic07d3e79dd597e18d87a56b45ddffa25e762beb9
Fixed caching issue where the
:paramref:`_sql.Select.with_for_update.key_share` element of
:meth:`_sql.Select.with_for_update` was not considered as part of the cache
key, leading to incorrect caching if different variations of this parameter
were used with an otherwise identical statement.
Also repairs a traversal issue where the ``of`` element of
``ForUpdateArg`` when set to ``None`` cannot be compared against a
non-None element because the traversal defines it as a clauselist.
Traversal in this case is adjusted to accommodate for this case so that
we dont need to create a risky-to-backport change to ``ForUpdateArg``
itself.
Fixes: #11544
Change-Id: Ie8a50716df06977af58b0c22a8c10e1b64d972b9
Enhanced the caching structure of the :paramref:`.over.rows` and
:paramref:`.over.range` so that different numerical values for the rows /
range fields are cached on the same cache key, to the extent that the
underlying SQL does not actually change (i.e. "unbounded", "current row",
negative/positive status will still change the cache key). This prevents
the use of many different numerical range/rows value for a query that is
otherwise identical from filling up the SQL cache.
Note that the semi-private compiler method ``_format_frame_clause()``
is removed by this fix, replaced with a new method
``visit_frame_clause()``. Third party dialects which may have referred
to this method will need to change the name and revise the approach to
rendering the correct SQL for that dialect.
This patch introduces a new ClauseElement called _FrameClause which
stores the integer range values separately and within cache-compatible
BindParameter objects from the "type" which
can be unbounded, current, preceding, or following, represented by
a _FrameClauseType enum. The negative
sign is also stripped from the integer and represented within the
_FrameClauseType. Tests from #11514 are adapted to include
a test for SQL Server's "literal_execute" flag taking effect so
that literal numeric values aren't stored in the cache.
Fixes: #11515
Change-Id: I8aad368ffef9f06cb5c3f8c4e971fadef029ffd5
Fixed additional issues in the event system triggered by unpickling of a
:class:`.Enum` datatype, continuing from 🎫`11365` and
🎫`11360`, where dynamically generated elements of the event
structure would not be present when unpickling in a new process.
Fixes: #11530
Change-Id: Ie1f2b3453d4891051f8719f6d3f6703302d5a86e
Fixed caching issue where using the :meth:`.TextualSelect.add_cte` method
of the :class:`.TextualSelect` construct would not set a correct cache key
which distinguished between different CTE expressions.
Fixes: #11471
Change-Id: Ia9ce2c8cfd128f0f130aa9b26448dc23d994c324
Fixed bug in :meth:`_sql.FunctionFilter.filter` that would mutate
the existing function in-place. It now behaves like the rest of the
SQLAlchemy API, returning a new instance instead of mutating the
original one.
Fixes: #11426
Change-Id: I46ffebaed82426cfb1623db066686cfb911055a1
Added :paramref:`_schema.Column.insert_default` as an alias of
:paramref:`_schema.Column.default` for compatibility with
func:`_orm.mapped_column`.
Fixes: #11374
Change-Id: I5509b6cbac7b37ac37430a88442b1319cc9c1024
Fixed issue where a :class:`.MetaData` collection would not be
serializable, if an :class:`.Enum` or :class:`.Boolean` datatype were
present which had been adapted. This specific scenario in turn could occur
when using the :class:`.Enum` or :class:`.Boolean` within ORM Annotated
Declarative form where type objects frequently get copied.
Fixes: #11365
Change-Id: Iaaa64baad79c41075d37cf53dade744d79e600a3
### Description
Fixes#11328 by adding an overload to ColumnCollection when a non-None default is provided.
### Checklist
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Closes: #11329
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11329
Pull-request-sha: 32db849e0d
Change-Id: I8bef91c423fb7048ec8d4a7c99f70f0b1588c37a
Fixed issue in cursor handling which affected handling of duplicate
:class:`_sql.Column` or similar objcts in the columns clause of
:func:`_sql.select`, both in combination with arbitary :func:`_sql.text()`
clauses in the SELECT list, as well as when attempting to retrieve
:meth:`_engine.Result.mappings` for the object, which would lead to an
internal error.
Fixes: #11306
Change-Id: I418073b2fdba86b2121b6d00eaa40b1805b69bb8
Ensure the ``PYTHONPATH`` variable is properly initialized when
using ``subprocess.run`` in the tests.
Fixes: #11268
Change-Id: Ie2db656364931b3be9033dcaaf7a7c56b383ecca
Fixed regression from the 1.4 series where the refactor of the
:meth:`_types.TypeEngine.with_variant` method introduced at
:ref:`change_6980` failed to accommodate for the ``.copy()`` method, which
will lose the variant mappings that are set up. This becomes an issue for
the very specific case of a "schema" type, which includes types such as
:class:`.Enum` and :class:`.ARRAY`, when they are then used in the context
of an ORM Declarative mapping with mixins where copying of types comes into
play. The variant mapping is now copied as well.
Fixes: #11176
Change-Id: Icf1a2752f60fce863c87ead8b0fe298b0f3d3766
Made a change to the adjustment made in version 2.0.10 for 🎫`9618`,
which added the behavior of reconciling RETURNING rows from a bulk INSERT
to the parameters that were passed to it. This behavior included a
comparison of already-DB-converted bound parameter values against returned
row values that was not always "symmetrical" for SQL column types such as
UUIDs, depending on specifics of how different DBAPIs receive such values
versus how they return them, necessitating the need for additional
"sentinel value resolver" methods on these column types. Unfortunately
this broke third party column types such as UUID/GUID types in libraries
like SQLModel which did not implement this special method, raising an error
"Can't match sentinel values in result set to parameter sets". Rather than
attempt to further explain and document this implementation detail of the
"insertmanyvalues" feature including a public version of the new
method, the approach is intead revised to no longer need this extra
conversion step, and the logic that does the comparison now works on the
pre-converted bound parameter value compared to the post-result-processed
value, which should always be of a matching datatype. In the unusual case
that a custom SQL column type that also happens to be used in a "sentinel"
column for bulk INSERT is not receiving and returning the same value type,
the "Can't match" error will be raised, however the mitigation is
straightforward in that the same Python datatype should be passed as that
returned.
Fixes: #11160
Change-Id: Ica62571e923ad9545eb90502e6732b11875b164a
Fixed issue in :ref:`engine_insertmanyvalues` feature where using a primary
key column with an "inline execute" default generator such as an explicit
:class:`.Sequence` with an explcit schema name, while at the same time
using the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
feature would fail to render the sequence or the parameters properly,
leading to errors.
Fixes: #11157
Change-Id: I35666af46d40996aff35d3d39f48c150d838e6e4
Fixed issue in ORM annotated declarative where using
:func:`_orm.mapped_column()` with an :paramref:`_orm.mapped_column.index`
or :paramref:`_orm.mapped_column.unique` setting of False would be
overridden by an incoming ``Annotated`` element that featured that
parameter set to ``True``, even though the immediate
:func:`_orm.mapped_column()` element is more specific and should take
precedence. The logic to reconcile the booleans has been enhanced to
accommodate a local value of ``False`` as still taking precedence over an
incoming ``True`` value from the annotated element.
Fixes: #11091
Change-Id: I15cda4a0a07a289015c0a09bbe3ca2849956604e
Adjusted the fix made in 🎫`10570`, released in 2.0.23, where new
logic was added to reconcile possibly changing bound parameter values
across cache key generations used within the :func:`_orm.with_expression`
construct. The new logic changes the approach by which the new bound
parameter values are associated with the statement, avoiding the need to
deep-copy the statement which can result in a significant performance
penalty for very deep / complex SQL constructs. The new approach no longer
requires this deep-copy step.
Fixes: #11085
Change-Id: Ia51eb4e949c8f37af135399925a9916b9ed4ad2f
Added new core execution option
paramref:`_engine.Connection.execution_options.preserve_rowcount`
to unconditionally save the ``rowcount`` attribute from the cursor in the
class:`_engine.Result` returned from an execution, regardless of the
statement being executed.
When this option is provided the correct value is also set when
an INSERT makes use of the "insertmanyvalues" mode, that may use
more than one actualy cursor execution.
Fixes: #10974
Change-Id: Icecef6b7539be9f0a1a02b9539864f5f163dcfbc
manually update the files to remove literal string concat on the same line,
since black does not seem to be making progress in handling these
Change-Id: I3c651374c5f3db5b8bc0c700328d67ca03743b7b
Fixed an issue regarding the use of the :class:`.Uuid` datatype with the
:paramref:`.Uuid.as_uuid` parameter set to False, when using the pymssql
dialect. ORM-optimized INSERT statements (e.g. the "insertmanyvalues"
feature) would not correctly align primary key UUID values for bulk INSERT
statements, resulting in errors.
This change also adds a small degree of generalization to the
Uuid datatype by adding the native/non-native compilation conditional
to the base compiler.
Patch is originally part of Ib920871102b9b64f2cba9697f5cb72b6263e4ed8
which is implementing native UUID for mariadb in 2.1 only.
Change-Id: I96cbec5c0ece312b345206aa5a5db2ffcf732d41
The :class:`.Row` object now no longer makes use of an intermediary
``Tuple`` in order to represent its individual element types; instead,
the individual element types are present directly, via new :pep:`646`
integration, now available in more recent versions of Mypy. Mypy
1.7 or greater is now required for statements, results and rows
to be correctly typed. Pull request courtesy Yurii Karabas.
Fixes: #10635Closes: #10634
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10634
Pull-request-sha: 430785c8a0
Change-Id: Ibd0ae31a98b4ea69dcb89f970e640920b2be6c48
Fixed issues in :func:`_sql.case` where the logic for determining the
type of the expression could result in :class:`.NullType` if the last
element in the "whens" had no type, or in other cases where the type
could resolve to ``None``. The logic has been updated to scan all
given expressions so that the first non-null type is used, as well as
to always ensure a type is present. Pull request courtesy David Evans.
updates to test suite to use modern fixture patterns by Mike
Fixes: #10843Closes: #10847
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10847
Pull-request-sha: 4fd5c39ab5
Change-Id: I40f905ac336a8a42b617ff9473dbd9c22ac57505
Improved compilation of :func:`_sql.any_` / :func:`_sql.all_` in the
context of a negation of boolean comparison, will now render ``NOT (expr)``
rather than reversing the equality operator to not equals, allowing
finer-grained control of negations for these non-typical operators.
Fixes: #10817
Change-Id: If0b324b1220ad3c7f053af91e8a61c81015f312a
Added support for :ref:`oracledb` in async mode.
The current implementation has some limitation, preventing
the support for :meth:`_asyncio.AsyncConnection.stream`.
Improved support if planned for the 2.1 release of SQLAlchemy.
Fixes: #10679
Change-Id: Iff123cf6241bcfa0fbac57529b80f933951be0a7
Fixed issue in stringify for SQL elements, where a specific dialect is not
passed, where a dialect-specific element such as the PostgreSQL "on
conflict do update" construct is encountered and then fails to provide for
a stringify dialect with the appropriate state to render the construct,
leading to internal errors.
Fixed issue where stringifying or compiling a :class:`.CTE` that was
against a DML construct such as an :func:`_sql.insert` construct would fail
to stringify, due to a mis-detection that the statement overall is an
INSERT, leading to internal errors.
Fixes: #10753
Change-Id: I783eca3fc7bbc1794fedd325d58181dbcc7e0b75
Deprecate Oracle only parameters :paramref:`_schema.Sequence.order`,
paramref:`_schema.Identity.order` and :paramref:`_schema.Identity.on_null`.
They should be configured using the dialect kwargs ``oracle_order`` and
oracle_on_null``.
Fixes: #10247
Change-Id: I124a16c9a482745e6f15669008968284fc435998
I updated flake8 in pre-commit a few days ago but forgot to do it in
tox.
this flake seems to be picking inside of f-strings, which black does
not fix, so fix these manually.
Change-Id: I9a641a99e280fbba9d893a6f1f051b5039d5d4eb
Fixed issue where error reporting for unexpected schema item when creating
objects like :class:`_schema.Table` would incorrectly handle an argument
that was itself passed as a tuple, leading to a formatting error. The
error message has been modernized to use f-strings.
this change necessitated an update to flake8 as version 5 was
mis-interpreting f-strings that had semicolons in them.
Black is also unable to format some of these f-strings which had
to be broken out, unclear if there is a newer Black available.
Fixes: #10654
Change-Id: I703e94282c27ccf06f4aa315e8a11bd97b719170
Fixed issue where use of :func:`_orm.foreign` annotation on a
non-initialized :func:`_orm.mapped_column` construct would produce an
expression without a type, which was then not updated at initialization
time of the actual column, leading to issues such as relationships not
determining ``use_get`` appropriately.
Fixes: #10597
Change-Id: I8339ba715ec6bd1f50888f8a424c3ac156e2364f
Fixed issue where using the same bound parameter more than once with
``literal_execute=True`` in some combinations with other literal rendering
parameters would cause the wrong values to render due to an iteration
issue.
Fixes: #10142
Change-Id: Idde314006568e3445558f0104aed9d2f4af72b56
the expression clauselist feature added in #7744 failed to accommodate
this parameter that is used only by the PostgreSQL JSON
operators.
Fixed 2.0 regression caused by 🎫`7744` where chains of expressions
involving PostgreSQL JSON operators combined with other operators such as
string concatenation would lose correct parenthesization, due to an
implementation detail specific to the PostgreSQL dialect.
Fixes: #10479
Change-Id: Ic168bf6afd8bf1cfa648f2bad22fdd7254feaa34
Fixed regression in recently revised "insertmanyvalues" feature (likely
issue 🎫`9618`) where the ORM would inadvertently attempt to
interpret a non-RETURNING result as one with RETURNING, in the case where
the ``implicit_returning=False`` parameter were applied to the mapped
:class:`.Table`, indicating that "insertmanyvalues" cannot be used if the
primary key values are not provided.
This includes a refinement to insertmanyvalues where we consider
return_defaults() with supplemental_cols to be the same as an explicit
returning(), since that's the purpose of this. This saves us some
extra exceptions that could be thrown per-dialect if implicit_returning
were set to False in some cases.
Fixed issue within some dialects where the dialect could incorrectly return
an empty result set for an INSERT statement that does not actually return
rows at all, due to artfacts from pre- or post-fetching the primary key of
the row or rows still being present. Affected dialects included asyncpg,
all mssql dialects.
Fixes: #10453
Change-Id: Ie2e7e4f4cd9180558f9da315d21895347ec6d4f7
Mike Bayer