the :class:`.Numeric` and :class:`.Float` SQL types have been separated out
so that :class:`.Float` no longer inherits from :class:`.Numeric`; instead,
they both extend from a common mixin :class:`.NumericCommon`. This
corrects for some architectural shortcomings where numeric and float types
are typically separate, and establishes more consistency with
:class:`.Integer` also being a distinct type. The change should not have
any end-user implications except for code that may be using
``isinstance()`` to test for the :class:`.Numeric` datatype; third party
dialects which rely upon specific implementation types for numeric and/or
float may also require adjustment to maintain compatibility.
Fixes: #5252
Change-Id: Iadc841340b3d97e3eb5f7e63f0a0cc3cb4e30f74
The ``.c`` and ``.columns`` attributes on the :class:`.Select` and
:class:`.TextualSelect` constructs, which are not instances of
:class:`.FromClause`, have been removed completely, in addition to the
``.select()`` method as well as other codepaths which would implicitly
generate a subquery from a :class:`.Select` without the need to explicitly
call the :meth:`.Select.subquery` method.
In the case of ``.c`` and ``.columns``, these attributes were never useful
in practice and have caused a great deal of confusion, hence were
deprecated back in version 1.4, and have emitted warnings since that
version. Accessing the columns that are specific to a :class:`.Select`
construct is done via the :attr:`.Select.selected_columns` attribute, which
was added in version 1.4 to suit the use case that users often expected
``.c`` to accomplish. In the larger sense, implicit production of
subqueries works against SQLAlchemy's modern practice of making SQL
structure as explicit as possible.
Note that this is **not related** to the usual :attr:`.FromClause.c` and
:attr:`.FromClause.columns` attributes, common to objects such as
:class:`.Table` and :class:`.Subquery`, which are unaffected by this
change.
Fixes: #10236
Change-Id: If241b8674ccacce7e860bfed25b5d266bfe1aca7
Fixed issue in "lambda SQL" feature where the tracking of bound parameters
could be corrupted if the same lambda were evaluated across multiple
compile phases, including when using the same lambda across multiple engine
instances or with statement caching disabled.
Fixes: #12084
Change-Id: I327aa93ce7feb2326a22113164bd834b96b6b889
The :class:`.Float` and :class:`.Numeric` types are no longer automatically
considered as auto-incrementing columns when the
:paramref:`_schema.Column.autoincrement` parameter is left at its default
of ``"auto"`` on a :class:`_schema.Column` that is part of the primary key.
When the parameter is set to ``True``, a :class:`.Numeric` type will be
accepted as an auto-incrementing datatype for primary key columns, but only
if its scale is explicitly given as zero; otherwise, an error is raised.
This is a change from 2.0 where all numeric types including floats were
automatically considered as "autoincrement" for primary key columns.
Fixes: #11811
Change-Id: Icdfe084d425166199d6647335c5b53ea5b4b416e
Fixed regression caused by an internal code change in response to recent
Mypy releases that caused the very unusual case of a list of ORM-mapped
attribute expressions passed to :meth:`.ColumnOperators.in_` to no longer
be accepted.
in this commit we had to revisit d8dd28c42e where mypy typing
didn't accept ColumnOperartors. the type here is the _HasClauseElement[_T]
protocol which means we need to use a duck type for a runtime check.
Fixes: #12019
Change-Id: Ib378e9cb8defb49d5ac4d726ec93d6bdc581b6a9
Fixed regression from 1.4 where some datatypes such as those derived from
:class:`.TypeDecorator` could not be pickled when they were part of a
larger SQL expression composition due to internal supporting structures
themselves not being pickleable.
Fixes: #12002
Change-Id: I016e37b0c62071413f24c9aac35f6ecf475becaa
Datatypes that are binary based such as :class:`.VARBINARY` will resolve to
:class:`.LargeBinary` when the :meth:`.TypeEngine.as_generic()` method is
called.
Fixes: #11978
Change-Id: I2e0586324fb0f1c367da61f0074b35c96fbe2fd0
Fixed bug where the ``metadata`` element of an ``Enum`` datatype would not
be transferred to the new :class:`.MetaData` object when the type had been
copied via a :meth:`.Table.to_metadata` operation, leading to inconsistent
behaviors within create/drop sequences.
Fixes: #11802
Change-Id: Ibbc93aa31bdfde0d67a9530f41a08e826c17d58e
Fixed regression in :meth:`_sql.Select.with_statement_hint` and others
where the generative behavior of the method stopped producing a copy of the
object.
Fixes: #11703
Change-Id: Ia4482f91f76fae9982dc6b075bf5cfec7042ffa6
Follow up of 🎫`11471` to fix caching issue where using the
:meth:`.CompoundSelectState.add_cte` method of the
:class:`.CompoundSelectState` construct would not set a correct cache key
which distinguished between different CTE expressions. Also added tests
that would detect issues similar to the one fixed in 🎫`11544`.
Fixes: #11471
Change-Id: Iae6a91077c987d83cd70ea826daff42855491330
Fixed bug where the :meth:`.Operators.nulls_first()` and
:meth:`.Operators.nulls_last()` modifiers would not be treated the same way
as :meth:`.Operators.desc()` and :meth:`.Operators.asc()` when determining
if an ORDER BY should be against a label name already in the statement. All
four modifiers are now treated the same within ORDER BY.
Fixes: #11592
Change-Id: I1de1aff679c56af1abfdfd07f9bcbc45ecc5a8cc
Fixed issue where the :func:`_sql.collate` construct, which explicitly sets
a collation for a given expression, would maintain collation settings for
the underlying type object from the expression, causing SQL expressions to
have both collations stated at once when used in further expressions for
specific dialects that render explicit type casts, such as that of asyncpg.
The :func:`_sql.collate` construct now assigns its own type to explicitly
include the new collation, assuming it's a string type.
Fixes: #11576
Change-Id: I6fc8904d2bcbc21f11bbca57e4a451ed0edbd879
The :paramref:`_orm.aliased.name` parameter to :func:`_orm.aliased` may now
be combined with the :paramref:`_orm.aliased.flat` parameter, producing
per-table names based on a name-prefixed naming convention. Pull request
courtesy Eric Atkin.
Fixes: #11575Closes: #11531
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11531
Pull-request-sha: f85535464b
Change-Id: If79679c7a9598fffe99c033894b7dffecef13939
Added new execution option
:paramref:`_engine.Connection.execution_options.driver_column_names`. This
option disables the "name normalize" step that takes place against the
DBAPI ``cursor.description`` for uppercase-default backends like Oracle,
and will cause the keys of a result set (e.g. named tuple names, dictionary
keys in :attr:`.Row._mapping`, etc.) to be exactly what was delivered in
cursor.description. This is mostly useful for plain textual statements
using :func:`_sql.text` or :meth:`_engine.Connection.exec_driver_sql`.
Fixes: #10789
Change-Id: Ib647b25bb53492fa839af04dd032d9f061e630af
Fixed issue in "insertmanyvalues" feature where a particular call to
``cursor.fetchall()`` were not wrapped in SQLAlchemy's exception wrapper,
which apparently can raise a database exception during fetch when using
pyodbc.
Fixes: #11532
Change-Id: Ic07d3e79dd597e18d87a56b45ddffa25e762beb9
Fixed caching issue where the
:paramref:`_sql.Select.with_for_update.key_share` element of
:meth:`_sql.Select.with_for_update` was not considered as part of the cache
key, leading to incorrect caching if different variations of this parameter
were used with an otherwise identical statement.
Also repairs a traversal issue where the ``of`` element of
``ForUpdateArg`` when set to ``None`` cannot be compared against a
non-None element because the traversal defines it as a clauselist.
Traversal in this case is adjusted to accommodate for this case so that
we dont need to create a risky-to-backport change to ``ForUpdateArg``
itself.
Fixes: #11544
Change-Id: Ie8a50716df06977af58b0c22a8c10e1b64d972b9
Enhanced the caching structure of the :paramref:`.over.rows` and
:paramref:`.over.range` so that different numerical values for the rows /
range fields are cached on the same cache key, to the extent that the
underlying SQL does not actually change (i.e. "unbounded", "current row",
negative/positive status will still change the cache key). This prevents
the use of many different numerical range/rows value for a query that is
otherwise identical from filling up the SQL cache.
Note that the semi-private compiler method ``_format_frame_clause()``
is removed by this fix, replaced with a new method
``visit_frame_clause()``. Third party dialects which may have referred
to this method will need to change the name and revise the approach to
rendering the correct SQL for that dialect.
This patch introduces a new ClauseElement called _FrameClause which
stores the integer range values separately and within cache-compatible
BindParameter objects from the "type" which
can be unbounded, current, preceding, or following, represented by
a _FrameClauseType enum. The negative
sign is also stripped from the integer and represented within the
_FrameClauseType. Tests from #11514 are adapted to include
a test for SQL Server's "literal_execute" flag taking effect so
that literal numeric values aren't stored in the cache.
Fixes: #11515
Change-Id: I8aad368ffef9f06cb5c3f8c4e971fadef029ffd5
Fixed additional issues in the event system triggered by unpickling of a
:class:`.Enum` datatype, continuing from 🎫`11365` and
🎫`11360`, where dynamically generated elements of the event
structure would not be present when unpickling in a new process.
Fixes: #11530
Change-Id: Ie1f2b3453d4891051f8719f6d3f6703302d5a86e
Fixed caching issue where using the :meth:`.TextualSelect.add_cte` method
of the :class:`.TextualSelect` construct would not set a correct cache key
which distinguished between different CTE expressions.
Fixes: #11471
Change-Id: Ia9ce2c8cfd128f0f130aa9b26448dc23d994c324
Fixed bug in :meth:`_sql.FunctionFilter.filter` that would mutate
the existing function in-place. It now behaves like the rest of the
SQLAlchemy API, returning a new instance instead of mutating the
original one.
Fixes: #11426
Change-Id: I46ffebaed82426cfb1623db066686cfb911055a1
Added :paramref:`_schema.Column.insert_default` as an alias of
:paramref:`_schema.Column.default` for compatibility with
func:`_orm.mapped_column`.
Fixes: #11374
Change-Id: I5509b6cbac7b37ac37430a88442b1319cc9c1024
Fixed issue where a :class:`.MetaData` collection would not be
serializable, if an :class:`.Enum` or :class:`.Boolean` datatype were
present which had been adapted. This specific scenario in turn could occur
when using the :class:`.Enum` or :class:`.Boolean` within ORM Annotated
Declarative form where type objects frequently get copied.
Fixes: #11365
Change-Id: Iaaa64baad79c41075d37cf53dade744d79e600a3
### Description
Fixes#11328 by adding an overload to ColumnCollection when a non-None default is provided.
### Checklist
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Closes: #11329
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11329
Pull-request-sha: 32db849e0d
Change-Id: I8bef91c423fb7048ec8d4a7c99f70f0b1588c37a
Fixed issue in cursor handling which affected handling of duplicate
:class:`_sql.Column` or similar objcts in the columns clause of
:func:`_sql.select`, both in combination with arbitary :func:`_sql.text()`
clauses in the SELECT list, as well as when attempting to retrieve
:meth:`_engine.Result.mappings` for the object, which would lead to an
internal error.
Fixes: #11306
Change-Id: I418073b2fdba86b2121b6d00eaa40b1805b69bb8
Ensure the ``PYTHONPATH`` variable is properly initialized when
using ``subprocess.run`` in the tests.
Fixes: #11268
Change-Id: Ie2db656364931b3be9033dcaaf7a7c56b383ecca
Fixed regression from the 1.4 series where the refactor of the
:meth:`_types.TypeEngine.with_variant` method introduced at
:ref:`change_6980` failed to accommodate for the ``.copy()`` method, which
will lose the variant mappings that are set up. This becomes an issue for
the very specific case of a "schema" type, which includes types such as
:class:`.Enum` and :class:`.ARRAY`, when they are then used in the context
of an ORM Declarative mapping with mixins where copying of types comes into
play. The variant mapping is now copied as well.
Fixes: #11176
Change-Id: Icf1a2752f60fce863c87ead8b0fe298b0f3d3766
Made a change to the adjustment made in version 2.0.10 for 🎫`9618`,
which added the behavior of reconciling RETURNING rows from a bulk INSERT
to the parameters that were passed to it. This behavior included a
comparison of already-DB-converted bound parameter values against returned
row values that was not always "symmetrical" for SQL column types such as
UUIDs, depending on specifics of how different DBAPIs receive such values
versus how they return them, necessitating the need for additional
"sentinel value resolver" methods on these column types. Unfortunately
this broke third party column types such as UUID/GUID types in libraries
like SQLModel which did not implement this special method, raising an error
"Can't match sentinel values in result set to parameter sets". Rather than
attempt to further explain and document this implementation detail of the
"insertmanyvalues" feature including a public version of the new
method, the approach is intead revised to no longer need this extra
conversion step, and the logic that does the comparison now works on the
pre-converted bound parameter value compared to the post-result-processed
value, which should always be of a matching datatype. In the unusual case
that a custom SQL column type that also happens to be used in a "sentinel"
column for bulk INSERT is not receiving and returning the same value type,
the "Can't match" error will be raised, however the mitigation is
straightforward in that the same Python datatype should be passed as that
returned.
Fixes: #11160
Change-Id: Ica62571e923ad9545eb90502e6732b11875b164a
Fixed issue in :ref:`engine_insertmanyvalues` feature where using a primary
key column with an "inline execute" default generator such as an explicit
:class:`.Sequence` with an explcit schema name, while at the same time
using the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
feature would fail to render the sequence or the parameters properly,
leading to errors.
Fixes: #11157
Change-Id: I35666af46d40996aff35d3d39f48c150d838e6e4
Fixed issue in ORM annotated declarative where using
:func:`_orm.mapped_column()` with an :paramref:`_orm.mapped_column.index`
or :paramref:`_orm.mapped_column.unique` setting of False would be
overridden by an incoming ``Annotated`` element that featured that
parameter set to ``True``, even though the immediate
:func:`_orm.mapped_column()` element is more specific and should take
precedence. The logic to reconcile the booleans has been enhanced to
accommodate a local value of ``False`` as still taking precedence over an
incoming ``True`` value from the annotated element.
Fixes: #11091
Change-Id: I15cda4a0a07a289015c0a09bbe3ca2849956604e
Adjusted the fix made in 🎫`10570`, released in 2.0.23, where new
logic was added to reconcile possibly changing bound parameter values
across cache key generations used within the :func:`_orm.with_expression`
construct. The new logic changes the approach by which the new bound
parameter values are associated with the statement, avoiding the need to
deep-copy the statement which can result in a significant performance
penalty for very deep / complex SQL constructs. The new approach no longer
requires this deep-copy step.
Fixes: #11085
Change-Id: Ia51eb4e949c8f37af135399925a9916b9ed4ad2f
Added new core execution option
paramref:`_engine.Connection.execution_options.preserve_rowcount`
to unconditionally save the ``rowcount`` attribute from the cursor in the
class:`_engine.Result` returned from an execution, regardless of the
statement being executed.
When this option is provided the correct value is also set when
an INSERT makes use of the "insertmanyvalues" mode, that may use
more than one actualy cursor execution.
Fixes: #10974
Change-Id: Icecef6b7539be9f0a1a02b9539864f5f163dcfbc
manually update the files to remove literal string concat on the same line,
since black does not seem to be making progress in handling these
Change-Id: I3c651374c5f3db5b8bc0c700328d67ca03743b7b
Fixed an issue regarding the use of the :class:`.Uuid` datatype with the
:paramref:`.Uuid.as_uuid` parameter set to False, when using the pymssql
dialect. ORM-optimized INSERT statements (e.g. the "insertmanyvalues"
feature) would not correctly align primary key UUID values for bulk INSERT
statements, resulting in errors.
This change also adds a small degree of generalization to the
Uuid datatype by adding the native/non-native compilation conditional
to the base compiler.
Patch is originally part of Ib920871102b9b64f2cba9697f5cb72b6263e4ed8
which is implementing native UUID for mariadb in 2.1 only.
Change-Id: I96cbec5c0ece312b345206aa5a5db2ffcf732d41
The :class:`.Row` object now no longer makes use of an intermediary
``Tuple`` in order to represent its individual element types; instead,
the individual element types are present directly, via new :pep:`646`
integration, now available in more recent versions of Mypy. Mypy
1.7 or greater is now required for statements, results and rows
to be correctly typed. Pull request courtesy Yurii Karabas.
Fixes: #10635Closes: #10634
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10634
Pull-request-sha: 430785c8a0
Change-Id: Ibd0ae31a98b4ea69dcb89f970e640920b2be6c48
Fixed issues in :func:`_sql.case` where the logic for determining the
type of the expression could result in :class:`.NullType` if the last
element in the "whens" had no type, or in other cases where the type
could resolve to ``None``. The logic has been updated to scan all
given expressions so that the first non-null type is used, as well as
to always ensure a type is present. Pull request courtesy David Evans.
updates to test suite to use modern fixture patterns by Mike
Fixes: #10843Closes: #10847
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10847
Pull-request-sha: 4fd5c39ab5
Change-Id: I40f905ac336a8a42b617ff9473dbd9c22ac57505
Improved compilation of :func:`_sql.any_` / :func:`_sql.all_` in the
context of a negation of boolean comparison, will now render ``NOT (expr)``
rather than reversing the equality operator to not equals, allowing
finer-grained control of negations for these non-typical operators.
Fixes: #10817
Change-Id: If0b324b1220ad3c7f053af91e8a61c81015f312a
Michael Bayer