Improved compilation of :func:`_sql.any_` / :func:`_sql.all_` in the
context of a negation of boolean comparison, will now render ``NOT (expr)``
rather than reversing the equality operator to not equals, allowing
finer-grained control of negations for these non-typical operators.
Fixes: #10817
Change-Id: If0b324b1220ad3c7f053af91e8a61c81015f312a
Added support for :ref:`oracledb` in async mode.
The current implementation has some limitation, preventing
the support for :meth:`_asyncio.AsyncConnection.stream`.
Improved support if planned for the 2.1 release of SQLAlchemy.
Fixes: #10679
Change-Id: Iff123cf6241bcfa0fbac57529b80f933951be0a7
Fixed issue in stringify for SQL elements, where a specific dialect is not
passed, where a dialect-specific element such as the PostgreSQL "on
conflict do update" construct is encountered and then fails to provide for
a stringify dialect with the appropriate state to render the construct,
leading to internal errors.
Fixed issue where stringifying or compiling a :class:`.CTE` that was
against a DML construct such as an :func:`_sql.insert` construct would fail
to stringify, due to a mis-detection that the statement overall is an
INSERT, leading to internal errors.
Fixes: #10753
Change-Id: I783eca3fc7bbc1794fedd325d58181dbcc7e0b75
Deprecate Oracle only parameters :paramref:`_schema.Sequence.order`,
paramref:`_schema.Identity.order` and :paramref:`_schema.Identity.on_null`.
They should be configured using the dialect kwargs ``oracle_order`` and
oracle_on_null``.
Fixes: #10247
Change-Id: I124a16c9a482745e6f15669008968284fc435998
I updated flake8 in pre-commit a few days ago but forgot to do it in
tox.
this flake seems to be picking inside of f-strings, which black does
not fix, so fix these manually.
Change-Id: I9a641a99e280fbba9d893a6f1f051b5039d5d4eb
Fixed issue where error reporting for unexpected schema item when creating
objects like :class:`_schema.Table` would incorrectly handle an argument
that was itself passed as a tuple, leading to a formatting error. The
error message has been modernized to use f-strings.
this change necessitated an update to flake8 as version 5 was
mis-interpreting f-strings that had semicolons in them.
Black is also unable to format some of these f-strings which had
to be broken out, unclear if there is a newer Black available.
Fixes: #10654
Change-Id: I703e94282c27ccf06f4aa315e8a11bd97b719170
Fixed issue where use of :func:`_orm.foreign` annotation on a
non-initialized :func:`_orm.mapped_column` construct would produce an
expression without a type, which was then not updated at initialization
time of the actual column, leading to issues such as relationships not
determining ``use_get`` appropriately.
Fixes: #10597
Change-Id: I8339ba715ec6bd1f50888f8a424c3ac156e2364f
Fixed issue where using the same bound parameter more than once with
``literal_execute=True`` in some combinations with other literal rendering
parameters would cause the wrong values to render due to an iteration
issue.
Fixes: #10142
Change-Id: Idde314006568e3445558f0104aed9d2f4af72b56
the expression clauselist feature added in #7744 failed to accommodate
this parameter that is used only by the PostgreSQL JSON
operators.
Fixed 2.0 regression caused by 🎫`7744` where chains of expressions
involving PostgreSQL JSON operators combined with other operators such as
string concatenation would lose correct parenthesization, due to an
implementation detail specific to the PostgreSQL dialect.
Fixes: #10479
Change-Id: Ic168bf6afd8bf1cfa648f2bad22fdd7254feaa34
Fixed regression in recently revised "insertmanyvalues" feature (likely
issue 🎫`9618`) where the ORM would inadvertently attempt to
interpret a non-RETURNING result as one with RETURNING, in the case where
the ``implicit_returning=False`` parameter were applied to the mapped
:class:`.Table`, indicating that "insertmanyvalues" cannot be used if the
primary key values are not provided.
This includes a refinement to insertmanyvalues where we consider
return_defaults() with supplemental_cols to be the same as an explicit
returning(), since that's the purpose of this. This saves us some
extra exceptions that could be thrown per-dialect if implicit_returning
were set to False in some cases.
Fixed issue within some dialects where the dialect could incorrectly return
an empty result set for an INSERT statement that does not actually return
rows at all, due to artfacts from pre- or post-fetching the primary key of
the row or rows still being present. Affected dialects included asyncpg,
all mssql dialects.
Fixes: #10453
Change-Id: Ie2e7e4f4cd9180558f9da315d21895347ec6d4f7
Fixed issue where referring to a FROM entry in the SET clause of an UPDATE
statement would not include it in the FROM clause of the UPDATE statement,
if that entry were nowhere else in the statement; this occurs currently for
CTEs that were added using :meth:`.Update.add_cte` to provide the desired
CTE at the top of the statement.
Fixes: #10408
Change-Id: I6e3c6ca7a00cc884bda7e0f24c62c34c75134e5b
Fixed a wide range of :func:`_orm.mapped_column` parameters that were not
being transferred when using the :func:`_orm.mapped_column` object inside
of a pep-593 ``Annotated`` object, including
:paramref:`_orm.mapped_column.sort_order`,
:paramref:`_orm.mapped_column.deferred`,
:paramref:`_orm.mapped_column.autoincrement`,
:paramref:`_orm.mapped_column.system`, :paramref:`_orm.mapped_column.info`
etc.
Additionally, it remains not supported to have dataclass arguments, such as
:paramref:`_orm.mapped_column.kw_only`,
:paramref:`_orm.mapped_column.default_factory` etc. indicated within the
:func:`_orm.mapped_column` received by ``Annotated``, as this is not
supported with pep-681 Dataclass Transforms. A warning is now emitted when
these parameters are used within ``Annotated`` in this way (and they
continue to be ignored).
Fixes: #10369Fixes: #10046
Change-Id: Ibcfb287cba0e764db0ae15fab8049bbb9f94dd1b
Fixed regression introduced in 2.0.20 via 🎫`9600` fix which
attempted to add more formal typing to
:paramref:`_schema.MetaData.naming_convention`. This change prevented basic
naming convention dictionaries from passing typing and has been adjusted so
that a plain dictionary of strings for keys as well as dictionaries that
use constraint types as keys or a mix of both, are again accepted.
As part of this change, lesser used forms of the naming convention
dictionary are also typed, including that it currently allows for
``Constraint`` type objects as keys as well.
Fixes: #9284Fixes: #10264
Change-Id: Ic6561dd65058e4de3a7a393295b9863fc065db13
The :class:`.Values` construct will now automatically create a proxy (i.e.
a copy) of a :class:`_sql.column` if the column were already associated
with an existing FROM clause. This allows that an expression like
``values_obj.c.colname`` will produce the correct FROM clause even in the
case that ``colname`` was passed as a :class:`_sql.column` that was already
used with a previous :class:`.Values` or other table construct.
Originally this was considered to be a candidate for an error condition,
however it's likely this pattern is already in widespread use so it's
now added to support.
* adjust unrelated dml test recently added for update..returning *
case to not rely upon ordering
Fixes: #10280
Change-Id: I6e60e5b7cb7abd6a7bbd4722970ebf025596ab9c
Adjusted the :class:`_types.Enum` datatype to accept an argument of
``None`` for the :paramref:`_types.Enum.length` parameter, resulting in a
VARCHAR or other textual type with no length in the resulting DDL. This
allows for new elements of any length to be added to the type after it
exists in the schema. Pull request courtesy Eugene Toder.
Fixes: #10269Closes: #10274
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10274
Pull-request-sha: 651afaaea7
Change-Id: I374ae9e5fa63da21a4cc87e323ce1a54acd6e39b
Adjusted the operator precedence for the string concatenation operator to
be equal to that of string matching operators, such as
:meth:`.ColumnElement.like`, :meth:`.ColumnElement.regexp_match`,
:meth:`.ColumnElement.match`, etc., as well as plain ``==`` which has the
same precedence as string comparison operators, so that parenthesis will be
applied to a string concatenation expression that follows a string match
operator. This provides for backends such as PostgreSQL where the "regexp
match" operator is apparently of higher precedence than the string
concatenation operator.
Fixes: #9610
Change-Id: I73640e40e445375177340e1ed8f45b5da98d6dfb
Fixes the rendering of the Oracle only ``order`` attribute in
Sequence and Identity that was passed also when rendering
the DDL in PostgreSQL.
Fixes: #10207
Change-Id: I5b918eab38ba68fa10a213a79e2bd0cc48401a02
Fixed issue where unpickling of a :class:`_schema.Column` or other
:class:`_sql.ColumnElement` would fail to restore the correct "comparator"
object, which is used to generate SQL expressions specific to the type
object.
Fixes: #10213
Change-Id: I74e805024bcc0d93d549bd94757c2865b3117d72
Fixed issue where internal cloning used by the ORM for expressions like
:meth:`_orm.relationship.Comparator.any` to produce correlated EXISTS
constructs would interfere with the "cartesian product warning" feature of
the SQL compiler, leading the SQL compiler to warn when all elements of the
statement were correctly joined.
Fixes: #10124
Change-Id: I31c1ba538e2b943278e8cc0b7fddc107968a0826
Renamed :attr:`_result.Row.t` and :meth:`_result.Row.tuple` to
:attr:`_result.Row._t` and :meth:`_result.Row._tuple`; this is to suit the
policy that all methods and pre-defined attributes on :class:`.Row` should
be in the style of Python standard library ``namedtuple`` where all fixed
names have a leading underscore, to avoid name conflicts with existing
column names. The previous method/attribute is now deprecated and will
emit a deprecation warning.
Fixes: #10093
Change-Id: Ibb0e9423b55590c9dda8ee554187abd9a9122590
Fixed issue where the :meth:`_sql.ColumnOperators.regexp_match`
when using "flags" would not produce a "stable" cache key, that
is, the cache key would keep changing each time causing cache pollution.
The same issue existed for :meth:`_sql.ColumnOperators.regexp_replace`
with both the flags and the actual replacement expression.
The flags are now represented as fixed modifier strings rendered as
safestrings rather than bound parameters, and the replacement
expression is established within the primary portion of the "binary"
element so that it generates an appropriate cache key.
Note that as part of this change, the
:paramref:`_sql.ColumnOperators.regexp_match.flags` and
:paramref:`_sql.ColumnOperators.regexp_replace.flags` have been modified to
render as literal strings only, whereas previously they were rendered as
full SQL expressions, typically bound parameters. These parameters should
always be passed as plain Python strings and not as SQL expression
constructs; it's not expected that SQL expression constructs were used in
practice for this parameter, so this is a backwards-incompatible change.
The change also modifies the internal structure of the expression
generated, for :meth:`_sql.ColumnOperators.regexp_replace` with or without
flags, and for :meth:`_sql.ColumnOperators.regexp_match` with flags. Third
party dialects which may have implemented regexp implementations of their
own (no such dialects could be located in a search, so impact is expected
to be low) would need to adjust the traversal of the structure to
accommodate.
Fixed issue in mostly-internal :class:`.CacheKey` construct where the
``__ne__()`` operator were not properly implemented, leading to nonsensical
results when comparing :class:`.CacheKey` instances to each other.
Fixes: #10042
Change-Id: I2e245f81d7ee7136ad04cf77be35f9745c5da5e5
Fixed issue where the :paramref:`.ColumnOperators.like.escape` and similar
parameters did not allow an empty string as an argument that would be
passed through as the "escape" character; this is a supported syntax by
PostgreSQL. Pull requset courtesy Martin Caslavsky.
Fixes: #9907Closes: #9908
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9908
Pull-request-sha: d7ecc1778a
Change-Id: I39adb765a1b9650fe891883ed0973df66adc4e81
Generalized the MSSQL :func:`_sql.try_cast` function into the
``sqlalchemy.`` import namespace so that it may be implemented by third
party dialects as well. Within SQLAlchemy, the :func:`_sql.try_cast`
function remains a SQL Server-only construct that will raise
:class:`.CompileError` if used with backends that don't support it.
:func:`_sql.try_cast` implements a CAST where un-castable conversions are
returned as NULL, instead of raising an error. Theoretically, the construct
could be implemented by third party dialects for Google BigQuery, DuckDB,
and Snowflake, and possibly others.
Pull request courtesy Nick Crews.
Fixes: #9752Closes: #9753
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9753
Pull-request-sha: 2e81b8d2c9
Change-Id: Ib57999b5947a2e34d5d305e294ff99dc08b01111
Fixed issue in :func:`_sql.values` construct where an internal compilation
error would occur if the construct were used inside of a scalar subquery.
Fixes: #9772
Change-Id: I4b0f756977abafbd2aabaaa0064baa875249ebe1
fix a handful of warnings that were emitting but not raising,
usually because they were inside an "expect_warnings" block.
modify "expect_warnings" to always use "raise_on_any_unexpected"
behavior; remove this parameter.
Fixed issue in semi-private ``await_only()`` and ``await_fallback()``
concurrency functions where the given awaitable would remain un-awaited if
the function threw a ``GreenletError``, which could cause "was not awaited"
warnings later on if the program continued. In this case, the given
awaitable is now cancelled before the exception is thrown.
Change-Id: I33668c5e8c670454a3d879e559096fb873b57244
Implemented the "cartesian product warning" for UPDATE and DELETE
statements, those which include multiple tables that are not correlated
together in some way.
Fixed issue where :func:`_dml.update` construct that included multiple
tables and no VALUES clause would raise with an internal error. Current
behavior for :class:`_dml.Update` with no values is to generate a SQL
UPDATE statement with an empty "set" clause, so this has been made
consistent for this specific sub-case.
Fixes: #9721
Change-Id: I556639811cc930d2e37532965d2ae751882af921
Fixed 2.0 regression where use of :func:`_sql.bindparam()` inside of
:meth:`_dml.Insert.values` would fail to be interpreted correctly when
executing the :class:`_dml.Insert` statement using the ORM
:class:`_orm.Session`, due to the new ORM-enabled insert feature not
implementing this use case.
In addition, the bulk INSERT and UPDATE features now add these
capabilities:
* The requirement that extra parameters aren't passed when using ORM
INSERT using the "orm" dml_strategy setting is lifted.
* The requirement that additional WHERE criteria is not passed when using
ORM UPDATE using the "bulk" dml_strategy setting is lifted. Note that
in this case, the check for expected row count is turned off.
Fixes: #9583
Change-Id: I539c18893b697caeab5a5f0195a27d4f0487e728
Improved :class:`_engine.Row` implementation to optimize
``__getattr__`` performance.
The serialization of a :class:`_engine.Row` to pickle has changed with
this change. Pickle saved by older SQLAlchemy versions can still be loaded,
but new pickle saved by this version cannot be loaded by older ones.
Fixes: #9678Closes: #9668
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9668
Pull-request-sha: 86b8ccd195
Change-Id: Ia85c26a59e1a57ba2bf0d65578c6168f82a559f2
Added support for slice access with :class:`.ColumnCollection`, e.g.
``table.c[0:5]``, ``subquery.c[:-1]`` etc. Slice access returns a sub
:class:`.ColumnCollection` in the same way as passing a tuple of keys. This
is a natural continuation of the key-tuple access added for 🎫`8285`,
which it appears to be an oversight that this usage was omitted.
Change-Id: I6378642f39501ffbbae4acadf1dc38a43c39d722
References: #8285
References: #9690
Repaired a major shortcoming which was identified in the
:ref:`engine_insertmanyvalues` performance optimization feature first
introduced in the 2.0 series. This was a continuation of the change in
2.0.9 which disabled the SQL Server version of the feature due to a
reliance in the ORM on apparent row ordering that is not guaranteed to take
place. The fix applies new logic to all "insertmanyvalues" operations,
which takes effect when a new parameter
:paramref:`_dml.Insert.returning.sort_by_parameter_order` on the
:meth:`_dml.Insert.returning` or :meth:`_dml.UpdateBase.return_defaults`
methods, that through a combination of alternate SQL forms, direct
correspondence of client side parameters, and in some cases downgrading to
running row-at-a-time, will apply sorting to each batch of returned rows
using correspondence to primary key or other unique values in each row
which can be correlated to the input data.
Performance impact is expected to be minimal as nearly all common primary
key scenarios are suitable for parameter-ordered batching to be
achieved for all backends other than SQLite, while "row-at-a-time"
mode operates with a bare minimum of Python overhead compared to the very
heavyweight approaches used in the 1.x series. For SQLite, there is no
difference in performance when "row-at-a-time" mode is used.
It's anticipated that with an efficient "row-at-a-time" INSERT with
RETURNING batching capability, the "insertmanyvalues" feature can be later
be more easily generalized to third party backends that include RETURNING
support but not necessarily easy ways to guarantee a correspondence
with parameter order.
Fixes: #9618
References: #9603
Change-Id: I1d79353f5f19638f752936ba1c35e4dc235a8b7c
### Description
I ran into this originally in sqlalchemy2-stubs: https://github.com/sqlalchemy/sqlalchemy2-stubs/pull/251, where `RowMapping` only supported string keys according to the type hints. I ran into a similar issue here upgrading our application where because `RowMapping` subclassed `Mapping[str, Any]`, `Row._mapping.get()` would fail to typecheck when used with `Column` objects.
This patch adds a test to verify that `Row._mapping.get()` continues to work with both strings and `Column`s, though it doesn't look like mypy checks types in the tests.
Fixes#9644.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Closes: #9643
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9643
Pull-request-sha: 6c33fe534c
Change-Id: I1009c6defff109d73f13a9e8c51641009e6a79e2
Fixed issue where the :meth:`_sql.BindParameter.render_literal_execute`
method would fail when called on a parameter that also had ORM annotations
associated with it. In practice, this would be observed as a failure of SQL
compilation when using some combinations of a dialect that uses "FETCH
FIRST" such as Oracle along with a :class:`_sql.Select` construct that uses
:meth:`_sql.Select.limit`, within some ORM contexts, including if the
statement were embedded within a relationship primaryjoin expression.
Fixes: #9526
Change-Id: I2f512b6760a90293d274e60b06a891f10b276ecc
Fixed critical regression in PostgreSQL dialects such as asyncpg which rely
upon explicit casts in SQL in order for datatypes to be passed to the
driver correctly, where a :class:`.String` datatype would be cast along
with the exact column length being compared, leading to implicit truncation
when comparing a ``VARCHAR`` of a smaller length to a string of greater
length regardless of operator in use (e.g. LIKE, MATCH, etc.). The
PostgreSQL dialect now omits the length from ``VARCHAR`` when rendering
these casts.
Fixes: #9511
Change-Id: If094146d8cfd989a0b780872f38e86fd41ebfec2
Fixed critical SQL caching issue where use of the :meth:`_sql.Operators.op`
custom operator function would not produce an appropriate cache key,
leading to reduce the effectiveness of the SQL cache.
Fixes: #9506
Change-Id: I3eab1ddb5e09a811ad717161a59df0884cdf70ed
in I4e0b627bfa187f1780dc68ec81b94db1c78f846a the 1.4 version has more
changes than the main version, which failed to get the entire change,
yet the whole thing was merged. Restore the missing mutex related
code to the main version.
Fixed regression where the fix for 🎫`8098`, which was released in
the 1.4 series and provided a layer of concurrency-safe checks for the
lambda SQL API, included additional fixes in the patch that failed to be
applied to the main branch. These additional fixes have been applied.
Change-Id: Id172e09c421dafa6ef1d40b383aa4371de343864
References: #8098Fixes: #9461
Fixed regression involving pickling of Python rows between the cython and
pure Python implementations of :class:`.Row`, which occurred as part of
refactoring code for version 2.0 with typing. A particular constant were
turned into a string based ``Enum`` for the pure Python version of
:class:`.Row` whereas the cython version continued to use an integer
constant, leading to deserialization failures.
Regression occurred in a4bb502cf9Fixes: #9423
Change-Id: Icbd85cacb2d589cef7c246de7064249926146f2e
Fixed regression where the :func:`_sql.select` construct would not be able
to render if it were given no columns and then used in the context of an
EXISTS, raising an internal exception instead. While an empty "SELECT" is
not typically valid SQL, in the context of EXISTS databases such as
PostgreSQL allow it, and in any case the condition now no longer raises
an internal exception.
For this case, also add an extra whitespace trim step for the unusual
case that there are no columns to render. This is done in such a
way as to not interfere with other test cases that are involving
custom compilation schemes.
Fixes: #9440
Change-Id: If65ba9ce15d371f09b4342ad0669143b7b082a78
if .post1 fails to work out, we'll just do 2.0.6
The test_pickle_rows_other_process test is failing during wheel
builds as it seems that the "subprocess" run is not using the
cython extensions, leading to a pickle mismatch between the
cythonized and the pure python version of a row. comment
out this test and attempt to release as 2.0.5.post1 so that
wheels can build.
Fixes: #9429
Change-Id: I6e1e9f2b9c4ef8fa67a88ff86ebdacbeb02b90df
Michael Bayer