these seem to date back to 0.9 and revs like #3148 but none
of it seems to be affecting things now, try removing it all
and seeing what fails.
we've noted that _resolve_label does not appear to be
needed, however allow_label_resolve remains relevant both
for non-column elements like text() as well as that it is
used explicitly by the joined eager loader.
Change-Id: Ic8a5d8001ef2a4133360f51a92a6f7b0cc389095
Fixed regression where the use of a :class:`_orm.hybrid_property` attribute
or a mapped :func:`_orm.composite` attribute as a key passed to the
:meth:`_dml.Update.values` method for an ORM-enabled :class:`_dml.Update`
statement, as well as when using it via the legacy
:meth:`_orm.Query.update` method, would be processed for incoming
ORM/hybrid/composite values within the compilation stage of the UPDATE
statement, which meant that in those cases where caching occurred,
subsequent invocations of the same statement would no longer receive the
correct values. This would include not only hybrids that use the
:meth:`_orm.hybrid_property.update_expression` method, but any use of a
plain hybrid attribute as well. For composites, the issue instead caused a
non-repeatable cache key to be generated, which would break caching and
could fill up the statement cache with repeated statements.
The :class:`_dml.Update` construct now handles the processing of key/value
pairs passed to :meth:`_dml.Update.values` and
:meth:`_dml.Update.ordered_values` up front when the construct is first
generated, before the cache key has been generated so that the key/value
pairs are processed each time, and so that the cache key is generated
against the individual column/value pairs that will ultimately be
used in the statement.
Fixes: #7209
Change-Id: I08f248d1d60ea9690b014c21439b775d951fb9e5
Fixed issue where "expanding IN" would fail to function correctly with
datatypes that use the :meth:`_types.TypeEngine.bind_expression` method,
where the method would need to be applied to each element of the
IN expression rather than the overall IN expression itself.
Fixed issue where IN expressions against a series of array elements, as can
be done with PostgreSQL, would fail to function correctly due to multiple
issues within the "expanding IN" feature of SQLAlchemy Core that was
standardized in version 1.4. The psycopg2 dialect now makes use of the
:meth:`_types.TypeEngine.bind_expression` method with :class:`_types.ARRAY`
to portably apply the correct casts to elements. The asyncpg dialect was
not affected by this issue as it applies bind-level casts at the driver
level rather than at the compiler level.
as part of this commit the "bind translate" feature has been
simplified and also applies to the names in the POSTCOMPILE tag to
accommodate for brackets.
Fixes: #7177
Change-Id: I08c703adb0a9bd6f5aeee5de3ff6f03cccdccdc5
Fixed regression where ORM loaded objects could not be pickled in cases
where loader options making use of ``"*"`` were used in certain
combinations, such as combining the :func:`_orm.joinedload` loader strategy
with ``raiseload('*')`` of sub-elements.
Fixes: #7134
Fixed issue where SQL queries using the
:meth:`_functions.FunctionElement.within_group` construct could not be
pickled, typically when using the ``sqlalchemy.ext.serializer`` extension
but also for general generic pickling.
Fixes: #6520
Change-Id: Ib73fd49c875e6da9898493c190f610e68b88ec72
Repaired issue in new :paramref:`_sql.HasCTE.cte.nesting` parameter
introduced with 🎫`4123` where a recursive :class:`_sql.CTE` using
:paramref:`_sql.HasCTE.cte.recursive` in typical conjunction with UNION
would not compile correctly. Additionally makes some adjustments so that
the :class:`_sql.CTE` construct creates a correct cache key.
Pull request courtesy Eric Masseran.
Fixes: #4123
> This has not been caught by the tests because the nesting recursive
queries there did not union against itself, eg there was only the i
root clause...
- Now tests are real recursive queries
- Add tests on aliased nested CTEs (recursive or not)
- Adapt the `_restates` attribute to use it as a reference
- Add some docs around to explain some variables usage
Closes: #7133
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7133
Pull-request-sha: 2633f34f7f
Change-Id: I15512c94e1bc1f52afc619d82057ca647d274e92
Adjusted the "column disambiguation" logic that's new in 1.4, where the
same expression repeated gets an "extra anonymous" label, so that the logic
more aggressively deduplicates those labels when the repeated element
is the same Python expression object each time, as occurs in cases like
when using "singleton" values like :func:`_sql.null`. This is based on
the observation that at least some databases (e.g. MySQL, but not SQLite)
will raise an error if the same label is repeated inside of a subquery.
Related to 🎫`7153`, fixed an issue where result column lookups
would fail for "adapted" SELECT statements that selected for
"constant" value expressions most typically the NULL expression,
as would occur in such places as joined eager loading in conjunction
with limit/offset. This was overall a regression due to issue
🎫`6259` which removed all "adaption" for constants like NULL,
"true", and "false", but this broke the case where the same adaption
logic were used to match the constant to a labeled expression referring
to the constant in a subquery.
Fixes: #7153Fixes: #7154
Change-Id: I43823343721b9e70524ea3f5e8f39dd543a3e92b
Fixed an inconsistency in the any_() / all_() functions / methods where the
special behavior these functions have of "flipping" the expression such
that the "ANY" / "ALL" expression is always on the right side would not
function if the comparison were against the None value, that is,
"column.any_() == None" should produce the same SQL expression as "null()
== column.any_()". Added more docs to clarify this as well, plus mentions
that any_() / all_() generally supersede the ARRAY version "any()" /
"all()".
Fixes: #7140
Change-Id: Ia5d55414ba40eb3fbda3598931fdd24c9b4a4411
Fixed issue where using ORM column expressions as keys in the list of
dictionaries passed to :meth:`_sql.Insert.values` for "multi-valued insert"
would not be processed correctly into the correct column expressions.
Fixes: #7060
Change-Id: I1c4c286c33ea6eeaafba617996828f5c88ff0a1c
Fixed a two issues where combinations of ``select()`` and ``join()`` when
adapted to form a copy of the element would not completely copy the state
of all column objects associated with subqueries. A key problem this caused
is that usage of the :meth:`_sql.ClauseElement.params` method (which should
probably be moved into a legacy category as it is inefficient and error
prone) would leave copies of the old :class:`_sql.BindParameter` objects
around, leading to issues in correctly setting the parameters at execution
time.
Fixes: #7055
Change-Id: Ib822a978a99561b4402da3fb727b370f5c58210b
Added new parameter :meth:`_sql.HasCte.cte.nesting` to the
:class:`_sql.CTE` constructor and :meth:`_sql.HasCTE.cte` method, which
flags the CTE as one which should remain nested within an enclosing CTE,
rather than being moved to the top level of the outermost SELECT. While in
the vast majority of cases there is no difference in SQL functionality,
users have identified various edge-cases where true nesting of CTE
constructs is desirable. Much thanks to Eric Masseran for lots of work on
this intricate feature.
Fixes: #4123Closes: #6709
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6709
Pull-request-sha: 64ab2f6ea2
Change-Id: Ic4dc25ab763af96d96632369e01527d48a654149
Fixed issue related to new ``add_cte()`` feature where pairing two
"INSERT..FROM SELECT" statements simultaneously would lose track of the two
independent SELECT statements, leading to the wrong SQL.
Fixes: #7036
Change-Id: I90fe47eb203bc5c1ea5810db0edba08250b2b7e6
Fixed issue where an engine that had ``implicit_returning`` set to False
would fail to function when PostgreSQL's "fast insertmany" feature were
used in conjunction with a ``Sequence``, as well as if any kind of
"executemany" with "return_defaults()" were used in conjunction with a
``Sequence``. Note that PostgreSQL "fast insertmany" uses "RETURNING" by
definition, when the SQL statement is passed to the driver; overall, the
``implicit_returning`` flag is legacy and has no real use in modern
SQLAlchemy, and will be deprecated in a separate change.
Fixes: #6963
Change-Id: Id8e3dd50a21b9124f338067b0fdb57b8f608dca8
Fixed issue in lambda caching system where an element of a query that
produces no cache key, like a custom option or clause element, would still
populate the expression in the "lambda cache" inappropriately.
This was discovered as part of 🎫`6887` but is a separate
issue.
References: #6887
Change-Id: I1665f4320254ddc63a0abf3088e9daeaffbd1840
Adjusted the "from linter" warning feature to accommodate for a chain of
joins more than one level deep where the ON clauses don't explicitly match
up the targets, such as an expression such as "ON TRUE". This mode of use
is intended to cancel the cartesian product warning simply by the fact that
there's a JOIN from "a to b", which was not working for the case where the
chain of joins had more than one element.
this incurs a bit more compiler overhead that comes out in profiling
but is not extensive.
Added the "is_comparison" flag to the PostgreSQL "overlaps",
"contained_by", "contains" operators, so that they work in relevant ORM
contexts as well as in conjunction with the "from linter" feature.
Fixes: #6886
Change-Id: I078dc3fe6d4f7871ffe4ebac3e71e62f3f213d12
Fix issue in :class:`_sql.CTE` where new :meth:`_sql.HasCTE.add_cte` method
added in version 1.4.21 / 🎫`6752` failed to function correctly for
"compound select" structures such as :func:`_sql.union`,
:func:`_sql.union_all`, :func:`_sql.except`, etc. Pull request courtesy
Eric Masseran.
Fixes: #6752Closes: #6849
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6849
Pull-request-sha: 1c4b4d72b2
Change-Id: I49a16a4fc2af8299502011f3a02d8a2ad93255e3
Added new attribute :attr:`_sql.Select.columns_clause_froms` that will
retrieve the FROM list implied by the columns clause of the
:class:`_sql.Select` statement. This differs from the old
:attr:`_sql.Select.froms` collection in that it does not perform any ORM
compilation steps, which necessarily deannotate the FROM elements and do
things like compute joinedloads etc., which makes it not an appropriate
candidate for the :meth:`_sql.Select.select_from` method. Additionally adds
a new parameter
:paramref:`_sql.Select.with_only_columns.maintain_column_froms` that
transfers this collection to :meth:`_sql.Select.select_from` before
replacing the columns collection.
In addition, the :attr:`_sql.Select.froms` is renamed to
:meth:`_sql.Select.get_final_froms`, to stress that this collection is not
a simple accessor and is instead calculated given the full state of the
object, which can be an expensive call when used in an ORM context.
Additionally fixes a regression involving the
:func:`_orm.with_only_columns` function to support applying criteria to
column elements that were replaced with either
:meth:`_sql.Select.with_only_columns` or :meth:`_orm.Query.with_entities` ,
which had broken as part of 🎫`6503` released in 1.4.19.
Fixes: #6808
Change-Id: Ib5d66cce488bbaca06dab4f68fb5cdaa73e8823e
this appears to be unnecessary and prevents end-user
literal_binds case from working.
Fixed issue where the ``literal_binds`` compiler flag, as used externally
to render bound parameters inline, would fail to work when used with a
certain class of parameters known as "literal_execute", which covers things
like LIMIT and OFFSET values for dialects where the drivers don't allow a
bound parameter, such as SQL Server's "TOP" clause. The issue locally
seemed to affect only the MSSQL dialect.
Fixes: #6863
Change-Id: Ia74cff5b0107b129a11b9b965883552b2962e449
Fixed an issue in the ``CacheKey.to_offline_string()`` method used by the
dogpile.caching example where attempting to create a proper cache key from
the special "lambda" query generated by the lazy loader would fail to
include the parameter values, leading to an incorrect cache key.
Fixes: #6858
Change-Id: Ice27087583c6f3ff79cf7d5b879e5dd0a4e58158
Fixed issue where a bound parameter object that was "cloned" would cause a
name conflict in the compiler, if more than one clone of this parameter
were used at the same time in a single statement. This could occur in
particular with things like ORM single table inheritance queries that
indicated the same "discriminator" value multiple times in one query.
Fixes: #6824
Change-Id: Iba7a786fc5a2341ff7d07fc666d24ed790ad4fe8
Fixed issue where the unit of work would internally use a 2.0-deprecated
SQL expression form, emitting a deprecation warning when SQLALCHEMY_WARN_20
were enabled.
Fixes: #6812
Change-Id: I0a031e728527a1c3382848b6ddc793939362b128
Fixed issue where use of the :paramref:`_sql.case.whens` parameter passing
a dictionary positionally and not as a keyword argument would emit a 2.0
deprecation warning, referring to the deprecation of passing a list
positionally. The dictionary format of "whens", passed positionally, is
still supported and was accidentally marked as deprecated.
Removes warning filter for case statement.
Fixes: #6786
Change-Id: I8efd1882563773bec89ae5e34f0dfede77fc4683
Fixed critical caching issue where the ORM's persistence feature using
INSERT..RETURNING would cache an incorrect query when mixing the "bulk
save" and standard "flush" forms of INSERT.
Fixes: #6793
Change-Id: Ifeb61c1226d3fa6d5e1c2e29b6f5ff77a27d6a2d
Fixed issue in new :meth:`_schema.Table.table_valued` method where the
resulting :class:`_sql.TableValuedColumn` construct would not respond
correctly to alias adaptation as is used throughout the ORM, such as for
eager loading, polymorphic loading, etc.
Fixes: #6775
Change-Id: I77cec4b6e1b1003f2b6be242b54ada8e4a435250
Fixed issue where type-specific bound parameter handlers would not be
called upon in the case of using the :meth:`_sql.Insert.values` method with
the Python ``None`` value; in particular, this would be noticed when using
the :class:`_types.JSON` datatype as well as related PostgreSQL specific
types such as :class:`_postgresql.JSONB` which would fail to encode the
Python ``None`` value into JSON null, however the issue was generalized to
any bound parameter handler in conjunction with this specific method of
:class:`_sql.Insert`.
The issue with coercions forcing out ``null()`` may still impact
SQL expression usage as well; the change here is limited to crud
as the behavior there is relevant to some use cases, which may
need to be evaluated separately.
Fixes: #6770
Change-Id: If53edad811b37dada7578a89daf395628db058a6
Fixed regression which appeared in version 1.4.3 due to 🎫`6060`
where rules that limit ORM adaptation of derived selectables interfered
with other ORM-adaptation based cases, in this case when applying
adaptations for a :func:`_orm.with_polymorphic` against a mapping which
uses a :func:`_orm.column_property` which in turn makes use of a scalar
select that includes a :func:`_orm.aliased` object of the mapped table.
Fixes: #6762
Change-Id: Ice3dc34b97d12b59f044bdc0c5faaefcc4015227
Fixed issue in CTE constructs where a recursive CTE that referred to a
SELECT that has duplicate column names, which are typically deduplicated
using labeling logic in 1.4, would fail to refer to the deduplicated label
name correctly within the WITH clause.
As part of this change we are also attempting to remove the
behavior of SelectStatementGrouping forcing off the "asfrom"
contextual flag, which will have the result of additional labeling
being applied to some UNION and similar statements when they are
interpreted as subqueries. To maintain compatibility with
"grouping", the Grouping/SelectStatementGrouping are now broken
out into two separate compiler cases, as the "asfrom" logic appears
to be tailored towards table valued SELECTS as column expressions.
Fixes: #6710
Change-Id: I8af07a5c670dbe5736cd9f16084ef82f5e4c8642
To service #6718 and #6710, the system by which columns are
given labels in a SELECT statement as well as the system that
gives them keys in a .c or .selected_columns collection have
been refactored to provide a single source of truth for
both, in constrast to the previous approach that included
similar logic repeated in slightly different ways.
Main ideas:
1. ColumnElement attributes ._label, ._anon_label, ._key_label
are renamed to include the letters "tq", meaning
"table-qualified" - these labels are only used when rendering
a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its
label style; as this label style is primarily legacy, the
"tq" names should be isolated so that in a 2.0 style application
these aren't being used at all
2. The means by which the "labels" and "proxy keys" for the elements
of a SELECT has been centralized to a single source of truth;
previously, the three of _generate_columns_plus_names,
_generate_fromclause_column_proxies, and _column_naming_convention
all had duplicated rules between them, as well as that there
were a little bit of labeling rules in compiler._label_select_column
as well; by this we mean that the various "anon_label" "anon_key"
methods on ColumnElement were called by all four of these methods,
where there were many cases where it was necessary that one method
comes up with the same answer as another of the methods. This
has all been centralized into _generate_columns_plus_names
for all the names except the "proxy key", which is generated
by _column_naming_convention.
3. compiler._label_select_column has been rewritten to both not make
any naming decisions nor any "proxy key" decisions, only whether
to label or not to label; the _generate_columns_plus_names method
gives it the information, where the proxy keys come from
_column_naming_convention; previously, these proxy keys were matched
based on restatement of similar (but not really the same) logic in
two places. The heuristics of "whether to label or not to label"
are also reorganized to be much easier to read and understand.
4. a new method compiler._label_returning_column is added for dialects
to use in their "generate returning columns" methods. A
github search reveals a small number of third party dialects also
doing this using the prior _label_select_column method so we
try to make sure _label_select_column continues to work the
exact same way for that specific use case; for the "SELECT" use
case it now needs
5. After some attempts to do it different ways, for the case where
_proxy_key is giving us some kind of anon label, we are hard
changing it to "_no_label" right now, as there's not currently
a way to fully match anonymized labels from stmt.c or
stmt.selected_columns to what will be in the result map. The
idea of "_no_label" is to encourage the user to use label('name')
for columns they want to be able to target by string name that
don't have a natural name.
Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
Added new method :meth:`_sql.HasCTE.add_cte` to each of the
:func:`_sql.select`, :func:`_sql.insert`, :func:`_sql.update` and
:func:`_sql.delete` constructs. This method will add the given
:class:`_sql.CTE` as an "independent" CTE of the statement, meaning it
renders in the WITH clause above the statement unconditionally even if it
is not otherwise referenced in the primary statement. This is a popular use
case on the PostgreSQL database where a CTE is used for a DML statement
that runs against database rows independently of the primary statement.
Fixes: #6752
Change-Id: Ibf635763e40269cbd10f4c17e208850d8e8d0188
Fixed regression where the :func:`_sql.tablesample` construct would fail to
be executable when constructed given a floating-point sampling value not
embedded within a SQL function.
Fixes: #6735
Change-Id: I557bcd4bdbffc4329ad69d5659ba99b1c8deb554
Fixed regression caused in 1.4.19 due to #6503 and related involving
:meth:`_orm.Query.with_entities` where the new structure used would be
inappropriately transferred to an enclosing :class:`_orm.Query` when making
use of set operations such as :meth:`_orm.Query.union`, causing the JOIN
instructions within to be applied to the outside query as well.
Fixes: #6698
Change-Id: Ia9f294215ebc01330d142a0a3e5be9d02be9380f
Fixed issue where passing ``None`` for the value of
:paramref:`_schema.Table.prefixes` would not store an empty list, but
rather the constant ``None``, which may be unexpected by third party
dialects. The issue is revealed by a usage in recent versions of Alembic
that are passing ``None`` for this value. Pull request courtesy Kai
Mueller.
Fixes: #6685Closes: #6672
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6672
Pull-request-sha: b79aca0ee4
Change-Id: I758641c6fbde6f2607d074fecea7efa6728aeea0
Add a impl parameter to :class:`_types.PickleType` constructor, allowing
any arbitary type to be used in place of the default implementation of
:class:`_types.LargeBinary`. Pull request courtesy jason3gb.
Fixes: #6646Closes: #6657
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6657
Pull-request-sha: e49bcd368d
Change-Id: Ib79f3b0ebbc94393f673f5a5ba6558260083d0cf
Fixed old issue where a :func:`_sql.select()` made against the token "*",
which then yielded exactly one column, would fail to correctly organize the
``cursor.description`` column name into the keys of the result object.
Fixes: #6665
Change-Id: Ie8c00f62998972ad4a19a750d2642d00fde006f6
Mike Bayer