backporting a small bit of the changes made for the
2.0 version of #8796.
See if the changes apply cleanly to the 1.4 branch.
Fixes: #8796
Change-Id: I8118511a10beb38c545a55c962a18a77611293af
Fixed regression for 1.4 in :func:`_orm.contains_eager` where the "wrap in
subquery" logic of :func:`_orm.joinedload` would be inadvertently triggered
for use of the :func:`_orm.contains_eager` function with similar statements
(e.g. those that use ``distinct()``, ``limit()`` or ``offset()``). This is
not appropriate for :func:`_orm.contains_eager` which has always had the
contract that the user-defined SQL statement is unmodified with the
exception of adding the appropriate columns.
Also includes an adjustment to the assertion in Label._make_proxy()
which was there to prevent a fixed label name from being anonymized;
if the label is already anonymous, the change should proceed.
This logic was being hit before the contains_eager behavior was
adjusted. With the adjustment, this code is not used.
Fixes: #8569
Change-Id: I161e65041c0162fd2b83cbef40f57a50fcfaf0fd
(cherry picked from commit 57b400f079)
Fixed bugs involving the :paramref:`.Table.include_columns` and the
:paramref:`.Table.resolve_fks` parameters on :class:`.Table`; these
little-used parameters were apparently not working for columns that refer
to foreign key constraints.
In the first case, not-included columns that refer to foreign keys would
still attempt to create a :class:`.ForeignKey` object, producing errors
when attempting to resolve the columns for the foreign key constraint
within reflection; foreign key constraints that refer to skipped columns
are now omitted from the table reflection process in the same way as
occurs for :class:`.Index` and :class:`.UniqueConstraint` objects with the
same conditions. No warning is produced however, as we likely want to
remove the include_columns warnings for all constraints in 2.0.
In the latter case, the production of table aliases or subqueries would
fail on an FK related table not found despite the presence of
``resolve_fks=False``; the logic has been repaired so that if a related
table is not found, the :class:`.ForeignKey` object is still proxied to the
aliased table or subquery (these :class:`.ForeignKey` objects are normally
used in the production of join conditions), but it is sent with a flag that
it's not resolvable. The aliased table / subquery will then work normally,
with the exception that it cannot be used to generate a join condition
automatically, as the foreign key information is missing. This was already
the behavior for such foreign key constraints produced using non-reflection
methods, such as joining :class:`.Table` objects from different
:class:`.MetaData` collections.
Fixes: #8100Fixes: #8101
Change-Id: Ifa37a91bd1f1785fca85ef163eec031660d9ea4d
(cherry picked from commit 40e3c0da5be7dd526866bfc63590fc5621a9bd6e)
Added new attributes :attr:`.ValuesBase.returning_column_descriptions` and
:attr:`.ValuesBase.entity_description` to allow for inspection of ORM
attributes and entities that are installed as part of an :class:`.Insert`,
:class:`.Update`, or :class:`.Delete` construct. The
:attr:`.Select.column_descriptions` accessor is also now implemented for
Core-only selectables.
Fixes: #7861
Change-Id: Ia6a1cd24c798ba61f4e8e8eac90a0fd00d738342
(cherry picked from commit 2f1df5f9105149d6cb01c8b6ab6b9ccffa020780)
Adjusted the compiler's generation of "post compile" symbols including
those used for "expanding IN" as well as for the "schema translate map" to
not be based directly on plain bracketed strings with underscores, as this
conflicts directly with SQL Server's quoting format of also using brackets,
which produces false matches when the compiler replaces "post compile" and
"schema translate" symbols. The issue created easy to reproduce examples
both with the :meth:`.Inspector.get_schema_names` method when used in
conjunction with the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
feature, as well in the unlikely case that a symbol overlapping with the
internal name "POSTCOMPILE" would be used with a feature like "expanding
in".
Fixes: #7300
Change-Id: I6255c850b140522a4aba95085216d0bca18ce230
(cherry picked from commit b919a0a85a)
Adjusted the "column disambiguation" logic that's new in 1.4, where the
same expression repeated gets an "extra anonymous" label, so that the logic
more aggressively deduplicates those labels when the repeated element
is the same Python expression object each time, as occurs in cases like
when using "singleton" values like :func:`_sql.null`. This is based on
the observation that at least some databases (e.g. MySQL, but not SQLite)
will raise an error if the same label is repeated inside of a subquery.
Related to 🎫`7153`, fixed an issue where result column lookups
would fail for "adapted" SELECT statements that selected for
"constant" value expressions most typically the NULL expression,
as would occur in such places as joined eager loading in conjunction
with limit/offset. This was overall a regression due to issue
🎫`6259` which removed all "adaption" for constants like NULL,
"true", and "false", but this broke the case where the same adaption
logic were used to match the constant to a labeled expression referring
to the constant in a subquery.
Fixes: #7153Fixes: #7154
Change-Id: I43823343721b9e70524ea3f5e8f39dd543a3e92b
Added new attribute :attr:`_sql.Select.columns_clause_froms` that will
retrieve the FROM list implied by the columns clause of the
:class:`_sql.Select` statement. This differs from the old
:attr:`_sql.Select.froms` collection in that it does not perform any ORM
compilation steps, which necessarily deannotate the FROM elements and do
things like compute joinedloads etc., which makes it not an appropriate
candidate for the :meth:`_sql.Select.select_from` method. Additionally adds
a new parameter
:paramref:`_sql.Select.with_only_columns.maintain_column_froms` that
transfers this collection to :meth:`_sql.Select.select_from` before
replacing the columns collection.
In addition, the :attr:`_sql.Select.froms` is renamed to
:meth:`_sql.Select.get_final_froms`, to stress that this collection is not
a simple accessor and is instead calculated given the full state of the
object, which can be an expensive call when used in an ORM context.
Additionally fixes a regression involving the
:func:`_orm.with_only_columns` function to support applying criteria to
column elements that were replaced with either
:meth:`_sql.Select.with_only_columns` or :meth:`_orm.Query.with_entities` ,
which had broken as part of 🎫`6503` released in 1.4.19.
Fixes: #6808
Change-Id: Ib5d66cce488bbaca06dab4f68fb5cdaa73e8823e
Fixed issue where use of the :paramref:`_sql.case.whens` parameter passing
a dictionary positionally and not as a keyword argument would emit a 2.0
deprecation warning, referring to the deprecation of passing a list
positionally. The dictionary format of "whens", passed positionally, is
still supported and was accidentally marked as deprecated.
Removes warning filter for case statement.
Fixes: #6786
Change-Id: I8efd1882563773bec89ae5e34f0dfede77fc4683
Fixed issue in CTE constructs where a recursive CTE that referred to a
SELECT that has duplicate column names, which are typically deduplicated
using labeling logic in 1.4, would fail to refer to the deduplicated label
name correctly within the WITH clause.
As part of this change we are also attempting to remove the
behavior of SelectStatementGrouping forcing off the "asfrom"
contextual flag, which will have the result of additional labeling
being applied to some UNION and similar statements when they are
interpreted as subqueries. To maintain compatibility with
"grouping", the Grouping/SelectStatementGrouping are now broken
out into two separate compiler cases, as the "asfrom" logic appears
to be tailored towards table valued SELECTS as column expressions.
Fixes: #6710
Change-Id: I8af07a5c670dbe5736cd9f16084ef82f5e4c8642
To service #6718 and #6710, the system by which columns are
given labels in a SELECT statement as well as the system that
gives them keys in a .c or .selected_columns collection have
been refactored to provide a single source of truth for
both, in constrast to the previous approach that included
similar logic repeated in slightly different ways.
Main ideas:
1. ColumnElement attributes ._label, ._anon_label, ._key_label
are renamed to include the letters "tq", meaning
"table-qualified" - these labels are only used when rendering
a SELECT that has LABEL_STYLE_TABLENAME_PLUS_COL for its
label style; as this label style is primarily legacy, the
"tq" names should be isolated so that in a 2.0 style application
these aren't being used at all
2. The means by which the "labels" and "proxy keys" for the elements
of a SELECT has been centralized to a single source of truth;
previously, the three of _generate_columns_plus_names,
_generate_fromclause_column_proxies, and _column_naming_convention
all had duplicated rules between them, as well as that there
were a little bit of labeling rules in compiler._label_select_column
as well; by this we mean that the various "anon_label" "anon_key"
methods on ColumnElement were called by all four of these methods,
where there were many cases where it was necessary that one method
comes up with the same answer as another of the methods. This
has all been centralized into _generate_columns_plus_names
for all the names except the "proxy key", which is generated
by _column_naming_convention.
3. compiler._label_select_column has been rewritten to both not make
any naming decisions nor any "proxy key" decisions, only whether
to label or not to label; the _generate_columns_plus_names method
gives it the information, where the proxy keys come from
_column_naming_convention; previously, these proxy keys were matched
based on restatement of similar (but not really the same) logic in
two places. The heuristics of "whether to label or not to label"
are also reorganized to be much easier to read and understand.
4. a new method compiler._label_returning_column is added for dialects
to use in their "generate returning columns" methods. A
github search reveals a small number of third party dialects also
doing this using the prior _label_select_column method so we
try to make sure _label_select_column continues to work the
exact same way for that specific use case; for the "SELECT" use
case it now needs
5. After some attempts to do it different ways, for the case where
_proxy_key is giving us some kind of anon label, we are hard
changing it to "_no_label" right now, as there's not currently
a way to fully match anonymized labels from stmt.c or
stmt.selected_columns to what will be in the result map. The
idea of "_no_label" is to encourage the user to use label('name')
for columns they want to be able to target by string name that
don't have a natural name.
Change-Id: I7a92a66f3a7e459ccf32587ac0a3c306650daf11
Fixed issue where query production for joinedload against a complex left
hand side involving joined-table inheritance could fail to produce a
correct query, due to a clause adaption issue.
Fixes: #6595
Change-Id: Id4b839d52447cdc103b392dd8946c4cfa7a829e1
The :class:`.TypeDecorator` class will now emit a warning when used in SQL
compilation with caching unless the ``.cache_ok`` flag is set to ``True``
or ``False``. ``.cache_ok`` indicates that all the parameters passed to the
object are safe to be used as a cache key, ``False`` means they are not.
Fixes: #6436
Change-Id: Ib1bb7dc4b124e38521d615c2e2e691e4915594fb
Fixed regression where usage of the :func:`_sql.text` construct inside the
columns clause of a :class:`_sql.Select` construct, which is better handled
by using a :func:`_sql.literal_column` construct, would nonetheless prevent
constructs like :func:`_sql.union` from working correctly. Other use cases,
such as constructing subuqeries, continue to work the same as in prior
versions where the :func:`_sql.text` construct is silently omitted from the
collection of exported columns. Also repairs similar use within the
ORM.
This adds a new internal method _all_selected_columns. The existing
"selected_columns" collection can't store a TextClause and this never
worked, so they are omitted. The TextClause is also not "exported",
i.e. available for SELECT from a subquery, as was already the case
in 1.3, so the "exported_columns" and "exported_columns_iterator"
accessors are where we now omit TextClause.
Fixed regression involving legacy methods such as
:meth:`_sql.Select.append_column` where internal assertions would fail.
Fixes: #6343Fixes: #6261
Change-Id: I7c2e5b9ae5d94131c77599a020f4310dcf812bcf
Repaired and solidified issues regarding custom functions and other
arbitrary expression constructs which within SQLAlchemy's column labeling
mechanics would seek to use ``str(obj)`` to get a string representation to
use as an anonymous column name in the ``.c`` collection of a subquery.
This is a very legacy behavior that performs poorly and leads to lots of
issues, so has been revised to no longer perform any compilation by
establishing specific methods on :class:`.FunctionElement` to handle this
case, as SQL functions are the only use case that it came into play. An
effect of this behavior is that an unlabeled column expression with no
derivable name will be given an arbitrary label starting with the prefix
``"_no_label"`` in the ``.c`` collection of a subquery; these were
previously being represented either as the generic stringification of that
expression, or as an internal symbol.
This change seeks to make the concept of "anon name" more private
and renames anon_label and anon_key_label to _anon_name_label
and _anon_key_label. There's no end-user utility to these accessors
and we need to be able to reorganize these as well.
Fixes: #6256
Change-Id: Ie63c86b20ca45873affea78500388da94cf8bf94
Enhanced the "expanding" feature used for :meth:`_sql.ColumnOperators.in_`
operations to infer the type of expression from the right hand list of
elements, if the left hand side does not have any explicit type set up.
This allows the expression to support stringification among other things.
In 1.3, "expanding" was not automatically used for
:meth:`_sql.ColumnOperators.in_` expressions, so in that sense this change
fixes a behavioral regression.
Fixes: #6222
Change-Id: Icdfda1e2c226a21896cafd6d8f251547794451c2
Fixed regression where the :func:`_orm.joinedload` loader strategy would
not successfully joinedload to a mapper that is mapper against a
:class:`.CTE` construct.
Fixes: #6172
Change-Id: I667e46d00d4209dab5a89171118a00a7c30fb542
Fixed regression where the :class:`.ConcreteBase` would fail to map at all
when a mapped column name overlapped with the discriminator column name,
producing an assertion error. The use case here did not function correctly
in 1.3 as the polymorphic union would produce a query that ignored the
discriminator column entirely, while emitting duplicate column warnings. As
1.4's architecture cannot easily reproduce this essentially broken behavior
of 1.3 at the ``select()`` level right now, the use case now raises an
informative error message instructing the user to use the
``.ConcreteBase._concrete_discriminator_name`` attribute to resolve the
conflict. To assist with this configuration,
``.ConcreteBase._concrete_discriminator_name`` may be placed on the base
class only where it will be automatically used by subclasses; previously
this was not the case.
Fixes: #6090
Change-Id: I8b7d01e4c9ea0dc97f30b8cd658b3505b24312a7
Fixed issue where the process of joining two tables could fail if one of
the tables had an unrelated, unresolvable foreign key constraint which
would raise :class:`_exc.NoReferenceError` within the join process, which
nonetheless could be bypassed to allow the join to complete. The logic
which tested the exception for signficance within the process would make
assumptions about the construct which would fail.
Fixes: #5952
Change-Id: I492dacd082ddcf8abb1310ed447a6ed734595bb7
Forked from I22f6cf0f0b3360e55299cdcb2452cead2b2458ea
we are attempting to decide the case for columns mapped
under a different name. since the .key feature of
Column seems to support this fully, see if an annotation
can be used to indicate an effective .key for a column.
The effective change is that the labeling of column expressions
in rows has been improved to retain the original name of the ORM
attribute even if used in a subquery.
References: #5933
Change-Id: If251f556f7d723f50d349f765f1690d6c679d2ef
Fixed issue in new 1.4/2.0 style ORM queries where a statement-level label
style would not be preserved in the keys used by result rows; this has been
applied to all combinations of Core/ORM columns / session vs. connection
etc. so that the linkage from statement to result row is the same in all
cases.
also repairs a cache key bug where query.from_statement()
vs. select().from_statement() would not be disambiguated; the
compile options were not included in the cache key for
FromStatement.
Fixes: #5933
Change-Id: I22f6cf0f0b3360e55299cdcb2452cead2b2458ea
Replace :meth:`_orm.Query.with_labels` and
:meth:`_sql.GenerativeSelect.apply_labels` with explicit getters and
setters ``get_label_style`` and ``set_label_style`` to accommodate the
three supported label styles: ``LABEL_STYLE_DISAMBIGUATE_ONLY`` (default),
``LABEL_STYLE_TABLENAME_PLUS_COL``, and ``LABEL_STYLE_NONE``.
In addition, for Core and "future style" ORM queries,
``LABEL_STYLE_DISAMBIGUATE_ONLY`` is now the default label style. This
style differs from the existing "no labels" style in that labeling is
applied in the case of column name conflicts; with ``LABEL_STYLE_NONE``, a
duplicate column name is not accessible via name in any case.
For legacy ORM queries using :class:`_query.Query`, the table-plus-column
names labeling style applied by ``LABEL_STYLE_TABLENAME_PLUS_COL``
continues to be used so that existing test suites and logging facilities
see no change in behavior by default, however this style of labeling is no
longer required for SQLAlchemy queries to function, as result sets are
commonly matched to columns using a positional approach since SQLAlchemy
1.0.
Within test suites, all use of apply_labels() / use_labels
now uses the new methods. New tests added to
test/sql/test_deprecations.py nad test/orm/test_deprecations.py
to cover just the old apply_labels() method call. Tests
in ORM that made explicit use apply_labels()/ etc. where it isn't needed
for the ORM to work correctly use default label style now.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Fixes: #4757
Change-Id: I5fdcd2ed4ae8c7fe62f8be2b6d0e8f66409b6a54
Improved support for column names that contain percent signs in the string,
including repaired issues involving anoymous labels that also embedded a
column name with a percent sign in it, as well as re-established support
for bound parameter names with percent signs embedded on the psycopg2
dialect, using a late-escaping process similar to that used by the
cx_Oracle dialect.
* Added new constructor for _anonymous_label() that ensures incoming
string tokens based on column or table names will have percent
signs escaped; abstracts away the format of the label.
* generalized cx_Oracle's quoted_bind_names facility into the compiler
itself, and leveraged this for the psycopg2 dialect's issue with
percent signs in names as well. the parameter substitution is now
integrated with compiler.construct_parameters() as well as the
recently reworked set_input_sizes(), reducing verbosity in the
cx_Oracle dialect.
Fixes: #5653
Change-Id: Ia2ad13ea68b4b0558d410026e5a33f5cb3fbab2c
It's better, the majority of these changes look more readable to me.
also found some docstrings that had formatting / quoting issues.
Change-Id: I582a45fde3a5648b2f36bab96bad56881321899b
The :meth:`_sql.Join.alias` method is deprecated and will be removed in
SQLAlchemy 2.0. An explicit select + subquery, or aliasing of the inner
tables, should be used instead.
Fixes: #5010
Change-Id: Ic913afc31f0d70b0605f9a7af2742a0de1f9ad19
This change includes mainly that the bracketed use within
select() is moved to positional, and keyword arguments are
removed from calls to the select() function. it does not
yet fully address other issues such as keyword arguments passed
to the table.select().
Additionally, allows False / None to both be considered
as "disable" for all of select.correlate(), select.correlate_except(),
query.correlate(), which establishes consistency with
passing of ``False`` for the legact select(correlate=False)
argument.
Change-Id: Ie6c6e6abfbd3d75d4c8de504c0cf0159e6999108
in order to accommodate relationship loaders
with lambda caching, a lot more is needed. This is
a full refactor of the lambda system such that it
now has two levels of caching; the first level caches what
can be known from the __code__ element, then the next level
of caching is against the lambda itself and the contents
of __closure__. This allows for the elements inside
the lambdas, like columns and entities, to change and
then be part of the cache key. Lazy/selectinloads' use of
baked queries had to add distinct cache key elements,
which was attempted here but overall things needed to be
more robust than that.
This commit is broken out from the very long and sprawling
commit at Id6b5c03b1ce9ddb7b280f66792212a0ef0a1c541 .
Change-Id: I29a513c98917b1d503abfdd61e6b6e8800851aa8
This is kind of a mixed bag of all kinds to help get us
to 1.4 betas. The documentation stuff is a work in
progress. Lots of other relatively small changes to
APIs and things. More commits will follow to continue
improving the documentation and transitioning to the
1.4/2.0 hybrid documentation. In particular some refinements
to Session usage models so that it can match Engine's
scoping / transactional patterns, and a decision to
start moving away from "subtransactions" completely.
* add select().from_statement() to produce FromStatement in an
ORM context
* begin referring to select() that has "plugins" for the few edge
cases where select() will have ORM-only behaviors
* convert dynamic.AppenderQuery to its own object that can use
select(), though at the moment it uses Query to support legacy
join calling forms.
* custom query classes for AppenderQuery are replaced by
do_orm_execute() hooks for custom actions, a separate gerrit
will document this
* add Session.get() to replace query.get()
* Deprecate session.begin->subtransaction. propose within the
test suite a hypothetical recipe for apps that rely on this
pattern
* introduce Session construction level context manager,
sessionmaker context manager, rewrite the whole top of the
session_transaction.rst documentation. Establish context manager
patterns for Session that are identical to engine
* ensure same begin_nested() / commit() behavior as engine
* devise all new "join into an external transaction" recipe,
add test support for it, add rules into Session so it
just works, write new docs. need to ensure this doesn't
break anything
* vastly reduce the verbosity of lots of session docs as
I dont think people read this stuff and it's difficult
to keep current in any case
* constructs like case(), with_only_columns() really need
to move to *columns, add a coercion rule to just change
these.
* docs need changes everywhere I look. in_() is not in
the Core tutorial? how do people even know about it?
Remove tons of cruft from Select docs, etc.
* build a system for common ORM options like populate_existing
and autoflush to populate from execution options.
* others?
Change-Id: Ia4bea0f804250e54d90b3884cf8aab8b66b82ecf
Several weeks of using the future_select() construct
has led to the proposal there be just one select() construct
again which features the new join() method, and otherwise accepts
both the 1.x and 2.x argument styles. This would make
migration simpler and reduce confusion.
However, confusion may be increased by the fact that select().join()
is different Current thinking is we may be better off
with a few hard behavioral changes to old and relatively unknown APIs
rather than trying to play both sides within two extremely similar
but subtly different APIs. At the moment, the .join() thing seems
to be the only behavioral change that occurs without the user
taking any explicit steps. Session.execute() will still
behave the old way as we are adding a future flag.
This change also adds the "future" flag to Session() and
session.execute(), so that interpretation of the incoming statement,
as well as that the new style result is returned, does not
occur for existing applications unless they add the use
of this flag.
The change in general is moving the "removed in 2.0" system
further along where we want the test suite to fully pass
even if the SQLALCHEMY_WARN_20 flag is set.
Get many tests to pass when SQLALCHEMY_WARN_20 is set; this
should be ongoing after this patch merges.
Improve the RemovedIn20 warning; these are all deprecated
"since" 1.4, so ensure that's what the messages read.
Make sure the inforamtion link is on all warnings.
Add deprecation warnings for parameters present and
add warnings to all FromClause.select() types of methods.
Fixes: #5379Fixes: #5284
Change-Id: I765a0b912b3dcd0e995426427d8bb7997cbffd51
References: #5159
This patch contains a variety of ORM and expression layer
tweaks to support ORM constructs in select() statements,
without the 1.3.x requiremnt in Query that a full
_compile_context() + new select() is needed in order to
get a working statement object.
Includes such tweaks as the ability to implement
aliased class of an aliased class,
as we are looking to fully support ACs against subqueries,
as well as the ability to access anonymously-labeled
ColumnProperty expressions within subqueries by
naming the ".key" of the label after the property
key. Some tuning to query.join() as well
as ORMJoin internals to allow things to work more
smoothly.
Change-Id: Id810f485c5f7ed971529489b84694e02a3356d6d
The type coerce element did not correctly apply grouping rules when using
in an expression
Fixes: #5344
Change-Id: Id67b0e60ac54f8992f931aaed62731672f60c96c
Supercedes: If78fbb557c6f2cae637799c3fec2cbc5ac248aaf
Trying to see if by making the cache key memoized, we
still can have the older "identity" form of caching
which is the cheapest of all, at the same time as the
newer "cache key each time" version that is not nearly
as cheap; but still much cheaper than no caching at all.
Also needed is a per-execution update of _keymap when
we invoke from a cached select, so that Column objects
that are anonymous or otherwise adapted will match up.
this is analogous to the adaption of bound parameters
from the cache key.
Adds test coverage for the keymap / construct_params()
changes related to caching. Also hones performance
to a large extent for statement construction and
cache key generation.
Also includes a new memoized attribute
approach that vastly simplifies the previous approach
of "group_expirable_memoized_property" and finally
integrates cleanly with _clone(), _generate(), etc.
no more hardcoding of attributes is needed, as well
as that most _reset_memoization() calls are no longer
needed as the reset is inherent in a _generate() call;
this also has dramatic performance improvements.
Change-Id: I95c560ffcbfa30b26644999412fb6a385125f663
Enhanced the disambiguating labels feature of the
:func:`~.sql.expression.select` construct such that when a select statement
is used in a subquery, repeated column names from different tables are now
automatically labeled with a unique label name, without the need to use the
full "apply_labels()" feature that conbines tablename plus column name.
The disambigated labels are available as plain string keys in the .c
collection of the subquery, and most importantly the feature allows an ORM
:func:`.orm.aliased` construct against the combination of an entity and an
arbitrary subquery to work correctly, targeting the correct columns despite
same-named columns in the source tables, without the need for an "apply
labels" warning.
The existing labeling style is now called
LABEL_STYLE_TABLENAME_PLUS_COL. This labeling style will remain used
throughout the ORM as has been the case for over a decade, however,
the new disambiguation scheme could theoretically replace this scheme
entirely. The new scheme would dramatically alter how SQL looks
when rendered from the ORM to be more succinct but arguably harder
to read.
The tablename_columnname scheme used by Join.c is unaffected here,
as that's still hardcoded to that scheme.
Fixes: #5221
Change-Id: Ib47d9e0f35046b3afc77bef6e65709b93d0c3026
Continuation of I408e0b8be91fddd77cf279da97f55020871f75a9
- add an options() method to the base Generative construct.
this will be where ORM options can go
- Change Null, False_, True_ to be singletons, so that
we aren't instantiating them and having to use isinstance.
The previous issue with this was that they would produce dupe
labels in SELECT statements. Apply the duplicate column
logic, newly added in 1.4, to these objects as well as to
non-apply-labels SELECT statements in general as a means of
improving this.
- create a revised system for generating ClauseList compilation
constructs that simplfies up front creation to not actually
use ClauseList; a simple tuple is rendered by the compiler
using the same constrcution rules as what are used for
ClauseList but without creating the actual object. Apply
to Select, CompoundSelect, revise Update, Delete
- Select, CompoundSelect get an initial CompileState
implementation. All methods used only within compilation
are moved here
- refine update/insert/delete compile state to not require
an outside boolean
- refine and simplify Select._copy_internals
- rework bind(), which is going away, to not use some
of the internal traversal stuff
- remove "autocommit", "for_update" parameters from Select,
references #4643
- remove "autocommit" parameter from TextClause ,
references #4643
- add deprecation warnings for statement.execute(),
engine.execute(), statement.scalar(), engine.scalar().
Fixes: #5193
Change-Id: I04ca0152b046fd42c5054ba10f37e43fc6e5a57b
Reorganization of Select() is the first major element
of the 2.0 restructuring. In order to start this we need
to first create the new Select constructor and apply legacy
elements to the old one. This in turn necessitates
starting up the RemovedIn20Warning concept which itself
need to refer to "sqlalchemy.future", so begin to establish
this basic framework. Additionally, update the
DML constructors with the newer no-keyword style. Remove
the use of the "pending deprecation" and fix Query.add_column()
deprecation which was not acting as deprecated.
Fixes: #4845Fixes: #4648
Change-Id: I0c7a22b2841a985e1c379a0bb6c94089aae6264c
Creating an :func:`.and_` or :func:`.or_` construct with no arguments or
empty ``*args`` will now emit a deprecation warning, as the SQL produced is
a no-op (i.e. it renders as a blank string). This behavior is considered to
be non-intuitive, so for empty or possibly empty :func:`.and_` or
:func:`.or_` constructs, an appropriate default boolean should be included,
such as ``and_(True, *args)`` or ``or_(False, *args)``. As has been the
case for many major versions of SQLAlchemy, these particular boolean
values will not render if the ``*args`` portion is non-empty.
As there are some internal cases where an empty and_() construct is used
in order to build an optional WHERE expression, a private
utility function is added to suit this use case.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Fixes: #5054Closes: #5062
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5062
Pull-request-sha: 5ca2f27281
Change-Id: I599b9c8befa64d9a59a35ad7dd84ff400e3aa647
The "expanding IN" feature, which generates IN expressions at query
execution time which are based on the particular parameters associated with
the statement execution, is now used for all IN expressions made against
lists of literal values. This allows IN expressions to be fully cacheable
independently of the list of values being passed, and also includes support
for empty lists. For any scenario where the IN expression contains
non-literal SQL expressions, the old behavior of pre-rendering for each
position in the IN is maintained. The change also completes support for
expanding IN with tuples, where previously type-specific bind processors
weren't taking effect.
As part of this change, a more explicit separation between
"literal execute" and "post compile" bound parameters is being made;
as the "ansi bind rules" feature is rendering bound parameters
inline, as we now support "postcompile" generically, these should
be used here, however we have to render literal values at
execution time even for "expanding" parameters. new test fixtures
etc. are added to assert everything goes to the right place.
Fixes: #4645
Change-Id: Iaa2b7bfbfaaf5b80799ee17c9b8507293cba6ed1
Created new visitor system called "internal traversal" that
applies a data driven approach to the concept of a class that
defines its own traversal steps, in contrast to the existing
style of traversal now known as "external traversal" where
the visitor class defines the traversal, i.e. the SQLCompiler.
The internal traversal system now implements get_children(),
_copy_internals(), compare() and _cache_key() for most Core elements.
Core elements with special needs like Select still implement
some of these methods directly however most of these methods
are no longer explicitly implemented.
The data-driven system is also applied to ORM elements that
take part in SQL expressions so that these objects, like mappers,
aliasedclass, query options, etc. can all participate in the
cache key process.
Still not considered is that this approach to defining traversibility
will be used to create some kind of generic introspection system
that works across Core / ORM. It's also not clear if
real statement caching using the _cache_key() method is feasible,
if it is shown that running _cache_key() is nearly as expensive as
compiling in any case. Because it is data driven, it is more
straightforward to optimize using inlined code, as is the case now,
as well as potentially using C code to speed it up.
In addition, the caching sytem now accommodates for anonymous
name labels, which is essential so that constructs which have
anonymous labels can be cacheable, that is, their position
within a statement in relation to other anonymous names causes
them to generate an integer counter relative to that construct
which will be the same every time. Gathering of bound parameters
from any cache key generation is also now required as there is
no use case for a cache key that does not extract bound parameter
values.
Applies-to: #4639
Change-Id: I0660584def8627cad566719ee98d3be045db4b8d
The :class:`.Join` construct no longer considers the "onclause" as a source
of additional FROM objects to be omitted from the FROM list of an enclosing
:class:`.Select` object as standalone FROM objects. This applies to an ON
clause that includes a reference to another FROM object outside the JOIN;
while this is usually not correct from a SQL perspective, it's also
incorrect for it to be omitted, and the behavioral change makes the
:class:`.Select` / :class:`.Join` behave a bit more intuitively.
Fixes: #4621
Change-Id: Iaa1e75b7c59b21e9701ab3c9b69e66930feaf8ee
As of #4753 we allow duplicate columns. This creates some new
problems that there can be duplicate columns in a subquery
which are then not addressible on the outside because they
are ambiguous (Postgresql has this behavior at least). Additionally
it creates situations where we are making an anon label of an
anon label which is leaking into the query.
New logic for generating anon labels handles this situation and
also alters the .c collection
of a subquery such that we are only getting the first column
from the derived selectable that has that name, the subsequent ones
have a new deduping label with two underscores and are not exposed
in .c. The dedupe logic when rendering the columns will handle
duplicate label names for different columns, vs. the same column
repeated, as separate cases.
Fixes: #4892
Change-Id: I929fbd8da14bcc239e0481c24bbd9b5ce826e8fa
generation is to be enhanced to include caching
functionality, so ensure that Query and all generative in Core
(e.g. select, DML etc) are using the same generations system.
Additionally, deprecate Select.append methods and state
Select methods independently of their append versions.
Mutability of expression objects is a special case only when
generating new objects during a visit.
Fixes: #4637
Change-Id: I3dfac00d5e0f710c833b236f7a0913e1ca24dde4
Added new entity-targeting capabilities to the :class:`.Query` object to
help with the case where the :class:`.Session` is using a bind dictionary
against mapped classes, rather than a single bind, and the :class:`.Query`
is against a Core statement that was ultimately generated from a method
such as :meth:`.Query.subquery`; a deep search is performed to locate
any ORM entity related to the query in order to locate a mapper if
one is not otherwise present.
Fixes: #4829
Change-Id: I95cf325a5aba21baec4b313246c6f4d692284820
Fixed issue where internal cloning of SELECT constructs could lead to a key
error if the copy of the SELECT changed its state such that its list of
columns changed. This was observed to be occurring in some ORM scenarios
which may be unique to 1.3 and above, so is partially a regression fix.
For 1.4, the _is_clone_of key will be removed entirely as it seems to
have no purpose. This commit is the initial backport to 1.3 which
includes tests.
Fixes: #4780
Change-Id: I0c64962a2eba3763bea3107fc7c7d7aed8244430
Mike Bayer