Commit Graph

64 Commits

Author SHA1 Message Date
Mike Bayer ec20f346a6 anonymize CRUD params if visiting_cte is present
Fixed issue in :class:`.CTE` constructs involving multiple DDL
:class:`.Insert` statements with multiple VALUES parameter sets where the
bound parameter names generated for these parameter sets would conflict,
generating a compile time error.

Fixes: #12363
Change-Id: If8344ff725d4e0ec58d3ff61f38a0edcfc5bdebd
2025-03-14 10:33:22 -04:00
Mike Bayer 42ddb1fd5f check that two CTEs aren't just annotated forms of the same thing
Fixed issue where using :func:`_orm.aliased` around a :class:`.CTE`
construct could cause inappropriate "duplicate CTE" errors in cases where
that aliased construct appeared multiple times in a single statement.

Fixes: #12364
Change-Id: I9625cd83e9baf5312cdc644b38951353708d3b86
2025-02-20 13:33:44 -05:00
Federico Caselli d539bc3a0f update the format_docs_code to also work on python files
Change-Id: I0a6c9610b3fd85365ed4c2c199e3cad87ee64022
2024-11-30 21:10:21 +01:00
Federico Caselli 3fbbe8d67b remove unnecessary string concat in same line
manually update the files to remove literal string concat on the same line,
since black does not seem to be making progress in handling these

Change-Id: I3c651374c5f3db5b8bc0c700328d67ca03743b7b
2024-02-06 19:44:47 +01:00
Federico Caselli 9b153ff18f Update black to 24.1.1
Change-Id: Iadaea7b798d8e99302e1acb430dc7b758ca61137
2024-01-31 21:54:59 +01:00
Mike Bayer 0248efb761 copy stack related elements to str compiler
Fixed issue in stringify for SQL elements, where a specific dialect is not
passed,  where a dialect-specific element such as the PostgreSQL "on
conflict do update" construct is encountered and then fails to provide for
a stringify dialect with the appropriate state to render the construct,
leading to internal errors.

Fixed issue where stringifying or compiling a :class:`.CTE` that was
against a DML construct such as an :func:`_sql.insert` construct would fail
to stringify, due to a mis-detection that the statement overall is an
INSERT, leading to internal errors.

Fixes: #10753
Change-Id: I783eca3fc7bbc1794fedd325d58181dbcc7e0b75
2023-12-12 17:07:12 -05:00
Mike Bayer a2804c6219 consider indpendent CTE for UPDATE..FROM
Fixed issue where referring to a FROM entry in the SET clause of an UPDATE
statement would not include it in the FROM clause of the UPDATE statement,
if that entry were nowhere else in the statement; this occurs currently for
CTEs that were added using :meth:`.Update.add_cte` to provide the desired
CTE at the top of the statement.

Fixes: #10408
Change-Id: I6e3c6ca7a00cc884bda7e0f24c62c34c75134e5b
2023-10-03 13:29:01 -04:00
Federico Caselli 058c230cea Update black to v23.3.0 and flake8 to v6
This change could be added to .git-blame-ignore-revs

Change-Id: I7ba10052b26bc3c178d23fb50a1123d0aae965ca
2023-05-30 21:44:26 +02:00
Mike Bayer d23dcbaea2 don't count / gather INSERT bind names inside of a CTE
Fixed regression related to the implementation for the new
"insertmanyvalues" feature where an internal ``TypeError`` would occur in
arrangements where a :func:`_sql.insert` would be referred towards inside
of another :func:`_sql.insert` via a CTE; made additional repairs for this
use case for positional dialects such as asyncpg when using
"insertmanyvalues".

at the core here is a change to positional insertmanyvalues
where we now get exactly the positions for the "manyvalues" within
the larger list, allowing non-"manyvalues" on the left and right
sides at the same time, not assuming anything about how RETURNING
renders etc., since CTEs are in the mix also.

Fixes: #9173
Change-Id: I5ff071fbef0d92a2d6046b9c4e609bb008438afd
2023-01-30 22:28:53 +01:00
Federico Caselli 06c234d037 Rewrite positional handling, test for "numeric"
Changed how the positional compilation is performed. It's rendered by the compiler
the same as the pyformat compilation. The string is then processed to replace
the placeholders with the correct ones, and to obtain the correct order of the
parameters.
This vastly simplifies the computation of the order of the parameters, that in
case of nested CTE is very hard to compute correctly.

Reworked how numeric paramstyle behavers:
- added support for repeated parameter, without duplicating them like in normal
positional dialects
- implement insertmany support. This requires that the dialect supports out of
order placehoders, since all parameters that are not part of the VALUES clauses
are placed at the beginning of the parameter tuple
- support for different identifiers for a numeric parameter. It's for example
possible to use postgresql style placeholder $1, $2, etc

Added two new dialect based on sqlite to test "numeric" fully using
both :1 style and $1 style. Includes a workaround for SQLite's
not-really-correct numeric implementation.

Changed parmstyle of asyncpg dialect to use numeric, rendering with its native
$ identifiers

Fixes: #8926
Fixes: #8849
Change-Id: I7c640467d49adfe6d795cc84296fc7403dcad4d6
2022-12-05 09:59:01 -05:00
Federico Caselli 0f2baae6bf Fix positional compiling bugs
Fixed a series of issues regarding positionally rendered bound parameters,
such as those used for SQLite, asyncpg, MySQL and others. Some compiled
forms would not maintain the order of parameters correctly, such as the
PostgreSQL ``regexp_replace()`` function as well as within the "nesting"
feature of the :class:`.CTE` construct first introduced in 🎫`4123`.

Fixes: #8827
Change-Id: I9813ed7c358cc5c1e26725c48df546b209a442cb
2022-12-01 23:50:30 +01:00
Mike Bayer 85fa363c84 deep compare CTEs before considering them conflicting
Fixed issue where referencing a CTE multiple times in conjunction with a
polymorphic SELECT could result in multiple "clones" of the same CTE being
constructed, which would then trigger these two CTEs as duplicates. To
resolve, the two CTEs are deep-compared when this occurs to ensure that
they are equivalent, then are treated as equivalent.

Fixes: #8357
Change-Id: I1f634a9cf7a6c4256912aac1a00506aecea3b0e2
2022-08-05 17:26:00 -04:00
Mike Bayer 3b4d62f4f7 pep484 - sql.selectable
the pep484 task becomes more intense as there is mounting
pressure to come up with a consistency in how data moves
from end-user to instance variable.

current thinking is coming into:

1. there are _typing._XYZArgument objects that represent "what the
   user sent"
2. there's the roles, which represent a kind of "filter" for different
   kinds of objects.   These are mostly important as the argument
   we pass to coerce().
3. there's the thing that coerce() returns, which should be what the
   construct uses as its internal representation of the thing.
   This is _typing._XYZElement.

but there's some controversy over whether or
not we should pass actual ClauseElements around by their role
or not.   I think we shouldn't at the moment, but this makes the
"role-ness" of something a little less portable. Like, we have
to set DMLTableRole for TableClause, Join, and Alias, but then
also we have to repeat those three types in order to set up
_DMLTableElement.

Other change introduced here, there was a deannotate=True
for the left/right of a sql.join().    All tests pass without that.
I'd rather not have that there as if we have a join(A, B) where
A, B are mapped classes, we want them inside of the _annotations.
The rationale seems to be performance, but this performance can
be illustrated to be on the compile side which we hope is cached
in the normal case.

CTEs now accommodate for text selects including recursive.

Get typing to accommodate "util.preloaded" cleanly; add "preloaded"
as a real module.  This seemed like we would have needed
pep562 `__getattr__()` but we don't, just set names in
globals() as we import them.

References: #6810
Change-Id: I34d17f617de2fe2c086fc556bd55748dc782faf0
2022-04-04 09:26:43 -04:00
Mike Bayer c36965ab21 pop the stack that we pushed
Fixed regression caused by 🎫`7760` where the new capabilities of
:class:`.TextualSelect` were not fully implemented within the compiler
properly, leading to issues with composed INSERT constructs such as "INSERT
FROM SELECT" and "INSERT...ON CONFLICT" when combined with CTE and textual
statements.

Fixes: #7798
Change-Id: Ia2ce92507e574dd36fd26dd38ec9dd2713584467
2022-03-08 13:40:12 -05:00
Mike Bayer 0fe8f4a3e7 Add more nesting features to add_cte()
Added new parameter :paramref:`.HasCTE.add_cte.nest_here` to
:meth:`.HasCTE.add_cte` which will "nest" a given :class:`.CTE` at the
level of the parent statement. This parameter is equivalent to using the
:paramref:`.HasCTE.cte.nesting` parameter, but may be more intuitive in
some scenarios as it allows the nesting attribute to be set simultaneously
along with the explicit level of the CTE.

The :meth:`.HasCTE.add_cte` method also accepts multiple CTE objects.

Fixes: #7759
Change-Id: I263c015f5a3f452cb54819aee12bc9bf2953a7bb
2022-02-24 18:43:50 -05:00
Mike Bayer bef67e5812 support add_cte() for TextualSelect
Fixed issue where the :meth:`.HasCTE.add_cte` method as called upon a
:class:`.TextualSelect` instance was not being accommodated by the SQL
compiler. The fix additionally adds more "SELECT"-like compiler behavior to
:class:`.TextualSelect` including that DML CTEs such as UPDATE and INSERT
may be accommodated.

Fixes: #7760
Change-Id: Id97062d882e9b2a81b8e31c2bfaa9cfc5f77d5c1
2022-02-23 15:25:34 -05:00
Mike Bayer 6d589ffbb5 consider truediv as truediv; support floordiv operator
Implemented full support for "truediv" and "floordiv" using the
"/" and "//" operators.  A "truediv" operation between two expressions
using :class:`_types.Integer` now considers the result to be
:class:`_types.Numeric`, and the dialect-level compilation will cast
the right operand to a numeric type on a dialect-specific basis to ensure
truediv is achieved.  For floordiv, conversion is also added for those
databases that don't already do floordiv by default (MySQL, Oracle) and
the ``FLOOR()`` function is rendered in this case, as well as for
cases where the right operand is not an integer (needed for PostgreSQL,
others).

The change resolves issues both with inconsistent behavior of the
division operator on different backends and also fixes an issue where
integer division on Oracle would fail to be able to fetch a result due
to inappropriate outputtypehandlers.

Fixes: #4926
Change-Id: Id54cc018c1fb7a49dd3ce1216d68d40f43fe2659
2021-12-26 19:32:53 -05:00
Eric Masseran 958f902b1f Add Non linear CTE support
"Compound select" methods like :meth:`_sql.Select.union`,
:meth:`_sql.Select.intersect_all` etc. now accept ``*other`` as an argument
rather than ``other`` to allow for multiple additional SELECTs to be
compounded with the parent statement at once. In particular, the change as
applied to :meth:`_sql.CTE.union` and :meth:`_sql.CTE.union_all` now allow
for a so-called "non-linear CTE" to be created with the :class:`_sql.CTE`
construct, whereas previously there was no way to have more than two CTE
sub-elements in a UNION together while still correctly calling upon the CTE
in recursive fashion. Pull request courtesy Eric Masseran.

Allow:

```sql
WITH RECURSIVE nodes(x) AS (
   SELECT 59
   UNION
   SELECT aa FROM edge JOIN nodes ON bb=x
   UNION
   SELECT bb FROM edge JOIN nodes ON aa=x
)
SELECT x FROM nodes;
```

Based on @zzzeek suggestion: https://github.com/sqlalchemy/sqlalchemy/pull/7133#issuecomment-933882348

Fixes: #7259
Closes: #7260
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7260
Pull-request-sha: 2565a5fd4b

Change-Id: I685c8379762b5fb6ab4107ff8f4d8a4de70c0ca6
2021-11-16 14:44:36 -05:00
Mike Bayer 89661c1a21 set within_columns_clause=False for all sub-elements of select()
Fixed issue where using the feature of using a string label for ordering or
grouping described at :ref:`tutorial_order_by_label` would fail to function
correctly if used on a :class:`.CTE` construct, when the CTE were embedded
inside of an enclosing :class:`_sql.Select` statement that itself was set
up as a scalar subquery.

Fixes: #7269
Change-Id: Ied6048a1c9a622374a418230c8cfedafa8d3f87e
2021-11-09 15:25:38 -05:00
Mike Bayer d050193daa fully implement future engine and remove legacy
The major action here is to lift and move future.Connection
and future.Engine fully into sqlalchemy.engine.base.   This
removes lots of engine concepts, including:

* autocommit
* Connection running without a transaction, autobegin
  is now present in all cases
* most "autorollback" is obsolete
* Core-level subtransactions (i.e. MarkerTransaction)
* "branched" connections, copies of connections
* execution_options() returns self, not a new connection
* old argument formats, distill_params(), simplifies calling
  scheme between engine methods
* before/after_execute() events (oriented towards compiled constructs)
  don't emit for exec_driver_sql().  before/after_cursor_execute()
  is still included for this
* old helper methods superseded by context managers, connection.transaction(),
  engine.transaction() engine.run_callable()
* ancient engine-level reflection methods has_table(), table_names()
* sqlalchemy.testing.engines.proxying_engine

References: #7257

Change-Id: Ib20ed816642d873b84221378a9ec34480e01e82c
2021-11-07 14:30:35 -05:00
Eric Masseran ee9b8836a1 Fix recursive CTE to support nesting
Repaired issue in new :paramref:`_sql.HasCTE.cte.nesting` parameter
introduced with 🎫`4123` where a recursive :class:`_sql.CTE` using
:paramref:`_sql.HasCTE.cte.recursive` in typical conjunction with UNION
would not compile correctly.  Additionally makes some adjustments so that
the :class:`_sql.CTE` construct creates a correct cache key.
Pull request courtesy Eric Masseran.

Fixes: #4123

> This has not been caught by the tests because the nesting recursive
queries there did not union against itself, eg there was only the i
root clause...

- Now tests are real recursive queries
- Add tests on aliased nested CTEs (recursive or not)
- Adapt the `_restates` attribute to use it as a reference
- Add some docs around to explain some variables usage

Closes: #7133
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7133
Pull-request-sha: 2633f34f7f

Change-Id: I15512c94e1bc1f52afc619d82057ca647d274e92
2021-10-12 22:46:57 +00:00
Mike Bayer b7226379ac fixes for usage of the null() and similar constants
Adjusted the "column disambiguation" logic that's new in 1.4, where the
same expression repeated gets an "extra anonymous" label, so that the logic
more aggressively deduplicates those labels when the repeated element
is the same Python expression object each time, as occurs in cases like
when using "singleton" values like :func:`_sql.null`.  This is based on
the observation that at least some databases (e.g. MySQL, but not SQLite)
will raise an error if the same label is repeated inside of a subquery.

Related to 🎫`7153`, fixed an issue where result column lookups
would fail for "adapted" SELECT statements that selected for
"constant" value expressions most typically the NULL expression,
as would occur in such places as joined eager loading in conjunction
with limit/offset.  This was overall a regression due to issue
🎫`6259` which removed all "adaption" for constants like NULL,
"true", and "false", but this broke the case where the same adaption
logic were used to match the constant to a labeled expression referring
to the constant in a subquery.

Fixes: #7153
Fixes: #7154
Change-Id: I43823343721b9e70524ea3f5e8f39dd543a3e92b
2021-10-08 17:09:33 +00:00
Eric Masseran a3884f36f6 Implement nesting CTE
Added new parameter :meth:`_sql.HasCte.cte.nesting` to the
:class:`_sql.CTE` constructor and :meth:`_sql.HasCTE.cte` method, which
flags the CTE as one which should remain nested within an enclosing CTE,
rather than being moved to the top level of the outermost SELECT. While in
the vast majority of cases there is no difference in SQL functionality,
users have identified various edge-cases where true nesting of CTE
constructs is desirable. Much thanks to Eric Masseran for lots of work on
this intricate feature.

Fixes: #4123
Closes: #6709
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6709
Pull-request-sha: 64ab2f6ea2

Change-Id: Ic4dc25ab763af96d96632369e01527d48a654149
2021-09-17 11:55:48 -04:00
Mike Bayer 8fca5b6e6a use the stack for insert_from_select
Fixed issue related to new ``add_cte()`` feature where pairing two
"INSERT..FROM SELECT" statements simultaneously would lose track of the two
independent SELECT statements, leading to the wrong SQL.

Fixes: #7036
Change-Id: I90fe47eb203bc5c1ea5810db0edba08250b2b7e6
2021-09-16 13:40:47 -04:00
Eric Masseran 770d50092f Dispatch independent ctes on compound select
Fix issue in :class:`_sql.CTE` where new :meth:`_sql.HasCTE.add_cte` method
added in version 1.4.21 / 🎫`6752` failed to function correctly for
"compound select" structures such as :func:`_sql.union`,
:func:`_sql.union_all`, :func:`_sql.except`, etc. Pull request courtesy
Eric Masseran.

Fixes: #6752
Closes: #6849
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/6849
Pull-request-sha: 1c4b4d72b2

Change-Id: I49a16a4fc2af8299502011f3a02d8a2ad93255e3
2021-08-08 10:44:14 -04:00
Gord Thompson c6b1d24fe7 Modernize tests - dml_whereclause
Fixed issue where the unit of work would internally use a 2.0-deprecated
SQL expression form, emitting a deprecation warning when SQLALCHEMY_WARN_20
were enabled.

Fixes: #6812
Change-Id: I0a031e728527a1c3382848b6ddc793939362b128
2021-07-26 10:08:30 -06:00
Mike Bayer a0953bb709 Adjust CTE recrusive col list to accommodate dupe col names
Fixed issue in CTE constructs where a recursive CTE that referred to a
SELECT that has duplicate column names, which are typically deduplicated
using labeling logic in 1.4, would fail to refer to the deduplicated label
name correctly within the WITH clause.

As part of this change we are also attempting to remove the
behavior of SelectStatementGrouping forcing off the "asfrom"
contextual flag, which will have the result of additional labeling
being applied to some UNION and similar statements when they are
interpreted as subqueries.  To maintain compatibility with
"grouping", the Grouping/SelectStatementGrouping are now broken
out into two separate compiler cases, as the "asfrom" logic appears
to be tailored towards table valued SELECTS as column expressions.

Fixes: #6710
Change-Id: I8af07a5c670dbe5736cd9f16084ef82f5e4c8642
2021-07-13 10:25:52 -04:00
Mike Bayer 204ff1f60c implement independent CTEs
Added new method :meth:`_sql.HasCTE.add_cte` to each of the
:func:`_sql.select`, :func:`_sql.insert`, :func:`_sql.update` and
:func:`_sql.delete` constructs. This method will add the given
:class:`_sql.CTE` as an "independent" CTE of the statement, meaning it
renders in the WITH clause above the statement unconditionally even if it
is not otherwise referenced in the primary statement. This is a popular use
case on the PostgreSQL database where a CTE is used for a DML statement
that runs against database rows independently of the primary statement.

Fixes: #6752
Change-Id: Ibf635763e40269cbd10f4c17e208850d8e8d0188
2021-07-12 17:10:58 -04:00
Mike Bayer 2cf8c5868c Export deferred columns but not col props; fix CTE labeling
Refined the behavior of ORM subquery rendering with regards to deferred
columns and column properties to be more compatible with that of 1.3 while
also providing for 1.4's newer features. As a subquery in 1.4 does not make
use of loader options, including :func:`_orm.deferred`, a subquery that is
against an ORM entity with deferred attributes will now render those
deferred attributes that refer directly to mapped table columns, as these
are needed in the outer SELECT if that outer SELECT makes use of these
columns; however a deferred attribute that refers to a composed SQL
expression as we normally do with :func:`_orm.column_property` will not be
part of the subquery, as these can be selected explicitly if needed in the
subquery. If the entity is being SELECTed from this subquery, the column
expression can still render on "the outside" in terms of the derived
subquery columns. This produces essentially the same behavior as when
working with 1.3. However in this case the fix has to also make sure that
the ``.selected_columns`` collection of an ORM-enabled :func:`_sql.select`
also follows these rules, which in particular allows recursive CTEs to
render correctly in this scenario, which were previously failing to render
correctly due to this issue.

As part of this change the _exported_columns_iterator() method has been
removed and logic simplified to use ._all_selected_columns from any
SelectBase object where _exported_columns_iterator() was used before.
Additionally sets up UpdateBase to include ReturnsRows in its hierarchy;
the literal point of ReturnsRows was to be a common base for UpdateBase
and SelectBase so it was kind of weird it wasn't there.

Fixes: #6661

Fixed issue in CTE constructs mostly relevant to ORM use cases where a
recursive CTE against "anonymous" labels such as those seen in ORM
``column_property()`` mappings would render in the
``WITH RECURSIVE xyz(...)`` section as their raw internal label and not a
cleanly anonymized name.

Fixes: #6663
Change-Id: I26219d4d8e6c0915b641426e9885540f74fae4d2
2021-06-22 17:25:37 -04:00
Federico Caselli debeea97ee Update black flak8 and zimports
Change-Id: I488c9557eda390e4a88319affd4c8813ee274f80
2021-05-12 22:10:19 +02:00
Mike Bayer 07ab6441ea allow CTE to be direct DML target
Implemented support for a :class:`_sql.CTE` construct to be used directly
as the target of a :func:`_sql.delete` construct, i.e. "WITH ... AS cte
DELETE FROM cte". This appears to be a useful feature of SQL Server.

The CTE is now generally usable as a DML target table however
it's not clear if this syntax is valid beyond the use case of
DELETE itself.

Fixes: #6464
Change-Id: I3aac6fae2a1abb39bc0ffa87a044f5eb4f90f026
2021-05-11 08:39:24 -04:00
Mike Bayer 901f7a2b53 pass asfrom correctly in compilers
Fixed an argument error in the default and PostgreSQL compilers that
would interfere with an UPDATE..FROM or DELETE..FROM..USING statement
that was then SELECTed from as a CTE.

The incorrect pattern was also fixed in the mysql and sybase dialects.
MySQL supports CTEs but not "returning".

Fixes: #6303
Change-Id: Ic94805611a5ec443749fb6b1fd8a1326b0d83ef7
2021-04-17 00:37:47 -04:00
Mike Bayer ac2ed15740 Disallow AliasedReturnsRows from execution
Executing a :class:`_sql.Subquery` using :meth:`_engine.Connection.execute`
is deprecated and will emit a deprecation warning; this use case was an
oversight that should have been removed from 1.4. The operation will now
execute the underlying :class:`_sql.Select` object directly for backwards
compatibility. Similarly, the :class:`_sql.CTE` class is also not
appropriate for execution. In 1.3, attempting to execute a CTE would result
in an invalid "blank" SQL statement being executed; since this use case was
not working it now raises :class:`_exc.ObjectNotExecutableError`.
Previously, 1.4 was attempting to execute the CTE as a statement however it
was working only erratically.

The change also breaks out StatementRole from ReturnsRowsRole, as these
roles should not be in the same lineage (some statements don't return
rows, the whole class of ReturnsRows that are from clauses are
not statements).    Consolidate StatementRole and
CoerceTextStatementRole as there's no usage difference between
these.   Simplify some old tests that were trying to make
sure that "execution options" didn't transmit from a cte/subquery
out to a select; as cte/subuqery() aren't executable in any case
the options are removed.

Fixes: #6204
Change-Id: I62613b7ab418afdd22c409eae75659e3f52fb65f
2021-04-05 23:35:53 -04:00
Gord Thompson 22f65156bb Replace with_labels() and apply_labels() in ORM/Core
Replace :meth:`_orm.Query.with_labels` and
:meth:`_sql.GenerativeSelect.apply_labels` with explicit getters and
setters ``get_label_style`` and ``set_label_style`` to accommodate the
three supported label styles: ``LABEL_STYLE_DISAMBIGUATE_ONLY`` (default),
``LABEL_STYLE_TABLENAME_PLUS_COL``, and ``LABEL_STYLE_NONE``.

In addition, for Core and "future style" ORM queries,
``LABEL_STYLE_DISAMBIGUATE_ONLY`` is now the default label style. This
style differs from the existing "no labels" style in that labeling is
applied in the case of column name conflicts; with ``LABEL_STYLE_NONE``, a
duplicate column name is not accessible via name in any case.

For legacy ORM queries using :class:`_query.Query`, the table-plus-column
names labeling style applied by ``LABEL_STYLE_TABLENAME_PLUS_COL``
continues to be used so that existing test suites and logging facilities
see no change in behavior by default, however this style of labeling is no
longer required for SQLAlchemy queries to function, as result sets are
commonly matched to columns using a positional approach since SQLAlchemy
1.0.

Within test suites, all use of apply_labels()  / use_labels
now uses the new methods.    New tests added to
test/sql/test_deprecations.py nad test/orm/test_deprecations.py
to cover just the old apply_labels() method call.  Tests
in ORM that made explicit use apply_labels()/ etc. where it isn't needed
for the ORM to work correctly use default label style now.

Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Fixes: #4757
Change-Id: I5fdcd2ed4ae8c7fe62f8be2b6d0e8f66409b6a54
2021-01-26 16:52:30 -05:00
Mike Bayer c3f102c9fe upgrade to black 20.8b1
It's better, the majority of these changes look more readable to me.
also found some docstrings that had formatting / quoting issues.

Change-Id: I582a45fde3a5648b2f36bab96bad56881321899b
2020-09-28 15:17:26 -04:00
Federico Caselli e860060866 Update select usage to use the new 1.4 format
This change includes mainly that the bracketed use within
select() is moved to positional, and keyword arguments are
removed from calls to the select() function.  it does not
yet fully address other issues such as keyword arguments passed
to the table.select().

Additionally, allows False / None to both be considered
as "disable" for all of select.correlate(), select.correlate_except(),
query.correlate(), which establishes consistency with
passing of ``False`` for the legact select(correlate=False)
argument.

Change-Id: Ie6c6e6abfbd3d75d4c8de504c0cf0159e6999108
2020-09-08 17:13:48 -04:00
Mike Bayer 7fe400f546 Restore crud flags if visiting_cte is set
Fixed bug where a CTE of an INSERT/UPDATE/DELETE that also uses RETURNING
could then not be SELECTed from directly, as the internal state of the
compiler would try to treat the outer SELECT as a DELETE statement itself
and access nonexistent state.

Fixes: #5181
Change-Id: Icba76f2148c8344baa1c04bac4ab6c6d24f23072
2020-03-03 17:23:56 -05:00
Mike Bayer 748bf710fd Source base cleanups
in trying to apply 2020 copyright to files, the pre-commit
hooks complain about random file issues.

- remove old corrections.py utility, this had something to do
  with repairing refs in the sphinx docs

- run pre commit hooks on all files

- formatting adjustments to work around code formatting collisions
  (long import lines that zimports can't rewrite correctly)

Change-Id: I260744866f69e902eb93665c7c728ee94d3371a2
2020-01-01 11:11:24 -05:00
Marat Sharafutdinov c18a04affd Add CTE prefixes
Added support for prefixes to the :class:`.CTE` construct, to allow
support for Postgresql 12 "MATERIALIZED" and "NOT MATERIALIZED" phrases.
Pull request courtesy Marat Sharafutdinov.

Fixes: #5040
Closes: #5043
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5043
Pull-request-sha: d1b9059a0b

Change-Id: I2e9cb5d7f85961ec98ee51965de5b3ec4a97be2f
2019-12-18 17:47:51 -05:00
Mike Bayer aceefb508c Allow duplicate columns in from clauses and selectables
The :func:`.select` construct and related constructs now allow for
duplication of column labels and columns themselves in the columns clause,
mirroring exactly how column expressions were passed in.   This allows
the tuples returned by an executed result to match what was SELECTed
for in the first place, which is how the ORM :class:`.Query` works, so
this establishes better cross-compatibility between the two constructs.
Additionally, it allows column-positioning-sensitive structures such as
UNIONs (i.e. :class:`.CompoundSelect`) to be more intuitively constructed
in those cases where a particular column might appear in more than one
place.   To support this change, the :class:`.ColumnCollection` has been
revised to support duplicate columns as well as to allow integer index
access.

Fixes: #4753
Change-Id: Ie09a8116f05c367995c1e43623c51e07971d3bf0
2019-07-11 14:20:10 -04:00
Mike Bayer f07e050c9c Implement new ClauseElement role and coercion system
A major refactoring of all the functions handle all detection of
Core argument types as well as perform coercions into a new class hierarchy
based on "roles", each of which identify a syntactical location within a
SQL statement.  In contrast to the ClauseElement hierarchy that identifies
"what" each object is syntactically, the SQLRole hierarchy identifies
the "where does it go" of each object syntactically.   From this we define
a consistent type checking and coercion system that establishes well
defined behviors.

This is a breakout of the patch that is reorganizing select()
constructs to no longer be in the FromClause hierarchy.

Also includes a rename of as_scalar() into scalar_subquery(); deprecates
automatic coercion to scalar_subquery().

Partially-fixes: #4617
Change-Id: I26f1e78898693c6b99ef7ea2f4e7dfd0e8e1a1bd
2019-05-18 17:46:10 -04:00
Mike Bayer 4ca3092c0a Prevent __init__ from being called for Alias, subclasses
The :class:`.Alias` class and related subclasses :class:`.CTE`,
:class:`.Lateral` and :class:`.TableSample` have been reworked so that it is
not possible for a user to construct the objects directly.  These constructs
require that the standalone construction function or selectable-bound method
be used to instantiate new objects.

Fixes: #4509
Change-Id: I74ae4786cb3ae625dab33b00bfd6bdc4e1219139
2019-02-21 11:18:19 -05:00
Mike Bayer 1e278de4cc Post black reformatting
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9,  this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.

Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.

Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
2019-01-06 18:23:11 -05:00
Mike Bayer 1e1a38e780 Run black -l 79 against all source files
This is a straight reformat run using black as is, with no edits
applied at all.

The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines.   The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.

Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
2019-01-06 17:34:50 +00:00
Mike Bayer ef2859bbc8 Track if we're rendering within the CTE recursively
Fixed a regression that occurred from the previous fix to 🎫`4204` in
version 1.2.5, where a CTE that refers to itself after the
:meth:`.CTE.alias` method has been called would not refer to iself
correctly.

Change-Id: Iaa63d65ad2b90c8693f9953fbb32dbb10c73a037
Fixes: #4204
2018-03-14 11:38:15 -04:00
Mike Bayer 9a13f007e2 Clone _cte_alias instead of assigning "self"
Fixed bug in :class:.`CTE` construct along the same lines as that of
🎫`4204` where a :class:`.CTE` that was aliased would not copy itself
correctly during a "clone" operation as is frequent within the ORM as well
as when using the :meth:`.ClauseElement.params` method.

Change-Id: Id68d72dd244dedfc7bd6116c9a5123c51a55ea20
Fixes: #4210
2018-03-05 21:37:12 -05:00
Mike Bayer 5f60dc649c Check existing CTE for an alias name when rendering FROM clause
Fixed bug in CTE rendering where a :class:`.CTE` that was also turned into
an :class:`.Alias` would not render its "ctename AS aliasname" clause
appropriately if there were more than one reference to the CTE in a FROM
clause.

Change-Id: If8cff27a2f4faa5eceb59aa86398db6edb3b9e72
Fixes: #4204
2018-03-01 10:54:35 -05:00
Eric Atkin b6e4a16809 Quote cte alias if needed
Fixed bug where CTE expressions would not have their name or alias name
quoted when the given name is case sensitive or otherwise requires quoting.
Pull request courtesy Eric Atkin.

Fixes: #4197
Change-Id: Ib8573e82b9a1ca94b50c7c5d73ee98b79465d689
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/426
2018-02-22 15:06:01 -05:00
Khairi Hafsham 772374735d Make all tests to be PEP8 compliant
tested using pycodestyle version 2.2.0

Fixes: #3885
Change-Id: I5df43adc3aefe318f9eeab72a078247a548ec566
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/343
2017-02-07 11:21:56 -05:00
Mike Bayer 71030d67bd Propagate execution_options at compile stage
Compiler can now set up execution options and additionally
will propagate autocommit from embedded CTEs.

Change-Id: I19db7b8fe4d84549ea95342e8d2040189fed1bbe
Fixes: #3805
2016-10-05 16:47:31 -04:00