Add support to ``FETCH {FIRST | NEXT} [ count ] {ROW | ROWS}
{ONLY | WITH TIES}`` in the select for the supported backends,
currently PostgreSQL, Oracle and MSSQL.
Fixes: #5576
Change-Id: Ibb5871a457c0555f82b37e354e7787d15575f1f7
It's better, the majority of these changes look more readable to me.
also found some docstrings that had formatting / quoting issues.
Change-Id: I582a45fde3a5648b2f36bab96bad56881321899b
For most from_self() tests, move them into
test/orm/test_deprecated.py and replace the existing
test with one that uses aliased() plus a subquery.
This then revealed a few more issues.
Related items:
* Added slice() method to GenerativeSelect, to match that
of orm.Query and to make possible migration of one of the
from_self() tests. moved the utility functions used for this
from orm/util into sql/util.
* repairs a caching issue related to subqueryload
where information being derived from the cached path info
was mixing up with query information based on the per-query
state, specifically an AliasedClass that is per query.
* for the above issue, it seemed like path_registry maybe
had to change so that it represents AliasedClass objects
as their cache key rather than on identity, but it wasn't
needed. still seems like it would be more correct.
* enhances the error message raised by coercions for a case
such as when an AliasedClass holds onto a select() object
and not a subquery(); will name the original and resolved
object for clarity (although how is AliasedClass able to
accept a Select() object in the first place?)
* Added _set_propagate_attrs() to Query so that again if
it's passed to AliasedClass, it doesn't raise an error
during coercion, but again maybe that should also be
rejected up front
Fixes: #5368
Change-Id: I5912aa611d899acc87a75eb5ee9f95990592f210
Improved support for covering indexes (with INCLUDE columns). Added the
ability for postgresql to render CREATE INDEX statements with an INCLUDE
clause from Core. Index reflection also report INCLUDE columns separately
for both mssql and postgresql (11+).
Fixes: #4458
Change-Id: If0b82103fbc898cdaeaf6a6d2d421c732744acd6
Added support for inspection / reflection of partial indexes / filtered
indexes, i.e. those which use the ``mssql_where`` or ``postgresql_where``
parameters, with :class:`_schema.Index`. The entry is both part of the
dictionary returned by :meth:`.Inspector.get_indexes` as well as part of a
reflected :class:`_schema.Index` construct that was reflected. Pull
request courtesy Ramon Williams.
**Have a nice day!**
Fixes: #4966Closes: #5504
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5504
Pull-request-sha: b3018bac98
Change-Id: Icbb2f93d1545700718ccb5222097185b815f5dbc
This change includes mainly that the bracketed use within
select() is moved to positional, and keyword arguments are
removed from calls to the select() function. it does not
yet fully address other issues such as keyword arguments passed
to the table.select().
Additionally, allows False / None to both be considered
as "disable" for all of select.correlate(), select.correlate_except(),
query.correlate(), which establishes consistency with
passing of ``False`` for the legact select(correlate=False)
argument.
Change-Id: Ie6c6e6abfbd3d75d4c8de504c0cf0159e6999108
Two operations have been defined:
* :meth:`~.ColumnOperators.regexp_match` implementing a regular
expression match like function.
* :meth:`~.ColumnOperators.regexp_replace` implementing a regular
expression string replace function.
Fixes: #1390
Change-Id: I44556846e4668ccf329023613bd26861d5c674e6
Added the :class:`_schema.Identity` construct that can be used to
configure identity columns rendered with GENERATED { ALWAYS |
BY DEFAULT } AS IDENTITY. Currently the supported backends are
PostgreSQL >= 10, Oracle >= 12 and MSSQL (with different syntax
and a subset of functionalities).
Fixes: #5362Fixes: #5324Fixes: #5360
Change-Id: Iecea6f3ceb36821e8b96f0b61049b580507a1875
Allow specifying the data type when creating a :class:`.Sequence` in
PostgreSQL by using the parameter :paramref:`.Sequence.data_type`.
Fixes: #5498
Change-Id: I2b4a80aa89b1503c56748dc3ecd2cf145faddd8b
Adjusted the :meth:`_types.ARRAY.Comparator.any` and
:meth:`_types.ARRAY.Comparator.all` methods to implement a straight "NOT"
operation for negation, rather than negating the comparison operator.
Fixes: #5518
Change-Id: I87ee9278c321aafe51a679fcfcbb5fbb11307fda
Several weeks of using the future_select() construct
has led to the proposal there be just one select() construct
again which features the new join() method, and otherwise accepts
both the 1.x and 2.x argument styles. This would make
migration simpler and reduce confusion.
However, confusion may be increased by the fact that select().join()
is different Current thinking is we may be better off
with a few hard behavioral changes to old and relatively unknown APIs
rather than trying to play both sides within two extremely similar
but subtly different APIs. At the moment, the .join() thing seems
to be the only behavioral change that occurs without the user
taking any explicit steps. Session.execute() will still
behave the old way as we are adding a future flag.
This change also adds the "future" flag to Session() and
session.execute(), so that interpretation of the incoming statement,
as well as that the new style result is returned, does not
occur for existing applications unless they add the use
of this flag.
The change in general is moving the "removed in 2.0" system
further along where we want the test suite to fully pass
even if the SQLALCHEMY_WARN_20 flag is set.
Get many tests to pass when SQLALCHEMY_WARN_20 is set; this
should be ongoing after this patch merges.
Improve the RemovedIn20 warning; these are all deprecated
"since" 1.4, so ensure that's what the messages read.
Make sure the inforamtion link is on all warnings.
Add deprecation warnings for parameters present and
add warnings to all FromClause.select() types of methods.
Fixes: #5379Fixes: #5284
Change-Id: I765a0b912b3dcd0e995426427d8bb7997cbffd51
References: #5159
Renamed the :meth:`_schema.Table.tometadata` method to
:meth:`_schema.Table.to_metadata`. The previous name remains with a
deprecation warning.
Updated the "decorate" utility function to support decoration
of functions that include non-builtins as default values.
Moves test for deprecated "databases" package into
test/dialect/test_deprecations.py
Fixes: #5413Fixes: #5426
Change-Id: I6ed899871c935f9e46360127c17ccb7cf97cea6e
The psycopg2 dialect now defaults to using the very performant
``execute_values()`` psycopg2 extension for compiled INSERT statements,
and also impements RETURNING support when this extension is used. This
allows INSERT statements that even include an autoincremented SERIAL
or IDENTITY value to run very fast while still being able to return the
newly generated primary key values. The ORM will then integrate this
new feature in a separate change.
Implements RETURNING for insert with executemany
Adds support to return_defaults() mode and inserted_primary_key
to support mutiple INSERTed rows, via return_defauls_rows
and inserted_primary_key_rows accessors.
within default execution context, new cached compiler
getters are used to fetch primary keys from rows
inserted_primary_key now returns a plain tuple. this
is not yet a row-like object however this can be
added.
Adds distinct "values_only" and "batch" modes, as
"values" has a lot of benefits but "batch" breaks
cursor.rowcount
psycopg2 minimum version 2.7 so we can remove the
large number of checks for very old versions of
psycopg2
simplify tests to no longer distinguish between
native and non-native json
Fixes: #5401
Change-Id: Ic08fd3423d4c5d16ca50994460c0c234868bd61c
The :paramref:`.Enum.create_constraint` and
:paramref:`.Boolean.create_constraint` parameters now default to False,
indicating when a so-called "non-native" version of these two datatypes is
created, a CHECK constraint will not be generated by default. These CHECK
constraints present schema-management maintenance complexities that should
be opted in to, rather than being turned on by default.
Fixes: #5367
Change-Id: I0a3fb608ce32143fa757546cc17ba2013e93272a
Fixes: #4860
# Description
Add nowait, skip_lock, of arguments to for_update_clause for mysql
### Checklist
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Closes: #5290
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5290
Pull-request-sha: 490e822e73
Change-Id: Ibd2acc47b538c601c69c8fb954776035ecab4c6c
Revised the :paramref:`.Connection.execution_options.schema_translate_map`
feature such that the processing of the SQL statement to receive a specific
schema name occurs within the execution phase of the statement, rather than
at the compile phase. This is to support the statement being efficiently
cached. Previously, the current schema being rendered into the statement
for a particular run would be considered as part of the cache key itself,
meaning that for a run against hundreds of schemas, there would be hundreds
of cache keys, rendering the cache much less performant. The new behavior
is that the rendering is done in a similar manner as the "post compile"
rendering added in 1.4 as part of 🎫`4645`, 🎫`4808`.
Fixes: #5004
Change-Id: Ia5c89eb27cc8dc2c5b8e76d6c07c46290a7901b6
Fixed issue where the "schema_translate_map" feature would not work with a
PostgreSQL native enumeration type (i.e. :class:`.Enum`,
:class:`.postgresql.ENUM`) in that while the "CREATE TYPE" statement would
be emitted with the correct schema, the schema would not be rendered in
the CREATE TABLE statement at the point at which the enumeration was
referenced.
Fixes: #5158
Change-Id: I41529785de2e736c70a142c2ae5705060bfed73e
Added DDL support for "computed columns"; these are DDL column
specifications for columns that have a server-computed value, either upon
SELECT (known as "virtual") or at the point of which they are INSERTed or
UPDATEd (known as "stored"). Support is established for Postgresql, MySQL,
Oracle SQL Server and Firebird. Thanks to Federico Caselli for lots of work
on this one.
ORM round trip tests included. The ORM makes use of existing
FetchedValue support and no additional ORM logic is present for
the basic feature.
It has been observed that Oracle RETURNING does not return the
new value of a computed column upon UPDATE; it returns the
prior value. As this is very dangerous, a warning is emitted
if a computed column is rendered into the RETURNING clause
of an UPDATE statement.
Fixes: #4894Closes: #4928
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4928
Pull-request-sha: d39c521d5a
Change-Id: I2610b2999a5b1b127ed927dcdaeee98b769643ce
Fixed bug where Postgresql operators such as
:meth:`.postgresql.ARRAY.Comparator.contains` and
:meth:`.postgresql.ARRAY.Comparator.contained_by` would fail to function
correctly for non-integer values when used against a
:class:`.postgresql.array` object, due to an erroneous assert statement.
Fixes: #4822
Change-Id: I886aca4c86dc7d64e9d0dfc1d910a0ae64d775a1
Fixed issue where the :class:`.array_agg` construct in combination with
:meth:`.FunctionElement.filter` would not produce the correct operator
precedence between the FILTER keyword and the array index operator.
Fixes: #4760
Change-Id: Ic662cd3da3330554ec673bafd80495b3f1506098
Modified the :paramref:`.Select.with_for_update.of` parameter so that if a
join or other composed selectable is passed, the individual :class:`.Table`
objects will be filtered from it, allowing one to pass a join() object to
the parameter, as occurs normally when using joined table inheritance with
the ORM. Pull request courtesy Raymond Lu.
Fixes: #4550
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Closes: #4551
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4551
Pull-request-sha: 452da77d15
Change-Id: If4b7c231f7b71190d7245543959fb5c3351125a1
Fully removed the behavior of strings passed directly as components of a
:func:`.select` or :class:`.Query` object being coerced to :func:`.text`
constructs automatically; the warning that has been emitted is now an
ArgumentError or in the case of order_by() / group_by() a CompileError.
This has emitted a warning since version 1.0 however its presence continues
to create concerns for the potential of mis-use of this behavior.
Note that public CVEs have been posted for order_by() / group_by() which
are resolved by this commit: CVE-2019-7164 CVE-2019-7548
Added "SQL phrase validation" to key DDL phrases that are accepted as plain
strings, including :paramref:`.ForeignKeyConstraint.on_delete`,
:paramref:`.ForeignKeyConstraint.on_update`,
:paramref:`.ExcludeConstraint.using`,
:paramref:`.ForeignKeyConstraint.initially`, for areas where a series of SQL
keywords only are expected.Any non-space characters that suggest the phrase
would need to be quoted will raise a :class:`.CompileError`. This change
is related to the series of changes committed as part of 🎫`4481`.
Fixed issue where using an uppercase name for an index type (e.g. GIST,
BTREE, etc. ) or an EXCLUDE constraint would treat it as an identifier to
be quoted, rather than rendering it as is. The new behavior converts these
types to lowercase and ensures they contain only valid SQL characters.
Quoting is applied to :class:`.Function` names, those which are usually but
not necessarily generated from the :attr:`.sql.func` construct, at compile
time if they contain illegal characters, such as spaces or punctuation. The
names are as before treated as case insensitive however, meaning if the
names contain uppercase or mixed case characters, that alone does not
trigger quoting. The case insensitivity is currently maintained for
backwards compatibility.
Fixes: #4481Fixes: #4473Fixes: #4467
Change-Id: Ib22a27d62930e24702e2f0f7c74a0473385a08eb
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
Added support for the :class:`.aggregate_order_by` function to receive
multiple ORDER BY elements, previously only a single element was accepted.
Fixes: #4337
Change-Id: I411ac31697a0d65b568ad65ce5b5181717afbd65
Fixed bug in PostgreSQL dialect where compiler keyword arguments such as
``literal_binds=True`` were not being propagated to a DISTINCT ON
expression.
Fixes: #4325
Change-Id: I9949387dceb7fabe889799f42e92423572368b29
Fixed the :func:`.postgresql.array_agg` function, which is a slightly
altered version of the usual :func:`.functions.array_agg` function, to also
accept an incoming "type" argument without forcing an ARRAY around it,
essentially the same thing that was fixed for the generic function in 1.1
in 🎫`4107`.
Fixes: #4324
Change-Id: I399a29f59c945a217cdd22c65ff0325edea8ea65
Added support for "PARTITION BY" in Postgresql table definitions,
using "postgresql_partition_by". Pull request courtesy
Vsevolod Solovyov.
Change-Id: Id74d6882d7193fae1e5fd44b6e12d6852866fcc4
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/430
Fixed bug in Postgresql :meth:`.postgresql.dml.Insert.on_conflict_do_update`
which would prevent the insert statement from being used as a CTE,
e.g. via :meth:`.Insert.cte`, within another statement.
Change-Id: Ie20972a05e194290bc9d92819750845872949ecc
Fixes: #4074
Since postgresql_ops explicitly states that it expects
string keys, to apply to a function call or expression one
needs to give the SQL expression a label that can be referred
to by name in the dictionary. test / document this.
Change-Id: I4bc4ade46dac27f9c1b92e7823433292beab97b9
Fixes: #3970
Fixed bug in Postgresql :class:`.ExcludeConstraint` where the
"whereclause" and "using" parameters would not be copied during an
operation like :meth:`.Table.tometadata`.
Change-Id: I2f704981d4d4862f9c82a50272006fab8becebb6
Fixes: #3900
Fixed bug in new "ON CONFLICT DO UPDATE" feature where the "set"
values for the UPDATE clause would not be subject to type-level
processing, as normally takes effect to handle both user-defined
type level conversions as well as dialect-required conversions, such
as those required for JSON datatypes. Additionally, clarified that
the keys in the set_ dictionary should match the "key" of the column,
if distinct from the column name. A warning is emitted
for remaining column names that don't match column keys; for
compatibility reasons, these are emitted as they were previously.
Fixes: #3888
Change-Id: I67a04c67aa5f65e6d29f27bf3ef2f8257088d073
Fixed regression caused by the fix in 🎫`3807` (version 1.1.0)
where we ensured that the tablename was qualified in the WHERE clause
of the DO UPDATE portion of PostgreSQL's ON CONFLICT, however you
*cannot* put the table name in the WHERE clause in the actual ON
CONFLICT itself. This was an incorrect assumption, so that portion
of the change in 🎫`3807` is rolled back.
Change-Id: I442d8629496a8e405b54711cfcf487761810ae8a
Fixes: #3846Fixes: #3807
Changed the naming convention used when generating bound parameters
for a multi-VALUES insert statement, so that the numbered parameter
names don't conflict with the anonymized parameters of a WHERE clause,
as is now common in a PostgreSQL ON CONFLICT construct.
Change-Id: I3188d100fe4d322a47d344d6a63d3e40b915f228
Fixes: #3828
Fixed issue in new PG "on conflict" construct where columns including
those of the "excluded" namespace would not be table-qualified
in the WHERE clauses in the statement.
Change-Id: Idfefc93e7e7b0d84805e23d5436d822d606f6a0a
Fixes: #3807
Also adds version detection so that DROP INDEX CONCURRENTLY
only renders if the Postgresql version is 9.2 or higher;
for CREATE INDEX CONCURRENTLY, version detection is added
to allow the phrase to omit if the Postgresql version is
less than 8.2.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Change-Id: I597287e0ebbbe256c957a3579b58ace6848ab4f4
Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/84
This supports the use case of an application that uses the same set of
:class:`.Table` objects in many schemas, such as schema-per-user.
A new execution option
:paramref:`.Connection.execution_options.schema_translate_map` is
added. fixes#2685
- latest tox doesn't like the {posargs} in the profile rerunner
For example, this query:
SELECT s1.users.name FROM s1.users FOR UPDATE OF s1.users
should actually be:
SELECT s1.users.name FROM s1.users FOR UPDATE OF users
fixes#3573
Federico Caselli