A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.
See the notes added to the changelog and migration notes for further
details.
Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
Fixes to the test suite, a few errant imports, and setup.py:
- mysql and postgresql have unused 'json' imports; remove
- postgresql is exporting the 'json' symbol, remove
- make sure setup.py can find __version__ using " or '
- retry logic in provision create database for postgresql fixed
- refactor test_magazine to use cls.tables rather than globals
- remove unused class in test_scoping
- add a comment to test_deprecations that this test suite itself
is deprecated
- don't use mapper() and orm_mapper() in test_unitofwork, just
use mapper()
- remove dupe test_scalar_set_None test in test_attributes
- Python 2.7 and above includes unittest.SkipTest, remove pre-2.7
fallback
- use imported SkipTest in profiling
- declarative test_reflection tests with "reflectable_autoincrement"
already don't run on oracle or firebird; remove conditional logic
for these, which also removes an "id" symbol
- clean up test in test_functions, remove print statement
- remove dupe test_literal_processor_coercion_native_int_out_of_range
in test/sql/test_types.py
- fix psycopg2_hstore ref
Change-Id: I7b3444f8546aac82be81cd1e7b6d8b2ad6834fe6
a few code changes ahead of time to handle some __all__
issues better. also include new flake8 rules, since the
existing flake8 doesn't pass in any case.
Change-Id: I1efdf75124ae7bcac719c22e505bb5b13db06c04
Fixed issue in "expanding IN" feature where using the same bound parameter
name more than once in a query would lead to a KeyError within the process
of rewriting the parameters in the query.
Fixes: #4394
Change-Id: Ibcadce9fefbcb060266d9447c2044ee6efeccf5a
test_compiler is mostly related to SELECT statements as well
as smaller SQL elements. While it still has some DDL related
tests, move out all the remaining insert/update tests into
the already present test_insert.py, test_update.py
Fixes: #2630
Change-Id: I4167618543fd1235d12d1717c8c629d2374b325a
Amended the :class:`.AnsiFunction` class, the base of common SQL
functions like ``CURRENT_TIMESTAMP``, to accept positional arguments
like a regular ad-hoc function. This to suit the case that many of
these functions on specific backends accept arguments such as
"fractional seconds" precision and such. If the function is created
with arguments, it renders the the parenthesis and the arguments. If
no arguents are present, the compiler generates the non-parenthesized form.
Fixes: #4386
Change-Id: Ic492ef177e4987cec99ec4d95f55292be8daa087
Added :class:`.Sequence` to the "string SQL" system that will render a
meaningful string expression (``"<next sequence value: my_sequence>"``)
when stringifying without a dialect a statement that includes a "sequence
nextvalue" expression, rather than raising a compilation error.
Fixes: #4144
Change-Id: Ia910f0e22008a7cde7597365954ede324101cf4d
Added new naming convention tokens ``column_0N_name``, ``column_0_N_name``,
etc., which will render the names / keys / labels for all columns referenced
by a particular constraint in a sequence. In order to accommodate for the
length of such a naming convention, the SQL compiler's auto-truncation
feature now applies itself to constraint names as well, which creates a
shortened, deterministically generated name for the constraint that will
apply to a target backend without going over the character limit of that
backend.
Additional notes:
1. the SQLite dialect had a format_index method that was apparently not
used, removed.
2. the naming convention logic has been applying the foreign key
remote column spec to the naming convention, and not the actual
column name. In the case where the referenced Table object uses
.key inside the columns and these are what ForeignKey() references,
the naming convention was doing the wrong thing. The patch here
fixes this, however this isn't noted in the migration notes.
Fixes: #3989
Change-Id: Ib24f4754b886676096c480fc54b2e5c2463ac99a
this test was using sysdate() and current_timestamp() together
in conjunction with a truncation to DAY, however for four hours
on saturday night (see commit time :) ) these two values will
have a different value if one side is EDT and the other is UTC.
tox does not transmit environment variables including TZ by
default, so even if the server is set up for EDT, running tox
will not set TZ and at least Oracle client seems to use this
value, producing UTC for session time but the database on CI
was configured for EDT, producing EDT for sysdate.
Change-Id: I56602d2402a475a0c4fdf61c1c5fc2618c82f915
Fixed additional warnings generated by Python 3.7 due to changes in the
organization of the Python ``collections`` and ``collections.abc`` packages.
Previous ``collections`` warnings were fixed in version 1.2.11. Pull request
courtesy xtreak.
See I2d1c0ef97c8ecac7af152cc56263422a40faa6bb for the original collections.abc
fixes.
Fixes: #4339
Change-Id: Ia92d2461f20309fb33ea6c6f592f7d4e7e32ae7a
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/475
Fixed bug where the :paramref:`.Enum.create_constraint` flag on the
:class:`.Enum` datatype would not be propagated to copies of the type, which
affects use cases such as declarative mixins and abstract bases.
Fixes: #4341
Change-Id: I978be65f33a616fe4d5f5de03fb3eaab6f6a2272
Refactored :class:`.SQLCompiler` to expose a
:meth:`.SQLCompiler.group_by_clause` method similar to the
:meth:`.SQLCompiler.order_by_clause` and :meth:`.SQLCompiler.limit_clause`
methods, which can be overridden by dialects to customize how GROUP BY
renders. Pull request courtesy Samuel Chou.
Change-Id: I0a7238e55032558c27a0c56a72907c7b883456f1
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/474
MariaDB seems to handle some additional UPDATE/DELETE FROM
syntaxes as well as some forms of INTERSECT and EXCEPT. Open
up tests that expect failure for MySQL to allow success for
MariaDB 10.3.
Change-Id: Ia9341a82485ef7201bb8130d8dbf4a9b6976035a
Added missing window function parameters
:paramref:`.WithinGroup.over.range_` and :paramref:`.WithinGroup.over.rows`
parameters to the :meth:`.WithinGroup.over` and
:meth:`.FunctionFilter.over` methods, to correspond to the range/rows
feature added to the "over" method of SQL functions as part of
🎫`3049` in version 1.1.
Fixes: #4322
Change-Id: I77dcdac65c699a4b52a3fc3ee09a100ffb4fc20e
Fixed issue that is closely related to 🎫`3639` where an expression
rendered in a boolean context on a non-native boolean backend would
be compared to 1/0 even though it is already an implcitly boolean
expression, when :meth:`.ColumnElement.self_group` were used. While this
does not affect the user-friendly backends (MySQL, SQLite) it was not
handled by Oracle (and possibly SQL Server). Whether or not the
expression is implicitly boolean on any database is now determined
up front as an additional check to not generate the integer comparison
within the compliation of the statement.
Fixes: #4320
Change-Id: Iae0a65e5c01bd576e64733c3651e1e1a1a1b240c
:meth:`.TypeEngine.column_expression` methods where these methods would not
work if the target type were part of a :class:`.Variant`, or other target
type of a :class:`.TypeDecorator`. Additionally, the SQL compiler now
calls upon the dialect-level implementation when it renders these methods
so that dialects can now provide for SQL-level processing for built-in
types.
Change-Id: Ic7b39575184db582e628e6ecee48dcda7d03a817
Fixes: #3981
Added new logic to the "expanding IN" bound parameter feature whereby if
the given list is empty, a special "empty set" expression that is specific
to different backends is generated, thus allowing IN expressions to be
fully dynamic including empty IN expressions.
Fixes: #4271
Change-Id: Icc3c73bbd6005206b9d06baaeb14a097af5edd36
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/432
Fixed bug where the multi-table support for UPDATE and DELETE statements
did not consider the additional FROM elements as targets for correlation,
when a correlated SELECT were also combined with the statement. This
change now includes that a SELECT statement in the WHERE clause for such a
statement will try to auto-correlate back to these additional tables in the
parent UPDATE/DELETE or unconditionally correlate if
:meth:`.Select.correlate` is used. Note that auto-correlation raises an
error if the SELECT statement would have no FROM clauses as a result, which
can now occur if the parent UPDATE/DELETE specifies the same tables in its
additional set of tables ; specify :meth:`.Select.correlate` explicitly to
resolve.
Change-Id: Ie11eaad7e49af3f59df11691b104d6359341bdae
Fixes: #4313
Started importing "collections" from "collections.abc" under Python 3.3 and
greater for Python 3.8 compatibility. Pull request courtesy Nathaniel
Knight.
In Python 3.3, the abstract base classes (Iterable, Mapping, etc.)
were moved from the `collections` module and put in the
`collections.abc` module. They remain in the `collections` module for
backwards compatibility, and will until Python 3.8.
This commit adds a variable (`collections_abc`) to the `util/compat`
module, which will be the `collections` module for Python < 3.3 and
before, or the `collections.abc` module for Python >= 3.3. It also
uses the new variable, getting rid of some deprecation warnings that
were seen when running under Python 3.7.
Change-Id: I2d1c0ef97c8ecac7af152cc56263422a40faa6bb
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/464
Fixed bug where a :class:`.Sequence` would be dropped explicitly before any
:class:`.Table` that refers to it, which breaks in the case when the
sequence is also involved in a server-side default for that table, when
using :meth:`.MetaData.drop_all`. The step which processes sequences
to be dropped via non server-side column default functions is now invoked
after the table itself is dropped.
Change-Id: I185f2cc76d2011ad4dd3ba9bde5d8aef0ec335ae
Fixes: #4300
Added new feature :meth:`.FunctionElement.as_comparison` which allows a SQL
function to act as a binary comparison operation that can work within the
ORM.
Change-Id: I07018e2065d09775c0406cabdd35fc38cc0da699
Fixes: #3831
Fixed bug in MySQLdb dialect and variants such as PyMySQL where an
additional "unicode returns" check upon connection makes explicit use of
the "utf8" character set, which in MySQL 8.0 emits a warning that utf8mb4
should be used. This is now replaced with a utf8mb4 equivalent.
Documentation is also updated for the MySQL dialect to specify utf8mb4 in
all examples. Additional changes have been made to the test suite to use
utf8mb3 charsets and databases (there seem to be collation issues in some
edge cases with utf8mb4), and to support configuration default changes made
in MySQL 8.0 such as explicit_defaults_for_timestamp as well as new errors
raised for invalid MyISAM indexes.
Change-Id: Ib596ea7de4f69f976872a33bffa4c902d17dea25
Fixes: #4283Fixes: #4192
Fixed INSERT FROM SELECT with CTEs for the Oracle and MySQL dialects, where
the CTE was being placed above the entire statement as is typical with
other databases, however Oracle and MariaDB 10.2 wants the CTE underneath
the "INSERT" segment. Note that the Oracle and MySQL dialects don't yet
work when a CTE is applied to a subquery inside of an UPDATE or DELETE
statement, as the CTE is still applied to the top rather than inside the
subquery.
Also adds test suite support CTEs against backends.
Change-Id: I8ac337104d5c546dd4f0cd305632ffb56ac8bf90
Fixes: #4275Fixes: #4230
Fixed regression in 1.2 due to 🎫`4147` where a :class:`.Table` that
has had some of its indexed columns redefined with new ones, as would occur
when overriding columns during reflection or when using
:paramref:`.Table.extend_existing`, such that the :meth:`.Table.tometadata`
method would fail when attempting to copy those indexes as they still
referred to the replaced column. The copy logic now accommodates for this
condition.
Change-Id: I521aa2c9f3baa0e84598bbdd6ffe4bf07b6e3ba8
Fixes: #4279
The :class:`.Update` construct now accommodates a :class:`.Join` object
as supported by MySQL for UPDATE..FROM. As the construct already
accepted an alias object for a similar purpose, the feature of UPDATE
against a non-table was already implied so this has been added.
Change-Id: I7b2bca627849384d5377abb0c94626463e4fad04
Fixes: #3645
As SQL Server is now non-native boolean as of
I4765d2a2a00b0d14f50282603cc4d48d4739dac1 but uses the BIT
type, we need to constrain this test to continue to not
run against SQL Server.
Change-Id: I214faf2b788a0e8e10725622e3e71f3b70805533
Fixed issue where the "ambiguous literal" error message used when
interpreting literal values as SQL expression values would encounter a
tuple value, and fail to format the message properly. Pull request courtesy
Miguel Ventura.
Change-Id: I50d5d32d5f80ec79703a42d4b19b42c2f9701f24
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/442
Fixed issue where the compilation of an INSERT statement with the
"literal_binds" option that also uses an explicit sequence and "inline"
generation, as on Postgresql and Oracle, would fail to accommodate the
extra keyword argument within the sequence processing routine.
Change-Id: Ibdab7d340aea7429a210c9535ccf1a3e85f074fb
Fixes: #4231
Fixed a regression that occurred from the previous fix to 🎫`4204` in
version 1.2.5, where a CTE that refers to itself after the
:meth:`.CTE.alias` method has been called would not refer to iself
correctly.
Change-Id: Iaa63d65ad2b90c8693f9953fbb32dbb10c73a037
Fixes: #4204
Fixed bug in :class:.`CTE` construct along the same lines as that of
🎫`4204` where a :class:`.CTE` that was aliased would not copy itself
correctly during a "clone" operation as is frequent within the ORM as well
as when using the :meth:`.ClauseElement.params` method.
Change-Id: Id68d72dd244dedfc7bd6116c9a5123c51a55ea20
Fixes: #4210
Fixed bug in CTE rendering where a :class:`.CTE` that was also turned into
an :class:`.Alias` would not render its "ctename AS aliasname" clause
appropriately if there were more than one reference to the CTE in a FROM
clause.
Change-Id: If8cff27a2f4faa5eceb59aa86398db6edb3b9e72
Fixes: #4204
Fixed bug in new "expanding IN parameter" feature where the bind parameter
processors for values wasn't working at all, tests failed to cover this
pretty basic case which includes that ENUM values weren't working.
Change-Id: I8e2420d7229a3e253e43b5227ebb98f9fe0bd14a
Fixes: #4198
Fixed bug where CTE expressions would not have their name or alias name
quoted when the given name is case sensitive or otherwise requires quoting.
Pull request courtesy Eric Atkin.
Fixes: #4197
Change-Id: Ib8573e82b9a1ca94b50c7c5d73ee98b79465d689
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/426
Added support for :class:`.Enum` to persist the values of the enumeration,
rather than the keys, when using a Python pep-435 style enumerated object.
The user supplies a callable function that will return the string values to
be persisted. This allows enumerations against non-string values to be
value-persistable as well. Pull request courtesy Jon Snyder.
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/410Fixes: #3906
Change-Id: Id385465d215d1e5baaad68368b168afdd846b82c
Fixed bug where the :class:`.Enum` type wouldn't handle
enum "aliases" correctly, when more than one key refers to the
same value. Pull request courtesy Daniel Knell.
Fixes: #4180
Change-Id: Ia716c00ca6c67aeab56965f0fdd575ecb7c71416
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/420
Fixed regression in 1.2 where newly repaired quoting
of collation names in 🎫`3785` breaks SQL Server,
which explicitly does not understand a quoted collation
name. Whether or not mixed-case collation names are
quoted or not is now deferred down to a dialect-level
decision so that each dialect can prepare these identifiers
directly.
Change-Id: Iaf0a8123d9bf4711219e320896bb28c5d2649304
Fixes: #4154
Fixed bug in :meth:`.Insert.values` where using the "multi-values"
format in combination with :class:`.Column` objects as keys rather
than strings would fail. Pull request courtesy Aubrey Stark-Toller.
Change-Id: I9d3b40b5950df8f5bfdc8b1d22f9c3afb277f17f
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/412Fixes: #4162
Fixed bug where the :meth:`.Table.tometadata` method would not properly
accommodate :class:`.Index` objects that didn't consist of simple
column expressions, such as indexes against a :func:`.text` construct,
indexes that used SQL expressions or :attr:`.func`, etc. The routine
now copies expressions fully to a new :class:`.Index` object while
substituting all table-bound :class:`.Column` objects for those
of the target table.
Also refined the means by which tometadata() checks if an Index
or UniqueConstraint is generated by a column-level flag, by propagating
an attribute "_column_flag=True" to such indexes/constraints.
Change-Id: I7ef1b8ea42f9933357ae35f241a5ba9838bac35b
Fixes: #4147
No SQLA built-in subclasses ColumnElement without specifying
an alternate visit_name, and user defined ColumnElement
subclasses should avoid being treated like ColumnClause,
e.g. where a Table is present.
Fixes: #4142
Change-Id: I15ed09ba8bdebae4cb0c7e5e5df3f59351477577
Mike Bayer