Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
Fixed issue where a :class:`.postgresql.ENUM` or a custom domain present
in a remote schema would not be recognized within column reflection if
the name of the enum/domain or the name of the schema required quoting.
A new parsing scheme now fully parses out quoted or non-quoted tokens
including support for SQL-escaped quotes.
Fixed issue where multiple :class:`.postgresql.ENUM` objects referred to
by the same :class:`.MetaData` object would fail to be created if
multiple objects had the same name under different schema names. The
internal memoization the Postgresql dialect uses to track if it has
created a particular :class:`.postgresql.ENUM` in the database during
a DDL creation sequence now takes schema name into account.
Fixes: #4416
Change-Id: I8cf03069e10b12f409e9b6796e24fc5850979955
test_compiler is mostly related to SELECT statements as well
as smaller SQL elements. While it still has some DDL related
tests, move out all the remaining insert/update tests into
the already present test_insert.py, test_update.py
Fixes: #2630
Change-Id: I4167618543fd1235d12d1717c8c629d2374b325a
Fixed issue where reflection of a PostgreSQL domain that is expressed as an
array would fail to be recognized. Pull request courtesy Jakub Synowiec.
Fixes: #4377
Change-Id: I252c79ca435b87d4d9172b1c84e0e74e789ef676
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4380
As MySQLdb support read_timeout and write_timeout option, and
sqlalchemy just support read_timeout option. So sqlalchemy need to add
write_timeout option.
Fixes: #4381
Change-Id: I2bea80bdd6f20fafc72b48fa0e5d795626d9d9b9
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4385
Updated the parameters that can be sent to the cx_Oracle DBAPI to both allow
for all current parameters as well as for future parameters not added yet.
In addition, removed unused parameters that were deprecated in version 1.2,
and additionally we are now defaulting "threaded" to False.
Fixes: #4369
Change-Id: I599668960e7b2d5bd1f5e6850e10b5b3ec215ed3
Deprecated the use of :class:`.Sequence` with SQL Server in order to affect
the "start" and "increment" of the IDENTITY value, in favor of new
parameters ``mssql_identity_start`` and ``mssql_identity_increment`` which
set these parameters directly. :class:`.Sequence` will be used to generate
real ``CREATE SEQUENCE`` DDL with SQL Server in a future release.
Fixes: #4362
Change-Id: I1e69378c5c960ff0bc28137c923589692f1a918f
Fixed regression caused by 🎫`4344` released in 1.2.13, where the fix
for MySQL 8.0's case sensitivity problem with referenced column names when
reflecting foreign key referents is worked around using the
``information_schema.columns`` view. The workaround was failing on OSX /
``lower_case_table_names=2`` which produces non-matching casing for the
``information_schema.columns`` vs. that of ``SHOW CREATE TABLE``, so in
case-insensitive SQL modes case-insensitive matching is now used.
Fixes: #4361
Change-Id: I748549bc4c27fad6394593f8ec93fc22bfd01f6c
Implemented the SQLite ``ON CONFLICT`` clause as understood at the DDL
level, e.g. for primary key, unique, and CHECK constraints as well as
specified on a :class:`.Column` to satisfy inline primary key and NOT NULL.
Pull request courtesy Denis Kataev.
Fixes: #4360
Change-Id: I4cd4bafa8fca41e3101c87dbbfe169741bbda3f4
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/431
Added a workaround for a MySQL bug #88718 introduced in the 8.0 series,
where the reflection of a foreign key constraint is not reporting the
correct case sensitivity for the referred column, leading to errors during
use of the reflected constraint such as when using the automap extension.
The workaround emits an additional query to the information_schema tables in
order to retrieve the correct case sensitive name.
Fixes: #4344
Change-Id: I08020d6eec43cbe8a56316660380d3739a0b45f7
Added rudimental support for reflection of Postgresql
partitioned tables, e.g. that relkind='p' is added to reflection
queries that return table information.
Fixes: #4237
Change-Id: I66fd10b002e4ed21ea13b13a7e35a85f66bdea75
Added support for the :class:`.aggregate_order_by` function to receive
multiple ORDER BY elements, previously only a single element was accepted.
Fixes: #4337
Change-Id: I411ac31697a0d65b568ad65ce5b5181717afbd65
The Oracle dialect will no longer use the NCHAR/NCLOB datatypes to
represent generic unicode strings or clob fields in conjunction with
:class:`.Unicode` and :class:`.UnicodeText` unless the flag
``use_nchar_for_unicode=True`` is passed to :func:`.create_engine`.
Additionally, string types under Oracle now coerce to unicode under Python
2 in all cases, however unlike in previous iterations, we use SQLAlchemy's
native unicode handlers which are very performant (when C extensions are
enabled; when they are not, we use cx_Oracle's handlers).
Change-Id: I3939012e9396520875bc52e69bf81f27393836ee
Fixes: #4242
Fixed bug in PostgreSQL dialect where compiler keyword arguments such as
``literal_binds=True`` were not being propagated to a DISTINCT ON
expression.
Fixes: #4325
Change-Id: I9949387dceb7fabe889799f42e92423572368b29
Fixed the :func:`.postgresql.array_agg` function, which is a slightly
altered version of the usual :func:`.functions.array_agg` function, to also
accept an incoming "type" argument without forcing an ARRAY around it,
essentially the same thing that was fixed for the generic function in 1.1
in 🎫`4107`.
Fixes: #4324
Change-Id: I399a29f59c945a217cdd22c65ff0325edea8ea65
Fixed bug in PostgreSQL ENUM reflection where a case-sensitive, quoted name
would be reported by the query including quotes, which would not match a
target column during table reflection as the quotes needed to be stripped
off.
Fixes: #4323
Change-Id: I668f3acccc578e58f23b70c82d31d5c1ec194913
Added support for the parameters in an ON DUPLICATE KEY UPDATE statement on
MySQL to be ordered, since parameter order in a MySQL UPDATE clause is
significant, in a similar manner as that described at
:ref:`updates_order_parameters`. Pull request courtesy Maxim Bublis.
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/462
Change-Id: If508d8e26dbd3c55ab1e83cf573fb4021e9d091e
For cx_Oracle, Integer datatypes will now be bound to "int", per advice
from the cx_Oracle developers. Previously, using cx_Oracle.NUMBER caused a
loss in precision within the cx_Oracle 6.x series.
Change-Id: I4c6b2cca490aff5b98b7ceff3414715202881c89
Fixes: #4309
Fixed issue where the "schema" name used for a SQLite database within table
reflection would not quote the schema name correctly. Pull request
courtesy Phillip Cloud.
Change-Id: I2770788c1f094a7743209250ec26b5ef5fb2d9e8
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/463
Added ``fast_executemany=True`` parameter to the SQL Server pyodbc dialect,
which enables use of pyodbc's new performance feature of the same name
when using Microsoft ODBC drivers.
Change-Id: I743fa7280e8f709addd330cfc7682623701cbb2e
Fixes: #4158
Added support for SQLite's json functionality via the new
SQLite implementation for :class:`.sqltypes.JSON`, :class:`.sqlite.JSON`.
The name used for the type is ``JSON``, following an example found at
SQLite's own documentation. Pull request courtesy Ilja Everilä.
Fixes: #3850
Change-Id: I3d2714fb8655343a99d13dc751b16b93d05d7dda
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/434
Fixed bug in MSSQL reflection where when two same-named tables in different
schemas had same-named primary key constraints, foreign key constraints
referring to one of the tables would have their columns doubled, causing
errors. Pull request courtesy Sean Dunn.
Fixes: #4228
Change-Id: I7dabaaee0944e1030048826ba39fc574b0d63031
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/457
Fixed bug in index reflection where on MySQL 8.0 an index that includes
ASC or DESC in an indexed column specfication would not be correctly
reflected, as MySQL 8.0 introduces support for returning this information
in a table definition string.
Change-Id: I21f64984ade690aac8c87dbe3aad0c1ee8e9727f
Fixes: #4293
Add CI support for MySQL connector and try to fix some of the
more obvious issues. CI tests will run against MySQL 5.7
only for starters as there appear to be issues with
MySQL 8.0
Change-Id: Id8971143a8385a5c84f0646c21c4c21e793ce3a2
Added a new event currently used only by the cx_Oracle dialect,
:meth:`.DialectEvents.setiputsizes`. The event passes a dictionary of
:class:`.BindParameter` objects to DBAPI-specific type objects that will be
passed, after conversion to parameter names, to the cx_Oracle
``cursor.setinputsizes()`` method. This allows both visibility into the
setinputsizes process as well as the ability to alter the behavior of what
datatypes are passed to this method.
Change-Id: I43b97c8e3c840cad6f01edb274dc9cfed19cb5fc
Fixes: #4290
Fixed issue in test suite where SQLite 3.24 added a new reserved word that
conflicted with a usage in TypeReflectionTest. Pull request courtesy Nils
Philippsen.
Change-Id: I396562cecb5ca774f29e9234845bcc6a399fc5cb
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/452
Fixed issue within the SQL Server dialect under Python 3 where when running
against a non-standard SQL server database that does not contain either the
"sys.dm_exec_sessions" or "sys.dm_pdw_nodes_exec_sessions" views, leading
to a failure to fetch the isolation level, the error raise would fail due
to an UnboundLocalError.
Fixes: #4273
Co-authored-by: wikiped <wikiped@yandex.ru>
Change-Id: I39877c1f65f9cf8602fb1dceaf03072357759564
Fixed bug in MySQLdb dialect and variants such as PyMySQL where an
additional "unicode returns" check upon connection makes explicit use of
the "utf8" character set, which in MySQL 8.0 emits a warning that utf8mb4
should be used. This is now replaced with a utf8mb4 equivalent.
Documentation is also updated for the MySQL dialect to specify utf8mb4 in
all examples. Additional changes have been made to the test suite to use
utf8mb3 charsets and databases (there seem to be collation issues in some
edge cases with utf8mb4), and to support configuration default changes made
in MySQL 8.0 such as explicit_defaults_for_timestamp as well as new errors
raised for invalid MyISAM indexes.
Change-Id: Ib596ea7de4f69f976872a33bffa4c902d17dea25
Fixes: #4283Fixes: #4192
The Oracle BINARY_FLOAT and BINARY_DOUBLE datatypes now participate within
cx_Oracle.setinputsizes(), passing along NATIVE_FLOAT, so as to support the
NaN value. Additionally, :class:`.oracle.BINARY_FLOAT`,
:class:`.oracle.BINARY_DOUBLE` and :class:`.oracle.DOUBLE_PRECISION` now
subclass :class:`.Float`, since these are floating point datatypes, not
decimal. These datatypes were already defaulting the
:paramref:`.Float.asdecimal` flag to False in line with what
:class:`.Float` already does.
Added reflection capabilities for the :class:`.oracle.BINARY_FLOAT`,
:class:`.oracle.BINARY_DOUBLE` datatypes.
Change-Id: Id99b912e83052654a17d07dc92b4dcb958cb7600
Fixes: #4264
Altered the Oracle dialect such that when an :class:`.Integer` type is in
use, the cx_Oracle.NUMERIC type is set up for setinputsizes(). In
SQLAlchemy 1.1 and earlier, cx_Oracle.NUMERIC was passed for all numeric
types unconditionally, and in 1.2 this was removed to allow for better
numeric precision. However, for integers, some database/client setups
will fail to coerce boolean values True/False into integers which introduces
regressive behavior when using SQLAlchemy 1.2. Overall, the setinputsizes
logic seems like it will need a lot more flexibility going forward so this
is a start for that.
Change-Id: Ida80cc2c2c37ffc0e05da4b5df2dadfab55a01f2
Fixes: #4259
The ORM now doubles the "FOR UPDATE" clause within the subquery that
renders in conjunction with joined eager loading in some cases, as it has
been observed that MySQL does not lock the rows from a subquery. This
means the query renders with two FOR UPDATE clauses; note that on some
backends such as Oracle, FOR UPDATE clauses on subqueries are silently
ignored since they are unnecessary. Additionally, in the case of the "OF"
clause used primarily with Postgresql, the FOR UPDATE is rendered only on
the inner subquery when this is used so that the selectable can be targeted
to the table within the SELECT statement.
Change-Id: Ie5520d08d82bf0afd9e1bd2d43a0b2a0db0de16d
Fixes: #4246
Support added for the "WITH PARSER" syntax of CREATE FULLTEXT INDEX
in MySQL, using the ``mysql_with_parser`` keyword argument. Reflection
is also supported, which accommodates MySQL's special comment format
for reporting on this option as well. Additionally, the "FULLTEXT" and
"SPATIAL" index prefixes are now reflected back into the ``mysql_prefix``
index option.
Change-Id: I0209291978125d8cee1bb5ed386d4f66578697a0
Fixes: #4219
The Oracle NUMBER datatype is reflected as INTEGER if the precision is NULL
and the scale is zero, as this is how INTEGER values come back when
reflected from Oracle's tables. Pull request courtesy Kent Bower.
Change-Id: I4627febd46cab7085299c0a5700ee0f0bdca513c
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/437
Fixed bug where the special "not equals" operator for the Postgresql
"range" datatypes such as DATERANGE would fail to render "IS NOT NULL" when
compared to the Python ``None`` value.
Also break up range tests into backend round trip and straight
compilation suites.
Change-Id: Ibaee132b1ea7dac8b799495a27f98f82a7d9c028
Fixes: #4229
Added support for "PARTITION BY" in Postgresql table definitions,
using "postgresql_partition_by". Pull request courtesy
Vsevolod Solovyov.
Change-Id: Id74d6882d7193fae1e5fd44b6e12d6852866fcc4
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/430
Adjusted the SQL Server version detection for pyodbc to only allow for
numeric tokens, filtering out non-integers, since the dialect does tuple-
numeric comparisons with this value. This is normally true for all known
SQL Server / pyodbc drivers in any case.
Change-Id: I4ab18a07e19231091b5e877ba1fccd5eda72a992
Fixes: #4227
The minimum cx_Oracle version supported is 5.2 (June 2015). Previously,
the dialect asserted against version 5.0 but as of 1.2.2 we are using some
symbols that did not appear until 5.2.
Change-Id: I6fa4238f1722789924f4a6473fdce6f524333825
Fixes: #4211
Added support for :class:`.Enum` to persist the values of the enumeration,
rather than the keys, when using a Python pep-435 style enumerated object.
The user supplies a callable function that will return the string values to
be persisted. This allows enumerations against non-string values to be
value-persistable as well. Pull request courtesy Jon Snyder.
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/410Fixes: #3906
Change-Id: Id385465d215d1e5baaad68368b168afdd846b82c
For some reason the dialects were not consistently
converted to relative imports. Also added a test to
ensure that __all__ is functioning within each dialect.
Change-Id: I8450ed724473be7e17678e9aba7ba0f661fdf134
pyodbc 4.0.22 is no longer allowing a datetime to be truncated
into a date, and additionally is asserting that numeric truncation
is not occurring; previously, it looks like we could send a
decimal of -1E-25 through the driver but we were only getting
back -1E-20, the test failed to check this. Not clear if the
larger precision worked fully at some point, but in any case,
it doesn't work now so just remove those values from the test.
Change-Id: I66c7863b1708eb72f48173083b4ef78c93893b52
Fixed regression in 1.2 where newly repaired quoting
of collation names in 🎫`3785` breaks SQL Server,
which explicitly does not understand a quoted collation
name. Whether or not mixed-case collation names are
quoted or not is now deferred down to a dialect-level
decision so that each dialect can prepare these identifiers
directly.
Change-Id: Iaf0a8123d9bf4711219e320896bb28c5d2649304
Fixes: #4154
Mike Bayer