Enhanced MySQL/MariaDB version string parsing to accommodate for exotic
MariaDB version strings where the "MariaDB" word is embedded among other
alphanumeric characters such as "MariaDBV1". This detection is critical in
order to correctly accomodate for API features that have split between MySQL
and MariaDB such as the "transaction_isolation" system variable.
Fixes: #4624
Change-Id: Iba4b56535855629a974b1e24e012b07383d24199
Fixed regression from release 1.3.2 caused by 🎫`4562` where a URL
that contained only a query string and no hostname, such as for the
purposes of specifying a service file with connection information, would no
longer be propagated to psycopg2 properly. The change in 🎫`4562`
has been adjusted to further suit psycopg2's exact requirements, which is
that if there are any connection parameters whatsoever, the "dsn" parameter
is no longer required, so in this case the query string parameters are
passed alone.
Fixes: #4601
Change-Id: Ic29a8b77bcf50ee996968bab25aaac7ae4bfc26f
Fixed issue in SQL Server dialect where if a bound parameter were present in
an ORDER BY expression that would ultimately not be rendered in the SQL
Server version of the statement, the parameters would still be part of the
execution parameters, leading to DBAPI-level errors. Pull request courtesy
Matt Lewellyn.
Fixes: #4587Closes: #4588
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4588
Pull-request-sha: 2992a473e0
Change-Id: Ie709aefdb1babf810bb81526289448f8cc7a4cb1
Added support for parameter-less connection URLs for the psycopg2 dialect,
meaning, the URL can be passed to :func:`.create_engine` as
``"postgresql+psycopg2://"`` with no additional arguments to indicate an
empty DSN passed to libpq, which indicates to connect to "localhost" with
no username, password, or database given. Pull request courtesy Julian
Mehnle.
Fixes: #4562Closes: #4563
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4563
Pull-request-sha: 8a05c96944
Change-Id: Ib6fca3c3c9eebeaf590d7f7fb0bc8cd4b6e4a55a
Modified the :paramref:`.Select.with_for_update.of` parameter so that if a
join or other composed selectable is passed, the individual :class:`.Table`
objects will be filtered from it, allowing one to pass a join() object to
the parameter, as occurs normally when using joined table inheritance with
the ORM. Pull request courtesy Raymond Lu.
Fixes: #4550
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Closes: #4551
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4551
Pull-request-sha: 452da77d15
Change-Id: If4b7c231f7b71190d7245543959fb5c3351125a1
A commit() is emitted after an isolation level change to SNAPSHOT, as both
pyodbc and pymssql open an implicit transaction which blocks subsequent SQL
from being emitted in the current transaction.
Fixes: #4536
Change-Id: If3ba70f495bce2a35a873a3a72d1b30406e678c8
Fixed regression in SQL Server reflection due to 🎫`4393` where the
removal of open-ended ``**kw`` from the :class:`.Float` datatype caused
reflection of this type to fail due to a "scale" argument being passed.
Fixes: #4525
Change-Id: Ief8bb535778055eff2ab0d71660f81e3676390a1
For very slow CI this test still fails so establish the date
as being between the range of what the operation took
Change-Id: I9204cd1915777ebc58bd598065b9015befb93345
Added support for reflection of the :class:`.NCHAR` datatype to the Oracle
dialect, and added :class:`.NCHAR` to the list of types exported by the
Oracle dialect.
Fixes: #4506
Change-Id: I3a120b9ad714cbd0ebd8064519f43cf57e99d920
Fixed bug where the SQL Server "IDENTITY_INSERT" logic that allows an INSERT
to proceed with an explicit value on an IDENTITY column was not detecting
the case where :meth:`.Insert.values` were used with a dictionary that
contained a :class:`.Column` as key and a SQL expression as a value.
Fixes: #4499
Change-Id: Ia61cd6524b030b40a665db9c20771f0c5aa5fcd7
Fixed a second regression caused by 🎫`4344` (the first was
🎫`4361`), which works around MySQL issue 88718, where the lower
casing function used was not correct for Python 2 with OSX/Windows casing
conventions, which would then raise ``TypeError``. Full coverage has been
added to this logic so that every codepath is exercised in a mock style for
all three casing conventions on all versions of Python. MySQL 8.0 has
meanwhile fixed issue 88718 so the workaround is only applies to a
particular span of MySQL 8.0 versions.
Fixes: #4492
Change-Id: I14e7237e0be4a9c21c58c921066304ae99ac4dc6
Fully removed the behavior of strings passed directly as components of a
:func:`.select` or :class:`.Query` object being coerced to :func:`.text`
constructs automatically; the warning that has been emitted is now an
ArgumentError or in the case of order_by() / group_by() a CompileError.
This has emitted a warning since version 1.0 however its presence continues
to create concerns for the potential of mis-use of this behavior.
Note that public CVEs have been posted for order_by() / group_by() which
are resolved by this commit: CVE-2019-7164 CVE-2019-7548
Added "SQL phrase validation" to key DDL phrases that are accepted as plain
strings, including :paramref:`.ForeignKeyConstraint.on_delete`,
:paramref:`.ForeignKeyConstraint.on_update`,
:paramref:`.ExcludeConstraint.using`,
:paramref:`.ForeignKeyConstraint.initially`, for areas where a series of SQL
keywords only are expected.Any non-space characters that suggest the phrase
would need to be quoted will raise a :class:`.CompileError`. This change
is related to the series of changes committed as part of 🎫`4481`.
Fixed issue where using an uppercase name for an index type (e.g. GIST,
BTREE, etc. ) or an EXCLUDE constraint would treat it as an identifier to
be quoted, rather than rendering it as is. The new behavior converts these
types to lowercase and ensures they contain only valid SQL characters.
Quoting is applied to :class:`.Function` names, those which are usually but
not necessarily generated from the :attr:`.sql.func` construct, at compile
time if they contain illegal characters, such as spaces or punctuation. The
names are as before treated as case insensitive however, meaning if the
names contain uppercase or mixed case characters, that alone does not
trigger quoting. The case insensitivity is currently maintained for
backwards compatibility.
Fixes: #4481Fixes: #4473Fixes: #4467
Change-Id: Ib22a27d62930e24702e2f0f7c74a0473385a08eb
Fixed bug in SQLite DDL where using an expression as a server side default
required that it be contained within parenthesis to be accepted by the
sqlite parser. Pull request courtesy Bartlomiej Biernacki.
Fixes: #4474Closes: #4475
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4475
Pull-request-sha: 0af3238c69
Change-Id: I888c69bfba5a5ab8ffa420f512557ac311a36b31
(cherry picked from commit f7ce37e9dd493d4b60b3156f08ef4c98fd63904d)
Revised the query used when reflecting CHECK constraints to make use of the
``pg_get_constraintdef`` function, as the ``consrc`` column is being
deprecated in PG 12. Thanks to John A Stevenson for the tip.
Fixes: #4463
Change-Id: Ie0ee9bdfddb0635db72b35c2e2e4b27f154162fd
A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.
See the notes added to the changelog and migration notes for further
details.
Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
Fixed regression in integer precision logic due to the refactor of the
cx_Oracle dialect in 1.2. We now no longer apply the cx_Oracle.NATIVE_INT
type to result columns sending integer values (detected as positive
precision with scale ==0) which encounters integer overflow issues with
values that go beyond the 32 bit boundary. Instead, the output variable
is left untyped so that cx_Oracle can choose the best option.
Fixes: #4457
Change-Id: I1e3114c2f37bf028fb1f521a3e9789a77e3a7491
The ``literal_processor`` for the :class:`.Unicode` and
:class:`.UnicodeText` datatypes now render an ``N`` character in front of
the literal string expression as required by SQL Server for Unicode string
values rendered in SQL expressions.
Note that this adds full unicode characters to the standard test suite,
which means we also need to bump MySQL provisioning up to utf8mb4.
Modern installs do not seem to be reproducing the 1271 issue locally,
if it reproduces in CI it would be better for us to skip those ORM-centric
tests for MySQL.
Also remove unused _StringType from SQL Server dialect
Fixes: #4442
Change-Id: Id55817b3e8a2d81ddc8b7b27f85e3f1dcc1cea7e
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
Fixed issue where a :class:`.postgresql.ENUM` or a custom domain present
in a remote schema would not be recognized within column reflection if
the name of the enum/domain or the name of the schema required quoting.
A new parsing scheme now fully parses out quoted or non-quoted tokens
including support for SQL-escaped quotes.
Fixed issue where multiple :class:`.postgresql.ENUM` objects referred to
by the same :class:`.MetaData` object would fail to be created if
multiple objects had the same name under different schema names. The
internal memoization the Postgresql dialect uses to track if it has
created a particular :class:`.postgresql.ENUM` in the database during
a DDL creation sequence now takes schema name into account.
Fixes: #4416
Change-Id: I8cf03069e10b12f409e9b6796e24fc5850979955
test_compiler is mostly related to SELECT statements as well
as smaller SQL elements. While it still has some DDL related
tests, move out all the remaining insert/update tests into
the already present test_insert.py, test_update.py
Fixes: #2630
Change-Id: I4167618543fd1235d12d1717c8c629d2374b325a
Fixed issue where reflection of a PostgreSQL domain that is expressed as an
array would fail to be recognized. Pull request courtesy Jakub Synowiec.
Fixes: #4377
Change-Id: I252c79ca435b87d4d9172b1c84e0e74e789ef676
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4380
As MySQLdb support read_timeout and write_timeout option, and
sqlalchemy just support read_timeout option. So sqlalchemy need to add
write_timeout option.
Fixes: #4381
Change-Id: I2bea80bdd6f20fafc72b48fa0e5d795626d9d9b9
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4385
Updated the parameters that can be sent to the cx_Oracle DBAPI to both allow
for all current parameters as well as for future parameters not added yet.
In addition, removed unused parameters that were deprecated in version 1.2,
and additionally we are now defaulting "threaded" to False.
Fixes: #4369
Change-Id: I599668960e7b2d5bd1f5e6850e10b5b3ec215ed3
Deprecated the use of :class:`.Sequence` with SQL Server in order to affect
the "start" and "increment" of the IDENTITY value, in favor of new
parameters ``mssql_identity_start`` and ``mssql_identity_increment`` which
set these parameters directly. :class:`.Sequence` will be used to generate
real ``CREATE SEQUENCE`` DDL with SQL Server in a future release.
Fixes: #4362
Change-Id: I1e69378c5c960ff0bc28137c923589692f1a918f
Fixed regression caused by 🎫`4344` released in 1.2.13, where the fix
for MySQL 8.0's case sensitivity problem with referenced column names when
reflecting foreign key referents is worked around using the
``information_schema.columns`` view. The workaround was failing on OSX /
``lower_case_table_names=2`` which produces non-matching casing for the
``information_schema.columns`` vs. that of ``SHOW CREATE TABLE``, so in
case-insensitive SQL modes case-insensitive matching is now used.
Fixes: #4361
Change-Id: I748549bc4c27fad6394593f8ec93fc22bfd01f6c
Implemented the SQLite ``ON CONFLICT`` clause as understood at the DDL
level, e.g. for primary key, unique, and CHECK constraints as well as
specified on a :class:`.Column` to satisfy inline primary key and NOT NULL.
Pull request courtesy Denis Kataev.
Fixes: #4360
Change-Id: I4cd4bafa8fca41e3101c87dbbfe169741bbda3f4
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/431
Added a workaround for a MySQL bug #88718 introduced in the 8.0 series,
where the reflection of a foreign key constraint is not reporting the
correct case sensitivity for the referred column, leading to errors during
use of the reflected constraint such as when using the automap extension.
The workaround emits an additional query to the information_schema tables in
order to retrieve the correct case sensitive name.
Fixes: #4344
Change-Id: I08020d6eec43cbe8a56316660380d3739a0b45f7
Added rudimental support for reflection of Postgresql
partitioned tables, e.g. that relkind='p' is added to reflection
queries that return table information.
Fixes: #4237
Change-Id: I66fd10b002e4ed21ea13b13a7e35a85f66bdea75
Added support for the :class:`.aggregate_order_by` function to receive
multiple ORDER BY elements, previously only a single element was accepted.
Fixes: #4337
Change-Id: I411ac31697a0d65b568ad65ce5b5181717afbd65
The Oracle dialect will no longer use the NCHAR/NCLOB datatypes to
represent generic unicode strings or clob fields in conjunction with
:class:`.Unicode` and :class:`.UnicodeText` unless the flag
``use_nchar_for_unicode=True`` is passed to :func:`.create_engine`.
Additionally, string types under Oracle now coerce to unicode under Python
2 in all cases, however unlike in previous iterations, we use SQLAlchemy's
native unicode handlers which are very performant (when C extensions are
enabled; when they are not, we use cx_Oracle's handlers).
Change-Id: I3939012e9396520875bc52e69bf81f27393836ee
Fixes: #4242
Fixed bug in PostgreSQL dialect where compiler keyword arguments such as
``literal_binds=True`` were not being propagated to a DISTINCT ON
expression.
Fixes: #4325
Change-Id: I9949387dceb7fabe889799f42e92423572368b29
Fixed the :func:`.postgresql.array_agg` function, which is a slightly
altered version of the usual :func:`.functions.array_agg` function, to also
accept an incoming "type" argument without forcing an ARRAY around it,
essentially the same thing that was fixed for the generic function in 1.1
in 🎫`4107`.
Fixes: #4324
Change-Id: I399a29f59c945a217cdd22c65ff0325edea8ea65
Fixed bug in PostgreSQL ENUM reflection where a case-sensitive, quoted name
would be reported by the query including quotes, which would not match a
target column during table reflection as the quotes needed to be stripped
off.
Fixes: #4323
Change-Id: I668f3acccc578e58f23b70c82d31d5c1ec194913
Added support for the parameters in an ON DUPLICATE KEY UPDATE statement on
MySQL to be ordered, since parameter order in a MySQL UPDATE clause is
significant, in a similar manner as that described at
:ref:`updates_order_parameters`. Pull request courtesy Maxim Bublis.
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/462
Change-Id: If508d8e26dbd3c55ab1e83cf573fb4021e9d091e
For cx_Oracle, Integer datatypes will now be bound to "int", per advice
from the cx_Oracle developers. Previously, using cx_Oracle.NUMBER caused a
loss in precision within the cx_Oracle 6.x series.
Change-Id: I4c6b2cca490aff5b98b7ceff3414715202881c89
Fixes: #4309
Fixed issue where the "schema" name used for a SQLite database within table
reflection would not quote the schema name correctly. Pull request
courtesy Phillip Cloud.
Change-Id: I2770788c1f094a7743209250ec26b5ef5fb2d9e8
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/463
Added ``fast_executemany=True`` parameter to the SQL Server pyodbc dialect,
which enables use of pyodbc's new performance feature of the same name
when using Microsoft ODBC drivers.
Change-Id: I743fa7280e8f709addd330cfc7682623701cbb2e
Fixes: #4158
Added support for SQLite's json functionality via the new
SQLite implementation for :class:`.sqltypes.JSON`, :class:`.sqlite.JSON`.
The name used for the type is ``JSON``, following an example found at
SQLite's own documentation. Pull request courtesy Ilja Everilä.
Fixes: #3850
Change-Id: I3d2714fb8655343a99d13dc751b16b93d05d7dda
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/434
Fixed bug in MSSQL reflection where when two same-named tables in different
schemas had same-named primary key constraints, foreign key constraints
referring to one of the tables would have their columns doubled, causing
errors. Pull request courtesy Sean Dunn.
Fixes: #4228
Change-Id: I7dabaaee0944e1030048826ba39fc574b0d63031
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/457
Fixed bug in index reflection where on MySQL 8.0 an index that includes
ASC or DESC in an indexed column specfication would not be correctly
reflected, as MySQL 8.0 introduces support for returning this information
in a table definition string.
Change-Id: I21f64984ade690aac8c87dbe3aad0c1ee8e9727f
Fixes: #4293
Add CI support for MySQL connector and try to fix some of the
more obvious issues. CI tests will run against MySQL 5.7
only for starters as there appear to be issues with
MySQL 8.0
Change-Id: Id8971143a8385a5c84f0646c21c4c21e793ce3a2
Added a new event currently used only by the cx_Oracle dialect,
:meth:`.DialectEvents.setiputsizes`. The event passes a dictionary of
:class:`.BindParameter` objects to DBAPI-specific type objects that will be
passed, after conversion to parameter names, to the cx_Oracle
``cursor.setinputsizes()`` method. This allows both visibility into the
setinputsizes process as well as the ability to alter the behavior of what
datatypes are passed to this method.
Change-Id: I43b97c8e3c840cad6f01edb274dc9cfed19cb5fc
Fixes: #4290
mollardthomas