Added support for MySQL 8.0.1 + ``FOR SHARE`` to be emitted for the
:meth:`.Select.with_for_uddate` method, which offers compatibility with
``NOWAIT`` and ``SKIP LOCKED``. The new syntax is used only for MySQL when
version 8.0.1 or higher is detected. Pull request courtesy JetDrag.
Fixes: #10134Closes: #12964
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12964
Pull-request-sha: 03d5e37cfd
Change-Id: Iafb7a24363284edcfeead94a348f50a470a88403
Added a new concept of "operator classes" to the SQL operators supported by
SQLAlchemy, represented within the enum :class:`.OperatorClass`. The
purpose of this structure is to provide an extra layer of validation when a
particular kind of SQL operation is used with a particular datatype, to
catch early the use of an operator that does not have any relevance to the
datatype in use; a simple example is an integer or numeric column used with
a "string match" operator.
Fixes: #12736
Change-Id: I44f46d7326aef6847dbf0cf7a325833f8e347da6
Fixed yet another regression caused by by the DEFAULT rendering changes in
2.0.40 🎫`12425`, similar to 🎫`12488`, this time where using a
CURRENT_TIMESTAMP function with a fractional seconds portion inside a
textual default value would also fail to be recognized as a
non-parenthesized server default.
There's no way to do this other than start hardcoding a list
of MySQL functions that demand that parenthesis are not added around
them, I can think of no other heuristic that will work here.
Suggestions welcome
Fixes: #12648
Change-Id: I75d274b56306089929b369ecfb23604e9d6fa9dd
Fixed regression caused by the DEFAULT rendering changes in 2.0.40
🎫`12425` where using lowercase `on update` in a MySQL server default
would incorrectly apply parenthesis, leading to errors when MySQL
interpreted the rendered DDL. Pull request courtesy Alexander Ruehe.
Fixes: #12488Closes: #12489
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12489
Pull-request-sha: b9008f747d
Change-Id: If5281c52415e4ddb6c2f8aee191d2335f6673b35
Expanded the rules for when to apply parenthesis to a server default in DDL
to suit the general case of a default string that contains non-word
characters such as spaces or operators and is not a string literal.
Fixed issue in MySQL server default reflection where a default that has
spaces would not be correctly reflected. Additionally, expanded the rules
for when to apply parenthesis to a server default in DDL to suit the
general case of a default string that contains non-word characters such as
spaces or operators and is not a string literal.
Fixes: #12425
Change-Id: Ie40703dcd5fdc135025d676c01baba57ff3b71ad
Support has been re-added for the MySQL-Connector/Python DBAPI using the
``mysql+mysqlconnector://`` URL scheme. The DBAPI now works against
modern MySQL versions as well as MariaDB versions (in the latter case it's
required to pass charset/collation explicitly). Note however that
server side cursor support is disabled due to unresolved issues with this
driver.
References: #12332
Change-Id: I81279478196e830d3c0d5f24ecb3fe2dc18d4ca6
Added new construct :func:`_mysql.limit` which can be applied to any
:func:`_sql.update` or :func:`_sql.delete` to provide the LIMIT keyword to
UPDATE and DELETE. This new construct supersedes the use of the
"mysql_limit" dialect keyword argument.
Change-Id: Ie10c2f273432b0c8881a48f5b287f0566dde6ec3
Added support for the ``LIMIT`` clause with ``DELETE`` for the MySQL and
MariaDB dialects, to complement the already present option for
``UPDATE``. The :meth:`.delete.with_dialect_options` method of the
`:func:`.delete` construct accepts parameters for ``mysql_limit`` and
``mariadb_limit``, allowing users to specify a limit on the number of rows
deleted. Pull request courtesy of Pablo Nicolás Estevez.
Added logic to ensure that the ``mysql_limit`` and ``mariadb_limit``
parameters of :meth:`.update.with_dialect_options` and
:meth:`.delete.with_dialect_options` when compiled to string will only
compile if the parameter is passed as an integer; a ``ValueError`` is
raised otherwise.
corrected mysql documentation for update/delete options which
must be specified using the ``with_dialect_options()`` method.
Fixes: #11764Closes: #12146
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12146
Pull-request-sha: e34708374c
Change-Id: I8681ddabaa192b672c7a9b9981c4fe9e4bdc8d03
the :class:`.Numeric` and :class:`.Float` SQL types have been separated out
so that :class:`.Float` no longer inherits from :class:`.Numeric`; instead,
they both extend from a common mixin :class:`.NumericCommon`. This
corrects for some architectural shortcomings where numeric and float types
are typically separate, and establishes more consistency with
:class:`.Integer` also being a distinct type. The change should not have
any end-user implications except for code that may be using
``isinstance()`` to test for the :class:`.Numeric` datatype; third party
dialects which rely upon specific implementation types for numeric and/or
float may also require adjustment to maintain compatibility.
Fixes: #5252
Change-Id: Iadc841340b3d97e3eb5f7e63f0a0cc3cb4e30f74
Improved foreign keys reflection logic in MySQL 8+ to use a better
optimized query. The previous query could be quite slow in databases
with a large number of columns.
Fixes: #11975
Change-Id: Ie8bcd810d4b37abf7fd5e497596e0ade52c3f82e
Fixed a bug that caused a syntax error when a function was specified
to server_default when creating a column in MySQL or MariaDB.
Fixes#11317Closes: #11953
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11953
Pull-request-sha: d93ac419a9
Change-Id: I67fc83867df2b7dcf591c8f53b7a97afb90ebba9
Fixed issue in mariadbconnector dialect where query string arguments that
weren't checked integer or boolean arguments would be ignored, such as
string arguments like ``unix_socket``, etc. As part of this change, the
argument parsing for particular elements such as ``client_flags``,
``compress``, ``local_infile`` has been made more consistent across all
MySQL / MariaDB dialect which accept each argument. Pull request courtesy
Tobias Alex-Petersen.
Fixes: #11870Closes: #11869
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11869
Pull-request-sha: 8fdcabc83b
Change-Id: I3a11a0e65e118c94928027478409488b0d5e94f8
Fixed issue in MySQL dialect where using INSERT..FROM SELECT in combination
with ON DUPLICATE KEY UPDATE would erroneously render on MySQL 8 and above
the "AS new" clause, leading to syntax failures. This clause is required
on MySQL 8 to follow the VALUES clause if use of the "new" alias is
present, however is not permitted to follow a FROM SELECT clause.
Fixes: #11731
Change-Id: I254a3db4e9dccd9a76b11fdfe6e38a064ba0b5cf
Modified the MariaDB dialect so that when using the :class:`_sqltypes.Uuid`
datatype with MariaDB >= 10.7, leaving the
:paramref:`_sqltypes.Uuid.native_uuid` parameter at its default of True,
the native ``UUID`` datatype will be rendered in DDL and used for database
communication, rather than ``CHAR(32)`` (the non-native UUID type) as was
the case previously. This is a behavioral change since 2.0, where the
generic :class:`_sqltypes.Uuid` datatype delivered ``CHAR(32)`` for all
MySQL and MariaDB variants. Support for all major DBAPIs is implemented
including support for less common "insertmanyvalues" scenarios where UUID
values are generated in different ways for primary keys. Thanks much to
Volodymyr Kochetkov for delivering the PR.
To support this fully without hacks, the mariadb dialect now supports
driver-specific mariadb dialects as well, where we add one here for the
mysqlconnector DBAPI that doesn't accept Python UUID objects, whereas
all the other ones do.
Fixes: #10339Closes: #10849
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10849
Pull-request-sha: 8490b08713
Change-Id: Ib920871102b9b64f2cba9697f5cb72b6263e4ed8
manually update the files to remove literal string concat on the same line,
since black does not seem to be making progress in handling these
Change-Id: I3c651374c5f3db5b8bc0c700328d67ca03743b7b
Fixed issue where NULL/NOT NULL would not be properly reflected from a
MySQL column that also specified the VIRTUAL or STORED directives. Pull
request courtesy Georg Wicke-Arndt.
Fixes: #10850Closes: #10851
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10851
Pull-request-sha: fb9a81020c
Change-Id: I9a80d0db722c15682e18f0390a7b58e5979e73a1
mariaDB does not support setting NOT NULL for generated column
ref: https://mariadb.com/kb/en/generated-columns/#statement-support
added a check in `get_column_specification` for mariadb, checking
if user has not specified nullable set it as True for computed column,
but if user has explicitly set as False raise a compile error.
added testcase for same
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
**Have a nice day!**
Closes: #10354
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10354
Pull-request-sha: 68035e0f77
Change-Id: I00188cf899c0a2efe759e06e510243fbdb1a6dcf
Fixed issue where the :meth:`_sql.ColumnOperators.regexp_match`
when using "flags" would not produce a "stable" cache key, that
is, the cache key would keep changing each time causing cache pollution.
The same issue existed for :meth:`_sql.ColumnOperators.regexp_replace`
with both the flags and the actual replacement expression.
The flags are now represented as fixed modifier strings rendered as
safestrings rather than bound parameters, and the replacement
expression is established within the primary portion of the "binary"
element so that it generates an appropriate cache key.
Note that as part of this change, the
:paramref:`_sql.ColumnOperators.regexp_match.flags` and
:paramref:`_sql.ColumnOperators.regexp_replace.flags` have been modified to
render as literal strings only, whereas previously they were rendered as
full SQL expressions, typically bound parameters. These parameters should
always be passed as plain Python strings and not as SQL expression
constructs; it's not expected that SQL expression constructs were used in
practice for this parameter, so this is a backwards-incompatible change.
The change also modifies the internal structure of the expression
generated, for :meth:`_sql.ColumnOperators.regexp_replace` with or without
flags, and for :meth:`_sql.ColumnOperators.regexp_match` with flags. Third
party dialects which may have implemented regexp implementations of their
own (no such dialects could be located in a search, so impact is expected
to be low) would need to adjust the traversal of the structure to
accommodate.
Fixed issue in mostly-internal :class:`.CacheKey` construct where the
``__ne__()`` operator were not properly implemented, leading to nonsensical
results when comparing :class:`.CacheKey` instances to each other.
Fixes: #10042
Change-Id: I2e245f81d7ee7136ad04cf77be35f9745c5da5e5
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
Added the `"uuid"` key to `ischema_names` to enable lookup of `UUID` columns in reflection. There's also a new short test (MariaDB 10.7+ only) in the mysql reflection tests to verify if this feature works correctly.
Fixes#10028
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Closes: #10029
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10029
Pull-request-sha: 7c2838b1dd
Change-Id: Ifa1f51f472809b35374a7e2c6b85412046133452
fix a handful of warnings that were emitting but not raising,
usually because they were inside an "expect_warnings" block.
modify "expect_warnings" to always use "raise_on_any_unexpected"
behavior; remove this parameter.
Fixed issue in semi-private ``await_only()`` and ``await_fallback()``
concurrency functions where the given awaitable would remain un-awaited if
the function threw a ``GreenletError``, which could cause "was not awaited"
warnings later on if the program continued. In this case, the given
awaitable is now cancelled before the exception is thrown.
Change-Id: I33668c5e8c670454a3d879e559096fb873b57244
Fixed issues regarding reflection of comments for :class:`_schema.Table`
and :class:`_schema.Column` objects, where the comments contained control
characters such as newlines. Additional testing support for these
characters as well as extended Unicode characters in table and column
comments (the latter of which aren't supported by MySQL/MariaDB) added to
testing overall.
Fixes: #9722
Change-Id: Id18bf758fdb6231eb705c61eeaf74bb9fa472601
Fixed issue where string datatypes such as :class:`.CHAR`,
:class:`.VARCHAR`, :class:`.TEXT`, as well as binary :class:`.BLOB`, could
not be produced with an explicit length of zero, which has special meaning
for MySQL. Pull request courtesy J. Nick Koston.
Fixes: #9544Closes: #9543
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9543
Pull-request-sha: dc17fc3e93
Change-Id: I96925d45f16887f5dfd68a5d4f9284b3abc46d25
asyncmy 0.2.7 has had a loss in float precision for even
very low numbers of significant digits.
Change-Id: Iec6d2650943eeaa8e854f21990f6565d73331f8c
References: https://github.com/long2ice/asyncmy/issues/56
Added support for MySQL 8's new ``AS <name> ON DUPLICATE KEY`` syntax when
using :meth:`_mysql.Insert.on_duplicate_key_update`, which is required for
newer versions of MySQL 8 as the previous syntax using ``VALUES()`` now
emits a deprecation warning with those versions. Server version detection
is employed to determine if traditional MariaDB / MySQL < 8 ``VALUES()``
syntax should be used, vs. the newer MySQL 8 required syntax. Pull request
courtesy Caspar Wylie.
Fixes: #8626Closes: #9210
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9210
Pull-request-sha: 1c8dfbf0b4
Change-Id: I42c463837af06bc15b60c534159804193df07f02
Added support to MySQL index reflection to correctly reflect the
``mysql_length`` dictionary, which previously was being ignored.
Fixes: #9047
Change-Id: I0a5e27123be68741e12af4464a0fa305052ec36e
command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>"
pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not
exists in sqlalchemy fixtures
Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
Adjusted the regular expression used to match "CREATE VIEW" when
testing for views to work more flexibly, no longer requiring the
special keyword "ALGORITHM" in the middle, which was intended to be
optional but was not working correctly. The change allows view reflection
to work more completely on MySQL-compatible variants such as StarRocks.
Pull request courtesy John Bodley.
Fixes: #8588Closes: #8589
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8589
Pull-request-sha: d85b2c5b51
Change-Id: I173137f0bf68639cad0d5c329055475b40ddb5e4
The ``ROLLUP`` function will now correctly render ``WITH ROLLUP`` on
MySql and MariaDB, allowing the use of group by rollup with these
backend.
Fixes: #8503
Change-Id: I9289af3a39ca667a2f0f84f73346ebd4b091fedd
Add support for Partitioning and Sample pages on MySQL and MariaDB
reflected options.
The options are stored in the table dialect options dictionary, so
the following keyword need to be prefixed with ``mysql_`` or ``mariadb_``
depending on the backend.
Supported options are:
* ``stats_sample_pages``
* ``partition_by``
* ``partitions``
* ``subpartition_by``
These options are also reflected when loading a table from database,
and will populate the table :attr:`_schema.Table.dialect_options`.
Pull request courtesy of Ramon Will.
Fixes: #4038Closes: #5536
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5536
Pull-request-sha: f8852cabe1
Change-Id: I69b60576532af04c725c998e9e8fec6e2040b149
just in my own testing, if I say insert().return_defaults()
and stringify, I should see it, so make sure all the dialects
default to "insert_returning" etc. , with downgrade on
server version check.
Change-Id: Id64e78fcb03c48b5dcb0feb21cb9cc495edd15e9
Rearchitected the schema reflection API to allow some dialects to make use
of high performing batch queries to reflect the schemas of many tables at
once using much fewer queries. The new performance features are targeted
first at the PostgreSQL and Oracle backends, and may be applied to any
dialect that makes use of SELECT queries against system catalog tables to
reflect tables (currently this omits the MySQL and SQLite dialects which
instead make use of parsing the "CREATE TABLE" statement, however these
dialects do not have a pre-existing performance issue with reflection. MS
SQL Server is still a TODO).
The new API is backwards compatible with the previous system, and should
require no changes to third party dialects to retain compatibility;
third party dialects can also opt into the new system by implementing
batched queries for schema reflection.
Along with this change is an updated reflection API that is fully
:pep:`484` typed, features many new methods and some changes.
Fixes: #4379
Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08
Added disconnect code for MySQL error 4031, introduced in MySQL >= 8.0.24,
indicating connection idle timeout exceeded. In particular this repairs an
issue where pre-ping could not reconnect on a timed-out connection. Pull
request courtesy valievkarim.
Fixes: #8036Closes: #8037
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8037
Pull-request-sha: 7ab605c2d2
Change-Id: I21249c9d8acb305ac43ce61b90b41daf7fabdfe8
Improved the construction of SQL binary expressions to allow for very long
expressions against the same associative operator without special steps
needed in order to avoid high memory use and excess recursion depth. A
particular binary operation ``A op B`` can now be joined against another
element ``op C`` and the resulting structure will be "flattened" so that
the representation as well as SQL compilation does not require recursion.
To implement this more cleanly, the biggest change here is that
column-oriented lists of things are broken away from ClauseList
in a new class ExpressionClauseList, that also forms the basis
of BooleanClauseList. ClauseList is still used for the generic
"comma-separated list" of things such as Tuple and things like
ORDER BY, as well as in some API endpoints.
Also adds __slots__ to the TypeEngine-bound Comparator
classes. Still can't really do __slots__ on ClauseElement.
Fixes: #7744
Change-Id: I81a8ceb6f8f3bb0fe52d58f3cb42e4b6c2bc9018
both black and click were released in the past
few hours, and black 21.5b1 seems to suddenly
be failing on a missing symbol from click. just
update to the latest
Change-Id: Idf76732479a264f7f2245699a6bdaff018e3a123
Fixed issues in :class:`_mysql.SET` datatype as well as :class:`.Enum`
where the ``__repr__()`` method would not render all optional parameters in
the string output, impacting the use of these types in Alembic
autogenerate. Pull request for MySQL courtesy Yuki Nishimine.
Fixes: #7720Fixes: #7789Closes: #7772
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7772
Pull-request-sha: d58845479f
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Change-Id: Idcec23eab4258511d9f32f4e3d78e511ea6021f1
Added :class:`.Double`, :class:`.DOUBLE`, :class:`.DOUBLE_PRECISION`
datatypes to the base ``sqlalchemy.`` module namespace, for explicit use of
double/double precision as well as generic "double" datatypes. Use
:class:`.Double` for generic support that will resolve to DOUBLE/DOUBLE
PRECISION/FLOAT as needed for different backends.
Implemented DDL and reflection support for ``FLOAT`` datatypes which
include an explicit "binary_precision" value. Using the Oracle-specific
:class:`_oracle.FLOAT` datatype, the new parameter
:paramref:`_oracle.FLOAT.binary_precision` may be specified which will
render Oracle's precision for floating point types directly. This value is
interpreted during reflection. Upon reflecting back a ``FLOAT`` datatype,
the datatype returned is one of :class:`_types.DOUBLE_PRECISION` for a
``FLOAT`` for a precision of 126 (this is also Oracle's default precision
for ``FLOAT``), :class:`_types.REAL` for a precision of 63, and
:class:`_oracle.FLOAT` for a custom precision, as per Oracle documentation.
As part of this change, the generic :paramref:`_sqltypes.Float.precision`
value is explicitly rejected when generating DDL for Oracle, as this
precision cannot be accurately converted to "binary precision"; instead, an
error message encourages the use of
:meth:`_sqltypes.TypeEngine.with_variant` so that Oracle's specific form of
precision may be chosen exactly. This is a backwards-incompatible change in
behavior, as the previous "precision" value was silently ignored for
Oracle.
Fixes: #5465Closes: #7674
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7674
Pull-request-sha: 5c68419e5a
Change-Id: I831f4af3ee3b23fde02e8f6393c83e23dd7cd34d
Fixed regression caused by 🎫`7518` where changing the syntax "SHOW
VARIABLES" to "SELECT @@" broke compatibility with MySQL versions older
than 5.6, including early 5.0 releases. While these are very old MySQL
versions, a change in compatibility was not planned, so version-specific
logic has been restored to fall back to "SHOW VARIABLES" for MySQL server
versions < 5.6.
includes unrelated orm/test_expire ordering issue , only showing
up on 1.4 / py2.7 but seems to be passing by luck otherwise
Fixes: #7518
Change-Id: Ia554080af742f2c3437f88cf3f7a4827b5e55da8
Finalize all remaining removed-in-2.0 changes so that we
can begin doing pep-484 typing without old things
getting in the way (we will also have to do public_factory).
note there are a few "moved_in_20()" and "became_legacy_in_20()"
warnings still in place. The SQLALCHEMY_WARN_20 variable
is now removed.
Also removed here are the legacy "in place mutators" for Select
statements, and some keyword-only argument signatures in Core
have been added.
Also in the big change department, the ORM mapper() function
is removed entirely; the Mapper class is otherwise unchanged,
just the public-facing API function. Mappers are now always
given a registry in which to participate, however the
argument signature of Mapper is not changed. ideally "registry"
would be the first positional argument.
Fixes: #7257
Change-Id: Ic70c57b9f1cf7eb996338af5183b11bdeb3e1623
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
Black's `target-version` was still set to `['py27', 'py36']`. Set it to `[py37]` instead.
Also update Black and other pre-commit hooks and re-format with Black.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Closes: #7536
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7536
Pull-request-sha: b3aedf5570
Change-Id: I8be85636fd2c9449b07a8626050c8bd35bd119d5
JetDrag