Fixed issue where the :class:`.JSON` type had a read-only
:attr:`.JSON.should_evaluate_none` attribute, which would cause failures
when making use of the :meth:`.TypeEngine.evaluates_none` method in
conjunction with this type. Pull request courtesy Sanjana S.
Fixes: #4485Closes: #4496
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4496
Pull-request-sha: 044beb2398
Change-Id: I1f3e1d7dec9d2ceb6ccaaa8cac158a062cf02710
(cherry picked from commit 12dad561a7)
This affects mostly docstrings, except in orm/events.py::dispose_collection()
where one parameter gets renamed: given that the method is
empty, it seemed reasonable to me to fix that too.
Closes: #4440
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4440
Pull-request-sha: 779ed75acb
Change-Id: Ic0553fe97853054b09c2453af76d96363de6eb0e
(cherry picked from commit 66e88d30a8)
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
Fixes to the test suite, a few errant imports, and setup.py:
- mysql and postgresql have unused 'json' imports; remove
- postgresql is exporting the 'json' symbol, remove
- make sure setup.py can find __version__ using " or '
- retry logic in provision create database for postgresql fixed
- refactor test_magazine to use cls.tables rather than globals
- remove unused class in test_scoping
- add a comment to test_deprecations that this test suite itself
is deprecated
- don't use mapper() and orm_mapper() in test_unitofwork, just
use mapper()
- remove dupe test_scalar_set_None test in test_attributes
- Python 2.7 and above includes unittest.SkipTest, remove pre-2.7
fallback
- use imported SkipTest in profiling
- declarative test_reflection tests with "reflectable_autoincrement"
already don't run on oracle or firebird; remove conditional logic
for these, which also removes an "id" symbol
- clean up test in test_functions, remove print statement
- remove dupe test_literal_processor_coercion_native_int_out_of_range
in test/sql/test_types.py
- fix psycopg2_hstore ref
Change-Id: I7b3444f8546aac82be81cd1e7b6d8b2ad6834fe6
(cherry picked from commit 404e69426b)
a few code changes ahead of time to handle some __all__
issues better. also include new flake8 rules, since the
existing flake8 doesn't pass in any case.
Change-Id: I1efdf75124ae7bcac719c22e505bb5b13db06c04
(cherry picked from commit d229360a8d)
Fixed issue in "expanding IN" feature where using the same bound parameter
name more than once in a query would lead to a KeyError within the process
of rewriting the parameters in the query.
Fixes: #4394
Change-Id: Ibcadce9fefbcb060266d9447c2044ee6efeccf5a
(cherry picked from commit c495769751)
test_compiler is mostly related to SELECT statements as well
as smaller SQL elements. While it still has some DDL related
tests, move out all the remaining insert/update tests into
the already present test_insert.py, test_update.py
Fixes: #2630
Change-Id: I4167618543fd1235d12d1717c8c629d2374b325a
(cherry picked from commit 87cdda0086)
this test was using sysdate() and current_timestamp() together
in conjunction with a truncation to DAY, however for four hours
on saturday night (see commit time :) ) these two values will
have a different value if one side is EDT and the other is UTC.
tox does not transmit environment variables including TZ by
default, so even if the server is set up for EDT, running tox
will not set TZ and at least Oracle client seems to use this
value, producing UTC for session time but the database on CI
was configured for EDT, producing EDT for sysdate.
Change-Id: I56602d2402a475a0c4fdf61c1c5fc2618c82f915
(cherry picked from commit ac358a04a7)
Fixed additional warnings generated by Python 3.7 due to changes in the
organization of the Python ``collections`` and ``collections.abc`` packages.
Previous ``collections`` warnings were fixed in version 1.2.11. Pull request
courtesy xtreak.
See I2d1c0ef97c8ecac7af152cc56263422a40faa6bb for the original collections.abc
fixes.
Fixes: #4339
Change-Id: Ia92d2461f20309fb33ea6c6f592f7d4e7e32ae7a
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/475
(cherry picked from commit 2d2fa49130)
Fixed bug where the :paramref:`.Enum.create_constraint` flag on the
:class:`.Enum` datatype would not be propagated to copies of the type, which
affects use cases such as declarative mixins and abstract bases.
Fixes: #4341
Change-Id: I978be65f33a616fe4d5f5de03fb3eaab6f6a2272
(cherry picked from commit 0737f45d4f)
Refactored :class:`.SQLCompiler` to expose a
:meth:`.SQLCompiler.group_by_clause` method similar to the
:meth:`.SQLCompiler.order_by_clause` and :meth:`.SQLCompiler.limit_clause`
methods, which can be overridden by dialects to customize how GROUP BY
renders. Pull request courtesy Samuel Chou.
Change-Id: I0a7238e55032558c27a0c56a72907c7b883456f1
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/474
(cherry picked from commit 33fccc4861)
MariaDB seems to handle some additional UPDATE/DELETE FROM
syntaxes as well as some forms of INTERSECT and EXCEPT. Open
up tests that expect failure for MySQL to allow success for
MariaDB 10.3.
Change-Id: Ia9341a82485ef7201bb8130d8dbf4a9b6976035a
(cherry picked from commit 081d4275cf)
Added missing window function parameters
:paramref:`.WithinGroup.over.range_` and :paramref:`.WithinGroup.over.rows`
parameters to the :meth:`.WithinGroup.over` and
:meth:`.FunctionFilter.over` methods, to correspond to the range/rows
feature added to the "over" method of SQL functions as part of
🎫`3049` in version 1.1.
Fixes: #4322
Change-Id: I77dcdac65c699a4b52a3fc3ee09a100ffb4fc20e
(cherry picked from commit 3e2f61c439)
Fixed issue that is closely related to 🎫`3639` where an expression
rendered in a boolean context on a non-native boolean backend would
be compared to 1/0 even though it is already an implcitly boolean
expression, when :meth:`.ColumnElement.self_group` were used. While this
does not affect the user-friendly backends (MySQL, SQLite) it was not
handled by Oracle (and possibly SQL Server). Whether or not the
expression is implicitly boolean on any database is now determined
up front as an additional check to not generate the integer comparison
within the compliation of the statement.
Fixes: #4320
Change-Id: Iae0a65e5c01bd576e64733c3651e1e1a1a1b240c
(cherry picked from commit 462ccd9ff1)
Fixed bug where the multi-table support for UPDATE and DELETE statements
did not consider the additional FROM elements as targets for correlation,
when a correlated SELECT were also combined with the statement. This
change now includes that a SELECT statement in the WHERE clause for such a
statement will try to auto-correlate back to these additional tables in the
parent UPDATE/DELETE or unconditionally correlate if
:meth:`.Select.correlate` is used. Note that auto-correlation raises an
error if the SELECT statement would have no FROM clauses as a result, which
can now occur if the parent UPDATE/DELETE specifies the same tables in its
additional set of tables ; specify :meth:`.Select.correlate` explicitly to
resolve.
Change-Id: Ie11eaad7e49af3f59df11691b104d6359341bdae
Fixes: #4313
(cherry picked from commit abeea1d82d)
Started importing "collections" from "collections.abc" under Python 3.3 and
greater for Python 3.8 compatibility. Pull request courtesy Nathaniel
Knight.
In Python 3.3, the abstract base classes (Iterable, Mapping, etc.)
were moved from the `collections` module and put in the
`collections.abc` module. They remain in the `collections` module for
backwards compatibility, and will until Python 3.8.
This commit adds a variable (`collections_abc`) to the `util/compat`
module, which will be the `collections` module for Python < 3.3 and
before, or the `collections.abc` module for Python >= 3.3. It also
uses the new variable, getting rid of some deprecation warnings that
were seen when running under Python 3.7.
Change-Id: I2d1c0ef97c8ecac7af152cc56263422a40faa6bb
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/464
(cherry picked from commit a59c3b8f302a34ab037ec445b7452b1f353b91af)
Fixed bug where a :class:`.Sequence` would be dropped explicitly before any
:class:`.Table` that refers to it, which breaks in the case when the
sequence is also involved in a server-side default for that table, when
using :meth:`.MetaData.drop_all`. The step which processes sequences
to be dropped via non server-side column default functions is now invoked
after the table itself is dropped.
Change-Id: I185f2cc76d2011ad4dd3ba9bde5d8aef0ec335ae
Fixes: #4300
(cherry picked from commit 532566ba1f)
Fixed bug in MySQLdb dialect and variants such as PyMySQL where an
additional "unicode returns" check upon connection makes explicit use of
the "utf8" character set, which in MySQL 8.0 emits a warning that utf8mb4
should be used. This is now replaced with a utf8mb4 equivalent.
Documentation is also updated for the MySQL dialect to specify utf8mb4 in
all examples. Additional changes have been made to the test suite to use
utf8mb3 charsets and databases (there seem to be collation issues in some
edge cases with utf8mb4), and to support configuration default changes made
in MySQL 8.0 such as explicit_defaults_for_timestamp as well as new errors
raised for invalid MyISAM indexes.
Change-Id: Ib596ea7de4f69f976872a33bffa4c902d17dea25
Fixes: #4283Fixes: #4192
(cherry picked from commit c99345ee99)
The :class:`.Update` construct now accommodates a :class:`.Join` object
as supported by MySQL for UPDATE..FROM. As the construct already
accepted an alias object for a similar purpose, the feature of UPDATE
against a non-table was already implied so this has been added.
Change-Id: I7b2bca627849384d5377abb0c94626463e4fad04
Fixes: #3645
(cherry picked from commit 58540ae93d)
Fixed INSERT FROM SELECT with CTEs for the Oracle and MySQL dialects, where
the CTE was being placed above the entire statement as is typical with
other databases, however Oracle and MariaDB 10.2 wants the CTE underneath
the "INSERT" segment. Note that the Oracle and MySQL dialects don't yet
work when a CTE is applied to a subquery inside of an UPDATE or DELETE
statement, as the CTE is still applied to the top rather than inside the
subquery.
Also adds test suite support CTEs against backends.
Change-Id: I8ac337104d5c546dd4f0cd305632ffb56ac8bf90
Fixes: #4275Fixes: #4230
(cherry picked from commit 3619edcb8a)
Fixed regression in 1.2 due to 🎫`4147` where a :class:`.Table` that
has had some of its indexed columns redefined with new ones, as would occur
when overriding columns during reflection or when using
:paramref:`.Table.extend_existing`, such that the :meth:`.Table.tometadata`
method would fail when attempting to copy those indexes as they still
referred to the replaced column. The copy logic now accommodates for this
condition.
Change-Id: I521aa2c9f3baa0e84598bbdd6ffe4bf07b6e3ba8
Fixes: #4279
(cherry picked from commit 8f7766cc61)
As SQL Server is now non-native boolean as of
I4765d2a2a00b0d14f50282603cc4d48d4739dac1 but uses the BIT
type, we need to constrain this test to continue to not
run against SQL Server.
Change-Id: I214faf2b788a0e8e10725622e3e71f3b70805533
(cherry picked from commit d2bacad469)
Fixed issue where the "ambiguous literal" error message used when
interpreting literal values as SQL expression values would encounter a
tuple value, and fail to format the message properly. Pull request courtesy
Miguel Ventura.
Change-Id: I50d5d32d5f80ec79703a42d4b19b42c2f9701f24
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/442
(cherry picked from commit 0b0b58c938)
Fixed issue where the compilation of an INSERT statement with the
"literal_binds" option that also uses an explicit sequence and "inline"
generation, as on Postgresql and Oracle, would fail to accommodate the
extra keyword argument within the sequence processing routine.
Change-Id: Ibdab7d340aea7429a210c9535ccf1a3e85f074fb
Fixes: #4231
(cherry picked from commit b4eb29253c)
Fixed a regression that occurred from the previous fix to 🎫`4204` in
version 1.2.5, where a CTE that refers to itself after the
:meth:`.CTE.alias` method has been called would not refer to iself
correctly.
Change-Id: Iaa63d65ad2b90c8693f9953fbb32dbb10c73a037
Fixes: #4204
(cherry picked from commit ef2859bbc8)
Fixed bug in :class:.`CTE` construct along the same lines as that of
🎫`4204` where a :class:`.CTE` that was aliased would not copy itself
correctly during a "clone" operation as is frequent within the ORM as well
as when using the :meth:`.ClauseElement.params` method.
Change-Id: Id68d72dd244dedfc7bd6116c9a5123c51a55ea20
Fixes: #4210
(cherry picked from commit 9a13f007e2)
Fixed bug in CTE rendering where a :class:`.CTE` that was also turned into
an :class:`.Alias` would not render its "ctename AS aliasname" clause
appropriately if there were more than one reference to the CTE in a FROM
clause.
Change-Id: If8cff27a2f4faa5eceb59aa86398db6edb3b9e72
Fixes: #4204
(cherry picked from commit 5f60dc649c)
Fixed bug in new "expanding IN parameter" feature where the bind parameter
processors for values wasn't working at all, tests failed to cover this
pretty basic case which includes that ENUM values weren't working.
Change-Id: I8e2420d7229a3e253e43b5227ebb98f9fe0bd14a
Fixes: #4198
Fixed bug where CTE expressions would not have their name or alias name
quoted when the given name is case sensitive or otherwise requires quoting.
Pull request courtesy Eric Atkin.
Fixes: #4197
Change-Id: Ib8573e82b9a1ca94b50c7c5d73ee98b79465d689
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/426
Added support for :class:`.Enum` to persist the values of the enumeration,
rather than the keys, when using a Python pep-435 style enumerated object.
The user supplies a callable function that will return the string values to
be persisted. This allows enumerations against non-string values to be
value-persistable as well. Pull request courtesy Jon Snyder.
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/410Fixes: #3906
Change-Id: Id385465d215d1e5baaad68368b168afdd846b82c
Fixed bug where the :class:`.Enum` type wouldn't handle
enum "aliases" correctly, when more than one key refers to the
same value. Pull request courtesy Daniel Knell.
Fixes: #4180
Change-Id: Ia716c00ca6c67aeab56965f0fdd575ecb7c71416
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/420
Fixed regression in 1.2 where newly repaired quoting
of collation names in 🎫`3785` breaks SQL Server,
which explicitly does not understand a quoted collation
name. Whether or not mixed-case collation names are
quoted or not is now deferred down to a dialect-level
decision so that each dialect can prepare these identifiers
directly.
Change-Id: Iaf0a8123d9bf4711219e320896bb28c5d2649304
Fixes: #4154
Fixed bug in :meth:`.Insert.values` where using the "multi-values"
format in combination with :class:`.Column` objects as keys rather
than strings would fail. Pull request courtesy Aubrey Stark-Toller.
Change-Id: I9d3b40b5950df8f5bfdc8b1d22f9c3afb277f17f
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/412Fixes: #4162
Fixed bug where the :meth:`.Table.tometadata` method would not properly
accommodate :class:`.Index` objects that didn't consist of simple
column expressions, such as indexes against a :func:`.text` construct,
indexes that used SQL expressions or :attr:`.func`, etc. The routine
now copies expressions fully to a new :class:`.Index` object while
substituting all table-bound :class:`.Column` objects for those
of the target table.
Also refined the means by which tometadata() checks if an Index
or UniqueConstraint is generated by a column-level flag, by propagating
an attribute "_column_flag=True" to such indexes/constraints.
Change-Id: I7ef1b8ea42f9933357ae35f241a5ba9838bac35b
Fixes: #4147
No SQLA built-in subclasses ColumnElement without specifying
an alternate visit_name, and user defined ColumnElement
subclasses should avoid being treated like ColumnClause,
e.g. where a Table is present.
Fixes: #4142
Change-Id: I15ed09ba8bdebae4cb0c7e5e5df3f59351477577
Fixed regression in :class:`.ARRAY` datatype caused by
🎫`3964`, which is essentially the same
issue as that of 🎫`3832`, where column attachment events
for :class:`.ARRAY` would not be invoked. This breaks the use case
of using declarative mixins that declare a :class:`.Column` which
makes use of :meth:`.MutableList.as_mutable`.
Change-Id: If8c57615860883837f6cf72661e46180a77778c1
Fixes: #4141
Fixed bug in new "expanding bind parameter" feature whereby if multiple
params were used in one statement, the regular expression would not
match the parameter name correctly.
Change-Id: Ifaf7d627aac4ead2a13c8dddccb5c515253d88e6
Fixes: #4140
Fixed bug where ``__repr__`` of :class:`.ColumnDefault` would fail
if the argument were a tuple. Pull request courtesy Nicolas Caniart.
Change-Id: I08aa2448ef91054c43d6068ac54cedbdf7a83d64
Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/1Fixes: #4126
Reworked the new "autoescape" feature introduced in
:ref:`change_2694` in 1.2.0b2 to be fully automatic; the escape
character now defaults to a forwards slash ``"/"`` and
is applied to percent, underscore, as well as the escape
character itself, for fully automatic escaping. The
character can also be changed using the "escape" parameter.
Change-Id: I74894a2576983c0f6eb89480c9e5727f49fa9c25
Fixes: #2694
In release 1.1, the :class:`.Boolean` type was broken in that
boolean coercion via ``bool()`` would occur for backends that did not
feature "native boolean", but would not occur for native boolean backends,
meaning the string ``"0"`` now behaved inconsistently. After a poll, a
consensus was reached that non-boolean values should be raising an error,
especially in the ambiguous case of string ``"0"``; so the :class:`.Boolean`
datatype will now raise ``ValueError`` if an incoming value is not
within the range ``None, True, False, 1, 0``.
Change-Id: If70c4f79c266f0dd1a0306c0ffe7acb9c66c4cc3
Fixes: #4102
Fixed bug in :func:`.array_agg` function where passing an argument
that is already of type :class:`.ARRAY`, such as a Postgresql
:obj:`.postgresql.array` construct, would produce a ``ValueError``, due
to the function attempting to nest the arrays.
Change-Id: Ibe5f6275d90e4868e6ef8a733de05acd44c05d78
Fixes: #4107
One test appears to use some awkward calling style
with the current_date function that isn't working in pg10
anymore, this looks like an extremely
old test that can be removed
Change-Id: I5f8aee0f5ed423461be5a9060c812eb0acdc7df5
sanjana