The "NO ACTION" keyword for foreign key "ON UPDATE" is now considered to be
the default cascade for a foreign key on all supporting backends (SQlite,
MySQL, PostgreSQL) and when detected is not included in the reflection
dictionary; this is already the behavior for PostgreSQL and MySQL for all
previous SQLAlchemy versions in any case. The "RESTRICT" keyword is
positively stored when detected; PostgreSQL does report on this keyword,
and MySQL as of version 8.0 does as well. On earlier MySQL versions, it is
not reported by the database.
Fixes: #4741
Change-Id: I6becf1f2450605c1991158bb8a04d954dcc7396c
This is a very useful assertion which prevents unused variables
from being set up allows code to be more readable and sometimes
even more efficient. test suites seem to be where the most
problems are and there do not seem to be documentation examples
that are using this, or at least the linter is not taking effect
within rst blocks.
Change-Id: I2b3341d8dd14da34879d8425838e66a4b9f8e27d
Added support for column sorting flags when reflecting indexes for
PostgreSQL, including ASC, DESC, NULLSFIRST, NULLSLAST. Also adds this
facility to the reflection system in general which can be applied to other
dialects in future releases. Pull request courtesy Eli Collins.
Fixes: #4717Closes: #4725
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4725
Pull-request-sha: 3cbb067bd4
Change-Id: I8b0617d68580cfe4ff79d758a077263f33e852c2
- don't call relatively expensive isEnabledFor(), just call _log_notices
- don't reset the list if it's empty
- fix the test to use a custom function to definitely create a notice, confirmed
that PG seems to no longer create the "implicit sequence" notices
- assert that the reset of the notices works too
- update the docs to illustrate for folks who haven't worked with logging before
Change-Id: I7291e647c177d338e0ad673f3106b4d503e4b3ea
Fixed bug where PostgreSQL dialect could not correctly reflect an ENUM
datatype that has no members, returning a list with ``None`` for the
``get_enums()`` call and raising a TypeError when reflecting a column which
has such a datatype. The inspection now returns an empty list.
Fixes: #4701
Change-Id: I202bab19728862cbc64deae211d5ba6a103b8317
Moved the "rollback" which occurs during dialect initialization so that it
occurs after additional dialect-specific initialize steps, in particular
those of the psycopg2 dialect which would inadvertently leave transactional
state on the first new connection, which could interfere with some
psycopg2-specific APIs which require that no transaction is started. Pull
request courtesy Matthew Wilkes.
Fixes: #4663Closes: #4664
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4664
Pull-request-sha: e544fe671d
Change-Id: If40a15a1679b4eec0b8b8222f678697728009c30
Fixed regression from release 1.3.2 caused by 🎫`4562` where a URL
that contained only a query string and no hostname, such as for the
purposes of specifying a service file with connection information, would no
longer be propagated to psycopg2 properly. The change in 🎫`4562`
has been adjusted to further suit psycopg2's exact requirements, which is
that if there are any connection parameters whatsoever, the "dsn" parameter
is no longer required, so in this case the query string parameters are
passed alone.
Fixes: #4601
Change-Id: Ic29a8b77bcf50ee996968bab25aaac7ae4bfc26f
Added support for parameter-less connection URLs for the psycopg2 dialect,
meaning, the URL can be passed to :func:`.create_engine` as
``"postgresql+psycopg2://"`` with no additional arguments to indicate an
empty DSN passed to libpq, which indicates to connect to "localhost" with
no username, password, or database given. Pull request courtesy Julian
Mehnle.
Fixes: #4562Closes: #4563
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4563
Pull-request-sha: 8a05c96944
Change-Id: Ib6fca3c3c9eebeaf590d7f7fb0bc8cd4b6e4a55a
Modified the :paramref:`.Select.with_for_update.of` parameter so that if a
join or other composed selectable is passed, the individual :class:`.Table`
objects will be filtered from it, allowing one to pass a join() object to
the parameter, as occurs normally when using joined table inheritance with
the ORM. Pull request courtesy Raymond Lu.
Fixes: #4550
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Closes: #4551
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4551
Pull-request-sha: 452da77d15
Change-Id: If4b7c231f7b71190d7245543959fb5c3351125a1
Fully removed the behavior of strings passed directly as components of a
:func:`.select` or :class:`.Query` object being coerced to :func:`.text`
constructs automatically; the warning that has been emitted is now an
ArgumentError or in the case of order_by() / group_by() a CompileError.
This has emitted a warning since version 1.0 however its presence continues
to create concerns for the potential of mis-use of this behavior.
Note that public CVEs have been posted for order_by() / group_by() which
are resolved by this commit: CVE-2019-7164 CVE-2019-7548
Added "SQL phrase validation" to key DDL phrases that are accepted as plain
strings, including :paramref:`.ForeignKeyConstraint.on_delete`,
:paramref:`.ForeignKeyConstraint.on_update`,
:paramref:`.ExcludeConstraint.using`,
:paramref:`.ForeignKeyConstraint.initially`, for areas where a series of SQL
keywords only are expected.Any non-space characters that suggest the phrase
would need to be quoted will raise a :class:`.CompileError`. This change
is related to the series of changes committed as part of 🎫`4481`.
Fixed issue where using an uppercase name for an index type (e.g. GIST,
BTREE, etc. ) or an EXCLUDE constraint would treat it as an identifier to
be quoted, rather than rendering it as is. The new behavior converts these
types to lowercase and ensures they contain only valid SQL characters.
Quoting is applied to :class:`.Function` names, those which are usually but
not necessarily generated from the :attr:`.sql.func` construct, at compile
time if they contain illegal characters, such as spaces or punctuation. The
names are as before treated as case insensitive however, meaning if the
names contain uppercase or mixed case characters, that alone does not
trigger quoting. The case insensitivity is currently maintained for
backwards compatibility.
Fixes: #4481Fixes: #4473Fixes: #4467
Change-Id: Ib22a27d62930e24702e2f0f7c74a0473385a08eb
Revised the query used when reflecting CHECK constraints to make use of the
``pg_get_constraintdef`` function, as the ``consrc`` column is being
deprecated in PG 12. Thanks to John A Stevenson for the tip.
Fixes: #4463
Change-Id: Ie0ee9bdfddb0635db72b35c2e2e4b27f154162fd
A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.
See the notes added to the changelog and migration notes for further
details.
Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
Fixed issue where a :class:`.postgresql.ENUM` or a custom domain present
in a remote schema would not be recognized within column reflection if
the name of the enum/domain or the name of the schema required quoting.
A new parsing scheme now fully parses out quoted or non-quoted tokens
including support for SQL-escaped quotes.
Fixed issue where multiple :class:`.postgresql.ENUM` objects referred to
by the same :class:`.MetaData` object would fail to be created if
multiple objects had the same name under different schema names. The
internal memoization the Postgresql dialect uses to track if it has
created a particular :class:`.postgresql.ENUM` in the database during
a DDL creation sequence now takes schema name into account.
Fixes: #4416
Change-Id: I8cf03069e10b12f409e9b6796e24fc5850979955
Fixed issue where reflection of a PostgreSQL domain that is expressed as an
array would fail to be recognized. Pull request courtesy Jakub Synowiec.
Fixes: #4377
Change-Id: I252c79ca435b87d4d9172b1c84e0e74e789ef676
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4380
Added rudimental support for reflection of Postgresql
partitioned tables, e.g. that relkind='p' is added to reflection
queries that return table information.
Fixes: #4237
Change-Id: I66fd10b002e4ed21ea13b13a7e35a85f66bdea75
Added support for the :class:`.aggregate_order_by` function to receive
multiple ORDER BY elements, previously only a single element was accepted.
Fixes: #4337
Change-Id: I411ac31697a0d65b568ad65ce5b5181717afbd65
Fixed bug in PostgreSQL dialect where compiler keyword arguments such as
``literal_binds=True`` were not being propagated to a DISTINCT ON
expression.
Fixes: #4325
Change-Id: I9949387dceb7fabe889799f42e92423572368b29
Fixed the :func:`.postgresql.array_agg` function, which is a slightly
altered version of the usual :func:`.functions.array_agg` function, to also
accept an incoming "type" argument without forcing an ARRAY around it,
essentially the same thing that was fixed for the generic function in 1.1
in 🎫`4107`.
Fixes: #4324
Change-Id: I399a29f59c945a217cdd22c65ff0325edea8ea65
Fixed bug in PostgreSQL ENUM reflection where a case-sensitive, quoted name
would be reported by the query including quotes, which would not match a
target column during table reflection as the quotes needed to be stripped
off.
Fixes: #4323
Change-Id: I668f3acccc578e58f23b70c82d31d5c1ec194913
Fixed bug where the special "not equals" operator for the Postgresql
"range" datatypes such as DATERANGE would fail to render "IS NOT NULL" when
compared to the Python ``None`` value.
Also break up range tests into backend round trip and straight
compilation suites.
Change-Id: Ibaee132b1ea7dac8b799495a27f98f82a7d9c028
Fixes: #4229
Added support for "PARTITION BY" in Postgresql table definitions,
using "postgresql_partition_by". Pull request courtesy
Vsevolod Solovyov.
Change-Id: Id74d6882d7193fae1e5fd44b6e12d6852866fcc4
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/430
Extends AUTOCOMMIT_REGEXP for the postgres dialect to include `TRUNCATE`.
Change-Id: I315e03674b89bb89aae669b8655481e4d890491e
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/407
An EXCLUDE constraint makes an index just like a UNIQUE does;
get_indexes() will receive this. Test that this works out the
same way as it does for a UNIQUE.
Change-Id: I02ac7cbbb1ca0d1fcdcdbe9a8b8bd1ffee3e496c
Fixes: #4122
Added a new flag ``use_batch_mode`` to the psycopg2 dialect. This flag
enables the use of psycopg2's ``psycopg2.extras.execute_batch``
extension when the :class:`.Engine` calls upon ``cursor.executemany()``.
This extension provides a critical performance increase by over an order of magnitude
when running INSERT statements in batch. The flag is False by default
as it is considered to be experimental for now.
Change-Id: Ib88d28bc792958d47109f644ff1d08c897db4ff7
Fixes: #4109
Make sure we clear the plugin registry before testing
that the "postgres" name raises. Also move non-backend
tests out of MiscTest into a new suite.
Change-Id: Icd1bb4745aa07f52d585fcf959f76fcd8bdc7f24
Made further fixes to the :class:`.ARRAY` class in conjunction with
COLLATE, as the fix made in 🎫`4006` failed to accommodate
for a multidimentional array.
Change-Id: If3e438f8ce94ebae2196671c88a4914f3b743e60
Fixes: #4006
Internal refinements to the :class:`.Enum`, :class:`.Interval`, and
:class:`.Boolean` types, which now extend a common mixin
:class:`.Emulated` that indicates a type that provides Python-side
emulation of a DB native type, switching out to the DB native type when a
supporting backend is in use. The Postgresql :class:`.INTERVAL` type
when used directly will now include the correct type coercion rules for
SQL expressions that also take effect for :class:`.sqltypes.Interval`
(such as adding a date to an interval yields a datetime).
Change-Id: Ifb9f9d7cbd9f5990dcb2abb583193e9e92b789ad
Fixes: #4088
Fixed bug in Postgresql :meth:`.postgresql.dml.Insert.on_conflict_do_update`
which would prevent the insert statement from being used as a CTE,
e.g. via :meth:`.Insert.cte`, within another statement.
Change-Id: Ie20972a05e194290bc9d92819750845872949ecc
Fixes: #4074
Enabled UUID support for the pg8000 driver, which supports native Python
uuid round trips for this datatype. Arrays of UUID are still not supported,
however.
Change-Id: I44ca323c5d9f2cd87327210233bc36a3556eb050
Fixes: #4016
Fixed bug where the pg8000 driver would fail if using
:meth:`.MetaData.reflect` with a schema name, since the schema name would
be sent as a "quoted_name" object that's a string subclass, which pg8000
doesn't recognize. The quoted_name type is added to pg8000's
py_types collection on connect.
Change-Id: Id0f838320cb66563685e094e4eae2d5116100d27
Fixes: #4041
Fixed bug where using :class:`.ARRAY` with a string type that
features a collation would fail to produce the correct syntax
within CREATE TABLE.
The "COLLATE" must appear to the right of the array dimensions,
so we are using regexp substitution to insert the brackets in the
appropriate place. A more heavyweight solution would be that datatypes
know how to split up their base type vs. modifiers, but as this is
so specific to Postgresql ARRAY it's better to handle these cases
more locally.
Change-Id: I394c3c673eb60689e51b5301e51651972cfdb4c0
Fixes: #4006
Continuing with the fix that correctly handles Postgresql
version string "10devel" released in 1.1.8, an additional regexp
bump to handle version strings of the form "10beta1". While
Postgresql now offers better ways to get this information, we
are sticking w/ the regexp at least through 1.1.x for the least
amount of risk to compatibility w/ older or alternate Postgresql
databases.
Change-Id: I12ddb06465f7dcf80563c27632441ef5963f60d4
Fixes: #4005
Extends `AUTOCOMMIT_REGEXP` for the postgres dialect to include `GRANT` and `REVOKE`.
Change-Id: Iba15f1ebf5bd7bc0fc1193fdf561417e53bf5d57
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/357
Since postgresql_ops explicitly states that it expects
string keys, to apply to a function call or expression one
needs to give the SQL expression a label that can be referred
to by name in the dictionary. test / document this.
Change-Id: I4bc4ade46dac27f9c1b92e7823433292beab97b9
Fixes: #3970
For some reason, when ARRAY was added to the base it was never linked
to postgresql.ARRAY. Link the two types and also make base
ARRAY the schema event target so that it supports the same
features as postgresql.ARRAY.
Change-Id: I82fa6c9d2b8c5028dba3a009715f7bc296b2bc0b
Fixes: #3964
Added support for all possible "fields" identifiers when reflecting the
Postgresql ``INTERVAL`` datatype, e.g. "YEAR", "MONTH", "DAY TO
MINUTE", etc.. In addition, the :class:`.postgresql.INTERVAL`
datatype itself now includes a new parameter
:paramref:`.postgresql.INTERVAL.fields` where these qualifiers can be
specified; the qualifier is also reflected back into the resulting
datatype upon reflection / inspection.
Change-Id: I33816e68c533b023e0632db6f4e73fefd2de4721
Fixes: #3959
Changed the mechanics of :class:`.ResultProxy` to unconditionally
delay the "autoclose" step until the :class:`.Connection` is done
with the object; in the case where Postgresql ON CONFLICT with
RETURNING returns no rows, autoclose was occurring in this previously
non-existent use case, causing the usual autocommit behavior that
occurs unconditionally upon INSERT/UPDATE/DELETE to fail.
Change-Id: I235a25daf4381b31f523331f810ea04450349722
Fixes: #3955
(cherry picked from commit 8ee363e4917b0dcd64a83b6d26e465c9e61e0ea5)
(cherry picked from commit f52fb5282a)
Added support for parsing the Postgresql version string for
a development version like "PostgreSQL 10devel". Pull request
courtesy Sean McCully.
Change-Id: I7bc18bc4d290349c23e9796367b7d694d0873096
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/351
Added support for SQL comments on :class:`.Table` and :class:`.Column`
objects, via the new :paramref:`.Table.comment` and
:paramref:`.Column.comment` arguments. The comments are included
as part of DDL on table creation, either inline or via an appropriate
ALTER statement, and are also reflected back within table reflection,
as well as via the :class:`.Inspector`. Supported backends currently
include MySQL, Postgresql, and Oracle.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Fixes: #1546
Change-Id: Ib90683850805a2b4ee198e420dc294f32f15d35d
Added support for the :class:`.Variant` and the :class:`.SchemaType`
objects to be compatible with each other. That is, a variant
can be created against a type like :class:`.Enum`, and the instructions
to create constraints and/or database-specific type objects will
propagate correctly as per the variant's dialect mapping.
Also added testing for some potential double-event scenarios
on TypeDecorator but it seems usually this doesn't occur.
Change-Id: I4a7e7c26b4133cd14e870f5bc34a1b2f0f19a14a
Fixes: #2892
Added regular expressions for the "IMPORT FOREIGN SCHEMA",
"REFRESH MATERIALIZED VIEW" Postgresql statements so that they
autocommit when invoked via a connection or engine without
an explicit transaction. Pull requests courtesy Frazer McLean
and Paweł Stiasny.
Fixes: #3840
Co-authored-by: Frazer McLean
Co-authored-by: Paweł Stiasny
Change-Id: I92b2b61683d29d57fa23a66a3559120cb1241c2f
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/323
Mike Bayer