Extends AUTOCOMMIT_REGEXP for the postgres dialect to include `TRUNCATE`.
Change-Id: I315e03674b89bb89aae669b8655481e4d890491e
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/407
(cherry picked from commit 756d578287)
One test appears to use some awkward calling style
with the current_date function that isn't working in pg10
anymore, this looks like an extremely
old test that can be removed
Also cherry-picks f846a789b7 from master
to repair 'postgres'-deprecation warning test for multiple
calls within a single process
Change-Id: I5f8aee0f5ed423461be5a9060c812eb0acdc7df5
(cherry picked from commit 67ee56be69)
Made further fixes to the :class:`.ARRAY` class in conjunction with
COLLATE, as the fix made in 🎫`4006` failed to accommodate
for a multidimentional array.
Change-Id: If3e438f8ce94ebae2196671c88a4914f3b743e60
Fixes: #4006
(cherry picked from commit 6652f72352)
Fixed bug in Postgresql :meth:`.postgresql.dml.Insert.on_conflict_do_update`
which would prevent the insert statement from being used as a CTE,
e.g. via :meth:`.Insert.cte`, within another statement.
Change-Id: Ie20972a05e194290bc9d92819750845872949ecc
Fixes: #4074
(cherry picked from commit 7051653610)
Fixed bug where using :class:`.ARRAY` with a string type that
features a collation would fail to produce the correct syntax
within CREATE TABLE.
The "COLLATE" must appear to the right of the array dimensions,
so we are using regexp substitution to insert the brackets in the
appropriate place. A more heavyweight solution would be that datatypes
know how to split up their base type vs. modifiers, but as this is
so specific to Postgresql ARRAY it's better to handle these cases
more locally.
Change-Id: I394c3c673eb60689e51b5301e51651972cfdb4c0
Fixes: #4006
(cherry picked from commit ec422fb70e)
Continuing with the fix that correctly handles Postgresql
version string "10devel" released in 1.1.8, an additional regexp
bump to handle version strings of the form "10beta1". While
Postgresql now offers better ways to get this information, we
are sticking w/ the regexp at least through 1.1.x for the least
amount of risk to compatibility w/ older or alternate Postgresql
databases.
Change-Id: I12ddb06465f7dcf80563c27632441ef5963f60d4
Fixes: #4005
(cherry picked from commit b6d3f60791)
Extends `AUTOCOMMIT_REGEXP` for the postgres dialect to include `GRANT` and `REVOKE`.
Change-Id: Iba15f1ebf5bd7bc0fc1193fdf561417e53bf5d57
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/357
(cherry picked from commit 328248e6e31100aae03f060b83488f3116cd3450)
Since postgresql_ops explicitly states that it expects
string keys, to apply to a function call or expression one
needs to give the SQL expression a label that can be referred
to by name in the dictionary. test / document this.
Change-Id: I4bc4ade46dac27f9c1b92e7823433292beab97b9
Fixes: #3970
(cherry picked from commit 029d0f7538)
Changed the mechanics of :class:`.ResultProxy` to unconditionally
delay the "autoclose" step until the :class:`.Connection` is done
with the object; in the case where Postgresql ON CONFLICT with
RETURNING returns no rows, autoclose was occurring in this previously
non-existent use case, causing the usual autocommit behavior that
occurs unconditionally upon INSERT/UPDATE/DELETE to fail.
Change-Id: I235a25daf4381b31f523331f810ea04450349722
Fixes: #3955
(cherry picked from commit 8ee363e4917b0dcd64a83b6d26e465c9e61e0ea5)
Added support for parsing the Postgresql version string for
a development version like "PostgreSQL 10devel". Pull request
courtesy Sean McCully.
Change-Id: I7bc18bc4d290349c23e9796367b7d694d0873096
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/351
(cherry picked from commit a86764d99b)
Added support for the :class:`.Variant` and the :class:`.SchemaType`
objects to be compatible with each other. That is, a variant
can be created against a type like :class:`.Enum`, and the instructions
to create constraints and/or database-specific type objects will
propagate correctly as per the variant's dialect mapping.
Also added testing for some potential double-event scenarios
on TypeDecorator but it seems usually this doesn't occur.
Change-Id: I4a7e7c26b4133cd14e870f5bc34a1b2f0f19a14a
Fixes: #2892
Added regular expressions for the "IMPORT FOREIGN SCHEMA",
"REFRESH MATERIALIZED VIEW" Postgresql statements so that they
autocommit when invoked via a connection or engine without
an explicit transaction. Pull requests courtesy Frazer McLean
and Paweł Stiasny.
Fixes: #3840
Co-authored-by: Frazer McLean
Co-authored-by: Paweł Stiasny
Change-Id: I92b2b61683d29d57fa23a66a3559120cb1241c2f
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/323
Fixed bug in Postgresql :class:`.ExcludeConstraint` where the
"whereclause" and "using" parameters would not be copied during an
operation like :meth:`.Table.tometadata`.
Change-Id: I2f704981d4d4862f9c82a50272006fab8becebb6
Fixes: #3900
Fixed bug in new "ON CONFLICT DO UPDATE" feature where the "set"
values for the UPDATE clause would not be subject to type-level
processing, as normally takes effect to handle both user-defined
type level conversions as well as dialect-required conversions, such
as those required for JSON datatypes. Additionally, clarified that
the keys in the set_ dictionary should match the "key" of the column,
if distinct from the column name. A warning is emitted
for remaining column names that don't match column keys; for
compatibility reasons, these are emitted as they were previously.
Fixes: #3888
Change-Id: I67a04c67aa5f65e6d29f27bf3ef2f8257088d073
Corrects some warnings and adds tox config. Adds DeprecationWarning
to the error category. Large sweep for string literals w/ backslashes
as this is common in docstrings
Co-authored-by: Andrii Soldatenko
Fixes: #3886
Change-Id: Ia7c838dfbbe70b262622ed0803d581edc736e085
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/337
This allows to skip buffering of the results on the client side, e.g.
the following snippet:
table = sa.Table(
'testtbl', sa.MetaData(),
sa.Column('id', sa.Integer, primary_key=True),
sa.Column('a', sa.Integer),
sa.Column('b', sa.String(512))
)
table.create(eng, checkfirst=True)
with eng.connect() as conn:
result = conn.execute(table.select().limit(1)).fetchone()
if result is None:
for _ in range(1000):
conn.execute(
table.insert(),
[{'a': random.randint(1, 100000),
'b': ''.join(random.choice(string.ascii_letters) for _ in range(100))}
for _ in range(1000)]
)
with eng.connect() as conn:
for row in conn.execution_options(stream_results=True).execute(table.select()):
pass
now uses ~23 MB of memory instead of ~327 MB on CPython 3.5.2 and
PyMySQL 0.7.9.
psycopg2 implementation and execution options (stream_results,
server_side_cursors) are reused.
Change-Id: I4dc23ce3094f027bdff51b896b050361991c62e2
Fixed regression caused by the fix in 🎫`3807` (version 1.1.0)
where we ensured that the tablename was qualified in the WHERE clause
of the DO UPDATE portion of PostgreSQL's ON CONFLICT, however you
*cannot* put the table name in the WHERE clause in the actual ON
CONFLICT itself. This was an incorrect assumption, so that portion
of the change in 🎫`3807` is rolled back.
Change-Id: I442d8629496a8e405b54711cfcf487761810ae8a
Fixes: #3846Fixes: #3807
Postgresql table reflection will ensure that the
:paramref:`.Column.autoincrement` flag is set to False when reflecting
a primary key column that is not of an :class:`.Integer` datatype,
even if the default is related to an integer-generating sequence.
This can happen if a column is created as SERIAL and the datatype
is changed. The autoincrement flag can only be True if the datatype
is of integer affinity in the 1.1 series.
This bug is related to a test failure in downstream sqlalchemy_migrate.
Change-Id: I40260e47e1927a1ac940538408983c943bbdba28
Fixes: #3835
Changed the naming convention used when generating bound parameters
for a multi-VALUES insert statement, so that the numbered parameter
names don't conflict with the anonymized parameters of a WHERE clause,
as is now common in a PostgreSQL ON CONFLICT construct.
Change-Id: I3188d100fe4d322a47d344d6a63d3e40b915f228
Fixes: #3828
An adjustment to ON CONFLICT such that the "inserted_primary_key"
logic is able to accommodate the case where there's no INSERT or
UPDATE and there's no net change. The value comes out as None
in this case, rather than failing on an exception.
Change-Id: I0794e95c3ca262cb1ab2387167d96b8984225fce
Fixes: #3813
Added compiler-level flags used by Postgresql to place additional
parenthesis than would normally be generated by precedence rules
around operations involving JSON, HSTORE indexing operators as well as
within their operands since it has been observed that Postgresql's
precedence rules for at least the HSTORE indexing operator is not
consistent between 9.4 and 9.5.
Fixes: #3806
Change-Id: I5899677b330595264543b055abd54f3c76bfabf2
Fixed issue in new PG "on conflict" construct where columns including
those of the "excluded" namespace would not be table-qualified
in the WHERE clauses in the statement.
Change-Id: Idfefc93e7e7b0d84805e23d5436d822d606f6a0a
Fixes: #3807
Some of the dialect impl memoization for TypeDecorator
necessarily keeps the top-level TypeDecorator type
around, since a user-defined type will have bind and result
set processing behavior. For both TypeDecorator and Variant,
PG dialect needs to ensure it's looking at the SQLAlchemy
type to check for SmallInteger / BigInteger.
Fixes: 3739
Change-Id: I2d45fb997f17c6272d6bb826a77d2dba665adae7
(cherry picked from commit 421fa6b8bf)
count() here is misleading in that it not only
counts from an arbitrary column in the table, it also
does not make accommodations for DISTINCT, JOIN, etc.
as the ORM-level function does. Core should not be
attempting to provide a function like this.
Change-Id: I9916fc51ef744389a92c54660ab08e9695b8afc2
Fixes: #3724
Also adds version detection so that DROP INDEX CONCURRENTLY
only renders if the Postgresql version is 9.2 or higher;
for CREATE INDEX CONCURRENTLY, version detection is added
to allow the phrase to omit if the Postgresql version is
less than 8.2.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Change-Id: I597287e0ebbbe256c957a3579b58ace6848ab4f4
Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/84
Previously, it was impossible to have a Table that has
None for a schema name when the "schema" parameter on
MetaData was set. A new symbol sqlalchemy.schema.BLANK_SCHEMA
is added which indicates that the schema name should unconditionally
be set to None. In particular, this value must be passed within
cross-schema foreign key reflection, so that a Table which
is in the "default" schema can be represented properly.
Fixes: #3716
Change-Id: I3d24f99c22cded206c5379fd32a225e74edb7a8e
override with a column expression (e.g. by using ``'x' in col``)
would cause an endless loop in the case of an ARRAY type, as Python
defers this to ``__getitem__`` access which never raises for this
type. Overall, all use of ``__contains__`` now raises
NotImplementedError.
fixes#3642
removed; this has emitted a warning for many years and projects
should be calling upon ``sqlalchemy.dialects.postgresql``.
Engine URLs of the form ``postgres://`` will still continue to function,
however.
This supports the use case of an application that uses the same set of
:class:`.Table` objects in many schemas, such as schema-per-user.
A new execution option
:paramref:`.Connection.execution_options.schema_translate_map` is
added. fixes#2685
- latest tox doesn't like the {posargs} in the profile rerunner
persistence of JSON values in MySQL as well as basic operator support
of "getitem" and "getpath", making use of the ``JSON_EXTRACT``
function in order to refer to individual paths in a JSON structure.
fixes#3547
- Added a new type to core :class:`.types.JSON`. This is the
base of the PostgreSQL :class:`.postgresql.JSON` type as well as that
of the new :class:`.mysql.JSON` type, so that a PG/MySQL-agnostic
JSON column may be used. The type features basic index and path
searching support.
fixes#3619
- reorganization of migration docs etc. to try to refer both to
the fixes to JSON that helps Postgresql while at the same time
indicating these are new features of the new base JSON type.
- a rework of the Array/Indexable system some more, moving things
that are specific to Array out of Indexable.
- new operators for JSON indexing added to core so that these can
be compiled by the PG and MySQL dialects individually
- rename sqltypes.Array to sqltypes.ARRAY - as there is no generic
Array implementation, this is an uppercase type for now, consistent
with the new sqltypes.JSON type that is also not a generic implementation.
There may need to be some convention change to handle the case of
datatypes that aren't generic, rely upon DB-native implementations,
but aren't necessarily all named the same thing.
Mike Bayer