Since postgresql_ops explicitly states that it expects
string keys, to apply to a function call or expression one
needs to give the SQL expression a label that can be referred
to by name in the dictionary. test / document this.
Change-Id: I4bc4ade46dac27f9c1b92e7823433292beab97b9
Fixes: #3970
(cherry picked from commit 029d0f7538)
Changed the mechanics of :class:`.ResultProxy` to unconditionally
delay the "autoclose" step until the :class:`.Connection` is done
with the object; in the case where Postgresql ON CONFLICT with
RETURNING returns no rows, autoclose was occurring in this previously
non-existent use case, causing the usual autocommit behavior that
occurs unconditionally upon INSERT/UPDATE/DELETE to fail.
Change-Id: I235a25daf4381b31f523331f810ea04450349722
Fixes: #3955
(cherry picked from commit 8ee363e4917b0dcd64a83b6d26e465c9e61e0ea5)
Added support for parsing the Postgresql version string for
a development version like "PostgreSQL 10devel". Pull request
courtesy Sean McCully.
Change-Id: I7bc18bc4d290349c23e9796367b7d694d0873096
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/351
(cherry picked from commit a86764d99b)
Added support for the :class:`.Variant` and the :class:`.SchemaType`
objects to be compatible with each other. That is, a variant
can be created against a type like :class:`.Enum`, and the instructions
to create constraints and/or database-specific type objects will
propagate correctly as per the variant's dialect mapping.
Also added testing for some potential double-event scenarios
on TypeDecorator but it seems usually this doesn't occur.
Change-Id: I4a7e7c26b4133cd14e870f5bc34a1b2f0f19a14a
Fixes: #2892
Added regular expressions for the "IMPORT FOREIGN SCHEMA",
"REFRESH MATERIALIZED VIEW" Postgresql statements so that they
autocommit when invoked via a connection or engine without
an explicit transaction. Pull requests courtesy Frazer McLean
and Paweł Stiasny.
Fixes: #3840
Co-authored-by: Frazer McLean
Co-authored-by: Paweł Stiasny
Change-Id: I92b2b61683d29d57fa23a66a3559120cb1241c2f
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/323
Fixed bug in Postgresql :class:`.ExcludeConstraint` where the
"whereclause" and "using" parameters would not be copied during an
operation like :meth:`.Table.tometadata`.
Change-Id: I2f704981d4d4862f9c82a50272006fab8becebb6
Fixes: #3900
Fixed bug in new "ON CONFLICT DO UPDATE" feature where the "set"
values for the UPDATE clause would not be subject to type-level
processing, as normally takes effect to handle both user-defined
type level conversions as well as dialect-required conversions, such
as those required for JSON datatypes. Additionally, clarified that
the keys in the set_ dictionary should match the "key" of the column,
if distinct from the column name. A warning is emitted
for remaining column names that don't match column keys; for
compatibility reasons, these are emitted as they were previously.
Fixes: #3888
Change-Id: I67a04c67aa5f65e6d29f27bf3ef2f8257088d073
Corrects some warnings and adds tox config. Adds DeprecationWarning
to the error category. Large sweep for string literals w/ backslashes
as this is common in docstrings
Co-authored-by: Andrii Soldatenko
Fixes: #3886
Change-Id: Ia7c838dfbbe70b262622ed0803d581edc736e085
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/337
This allows to skip buffering of the results on the client side, e.g.
the following snippet:
table = sa.Table(
'testtbl', sa.MetaData(),
sa.Column('id', sa.Integer, primary_key=True),
sa.Column('a', sa.Integer),
sa.Column('b', sa.String(512))
)
table.create(eng, checkfirst=True)
with eng.connect() as conn:
result = conn.execute(table.select().limit(1)).fetchone()
if result is None:
for _ in range(1000):
conn.execute(
table.insert(),
[{'a': random.randint(1, 100000),
'b': ''.join(random.choice(string.ascii_letters) for _ in range(100))}
for _ in range(1000)]
)
with eng.connect() as conn:
for row in conn.execution_options(stream_results=True).execute(table.select()):
pass
now uses ~23 MB of memory instead of ~327 MB on CPython 3.5.2 and
PyMySQL 0.7.9.
psycopg2 implementation and execution options (stream_results,
server_side_cursors) are reused.
Change-Id: I4dc23ce3094f027bdff51b896b050361991c62e2
Fixed regression caused by the fix in 🎫`3807` (version 1.1.0)
where we ensured that the tablename was qualified in the WHERE clause
of the DO UPDATE portion of PostgreSQL's ON CONFLICT, however you
*cannot* put the table name in the WHERE clause in the actual ON
CONFLICT itself. This was an incorrect assumption, so that portion
of the change in 🎫`3807` is rolled back.
Change-Id: I442d8629496a8e405b54711cfcf487761810ae8a
Fixes: #3846Fixes: #3807
Postgresql table reflection will ensure that the
:paramref:`.Column.autoincrement` flag is set to False when reflecting
a primary key column that is not of an :class:`.Integer` datatype,
even if the default is related to an integer-generating sequence.
This can happen if a column is created as SERIAL and the datatype
is changed. The autoincrement flag can only be True if the datatype
is of integer affinity in the 1.1 series.
This bug is related to a test failure in downstream sqlalchemy_migrate.
Change-Id: I40260e47e1927a1ac940538408983c943bbdba28
Fixes: #3835
Changed the naming convention used when generating bound parameters
for a multi-VALUES insert statement, so that the numbered parameter
names don't conflict with the anonymized parameters of a WHERE clause,
as is now common in a PostgreSQL ON CONFLICT construct.
Change-Id: I3188d100fe4d322a47d344d6a63d3e40b915f228
Fixes: #3828
An adjustment to ON CONFLICT such that the "inserted_primary_key"
logic is able to accommodate the case where there's no INSERT or
UPDATE and there's no net change. The value comes out as None
in this case, rather than failing on an exception.
Change-Id: I0794e95c3ca262cb1ab2387167d96b8984225fce
Fixes: #3813
Added compiler-level flags used by Postgresql to place additional
parenthesis than would normally be generated by precedence rules
around operations involving JSON, HSTORE indexing operators as well as
within their operands since it has been observed that Postgresql's
precedence rules for at least the HSTORE indexing operator is not
consistent between 9.4 and 9.5.
Fixes: #3806
Change-Id: I5899677b330595264543b055abd54f3c76bfabf2
Fixed issue in new PG "on conflict" construct where columns including
those of the "excluded" namespace would not be table-qualified
in the WHERE clauses in the statement.
Change-Id: Idfefc93e7e7b0d84805e23d5436d822d606f6a0a
Fixes: #3807
Some of the dialect impl memoization for TypeDecorator
necessarily keeps the top-level TypeDecorator type
around, since a user-defined type will have bind and result
set processing behavior. For both TypeDecorator and Variant,
PG dialect needs to ensure it's looking at the SQLAlchemy
type to check for SmallInteger / BigInteger.
Fixes: 3739
Change-Id: I2d45fb997f17c6272d6bb826a77d2dba665adae7
(cherry picked from commit 421fa6b8bf)
count() here is misleading in that it not only
counts from an arbitrary column in the table, it also
does not make accommodations for DISTINCT, JOIN, etc.
as the ORM-level function does. Core should not be
attempting to provide a function like this.
Change-Id: I9916fc51ef744389a92c54660ab08e9695b8afc2
Fixes: #3724
Also adds version detection so that DROP INDEX CONCURRENTLY
only renders if the Postgresql version is 9.2 or higher;
for CREATE INDEX CONCURRENTLY, version detection is added
to allow the phrase to omit if the Postgresql version is
less than 8.2.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Change-Id: I597287e0ebbbe256c957a3579b58ace6848ab4f4
Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/84
Previously, it was impossible to have a Table that has
None for a schema name when the "schema" parameter on
MetaData was set. A new symbol sqlalchemy.schema.BLANK_SCHEMA
is added which indicates that the schema name should unconditionally
be set to None. In particular, this value must be passed within
cross-schema foreign key reflection, so that a Table which
is in the "default" schema can be represented properly.
Fixes: #3716
Change-Id: I3d24f99c22cded206c5379fd32a225e74edb7a8e
override with a column expression (e.g. by using ``'x' in col``)
would cause an endless loop in the case of an ARRAY type, as Python
defers this to ``__getitem__`` access which never raises for this
type. Overall, all use of ``__contains__`` now raises
NotImplementedError.
fixes#3642
removed; this has emitted a warning for many years and projects
should be calling upon ``sqlalchemy.dialects.postgresql``.
Engine URLs of the form ``postgres://`` will still continue to function,
however.
This supports the use case of an application that uses the same set of
:class:`.Table` objects in many schemas, such as schema-per-user.
A new execution option
:paramref:`.Connection.execution_options.schema_translate_map` is
added. fixes#2685
- latest tox doesn't like the {posargs} in the profile rerunner
persistence of JSON values in MySQL as well as basic operator support
of "getitem" and "getpath", making use of the ``JSON_EXTRACT``
function in order to refer to individual paths in a JSON structure.
fixes#3547
- Added a new type to core :class:`.types.JSON`. This is the
base of the PostgreSQL :class:`.postgresql.JSON` type as well as that
of the new :class:`.mysql.JSON` type, so that a PG/MySQL-agnostic
JSON column may be used. The type features basic index and path
searching support.
fixes#3619
- reorganization of migration docs etc. to try to refer both to
the fixes to JSON that helps Postgresql while at the same time
indicating these are new features of the new base JSON type.
- a rework of the Array/Indexable system some more, moving things
that are specific to Array out of Indexable.
- new operators for JSON indexing added to core so that these can
be compiled by the PG and MySQL dialects individually
- rename sqltypes.Array to sqltypes.ARRAY - as there is no generic
Array implementation, this is an uppercase type for now, consistent
with the new sqltypes.JSON type that is also not a generic implementation.
There may need to be some convention change to handle the case of
datatypes that aren't generic, rely upon DB-native implementations,
but aren't necessarily all named the same thing.
For example, this query:
SELECT s1.users.name FROM s1.users FOR UPDATE OF s1.users
should actually be:
SELECT s1.users.name FROM s1.users FOR UPDATE OF users
fixes#3573
to return ``datetime.timedelta`` in the same way as that of
:obj:`.types.Interval.python_type`, rather than raising
``NotImplementedError``.
fixes#3571
(cherry picked from commit 29d6f6e19b)
"auto increment" column has been changed, such that autoincrement
is no longer implicitly enabled for a :class:`.Table` that has a
composite primary key. In order to accommodate being able to enable
autoincrement for a composite PK member column while at the same time
maintaining SQLAlchemy's long standing behavior of enabling
implicit autoincrement for a single integer primary key, a third
state has been added to the :paramref:`.Column.autoincrement` parameter
``"auto"``, which is now the default. fixes#3216
- The MySQL dialect no longer generates an extra "KEY" directive when
generating CREATE TABLE DDL for a table using InnoDB with a
composite primary key with AUTO_INCREMENT on a column that isn't the
first column; to overcome InnoDB's limitation here, the PRIMARY KEY
constraint is now generated with the AUTO_INCREMENT column placed
first in the list of columns.
to a :class:`.types.Enum` or :class:`.postgresql.ENUM` subtype
will now emit the expected "CREATE TYPE" and "DROP TYPE" DDL when
the type is used within a "CREATE TABLE" or "DROP TABLE".
fixes#2729
Mike Bayer