Added a full suite of new SQL bitwise operators, for performing
database-side bitwise expressions on appropriate data values such as
integers, bit-strings, and similar. Pull request courtesy Yegor Statkevich.
Fixes: #8780Closes: #9204
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9204
Pull-request-sha: a4541772a6
Change-Id: I4c70e80f9548dcc1b4e3dccd71bd59d51d3ed46e
Added support to the asyncpg dialect to return the ``cursor.rowcount``
value for SELECT statements when available. While this is not a typical use
for ``cursor.rowcount``, the other PostgreSQL dialects generally provide
this value. Pull request courtesy Michael Gorven.
Fixes: #9048Closes: #9049
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9049
Pull-request-sha: df16160530
Change-Id: I095b866779ccea7e4d50bc841fef7605e61c667f
Fixed bug where the PostgreSQL
:paramref:`_postgresql.OnConflictClause.constraint` parameter would accept
an :class:`.Index` object, however would not expand this index out into its
individual index expressions, instead rendering its name in an ON CONFLICT
ON CONSTRAINT clause, which is not accepted by PostgreSQL; the "constraint
name" form only accepts unique or exclude constraint names. The parameter
continues to accept the index but now expands it out into its component
expressions for the render.
Fixes: #9023
Change-Id: I6baf243e26bfe578bf3f193c162dd7a623b6ede9
Fixed regression where newly revised PostgreSQL range types such as
:class:`_postgresql.INT4RANGE` could not be set up as the impl of a
:class:`.TypeDecorator` custom type, instead raising a ``TypeError``.
Fixes: #9020
Change-Id: Ib881c3c7f63d000f49a09185a8663659a9970aa9
Added support for explicit use of PG full text functions with asyncpg and
psycopg (SQLAlchemy 2.0 only), with regards to the ``REGCONFIG`` type cast
for the first argument, which previously would be incorrectly cast to a
VARCHAR, causing failures on these dialects that rely upon explicit type
casts. This includes support for :class:`_postgresql.to_tsvector`,
:class:`_postgresql.to_tsquery`, :class:`_postgresql.plainto_tsquery`,
:class:`_postgresql.phraseto_tsquery`,
:class:`_postgresql.websearch_to_tsquery`,
:class:`_postgresql.ts_headline`, each of which will determine based on
number of arguments passed if the first string argument should be
interpreted as a PostgreSQL "REGCONFIG" value; if so, the argument is typed
using a newly added type object :class:`_postgresql.REGCONFIG` which is
then explicitly cast in the SQL expression.
Fixes: #8977
Change-Id: Ib36698a984fd4194bd6e0eb663105f790f3db7d3
This fixes issue #8984, making the method `Range.__eq__` return `NotImplemented` when the argument is an instance of a different class.
Closes: #8985
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8985
Pull-request-sha: b8f601f038
Change-Id: Iaf0b651a22a9f346c8f18b3a53e3339bf61bb33f
This adds a bunch of properties to new PG Range class for compatibility
with other implementations, providing a more similar API to access
emptiness and bounds status.
The naming conventions here derive from PostgreSQL itself,
see https://www.postgresql.org/docs/9.3/functions-range.html .
pep-484 also implemented by Mike, as this is a pretty type-intensive
module.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Closes: #8927
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8927
Pull-request-sha: 8b9e7b7e33
Change-Id: I0b1d49311517ee1cc1377a974ed0a860ea5756e4
Changed how the positional compilation is performed. It's rendered by the compiler
the same as the pyformat compilation. The string is then processed to replace
the placeholders with the correct ones, and to obtain the correct order of the
parameters.
This vastly simplifies the computation of the order of the parameters, that in
case of nested CTE is very hard to compute correctly.
Reworked how numeric paramstyle behavers:
- added support for repeated parameter, without duplicating them like in normal
positional dialects
- implement insertmany support. This requires that the dialect supports out of
order placehoders, since all parameters that are not part of the VALUES clauses
are placed at the beginning of the parameter tuple
- support for different identifiers for a numeric parameter. It's for example
possible to use postgresql style placeholder $1, $2, etc
Added two new dialect based on sqlite to test "numeric" fully using
both :1 style and $1 style. Includes a workaround for SQLite's
not-really-correct numeric implementation.
Changed parmstyle of asyncpg dialect to use numeric, rendering with its native
$ identifiers
Fixes: #8926Fixes: #8849
Change-Id: I7c640467d49adfe6d795cc84296fc7403dcad4d6
Fixed a series of issues regarding positionally rendered bound parameters,
such as those used for SQLite, asyncpg, MySQL and others. Some compiled
forms would not maintain the order of parameters correctly, such as the
PostgreSQL ``regexp_replace()`` function as well as within the "nesting"
feature of the :class:`.CTE` construct first introduced in 🎫`4123`.
Fixes: #8827
Change-Id: I9813ed7c358cc5c1e26725c48df546b209a442cb
Added additional type-detection for the new PostgreSQL
:class:`_postgresql.Range` type, where previous cases that allowed the
psycopg2-native range objects to be received directly by the DBAPI without
SQLAlchemy intercepting them stopped working, as we now have our own value
object. The :class:`_postgresql.Range` object has been enhanced such that
SQLAlchemy Core detects it in otherwise ambiguous situations (such as
comparison to dates) and applies appropriate bind handlers. Pull request
courtesy Lele Gaifax.
Fixes: #8884Closes: #8886
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8886
Pull-request-sha: 6e95e08a30
Change-Id: I3ca277c826dcf4b5644f44eb251345b439a84ee4
The boolness of the range was defined to be equal to its emptiness. As this has been identified as a typo rather than the intended, this inverts the logic, to match common Python behaviour as well as how other popular PG drivers do.
Closes: #8885
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8885
Pull-request-sha: 5670cdb920
Change-Id: I6f4a40168b2f037c578e84f7550370411bd42160
Made an adjustment to how the PostgreSQL dialect considers column types
when it reflects columns from a table, to accommodate for alternative
backends which may return NULL from the PG ``format_type()`` function.
Fixes: #8748
Change-Id: I6178287aac567210a76afaa5805b825daa7fa4db
command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>"
pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not
exists in sqlalchemy fixtures
Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
### Description
This PR implements missing methods on the PG `Range` class, as described by issue #8765.
### Checklist
This pull request is:
- [ ] A documentation / typographical error fix
- [ ] A short code fix
- [x] A new feature implementation
Closes: #8766
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8766
Pull-request-sha: 21c0df86cc
Change-Id: I86fabd966ad1f14a3a86132be741df46965b9aa9
An informative re-raise is now thrown in the case where any "literal
bindparam" render operation fails, indicating the value itself and
the datatype in use, to assist in debugging when literal params
are being rendered in a statement.
Fixes: #8800
Change-Id: Id658f8b03359312353ddbb0c7563026239579f7b
The RETURNING clause now renders columns using the routine as that of the
:class:`.Select` to generate labels, which will include disambiguating
labels, as well as that a SQL function surrounding a named column will be
labeled using the column name itself. This is a more comprehensive change
than a similar one made for the 1.4 series that adjusted the function label
issue only.
includes 1.4's changelog for the backported version which also
fixes an Oracle issue independently of the 2.0 series.
Fixes: #8770
Change-Id: I2ab078a214a778ffe1720dbd864ae4c105a0691d
Added new methods :meth:`_postgresql.Range.contains` and
:meth:`_postgresql.Range.contained_by` to the new :class:`.Range` data
object, which mirror the behavior of the PostgreSQL ``@>`` and ``<@``
operators, as well as the
:meth:`_postgresql.AbstractRange.comparator_factory.contains` and
:meth:`_postgresql.AbstractRange.comparator_factory.contained_by` SQL
operator methods. Pull request courtesy Lele Gaifax.
Fixes: #8706Closes: #8707
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8707
Pull-request-sha: 3a74a0d93e
Change-Id: Ief81ca5c31448640b26dfbc3defd4dde1d51e366
Refined the new approach to range objects described at :ref:`change_7156`
to accommodate driver-specific range and multirange objects, to better
accommodate both legacy code as well as when passing results from raw SQL
result sets back into new range or multirange expressions.
Fixes: #8690
Change-Id: I7e62c47067f695c6380ad0fe2fe19deaf33594d1
The :class:`.Sequence` construct restores itself to the DDL behavior it
had prior to the 1.4 series, where creating a :class:`.Sequence` with
no additional arguments will emit a simple ``CREATE SEQUENCE`` instruction
**without** any additional parameters for "start value". For most backends,
this is how things worked previously in any case; **however**, for
MS SQL Server, the default value on this database is
``-2**63``; to prevent this generally impractical default
from taking effect on SQL Server, the :paramref:`.Sequence.start` parameter
should be provided. As usage of :class:`.Sequence` is unusual
for SQL Server which for many years has standardized on ``IDENTITY``,
it is hoped that this change has minimal impact.
Fixes: #7211
Change-Id: I1207ea10c8cb1528a1519a0fb3581d9621c27b31
also adjusted CacheKeyFixture to be a general purpose
fixture so that sub-components / dialects can run
their own cache key tests.
Fixes: #8574
Change-Id: I6c66107856aee11e548d357cea77bceee3e316a0
the feature is enabled for all built in backends
when RETURNING is used,
except for Oracle that doesn't need it, and on
psycopg2 and mssql+pyodbc it is used for all INSERT statements,
not just those that use RETURNING.
third party dialects would need to opt in to the new feature
by setting use_insertmanyvalues to True.
Also adds dialect-level guards against using returning
with executemany where we dont have an implementation to
suit it. execute single w/ returning still defers to the
server without us checking.
Fixes: #6047Fixes: #7907
Change-Id: I3936d3c00003f02e322f2e43fb949d0e6e568304
Range type handling has been enhanced so that it automatically
renders type casts, so that in-place round trips for statements that don't
provide the database with any context don't require the :func:`_sql.cast`
construct to be explicit for the database to know the desired type.
Change-Id: Id630b726f8a23059dd2f4cbc410bf5229d89cbfb
References: #8540
The :class:`_functions.array_agg` will now set the array dimensions to 1.
Improved :class:`_types.ARRAY` processing to accept ``None`` values as
value of a multi-array.
Fixes: #7083
Change-Id: Iafec4f77fde9719ccc7c8535bf6235dbfbc62102
Implemented the DDL event hooks :meth:`.DDLEvents.before_create`,
:meth:`.DDLEvents.after_create`, :meth:`.DDLEvents.before_drop`,
:meth:`.DDLEvents.after_drop` for all :class:`.SchemaItem` objects that
include a distinct CREATE or DROP step, when that step is invoked as a
distinct SQL statement, including for :class:`.ForeignKeyConstraint`,
:class:`.Sequence`, :class:`.Index`, and PostgreSQL's
:class:`_postgresql.ENUM`.
Fixes: #8394
Change-Id: Iee6274984e794f50103451a04d089641d6ac443a
Introduced the type :class:`_postgresql.JSONPATH` that can be used
in cast expressions. This is required by some PostgreSQL dialects
when using functions such as ``jsonb_path_exists`` or
``jsonb_path_match`` that accept a ``jsonpath`` as input.
Fixes: #8216
Change-Id: I3e7337eab91680cab1604e1f3058854a0a19c5be
Ranges now work using a new Range object,
multiranges as lists of Range objects (this is what
asyncpg does. not sure why psycopg has a "Multirange"
type).
psycopg, psycopg2, and asyncpg are currently supported.
It's not clear how to make ranges work with pg8000, likely
needs string conversion; this is straightforward with the
new archicture and can be added later.
Fixes: #8178
Change-Id: Iab8d8382873d5c14199adbe3f09fd0dc17e2b9f1
This adds functionality for PostgreSQL MultiRange type, as discussed in Issue #7156.
As far as I can tell, only psycopg provides a [Multirange adaptation](https://www.psycopg.org/psycopg3/docs/basic/pgtypes.html#multirange-adaptation). Psycopg2 only supports a [Range adaptation/data type](https://www.psycopg.org/psycopg3/docs/basic/pgtypes.html#multirange-adaptation).
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Closes: #7816
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7816
Pull-request-sha: 7e9e0c858d
Change-Id: I345e0f58f534ac37709a7a4627b6de8ddd8fa89e
just in my own testing, if I say insert().return_defaults()
and stringify, I should see it, so make sure all the dialects
default to "insert_returning" etc. , with downgrade on
server version check.
Change-Id: Id64e78fcb03c48b5dcb0feb21cb9cc495edd15e9
Fixed issue in psycopg2 dialect where the "multiple hosts" feature
implemented for 🎫`4392`, where multiple ``host:port`` pairs could be
passed in the query string as
``?host=host1:port1&host=host2:port2&host=host3:port3`` was not implemented
correctly, as it did not propagate the "port" parameter appropriately.
Connections that didn't use a different "port" likely worked without issue,
and connections that had "port" for some of the entries may have
incorrectly passed on that hostname. The format is now corrected to pass
hosts/ports appropriately.
As part of this change, maintained support for another multihost style that
worked unintentionally, which is comma-separated
``?host=h1,h2,h3&port=p1,p2,p3``. This format is more consistent with
libpq's query-string format, whereas the previous format is inspired by a
different aspect of libpq's URI format but is not quite the same thing.
If the two styles are mixed together, an error is raised as this is
ambiguous.
Fixes: #4392
Change-Id: Ic9cc0b0e6e90725e158d9efe73e042853dd1263f
The PostgreSQL dialect now supports reflection of expression based indexes.
The reflection is supported both when using
:meth:`_engine.Inspector.get_indexes` and when reflecting a
:class:`_schema.Table` using :paramref:`_schema.Table.autoload_with`.
Thanks to immerrr and Aidan Kane for the help on this ticket.
Fixes: #7442
Change-Id: I3e36d557235286c0f7f6d8276272ff9225058d48
Adds support for comments on named constraints, including `ForeignKeyConstraint`, `PrimaryKeyConstraint`, `CheckConstraint`, `UniqueConstraint`, solving the [Issue 5667](https://github.com/sqlalchemy/sqlalchemy/issues/5667).
Supports only PostgreSQL backend.
### Description
Following the example of [Issue 1546](https://github.com/sqlalchemy/sqlalchemy/issues/1546), supports comments on constraints. Specifically, enables comments on _named_ ones — as I get it, PostgreSQL prohibits comments on unnamed constraints.
Enables setting the comments for named constraints like this:
```
Table(
'example', metadata,
Column('id', Integer),
Column('data', sa.String(30)),
PrimaryKeyConstraint(
"id", name="id_pk", comment="id_pk comment"
),
CheckConstraint('id < 100', name="cc1", comment="Id value can't exceed 100"),
UniqueConstraint(['data'], name="uc1", comment="Must have unique data field"),
)
```
Provides the DDL representation for constraint comments and routines to create and drop them. Class `.Inspector` reflects constraint comments via methods like `get_check_constraints` .
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- [ ] A short code fix
- [x] A new feature implementation
- Solves the issue 5667.
- The commit message includes `Fixes: 5667`.
- Includes tests based on comment reflection.
**Have a nice day!**
Fixes: #5667Closes: #7742
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7742
Pull-request-sha: 42a5d3c3e9
Change-Id: Ia60f578595afdbd6089541c9a00e37997ef78ad3
given 017fd9ae06 has changed 'name'
to be required for ENUM, add some more adapt from Enum types
of tests.
Change-Id: I447983e640d6466f969aa51c8dff5a0cb8074ca8
Added a new Postgresql :class:`_postgresql.DOMAIN` datatype, which follows
the same CREATE TYPE / DROP TYPE behaviors as that of PostgreSQL
:class:`_postgresql.ENUM`. Much thanks to David Baumgold for the efforts on
this.
Fixes: #7316Closes: #7317
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7317
Pull-request-sha: bc9a82f010
Change-Id: Id8d7e48843a896de17d20cc466b115b3cc065132
Rearchitected the schema reflection API to allow some dialects to make use
of high performing batch queries to reflect the schemas of many tables at
once using much fewer queries. The new performance features are targeted
first at the PostgreSQL and Oracle backends, and may be applied to any
dialect that makes use of SELECT queries against system catalog tables to
reflect tables (currently this omits the MySQL and SQLite dialects which
instead make use of parsing the "CREATE TABLE" statement, however these
dialects do not have a pre-existing performance issue with reflection. MS
SQL Server is still a TODO).
The new API is backwards compatible with the previous system, and should
require no changes to third party dialects to retain compatibility;
third party dialects can also opt into the new system by implementing
batched queries for schema reflection.
Along with this change is an updated reflection API that is fully
:pep:`484` typed, features many new methods and some changes.
Fixes: #4379
Change-Id: I897ec09843543aa7012bcdce758792ed3d415d08
as we already implement stringification for the contents,
provide a bracketed syntax for default and ARRAY literal
for PG specifically. ARRAY literal seems much simpler to
render than their quoted syntax which requires double quotes
for strings.
also open up testing for pg8000 which has likely been
fine with arrays for awhile now, bump the version pin
also.
Fixes: #8138
Change-Id: Id85b052b0a9564d6aa1489160e58b7359f130fdd
Added new backend-agnostic :class:`_types.Uuid` datatype generalized from
the PostgreSQL dialects to now be a core type, as well as migrated
:class:`_types.UUID` from the PostgreSQL dialect. Thanks to Trevor Gross
for the help on this.
also includes:
* corrects some missing behaviors in the suite literal fixtures
test where row round trips weren't being correctly asserted.
* fixes some of the ISO literal date rendering added in
952383f9ee for #5052 to truncate datetime strings for date/time
datatypes in the same way that drivers typically do for bound
parameters; this was not working fully and wasn't caught by the
broken test fixture
Fixes: #7212
Change-Id: I981ac6d34d278c18281c144430a528764c241b04
jazzthief