- fix mypy and linter issues
- expand BitString tests for int and bytes conversion
- Only run BIT test types on postgresql dialect
- Fix problems in `BitString.from_int` and `BitString.from_bytes`
- Add `BitString.to_bytes` method to allow specifying the byte
length of the returned instances
- Fixed problems testing operators in BIT type tests
- Added custom `BitString` implementation which inherits from `str`
and overrides methods and operators appropriately and
implements the bitwise operations in a manner consistent with
postgresql. Also exposes class and instance methods ensuring
instances can be converted to/from `int` and `bytes` sensibly.
- Added default implementations of `bind_processor` and
`result_processor` to `postgresql.BIT` type to convert `BitString`
to/from string for PG drivers.
- Override `bind_processor` and `result_processor` in AsyncpgBit
in order to convert `BitString` to/from `asynpg.BitString`
in `asyncpg` driver.
- Added support for bitwise comparators to postgresql `BIT` instances
Fixes: 10556
Added support for the pow operator (``**``), with a default SQL
implementation of the ``POW()`` function. On Oracle Database, PostgreSQL
and MSSQL it renders as ``POWER()``. As part of this change, the operator
routes through a new first class ``func`` member :class:`_functions.pow`,
which renders on Oracle Database, PostgreSQL and MSSQL as ``POWER()``.
Fixes: #8579Closes: #8580
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8580
Pull-request-sha: 041b2ef474
Change-Id: I371bd44ed3e58f2d55ef705aeec7d04710c97f23
This is supported both for schema definition and reflection.
Fixes#10665.
Closes: #12485
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12485
Pull-request-sha: 1aabea7b55ece9fc0c6e069b777d4404ac01f964
Change-Id: I81d23966f84390dd1b03f0d13284ce6d883ee24e
When building a PostgreSQL ``ARRAY`` literal using
:class:`_postgresql.array` with an empty ``clauses`` argument, the
:paramref:`_postgresql.array.type_` parameter is now significant in that it
will be used to render the resulting ``ARRAY[]`` SQL expression with a
cast, such as ``ARRAY[]::INTEGER``. Pull request courtesy Denis Laxalde.
Fixes: #12432Closes: #12435
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12435
Pull-request-sha: 9633d3c15d
Change-Id: I29ed7bd0562b82351d22de0658fb46c31cfe44f6
Added syntax extension :func:`_postgresql.distinct_on` to build ``DISTINCT
ON`` clauses. The old api, that passed columns to
:meth:`_sql.Select.distinct`, is now deprecated.
Fixes: #12342
Change-Id: Ia6a7e647a11e57b6ac2f50848778c20dc55eaf54
Added support for specifying a list of columns for ``SET NULL`` and ``SET
DEFAULT`` actions of ``ON DELETE`` clause of foreign key definition on
PostgreSQL. Pull request courtesy Denis Laxalde.
Fixes: #11595Closes: #12421
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12421
Pull-request-sha: d0394db706
Change-Id: I036a559ae4a8efafe9ba64d776a840bd785a7397
Fixed issue in PostgreSQL network types :class:`_postgresql.INET`,
:class:`_postgresql.CIDR`, :class:`_postgresql.MACADDR`,
:class:`_postgresql.MACADDR8` where sending string values to compare to
these types would render an explicit CAST to VARCHAR, causing some SQL /
driver combinations to fail. Pull request courtesy Denis Laxalde.
Fixes: #12060Closes: #12412
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12412
Pull-request-sha: 029fda7f2d
Change-Id: Id4a502ebc119775567cacddbabef2ce9715c1a9f
Fixed compiler issue in the PostgreSQL dialect where incorrect keywords
would be passed when using "FOR UPDATE OF" inside of a subquery.
Fixes: #12417
Change-Id: I6255d165e8e719e1786e78aa60ee8e6a95af1dcb
Add SQL typing to reflection query used to retrieve a the structure
of IDENTITY columns, adding explicit JSON typing to the query to suit
unusual PostgreSQL driver configurations that don't support JSON natively.
Fixed issue affecting PostgreSQL 17.3 and greater where reflection of
domains with "NOT NULL" as part of their definition would include an
invalid constraint entry in the data returned by
:meth:`_postgresql.PGInspector.get_domains` corresponding to an additional
"NOT NULL" constraint that isn't a CHECK constraint; the existing
``"nullable"`` entry in the dictionary already indicates if the domain
includes a "not null" constraint. Note that such domains also cannot be
reflected on PostgreSQL 17.0 through 17.2 due to a bug on the PostgreSQL
side; if encountering errors in reflection of domains which include NOT
NULL, upgrade to PostgreSQL server 17.3 or greater.
Fixes: #11751
Change-Id: I8e69de51601dca3257186e38c6f699fbfd9014c6
Adjusted the asyncpg connection wrapper so that the asyncpg
``.transaction()`` call sends ``None`` for isolation_level if not otherwise
set in the SQLAlchemy dialect/wrapper, thereby allowing asyncpg to make use
of the server level setting for isolation_level in the absense of a
client-level setting. Previously, this behavior of asyncpg was blocked by a
hardcoded ``read_committed``.
Fixes: #12159
Change-Id: I2cd878a5059a8fefc9557a9b8e056fedaee2e9a4
Adjusted the asyncpg dialect so that an empty SQL string, which is valid
for PostgreSQL server, may be successfully processed at the dialect level,
such as when using :meth:`.Connection.exec_driver_sql`. Pull request
courtesy Andrew Jackson.
Closes: #12220
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12220
Pull-request-sha: 45c94febee
Change-Id: I870df9e31f4a229939e76c702724c25073329282
in 2.1 we want these structures to be cacheable, so start
by cleaning up types and adding coercions to enforce those types.
these will be more locked down in 2.1 as we will need to move
bound parameter coercion outside of compilation, but here
do some small starts and introduce in 2.0.
in one interest of cachability, a "literal_binds" that found
its way into SQLite's compiler is replaced with "literal_execute",
the difference being that the latter is cacheable. This literal
is apparently necessary to suit SQLite's query planner for
the "index criteria" portion of the on conflict clause that otherwise
can't work with a real bound parameter.
Change-Id: I4d66ec1473321616a1707da324a7dfe7a61ec94e
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
Fixes#12093
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Closes: #12094
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12094
Pull-request-sha: 3f900e96b9
Change-Id: I4c3945eec6a931acd0a8c1682988c5f26e96a499
Render bind cast for ``JSON`` and ``JSONB`` datatype on every dialect.
Previously this was only enabled in a subset of dialects.
Fixes: #11994
Change-Id: Ib085deb3e84034dac9e4f4057d32f055d5533e52
Fixed critical issue in the asyncpg driver where a rollback or commit that
fails specifically for the ``MissingGreenlet`` condition or any other error
that is not raised by asyncpg itself would discard the asyncpg transaction
in any case, even though the transaction were still idle, leaving to a
server side condition with an idle transaction that then goes back into the
connection pool. The flags for "transaction closed" are now not reset for
errors that are raised outside of asyncpg itself. When asyncpg itself
raises an error for ``.commit()`` or ``.rollback()``, asyncpg does then
discard of this transaction.
Fixes: #11819
Change-Id: I12f0532788b03ea63fb47a7af21e07c37effb070
The PostgreSQL dialect now returns :class:`_postgresql.DOMAIN` instances
when reflecting a column that has a domain as type.
Previously the domain data type was returned instead.
As part of this change, the domain reflection was improved to also
return the collation of the text types.
Fixes: #10693Closes: #10729
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10729
Pull-request-sha: adac164d19
Change-Id: I8730840de2e7e9649067191430eefa086bcf5e7b
Fixed regression caused by just-released fix for 🎫`10863` where an
invalid exception class were added to the "except" block, which does not
get exercised unless such a catch actually happens. A mock-style test has
been added to ensure this catch is exercised in unit tests.
Fixes: #11005
Change-Id: I5a65403fb7bb35296ff44ae3cf6a336f8e0bda97
Correctly type PostgreSQL RANGE and MULTIRANGE types as ``Range[T]``
and ``Sequence[Range[T]]``.
Introduced utility sequence ``MultiRange`` to allow better
interoperability of MULTIRANGE types.
Fixes#9736Closes: #10625
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10625
Pull-request-sha: 2c17bc5f92
Change-Id: I4f91d0233b29fd8101e67bdd4cd0aa2524ab788a
manually update the files to remove literal string concat on the same line,
since black does not seem to be making progress in handling these
Change-Id: I3c651374c5f3db5b8bc0c700328d67ca03743b7b
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Fixes#10904
**Have a nice day!**
Closes: #10905
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10905
Pull-request-sha: 85f232a303
Change-Id: I5e2fc05a696eb6da71bbd695f0466e8552d203b6
Added support for reflection of PostgreSQL CHECK constraints marked with
"NO INHERIT", setting the key ``no_inherit=True`` in the reflected data.
Pull request courtesy Ellis Valentiner.
Fixes: #10777Closes: #10778
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10778
Pull-request-sha: 058082ff62
Change-Id: Ia33e29c0c57cf0076e8819311f4628d712fdc332
Fixed SQL handling for "insertmanyvalues" when using the
:class:`.postgresql.BIT` datatype with the asyncpg backend. The
:class:`.postgresql.BIT` on asyncpg apparently requires the use of an
asyncpg-specific `BitString` type which is currently exposed when using
this DBAPI, making it incompatible with other PostgreSQL DBAPIs that all
work with plain bitstrings here. A future fix in version 2.1 will
normalize this datatype across all PG backends. Pull request courtesy
Sören Oldag.
Fixes: #10532Closes: #10533
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10533
Pull-request-sha: 528607d5b5
Change-Id: I879f2fc24b5650467b85f170c0def15834f75456
Implemented "literal value processing" for the :class:`.Interval` datatype
for both the PostgreSQL and Oracle dialects, allowing literal rendering of
interval values. Pull request courtesy Indivar Mishra.
Fixes: #9737Closes: #10383
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10383
Pull-request-sha: bf3a73dfd9d329779e12037ae62dc1032e76d0f6
Change-Id: Ic1a1c505f23eeb681059303799d5fc8821aadacf
the expression clauselist feature added in #7744 failed to accommodate
this parameter that is used only by the PostgreSQL JSON
operators.
Fixed 2.0 regression caused by 🎫`7744` where chains of expressions
involving PostgreSQL JSON operators combined with other operators such as
string concatenation would lose correct parenthesization, due to an
implementation detail specific to the PostgreSQL dialect.
Fixes: #10479
Change-Id: Ic168bf6afd8bf1cfa648f2bad22fdd7254feaa34
Fixed a series of reflection issues affecting the PostgreSQL,
MySQL/MariaDB, and SQLite dialects when reflecting foreign key constraints
where the target column contained parenthesis in one or both of the table
name or column name.
Fixes: #10275
Change-Id: Ia2393d45416af6b36e7cab4ee10c2ade7a7e49b3
Fixed regression which appeared in 2.0 due to 🎫`8491` where the
revised "ping" used for PostgreSQL dialects when the
:paramref:`_sa.create_engine.pool_pre_ping` parameter is in use would
interfere with the use of asyncpg with PGBouncer "transaction" mode, as the
multiple PostgreSQL commands emitted by asnycpg could be broken out among
multiple connections leading to errors, due to the lack of any transaction
around this newly revised "ping". The ping is now invoked within a
transaction, in the same way that is implicit with all other backends that
are based on the pep-249 DBAPI; this guarantees that the series of PG
commands sent by asyncpg for this command are invoked on the same backend
connection without it jumping to a different connection mid-command. The
transaction is not used if the asyncpg dialect is used in "AUTOCOMMIT"
mode, which remains incompatible with pgbouncer transaction mode.
Fixes: #10226
Change-Id: I93156ad7c353a865d93e5296bd7bc137f0350f3c
Fixed issue where comparisons to the :class:`_postgresql.CITEXT` datatype
would cast the right side to ``VARCHAR``, leading to the right side not
being interpreted as a ``CITEXT`` datatype, for the asyncpg, psycopg3 and
pg80000 dialects. This led to the :class:`_postgresql.CITEXT` type being
essentially unusable for practical use; this is now fixed and the test
suite has been corrected to properly assert that expressions are rendered
correctly.
Fixes: #10096
Change-Id: I49129e50261cf09942c0c339d581ce17a26d8181
silently ignore the host portion of the URL if host=xyz
is present, and document this.
Fixes: #10076
Change-Id: I370b138aec63a9a2aa6029fe9e66eb1659434a43
update for a2c06a2a0a
updates test suite to include direct expected data / errors in
the test data
Fixes: #10069
Change-Id: I1e689101b90b7469608b74ed37abd7c2122151a4
Fixed regression caused by improvements to PostgreSQL URL parsing in
🎫`10004` where "host" query string arguments that had colons in
them, to support various third party proxy servers and/or dialects, would
not parse correctly as these were evaluted as ``host:port`` combinations.
Parsing has been updated to consider a colon as indicating a ``host:port``
value only if the hostname contains only alphanumeric characters with dots
or dashes only (e.g. no slashes), followed by exactly one colon followed by
an all-integer token of zero or more integers. In all other cases, the
full string is taken as a host.
Fixes: #10069
Change-Id: I77beb27e44abc0a66aa0810de855daa4186dacfd
Added new parameter ``native_inet_types=False`` to the all PostgreSQL
dialects, which indicates the all converters used by the DBAPI to
convert rows from PostgreSQL :class:`.INET` and :class:`.CIDR` columns
into Python ``ipaddress`` datatypes should be disabled, returning strings
instead. This allows code written to work with strings for these datatypes
to be migrated to asyncpg, psycopg, or pg8000 without code changes
beyond the engine parameter.
Currently, some DBAPIs return ``ipaddress`` objects while others return
strings for one or both of these datatypes. A future release of
SQLAlchemy will attempt to normalize support for Python's ``ipaddress``
across all DBAPIs.
Fixes: #9945
Change-Id: Id59e9982e2bfd6706fe335e4e700902abfb63663
Fixed issue where the :meth:`_sql.ColumnOperators.regexp_match`
when using "flags" would not produce a "stable" cache key, that
is, the cache key would keep changing each time causing cache pollution.
The same issue existed for :meth:`_sql.ColumnOperators.regexp_replace`
with both the flags and the actual replacement expression.
The flags are now represented as fixed modifier strings rendered as
safestrings rather than bound parameters, and the replacement
expression is established within the primary portion of the "binary"
element so that it generates an appropriate cache key.
Note that as part of this change, the
:paramref:`_sql.ColumnOperators.regexp_match.flags` and
:paramref:`_sql.ColumnOperators.regexp_replace.flags` have been modified to
render as literal strings only, whereas previously they were rendered as
full SQL expressions, typically bound parameters. These parameters should
always be passed as plain Python strings and not as SQL expression
constructs; it's not expected that SQL expression constructs were used in
practice for this parameter, so this is a backwards-incompatible change.
The change also modifies the internal structure of the expression
generated, for :meth:`_sql.ColumnOperators.regexp_replace` with or without
flags, and for :meth:`_sql.ColumnOperators.regexp_match` with flags. Third
party dialects which may have implemented regexp implementations of their
own (no such dialects could be located in a search, so impact is expected
to be low) would need to adjust the traversal of the structure to
accommodate.
Fixed issue in mostly-internal :class:`.CacheKey` construct where the
``__ne__()`` operator were not properly implemented, leading to nonsensical
results when comparing :class:`.CacheKey` instances to each other.
Fixes: #10042
Change-Id: I2e245f81d7ee7136ad04cf77be35f9745c5da5e5
Thomas Stephenson