Added support for the pow operator (``**``), with a default SQL
implementation of the ``POW()`` function. On Oracle Database, PostgreSQL
and MSSQL it renders as ``POWER()``. As part of this change, the operator
routes through a new first class ``func`` member :class:`_functions.pow`,
which renders on Oracle Database, PostgreSQL and MSSQL as ``POWER()``.
Fixes: #8579Closes: #8580
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8580
Pull-request-sha: 041b2ef474
Change-Id: I371bd44ed3e58f2d55ef705aeec7d04710c97f23
This is supported both for schema definition and reflection.
Fixes#10665.
Closes: #12485
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12485
Pull-request-sha: 1aabea7b55ece9fc0c6e069b777d4404ac01f964
Change-Id: I81d23966f84390dd1b03f0d13284ce6d883ee24e
When building a PostgreSQL ``ARRAY`` literal using
:class:`_postgresql.array` with an empty ``clauses`` argument, the
:paramref:`_postgresql.array.type_` parameter is now significant in that it
will be used to render the resulting ``ARRAY[]`` SQL expression with a
cast, such as ``ARRAY[]::INTEGER``. Pull request courtesy Denis Laxalde.
Fixes: #12432Closes: #12435
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12435
Pull-request-sha: 9633d3c15d
Change-Id: I29ed7bd0562b82351d22de0658fb46c31cfe44f6
Added syntax extension :func:`_postgresql.distinct_on` to build ``DISTINCT
ON`` clauses. The old api, that passed columns to
:meth:`_sql.Select.distinct`, is now deprecated.
Fixes: #12342
Change-Id: Ia6a7e647a11e57b6ac2f50848778c20dc55eaf54
Added support for specifying a list of columns for ``SET NULL`` and ``SET
DEFAULT`` actions of ``ON DELETE`` clause of foreign key definition on
PostgreSQL. Pull request courtesy Denis Laxalde.
Fixes: #11595Closes: #12421
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12421
Pull-request-sha: d0394db706
Change-Id: I036a559ae4a8efafe9ba64d776a840bd785a7397
Fixed compiler issue in the PostgreSQL dialect where incorrect keywords
would be passed when using "FOR UPDATE OF" inside of a subquery.
Fixes: #12417
Change-Id: I6255d165e8e719e1786e78aa60ee8e6a95af1dcb
in 2.1 we want these structures to be cacheable, so start
by cleaning up types and adding coercions to enforce those types.
these will be more locked down in 2.1 as we will need to move
bound parameter coercion outside of compilation, but here
do some small starts and introduce in 2.0.
in one interest of cachability, a "literal_binds" that found
its way into SQLite's compiler is replaced with "literal_execute",
the difference being that the latter is cacheable. This literal
is apparently necessary to suit SQLite's query planner for
the "index criteria" portion of the on conflict clause that otherwise
can't work with a real bound parameter.
Change-Id: I4d66ec1473321616a1707da324a7dfe7a61ec94e
Correctly type PostgreSQL RANGE and MULTIRANGE types as ``Range[T]``
and ``Sequence[Range[T]]``.
Introduced utility sequence ``MultiRange`` to allow better
interoperability of MULTIRANGE types.
Fixes#9736Closes: #10625
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10625
Pull-request-sha: 2c17bc5f92
Change-Id: I4f91d0233b29fd8101e67bdd4cd0aa2524ab788a
manually update the files to remove literal string concat on the same line,
since black does not seem to be making progress in handling these
Change-Id: I3c651374c5f3db5b8bc0c700328d67ca03743b7b
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [x] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Fixes#10904
**Have a nice day!**
Closes: #10905
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10905
Pull-request-sha: 85f232a303
Change-Id: I5e2fc05a696eb6da71bbd695f0466e8552d203b6
the expression clauselist feature added in #7744 failed to accommodate
this parameter that is used only by the PostgreSQL JSON
operators.
Fixed 2.0 regression caused by 🎫`7744` where chains of expressions
involving PostgreSQL JSON operators combined with other operators such as
string concatenation would lose correct parenthesization, due to an
implementation detail specific to the PostgreSQL dialect.
Fixes: #10479
Change-Id: Ic168bf6afd8bf1cfa648f2bad22fdd7254feaa34
Fixed issue where the :meth:`_sql.ColumnOperators.regexp_match`
when using "flags" would not produce a "stable" cache key, that
is, the cache key would keep changing each time causing cache pollution.
The same issue existed for :meth:`_sql.ColumnOperators.regexp_replace`
with both the flags and the actual replacement expression.
The flags are now represented as fixed modifier strings rendered as
safestrings rather than bound parameters, and the replacement
expression is established within the primary portion of the "binary"
element so that it generates an appropriate cache key.
Note that as part of this change, the
:paramref:`_sql.ColumnOperators.regexp_match.flags` and
:paramref:`_sql.ColumnOperators.regexp_replace.flags` have been modified to
render as literal strings only, whereas previously they were rendered as
full SQL expressions, typically bound parameters. These parameters should
always be passed as plain Python strings and not as SQL expression
constructs; it's not expected that SQL expression constructs were used in
practice for this parameter, so this is a backwards-incompatible change.
The change also modifies the internal structure of the expression
generated, for :meth:`_sql.ColumnOperators.regexp_replace` with or without
flags, and for :meth:`_sql.ColumnOperators.regexp_match` with flags. Third
party dialects which may have implemented regexp implementations of their
own (no such dialects could be located in a search, so impact is expected
to be low) would need to adjust the traversal of the structure to
accommodate.
Fixed issue in mostly-internal :class:`.CacheKey` construct where the
``__ne__()`` operator were not properly implemented, leading to nonsensical
results when comparing :class:`.CacheKey` instances to each other.
Fixes: #10042
Change-Id: I2e245f81d7ee7136ad04cf77be35f9745c5da5e5
Fixed issue where the :paramref:`.ColumnOperators.like.escape` and similar
parameters did not allow an empty string as an argument that would be
passed through as the "escape" character; this is a supported syntax by
PostgreSQL. Pull requset courtesy Martin Caslavsky.
Fixes: #9907Closes: #9908
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9908
Pull-request-sha: d7ecc1778a
Change-Id: I39adb765a1b9650fe891883ed0973df66adc4e81
Added support for PostgreSQL 10 ``NULLS NOT DISTINCT`` feature of
unique indexes and unique constraint using the dialect option
postgresql_nulls_not_distinct``.
Updated the reflection logic to also correctly take this option
into account.
Fixes: #8240Closes: #9834
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9834
Pull-request-sha: 825a4ff13a
Change-Id: I6585fbb05ad32a131cf9ba25a59f7b229bce5b52
Added a full suite of new SQL bitwise operators, for performing
database-side bitwise expressions on appropriate data values such as
integers, bit-strings, and similar. Pull request courtesy Yegor Statkevich.
Fixes: #8780Closes: #9204
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/9204
Pull-request-sha: a4541772a6
Change-Id: I4c70e80f9548dcc1b4e3dccd71bd59d51d3ed46e
Added support for explicit use of PG full text functions with asyncpg and
psycopg (SQLAlchemy 2.0 only), with regards to the ``REGCONFIG`` type cast
for the first argument, which previously would be incorrectly cast to a
VARCHAR, causing failures on these dialects that rely upon explicit type
casts. This includes support for :class:`_postgresql.to_tsvector`,
:class:`_postgresql.to_tsquery`, :class:`_postgresql.plainto_tsquery`,
:class:`_postgresql.phraseto_tsquery`,
:class:`_postgresql.websearch_to_tsquery`,
:class:`_postgresql.ts_headline`, each of which will determine based on
number of arguments passed if the first string argument should be
interpreted as a PostgreSQL "REGCONFIG" value; if so, the argument is typed
using a newly added type object :class:`_postgresql.REGCONFIG` which is
then explicitly cast in the SQL expression.
Fixes: #8977
Change-Id: Ib36698a984fd4194bd6e0eb663105f790f3db7d3
Fixed a series of issues regarding positionally rendered bound parameters,
such as those used for SQLite, asyncpg, MySQL and others. Some compiled
forms would not maintain the order of parameters correctly, such as the
PostgreSQL ``regexp_replace()`` function as well as within the "nesting"
feature of the :class:`.CTE` construct first introduced in 🎫`4123`.
Fixes: #8827
Change-Id: I9813ed7c358cc5c1e26725c48df546b209a442cb
Added additional type-detection for the new PostgreSQL
:class:`_postgresql.Range` type, where previous cases that allowed the
psycopg2-native range objects to be received directly by the DBAPI without
SQLAlchemy intercepting them stopped working, as we now have our own value
object. The :class:`_postgresql.Range` object has been enhanced such that
SQLAlchemy Core detects it in otherwise ambiguous situations (such as
comparison to dates) and applies appropriate bind handlers. Pull request
courtesy Lele Gaifax.
Fixes: #8884Closes: #8886
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8886
Pull-request-sha: 6e95e08a30
Change-Id: I3ca277c826dcf4b5644f44eb251345b439a84ee4
command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>"
pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not
exists in sqlalchemy fixtures
Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
The RETURNING clause now renders columns using the routine as that of the
:class:`.Select` to generate labels, which will include disambiguating
labels, as well as that a SQL function surrounding a named column will be
labeled using the column name itself. This is a more comprehensive change
than a similar one made for the 1.4 series that adjusted the function label
issue only.
includes 1.4's changelog for the backported version which also
fixes an Oracle issue independently of the 2.0 series.
Fixes: #8770
Change-Id: I2ab078a214a778ffe1720dbd864ae4c105a0691d
The :class:`.Sequence` construct restores itself to the DDL behavior it
had prior to the 1.4 series, where creating a :class:`.Sequence` with
no additional arguments will emit a simple ``CREATE SEQUENCE`` instruction
**without** any additional parameters for "start value". For most backends,
this is how things worked previously in any case; **however**, for
MS SQL Server, the default value on this database is
``-2**63``; to prevent this generally impractical default
from taking effect on SQL Server, the :paramref:`.Sequence.start` parameter
should be provided. As usage of :class:`.Sequence` is unusual
for SQL Server which for many years has standardized on ``IDENTITY``,
it is hoped that this change has minimal impact.
Fixes: #7211
Change-Id: I1207ea10c8cb1528a1519a0fb3581d9621c27b31
also adjusted CacheKeyFixture to be a general purpose
fixture so that sub-components / dialects can run
their own cache key tests.
Fixes: #8574
Change-Id: I6c66107856aee11e548d357cea77bceee3e316a0
Introduced the type :class:`_postgresql.JSONPATH` that can be used
in cast expressions. This is required by some PostgreSQL dialects
when using functions such as ``jsonb_path_exists`` or
``jsonb_path_match`` that accept a ``jsonpath`` as input.
Fixes: #8216
Change-Id: I3e7337eab91680cab1604e1f3058854a0a19c5be
just in my own testing, if I say insert().return_defaults()
and stringify, I should see it, so make sure all the dialects
default to "insert_returning" etc. , with downgrade on
server version check.
Change-Id: Id64e78fcb03c48b5dcb0feb21cb9cc495edd15e9
Added a new Postgresql :class:`_postgresql.DOMAIN` datatype, which follows
the same CREATE TYPE / DROP TYPE behaviors as that of PostgreSQL
:class:`_postgresql.ENUM`. Much thanks to David Baumgold for the efforts on
this.
Fixes: #7316Closes: #7317
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7317
Pull-request-sha: bc9a82f010
Change-Id: Id8d7e48843a896de17d20cc466b115b3cc065132
The :meth:`.Operators.match` operator now uses ``plainto_tsquery()`` for
PostgreSQL full text search, rather than ``to_tsquery()``. The rationale
for this change is to provide better cross-compatibility with match on
other database backends. Full support for all PostgreSQL full text
functions remains available through the use of :data:`.func` in
conjunction with :meth:`.Operators.bool_op` (an improved version of
:meth:`.Operators.op` for boolean operators).
Additional doc updates here apply to 1.4 so will backport these
out to a separate commit.
Fixes: #7086
Change-Id: I1946075daf5d9c558e85f73f1bf852604b3b1b8c
Fixed bug where the PostgreSQL :meth:`_postgresql.Insert.on_conflict`
method and the SQLite :meth:`_sqlite.Insert.on_conflict` method would both
fail to correctly accommodate a column with a separate ".key" when
specifying the column using its key name in the dictionary passed to
``set_``, as well as if the :attr:`_sqlite.Insert.excluded` or
:attr:`_postgresql.Insert.excluded` collection were used as the dictionary
directly.
Fixes: #8014
Change-Id: I67226aeedcb2c683e22405af64720cc1f990f274
Fixed bug in :class:`.ARRAY` datatype in combination with :class:`.Enum` on
PostgreSQL where using the ``.any()`` method to render SQL ANY(), given
members of the Python enumeration as arguments, would produce a type
adaptation failure on all drivers.
Fixes: #6515
Change-Id: Ia1e3b4e10aaf264ed436ce6030d105fc60023433
Fixed regression caused by 🎫`7760` where the new capabilities of
:class:`.TextualSelect` were not fully implemented within the compiler
properly, leading to issues with composed INSERT constructs such as "INSERT
FROM SELECT" and "INSERT...ON CONFLICT" when combined with CTE and textual
statements.
Fixes: #7798
Change-Id: Ia2ce92507e574dd36fd26dd38ec9dd2713584467
Added :class:`.Double`, :class:`.DOUBLE`, :class:`.DOUBLE_PRECISION`
datatypes to the base ``sqlalchemy.`` module namespace, for explicit use of
double/double precision as well as generic "double" datatypes. Use
:class:`.Double` for generic support that will resolve to DOUBLE/DOUBLE
PRECISION/FLOAT as needed for different backends.
Implemented DDL and reflection support for ``FLOAT`` datatypes which
include an explicit "binary_precision" value. Using the Oracle-specific
:class:`_oracle.FLOAT` datatype, the new parameter
:paramref:`_oracle.FLOAT.binary_precision` may be specified which will
render Oracle's precision for floating point types directly. This value is
interpreted during reflection. Upon reflecting back a ``FLOAT`` datatype,
the datatype returned is one of :class:`_types.DOUBLE_PRECISION` for a
``FLOAT`` for a precision of 126 (this is also Oracle's default precision
for ``FLOAT``), :class:`_types.REAL` for a precision of 63, and
:class:`_oracle.FLOAT` for a custom precision, as per Oracle documentation.
As part of this change, the generic :paramref:`_sqltypes.Float.precision`
value is explicitly rejected when generating DDL for Oracle, as this
precision cannot be accurately converted to "binary precision"; instead, an
error message encourages the use of
:meth:`_sqltypes.TypeEngine.with_variant` so that Oracle's specific form of
precision may be chosen exactly. This is a backwards-incompatible change in
behavior, as the previous "precision" value was silently ignored for
Oracle.
Fixes: #5465Closes: #7674
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7674
Pull-request-sha: 5c68419e5a
Change-Id: I831f4af3ee3b23fde02e8f6393c83e23dd7cd34d
Added compiler support for the PostgreSQL ``NOT VALID`` phrase when rendering
DDL for the :class:`.CheckConstraint`, :class:`.ForeignKeyConstraint`
and :class:`.ForeignKey` schema constructs. Pull request courtesy
Gilbert Gilb's.
Fixes: #7600Closes: #7601
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7601
Pull-request-sha: 78eecd55fd
Change-Id: I84bfe84596856eeea2bcca45c04ad23d980a75ec
Implemented full support for "truediv" and "floordiv" using the
"/" and "//" operators. A "truediv" operation between two expressions
using :class:`_types.Integer` now considers the result to be
:class:`_types.Numeric`, and the dialect-level compilation will cast
the right operand to a numeric type on a dialect-specific basis to ensure
truediv is achieved. For floordiv, conversion is also added for those
databases that don't already do floordiv by default (MySQL, Oracle) and
the ``FLOOR()`` function is rendered in this case, as well as for
cases where the right operand is not an integer (needed for PostgreSQL,
others).
The change resolves issues both with inconsistent behavior of the
division operator on different backends and also fixes an issue where
integer division on Oracle would fail to be able to fetch a result due
to inappropriate outputtypehandlers.
Fixes: #4926
Change-Id: Id54cc018c1fb7a49dd3ce1216d68d40f43fe2659
Add a new system so that PostgreSQL and other dialects have a
reliable way to add casts to bound parameters in SQL statements,
replacing previous use of setinputsizes() for PG dialects.
rationale:
1. psycopg3 will be using the same SQLAlchemy-side "setinputsizes"
as asyncpg, so we will be seeing a lot more of this
2. the full rendering that SQLAlchemy's compilation is performing
is in the engine log as well as error messages. Without this,
we introduce three levels of SQL rendering, the compiler, the
hidden "setinputsizes" in SQLAlchemy, and then whatever the DBAPI
driver does. With this new approach, users reporting bugs etc.
will be less confused that there are as many as two separate
layers of "hidden rendering"; SQLAlchemy's rendering is again
fully transparent
3. calling upon a setinputsizes() method for every statement execution
is expensive. this way, the work is done behind the caching layer
4. for "fast insertmany()", I also want there to be a fast approach
towards setinputsizes. As it was, we were going to be taking
a SQL INSERT with thousands of bound parameter placeholders and
running a whole second pass on it to apply typecasts. this way,
we will at least be able to build the SQL string once without a huge
second pass over the whole string
5. psycopg2 can use this same system for its ARRAY casts
6. the general need for PostgreSQL to have lots of type casts
is now mostly in the base PostgreSQL dialect and works independently
of a DBAPI being present. dependence on DBAPI symbols that aren't
complete / consistent / hashable is removed
I was originally going to try to build this into bind_expression(),
but it was revealed this worked poorly with custom bind_expression()
as well as empty sets. the current impl also doesn't need to
run a second expression pass over the POSTCOMPILE sections, which
came out better than I originally thought it would.
Change-Id: I363e6d593d059add7bcc6d1f6c3f91dd2e683c0c
the _CompileLabel class included ``__slots__`` but these
weren't used as the superclasses included slots.
Create a ``__slots__`` superclass for ``ClauseElement``,
creating a new class of compilable SQL elements that don't
include heavier features like caching, annotations and
cloning, which are meant to be used only in an ad-hoc
compiler fashion. Create new ``CompilerColumnElement``
from that which serves in column-oriented contexts, but
similarly does not include any expression operator support
as it is intended to be used only to generate a string.
Apply this to both
``_CompileLabel`` as well as PostgreSQL ``_ColonCast``,
which does not actually subclass ``ColumnElement`` as this
class has memoized attributes that aren't worth changing,
and does not include SQL operator capabilities as these
are not needed for these compiler-only objects.
this allows us to more inexpensively add new ad-hoc
labels / casts etc. at compile time, as we will be seeking
to expand out the typecasts that are needed for PostgreSQL
dialects in a subsequent patch.
Change-Id: I52973ae3295cb6e2eb0d7adc816c678a626643ed
The :paramref:`_sa.create_engine.implicit_returning` parameter is
deprecated on the :func:`_sa.create_engine` function only; the parameter
remains available on the :class:`_schema.Table` object. This parameter was
originally intended to enable the "implicit returning" feature of
SQLAlchemy when it was first developed and was not enabled by default.
Under modern use, there's no reason this parameter should be disabled, and
it has been observed to cause confusion as it degrades performance and
makes it more difficult for the ORM to retrieve recently inserted server
defaults. The parameter remains available on :class:`_schema.Table` to
specifically suit database-level edge cases which make RETURNING
infeasible, the sole example currently being SQL Server's limitation that
INSERT RETURNING may not be used on a table that has INSERT triggers on it.
Also removed from the Oracle dialect some logic that would upgrade
an Oracle 8/8i server version to use implicit returning if the
parameter were explictly passed; these versions of Oracle
still support RETURNING so the feature is now enabled for all
Oracle versions.
Fixes: #6962
Change-Id: Ib338e300cd7c8026c3083043f645084a8211aed8
a few changes for py2k:
* map_imperatively() includes the check that a class
is being sent, this was only working for mapper() before
* the test suite didn't place the py2k "autouse" workaround
in the correct order, seemingly, tried to adjust the
per-test ordering setup in pytestplugin.py
Change-Id: I4cc39630724e810953cfda7b2afdadc8b948e3c2
Adjusted the "from linter" warning feature to accommodate for a chain of
joins more than one level deep where the ON clauses don't explicitly match
up the targets, such as an expression such as "ON TRUE". This mode of use
is intended to cancel the cartesian product warning simply by the fact that
there's a JOIN from "a to b", which was not working for the case where the
chain of joins had more than one element.
this incurs a bit more compiler overhead that comes out in profiling
but is not extensive.
Added the "is_comparison" flag to the PostgreSQL "overlaps",
"contained_by", "contains" operators, so that they work in relevant ORM
contexts as well as in conjunction with the "from linter" feature.
Fixes: #6886
Change-Id: I078dc3fe6d4f7871ffe4ebac3e71e62f3f213d12
Federico Caselli