Adjusted the :meth:`_types.ARRAY.Comparator.any` and
:meth:`_types.ARRAY.Comparator.all` methods to implement a straight "NOT"
operation for negation, rather than negating the comparison operator.
Fixes: #5518
Change-Id: I87ee9278c321aafe51a679fcfcbb5fbb11307fda
(cherry picked from commit 8bc793c4db)
Added new accessors to expressions of type :class:`.JSON` to allow for
specific datatype access and comparison, covering strings, integers,
numeric, boolean elements. This revises the documented approach of
CASTing to string when comparing values, instead adding specific
functionality into the PostgreSQL, SQlite, MySQL dialects to reliably
deliver these basic types in all cases.
The change also delivers a new feature to the test exclusions
system so that combinations and exclusions can be used together.
Fixes: #4276
Change-Id: Ica5a926c060feb40a0a7cd60b9d6e061d7825728
(cherry picked from commit 01cbf4d7b8)
Add factilities to implement pytest.mark.parametrize and
pytest.fixtures patterns, which largely resemble things we are
already doing.
Ensure a facade is used, so that the test suite remains independent
of py.test, but also tailors the functions to the more limited
scope in which we are using them.
Additionally, create a class-based version that works from the
same facade.
Several old polymorphic tests as well as two of the sql test
are refactored to use the new features.
Change-Id: I6ef8af1dafff92534313016944d447f9439856cf
References: #4896
(cherry picked from commit 696ef9421da763a0ffcfd98083c0af78db3ea756)
Fixed issue where double negation of a boolean column wouldn't reset
the "NOT" operator.
Fixes: #4618
Change-Id: Ica280a0d6b5b0870aa2d05c4d059a1e559e6b12a
(cherry picked from commit 18f25f50353d9736e6638266585b2cb3ef7b0ea4)
This affects mostly docstrings, except in orm/events.py::dispose_collection()
where one parameter gets renamed: given that the method is
empty, it seemed reasonable to me to fix that too.
Closes: #4440
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4440
Pull-request-sha: 779ed75acb
Change-Id: Ic0553fe97853054b09c2453af76d96363de6eb0e
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
Fixed issue that is closely related to 🎫`3639` where an expression
rendered in a boolean context on a non-native boolean backend would
be compared to 1/0 even though it is already an implcitly boolean
expression, when :meth:`.ColumnElement.self_group` were used. While this
does not affect the user-friendly backends (MySQL, SQLite) it was not
handled by Oracle (and possibly SQL Server). Whether or not the
expression is implicitly boolean on any database is now determined
up front as an additional check to not generate the integer comparison
within the compliation of the statement.
Fixes: #4320
Change-Id: Iae0a65e5c01bd576e64733c3651e1e1a1a1b240c
Started importing "collections" from "collections.abc" under Python 3.3 and
greater for Python 3.8 compatibility. Pull request courtesy Nathaniel
Knight.
In Python 3.3, the abstract base classes (Iterable, Mapping, etc.)
were moved from the `collections` module and put in the
`collections.abc` module. They remain in the `collections` module for
backwards compatibility, and will until Python 3.8.
This commit adds a variable (`collections_abc`) to the `util/compat`
module, which will be the `collections` module for Python < 3.3 and
before, or the `collections.abc` module for Python >= 3.3. It also
uses the new variable, getting rid of some deprecation warnings that
were seen when running under Python 3.7.
Change-Id: I2d1c0ef97c8ecac7af152cc56263422a40faa6bb
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/464
Reworked the new "autoescape" feature introduced in
:ref:`change_2694` in 1.2.0b2 to be fully automatic; the escape
character now defaults to a forwards slash ``"/"`` and
is applied to percent, underscore, as well as the escape
character itself, for fully automatic escaping. The
character can also be changed using the "escape" parameter.
Change-Id: I74894a2576983c0f6eb89480c9e5727f49fa9c25
Fixes: #2694
Fixed bug where the recently added :meth:`.ColumnOperators.any_`
and :meth:`.ColumnOperators.all_` methods didn't work when called
as methods, as opposed to using the standalone functions
:func:`~.expression.any_` and :func:`~.expression.all_`. Also
added documentation examples for these relatively unintuitive
SQL operators.
Change-Id: I3e56b463e9fd146a077b9970624f50cba27f9811
Fixes: #4093
Refined the behavior of :meth:`.Operators.op` such that in all cases,
if the :paramref:`.Operators.op.is_comparison` flag is set to True,
the return type of the resulting expression will be
:class:`.Boolean`, and if the flag is False, the return type of the
resulting expression will be the same type as that of the left-hand
expression, which is the typical default behavior of other operators.
Also added a new parameter :paramref:`.Operators.op.return_type` as well
as a helper method :meth:`.Operators.bool_op`.
Change-Id: Ifc8553cd4037d741b84b70a9702cbd530f1a9de0
Fixes: #4063
The operator precedence for all comparison operators such as LIKE, IS,
IN, MATCH, equals, greater than, less than, etc. has all been merged
into one level, so that expressions which make use of these against
each other will produce parentheses between them. This suits the
stated operator precedence of databases like Oracle, MySQL and others
which place all of these operators as equal precedence, as well as
Postgresql as of 9.5 which has also flattened its operator precendence.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Fixes: #3999
Change-Id: I3f3d5124a64af0d376361cdf15a97e2e703be56f
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/367
Repaired issue where the type of an expression that used
:meth:`.ColumnOperators.is_` or similar would not be a "boolean" type,
instead the type would be "nulltype", as well as when using custom
comparison operators against an untyped expression. This typing can
impact how the expression behaves in larger contexts as well as
in result-row-handling.
Change-Id: Ib810ff686de500d8db26ae35a51005fab29603b6
Fixes: #3873
Fixed the negation of a :class:`.Label` construct so that the
inner element is negated correctly, when the :func:`.not_` modifier
is applied to the labeled expression.
Change-Id: Ia99917b2959bdfbff28689c962b4203911c57b85
Fixes: #3969
Added a new kind of :func:`.bindparam` called "expanding". This is
for use in ``IN`` expressions where the list of elements is rendered
into individual bound parameters at statement execution time, rather
than at statement compilation time. This allows both a single bound
parameter name to be linked to an IN expression of multiple elements,
as well as allows query caching to be used with IN expressions. The
new feature allows the related features of "select in" loading and
"polymorphic in" loading to make use of the baked query extension
to reduce call overhead. This feature should be considered to be
**experimental** for 1.2.
Fixes: #3953
Change-Id: Ie708414a3ab9c0af29998a2c7f239ff7633b1f6e
This patch moves the "doubling" of percent signs into
the base compiler and makes it completely a product
of whether or not the paramstyle is format/pyformat or
not. Without this paramstyle, percent signs
are not doubled across text(), literal_column(), and
column().
Change-Id: Ie2f278ab1dbb94b5078f85c0096d74dbfa049197
Fixes: #3740
The expression used for COLLATE as rendered by the column-level
:func:`.expression.collate` and :meth:`.ColumnOperators.collate` is now
quoted as an identifier when the name is case sensitive, e.g. has
uppercase characters. Note that this does not impact type-level
collation, which is already quoted.
Change-Id: I83d5d9cd1e66a4f20b96303bb84c5f360d5d6a1a
Fixes: #3785
The longstanding behavior of the :meth:`.Operators.in_` and
:meth:`.Operators.not_in_` operators emitting a warning when
the right-hand condition is an empty sequence has been revised;
a new flag :paramref:`.create_engine.empty_in_strategy` allows an
empty "IN" expression to generate a simple boolean expression, or
to invoke the previous behavior of dis-equating the expression to
itself, with or without a warning. The default behavior is now
to emit the simple boolean expression, allowing an empty IN to
be evaulated without any performance penalty.
Change-Id: I65cc37f2d7cf65a59bf217136c42fee446929352
Fixes: #3907
Added compiler-level flags used by Postgresql to place additional
parenthesis than would normally be generated by precedence rules
around operations involving JSON, HSTORE indexing operators as well as
within their operands since it has been observed that Postgresql's
precedence rules for at least the HSTORE indexing operator is not
consistent between 9.4 and 9.5.
Fixes: #3806
Change-Id: I5899677b330595264543b055abd54f3c76bfabf2
The "eq" and "ne" operators are no longer part of the list of
"associative" operators, while they remain considered to be
"commutative". This allows an expression like ``(x == y) == z``
to be maintained at the SQL level with parenthesis. Pull request
courtesy John Passaro.
Fixes: #3799
Change-Id: I3759d8987b35649d7418b6524316c9e70c857e68
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/308
Fixed issue in SQL math negation operator where the type of the
expression would no longer be the numeric type of the original.
This would cause issues where the type determined result set
behaviors.
Change-Id: If0e339614a3686e251235fc94b6f59310c4630a5
Fixes: #3735
None / True / False render as literals.
For SQLite, "IS" is used as SQLite lacks
"IS DISTINCT FROM" but its "IS" operator acts
this way for NULL.
Doctext-author: Mike Bayer <mike_mp@zzzcomputing.com>
Change-Id: I9227b81f7207b42627a0349d14d40b46aa756cce
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/248
override with a column expression (e.g. by using ``'x' in col``)
would cause an endless loop in the case of an ARRAY type, as Python
defers this to ``__getitem__`` access which never raises for this
type. Overall, all use of ``__contains__`` now raises
NotImplementedError.
fixes#3642
persistence of JSON values in MySQL as well as basic operator support
of "getitem" and "getpath", making use of the ``JSON_EXTRACT``
function in order to refer to individual paths in a JSON structure.
fixes#3547
- Added a new type to core :class:`.types.JSON`. This is the
base of the PostgreSQL :class:`.postgresql.JSON` type as well as that
of the new :class:`.mysql.JSON` type, so that a PG/MySQL-agnostic
JSON column may be used. The type features basic index and path
searching support.
fixes#3619
- reorganization of migration docs etc. to try to refer both to
the fixes to JSON that helps Postgresql while at the same time
indicating these are new features of the new base JSON type.
- a rework of the Array/Indexable system some more, moving things
that are specific to Array out of Indexable.
- new operators for JSON indexing added to core so that these can
be compiled by the PG and MySQL dialects individually
- rename sqltypes.Array to sqltypes.ARRAY - as there is no generic
Array implementation, this is an uppercase type for now, consistent
with the new sqltypes.JSON type that is also not a generic implementation.
There may need to be some convention change to handle the case of
datatypes that aren't generic, rely upon DB-native implementations,
but aren't necessarily all named the same thing.
"super" instead of hardcoding to "self.type" for the default return
value, the base Comparator was returning other_comparator.type. It's
not clear what the rationale for this was, though in theory the
base Comparator should possibly even throw an exception if the two
types aren't the same (or of the same affinity?) .
- mysql.SET was broken on this because the bitwise version adds "0"
to the value to force an integer within column_expression, we are doing type_coerces here
now in any case so that there is no type ambiguity for this
operation
- The "hashable" flag on special datatypes such as :class:`.postgresql.ARRAY`,
:class:`.postgresql.JSON` and :class:`.postgresql.HSTORE` is now
set to False, which allows these types to be fetchable in ORM
queries that include entities within the row. fixes#3499
- The Postgresql :class:`.postgresql.ARRAY` type now supports multidimensional
indexed access, e.g. expressions such as ``somecol[5][6]`` without
any need for explicit casts or type coercions, provided
that the :paramref:`.postgresql.ARRAY.dimensions` parameter is set to the
desired number of dimensions. fixes#3487
- The return type for the :class:`.postgresql.JSON` and :class:`.postgresql.JSONB`
when using indexed access has been fixed to work like Postgresql itself,
and returns an expression that itself is of type :class:`.postgresql.JSON`
or :class:`.postgresql.JSONB`. Previously, the accessor would return
:class:`.NullType` which disallowed subsequent JSON-like operators to be
used. part of fixes#3503
- The :class:`.postgresql.JSON`, :class:`.postgresql.JSONB` and
:class:`.postgresql.HSTORE` datatypes now allow full control over the
return type from an indexed textual access operation, either ``column[someindex].astext``
for a JSON type or ``column[someindex]`` for an HSTORE type,
via the :paramref:`.postgresql.JSON.astext_type` and
:paramref:`.postgresql.HSTORE.text_type` parameters. also part of fixes#3503
- The :attr:`.postgresql.JSON.Comparator.astext` modifier no longer
calls upon :meth:`.ColumnElement.cast` implicitly, as PG's JSON/JSONB
types allow cross-casting between each other as well. Code that
makes use of :meth:`.ColumnElement.cast` on JSON indexed access,
e.g. ``col[someindex].cast(Integer)``, will need to be changed
to call :attr:`.postgresql.JSON.Comparator.astext` explicitly. This is
part of the refactor in references #3503 for consistency in operator
use.
a type that was also a :class:`.TypeDecorator` would fail with
Python's "Cannot create a consistent method resolution order (MRO)"
error, when any kind of SQL comparison expression were used against
an object using this type.
return type is not strictly assumed to be boolean; it now
returns a :class:`.Boolean` subclass called :class:`.MatchType`.
The type will still produce boolean behavior when used in Python
expressions, however the dialect can override its behavior at
result time. In the case of MySQL, while the MATCH operator
is typically used in a boolean context within an expression,
if one actually queries for the value of a match expression, a
floating point value is returned; this value is not compatible
with SQLAlchemy's C-based boolean processor, so MySQL's result-set
behavior now follows that of the :class:`.Float` type.
A new operator object ``notmatch_op`` is also added to better allow
dialects to define the negation of a match operation.
fixes#3263
of the "constants" :func:`.null`, :func:`.true`, and :func:`.false`
has been reverted. These functions returning a "singleton" object
had the effect that different instances would be treated as the
same regardless of lexical use, which in particular would impact
the rendering of the columns clause of a SELECT statement.
fixes#3170
system, whereby using a :class:`.TypeDecorator` in conjunction with
variant would fail with an MRO error when a comparison operator was used.
fixes#3102
:meth:`.Operators.__or__` and :meth:`.Operators.__invert__`
operator overload methods could not be overridden within a custom
:class:`.TypeEngine.Comparator` implementation.
fixes#3012
the new rules for "where" and "having" woudn't take effect for the
"whereclause" and "having" kw arguments of the :func:`.select` construct,
which is also what :class:`.Query` uses so wasn't working in the
ORM either. fixes#3013 re: #2804
renders "BETWEEN SYMMETRIC". Also added a new negation operator
"notbetween_op", which now allows an expression like ``~col.between(x, y)``
to render as "col NOT BETWEEN x AND y", rather than a parentheiszed NOT
string. fixes#2990
the first SQL expression would be applied as the "comparison type"
to a compared tuple value; this has the effect in some cases of an
inappropriate "type coersion" occurring, such as when a tuple that
has a mix of String and Binary values improperly coerces target
values to Binary even though that's not what they are on the left
side. :func:`.tuple_` now expects heterogeneous types within its
list of values.
fixes#2977
erroneously passed a column expression whose comparator included
the ``__getitem__()`` method, such as a column that uses the
:class:`.postgresql.ARRAY` type. [ticket:2957]
flag allows a custom op from :meth:`.Operators.op` to be considered
as a "comparison" operator, thus usable for custom
:paramref:`.relationship.primaryjoin` conditions.
have been modified, such that the COLLATE operator is now of lower
precedence than the comparison operators. This has the effect that
a COLLATE applied to a comparison will not render parenthesis
around the comparison, which is not parsed by backends such as
MSSQL. The change is backwards incompatible for those setups that
were working around the issue by applying :meth:`.Operators.collate`
to an individual element of the comparison expression,
rather than the comparison expression as a whole. [ticket:2879]
with conjunctions, e.g.
``None`` :func:`.expression.null` :func:`.expression.true`
:func:`.expression.false`, including consistency in rendering NULL
in conjunctions, "short-circuiting" of :func:`.and_` and :func:`.or_`
expressions which contain boolean constants, and rendering of
boolean constants and expressions as compared to "1" or "0" for backends
that don't feature ``true``/``false`` constants. [ticket:2804]
regards to the True/False constants. An expression like
``col.is_(True)`` will now render ``col IS true``
on the target platform, rather than converting the True/
False constant to an integer bound parameter.
This allows the ``is_()`` operator to work on MySQL when
given True/False constants.
[ticket:2682]
Mike Bayer