In order for text(), custom compiled objects, etc. to be usable
by Query(), they are all targeted by object key in the result map.
As we no longer want Query to implicitly label these, as well as that
text() has no label feature, support adding entries to the result
map that have no name, key, or type, only the object itself, and
then ensure that the compiler sets up for positional targeting
when this condition is detected.
Allows for more flexible ORM query usage with custom expressions
and text() while having less special logic in query itself.
Fixes: #4887
Change-Id: Ie073da127d292d43cb132a2b31bc90af88bfe2fd
Fixed issue where the :class:`.array_agg` construct in combination with
:meth:`.FunctionElement.filter` would not produce the correct operator
precedence between the FILTER keyword and the array index operator.
Fixes: #4760
Change-Id: Ic662cd3da3330554ec673bafd80495b3f1506098
As part of the SQLAlchemy 2.0 migration project, a conceptual change has
been made to the role of the :class:`.SelectBase` class hierarchy,
which is the root of all "SELECT" statement constructs, in that they no
longer serve directly as FROM clauses, that is, they no longer subclass
:class:`.FromClause`. For end users, the change mostly means that any
placement of a :func:`.select` construct in the FROM clause of another
:func:`.select` requires first that it be wrapped in a subquery first,
which historically is through the use of the :meth:`.SelectBase.alias`
method, and is now also available through the use of
:meth:`.SelectBase.subquery`. This was usually a requirement in any
case since several databases don't accept unnamed SELECT subqueries
in their FROM clause in any case.
See the documentation in this change for lots more detail.
Fixes: #4617
Change-Id: I0f6174ee24b9a1a4529168e52e855e12abd60667
Fixed that the :class:`.GenericFunction` class was inadvertently
registering itself as one of the named functions. Pull request courtesy
Adrien Berchet.
Fixes: #4653Closes: #4654
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4654
Pull-request-sha: 1112b89f0d
Change-Id: Ia0d366d3bff44a763aa496287814278dff732a19
Registered function names based on :class:`.GenericFunction` are now
retrieved in a case-insensitive fashion in all cases, removing the
deprecation logic from 1.3 which temporarily allowed multiple
:class:`.GenericFunction` objects to exist with differing cases. A
:class:`.GenericFunction` that replaces another on the same name whether or
not it's case sensitive emits a warning before replacing the object.
Fixes: #4649
Change-Id: I265ae19833132db07ed5b5ae40c4d24f659b1ab3
The :class:`.GenericFunction` namespace is being migrated so that function
names are looked up in a case-insensitive manner, as SQL functions do not
collide on case sensitive differences nor is this something which would
occur with user-defined functions or stored procedures. Lookups for
functions declared with :class:`.GenericFunction` now use a case
insensitive scheme, however a deprecation case is supported which allows
two or more :class:`.GenericFunction` objects with the same name of
different cases to exist, which will cause case sensitive lookups to occur
for that particular name, while emitting a warning at function registration
time. Thanks to Adrien Berchet for a lot of work on this complicated
feature.
Fixes: #4569Closes: #4570
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4570
Pull-request-sha: 37d4f3322b
Change-Id: Ief07c6eb55bf398f6aad85b60ef13ee6d1173109
Fully removed the behavior of strings passed directly as components of a
:func:`.select` or :class:`.Query` object being coerced to :func:`.text`
constructs automatically; the warning that has been emitted is now an
ArgumentError or in the case of order_by() / group_by() a CompileError.
This has emitted a warning since version 1.0 however its presence continues
to create concerns for the potential of mis-use of this behavior.
Note that public CVEs have been posted for order_by() / group_by() which
are resolved by this commit: CVE-2019-7164 CVE-2019-7548
Added "SQL phrase validation" to key DDL phrases that are accepted as plain
strings, including :paramref:`.ForeignKeyConstraint.on_delete`,
:paramref:`.ForeignKeyConstraint.on_update`,
:paramref:`.ExcludeConstraint.using`,
:paramref:`.ForeignKeyConstraint.initially`, for areas where a series of SQL
keywords only are expected.Any non-space characters that suggest the phrase
would need to be quoted will raise a :class:`.CompileError`. This change
is related to the series of changes committed as part of 🎫`4481`.
Fixed issue where using an uppercase name for an index type (e.g. GIST,
BTREE, etc. ) or an EXCLUDE constraint would treat it as an identifier to
be quoted, rather than rendering it as is. The new behavior converts these
types to lowercase and ensures they contain only valid SQL characters.
Quoting is applied to :class:`.Function` names, those which are usually but
not necessarily generated from the :attr:`.sql.func` construct, at compile
time if they contain illegal characters, such as spaces or punctuation. The
names are as before treated as case insensitive however, meaning if the
names contain uppercase or mixed case characters, that alone does not
trigger quoting. The case insensitivity is currently maintained for
backwards compatibility.
Fixes: #4481Fixes: #4473Fixes: #4467
Change-Id: Ib22a27d62930e24702e2f0f7c74a0473385a08eb
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
Fixes to the test suite, a few errant imports, and setup.py:
- mysql and postgresql have unused 'json' imports; remove
- postgresql is exporting the 'json' symbol, remove
- make sure setup.py can find __version__ using " or '
- retry logic in provision create database for postgresql fixed
- refactor test_magazine to use cls.tables rather than globals
- remove unused class in test_scoping
- add a comment to test_deprecations that this test suite itself
is deprecated
- don't use mapper() and orm_mapper() in test_unitofwork, just
use mapper()
- remove dupe test_scalar_set_None test in test_attributes
- Python 2.7 and above includes unittest.SkipTest, remove pre-2.7
fallback
- use imported SkipTest in profiling
- declarative test_reflection tests with "reflectable_autoincrement"
already don't run on oracle or firebird; remove conditional logic
for these, which also removes an "id" symbol
- clean up test in test_functions, remove print statement
- remove dupe test_literal_processor_coercion_native_int_out_of_range
in test/sql/test_types.py
- fix psycopg2_hstore ref
Change-Id: I7b3444f8546aac82be81cd1e7b6d8b2ad6834fe6
Amended the :class:`.AnsiFunction` class, the base of common SQL
functions like ``CURRENT_TIMESTAMP``, to accept positional arguments
like a regular ad-hoc function. This to suit the case that many of
these functions on specific backends accept arguments such as
"fractional seconds" precision and such. If the function is created
with arguments, it renders the the parenthesis and the arguments. If
no arguents are present, the compiler generates the non-parenthesized form.
Fixes: #4386
Change-Id: Ic492ef177e4987cec99ec4d95f55292be8daa087
Added missing window function parameters
:paramref:`.WithinGroup.over.range_` and :paramref:`.WithinGroup.over.rows`
parameters to the :meth:`.WithinGroup.over` and
:meth:`.FunctionFilter.over` methods, to correspond to the range/rows
feature added to the "over" method of SQL functions as part of
🎫`3049` in version 1.1.
Fixes: #4322
Change-Id: I77dcdac65c699a4b52a3fc3ee09a100ffb4fc20e
Added new feature :meth:`.FunctionElement.as_comparison` which allows a SQL
function to act as a binary comparison operation that can work within the
ORM.
Change-Id: I07018e2065d09775c0406cabdd35fc38cc0da699
Fixes: #3831
Fixed bug in :func:`.array_agg` function where passing an argument
that is already of type :class:`.ARRAY`, such as a Postgresql
:obj:`.postgresql.array` construct, would produce a ``ValueError``, due
to the function attempting to nest the arrays.
Change-Id: Ibe5f6275d90e4868e6ef8a733de05acd44c05d78
Fixes: #4107
One test appears to use some awkward calling style
with the current_date function that isn't working in pg10
anymore, this looks like an extremely
old test that can be removed
Change-Id: I5f8aee0f5ed423461be5a9060c812eb0acdc7df5
persistence of JSON values in MySQL as well as basic operator support
of "getitem" and "getpath", making use of the ``JSON_EXTRACT``
function in order to refer to individual paths in a JSON structure.
fixes#3547
- Added a new type to core :class:`.types.JSON`. This is the
base of the PostgreSQL :class:`.postgresql.JSON` type as well as that
of the new :class:`.mysql.JSON` type, so that a PG/MySQL-agnostic
JSON column may be used. The type features basic index and path
searching support.
fixes#3619
- reorganization of migration docs etc. to try to refer both to
the fixes to JSON that helps Postgresql while at the same time
indicating these are new features of the new base JSON type.
- a rework of the Array/Indexable system some more, moving things
that are specific to Array out of Indexable.
- new operators for JSON indexing added to core so that these can
be compiled by the PG and MySQL dialects individually
- rename sqltypes.Array to sqltypes.ARRAY - as there is no generic
Array implementation, this is an uppercase type for now, consistent
with the new sqltypes.JSON type that is also not a generic implementation.
There may need to be some convention change to handle the case of
datatypes that aren't generic, rely upon DB-native implementations,
but aren't necessarily all named the same thing.
``<function> WITHIN GROUP (ORDER BY <criteria>)``, using the
method :class:`.FunctionElement.within_group`. A series of common
set-aggregate functions with return types derived from the set have
been added. This includes functions like :class:`.percentile_cont`,
:class:`.dense_rank` and others.
fixes#1370
- make sure we use func.name for all _literal_as_binds in functions.py
so we get consistent naming behavior for parameters.
which automatically returns an :class:`.Array` of the correct type
and supports index / slice operations. As arrays are only
supported on Postgresql at the moment, only actually works on
Postgresql. fixes#3132
of :class:`.FunctionElement` or other column element that incorrectly
states 'None' or any other invalid object as the ``.type``
attribute will report this exception instead of recursion overflow.
fixes#3485
constructs are now importable from the "from sqlalchemy" namespace,
just like every other Core construct.
- The implicit conversion of strings to :func:`.text` constructs
when passed to most builder methods of :func:`.select` as
well as :class:`.Query` now emits a warning with just the
plain string sent. The textual conversion still proceeds normally,
however. The only method that accepts a string without a warning
are the "label reference" methods like order_by(), group_by();
these functions will now at compile time attempt to resolve a single
string argument to a column or label expression present in the
selectable; if none is located, the expression still renders, but
you get the warning again. The rationale here is that the implicit
conversion from string to text is more unexpected than not these days,
and it is better that the user send more direction to the Core / ORM
when passing a raw string as to what direction should be taken.
Core/ORM tutorials have been updated to go more in depth as to how text
is handled.
fixes#2992
e.g. the ``func`` construct. Previously, behavior for this method
was undefined. The current behavior mimics that of pre-0.9.4,
which is that the function is turned into a single-column FROM
clause with the given alias name, where the column itself is
anonymously named.
fixes#3137
is currently being supported in addition to nose, and will likely
be preferred to nose going forward. The nose plugin system used
by SQLAlchemy has been split out so that it works under pytest as
well. There are no plans to drop support for nose at the moment
and we hope that the test suite itself can continue to remain as
agnostic of testing platform as possible. See the file
README.unittests.rst for updated information on running tests
with pytest.
The test plugin system has also been enhanced to support running
tests against mutiple database URLs at once, by specifying the ``--db``
and/or ``--dburi`` flags multiple times. This does not run the entire test
suite for each database, but instead allows test cases that are specific
to certain backends make use of that backend as the test is run.
When using pytest as the test runner, the system will also run
specific test suites multiple times, once for each database, particularly
those tests within the "dialect suite". The plan is that the enhanced
system will also be used by Alembic, and allow Alembic to run
migration operation tests against multiple backends in one run, including
third-party backends not included within Alembic itself.
Third party dialects and extensions are also encouraged to standardize
on SQLAlchemy's test suite as a basis; see the file README.dialects.rst
for background on building out from SQLAlchemy's test platform.
into the names used by standard functions in :mod:`sqlalchemy.sql.functions`,
such as ``func.coalesce()`` and ``func.max()``. Using these functions
in ORM attributes and thus producing annotated versions of them could
corrupt the actual function name rendered in the SQL. [ticket:2927]
become an externally usable package but still remains within the main sqlalchemy parent package.
in this system, we use kind of an ugly hack to get the noseplugin imported outside of the
"sqlalchemy" package, while still making it available within sqlalchemy for usage by
third party libraries.
will now be produced via the func.* accessor
as well, as users naturally try to access these
names from func.* they might as well do
what's expected, even though the returned
object is not a FunctionElement.
[ticket:2562]
True by default, if not passed explicitly,
on bindparam() if the "value" or "callable"
parameters are not passed.
This will cause statement execution to check
for the parameter being present in the final
collection of bound parameters, rather than
implicitly assigning None. [ticket:2556]
to allow for user-defined GenericFunction
subclasses to be available via the func.*
namespace automatically by classname,
optionally using a package name as well.
- add --with-cdecimal flag to tests, monkeypatches cdecimal in
- fix mssql/pyodbc.py to not use private '_int' accessor in decimal conversion
routines
- pyodbc version 2.1.8 is needed for cdecimal in any case as
previous versions also called '_int', 2.1.8 adds the same string
logic as our own dialect, so that logic is skipped for modern
pyodbc version
- make the imports for "Decimal" consistent across the whole lib. not sure
yet how we should be importing "Decimal" or what the best way forward
is that would allow a clean user-invoked swap of cdecimal; for now,
added docs suggesting a global monkeypatch - the two decimal libs
are not compatible with each other so any chance of mixing produces
serious issues. adding adapters to DBAPIs tedious and adds in-python
overhead. suggestions welcome on how we should be doing
Decimal/cdecimal.
API, used for any expression construct that can be sent to
execute(). FunctionElement now inherits Executable so that
it gains execution_options(), which are also propagated
to the select() that's generated within execute().
Executable in turn subclasses _Generative which marks
any ClauseElement that supports the @_generative
decorator - these may also become "public" for the benefit
of the compiler extension at some point.
Mike Bayer