Add SelectBase.exists() method as it seems strange this is
not available already. The Exists construct itself does
not provide full SELECT-building capabilities so it makes
sense this should be used more like a scalar_subquery.
Make sure stream_results is getting set up when yield_per
is used, for 2.0 style statements as well. this was
hardcoded inside of Query.yield_per() and is now moved
to take place within QueryContext.
Change-Id: Icafcd4fd9b708772343d56edf40995c9e8f835d6
The operator changes are:
* `isfalse` is now `is_false`
* `isnot_distinct_from` is now `is_not_distinct_from`
* `istrue` is now `is_true`
* `notbetween` is now `not_between`
* `notcontains` is now `not_contains`
* `notendswith` is now `not_endswith`
* `notilike` is now `not_ilike`
* `notlike` is now `not_like`
* `notmatch` is now `not_match`
* `notstartswith` is now `not_startswith`
* `nullsfirst` is now `nulls_first`
* `nullslast` is now `nulls_last`
Because these are core operators, the internal migration strategy for this
change is to support legacy terms for an extended period of time -- if not
indefinitely -- but update all documentation, tutorials, and internal usage
to the new terms. The new terms are used to define the functions, and
the legacy terms have been deprecated into aliases of the new terms.
Fixes: #5435
Change-Id: Ifbd7cb1cdda5981990243c4fc4b4ff467dc132ac
Fixed structural compiler issue where some constructs such as MySQL /
PostgreSQL "on conflict / on duplicate key" would rely upon the state of
the :class:`_sql.Compiler` object being fixed against their statement as
the top level statement, which would fail in cases where those statements
are branched from a different context, such as a DDL construct linked to a
SQL statement.
Fixes: #5656
Change-Id: I568bf40adc7edcf72ea6c7fd6eb9d07790de189e
Improved support for column names that contain percent signs in the string,
including repaired issues involving anoymous labels that also embedded a
column name with a percent sign in it, as well as re-established support
for bound parameter names with percent signs embedded on the psycopg2
dialect, using a late-escaping process similar to that used by the
cx_Oracle dialect.
* Added new constructor for _anonymous_label() that ensures incoming
string tokens based on column or table names will have percent
signs escaped; abstracts away the format of the label.
* generalized cx_Oracle's quoted_bind_names facility into the compiler
itself, and leveraged this for the psycopg2 dialect's issue with
percent signs in names as well. the parameter substitution is now
integrated with compiler.construct_parameters() as well as the
recently reworked set_input_sizes(), reducing verbosity in the
cx_Oracle dialect.
Fixes: #5653
Change-Id: Ia2ad13ea68b4b0558d410026e5a33f5cb3fbab2c
These attributes will be removed in SQLAlchemy 2.0.
Also alters the deprecation message to qualify the
type of object correctly. this in turn requires changes
in the warnings filter and deprecation tests.
Change-Id: I5779d9813e88f42e5db0c7b5e3ffff1d1535c203
The :paramref:`_schema.MetaData.bind` argument as well as the overall
concept of "bound metadata" is deprecated in SQLAlchemy 1.4 and will be
removed in SQLAlchemy 2.0. The parameter as well as related functions now
emit a :class:`_exc.RemovedIn20Warning` when :ref:`deprecation_20_mode` is
in use.
Added new parameter :paramref:`_automap.AutomapBase.prepare.autoload_with`
which supersedes :paramref:`_automap.AutomapBase.prepare.reflect`
and :paramref:`_automap.AutomapBase.prepare.engine`.
Fixes: #4634Fixes: #5142
Change-Id: Iaabf9b481931e2fb68b97b5954c32e65772a298e
The :class:`_schema.Table` class now raises a deprecation warning
when columns with the same name are defined. To replace a column a new
parameter :paramref:`_schema.Table.append_column.replace_existing` was
added to the :meth:`_schema.Table.append_column` method.
The :meth:`_expression.ColumnCollection.contains_column` will now
raises an error when called with a string, suggesting the caller
to use ``in`` instead.
Co-authored-by: Federico Caselli <cfederico87@gmail.com>
Change-Id: I1d58c8ebe081079cb669e7ead60886ffc1b1a7f5
Fixed issue where a plain pickle dumps call of the :class:`_sql.Over`
construct didn't work.
Fixes: #5644
Change-Id: I4b07f74ecd5d52f0794128585367012200a38a36
this test can't require "skip identity_columns" because
older Postgresql and Oracle report false for "identity_columns",
but their dialects won't skip actually rendering.
for now the only option is to hardcode to a non-identity
dialect.
Change-Id: Ia4f39f393b4ba10b3e82601a22ab75200cd52909
this test was not actually working correctly against
mariadb and was "failing" (i.e. "passing") inappropriately
for pymysql, mysqlclient. it also started "passing" (i.e. "failing")
for mariadb as of 1.0.3.
modernize reqs here including for mssql bit behavior.
Change-Id: Ia88ee607ec8d59f6140a7e4a1174910382d53674
Add support to ``FETCH {FIRST | NEXT} [ count ] {ROW | ROWS}
{ONLY | WITH TIES}`` in the select for the supported backends,
currently PostgreSQL, Oracle and MSSQL.
Fixes: #5576
Change-Id: Ibb5871a457c0555f82b37e354e7787d15575f1f7
Fixed bug where an error was not raised for lower-case
:func:`_column` added to lower-case :func:`_table` object. This now raises
:class:`_exc.ArgumentError` which has always been the case for
upper-case :class:`_schema.Column` and :class:`_schema.Table`.
Fixes: #5618
Change-Id: Ifcbdf27c022fd2996a5b99559df71fc1c1e0f19c
Fixed bug where an UPDATE statement against a JOIN using MySQL multi-table
format would fail to include the table prefix for the target table if the
statement had no WHERE clause, as only the WHERE clause were scanned to
detect a "multi table update" at that particular point. The target
is now also scanned if it's a JOIN to get the leftmost table as the
primary table and the additional entries as additional FROM entries.
Fixes: #5617
Change-Id: I26d74afebe06e28af28acf960258f170a1627823
Added support for reflecting "identity" columns, which are now returned
as part of the structure returned by :meth:`_reflection.Inspector.get_columns`.
When reflecting full :class:`_schema.Table` objects, identity columns will
be represented using the :class:`_schema.Identity` construct.
Fixed compilation error on oracle for sequence and identity column
``nominvalue`` and ``nomaxvalue`` options that require no space in them.
Improved test compatibility with oracle 18.
As part of the support for reflecting :class:`_schema.Identity` objects,
the method :meth:`_reflection.Inspector.get_columns` no longer returns
``mssql_identity_start`` and ``mssql_identity_increment`` as part of the
``dialect_options``. Use the information in the ``identity`` key instead.
The mssql dialect will assume that at least MSSQL 2005 is used.
There is no hard exception raised if a previous version is detected,
but operations may fail for older versions.
Fixes: #5527Fixes: #5324
Change-Id: If039fe637c46b424499e6bac54a2cbc0dc54cb57
It's better, the majority of these changes look more readable to me.
also found some docstrings that had formatting / quoting issues.
Change-Id: I582a45fde3a5648b2f36bab96bad56881321899b
The :meth:`_sql.Join.alias` method is deprecated and will be removed in
SQLAlchemy 2.0. An explicit select + subquery, or aliasing of the inner
tables, should be used instead.
Fixes: #5010
Change-Id: Ic913afc31f0d70b0605f9a7af2742a0de1f9ad19
As the test suite has widespread use of many patterns
that are deprecated, enable SQLALCHEMY_WARN_20 globally
for the test suite but then break the warnings filter
out into a whole list of all the individual warnings
we are looking for. this way individual changesets
can target a specific class of warning, as many of these
warnings will indivdidually affect dozens of files
and potentially hundreds of lines of code.
Many warnings are also resolved here as this
patch started out that way. From this point
forward there should be changesets that target a
subset of the warnings at a time.
For expediency, updates some migration 2.0 docs
for ORM as well.
Change-Id: I98b8defdf7c37b818b3824d02f7668e3f5f31c94
Several operators are renamed to achieve more consistent naming across
SQLAlchemy.
The operator changes are:
* `isnot` is now `is_not`
* `not_in_` is now `not_in`
Because these are core operators, the internal migration strategy for this
change is to support legacy terms for an extended period of time -- if not
indefinitely -- but update all documentation, tutorials, and internal usage
to the new terms. The new terms are used to define the functions, and
the legacy terms have been deprecated into aliases of the new terms.
Fixes: #5429
Change-Id: Ia1e66e7a50ac35d3f6260d8bf6ba3ce8087cbad2
Make optional sequences render as identity in mssql
Remove unused dialect option sequence_default_column_type
Change-Id: I821eeffcb442f8d1b69186a9b798b15c3d8d6ff3
This change includes mainly that the bracketed use within
select() is moved to positional, and keyword arguments are
removed from calls to the select() function. it does not
yet fully address other issues such as keyword arguments passed
to the table.select().
Additionally, allows False / None to both be considered
as "disable" for all of select.correlate(), select.correlate_except(),
query.correlate(), which establishes consistency with
passing of ``False`` for the legact select(correlate=False)
argument.
Change-Id: Ie6c6e6abfbd3d75d4c8de504c0cf0159e6999108
fixed an issue where even though the method claims to be
matching up columns positionally, it was failing on that by
looking in "keymap" based on string name.
Adds a new member to the _keymap recs MD_RESULT_MAP_INDEX
so that we can efficiently link from the generated keymap
back to the compiled._result_columns structure without
any ambiguity.
Fixes: #5559
Change-Id: Ie2fa9165c16625ef860ffac1190e00575e96761f
:class:`_schema.Table` parameter ``mustexist`` has been renamed
to :paramref:`_schema.Table.must_exist` and will now warn when used.
Change-Id: I0b0ca6021f9f7cfbe2040bbc1125a2236ac79f53
Two operations have been defined:
* :meth:`~.ColumnOperators.regexp_match` implementing a regular
expression match like function.
* :meth:`~.ColumnOperators.regexp_replace` implementing a regular
expression string replace function.
Fixes: #1390
Change-Id: I44556846e4668ccf329023613bd26861d5c674e6
MariaDB should not run a Sequence if it has optional=True.
Additionally, rework the rules in crud.py to accommodate the
new combination MariaDB brings us, which is a dialect
that supports both cursor.lastrowid, explicit sequences,
*and* no support for returning.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Fixes: #5528
Change-Id: I9a8ea69a34983affa95dfd22186e2908fdf0d58c
Added the :class:`_schema.Identity` construct that can be used to
configure identity columns rendered with GENERATED { ALWAYS |
BY DEFAULT } AS IDENTITY. Currently the supported backends are
PostgreSQL >= 10, Oracle >= 12 and MSSQL (with different syntax
and a subset of functionalities).
Fixes: #5362Fixes: #5324Fixes: #5360
Change-Id: Iecea6f3ceb36821e8b96f0b61049b580507a1875
Adjusted the :meth:`_types.ARRAY.Comparator.any` and
:meth:`_types.ARRAY.Comparator.all` methods to implement a straight "NOT"
operation for negation, rather than negating the comparison operator.
Fixes: #5518
Change-Id: I87ee9278c321aafe51a679fcfcbb5fbb11307fda
Improved the :func:`_sql.tuple_` construct such that it behaves predictably
when used in a columns-clause context. The SQL tuple is not supported as a
"SELECT" columns clause element on most backends; on those that do
(PostgreSQL, not surprisingly), the Python DBAPI does not have a "nested
type" concept so there are still challenges in fetching rows for such an
object. Use of :func:`_sql.tuple_` in a :func:`_sql.select` or
:class:`_orm.Query` will now raise a :class:`_exc.CompileError` at the
point at which the :func:`_sql.tuple_` object is seen as presenting itself
for fetching rows (i.e., if the tuple is in the columns clause of a
subquery, no error is raised). For ORM use,the :class:`_orm.Bundle` object
is an explicit directive that a series of columns should be returned as a
sub-tuple per row and is suggested by the error message. Additionally ,the
tuple will now render with parenthesis in all contexts. Previously, the
parenthesization would not render in a columns context leading to
non-defined behavior.
As part of this change, Tuple receives a dedicated datatype
which appears to allow us the very desirable change of removing
the bindparam._expanding_in_types attribute as well as
ClauseList._tuple_values (which might already have not been
needed due to #4645).
Fixes: #5127
Change-Id: Iecafa0e0aac2f1f37ec8d0e1631d562611c90200
Thr proposed change will provide the user with the target Enum Class name as well as up to four possible enum values when a LookupError is raised in the Enum Class.
A user requested that the enum name and possible values are included to the LookupError message to make debugging easier. The criteria included using ellipses for Enums containing more than four values and using ellipses for enum values that were greater than a certain number of characters (for this resolution the limit is 11 characters).
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Fixes: #4733Closes: #5490
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5490
Pull-request-sha: 55e76f2ae7
Change-Id: I4541f9efed1c05401587a413e9e748d46938bcd1
MySQL dialect's server_version_info tuple is now all numeric. String
tokens like "MariaDB" are no longer present so that numeric comparison
works in all cases. The .is_mariadb flag on the dialect should be
consulted for whether or not mariadb was detected. Additionally removed
structures meant to support extremely old MySQL versions 3.x and 4.x;
the minimum MySQL version supported is now version 5.0.2.
In addition, as the "MariaDB" name goes away from server version,
expand upon the change in I330815ebe572b6a9818377da56621397335fa702
to support the name "mariadb" throughout the dialect and test suite
when mariadb-only mode is used. This changes the "name" field
on the MariaDB dialect to "mariadb", which then implies a change
throughout the testing requirements system as well as all the
dialect-specific DDL argument names such as "mysql_engine" is
now specified as "mariadb_engine", etc. Make use of the
recent additions to test suite URL provisioning so that we can
force MariaDB databases to have a "mariadb-only" dialect which
allows us to test this name change fully.
Update documentation to refer to MySQL / MariaDB explicitly
as well as indicating the "mariadb_" prefix used for options.
It seems likely that MySQL and MariaDB version numbers are going to
start colliding at some point so having the "mariadb" name
be available as a totally separate dialect name should give us
some options in this regard.
Currently also includes a date related fix to a test for
the postgresql dialect that was implicitly assuming a
non-UTC timezone
Fixes: #4189
Change-Id: I00e76d00f62971e1f067bd61915fa6cc1cf64e5e
Using the approach introduced at
https://gist.github.com/zzzeek/6287e28054d3baddc07fa21a7227904e
We can now create asyncio endpoints that are then handled
in "implicit IO" form within the majority of the Core internals.
Then coroutines are re-exposed at the point at which we call
into asyncpg methods.
Patch includes:
* asyncpg dialect
* asyncio package
* engine, result, ORM session classes
* new test fixtures, tests
* some work with pep-484 and a short plugin for the
pyannotate package, which seems to have so-so results
Change-Id: Idbcc0eff72c4cad572914acdd6f40ddb1aef1a7d
Fixes: #3414
The fix in 180ae7c1a5
didn't do much as the code was not preserving parameter
order at all, in fact. Reworked stmt_parameters to be
delivered in the correct order up front and preserve
throughout crud.py which was not being done at all
before.
Fixes: #5510
Change-Id: I0795c71df73005a25d1bbf216732d41b41e11a5f
The issue of Result.fetchXXX() methods returning Row
objects unless filtering is applied will not provide a
clear enough API story when type annotations are applied,
so break out scalars/mappings into separate wrapper objects.
this makes some things more intuitive and other things a little
more bumpy. however the return type story is now clearer.
Fixes: #5503
Change-Id: I629a061823179680dc0723559183859a67ea4db1
The test for the parameter fix in c0685e5f41 was not working
deterministically on Python 2, so use ordered_values(), however
a second issue in ordered_values() was assuming each element was a
column, so also test for array-assignment expressions with
ordered_values.
Change-Id: I944c72a52700ffb4ab5ae1a83ae21f1efc84b505
Fixes related to rendering of complex UPDATE DML
which was not correctly preserving positional parameter
order in conjunction with DML features that are only known
to work on the PostgreSQL database. Both pg8000
and asyncpg use positional parameters which is why these
issues are suddenly apparent.
crud.py now takes on the task of rendering the column
expressions for SET or VALUES so that for the very unusual
case that the column expression is a compound expression
that includes a bound parameter (namely an array index),
the bound parameter order is preserved.
Additionally, crud.py passes through the positional_names
keyword argument into bindparam_string() which is necessary
when CTEs are being rendered, as PG supports complex
CTE / INSERT / UPDATE scenarios.
Change-Id: I7f03920500e19b721636b84594de78a5bfdcbc82
Fixed issue where the
:paramref:`_engine.Connection.execution_options.schema_translate_map`
feature would not take effect when the :meth:`_schema.Sequence.next_value`
function function for a :class:`_schema.Sequence` were used in the
:paramref:`_schema.Column.server_default` parameter and the create table
DDL were emitted.
Fixes: #5500
Change-Id: I74a9fa13d22749d06c8202669f9ea220d9d984d9
Added the ability to add arbitrary criteria to the ON clause generated
by a relationship attribute in a query, which applies to methods such
as :meth:`_query.Query.join` as well as loader options like
:func:`_orm.joinedload`. Additionally, a "global" version of the option
allows limiting criteria to be applied to particular entities in
a query globally.
Documentation is minimal at this point, new examples will
be coming in a subsequent commit.
Some adjustments to execution options in how they are represented
in the ORMExecuteState as well as well as a few ORM tests that
forgot to get merged in a preceding commit.
Fixes: #4472
Change-Id: I2b8fc57092dedf35ebd16f6343ad0f0d7d332beb
in order to accommodate relationship loaders
with lambda caching, a lot more is needed. This is
a full refactor of the lambda system such that it
now has two levels of caching; the first level caches what
can be known from the __code__ element, then the next level
of caching is against the lambda itself and the contents
of __closure__. This allows for the elements inside
the lambdas, like columns and entities, to change and
then be part of the cache key. Lazy/selectinloads' use of
baked queries had to add distinct cache key elements,
which was attempted here but overall things needed to be
more robust than that.
This commit is broken out from the very long and sprawling
commit at Id6b5c03b1ce9ddb7b280f66792212a0ef0a1c541 .
Change-Id: I29a513c98917b1d503abfdd61e6b6e8800851aa8
Remove lookup logic that attempts to locate a dialect for a type,
just use StrSQLTypeCompiler.
Cleaned up the internal ``str()`` for datatypes so that all types produce a
string representation without any dialect present, including that it works
for third-party dialect types without that dialect being present. The
string representation defaults to being the UPPERCASE name of that type
with nothing else.
Fixes: #4262
Change-Id: I02149e8a1ba1e7336149e962939b07ae0df83c6b
mike bayer