Commit Graph

4243 Commits

Author SHA1 Message Date
Mike Bayer f3268b6f2f Use preloaded for sql.util import in exc
Repaired a function-level import that was not using SQLAlchemy's standard
late-import system within the sqlalchemy.exc module.

Moved preloaded to sqlalchemy.util.preloaded so that it
does not depend on langhelpers which depends on exc.

Fixes: #5632
Change-Id: I61b7ce9cd461071ce543714739f67aa5aeb47fd6
2020-10-06 22:25:23 -04:00
mike bayer 402cca8f2a Merge "Fetch first support" 2020-10-05 13:55:55 +00:00
mike bayer 7ccbdc31f8 Merge "Improve some documentations" 2020-10-05 13:35:47 +00:00
Federico Caselli 34e6b732a1 Fetch first support
Add support to ``FETCH {FIRST | NEXT} [ count ] {ROW | ROWS}
 {ONLY | WITH TIES}`` in the select for the supported backends,
currently PostgreSQL, Oracle and MSSQL.

Fixes: #5576
Change-Id: Ibb5871a457c0555f82b37e354e7787d15575f1f7
2020-10-02 21:34:24 +02:00
Mike Bayer 9586594754 raise on lower-case column shared to multiple tables
Fixed bug where an error was not raised for lower-case
:func:`_column` added to lower-case :func:`_table` object. This now raises
:class:`_exc.ArgumentError` which has always been the case for
upper-case :class:`_schema.Column` and :class:`_schema.Table`.

Fixes: #5618
Change-Id: Ifcbdf27c022fd2996a5b99559df71fc1c1e0f19c
2020-09-30 09:37:02 -04:00
mike bayer b21a03316f Merge "Support for multiple hosts in PostgreSQL connection string" 2020-09-30 12:07:25 +00:00
Mike Bayer f483573aa6 Scan for tables without relying upon whereclause
Fixed bug where an UPDATE statement against a JOIN using MySQL multi-table
format would fail to include the table prefix for the target table if the
statement had no WHERE clause, as only the WHERE clause were scanned to
detect a "multi table update" at that particular point.  The target
is now also scanned if it's a JOIN to get the leftmost table as the
primary table and the additional entries as additional FROM entries.

Fixes: #5617
Change-Id: I26d74afebe06e28af28acf960258f170a1627823
2020-09-29 16:46:56 -04:00
Federico Caselli 50fb226903 Improve some documentations
Change-Id: Ibcb0da3166b94aa58fa92d544c3e5cf75844546e
2020-09-29 19:27:29 +02:00
Federico Caselli 7362d454f4 Add reflection for Identity columns
Added support for reflecting "identity" columns, which are now returned
as part of the structure returned by :meth:`_reflection.Inspector.get_columns`.
When reflecting full :class:`_schema.Table` objects, identity columns will
be represented using the :class:`_schema.Identity` construct.

Fixed compilation error on oracle for sequence and identity column
``nominvalue`` and ``nomaxvalue`` options that require no space in them.

Improved test compatibility with oracle 18.

As part of the support for reflecting :class:`_schema.Identity` objects,
the method :meth:`_reflection.Inspector.get_columns` no longer returns
``mssql_identity_start`` and ``mssql_identity_increment`` as part of the
``dialect_options``. Use the information in the ``identity`` key instead.

The mssql dialect will assume that at least MSSQL 2005 is used.
There is no hard exception raised if a previous version is detected,
but operations may fail for older versions.

Fixes: #5527
Fixes: #5324
Change-Id: If039fe637c46b424499e6bac54a2cbc0dc54cb57
2020-09-28 18:11:12 -04:00
Gord Thompson 75ac0abc7d Add deprecation warning for .join().alias()
The :meth:`_sql.Join.alias` method is deprecated and will be removed in
SQLAlchemy 2.0.   An explicit select + subquery, or aliasing of the inner
tables, should be used instead.

Fixes: #5010
Change-Id: Ic913afc31f0d70b0605f9a7af2742a0de1f9ad19
2020-09-28 12:08:04 -04:00
Mike Bayer 17dd7fce61 Accommodate for same base class multiple times in inherits list
Improved declarative inheritance scanning to not get tripped up when the
same base class appears multiple times in the base inheritance list.

Fixes: #4699
Change-Id: I932e735cd2e2c1efa935936c84219924225d10f1
2020-09-27 11:45:49 -04:00
Mike Bayer 75a8fb5643 formatting fixes, some content addition
Change-Id: I807f9763252cdb18ba8702cf42fdb2a7d4c72b52
2020-09-25 00:10:04 -04:00
Mike Bayer 5e7acf14a4 new docs WIP
This WIP is part of the final push for 1.4's docs
to fully "2.0-ize" what we can, and have it all ready.

So far this includes a rewrite of the 2.0 migration,
set up for the 1.4 /2.0 docs style, and a total redesign
of the index page using a new flex layout in zzzeeksphinx.

It also reworks some of the API reference sections
to have more subheaders.   zzzeeksphinx is also enhanced
to provide automatic summaries for all api doc section.

Change-Id: I01d360cb9c8749520246b96ee6496143c6037918
2020-09-24 23:13:46 -04:00
RamonWill f9ef7fd30b Support for multiple hosts in PostgreSQL connection string
Provide support for multiple hosts in the PostgreSQL connection string.

A user requested for SQLAlchemy to support multiple hosts within a PostgreSQL URL string. The proposed fix allows this. In the event that the url contains multiple hosts the proposed code will convert the query["hosts"] tuple into a single string. This allows the hosts to then get converted into a valid dsn variable in the psycopg2 connect function.

This pull request is:

- [ ] A documentation / typographical error fix
	- Good to go, no issue or tests are needed
- [X ] A short code fix
	- please include the issue number, and create an issue if none exists, which
	  must include a complete example of the issue.  one line code fixes without an
	  issue and demonstration will not be accepted.
	- Please include: `Fixes: #<issue number>` in the commit message
	- please include tests.   one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
	- please include the issue number, and create an issue if none exists, which must
	  include a complete example of how the feature would look.
	- Please include: `Fixes: #<issue number>` in the commit message
	- please include tests.

**Have a nice day!**
Fixes: #4392

Closes: #5554
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5554
Pull-request-sha: 3f7a0ab8df

Change-Id: I3f3768d51b8331de786ffdc025b7ecfc662eafe5
2020-09-23 11:20:54 +01:00
Mike Bayer 96888aee76 Deprecate negative slice indexes
The "slice index" feature used by :class:`_orm.Query` as well as by the
dynamic relationship loader will no longer accept negative indexes in
SQLAlchemy 2.0.  These operations do not work efficiently and load the
entire collection in, which is both surprising and undesirable.   These
will warn in 1.4 unless the :paramref:`_orm.Session.future` flag is set in
which case they will raise IndexError.

Fixes: #5606
Change-Id: I5f5dcf984a8f41ab3d0e233ef7553e77fd99a771
2020-09-22 14:26:18 -04:00
Mike Bayer 0d1efeec47 Raise if unique() not applied to 2.0 joined eager load results
The automatic uniquing of rows on the client side is turned off for the new
:term:`2.0 style` of ORM querying.  This improves both clarity and
performance.  However, uniquing of rows on the client side is generally
necessary when using joined eager loading for collections, as there
will be duplicates of the primary entity for each element in the
collection because a join was used.  This uniquing must now be manually
enabled and can be achieved using the new
:meth:`_engine.Result.unique` modifier.   To avoid silent failure, the ORM
explicitly requires the method be called when the result of an ORM
query in 2.0 style makes use of joined load collections.    The newer
:func:`_orm.selectinload` strategy is likely preferable for eager loading
of collections in any case.

This changeset also fixes an issue where ORM-style "single entity"
results would not apply unique() correctly if results were returned
as tuples.

Fixes: #4395
Change-Id: Ie62e0cb68ef2a6d2120e968b79575a70d057212e
2020-09-21 19:55:59 -04:00
mike bayer 73ab000007 Merge "Stringify correctly for non-str exception argument" 2020-09-20 19:01:53 +00:00
Andrzej Bartosiński c4f28fb414 Stringify correctly for non-str exception argument
Fixed issue where a non-string object sent to
:class:`_exc.SQLAlchemyError` or a subclass, as occurs with some third
party dialects, would fail to stringify correctly. Pull request
courtesy Andrzej Bartosiński.

Fixes: #5599
Closes: #5600
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5600
Pull-request-sha: cdccccc42a

Change-Id: Icd710d9015abc80f61a84893d75fbb33ee0fe46e
2020-09-20 14:09:23 -04:00
Gord Thompson cec14b77e6 Fix has_table() for mssql temporary tables
Fixes: #5597

Fixes the issue where :meth:`_reflection.has_table` always returns
``False`` for temporary tables.

Change-Id: I03ab04c849a157ce8fd28c07ec3bf4407b0f2c94
2020-09-19 10:18:52 -06:00
Gord Thompson dcd2be16b9 Add support for Azure authentication options
Fixes: #5592
Change-Id: I0688e5ea0fc6b01a0b72f397daea8f57a2ec0766
2020-09-17 11:42:06 -06:00
Mike Bayer 7e864fc7b1 Create a framework to allow all SQLALCHEMY_WARN_20 to pass
As the test suite has widespread use of many patterns
that are deprecated, enable SQLALCHEMY_WARN_20 globally
for the test suite but then break the warnings filter
out into a whole list of all the individual warnings
we are looking for.  this way individual changesets
can target a specific class of warning, as many of these
warnings will indivdidually affect dozens of files
and potentially hundreds of lines of code.

Many warnings are also resolved here as this
patch started out that way.   From this point
forward there should be changesets that target a
subset of the warnings at a time.

For expediency, updates some migration 2.0 docs
for ORM as well.

Change-Id: I98b8defdf7c37b818b3824d02f7668e3f5f31c94
2020-09-16 12:31:05 -04:00
mike bayer 7074010449 Merge "Make :class:_orm.registry `bind` a private parameter." 2020-09-16 14:18:59 +00:00
Federico Caselli 43b486917b Make :class:_orm.registry `bind` a private parameter.
Explicitly deprecate ``bind`` from :func:`_orm.declarative_base`
and :func:`_orm.as_declarative`.

Some other documentation cleanup on declarative documentation.

Change-Id: I3f7918d23833b2778ab7009ac5018841deb19f75
2020-09-15 22:12:22 +02:00
Gord Thompson 31c3ed715a Remove silent ignore for skip_locked w/ unsupported backends
For SQLAlchemy 1.4:

The "skip_locked" keyword used with ``with_for_update()`` will render "SKIP
LOCKED" on all MySQL backends, meaning it will fail for MySQL less than
version 8 and on current MariaDB backends.  This is because those backends
do not support "SKIP LOCKED" or any equivalent, so this error should not be
silently ignored.   This is upgraded from a warning in the 1.3 series.

For SQLAlchemy 1.3:

The "skip_locked" keyword used with ``with_for_update()`` will emit a
warning when used on MariaDB backends, and will then be ignored.   This is
a deprecated behavior that will raise in SQLAlchemy 1.4, as an application
that requests "skip locked" is looking for a non-blocking operation which
is not available on those backends.

Fixes: #5578
Change-Id: I49ccb6c6ff46eafed12b77f51e1da8e0e397966c
2020-09-15 09:43:00 -04:00
mike bayer 8455a11bcc Merge "Rename Core expression isnot, not_in_" 2020-09-14 20:35:51 +00:00
Mike Bayer 3d803a8ed3 Pass all pool parameters in recreate()
The following pool parameters were not being propagated to the new pool
created when :meth:`_engine.Engine.dispose` were called: ``pre_ping``,
``use_lifo``.  Additionally the ``recycle`` and ``reset_on_return``
parameters were not propagated for the :class:`_engine.AssertionPool`
class.  These issues have been fixed.

Fixes: #5582
Change-Id: Ifdb703aa7e849652242b9ff8071c854cd1d77e71
2020-09-14 11:36:31 -04:00
jonathan vanasco fe413084c5 Rename Core expression isnot, not_in_
Several operators are renamed to achieve more consistent naming across
SQLAlchemy.

The operator changes are:

* `isnot` is now `is_not`
* `not_in_` is now `not_in`

Because these are core operators, the internal migration strategy for this
change is to support legacy terms for an extended period of time -- if not
indefinitely -- but update all documentation, tutorials, and internal usage
to the new terms.  The new terms are used to define the functions, and
the legacy terms have been deprecated into aliases of the new terms.

Fixes: #5429
Change-Id: Ia1e66e7a50ac35d3f6260d8bf6ba3ce8087cbad2
2020-09-14 11:34:49 -04:00
Mike Bayer 25b8e89d3d Deprecate engine-wise ss cursors; repair mariadbconnector
The server_side_cursors engine-wide feature relies upon
regexp parsing of statements a well as general guessing as
to when the feature should be used.   This is not within the
2.0 way of doing things and should be removed.

Additionally, mariadbconnector defaults to unbuffered cursors;
add new cursor hooks so that mariadbconnector can specify
buffered or unbuffered cursors without too much difficulty.
This will also correctly default mariadbconnector to buffered
cursors which should repair the segfaults we've been getting.

Try to restore the assert_raises that was removed in
5b6dfc0c38 to see if mariadbconnector segfaults
are resolved.

Change-Id: I77f1c972c742e40694972f578140bb0cac8c39eb
2020-09-13 13:06:02 -04:00
mike bayer 645be4aa24 Merge "Improve handling of covering indexes" 2020-09-12 19:46:50 +00:00
Gord Thompson 1a08d1aade Improve handling of covering indexes
Improved support for covering indexes (with INCLUDE columns). Added the
ability for postgresql to render CREATE INDEX statements with an INCLUDE
clause from Core. Index reflection also report INCLUDE columns separately
for both mssql and postgresql (11+).

Fixes: #4458
Change-Id: If0b82103fbc898cdaeaf6a6d2d421c732744acd6
2020-09-12 13:00:16 -04:00
mike bayer a2f2863f5a Merge "Raname `mustexit to must_exist`" 2020-09-12 16:36:00 +00:00
mike bayer f53e5fc7cb Merge "Reflect mssql/postgresql filtered/partial indexes" 2020-09-12 16:26:28 +00:00
RamonWill 69502725db Reflect mssql/postgresql filtered/partial indexes
Added support for inspection / reflection of partial indexes / filtered
indexes, i.e. those which use the ``mssql_where`` or ``postgresql_where``
parameters, with :class:`_schema.Index`.   The entry is both part of the
dictionary returned by :meth:`.Inspector.get_indexes` as well as part of a
reflected :class:`_schema.Index` construct that was reflected.  Pull
request courtesy Ramon Williams.

**Have a nice day!**
Fixes: #4966

Closes: #5504
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5504
Pull-request-sha: b3018bac98

Change-Id: Icbb2f93d1545700718ccb5222097185b815f5dbc
2020-09-12 10:30:43 -04:00
mike bayer 34870a0f6e Merge "Emit deprecation warning for **kw passed to session.execute()" 2020-09-12 13:10:19 +00:00
Mike Bayer 53ad41cbd9 fix typos
Change-Id: I7649576bd1e100fd6f8968f0256b074263943a44
2020-09-11 11:07:33 -04:00
Mike Bayer fe6c3605df Emit deprecation warning for **kw passed to session.execute()
Passing keyword arguments to methods such as :meth:`_orm.Session.execute`
to be passed into the :meth:`_orm.Session.get_bind` method is deprecated;
the new :paramref:`_orm.Session.execute.bind_arguments` dictionary should
be passed instead.

Fixes: #5573
Change-Id: I555bda84384dbf6d12ba4483c486f9488be0fa25
2020-09-11 10:50:58 -04:00
Mike Bayer 450f5c0d65 Build out new declarative systems; deprecate mapper()
The ORM Declarative system is now unified into the ORM itself, with new
import spaces under ``sqlalchemy.orm`` and new kinds of mappings.  Support
for decorator-based mappings without using a base class, support for
classical style-mapper() calls that have access to the declarative class
registry for relationships, and full integration of Declarative with 3rd
party class attribute systems like ``dataclasses`` and ``attrs`` is now
supported.

Fixes: #5508
Change-Id: I130b2b6edff6450bfe8a3e6baa099ff04b5471ff
2020-09-10 17:53:53 -04:00
Mike Bayer 96bb6dc56d Add 3rd party integration issues FAQ; add numpy
Users of numpy and pandas should refer to this section
where we will try to catalog the common issues that don't
really have an automatic solution.

Fixes: #5552
Change-Id: I02990e328616ccb21933ffda1167c52256bace07
2020-09-10 15:48:14 -04:00
Mike Bayer e78c8f8495 Add more docs for populate_existing(); link with_for_update
The populate_existing() method is actually changing
to be an execution option, however it has almost no
mention in the narrative docs so add docs in terms of the
1.x version first, including that we mention you almost
definitely want to use this method if you are also using
with_for_update().

Fixes: #5572
Fixes: #4774
Change-Id: Ieca916400622c1ffc1ae81204132a02a0983594c
2020-09-10 12:15:16 -04:00
Mike Bayer 5ed90cb05a Repair documentation issues
Fixes some remaining issues detailed at #5428.

Fixes: #5428
Change-Id: I942a64411766fc82f30791eee570747a218af77d
2020-09-09 11:01:01 -04:00
Federico Caselli e860060866 Update select usage to use the new 1.4 format
This change includes mainly that the bracketed use within
select() is moved to positional, and keyword arguments are
removed from calls to the select() function.  it does not
yet fully address other issues such as keyword arguments passed
to the table.select().

Additionally, allows False / None to both be considered
as "disable" for all of select.correlate(), select.correlate_except(),
query.correlate(), which establishes consistency with
passing of ``False`` for the legact select(correlate=False)
argument.

Change-Id: Ie6c6e6abfbd3d75d4c8de504c0cf0159e6999108
2020-09-08 17:13:48 -04:00
mike bayer 0d56a62f72 Merge "Create connection characteristics API; implement postgresql flags" 2020-09-08 15:17:37 +00:00
Mike Bayer e3716012c5 Create connection characteristics API; implement postgresql flags
Added support for PostgreSQL "readonly" and "deferrable" flags for all of
psycopg2, asyncpg and pg8000 dialects.   This takes advantage of a newly
generalized version of the "isolation level" API to support other kinds of
session attributes set via execution options that are reliably reset
when connections are returned to the connection pool.

Fixes: #5549
Change-Id: I0ad6d7a095e49d331618274c40ce75c76afdc7dd
2020-09-08 11:16:53 -04:00
Mike Bayer 6dc8d1dc69 PostgreSQL dialect-level isolation_level parameter is legacy
The isolation level section in the docs inadvertently
copied the PostgreSQL example using the PGDialect.isolation_level
parameter and not the execution_options.  ensure only
the execution_options version is documented.

Change-Id: I94e02ede62d3dded40e3fcbce8d04608dc063108
2020-09-08 11:01:28 -04:00
Federico Caselli bcfb12f813 Raname `mustexit to must_exist`
:class:`_schema.Table` parameter ``mustexist`` has been renamed
to :paramref:`_schema.Table.must_exist` and will now warn when used.

Change-Id: I0b0ca6021f9f7cfbe2040bbc1125a2236ac79f53
2020-09-03 23:46:35 +02:00
mike bayer fd1129efd4 Merge "Add new MySQL reserved words: cube, lateral." 2020-09-03 16:17:23 +00:00
Mike Bayer b050169600 Add caveat re: with_expression and already-loaded object
Adds a test for the populate_existing() behavior as well.

Fixes: #5553
Change-Id: Ib0db6227c3fec7d0065f2a7caa36b3fd94ef14fd
2020-09-02 14:56:49 -04:00
mike bayer 7439697ca3 Merge "Improve reflection for mssql temporary tables" 2020-09-01 17:12:11 +00:00
mike bayer d61cf0a9ad Merge "Add support for classical mapping of dataclasses" 2020-09-01 17:10:53 +00:00
mike bayer 3fe3b88cc0 Merge "MariaDB dialect implementation" 2020-09-01 17:09:19 +00:00