Commit Graph

15358 Commits

Author SHA1 Message Date
Mike Bayer 0eea54e84d remove erroneous note about future metadata
this change was already applied

Change-Id: I24cfdc1912b77f98ae6d0f3865cabd223553fc79
2022-08-17 09:17:33 -04:00
RamonWill a134ec1760 Add support for Partitioning and Sample pages on mysql
Add support for Partitioning and Sample pages on MySQL and MariaDB
reflected options.
The options are stored in the table dialect options dictionary, so
the following keyword need to be prefixed with ``mysql_`` or ``mariadb_``
depending on the backend.
Supported options are:

* ``stats_sample_pages``
* ``partition_by``
* ``partitions``
* ``subpartition_by``

These options are also reflected when loading a table from database,
and will populate the table :attr:`_schema.Table.dialect_options`.
Pull request courtesy of Ramon Will.

Fixes: #4038

Closes: #5536
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5536
Pull-request-sha: f8852cabe1

Change-Id: I69b60576532af04c725c998e9e8fec6e2040b149
2022-08-11 21:39:45 +02:00
Mike Bayer 6f75807063 reorg bulk persistence into a separate module
This restores persistence.py to only functions that are used
by unitofwork.py, and all the "bulk" stuff gets its own
module bulk_persistence.py.  Also fixes up the ORM context
class hierarchy for bulk.

This is all ahead of the ORM-insert changes coming in, so that
the later review can be about logic and not about reorganization.

Change-Id: I035896e9e77fcece866d246edf30097cccad0182
2022-08-11 11:04:01 -04:00
Mike Bayer c4b7f1f7c9 doc fixes
* fixed erroneous use of mapped_column() in m2m relationship Table
* Fill in full imports for some relationship examples that had
  partial imports; examples that have no imports, leave empty for now
* converted joined/single inh mappings to annotated style
* We have a problem with @declared_attr in that the error message
  is wrong if the mapped_column() returned doesnt have a type, and/or
  mapped_column() with @declared_attr doesnt use the annotation
* fix thing where sphinx with undoc-members global setting seems to
  no longer tolerate ":attribute:" entries in autodoc classes, which
  is fine we can document the annotations now
* Fix mapper params in inheritance to be on Mapper
* add missing changelog file for instances remove

Change-Id: I9b70b25a320d8122fade68bc4d1f82f8b72b26f3
2022-08-10 12:55:19 -04:00
Mike Bayer 7e442cd0a9 improve names for datatypes section
Continuing along #8362, if we look at the document here without
"this is the same text I've read for 14 years", we begin to see
that the title "Column and Data types" makes no sense at all,
is there a "column type" and a "Data type"?  I guess what I was
thinking at that time is that a type can be placed on a
Column, or it can be used whenever you have, you know, "data".
The phrase "SQL expression" wasn't discovered yet.

"SQL Datatype" is not spectacular but at least it's one term.
the new intro then is focused on the hierarchy layout so let's
name it that.

not amazing, but better than the duplicate name that made
no sense before.

Fixes: #8362
Change-Id: Iab37ef5605ec55f30284ac9a98bf7246f736675d
2022-08-09 09:50:57 -04:00
Mike Bayer ac1d775d86 improve typing intro
Fixes: #8362
Change-Id: I38aa1727e94c50a9f06bd75d57ea1ca1cfffd2f3
2022-08-08 14:50:29 -04:00
Mike Bayer 3af0765529 cherry-pick changelog update for 1.4.41 2022-08-08 12:16:26 -04:00
Mike Bayer 463133a7c2 cherry-pick changelog from 1.4.40 2022-08-08 12:16:25 -04:00
Mike Bayer b7c7864490 repair doc warnings
Change-Id: I446105028539a34da90d6b8ae4812965cc398ee5
(cherry picked from commit c539ee3522)
2022-08-08 12:08:55 -04:00
Mike Bayer 85fa363c84 deep compare CTEs before considering them conflicting
Fixed issue where referencing a CTE multiple times in conjunction with a
polymorphic SELECT could result in multiple "clones" of the same CTE being
constructed, which would then trigger these two CTEs as duplicates. To
resolve, the two CTEs are deep-compared when this occurs to ensure that
they are equivalent, then are treated as equivalent.

Fixes: #8357
Change-Id: I1f634a9cf7a6c4256912aac1a00506aecea3b0e2
2022-08-05 17:26:00 -04:00
mike bayer f085762744 Merge "translate joined inheritance cols in UPDATE/DELETE" into main 2022-08-05 19:29:26 +00:00
Mike Bayer 8cb6fbfb65 credit @zeeeeeb for PG Multirange patch
Also move some of the PG docs to the .rst
page so we can link to sections.

References: #7156

Change-Id: If57abc768d4768058ffa768f9bf72f83c1ee6c29
2022-08-05 13:30:32 -04:00
mike bayer bc5b198c6f Merge "implement PG ranges/multiranges agnostically" into main 2022-08-05 17:16:18 +00:00
mike bayer 83ead60141 Merge "fixes: #7156 - Adds support for PostgreSQL MultiRange type" into main 2022-08-05 17:15:56 +00:00
Mike Bayer 9f992e4218 translate joined inheritance cols in UPDATE/DELETE
Fixed issue in ORM enabled UPDATE when the statement is created against a
joined-inheritance subclass, updating only local table columns, where the
"fetch" synchronization strategy would not render the correct RETURNING
clause for databases that use RETURNING for fetch synchronization.
Also adjusts the strategy used for RETURNING in UPDATE FROM and
DELETE FROM statements.

Also fixes MariaDB which does not support RETURNING with
DELETE..USING.  this was not caught in tests because
"fetch" strategy wasn't tested.  so also adjust the ORMDMLState
classes to look for "extra froms" first before adding
RETURNING, add new parameters to interfaces for
"update_returning_multitable" and "delete_returning_multitable".
A new execution option is_delete_using=True, described in the
changelog message, is added to allow the ORM to know up front
if a certain statement should have a SELECT up front
for "fetch" strategy.

Fixes: #8344
Change-Id: I3dcdb68e6e97ab0807a573c2fdb3d53c16d063ba
2022-08-05 12:53:35 -04:00
mike bayer b04a05f6f7 Merge "include column.default, column.onupdate in eager_defaults" into main 2022-08-05 14:54:12 +00:00
mike bayer e332909b10 Merge "Support kw_only and match_args in dataclass mapping" into main 2022-08-05 14:53:22 +00:00
Mike Bayer fce1d954aa implement PG ranges/multiranges agnostically
Ranges now work using a new Range object,
multiranges as lists of Range objects (this is what
asyncpg does.  not sure why psycopg has a "Multirange"
type).

psycopg, psycopg2, and asyncpg are currently supported.
It's not clear how to make ranges work with pg8000, likely
needs string conversion; this is straightforward with the
new archicture and can be added later.

Fixes: #8178
Change-Id: Iab8d8382873d5c14199adbe3f09fd0dc17e2b9f1
2022-08-05 10:39:39 -04:00
Mike Bayer 82a1d4096f include column.default, column.onupdate in eager_defaults
Fixed bug in the behavior of the :paramref:`_orm.Mapper.eager_defaults`
parameter such that client-side SQL default or onupdate expressions in the
table definition alone will trigger a fetch operation using RETURNING or
SELECT when the ORM emits an INSERT or UPDATE for the row. Previously, only
server side defaults established as part of table DDL and/or server-side
onupdate expressions would trigger this fetch, even though client-side SQL
expressions would be included when the fetch was rendered.

Fixes: #7438
Change-Id: Iba719298ba4a26d185edec97ba77d2d54585e5a4
2022-08-05 10:07:15 -04:00
mike bayer 828a1a0d14 Merge "deprecate Query.instances()" into main 2022-08-04 20:30:19 +00:00
Federico Caselli d2887d03a2 Support kw_only and match_args in dataclass mapping
Fixes: #8346
Change-Id: I964629e3bd25221bf6df6ab31c59b3ce1983cd9a
2022-08-04 18:28:57 +02:00
zeeeeb eeff036db6 fixes: #7156 - Adds support for PostgreSQL MultiRange type
This adds functionality for PostgreSQL MultiRange type, as discussed in Issue #7156.

As far as I can tell, only psycopg provides a [Multirange adaptation](https://www.psycopg.org/psycopg3/docs/basic/pgtypes.html#multirange-adaptation). Psycopg2 only supports a [Range adaptation/data type](https://www.psycopg.org/psycopg3/docs/basic/pgtypes.html#multirange-adaptation).

This pull request is:

- [ ] A documentation / typographical error fix
	- Good to go, no issue or tests are needed
- [ ] A short code fix
	- please include the issue number, and create an issue if none exists, which
	  must include a complete example of the issue.  one line code fixes without an
	  issue and demonstration will not be accepted.
	- Please include: `Fixes: #<issue number>` in the commit message
	- please include tests.   one line code fixes without tests will not be accepted.
- [x] A new feature implementation
	- please include the issue number, and create an issue if none exists, which must
	  include a complete example of how the feature would look.
	- Please include: `Fixes: #<issue number>` in the commit message
	- please include tests.

Closes: #7816
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7816
Pull-request-sha: 7e9e0c858d

Change-Id: I345e0f58f534ac37709a7a4627b6de8ddd8fa89e
2022-08-04 09:39:38 -04:00
mike bayer 7c8572f004 Merge "Fix 'No transaction found' error on Synapse." into main 2022-08-04 13:37:38 +00:00
Mike Bayer 8032bfdcee deprecate Query.instances()
this method no longer does the thing that it was originally
intended to do, which is to get ORM results from arbitrary
result sets.   Modern patterns should supersede the
use of this construct.

Change-Id: Ia1656c84d7c323f55e3a9594b950a40763d63d90
References: #8347
2022-08-04 13:32:30 +00:00
mike bayer 0027b3a4bc Merge "ensure RETURNING renders in stringify w/ no server version" into main 2022-08-04 13:32:17 +00:00
Mike Bayer dc5a1c482e update quoted_name doc
Fixes: #8339
Change-Id: If78bc9babfdc6a4dde4e65d72858ac7a402cbb4d
2022-08-04 09:26:47 -04:00
Mike Bayer 0b57cc9564 more mysql 8.0.30 fixes
Change-Id: I9df3506f364f4721404cf2022486bc31fd5c2ce6
2022-08-03 20:47:50 -04:00
Mike Bayer f684bb7659 ensure RETURNING renders in stringify w/ no server version
just in my own testing, if I say insert().return_defaults()
and stringify, I should see it, so make sure all the dialects
default to "insert_returning" etc. , with downgrade on
server version check.

Change-Id: Id64e78fcb03c48b5dcb0feb21cb9cc495edd15e9
2022-08-03 20:47:27 -04:00
Mike Bayer ea6fb4ff5b adjust mysql utf test
we've updated mysql on jenkins and this test seems to need
a small adjustment

Change-Id: I21508f667700cf8f3200f15af501a66a85f48779
2022-08-03 18:09:39 -04:00
Fabian Preiß b5485fe418 glossary/association relationship: role->role_name (#8331) 2022-08-03 22:25:19 +02:00
Mike Bayer c2327ec60f send in the dragons on async_scoped_session
make it clear that async_scoped_session.remove() must
be called, else memory will build up.    Generally
discourage the whole pattern as well, as this is a
"framework" pattern and we don't really want to be supporting
frameworks.    Also indicate that scopefunc must be idempotent
and lightweight.

Fixes: #8340
Change-Id: Ibc3d21124ae73c3b25ee51966504bbb1975c36b2
2022-08-03 12:17:54 -04:00
Mike Bayer 3ef9fa6d4f reword yield_per a bit more
I'm still not satisified with this section as it is still
too wordy and dense, but at least let's put a better description
of what yield_per actually is and why one might use it at the top.

Change-Id: I10f4d862d9c499044f5718fca0d27ac106289717
2022-08-02 14:51:49 -04:00
Gord Thompson 5741068012 Fix 'No transaction found' error on Synapse.
Fixed issue where the SQL Server dialect's query for the current isolation
level would fail on Azure Synapse Analytics, due to the way in which this
database handles transaction rollbacks after an error has occurred. The
initial query has been modified to no longer rely upon catching an error
when attempting to detect the appropriate system view. Additionally, to
better support this database's very specific "rollback" behavior,
implemented new parameter ``ignore_no_transaction_on_rollback`` indicating
that a rollback should ignore Azure Synapse error 'No corresponding
transaction found. (111214)', which is raised if no transaction is present
in conflict with the Python DBAPI.

Fixes: #8231
Closes: #8233
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8233
Pull-request-sha: c48bd44a9f

Change-Id: I6407a03148f45cc9eba8fe1d31d4f59ebf9c7ef7
2022-08-02 13:59:16 -04:00
mike bayer 0d27f6c25d Merge "SQLite reflection ignores schema internal names" into main 2022-08-02 15:48:49 +00:00
mike bayer 66e58afe2a Merge "implement tuple-slices from .c collections" into main 2022-08-02 15:46:28 +00:00
Mike Bayer 37b8c5e755 fix up SAVEPOINT docs
these contained a factual error that the entire session is
expired, which is no longer the case (I can't find exactly
when this was changed).  Additionally, added a PostgreSQL
specific example w/ IntegrityError as this is the most
common case for this.   Tried to tighten up other language
and make it as clear as possible.

Change-Id: I39160e7443964db59d1d5a2e0616084767813eea
2022-08-02 11:34:35 -04:00
mike bayer 44f54bf4c0 Merge "repair psycopg2 (and psycopg) multiple hosts format" into main 2022-08-02 13:49:42 +00:00
Mike Bayer 7f45bcd114 document @ sign in issue template, docs
Fixes: #8328
Change-Id: I69a48c4499fe7e57aad242403186e69c4452b84b
2022-08-02 09:33:51 -04:00
Federico Caselli 63d90b0f44 SQLite reflection ignores schema internal names
Added new parameter to SQLite for reflection methods called
``sqlite_include_internal=True``; when omitted, local tables that start
with the prefix ``sqlite_``, which per SQLite documentation are noted as
"internal schema" tables such as the ``sqlite_sequence`` table generated to
support "AUTOINCREMENT" columns, will not be included in reflection methods
that return lists of local objects. This prevents issues for example when
using Alembic autogenerate, which previously would consider these
SQLite-generated tables as being remove from the model.

Fixes: #8234
Change-Id: I36ee7a053e04b6c46c912aaa0d7e035a5b88a4f9
2022-08-01 21:47:36 +00:00
Mike Bayer 1ecbf14cc2 implement tuple-slices from .c collections
Added new syntax to the ``.c`` collection on all :class:`.FromClause`
objects allowing tuples of keys to be passed to ``__getitem__()``, along
with support for ``select()`` handling of ``.c`` collections directly,
allowing the syntax ``select(table.c['a', 'b', 'c'])`` to be possible. The
sub-collection returned is itself a :class:`.ColumnCollection` which is
also directly consumable by :func:`_sql.select` and similar now.

Fixes: #8285
Change-Id: I2236662c477ffc50af079310589e213323c960d1
2022-08-01 21:46:33 +00:00
Mike Bayer ddc326585a repair psycopg2 (and psycopg) multiple hosts format
Fixed issue in psycopg2 dialect where the "multiple hosts" feature
implemented for 🎫`4392`, where multiple ``host:port`` pairs could be
passed in the query string as
``?host=host1:port1&host=host2:port2&host=host3:port3`` was not implemented
correctly, as it did not propagate the "port" parameter appropriately.
Connections that didn't use a different "port" likely worked without issue,
and connections that had "port" for some of the entries may have
incorrectly passed on that hostname. The format is now corrected to pass
hosts/ports appropriately.

As part of this change, maintained support for another multihost style that
worked unintentionally, which is comma-separated
``?host=h1,h2,h3&port=p1,p2,p3``. This format is more consistent with
libpq's query-string format, whereas the previous format is inspired by a
different aspect of libpq's URI format but is not quite the same thing.

If the two styles are mixed together, an error is raised as this is
ambiguous.

Fixes: #4392
Change-Id: Ic9cc0b0e6e90725e158d9efe73e042853dd1263f
2022-08-01 15:21:04 -04:00
Mike Bayer 3ff18812d8 exclude __new__
due to Generic this method seems to be coming out everywhere
and it's just noise

Change-Id: I8fabb462d5faebb156b147fbd8f89dbb1b1ba380
2022-07-31 10:34:26 -04:00
Nikita Sobolev 1657312ba7 Remove __cmp__ methods (#8313) 2022-07-31 12:08:36 +02:00
Federico Caselli f8c4dba4e9 Update to flake8 5.
Change-Id: I5a241a70efba68bcea9819ddce6aebc25703e68d
2022-07-31 11:56:07 +02:00
Mike Bayer 14bfbadfdf use inherited members for Query
not sure why these four methods were here, they don't
get sorted when doing it this way.

Change-Id: I554f132df3f299858ca5b451a79fbd9dd1f520ee
2022-07-30 15:15:19 -04:00
Nikita Sobolev 5de71c6302 Remove all __nonzero__ methods (#8308) 2022-07-30 20:48:26 +02:00
bbben b86112fd85 glossary: update the acronym definition (#8306)
* glossary: fix typo

* add 'Read' to the CRUD definition
2022-07-30 20:43:59 +02:00
dependabot[bot] 7dfa644d06 Bump actions/checkout from 2 to 3 (#8310)
Bumps [actions/checkout](https://github.com/actions/checkout) from 2 to 3.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v2...v3)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-30 16:10:19 +02:00
dependabot[bot] 82e9e9dd8e Bump actions/setup-python from 2 to 4 (#8309)
Bumps [actions/setup-python](https://github.com/actions/setup-python) from 2 to 4.
- [Release notes](https://github.com/actions/setup-python/releases)
- [Commits](https://github.com/actions/setup-python/compare/v2...v4)

---
updated-dependencies:
- dependency-name: actions/setup-python
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-30 13:44:54 +02:00
Naveen b71cf3517e chore: Included githubactions in the dependabot config
This should help with keeping the GitHub actions updated on new releases. This will also help with keeping it secure.

Dependabot helps in keeping the supply chain secure https://docs.github.com/en/code-security/dependabot

GitHub actions up to date https://docs.github.com/en/code-security/dependabot/working-with-dependabot/keeping-your-actions-up-to-date-with-dependabot

https://github.com/ossf/scorecard/blob/main/docs/checks.md#dependency-update-tool
Signed-off-by: naveen <172697+naveensrinivasan@users.noreply.github.com>

Closes: #8180
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8180
Pull-request-sha: ef796e21f3

Change-Id: I8252914951985de81dce944c3ab076dedf439cde
2022-07-30 13:17:49 +02:00