Commit Graph

5931 Commits

Author SHA1 Message Date
mike bayer 372ea0893b Merge "Add recommendation for URL.create() re: escaping" into main 2022-11-27 16:32:53 +00:00
Gord Thompson 14c73685ba Add recommendation for URL.create() re: escaping
Let users know that URL.create() can build the
whole connection URL instead of making them
escape things like passwords ad-hoc.

includes some general cleanup of URL docstring
by mike

Change-Id: Ic71bb0201fecf30e1db11e006c269f2d041b5439
2022-11-27 10:41:33 -05:00
Federico Caselli 61443aa62b Implement ScalarValue
Added :class:`_expression.ScalarValues` that can be used as a column
element allowing using :class:`_expression.Values` inside IN clauses
or in conjunction with ``ANY`` or ``ALL`` collection aggregates.
This new class is generated using the method
:meth:`_expression.Values.scalar_values`.
The :class:`_expression.Values` instance is now coerced to a
:class:`_expression.ScalarValues` when used in a ``IN`` or ``NOT IN``
operation.

Fixes: #6289
Change-Id: Iac22487ccb01553684b908e54d01c0687fa739f1
2022-11-26 18:49:06 -05:00
Mike Bayer df3bd8d297 add "merge" to viewonly cascades; propagate NO_RAISE when merging
Fixed bug where :meth:`_orm.Session.merge` would fail to preserve the
current loaded contents of relationship attributes that were indicated with
the :paramref:`_orm.relationship.viewonly` parameter, thus defeating
strategies that use :meth:`_orm.Session.merge` to pull fully loaded objects
from caches and other similar techniques. In a related change, fixed issue
where an object that contains a loaded relationship that was nonetheless
configured as ``lazy='raise'`` on the mapping would fail when passed to
:meth:`_orm.Session.merge`; checks for "raise" are now suspended within
the merge process assuming the :paramref:`_orm.Session.merge.load`
parameter remains at its default of ``True``.

Overall, this is a behavioral adjustment to a change introduced in the 1.4
series as of 🎫`4994`, which took "merge" out of the set of cascades
applied by default to "viewonly" relationships. As "viewonly" relationships
aren't persisted under any circumstances, allowing their contents to
transfer during "merge" does not impact the persistence behavior of the
target object. This allows :meth:`_orm.Session.merge` to correctly suit one
of its use cases, that of adding objects to a :class:`.Session` that were
loaded elsewhere, often for the purposes of restoring from a cache.

Fixes: #8862
Change-Id: I8731c7810460e6a71f8bf5e8ded59142b9b02956
2022-11-24 11:50:48 -05:00
Mike Bayer 1c9aa8c046 improve docs for event listening with asyncio
add examples, including how to listen on an async_sessionmaker

Change-Id: I1d68324458c23bc830508c8ebfd3224d434c16a5
2022-11-23 13:11:26 -05:00
mike bayer 3b6a5fb05c Merge "ensure implicit_returning is checked on a Table instance" into main 2022-11-22 23:30:36 +00:00
Mike Bayer ab84559ca0 ensure implicit_returning is checked on a Table instance
Fixed regression where flushing a mapped class that's mapped against a
subquery, such as a direct mapping or some forms of concrete table
inheritance, would fail if the :paramref:`_orm.Mapper.eager_defaults`
parameter were used.

mapper.local_table can still be a non-table instance.
correct the check first added in 466ed5b53a.

Fixes: #8812
Change-Id: I1bb2b83c31b910fbd96fcd3ac43e789b657aebf7
2022-11-22 14:06:09 -05:00
mike bayer 38db017a2c Merge "Type annotations for sqlalchemy.ext.mutable" into main 2022-11-22 16:42:45 +00:00
mike bayer cbc42334b1 Merge "fix optionalized forms for dict[]" into main 2022-11-22 16:07:49 +00:00
Gleb Kisenkov ba0e508141 Type annotations for sqlalchemy.ext.mutable
The ``sqlalchemy.ext.mutable`` extension is now fully pep-484 typed. Huge
thanks to Gleb Kisenkov for their efforts on this.

Fixes: #8667
Closes: #8775
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8775
Pull-request-sha: b907888ec6

Change-Id: Id9224e03201e6970b1ec56eb546ece4b2f3e0edd
2022-11-22 10:39:41 -05:00
Mike Bayer 509ffeedef fix optionalized forms for dict[]
Fixed a suite of issues involving :class:`.Mapped` use with dictionary
types, such as ``Mapped[dict[str, str] | None]``, would not be correctly
interpreted in Declarative ORM mappings. Support to correctly
"de-optionalize" this type including for lookup in type_annotation_map has
been fixed.

Fixes: #8777
Change-Id: Iaba90791dea341d00eaff788d40b0a4e48dab02e
2022-11-22 09:49:02 -05:00
mike bayer 4f0da8f07a Merge "Fix inferred type of async_sessionmaker" into main 2022-11-21 21:24:11 +00:00
mike bayer 5ff55f7078 Merge "add common base class for all SQL col expression objects" into main 2022-11-21 21:23:12 +00:00
Sam Bull e43e240a00 Fix inferred type of async_sessionmaker
Improved the typing for :class:`.sessionmaker` and
:class:`.asyncsessionmaker`, so that the default type of their return value
will be :class:`.Session` or :class:`.AsyncSession`, without the need to
type this explicitly. Previously, Mypy would not automaticaly infer these
return types from its generic base.

As part of this change, arguments for :class:`.Session`,
:class:`.AsyncSession`, :class:`.sessionmaker` and
:class:`.asyncsessionmaker` beyond the initial "bind" argument have been
made keyword-only, which includes parameters that have always been
documented as keyword arguments, such as :paramref:`.Session.autoflush`,
:paramref:`.Session.class_`, etc.

Pull request courtesy Sam Bull.

Closes: #8842
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8842
Pull-request-sha: fa6d1a8468

Change-Id: Iaaabc4572a87489d61617d970f62b9b50db4fac7
2022-11-21 12:01:56 -05:00
Mike Bayer caa9293e2e add common base class for all SQL col expression objects
Added a new type :class:`.SQLColumnExpression` which may be indicated in
user code to represent any SQL column oriented expression, including both
those based on :class:`.ColumnElement` as well as on ORM
:class:`.QueryableAttribute`. This type is a real class, not an alias, so
can also be used as the foundation for other objects.

Fixes: #8847
Change-Id: I3161bdff1c9f447793fce87864e1774a90cd4146
2022-11-21 11:55:23 -05:00
Mike Bayer ece524c005 fall back to eval() for names that have dots
Fixed regression in 2.0.0b3 caused by 🎫`8759` where indicating the
:class:`.Mapped` name using a qualified name such as
``sqlalchemy.orm.Mapped`` would fail to be recognized by Declarative as
indicating the :class:`.Mapped` construct.

Fixes: #8853
Change-Id: Iddb6efaae864d4545e80c54658244670f81ef6cc
2022-11-21 08:52:12 -05:00
Mike Bayer 46e6693cb3 clarify role of __allow_unmapped__
A different note will be needed in the 1.4 version
of these docs.

Fixes: #8845
Change-Id: I48651c64d511684077c80a317349dd230424b575
2022-11-19 13:10:31 -05:00
mike bayer 1fd68ac20a Merge "Issue #8765: implement missing methods on PG Range" into main 2022-11-17 14:37:57 +00:00
Mike Bayer 200e70b974 accommodate NULL format_type()
Made an adjustment to how the PostgreSQL dialect considers column types
when it reflects columns from a table, to accommodate for alternative
backends which may return NULL from the PG ``format_type()`` function.

Fixes: #8748
Change-Id: I6178287aac567210a76afaa5805b825daa7fa4db
2022-11-16 20:11:18 -05:00
mike bayer 3fc6c40ea7 Merge "perf improvements related to corresponding_column (2)" into main 2022-11-16 00:03:08 +00:00
Lele Gaifax bc696220bb Issue #8765: implement missing methods on PG Range
### Description
This PR implements missing methods on the PG `Range` class, as described by issue #8765.

### Checklist
This pull request is:

- [ ] A documentation / typographical error fix
- [ ] A short code fix
- [x] A new feature implementation

Closes: #8766
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8766
Pull-request-sha: 21c0df86cc

Change-Id: I86fabd966ad1f14a3a86132be741df46965b9aa9
2022-11-15 15:27:34 -05:00
Mike Bayer d5be2cc139 perf improvements related to corresponding_column (2)
commit two of two. this reorganizes ColumnCollection
to build a new index up front that's used to optimize
the corresponding_column() method.

Additional performance enhancements within ORM-enabled SQL statements,
specifically targeting callcounts within the construction of ORM
statements, using combinations of :func:`_orm.aliased` with
:func:`_sql.union` and similar "compound" constructs, in addition to direct
performance improvements to the ``corresponding_column()`` internal method
that is used heavily by the ORM by constructs like :func:`_orm.aliased` and
similar.

Fixes: #8796
Change-Id: I4a76788007d5a802b9a4081e6a0f6e4b52497b50
2022-11-15 14:16:06 -05:00
mike bayer e723db6c46 Merge "Fix relationship.order_by typing" into main 2022-11-15 18:14:24 +00:00
Andy Garfield 36954a3dbb Small tutorial rewording
The language is this sentence took me a few reads to understand. This is just a rewording.
2022-11-15 11:28:58 -05:00
Bryan Forbes 2fdfb5527e Fix relationship.order_by typing
Fixes: #8776

### Checklist

This pull request is:

- [ ] A documentation / typographical error fix
	- Good to go, no issue or tests are needed
- [x] A short code fix
	- please include the issue number, and create an issue if none exists, which
	  must include a complete example of the issue.  one line code fixes without an
	  issue and demonstration will not be accepted.
	- Please include: `Fixes: #<issue number>` in the commit message
	- please include tests.   one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
	- please include the issue number, and create an issue if none exists, which must
	  include a complete example of how the feature would look.
	- Please include: `Fixes: #<issue number>` in the commit message
	- please include tests.

Closes: #8779
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8779
Pull-request-sha: 2bc64555a1

Change-Id: I52fe6db5ba954a6039836b45f0e89ad96049df03
2022-11-15 09:03:25 -05:00
Mike Bayer eea0f44bbd add informative exception context for literal render
An informative re-raise is now thrown in the case where any "literal
bindparam" render operation fails, indicating the value itself and
the datatype in use, to assist in debugging when literal params
are being rendered in a statement.

Fixes: #8800
Change-Id: Id658f8b03359312353ddbb0c7563026239579f7b
2022-11-14 10:28:40 -05:00
Mike Bayer 1a81cc3f39 cherry-pick changelog update for 1.4.45 2022-11-12 09:13:38 -05:00
Mike Bayer d89659aef8 cherry-pick changelog from 1.4.44 2022-11-12 09:13:36 -05:00
Mike Bayer 5fc22d0e64 port mypy 0.990 fixes changelog from 1.4 to 2.0
for 2.0, we fixed for mypy 0.990 in ebb54e80a5.
In 1.4, we did a smaller version specific to the
test runner in 910504b67c.  Since people
run the tests sometimes, include that this was fixed
for the 1.4/2.0 series.

Change-Id: I18b2aa10bd8a128b5aa093af5e219bc987271139
2022-11-11 21:13:06 -05:00
mike bayer 6b140afe7d Merge "ensure anon_map is passed for most annotated traversals" into main 2022-11-11 21:30:43 +00:00
mike bayer 1dd0f23e8d Merge "establish consistency for RETURNING column labels" into main 2022-11-11 21:00:21 +00:00
Mike Bayer 1d8833a9c1 ensure anon_map is passed for most annotated traversals
We can cache the annotated cache key for Table, but
for selectables it's not safe, as it fails to pass the
anon_map along and creates many redudant structures in
observed test scenario.  It is likely safe for a
Column that's mapped to a Table also, however this is
not implemented here.   Will have to see if that part
needs adjusting.

Fixed critical memory issue identified in cache key generation, where for
very large and complex ORM statements that make use of lots of ORM aliases
with subqueries, cache key generation could produce excessively large keys
that were orders of magnitude bigger than the statement itself. Much thanks
to Rollo Konig Brock for their very patient, long term help in finally
identifying this issue.

Also within TypeEngine objects, when we generate elements
for instance variables, skip the None elements at least.
this also saves on tuple complexity.

Fixes: #8790
Change-Id: I448ddbfb45ae0a648815be8dad4faad7d1977427
2022-11-11 15:25:35 -05:00
Mike Bayer 604611e7e5 repair --disable-asyncio parameter
Fixed issue where the ``--disable-asyncio`` parameter to the test suite
would fail to not actually run greenlet tests and would also not prevent
the suite from using a "wrapping" greenlet for the whole suite. This
parameter now ensures that no greenlet or asyncio use will occur within the
entire run when set.

Fixes: #8793
Change-Id: I87b510846b2cc24413cd54e7b7136e91aad3c309
2022-11-11 11:20:34 -05:00
Mike Bayer 8e91cfe529 establish consistency for RETURNING column labels
The RETURNING clause now renders columns using the routine as that of the
:class:`.Select` to generate labels, which will include disambiguating
labels, as well as that a SQL function surrounding a named column will be
labeled using the column name itself. This is a more comprehensive change
than a similar one made for the 1.4 series that adjusted the function label
issue only.

includes 1.4's changelog for the backported version which also
fixes an Oracle issue independently of the 2.0 series.

Fixes: #8770
Change-Id: I2ab078a214a778ffe1720dbd864ae4c105a0691d
2022-11-11 16:20:00 +00:00
Mike Bayer e3a8d19891 work around Python 3.11 IntEnum issue; update FastIntFlag
in [1], Python 3.11 seems to have changed the behavior of
IntEnum.  We didn't notice this because we have our own
workaround class already, but typing did.   Ensure we remain
compatible with IntFlag.

This change also modifies FastIntFlag to no longer use
global symbols; this is unnecessary as we assign FastIntFlag
members explicitly.  Use of ``symbol()`` should probably
be phased out.

[1] https://github.com/python/cpython/issues/99304
Fixes: #8783

Change-Id: I8ae2e871ff1467ae5ca1f63e66b5dae45d4a6c93
2022-11-10 12:19:02 -05:00
Mike Bayer 157af86615 Version 2.0.0b4 placeholder 2022-11-04 17:15:01 -04:00
Mike Bayer 35648a1094 - 2.0.0b3 2022-11-04 17:09:03 -04:00
Mike Bayer 2b22acd924 cherry-pick changelog update for 1.4.44 2022-11-04 17:08:40 -04:00
Mike Bayer 0077eb1274 cherry-pick changelog from 1.4.43 2022-11-04 17:08:39 -04:00
Mike Bayer 6410e372cb typo
Change-Id: I41cb128767c0cba48255cca0904ae1bff1b357ac
2022-11-04 17:00:29 -04:00
Mike Bayer 6567d8cd7a fixes
Change-Id: Id4c3c24bbff0a8bc1f20ff8deb10a098d2cca8b7
2022-11-04 16:59:37 -04:00
Mike Bayer 751469240a changelog updates
as this release cycle was fairly frenetic, a lot of these
changelogs were very poorly worded (by me).

Change-Id: Idb796cf3e25975fb2f75bacf26f1cb57ef0e4cad
2022-11-04 16:16:48 -04:00
mike bayer 4eb99ef08c Merge "support renamed symbols in annotation scans" into main 2022-11-04 18:37:56 +00:00
Mike Bayer c0b8bdcaf0 support renamed symbols in annotation scans
Added support in ORM declarative annotations for class names specified for
:func:`_orm.relationship`, as well as the name of the :class:`_orm.Mapped`
symbol itself, to be different names than their direct class name, to
support scenarios such as where :class:`_orm.Mapped` is imported as
``from sqlalchemy.orm import Mapped as M``, or where related class names
are imported with an alternate name in a similar fashion. Additionally, a
target class name given as the lead argument for :func:`_orm.relationship`
will always supersede the name given in the left hand annotation, so that
otherwise un-importable names that also don't match the class name can
still be used in annotations.

Fixes: #8759
Change-Id: I74a00de7e1a45bf62dad50fd385bb75cf343f9f3
2022-11-04 13:38:06 -04:00
Mike Bayer 7b6259c0f3 resolve synonyms in dictionary form of Session.get()
Improved "dictionary mode" for :meth:`_orm.Session.get` so that synonym
names which refer to primary key attribute names may be indicated in the
named dictionary.

Fixes: #8753
Change-Id: I56112564a5c23b51b26e01c64087cbf4399cd951
2022-11-04 12:49:49 -04:00
Mike Bayer b96321ae79 Support result.close() for all iterator patterns
This change contains new features for 2.0 only as well as some
behaviors that will be backported to 1.4.

For 1.4 and 2.0:

Fixed issue where the underlying DBAPI cursor would not be closed when
using :class:`_orm.Query` with :meth:`_orm.Query.yield_per` and direct
iteration, if a user-defined exception case were raised within the
iteration process, interrupting the iterator. This would lead to the usual
MySQL-related issues with server side cursors out of sync.

For 1.4 only:

A similar scenario can occur when using :term:`2.x` executions with direct
use of :class:`.Result`, in that case the end-user code has access to the
:class:`.Result` itself and should call :meth:`.Result.close` directly.
Version 2.0 will feature context-manager calling patterns to address this
use case.  However within the 1.4 scope, ensured that ``.close()`` methods
are available on all :class:`.Result` implementations including
:class:`.ScalarResult`, :class:`.MappingResult`.

For 2.0 only:

To better support the use case of iterating :class:`.Result` and
:class:`.AsyncResult` objects where user-defined exceptions may interrupt
the iteration, both objects as well as variants such as
:class:`.ScalarResult`, :class:`.MappingResult`,
:class:`.AsyncScalarResult`, :class:`.AsyncMappingResult` now support
context manager usage, where the result will be closed at the end of
iteration.

Corrected various typing issues within the engine and async engine
packages.

Fixes: #8710
Change-Id: I3166328bfd3900957eb33cbf1061d0495c9df670
2022-11-03 18:42:52 -04:00
mike bayer a5eed23c06 Merge "Implement contains_value(), issubset() and issuperset() on PG Range" into main 2022-11-03 20:11:58 +00:00
Patrick Hoefler d4e846f969 DOC: Fix typo avoiding rendering of legacy marker (#8752) 2022-11-03 19:36:35 +01:00
Lele Gaifax e8124b29b0 Implement contains_value(), issubset() and issuperset() on PG Range
Added new methods :meth:`_postgresql.Range.contains` and
:meth:`_postgresql.Range.contained_by` to the new :class:`.Range` data
object, which mirror the behavior of the PostgreSQL ``@>`` and ``<@``
operators, as well as the
:meth:`_postgresql.AbstractRange.comparator_factory.contains` and
:meth:`_postgresql.AbstractRange.comparator_factory.contained_by` SQL
operator methods. Pull request courtesy Lele Gaifax.

Fixes: #8706
Closes: #8707
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8707
Pull-request-sha: 3a74a0d93e

Change-Id: Ief81ca5c31448640b26dfbc3defd4dde1d51e366
2022-11-03 09:30:38 -04:00
Mike Bayer 32b2b31da2 use simple decimal query to detect decimal char
Fixed issue where the ``nls_session_parameters`` view queried on first
connect in order to get the default decimal point character may not be
available depending on Oracle connection modes, and would therefore raise
an error.  The approach to detecting decimal char has been simplified to
test a decimal value directly, instead of reading system views, which
works on any backend / driver.

Fixes: #8744
Change-Id: I39825131c13513798863197d0c180dd5a18b32dc
2022-11-01 17:05:52 -04:00