The is_pep695() function incorrectly identified
Annotated[TypeAliasType, ...] as a PEP 695 type alias because
Annotated's __origin__ attribute returns the first type argument
(the TypeAliasType) rather than Annotated itself. This caused
_init_column_for_annotation to crash with AttributeError when
attempting to access __value__ on the Annotated wrapper.
Added a check for is_pep593() before recursing through __origin__
in is_pep695(), so Annotated types are correctly excluded.
Fixes: #13386
Change-Id: I36ef83ebbab5abc08bed0131efb552c3fc001911
* in selectinloader, dont use Bundle() to represent the PK portion
* use more efficient mapper._state_ident_getter() method in selectinloader
which pre-resolves keys and only calls upon _get_state_attr_by_column when
an attribute is not locally present
* removed use of groupby() + lambda against Row objects in subqueryloader; converts
to tuple and builds lists via append()
Adds tests pinning behavior of the rewritten result handling: the uselist=False multiple-rows warning, and many-to-one loads where the foreign key value matches no row or is NULL.
Benchmarked on an in-memory SQLite database (median of 30 runs, ms):
case | baseline | branch | Δ
-- | -- | -- | --
selectin_m2m | 9.747 | 6.958 | -28.6%
selectin_m2o | 13.761 | 12.538 | -8.9%
selectin_nested | 14.670 | 11.473 | -21.8%
selectin_o2m | 14.686 | 11.823 | -19.5%
selectin_o2m_few_big | 25.771 | 19.787 | -23.2%
subquery_o2m | 15.946 | 15.736 | -1.3%
Fixes: #13363Closes: #13364
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13364
Pull-request-sha: 3b858fc54b
Change-Id: I2e772c7ee9fa9e1b50026cab8c7997b861f1acda
ORM result row fetching now processes rows as plain tuples rather than
constructing :class:`.Row` objects, as ORM loaders use position-based
access and do not require the :class:`.Row` interface. :class:`.Row`
construction is still used when engine-level debug logging is enabled so
that individual rows can be logged. Benchmarks show a 3-16% improvement in
ORM entity load times depending on query shape. Pull request courtesy
Oliver Parker.
Adds Result._all_interim_rows(), which returns the remaining rows as processed plain tuples, applying result processors and tuple filters but skipping Row object construction. ORM loading uses this for its row fetch; its row getters are position-based itemgetters that accept any tuple-like row. Results that require row logging or have scalar sources fall back to Row construction.
Adds tests covering the new behavior: rows are plain tuples with result processors applied, and Row construction still occurs when engine-level row logging is enabled, at both the Result and ORM loading level.
Benchmarked on an in-memory SQLite database with this change alone (median of 30 runs, ms); this path is used by all ORM entity loads:
plain_small (500 rows x 5 cols) 2.43 -> 2.28 -6%
plain_wide (2000 rows x 25 cols) 8.17 -> 7.24 -11%
joined_o2m (500 x 10) 18.56 -> 16.95 -9%
selectin_o2m (500 x 10) 18.45 -> 17.79 -4%
selectin_nested (50 x 10 x 10) 18.67 -> 17.33 -7%
selectin_m2m (500 x 10) 12.70 -> 11.53 -9%
selectin_m2o (5000 -> 200) 15.83 -> 15.29 -3%
selectin_o2m_few_big (20 x 500) 32.96 -> 31.29 -5%
subquery_o2m (500 x 10) 21.51 -> 18.16 -16%
Fixes: #13363Closes: #13365
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13365
Pull-request-sha: 46f4125ead
Change-Id: Ia91a89c8cac78d4790391bc5b171b76d4aaca71b
Fixed regular expression in the pure Python hstore result processor,
used when ``use_native_hstore=False`` is set, which could hang on
malformed hstore text containing unterminated quoted segments with
backslashes. Pull request courtesy dxbjavid.
Fixes: #13370Closes: #13371
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13371
Pull-request-sha: f5eddae11c
Change-Id: I0d2d7565dc88f56a73b41e2ad20ca1c5a6f738bb
ensure all the main result methods as well as the ORM-used
_raw_all_rows() perform equivalent row logging.
Change-Id: Iee0dc3c75a9c71f6d1a73fa058c5edf0b16348ec
1.9.0 seems to fix the picklable statement exception fail,
so fix for that test and pin to lower version
Change-Id: Ic561b1a6b688f38b9f5189508d9203bca47fef79
Introduced _BackendsMultiReflection mixin in engine/default.py
that provides the get_columns(), get_pk_constraint(),
get_foreign_keys(), get_indexes(), get_unique_constraints(),
get_check_constraints(), get_table_comment(), and
get_table_options() single-table methods, each delegating to
the corresponding get_multi_* method with
filter_names=[table_name].
PostgreSQL, Oracle, and MSSQL dialects now inherit from this
mixin instead of duplicating the wrapper pattern. Oracle
retains its _value_or_raise() override which applies
normalize_name() for case-folding. MSSQL overrides
get_unique_constraints(), get_check_constraints(), and
get_table_options() with NotImplementedError since it has no
native get_multi_* for those yet.
Change-Id: If68bbf94b3956348fc7ae8179ff093d63dcdfbe2
When tpc_prepare() raised during SessionTransaction._prepare_impl(),
the error handler's call to self.rollback() was blocked by the
@declare_states decorator, which had set _next_state to
CHANGE_IN_PROGRESS. This caused IllegalStateChangeError to be raised
instead of the original database exception, masking the real error
and preventing proper cleanup.
Used _expect_state(SessionTransactionState.CLOSED) to temporarily
allow the rollback state transition, matching the existing pattern
used in commit() for the close() call.
Fixes: #13356
Change-Id: Ie8212d5b6f8515340cf9d83c56dcbfa5a7415812
Repaired bug introduced in 🎫`13229` where a two-phase
transaction recovery would not return the correct transaction
identifier when generating the identifiers using the ``xid()``
method of the psycopg connection.
Fixes: #13355
Change-Id: Iffe68c1701afaa678fa7b598559dd396d3f8db41
Removed the legacy ``include_columns`` key from the dictionary returned
by the index reflection methods of some dialects.
This information is now part of the ``dialect_options`` dictionary under the key
``{dialect_name}_include``, such as ``postgresql_include`` or ``mssql_include``.
Fixes: #13350
Change-Id: I9535690350d97e19477f7e9919dc4994b876af47
Session level :paramref:`_orm.Session.execution_options` now take
effect for Core level SQL emitted by unit of work operations, in
addition to their existing use within ORM statement executions.
This is to provide for Core options such as
:paramref:`_engine.Connection.execution_options.schema_translate_map`
to be applicable to a :class:`.Session` overall.
Fixes: #13346
Change-Id: Icb453b7925f1bc38f30538d7726d222842bc65bd
Added CollectionAggregateFunction base class that sets
_is_collection_aggregate = True, and registered any_, all_, some_
as subclasses so that func.any(), func.all(), and func.some() correctly
prevent operator flipping on negation. Previously ~(col == func.any(arr))
would incorrectly compile to col != any(arr) instead of
NOT (col = any(arr)), which has different semantics for collection
aggregate comparison modifiers.
Also extended the _construct_for_op guard to check both left and right
operands for _is_collection_aggregate, since func.any(arr) can appear
on either side of a comparison unlike the standalone any_() construct
which auto-reverses operands.
Fixes: #13343
Change-Id: Id4774938876dc7f1f38cf143dccfe3c8ddba464d
Optimized :func:`_orm.selectinload` to skip the ``.unique()`` call on inner
result sets when no nested :func:`_orm.joinedload` on a collection is
present. The uniquing pass is only required when a joined eager load
inflates rows due to a one-to-many or many-to-many JOIN; in the common case
of a leaf selectin load, rows are already unique by construction and the
per-row hashing overhead can be avoided. As a side effect, ``yield_per``
set in a ``do_orm_execute`` event for a :func:`_orm.selectinload`
relationship load no longer raises ``InvalidRequestError`` when no nested
collection joinedload is in effect, since ``.unique()`` is no longer called
in that path. Pull request courtesy Oliver Parker.
`_SelectInLoader._load_via_parent` and `_load_via_child` currently call
`.unique()` unconditionally on the inner `Result`. The uniqueness pass is only
required when a nested `joinedload` on a collection is in effect — in that case
the `JOIN` inflates rows (one row per `(child, grandchild)` instead of one per
child) and the outer `groupby` would produce duplicated entries without dedup.
`loading.instances()` already signals exactly this condition: it sets
`Result._unique_filter_state` to a `require_unique` guard when the inner
compile state has `multi_row_eager_loaders=True`. When that flag is unset (no
nested collection `joinedload`), `_unique_filter_state` stays `None`, meaning
the inner query produces unique rows by construction:
- `omit_join` 1:N — one row per child
- `omit_join` M2O — one row per parent
- `omit_join` M2M — one row per (parent, entity)
- `load_with_join` (non-omit) — one row per (parent, child)
This PR makes the `.unique()` call conditional, via a new `_has_unique_filter`
property on `Result` that exposes this state without reaching into the private
`_unique_filter_state` attribute directly:
```python
if result._has_unique_filter:
result = result.unique()
```
Per-row hashing in `_iterator_getter` is avoided in the common leaf-load case.
On an in-memory SQLite bench (2000 parents × 5 children + M:N tags, Python
3.14, n=1000 iterations):
| Workload | before avg | after avg | delta | before sd | after sd |
|---|---|---|---|---|---|
| selectin children (1:N) | 56.0 ms | 41.4 ms | **−26%** | 5.2 ms | 1.9 ms |
| selectin tags M2M | 25.6 ms | 20.0 ms | **−22%** | 4.2 ms | 3.1 ms |
| joined children (1:N) | 42.9 ms | 37.6 ms | ~0% | — | — |
| plain parent load | 4.2 ms | 3.6 ms | ~0% | — | — |
**Behaviour change:** `yield_per` set in a `do_orm_execute` event for a
relationship load no longer raises
`InvalidRequestError("Can't use yield_per in conjunction with unique")` for
`selectinload` without a nested collection `joinedload` — because `.unique()`
is no longer called in that path. `immediateload` is unaffected (it still calls
`.unique()` unconditionally).
Fixes: #13339Closes: #13341
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13341
Pull-request-sha: 980af0383b
Change-Id: I12e993f104d354ae894f81f9b09fbcf9a95b0027
Fixes#13336.
`SelectState._get_display_froms` recomputed a loop-invariant `_cloned_intersection(...)` once per FROM element in each of the three correlation comprehensions, making each branch O(N²) in the number of FROM elements. This hoists the call so it runs once, which is O(N).
`_cloned_intersection` / `_cloned_difference` are pure and return a set, and neither argument changes during the comprehension, so the result is identical. A function-level benchmark asserts `old == new` at every N (full numbers in #13336), and `test/sql/` plus the ORM compilation/query tests pass: 7442 passed, 359 skipped. Net -14 lines.
Per the issue discussion, no changelog entry is included.
### Checklist
This pull request is:
- [x] A short code fix
- Issue with a runnable demonstration: #13336
- Behavior-preserving (no logic change), so it is covered by the existing `test/sql/` and ORM compilation/query suites rather than adding new tests.
Closes: #13337
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13337
Pull-request-sha: beba43e77a
Change-Id: I95b02ffa66ef709c16bf5275924ec03244c19ecb
Fixed issue where :meth:`.Result.freeze` would lose track of ambiguous
column names present in the original :class:`.CursorResult`, causing
key-based access on the thawed result to silently return a value instead of
raising :class:`.InvalidRequestError`. The
:class:`.SimpleResultMetaData` now accepts and propagates ambiguous key
information so that frozen, thawed, and pickled results raise consistently
for duplicate column names. Pull request courtesy Saurabh Kohli.
Fixes: #9427Closes: #13335
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13335
Pull-request-sha: c03904ece2
Change-Id: Ia184f77b442b069e6f9a4f94a967ead41a1704b6
Fixed regression caused by 🎫`8068` where a ``backref``
named ``'metadata'`` on a mapped class would cause an
``AssertionError`` when the class also used string-based
relationship references (e.g. ``secondary="some_table"``).
The ``_metadata_for_cls()`` helper now checks
``isinstance(meta, MetaData)`` as a condition rather than
asserting, falling back to ``registry.metadata`` when the
class attribute has been overwritten by a backref.
A warning is now emitted when a Declarative attribute name is named
``metadata`` or ``registry``. Previously, no warning was emitted for
``registry``, and using the name ``metadata`` would raise an
InvalidRequestError. Since these names can be used for attributes
that are mapped as backrefs or using imperative mappings, usage
under Declarative has been relaxed for ``metadata`` but also warns
for both names as they may have unintended interactions with the
Declarative reserved names.
References: https://bugs.launchpad.net/nova/+bug/2154165
References: https://github.com/sqlalchemy/sqlalchemy/discussions/8619Fixes: #13333
Change-Id: I0f3173bce9c8c8881bd6b8ea75102bb8ec92be8b
Fixed issue where :class:`_sql.StatementLambdaElement` would proxy
attribute access through the cached "expected" expression rather than the
resolved expression, causing stale closure-bound parameter values to be
used when a lambda statement was extended with non-lambda criteria such as
an additional ``.where()`` clause. Courtesy cjc0013.
Fixes: #10827Closes: #13327
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13327
Pull-request-sha: ec3e6735bf
Change-Id: I8a32c11f3da63109cf37c39541df8ebfee52b8c5
Fixed issue where :func:`_orm.subqueryload` combined with
:meth:`.PropComparator.of_type` and :meth:`.PropComparator.and_` would
silently drop the additional filter criteria, causing all related objects
to be loaded instead of only those matching the filter. The
:class:`.LoaderCriteriaOption` was being constructed against the base
entity rather than the effective entity indicated by
:meth:`.PropComparator.of_type`. Pull request courtesy Arya Rizky.
Fixes: #13207Closes: #13290
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13290
Pull-request-sha: b7a8617cde
Change-Id: I2c24652ec112511deaf39dbb9d6197e2097904ed
The :func:`.selectinload` loader strategy now selects the ``omit_join``
optimization for many-to-many non-self-referential relationships, reducing
the number of joins in the secondary SELECT by selecting from the secondary
table directly rather than joining back to the parent entity. ``omit_join``
is enabled automatically when the join condition determines that the
secondary table's foreign keys fully cover the parent's primary key. As
always, ``omit_join`` can be disabled by setting
:paramref:`.relationship.omit_join` to ``False``. Pull request courtesy
bekapono.
Fixes: #5987Closes: #13278
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13278
Pull-request-sha: fdd9847e7d
Change-Id: Ib68f8e2be1399222383cdd7b55793fe88402212c
Fixes#9256.
This updates the annotations for Session.bulk_insert_mappings() and Session.bulk_update_mappings().
The docstrings and runtime behavior already allow either a mapped class or a Mapper object, but the previous annotations only accepted Mapper[Any].
This patch switches those arguments to the existing _EntityBindKey alias, which matches the inputs accepted by _class_to_mapper(): mapped classes and Mapper objects, but not AliasedClass or AliasedInsp.
I also updated the internal _bulk_save_mappings() annotation so the public methods and the private helper stay consistent. The scoped_session proxy output has been kept in sync with tools/generate_proxy_methods.py, and the generator check passes.
I added a typing regression test covering both mapped classes and Mapper objects for the two bulk mapping methods. I confirmed that the mapped-class cases fail with the old annotation and pass with this change.
Checked locally:
python -m pytest -m mypy test/typing/test_mypy.py -k "session.py" -q
python -m mypy ./lib/sqlalchemy
python tools/generate_proxy_methods.py --check
python -m pytest test/orm/dml/test_bulk.py -q
python -m pytest -m mypy test/typing/test_mypy.py -k "not typed_queries.py" -q
I could not run the full typing suite locally because my local Python 3.12 environment does not include string.templatelib. I only skipped typed_queries.py; that file is expected to be covered by SQLAlchemy's Python 3.14 mypy CI job.
Closes: #13322
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/13322
Pull-request-sha: bb730f3427
Change-Id: I4c5d516b3933b4e7fae9c844881a61f557a8bb5e
Mike Bayer