The string argument accepted as the first positional argument by the
:func:`.relationship` function when using the Declarative API is no longer
interpreted using the Python ``eval()`` function; instead, the name is dot
separated and the names are looked up directly in the name resolution
dictionary without treating the value as a Python expression. However,
passing a string argument to the other :func:`.relationship` parameters
that necessarily must accept Python expressions will still use ``eval()``;
the documentation has been clarified to ensure that there is no ambiguity
that this is in use.
Fixes: #5238
Change-Id: Id802f403190adfab0ca034afe2214ba10fd9cfbb
(cherry picked from commit 17e31604ae)
- Added pyproject.toml with black arguments
- Updated black version in precommit hook
- Reformatted the code
Fixes: #5100Closes: #5103
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5103
Pull-request-sha: 795fd5f896
Change-Id: I14eedbaa51fb531cbf90fcefe6a1e07c8a565625
(cherry picked from commit f49a3fc02c03e39b3d34da9dfba4fe46d71c4aff)
Added keyword arguments to the :meth:`.MutableList.sort` function so that a
key function as well as the "reverse" keyword argument can be provided.
Fixes: #5114
Change-Id: Iefb29e1ccadfad6ecba558ce575029307001b88e
(cherry picked from commit b98403f0c2)
Fixed bug in sqlalchemy.ext.serializer where a unique
:class:`.BindParameter` object could conflict with itself if it were
present in the mapping itself, as well as the filter condition of the
query, as one side would be used against the non-deserialized version and
the other side would use the deserialized version. Logic is added to
:class:`.BindParameter` similar to its "clone" method which will uniquify
the parameter name upon deserialize so that it doesn't conflict with its
original.
Fixes: #5086
Change-Id: Ie1edce137e92ac496c822831d038999be5d1fc2d
(cherry picked from commit 4b17d03064)
in trying to apply 2020 copyright to files, the pre-commit
hooks complain about random file issues.
- remove old corrections.py utility, this had something to do
with repairing refs in the sphinx docs
- run pre commit hooks on all files
- formatting adjustments to work around code formatting collisions
(long import lines that zimports can't rewrite correctly)
- for 1.3, update conf.py with formatted version
Change-Id: I260744866f69e902eb93665c7c728ee94d3371a2
(cherry picked from commit 748bf710fd)
Fixed a few test failures which would occur on Windows due to SQLite file
locking issues, as well as some timing issues in connection pool related
tests; pull request courtesy Federico Caselli.
Note the pool related issues were fixed by Mike in
I1a7162e67912d22c135fa517b687a073f8fd9151 but are being ticketed
here.
Fixes: #4946Closes: #5055
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5055
Pull-request-sha: 36925573af
Change-Id: Ic53ec82f5d588d0e26a2d033a17c6109900d7f63
(cherry picked from commit f461d710f9f848029603cf76f7c26725afdea11e)
Identified a performance issue in the system by which a join is constructed
based on a mapped relationship. The clause adaption system would be used
for the majority of join expressions including in the common case where no
adaptation is needed. The conditions under which this adaptation occur
have been refined so that average non-aliased joins along a simple
relationship without a "secondary" table use about 70% less function calls.
Change-Id: Ifbe04214576e5a9fac86ca80c1dc7145c27cd50a
(cherry picked from commit 7e9f273835)
Previously, uses_objects was specific to the SynonymAttribute;
generalize it so that it defaults to False for other DescriptorProps.
Immediate fix is against CompositeProperty.
Fixed regression introduced in 1.3.0 related to the association proxy
refactor in 🎫`4351` that prevented :func:`.composite` attributes
from working in terms of an association proxy that references them.
Add test coverage for association proxies that refer to Composite
attributes as endpoints.
Fixes: #5000
Change-Id: Iea6fb1bd3314d861a9bc22491b0ae1e6c5e6340d
(cherry picked from commit c2432d9d19)
The :class:`.BakedQuery` will not cache a query that was modified by a
:meth:`.QueryEvents.before_compile` event, so that compilation hooks that
may be applying ad-hoc modifications to queries will take effect on each
run. In particular this is helpful for events that modify queries used in
lazy loading as well as eager loading such as "select in" loading. In
order to re-enable caching for a query modified by this event, a new
flag ``bake_ok`` is added; see :ref:`baked_with_before_compile` for
details.
A longer term plan to provide a new form of SQL caching should solve this
kind of issue more comprehensively.
Fixes: #4947
Change-Id: I5823c4fa00e7b6d46a2e8461b02d8b16605a6ed0
(cherry picked from commit d6db28556b)
This is a very useful assertion which prevents unused variables
from being set up allows code to be more readable and sometimes
even more efficient. test suites seem to be where the most
problems are and there do not seem to be documentation examples
that are using this, or at least the linter is not taking effect
within rst blocks.
Change-Id: I2b3341d8dd14da34879d8425838e66a4b9f8e27d
(cherry picked from commit 190e0139e8)
Fixed bug where the :attr:`.Mapper.all_orm_descriptors` accessor would
return an entry for the :class:`.Mapper` itself under the declarative
``__mapper___`` key, when this is not a descriptor. The ``.is_attribute``
flag that's present on all :class:`.InspectionAttr` objects is now
consulted, which has also been modified to be ``True`` for an association
proxy, as it was erroneously set to False for this object.
Fixes: #4729
Change-Id: Ia02388cc25d004e32d337140b62a587f3e5a0b7b
(cherry picked from commit 79d07c9abc)
The test added for #4686 can raise for "B" missing which
is normal and should not fail the test. Also ensure mappers are
cleared to prevent subsequent tests elsewhere from being
affected.
Change-Id: I4c5791223e7fd21e04dcd095daa7d868e77dbd97
(cherry picked from commit 8b2eb2a2d0)
Fixed regression where new association proxy system was still not proxying
hybrid attributes when they made use of the ``@hybrid_property.expression``
decorator to return an alternate SQL expression, or when the hybrid
returned an arbitrary :class:`.PropComparator`, at the expression level.
This involved futher generalization of the heuristics used to detect the
type of object being proxied at the level of :class:`.QueryableAttribute`,
to better detect if the descriptor ultimately serves mapped classes or
column expressions.
Fixes: #4690
Change-Id: I5b5300661291c94a23de53bcf92d747701720aa1
(cherry picked from commit c785a528ea)
Applied the mapper "configure mutex" against the declarative class mapping
process, to guard against the race which can occur if mappers are used
while dynamic module import schemes are still in the process of configuring
mappers for related classes. This does not guard against all possible race
conditions, such as if the concurrent import has not yet encountered the
dependent classes as of yet, however it guards against as much as possible
within the SQLAlchemy declarative process.
Fixes: #4686
Change-Id: I0349036b8078bd42265ab40862cfbfe5bf9d5b44
(cherry picked from commit 5039c6f01d)
Fixed bug where using ``copy.copy()`` or ``copy.deepcopy()`` on
:class:`.MutableList` would cause the items within the list to be
duplicated, due to an inconsistency in how Python pickle and copy both make
use of ``__getstate__()`` and ``__setstate__()`` regarding lists. In order
to resolve, a ``__reduce_ex__`` method had to be added to
:class:`.MutableList`. In order to maintain backwards compatibility with
existing pickles based on ``__getstate__()``, the ``__setstate__()`` method
remains as well; the test suite asserts that pickles made against the old
version of the class can still be deserialized by the pickle module.
Also modified sqlalchemy.testing.util.picklers to return picklers all the way through
pickle.HIGHEST_PROTOCOL.
Fixes: #4603
Change-Id: I7f78b9cfb89d59a706248536c553dc5e1d987b88
Restored instance-level support for plain Python descriptors, e.g.
``@property`` objects, in conjunction with association proxies, in that if
the proxied object is not within ORM scope at all, it gets classified as
"ambiguous" but is proxed directly. For class level access, a basic class
level``__get__()`` now returns the
:class:`.AmbiguousAssociationProxyInstance` directly, rather than raising
its exception, which is the closest approximation to the previous behavior
that returned the :class:`.AssociationProxy` itself that's possible. Also
improved the stringification of these objects to be more descriptive of
current state.
Fixes: #4574
Change-Id: I787a22806b5530c146ae6ee66b588e5b191ae689
Fixed regression where an association proxy linked to a synonym would no
longer work, both at instance level and at class level.
Fixes: #4522
Change-Id: I949079229ef87c12736c362df35444f6e30c8038
Fixed a regression in 1.2 due to the introduction of baked queries for
relationship lazy loaders, where a race condition is created during the
generation of the "lazy clause" which occurs within a memoized attribute. If
two threads initialize the memoized attribute concurrently, the baked query
could be generated with bind parameter keys that are then replaced with new
keys by the next run, leading to a lazy load query that specifies the
related criteria as ``None``. The fix establishes that the parameter names
are fixed before the new clause and parameter objects are generated, so that
the names are the same every time.
Fixes: #4507
Change-Id: I605b824e028c87bc20ca8c2577227cdf6a591064
Implemented a more comprehensive assignment operation (e.g. "bulk replace")
when using association proxy with sets or dictionaries. Fixes the problem
of redundant proxy objects being created to replace the old ones, which
leads to excessive events and SQL and in the case of unique constraints
will cause the flush to fail.
Fixes: #2642
Change-Id: I57ab27dd9feba057e539267722cce92254fca777
in b8f9517cdd we disabled this new error
coming up, but it's just this one line in a test that is in fact wrong.
Change-Id: I4e3612596e9735c7db066832d13ccb454c93808f
Added some helper exceptions that invoke when a mapping based on
:class:`.AbstractConcreteBase`, :class:`.DeferredReflection`, or
:class:`.AutoMap` is used before the mapping is ready to be used, which
contain descriptive information on the class, rather than falling through
into other failure modes that are less informative.
Fixes: #4470
Change-Id: I9bc51697f63cedaa7809a0adb17b2398c209e289
Improved error messages emitted by the ORM in the area of loader option
traversal. This includes early detection of mis-matched loader strategies
along with a clearer explanation why these strategies don't match.
Fixes: #4433
Change-Id: I3351b64241f7f62ca141a0be95085e6ef8ca6d32
Implemented a new feature whereby the :class:`.AliasedClass` construct can
now be used as the target of a :func:`.relationship`. This allows the
concept of "non primary mappers" to no longer be necessary, as the
:class:`.AliasedClass` is much easier to configure and automatically inherits
all the relationships of the mapped class, as well as preserves the
ability for loader options to work normally.
- introduce new name for mapped_table, "persist_selectable". this is
the selectable that selects against the local mapper and its superclasses,
but does not include columns local only to subclasses.
- relationship gains "entity" which is the mapper or aliasedinsp.
- clarfiy name "entity" vs. "query_entity" in loader strategies.
Fixes: #4423Fixes: #4422Fixes: #4421Fixes: #3348
Change-Id: Ic3609b43dc4ed115006da9ad9189e574dc0c72d9
A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.
See the notes added to the changelog and migration notes for further
details.
Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
Fixed issue in association proxy due to 🎫`3423` which caused the use
of custom :class:`.PropComparator` objects with hybrid attribites, such as
the one demonstrated in the ``dictlike-polymorphic`` example to not
function within an association proxy. The strictness that was added in
🎫`3423` has been relaxed, and additional logic to accomodate for
an association proxy that links to a custom hybrid have been added.
Fixes: #4446
Change-Id: I8addc80f51094769915ac2dce1a301bd72ee7433
Added a new function :func:`.close_all_sessions` which takes
over the task of the :meth:`.Session.close_all` method, which
is now deprecated as this is confusing as a classmethod.
Pull request courtesy Augustin Trancart.
Fixes: #4412Closes: #4438
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4438
Pull-request-sha: 7833d12a98
Change-Id: Ib35eaa520ae886f3f8f550f9712fc3b139e00b60
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
Fixes to the test suite, a few errant imports, and setup.py:
- mysql and postgresql have unused 'json' imports; remove
- postgresql is exporting the 'json' symbol, remove
- make sure setup.py can find __version__ using " or '
- retry logic in provision create database for postgresql fixed
- refactor test_magazine to use cls.tables rather than globals
- remove unused class in test_scoping
- add a comment to test_deprecations that this test suite itself
is deprecated
- don't use mapper() and orm_mapper() in test_unitofwork, just
use mapper()
- remove dupe test_scalar_set_None test in test_attributes
- Python 2.7 and above includes unittest.SkipTest, remove pre-2.7
fallback
- use imported SkipTest in profiling
- declarative test_reflection tests with "reflectable_autoincrement"
already don't run on oracle or firebird; remove conditional logic
for these, which also removes an "id" symbol
- clean up test in test_functions, remove print statement
- remove dupe test_literal_processor_coercion_native_int_out_of_range
in test/sql/test_types.py
- fix psycopg2_hstore ref
Change-Id: I7b3444f8546aac82be81cd1e7b6d8b2ad6834fe6
Fixed a regression in 1.3.0b1 caused by 🎫`3423` where association
proxy objects that access an attribute that's only present on a polymorphic
subclass would raise an ``AttributeError`` even though the actual instance
being accessed was an instance of that subclass.
Fixes: #4401
Change-Id: Ie62c48aa9142adff45cbf9a297184987c72f30f3
since "left" is kind of ambiguous, use more explicit terminology
here.
Also update the test to use a positive assertion that the
warning is emitted; quote the attribute name.
Change-Id: Ic2284c200a26b32b2da063cfaf6d59547309d587
References: https://github.com/zzzeek/sqlalchemy/pull/488
Added a ``__clause_element__()`` method to :class:`.ColumnProperty` which
can allow the usage of a not-fully-declared column or deferred attribute in
a declarative mapped class slightly more friendly when it's used in a
constraint or other column-oriented scenario within the class declaration,
though this still can't work in open-ended expressions; prefer to call the
:attr:`.ColumnProperty.expression` attribute if receiving ``TypeError``.
Fixes: #4372
Change-Id: I5d3d1adb9c77de0566298bc2c46e9001d314b0c7
A warning is emitted in the case that a :func:`.column` object is applied to
a declarative class, as it seems likely this intended to be a
:class:`.Column` object.
Fixes: #4374
Change-Id: I2e617ef65547162e3ba6587c168548ad0cf6203d
Added missing ``.index()`` method to list-based association collections
in the association proxy extension.
Change-Id: Ice81dc4bcccd520638c5bc9a0f8bb2027946c846
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/485
The column conflict resolution technique discussed at
:ref:`declarative_column_conflicts` is now functional for a :class:`.Column`
that is also a primary key column. Previously, a check for primary key
columns declared on a single-inheritance subclass would occur before the
column copy were allowed to pass.
Fixes: #4352
Change-Id: Id4c025da53c28e58db6b549fe398f25f8a90d355
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/483
The :class:`.AssociationProxy` now has standard column comparison operations
such as :meth:`.ColumnOperators.like` and
:meth:`.ColumnOperators.startswith` available when the target attribute is a
plain column - the EXISTS expression that joins to the target table is
rendered as usual, but the column expression is then use within the WHERE
criteria of the EXISTS. Note that this alters the behavior of the
``.contains()`` method on the association proxy to make use of
:meth:`.ColumnOperators.contains` when used on a column-based attribute.
Fixes: #4351
Change-Id: I310941f4e8f778c200f8144a26a89e5364cd4dfb
Fixed regression caused by 🎫`4326` in version 1.2.12 where using
:class:`.declared_attr` with a mixin in conjunction with
:func:`.orm.synonym` would fail to map the synonym properly to an inherited
subclass.
Fixes: #4350
Change-Id: Ib2a9b6a125a2ac7c7ff80201746b7f10e5596226
The "selectin" loader strategy now omits the JOIN in the case of a
simple one-to-many load, where it instead relies upon the foreign key
columns of the related table in order to match up to primary keys in
the parent table. This optimization can be disabled by setting
the :paramref:`.relationship.omit_join` flag to False.
Many thanks to Jayson Reis for the efforts on this.
As part of this change, horizontal shard no longer relies upon
the _mapper_zero() method to get the query-bound mapper, instead
using the more generalized _bind_mapper() (which will use mapper_zero
if no explicit FROM is present). A short check for the particular
recursive condition is added to BundleEntity and it no longer assigns
itself as the "namespace" to its ColumnEntity objects which creates
a reference cycle.
Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Fixes: #4340
Change-Id: I649587e1c07b684ecd63f7d10054cd165891baf4
Pull-request: https://bitbucket.org/zzzeek/sqlalchemy/pull-requests/7
Added support for bulk :meth:`.Query.update` and :meth:`.Query.delete`
to the :class:`.ShardedQuery` class within the horiziontal sharding
extension. This also adds an additional expansion hook to the
bulk update/delete methods :meth:`.Query._execute_crud`.
Fixes: #4196
Change-Id: I65f56458176497a8cbdd368f41b879881f06348b
Considering the reversal of #597 as well as
84420a1d0f as I am unable to reproduce
the original issues from that release.
The long-standing behavior of the association proxy collection maintaining
only a weak reference to the parent object is reverted; the proxy will now
maintain a strong reference to the parent for as long as the proxy
collection itself is also in memory, eliminating the "stale association
proxy" error. This change is being made on an experimental basis to see if
any use cases arise where it causes side effects.
Change-Id: I051334be90a343dd0e8a1f35e072075eb14b14a7
Fixes: #4268
Reworked :class:`.AssociationProxy` to store state that's specific to a
parent class in a separate object, so that a single
:class:`.AssocationProxy` can serve for multiple parent classes, as is
intrinsic to inheritance, without any ambiguity in the state returned by it.
A new method :meth:`.AssociationProxy.for_class` is added to allow
inspection of class-specific state.
Change-Id: I634f88aae6306ac5c5237a0e1acbe07d0481d6b6
Fixes: #3423
Added new feature :meth:`.BakedQuery.to_query`, which allows for a
clean way of using one :class:`.BakedQuery` as a subquery inside of another
:class:`.BakedQuery` without needing to refer explicitly to a
:class:`.Session`.
Fixes: #4318
Change-Id: I528056c7d140036c27b95500d7a60dcd14784016
Fixed issue where :class:`.BakedQuery` did not include the specific query
class used by the :class:`.Session` as part of the cache key, leading to
incompatibilities when using custom query classes, in particular the
:class:`.ShardedQuery` which has some different argument signatures.
Fixes: #4328
Change-Id: I829c2a8b09c91e91c8dc8ea5476c0d7aa47028bd
Fixed bug where the declarative scan for attributes would receive the
expression proxy delivered by a hybrid attribute at the class level, and
not the hybrid attribute itself, when receiving the descriptor via the
``@declared_attr`` callable on a subclass of an already-mapped class. This
would lead to an attribute that did not report itself as a hybrid when
viewed within :attr:`.Mapper.all_orm_descriptors`.
Fixes: #4326
Change-Id: I582d03f05c3768b3344f93e3791240e9e69b9d1e
Fixed issue in previously untested use case, allowing a declarative mapped
class to inherit from a classically-mapped class outside of the declarative
base, including that it accommodates for unmapped intermediate classes. An
unmapped intermediate class may specify ``__abstract__``, which is now
interpreted correctly, or the intermediate class can remain unmarked, and
the classically mapped base class will be detected within the hierarchy
regardless. In order to anticipate existing scenarios which may be mixing
in classical mappings into existing declarative hierarchies, an error is
now raised if multiple mapped bases are detected for a given class.
Fixes: #4321
Change-Id: I8604ecfd170d2589d9d1b1c87ba303762071fc30
Fixed multiple issues regarding de-association of scalar objects with the
association proxy. ``del`` now works, and additionally a new flag
:paramref:`.AssociationProxy.cascade_scalar_deletes` is added, which when
set to True indicates that setting a scalar attribute to ``None`` or
deleting via ``del`` will also set the source association to ``None``.
Change-Id: I1580d761571d63eb03a7e8df078cef97d265b85c
Fixes: #4308
The sharding tests created named sqlite databases that were
shared across test suites. It is unknown why these suddenly
started failing and were not failing before.
Change-Id: If2044f914ddaea0db594aa18b9278e24e2c818ea
Mike Bayer