### Description
Follows initial attempt in #10450 - but starts with simpler association examples.
### Checklist
This pull request is:
- [x] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
I was curious how to add these selectively to any of the type hint test suites, to prevent future drift, but didn't see anything too obvious.
Closes: #12031
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/12031
Pull-request-sha: dad6239370
Change-Id: Id5c2d65137c5e9d7e87778acd51b965c2bcf315a
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
When reading more of the Generic Associations, I found that the examples state "Annotated example", and wondered what that meant, since in other parts of the docs that means they are using the new 2.0 style.
I tried to update this example to be more in line with the new style, including a little f-string update.
I completely understand this is unlikely to be merged as-is - but wanted to understand more about the right way to use modern styles to properly code well-hinted, more "exotic" implementations.
Outstanding questions:
- Should examples pass pass `mypy --strict` ? It doesn't right now.
- Are there better ways to apply `Mapped` within the `type(...)` definition, so we could skip importing `Integer`?
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
(Do non-running examples count as code fixes/features?)
- [x] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
**Have a nice day!**
Closes: #10450
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10450
Pull-request-sha: a6a54d6948
Change-Id: Ie7c7526daed5bf907d41f666459759b9a986b8c2
Added the ability to create custom SQL constructs that can define new
clauses within SELECT, INSERT, UPDATE, and DELETE statements without
needing to modify the construction or compilation code of of
:class:`.Select`, :class:`.Insert`, :class:`.Update`, or :class:`.Delete`
directly. Support for testing these constructs, including caching support,
is present along with an example test suite. The use case for these
constructs is expected to be third party dialects for NewSQL or other novel
styles of database that introduce new clauses to these statements. A new
example suite is included which illustrates the ``QUALIFY`` SQL construct
used by several NewSQL databases which includes a cachable implementation
as well as a test suite.
Since these extensions start to make it a bit crowded with how many
kinds of "options" we have on statements, did some naming /
documentation changes with existing constructs on Executable, in
particular to distinguish ExecutableOption from SyntaxExtension.
Fixes: #12195
Change-Id: I4a44ee5bbc3d8b1b640837680c09d25b1b7077af
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
I'm chasing some loose datetime.datetime.utcnow() deprecation warning in some test suites, and one of these was seemingly coming from sqlalchemy. It wasn't, but nevertheless these minor cleanup changes may still be found useful.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [x] A documentation / typographical / small typing error fix
- Good to go, no issue or tests are needed
**Have a nice day!**
Closes: #11736
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/11736
Pull-request-sha: 9bee8af8d1
Change-Id: Ib1b85fa3d66b665165d908e7c8394482b714c57f
Fixed issue in history_meta example where the "version" column in the
versioned table needs to default to the most recent version number in the
history table on INSERT, to suit the use case of a table where rows are
deleted, and can then be replaced by new rows that re-use the same primary
key identity. This fix adds an additonal SELECT query per INSERT in the
main table, which may be inefficient; for cases where primary keys are not
re-used, the default function may be omitted. Patch courtesy Philipp H.
v. Loewenfeld.
Fixes: #10267
Change-Id: I6b0737a7e871763f95fd636c9ad98b80f3b5808e
Fixed regression in history_meta example where the use of
:meth:`_schema.MetaData.to_metadata` to make a copy of the history table
would also copy indexes (which is a good thing), but causing naming
conflicts indexes regardless of naming scheme used for those indexes. A
"_history" suffix is now added to these indexes in the same way as is
achieved for the table name.
Fixes: #10920
Change-Id: I78823650956ff979d500bedbdbce261048894ce9
Fixed the performance example scripts in examples/performance to mostly
work with the Oracle database, by adding the :class:`.Identity` construct
to all the tables and allowing primary generation to occur on this backend.
A few of the "raw DBAPI" cases still are not compatible with Oracle.
Change-Id: I7ce19645ea78736dddfda6f33b9356ad75dee68f
Reverting change from b3216486 that causes error
sqlalchemy.exc.MultipleResultsFound: Multiple rows were found when exactly one was required
Change-Id: I9b249ff6e8e336f9c224e670359b4df99bc06092
changed gather orm example to avoid passing the same session to the
function called in the gather. While this is not an issue since no
sql is executed on it, it's still confusing for some users
References: #9312
Change-Id: I7f86e9c1640649ac13fd459b761d67718d4c3dc0
Initial fixes to test to accommodate py312
Here we are pulling in a current fork of greenlet that works
for python 3.12. It works though there is one issue with aiosqlite
that might be related.
a non-voting py312 job is added to gerrit / jenkins
Fixes: #9819
Change-Id: I91a51dcbad2902f7c4c7cec88ebbf42c2417b512
fix a handful of warnings that were emitting but not raising,
usually because they were inside an "expect_warnings" block.
modify "expect_warnings" to always use "raise_on_any_unexpected"
behavior; remove this parameter.
Fixed issue in semi-private ``await_only()`` and ``await_fallback()``
concurrency functions where the given awaitable would remain un-awaited if
the function threw a ``GreenletError``, which could cause "was not awaited"
warnings later on if the program continued. In this case, the given
awaitable is now cancelled before the exception is thrown.
Change-Id: I33668c5e8c670454a3d879e559096fb873b57244
Added a new helper mixin :class:`_asyncio.AsyncAttrs` that seeks to improve
the use of lazy-loader and other expired or deferred ORM attributes with
asyncio, providing a simple attribute accessor that provides an ``await``
interface to any ORM attribute, whether or not it needs to emit SQL.
Change-Id: I1427b288dc28319c854372643066c491b9ee8dc0
References: #9731
Fixed issue in "versioned history" example where using a declarative base
that is derived from :class:`_orm.DeclarativeBase` would fail to be mapped.
Additionally, repaired the given test suite so that the documented
instructions for running the example using Python unittest now work again.
Change-Id: I164a5b8dbdd01e3d815eb356f7b7cadf226ca296
References: #9546
This adds the very small plugin flake8-import-single which
will prevent us from having an import with more than one symbol
on a line.
Flake8 by itself prevents this pattern with E401:
import collections, os, sys
However does not do anything with this:
from sqlalchemy import Column, text
Both statements have the same issues generating merge artifacts
as well as presenting a manual decision to be made. While
zimports generally cleans up such imports at the top level, we
don't enforce zimports / pre-commit use.
the plugin finds the same issue for imports that are inside of
test methods. We shouldn't usually have imports in test methods
so most of them here are moved to be top level.
The version is pinned at 0.1.5; the project seems to have no
activity since 2019, however there are three 0.1.6dev releases
on pypi which stopped in September 2019, they seem to be
experiments with packaging. The source for 0.1.5
is extremely simple and only reveals one method to flake8
(the run() method).
Change-Id: Icea894e43bad9c0b5d4feb5f49c6c666d6ea6aa1
first change:
Reworked the :ref:`examples_versioned_history` to work with
version 2.0, while at the same time improving the overall working of
this example to use newer APIs, including a newly added hook
:meth:`_orm.MapperEvents.after_mapper_constructed`.
second change:
Added new event hook :meth:`_orm.MapperEvents.after_mapper_constructed`,
which supplies an event hook to take place right as the
:class:`_orm.Mapper` object has been fully constructed, but before the
:meth:`_orm.registry.configure` call has been called. This allows code that
can create additional mappings and table structures based on the initial
configuration of a :class:`_orm.Mapper`, which also integrates within
Declarative configuration. Previously, when using Declarative, where the
:class:`_orm.Mapper` object is created within the class creation process,
there was no documented means of running code at this point. The change
is to immediately benefit custom mapping schemes such as that
of the :ref:`examples_versioned_history` example, which generate additional
mappers and tables in response to the creation of mapped classes.
third change:
The infrequently used :attr:`_orm.Mapper.iterate_properties` attribute and
:meth:`_orm.Mapper.get_property` method, which are primarily used
internally, no longer implicitly invoke the :meth:`_orm.registry.configure`
process. Public access to these methods is extremely rare and the only
benefit to having :meth:`_orm.registry.configure` would have been allowing
"backref" properties be present in these collections. In order to support
the new :meth:`_orm.MapperEvents.after_mapper_constructed` event, iteration
and access to the internal :class:`_orm.MapperProperty` objects is now
possible without triggering an implicit configure of the mapper itself.
The more-public facing route to iteration of all mapper attributes, the
:attr:`_orm.Mapper.attrs` collection and similar, will still implicitly
invoke the :meth:`_orm.registry.configure` step thus making backref
attributes available.
In all cases, the :meth:`_orm.registry.configure` is always available to
be called directly.
fourth change:
Fixed obscure ORM inheritance issue caused by 🎫`8705` where some
scenarios of inheriting mappers that indicated groups of columns from the
local table and the inheriting table together under a
:func:`_orm.column_property` would nonetheless warn that properties of the
same name were being combined implicitly.
Fixes: #9220Fixes: #9232
Change-Id: Id335b8e8071c8ea509c057c389df9dcd2059437d
Added new option to horizontal sharding API
:class:`_horizontal.set_shard_id` which sets the effective shard identifier
to query against, for both the primary query as well as for all secondary
loaders including relationship eager loaders as well as relationship and
column lazy loaders.
Modernize sharding examples with new-style mappings, add new asyncio example.
Fixes: #7226Fixes: #7028
Change-Id: Ie69248060c305e8de04f75a529949777944ad511
Allow do_orm_execute() events to both receive the complete
state of bind_argments, load_options, update_delete_options
as they do already, but also allow them to *change* all those
things via new execution options. Options like autoflush,
populate_existing etc. can now be updated within a
do_orm_execute() hook and those changes will take effect
all the way through.
Took a few tries to get something that covers every case here,
in particular horizontal sharding which is consuming those
options as well as using context.invoke(), without excess
complexity. The good news seems to be that a simple
reorg and replacing the "reentrant" boolean with
"is this before do_orm_execute is invoked" was all that was
needed.
As part of this we add a new "identity_token" option allowing
this option to be controlled from do_orm_execute() as well
as from the outside.
WIP
Fixes: #7837
Change-Id: I087728215edec8d1b1712322ab389e3f52ff76ba
command run is "pyupgrade --py37-plus --keep-runtime-typing --keep-percent-format <files...>"
pyupgrade will change assert_ to assertTrue. That was reverted since assertTrue does not
exists in sqlalchemy fixtures
Change-Id: Ie1ed2675c7b11d893d78e028aad0d1576baebb55
* requirements needs typing_extensions
* update all "future=True" references to be appropriate to 2.0
Change-Id: I2eeb0ae65afdb587f21aeb0020f1d8a292f67c21
For consistency with the prominent ORM concept :class:`_orm.Mapped`, the
names of the dictionary-oriented collections,
:func:`_orm.attribute_mapped_collection`,
:func:`_orm.column_mapped_collection`, and :class:`_orm.MappedCollection`,
are changed to :func:`_orm.attribute_keyed_dict`,
:func:`_orm.column_keyed_dict` and :class:`_orm.KeyFuncDict`, using the
phrase "dict" to minimize any confusion against the term "mapped". The old
names will remain indefinitely with no schedule for removal.
Docs here are also updated for typing as we can type
these collections as ``Mapped[dict[str, cls]]``, don't need
KeyFuncDict / MappedCollection for these
Fixes: #8608
Change-Id: Ib5cf63e0aef1c389e023a75e454bb21f9d779b54
reviewers: these docs publish periodically at:
https://docs.sqlalchemy.org/en/gerrit/4042/orm/queryguide/index.html
See the "last generated" timestamp near the bottom of the
page to ensure the latest version is up
Change includes some other adjustments:
* small typing fixes for end-user benefit
* removal of a bunch of old examples for patterns that nobody
uses or aren't really what we promote now
* modernization of some examples, including inheritance
Change-Id: I9929daab7797be9515f71c888b28af1209e789ff
the feature is enabled for all built in backends
when RETURNING is used,
except for Oracle that doesn't need it, and on
psycopg2 and mssql+pyodbc it is used for all INSERT statements,
not just those that use RETURNING.
third party dialects would need to opt in to the new feature
by setting use_insertmanyvalues to True.
Also adds dialect-level guards against using returning
with executemany where we dont have an implementation to
suit it. execute single w/ returning still defers to the
server without us checking.
Fixes: #6047Fixes: #7907
Change-Id: I3936d3c00003f02e322f2e43fb949d0e6e568304
This is so that custom Session and AsyncSession classes
can be typed for these factories. Added appropriate
typevars to `__call__()`, `__enter__()` and other methods
so that a custom Session or AsyncSession subclass is carried
through.
Fixes: #7656
Change-Id: Ia2b8c1f22b4410db26005c3285f6ba3d13d7f0e0
in this patch the asyncio/events.py module, which
existed only to raise errors when trying to attach event
listeners, is removed, as we were already coding an asyncio-specific
workaround in upstream Pool / Session to raise this error,
just moved the error out to the target and did the same thing
for Engine.
We also add an async_sessionmaker class. The initial rationale
here is because sessionmaker() is hardcoded to Session subclasses,
and there's not a way to get the use case of
sessionmaker(class_=AsyncSession) to type correctly without changing
the sessionmaker() symbol itself to be a function and not a class,
which gets too complicated for what this is. Additionally,
_SessionClassMethods has only three methods on it, one of which
is not usable with asyncio (close_all()), the others
not generally used from the session class.
Change-Id: I064a5fa5d91cc8d5bbe9597437536e37b4e801fe
both black and click were released in the past
few hours, and black 21.5b1 seems to suddenly
be failing on a missing symbol from click. just
update to the latest
Change-Id: Idf76732479a264f7f2245699a6bdaff018e3a123
the horizontal sharding API needs some work as it is
still exposing some legacy details, but in any case illustrate
how we can, for the moment, to use multiple schema translate
maps in a single session.
A lot more cleanup is needed in horizontal sharding, see #7837
Change-Id: Ia925e2226ecee9d747a8c4fc1772917f10bc505f
References: #7832
References: #7837
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
There's a few bits and pieces of code to support Python <= 3.6 which are no longer needed and can be removed, to slightly simplify the codebase.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Closes: #7544
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7544
Pull-request-sha: 282b4a9128
Change-Id: I9ddf15fcf72551d52e3f027f337c7fee4aa9083b
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
Black's `target-version` was still set to `['py27', 'py36']`. Set it to `[py37]` instead.
Also update Black and other pre-commit hooks and re-format with Black.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Closes: #7536
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7536
Pull-request-sha: b3aedf5570
Change-Id: I8be85636fd2c9449b07a8626050c8bd35bd119d5
This patch adds new warnings for all elements that
don't indicate their caching behavior, including user-defined
ClauseElement subclasses and third party dialects.
it additionally adds new documentation to discuss an apparent
performance degradation in 1.4 when caching is disabled as a
result in the significant expense incurred by ORM
lazy loaders, which in 1.3 used BakedQuery so were actually
cached.
As a result of adding the warnings, a fair degree of
lesser used SQL expression objects identified that they did not
define caching behavior so would have been producing
``[no key]``, including PostgreSQL constructs ``hstore``
and ``array``. These have been amended to use inherit
cache where appropriate. "on conflict" constructs in
PostgreSQL, MySQL, SQLite still explicitly don't generate
a cache key at this time.
The change also adds a test for all constructs via
assert_compile() to assert they will not generate cache
warnings.
Fixes: #7394
Change-Id: I85958affbb99bfad0f5efa21bc8f2a95e7e46981
Mike Fiedler