Commit Graph

15018 Commits

Author SHA1 Message Date
Mike Bayer 428ea01f00 read from cls.__dict__ so init_subclass works
Modified the :class:`.DeclarativeMeta` metaclass to pass ``cls.__dict__``
into the declarative scanning process to look for attributes, rather than
the separate dictionary passed to the type's ``__init__()`` method. This
allows user-defined base classes that add attributes within an
``__init_subclass__()`` to work as expected, as ``__init_subclass__()`` can
only affect the ``cls.__dict__`` itself and not the other dictionary. This
is technically a regression from 1.3 where ``__dict__`` was being used.

Additionally makes the reference between ClassManager and
the declarative configuration object a weak reference, so that it
can be discarded after mappers are set up.

Fixes: #7900
Change-Id: I3c2fd4e227cc1891aa4bb3d7d5b43d5686f9f27c
2022-04-12 15:10:32 -04:00
Mike Bayer a45e2284da pep-484: asyncio
in this patch the asyncio/events.py module, which
existed only to raise errors when trying to attach event
listeners, is removed, as we were already coding an asyncio-specific
workaround in upstream Pool / Session to raise this error,
just moved the error out to the target and did the same thing
for Engine.

We also add an async_sessionmaker class.  The initial rationale
here is because sessionmaker() is hardcoded to Session subclasses,
and there's not a way to get the use case of
sessionmaker(class_=AsyncSession) to type correctly without changing
the sessionmaker() symbol itself to be a function and not a class,
which gets too complicated for what this is. Additionally,
_SessionClassMethods has only three methods on it, one of which
is not usable with asyncio (close_all()), the others
not generally used from the session class.

Change-Id: I064a5fa5d91cc8d5bbe9597437536e37b4e801fe
2022-04-11 22:11:07 -04:00
Mike Bayer aa9cd878e8 pep-484: session, instancestate, etc
Also adds some fixes to annotation-based mapping
that have come up, as well as starts to add more
pep-484 test cases

Change-Id: Ia722bbbc7967a11b23b66c8084eb61df9d233fee
2022-04-12 02:09:50 +00:00
Mike Bayer 98eae4e181 use code generation for scoped_session
our decorator thing generates code in any case,
so point it at the file itself to generate real code
for the blocks rather than doing things dynamically.

this will allow typing tools to have no problem
whatsoever and we also reduce import time overhead.
file size will be a lot bigger though, shrugs.

syntax / dupe method / etc. checking will be accomplished
by our existing linting / typing / formatting tools.

As we are also using "from __future__ import annotations",
we also no longer have to apply quotes to generated
annotations.

Change-Id: I20962cb65bda63ff0fb67357ab346e9b1ef4f108
2022-04-12 02:09:42 +00:00
Mike Bayer 15ef11e0ed level up pep-484 on engine result
Have each result subclass be generic to the
kind of row/object it returns.  rework things so that
a significant number of "type ignores" can go away
and also allow asyncio to more cleanly proxy
the result objects.

Change-Id: Ia3ddb2cb52f5856839bd8c9c46f0289ab4b10405
2022-04-12 02:09:36 +00:00
Mike Bayer 8254e3a28a repair ancient and incorrect comment
it referred towards _columntoproperty refering to
lists of MapperProperty.  this comment goes all the
way to pre 0.1 being released.   it's likely been
wrong for nearly all that time.

Change-Id: I71234ae58a6253249d92224356e38372e4aff148
2022-04-11 22:05:42 -04:00
Federico Caselli 11bf824474 update flake8 noqa skips with proper syntax
Change-Id: I42ed77f559e3ee5b8c600d98457ee37803ef0ea6
2022-04-11 22:48:23 +02:00
Mike Bayer fd05273260 simplify coverage config
I *think* we don't need to tell .coverage to look in lib,
it seems to be covering from whereever successfuly.  not
sure, but want to run on jenkins to see how it comes out.

Change-Id: Ib74f427a5b09fcbeba0188dcf02dab27c329a8cd
2022-04-09 11:32:50 -04:00
Mike Bayer 508f4e0c36 add mypy coverage task
get mypy plugin to be able to have coverage

Change-Id: I8c2d57bb97bd522b83d20a0f97576c7ae9ea71a5
2022-04-09 11:15:29 -04:00
mike bayer c2aa65e70e Merge "add sane_rowcount to SimpleUpdateDeleteTest" into main 2022-04-09 14:21:28 +00:00
Mike Bayer 1eeded1687 update DDL examples removing execute_if
Fixes: #7924
Change-Id: I684bf3720010ffe34dbdd39ec8c7e0c4af94e620
2022-04-09 10:02:35 -04:00
Mike Bayer f3a65eb903 add sane_rowcount to SimpleUpdateDeleteTest
For third party dialects, repaired a missing requirement for the
``SimpleUpdateDeleteTest`` suite test which was not checking for a working
"rowcount" function on the target dialect.

Fixes: #7919
Change-Id: I2bc68132131eb36c43b8dabec0fac86272e26df5
2022-04-09 09:50:48 -04:00
Mike Bayer 952383f9ee implement iso date literals for all backends
Added modified ISO-8601 rendering (i.e. ISO-8601 with the T converted to a
space) when using ``literal_binds`` with the SQL compilers provided by the
PostgreSQL, MySQL, MariaDB, MSSQL, Oracle dialects. For Oracle, the ISO
format is wrapped inside of an appropriate TO_DATE() function call.
Previously this rendering was not implemented for dialect-specific
compilation.

Fixes: #5052
Change-Id: I7af15a51fedf5c5a8e76e645f7c3be997ece35f0
2022-04-08 14:13:51 -04:00
Mike Bayer 4acf50c8e8 i forgot about begin_nested()
make bulleted lists of what is autoflush and what's not.

Change-Id: Id3bc4714013e9df243d804d7b5b60c6ef75e1316
2022-04-08 10:56:54 -04:00
Mike Bayer ed2b29dc34 clarify autoflush setting does not apply to commit
Change-Id: Icad0f3bd071422b8d1af204c9a1193a9ce6124ba
References: #7916
2022-04-08 10:36:50 -04:00
Mike Bayer 9b32d9040a update comment re: #2703
Change-Id: Ia78db7601e98acd2562872e0fda639aac99650dd
2022-04-08 09:48:26 -04:00
Mike Bayer f7ebfa6072 explicitly refer to Apple M1 for greenlet issue
As developers are now buying lots of Apple M1
machines, and AFAWK greenlet is still not able
to provide a pre-built wheel, we are going to get a lot
of devs trying to use asyncio on their Apple M1s, in
greater proportions compared to devs running containers
etc. on other various less popular CPU architectures.

Add a new FAQ section for installation, add new
red dragon to the very top of asyncio docs,
add new verbiage, all of which includes
"Apple M1" in bold text, to minimize the chance of anyone
missing this.

Fixes: #7922
Fixes: #7714
Change-Id: I106923a2860a4efd77d1b999197be102afc1f73d
2022-04-08 09:28:47 -04:00
mike bayer 735792d756 Merge "cx_Oracle modernize" into main 2022-04-07 15:19:10 +00:00
Mike Bayer 2acc9ec128 cx_Oracle modernize
Full "RETURNING" support is implemented for the cx_Oracle dialect, meaning
multiple RETURNING rows are now recived for DML statements that produce
more than one row for RETURNING.

cx_Oracle 7 is now the minimum version for cx_Oracle.

Getting Oracle to do multirow returning took about 5 minutes.  however,
getting Oracle's RETURNING system to integrate with ORM-enabled
insert, update, delete, is a big deal because that architecture wasn't
really working very robustly, including some recent changes in 1.4
for FromStatement were done in a hurry, so this patch also cleans up
the FromStatement situation and begins to establish it more concretely
as the base for all ReturnsRows / TextClause ORM scenarios.

Fixes: #6245
Change-Id: I2b4e6007affa51ce311d2d5baa3917f356ab961f
2022-04-07 10:47:53 -04:00
Mike Bayer 9edcdd03e8 cherry-pick changelog update for 1.4.36 2022-04-06 11:34:25 -04:00
Mike Bayer 975bcecb74 cherry-pick changelog from 1.4.35 2022-04-06 11:34:24 -04:00
Mike Bayer 31ccec00da classify issue
Change-Id: Id1e59fee382d1d72d78e99b1c56a5837b5503fed
2022-04-06 11:17:05 -04:00
Mike Bayer 2168a64aff maintain complete cloned_set for BindParameter
Fixed regression caused by 🎫`7823` which impacted the caching
system, such that bound parameters that had been "cloned" within ORM
operations, such as polymorphic loading, would in some cases not acquire
their correct execution-time value leading to incorrect bind values being
rendered.

Fixes: #7903
Change-Id: I61c802749b859bebeb127d24e66d6e77d13ce57a
2022-04-06 10:01:43 -04:00
Mike Bayer 7935b76d9e fix typo in lambda example
we might have an actual customer so clean up shop

Change-Id: I0b1e36ad78f364805a3a7bfd6fac953cf94b838f
2022-04-05 14:39:32 -04:00
Mike Bayer 3b4d62f4f7 pep484 - sql.selectable
the pep484 task becomes more intense as there is mounting
pressure to come up with a consistency in how data moves
from end-user to instance variable.

current thinking is coming into:

1. there are _typing._XYZArgument objects that represent "what the
   user sent"
2. there's the roles, which represent a kind of "filter" for different
   kinds of objects.   These are mostly important as the argument
   we pass to coerce().
3. there's the thing that coerce() returns, which should be what the
   construct uses as its internal representation of the thing.
   This is _typing._XYZElement.

but there's some controversy over whether or
not we should pass actual ClauseElements around by their role
or not.   I think we shouldn't at the moment, but this makes the
"role-ness" of something a little less portable. Like, we have
to set DMLTableRole for TableClause, Join, and Alias, but then
also we have to repeat those three types in order to set up
_DMLTableElement.

Other change introduced here, there was a deannotate=True
for the left/right of a sql.join().    All tests pass without that.
I'd rather not have that there as if we have a join(A, B) where
A, B are mapped classes, we want them inside of the _annotations.
The rationale seems to be performance, but this performance can
be illustrated to be on the compile side which we hope is cached
in the normal case.

CTEs now accommodate for text selects including recursive.

Get typing to accommodate "util.preloaded" cleanly; add "preloaded"
as a real module.  This seemed like we would have needed
pep562 `__getattr__()` but we don't, just set names in
globals() as we import them.

References: #6810
Change-Id: I34d17f617de2fe2c086fc556bd55748dc782faf0
2022-04-04 09:26:43 -04:00
mike bayer 4e603e2375 Merge "use .fromisoformat() for sqlite datetime, date, time parsing" into main 2022-04-03 19:20:32 +00:00
Mike Bayer ccadbec825 use .fromisoformat() for sqlite datetime, date, time parsing
SQLite datetime, date, and time datatypes now use Python standard lib
``fromisoformat()`` methods in order to parse incoming datetime, date, and
time string values. This improves performance vs. the previous regular
expression-based approach, and also automatically accommodates for datetime
and time formats that contain either a six-digit "microseconds" format or a
three-digit "milliseconds" format.

Fixes: #7029
Change-Id: I67aab4fe5ee3055e5996050cf4564981413cc221
2022-04-03 14:47:52 -04:00
Mike Bayer c315c7401a TableValuedAlias generation fixes
Fixed bug in newly implemented
:paramref:`.FunctionElement.table_valued.joins_implicitly` feature where
the parameter would not automatically propagate from the original
:class:`.TableValuedAlias` object to the secondary object produced when
calling upon :meth:`.TableValuedAlias.render_derived` or
:meth:`.TableValuedAlias.alias`.

Additionally repaired these issues in :class:`.TableValuedAlias`:

* repaired a potential memory issue which could occur when
repeatedly calling :meth:`.TableValuedAlias.render_derived` against
successive copies of the same object (for .alias(), we currently
have to still continue chaining from the previous element.  not sure
if this can be improved but this is standard behavior for .alias()
elsewhere)
* repaired issue where the individual element types would be lost when
calling upon :meth:`.TableValuedAlias.render_derived` or
:meth:`.TableValuedAlias.alias`.

Fixes: #7890
Change-Id: Ie5120c7ff1e5c1bba5aaf77c782a51c637860208
2022-04-03 11:30:09 -04:00
Mike Bayer 1dffb7cede clarify alternative mapping example
this second example is not part of the doctest steps,
clarify that it's not part of code examples to be present
in execution steps.   Add an extra registry +
declarative base on top
so that even if someone does run it, the Base will have
been reset and the examples will continue to work
(noting that column order in statements may change, but
probably nothing else).

Fixes: #7891
Change-Id: Icb1ba310230841e502185d9d0cadd3c18d467292
2022-04-03 10:52:30 -04:00
Maple 2ea12b2f5c fix small indentation typo (#7882) 2022-04-03 11:30:08 +02:00
Mike Bayer 990715adb4 cherry-pick changelog update for 1.4.35 2022-03-31 18:28:33 -04:00
Mike Bayer 9c4ba1a37c cherry-pick changelog from 1.4.34 2022-03-31 18:28:33 -04:00
Mike Bayer 44f5591326 allow executemany values for ON CONFLICT DO NOTHING
Scaled back a fix made for 🎫`6581` where "executemany values" mode
for psycopg2 were disabled for all "ON CONFLICT" styles of INSERT, to
not apply to the "ON CONFLICT DO NOTHING" clause, which does not include
any parameters and is safe for "executemany values" mode.  "ON CONFLICT
DO UPDATE" is still blocked from "executemany values" as there may
be additional parameters in the DO UPDATE clause that cannot be batched
(which is the original issue fixed by 🎫`6581`).

Fixes: #7880
Change-Id: Id3e23a0c6699333409a50148fa8923cb8e564bdc
2022-03-31 17:01:40 -04:00
Mike Bayer cbe38dbc66 add template methods for ORMInsert
Fixed regression caused by 🎫`7861` where invoking an
:class:`.Insert` construct which contained ORM entities via
:meth:`_orm.Session.execute` would fail.

Fixes: #7878
Change-Id: Icc4d8028249cc417f504fdd3e31e206b5bbc89f8
2022-03-31 14:58:13 -04:00
Mike Bayer 00659c72c0 cherry-pick changelog update for 1.4.34 2022-03-31 10:38:05 -04:00
Mike Bayer edfa54147a cherry-pick changelog from 1.4.33 2022-03-31 10:38:05 -04:00
mike bayer 575cb63520 Merge "add close=False parameter to engine.dispose()" into main 2022-03-31 14:05:09 +00:00
Mike Bayer 169ab546a9 add close=False parameter to engine.dispose()
Added new parameter :paramref:`.Engine.dispose.close`, defaulting to True.
When False, the engine disposal does not touch the connections in the old
pool at all, simply dropping the pool and replacing it. This use case is so
that when the original pool is transferred from a parent process, the
parent process may continue to use those connections.

Fixes: #7877
Change-Id: I88b0808442381ba5e50674787cdb64f0e77d8b54
2022-03-31 09:19:46 -04:00
Federico Caselli 3ec009eefd Update bindparam cache key
The ``literal_execute`` parameter now takes part of the cache
generation of a bindparam, since it changes the sql string generated
by the compiler.
Previously the correct bind values were used, but the ``literal_execute``
would be ignored on subsequent executions of the same query.

Fixes: #7876
Change-Id: I6bf887f1a2fe31f9d0ab68f5b4ff315004d006b2
2022-03-30 23:36:18 +02:00
mike bayer 12b67ae683 Merge "Add links to database url section from URL docs" into main 2022-03-30 21:02:59 +00:00
mike bayer b7ca7ed1b1 Merge "pep-484: the pep-484ening, SQL part three" into main 2022-03-30 18:36:52 +00:00
Mike Bayer 4e754a8914 pep-484: the pep-484ening, SQL part three
hitting DML which is causing us to open up the
ColumnCollection structure a bit, as we do put anonymous
column expressions with None here.  However, we still want
Table /TableClause to have named column collections that
don't return None, so parametrize the "key" in this
collection also.

* rename some "immutable" elements to "readonly".  we change
  the contents of immutablecolumncollection underneath, so it's
  not "immutable"

Change-Id: I2593995a4e5c6eae874bed5bf76117198be8ae97
2022-03-30 14:04:52 -04:00
Mike Bayer 9731484f5c further changelog / doc fixes
Change-Id: I959f242272ff5147b7c1f721cf119d09309a8c57
2022-03-30 11:44:23 -04:00
Mike Bayer 85a4c7f3d5 dont use cyext for sphinx builds
if these files are stale, imports fail and the build
silently excludes docstrings.  also , typing / docstrings
are generally in the _py version of things, so just
don't include cyexts in doc builds.

Change-Id: Ibec927c7646d4b870a84a297d1b199ccecf3afef
2022-03-30 11:40:21 -04:00
Mike Bayer 731262b17e changelog fixes
Change-Id: I24e3de82d656d59719fd067bc81b2948e725b487
2022-03-30 10:52:29 -04:00
Federico Caselli 8550d182a8 Add links to database url section from URL docs
Change-Id: Ifa6dc14bf95762b9477560f485f43efa11af421b
2022-03-29 23:00:00 +02:00
Federico Caselli 9acd3dc6b7 Fix spacing on `bindparam` docs
Change-Id: I9daad5293a4887734f14e6e5de6014aca415453c
2022-03-29 22:51:43 +02:00
mike bayer 579422621b Merge "Remove "autoload" references in Table docs" into main 2022-03-29 20:04:29 +00:00
mike bayer 470ca6d3fb Merge "use annotated entity when adding secondary" into main 2022-03-29 17:56:29 +00:00
Gord Thompson c3e16d3e88 Remove "autoload" references in Table docs
Replace with "autoload_with" as appropriate.

Change-Id: I8bb2a7e40f116e8e5dea2b8e25d0c160bb6da94e
2022-03-29 17:54:03 +00:00