Adjustments made to the BLOB / CLOB / NCLOB datatypes in the cx_Oracle and
oracledb dialects, to improve performance based on recommendations from
Oracle developers.
References: https://github.com/oracle/python-cx_Oracle/issues/596Fixes: #7494
Change-Id: I0d8cc3579140aa65cacf5b7d3373f7e1929a8f85
ROLLBACK TO SAVEPOINT is re-enabled
in https://github.com/tlocke/pg8000/issues/111.
we still have to add savepoint support to our fixture that
deletes from tables without checking for them.
this is inconvenient but not incorrect.
Change-Id: I2f4a0a3e18db93c3e6794ade9b0fee33d2e4b7dc
Implement a new means of creating a mapped dataclass where
instead of applying the `@dataclass` decorator distinctly,
the declarative process itself can create the dataclass.
MapperProperty and MappedColumn objects themselves take
the place of the dataclasses.Field object when constructing
the class.
The overall approach is made possible at the typing level
using pep-681 dataclass transforms [1].
This new approach should be able to completely supersede the
previous "dataclasses" approach of embedding metadata into
Field() objects, which remains a mutually exclusive declarative
setup style (mixing them introduces new issues that are not worth
solving).
[1] https://peps.python.org/pep-0681/#transform-descriptor-types-exampleFixes: #7642
Change-Id: I6ba88a87c5df38270317b4faf085904d91c8a63c
Full "RETURNING" support is implemented for the cx_Oracle dialect, meaning
multiple RETURNING rows are now recived for DML statements that produce
more than one row for RETURNING.
cx_Oracle 7 is now the minimum version for cx_Oracle.
Getting Oracle to do multirow returning took about 5 minutes. however,
getting Oracle's RETURNING system to integrate with ORM-enabled
insert, update, delete, is a big deal because that architecture wasn't
really working very robustly, including some recent changes in 1.4
for FromStatement were done in a hurry, so this patch also cleans up
the FromStatement situation and begins to establish it more concretely
as the base for all ReturnsRows / TextClause ORM scenarios.
Fixes: #6245
Change-Id: I2b4e6007affa51ce311d2d5baa3917f356ab961f
asyncmy made a backwards incompatible change to the
TIME datatype in 0.2.4 which hopefully will be reverted
as this change does not belong in a minor point release
Change-Id: Id8bf932d70f52caba06094308a5b87387e69b94b
References: https://github.com/long2ice/asyncmy/issues/37
Fixed regression in mariadbconnector dialect as of mariadb connector 1.0.10
where the DBAPI no longer pre-buffers cursor.lastrowid. The dialect now
fetches this value proactively for situations where it applies.
test_invalidate_on_results seems to pass for mariadbconnector now.
the driver has likely changed how it buffers result sets. This is
a major change for them to make in a point release so we might
want to watch this in case they reverse course again.
Fixes: #7738
Change-Id: I9610aae01d1ae42fa92ffbc7123a6948e40ec9dd
__future__.annotations mode allows us to use non-string
annotations for argument and return types in most cases,
but more importantly it removes a large amount of runtime
overhead that would be spent in evaluating the annotations.
Change-Id: I2f5b6126fe0019713fc50001be3627b664019ede
References: #6810
large patch to get ORM / typing efforts started.
this is to support adding new test cases to mypy,
support dropping sqlalchemy2-stubs entirely from the
test suite, validate major ORM typing reorganization
to eliminate the need for the mypy plugin.
* New declarative approach which uses annotation
introspection, fixes: #7535
* Mapped[] is now at the base of all ORM constructs
that find themselves in classes, to support direct
typing without plugins
* Mypy plugin updated for new typing structures
* Mypy test suite broken out into "plugin" tests vs.
"plain" tests, and enhanced to better support test
structures where we assert that various objects are
introspected by the type checker as we expect.
as we go forward with typing, we will
add new use cases to "plain" where we can assert that
types are introspected as we expect.
* For typing support, users will be much more exposed to the
class names of things. Add these all to "sqlalchemy" import
space.
* Column(ForeignKey()) no longer needs to be `@declared_attr`
if the FK refers to a remote table
* composite() attributes mapped to a dataclass no longer
need to implement a `__composite_values__()` method
* with_variant() accepts multiple dialect names
Change-Id: I22797c0be73a8fbbd2d6f5e0c0b7258b17fe145d
Fixes: #7535Fixes: #7551
References: #6810
Starting to set up practices and conventions to
get the library typed.
Key goals for typing are:
1. whole library can pass mypy without any strict
turned on.
2. we can incrementally turn on some strict flags on a per-package/
module basis, as here we turn on more strictness for sqlalchemy.util, exc,
and log
3. mypy ORM plugin tests work fully without sqlalchemy2-stubs
installed
4. public facing methods all have return types, major parameter
signatures filled in also
5. Foundational elements like util etc. are typed enough so that
we can use them in fully typed internals higher up the stack.
Conventions set up here:
1. we can use lots of config in setup.cfg to limit where mypy
is throwing errors and how detailed it should be in different
packages / modules. We can use this to push up gerrits
that will pass tests fully without everything being typed.
2. a new tox target pep484 is added. this links to a new jenkins
pep484 job that works across all projects (alembic, dogpile, etc.)
We've worked around some mypy bugs that will likely
be around for awhile, and also set up some core practices
for how to deal with certain things such as public_factory
modules (mypy won't accept a module from a callable at all,
so need to use simple type checking conditionals).
References: #6810
Change-Id: I80be58029896a29fd9f491aa3215422a8b705e12
The warnings plugin lets us set the filters up
in the config, and as our filter requirements are now
simple we can just set this up.
additionally pytest now recommends pyproject.toml, since
we fully include this now, let's move it there.
the pytest logging plugin seems to not be any problem either
at the moment, so re-enable that. if it becomes apparent
whatever the problem was (which was probably that it was just
surprising, or something) we can disable it again and comment
what the reason was.
Change-Id: Ia9715533b01f72aa5fdcf6a27ce75b76f829fa43
Fixed issue in :meth:`_orm.Session.bulk_save_mappings` where the sorting
that takes place when the ``preserve_order`` parameter is set to False
would sort partially on ``Mapper`` objects, which is rejected in Python
3.11.
Also uses typing_extensions for NotRequired as this symbol
does not seem to be in Python 3.11.0a4 yet.
For interim 3.11 support, adds the git main build of greenlet
for Python 3.11
Fixes: #7591
Change-Id: I24a62f2322ad7dac5d8e4a00853f8a9408877c9c
introduces:
1. new mapped_column() helper
2. DeclarativeBase helper
3. declared_attr has been re-typed
4. rework of Mapped[] to return InstrumentedAtribute for
class get, so works without Mapped itself having expression
methods
5. ORM constructs now generic on [_T]
also includes some early typing work, most of which will
be in later commits:
1. URL and History become typing.NamedTuple
2. come up with type-checking friendly way of type
checking cy extensions, where type checking will be applied
to the py versions, just needed to come up with a succinct
conditional pattern for the imports
References: #6810
References: #7535
References: #7562
Change-Id: Ie5d9a44631626c021d130ca4ce395aba623c71fb
start applying foundational annotations to key
elements.
two main elements addressed here:
1. removal of public_factory() and replacement with
explicit functions. this just works much better with
typing.
2. typing support for column expressions and operators.
The biggest part of this involves stubbing out all the
ColumnOperators methods under ColumnElement in a
TYPE_CHECKING section. Took me a while to see this
method vs. much more complicated things I thought
I needed.
Also for this version implementing #7519, ColumnElement
types against the Python type and not TypeEngine. it is
hoped this leads to easier transferrence between ORM/Core
as well as eventual support for result set typing.
Not clear yet how well this approach will work and what
new issues it may introduce.
given the current approach we now get full, rich typing for
scenarios like this:
from sqlalchemy import column, Integer, String, Boolean
c1 = column('a', String)
c2 = column('a', Integer)
expr1 = c2.in_([1, 2, 3])
expr2 = c2 / 5
expr3 = -c2
expr4_a = ~(c2 == 5)
expr4_b = ~column('q', Boolean)
expr5 = c1 + 'x'
expr6 = c2 + 10
Fixes: #7519Fixes: #6810
Change-Id: I078d9f57955549f6f7868314287175f6c61c44cb
Good new is that pylance likes it and copies over the
singature and everything.
Bad news is that mypy does not support this yet https://github.com/python/mypy/issues/8645
Other minor bad news is that non_generative is not typed. I've tried using a protocol
like the one in the comment but the signature is not ported over by pylance, so it's
probably best to just live without it to have the correct signature.
notes from mike: these three decorators are at the core of getting
the library to be typed, more good news is that pylance will
do all the things we like re: public_factory, see
https://github.com/microsoft/pyright/issues/2758#issuecomment-1002788656
.
For @_generative, we will likely move to using pep 673 once mypy
supports it which may be soon. but overall having the explicit
"return self" in the methods, while a little inconvenient, makes
the typing more straightforward and locally present in the files
rather than being decided at a distance. having "return self"
present, or not, both have problems, so maybe we will be able
to change it again if things change as far as decorator support.
As it is, I feel like we are barely squeaking by with our decorators,
the typing is already pretty out there.
Change-Id: Ic77e13fc861def76a5925331df85c0aa48d77807
References: #6810
### Description
Found via `codespell -q 3 -L ba,crate,datas,froms,gord,hist,inh,nd,selectin,strat,ue`
Also added codespell to the pep8 tox env
### Checklist
This pull request is:
- [x] A documentation / typographical error fix
- Good to go, no issue or tests are needed
Closes: #7338
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7338
Pull-request-sha: 0deac22193
Change-Id: Icd61db31c8dc655d4a39d8a304194804d08555fe
pytest 7 is on rc, so in order to get it installed
when I run tox, bump min version to 7.0.0rc1. The performance
issues with version 6 are not tolerable.
Additionally remove all python_version<3.7 specs from setup.cfg,
tox, these are no longer needed.
Change-Id: Id5523bc920c562a9c2a59aba51a36dda3b11a079
Re-implement c version immutabledict / processors / resultproxy / utils with cython.
Performance is in general in par or better than the c version
Added a collection module that has cython version of OrderedSet and IdentitySet
Added a new test/perf file to compare the implementations.
Run ``python test/perf/compiled_extensions.py all`` to execute the comparison test.
See results here: https://docs.google.com/document/d/1nOcDGojHRtXEkuy4vNXcW_XOJd9gqKhSeALGG3kYr6A/edit?usp=sharingFixes: #7256
Change-Id: I2930ef1894b5048210384728118e586e813f6a76
Signed-off-by: Federico Caselli <cfederico87@gmail.com>
This is so that dialect methods that are called within init
can assume the same argument structure as when they are called
in other places; we can nail down the type of object as well.
This change seems to mostly impact the isolation level routines
in the dialects, as these are called during initialize()
as well as on established connections. these methods can now
assume a non-proxied DBAPI connection object in all cases,
as it is commonly required that attributes like ".autocommit"
are set on the object which don't work well in a proxied
situation.
Other changes:
* adds an interface for the "connectionfairy" concept
called PoolProxiedConnection.
* Removes ``Connectable`` superclass of Connection.
``Connectable`` was originally meant to provide for the
"method which accepts connection or engine" theme. As this
pattern is greatly reduced in 2.0 and Engine no longer extends
from it, the ``Connectable`` superclass doesnt serve any real
purpose.
Leading from that, to set this in I also applied pep 484 annotations
to the Dialect base, and then in the interests of seeing some
of the typing information show up in my IDE did a little bit for Engine,
Connection and others. I hope that it's feasible that we can
add annotations to specific classes and attributes ahead of when we
actually try to mass-populate the whole library. This was
the original spirit of pep-484 that we can apply annotations
gradually. I do of course want to try to do a mass-populate
although i think even in that case we will end up doing a lot
of manual work anyway (in particular for the changes here which
are distinct from what the stubs have).
Fixes: #7122
Change-Id: I5dd7fbff8a7ae520a81c165091af12a6a68826db
Fixes: #6960
Even though a default driver still exists for
each dialect, remove most usages of `dialect://`
to encourage users to explicitly specify
`dialect+driver://`
Change-Id: I0ad42167582df509138fca64996bbb53e379b1af
Repairs one in-library deprecation warning regarding
mapper propagation of options
raises maxfail to 250, as 25 is too low when we are trying
to address many errors at once. the 25 was originally
due to the fact that our fixtures would be broken after
that many failures in most cases, which today should not
be the case nearly as often.
Change-Id: I26affddf42e2cae2aaf9561633e9b8cd431eb189
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
Python 3.10 was released last week.
Test on Python 3.10 and add the Trove classifier.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [x] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
^ Which checklist option should I use?
Should I do something to trigger/test all the workflows?
Closes: #7179
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7179
Pull-request-sha: 5df7ff0e0d
Change-Id: Ib7ecc0de75f7dffdf05db13563f7be10898e67ba
Fixed regression due to #7024 where the reorganization of the "platform
machine" names for greenlet dependency mis-spelled "aarch64" and
additionally omitted uppercase "AMD64" as is needed for windows machines.
Pull request courtesy James Dow.
Fixes: #7024Closes: #7069
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7069
Pull-request-sha: dbd7419e21
Change-Id: Ia2793376a8fbb71d9b5b02fdc955dddbc0a0e80e
Added initial support for the ``asyncmy`` asyncio database driver for MySQL
and MariaDB. This driver is very new, however appears to be the only
current alternative to the ``aiomysql`` driver which currently appears to
be unmaintained and is not working with current Python versions. Much
thanks to long2ice for the pull request for this dialect.
Fixes: #6993Closes: #7000
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7000
Pull-request-sha: f7d6c811fc
Change-Id: I4ef54b43334feff7e3a710fc4de6821437f3bb68
Further adjusted the "greenlet" package specifier in setup.cfg to use a
long chain of "or" expressions, so that the comparison of
``platform_machine`` to a specific identifier matches only the complete
string.
Fixes: #7024
Change-Id: I1cd40f5e14c62c21bd2e530ca8e38a5a3d50d353
windows seems to be returning the name in uppercase,
we might have to revert this whole thing if things
continue to not work
Change-Id: I365a07e3521f9f43a3d9d5f93da7788a8a052ff0
The setup requirements have been modified such ``greenlet`` is a default
requirement only for those platforms that are well known for ``greenlet``
to be installable and for which there is already a pre-built binary on
pypi; the current list is ``x86_64 aarch64 ppc64le amd64 win32``. For other
platforms, greenlet will not install by default, which should enable
installation and test suite running of SQLAlchemy 1.4 on platforms that
don't support ``greenlet``, excluding any asyncio features. In order to
install with the ``greenlet`` dependency included on a machine architecture
outside of the above list, the ``[asyncio]`` extra may be included by
running ``pip install sqlalchemy[asyncio]`` which will then attempt to
install ``greenlet``.
Additionally, the test suite has been repaired so that tests can complete
fully when greenlet is not installed, with appropriate skips for
asyncio-related tests.
Fixes: #6136
Change-Id: I8f3a1c00a4a8b6a273484af1da1f7aaadf588ae7
Fixed issue where mypy plugin would not correctly interpret an explicit
:class:`_orm.Mapped` annotation in conjunction with a
:func:`_orm.relationship` that refers to a class by string name; the
correct annotation would be downgraded to a less specific one leading to
typing errors.
The thing figured out here is that after we've already scanned
a class in the semanal stage and created DeclClassApplied,
when we are called again with that same DeclClassApplied, for this
specific kind of case we actually now have *better* types than
we did before, where the left side that looked like
List?[Address?] now seems to say
builtins.list[official.module.Address] - so let's take the
right side expression again, this time embedded in our
Mapped._empty_constructor() expression, and run the infer
all over again just like mypy would. Just not setting the
"wrong" type here fixed the test cases but by re-applying the
whole infer we get the correct Mapped[] on the left side too.
Fixes: #6255
Change-Id: Iafe7254374f685a8458c7a1db82aafc2ed6d0232
Revised the fix for ``OrderingList`` from version 1.4.7 which was testing
against the incorrect API.
Fixes: #6205
Change-Id: I1d3f8b6534b70ae000294c2a67f08117cb77ee99
The ``pysqlcipher`` dialect now imports the ``sqlcipher3`` module
for Python 3 by default. Regressions have been repaired such that
the connection routine was not working.
To better support the post-connection steps of the pysqlcipher
dialect, a new hook Dialect.on_connect_url() is added, which
supersedes Dialect.on_connect() and is passed the URL object.
The dialect now pulls the passphrase and other cipher args
from the URL directly without including them in the
"connect" args. This will allow any user-defined extensibility
to connecting to work as it would for other dialects.
The commit also builds upon the extended routines in
sqlite/provisioning.py to better support running tests against
multiple simultaneous SQLite database files. Additionally enables
backend for test_sqlite which was skipping everything
for aiosqlite too, fortunately everything there is passing.
Fixes: #5848
Change-Id: I43f53ebc62298a84a4abe149e1eb699a027b7915
Added support for the aiosqlite database driver for use with the
SQLAlchemy asyncio extension.
Fixes: #5920
Change-Id: Id11a320516a44e886a6f518d2866a0f992413e55
Rudimentary and experimental support for Mypy has been added in the form of
a new plugin, which itself depends on new typing stubs for SQLAlchemy. The
plugin allows declarative mappings in their standard form to both be
compatible with Mypy as well as to provide typing support for mapped
classes and instances.
Fixes: #4609
Change-Id: Ia035978c02ad3a5c0e5b3c6c30044dd5a3155170
Mike Bayer