Commit Graph

6797 Commits

Author SHA1 Message Date
Mike Bayer e3716012c5 Create connection characteristics API; implement postgresql flags
Added support for PostgreSQL "readonly" and "deferrable" flags for all of
psycopg2, asyncpg and pg8000 dialects.   This takes advantage of a newly
generalized version of the "isolation level" API to support other kinds of
session attributes set via execution options that are reliably reset
when connections are returned to the connection pool.

Fixes: #5549
Change-Id: I0ad6d7a095e49d331618274c40ce75c76afdc7dd
2020-09-08 11:16:53 -04:00
Gord Thompson 5b6dfc0c38 Modernize some .execute calls in EnumSetTest
Change-Id: I9bbc3be4ec1797686ddf4a559f5b9e1679c62f58
2020-09-07 22:23:06 -04:00
Mike Bayer b0e9083eb2 Don't rely on string col name in adapt_to_context
fixed an issue where even though the method claims to be
matching up columns positionally, it was failing on that by
looking in "keymap" based on string name.

Adds a new member to the _keymap recs MD_RESULT_MAP_INDEX
so that we can efficiently link from the generated keymap
back to the compiled._result_columns structure without
any ambiguity.

Fixes: #5559

Change-Id: Ie2fa9165c16625ef860ffac1190e00575e96761f
2020-09-06 09:55:27 -04:00
Mike Bayer fc612d1714 Revert "Block non-mysqldb / pymysql from testing Enum"
This reverts commit 55843040b1.
2020-09-05 09:47:45 -04:00
Mike Bayer 55843040b1 Block non-mysqldb / pymysql from testing Enum
The mariadbconnector driver is having sporadic interpreter
crashes within this test suite.  hopefully the crashes
are specific to these tests else we'll have to remove
mariadb from CI.

Change-Id: Idb7a9521e795d39957bce45415013eb4e771e560
2020-09-03 14:50:10 -04:00
Mike Bayer b050169600 Add caveat re: with_expression and already-loaded object
Adds a test for the populate_existing() behavior as well.

Fixes: #5553
Change-Id: Ib0db6227c3fec7d0065f2a7caa36b3fd94ef14fd
2020-09-02 14:56:49 -04:00
mike bayer 7439697ca3 Merge "Improve reflection for mssql temporary tables" 2020-09-01 17:12:11 +00:00
mike bayer d61cf0a9ad Merge "Add support for classical mapping of dataclasses" 2020-09-01 17:10:53 +00:00
mike bayer 3fe3b88cc0 Merge "MariaDB dialect implementation" 2020-09-01 17:09:19 +00:00
Václav Klusák f806491fca Add support for classical mapping of dataclasses
Added support for direct mapping of Python classes that are defined using
the Python ``dataclasses`` decorator.    See the section
:ref:`mapping_dataclasses` for background.  Pull request courtesy Václav
Klusák.

Fixes: #5027
Closes: #5516
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5516
Pull-request-sha: bb48c63d15

Change-Id: Ie33db2aae4adeeb5d99633fe926b9c30bab0b885
2020-09-01 10:59:07 -04:00
Gord Thompson 516131c40d Improve reflection for mssql temporary tables
Fixes: #5506
Change-Id: I718474d76e3c630a1b71e07eaa20cefb104d11de
2020-09-01 08:05:51 -06:00
mike bayer 301c3f3579 Merge "Provide a more detailed error message for Query.join()" 2020-09-01 13:50:29 +00:00
RamonWill 17090c004e Provide a more detailed error message for Query.join()
An :class:`.ArgumentError` with more detail is now raised if the target
parameter for :meth:`_query.Query.join` is set to an unmapped object.
Prior to this change a less detailed ``AttributeError`` was raised.
Pull request courtesy Ramon Williams.

Fixes: #4428
Closes: #5452
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5452
Pull-request-sha: b148df5470

Change-Id: I873453d1fdb651178216aac698baac63ae5a94e8
2020-08-31 19:01:15 -04:00
Fantix King 9c5b989cf0 Fix a mis-reference in create_async_engine().
`AsyncMethodRequired` is actually from
`sqlalchemy.ext.asyncio.exc`, so here it
should be referenced as `async_exc.AsyncMethodRequired`,
instead of `exc.AsyncMethodRequired`.

Fixes: #5529
Closes: #5545
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5545
Pull-request-sha: d8f885c587

Change-Id: I6886558bfd33d3e9e283fbd60c0ec971a1f22c0c
2020-08-31 18:23:41 -04:00
Georg Richter 603f7d30f6 MariaDB dialect implementation
Fixes: #5459

Closes: #5515
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5515
Pull-request-sha: 760090b906

Change-Id: I30e8fbc02b7b5329ca228cd39f6fb7cfd0e43092
2020-08-31 18:19:35 -04:00
mike bayer ed5e015ef4 Merge "Include PostgreSQL in trigger test and correct documentation example" 2020-08-31 16:34:31 +00:00
RamonWill 2e887c7a41 Include PostgreSQL in trigger test and correct documentation example
Include PostgreSQL dialect in trigger test and correct DDL example in documentation

A user highlighted that the syntax in the DDL trigger example was incorrect for PostgreSQL. The trigger tests where also skipping the PostgreSQL dialect until the syntax was corrected. This PR fixes both of these issues.

This pull request is:

- [X ] A documentation / typographical error fix
	- Good to go, no issue or tests are needed
- [ ] A short code fix
	- please include the issue number, and create an issue if none exists, which
	  must include a complete example of the issue.  one line code fixes without an
	  issue and demonstration will not be accepted.
	- Please include: `Fixes: #<issue number>` in the commit message
	- please include tests.   one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
	- please include the issue number, and create an issue if none exists, which must
	  include a complete example of how the feature would look.
	- Please include: `Fixes: #<issue number>` in the commit message
	- please include tests.

**Have a nice day!**
Fixes: #4037

Closes: #5548
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5548
Pull-request-sha: 1db5e47adb

Change-Id: I06edbcab99c82a3ce25581b81f8d2a4a028c07c3
2020-08-31 16:13:29 +01:00
Fantix King cae33a7901 Fix AsyncEngine connect() bug when pool is exhausted
### Description

Decorating the referenced `await_fallback` with `staticmethod` would stop `AsyncAdaptedQueue.await_` from being treated as a bound method.

### Checklist

This pull request is:

- [x] A short code fix
    Fixes #5546

**Have a nice day!**

Closes: #5547
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5547
Pull-request-sha: 6f18ee290e

Change-Id: Ie335ee650f1dee0d1fce59e448217a48307b3435
2020-08-31 03:41:00 -04:00
Mike Bayer 575b6dded9 Support extra / single inh criteria with ORM update/delete
The ORM bulk update and delete operations, historically available via the
:meth:`_orm.Query.update` and :meth:`_orm.Query.delete` methods as well as
via the :class:`_dml.Update` and :class:`_dml.Delete` constructs for
:term:`2.0 style` execution, will now automatically accommodate for the
additional WHERE criteria needed for a single-table inheritance
discrminiator.   Joined-table inheritance is still not directly
supported. The new :func:`_orm.with_loader_criteria` construct is also
supported for all mappings with bulk update/delete.

Fixes: #5018
Fixes: #3903
Change-Id: Id90827cc7e2bc713d1255127f908c8e133de9295
2020-08-30 19:45:04 -04:00
mike bayer 406034d41a Merge "internal test framework files for standardization of is_not/not_in;" 2020-08-30 14:33:08 +00:00
Mike Bayer 34dfd7eb88 Update connection docs for migrating off of nesting
Change-Id: I3a81140f00a4a9945121bfb8ec4c0e3953b4085f
2020-08-29 18:55:02 -04:00
jonathan vanasco 672087176e internal test framework files for standardization of is_not/not_in;
this is safe for 1.3.x

Change-Id: Icba38fdc20f5d8ac407383a4278ccb346e09af38
2020-08-29 12:05:58 -04:00
Mike Bayer 1e80028550 See if the future is here
The docs are going to talk a lot about session.execute(select())
for ORM queries, and additionally it's much easier to help
users with queries and such if we can use this new syntax.
I'm hoping to see how hard it is to get a unified tutorial
started that switches to new syntax.  Basically, new syntax
is much easier to explain and less buggy.   But, if we
are starting to present new syntax with the explicit goal
of being easier to explain for less experienced programmers,
the "future" thing is going to just be an impediment
to that.

See if we can remove "future" from session.execute(),
so that ORM-enabled select() statements return ORM results
at that level.  This does not change the presence of the
"future" flag for the Session's construction and for its
transactional behaviors.

The only perceptible change of the future flag for
session.execute() is that session.execute(select()) where the
statement has ORM entities in it now returns ORM new
style tuples rather than old style tuples.   Like
mutating a URL, it's hopefully not very common that people
are doing this.

Change-Id: I0aa10322bb787d554d32772e3bc60548f1bf6206
2020-08-28 18:14:11 -04:00
Gord Thompson dc91c7db7f Emit v2.0 deprecation warning for "implicit autocommit"
"Implicit autocommit", which is the COMMIT that occurs when a DML or DDL
statement is emitted on a connection, is deprecated and won't be part of
SQLAlchemy 2.0.   A 2.0-style warning is emitted when autocommit takes
effect, so that the calling code may be adjusted to use an explicit
transaction.

As part of this change, DDL methods such as
:meth:`_schema.MetaData.create_all` when used against a
:class:`_engine.Engine` or :class:`_engine.Connection` will run the
operation in a BEGIN block if one is not started already.

The MySQL and MariaDB dialects now query from the information_schema.tables
system view in order to determine if a particular table exists or not.
Previously, the "DESCRIBE" command was used with an exception catch to
detect non-existent,  which would have the undesirable effect of emitting a
ROLLBACK on the connection. There appeared to be legacy encoding issues
which prevented the use of "SHOW TABLES", for this, but as MySQL support is
now at 5.0.2  or above due to 🎫`4189`, the information_schema tables
are now available in all cases.

Fixes: #4846
Change-Id: I733a7e0e17477a63607fb9931c87c393bbd7ac57
2020-08-28 16:32:05 -04:00
mike bayer 67a2afbed5 Merge "Add support for regular expression on supported backend." 2020-08-27 22:28:39 +00:00
Federico Caselli b1b97ed1fc Add support for regular expression on supported backend.
Two operations have been defined:

* :meth:`~.ColumnOperators.regexp_match` implementing a regular
  expression match like function.
* :meth:`~.ColumnOperators.regexp_replace` implementing a regular
  expression string replace function.

Fixes: #1390
Change-Id: I44556846e4668ccf329023613bd26861d5c674e6
2020-08-27 17:30:18 -04:00
Mike Bayer d78686b4f1 Raise NotImplemenedError for association proxy __clause_element__
It's not possible right now to use an association proxy element as a plain
column expression to be SELECTed from or used in a SQL function.  An
informative error is now raised when this occurs.

Fixes: #5542
Change-Id: I334e767ebc0b56c1dccc4a1e5185b0435af77b93
2020-08-27 15:53:01 -04:00
mike bayer 91100022ec Merge "make URL immutable" 2020-08-26 02:44:17 +00:00
mike bayer 014879fdc6 Merge "More descriptive error for non-mapped string prop name" 2020-08-26 02:43:32 +00:00
Mike Bayer 3668b3a30c make URL immutable
it's not really correct that URL is mutable and doesn't do
any argument checking.   propose replacing it with an immutable
named tuple with rich copy-and-mutate methods.

At the moment this makes a hard change to the CreateEnginePlugin
docs that previously recommended url.query.pop().  I can't find
any plugins on github other than my own that are using this
feature, so see if we can just make a hard change on this one.

Fixes: #5526
Change-Id: I28a0a471d80792fa8c28f4fa573d6352966a4a79
2020-08-25 20:10:16 -04:00
mike bayer 1d18c46512 Merge "Updates for MariaDB sequences" 2020-08-25 22:14:19 +00:00
jonathan vanasco aff9d06e00 More descriptive error for non-mapped string prop name
Fixed issue where using a loader option against a string attribute name
that is not actually a mapped attribute, such as a plain Python descriptor,
would raise an uninformative AttributeError;  a descriptive error is now
raised.

Fixes: #4589
Closes: #4594
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4594
Pull-request-sha: 2b7ed5240f

Change-Id: I66b9937991eb7cdbe074a92f490af1c80d16449e
2020-08-25 11:02:43 -04:00
Gord Thompson a1a081398e Add deprecation warning for mssql legacy_schema_aliasing
Fixes: #4809
Change-Id: I9ce2a5dfb79d86624c187ee28b5911fd14328ce2
2020-08-24 11:23:33 -04:00
Federico Caselli 9ab4da7018 Updates for MariaDB sequences
MariaDB should not run a Sequence if it has optional=True.
Additionally, rework the rules in crud.py to accommodate the
new combination MariaDB brings us, which is a dialect
that supports both cursor.lastrowid, explicit sequences,
*and* no support for returning.

Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>
Fixes: #5528
Change-Id: I9a8ea69a34983affa95dfd22186e2908fdf0d58c
2020-08-22 12:46:12 -04:00
mike bayer 317f2e1be2 Merge "Add support for identity columns" 2020-08-21 21:39:40 +00:00
mike bayer 9b6b867fe5 Merge "Raise UnmappedInstanceError if the attribute of a database object is an unmapped object." 2020-08-20 19:24:12 +00:00
mike bayer 544ef23cd3 Merge "normalize execute style for events, 2.0" 2020-08-20 15:22:08 +00:00
Mike Bayer a1939719a6 normalize execute style for events, 2.0
The _execute_20 and exec_driver_sql methods should wrap
up the parameters so that they represent the single list / single
dictionary style of invocation into the legacy methods.  then
the before_ after_ execute event handlers should be receiving
the parameter dictionary as a single dictionary.   this requires
that we break out distill_params to work differently if event
handlers are present.

additionally, add deprecation warnings for old argument passing
styles.

Change-Id: I97cb4d06adfcc6b889f10d01cc7775925cffb116
2020-08-20 10:14:21 -04:00
mike bayer 6234792375 Merge "Implement DDL visitor for PG ENUM with schema translate support" 2020-08-19 17:38:07 +00:00
mike bayer 348afaf742 Merge "Add JSON support for mssql" 2020-08-19 17:25:06 +00:00
Gord Thompson 0c7a867ac7 Add JSON support for mssql
Added support for the :class:`_types.JSON` datatype on the SQL Server
dialect using the :class:`_mssql.JSON` implementation, which implements SQL
Server's JSON functionality against the ``NVARCHAR(max)`` datatype as per
SQL Server documentation. Implementation courtesy Gord Thompson.

Fixes: #4384
Change-Id: I28af79a4d8fafaa68ea032228609bba727784f18
2020-08-19 11:05:52 -04:00
Mike Bayer d1005e1305 Implement DDL visitor for PG ENUM with schema translate support
Fixed issue where the :class:`_postgresql.ENUM` type would not consult the
schema translate map when emitting a CREATE TYPE or DROP TYPE during the
test to see if the type exists or not.  Additionally, repaired an issue
where if the same enum were encountered multiple times in a single DDL
sequence, the "check" query would run repeatedly rather than relying upon a
cached value.

Fixes: #5520
Change-Id: I79f46e29ac0168e873ff178c242f8d78f6679aeb
2020-08-19 10:43:38 -04:00
Federico Caselli 26e8d3b5bd Add support for identity columns
Added the :class:`_schema.Identity` construct that can be used to
configure identity columns rendered with GENERATED { ALWAYS |
BY DEFAULT } AS IDENTITY. Currently the supported backends are
PostgreSQL >= 10, Oracle >= 12 and MSSQL (with different syntax
and a subset of functionalities).

Fixes: #5362
Fixes: #5324
Fixes: #5360

Change-Id: Iecea6f3ceb36821e8b96f0b61049b580507a1875
2020-08-19 00:34:23 +02:00
Federico Caselli 8e01a928d9 Support data types for CREATE SEQUENCE in PostgreSQL
Allow specifying the data type when creating a :class:`.Sequence` in
PostgreSQL by using the parameter :paramref:`.Sequence.data_type`.

Fixes: #5498

Change-Id: I2b4a80aa89b1503c56748dc3ecd2cf145faddd8b
2020-08-18 22:03:28 +00:00
mike bayer 0901190bb4 Merge "Deliver straight BinaryExpr w/ no negate for any() / all()" 2020-08-18 19:07:19 +00:00
Mike Bayer 8bc793c4db Deliver straight BinaryExpr w/ no negate for any() / all()
Adjusted the :meth:`_types.ARRAY.Comparator.any` and
:meth:`_types.ARRAY.Comparator.all` methods to implement a straight "NOT"
operation for negation, rather than negating the comparison operator.

Fixes: #5518
Change-Id: I87ee9278c321aafe51a679fcfcbb5fbb11307fda
2020-08-18 14:18:02 -04:00
Tony Locke 06f1929b86 Update dialect for pg8000 version 1.16.0
The pg8000 dialect has been revised and modernized for the most recent
version of the pg8000 driver for PostgreSQL.  Changes to the dialect
include:

* All data types are now sent as text rather than binary.

* Using adapters, custom types can be plugged in to pg8000.

* Previously, named prepared statements were used for all statements.
  Now unnamed prepared statements are used by default, and named
  prepared statements can be used explicitly by calling the
  Connection.prepare() method, which returns a PreparedStatement
  object.

Pull request courtesy Tony Locke.

Notes by Mike: to get this all working it was needed to break
up JSONIndexType into "str" and "int" subtypes; this will be
needed for any dialect that is dependent on setinputsizes().

also includes @caselit's idea to include query params
in the dbdriver parameter.

Co-authored-by: Mike Bayer <mike_mp@zzzcomputing.com>

Closes: #5451
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5451
Pull-request-sha: 639751ca9c

Change-Id: I2869bc52c330916773a41d11d12c297aecc8fcd8
2020-08-18 11:12:16 -04:00
mike bayer 1c1e038ac4 Merge "Create a real type for Tuple() and handle appropriately in compiler" 2020-08-17 19:56:30 +00:00
mike bayer 95b6180421 Merge "Make discriminator column used by ConcreteBase configurable" 2020-08-17 19:55:38 +00:00
mike bayer ee99a879ac Merge "Bump minimum MySQL version to 5.0.2; use all-numeric server version" 2020-08-17 15:53:20 +00:00