Commit Graph

12846 Commits

Author SHA1 Message Date
mike bayer 906bb6533d Merge "Correctly apply self_group in type_coerce element." 2020-05-23 21:21:39 +00:00
mike bayer a2e8e773c5 Merge "Add with_for_update mysql new functionalities" 2020-05-23 21:14:27 +00:00
mike bayer 56f3021386 Merge "Improve error message when using :meth:_query.Query.filter_by in a query where the first entity is not a mapped class." 2020-05-23 21:11:50 +00:00
mike bayer 330381a331 Merge "Add note re: pymssql not in CI" 2020-05-23 21:10:59 +00:00
mike bayer c82540033c Merge "fix typo in reference documentation" 2020-05-23 21:10:14 +00:00
Gord Thompson 9a04c2f657 Stop converting mssql datetime.time parameters to datetime.datetime
Fixes: #5339
Change-Id: Ida75422d8c3fdfc7adae68e547d88df49368a693
2020-05-23 07:03:41 -06:00
mike bayer ebceb618ef Merge "Add immutabledict C code" 2020-05-23 12:51:07 +00:00
Mike Bayer fcbd03e48a Add immutabledict C code
Start trying to convert fundamental objects to
C as we now rely on a fairly small core of things,
and 1.4 is having problems with complexity added being
slower than the performance gains we are trying to build in.

immutabledict here does seem to bench as twice as fast as the
Python one, see below.  However, it does not appear to be
used prominently enough to make any dent in the performance
tests.

at the very least it may provide us some more lift-and-copy
code for more C extensions.

import timeit

from sqlalchemy.util._collections import not_immutabledict, immutabledict

def run(dict_cls):
    for i in range(1000000):
        d1 = dict_cls({"x": 5, "y": 4})

        d2 = d1.union({"x": 17, "new key": "some other value"}, None)

        assert list(d2) == ["x", "y", "new key"]

print(
    timeit.timeit(
        "run(d)", "from __main__ import run, not_immutabledict as d", number=1
    )
)
print(
    timeit.timeit(
        "run(d)", "from __main__ import run, immutabledict as d", number=1
    )
)

output:

python: 1.8799766399897635
C code: 0.8880784640205093

Change-Id: I29e7104dc21dcc7cdf895bf274003af2e219bf6d
2020-05-23 00:05:13 -04:00
Federico Caselli 2632a8db05 fix typo in reference documentation
Change-Id: I5720635863f0d8a8ebfa4ddc51008bb93c1067b4
2020-05-23 00:11:46 +02:00
Federico Caselli d163088de1 Correctly apply self_group in type_coerce element.
The type coerce element did not correctly apply grouping rules when using
in an expression

Fixes: #5344
Change-Id: Id67b0e60ac54f8992f931aaed62731672f60c96c
2020-05-22 23:56:50 +02:00
mike bayer 3fb0eb5a15 Merge "Fix query string escaping in engine URLs" 2020-05-22 20:17:49 +00:00
mike bayer 16fcbd3be2 Merge "Don't emit pyodbc "no driver" warning for empty URL" 2020-05-22 20:12:42 +00:00
Mike Bayer 88ae6a6cdd Cross reference do_connect() event w/ creator
two questions today involving creator / do_connect,
do_connect is not well known enough, ensure docs are present
and prominent.

Change-Id: I85d518b9fc7b9b069a18010969abefa360134fe9
2020-05-22 16:04:17 -04:00
Mike Bayer 29534cf139 Don't emit pyodbc "no driver" warning for empty URL
Fixed an issue in the pyodbc connector such that a warning about pyodbc
"drivername" would be emitted when using a totally empty URL.  Empty URLs
are normal when producing a non-connected dialect object or when using the
"creator" argument to create_engine(). The warning now only emits if the
driver name is missing but other parameters are still present.

Fixes: #5346
Change-Id: I0ee6f5fd5af7faca63bf0d7034410942f40834a8
2020-05-22 13:15:30 -04:00
mike bayer 1d320fa480 Merge "Structural / performance refinements" 2020-05-22 15:45:57 +00:00
Mike Bayer 5b2abf4d26 Structural / performance refinements
* state connection schema_translate_map entirely in terms of
  execution options, support for per-execution options as well

* use slots for role impls, remove superclass of the roles
  themselves as this is not needed

* tighten loop in resolve, might become a C function

Change-Id: Ib98ac9b65022fbf976e49c6060e4c37573528c5f
2020-05-22 10:30:38 -04:00
Mike Bayer 5910f9ca3f Don't incref on new reference key_style
in 4550983e0c we
added a new attribute key_style.  remove an erroneous
Py_INCREF when we acquire it from PyLong_FromLong
as we already own the reference.    since this
is a new reference we actualy need to Py_DECREF
it because we aren't returning it.

Change-Id: I61470513a173c76863ec6f7f5ff9b2ec13582f08
2020-05-22 10:19:57 -04:00
mike bayer d45657a2f5 Merge "Performance fixes for new result set" 2020-05-21 19:03:32 +00:00
Miguel Grinberg 2644693c0b Fix query string escaping in engine URLs
Fixed issue in :class:`.URL` object where stringifying the object
would not URL encode special characters, preventing the URL from being
re-consumable as a real URL.  Pull request courtesy Miguel Grinberg.

Fixes: #5341
Closes: #5342
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5342
Pull-request-sha: 362ca33983

Change-Id: Ief6218122d1ec0c70479eb1a90e1c16433801924
2020-05-21 14:27:07 -04:00
Mike Bayer 4550983e0c Performance fixes for new result set
A few small mistakes led to huge callcounts.   Additionally,
the warn-on-get behavior which is attempting to warn for
deprecated access in SQLAlchemy 2.0 is very expensive; it's not clear
if its feasible to have this warning or to somehow alter how it
works.

Fixes: #5340
Change-Id: I73bdd2d7b6f1b25cc0222accabd585cf761a5af4
2020-05-21 14:16:03 -04:00
Federico Caselli b67548ad78 Target real pypi in the create wheel workflow
Change-Id: Ic764aae08b9d45011048f21c8ee992222c078bd2
2020-05-20 21:31:16 +02:00
Mike Bayer 3d55263c92 fix table.schema version to 1.3.18
Fixes: #5338
Change-Id: I12ce8021f5627c306c768b74da991ef4af0a5b55
2020-05-19 18:56:11 -04:00
mike bayer 6287e1df65 Merge "SQLite 3.31 added support for computed column." 2020-05-19 20:18:40 +00:00
mike bayer fa7075861e Merge "Introduce :class:.IdentityOptions to store common parameters for sequences and identity columns." 2020-05-19 19:47:16 +00:00
Mike Bayer af0e59ff16 Disable "check unicode returns" under Python 3
Disabled the "unicode returns" check that runs on dialect startup when
running under Python 3, which for many years has occurred in order to test
the current DBAPI's behavior for whether or not it returns Python Unicode
or Py2K strings for the VARCHAR and NVARCHAR datatypes.  The check still
occurs by default under Python 2, however the mechanism to test the
behavior will be removed in SQLAlchemy 2.0 when Python 2 support is also
removed.

This logic was very effective when it was needed, however now that Python 3
is standard, all DBAPIs are expected to return Python 3 strings for
character datatypes.  In the unlikely case that a third party DBAPI does
not support this, the conversion logic within :class:`.String` is still
available and the third party dialect may specify this in its upfront
dialect flags by setting the dialect level flag ``returns_unicode_strings``
to one of :attr:`.String.RETURNS_CONDITIONAL` or
:attr:`.String.RETURNS_BYTES`, both of which will enable Unicode conversion
even under Python 3.

As part of this change, disabling testing of the doctest tutorials under
Python 2.

Fixes: #5315
Change-Id: I1260e894611409d3b7fe1a92bd90c52043bbcf19
2020-05-19 13:53:39 -04:00
Federico Caselli 87949deb04 SQLite 3.31 added support for computed column.
This change enables their support in SQLAlchemy when targeting SQLite.

Fixes: #5297
Change-Id: Ia9f21a49e58fc977e3c669b8176036c95d93b9c8
2020-05-19 19:33:02 +02:00
Mike Bayer 53af60b353 Streamline visitors.iterate
This method might be used more significantly in the
ORM refactor, so further refine it.

* all get_children() methods now work entirely based on iterators.
  Basically only select() was sensitive to this anymore and it now
  chains the iterators together

* remove all kinds of flags like column_collections, schema_visitor
  that apparently aren't used anymore.

* remove the "depthfirst" visitors as these don't seem to be
  used either.

* make sure select() yields its columns first as these will be used
  to determine the current mapper.

Change-Id: I05273a2d5306a57c2d1b0979050748cf3ac964bf
2020-05-18 16:21:54 -04:00
mike bayer de1fd55af4 Merge "Update transaction / connection handling" 2020-05-17 22:00:16 +00:00
Mike Bayer 0e53221eef Update transaction / connection handling
step one, do away with __connection attribute and using
awkward AttributeError logic

step two, move all management of "connection._transaction"
into the transaction objects themselves where it's easier
to follow.

build MarkerTransaction that takes the role of
"do-nothing block"

new connection datamodel is: connection._transaction, always
a root, connection._nested_transaction, always a nested.

nested transactions still chain to each other as this
is still sort of necessary but they consider the root
transaction separately, and the marker transactions
not at all.

introduce new InvalidRequestError subclass
PendingRollbackError.  Apply to connection and session
for all cases where a transaction needs to be rolled
back before continuing.   Within Connection,
both PendingRollbackError as well as ResourceClosedError
are now raised directly without being handled by
handle_dbapi_error();  this removes these two exception
cases from the handle_error event handler as well as
from StatementError wrapping, as these two exceptions are
not statement oriented and are instead programmatic
issues, that the application is failing to handle database
errors properly.

Revise savepoints so that when a release fails, they set
themselves as inactive so that their rollback() method
does not throw another exception.

Give savepoints another go on MySQL, can't get release working
however get support for basic round trip going

Fixes: #5327
Change-Id: Ia3cbbf56d4882fcc7980f90519412f1711fae74d
2020-05-17 16:32:22 -04:00
Mike Bayer 0d611b8891 Reword delete-orphan on many error message and document
For many years we have encountered users making use of the
"single_parent" flag in response to the error message for
"delete-orphan" expressing this as a means to cancel the current
error.   However, the actual issue here is usually a misuse
of the delete-orphan cascade setting.  Reword the error message to
be much more descriptive about what this means and add new
error link sections describing the situation in as much detail
as possible.

Fixes: #5329
Change-Id: I7ba710378b2935479ab22ff9a0a79c692dbf69a6
2020-05-16 19:43:46 -04:00
Gord Thompson 88bffda3fe Add note re: pymssql not in CI
Change-Id: I45f78131ffe1881a3965e8aa41bbc46da7d43a5b
2020-05-16 08:57:06 -06:00
Federico Caselli 1160d60a14 Introduce :class:.IdentityOptions to store common parameters for
sequences and identity columns.

References: #5324
Change-Id: I72f7fc1a003456206b004d3d26306940f9c36414
2020-05-15 23:57:28 +02:00
RobotScribe 103260ddb4 Add with_for_update mysql new functionalities
Fixes: #4860

# Description
Add nowait, skip_lock, of arguments to for_update_clause for mysql

### Checklist

This pull request is:

- [ ] A documentation / typographical error fix
	- Good to go, no issue or tests are needed
- [ ] A short code fix
	- please include the issue number, and create an issue if none exists, which
	  must include a complete example of the issue.  one line code fixes without an
	  issue and demonstration will not be accepted.
	- Please include: `Fixes: #<issue number>` in the commit message
	- please include tests.   one line code fixes without tests will not be accepted.
- [x] A new feature implementation
	- please include the issue number, and create an issue if none exists, which must
	  include a complete example of how the feature would look.
	- Please include: `Fixes: #<issue number>` in the commit message
	- please include tests.

**Have a nice day!**

Closes: #5290
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5290
Pull-request-sha: 490e822e73

Change-Id: Ibd2acc47b538c601c69c8fb954776035ecab4c6c
2020-05-15 15:50:32 -06:00
mike bayer d4182d4da9 Merge "Add 'schema' parameter to table" 2020-05-15 17:11:20 +00:00
Federico Caselli 8c10e29dc7 Improve error message when using :meth:_query.Query.filter_by in
a query where the first entity is not a mapped class.

Fixes: #5260
Change-Id: I18cfccc0f3e587cb3df3e1f5f23b6885d5f2f466
2020-05-15 18:42:41 +02:00
mike bayer 393fa8417d Merge "Adjust documentation wording for firebird and sybase" 2020-05-15 15:15:45 +00:00
mike bayer 0fe13a5fbe Merge "Move supports_sane_rowcount_returning = False to dialect level" 2020-05-15 15:14:55 +00:00
Gord Thompson 3f08d54d30 Adjust documentation wording for firebird and sybase
Change-Id: Ied2cdab1c2dbdee681a5500285a5bcdd4d589afa
2020-05-15 07:16:35 -06:00
Mike Bayer 79de84b25e Actively unset reset agent in discard transaction
The assumptions in _discard_transaction from
916e1fea25 were too narrow,
assuming that if the given transaction were not our
"current" one, that this would not be the reset agent.  however
as the legacy behvaior is that even a "nested" transaction gets
set as "self._transaction", this did not accommodate for the nested
transaction being thrown away.   We will attempt to refine all of this
logic in #5327 for 1.4 /master assuming this is feasible for the
full suite of current use cases.

Fixes: #5326
Change-Id: I6787e82c9e50c23317f87d0d094122c6a6f066da
2020-05-14 10:51:29 -04:00
Mike Bayer 6bd171cbc8 Restore unreleased_XY directories
these directories were removed in 4e5b7e6e62
which breaks the documentation builds.

Change-Id: I17aa66bf95c69a075a68e78245fad3bef85dc29e
2020-05-14 10:42:08 -04:00
Mike Bayer 19f5e21713 cherry-pick changelog update for 1.3.18 2020-05-13 17:40:41 -04:00
Mike Bayer f3f80a37dc cherry-pick changelog from 1.3.17 2020-05-13 17:40:40 -04:00
mike bayer 31807c2637 Merge "Assert reset agent always set correctly and is active" 2020-05-13 21:34:14 +00:00
Mike Bayer 916e1fea25 Assert reset agent always set correctly and is active
Fixed fairly critical issue where the DBAPI connection could be returned to
the connection pool while still in an un-rolled-back state. The reset agent
responsible for rolling back the connection could be corrupted in the case
that the transaction was "closed" without being rolled back or committed,
which can occur in some scenarios when using ORM sessions and emitting
.close() in a certain pattern involving savepoints.   The fix ensures that
the reset agent is always active.

note that the reset agent will go away in 2.0 and the only real
purpose of it is for logging of ROLLBACK.   Apparently with the
SQLite singleton engine in the test suite, there are some strucutral
mismatches in the test fixtures where the reset agent is getting
set differently than the transaction likely due to the same connection
being shared in multiple context, though it's unclear.

Fixes: #5326
Change-Id: If056870ea70a2d9a1749768988d5e023f3061b31
2020-05-13 16:12:42 -04:00
mike bayer dabc226af6 Merge "Load external firebird or sybase dialect if available" 2020-05-13 17:15:59 +00:00
mike bayer ce6619756e Merge "Fix typo in changelog of 5237" 2020-05-12 22:31:49 +00:00
Federico Caselli 14e0ba2144 Fix typo in changelog of 5237
Change-Id: I9e526b133c130c1656fd09e5fecdc3a84c425d46
2020-05-12 21:37:50 +02:00
Gord Thompson cc97dade0d Move supports_sane_rowcount_returning = False to dialect level
Fixes: #5321
Change-Id: Id83e98e9013818424c133297a850746302633158
2020-05-11 14:44:37 -06:00
Federico Caselli 447f16bc7f Ensure order in doctest of core/tutorial.rst
Fix doctest error caused by the missing order by clause in the
tests introduced in Ia1bbe3248b4f7c74fbee06fedf76a6ce94cd28a6

Change-Id: I55b45690879ebbaa57bc62765fcdab06f5e9c6f3
2020-05-11 21:47:08 +02:00
Gord Thompson 587f6c7715 Load external firebird or sybase dialect if available
Fixes: #5318

Extension of I1660abb11c02656fbf388f2f9c4257075111be58

Change-Id: I32b678430497327f9b08f821bd345a2557e34b1f
2020-05-11 07:23:54 -06:00