in all cases unconditionally, the number of use cases that go beyond what
dbapi_error() is expecting has gone too far for an 0.9 release.
Additionally, the number of things we'd like to track is really a lot
more than the five arguments here, and ExecutionContext is really not
suitable as totally public API for this. So restore dbapi_error
to its old version, deprecate, and build out handle_error instead.
This is a lot more extensible and doesn't get in the way of anything
compatibility-wise.
have been enhanced such that the function handler is now capable
of raising or returning a new exception object, which will replace
the exception normally being thrown by SQLAlchemy.
fixes#3076
- Fixed bug which would occur if a DBAPI exception
occurs when the engine first connects and does its initial checks,
and the exception is not a disconnect exception, yet the cursor
raises an error when we try to close it. In this case the real
exception would be quashed as we tried to log the cursor close
exception via the connection pool and failed, as we were trying
to access the pool's logger in a way that is inappropriate
in this very specific scenario. fixes#3063
1. make sure pool._invalidate() sets the timestamp up before
invalidating the target connection. we can otherwise show how the
conn.invalidate() + pool._invalidate() can lead to an extra connection
being made.
2. to help with that, soften up the check on connection.invalidate()
when connection is already closed. a warning is fine here
3. add a mutex to test_max_overflow() when we connect, because the way
we're using mock depends on an iterator, that needs to be synchronized
implementation allows an event handler to redefine the specific mechanics
by which an arbitrary dialect invokes execute() or executemany() on a
DBAPI cursor. The new events, at this point semi-public and experimental,
are in support of some upcoming transaction-related extensions.
after one or more :class:`.Connection` objects have been created
(such as by an orm :class:`.Session` or via explicit connect)
and the listener will pick up events from those connections.
Previously, performance concerns pushed the event transfer from
:class:`.Engine` to :class:`.Connection` at init-time only, but
we've inlined a bunch of conditional checks to make this possible
without any additional function calls. fixes#2978
within userland engine.dispose(); as some SQLA tests already failed when the replace step
was removed, due to those conns still being referenced, it's likely this will
create surprises for all those users that incorrectly use dispose()
and it's not really worth dealing with. This doesn't affect the change
we made for ref: #2985.
recycles the connection pool when a "disconnect" condition is detected;
instead of discarding the pool and explicitly closing out connections,
the pool is retained and a "generational" timestamp is updated to
reflect the current time, thereby causing all existing connections
to be recycled when they are next checked out. This greatly simplifies
the recycle process, removes the need for "waking up" connect attempts
waiting on the old pool and eliminates the race condition that many
immediately-discarded "pool" objects could be created during the
recycle operation. fixes#2985
:class:`.DateTime`. As Oracle has no "datetime" type per se,
it instead has only ``DATE``, it is appropriate here that the
``DATE`` type as present in the Oracle dialect be an instance of
:class:`.DateTime`. This issue doesn't change anything as far as
the behavior of the type, as data conversion is handled by the
DBAPI in any case, however the improved subclass layout will help
the use cases of inspecting types for cross-database compatibility.
Also removed uppercase ``DATETIME`` from the Oracle dialect as this
type isn't functional in that context. fixes#2987
emitted for the "_cursor_execute()" method of :class:`.Connection`;
this is the "quick" executor that is used for things like
when a sequence is executed ahead of an INSERT statement, as well as
for dialect startup checks like unicode returns, charset, etc.
the :meth:`.ConnectionEvents.before_cursor_execute` event was already
invoked here. The "executemany" flag is now always set to False
here, as this event always corresponds to a single execution.
Previously the flag could be True if we were acting on behalf of
an executemany INSERT statement.
If autoloading of a table fails, don't register it in a metadata
instance. It seems that the original behaviour was accidentally
changed in f6198d9abf, restore it.
Closes issue #2988
is currently being supported in addition to nose, and will likely
be preferred to nose going forward. The nose plugin system used
by SQLAlchemy has been split out so that it works under pytest as
well. There are no plans to drop support for nose at the moment
and we hope that the test suite itself can continue to remain as
agnostic of testing platform as possible. See the file
README.unittests.rst for updated information on running tests
with pytest.
The test plugin system has also been enhanced to support running
tests against mutiple database URLs at once, by specifying the ``--db``
and/or ``--dburi`` flags multiple times. This does not run the entire test
suite for each database, but instead allows test cases that are specific
to certain backends make use of that backend as the test is run.
When using pytest as the test runner, the system will also run
specific test suites multiple times, once for each database, particularly
those tests within the "dialect suite". The plan is that the enhanced
system will also be used by Alembic, and allow Alembic to run
migration operation tests against multiple backends in one run, including
third-party backends not included within Alembic itself.
Third party dialects and extensions are also encouraged to standardize
on SQLAlchemy's test suite as a basis; see the file README.dialects.rst
for background on building out from SQLAlchemy's test platform.
concurrent ability to return connections from the pool means that the
"first_connect" event is now no longer synchronized either, thus leading
to dialect mis-configurations under even minimal concurrency situations.
SQLite allows column types that aren't technically understood in sqlite
by using 'data affinity', which is an algorithm for converting column
types in to some sort of useful type that can be stored and retrieved
from the db. Unfortunatly, this breaks reflection since we (previously)
expected a sqlite db to reflect column types that we permit in the
`ischema_names` for that dialect.
This patch changes the logic for 'unknown' column types during
reflection to instead run through SQLite's data affinity algorithm, and
assigns appropriate types from that.
It also expands the matching for column type to include column types
with spaces (strongly discouraged but allowed by sqlite) and also
completely empty column types (in which case the NullType is assigned,
which sqlite will treat as a Blob - or rather, Blob is treated as
NullType). These changes mean that SQLite will never raise an error for
an unknown type during reflection - there will always be some 'useful'
type returned, which follows the spirit of SQLite (accomodation before
sanity!).
applied to :class:`.Constraint` and :class:`.Index` objects. Based
on a recipe in the wiki, the new feature uses schema-events to set up
names as various schema objects are associated with each other. The
events then expose a configuration system through a new argument
:paramref:`.MetaData.naming_convention`. This system allows production
of both simple and custom naming schemes for constraints and indexes
on a per-:class:`.MetaData` basis. [ticket:2923]
commit 7e65e52c086652de3dd3303c723f98f09af54db8
Author: Mike Bayer <mike_mp@zzzcomputing.com>
Date: Sat Feb 1 15:09:04 2014 -0500
- first pass at new naming approach
reflection now updates the PKC in place.
- support the use case of the empty PrimaryKeyConstraint in order to specify
constraint options; the columns marked as primary_key=True will now be gathered
into the columns collection, rather than being ignored. [ticket:2910]
- add validation such that column specification should only take place
in the PrimaryKeyConstraint directly, or by using primary_key=True flags;
if both are present, they have to match exactly, otherwise the condition is
assumed to be ambiguous, and a warning is emitted; the old behavior of
using the PKC columns only is maintained.
type such as "charset" and "collation". While MySQL wants all character-
based CAST calls to use the CHAR type, we now create a real CHAR
object at CAST time and copy over all the parameters it has, so that
an expression like ``cast(x, mysql.TEXT(charset='utf8'))`` will
render ``CAST(t.col AS CHAR CHARACTER SET utf8)``.
- Added new "unicode returns" detection to the MySQL dialect and
to the default dialect system overall, such that any dialect
can add extra "tests" to the on-first-connect "does this DBAPI
return unicode directly?" detection. In this case, we are
adding a check specifically against the "utf8" encoding with
an explicit "utf8_bin" collation type (after checking that
this collation is available) to test for some buggy unicode
behavior observed with MySQLdb version 1.2.3. While MySQLdb
has resolved this issue as of 1.2.4, the check here should
guard against regressions. The change also allows the "unicode"
checks to log in the engine logs, which was not previously
the case. [ticket:2906]
_reset_agent, so that it's local to the various begin_impl(),
rollback_impl(), etc. this allows setting/resetting of the flag
to be symmetric.
- don't set _reset_agent if it's not None, don't unset it if it isn't
our own transaction.
- make sure we clean it out in close().
- basically, we're dealing here with pools using "threadlocal" that have a
counter, other various mismatches that the tests bring up
- test for recover() now has to invalidate() the previous connection,
because closing it actually rolls it back (e.g. this test was relying
on the broken behavior).
:class:`.RootTransaction` or :class:`.TwoPhaseTransaction`
with its immediate :class:`._ConnectionFairy` as a "reset handler"
for the span of that transaction, which takes over the task
of calling commit() or rollback() for the "reset on return" behavior
of :class:`.Pool` if the transaction was not otherwise completed.
This resolves the issue that a picky transaction
like that of MySQL two-phase will be
properly closed out when the connection is closed without an
explicit rollback or commit (e.g. no longer raises "XAER_RMFAIL"
in this case - note this only shows up in logging as the exception
is not propagated within pool reset).
This issue would arise e.g. when using an orm
:class:`.Session` with ``twophase`` set, and then
:meth:`.Session.close` is called without an explicit rollback or
commit. The change also has the effect that you will now see
an explicit "ROLLBACK" in the logs when using a :class:`.Session`
object in non-autocommit mode regardless of how that session was
discarded. Thanks to Jeff Dairiki and Laurence Rowe for isolating
the issue here. [ticket:2907]
events including auto-invalidation, which is useful both for tests here as well as
detecting failure conditions within the "reset" or "close" cases.
- rename the argument for PoolEvents.reset() to dbapi_connection and connection_record
to be consistent with everything else.
- add new documentation sections on invalidation, including auto-invalidation
and the invalidation process within the pool.
- add _ConnectionFairy and _ConnectionRecord to the pool documentation. Establish
docs for common _ConnectionFairy/_ConnectionRecord methods and accessors and
have PoolEvents docs refer to _ConnectionRecord,
since it is passed to all events. Rename a few _ConnectionFairy methods that are actually
private to pool such as _checkout(), _checkin() and _checkout_existing(); there should not
be any external code calling these
we will be able to parse dialect-specific arguments from string
configuration dictionaries. Dialect classes can now provide their
own list of parameter types and string-conversion routines.
The feature is not yet used by the built-in dialects, however.
[ticket:2875]
of dbapi.Error (such as ``TypeError``, ``NotImplementedError``, etc.)
will propagate the exception unchanged. Previously,
the error handling specific to the ``connect()`` routine would both
inappropriately run the exception through the dialect's
:meth:`.Dialect.is_disconnect` routine as well as wrap it in
a :class:`sqlalchemy.exc.DBAPIError`. It is now propagated unchanged
in the same way as occurs within the execute process. [ticket:2881]
- add tests for this in test_parseconnect, but also add tests in test_execute
to ensure the execute() behavior as well
attempts when an existing connection attempt is blocking. Previously,
the production of new connections was serialized within the block
that monitored overflow; the overflow counter is now altered within
it's own critical section outside of the connection process itself.
[ticket:2880]
connection to be available, such that for a connection pool
with no timeout specified, it will every half a second break out of
the wait to check for the so-called "abort" flag, which allows the
waiter to break out in case the whole connection pool was dumped;
normally the waiter should break out due to a notify_all() but it's
possible this notify_all() is missed in very slim cases.
This is an extension of logic first introduced in 0.8.0, and the
issue has only been observed occasionally in stress tests.
"@", or "/" must be encoded." - so re-apply encoding to both password
and username, don't encode spaces as plus signs, don't encode any chars
outside of :, @, / on stringification - but we still parse for any
%XX character (is that right?)
:func:`.make_url` function **no longer URL encode the password**.
Database passwords that include characters like spaces, plus signs
and anything else should now represent these characters directly,
without any URL escaping. [ticket:2873]
when a pre-DBAPI :class:`.StatementError` were raised within
:meth:`.Connection.execute`, causing encoding errors for
non-ASCII statements. The stringification now remains within
Python unicode thus avoiding encoding errors. [ticket:2871]
mike bayer