1. ensure provision.py loads dialect implementations when running
reap_dbs.py. Reapers haven't been working since
598f2f7e55 .
2. add some exclusion rules to allow the sqlite_file target to work;
add to tox.
3. add reap dbs target for SQLite, repair SQLite drop_db routine
which also wasn't doing the right thing for memory databases
etc.
4. Fix logging in provision files, as the main provision logger
is the one that's enabled by reap_dbs and maybe others, have all
the provision files use the provision logger.
Fixes: #5180Fixes: #5168
Change-Id: Ibc1b0106394d20f5bcf847f37b09d185f26ac9b5
(cherry picked from commit 08fdf6da228aab54263abe043c21bcebf3513e72)
Observing a SQLite connection/cursor being hung on
test_resultset -> PositionalTextTest -> test_dupe_col_obj.
this uses connectionless execution and the result object
fails to be constructed. When that happens, there is no path
for the cursor or connection to be closed / released.
Recent changes with the exception assertions in #4849 seem to
be causing a cycle to last a little longer than usual which
is exposing this issue for one particular test on SQLite.
As we want to get rid of weakref cleanup, evaluate
why we dont have handle_dbapi_exception for this whole
block, as after_cursor_execute can raise, result construction
can raise, autocommit can raise, close can raise, there
does not seem to be a reason these things should be outside
of the block that gets cleaned up.
Fixes: #5182
Change-Id: I640ac55e8c5f39d287f779fbb5dc0ab727218ca3
(cherry picked from commit 4cd33f18a21d9e33b37ef7163822c327453d1e62)
Applied an explicit "cause" to most if not all internally raised exceptions
that are raised from within an internal exception catch, to avoid
misleading stacktraces that suggest an error within the handling of an
exception. While it would be preferable to suppress the internally caught
exception in the way that the ``__suppress_context__`` attribute would,
there does not as yet seem to be a way to do this without suppressing an
enclosing user constructed context, so for now it exposes the internally
caught exception as the cause so that full information about the context
of the error is maintained.
Fixes: #4849
Change-Id: I55a86b29023675d9e5e49bc7edc5a2dc0bcd4751
(cherry picked from commit 8be0dae77a7e0747f0d0fb4282db4aea7f41e03a)
Added new parameter :paramref:`.create_engine.hide_parameters` which when
set to True will cause SQL parameters to no longer be logged, nor rendered
in the string representation of a :class:`.StatementError` object.
Fixes: #4815
Change-Id: Ib87f868b6936cf6b42b192644e9d732ec24266c2
(cherry picked from commit 4b321e8a5e)
Fixed bug where using reflection function such as :meth:`.MetaData.reflect`
with an :class:`.Engine` object that had execution options applied to it
would fail, as the resulting :class:`.OptionEngine` proxy object failed to
include a ``.engine`` attribute used within the reflection routines.
Fixes: #4754
Change-Id: I6c342af5c6db6fe362b9d25f3f26d6859f62f87a
(cherry picked from commit 3238d953b4)
This is a very useful assertion which prevents unused variables
from being set up allows code to be more readable and sometimes
even more efficient. test suites seem to be where the most
problems are and there do not seem to be documentation examples
that are using this, or at least the linter is not taking effect
within rst blocks.
Change-Id: I2b3341d8dd14da34879d8425838e66a4b9f8e27d
(cherry picked from commit 190e0139e8)
Moved the "rollback" which occurs during dialect initialization so that it
occurs after additional dialect-specific initialize steps, in particular
those of the psycopg2 dialect which would inadvertently leave transactional
state on the first new connection, which could interfere with some
psycopg2-specific APIs which require that no transaction is started. Pull
request courtesy Matthew Wilkes.
Fixes: #4663Closes: #4664
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4664
Pull-request-sha: e544fe671d
Change-Id: If40a15a1679b4eec0b8b8222f678697728009c30
(cherry picked from commit f601791a91)
Revised the formatting for :class:`.StatementError` when stringified. Each
error detail is broken up over multiple newlines instead of spaced out on a
single line. Additionally, the SQL representation now stringifies the SQL
statement rather than using ``repr()``, so that newlines are rendered as is.
Pull request courtesy Nate Clark.
Fixes: #4500Closes: #4501
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4501
Pull-request-sha: 60cc0ee68d
Change-Id: I79d8418b7495e5691c9a56f41e79495c26a967ff
Added accessors for execution options to Core and ORM, via
:meth:`.Query.get_execution_options`,
:meth:`.Connection.get_execution_options`,
:meth:`.Engine.get_execution_options`, and
:meth:`.Executable.get_execution_options`. PR courtesy Daniel Lister.
Fixes: #4406Closes: #4465
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4465
Pull-request-sha: 9674688bb5
Change-Id: I93ba51d7a2d687e255edd6938db15615e56dd237
A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.
See the notes added to the changelog and migration notes for further
details.
Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
Fixed a regression introduced in version 1.2 where a refactor
of the :class:`.SQLAlchemyError` base exception class introduced an
inappropriate coercion of a plain string message into Unicode under
python 2k, which is not handled by the Python interpreter for characters
outside of the platform's encoding (typically ascii). The
:class:`.SQLAlchemyError` class now passes a bytestring through under
Py2K for ``__str__()`` as is the behavior of exception objects in general
under Py2K, does a safe coercion to unicode utf-8 with
backslash fallback for ``__unicode__()``. For Py3K the message is
typically unicode already, but if not is again safe-coerced with utf-8
with backslash fallback for the ``__str__()`` method.
Fixes: #4429
Change-Id: I2289da3f2c45c7d0041fa43d838958f7614defc3
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
Fixed bug where a :class:`.Sequence` would be dropped explicitly before any
:class:`.Table` that refers to it, which breaks in the case when the
sequence is also involved in a server-side default for that table, when
using :meth:`.MetaData.drop_all`. The step which processes sequences
to be dropped via non server-side column default functions is now invoked
after the table itself is dropped.
Change-Id: I185f2cc76d2011ad4dd3ba9bde5d8aef0ec335ae
Fixes: #4300
Fixed a reference leak issue where the values of the parameter dictionary
used in a statement execution would remain referenced by the "compiled
cache", as a result of storing the key view used by Python 3 dictionary
keys(). Pull request courtesy Olivier Grisel.
Change-Id: Icfb0f38111a165780f6dd3e4e3382a03df79ce26
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/441
Fixed regression caused in 1.2.3 due to fix from 🎫`4181` where
the changes to the event system involving :class:`.Engine` and
:class:`.OptionEngine` did not accommodate for event removals, which
would raise an ``AttributeError`` when invoked at the class
level.
Change-Id: I1c9083829d74dd710716d28b0eaca4fa15e86313
Fixes: #4190
Fixed bug where events associated with an :class:`Engine`
at the class level would be doubled when the
:meth:`.Engine.execution_options` method were used. To
achieve this, the semi-private class :class:`.OptionEngine`
no longer accepts events directly at the class level
and will raise an error; the class only propagates class-level
events from its parent :class:`.Engine`. Instance-level
events continue to work as before.
The comments present another way of doing this where we would
copy events from the parent engine at option time rather
than linking the event listeners, but this would be a behavioral
change that adding new events to the parent engine would not
take effect for an already-created OptionEngine.
Change-Id: Id128516f54103fbad9a2210d6571eceb59c8b0cb
Fixes: #4181
Fixed bug in cx_Oracle disconnect detection, used by pre_ping and other
features, where an error could be raised as DatabaseError which includes a
numeric error code; previously we weren't checking in this case for a
disconnect code.
Change-Id: I359bb5ede25a4726ea632b81af83c6391f405ae1
Fixes: #4182
Fixed bug where a SQL-oriented Python-side column default could fail to
be executed properly upon INSERT in the "pre-execute" codepath, if the
SQL itself were an untyped expression, such as plain text. The "pre-
execute" codepath is fairly uncommon however can apply to non-integer
primary key columns with SQL defaults when RETURNING is not used.
Tests exist here to ensure typing is applied to
a typed expression for default, but in the case of
an untyped SQL value, we know the type from the column,
so apply this.
Change-Id: I5d8b391611c137b9f700115a50a2bf5b30abfe94
Fixes: #3923
Added an exception handler that will warn for the "cause" exception on
Py2K when the "autorollback" feature of :class:`.Connection` itself
raises an exception. In Py3K, the two exceptions are naturally reported
by the interpreter as one occurring during the handling of the other.
This is continuing with the series of changes for rollback failure
handling that were last visited as part of 🎫`2696` in 1.0.12.
Change-Id: I600ba455a14ebaea27c6189889181f97c632f179
Fixes: #3946
Fixed bug where in the unusual case of passing a
:class:`.Compiled` object directly to :meth:`.Connection.execute`,
the dialect with which the :class:`.Compiled` object were generated
was not consulted for the paramstyle of the string statement, instead
assuming it would match the dialect-level paramstyle, causing
mismatches to occur.
Change-Id: I114e4db2183fbb75bb7c0b0641f5a161855696ee
Fixes: #3938
Corrects some warnings and adds tox config. Adds DeprecationWarning
to the error category. Large sweep for string literals w/ backslashes
as this is common in docstrings
Co-authored-by: Andrii Soldatenko
Fixes: #3886
Change-Id: Ia7c838dfbbe70b262622ed0803d581edc736e085
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/337
Raise a more descriptive exception / message when ClauseElement
or non-SQLAlchemy objects that are not "executable" are erroneously
passed to ``.execute()``; a new exception ObjectNotExecutableError
is raised consistently in all cases.
Change-Id: I2dd393121e2c7e5b6b9e40286a2f25670876e8e4
Fixes: #3786
count() here is misleading in that it not only
counts from an arbitrary column in the table, it also
does not make accommodations for DISTINCT, JOIN, etc.
as the ORM-level function does. Core should not be
attempting to provide a function like this.
Change-Id: I9916fc51ef744389a92c54660ab08e9695b8afc2
Fixes: #3724
This supports the use case of an application that uses the same set of
:class:`.Table` objects in many schemas, such as schema-per-user.
A new execution option
:paramref:`.Connection.execution_options.schema_translate_map` is
added. fixes#2685
- latest tox doesn't like the {posargs} in the profile rerunner
"max_row_buffer" execution option for BufferedRowResultProxy
- also add documentation, changelog and version notes
- rework the max_row_buffer argument to be interpreted from
the execution options upfront when the BufferedRowResultProxy
is first initialized.
functionality. Added a new "soft invalidate" feature to the
connection pool at the level of the checked out connection wrapper
as well as the :class:`._ConnectionRecord`. This works similarly
to a modern pool invalidation in that connections aren't actively
closed, but are recycled only on next checkout; this is essentially
a per-connection version of that feature. A new event
:class:`.PoolEvents.soft_invalidate` is added to complement it.
fixes#3379
- Added new flag
:attr:`.ExceptionContext.invalidate_pool_on_disconnect`.
Allows an error handler within :meth:`.ConnectionEvents.handle_error`
to maintain a "disconnect" condition, but to handle calling invalidate
on individual connections in a specific manner within the event.
- Added new event :class:`.DialectEvents.do_connect`, which allows
interception / replacement of when the :meth:`.Dialect.connect`
hook is called to create a DBAPI connection. Also added
dialect plugin hooks :meth:`.Dialect.get_dialect_cls` and
:meth:`.Dialect.engine_created` which allow external plugins to
add events to existing dialects using entry points.
fixes#3355
That is, after exhausing all rows using the fetch methods, the
DBAPI cursor is released as before and the object may be safely
discarded, but the fetch methods may continue to be called for which
they will return an end-of-result object (None for fetchone, empty list
for fetchmany and fetchall). Only if :meth:`.ResultProxy.close`
is called explicitly will these methods raise the "result is closed"
error.
fixes#3330fixes#3329
consists mainly of adjusting fixtures to ensure connections are closed
explicitly. psycopg2cffi also handles unicode bind parameter
names differently than psycopg2, and seems to possibly have a little less
control over floating point values at least in one test which is
marked as a "fail", though will see if it runs differently on linux
than osx..
- changelog for psycopg2cffi, fixes#3052
levels; :meth:`.Connection.get_isolation_level`,
:attr:`.Connection.default_isolation_level`.
- enhance documentation inter-linkage between new accessors,
existing isolation_level parameters, as well as in
the dialect-level methods which should be fully covered
by Engine/Connection level APIs now.
and parameters are not displayed if None, reducing confusion for
error messages that weren't related to a statement. The full
module and classname for the DBAPI-level exception is displayed,
making it clear that this is a wrapped DBAPI exception. The
statement and parameters themselves are bounded within a bracketed
sections to better isolate them from the error message and from
each other.
fixes#3172
:paramref:`.create_engine.execution_options` or
:meth:`.Engine.update_execution_options` are not passed to the
special :class:`.Connection` used to initialize the dialect
within the "first connect" event; dialects will usually
perform their own queries in this phase, and none of the
current available options should be applied here. In
particular, the "autocommit" option was causing an attempt to
autocommit within this initial connect which would fail with
an AttributeError due to the non-standard state of the
:class:`.Connection`.
fixes#3200
when you call :meth:`.Connection.connect`, would not share transaction
status with the parent. The architecture of branching has been tweaked
a bit so that the branched connection defers to the parent for
all transactional status and operations.
fixes#3190
as that of 🎫`3199`, when the ``named=True`` parameter
would be used. Some events would fail to register, and others
would not invoke the event arguments correctly, generally in the
case of when an event was "wrapped" for adaption in some other way.
The "named" mechanics have been rearranged to not interfere with
the argument signature expected by internal wrapper functions.
fixes#3197
Gord Thompson