The error message that is emitted when a URL cannot be parsed no longer
includes the URL itself within the error message.
Fixes: #12579
Change-Id: Icd17bd4fe0930036662b6a4fe0264cb13df04ba7
Adjusted URL parsing and stringification to apply url quoting to the
"database" portion of the URL. This allows a URL where the "database"
portion includes special characters such as question marks to be
accommodated.
Fixes: #11234
Change-Id: If868c96969b70f1090f0b474403d22fd3a2cc529
manually update the files to remove literal string concat on the same line,
since black does not seem to be making progress in handling these
Change-Id: I3c651374c5f3db5b8bc0c700328d67ca03743b7b
Fixed URL-encoding of the username and password components of
:class:`.engine.URL` objects when converting them to string using the
:meth:`_engine.URL.render_as_string` method, by using Python standard
library ``urllib.parse.quote`` while allowing for plus signs and spaces to
remain unchanged as supported by SQLAlchemy's non-standard URL parsing,
rather than the legacy home-grown routine from many years ago. Pull request
courtesy of Xavier NUNN.
Fixes: #10662Closes: #10726
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10726
Pull-request-sha: 82219041b8
Change-Id: Iedca4929579d4d26ef8cce083252dcd1e476286b
Fixed URL-encoding of the username and password components of
:class:`.engine.URL` objects when converting them to string using the
:meth:`_engine.URL.render_as_string` method, by using Python standard
library ``urllib.parse.quote_plus``, rather than the legacy home-grown
routine from many years ago. Pull request courtesy of Xavier NUNN.
Fixes: #10662Closes: #10726
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10726
Pull-request-sha: 82219041b8
Change-Id: I90b7a9f4dfdb719082b4b178ad4e009a8531a18e
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [X] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
Fixes#10079
**Have a nice day!**
Closes: #10080
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/10080
Pull-request-sha: 133c7d6c35
Change-Id: Ie3d998385743680756bc10fbb4f4227669d57648
Fixed a bug that prevented use of :attr:`_engine.URL.normalized_query` in
SQLAlchemy v2.
Fixes: #9682
Change-Id: I2704154af34f438b4cbb290602fc936c1184c074
Added :func:`_sa.create_pool_from_url` and
:func:`_asyncio.create_async_pool_from_url` to create
a :class:`_pool.Pool` instance from an input url passed as string
or :class:`_sa.URL`.
Fixes: #9613
Change-Id: Icd8aa3f2849e6fd1bc5341114f3ef8d216a2c543
For improved security, the :class:`_url.URL` object will now use password
obfuscation by default when ``str(url)`` is called. To stringify a URL with
cleartext password, the :meth:`_url.URL.render_as_string` may be used,
passing the :paramref:`_url.URL.render_as_string.hide_password` parameter
as ``False``. Thanks to our contributors for this pull request.
Fixes: #8567Closes: #8563
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/8563
Pull-request-sha: d1f1127f75
Change-Id: If756c8073ff99ac83876d9833c8fe1d7c76211f9
All modules in sqlalchemy.engine are strictly
typed with the exception of cursor, default, and
reflection. cursor and default pass with non-strict
typing, reflection is waiting on the multi-reflection
refactor.
Behavioral changes:
* create_connect_args() methods return a tuple of list,
dict, rather than a list of list, dict
* removed allow_chars parameter from
pyodbc connector ._get_server_version_info()
method
* the parameter list passed to do_executemany is now
a list in all cases. previously, this was being run
through dialect.execute_sequence_format, which
defaults to tuple and was only intended for individual
tuple params.
* broke up dialect.dbapi into dialect.import_dbapi
class method and dialect.dbapi module object. added
a deprecation path for legacy dialects. it's not
really feasible to type a single attr as a classmethod
vs. module type. The "type_compiler" attribute also
has this problem with greater ability to work around,
left that one for now.
* lots of constants changing to be Enum, so that we can
type them. for fixed tuple-position constants in
cursor.py / compiler.py (which are used to avoid the
speed overhead of namedtuple), using Literal[value]
which seems to work well
* some tightening up in Row regarding __getitem__, which
we can do since we are on full 2.0 style result use
* altered the set_connection_execution_options and
set_engine_execution_options event flows so that the
dictionary of options may be mutated within the event
hook, where it will then take effect as the actual
options used. Previously, changing the dict would
be silently ignored which seems counter-intuitive
and not very useful.
* A lot of DefaultDialect/DefaultExecutionContext
methods and attributes, including underscored ones, move
to interfaces. This is not fully ideal as it means
the Dialect/ExecutionContext interfaces aren't publicly
subclassable directly, but their current purpose
is more of documentation for dialect authors who should
(and certainly are) still be subclassing the DefaultXYZ
versions in all cases
Overall, Result was the most extremely difficult class
hierarchy to type here as this hierarchy passes through
largely amorphous "row" datatypes throughout, which
can in fact by all kinds of different things, like
raw DBAPI rows, or Row objects, or "scalar"/Any, but
at the same time these types have meaning so I tried still
maintaining some level of semantic markings for these,
it highlights how complex Result is now, as it's trying
to be extremely efficient and inlined while also being
very open-ended and extensible.
Change-Id: I98b75c0c09eab5355fc7a33ba41dd9874274f12a
introduces:
1. new mapped_column() helper
2. DeclarativeBase helper
3. declared_attr has been re-typed
4. rework of Mapped[] to return InstrumentedAtribute for
class get, so works without Mapped itself having expression
methods
5. ORM constructs now generic on [_T]
also includes some early typing work, most of which will
be in later commits:
1. URL and History become typing.NamedTuple
2. come up with type-checking friendly way of type
checking cy extensions, where type checking will be applied
to the py versions, just needed to come up with a succinct
conditional pattern for the imports
References: #6810
References: #7535
References: #7562
Change-Id: Ie5d9a44631626c021d130ca4ce395aba623c71fb
<!-- Provide a general summary of your proposed changes in the Title field above -->
### Description
<!-- Describe your changes in detail -->
Black's `target-version` was still set to `['py27', 'py36']`. Set it to `[py37]` instead.
Also update Black and other pre-commit hooks and re-format with Black.
### Checklist
<!-- go over following points. check them with an `x` if they do apply, (they turn into clickable checkboxes once the PR is submitted, so no need to do everything at once)
-->
This pull request is:
- [ ] A documentation / typographical error fix
- Good to go, no issue or tests are needed
- [ ] A short code fix
- please include the issue number, and create an issue if none exists, which
must include a complete example of the issue. one line code fixes without an
issue and demonstration will not be accepted.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests. one line code fixes without tests will not be accepted.
- [ ] A new feature implementation
- please include the issue number, and create an issue if none exists, which must
include a complete example of how the feature would look.
- Please include: `Fixes: #<issue number>` in the commit message
- please include tests.
**Have a nice day!**
Closes: #7536
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7536
Pull-request-sha: b3aedf5570
Change-Id: I8be85636fd2c9449b07a8626050c8bd35bd119d5
Added support for ``copy()`` and ``deepcopy()`` to the :class:`_url.URL`
class. Pull request courtesy Tom Ritchford.
Fixes: #7400Closes: #7401
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/7401
Pull-request-sha: a2c1b8992f
Change-Id: I55977338b2655a7d4f733ae786d31e589185e9ca
Fixes: #6960
Even though a default driver still exists for
each dialect, remove most usages of `dialect://`
to encourage users to explicitly specify
`dialect+driver://`
Change-Id: I0ad42167582df509138fca64996bbb53e379b1af
The major action here is to lift and move future.Connection
and future.Engine fully into sqlalchemy.engine.base. This
removes lots of engine concepts, including:
* autocommit
* Connection running without a transaction, autobegin
is now present in all cases
* most "autorollback" is obsolete
* Core-level subtransactions (i.e. MarkerTransaction)
* "branched" connections, copies of connections
* execution_options() returns self, not a new connection
* old argument formats, distill_params(), simplifies calling
scheme between engine methods
* before/after_execute() events (oriented towards compiled constructs)
don't emit for exec_driver_sql(). before/after_cursor_execute()
is still included for this
* old helper methods superseded by context managers, connection.transaction(),
engine.transaction() engine.run_callable()
* ancient engine-level reflection methods has_table(), table_names()
* sqlalchemy.testing.engines.proxying_engine
References: #7257
Change-Id: Ib20ed816642d873b84221378a9ec34480e01e82c
Fixed issue where the deprecation warning for the :class:`.URL` constructor
which indicates that the :meth:`.URL.create` method should be used would
not emit if a full positional argument list of seven arguments were passed;
additionally, validation of URL arguments will now occur if the constructor
is called in this way, which was being skipped previously.
Fixes: #7130
Change-Id: I8c8491d8aa7774afaf67c22b4f8e9859f780f2d9
Ensure that ``str()`` is called on the an ``URL.password`` argument,
allowing usage of objects that implement the ``__str__()`` method
as password attributes.
Also clarified that one such object is not appropriate to dynamically
change the password.
Fixes: #6958
Change-Id: Id0690990a64b9e0935537b7b8f5a73efe6a9e3dc
Fixed issue in ``URL`` where validation of "drivername" would not
appropriately respond to the ``None`` value where a string were expected.
Fixes: #6983
Change-Id: If546c373a60533779595a9e393ea9a59a9b8a96f
The fix for pysqlcipher released in version 1.4.3 🎫`5848` was
unfortunately non-working, in that the new ``on_connect_url`` hook was
erroneously not receiving a ``URL`` object under normal usage of
:func:`_sa.create_engine` and instead received a string that was unhandled;
the test suite failed to fully set up the actual conditions under which
this hook is called. This has been fixed.
Fixes: #6586
Change-Id: I3bf738daec35877a10fdad740f08dca9e7420829
Fixed a long-standing issue with :class:`.URL` where query parameters
following the question mark would not be parsed correctly if the URL did
not contain a database portion with a backslash.
Fixed issue where an ``@`` sign in the database portion of a URL would not
be interpreted correctly if the URL also had a username:password section.
Fixes: #6329Fixes: #6482
Change-Id: I6cb6478affa49b618335b947a74e64090657a98c
Added the "future" keyword to the list of words that are known by the
:func:`_sa.engine_from_config` function, so that the values "true" and
"false" may be configured as "boolean" values when using a key such
as ``sqlalchemy.future = true`` or ``sqlalchemy.future = false``.
Change-Id: Ib4bba748497cc68e4c913dde54c23a4bb08b4deb
It's better, the majority of these changes look more readable to me.
also found some docstrings that had formatting / quoting issues.
Change-Id: I582a45fde3a5648b2f36bab96bad56881321899b
it's not really correct that URL is mutable and doesn't do
any argument checking. propose replacing it with an immutable
named tuple with rich copy-and-mutate methods.
At the moment this makes a hard change to the CreateEnginePlugin
docs that previously recommended url.query.pop(). I can't find
any plugins on github other than my own that are using this
feature, so see if we can just make a hard change on this one.
Fixes: #5526
Change-Id: I28a0a471d80792fa8c28f4fa573d6352966a4a79
Fixed issue in :class:`.URL` object where stringifying the object
would not URL encode special characters, preventing the URL from being
re-consumable as a real URL. Pull request courtesy Miguel Grinberg.
Fixes: #5341Closes: #5342
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/5342
Pull-request-sha: 362ca33983
Change-Id: Ief6218122d1ec0c70479eb1a90e1c16433801924
Comparing two objects of :class:`.URL` using ``__eq__()`` did not take port
number into consideration, two objects differing only by port number were
considered equal. Port comparison is now added in ``__eq__()`` method of
:class:`.URL`, objects differing by port number are now not equal.
Additionally, ``__ne__()`` was not implemented for :class:`.URL` which
caused unexpected result when ``!=`` was used in Python2, since there are no
implied relationships among the comparison operators in Python2.
Fixes: #4406Closes: #4515
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4515
Pull-request-sha: 0f15b805f0
Change-Id: Iba7d224f1282dc3f4b884d1a746f2d46669f551e
A large change throughout the library has ensured that all objects, parameters,
and behaviors which have been noted as deprecated or legacy now emit
``DeprecationWarning`` warnings when invoked. As the Python 3 interpreter now
defaults to displaying deprecation warnings, as well as that modern test suites
based on tools like tox and pytest tend to display deprecation warnings,
this change should make it easier to note what API features are obsolete.
See the notes added to the changelog and migration notes for further
details.
Fixes: #4393
Change-Id: If0ea11a1fc24f9a8029352eeadfc49a7a54c0a1b
Applied on top of a pure run of black -l 79 in
I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9, this set of changes
resolves all remaining flake8 conditions for those codes
we have enabled in setup.cfg.
Included are resolutions for all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I4f72d3ba1380dd601610ff80b8fb06a2aff8b0fe
This is a straight reformat run using black as is, with no edits
applied at all.
The black run will format code consistently, however in
some cases that are prevalent in SQLAlchemy code it produces
too-long lines. The too-long lines will be resolved in the
following commit that will resolve all remaining flake8 issues
including shadowed builtins, long lines, import order, unused
imports, duplicate imports, and docstring issues.
Change-Id: I7eda77fed3d8e73df84b3651fd6cfcfe858d4dc9
The :class:`.URL` object now allows query keys to be specified multiple
times where their values will be joined into a list. This is to support
the plugins feature documented at :class:`.CreateEnginePlugin` which
documents that "plugin" can be passed multiple times. Additionally, the
plugin names can be passed to :func:`.create_engine` outside of the URL
using the new :paramref:`.create_engine.plugins` parameter.
Change-Id: Ifc48ad120bd6c6204eda567492caf79832aeeaa5
Fixes: #4170
The "password" attribute of the :class:`.url.URL` object can now be
any user-defined or user-subclassed string object that responds to the
Python ``str()`` builtin. The object passed will be maintained as the
datamember :attr:`.url.URL.password_original` and will be consulted
when the :attr:`.url.URL.password` attribute is read to produce the
string value.
Change-Id: I91d101c3b10e135ae7e4de60a5104b51776db84f
Fixes: #4089
- add a connect=True key to connection record to support
pre-loading of _ConnectionRecord objects
- ensure _ConnectionRecord.close() leaves the record in a good
state for reopening
- add _ConnectionRecord.record_info for persistent storage
- add "in_use" accessor based on fairy_ref being present or not
- allow for the exclusions system and SuiteRequirements to be
usable without the full plugin_base setup.
- move some Python-env requirements to the importable
requirements.py module.
- allow starttime to be queried
- add additional events for engine plugins
- have "dialect" be a first-class parameter to the pool,
ensure the engine strategy supplies it up front
Change-Id: Ibf549f7a1766e49d335cd6f5e26bacfaef9a8229
be stated in the query string for a URL. Custom plugins can
be written which will be given the chance up front to alter and/or
consume the engine's URL and keyword arguments, and then at engine
create time will be given the engine itself to allow additional
modifications or event registration. Plugins are written as a
subclass of :class:`.CreateEnginePlugin`; see that class for
details.
fixes#3536
:func:`.engine_from_config` were not being parsed correctly;
these included ``pool_threadlocal`` and the psycopg2 argument
``use_native_unicode``. fixes#3435
- add legacy_schema_aliasing config parsing for mssql
- move use_native_unicode config arg to the psycopg2 dialect
functionality. Added a new "soft invalidate" feature to the
connection pool at the level of the checked out connection wrapper
as well as the :class:`._ConnectionRecord`. This works similarly
to a modern pool invalidation in that connections aren't actively
closed, but are recycled only on next checkout; this is essentially
a per-connection version of that feature. A new event
:class:`.PoolEvents.soft_invalidate` is added to complement it.
fixes#3379
- Added new flag
:attr:`.ExceptionContext.invalidate_pool_on_disconnect`.
Allows an error handler within :meth:`.ConnectionEvents.handle_error`
to maintain a "disconnect" condition, but to handle calling invalidate
on individual connections in a specific manner within the event.
- Added new event :class:`.DialectEvents.do_connect`, which allows
interception / replacement of when the :meth:`.Dialect.connect`
hook is called to create a DBAPI connection. Also added
dialect plugin hooks :meth:`.Dialect.get_dialect_cls` and
:meth:`.Dialect.engine_created` which allow external plugins to
add events to existing dialects using entry points.
fixes#3355
:paramref:`.Pool.reset_on_return` parameter as a synonym for ``None``,
so that string values can be used for all settings, allowing
.ini file utilities like :func:`.engine_from_config` to be usable
without issue.
fixes#3375
to it, so that we say we will do the wrapping just once right here
in _execute_context() / _execute_default(). An adjustment is made
to _handle_dbapi_error() to not assume self.__connection in case
we are already in an invalidated state
further adjustment to
0639c199a5, 41e7253dee, #3266
take effect in all engine connection use cases, including
when user-custom connect routines are used via the
:paramref:`.create_engine.creator` parameter, as well as when
the :class:`.Connection` encounters a connection error on
revalidation.
fixes#3266
we will be able to parse dialect-specific arguments from string
configuration dictionaries. Dialect classes can now provide their
own list of parameter types and string-conversion routines.
The feature is not yet used by the built-in dialects, however.
[ticket:2875]
Mike Bayer