Changed the ``repr()`` of the :class:`.quoted_name` construct to use
regular string repr() under Python 3, rather than running it through
"backslashreplace" escaping, which can be misleading.
Modified the approach of "name normalization" for the Oracle and Firebird
dialects, which converts from the UPPERCASE-as-case-insensitive convention
of these dialects into lowercase-as-case-insensitive for SQLAlchemy, to not
automatically apply the :class:`.quoted_name` construct to a name that
matches itself under upper or lower case conversion, as is the case for
many non-european characters. All names used within metadata structures
are converted to :class:`.quoted_name` objects in any case; the change
here would only affect the output of some inspection functions.
Moved name normalize to be under default dialect, added test coverage
in test/sql/test_quote.py
Fixes: #4931
Change-Id: Ic121b20e07249824710a54423e321d94a425362f
(cherry picked from commit f9000e2a38)
When mapping to a construct like OUTER JOIN, an UPDATE from the
ORM expects that all involved tables have a row already present;
document this as well as a potential workaround.
Fixes: #4927
Change-Id: Ie99f9f53cea33d9df2513f384a7c68676b197fb7
(cherry picked from commit 2cae55269b)
Added dialect-level flag ``encoding_errors`` to the cx_Oracle dialect,
which can be specified as part of :func:`.create_engine`. This is passed
to SQLAlchemy's unicode decoding converter under Python 2, and to
cx_Oracle's ``cursor.var()`` object as the ``encodingErrors`` parameter
under Python 3, for the very unusual case that broken encodings are present
in the target database which cannot be fetched unless error handling is
relaxed. The value is ultimately one of the Python "encoding errors"
parameters passed to ``decode()``.
Closes: #4801Fixes: #4799
Change-Id: I1d542ba367bcd187347c54db1fee815f7890e71c
(cherry picked from commit d76cb72135)
Fixed an issue in the :meth:`.Engine.table_names` method where it would
feed the dialect's default schema name back into the dialect level table
function, which in the case of SQL Server would interpret it as a
dot-tokenized schema name as viewed by the mssql dialect, which would
cause the method to fail in the case where the database username actually
had a dot inside of it. In 1.3, this method is still used by the
:meth:`.MetaData.reflect` function so is a prominent codepath. In 1.4,
which is the current master development branch, this issue doesn't exist,
both because :meth:`.MetaData.reflect` isn't using this method nor does the
method pass the default schema name explicitly. The fix nonetheless
guards against the default server name value returned by the dialect from
being interpreted as dot-tokenized name under any circumstances by
wrapping it in quoted_name().
Fixes: #4923
Change-Id: I821bd38ed89b767eaca0bdffee7f8ba3baf82560
(cherry picked from commit f50c6a04067acf2cd2fc5e42d5acaa9206d9a078)
The doc edits in e45878bf4f
triggered a doctest and a pep8 failure.
Change-Id: I41c6cba9bab2d3721d9c6280be47eec32f4b4736
(cherry picked from commit d25c03310f)
sphinx is not generating the docs for func. cross-copy
__doc__ and also add more links.
Fixes: #4922
Change-Id: I5512111d726b6fcf9821be730c9e29adc73c95cb
(cherry picked from commit e45878bf4f)
The :class:`.sqltypes.NCHAR` datatype will now bind to the
``cx_Oracle.FIXED_NCHAR`` DBAPI data bindings when used in a bound
parameter, which supplies proper comparison behavior against a
variable-length string. Previously, the :class:`.sqltypes.NCHAR` datatype
would bind to ``cx_oracle.NCHAR`` which is not fixed length; the
:class:`.sqltypes.CHAR` datatype already binds to ``cx_Oracle.FIXED_CHAR``
so it is now consistent that :class:`.sqltypes.NCHAR` binds to
``cx_Oracle.FIXED_NCHAR``.
Fixes: #4913
Change-Id: I5bb111f2e06bbdd525bc5f716579baad31bbb3db
(cherry picked from commit e4a5da1c5a)
Fixed bug where a table that would have a column label overlap with a plain
column name, such as "foo.id AS foo_id" vs. "foo.foo_id", would prematurely
generate the ``._label`` attribute for a column before this overlap could
be detected due to the use of the ``index=True`` or ``unique=True`` flag on
the column in conjunction with the default naming convention of
``"column_0_label"``. This would then lead to failures when ``._label``
were used later to generate a bound parameter name, in particular those
used by the ORM when generating the WHERE clause for an UPDATE statement.
The issue has been fixed by using an alternate ``._label`` accessor for DDL
generation that does not affect the state of the :class:`.Column`. The
accessor also bypasses the key-deduplication step as it is not necessary
for DDL, the naming is now consistently ``"<tablename>_<columnname>"``
without any subsequent numeric symbols when used in DDL.
Fixes: #4911
Change-Id: Iabf5fd3250738d800d6e41a2a3a27a7ce2405e7d
(cherry picked from commit 41dc71ad2f)
Fixed bug where parameter repr as used in logging and error reporting needs
additional context in order to distinguish between a list of parameters for
a single statement and a list of parameter lists, as the "list of lists"
structure could also indicate a single parameter list where the first
parameter itself is a list, such as for an array parameter. The
engine/connection now passes in an additional boolean indicating how the
parameters should be considered. The only SQLAlchemy backend that expects
arrays as parameters is that of psycopg2 which uses pyformat parameters,
so this issue has not been too apparent, however as other drivers that use
positional gain more features it is important that this be supported. It
also eliminates the need for the parameter repr function to guess based on
the parameter structure passed.
Fixes: #4902
Change-Id: I086246ee0eb51484adbefd83e07295fa56576c5f
(cherry picked from commit 9488480abe)
Fixed regression in Oracle dialect that was inadvertently using max
identifier length of 128 characters on Oracle server 12.2 and greater even
though the stated contract for the remainder of the 1.3 series is that
this value stays at 30 until version SQLAlchemy 1.4. Also repaired issues
with the retrieval of the "compatibility" version, and removed the warning
emitted when the "v$parameter" view was not accessible as this was causing
user confusion.
Fixes: #4898
Change-Id: Ieb7b3e093610896c5aa12d0789b63262e0ecf9d8
Fixed bug in SQL Server dialect with new "max_identifier_length" feature
where the mssql dialect already featured this flag, and the implementation
did not accommodate for the new initialization hook correctly.
Fixes: #4857
Change-Id: I96a9c6ca9549d8f6fb167c0333f684e8d922a3bf
(cherry picked from commit 2ba70aba90)
A warning is emitted for a condition in which the :class:`.Session` may
implicitly swap an object out of the identity map for another one with the
same primary key, detaching the old one, which can be an observed result of
load operations which occur within the :meth:`.SessionEvents.after_flush`
hook. The warning is intended to notify the user that some special
condition has caused this to happen and that the previous object may not be
in the expected state.
Fixes: #4890
Change-Id: Ide11c6b9f21ca67ff5a96266c521d0c56fd6af8d
(cherry picked from commit 4aa43ecbd7)
Restored adding cx_Oracle.DATETIME to the setinputsizes() call when a
SQLAlchemy :class:`.Date`, :class:`.DateTime` or :class:`.Time` datatype is
used, so that in the case where a bound parameter is passed as NULL
in some complex queries (in particular this happens with some lazy load
situations), the type is still present. This was removed
in the 1.2 series for arbitrary reasons.
Also adds a suite test for this generic situation.
What's not clear is that do we really need setinputsizes() for all
datatypes if we are supporting NULL in bound parameters.
Fixes: #4886
Change-Id: If99215c31861f9ea6f60a30d47f2f320adc4797f
(cherry picked from commit e21afbcd8c)
Added identifier quoting to the schema name applied to the "use" statement
which is invoked when a SQL Server multipart schema name is used within a
:class:`.Table` that is being reflected, as well as for :class:`.Inspector`
methods such as :meth:`.Inspector.get_table_names`; this accommodates for
special characters or spaces in the database name. Additionally, the "use"
statement is not emitted if the current database matches the target owner
database name being passed.
Fixes: #4883
Change-Id: I84419730e94aac3a88d331ad8c24d10aabbc34af
(cherry picked from commit 66a7befa0c)
Added new :func:`.create_engine` parameter
:paramref:`.create_engine.max_identifier_length`. This overrides the
dialect-coded "max identifier length" in order to accommodate for databases
that have recently changed this length and the SQLAlchemy dialect has
not yet been adjusted to detect for that version. This parameter interacts
with the existing :paramref:`.create_engine.label_length` parameter in that
it establishes the maximum (and default) value for anonymously generated
labels.
The Oracle dialect now emits a warning if Oracle version 12.2 or greater is
used, and the :paramref:`.create_engine.max_identifier_length` parameter is
not set. The version in this specific case defaults to that of the
"compatibility" version set in the Oracle server configuration, not the
actual server version. In version 1.4, the default max_identifier_length
for 12.2 or greater will move to 128 characters. In order to maintain
forwards compatibility, applications should set
:paramref:`.create_engine.max_identifier_length` to 30 in order to maintain
the same length behavior, or to 128 in order to test the upcoming behavior.
This length determines among other things how generated constraint names
are truncated for statements like ``CREATE CONSTRAINT`` and ``DROP
CONSTRAINT``, which means a the new length may produce a name-mismatch
against a name that was generated with the old length, impacting database
migrations.
Fixes: #4857
Change-Id: Ib62efb00c6180c375869029b57353d90385d7950
(cherry picked from commit 88761b8b0b0cfa67cdd6a4913e3a0ea5cba93cbb)
Passing a plain string expression to :meth:`.Session.query` is deprecated,
as all string coercions were removed in 🎫`4481` and this one should
have been included. The :func:`.literal_column` function may be used to
produce a textual column expression.
Note this changeset is in the 1.3 branch only. The 1.4 (currently master)
branch will already have removed this behavior.
Fixes: #4873
Change-Id: I20067a6707c7e2f028c67b1fbf0e8e4d9d136233
Fixed regression in selectinload loader strategy caused by #4775 (released
in version 1.3.6) where a many-to-one attribute of None would no longer be
populated by the loader. While this was usually not noticeable due to the
lazyloader populating None upon get, it would lead to a detached instance
error if the object were detached.
Fixes: #4872
Change-Id: I18466841683111643810ebc4331376df38c4767b
(cherry picked from commit 9f3539b174)
Added support for sqlite "URI" connections, which allow for sqlite-specific
flags to be passed in the query string such as "read only" for Python
sqlite3 drivers that support this.
Fixes: #4863
Change-Id: I7740b55ee8f2ede72a5c49ee94a7540e4d0250f2
(cherry picked from commit adb495503d)
As we are going to be adding a lot of new features to the visitors
module which may impact end-user custom constructs, start documenting
the package for now.
Change-Id: Ibae471c2cb861f6280f601b7b2382d61968825cd
(cherry picked from commit ed02d924dd)
Added an explicit error message for the case when objects passed to
:class:`.Table` are not :class:`.SchemaItem` objects, rather than resolving
to an attribute error.
Fixes: #4847
Change-Id: I4dcdcee86b64c85ccf12e2ddc3d638563d307991
(cherry picked from commit 7afcb39a96)
Users are frequently confused why count() does not return the same
number as the number of objects returned by all(). While we continue
to want to find a better solution to this problem, in the meantime
this has never been clearly documented. Add an FAQ section with
links from .count() , .all(), ORM tutorial.
Change-Id: I6eff36b686ff6cdd55489036fc48a981bc47d5ee
(cherry picked from commit 75c3950732)
Characters that interfere with "pyformat" or "named" formats in bound
parameters, namely ``%, (, )`` and the space character, as well as a few
other typically undesirable characters, are stripped early for a
:func:`.bindparam` that is using an anonymized name, which is typically
generated automatically from a named column which itself includes these
characters in its name and does not use a ``.key``, so that they do not
interfere either with the SQLAlchemy compiler's use of string formatting or
with the driver-level parsing of the parameter, both of which could be
demonstrated before the fix. The change only applies to anonymized
parameter names that are generated and consumed internally, not end-user
defined names, so the change should have no impact on any existing code.
Applies in particular to the psycopg2 driver which does not otherwise quote
special parameter names, but also strips leading underscores to suit Oracle
(but not yet leading numbers, as some anon parameters are currently
entirely numeric/underscore based); Oracle in any case continues to quote
parameter names that include special characters.
Fixes: #4837
Change-Id: I21cb654c3e4ef786114160b8b4295242720bf3f9
(cherry picked from commit d7aa017d83)
Fixed unit test regression released in 1.3.8 that would cause failure for
Oracle, SQL Server and other non-native ENUM platforms due to new
enumeration tests added as part of 🎫`4285` enum sortability in the
unit of work; the enumerations created constraints that were duplicated on
name.
Fixes: #4285
Change-Id: I34f51e82be9b6bb43b0df8c54bb36822e6eeb73e
(cherry picked from commit 2df613243d)
Added support for the use of an :class:`.Enum` datatype using Python
pep-435 enumeration objects as values for use as a primary key column
mapped by the ORM. As these values are not inherently sortable, as
required by the ORM for primary keys, a new
:attr:`.TypeEngine.sort_key_function` attribute is added to the typing
system which allows any SQL type to implement a sorting for Python objects
of its type which is consulted by the unit of work. The :class:`.Enum`
type then defines this using the database value of a given enumeration.
The sorting scheme can be also be redefined by passing a callable to the
:paramref:`.Enum.sort_key_function` parameter. Pull request courtesy
Nicolas Caniart.
Fixes: #4285Closes: #4816
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4816
Pull-request-sha: 42266b766c
Change-Id: Iadcc16173c1ba26ffac5830db57743a4cb987c55
(cherry picked from commit 75b2518b26)
Added support for reflection of CHECK constraints that include the special
PostgreSQL qualifier "NOT VALID", which can be present for CHECK
constraints that were added to an exsiting table with the directive that
they not be applied to existing data in the table. The PostgreSQL
dictionary for CHECK constraints as returned by
:meth:`.Inspector.get_check_constraints` may include an additional entry
``dialect_options`` which within will contain an entry ``"not_valid":
True`` if this symbol is detected. Pull request courtesy Bill Finn.
Fixes: #4824Closes: #4825
Pull-request: https://github.com/sqlalchemy/sqlalchemy/pull/4825
Pull-request-sha: a0e1ab133c
Change-Id: I78365f50055c95474c92124b85df66c5c80c00c8
(cherry picked from commit 3980a9a455)
Fixed bug where :class:`.Load` objects were not pickleable due to
mapper/relationship state in the internal context dictionary. These
objects are now converted to picklable using similar techniques as that of
other elements within the loader option system that have long been
serializable.
Fixes: #4823
Change-Id: Id2a0d8b640ac475c86d6416ad540671e66d410e5
(cherry picked from commit cd2ccee9d8)
Fixed bug where Postgresql operators such as
:meth:`.postgresql.ARRAY.Comparator.contains` and
:meth:`.postgresql.ARRAY.Comparator.contained_by` would fail to function
correctly for non-integer values when used against a
:class:`.postgresql.array` object, due to an erroneous assert statement.
Fixes: #4822
Change-Id: I886aca4c86dc7d64e9d0dfc1d910a0ae64d775a1
(cherry picked from commit 4b5d162ed0)
Added new parameter :paramref:`.create_engine.hide_parameters` which when
set to True will cause SQL parameters to no longer be logged, nor rendered
in the string representation of a :class:`.StatementError` object.
Fixes: #4815
Change-Id: Ib87f868b6936cf6b42b192644e9d732ec24266c2
(cherry picked from commit 4b321e8a5e)
The changelog for #4793 has a typo in the RST and didn't show up
in the changelog, causing confusion as seen in #4814
Change-Id: I5833097c1576120fbc2ea27c54bbca685ae89f88
References: #4793
References: #4814
(cherry picked from commit 94385b031c)
Describe the link between table reflection, datatype lookups,
and what approaches are needed in the case where in-Python datatypes
are needed in the reflected table metadata.
Fixes: #4812
Change-Id: I68bef2bf472811797d4f5d9a625c6b9bca902f78
(cherry picked from commit a3df16995b)
Mike Bayer