Commit Graph

234 Commits

Author SHA1 Message Date
Mike Bayer f4c4f784cd Don't cache savepoint identifiers
Fixed bug in compiler where the string identifier of a savepoint would
be cached in the identifier quoting dictionary; as these identifiers
are arbitrary, a small memory leak could occur if a single
:class:`.Connection` had an unbounded number of savepoints used,
as well as if the savepoint clause constructs were used directly
with an unbounded umber of savepoint names.   The memory leak does
**not** impact the vast majority of cases as normally the
:class:`.Connection`, which renders savepoint names with a simple
counter starting at "1", is used on a per-transaction or
per-fixed-number-of-transactions basis before being discarded.

The savepoint name in virtually all cases does not require quoting
at all, however to support potential third party use cases
the "check for quotes needed" logic is retained, at a small
performance cost.   Uncondtionally quoting the name is another
option, but this would turn the name into a case sensitive name
which runs the risk of poor interactions with existing deployments
that may be looking at these names in other contexts.

Change-Id: I6b53c96abf7fdf1840592bbca5da81347911844c
Fixes: #3931
2017-03-06 17:20:06 -05:00
Mike Bayer fe1cabb88a Performance within instances()
Continuing from Ie43beecf37945b2bb7fff0aaa597a597293daa18,
also observed is the overhead of PathRegsitry memoized token
functions, as these paths are not cached in the case of
long joinedloader chains.  The memoizations here were made
with short paths in mind, and have been replaced with
an inlined straight create of these paths up front, producing
callcounts very similar to 0.8. Combined with the previous
optimizations, 1.1 now runs the "joined eager load of one row"
worst case test in about 40% fewer calls than 0.8 and 60%
fewer than 1.1.5.

Change-Id: Ib5e1c1345a1dd8edfbdb3fed06eb717d4e164d31
Fixes: #3915
2017-02-23 16:56:20 -05:00
Mike Bayer 1a7e8d77ba Memoize AliasedClass, ClauseAdapter objects in joined eager load
Addressed some long unattended performance concerns within the
joined eager loader query construction system.  The use of ad-hoc
:class:`.AliasedClass` objects per query, which produces lots of column
lookup overhead each time, has been replaced with a cached approach
that makes use of a small pool of :class:`.AliasedClass` objects
that are reused between invocations of joined eager loading.
Callcount reduction of SQL query generation for worst-case joined
loader scenarios (lots of joins, lots of columns) is reduced by
approximately 270%.

Change-Id: Ie43beecf37945b2bb7fff0aaa597a597293daa18
Fixes: #3915
2017-02-16 11:31:26 -05:00
Khairi Hafsham 772374735d Make all tests to be PEP8 compliant
tested using pycodestyle version 2.2.0

Fixes: #3885
Change-Id: I5df43adc3aefe318f9eeab72a078247a548ec566
Pull-request: https://github.com/zzzeek/sqlalchemy/pull/343
2017-02-07 11:21:56 -05:00
Mike Bayer 23825a2591 Deprecate Mapper.order_by
This is an old parameter no longer relevant to how SQLAlchemy
works, once the Query object was introduced.  By deprecating it
we establish that we aren't supporting non-working use cases
and that we encourage applications to move off of the use of this
parameter.

Fixes: #3394
Change-Id: I25b9a38142a1537bbcb27d3e8b66a8b265140072
2016-04-19 18:17:45 -04:00
Mike Bayer 2419bfe34c - fix stray connection 2016-01-23 17:31:02 -05:00
Mike Bayer 8712ef2f81 - Added new checks for the common error case of passing mapped classes
or mapped instances into contexts where they are interpreted as
SQL bound parameters; a new exception is raised for this.
fixes #3321
2015-08-22 12:47:13 -04:00
Mike Bayer 92ac4a9f88 - revert this change which was inadvertently included in the
fix for #3451
2015-06-13 20:47:18 -04:00
Mike Bayer 2497d559dd - add changelog for #3451, with 09485d7331 fixes #3451
- also add a bulk_insert_mappings test
2015-06-13 20:15:17 -04:00
Eric Streeper 45876612d2 PEP8 cleanup in /test/aaa_profiling 2015-03-18 22:56:18 -07:00
Mike Bayer fcfa62bd76 - start locking down querying for cols after observing yesterdays
callcount bump due to the slots thing
- rewrite profiles using new technique
2015-03-09 13:07:09 -04:00
Mike Bayer 3eff76c4b0 - add the test_expire_lots test for comparison 2015-02-18 15:50:37 -05:00
Mike Bayer 026449c15f - Fixed a leak which would occur in the unsupported and highly
non-recommended use case of replacing a relationship on a fixed
mapped class many times, referring to an arbitrarily growing number of
target mappers.  A warning is emitted when the old relationship is
replaced, however if the mapping were already used for querying, the
old relationship would still be referenced within some registries.
fixes #3251
2014-11-13 13:17:38 -05:00
Mike Bayer 7c6a45c480 - The :func:~.expression.column and :func:~.expression.table
constructs are now importable from the "from sqlalchemy" namespace,
just like every other Core construct.
- The implicit conversion of strings to :func:`.text` constructs
when passed to most builder methods of :func:`.select` as
well as :class:`.Query` now emits a warning with just the
plain string sent.   The textual conversion still proceeds normally,
however.  The only method that accepts a string without a warning
are the "label reference" methods like order_by(), group_by();
these functions will now at compile time attempt to resolve a single
string argument to a column or label expression present in the
selectable; if none is located, the expression still renders, but
you get the warning again. The rationale here is that the implicit
conversion from string to text is more unexpected than not these days,
and it is better that the user send more direction to the Core / ORM
when passing a raw string as to what direction should be taken.
Core/ORM tutorials have been updated to go more in depth as to how text
is handled.
fixes #2992
2014-09-01 20:19:54 -04:00
Mike Bayer 3c60d3b1ca - A new style of warning can be emitted which will "filter" up to
N occurrences of a parameterized string.   This allows parameterized
warnings that can refer to their arguments to be delivered a fixed
number of times until allowing Python warning filters to squelch them,
and prevents memory from growing unbounded within Python's
warning registries.
fixes #3178
2014-08-31 15:22:00 -04:00
Mike Bayer 51c7005dc5 - clean up zoomark a little and try to get new profiles written 2014-08-30 00:11:32 -04:00
Mike Bayer b577afcb2b - rework profiling, zoomark tests into single tests so that
they can be used under xdist
2014-08-16 19:49:07 -04:00
Mike Bayer ab7c785d96 - turn off the testing reaper here, that's the source of the leaks
when we are running with pydist, and even when we are running without it
in fact...
2014-08-08 13:06:41 -04:00
Mike Bayer 54592942c4 - add support for tags, including include/exclude support.
simplify tox again now that we can exclude tests more easily
2014-07-27 18:46:20 -04:00
Mike Bayer a007c6e616 put a greater variance into this test to prevent sporadic failures 2014-07-21 12:09:12 -04:00
Mike Bayer ef968470c9 imports gone bad 2014-07-10 11:36:57 -04:00
Mike Bayer 1fc440393a - fully flake8 test/aaa_profiling 2014-07-09 18:59:16 -04:00
Matthias Urlichs f341cfbfe1 Documentation fix-up: "its" vs. "it's"
Removed ungrammatical apostrophes from documentation, replacing
"it's" with "its" where appropriate (but in a few cases with "it is"
when that read better).

While doing that, I also fixed a couple of minor typos etc.
as I noticed them.
2014-05-11 16:49:56 +02:00
Alex Gaynor 1caa7fafbd Fix many typos throughout the codebase
Found using: https://github.com/intgr/topy
2014-04-26 13:13:13 -07:00
Mike Bayer b6411ee3cf fixes to get profiling tests working again 2014-03-26 20:43:20 -04:00
Mike Bayer a465f3d031 use integer division here 2014-03-26 16:39:43 -04:00
Mike Bayer dc0d581d5d - rework memusage tests so that it only runs five iterations at a time, if it sees success within
that period, it's done.  memusage tests have become very slow
2014-03-26 15:18:29 -04:00
Mike Bayer 2f150bee28 - rename __multiple__ to __backend__, and apply __backend__ to a large number of tests.
- move out logging tests from test_execute to test_logging
2014-03-24 11:33:53 -04:00
Mike Bayer 30a50cc46a - the wrapped memoized_property here was not working, as the attribute name
didn't match.  use straight memoized_props here for now, add a perf test to check it
2013-11-26 23:24:13 -05:00
Mike Bayer 02f21ffcf3 - The :class:.RowProxy object is now sortable in Python as a regular
tuple is; this is accomplished via ensuring tuple() conversion on
both sides within the ``__eq__()`` method as well as
the addition of a ``__lt__()`` method. [ticket:2848]
2013-11-19 19:29:18 -05:00
Mike Bayer 08a6a8b519 - Removed some now unneeded version checks [ticket:2829] courtesy alex gaynor 2013-09-22 20:35:40 -04:00
Mike Bayer b83a1564f1 Improved support for the cymysql driver, supporting version 0.6.5,
courtesy Hajime Nakagami.
2013-08-17 00:31:57 -04:00
Mike Bayer 4505425a38 - Removal of event listeners is now implemented. The feature is
provided via the :func:`.event.remove` function.
[ticket:2268]
- reorganization of event.py module into a package; with the addition of the
docstring work as well as the new registry for removal, there's a lot more code now.
the package separates concerns and provides a top-level doc for each subsection
of functionality
- the remove feature works by providing the EventKey object which associates
the user-provided arguments to listen() with a global, weak-referencing registry.
This registry stores a collection of _ListenerCollection and _DispatchDescriptor
objects associated with each set of arguments, as well as the wrapped function
which was applied to that collection.  The EventKey can then be recreated for
a removal, all the _ListenerCollection and _DispatchDescriptor objects are located,
and the correct wrapped function is removed from each one.
2013-07-26 14:21:58 -04:00
Mike Bayer 731a4daf63 A performance fix related to the usage of the :func:.defer option
when loading mapped entities.   The function overhead of applying
a per-object deferred callable to an instance at load time was
significantly higher than that of just loading the data from the row
(note that ``defer()`` is meant to reduce DB/network overhead, not
necessarily function call count); the function call overhead is now
less than that of loading data from the column in all cases.  There
is also a reduction in the number of "lazy callable" objects created
per load from N (total deferred values in the result) to 1 (total
number of deferred cols).
[ticket:2778]
2013-07-13 16:29:07 -04:00
Mike Bayer b38a76cd1d - replace most explicitly-named test objects called "Mock..." with
actual mock objects from the mock library.  I'd like to use mock
for new tests so we might as well use it in obvious places.
- use unittest.mock in py3.3
- changelog
- add a note to README.unittests
- add tests_require in setup.py
- have tests import from sqlalchemy.testing.mock
- apply usage of mock to one of the event tests.  we can be using
this approach all over the place.
2013-06-30 18:35:12 -04:00
Mike Bayer 3b4168875d - zoomark tests
- rewrite all profiles, we'll review the diffs to see if anything is too far out
2013-05-26 19:47:13 -04:00
Mike Bayer 6cde27fe91 a pass where we try to squash down as many list()/keys() combinations
as possible
2013-05-26 19:06:13 -04:00
Mike Bayer 4b614b9b35 - the raw 2to3 run
- went through examples/ and cleaned out excess list() calls
2013-04-27 19:53:57 -04:00
Hajime Nakagami af9ed1ff16 testing.crashes on cymysql >= 0.6 2013-04-20 16:33:32 +09:00
Hajime Nakagami a6a311cd1a skip with cymysql 0.6 2013-04-20 15:54:33 +09:00
Mike Bayer a5ede47f12 A major fix to the way in which a select() object produces
labeled columns when apply_labels() is used; this mode
produces a SELECT where each column is labeled as in
<tablename>_<columnname>, to remove column name collisions
for a multiple table select.   The fix is that if two labels
collide when combined with the table name, i.e.
"foo.bar_id" and "foo_bar.id", anonymous aliasing will be
applied to one of the dupes.  This allows the ORM to handle
both columns independently; previously, 0.7
would in some cases silently emit a second SELECT for the
column that was "duped", and in 0.8 an ambiguous column error
would be emitted.   The "keys" applied to the .c. collection
of the select() will also be deduped, so that the "column
being replaced" warning will no longer emit for any select()
that specifies use_labels, though the dupe key will be given
an anonymous label which isn't generally user-friendly.
[ticket:2702]
2013-04-11 16:14:23 -04:00
Mike Bayer c8a80e2130 - remove all compat items that are pre-2.5 (hooray)
- other cleanup
- don't need compat.decimal, that approach never panned out. hopefully
outside libs aren't pulling it in, they shouldn't be
2013-03-09 17:26:16 -05:00
Mike Bayer 576224edd0 - add workaround for sqlite memusage tests, so no longer need to count to 220/skip tests
- Fixed potential memory leak which could occur if an
arbitrary number of :class:`.sessionmaker` objects
were created.   The anonymous subclass created by
the sessionmaker, when dereferenced, would not be garbage
collected due to remaining class-level references from the
event package.  This issue also applies to any custom system
that made use of ad-hoc subclasses in conjunction with
an event dispatcher.  Also in 0.7.10. [ticket:2650]
2013-01-12 17:21:35 -05:00
Mike Bayer b66dad46f3 - refactor of pathing mechanics, to address #2614, #2617
- paths now store Mapper + MapperProperty now instead of string key,
so that the parent mapper for the property is known, supports same-named
properties on multiple subclasses
- the Mapper within the path is now always relevant to the property
to the right of it.  PathRegistry does the translation now, instead
of having all the outside users of PathRegistry worry about it,
to produce a path that is much more consistent.  Paths are now
consistent with mappings in all cases.  Special logic to get at
"with_polymorphic" structures and such added also.
- AliasedClass now has two modes, "use_mapper_path" and regular;
"use_mapper_path" is for all those situations where we put an AliasedClass
in for a plain class internally, and want it to "path" with the
plain mapper.
- The AliasedInsp is now the first class "entity" for an AliasedClass,
and is passed around internally and used as attr._parententity
and such.  it is the AliasedClass analogue for Mapper.
2012-12-01 20:12:23 -05:00
Mike Bayer 0f56aa56c5 need to disable hstore for the "player" here. should really get rid of the recorder/player thing
at this point
2012-11-18 03:01:49 -05:00
Mike Bayer 08f104f7e7 - import fixes 2012-09-29 16:58:00 -04:00
Mike Bayer be40b51cb0 getting everything to pass again 2012-09-27 16:33:54 -04:00
Mike Bayer 20cdc64588 trying different approaches to test layout. in this one, the testing modules
become an externally usable package but still remains within the main sqlalchemy parent package.
in this system, we use kind of an ugly hack to get the noseplugin imported outside of the
"sqlalchemy" package, while still making it available within sqlalchemy for usage by
third party libraries.
2012-09-27 02:37:33 -04:00
Mike Bayer a2468c8a31 - [feature] To complement [ticket:2547], types
can now provide "bind expressions" and
"column expressions" which allow compile-time
injection of SQL expressions into statements
on a per-column or per-bind level.   This is
to suit the use case of a type which needs
to augment bind- and result- behavior at the
SQL level, as opposed to in the Python level.
Allows for schemes like transparent encryption/
decryption, usage of Postgis functions, etc.
[ticket:1534]
- update postgis example fully.
- still need to repair the result map propagation
here to be transparent for cases like "labeled column".
2012-08-17 18:35:25 -04:00
Mike Bayer a4c651420b - this is a refcount test 2012-08-11 17:55:37 -04:00