pyramid_tm is a package which allows Pyramid requests to join
the active transaction as provided by the Python transaction package. (See the documentation
for the transaction package for an
explanation of what “joining the active transaction” means).
Install using pip, e.g. (within a virtualenv):
$ pip install pyramid_tm
pyramid_tm is installed, you must use the
mechanism to include it into your Pyramid project’s configuration. In your
config = Configurator(.....) config.include('pyramid_tm')
Or use the
pyramid.includes configuration setting in your
[app:myapp] pyramid.includes = pyramid_tm
After the package is included, whenever a new request enters the application, a new transaction is associated with that request.
repoze.tm2 middleware is in the WSGI
pyramid_tm becomes inactive.
At the beginning of a request a new transaction is started
request.tm.begin() function. Once the request has
finished all of its works (ie views have finished running), a few checks
- Did some a transaction.doom() cause the transaction to become “doomed”? if so,
- Did an exception occur in the underlying code? if so,
- If the
tm.commit_vetoconfiguration setting was used, did the commit veto callback, called with the response generated by the application, return a result that evaluates to
True? if so,
If none of these checks calls
transaction.abort() then the transaction is
instead committed using
By itself, this transaction machinery doesn’t do much. It is up to
third-party code to join the active transaction to benefit. See
repoze.filesafe for an
example of how files creation can be committed or rolled back based on
transaction and the pyramid_mailer package to see
how you can prevent emails from being sent until a transaction succeeds.
ZODB database connections are automatically joined to the transaction, as
well as SQLAlchemy connections which are configured with
zope.sqlalchemy.register(session) from the zope.sqlalchemy package.
Custom Transaction Managers¶
pyramid_tm will use the threadlocal
to associate one transaction manager per thread. If you wish to override this
and provide your own transaction manager you can create your own manager hook
that will return the manager it should use.
1 2 3 4
import transaction def manager_hook(request): return transaction.TransactionManager(explicit=True)
To enable this hook, add it as the
tm.manager_hook setting in your app.
1 2 3 4 5 6 7
from pyramid.config import Configurator def app(global_conf, **settings): settings['tm.manager_hook'] = manager_hook config = Configurator(settings=settings) config.include('pyramid_tm') # ...
This specific example, using an explicit mode non-threadlocal manager, is
highly recommended and is shipped as
tm.manager_hook = pyramid_tm.explicit_manager in your settings
to enable it.
The current transaction manager being used for any particular request can
always be accessed on the request as
request.tm so long as it is accessed
pyramid_tm tween is active. If you try to access
outside of the tween or during a request in which
pyramid_tm was disabled,
request.tm will raise an
It is recommended to use a custom transaction manager with
explicit=True, as in the example above, instead of the threadlocal
transaction.manager to give greater control over the transaction’s
lifecycle and to weed out potential bugs in your application. For example,
you may have some parts of your app that access the manager after it has
already been committed. This will open an implicit transaction that is
never committed, and will even hang around until a subsequent request
aborts the implicit transaction. Instead, if you set
any code affecting the manager outside of the lifecycle of the transaction
will cause an error and will be noticed quickly.
Adding an Activation Hook¶
It may not always be desirable to have every request managed by the
transaction manager automatically. It is possible to configure
with an “activate” hook. The callback function receives the request. It
can then examine it and return
False if the transaction manager should
be disabled for that request.
1 2 3 4 5 6
def activate_hook(request): if request.path_info.startswith('/long-poll'): # Allow the long-poll class to manage its own connections to avoid # long-lived transactions. return False return True
To enable this hook, add it as the
tm.activate_hook setting in your app.
1 2 3 4 5 6 7
from pyramid.config import Configurator def app(global_conf, **settings): settings['tm.activate_hook'] = activate_hook config = Configurator(settings=settings) config.include('pyramid_tm') # ...
Or via PasteDeploy:
[app:myapp] tm.activate_hook = myapp.activate_hook
In either configuration the value for
tm.activate_hook is a
dotted Python name.
Adding a Commit Veto Hook¶
It is possible to configure
pyramid_tm with a “commit veto” hook. The
commit veto hook receives the request and the response. It can examine both
of them, and return
True if the transaction should be vetoed. If the
transaction is vetoed, it will be aborted instead of committed. By default,
pyramid_tm does not configure a commit veto into the system; you must do
pyramid_tm contains a
pyramid_tm.default_commit_veto() that is
suitable for use when you want to abort when the response’s status code
indicates non-success or if you’d like to signal that the transaction should
be aborted or committed using a response header. The default commit veto
vetoes a commit if the status code starts with
5 or there is a
X-Tm response header with a value that does not equal
1 2 3 4 5
def default_commit_veto(request, response): xtm = response.headers.get('x-tm') if xtm is not None: return xtm != 'commit' return response.status.startswith(('4', '5'))
If you’d like to use this commit veto in your system, you can do it via Python:
1 2 3 4 5 6 7
from pyramid.config import Configurator def app(global_conf, **settings): settings['tm.commit_veto'] = 'pyramid_tm.default_commit_veto' config = Configurator(settings=settings) config.include('pyramid_tm') # ...
Or via PasteDeploy:
[app:myapp] tm.commit_veto = pyramid_tm.default_commit_veto
If you’d like to use a different “commit veto” callback, create a function
with the same signature (
response) and return value
False), then pass a
pair in your settings which points at the Python dotted name of this commit
1 2 3 4 5 6
from pyramid.config import Configurator def app(global_conf, settings): settings['tm.commit_veto'] = 'my.package.commit_veto' config = Configurator(settings=settings) config.include('pyramid_tm')
[app:myapp] tm.commit_veto = my.package.commit_veto
In the PasteDeploy example, the path is a dotted Python name, where the dots separate module and package names, and the colon separates a module from its contents. In the above example, the code would be implemented as a “commit_veto” function which lives in the “package” submodule of the “my” package.
pyramid_tm ships with support for pyramid_retry which is an
execution policy that will retry requests when they fail with exceptions
marked as retryable. By default, retrying is turned off. In order to turn it
on you must update your app’s configuration:
from pyramid.config import Configurator def main(global_config, **settings): config = Configurator(settings=settings) config.include('pyramid_retry') config.include('pyramid_tm')
Finally, ensure that your application’s settings have
set to a value greater than
When the transaction manager calls the downstream handler, if the handler
raises a retryable exception,
pyramid_tm will mark the exception
as retryable by
pyramid_retry. The execution policy will detect a
retryable error and create a new copy of the request with new state.
Retryable exceptions include
certain exceptions raised by various data managers, such as
where the exception’s code is 8877. Any exception which inherits from
transaction.interfaces.TransientError will be marked as retryable.
Read more about retrying requests in the pyramid_retry documentation.
Explicit Tween Configuration¶
Note that the transaction manager is a Pyramid “tween”, and it can be used in
the explicit tween list if its implicit position in the tween chain is
incorrect (see the output of
[app:myapp] pyramid.tweens = someothertween pyramid_tm.tm_tween_factory pyramid.tweens.excview_tween_factory
It usually belongs directly above the
“pyramid.tweens.excview_tween_factory” entry in the `` ptweens``
output, and will attempt to sort there by default as the result of having
Avoid Accessing the Authentication Policy¶
By default the tween will access
pyramid.request.Request.unauthenticated_userid in order to annotate
the transaction with information about the user. This can be turned off
by setting the ini option
tm.annotate_user = false.
Reporting Bugs / Development Versions¶
Visit http://github.com/Pylons/pyramid_tm to download development or tagged versions.
Visit http://github.com/Pylons/pyramid_tm/issues to report bugs.
pyramid_tmtween has been moved over the
EXCVIEWtween. This means the transaction is open during exception view execution. See https://github.com/Pylons/pyramid_tm/pull/55
- Added a
tm_activeview predicate which may be useful in exception views that require access to the database. See https://github.com/Pylons/pyramid_tm/pull/60
tm.attemptssetting has been removed and retry support has been moved into a new package named
pyramid_retry. If you want retry support then please look at that library for more information about installing and enabling it. See https://github.com/Pylons/pyramid_tm/pull/55
pyramid_tmtween has been moved over the
EXCVIEWtween. If you have any hacks in your application that are opening a new transaction inside your exception views then it’s likely you will want to remove them or re-evaluate when upgrading. See https://github.com/Pylons/pyramid_tm/pull/55
- Drop support for Pyramid < 1.5.
- Support for Python 3.6.
pyramid_tm1.1.0 failed to fix a unicode issue related to undecodable request paths. The placeholder message was not unicode. See https://github.com/Pylons/pyramid_tm/pull/52
- Include Changes in the main docs.
- The transaction’s request path and userid are now coerced to unicode by
first decoding as
utf-8and falling back to
latin-1. If the userid does not conform to these restrictions then set
tm.annotate_user = noin your settings. See https://github.com/Pylons/pyramid_tm/pull/50
- Pin to
transaction < 1.99as pyramid_tm is currently incompatible with the new 2.x release of transaction. See https://github.com/Pylons/pyramid_tm/issues/49
- Removes the
request.tmis accessed outside the tween. It turns out this broke subrequests as well as
pyramid.paster.bootstrappCLI scripts, especially when using the global transaction manager which can be tracked outside of the tween. See https://github.com/Pylons/pyramid_tm/pull/48
- Drop Python 2.6, 3.2 and 3.3 support.
- Add Python 3.5 support.
- Subtle bugs can occur if you use the transaction manager during a request
pyramid_tmis disabled via an
activate_hook. To combat these types of errors, attempting to access
request.tmwill now raise an
pyramid_tmis inactive. See https://github.com/Pylons/pyramid_tm/pull/46
- Fix compatibility with 1.2 and 1.3 again. This wasn’t fully fixed in the 0.12 release as the tween was relying on request properties working (which they do not inside tweens in older versions). See https://github.com/Pylons/pyramid_tm/pull/39
- Expose a
tm.annotate_useroption to avoid computing
request.unauthenticated_useridon every request. See https://github.com/Pylons/pyramid_tm/pull/36
- Restore compatibility with Pyramid 1.2 and 1.3.
- Add a hook to override creation of the transaction manager (the default
remains the thread-local one accessed through
transaction.manager). See: https://github.com/Pylons/pyramid_tm/pull/31
- Fix recording transactions with non-text, non-bytes userids. See: https://github.com/Pylons/pyramid_tm/issues/28
- Add a new
tm.activate_hookhook which can control when the transaction manager is active. For example, this may be useful in situations where the manager should be disabled for a particular URL. https://github.com/Pylons/pyramid_tm/pull/12
- Fix unit tests under Pyramid 1.5.
- Fix a bug preventing retryable exceptions from actually being retried. https://github.com/Pylons/pyramid_tm/pull/8
- Don’t call
setUseron transaction if there is no user logged in. This could cause the username set on the transaction to be a strange string: ” None”. https://github.com/Pylons/pyramid_tm/pull/9
- Avoid crash when the
path_infocannot be decoded from the request object. https://github.com/Pylons/pyramid_tm/pull/19
- Write unauthenticated userid and
request.path_infoas transaction metadata via
t.noterespectively during a commit.
- Disuse the confusing and bug-ridden generator-plus-context-manager “attempts” mechanism from the transaction package for retrying retryable exceptions (e.g. ZODB ConflictError). Use a simple while loop plus a counter and imperative logic instead.
- When a non-retryable exception was raised as the result of a call to
transaction.manager.commit, the exception was not reraised properly. Symptom: an unrecoverable exception such as
Unsupported: Storing blobs in <somestorage> is not supported.would be swallowed inappropriately.
- Work around failure to retry ConflictError properly at commit time by the
transaction1.2.0 package. See https://mail.zope.org/pipermail/zodb-dev/2012-March/014603.html for details.
- No longer tested under Python 2.5 by
tox.ini(and therefore no longer tested under 2.5 by the Pylons Jenkins server). The package may still work under 2.5, but automated tests will no longer show breakage when it changes in ways that break 2.5 support.
- Squash test deprecation warnings under Python 3.2.
- The transaction manager has been converted to a Pyramid 1.2 “tween” (instead of an event subscriber). It will be slotted directly “below” the exception view handler, meaning it will have a chance to handle exceptions before they are turned into responses. This means it’s best to “raise HTTPFound(...)” instead of “return HTTPFound(...)” if you want an HTTP exception to abort the transaction.
- The transaction manager will now retry retryable exceptions (such as a ZODB
conflict error) if
tm.attemptsis configured to be more than the default of
1. See the
Retryingsection of the documentation.
- Python 3.2 compatibility (requires Pyramid 1.3dev+).
Incompatible with Pyramid < 1.2a1. Use
pyramid_tmversion 0.2 if you need compatibility with an older Pyramid installation.
default_commit_vetocommit veto callback is no longer configured into the system by default. Use
tm.commit_veto = pyramid_tm.default_commit_vetoin the deployment settings to add it. This is for parity with
repoze.tm2, which doesn’t configure in a commit veto by default either.
default_commit_vetono longer checks for the presence of the
X-Tm-Abortheader when attempting to figure out whether the transaction should be aborted (although it still checks for the
X-Tmheader). Use version 0.2 or a custom commit veto function if your application depends on the
A commit veto is now called with two arguments:
requestis the webob request that caused the transaction manager to become active. The
responseis the response returned by the Pyramid application. This call signature is incompatible with older versions. The call signature of a
pyramid_tm0.2 and older commit veto accepted three arguments:
headers. If you’re using a custom
commit_vetofunction, you’ll need to either convert your existing function to use the new calling convention or use a wrapper to make it compatible with the new calling convention. Here’s a simple wrapper function (
bwcompat_commit_veto_wrapper) that will allow you to use your existing custom commit veto function:
def bwcompat_commit_veto_wrapper(request, response): return my_custom_commit_veto(request.environ, response.status, response.headerlist)
pyramid_tm.commit_vetoconfiguration setting is now canonically spelled as
tm.commit_veto. The older spelling will continue to work, but may raise a deprecation error when used.
- A new header
X-Tmis now honored by the
default_commit_vetocommit veto hook. If this header exists in the headerlist, its value must be a string. If its value is
commit, the transaction will be committed regardless of the status code or the value of
X-Tm-Abort. If the value of the
abort(or any other string value except
commit), the transaction will be aborted, regardless of the status code or the value of
- Initial release, based on repoze.tm2