pyramid_tm¶
Overview¶
pyramid_tm
is a package which allows Pyramid requests to join
the active transaction as provided by the Python transaction package. (See the documentation
for the transaction package for an
explanation of what “joining the active transaction” means).
Setup¶
Once pyramid_tm
is installed, you must use the config.include
mechanism to include it into your Pyramid project’s configuration. In your
Pyramid project’s __init__.py
:
1 2 | config = Configurator(.....)
config.include('pyramid_tm')
|
Or use the pyramid.includes
configuration setting in your .ini
file:
1 2 | [app:myapp]
pyramid.includes = pyramid_tm
|
After the package is included, whenever a new request enters the application, a new transaction is associated with that request.
Note
When the repoze.tm
or repoze.tm2
middleware is in the WSGI
pipeline, pyramid_tm
becomes inactive.
transaction Usage¶
At the beginning of a request a new transaction is started
using the transaction.begin()
function. Once the request has
finished all of its works (ie views have finished running), a few checks
are tested:
- Did some a transaction.doom() cause the transaction to become “doomed”? if so,
transaction.abort()
.- Did an exception occur in the underlying code? if so,
transaction.abort()
- If the
tm.commit_veto
configuration setting was used, did the commit veto callback, called with the response generated by the application, return a result that evaluates toTrue
? if so,transaction.abort()
.
If none of these checks calls transaction.abort()
then the transaction is
instead committed using transaction.commit()
.
By itself, this transaction machinery doesn’t do much. It is up to
third-party code to join the active transaction to benefit. See
repoze.filesafe for an
example of how files creation can be committed or rolled back based on
transaction and the pyramid_mailer package to see
how you can prevent emails from being sent until a transaction succeeds.
ZODB database connections are automatically joined to the transaction, as
well as SQLAlchemy connections which are configured with the
ZopeTransactionExtension
extension from the zope.sqlalchemy package.
Custom Transaction Managers¶
By default pyramid_tm
will use the default transaction manager which uses
thread locals to associate one transaction manager per thread. If you wish
to override this and provide your own transaction manager you can create your
own manager hook that will return the manager it should use.
1 2 3 4 | import transaction
def manager_hook(request):
return transaction.TransactionManager()
|
To enable this hook, add it as the tm.manager_hook
setting in your app.
1 2 3 4 5 6 7 | from pyramid.config import Configurator
def app(global_conf, **settings):
settings['tm.manager_hook'] = manager_hook
config = Configurator(settings=settings)
config.include('pyramid_tm')
# ...
|
The current transaction manager being used for any particular request can
always be accessed on the request as request.tm
.
Adding an Activation Hook¶
It may not always be desireable to have every request managed by the
transaction manager automatically. It is possible to configure pyramid_tm
with an “activate” hook. The callback function receives the request. It
can then examine it and return False
if the transaction manager should
be disabled for that request.
1 2 3 4 5 6 | def activate_hook(request):
if request.path_info.startswith('/long-poll'):
# Allow the long-poll class to manage its own connections to avoid
# long-lived transactions.
return False
return True
|
To enable this hook, add it as the tm.activate_hook
setting in your app.
1 2 3 4 5 6 7 | from pyramid.config import Configurator
def app(global_conf, **settings):
settings['tm.activate_hook'] = activate_hook
config = Configurator(settings=settings)
config.include('pyramid_tm')
# ...
|
Or via PasteDeploy:
1 2 | [app:myapp]
tm.activate_hook = myapp.activate_hook
|
In either configuration the value for tm.activate_hook
is a
dotted Python name.
Adding a Commit Veto Hook¶
It is possible to configure pyramid_tm
with a “commit veto” hook. The
commit veto hook receives the request and the response. It can examine both
of them, and return True
if the transaction should be vetoed. If the
transaction is vetoed, it will be aborted instead of committed. By default,
pyramid_tm
does not configure a commit veto into the system; you must do
it explicitly.
pyramid_tm
contains a pyramid_tm.default_commit_veto()
that is
suitable for use when you want to abort when the response’s status code
indicates non-success or if you’d like to signal that the transaction should
be aborted or committed using a response header. The default commit veto
vetoes a commit if the status code starts with 4
or 5
or there is a
X-Tm
response header with a value that does not equal commit
.
1 2 3 4 5 | def default_commit_veto(request, response):
xtm = response.headers.get('x-tm')
if xtm is not None:
return xtm != 'commit'
return response.status.startswith(('4', '5'))
|
If you’d like to use this commit veto in your system, you can do it via Python:
1 2 3 4 5 6 7 | from pyramid.config import Configurator
def app(global_conf, **settings):
settings['tm.commit_veto'] = 'pyramid_tm.default_commit_veto'
config = Configurator(settings=settings)
config.include('pyramid_tm')
# ...
|
Or via PasteDeploy:
1 2 | [app:myapp]
tm.commit_veto = pyramid_tm.default_commit_veto
|
If you’d like to use a different “commit veto” callback, create a function
with the same signature (request
, response
) and return value
(True
or False
), then pass a tm.commit_veto
key/value
pair in your settings which points at the Python dotted name of this commit
veto.
Via Python:
1 2 3 4 5 6 | from pyramid.config import Configurator
def app(global_conf, settings):
settings['tm.commit_veto'] = 'my.package.commit_veto'
config = Configurator(settings=settings)
config.include('pyramid_tm')
|
Via PasteDeploy:
1 2 | [app:myapp]
tm.commit_veto = my.package.commit_veto
|
In the PasteDeploy example, the path is a dotted Python name, where the dots separate module and package names, and the colon separates a module from its contents. In the above example, the code would be implemented as a “commit_veto” function which lives in the “package” submodule of the “my” package.
Retrying¶
When the transaction manager calls the downstream handler, if the handler raises a “retryable” exception, the transaction manager can be configured to attempt to call the downstream handler again with the same request, in effect “replaying” the request.
By default, retrying is turned off. To turn it on, use the
tm.attempts
configuration setting. By default this setting is
1
, meaning only one attempt will be tried, and no retry will happen even
if a retryable error is raised by the handler. But if the value, for
example, is set to 3
, the following set of events might happen.
- The first attempt to call the handler raises a retryable exception; a second attempt will be tried.
- The second attempt raises a retryable exception, the transaction manager will try the request again one more time.
- The third attempt also raises a retryable exception, at this point all attempts are used up and the “retryable” exception will be raised to its caller.
Or this might happen:
- The first attempt to call the handler raises a retryable exception; a second attempt will be tried.
- The second attempt returns a response without raising any exception.
- The response is returned to the caller.
Retryable exceptions include `ZODB.POSException.ConflictError
, and
certain exceptions raised by various data managers, such as
psycopg2.extensions.TransactionRollbackError
, cx_Oracle.DatabaseError
where the exception’s code is 8877. Any exception which inherits from
transaction.interfaces.TransientError
will be treated with retry
behavior.
Explicit Tween Configuration¶
Note that the transaction manager is a Pyramid “tween”, and it can be used in
the explicit tween list if its implicit position in the tween chain is
incorrect (see the output of paster ptweens
):
[app:myapp]
pyramid.tweens = someothertween
pyramid.tweens.excview_tween_factory
pyramid_tm.tm_tween_factory
It usually belongs directly above the “MAIN” entry in the paster ptweens
output, and will attempt to sort there by default as the result of having
include('pyramid_tm')
invoked.
Avoid Accessing the Authentication Policy¶
By default the tween will access
pyramid.request.Request.unauthenticated_userid
in order to annotate
the transaction with information about the user. This can be turned off
by setting the ini option tm.annotate_user = false
.
More Information¶
Reporting Bugs / Development Versions¶
Visit http://github.com/Pylons/pyramid_tm to download development or tagged versions.
Visit http://github.com/Pylons/pyramid_tm/issues to report bugs.
Changes¶
1.1.1 (2016-11-21)¶
pyramid_tm
1.1.0 failed to fix a unicode issue related to undecodable request paths. The placeholder message was not unicode. See https://github.com/Pylons/pyramid_tm/pull/52- Include Changes in the main docs.
1.1.0 (2016-11-19)¶
- Support
transaction
2.x. - The transaction’s request path and userid are now coerced to unicode by
first decoding as
utf-8
and falling back tolatin-1
. If the userid does not conform to these restrictions then settm.annotate_user = no
in your settings. See https://github.com/Pylons/pyramid_tm/pull/50
1.0.2 (2016-11-18)¶
- Pin to
transaction < 1.99
as pyramid_tm is currently incompatible with the new 2.x release of transaction. See https://github.com/Pylons/pyramid_tm/issues/49
1.0.1 (2016-10-24)¶
- Removes the
AttributeError
whenrequest.tm
is accessed outside the tween. It turns out this broke subrequests as well aspshell
andpyramid.paster.bootstrapp
CLI scripts, especially when using the global transaction manager which can be tracked outside of the tween. See https://github.com/Pylons/pyramid_tm/pull/48
1.0 (2016-09-12)¶
- Drop Python 2.6, 3.2 and 3.3 support.
- Add Python 3.5 support.
- Subtle bugs can occur if you use the transaction manager during a request
in which
pyramid_tm
is disabled via anactivate_hook
. To combat these types of errors, attempting to accessrequest.tm
will now raise anAttributeError
whenpyramid_tm
is inactive. See https://github.com/Pylons/pyramid_tm/pull/46
0.12.1 (2015-11-25)¶
- Fix compatibility with 1.2 and 1.3 again. This wasn’t fully fixed in the 0.12 release as the tween was relying on request properties working (which they do not inside tweens in older versions). See https://github.com/Pylons/pyramid_tm/pull/39
0.12 (2015-05-20)¶
- Expose a
tm.annotate_user
option to avoid computingrequest.unauthenticated_userid
on every request. See https://github.com/Pylons/pyramid_tm/pull/36 - Restore compatibility with Pyramid 1.2 and 1.3.
0.11 (2015-02-04)¶
- Add a hook to override creation of the transaction manager (the default
remains the thread-local one accessed through
transaction.manager
). See: https://github.com/Pylons/pyramid_tm/pull/31
0.10 (2015-01-06)¶
- Fix recording transactions with non-text, non-bytes userids. See: https://github.com/Pylons/pyramid_tm/issues/28
0.9 (2014-12-30)¶
- Work around recording transaction userid containing unicode. See https://github.com/Pylons/pyramid_tm/pull/15, although the fix is different, to ensure Python3 compatibility.
- Work around recording transaction notes containing unicode. https://github.com/Pylons/pyramid_tm/pull/25
0.8 (2014-11-12)¶
- Add a new
tm.activate_hook
hook which can control when the transaction manager is active. For example, this may be useful in situations where the manager should be disabled for a particular URL. https://github.com/Pylons/pyramid_tm/pull/12 - Fix unit tests under Pyramid 1.5.
- Fix a bug preventing retryable exceptions from actually being retried. https://github.com/Pylons/pyramid_tm/pull/8
- Don’t call
setUser
on transaction if there is no user logged in. This could cause the username set on the transaction to be a strange string: ” None”. https://github.com/Pylons/pyramid_tm/pull/9 - Avoid crash when the
path_info
cannot be decoded from the request object. https://github.com/Pylons/pyramid_tm/pull/19
0.7 (2012-12-30)¶
- Write unauthenticated userid and
request.path_info
as transaction metadata viat.setUser
andt.note
respectively during a commit.
0.6 (2012-12-26)¶
- Disuse the confusing and bug-ridden generator-plus-context-manager “attempts” mechanism from the transaction package for retrying retryable exceptions (e.g. ZODB ConflictError). Use a simple while loop plus a counter and imperative logic instead.
0.5 (2012-06-26)¶
Bug Fixes¶
- When a non-retryable exception was raised as the result of a call to
transaction.manager.commit
, the exception was not reraised properly. Symptom: an unrecoverable exception such asUnsupported: Storing blobs in <somestorage> is not supported.
would be swallowed inappropriately.
0.4 (2012-03-28)¶
Bug Fixes¶
- Work around failure to retry ConflictError properly at commit time by the
transaction
1.2.0 package. See https://mail.zope.org/pipermail/zodb-dev/2012-March/014603.html for details.
Testing¶
- No longer tested under Python 2.5 by
tox.ini
(and therefore no longer tested under 2.5 by the Pylons Jenkins server). The package may still work under 2.5, but automated tests will no longer show breakage when it changes in ways that break 2.5 support. - Squash test deprecation warnings under Python 3.2.
0.3 (2011-09-27)¶
Features¶
- The transaction manager has been converted to a Pyramid 1.2 “tween” (instead of an event subscriber). It will be slotted directly “below” the exception view handler, meaning it will have a chance to handle exceptions before they are turned into responses. This means it’s best to “raise HTTPFound(...)” instead of “return HTTPFound(...)” if you want an HTTP exception to abort the transaction.
- The transaction manager will now retry retryable exceptions (such as a ZODB
conflict error) if
tm.attempts
is configured to be more than the default of1
. See theRetrying
section of the documentation. - Python 3.2 compatibility (requires Pyramid 1.3dev+).
Backwards Incompatibilities¶
Incompatible with Pyramid < 1.2a1. Use
pyramid_tm
version 0.2 if you need compatibility with an older Pyramid installation.The
default_commit_veto
commit veto callback is no longer configured into the system by default. Usetm.commit_veto = pyramid_tm.default_commit_veto
in the deployment settings to add it. This is for parity withrepoze.tm2
, which doesn’t configure in a commit veto by default either.The
default_commit_veto
no longer checks for the presence of theX-Tm-Abort
header when attempting to figure out whether the transaction should be aborted (although it still checks for theX-Tm
header). Use version 0.2 or a custom commit veto function if your application depends on theX-Tm-Abort
header.A commit veto is now called with two arguments:
request
andresponse
. Therequest
is the webob request that caused the transaction manager to become active. Theresponse
is the response returned by the Pyramid application. This call signature is incompatible with older versions. The call signature of apyramid_tm
0.2 and older commit veto accepted three arguments:environ
,status
, andheaders
. If you’re using a customcommit_veto
function, you’ll need to either convert your existing function to use the new calling convention or use a wrapper to make it compatible with the new calling convention. Here’s a simple wrapper function (bwcompat_commit_veto_wrapper
) that will allow you to use your existing custom commit veto function:def bwcompat_commit_veto_wrapper(request, response): return my_custom_commit_veto(request.environ, response.status, response.headerlist)
Deprecations¶
- The
pyramid_tm.commit_veto
configuration setting is now canonically spelled astm.commit_veto
. The older spelling will continue to work, but may raise a deprecation error when used.
0.2 (2011-07-18)¶
- A new header
X-Tm
is now honored by thedefault_commit_veto
commit veto hook. If this header exists in the headerlist, its value must be a string. If its value iscommit
, the transaction will be committed regardless of the status code or the value ofX-Tm-Abort
. If the value of theX-Tm
header isabort
(or any other string value exceptcommit
), the transaction will be aborted, regardless of the status code or the value ofX-Tm-Abort
.
0.1 (2011-02-23)¶
- Initial release, based on repoze.tm2