pyramid_retry

pyramid_retry is an execution policy for Pyramid that wraps requests and can retry them a configurable number of times under certain “retryable” error conditions before indicating a failure to the client.

Installation

Stable release

To install pyramid_retry, run this command in your terminal:

$ pip install pyramid_retry

If you don’t have pip installed, this Python installation guide can guide you through the process.

From sources

The sources for pyramid_retry can be downloaded from the Github repo.

$ git clone https://github.com/Pylons/pyramid_retry.git

Once you have a copy of the source, you can install it with:

$ pip install -e .

Usage

Activate pyramid_retry by including it in your application:

def main(global_config, **settings):
    config = Configurator(settings=settings)
    config.include('pyramid_retry')
    # ...
    config.add_route('home', '/')

By default pyramid_retry will register an instance of pyramid_retry.RetryableExecutionPolicy() as an execution policy in your application using the retry.attempts setting as the maximum number of attempts per request. The default number of attempts is 1.

The policy will handle any requests that fail because the application raised an instance of pyramid_retry.RetryableException or another exception implementing the pyramid_retry.IRetryableError interface.

The below, very contrived example, shows conceptually what’s going on when a request is retried. The failing_view is executed initially and for the final attempt the recovery_view is executed.

@view_config(route_name='home')
def failing_view(request):
    raise RetryableException

@view_config(route_name='home', is_last_attempt=True, renderer='string')
def recovery_view(request):
    return 'success'

Of course you probably wouldn’t write actual code that expects to fail like this. More realistically you may use a library like pyramid_tm to translate certain transactional errors marked as “transient” into retryable errors.

Custom Retryable Errors

The simple approach to marking errors as retryable is to simply catch the error and raise a pyramid_retry.RetryableException instead:

from pyramid_retry import RetryableException
import requests

def view(request):
    try:
        response = requests.get('https://www.google.com')
    except requests.Timeout:
        raise RetryableException

This will work but if this is the last attempt then the failed request will not actually be retried and on top of that the original exception is lost.

A better approach is to preserve the original exception and simply mark it as retryable using the pyramid_retry.IRetryableError marker interface:

from pyramid_retry import IRetryableError
import requests
import zope.interface

# mark requests.Timeout errors as retryable
zope.interface.classImplements(requests.Timeout, IRetryableError)

def view(request):
    response = requests.get('https://www.google.com')

View Predicates

When the library is included in your application it registers two new view predicates which are especially useful on exception views to determine when to handle certain errors.

is_exc_retryable will match the exception view only if the exception is both an retryable error and there are remaining attempts in which the request would be retried. See pyramid_retry.RetryableExceptionPredicate for more information.

is_last_attempt will match only if, when the view is executed, there will not be another attempt for this request. See pyramid_retry.LastAttemptPredicate for more information.

Caveats

  • In order to guarantee that a request can be retried it must make the body seekable. This is done via request.make_body_seekable(). Generally the body is loaded directly from environ['wsgi.input'] which is controlled by the WSGI server. However to make the body seekable it is copied into a seekable wrapper. In some cases this can lead to a very large copy operation before the request is executed.
  • pyramid_retry does not copy the environ or make any attempt to restore it to its original state before retrying a request. This means anything stored on the environ will persist across requests created for that environ.

Indices and tables