What’s New In Pyramid 1.0

This article explains the new features in Pyramid version 1.0 as compared to its predecessor, repoze.bfg 1.3. It also documents backwards incompatibilities between the two versions and deprecations added to Pyramid 1.0, as well as software dependency changes and notable documentation additions.

Major Feature Additions

The major feature additions in Pyramid 1.0 are:

  • New name and branding association with the Pylons Project.
  • BFG conversion script
  • Scaffold improvements
  • Terminology changes
  • Better platform compatibility and support
  • Direct built-in support for the Mako templating language.
  • Built-in support for sessions.
  • Updated URL dispatch features
  • Better imperative extensibility
  • ZCML externalized
  • Better support for global template variables during rendering
  • View mappers
  • Testing system improvements
  • Authentication support improvements
  • Documentation improvements

New Name and Branding

The name of repoze.bfg has been changed to Pyramid. The project is also now a subproject of a new entity, “The Pylons Project”. The Pylons Project is the project name for a collection of web-framework-related technologies. Pyramid was the first package in the Pylons Project. Other packages to the collection have been added over time, such as support packages useful for Pylons 1 users as well as ex-Zope users. Pyramid is the successor to both repoze.bfg and Pylons version 1.

The Pyramid codebase is derived almost entirely from repoze.bfg with some changes made for the sake of Pylons 1 compatibility.

Pyramid is technically backwards incompatible with repoze.bfg, as it has a new package name, so older imports from the repoze.bfg module will fail if you do nothing to your existing repoze.bfg application. However, you won’t have to do much to use your existing BFG applications on Pyramid. There’s automation which will change most of your import statements and ZCML declarations. See http://docs.pylonsproject.org/projects/pyramid/current/tutorials/bfg/index.html for upgrade instructions.

Pylons 1 users will need to do more work to use Pyramid, as Pyramid shares no “DNA” with Pylons. It is hoped that over time documentation and upgrade code will be developed to help Pylons 1 users transition to Pyramid more easily.

repoze.bfg version 1.3 will be its last major release. Minor updates will be made for critical bug fixes. Pylons version 1 will continue to see maintenance releases, as well.

The Repoze project will continue to exist. Repoze will be able to regain its original focus: bringing Zope technologies to WSGI. The popularity of repoze.bfg as its own web framework hindered this goal.

We hope that people are attracted at first by the spirit of cooperation demonstrated by the Pylons Project and the merging of development communities. It takes humility to sacrifice a little sovereignty and work together. The opposite, forking or splintering of projects, is much more common in the open source world. We feel there is a limited amount of oxygen in the space of “top-tier” Python web frameworks and we don’t do the Python community a service by over-crowding. By merging the repoze.bfg and the philosophically-similar Pylons communities, both gain an expanded audience and a stronger chance of future success.

BFG Conversion Script

The bfg2pyramid conversion script performs a mostly automated conversion of an existing repoze.bfg application to Pyramid. The process is described in Converting a repoze.bfg Application to Pyramid.

Scaffold Improvements

  • The scaffolds now have much nicer CSS and graphics.
  • The development.ini, generated by all scaffolds, is now configured to use the WebError interactive exception debugger by default.
  • All scaffolds have been normalized: each now uses the name main to represent the function that returns a WSGI application, and each now has roughly the same shape of development.ini style.
  • All preexisting scaffolds now use “imperative” configuration (starter, routesalchemy, alchemy, zodb) instead of ZCML configuration.
  • The pyramid_zodb, pyramid_routesalchemy and pyramid_alchemy scaffolds now use a default “commit veto” hook when configuring the repoze.tm2 transaction manager in development.ini. This prevents a transaction from being committed when the response status code is within the 400 or 500 ranges. See also http://docs.repoze.org/tm2/#using-a-commit-veto.

Terminology Changes

  • The Pyramid concept previously known as “model” is now known as “resource”. As a result, the following API renames have been made. Backwards compatibility shims for the old names have been left in place in all cases:

    pyramid.url.model_url ->
                      pyramid.url.resource_url
    
    pyramid.traversal.find_model ->
                      pyramid.url.find_resource
    
    pyramid.traversal.model_path ->
                      pyramid.traversal.resource_path
    
    pyramid.traversal.model_path_tuple ->
                      pyramid.traversal.resource_path_tuple
    
    pyramid.traversal.ModelGraphTraverser ->
                      pyramid.traversal.ResourceTreeTraverser
    
    pyramid.config.Configurator.testing_models ->
                      pyramid.config.Configurator.testing_resources
    
    pyramid.testing.registerModels ->
                      pyramid.testing.registerResources
    
    pyramid.testing.DummyModel ->
                      pyramid.testing.DummyResource
    
  • All documentation which previously referred to “model” now refers to “resource”.

  • The starter scaffold now has a resources.py module instead of a models.py module.

  • Positional argument names of various APIs have been changed from model to resource.

  • The Pyramid concept previously known as “resource” is now known as “asset”. As a result, the following API changes were made. Backwards compatibility shims have been left in place as necessary:

    pyramid.config.Configurator.absolute_resource_spec ->
                      pyramid.config.Configurator.absolute_asset_spec
    
    pyramid.config.Configurator.override_resource ->
                      pyramid.config.Configurator.override_asset
    
  • The (non-API) module previously known as pyramid.resource is now known as pyramid.asset.

  • All docs that previously referred to “resource specification” now refer to “asset specification”.

  • The setting previously known as BFG_RELOAD_RESOURCES (envvar) or reload_resources (config file) is now known, respectively, as PYRAMID_RELOAD_ASSETS and reload_assets.

Better Platform Compatibility and Support

We’ve made Pyramid’s test suite pass on both Jython and PyPy. However, Chameleon doesn’t work on either, so you’ll need to use Mako or Jinja2 templates on these platforms.

Sessions

Pyramid now has built-in sessioning support, documented in Sessions. The sessioning implementation is pluggable. It also provides flash messaging and cross-site-scripting prevention features.

Using request.session now returns a (dictionary-like) session object if a session factory has been configured.

A new argument to the Configurator constructor exists: session_factory and a new method on the configurator exists: pyramid.config.Configurator.set_session_factory().

Mako

In addition to Chameleon templating, Pyramid now also provides built-in support for Mako templating. See Templating With Mako Templates for more information.

URL Dispatch

  • URL Dispatch now allows for replacement markers to be located anywhere in the pattern, instead of immediately following a /.
  • URL Dispatch now uses the form {marker} to denote a replace marker in the route pattern instead of :marker. The old colon-style marker syntax is still accepted for backwards compatibility. The new format allows a regular expression for that marker location to be used instead of the default [^/]+, for example {marker:\d+} is now valid to require the marker to be digits.
  • Addded a new API pyramid.url.current_route_url(), which computes a URL based on the “current” route (if any) and its matchdict values.
  • Added a paster proute command which displays a summary of the routing table. See the narrative documentation section entitled Displaying All Application Routes.
  • Added debug_routematch configuration setting (settable in your .ini file) that logs matched routes including the matchdict and predicates.
  • Add a pyramid.url.route_path() API, allowing folks to generate relative URLs. Calling route_path is the same as calling pyramid.url.route_url() with the argument _app_url equal to the empty string.
  • Add a pyramid.request.Request.route_path() API. This is a convenience method of the request which calls pyramid.url.route_url().
  • Added class vars matchdict and matched_route to pyramid.request.Request. Each is set to None when a route isn’t matched during a request.

ZCML Externalized

  • The load_zcml method of a Configurator has been removed from the Pyramid core. Loading ZCML is now a feature of the pyramid_zcml package, which can be downloaded from PyPI. Documentation for the package should be available via http://pylonsproject.org/projects/pyramid_zcml/dev/, which describes how to add a configuration statement to your main block to reobtain this method. You will also need to add an install_requires dependency upon the pyramid_zcml distribution to your setup.py file.
  • The “Declarative Configuration” narrative chapter has been removed (it was moved to the pyramid_zcml package).
  • Most references to ZCML in narrative chapters have been removed or redirected to pyramid_zcml locations.
  • The starter_zcml paster scaffold has been moved to the pyramid_zcml package.

Imperative Two-Phase Configuration

To support application extensibility, the Pyramid Configurator, by default, now detects configuration conflicts and allows you to include configuration imperatively from other packages or modules. It also, by default, performs configuration in two separate phases. This allows you to ignore relative configuration statement ordering in some circumstances. See Advanced Configuration for more information.

The pyramid.config.Configurator.add_directive() allows framework extenders to add methods to the configurator, which allows extenders to avoid subclassing a Configurator just to add methods. See Adding Methods to the Configurator via add_directive for more info.

Surrounding application configuration with config.begin() and config.end() is no longer necessary. All scaffolds have been changed to no longer call these functions.

Better Support for Global Template Variables During Rendering

A new event type named pyramid.interfaces.IBeforeRender is now sent as an event before a renderer is invoked. Applications may now subscribe to the IBeforeRender event type in order to introspect the and modify the set of renderer globals before they are passed to a renderer. The event object iself has a dictionary-like interface that can be used for this purpose. For example:

from repoze.events import subscriber
from pyramid.interfaces import IRendererGlobalsEvent

@subscriber(IRendererGlobalsEvent)
def add_global(event):
    event['mykey'] = 'foo'

View Mappers

A “view mapper” subsystem has been extracted, which allows framework extenders to control how view callables are constructed and called. This feature is not useful for “civilians”, only for extension writers. See Using a View Mapper for more information.

Testing Support Improvements

The pyramid.testing.setUp() and pyramid.testing.tearDown() APIs have been undeprecated. They are now the canonical setup and teardown APIs for test configuration, replacing “direct” creation of a Configurator. This is a change designed to provide a facade that will protect against any future Configurator deprecations.

Authentication Support Improvements

  • The pyramid.interfaces.IAuthenticationPolicy interface now specifies an unauthenticated_userid method. This method supports an important optimization required by people who are using persistent storages which do not support object caching and whom want to create a “user object” as a request attribute.
  • A new API has been added to the pyramid.security module named unauthenticated_userid. This API function calls the unauthenticated_userid method of the effective security policy.
  • The class pyramid.authentication.AuthTktCookieHelper is now an API. This class can be used by third-party authentication policy developers to help in the mechanics of authentication cookie-setting.
  • The pyramid.authentication.AuthTktAuthenticationPolicy now accepts a tokens parameter via pyramid.security.remember(). The value must be a sequence of strings. Tokens are placed into the auth_tkt “tokens” field and returned in the auth_tkt cookie.
  • Added a wild_domain argument to pyramid.authentication.AuthTktAuthenticationPolicy, which defaults to True. If it is set to False, the feature of the policy which sets a cookie with a wilcard domain will be turned off.

Documentation Improvements

  • Casey Duncan, a good friend, and an excellent technical writer has given us the gift of professionally editing the entire Pyramid documentation set. Any faults in the documentation are the development team’s, and all improvements are his.
  • The “Resource Location and View Lookup” chapter has been replaced with a variant of Rob Miller’s “Much Ado About Traversal” (originally published at http://blog.nonsequitarian.org/2010/much-ado-about-traversal/).
  • Many users have contributed documentation fixes and improvements including Ben Bangert, Blaise Laflamme, Rob Miller, Mike Orr, Carlos de la Guardia, Paul Everitt, Tres Seaver, John Shipman, Marius Gedminas, Chris Rossi, Joachim Krebs, Xavier Spriet, Reed O’Brien, William Chambers, Charlie Choiniere, and Jamaludin Ahmad.

Minor Feature Additions

  • The settings dictionary passed to the Configurator is now available as config.registry.settings in configuration code and request.registry.settings in view code).
  • pyramid.config.Configurator.add_view() now accepts a decorator keyword argument, a callable which will decorate the view callable before it is added to the registry.
  • Allow static renderer provided during view registration to be overridden at request time via a request attribute named override_renderer, which should be the name of a previously registered renderer. Useful to provide “omnipresent” RPC using existing rendered views.
  • If a resource implements a __resource_url__ method, it will be called as the result of invoking the pyramid.url.resource_url() function to generate a URL, overriding the default logic. See Generating The URL Of A Resource for more information.
  • The name registry is now available in a pshell environment by default. It is the application registry object.
  • Added support for json on Google App Engine by catching NotImplementedError and importing simplejson from django.utils.
  • Added the pyramid.httpexceptions module, which is a facade for the webob.exc module.
  • New class: pyramid.response.Response. This is a pure facade for webob.Response (old code need not change to use this facade, it’s existence is mostly for vanity and documentation-generation purposes).
  • The request now has a new attribute: tmpl_context for benefit of Pylons users.
  • New API methods for pyramid.request.Request: model_url, route_url, and static_url. These are simple passthroughs for their respective functions in pyramid.url.

Backwards Incompatibilities

  • When a pyramid.exceptions.Forbidden error is raised, its status code now 403 Forbidden. It was previously 401 Unauthorized, for backwards compatibility purposes with repoze.bfg. This change will cause problems for users of Pyramid with repoze.who, which intercepts 401 Unauthorized by default, but allows 403 Forbidden to pass through. Those deployments will need to configure repoze.who to also react to 403 Forbidden. To do so, use a repoze.who challenge_decider that looks like this:

    import zope.interface
    from repoze.who.interfaces import IChallengeDecider
    
    def challenge_decider(environ, status, headers):
        return status.startswith('403') or status.startswith('401')
    zope.interface.directlyProvides(challenge_decider, IChallengeDecider)
    
  • The paster bfgshell command is now known as paster pshell.

  • There is no longer an IDebugLogger object registered as a named utility with the name repoze.bfg.debug.

  • These deprecated APIs have been removed: pyramid.testing.registerViewPermission, pyramid.testing.registerRoutesMapper, pyramid.request.get_request, pyramid.security.Unauthorized, pyramid.view.view_execution_permitted, pyramid.view.NotFound

  • The Venusian “category” for all built-in Venusian decorators (e.g. subscriber and view_config/bfg_view) is now pyramid instead of bfg.

  • The pyramid.renderers.rendered_response function removed; use pyramid.renderers.render_to_response() instead.

  • Renderer factories now accept a renderer info object rather than an absolute resource specification or an absolute path. The object has the following attributes: name (the renderer= value), package (the ‘current package’ when the renderer configuration statement was found), type: the renderer type, registry: the current registry, and settings: the deployment settings dictionary. Third-party repoze.bfg renderer implementations that must be ported to Pyramid will need to account for this. This change was made primarily to support more flexible Mako template rendering.

  • The presence of the key repoze.bfg.message in the WSGI environment when an exception occurs is now deprecated. Instead, code which relies on this environ value should use the exception attribute of the request (e.g. request.exception[0]) to retrieve the message.

  • The values bfg_localizer and bfg_locale_name kept on the request during internationalization for caching purposes were never APIs. These however have changed to localizer and locale_name, respectively.

  • The default cookie_name value of the pyramid.authentication.AuthTktAuthenticationPolicy now defaults to auth_tkt (it used to default to repoze.bfg.auth_tkt).

  • The pyramid.testing.zcml_configure() API has been removed. It had been advertised as removed since repoze.bfg 1.2a1, but hadn’t actually been.

  • All environment variables which used to be prefixed with BFG_ are now prefixed with PYRAMID_ (e.g. BFG_DEBUG_NOTFOUND is now PYRAMID_DEBUG_NOTFOUND)

  • Since the pyramid.interfaces.IAuthenticationPolicy interface now specifies that a policy implementation must implement an unauthenticated_userid method, all third-party custom authentication policies now must implement this method. It, however, will only be called when the global function named pyramid.security.unauthenticated_userid() is invoked, so if you’re not invoking that, you will not notice any issues.

  • The configure_zcml setting within the deployment settings (within **settings passed to a Pyramid main function) has ceased to have any meaning.

  • The make_app function has been removed from the pyramid.router module. It continues life within the pyramid_zcml package. This leaves the pyramid.router module without any API functions.

Deprecations and Behavior Differences

  • pyramid.configuration.Configurator is now deprecated. Use pyramid.config.Configurator, passing its constructor autocommit=True instead. The pyramid.configuration.Configurator alias will live for a long time, as every application uses it, but its import now issues a deprecation warning. The pyramid.config.Configurator class has the same API as the pyramid.configuration.Configurator class, which it means to replace, except by default it is a non-autocommitting configurator. The now-deprecated pyramid.configuration.Configurator will autocommit every time a configuration method is called. The pyramid.configuration module remains, but it is deprecated. Use pyramid.config instead.
  • The pyramid.settings.get_settings() API is now deprecated. Use pyramid.threadlocals.get_current_registry().settings instead or use the settings attribute of the registry available from the request (request.registry.settings).
  • The decorator previously known as pyramid.view.bfg_view is now known most formally as pyramid.view.view_config in docs and scaffolds.
  • Obtaining the settings object via registry.{get|query}Utility(ISettings) is now deprecated. Instead, obtain the settings object via the registry.settings attribute. A backwards compatibility shim was added to the registry object to register the settings object as an ISettings utility when setattr(registry, 'settings', foo) is called, but it will be removed in a later release.
  • Obtaining the settings object via pyramid.settings.get_settings() is now deprecated. Obtain it instead as the settings attribute of the registry now (obtain the registry via pyramid.threadlocal.get_registry() or as request.registry).

Dependency Changes

  • Depend on Venusian >= 0.5 (for scanning conflict exception decoration).

Documentation Enhancements

  • Added a pyramid.httpexceptions API documentation chapter.
  • Added a pyramid.session API documentation chapter.
  • Added an API chapter for the pyramid.response module.
  • Added a Sessions narrative documentation chapter.
  • All documentation which previously referred to webob.Response now uses pyramid.response.Response instead.
  • The documentation has been overhauled to use imperative configuration, moving declarative configuration (ZCML) explanations to an external package, pyramid_zcml.
  • Removed zodbsessions tutorial chapter. It’s still useful, but we now have a SessionFactory abstraction which competes with it, and maintaining documentation on both ways to do it is a distraction.
  • Added an example of WebTest functional testing to the testing narrative chapter at Creating Functional Tests.
  • Extended the Resources chapter with examples of calls to resource-related APIs.
  • Add “Pyramid Provides More Than One Way to Do It” to Design Defense documentation.
  • The (weak) “Converting a CMF Application to Pyramid” tutorial has been removed from the tutorials section. It was moved to the pyramid_tutorials Github repository at http://docs.pylonsproject.org/projects/pyramid_tutorials/dev/.
  • Moved “Using ZODB With ZEO” and “Using repoze.catalog Within Pyramid” tutorials out of core documentation and into the Pyramid Tutorials site (http://docs.pylonsproject.org/projects/pyramid_tutorials/dev/).
  • Removed API documentation for deprecated pyramid.testing APIs named registerDummySecurityPolicy, registerResources, registerModels, registerEventListener, registerTemplateRenderer, registerDummyRenderer, registerView, registerUtility, registerAdapter, registerSubscriber, registerRoute, and registerSettings.