Defining the Domain Model

The first change we'll make to our stock cookiecutter-generated application will be to define a wiki page domain model.

Note

There is nothing special about the filename user.py or page.py except that they are Python modules. A project may have many models throughout its codebase in arbitrarily named modules. Modules implementing models often have model in their names or they may live in a Python subpackage of your application package named models (as we've done in this tutorial), but this is only a convention and not a requirement.

Declaring dependencies in our setup.py file

The models code in our application will depend on a package which is not a dependency of the original "tutorial" application. The original "tutorial" application was generated by the cookiecutter; it doesn't know about our custom application requirements.

We need to add a dependency, the bcrypt package, to our tutorial package's setup.py file by assigning this dependency to the requires parameter in the setup() function.

Open tutorial/setup.py and edit it to look like the following:

import os

from setuptools import setup, find_packages

here = os.path.abspath(os.path.dirname(__file__))
with open(os.path.join(here, 'README.txt')) as f:
    README = f.read()
with open(os.path.join(here, 'CHANGES.txt')) as f:
    CHANGES = f.read()

requires = [
    'bcrypt',
    'plaster_pastedeploy',
    'pyramid >= 1.9a',
    'pyramid_debugtoolbar',
    'pyramid_jinja2',
    'pyramid_retry',
    'pyramid_tm',
    'SQLAlchemy',
    'transaction',
    'zope.sqlalchemy',
    'waitress',
]

tests_require = [
    'WebTest >= 1.3.1',  # py3 compat
    'pytest',
    'pytest-cov',
]

setup(
    name='tutorial',
    version='0.0',
    description='myproj',
    long_description=README + '\n\n' + CHANGES,
    classifiers=[
        'Programming Language :: Python',
        'Framework :: Pyramid',
        'Topic :: Internet :: WWW/HTTP',
        'Topic :: Internet :: WWW/HTTP :: WSGI :: Application',
    ],
    author='',
    author_email='',
    url='',
    keywords='web pyramid pylons',
    packages=find_packages(),
    include_package_data=True,
    zip_safe=False,
    extras_require={
        'testing': tests_require,
    },
    install_requires=requires,
    entry_points={
        'paste.app_factory': [
            'main = tutorial:main',
        ],
        'console_scripts': [
            'initialize_tutorial_db = tutorial.scripts.initializedb:main',
        ],
    },
)

Only the highlighted line needs to be added.

Note

We are using the bcrypt package from PyPI to hash our passwords securely. There are other one-way hash algorithms for passwords if bcrypt is an issue on your system. Just make sure that it's an algorithm approved for storing passwords versus a generic one-way hash.

Running pip install -e .

Since a new software dependency was added, you will need to run pip install -e . again inside the root of the tutorial package to obtain and register the newly added dependency distribution.

Make sure your current working directory is the root of the project (the directory in which setup.py lives) and execute the following command.

On UNIX:

$ $VENV/bin/pip install -e .

On Windows:

c:\tutorial> %VENV%\Scripts\pip install -e .

Success executing this command will end with a line to the console something like the following.

Successfully installed bcrypt-3.1.2 cffi-1.9.1 pycparser-2.17 tutorial

Remove mymodel.py

Let's delete the file tutorial/models/mymodel.py. The MyModel class is only a sample and we're not going to use it.

Add user.py

Create a new file tutorial/models/user.py with the following contents:

import bcrypt
from sqlalchemy import (
    Column,
    Integer,
    Text,
)

from .meta import Base


class User(Base):
    """ The SQLAlchemy declarative model class for a User object. """
    __tablename__ = 'users'
    id = Column(Integer, primary_key=True)
    name = Column(Text, nullable=False, unique=True)
    role = Column(Text, nullable=False)

    password_hash = Column(Text)

    def set_password(self, pw):
        pwhash = bcrypt.hashpw(pw.encode('utf8'), bcrypt.gensalt())
        self.password_hash = pwhash.decode('utf8')

    def check_password(self, pw):
        if self.password_hash is not None:
            expected_hash = self.password_hash.encode('utf8')
            return bcrypt.checkpw(pw.encode('utf8'), expected_hash)
        return False

This is a very basic model for a user who can authenticate with our wiki.

We discussed briefly in the previous chapter that our models will inherit from an SQLAlchemy sqlalchemy.ext.declarative.declarative_base(). This will attach the model to our schema.

As you can see, our User class has a class-level attribute __tablename__ which equals the string users. Our User class will also have class-level attributes named id, name, password_hash, and role (all instances of sqlalchemy.schema.Column). These will map to columns in the users table. The id attribute will be the primary key in the table. The name attribute will be a text column, each value of which needs to be unique within the column. The password_hash is a nullable text attribute that will contain a securely hashed password. Finally, the role text attribute will hold the role of the user.

There are two helper methods that will help us later when using the user objects. The first is set_password which will take a raw password and transform it using bcrypt into an irreversible representation, a process known as "hashing". The second method, check_password, will allow us to compare the hashed value of the submitted password against the hashed value of the password stored in the user's record in the database. If the two hashed values match, then the submitted password is valid, and we can authenticate the user.

We hash passwords so that it is impossible to decrypt them and use them to authenticate in the application. If we stored passwords foolishly in clear text, then anyone with access to the database could retrieve any password to authenticate as any user.

Add page.py

Create a new file tutorial/models/page.py with the following contents:

from sqlalchemy import (
    Column,
    ForeignKey,
    Integer,
    Text,
)
from sqlalchemy.orm import relationship

from .meta import Base


class Page(Base):
    """ The SQLAlchemy declarative model class for a Page object. """
    __tablename__ = 'pages'
    id = Column(Integer, primary_key=True)
    name = Column(Text, nullable=False, unique=True)
    data = Column(Text, nullable=False)

    creator_id = Column(ForeignKey('users.id'), nullable=False)
    creator = relationship('User', backref='created_pages')

As you can see, our Page class is very similar to the User defined above, except with attributes focused on storing information about a wiki page, including id, name, and data. The only new construct introduced here is the creator_id column, which is a foreign key referencing the users table. Foreign keys are very useful at the schema-level, but since we want to relate User objects with Page objects, we also define a creator attribute as an ORM-level mapping between the two tables. SQLAlchemy will automatically populate this value using the foreign key referencing the user. Since the foreign key has nullable=False, we are guaranteed that an instance of page will have a corresponding page.creator, which will be a User instance.

Edit models/__init__.py

Since we are using a package for our models, we also need to update our __init__.py file to ensure that the models are attached to the metadata.

Open the tutorial/models/__init__.py file and edit it to look like the following:

from sqlalchemy import engine_from_config
from sqlalchemy.orm import sessionmaker
from sqlalchemy.orm import configure_mappers
import zope.sqlalchemy

# import or define all models here to ensure they are attached to the
# Base.metadata prior to any initialization routines
from .page import Page  # noqa
from .user import User  # noqa

# run configure_mappers after defining all of the models to ensure
# all relationships can be setup
configure_mappers()


def get_engine(settings, prefix='sqlalchemy.'):
    return engine_from_config(settings, prefix)


def get_session_factory(engine):
    factory = sessionmaker()
    factory.configure(bind=engine)
    return factory


def get_tm_session(session_factory, transaction_manager):
    """
    Get a ``sqlalchemy.orm.Session`` instance backed by a transaction.

    This function will hook the session to the transaction manager which
    will take care of committing any changes.

    - When using pyramid_tm it will automatically be committed or aborted
      depending on whether an exception is raised.

    - When using scripts you should wrap the session in a manager yourself.
      For example::

          import transaction

          engine = get_engine(settings)
          session_factory = get_session_factory(engine)
          with transaction.manager:
              dbsession = get_tm_session(session_factory, transaction.manager)

    """
    dbsession = session_factory()
    zope.sqlalchemy.register(
        dbsession, transaction_manager=transaction_manager)
    return dbsession


def includeme(config):
    """
    Initialize the model for a Pyramid app.

    Activate this setup using ``config.include('tutorial.models')``.

    """
    settings = config.get_settings()
    settings['tm.manager_hook'] = 'pyramid_tm.explicit_manager'

    # use pyramid_tm to hook the transaction lifecycle to the request
    config.include('pyramid_tm')

    # use pyramid_retry to retry a request when transient exceptions occur
    config.include('pyramid_retry')

    session_factory = get_session_factory(get_engine(settings))
    config.registry['dbsession_factory'] = session_factory

    # make request.dbsession available for use in Pyramid
    config.add_request_method(
        # r.tm is the transaction manager used by pyramid_tm
        lambda r: get_tm_session(session_factory, r.tm),
        'dbsession',
        reify=True
    )

Here we align our imports with the names of the models, Page and User.

Edit scripts/initializedb.py

We haven't looked at the details of this file yet, but within the scripts directory of your tutorial package is a file named initializedb.py. Code in this file is executed whenever we run the initialize_tutorial_db command, as we did in the installation step of this tutorial.

Note

The command is named initialize_tutorial_db because of the mapping defined in the [console_scripts] entry point of our project's setup.py file.

Since we've changed our model, we need to make changes to our initializedb.py script. In particular, we'll replace our import of MyModel with those of User and Page. We'll also change the very end of the script to create two User objects (basic and editor) as well as a Page, rather than a MyModel, and add them to our dbsession.

Open tutorial/scripts/initializedb.py and edit it to look like the following:

import os
import sys
import transaction

from pyramid.paster import (
    get_appsettings,
    setup_logging,
    )

from pyramid.scripts.common import parse_vars

from ..models.meta import Base
from ..models import (
    get_engine,
    get_session_factory,
    get_tm_session,
    )
from ..models import Page, User


def usage(argv):
    cmd = os.path.basename(argv[0])
    print('usage: %s <config_uri> [var=value]\n'
          '(example: "%s development.ini")' % (cmd, cmd))
    sys.exit(1)


def main(argv=sys.argv):
    if len(argv) < 2:
        usage(argv)
    config_uri = argv[1]
    options = parse_vars(argv[2:])
    setup_logging(config_uri)
    settings = get_appsettings(config_uri, options=options)

    engine = get_engine(settings)
    Base.metadata.create_all(engine)

    session_factory = get_session_factory(engine)

    with transaction.manager:
        dbsession = get_tm_session(session_factory, transaction.manager)

        editor = User(name='editor', role='editor')
        editor.set_password('editor')
        dbsession.add(editor)

        basic = User(name='basic', role='basic')
        basic.set_password('basic')
        dbsession.add(basic)

        page = Page(
            name='FrontPage',
            creator=editor,
            data='This is the front page',
        )
        dbsession.add(page)

Only the highlighted lines need to be changed.

Installing the project and re-initializing the database

Because our model has changed, and in order to reinitialize the database, we need to rerun the initialize_tutorial_db command to pick up the changes we've made to both the models.py file and to the initializedb.py file. See Initializing the database for instructions.

Success will look something like this:

2016-12-20 02:51:11,195 INFO  [sqlalchemy.engine.base.Engine:1235][MainThread] SELECT CAST('test plain returns' AS VARCHAR(60)) AS anon_1
2016-12-20 02:51:11,195 INFO  [sqlalchemy.engine.base.Engine:1236][MainThread] ()
2016-12-20 02:51:11,195 INFO  [sqlalchemy.engine.base.Engine:1235][MainThread] SELECT CAST('test unicode returns' AS VARCHAR(60)) AS anon_1
2016-12-20 02:51:11,195 INFO  [sqlalchemy.engine.base.Engine:1236][MainThread] ()
2016-12-20 02:51:11,196 INFO  [sqlalchemy.engine.base.Engine:1140][MainThread] PRAGMA table_info("pages")
2016-12-20 02:51:11,196 INFO  [sqlalchemy.engine.base.Engine:1143][MainThread] ()
2016-12-20 02:51:11,196 INFO  [sqlalchemy.engine.base.Engine:1140][MainThread] PRAGMA table_info("users")
2016-12-20 02:51:11,197 INFO  [sqlalchemy.engine.base.Engine:1143][MainThread] ()
2016-12-20 02:51:11,197 INFO  [sqlalchemy.engine.base.Engine:1140][MainThread]
CREATE TABLE users (
        id INTEGER NOT NULL,
        name TEXT NOT NULL,
        role TEXT NOT NULL,
        password_hash TEXT,
        CONSTRAINT pk_users PRIMARY KEY (id),
        CONSTRAINT uq_users_name UNIQUE (name)
)


2016-12-20 02:51:11,197 INFO  [sqlalchemy.engine.base.Engine:1143][MainThread] ()
2016-12-20 02:51:11,198 INFO  [sqlalchemy.engine.base.Engine:719][MainThread] COMMIT
2016-12-20 02:51:11,199 INFO  [sqlalchemy.engine.base.Engine:1140][MainThread]
CREATE TABLE pages (
        id INTEGER NOT NULL,
        name TEXT NOT NULL,
        data TEXT NOT NULL,
        creator_id INTEGER NOT NULL,
        CONSTRAINT pk_pages PRIMARY KEY (id),
        CONSTRAINT uq_pages_name UNIQUE (name),
        CONSTRAINT fk_pages_creator_id_users FOREIGN KEY(creator_id) REFERENCES users (id)
)


2016-12-20 02:51:11,199 INFO  [sqlalchemy.engine.base.Engine:1143][MainThread] ()
2016-12-20 02:51:11,200 INFO  [sqlalchemy.engine.base.Engine:719][MainThread] COMMIT
2016-12-20 02:51:11,755 INFO  [sqlalchemy.engine.base.Engine:679][MainThread] BEGIN (implicit)
2016-12-20 02:51:11,755 INFO  [sqlalchemy.engine.base.Engine:1140][MainThread] INSERT INTO users (name, role, password_hash) VALUES (?, ?, ?)
2016-12-20 02:51:11,755 INFO  [sqlalchemy.engine.base.Engine:1143][MainThread] ('editor', 'editor', '$2b$12$ds7h2Zb7.l6TEFup5h8f4ekA9GRfEpE1yQGDRvT9PConw73kKuupG')
2016-12-20 02:51:11,756 INFO  [sqlalchemy.engine.base.Engine:1140][MainThread] INSERT INTO users (name, role, password_hash) VALUES (?, ?, ?)
2016-12-20 02:51:11,756 INFO  [sqlalchemy.engine.base.Engine:1143][MainThread] ('basic', 'basic', '$2b$12$KgruXP5Vv7rikr6dGB3TF.flGXYpiE0Li9K583EVomjY.SYmQOsyi')
2016-12-20 02:51:11,757 INFO  [sqlalchemy.engine.base.Engine:1140][MainThread] INSERT INTO pages (name, data, creator_id) VALUES (?, ?, ?)
2016-12-20 02:51:11,757 INFO  [sqlalchemy.engine.base.Engine:1143][MainThread] ('FrontPage', 'This is the front page', 1)
2016-12-20 02:51:11,757 INFO  [sqlalchemy.engine.base.Engine:719][MainThread] COMMIT

View the application in a browser

We can't. At this point, our system is in a "non-runnable" state; we'll need to change view-related files in the next chapter to be able to start the application successfully. If you try to start the application (see Start the application), you'll wind up with a Python traceback on your console that ends with this exception:

ImportError: cannot import name MyModel

This will also happen if you attempt to run the tests.