The Flask Mega-Tutorial, Part IV: Database (2012)

Posted by
on under

(Great news! There is a new version of this tutorial!)

This is the fourth article in the series in which I document my experience writing web applications in Python using the Flask microframework.

The goal of the tutorial series is to develop a decently featured microblogging application that demonstrating total lack of originality I have decided to call microblog.

NOTE: This article was revised in September 2014 to be in sync with current versions of Python and Flask.

Here is an index of all the articles in the series that have been published to date:


In the previous chapter of the series we created our login form, complete with submission and validation.

In this article we are going to create our database and set it up so that we can record our users in it.

To follow this chapter along you need to have the microblog app as we left it at the end of the previous chapter. Please make sure the app is installed and running.

Running Python scripts from the command line

In this chapter we are going to write a few scripts that simplify the management of our database. Before we get into that let's review how a Python script is executed on the command line.

If you are on Linux or OS X, then scripts have to be given executable permission, like this:

$ chmod a+x

The script has a shebang line, which points to the interpreter that should be used. A script that has been given executable permission and has a shebang line can be executed simply like this:

./ <arguments>

On Windows, however, this does not work, and instead you have to provide the script as an argument to the chosen Python interpreter:

$ flask\Scripts\python <arguments>

To avoid having to type the path to the Python interpreter you can add your microblog/flask/Scripts directory to the system path, making sure it appears before your regular Python interpreter. This can be temporarily achieved by activating the virtual environment with the following command:

$ flask\Scripts\activate

From now on, in this tutorial the Linux/OS X syntax will be used for brevity. If you are on Windows remember to convert the syntax appropriately.

Databases in Flask

We will use the Flask-SQLAlchemy extension to manage our application. This extension provides a wrapper for the SQLAlchemy project, which is an Object Relational Mapper or ORM.

ORMs allow database applications to work with objects instead of tables and SQL. The operations performed on the objects are translated into database commands transparently by the ORM. Knowing SQL can be very helpful when working with ORMs, but we will not be learning SQL in this tutorial, we will let Flask-SQLAlchemy speak SQL for us.


Most database tutorials I've seen cover creation and use of a database, but do not adequately address the problem of updating a database as the application grows. Typically you end up having to delete the old database and create a new one each time you need to make updates, losing all the data. And if the data cannot be recreated easily you may be forced to write export and import scripts yourself.

Luckily, we have a much better option.

We are going to use SQLAlchemy-migrate to keep track of database updates for us. It adds a bit of work to get a database started, but that is a small price to pay for never having to worry about manual database migrations.

Enough theory, let's get started!


For our little application we will use a sqlite database. The sqlite databases are the most convenient choice for small applications, as each database is stored in a single file and there is no need to start a database server.

We have a couple of new configuration items to add to our config file (file

import os
basedir = os.path.abspath(os.path.dirname(__file__))

SQLALCHEMY_DATABASE_URI = 'sqlite:///' + os.path.join(basedir, 'app.db')
SQLALCHEMY_MIGRATE_REPO = os.path.join(basedir, 'db_repository')

The SQLALCHEMY_DATABASE_URI is required by the Flask-SQLAlchemy extension. This is the path of our database file.

The SQLALCHEMY_MIGRATE_REPO is the folder where we will store the SQLAlchemy-migrate data files.

Finally, when we initialize our app we also need to initialize our database. Here is our updated package init file (file app/

from flask import Flask
from flask_sqlalchemy import SQLAlchemy

app = Flask(__name__)
db = SQLAlchemy(app)

from app import views, models

Note the two changes we have made to our init script. We are now creating a db object that will be our database, and we are also importing a new module called models. We will write this module next.

The database model

The data that we will store in our database will be represented by a collection of classes that are referred to as the database models. The ORM layer will do the translations required to map objects created from these classes into rows in the proper database table.

Let's start by creating a model that will represent our users. Using the WWW SQL Designer tool, I have made the following diagram to represent our users table:

users table

The id field is usually in all models, and is used as the primary key. Each user in the database will be assigned a unique id value, stored in this field. Luckily this is done automatically for us, we just need to provide the id field.

The nickname and email fields are defined as strings (or VARCHAR in database jargon), and their maximum lengths are specified so that the database can optimize space usage.

Now that we have decided what we want our users table to look like, the job of translating that into code is pretty easy (file app/

from app import db

class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    nickname = db.Column(db.String(64), index=True, unique=True)
    email = db.Column(db.String(120), index=True, unique=True)

    def __repr__(self):
        return '<User %r>' % (self.nickname)

The User class that we just created contains several fields, defined as class variables. Fields are created as instances of the db.Column class, which takes the field type as an argument, plus other optional arguments that allow us, for example, to indicate which fields are unique and indexed.

The __repr__ method tells Python how to print objects of this class. We will use this for debugging.

Creating the database

With the configuration and model in place we are now ready to create our database file. The SQLAlchemy-migrate package comes with command line tools and APIs to create databases in a way that allows easy updates in the future, so that is what we will use. I find the command line tools a bit awkward to use, so instead I have written my own set of little Python scripts that invoke the migration APIs.

Here is a script that creates the database (file

from migrate.versioning import api
from config import SQLALCHEMY_DATABASE_URI
from config import SQLALCHEMY_MIGRATE_REPO
from app import db
import os.path
if not os.path.exists(SQLALCHEMY_MIGRATE_REPO):
    api.create(SQLALCHEMY_MIGRATE_REPO, 'database repository')

Note how this script is completely generic. All the application specific pathnames are imported from the config file. When you start your own project you can just copy the script to the new app's directory and it will work right away.

To create the database you just need to execute this script (remember that if you are on Windows the command is slightly different):


After you run the command you will have a new app.db file. This is an empty sqlite database, created from the start to support migrations. You will also have a db_repository directory with some files inside. This is the place where SQLAlchemy-migrate stores its data files. Note that we do not regenerate the repository if it already exists. This will allow us to recreate the database while leaving the existing repository if we need to.

Our first migration

Now that we have defined our model, we can incorporate it into our database. We will consider any changes to the structure of the app database a migration, so this is our first, which will take us from an empty database to a database that can store users.

To generate a migration I use another little Python helper script (file

import imp
from migrate.versioning import api
from app import db
from config import SQLALCHEMY_DATABASE_URI
from config import SQLALCHEMY_MIGRATE_REPO
migration = SQLALCHEMY_MIGRATE_REPO + ('/versions/' % (v+1))
tmp_module = imp.new_module('old_model')
exec(old_model, tmp_module.__dict__)
script = api.make_update_script_for_model(SQLALCHEMY_DATABASE_URI, SQLALCHEMY_MIGRATE_REPO, tmp_module.meta, db.metadata)
open(migration, "wt").write(script)
print('New migration saved as ' + migration)
print('Current database version: ' + str(v))

The script looks complicated, but it doesn't really do much. The way SQLAlchemy-migrate creates a migration is by comparing the structure of the database (obtained in our case from file app.db) against the structure of our models (obtained from file app/ The differences between the two are recorded as a migration script inside the migration repository. The migration script knows how to apply a migration or undo it, so it is always possible to upgrade or downgrade a database format.

While I have never had problems generating migrations automatically with the above script, I could see that sometimes it would be hard to determine what changes were made just by comparing the old and the new format. To make it easy for SQLAlchemy-migrate to determine the changes I never rename existing fields, I limit my changes to adding or removing models or fields, or changing types of existing fields. And I always review the generated migration script to make sure it is right.

It goes without saying that you should never attempt to migrate your database without having a backup, in case something goes wrong. Also never run a migration for the first time on a production database, always make sure the migration works correctly on a development database.

So let's go ahead and record our migration:

$ ./

And the output from the script will be:

New migration saved as db_repository/versions/
Current database version: 1

The script shows where the migration script was stored, and also prints the current database version. The empty database version was version 0, after we migrated to include users we are at version 1.

Database upgrades and downgrades

By now you may be wondering why is it that important to go through the extra hassle of recording database migrations.

Imagine that you have your application in your development machine, and also have a copy deployed to a production server that is online and in use.

Let's say that for the next release of your app you have to introduce a change to your models, for example a new table needs to be added. Without migrations you would need to figure out how to change the format of your database, both in your development machine and then again in your server, and this could be a lot of work.

If you have database migration support, then when you are ready to release the new version of the app to your production server you just need to record a new migration, copy the migration scripts to your production server and run a simple script that applies the changes for you. The database upgrade can be done with this little Python script (file

from migrate.versioning import api
from config import SQLALCHEMY_DATABASE_URI
from config import SQLALCHEMY_MIGRATE_REPO
print('Current database version: ' + str(v))

When you run the above script, the database will be upgraded to the latest revision, by applying the migration scripts stored in the database repository.

It is not a common need to have to downgrade a database to an old format, but just in case, SQLAlchemy-migrate supports this as well (file

from migrate.versioning import api
from config import SQLALCHEMY_DATABASE_URI
from config import SQLALCHEMY_MIGRATE_REPO
print('Current database version: ' + str(v))

This script will downgrade the database one revision. You can run it multiple times to downgrade several revisions.

Database relationships

Relational databases are good at storing relations between data items. Consider the case of a user writing a blog post. The user will have a record in the users table, and the post will have a record in the posts table. The most efficient way to record who wrote a given post is to link the two related records.

Once a link between a user and a post is established there are two types of queries that we may need to use. The most trivial one is when you have a blog post and need to know what user wrote it. A more complex query is the reverse of this one. If you have a user, you may want to know all the posts that the user wrote. Flask-SQLAlchemy will help us with both types of queries.

Let's expand our database to store posts, so that we can see relationships in action. For this we go back to our database design tool and create a posts table:

users table

Our posts table will have the required id, the body of the post and a timestamp. Not much new there. But the user_id field deserves an explanation.

We said we wanted to link users to the posts that they write. The way to do that is by adding a field to the post that contains the id of the user that wrote it. This id is called a foreign key. Our database design tool shows foreign keys as a link between the foreign key and the id field of the table it refers to. This kind of link is called a one-to-many relationship, one user writes many posts.

Let's modify our models to reflect these changes (app/

from app import db

class User(db.Model):
    id = db.Column(db.Integer, primary_key=True)
    nickname = db.Column(db.String(64), index=True, unique=True)
    email = db.Column(db.String(120), index=True, unique=True)
    posts = db.relationship('Post', backref='author', lazy='dynamic')

    def __repr__(self):
        return '<User %r>' % (self.nickname)

class Post(db.Model):
    id = db.Column(db.Integer, primary_key = True)
    body = db.Column(db.String(140))
    timestamp = db.Column(db.DateTime)
    user_id = db.Column(db.Integer, db.ForeignKey(''))

    def __repr__(self):
        return '<Post %r>' % (self.body)

We have added the Post class, which will represent blog posts written by users. The user_id field in the Post class was initialized as a foreign key, so that Flask-SQLAlchemy knows that this field will link to a user.

Note that we have also added a new field to the User class called posts, that is constructed as a db.relationship field. This is not an actual database field, so it isn't in our database diagram. For a one-to-many relationship a db.relationship field is normally defined on the "one" side. With this relationship we get a user.posts member that gets us the list of posts from the user. The first argument to db.relationship indicates the "many" class of this relationship. The backref argument defines a field that will be added to the objects of the "many" class that points back at the "one" object. In our case this means that we can use to get the User instance that created a post. Don't worry if these details don't make much sense just yet, we'll see examples of this at the end of this article.

Let's record another migration with this change. Simply run:

$ ./

And the script will respond:

New migration saved as db_repository/versions/
Current database version: 2

It isn't really necessary to record each little change to the database model as a separate migration, a migration is normally only recorded at significant points in the history of the project. We are doing more migrations than necessary here only to show how the migration system works.

Play time

We have spent a lot of time defining our database, but we haven't seen how it works yet. Since our app does not have database code yet let's make use of our brand new database in the Python interpreter.

So go ahead and fire up Python. On Linux or OS X:


Or on Windows:


Once in the Python prompt enter the following:

>>> from app import db, models

This brings our database and models into memory.

Let's create a new user:

>>> u = models.User(nickname='john', email='')
>>> db.session.add(u)
>>> db.session.commit()

Changes to a database are done in the context of a session. Multiple changes can be accumulated in a session and once all the changes have been registered you can issue a single db.session.commit(), which writes the changes atomically. If at any time while working on a session there is an error, a call to db.session.rollback() will revert the database to its state before the session was started. If neither commit nor rollback are issued then the system by default will roll back the session. Sessions guarantee that the database will never be left in an inconsistent state.

Let's add another user:

>>> u = models.User(nickname='susan', email='')
>>> db.session.add(u)
>>> db.session.commit()

Now we can query what our users are:

>>> users = models.User.query.all()
>>> users
[<User u'john'>, <User u'susan'>]
>>> for u in users:
...     print(,u.nickname)
1 john
2 susan

For this we have used the query member, which is available in all model classes. Note how the id member was automatically set for us.

Here is another way to do queries. If we know the id of a user we can find the data for that user as follows:

>>> u = models.User.query.get(1)
>>> u
<User u'john'>

Now let's add a blog post:

>>> import datetime
>>> u = models.User.query.get(1)
>>> p = models.Post(body='my first post!', timestamp=datetime.datetime.utcnow(), author=u)
>>> db.session.add(p)
>>> db.session.commit()

Here we set our timestamp in UTC time zone. All timestamps stored in our database will be in UTC. We can have users from all over the world writing posts and we need to use uniform time units. In a future tutorial we will see how to show these times to users in their local timezone.

You may have noticed that we have not set the user_id field of the Post class. Instead, we are storing a User object inside the author field. The author field is a virtual field that was added by Flask-SQLAlchemy to help with relationships, we have defined the name of this field in the backref argument to db.relationship in our model. With this information the ORM layer will know how to complete the user_id for us.

To complete this session, let's look at a few more database queries that we can do:

# get all posts from a user
>>> u = models.User.query.get(1)
>>> u
<User u'john'>
>>> posts = u.posts.all()
>>> posts
[<Post u'my first post!'>]

# obtain author of each post
>>> for p in posts:
...     print(,,p.body)
1 john my first post!

# a user that has no posts
>>> u = models.User.query.get(2)
>>> u
<User u'susan'>
>>> u.posts.all()

# get all users in reverse alphabetical order
>>> models.User.query.order_by('nickname desc').all()
[<User u'susan'>, <User u'john'>]

The Flask-SQLAlchemy documentation is the best place to learn about the many options that are available to query the database.

Before we close, let's erase the test users and posts we have created, so that we can start from a clean database in the next chapter:

>>> users = models.User.query.all()
>>> for u in users:
...     db.session.delete(u)
>>> posts = models.Post.query.all()
>>> for p in posts:
...     db.session.delete(p)
>>> db.session.commit()

Final words

This was a long tutorial. We have learned the basics of working with a database, but we haven't incorporated the database into our application yet. In the next chapter we will put all we have learned about databases into practice when we look at our user login system.

In the meantime, if you haven't been writing the application along, you may want to download it in its current state:


Note that I have not included a database in the zip file above, but the repository with the migrations is there. To create a new database just use the script, then use to upgrade the database to the latest revision.

I hope to see you next time!


Become a Patron!

Hello, and thank you for visiting my blog! If you enjoyed this article, please consider supporting my work on this blog on Patreon!

  • #26 drew said

    I'm trying to follow along this tutorial only 'm deploying my app on heroku. Any hints on what i might need to alter in order to get the migrations,db_create,db_upgrade,etc... to work?

  • #27 Miguel Grinberg said

    @drew: to run on Heroku you will be using a Postgres database, so you will need to change the app's configuration to tell SQLAlchemy that you are using Postgres. You may want to test this locally first, to make sure everything works. If you get the database scripts to work locally, then I expect they'll also work via 'heroku run'.

  • #28 OG said

    I get the following error and I am not sure why:

    Traceback (most recent call last):
    File "./", line 2, in <module>
    from migrate.versioning import api
    File "~/microblog/flask/local/lib/python2.7/site-packages/migrate/versioning/", line 33, in <module>
    from migrate.versioning import (repository, schema, version,
    File "~/microblog/flask/local/lib/python2.7/site-packages/migrate/versioning/", line 10, in <module>
    from sqlalchemy import exceptions as sa_exceptions
    ImportError: cannot import name exceptions

    What am I doing wrong?

  • #29 Napoleon said

    To whom it may concern, sqlalchemy-migrate==0.7.2 is not compatible with SQLAlchemy 0.8.x. install SQLAlchemy 0.7.x instead with pip install SQLAlchemy==0.7.x. Don't forget sudo if you are performing a global install.

  • #30 Miguel Grinberg said

    @Napoleon: You are correct, looks like sqlalchemy-migrate does not work with SQLAlchemy 0.8, so your advice is good until compatibility problems are resolved.

  • #31 Miguel Grinberg said

    @OG: Unfortunately the SQLAlchemy guys released 0.8 a few days ago, and they did not preserve compatibility with the 0.7 releases (shame on them!). The sqlalchemy-migrate module does not work with the new 0.8 release. The workaround is to force the 0.7 version to be installed, using "flask/bin/pip install sqlalchemy==0.7.9".

  • #32 Rob said

    Thank you. This has been a great tutorial. It is helping me understand how to put together a web app and how they work. I tried Rails and Django while I could get an app up pretty quickly I was soon lost in all the hocus pocus magic so it was difficult to learn what was actually happening. Great stuff this is forcing me to learn what id going on.

    One thing that tripped me up in this chapter, for some reason the SQLALCHEMY_DATABASE_URI config lines had to be at the bottom of my config file. If it was above any of the other lines in the config it would throw an import error when the db_create was run. Do you have any thoughts as to why this is? Thanks again. Small world I am in portland too. Take care

  • #33 Miguel Grinberg said

    @Rob: I can't think of a reason why the position of the constant within the file would change things. I certainly do not have it at the bottom (see it here: so I'm not sure what the problem is. Feel free to paste your stack trace, maybe that'll give me a clue.

  • #34 Indjeet said

    Seems migrate is no longer maintained as there was no commit in the last year. seems to be a viable alternative. It is from the author of SQLAlchemy.

  • #35 Miguel Grinberg said

    @Indjeet: Yes, the fact that the migrate project is incompatible with SQLAlchemy 0.8 is a problem. At some point I will look at Alembic and maybe I'll switch. Thanks.

  • #36 Yifan Wu said

    Hi! So I was wondering how is recognized; is it because it's part of the User class? (the fact it's not in caps is a bit weird; thanks!)

  • #37 Miguel Grinberg said

    @Yifan: I agree it is a bit weird. The mapping is done using the table name, which SQLAlchemy defaults to the lower case version of the class name.

  • #38 Luis Villamarin said

    Miguel, one question: If I was to use inheritance I was reviewing the documentation and found that the entities inherit from 'Base' instead of 'Model'. Could you explain why you used 'Base' and how can use inheritance in this case. Thanks in advance.

  • #39 Miguel Grinberg said

    @Luis: the 'Base' class is required for using the declarative APIs of SQLAlchemy. What is the problem with inheritance that you noticed? Have you seen the discussion on inheritance at the SQLAlchemy site?

  • #40 Lunamystry said

    Thank you for a great tutorial. I thought I would share this, since I just found it.
    I have an existing database and I wanted to use Flask-SQLAlchemy with it. The database more than 10 table (I don't claim to be good at SQL). I found this:

    I installed it using pip install sqlautocode
    then I did sqlautocode mysql://root:secret@localhost:3306/existingdb -d -o
    With the same configuration you gave for the database and Flask-SQLAlchemy, I changed the generated thus:

    from app import db
    from sqlalchemy import * from sqlalchemy.orm import relation

    engine = db.engine
    DeclarativeBase = db.Model
    metadata = db.metadata
    metadata.bind = db.engine

    This is all at the top of the file and I left the rest of the file. I am about to change some of the generated table names using search and replace and add the required Flask-Login functions. Hope this helps someone.

  • #41 somit said

    "sqlalchemy.exc.InvalidRequestError: One or more mappers failed to initialize - can't proceed with initialization of other mappers. Original exception was: Could not determine join condition between parent/child tables on relationship User.posts. Specify a 'primaryjoin' expression. If 'secondary' is present, 'secondaryjoin' is needed as well."

    I get this error on inserting first record in user table

  • #42 Miguel Grinberg said

    @somit: you may want to compare your with mine (on github) to see if you find any mistakes.

  • #43 Isxek said

    Thanks for creating the tutorial. I'm getting the following error when I run

    $ flask\Scripts\python
    Traceback (most recent call last):
    File "", line 2, in <module>
    from migrate.versioning import api
    ImportError: No module named migrate.versioning

    Where does migrate.versioning come from?

  • #44 Miguel Grinberg said

    @Isxek: you need to install sqlalchemy-migrate. See the first article in the series for installation instructions.

  • #45 Isxek said

    Hi Miguel, please ignore my previous comment. I backtracked my steps, and it looks like I neglected to run the sqlalchemy-migrate install. Thanks again for writing this tutorial!

  • #46 Aniruddha said


    Thanks for in dept tutorial. I am following the tutorial. I created the,, & But when I run db_create script or db_migrate script or db_upgrade script. It wont create user table or post table, niether it shows any error. The only table I have in database in migrate_version.

    One more point : I am using mysql

  • #47 Miguel Grinberg said

    @Aniruddha: Do you get the same problem if you use a sqlite database like I? I have tried at some point to run this application with a mysql database and didn't find any problems, but that was a while ago.

  • #48 Raphael said

    Hello Miguel...
    I'm having problem with the db_create...


    Traceback (most recent call last):
    File "", line 7, in <module>
    File "C:\flaskr\venv\lib\site-packages\", line 822, in crea
    self.execute_for_all_tables(app, bind, 'create_all')
    File "C:\flaskr\venv\lib\site-packages\", line 814, in _exe
    op(bind=self.get_engine(app, bind), tables=tables)
    File "C:\flaskr\venv\lib\site-packages\", line 763, in get

    return connector.get_engine()
    File "C:\flaskr\venv\lib\site-packages\", line 435, in get_
    info = make_url(uri)
    File "C:\flaskr\venv\lib\site-packages\sqlalchemy\engine\", line 178, in
    return _parse_rfc1738_args(name_or_url)
    File "C:\flaskr\venv\lib\site-packages\sqlalchemy\engine\", line 216, in
    return URL(name, **components)
    File "C:\flaskr\venv\lib\site-packages\sqlalchemy\engine\", line 56, in
    self.port = int(port)
    ValueError: invalid literal for int() with base 10: '\flaskr\app.db'

  • #49 Miguel Grinberg said

    @Raphael: The error is pretty clear. It appears you are not using sqlite but another database that takes a port as part of the configuration. The error says that you assigned the string "\flaskr\app.db" to the port in the configuration, but the port is expected to be an integer.

  • #50 andy matthews said

    Just FYI, I'm following along with your tutorial and got an error trying to import exceptions from sqlalchemy. Looks like that package was moved:

    In case you want to update your tutorial