The Flask Mega-Tutorial, Part IX: Pagination

This is the ninth article in the series in which I document my experience writing web applications in Python using the Flask microframework.

The goal of the tutorial series is to develop a decently featured microblogging application that demonstrating total lack of originality I have decided to call microblog.

NOTE: This article was revised in September 2014 to be in sync with current versions of Python and Flask.

Here is an index of all the articles in the series that have been published to date:

Recap

In the previous article in the series we've made all the database changes necessary to support the 'follower' paradigm, where users choose other users to follow.

Today we will build on what we did last time and enable our application to accept and deliver real content to its users. We are saying goodbye to the last of our fake objects today!

Submission of blog posts

Let's start with something simple. The home page should have a form for users to submit new posts.

First we define a single field form object (file app/forms.py):

class PostForm(Form):
    post = StringField('post', validators=[DataRequired()])

Next, we add the form to the template (file app/templates/index.html):

<!-- extend base layout -->
{% extends "base.html" %}

{% block content %}
  <h1>Hi, {{ g.user.nickname }}!</h1>
  <form action="" method="post" name="post">
      {{ form.hidden_tag() }}
      <table>
          <tr>
              <td>Say something:</td>
              <td>{{ form.post(size=30, maxlength=140) }}</td>
              <td>
              {% for error in form.post.errors %}
              <span style="color: red;">[{{ error }}]</span><br>
              {% endfor %}
              </td>
          </tr>
          <tr>
              <td></td>
              <td><input type="submit" value="Post!"></td>
              <td></td>
          </tr>
      </table>
  </form>
  {% for post in posts %}
  <p>
    {{ post.author.nickname }} says: <b>{{ post.body }}</b>
  </p>
  {% endfor %}
{% endblock %}

Nothing earth shattering so far, as you can see. We are simply adding yet another form, like the ones we've done before.

Last of all, the view function that ties everything together is expanded to handle the form (file app/views.py):

from forms import LoginForm, EditForm, PostForm
from models import User, Post

@app.route('/', methods=['GET', 'POST'])
@app.route('/index', methods=['GET', 'POST'])
@login_required
def index():
    form = PostForm()
    if form.validate_on_submit():
        post = Post(body=form.post.data, timestamp=datetime.utcnow(), author=g.user)
        db.session.add(post)
        db.session.commit()
        flash('Your post is now live!')
        return redirect(url_for('index'))
    posts = [
        { 
            'author': {'nickname': 'John'}, 
            'body': 'Beautiful day in Portland!' 
        },
        { 
            'author': {'nickname': 'Susan'}, 
            'body': 'The Avengers movie was so cool!' 
        }
    ]
    return render_template('index.html',
                           title='Home',
                           form=form,
                           posts=posts)

Let's review the changes in this function one by one:

  • We are now importing the Post and PostForm classes
  • We accept POST requests in both routes associated with the index view function, since that is how we will receive submitted posts.
  • When we arrive at this view function through a form submission we insert a new Post record into the database. When we arrive at it via a regular GET request we do as before.
  • The template now receives an additional argument, the form, so that it can render the text field.

One final comment before we continue. Notice how after we insert a new Post into the detabase we do this:

return redirect(url_for('index'))

We could have easily skipped the redirect and allowed the function to continue down into the template rendering part, and it would have been more efficient. Because really, all the redirect does is return to this same view function to do that, after an extra trip to the client web browser.

So, why the redirect? Consider what happens after the user writes a blog post, submits it and then hits the browser's refresh key. What will the refresh command do? Browsers resend the last issued request as a result of a refresh command.

Without the redirect, the last request is the POST request that submitted the form, so a refresh action will resubmit the form, causing a second Post record that is identical to the first to be written to the database. Not good.

By having the redirect, we force the browser to issue another request after the form submission, the one that grabs the redirected page. This is a simple GET request, so a refresh action will now repeat the GET request instead of submitting the form again.

This simple trick avoids inserting duplicate posts when a user inadvertently refreshes the page after submitting a blog post.

Displaying blog posts

And now we get to the fun part. We are going to grab blog posts from the database and display them.

If you recall from a few articles ago, we created a couple of fake posts and we've been displaying those in our home page for a long time. The fake objects were created explicitly in the index view function as a simply Python list:

    posts = [
        { 
            'author': {'nickname': 'John'}, 
            'body': 'Beautiful day in Portland!' 
        },
        { 
            'author': {'nickname': 'Susan'}, 
            'body': 'The Avengers movie was so cool!' 
        }
    ]

But in the last article we created the query that allows us to get all the posts from followed users, so now we can simply replace the above with this (file app/views.py):

    posts = g.user.followed_posts().all()

And when you run the application you will be seeing blog posts from the database!

The followed_posts method of the User class returns a sqlalchemy query object that is configured to grab the posts we are interested in. Calling all() on this query just retrieves all the posts into a list, so we end up with a structure that is very much alike the fake one we've been using until now. It's so close that the template does not even notice.

At this point feel free to play with the application. You can create a few users, make them follow others, and finally post some messages to see how each user sees its blog post stream.

Pagination

The application is looking better than ever, but we have a problem. We are showing all of the followed posts in the home page. What happens if a user has a thousand followed posts? Or a million? As you can imagine, grabbing and handling such a large list of objects will be extremely inefficient.

Instead, we are going to show this potentially large number of posts in groups, or pages.

Flask-SQLAlchemy comes with very good support for pagination. If for example, we wanted to get the first three followed posts of some user we can do this:

    posts = g.user.followed_posts().paginate(1, 3, False).items

The paginate method can be called on any query object. It takes three arguments:

  • the page number, starting from 1,
  • the number of items per page,
  • an error flag. If True, when an out of range page is requested a 404 error will be automatically returned to the client web browser. If False, an empty list will be returned instead of an error.

The return value from paginate is a Pagination object. The items member of this object contains the list of items in the requested page. There are other useful things in the Pagination object that we will see a bit later.

Now let's think about how we can implement pagination in our index view function. We can start by adding a configuration item to our application that determines how many items per page we will display (file config.py):

# pagination
POSTS_PER_PAGE = 3

It is a good idea to have these global knobs that can change the behavior of our application in the configuration file all together, because then we can go to a single place to revise them all.

In the final application we will of course use a much larger number than 3, but for testing it is useful to work with small numbers.

Next, let's decide how the URLs that request different pages will look. We've seen before that Flask routes can take arguments, so we can add a suffix to the URL that indicates the desired page:

http://localhost:5000/         <-- page #1 (default)
http://localhost:5000/index    <-- page #1 (default)
http://localhost:5000/index/1  <-- page #1
http://localhost:5000/index/2  <-- page #2

This format of URLs can be easily implemented with an additional route added to our view function (file app/views.py):

from config import POSTS_PER_PAGE

@app.route('/', methods=['GET', 'POST'])
@app.route('/index', methods=['GET', 'POST'])
@app.route('/index/<int:page>', methods=['GET', 'POST'])
@login_required
def index(page=1):
    form = PostForm()
    if form.validate_on_submit():
        post = Post(body=form.post.data, timestamp=datetime.utcnow(), author=g.user)
        db.session.add(post)
        db.session.commit()
        flash('Your post is now live!')
        return redirect(url_for('index'))
    posts = g.user.followed_posts().paginate(page, POSTS_PER_PAGE, False).items
    return render_template('index.html',
                           title='Home',
                           form=form,
                           posts=posts)

Our new route takes the page argument, and declares it as an integer. We also need to add the page argument to the index function, and we have to give it a default value because two of the three routes do not have this argument, so for those the default will always be used.

And now that we have a page number available to us we can easily hook it up to our followed_posts query, along with the POSTS_PER_PAGE configuration constant we defined earlier.

Note how easy these changes are, and how little code is affected each time we make a change. We are trying to write each part of the application without making any assumptions regarding how the other parts work, and this enables us to write modular and robust applications that are easier to test and are less likely to fail or have bugs.

At this point you can try the pagination by entering URLs for the different pages by hand into your browser's address bar. Make sure you have more than three posts available so that you can see more than one page.

Page navigation

We now need to add links that allow users to navigate to the next and/or previous pages, and luckily this is extremely easy to do, Flask-SQLAlchemy does most of the work for us.

We are going to start by making a small change in the view function. In our current version we use the paginate method as follows:

posts = g.user.followed_posts().paginate(page, POSTS_PER_PAGE, False).items

By doing this we are only keeping the items member of the Pagination object returned by paginate. But this object has a number of other very useful things in it, so we will instead keep the whole object (file app/views.py):

posts = g.user.followed_posts().paginate(page, POSTS_PER_PAGE, False)

To compensate for this change, we have to modify the template (file app/templates/index.html):

<!-- posts is a Paginate object -->
{% for post in posts.items %}
<p>
  {{ post.author.nickname }} says: <b>{{ post.body }}</b>
</p>
{% endfor %}

What this change does is make the full Paginate object available to our template. The members of this object that we will use are:

  • has_next: True if there is at least one more page after the current one
  • has_prev: True if there is at least one more page before the current one
  • next_num: page number for the next page
  • prev_num: page number for the previous page

With these for elements we can produce the following (file app/templates/index.html):

<!-- posts is a Paginate object -->
{% for post in posts.items %}
<p>
  {{ post.author.nickname }} says: <b>{{ post.body }}</b>
</p>
{% endfor %}
{% if posts.has_prev %}<a href="{{ url_for('index', page=posts.prev_num) }}"><< Newer posts</a>{% else %}<< Newer posts{% endif %} | 
{% if posts.has_next %}<a href="{{ url_for('index', page=posts.next_num) }}">Older posts >></a>{% else %}Older posts >>{% endif %}

So we have two links. First we have one labeled "Newer posts" that sends us to the previous page (keep in mind we show posts sorted by newest first, so the first page is the one with the newest stuff). Conversely, the "Older posts" points to the next page.

When we are looking at the first page we do not want to show a link to go to the previous page, since there isn't one. This is easy to detect because posts.has_prev will be False. We handle that case simply by showing the same text of the link but without the link itself. The link to the next page is handled in the same way.

Implementing the Post sub-template

Back in the article where we added avatar pictures we defined a sub-template with the HTML rendering of a single post. The reason we created this sub-template was so that we can render posts with a consistent look in multiple pages, without having to duplicate the HTML code.

It is now time to implement this sub-template in our index page. And, as most of the things we are doing today, it is surprisingly simple (file app/templates/index.html):

<!-- posts is a Paginate object -->
{% for post in posts.items %}
    {% include 'post.html' %}
{% endfor %}

Amazing, huh? We just discarded our old rendering code and replaced it with an include of the sub-template. Just with this, we get the nicer version of the post that includes the user's avatar.

Here is a screenshot of the index page of our application in its current state:

microblog profile page

The user profile page

We are done with the index page for now. However, we have also included posts in the user profile page, not posts from everyone but just from the owner of the profile. To be consistent the user profile page should be changed to match the index page.

The changes are similar to those we made on the index page. Here is a summary of what we need to do:

  • add an additional route that takes the page number
  • add a page argument to the view function, with a default of 1
  • replace the list of fake posts with the proper database query and pagination
  • update the template to use the pagination object

Here is the updated view function (file app/views.py):

@app.route('/user/<nickname>')
@app.route('/user/<nickname>/<int:page>')
@login_required
def user(nickname, page=1):
    user = User.query.filter_by(nickname=nickname).first()
    if user is None:
        flash('User %s not found.' % nickname)
        return redirect(url_for('index'))
    posts = user.posts.paginate(page, POSTS_PER_PAGE, False)
    return render_template('user.html',
                           user=user,
                           posts=posts)

Note that this function already had an argument (the nickname of the user), so we add the page number as a second argument.

The changes to the template are also pretty simple (file app/templates/user.html):

<!-- posts is a Paginate object -->
{% for post in posts.items %}
    {% include 'post.html' %}
{% endfor %}
{% if posts.has_prev %}<a href="{{ url_for('user', nickname=user.nickname, page=posts.prev_num) }}"><< Newer posts</a>{% else %}<< Newer posts{% endif %} | 
{% if posts.has_next %}<a href="{{ url_for('user', nickname=user.nickname, page=posts.next_num) }}">Older posts >></a>{% else %}Older posts >>{% endif %}

Final words

Below I'm making available the updated version of the microblog application with all the pagination changes introduced in this article.

Download microblog-0.9.zip.

As always, a database isn't provided so you have to create your own. If you are following this series of articles you know how to do it. If not, then go back to the database article to find out.

As always, I thank you for following my tutorial. I hope to see you again in the next one!

Miguel

74 comments

  • #1 Siros said :

    Thank you so much again , this is very helpful.

  • #2 Sean said :

    Great series! This is the most helpful Python/Flask tutorial I have read. Thank you very much!!

  • #3 Bobby said :

    This tutorial is amazing....really great!! Thanks for sharing your skills with us. Two small points: your index function example above is missing "user = g.user"; it might be helpful to explain to people what the "form.hidden_tag()" does.

  • #4 Bobby said :

    can you tell me why my route decorators need the trailing slash to work? @app.route('/login/') I'm sure I'm doing something silly??

  • #5 Miguel Grinberg said :

    @Bobby: hidden tags were covered in part 3 of the series. Can you expand on the "user = g.user" comment? What would that achieve?

  • #6 Bobby said :

    in an earlier part of the tutorial, we were passing the user into the index template to say "Hello Bobby"...I guess that got dropped somewhere and I missed it. Can you tell me why my routes need the trailing "/"?

  • #7 Miguel Grinberg said :

    @Bobby: I'm not sure why you need trailing slashes. Are you using the development web server when you run the application? The only idea I can offer is that a different web server might be redirecting requests without a trailing slash. Check in the debugging console of your browser to see what's happening. Then please let me know as I would like to know!

  • #8 Dogukan Tufekci said :

    @Miguel thanks for another great tutorial. You are making my life so easy in this journey to understand Flask. I noticed that posts on a user's profile are not sorted by date. So I tweaked the code this way. Not sure if there's a better way to do this: class User(db.Model): #.... def sorted_posts(self): return Post.query.filter(Post.user_id == self.id).order_by(Post.timestamp.desc())

  • #9 Miguel Grinberg said :

    @Dogukan: you are right, I missed the sorting! You could simplify your solution a bit, the sorted_posts() method can be implemented as "return self.posts.order_by(Post.timestamp.desc())". I will update the article to include this. Thanks.

  • #10 Dogukan Tufekci said :

    @Miguel That's much simpler indeed! Thanks!

  • #11 George Mabley said :

    Hello, I'm not sure if this is the best article to ask this on, but it at least uses the concept. Is there a function similar to redirect which either redirects you to the current page, or refreshes the page you are on? Say I have a view that I can call on both the index and user page. If an arbitrary condition is met, I would like a flash message to occur on the page, and for the page to basically start fresh. However, I have to return something, so I must choose to redirect to url_for('index') or url_for('user'). If what I am asking is not clear, I will gladly provide some code as an example. Thank you!

  • #12 Miguel Grinberg said :

    @George: I'm not completely sure I understand, but I think a good example of what you are asking is the login view. Let's say the user wants to visit some password protected page, so he gets redirected to the login page. Once he enters his credentials you have two options, if the credentials are valid you have to go to the page the user wanted to visit originally, if the credentials are invalid you have to redirect back to the login page. This is implemented with an additional argument sent to the view that needs to decide where to redirect. In the login example if the user needs to access the index page the server will redirect to http://server/login?next=index. If instead the user went to the profile page the redirect will be http://server/login?next=profile. The login page then has the "next" argument in the request to know where to redirect. I hope this helps.

  • #13 George Mabley said :

    Wow, thanks for the quick reply. I think that could work, but I am still hoping there is a simpler solution, Let me try to explain with code here: http://i.imgur.com/OGYJyo2.png. Is there not a way for flask to, if those conditions are met, redirect you to the user page if you are on the user page, and the index page if you are on the index?

  • #14 Miguel Grinberg said :

    Ah, I think I understand it better now. I can think of two ways to handle the problem. One is similar to what I said before, you have to insert something in the request that tells the view function what is the originating page. For example, you could use a more complex route, like "/repost/<source>/<id>" so then the repost view function gets an additional argument that can be "index" or "user". The problem is that you have to build different URLs depending on what view you are in. A more sophisticated solution would be to let the client handle this via an ajax call, which does not trigger a page reload. Then it is up to the Javascript client code to stay on the same page or trigger a reload, based on instructions provided by the Ajax handler on the server. (hint: next article in the series covers ajax). Good luck,

  • #15 abenrob said :

    @Dogukan, Miguel - how are you modifying the User view for the sorted object? we ahve "posts = user.posts.paginate(page, POSTS_PER_PAGE, False)" I tried "user.sorted_posts.paginate(page, POSTS_PER_PAGE, False)" after implenting sorted_view() into the User model, but that isn't working...

  • #16 Miguel Grinberg said :

    @abenrob: sorted_posts is a method, you have to add the parenthesis at the end: "user.sorted_posts().paginate(page, POSTS_PER_PAGE, False)".

  • #17 abenrob said :

    Of course. Thanks again!

  • #18 uldis said :

    How to solve the problem when the user double click the "post!" button? The post with the same content is inserted twice.

  • #19 Miguel Grinberg said :

    @uldis: the standard solution for the double click to a form submit button is to use Javascript to disable the button when it is clicked the first time. You can achieve that simply by adding onclick="this.disabled=true;this.form.submit();" inside the submit button's input element.

  • #20 Napoleon Ahiable said :

    Thank you so much for these amazing tutorials. YOU my friend, are my new favourite dude and I'll be hanging out with you a lot. God bless you for your kind giving heart.

  • #21 Saber Rastikerdar said :

    Thank you for this great tutorial series.

  • #22 Tri said :

    Miguel, Great tutorial once again! Could you please walk me through how to create other users and have them follow other users to test out the functionalities we just made? Kind of like in your screenshot. Thank you!

  • #23 Miguel Grinberg said :

    @Tri: the easiest way is for you to play different users. For example, login to the server with two different browsers, using a different OpenID on each. Then each of these users can follow the other.

  • #24 Tri said :

    I tried that but I keep on getting errors saying, "Invalid login. Please try again", even though it says, "Hi, Tri" underneath. And even when I logged in with another OpenID, the page always says, "Hi, Tri". Then when I click on 'your profile', there's an error that says, "TypeError: must be string or buffer, not None". I tried with many different OpenIDs and different browsers...logging out after each one, but I always get that same exact error where "Tri" is always the one that comes up. However, when I log in through Google with my email, there's no error. That's the only account that has no error, which is why I can't do multiple logins. I don't know where the problem comes from. Hopefully, you know what's going on. Thanks again.

  • #25 Miguel Grinberg said :

    @Tri: you may have found a bug, but the information that you are giving me isn't enough for me to figure out where or why. Could you show me stack traces of the errors that you get?

Leave a Comment

Note: all comments are screened before they are published. Thank you for your patience!