Dynamically Update Your Flask Web Pages Using Turbo-Flask

How can you update a part of a web page dynamically, without the user having to hit the refresh button on the browser?

This is a question that is frequently asked on the Internet. If you have some knowledge of JavaScript this is relatively easy to do, more so if you use a front end web framework such as React or Vue. But what if you have a standard web application written in Flask and Jinja templates?

In this article I'm going to introduce you to my Turbo-Flask extension, which will allow your Flask application to easily push asynchronous page updates to the browser, without having to write any JavaScript code. In the screenshot below, note how the CPU load numbers update without the user doing anything.

What is Turbo-Flask?

Turbo-Flask is a Flask extension that integrates Hotwire's turbo.js JavaScript library with your Flask application.

This library bundles a number of different features, all with the goal of making server generated web pages behave more like single-page applications, but without requiring the application to write any front end code in JavaScript. Here is a list of the four modules included in turbo.js:

• Turbo Drive: to accelerate page navigation by transparently updating pages in the background, without letting the browser ever do a full page reload.
• Turbo Frames: to only update predefined parts of the page when the user clicks on a link or submits a form.
• Turbo Streams: to let the server-side application update parts of the page by submitting HTML fragments to the client.
• Turbo Native: to wrap your application as a native iOS or Android app. This section of turbo.js is out of scope for this article and I have very little experience with it, so I will not discuss it at all.

If you are interested in the Turbo Drive, and Frames features, I suggest you review the turbo.js documentation links above, as these do not require a Flask integration and can be used directly in your Jinja templates.

In this article I want to concentrate on the Turbo Streams feature, which is, in my opinion, the most interesting of the set.

An Example Application

This is going to be a hands-on tutorial, so let's create a short Flask application to which we can later add Turbo-Flask. If you are too lazy to type or copy/paste the code examples below, you can find the complete code for this article in the Turbo-Flask repository on GitHub.

To begin, create a directory in which you will work on this application. You may also want to create a virtual environment and activate it. Once you are ready, install Flask and Turbo-Flask:

(venv) $ pip install flask turbo-flask

Note: The Turbo-Flask package requires Flask version 2.

The Flask application

The code for the base application is shown below. Copy it to a file named app.py:

import random
import re
import sys
from flask import Flask, render_template

app = Flask(__name__)

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/page2')
def page2():
    return render_template('page2.html')

@app.context_processor
def inject_load():
    if sys.platform.startswith('linux'): 
        with open('/proc/loadavg', 'rt') as f:
            load = f.read().split()[0:3]
    else:
        load = [int(random.random() * 100) / 100 for _ in range(3)]
    return {'load1': load[0], 'load5': load[1], 'load15': load[2]}

The application defines two routes on the / and /page2 URLs. Both pages are implemented with Jinja templates named index.html and page2.html.

So far this does not show anything out of the ordinary. The last part of the application is something you may not be familiar with. The inject_load() function is decorated with the @app.context_processor decorator from Flask, which allows the application to add symbols to the Jinja scope. The dictionary that this function returns contains three keys, load1, load5 and load15, which Flask will make available to all Jinja templates rendered with the render_template() function.

The three load numbers are a standard metric that can be obtained in Unix based systems. These numbers give you the CPU load of the system in the last 1, 5 and 15 minutes. The method that I use to obtain these numbers is specific to Linux servers:

        with open('/proc/loadavg', 'rt') as f:
            load = f.read().split()[0:3]

To ensure that you can use this example application under other operating systems, the function provides an alternative implementation that just generates three fake load numbers:

        load = [int(random.random() * 100) / 100 for _ in range(3)]

The Base Template

Since the application will have two pages with largely the same structure, it is a good idea to extract the base layout into a base template that both pages can inherit from. Here is the base.html template, which you will need to store in a templates sub-directory:

<!doctype html>
<html>
  <head>
    <title>Turbo-Flask Streams Demo</title>
    <style>
      .load {
        float: right;
        width: 300px;
        border: 1px solid black;
        margin-left: 10px;
        margin-bottom: 10px;
        padding: 10px;
      }
      .load th, .load td {
        padding: 6px;
        text-align: center;
      }
    </style>
  </head>
  <body>
    {% include "loadavg.html" %}
    {% block content %}{% endblock %}
  </body>
</html>

The base template defines a few styles that create the design of the box with the load average metrics as you see in the screenshot at the start of the article. In particular, note how the float: right CSS attribute will make this part of the page appear on the top right corner of the page, wrapped with content.

The body of the template includes the loadavg.html template, which implements this box, and then defines a content Jinja block that the derived pages can use to provide their page bodies.

The Load Averages Template

The template referenced by the base template implementes the box that shows the three load average values as a table. Save the following page in the templates directory with the name loadavg.html:

<div id="load" class="load">
  <table>
    <tr><th></th><th>1 min</th><th>5 min</th><th>15 min</th></tr>
    <tr><th>Load averages:</th><td>{{ load1 }}</td><td>{{ load5 }}</td><td>{{ load15 }}</td></tr>
  </table>
</div>

Here notice the load1, load5 and load15 variables, which were the ones injected into the Jinja context by the application.

The Page Templates

To complete the base version of the application we need to create the two templates that are rendered by the index() and page2() routes. Since the content of the pages does not really matter much, I created a placeholder page with some dummy text in them, plus a link to the other page. Below you can see the index.html and page2.html templates, which you should copy to the templates sub-directory:

{% extends "base.html" %}

{% block content %}
<h1>Main Page</h1>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>You can visit <a href="/page2">Page 2</a>.</p>
{% endblock %}

{% extends "base.html" %}

{% block content %}
<h1>Page 2</h1>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Go back to the <a href="/">home page</a>.</p>
{% endblock %}

Both templates inherit from the base.html template created earlier, which means that they are going to have the box with the CPU metrics automatically added in the top-right corner of the page.

Running the Application

The application is now complete. You can start it with the flask run command:

(venv) $ flask run

Open a web browser and type http://localhost:5000 in the address bar. You should now have a two-page application that shows (static) CPU metrics in the top-right corner of the page.

Configuring Turbo-Flask

The goal is to have the CPU load values update every five seconds without the user having to refresh the page. The first step in achieving this goal is to add the Turbo-Flask extension to the application.

Turbo-Flask is initialized like most other Flask extensions. In the imports section of app.py, import the Turbo class:

from turbo_flask import Turbo

Right after the Flask application instance is initialized, create an instance of this class and initialized it by passing the Flask application instance:

turbo = Turbo(app)

In the base.html template, add {{ turbo() }} inside the <head> element:

<!doctype html>
<html>
  <head>
    ...
    {{ turbo() }}
  </head>
</html>

The call to the turbo() function in the base template initializes the turbo.js library. While Turbo Drive and Turbo Frames are not the focus of this article, you should know that this is all that is needed from the Python side to enable these modules in your application.

Adding a Background Updater Thread

And now we reach the most interesting part of this tutorial. We want the application to push updates to the CPU load numbers every 5 seconds, so we will need a background thread that can issue these updates to the connected clients.

A convenient way to start this background thread is to do it as a before_first_request handler, as this ensures that the thread will be up and running as soon as the first client connects. The changes to start a background thread on first request are shown below. Add these changes to app.py:

import threading

# ...

@app.before_first_request
def before_first_request():
    threading.Thread(target=update_load).start()

The update_load() function is where the page updates are going to be generated. The function is going to have a loop, and in each loop iteration an update will be pushed to all connected clients. Below you can see the implementation of this function, which you should also add to app.py:

import time

# ...

def update_load():
    with app.app_context():
        while True:
            time.sleep(5)
            turbo.push(turbo.replace(render_template('loadavg.html'), 'load'))

This function is going to call the render_template() function from Flask to generate the updated CPU metrics fragment, so it needs to have an application context. The loop includes a 5 second sleep that sets the frequency of the updates.

An update is sent to all clients with the turbo.push() method. This method has only one required argument, the stream to push to clients. A stream is composed of one or more page update operations. There are 5 supported operations that can be used with Turbo Streams:

• Append, or turbo.append(content, target): add content at the end of target
• Prepend, or turbo.prepend(content, target): add content at the start of target
• Replace, or turbo.replace(content, target): replace target with content
• Update, or turbo.update(content, target): replace the contents of target with content
• Remove, or turbo.remove(target): remove target

When you want to push a single operation, you can pass the return value of the corresponding operation directly to turbo.push(), as in the example above. If you would like to send several updates in different parts of the page, you can also pass a list of several operations, and turbo.js will apply all the updates for you.

In this application we want to replace the old version of the CPU metrics with an updated version, so the most appropriate operation is turbo.replace(). The append and prepend options are very useful when the page update needs to add information, without removing any of the existing contents. The update operation is similar to replace, but only the contents of the target element are replaced, leaving the element itself alone. Finally, the remove operation is useful to delete a part of the page.

In all these operations the content argument is the HTML fragment that needs to be updated in the page, given as a string. When working with Flask this is easy to generate, you just need to have a template that renders only this section of the page. In this application we already have the CPU metrics in a separate template because we needed to include it in two different pages, so all we need to do to generate the HTML fragment is to render this template directly in a render_template() call.

The target argument accepted for these operations is the id of the target element in the page that receives the update. If you review the loadavg.html template you will notice that the top-level <div> in this template has id="load".

After you make the updates to the application, restart the Flask server and refresh the page on your browser. You should now see the CPU load numbers update at 5 second intervals. It was easy, right?

The turbo.push() method has an optional argument called to, that can be used to specify which client(s) should receive the update. in this application this argument was not included because the update needs to go to all connected clients. If you are interested in sending updates to a subset of the connected clients the to argument, along with the @turbo.user_id decorator will come in handy.

Great, But How Does This Work?

You may be curious to know how does the turbo.push() method do its magic.

One of the things that the {{ turbo() }} call added in the base template does is open a WebSocket connection with the server. This connection is transparent to the application, as the Turbo-Flask extension manages it for you.

A WebSocket connection is bi-directional, so the server can send data to the client without the client having to ask for it. The turbo.push() method takes advantage of this WebSocket connection to submit your page updates. The turbo.js library is listening on this connection and whenever data is received from the server it executes the operations included in the stream.

Deployment Considerations

Turbo-Flask tries to make the set up and use of the WebSocket endpoint completely transparent to your application. At least during development, you should not need to worry about this, as the extension takes care of integrating WebSocket support with the Flask development web server.

If you intend to deploy a Turbo-Flask application to production you will need to decide on a deployment strategy that is compatible with WebSocket. There are 5 different production-ready configurations that are supported:

• Gunicorn with the threaded worker
• Gunicorn with the eventlet worker
• Gunicorn with the gevent worker
• The eventlet WSGI web server with monkey patching
• The gevent WSGI web server with monkey patching

To learn more about these options, consult the deployment documentation for the Flask-Sock package, used by Turbo-Flask to implement the WebSocket route.

Conclusion

I hope this tutorial gave you a good overview of Turbo-Flask and turbo.js, and how to use them to trigger dynamic updates to your Flask web pages.

As mentioned a few times throughout the article, turbo.js is composed of a collection of utilities that speed up navigation, form submissions and updates to web pages rendered in the server, so I encourage you to read the turbo.js documentation to learn about how to take advantage of the remaining parts of this library.