Dynamically Update Your Flask Web Pages Using Turbo-Flask

Posted by
on under

How can you update a part of a web page dynamically, without the user having to hit the refresh button on the browser?

This is a question that is frequently asked on the Internet. If you have some knowledge of JavaScript this is relatively easy to do, more so if you use a front end web framework such as React or Vue. But what if you have a standard web application written in Flask and Jinja templates?

In this article I'm going to introduce you to my Turbo-Flask extension, which will allow your Flask application to easily push asynchronous page updates to the browser, without having to write any JavaScript code. In the screenshot below, note how the CPU load numbers update without the user doing anything.

Turbo-Flask Demo Application

What is Turbo-Flask?

Turbo-Flask is a Flask extension that integrates Hotwire's turbo.js JavaScript library with your Flask application.

This library bundles a number of different features, all with the goal of making server generated web pages behave more like single-page applications, but without requiring the application to write any front end code in JavaScript. Here is a list of the four modules included in turbo.js:

  • Turbo Drive: to accelerate page navigation by transparently updating pages in the background, without letting the browser ever do a full page reload.
  • Turbo Frames: to only update predefined parts of the page when the user clicks on a link or submits a form.
  • Turbo Streams: to let the server-side application update parts of the page by submitting HTML fragments to the client.
  • Turbo Native: to wrap your application as a native iOS or Android app. This section of turbo.js is out of scope for this article and I have very little experience with it, so I will not discuss it at all.

If you are interested in the Turbo Drive, and Frames features, I suggest you review the turbo.js documentation links above, as these do not require a Flask integration and can be used directly in your Jinja templates.

In this article I want to concentrate on the Turbo Streams feature, which is, in my opinion, the most interesting of the set.

An Example Application

This is going to be a hands-on tutorial, so let's create a short Flask application to which we can later add Turbo-Flask. If you are too lazy to type or copy/paste the code examples below, you can find the complete code for this article in the Turbo-Flask repository on GitHub.

To begin, create a directory in which you will work on this application. You may also want to create a virtual environment and activate it. Once you are ready, install Flask and Turbo-Flask:

(venv) $ pip install flask turbo-flask

Note: The Turbo-Flask package requires Flask version 2.

The Flask application

The code for the base application is shown below. Copy it to a file named app.py:

import random
import re
import sys
from flask import Flask, render_template

app = Flask(__name__)

@app.route('/')
def index():
    return render_template('index.html')

@app.route('/page2')
def page2():
    return render_template('page2.html')

@app.context_processor
def inject_load():
    if sys.platform.startswith('linux'): 
        with open('/proc/loadavg', 'rt') as f:
            load = f.read().split()[0:3]
    else:
        load = [int(random.random() * 100) / 100 for _ in range(3)]
    return {'load1': load[0], 'load5': load[1], 'load15': load[2]}

The application defines two routes on the / and /page2 URLs. Both pages are implemented with Jinja templates named index.html and page2.html.

So far this does not show anything out of the ordinary. The last part of the application is something you may not be familiar with. The inject_load() function is decorated with the @app.context_processor decorator from Flask, which allows the application to add symbols to the Jinja scope. The dictionary that this function returns contains three keys, load1, load5 and load15, which Flask will make available to all Jinja templates rendered with the render_template() function.

The three load numbers are a standard metric that can be obtained in Unix based systems. These numbers give you the CPU load of the system in the last 1, 5 and 15 minutes. The method that I use to obtain these numbers is specific to Linux servers:

        with open('/proc/loadavg', 'rt') as f:
            load = f.read().split()[0:3]

To ensure that you can use this example application under other operating systems, the function provides an alternative implementation that just generates three fake load numbers:

        load = [int(random.random() * 100) / 100 for _ in range(3)]

The Base Template

Since the application will have two pages with largely the same structure, it is a good idea to extract the base layout into a base template that both pages can inherit from. Here is the base.html template, which you will need to store in a templates sub-directory:

<!doctype html>
<html>
  <head>
    <title>Turbo-Flask Streams Demo</title>
    <style>
      .load {
        float: right;
        width: 300px;
        border: 1px solid black;
        margin-left: 10px;
        margin-bottom: 10px;
        padding: 10px;
      }
      .load th, .load td {
        padding: 6px;
        text-align: center;
      }
    </style>
  </head>
  <body>
    {% include "loadavg.html" %}
    {% block content %}{% endblock %}
  </body>
</html>

The base template defines a few styles that create the design of the box with the load average metrics as you see in the screenshot at the start of the article. In particular, note how the float: right CSS attribute will make this part of the page appear on the top right corner of the page, wrapped with content.

The body of the template includes the loadavg.html template, which implements this box, and then defines a content Jinja block that the derived pages can use to provide their page bodies.

The Load Averages Template

The template referenced by the base template implementes the box that shows the three load average values as a table. Save the following page in the templates directory with the name loadavg.html:

<div id="load" class="load">
  <table>
    <tr><th></th><th>1 min</th><th>5 min</th><th>15 min</th></tr>
    <tr><th>Load averages:</th><td>{{ load1 }}</td><td>{{ load5 }}</td><td>{{ load15 }}</td></tr>
  </table>
</div>

Here notice the load1, load5 and load15 variables, which were the ones injected into the Jinja context by the application.

The Page Templates

To complete the base version of the application we need to create the two templates that are rendered by the index() and page2() routes. Since the content of the pages does not really matter much, I created a placeholder page with some dummy text in them, plus a link to the other page. Below you can see the index.html and page2.html templates, which you should copy to the templates sub-directory:

{% extends "base.html" %}

{% block content %}
<h1>Main Page</h1>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>You can visit <a href="/page2">Page 2</a>.</p>
{% endblock %}
{% extends "base.html" %}

{% block content %}
<h1>Page 2</h1>
<p>Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.</p>
<p>Go back to the <a href="/">home page</a>.</p>
{% endblock %}

Both templates inherit from the base.html template created earlier, which means that they are going to have the box with the CPU metrics automatically added in the top-right corner of the page.

Running the Application

The application is now complete. You can start it with the flask run command:

(venv) $ flask run

Open a web browser and type http://localhost:5000 in the address bar. You should now have a two-page application that shows (static) CPU metrics in the top-right corner of the page.

Turbo-Flask Static Demo

Configuring Turbo-Flask

The goal is to have the CPU load values update every five seconds without the user having to refresh the page. The first step in achieving this goal is to add the Turbo-Flask extension to the application.

Turbo-Flask is initialized like most other Flask extensions. In the imports section of app.py, import the Turbo class:

from turbo_flask import Turbo

Right after the Flask application instance is initialized, create an instance of this class and initialized it by passing the Flask application instance:

turbo = Turbo(app)

In the base.html template, add {{ turbo() }} inside the <head> element:

<!doctype html>
<html>
  <head>
    ...
    {{ turbo() }}
  </head>
</html>

The call to the turbo() function in the base template initializes the turbo.js library. While Turbo Drive and Turbo Frames are not the focus of this article, you should know that this is all that is needed from the Python side to enable these modules in your application.

Adding a Background Updater Thread

And now we reach the most interesting part of this tutorial. We want the application to push updates to the CPU load numbers every 5 seconds, so we will need a background thread that can issue these updates to the connected clients.

A convenient way to start this background thread is to do it as a before_first_request handler, as this ensures that the thread will be up and running as soon as the first client connects. The changes to start a background thread on first request are shown below. Add these changes to app.py:

import threading

# ...

@app.before_first_request
def before_first_request():
    threading.Thread(target=update_load).start()

The update_load() function is where the page updates are going to be generated. The function is going to have a loop, and in each loop iteration an update will be pushed to all connected clients. Below you can see the implementation of this function, which you should also add to app.py:

import time

# ...

def update_load():
    with app.app_context():
        while True:
            time.sleep(5)
            turbo.push(turbo.replace(render_template('loadavg.html'), 'load'))

This function is going to call the render_template() function from Flask to generate the updated CPU metrics fragment, so it needs to have an application context. The loop includes a 5 second sleep that sets the frequency of the updates.

An update is sent to all clients with the turbo.push() method. This method has only one required argument, the stream to push to clients. A stream is composed of one or more page update operations. There are 5 supported operations that can be used with Turbo Streams:

  • Append, or turbo.append(content, target): add content at the end of target
  • Prepend, or turbo.prepend(content, target): add content at the start of target
  • Replace, or turbo.replace(content, target): replace target with content
  • Update, or turbo.update(content, target): replace the contents of target with content
  • Remove, or turbo.remove(target): remove target

When you want to push a single operation, you can pass the return value of the corresponding operation directly to turbo.push(), as in the example above. If you would like to send several updates in different parts of the page, you can also pass a list of several operations, and turbo.js will apply all the updates for you.

In this application we want to replace the old version of the CPU metrics with an updated version, so the most appropriate operation is turbo.replace(). The append and prepend options are very useful when the page update needs to add information, without removing any of the existing contents. The update operation is similar to replace, but only the contents of the target element are replaced, leaving the element itself alone. Finally, the remove operation is useful to delete a part of the page.

In all these operations the content argument is the HTML fragment that needs to be updated in the page, given as a string. When working with Flask this is easy to generate, you just need to have a template that renders only this section of the page. In this application we already have the CPU metrics in a separate template because we needed to include it in two different pages, so all we need to do to generate the HTML fragment is to render this template directly in a render_template() call.

The target argument accepted for these operations is the id of the target element in the page that receives the update. If you review the loadavg.html template you will notice that the top-level <div> in this template has id="load".

After you make the updates to the application, restart the Flask server and refresh the page on your browser. You should now see the CPU load numbers update at 5 second intervals. It was easy, right?

The turbo.push() method has an optional argument called to, that can be used to specify which client(s) should receive the update. in this application this argument was not included because the update needs to go to all connected clients. If you are interested in sending updates to a subset of the connected clients the to argument, along with the @turbo.user_id decorator will come in handy.

Great, But How Does This Work?

You may be curious to know how does the turbo.push() method do its magic.

One of the things that the {{ turbo() }} call added in the base template does is open a WebSocket connection with the server. This connection is transparent to the application, as the Turbo-Flask extension manages it for you.

A WebSocket connection is bi-directional, so the server can send data to the client without the client having to ask for it. The turbo.push() method takes advantage of this WebSocket connection to submit your page updates. The turbo.js library is listening on this connection and whenever data is received from the server it executes the operations included in the stream.

Deployment Considerations

Turbo-Flask tries to make the set up and use of the WebSocket endpoint completely transparent to your application. At least during development, you should not need to worry about this, as the extension takes care of integrating WebSocket support with the Flask development web server.

If you intend to deploy a Turbo-Flask application to production you will need to decide on a deployment strategy that is compatible with WebSocket. There are 5 different production-ready configurations that are supported:

  • Gunicorn with the threaded worker
  • Gunicorn with the eventlet worker
  • Gunicorn with the gevent worker
  • The eventlet WSGI web server with monkey patching
  • The gevent WSGI web server with monkey patching

To learn more about these options, consult the deployment documentation for the Flask-Sock package, used by Turbo-Flask to implement the WebSocket route.

Conclusion

I hope this tutorial gave you a good overview of Turbo-Flask and turbo.js, and how to use them to trigger dynamic updates to your Flask web pages.

As mentioned a few times throughout the article, turbo.js is composed of a collection of utilities that speed up navigation, form submissions and updates to web pages rendered in the server, so I encourage you to read the turbo.js documentation to learn about how to take advantage of the remaining parts of this library.

Become a Patron!

Hello, and thank you for visiting my blog! If you enjoyed this article, please consider supporting my work on this blog on Patreon!

81 comments
  • #1 Emmanuel said

    Thanks a lot for dedicating your time to helping the flask community.
    I really love the idea behind turbo flask.

  • #2 Emil Eliasson said

    Hi Miguel, thanks for posting this. Have you tried running this in different browsers? Your load-example runs perfectly fine on Chrome and Edge, but when I load the same page using Firefox (all addons disabled, I also tried private mode), I don't get the loadavg to update every five seconds.
    Firefox console gives this message: "Firefox can’t establish a connection to the server at ws://x.x.x.x:5000/turbo-stream."

    With Firefox, the flask application also gives this 400-message for the /turbo-stream URL in the flask server output: "x.x.x.x - - [27/May/2021 14:38:35] "GET /turbo-stream HTTP/1.1" 400 -"

    The 400-message does not show up if I load the page in Chrome or Edge. Just wanted to know your thoughts on why this might be happening.

    Once again, thanks for sharing this cool project!

  • #3 Miguel Grinberg said

    @Emil: there is currently a bug in Werkzeug that causes this. Once Werkzeug 2.0.2 is released Firefox will work the same as other browsers. You can install the main branch of Werkzeug if you need the fix right away.

  • #4 Rafael Gonçalves said

    Hi Miguel,

    Thanks for sharing your knowlegment!

    I have a problem to run flask_turbo on my app:

    from turbo_flask import Turbo
    

    ModuleNotFoundError: No module named 'turbo_flask'

    But i has installed it:
    Requirement already satisfied: turbo_flask in ./venv_bridge_aws/lib/python3.9/site-packages (0.6.0)

    You know what i'm doing wrong?

  • #5 Miguel Grinberg said

    @Rafael: I don't. Something in your virtual environment must be broken. I would recreate the virtual environment from scratch.

  • #6 Niko Konzack said

    Hi Miguel, thanks for your post. It has already helped me a lot.

    However, I cannot find a way to execute Javascript every time I reload a certain <div>.
    Do you know how I could do that?

    Thank you in advance for your answer!

  • #7 Miguel Grinberg said

    @Niko: Have you asked in a Hotwired Turbo forum? That's really the best place, I think they're planning to allow JS to be included in stream updates.

  • #8 Zubin Madon said

    Can you please explain how this method is better than simply refreshing from HTML/JS meta code like this: https://www.programmersought.com/article/62861232605/

  • #9 Miguel Grinberg said

    @Zubin: this library is not intended to auto-refresh your page. If that's what you want, then use the meta-refresh tag.

  • #10 Malik said

    Hi Miguel! Thank you for taking your time! I'm having trouble to get turbo to work because of the Talisman CSP. I wonder if there's any workaround?

  • #11 Miguel Grinberg said

    @Malik: I don't really know which change added by Talisman affects Turbo-Flask. I suppose the workaround is to disable whatever prevents Turbo-Flask from running? You have to find out what it is specifically, and then decide if you are comfortable disabling it. Without knowing what is it I cannot give you any more specific advice.

  • #12 Ling said

    Hi Miguel! Trying to use flask-restx with Turbo now, when using
    make_response(render_template('index.html'), 200, headers) with header text/html. Js console alert "Failed to load module script: Expected a JavaScript module script but the server responded with a MIME type of "text/html". Strict MIME type checking is enforced for module scripts per HTML spec."
    Any idea?

  • #13 Miguel Grinberg said

    @Ling: Can't tell for sure, but it seems you are invoking an endpoint that returns HTML from a <script> element in your page.

  • #14 Ling said

    @Miguel I have tried to use pure javascript to connect instead of module.

    <script type="text/javascript" src="https://unpkg.com/@hotwired/turbo@7.0.0-beta.5/dist/turbo.es5-umd.js"></script>
    <script type="text/javascript">
    
      if (window["WebSocket"] && window["Turbo"]) {
        Turbo.connectStreamSource(new WebSocket(`ws${location.protocol.substring(4)}//${location.host}/turbo-stream`))
      } else {
        console.warn("Turbo Streams not available");
      }
    </script>
    

    It passed the error I mentioned yesterday

  • #15 Miguel Grinberg said

    @Ling: with turbo-flask you do not need to make a WebSocket connection, this is all handled by the extension when you add {{ turbo() }} to your Jinja template. Aside from that, the error means what I said before, the route appears to be returning HTML instead of WebSocket. I suggest you start from one of the example apps in the Turbo-Flask repository and confirm that everything works, then start making changes according to your needs.

  • #16 Ling said

    @Miguel I guess CDN website for turbo.js is blocked by my company's network. So it returns error html page:)
    It would be great if we can define the javascript source module somewhere.

  • #17 Miguel Grinberg said

    @Ling: See the url argument to the turbo() function: https://turbo-flask.readthedocs.io/en/latest/api.html#turbo_flask.Turbo.turbo.

  • #18 Kerem said

    Hello Miguel thanks for this blog post.
    First thing that popped into my mind was using this for getting user notifications. I wonder if it'd be wise to use Turbo-Flask instead of sending requests for every ten seconds like you implemented in Mega Flask tutorial.

  • #19 Miguel Grinberg said

    @Kerem: Yes, it's a valid approach. Though keep in mind that this sets up a WebSocket connection for each client, so it's not free. Instead of the constant polling you'll be maintaining a permanent connection for each client.

  • #20 Andy said

    Hi Miguel, thanks for this blog, and all your work. I'm trying to create a page with a form, which executes some code based on form data, and the progress then displayed underneath the form. turbo-flask seems to fit this requirement well, but I am a bit stuck. What is the logic when using turbo-flask to update a page multiple times and not just triggered by the first POST. Once I do the first "return turbo.stream..." how do I then repeat that after further code has executed? I've looked through your 'load' and 'todo' examples but unable to work it out. I'm still fairly green with python so I might be missing some simple logic. Thanks.

  • #21 Andy said

    Hi Miguel, I think I have worked it out now. Rather than use turbo.stream, I am using turbo.push. Although I don't fully understand the app.app_context() line, it does now do what I expect. Many thanks again.

  • #22 Miguel Grinberg said

    @Andy: your solution is the correct one. When you want to send updates to a page initiated by the server you have to use turbo.push().

  • #23 Jean-Francois Theoret said

    Thanks for sharing this code! Finally a python solution for async!

    I am running into a problem however that I hope you may be to help me with. I am using blueprints to serve some paths. When I try to push and and render to my webpage in the update thread:

    turbo.push(turbo.replace(render_template('monitor.html'), 'alive'))

    I get:

    RuntimeError: Application was not able to create a URL adapter for request independent URL generation. You might be able to fix this by setting the SERVER_NAME config variable.

    Would you have an idea why? Thanks

  • #24 Miguel Grinberg said

    @Jean-Francois: The problem is that you are using the url_for() function, probably somewhere in your HTML template. This normal usage of this function relies on having a request context, which your background thread does not have. You can follow the advice in the error message and set SERVER_NAME in the configuration so that url_for knows about your server without having to look information up in the request context.

  • #25 Jean-Francois Theoret said

    @Miguel: Thanks a mil! Removing the url_for solved the issue!

    Now I just need to undestand context so that I can fix my flash messages which arent working anymore :)

Leave a Comment