Migrating from Flask-Script to the New Flask CLI

In release 0.11, Flask introduced new command-line functionality based on Click, which includes the flask command. Before then, Flask did not provide any support for building command-line interfaces (CLIs), but Flask-Script provided similar functionality as a third party extension.

It's been more than a year since the Flask CLI has been released, and I still see a lot of projects out there based on Flask-Script. My guess is that there aren't really any important reasons that motivate people to migrate, since Flask-Script worked well, or at least well enough. But the reality is that Flask-Script hasn't had an official release since 2014 and appears to be unmaintained. In this article I want to show you how I migrated the Flasky application from my Flask book from Flask-Script to Click (one of the changes that are coming in the second edition of the book!), so that you can learn what the differences are, and decide if it is time to migrate your applications.

First, A Confession

You may think that with this article I want to convince you that the Flask CLI is the greatest thing since sliced bread, but if you think that, you are actually wrong. Back in 2014, when Armin Ronacher introduced the Click integration in Flask, I was (and still am) opposed to the idea of adding yet another dependency to the Flask core project, and suggested that Click-based CLIs be introduced as an extension that can become a fair competitor to Flask-Script. This is all water under the bridge now, but you can see an exchange I had with Armin on this topic here: https://github.com/smurfix/flask-script/issues/97.

So really, I believe forcing a CLI based on Click was a mistake, and against the "pick the best tool for each task" spirit Flask is known for. Unfortunately, making the Click-based CLIs part of the Flask core crushed any steam the Flask-Script maintainers had left, and at least from looking at their GitHub repository, my impression is that the project is now dying a slow death.

What's Wrong With Flask-Script?

Flask-Script has, in my opinion, one important flaw. But interestingly enough, it is a problem that is directly caused by an old design issue in the Flask reloader, also present if you start your application with app.run(), without using Flask-Script.

If you start your application in debug mode, there will be two Flask processes running. The first process is watching the source files for changes, and the second is the actual Flask server. When any source files change, the watcher process kills the server process, and then starts another one, which will now use the updated source files.

The problem occurs when you inadvertently introduce a syntax error in one of your source files. The watcher process will not know any better, so it will kill the older server, and launch a new one. But the new server is not going to start, since Python will raise an error while parsing the modified file. When the server process exits in error, the watcher process gives up and exits too, forcing you to restart the whole thing once you fix the error in your code.

This works much better when you start the reloader with the new flask run command. The application is not imported until the first request is sent, and at that point, if an error occurs while importing the source files, it is handled exactly as if it was an error that occurred during runtime. This means that if you have the web-based debugger enabled, you will get the error reported there. The new watcher process is also smarter, and does not exit if the server process dies. Instead, it will continue to monitor source files, and try to start the server again after more changes are made.

While this was an annoyance, I got used to it and was always prepared to restart the application when it exited due to an error on my part.

The New Flask CLI

Is the Flask CLI better than Flask-Script? Let's start with a review of the default commands that all Flask applications get and see how they fare.

When you install Flask 0.11 or newer, you get a flask command installed on your virtual environment:

(venv) $ flask
Usage: flask [OPTIONS] COMMAND [ARGS]...

  This shell command acts as general utility script for Flask applications.

  It loads the application configured (through the FLASK_APP environment
  variable) and then provides commands either provided by the application or
  Flask itself.

  The most useful commands are the "run" and "shell" command.

  Example usage:

    $ export FLASK_APP=hello.py
    $ export FLASK_DEBUG=1
    $ flask run

Options:
  --version  Show the flask version
  --help     Show this message and exit.

Commands:
  run    Runs a development server.
  shell  Runs a shell in the app context.

When using Flask-Script, you had to create a driver script, typically called manage.py. Looking at the above, you can see that ./manage.py runserver maps to flask run, and ./manage.py shell maps to flask shell. Not much of a difference, right?

There is actually one gotcha. The manage.py and flask commands differ in the way they discover the Flask application instance. For Flask-Script, the application is provided to the Manager class as an argument, either directly or in the form of an application factory function. The new Flask CLI however, expects the application instance to be provided in the FLASK_APP environment variable, which you typically set to the filename of the module that defines it. Flask will look for an app or application object in that module and use that as the application.

Unfortunately, there is no direct support for application factory functions in the new Flask CLI. The approach that you need to follow to use a factory function is to define a module that calls the factory function to create the app object, and then reference that module in FLASK_APP. This is similar in concept to the wsgi.py module in Django applications.

If you wanted to support the new Flask CLI for Flasky, you could write a flasky.py file as follows:

import os
from app import create_app

app = create_app(os.getenv('FLASK_CONFIG') or 'default')

Note that I copied these lines from manage.py, which we are not going to use once we are fully migrated to the new CLI.

The "flask run" Command

After adding the flasky.py module, you can start the Flask development server with the following command:

$ export FLASK_APP=flasky.py
$ flask run

If you are using Microsoft Windows, the FLASK_APP environment variable is set in a slightly different way:

> set FLASK_APP=flasky.py
> flask run

The flask run command has options to enable or disable the reloader and the debugger, and also to set the IP address and port where the server will be listening for client requests. The options are not identical to those from Flask-Script, but they are close enough. You can use flask run --help to see all the available options.

As I'm sure you are aware, most Flask applications have this at the bottom of the main script:

if __name__ == '__main__':
    app.run()

This is what actually starts the server when you don't use any CLI support. If you leave this in your script and start using the new CLI it is not going to cause any problem, but it isn't really needed, so you can go ahead and remove it.

The "flask shell" Command

The shell command is basically the same, but there is a small difference in how you define additional symbols that you want auto-imported into the shell context. This is a feature that can save you a lot of time when working on an application. Normally you add your model classes, database instance and other objects you are likely to interact with in a testing or debugging session in the shell.

For Flask-Script, the Flasky application had the following shell context definition:

def make_shell_context():
    return dict(app=app, db=db, User=User, Follow=Follow, Role=Role,
                Permission=Permission, Post=Post, Comment=Comment)
manager.add_command("shell", Shell(make_context=make_shell_context))

As you see above, the make_shell_context() function is referenced in the add_command call that defines the shell option. Flask-Script calls this function right before starting a shell session, and includes all the symbols returned in the dictionary in it.

The Flask CLI offers the same functionality, but uses a decorator to identify the function that provides shell context items (there can be multiple functions, actually). To have the equivalent functionality to the above, I've expanded the flasky.py module:

import os
from app import create_app, db
from app.models import User, Follow, Role, Permission, Post, Comment

app = create_app(os.getenv('FLASK_CONFIG') or 'default')

@app.shell_context_processor
def make_shell_context():
    return dict(app=app, db=db, User=User, Follow=Follow, Role=Role,
                Permission=Permission, Post=Post, Comment=Comment)

So as you see, the function is identical, but needs the @app.shell_context_processor decorator so that Flask knows about it.

Commands From Flask Extensions

A big part of the success of Flask-Script was that it allowed other Flask extensions to add their own commands. I have actually taken advantage of this functionality in my Flask-Migrate extension.

So how do you migrate those commands to the Click CLI? Unfortunately you depend on the extension author to do the migration for you. If you use any extensions that work with Flask-Script that haven't been updated to also work with the Flask CLI, you are pretty much out of luck, you will need to continue using Flask-Script, at least when you interact with the extensions in question. If you use Flask-Migrate, you do not have to worry, as I have updated it to support both the Click and Flask-Script CLIs. Just make sure you are using a recent version.

Specifically for Flask-Migrate, ./manage.py db ... directly translates to flask db .... In the Flask-Script version, the Flask-Migrate extension was initialized in manage.py. For the Flask CLI version, we can move that to the new flasky.py module, which I'm sure you realized by now that it is becoming a replacement of Flask-Script's manage.py, with the only difference that it is not a script you execute directly. Here is that module with the Flask-Migrate integration added:

import os
from app import create_app, db
from app.models import User, Follow, Role, Permission, Post, Comment
from flask_migrate import Migrate

app = create_app(os.getenv('FLASK_CONFIG') or 'default')
migrate = Migrate(app, db)

@app.shell_context_processor
def make_shell_context():
    return dict(app=app, db=db, User=User, Follow=Follow, Role=Role,
                Permission=Permission, Post=Post, Comment=Comment)

As an example, you can upgrade the Flasky database to the latest revision with the following command, but remember that you need to have FLASK_APP set for it to work:

$ flask db upgrade

Application Commands

Another nice feature in Flask-Script was that it allowed you to write your custom tasks as functions, which then were automatically converted to commands in the CLI just by adding a decorator. In Flasky, I've used this feature to add a few commands. For example, here is how I implemented the ./manage.py test command for Flask-Script:

@manager.command
def test(coverage=False):
  """Run the unit tests."""
  # ...

The @manager.command decorator was all that was needed to expose the function as a test command, and it even detected the coverage argument and added it as a --coverage option.

Decorator-based commands are actually Click's bread and butter, so this is something that can be migrated without a problem. The equivalent function for the Flask CLI can go in flasky.py, and would be as follows:

import click

# ...

@app.cli.command()
@click.option('--coverage/--no-coverage', default=False, help='Enable code coverage')
def test(coverage):
    """Run the unit tests."""
    # ...

The @app.cli.command() decorator provides an interface to Click. As in Flask-Script, commands are executed with an application context installed, but with the Flask CLI the app context can be disabled if you don't need it. The actual code of the function does not need to change, but note how the --coverage option needs to be given explicitly using the @click.option decorator, unlike the Flask-Script case, in which the option is automatically derived from the function argument list.

The same approach can be taken with the other two custom functions that I have in my Flasky application. The complete implementation of the flasky.py module is below:

import os
from app import create_app, db
from app.models import User, Follow, Role, Permission, Post, Comment
from flask_migrate import Migrate
import click

app = create_app(os.getenv('FLASK_CONFIG') or 'default')
migrate = Migrate(app, db)

@app.shell_context_processor
def make_shell_context():
    return dict(app=app, db=db, User=User, Follow=Follow, Role=Role,
                Permission=Permission, Post=Post, Comment=Comment)

@app.cli.command()
@click.option('--coverage/--no-coverage', default=False, help='aaa')
def test(coverage=False):
    "Test coverage"
    # ...

@app.cli.command()
@click.option('--length', default=25, help='Profile stack length')
@click.option('--profile-dir', default=None, help='Profile directory')
def profile(length, profile_dir):
    """Start the application under the code profiler."""
    # ...

@app.cli.command()
def deploy():
    """Run deployment tasks."""
    # ...

Conclusion

In this article I showed you the functionality offered by the new Flask CLI that will be useful if you want to migrate your Flask-Script based applications. There are a couple more things that you can do that I haven't included here, however. The documentation describes how you can create your own driver scripts, similar to the manage.py of Flask-Script, in case you don't want to use the flask command (this is surprisingly harder than it sounds!). It also tells you how to register commands through "entry points" defined in your project's setup.py file, if you use one for your project. Be sure to check the CLI documenation to learn about those if you are interested.

So what do you think? Will you be migrating your Flask-Script applications soon? Let me know below in the comments!