Migrating from Flask-Script to the New Flask CLI
Posted by
on underIn release 0.11, Flask introduced new command-line functionality based on Click, which includes the flask
command. Before then, Flask did not provide any support for building command-line interfaces (CLIs), but Flask-Script provided similar functionality as a third party extension.
It's been more than a year since the Flask CLI has been released, and I still see a lot of projects out there based on Flask-Script. My guess is that there aren't really any important reasons that motivate people to migrate, since Flask-Script worked well, or at least well enough. But the reality is that Flask-Script hasn't had an official release since 2014 and appears to be unmaintained. In this article I want to show you how I migrated the Flasky application from my Flask book from Flask-Script to Click (one of the changes that are coming in the second edition of the book!), so that you can learn what the differences are, and decide if it is time to migrate your applications.
First, A Confession
You may think that with this article I want to convince you that the Flask CLI is the greatest thing since sliced bread, but if you think that, you are actually wrong. Back in 2014, when Armin Ronacher introduced the Click integration in Flask, I was (and still am) opposed to the idea of adding yet another dependency to the Flask core project, and suggested that Click-based CLIs be introduced as an extension that can become a fair competitor to Flask-Script. This is all water under the bridge now, but you can see an exchange I had with Armin on this topic here: https://github.com/smurfix/flask-script/issues/97.
So really, I believe forcing a CLI based on Click was a mistake, and against the "pick the best tool for each task" spirit Flask is known for. Unfortunately, making the Click-based CLIs part of the Flask core crushed any steam the Flask-Script maintainers had left, and at least from looking at their GitHub repository, my impression is that the project is now dying a slow death.
What's Wrong With Flask-Script?
Flask-Script has, in my opinion, one important flaw. But interestingly enough, it is a problem that is directly caused by an old design issue in the Flask reloader, also present if you start your application with app.run()
, without using Flask-Script.
If you start your application in debug mode, there will be two Flask processes running. The first process is watching the source files for changes, and the second is the actual Flask server. When any source files change, the watcher process kills the server process, and then starts another one, which will now use the updated source files.
The problem occurs when you inadvertently introduce a syntax error in one of your source files. The watcher process will not know any better, so it will kill the older server, and launch a new one. But the new server is not going to start, since Python will raise an error while parsing the modified file. When the server process exits in error, the watcher process gives up and exits too, forcing you to restart the whole thing once you fix the error in your code.
This works much better when you start the reloader with the new flask run
command. The application is not imported until the first request is sent, and at that point, if an error occurs while importing the source files, it is handled exactly as if it was an error that occurred during runtime. This means that if you have the web-based debugger enabled, you will get the error reported there. The new watcher process is also smarter, and does not exit if the server process dies. Instead, it will continue to monitor source files, and try to start the server again after more changes are made.
While this was an annoyance, I got used to it and was always prepared to restart the application when it exited due to an error on my part.
The New Flask CLI
Is the Flask CLI better than Flask-Script? Let's start with a review of the default commands that all Flask applications get and see how they fare.
When you install Flask 0.11 or newer, you get a flask
command installed on your virtual environment:
(venv) $ flask
Usage: flask [OPTIONS] COMMAND [ARGS]...
This shell command acts as general utility script for Flask applications.
It loads the application configured (through the FLASK_APP environment
variable) and then provides commands either provided by the application or
Flask itself.
The most useful commands are the "run" and "shell" command.
Example usage:
$ export FLASK_APP=hello.py
$ export FLASK_DEBUG=1
$ flask run
Options:
--version Show the flask version
--help Show this message and exit.
Commands:
run Runs a development server.
shell Runs a shell in the app context.
When using Flask-Script, you had to create a driver script, typically called manage.py
. Looking at the above, you can see that ./manage.py runserver
maps to flask run
, and ./manage.py shell
maps to flask shell
. Not much of a difference, right?
There is actually one gotcha. The manage.py
and flask
commands differ in the way they discover the Flask application instance. For Flask-Script, the application is provided to the Manager
class as an argument, either directly or in the form of an application factory function. The new Flask CLI however, expects the application instance to be provided in the FLASK_APP
environment variable, which you typically set to the filename of the module that defines it. Flask will look for an app
or application
object in that module and use that as the application.
Unfortunately, there is no direct support for application factory functions in the new Flask CLI. The approach that you need to follow to use a factory function is to define a module that calls the factory function to create the app object, and then reference that module in FLASK_APP
. This is similar in concept to the wsgi.py
module in Django applications.
If you wanted to support the new Flask CLI for Flasky, you could write a flasky.py
file as follows:
import os
from app import create_app
app = create_app(os.getenv('FLASK_CONFIG') or 'default')
Note that I copied these lines from manage.py
, which we are not going to use once we are fully migrated to the new CLI.
The "flask run" Command
After adding the flasky.py
module, you can start the Flask development server with the following command:
$ export FLASK_APP=flasky.py
$ flask run
If you are using Microsoft Windows, the FLASK_APP
environment variable is set in a slightly different way:
> set FLASK_APP=flasky.py
> flask run
The flask run
command has options to enable or disable the reloader and the debugger, and also to set the IP address and port where the server will be listening for client requests. The options are not identical to those from Flask-Script, but they are close enough. You can use flask run --help
to see all the available options.
As I'm sure you are aware, most Flask applications have this at the bottom of the main script:
if __name__ == '__main__':
app.run()
This is what actually starts the server when you don't use any CLI support. If you leave this in your script and start using the new CLI it is not going to cause any problem, but it isn't really needed, so you can go ahead and remove it.
The "flask shell" Command
The shell
command is basically the same, but there is a small difference in how you define additional symbols that you want auto-imported into the shell context. This is a feature that can save you a lot of time when working on an application. Normally you add your model classes, database instance and other objects you are likely to interact with in a testing or debugging session in the shell.
For Flask-Script, the Flasky application had the following shell context definition:
def make_shell_context():
return dict(app=app, db=db, User=User, Follow=Follow, Role=Role,
Permission=Permission, Post=Post, Comment=Comment)
manager.add_command("shell", Shell(make_context=make_shell_context))
As you see above, the make_shell_context()
function is referenced in the add_command
call that defines the shell
option. Flask-Script calls this function right before starting a shell session, and includes all the symbols returned in the dictionary in it.
The Flask CLI offers the same functionality, but uses a decorator to identify the function that provides shell context items (there can be multiple functions, actually). To have the equivalent functionality to the above, I've expanded the flasky.py
module:
import os
from app import create_app, db
from app.models import User, Follow, Role, Permission, Post, Comment
app = create_app(os.getenv('FLASK_CONFIG') or 'default')
@app.shell_context_processor
def make_shell_context():
return dict(app=app, db=db, User=User, Follow=Follow, Role=Role,
Permission=Permission, Post=Post, Comment=Comment)
So as you see, the function is identical, but needs the @app.shell_context_processor
decorator so that Flask knows about it.
Commands From Flask Extensions
A big part of the success of Flask-Script was that it allowed other Flask extensions to add their own commands. I have actually taken advantage of this functionality in my Flask-Migrate extension.
So how do you migrate those commands to the Click CLI? Unfortunately you depend on the extension author to do the migration for you. If you use any extensions that work with Flask-Script that haven't been updated to also work with the Flask CLI, you are pretty much out of luck, you will need to continue using Flask-Script, at least when you interact with the extensions in question. If you use Flask-Migrate, you do not have to worry, as I have updated it to support both the Click and Flask-Script CLIs. Just make sure you are using a recent version.
Specifically for Flask-Migrate, ./manage.py db ...
directly translates to flask db ...
. In the Flask-Script version, the Flask-Migrate extension was initialized in manage.py
. For the Flask CLI version, we can move that to the new flasky.py
module, which I'm sure you realized by now that it is becoming a replacement of Flask-Script's manage.py
, with the only difference that it is not a script you execute directly. Here is that module with the Flask-Migrate integration added:
import os
from app import create_app, db
from app.models import User, Follow, Role, Permission, Post, Comment
from flask_migrate import Migrate
app = create_app(os.getenv('FLASK_CONFIG') or 'default')
migrate = Migrate(app, db)
@app.shell_context_processor
def make_shell_context():
return dict(app=app, db=db, User=User, Follow=Follow, Role=Role,
Permission=Permission, Post=Post, Comment=Comment)
As an example, you can upgrade the Flasky database to the latest revision with the following command, but remember that you need to have FLASK_APP
set for it to work:
$ flask db upgrade
Application Commands
Another nice feature in Flask-Script was that it allowed you to write your custom tasks as functions, which then were automatically converted to commands in the CLI just by adding a decorator. In Flasky, I've used this feature to add a few commands. For example, here is how I implemented the ./manage.py test
command for Flask-Script:
@manager.command
def test(coverage=False):
"""Run the unit tests."""
# ...
The @manager.command
decorator was all that was needed to expose the function as a test
command, and it even detected the coverage
argument and added it as a --coverage
option.
Decorator-based commands are actually Click's bread and butter, so this is something that can be migrated without a problem. The equivalent function for the Flask CLI can go in flasky.py
, and would be as follows:
import click
# ...
@app.cli.command()
@click.option('--coverage/--no-coverage', default=False, help='Enable code coverage')
def test(coverage):
"""Run the unit tests."""
# ...
The @app.cli.command()
decorator provides an interface to Click. As in Flask-Script, commands are executed with an application context installed, but with the Flask CLI the app context can be disabled if you don't need it. The actual code of the function does not need to change, but note how the --coverage
option needs to be given explicitly using the @click.option
decorator, unlike the Flask-Script case, in which the option is automatically derived from the function argument list.
The same approach can be taken with the other two custom functions that I have in my Flasky application. The complete implementation of the flasky.py
module is below:
import os
from app import create_app, db
from app.models import User, Follow, Role, Permission, Post, Comment
from flask_migrate import Migrate
import click
app = create_app(os.getenv('FLASK_CONFIG') or 'default')
migrate = Migrate(app, db)
@app.shell_context_processor
def make_shell_context():
return dict(app=app, db=db, User=User, Follow=Follow, Role=Role,
Permission=Permission, Post=Post, Comment=Comment)
@app.cli.command()
@click.option('--coverage/--no-coverage', default=False, help='aaa')
def test(coverage=False):
"Test coverage"
# ...
@app.cli.command()
@click.option('--length', default=25, help='Profile stack length')
@click.option('--profile-dir', default=None, help='Profile directory')
def profile(length, profile_dir):
"""Start the application under the code profiler."""
# ...
@app.cli.command()
def deploy():
"""Run deployment tasks."""
# ...
Conclusion
In this article I showed you the functionality offered by the new Flask CLI that will be useful if you want to migrate your Flask-Script based applications. There are a couple more things that you can do that I haven't included here, however. The documentation describes how you can create your own driver scripts, similar to the manage.py
of Flask-Script, in case you don't want to use the flask
command (this is surprisingly harder than it sounds!). It also tells you how to register commands through "entry points" defined in your project's setup.py
file, if you use one for your project. Be sure to check the CLI documenation to learn about those if you are interested.
So what do you think? Will you be migrating your Flask-Script applications soon? Let me know below in the comments!
-
#1 Dennis said
Flask cli is good, but it seem like flask shell didn't support ipython?
-
#2 Miguel Grinberg said
@Dennis: No, ipython is not supported by the Flask CLI, Flask-Script wins hands down on that category. But note that there are extensions that add this. Take a look at https://github.com/ei-grad/flask-shell-ipython for example.
-
#3 Mo said
Hi! When will the second edition of your book be released?
-
#4 Miguel Grinberg said
@Mo: There is no set schedule yet, but I expect it'll happen around the end of the year.
-
#5 Jason said
Hi, Miguel
I want to do a blog like yours, I encounter a problem about defining a database model.class User(db.Model): pass class Admin(db.Model): pass class Comment(db.Model): pass
The problem is User can comment on posts, they are one-to-many relationship. Admin also can comment on posts. If I define two one-to-many relationships, it becomes in a mess.
Should I combine User and Admin to a single table?
Could you give me some advice? Thank you! -
#6 Miguel Grinberg said
@Jason: users and admins can certainly be in the same table, with a "role" column that defines if the user has admin powers or not. That is going to make everything simpler.
-
#8 Miguel Grinberg said
@Kiko: You are correct, I have fixed the mistake. Thanks!
-
#9 I Che said
Hello Miguel,
Thank you for this update.
But seems like when I useflask run
instead ofmanage.py
config variableDEBUG
ignored.
Is any way to avoid usage environment variableexport FLASK_DEBUG=1
?Thanks.
-
#10 Miguel Grinberg said
@I Che: Yes, that is currently a limitation of the new method of starting the application. I'm thinking about how that can be improved.
-
#11 I Che said
@Miguel,
Possibly I can help you with this? Do you have any ideas?
Thanks. -
#12 Miguel Grinberg said
@I Che: if you have any improvements that you would like to propose, I'd say submit a PR to the Flask project on GitHub. This is not a trivial problem to solve, I don't have any solutions at the moment.
-
#13 Yuri said
Hi!
Good article!
I am new at Flask, and I like that I can extends my app as I want. But I have one question. There are a lot of extensions and is it good to use its all at my project? Because many extensions are not longer maintains. And in future I will be forced to refuse its.
For example, I will use in my app such extensions like Flask-Bootstrap or Flask-Nav or even Flask-Moment. But I can do my app without its. I can add references to libraries in templates by myself. And app will not be be dependent on others extensions.
What do you think about it? -
#14 Miguel Grinberg said
@Yuri: None of the extensions you mention are unmaintained. How did you conclude they are?
-
#15 Yuri said
@Miguel Grinberg: I know that its are maintained. But if they cease to be supported in the future? Then I will be forced to refuse them.
And I say that it is not difficult to realize them themselves. And because of that in the future I will not have problems with these extensions. -
#16 wowebookpro said
how to implement the function create admin of flask_script in Flask CLI
@manager.command
def create_admin():
"""Creates the admin user."""
db.session.add(User(
email="ad@min.com",
password="admin",
admin=True,
confirmed=True,
confirmed_on=datetime.datetime.now())
)
db.session.commit() -
#17 Miguel Grinberg said
@wowebookpro: I don't see any issues porting this command to Flask CLI using the guidelines in this article? Have you tried? What error do you get?
-
#18 colby said
I can't get 'flask run' to work with any configuration from the Config class. It's loading it, but not applying it. For example 'DEBUG = True' is there, but doesn't actually run in debug. If I run app.run() and 'python myfile.py' everything works. The only settings that work with flask run are ones that I set in either the venv/bin/activate file or from the command line (i.e., export SECRET_KEY='blahblah').
-
#19 Miguel Grinberg said
Debug mode in newer flask releases is set with the FLASK_DEBUG=1 environment variable.
-
#20 Arthur said
Is there any way to implement something like Flask-Script subcommands with the New Flask CLI?
For example, it was possible to do this with Flask-Script:
from flask_script import Manager from app import create_app from app.manage import manager as app_manager app = create_app() manager = Manager(app) manager.add_command('app', app_manager) if __name__ == '__main__': manager.run()
app/manage.py
from flask_script import Manager manager = Manager(usage='Management for the application') @manager.command def hello(arg): print('Hello ' + arg)
So you would call the
hello
function using this argument:$python manage.py app hello xyz
and it would function as it you would expect.
I understand that you can just put your functions in the
app.py
file, as you did in the flasky example, but in my experience this makes the app.py file really messy really quickly. -
#21 Miguel Grinberg said
@Arthur: Yes, you have full access to the click package to create your CLI in any way you want. If you want to see an example of a CLI written in a separate module, the app that I built as part of the Flask Mega-Tutorial does that: https://github.com/miguelgrinberg/microblog/blob/master/app/cli.py.
-
#22 Mark E said
Hi Miguel, thanks so much for all of the great tutorials. I'm trying to get Flask to work with gunicorn, but I'm having trouble getting it set up. I made a hello world app that functions just fine, but I can't seem to get my real app (which uses flask-script and a manage.py file, created using your mega-tutorial) up and running.
Do you know of a site/post/book with good references or instructions for this? Or, would you recommend I migrate over to Flask CLI, as shown in this post?
Thanks for the advice
-
#23 Miguel Grinberg said
@Mark: It should work with Flask-Script. You need to pass the location and name of your app, which for Flask-Script is likely "manage:app". Did you try that?
-
#24 Sanjeev said
in django we have cli that can even setup project structure beside just other things, default structure of project comes handy to get going flask cli lacks that
-
#25 Miguel Grinberg said
@Sanjeev: Yes. Flask does not impose a specific file structure like Django does. You can structure the application in any way you like. If you want to have a robust structure, you can use the Flask template for cookiecutter, or use one of my apps as a base.