The Flask Mega-Tutorial Part XIX: Deployment on Docker Containers
Posted by
on underThis is the nineteenth installment of the Flask Mega-Tutorial series, in which I'm going to deploy Microblog to the Docker container platform.
For your reference, below is a list of the articles in this series.
- Chapter 1: Hello, World!
- Chapter 2: Templates
- Chapter 3: Web Forms
- Chapter 4: Database
- Chapter 5: User Logins
- Chapter 6: Profile Page and Avatars
- Chapter 7: Error Handling
- Chapter 8: Followers
- Chapter 9: Pagination
- Chapter 10: Email Support
- Chapter 11: Facelift
- Chapter 12: Dates and Times
- Chapter 13: I18n and L10n
- Chapter 14: Ajax
- Chapter 15: A Better Application Structure
- Chapter 16: Full-Text Search
- Chapter 17: Deployment on Linux
- Chapter 18: Deployment on Heroku
- Chapter 19: Deployment on Docker Containers (this article)
- Chapter 20: Some JavaScript Magic
- Chapter 21: User Notifications
- Chapter 22: Background Jobs
- Chapter 23: Application Programming Interfaces (APIs)
In Chapter 17 you learned about traditional deployments, in which you have to take care of every little aspect of the server configuration. Then in Chapter 18 I took you to the other extreme when I introduced you to Heroku, a service that takes complete control of the configuration and deployment tasks, allowing you to fully concentrate on your application. In this chapter you are going to learn about a third application deployment strategy based on containers, more particularly on the Docker container platform. This third option sits somewhere in between the other two in terms of the amount of deployment work needed on your part.
Containers are built on a lightweight virtualization technology that allows an application, along with its dependencies and configuration to run in complete isolation, but without the need to use a full blown virtualization solution such as virtual machines, which need a lot more resources and can sometimes have a significant performance degradation in comparison to the host. A system configured as a container host can execute many containers, all of them sharing the host's kernel and direct access to the host's hardware. This is in contrast to virtual machines, which have to emulate a complete system, including CPU, disk, other hardware, kernel, etc.
In spite of having to share the kernel, the level of isolation in a container is pretty high. A container has its own file system, and can be based on an operating system that is different than the one used by the container host. For example, you can run containers based on Ubuntu Linux on a Fedora host, or vice versa. While containers are a technology that is native to the Linux operating system, thanks to virtualization it is also possible to run Linux containers on Windows and Mac OS X hosts. This allows you to test your deployments on your development system, and also incorporate containers in your development workflow if you wish to do so.
The GitHub links for this chapter are: Browse, Zip, Diff.
Installing Docker
While Docker isn't the only container platform, it is by far the most popular, so that's going to be my choice.
To work with Docker, you first have to install it on your system. There are installers for Windows, Mac OS X and several Linux distributions available at the Docker website. If you are working on a Microsoft Windows system, it is important to note that Docker requires Hyper-V. The installer will enable this for you if necessary, but keep in mind that enabling Hyper-V prevents other virtualization technologies such as VirtualBox from working.
Once Docker is installed on your system, you can verify that the install was successful by typing the following command on a terminal window or command prompt:
$ docker version
Client:
Cloud integration: 1.0.14
Version: 20.10.6
API version: 1.41
Go version: go1.16.3
Git commit: 370c289
Built: Fri Apr 9 22:46:57 2021
OS/Arch: darwin/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.6
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: 8728dd2
Built: Fri Apr 9 22:44:56 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.4
GitCommit: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
runc:
Version: 1.0.0-rc93
GitCommit: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Building a Container Image
The first step in creating a container for Microblog is to build an image for it. A container image is a template that is used to create a container. It contains a complete representation of the container file system, along with various settings pertaining to networking, start up options, etc.
The most basic way to create a container image for your application is to start a container for the base operating system you want to use (Ubuntu, Fedora, etc.), connect to a bash shell process running in it, and then manually install your application, maybe following the guidelines I presented in Chapter 17 for a traditional deployment. After you install everything, you can take a snapshot of the container and that becomes the image. This type of workflow is supported with the docker
command, but I'm not going to discuss it because it is not convenient to have to manually install the application every time you need to generate a new image.
A better approach is to generate the container image through a script. The command that creates scripted container images is docker build
. This command reads and executes build instructions from a file called Dockerfile, which I will need to create. The Dockerfile is basically an installer script of sorts that executes the installation steps to get the application deployed, plus some container specific settings.
Here is a basic Dockerfile for Microblog:
Dockerfile: Dockerfile for Microblog.
FROM python:slim
RUN useradd microblog
WORKDIR /home/microblog
COPY requirements.txt requirements.txt
RUN python -m venv venv
RUN venv/bin/pip install -r requirements.txt
RUN venv/bin/pip install gunicorn
COPY app app
COPY migrations migrations
COPY microblog.py config.py boot.sh ./
RUN chmod +x boot.sh
ENV FLASK_APP microblog.py
RUN chown -R microblog:microblog ./
USER microblog
EXPOSE 5000
ENTRYPOINT ["./boot.sh"]
Each line in the Dockerfile is a command. The FROM
command specifies the base container image on which the new image will be built. The idea is that you start from an existing image, add or change some things, and you end up with a derived image. Images are referenced by a name and a tag, separated by a colon. The tag is used as a versioning mechanism, allowing a container image to provide more than one variant. The name of my chosen image is python
, which is the official Docker image for Python. The tags for this image allow you to specify the interpreter version and base operating system. The slim
tag selects a container image that has only the minimal packages required to run the Python interpreter. You can see what other tags are available for Python in the Python image repository.
The RUN
command executes an arbitrary command in the context of the container. This would be similar to you typing the command in a shell prompt. The useradd microblog
command creates a new user named microblog
. Most container images have root
as the default user, but it is not a good practice to run an application as root, so I create my own user.
The WORKDIR
command sets a default directory where the application is going to be installed. When I created the microblog
user above, a home directory was created, so now I'm making that directory the default. The new default directory is going to apply to any remaining commands in the Dockerfile, and also later when the container is executed.
The COPY
command transfers files from your machine to the container file system. This command takes two or more arguments, the source and destination files or directories. The source file(s) must be relative to the directory where the Dockerfile is located. The destination can be an absolute path, or a path relative to the directory that was set in a previous WORKDIR
command. In this first COPY
command, I'm copying the requirements.txt file to the microblog
user's home directory in the container file system.
Now that I have the requirements.txt file in the container, I can create a virtual environment, using the RUN
command. First I create it, and then I install all the requirements in it. Because the requirements file contains only generic dependencies, I then explicitly install gunicorn, which I'm going to use as a web server. Alternatively, I could have added gunicorn to my requirements.txt file.
The three COPY
commands that follow install the application in the container, by copying the app package, the migrations directory with the database migrations, and the microblog.py and config.py scripts from the top-level directory. I'm also copying a new file, boot.sh that I will discuss below.
The RUN chmod
command ensures that this new boot.sh file is correctly set as an executable file. If you are in a Unix based file system and your source file is already marked as executable, then the copied file will also have the executable bit set. I added an explicit set because on Windows it is harder to set executable bits. If you are working on Mac OS X or Linux you probably don't need this statement, but it does not hurt to have it anyway.
The ENV
command sets an environment variable inside the container. I need to set FLASK_APP
, which is required to use the flask
command.
The RUN chown
command that follows sets the owner of all the directories and files that were stored in /home/microblog as the new microblog
user. Even though I created this user near the top of the Dockerfile, the default user for all the commands remained root
, so all these files need to be switched to the microblog
user so that this user can work with them when the container is started.
The USER
command in the next line makes this new microblog
user the default for any subsequent instructions, and also for when the container is started.
The EXPOSE
command configures the port that this container will be using for its server. This is necessary so that Docker can configure the network in the container appropriately. I've chosen the standard Flask port 5000, but this can be any port.
Finally, the ENTRYPOINT
command defines the default command that should be executed when the container is started. This is the command that will start the application web server. To keep things well organized, I decided to create a separate script for this, and this is the boot.sh file that I copied to the container earlier. Here are the contents of this script:
boot.sh: Docker container start-up script.
#!/bin/bash
source venv/bin/activate
flask db upgrade
flask translate compile
exec gunicorn -b :5000 --access-logfile - --error-logfile - microblog:app
This is a fairly standard start up script that is fairly similar to how the deployments in Chapter 17 and Chapter 18 were started. I activate the virtual environment, upgrade the database though the migration framework, compile the language translations, and finally run the server with gunicorn.
Note the exec
that precedes the gunicorn command. In a shell script, exec
triggers the process running the script to be replaced with the command given, instead of starting it as a new process. This is important, because Docker associates the life of the container to the first process that runs on it. In cases like this one, where the start up process is not the main process of the container, you need to make sure that the main process takes the place of that first process to ensure that the container is not terminated early by Docker.
An interesting aspect of Docker is that anything that the container writes to stdout
or stderr
will be captured and stored as logs for the container. For that reason, the --access-logfile
and --error-logfile
are both configured with a -
, which sends the log to standard output so that they are stored as logs by Docker.
With the Dockerfile created, I can now build a container image:
$ docker build -t microblog:latest .
The -t
argument that I'm giving to the docker build
command sets the name and tag for the new container image. The .
indicates the base directory where the container is to be built. This is the directory where the Dockerfile is located. The build process is going to evaluate all the commands in the Dockerfile and create the image, which will be stored on your own machine.
You can obtain a list of the images that you have locally with the docker images
command:
$ docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
microblog latest 03978d7e1007 27 seconds ago 283MB
python slim c2f204720fdd 11 days ago 115MB
This listing will include your new image, and also the base image on which it was built. Any time you make changes to the application, you can update the container image by running the build command again.
Starting a Container
With an image already created, you can now run the container version of the application. This is done with the docker run
command, which usually takes a large number of arguments. I'm going to start by showing you a basic example:
$ docker run --name microblog -d -p 8000:5000 --rm microblog:latest
021da2e1e0d390320248abf97dfbbe7b27c70fefed113d5a41bb67a68522e91c
The --name
option provides a name for the new container. The -d
option tells Docker to run the container in the background. Without -d
the container runs as a foreground application, blocking your command prompt. The -p
option maps container ports to host ports. The first port is the port on the host computer, and the one on the right is the port inside the container. The above example exposes port 5000 in the container on port 8000 in the host, so you will access the application on 8000, even though internally the container is using 5000. The --rm
option will delete the container once it is terminated. While this isn't required, containers that finish or are interrupted are usually not needed anymore, so they can be automatically deleted. The last argument is the container image name and tag to use for the container. After you run the above command, you can access the application at http://localhost:8000.
The output of docker run
is the ID assigned to the new container. This is a long hexadecimal string, that you can use whenever you need to refer to the container in subsequent commands. In fact, only the first few characters are necessary, enough to make the ID unique.
If you want to see what containers are running, you can use the docker ps
command:
$ docker ps
CONTAINER ID IMAGE COMMAND PORTS NAMES
021da2e1e0d3 microblog:latest "./boot.sh" 0.0.0.0:8000->5000/tcp microblog
You can see that even the docker ps
command shortens container IDs. If you now want to stop the container, you can use docker stop
:
$ docker stop 021da2e1e0d3
021da2e1e0d3
If you recall, there are a number of options in the application's configuration that are sourced from environment variables. For example, the Flask secret key, database URL and email server options are all imported from environment variables. In the docker run
example above I have not worried about those, so all those configuration options are going to use defaults.
In a more realistic example, you will be setting those environment variables inside the container. You saw in the previous section that the ENV
command in the Dockerfile sets environment variables, and it is a handy option for variables that are going to be static. For variables that depend on the installation, however, it isn't convenient to have them as part of the build process, because you want to have a container image that is fairly portable. If you want to give your application to another person as a container image, you would want that person to be able to use it as is, and not have to rebuild it with different variables.
So build-time environment variables can be useful, but there is also a need to have run-time environment variables that can be set via the docker run
command, and for these variables, the -e
option can be used. The following example sets a secret key and sends email through a gmail account:
$ docker run --name microblog -d -p 8000:5000 --rm -e SECRET_KEY=my-secret-key \
-e MAIL_SERVER=smtp.googlemail.com -e MAIL_PORT=587 -e MAIL_USE_TLS=true \
-e MAIL_USERNAME=<your-gmail-username> -e MAIL_PASSWORD=<your-gmail-password> \
microblog:latest
It is not uncommon for docker run
command lines to be extremely long due to having many environment variable definitions.
Using Third-Party "Containerized" Services
The container version of Microblog is looking good, but I haven't really thought much about storage yet. In fact, since I haven't set a DATABASE_URL
environment variable, the application is using the default SQLite database, which is supported by a file on disk. What do you think is going to happen to that SQLite file when you stop and delete the container? The file is going to disappear!
The file system in a container is ephemeral, meaning that it goes away when the container goes away. You can write data to the file system, and the data is going to be there if the container needs to read it, but if for any reason you need to recycle your container and replace it with a new one, any data that the application saved to disk is going to be lost forever.
A good design strategy for a container application is to make the application containers stateless. If you have a container that has application code and no data, you can throw it away and replace it with a new one without any problems, the container becomes truly disposable, which is great in terms of simplifying the deployment of upgrades.
But of course, this means that the data must be put somewhere outside of the application container. This is where the fantastic Docker ecosystem comes into play. The Docker Container Registry contains a large variety of container images. I have already told you about the Python container image, which I'm using as a base image for my Microblog container. In addition to that, Docker maintains images for many other languages, databases and other services in the Docker registry and if that isn't enough, the registry also allows companies to publish container images for their products, and also regular users like you or me to publish your own images. That means that the effort to install third party services is reduced to finding an appropriate image in the registry, and starting it with a docker run
command with proper arguments.
So what I'm going to do now is create two additional containers, one for a MySQL database, and another one for the Elasticsearch service, and then I'm going to make the command line that starts the Microblog container even longer with options that enable it to access these two new containers.
Adding a MySQL Container
Like many other products and services, MySQL has public container images available on the Docker registry. Like my own Microblog container, MySQL relies on environment variables that need to be passed to docker run
. These configure passwords, database names etc. While there are many MySQL images in the registry, I decided to use one that is officially maintained by the MySQL team. You can find detailed information about the MySQL container image in its registry page: https://hub.docker.com/r/mysql/mysql-server/.
If you remember the laborious process to set up MySQL in Chapter 17, you are going to appreciate Docker when you see how easy it is to deploy MySQL. Here is the docker run
command that starts a MySQL server:
$ docker run --name mysql -d -e MYSQL_RANDOM_ROOT_PASSWORD=yes \
-e MYSQL_DATABASE=microblog -e MYSQL_USER=microblog \
-e MYSQL_PASSWORD=<database-password> \
mysql/mysql-server:latest
That is it! On any machine that you have Docker installed, you can run the above command and you'll get a fully installed MySQL server with a randomly generated root password, a brand new database called microblog
, and a user with the same name that is configured with full permissions to access the database. Note that you will need to enter a proper password as the value for the MYSQL_PASSWORD
environment variable.
Now on the application side, I need to add a MySQL client package, like I did for the traditional deployment on Ubuntu. I'm going to use pymysql
once again, which I can add to the Dockerfile, along with the cryptography
package that it uses for authentication against the MySQL server:
Dockerfile: Add pymysql and cryptography to Dockerfile.
# ...
RUN venv/bin/pip install gunicorn pymysql cryptography
# ...
Any time a change is made to the application or the Dockerfile, the container image needs to be rebuilt:
$ docker build -t microblog:latest .
Any now I can start Microblog again, but this time with a link to the database container so that both can communicate through the network:
$ docker run --name microblog -d -p 8000:5000 --rm -e SECRET_KEY=my-secret-key \
-e MAIL_SERVER=smtp.googlemail.com -e MAIL_PORT=587 -e MAIL_USE_TLS=true \
-e MAIL_USERNAME=<your-gmail-username> -e MAIL_PASSWORD=<your-gmail-password> \
--link mysql:dbserver \
-e DATABASE_URL=mysql+pymysql://microblog:<database-password>@dbserver/microblog \
microblog:latest
The --link
option tells Docker to make another container accessible to this one. The argument contains two names separated by a colon. The first part is the name or ID of the container to link, in this case the one named mysql
that I created above. The second part defines a hostname that can be used in this container to refer to the linked one. Here I'm using dbserver
as generic name that represents the database server.
With the link between the two containers established, I can set the DATABASE_URL
environment variable so that SQLAlchemy is directed to use the MySQL database in the other container. The database URL is going to use dbserver
as the database hostname, microblog
as the database name and user, and the password that you selected when you started MySQL.
One thing I noticed when I was experimenting with the MySQL container is that it takes a few seconds for this container to be fully running and ready to accept database connections. If you start the MySQL container and then start the application container immediately after, when the boot.sh script tries to run flask db upgrade
it may fail due to the database not being ready to accept connections. To make my solution more robust, I decided to add a retry loop in boot.sh:
boot.sh: Retry database connection.
#!/bin/bash
source venv/bin/activate
while true; do
flask db upgrade
if [[ "$?" == "0" ]]; then
break
fi
echo Upgrade command failed, retrying in 5 secs...
sleep 5
done
flask translate compile
exec gunicorn -b :5000 --access-logfile - --error-logfile - microblog:app
This loop checks the exit code of the flask db upgrade
command, and if it is non-zero it assumes that something went wrong, so it waits five seconds and then retries.
Adding a Elasticsearch Container
The Elasticsearch documentation for Docker shows how to run the service as a single-node for development, and as a two-node production-ready deployment. For now I'm going to go with the single-node option and use the "oss" image, which only has the open source engine. The container is started with the following command:
$ docker run --name elasticsearch -d -p 9200:9200 -p 9300:9300 --rm \
-e "discovery.type=single-node" \
docker.elastic.co/elasticsearch/elasticsearch-oss:7.10.2
This docker run
command has many similarities with the ones I've used for Microblog and MySQL, but there are a couple of interesting differences. First, there are two -p
options, which means that this container is going to listen on two ports instead of just one. Both ports 9200 and 9300 are mapped to the same ports in the host machine.
The other difference is in the syntax used to refer to the container image. For the images that I've been building locally, the syntax was <name>:<tag>
. The MySQL container uses a slightly more complete syntax with the format <account>/<name>:<tag>
, which is appropriate to reference container images on the Docker registry. The Elasticsearch image that I'm using follows the pattern <registry>/<account>/<name>:<tag>
, which includes the address of the registry as the first component. This syntax is used for images that are not hosted in the Docker registry. In this case Elasticsearch runs their own container registry service at docker.elastic.co instead of using the main registry maintained by Docker.
So now that I have the Elasticsearch service up and running, I can modify the start command for my Microblog container to create a link to it and set the Elasticsearch service URL:
$ docker run --name microblog -d -p 8000:5000 --rm -e SECRET_KEY=my-secret-key \
-e MAIL_SERVER=smtp.googlemail.com -e MAIL_PORT=587 -e MAIL_USE_TLS=true \
-e MAIL_USERNAME=<your-gmail-username> -e MAIL_PASSWORD=<your-gmail-password> \
--link mysql:dbserver \
-e DATABASE_URL=mysql+pymysql://microblog:<database-password>@dbserver/microblog \
--link elasticsearch:elasticsearch \
-e ELASTICSEARCH_URL=http://elasticsearch:9200 \
microblog:latest
Before you run this command, remember to stop your previous Microblog container if you still have it running. Also be careful in setting the correct passwords for the database and the Elasticsearch service in the proper places in the command.
Now you should be able to visit http://localhost:8000 and use the search feature. If you experience any errors, you can troubleshoot them by looking at the container logs. You'll most likely want to see logs for the Microblog container, where any Python stack traces will appear:
$ docker logs microblog
The Docker Container Registry
So now I have the complete application up and running on Docker, using three containers, two of which come from publicly available third-party images. If you would like to make your own container images available to others, then you have to push them to the Docker registry from where anybody can obtain images.
To have access to the Docker registry you need to go to https://hub.docker.com and create an account for yourself. Make sure you pick a username that you like, because that is going to be used in all the images that you publish.
To be able to access your account from the command line, you need to log in with the docker login
command:
$ docker login
If you've been following my instructions, you now have an image called microblog:latest
stored locally on your computer. To be able to push this image to the Docker registry, it needs to be renamed to include the account, like the image from MySQL. This is done with the docker tag
command:
$ docker tag microblog:latest <your-docker-registry-account>/microblog:latest
If you list your images again with docker images
you are now going to see two entries for Microblog, the original one with the microblog:latest
name, and a new one that also includes your account name. These are really two alias for the same image.
To publish your image to the Docker registry, use the docker push
command:
$ docker push <your-docker-registry-account>/microblog:latest
Now your image is publicly available and you can document how to install it and run from the Docker registry in the same way MySQL and others do.
Deployment of Containerized Applications
One of the best things about having your application running in Docker containers is that once you have the containers tested locally, you can take them to any platform that offers Docker support. For example, you could use the same servers I recommended in Chapter 17 from Digital Ocean, Linode or Amazon Lightsail. Even the cheapest offering from these providers is sufficient to run Docker with a handful of containers.
The Amazon Container Service (ECS) gives you the ability to create a cluster of container hosts on which to run your containers, in a fully integrated AWS environment, with support for scaling and load balancing, plus the option to use a private container registry for your container images.
Finally, a container orchestration platform such as Kubernetes provides an even greater level of automation and convenience, by allowing you to describe your multi-container deployments in simple text files in YAML format, with load balancing, scaling, secure management of secrets and rolling upgrades and rollbacks.
-
#1 Rodrigo said
Great!! Congratulations!!!
-
#2 rodel said
Any idea why am i getting this error?
Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-1ga4o9is/cffi/
this happens during installation of:
Collecting cffi (from -r requirements.txt (line 8))
Downloading cffi-1.11.5.tar.gz (438kB) -
#3 Miguel Grinberg said
@rodel: not sure, it seems something is not installing correctly. Are you doing this project or something else?
-
#4 John Smith said
Hi Miguel,
Geat post. Since Docker is an isolation on itself like a virtualenv, why are we creating a venv inside that ? Is there a specific reason?
Thanks -
#5 rodel said
Thanks for this really great tutorial.
Its looking for compiler or something , but I fixed by adding this on dockerfile:
RUN apk add --no-cache --virtual .pynacl_deps build-base python3-dev libffi-dev
source from:
https://stackoverflow.com/questions/45028650/docker-image-with-python-alpine-failure-due-missing-compiler-errorThis is the complete error when installing the ccfi part:
Complete output from command python setup.py egg_info:No working compiler found, or bogus compiler options passed to the compiler from Python's standard "distutils" module. See the error messages above. Likely, the problem is not related to CFFI but generic to the setup.py of any Python package that tries to compile C code. (Hints: on OS/X 10.8, for errors about -mno-fused-madd see http://stackoverflow.com/questions/22313407/ Otherwise, see https://wiki.python.org/moin/CompLangPython or the IRC channel #python on irc.freenode.net.)
Using archlinux as docker host and also tried in opensuse leap 42.2 both got me above error.
-
#6 Tolis said
Great tutorial, thank you for all the great articles and talks. I have a question concerning the dockerfile. Since we're using a container to run the application, doesn't make the virtual environment within the image unnecessary? It feels like an extra layer of abstraction.
I haven't run any python app using containers so I don't have much experience to be honest. -
#7 Miguel Grinberg said
@John: It's not technically necessary. I like to do it, because a virtualenv makes the names of the python interpreter and pip uniform. The container image may have them as python3 and pip3, for example. But it's a small thing, you should be fine installing everything on the global interpreter as well.
-
#8 Miguel Grinberg said
@Tolis: I replied to this right above. Yes, a virtualenv is optional in a container image. It does not make anything slower though.
-
#9 Graham said
Hi Miguel, I'm a little confused.
As you say, files created by a container disappear when the container is taken down and this is your motivation for using a second, separate container running a MySQL image.
How does this help? When the MySQL container is (eventually) taken down, won't the data held there disappear too?
Or does the MySQL container do something cunning like mount a file from the underlying O/S inside the container?
-
#10 Miguel Grinberg said
@Graham: correct, the MySQL container does not write data to its own file system, it creates an external volume and mounts it on the container file system. So you can kill MySQL, upgrade it, and when it restarts, it will attach to the same volume that holds the data.
-
#11 Joel Tang said
First of all, I want to show my respect to you and thank you for all the efforts again though I already did that before. And then I will make my comment on this section.
If you are a beginner on Docker as me, be cautious using --rm. The first and most frustrating bug I met is I cannot let the container up and run. the reason for that is I am not exactly following the tutorial and made some modification in both Dockerfile and boot.sh, so it messed up. the container will be up for several seconds and removed. So if your container failed to run somehow, consider removing --rm and using docker logs <container_id> to see what is going on. Although the author mentioned in later part, it will save you time when you know this trick earlier. -
#12 med said
hello Miguel, thank you for the amazing tutorial.
To deploy the application properly don't we need to use a web server like nginx or apache? should it be configured on the app container? and how? -
#13 Miguel Grinberg said
@med: Sure, you can add nginx/apache in front of the gunicorn web server if you like. I would not say that this is a must though. In many cases where you deploy your containers to a cloud orchestration platform the web server will be provided for you (Kubernetes, ECS, etc.).
-
#14 Josh said
First, I have to say thanks because this series has been extremely helpful!
Second, if anyone else decides to try an Ubuntu image instead of Alpine, the boot.sh script works if you change #!/bin/sh to #!/bin/bash
-
#15 Tim said
Excellent, really excellent, thank you.
-
#16 Will said
Do you have any tips or help for debugging flask within docker while using the application factory pattern?
The application is starting using
pipenv run flask run
and trying things likepipenv run flask shell
and running causes a failure without error.
There's an issue tracker that I think's related to it, but for now looking for a decent solution: https://bugs.python.org/issue16446 -
#17 Miguel Grinberg said
@Will: So the flask shell command that fails in the container runs fine when docker isn't involved? Maybe you can start a bash shell on a running container and then try to run and/or debug the failing command like you would normally do, but do it inside the container?
-
#18 Sebastian said
I want to follow this tutorial since using docker seems a better way to go for hosting my app in digital ocean.
I read the 17th chapter where you specify is not good to publish gunicorn and that's better to route the traffic from nginx to gunicorn, however in this case you don't do that.
I saw your answer to other comment telling that it is not neccesary since the orchestration platforms already offer webserver, is this the case with digital ocean? Could you point me in the right direction? -
#19 Miguel Grinberg said
@Sebastian: digital ocean does not provide anything. You can host the site directly on Gunicorn if you want, I never said it is not good, just that there are some advantages on having nginx in front, such as being able to serve static files directly, or handling SSL outside of the application. There are many different ways to deploy an application based on containers. You can run nginx in another container, for example. You can see some other ideas in the last section of this article.
-
#20 Stan Prokop said
I never said it is not good, just that there are some advantages on having nginx in front, such as being able to serve static files directly, or handling SSL outside of the application.
Actually, Gunicorn directly facing the internet is really not good. Consider these facts:
Gunicorn is susceptible to denial-of-service attacks: http://docs.gunicorn.org/en/stable/deploy.html
Gunicorn does not support HTTP/2 (one connection for parallelism, binary protocol, etc., etc.): https://github.com/benoitc/gunicorn/issues/1195Moreover I think that any Security specialist would consider usage of Gunicorn directly as a security risk and would strongly recommend to hide it behind an HTTP proxy.
-
#21 Rumolu said
@rodel
The error
Collecting cffi>=1.7 (from cryptography->pymysql) Downloading https://files.pythonhosted.org/packages/e7/a7/cd45...f52b/cffi-1.11.5.tar.gz (438kB) Complete output from command python setup.py egg_info: No working compiler found, or bogus compiler options passed to the compiler from Python's standard "distutils" module. See the error messages above. Likely, the problem is not related to CFFI but generic to the setup.py of any Python package that tries to compile C code. (Hints: on OS/X 10.8, for errors about -mno-fused-madd see http://stackoverflow.com/questions/22313407/ Otherwise, see https://wiki.python.org/moin/CompLangPython or the IRC channel #python on irc.freenode.net.) ---------------------------------------- Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-install-_tu8nom_/cffi/ The command '/bin/sh -c venv/bin/pip install gunicorn pymysql' returned a non-zero code: 1
is somewhat misleading. It results quite simply from calling
docker build -t microblog:latest .
with
RUN venv/bin/pip install gunicorn pymysql
rather than
RUN venv/bin/pip install gunicorn
in the Dockerfile.
-
#22 Miguel Grinberg said
@Rumolu: that's a workaround, but you end up not having the MySQL drivers, which are needed. This appears to be an issue in a recent release of PyMySQL, see https://github.com/PyMySQL/PyMySQL/issues/697. A better workaround is to use pymysql<0.9 to get a slightly older version that does not have this problem.
-
#23 Michael Rodgers said
Is it possible to store the static directory in a separate container so they don't get deleted with the container? I'm allowing users to upload profile pics so I don't want to have this deleted.
-
#24 Rumolu said
@Miguel#22: I see that it's all right, necessary even, to start pymysql before the service is running. An image is static and cannot depend, nor know about, anything else that will go concurrently "live".
But one question remains. If Xcode and gcc are installed and running just fine, how does one avert the "No working compiler found" error and point to one of the installed compilers? Clearly a question for another forum though.
-
#25 Miguel Grinberg said
@Michael: a better solution is to create a volume and mount this volume on your container in the correct path. The volume will survive the container, and can be transferred to the new container when you do an upgrade.