The Flask Mega-Tutorial, Part XI: Email Support (2012)

Posted by
on under

(Great news! There is a new version of this tutorial!)

This is the eleventh article in the series in which I document my experience writing web applications in Python using the Flask microframework.

The goal of the tutorial series is to develop a decently featured microblogging application that demonstrating total lack of originality I have decided to call microblog.

NOTE: This article was revised in September 2014 to be in sync with current versions of Python and Flask.

Here is an index of all the articles in the series that have been published to date:

Recap

In the most recent installments of this tutorial we've been looking at improvements that mostly had to do with our database.

Today we are letting our database rest for a bit, and instead we'll look at another important function that most web applications have: the ability to send emails to its users.

In our little microblog application we are going to implement one email related function, we will send an email to a user each time he/she gets a new follower. There are several more ways in which email support can be useful, so we'll make sure we design a generic framework for sending emails that can be reused.

Configuration

Luckily for us, Flask already has an extension that handles email called Flask-Mail, and while it will not take us 100% of the way, it gets us pretty close.

Back when we looked at unit testing, we added configuration for Flask to send us an email should an error occur in the production version of our application. That same information is used for sending application related emails.

Just as a reminder, what we need is two pieces of information:

  • the email server that will be used to send the emails, along with any required authentication
  • the email address(es) of the admins

This is what we did in the previous article (file config.py):

# email server
MAIL_SERVER = 'your.mailserver.com'
MAIL_PORT = 25
MAIL_USERNAME = None
MAIL_PASSWORD = None

# administrator list
ADMINS = ['you@example.com']

It goes without saying that you have enter the details of an actual email server and administrator above before the application can actually send emails. We are not going to enhance the server setup to allow those that require an encrypted communication through TLS or SSL. For example, if you want the application to send emails via your gmail account you would enter the following:

# email server
MAIL_SERVER = 'smtp.googlemail.com'
MAIL_PORT = 465
MAIL_USE_TLS = False
MAIL_USE_SSL = True
MAIL_USERNAME = os.environ.get('MAIL_USERNAME')
MAIL_PASSWORD = os.environ.get('MAIL_PASSWORD')

# administrator list
ADMINS = ['your-gmail-username@gmail.com']

Note that the username and password are read from environment variables. You will need to set MAIL_USERNAME and MAIL_PASSWORD to your Gmail login credentials. Putting sensitive information in environment variables is safer than writing down the information on a source file.

We also need to initialize a Mail object, as this will be the object that will connect to the SMTP server and send the emails for us (file app/__init__.py):

from flask_mail import Mail
mail = Mail(app)

Let's send an email!

To learn how Flask-Mail works we'll just send an email from the command line. So let's fire up Python from our virtual environment and run the following:

>>> from flask_mail import Message
>>> from app import app, mail
>>> from config import ADMINS
>>> msg = Message('test subject', sender=ADMINS[0], recipients=ADMINS)
>>> msg.body = 'text body'
>>> msg.html = '<b>HTML</b> body'
>>> with app.app_context():
...     mail.send(msg)
....

The snippet of code above will send an email to the list of admins that are configured in config.py. The sender will be the first admin in the list. The email will have text and HTML versions, so depending on how your email client is setup you may see one or the other. Note that we needed to create an app_context to send the email. Recent releases of Flask-Mail require this. An application context is created automatically when a request is handled by Flask. Since we are not inside a request we have to create the context by hand just so that Flask-Mail can do its job.

Pretty, neat. Now it's time to integrate this code into our application!

A simple email framework

We will now write a helper function that sends an email. This is just a generic version of the above test. We'll put this function in a new source file that will be dedicated to our email support functions (file app/emails.py):

from flask_mail import Message
from app import mail

def send_email(subject, sender, recipients, text_body, html_body):
    msg = Message(subject, sender=sender, recipients=recipients)
    msg.body = text_body
    msg.html = html_body
    mail.send(msg)

Note that Flask-Mail support goes beyond what we are using. Bcc lists and attachments are available, for example, but we won't use them in this application.

Follower notifications

Now that we have the basic framework to send an email in place, we can write the function that sends out the follower notification (file app/emails.py):

from flask import render_template
from config import ADMINS

def follower_notification(followed, follower):
    send_email("[microblog] %s is now following you!" % follower.nickname,
               ADMINS[0],
               [followed.email],
               render_template("follower_email.txt", 
                               user=followed, follower=follower),
               render_template("follower_email.html", 
                               user=followed, follower=follower))

Do you find any surprises in here? Our old friend the render_template function is making an appearance. If you recall, we used this function to render all the HTML templates from our views. Like the HTML from our views, the bodies of email messages are an ideal candidate for using templates. As much as possible we want to keep logic separate from presentation, so emails will also go into the templates folder along with our views.

So we now need to write the templates for the text and HTML versions of our follower notification email. Here is the text version (file app/templates/follower_email.txt):

Dear {{ user.nickname }},

{{ follower.nickname }} is now a follower. Click on the following link to visit {{ follower.nickname }}'s profile page:

{{ url_for('user', nickname=follower.nickname, _external=True) }}

Regards,

The microblog admin

For the HTML version we can do a little bit better and even show the follower's avatar and profile information (file app/templates/follower_email.html):

<p>Dear {{ user.nickname }},</p>
<p><a href="{{ url_for('user', nickname=follower.nickname, _external=True) }}">{{ follower.nickname }}</a> is now a follower.</p>
<table>
    <tr valign="top">
        <td><img src="{{ follower.avatar(50) }}"></td>
        <td>
            <a href="{{ url_for('user', nickname=follower.nickname, _external=True) }}">{{ follower.nickname }}</a><br />
            {{ follower.about_me }}
        </td>
    </tr>
</table>
<p>Regards,</p>
<p>The <code>microblog</code> admin</p>

Note the _external=True argument to url_for in the above templates. By default, the url_for function generates URLs that are relative to the domain from which the current page comes from. For example, the return value from url_for("index") will be /index, while in this case we want http://localhost:5000/index. In an email there is no domain context, so we have to force fully qualified URLs that include the domain, and the _external argument is just for that.

The final step is to hook up the sending of the email with the actual view function that processes the "follow" (file app/views.py):

from .emails import follower_notification

@app.route('/follow/<nickname>')
@login_required
def follow(nickname):
    user = User.query.filter_by(nickname=nickname).first()
    # ...
    follower_notification(user, g.user)
    return redirect(url_for('user', nickname=nickname))

Now you can create two users (if you haven't yet) and make one follow the other to see how the email notification works.

So that's it? Are we done?

We could now pat ourselves in the back for a job well done and take email notifications out of our list of features yet to implement.

But if you played with the application for some time and paid attention you may have noticed that now that we have email notifications when you click the follow link it takes 2 to 3 seconds for the browser to refresh the page, whereas before it was almost instantaneous.

So what happened?

The problem is that Flask-Mail sends emails synchronously. The web server blocks while the email is being sent and only returns its response back to the browser once the email has been delivered. Can you imagine what would happen if we try to send an email to a server that is slow, or even worse, temporarily offline? Not good.

This is a terrible limitation, sending an email should be a background task that does not interfere with the web server, so let's see how we can fix this.

Asynchronous calls in Python

What we really want is for the send_email function to return immediately, while the work of sending the email is moved to a background process.

Turns out Python already has support for running asynchronous tasks, actually in more than one way. The threading and multiprocessing modules can both do this.

Starting a thread each time we need to send an email is much less resource intensive than starting a brand new process, so let's move the mail.send(msg) call into thread (file app/emails.py):

from threading import Thread
from app import app

def send_async_email(app, msg):
    with app.app_context():
        mail.send(msg)

def send_email(subject, sender, recipients, text_body, html_body):
    msg = Message(subject, sender=sender, recipients=recipients)
    msg.body = text_body
    msg.html = html_body
    thr = Thread(target=send_async_email, args=[app, msg])
    thr.start()

The send_async_email function now runs in a background thread. Because it is a separate thread, the application context required by Flask-Mail will not be automatically set for us, so the app instance is passed to the thread, and the application context is set up manually, like we did above when we sent an email from the Python console.

If you test the 'follow' function of our application now you will notice that the web browser shows the refreshed page before the email is actually sent.

So now we have asynchronous emails implemented, but what if in the future we need to implement other asynchronous functions? The procedure would be identical, but we would need to duplicate the threading code for each particular case, which is not good.

We can improve our solution by implementing a decorator. With a decorator the above code would change to this:

from .decorators import async

@async
def send_async_email(app, msg):
    with app.app_context():
        mail.send(msg)

def send_email(subject, sender, recipients, text_body, html_body):
    msg = Message(subject, sender=sender, recipients=recipients)
    msg.body = text_body
    msg.html = html_body
    send_async_email(app, msg)

Much nicer, right?

The code that allows this magic is actually pretty simple. We will put it in a new source file (file app/decorators.py):

from threading import Thread

def async(f):
    def wrapper(*args, **kwargs):
        thr = Thread(target=f, args=args, kwargs=kwargs)
        thr.start()
    return wrapper

And now that we indirectly have created a useful framework for asynchronous tasks we can say we are done!

Just as an exercise, let's consider how this solution would look using processes instead of threads. We do not want a new process started for each email that we need to send, so instead we could use the Pool class from the multiprocessing module. This class creates a specified number of processes (which are forks of the main process) and all those processes wait to receive jobs to run, given to the pool via the apply_async method. This could be an interesting approach for a busy site, but we will stay with the threads for now.

Final words

The source code for the updated microblog application is available below:

Download microblog-0.11.zip.

I've got a few requests for putting this application up on github or similar, which I think is a pretty good idea. I will be working on that in the near future. Stay tuned.

Thank you again for following me on this tutorial series. I look forward to see you on the next chapter.

Miguel

Become a Patron!

Hello, and thank you for visiting my blog! If you enjoyed this article, please consider supporting my work on this blog on Patreon!

110 comments
  • #1 mordecai said

    Hi, Miguel. Thanks for your tutorial. When I follow this tutorial, I think that in send_email:
    "msg = Message(subject, sender, recipients)"
    shoud be:
    "msg = Message(subject, sender = sender, recipients = recipients)"

  • #2 Miguel Grinberg said

    @mordecai: thanks, you are correct, I actually have it correctly later in the article. I have made the correction above.

  • #3 Kundan Singh said

    Hi.. The code in emails.py is <c>thr = threading.Thread(target = send_async_email, args = [msg])</c> which should be <c>thr = Thread(target = send_async_email, args = [msg])</c> i suppose. You have misplaced threading

  • #4 Miguel Grinberg said

    @Kundan: Yes, you are right. I have made the correction. Thanks!

  • #5 Zach Kagin said

    Hi Miguel,

    First off, thank you so much for this tutorial. I am well on my way to completion of it, and have run into very few speed bumps. It is clear, concise, and incredibly useful for what I have in mind for building next.

    With that in mind, I have recently run into an error with this last step. Whenever I try and send an error - whether it was via python at the beginning or when auto-sending on follow - I get the following error:
    File "<REDACTED>/microblog/flask/lib/python2.7/site-packages/flask_mail.py", line 141, in send
    email_dispatched.send(message, app=current_app._get_current_object())
    File "<REDACTED>/microblog/flask/lib/python2.7/site-packages/werkzeug/local.py", line 295, in _get_current_object
    return self.__local()
    File "<REDACTED>/microblog/flask/lib/python2.7/site-packages/flask/globals.py", line 26, in _find_app
    raise RuntimeError('working outside of application context')
    RuntimeError: working outside of application context

    Any idea what might be the issue? I believe my code matches your identically. The interesting part is that the email still goes through, so it's not as much a blocker as an annoyance. Thanks for any help you can offer!

    Zach

  • #6 Miguel Grinberg said

    @Zach: it seems you are trying to send an email outside of the context of a request. Can you show me the missing frames in your stack trace? I need to see from where you are trying to send an email.

  • #7 Olga said

    hello, Miguel!

    First of all thank you for such great tutorial!
    Can you help me? I've got an error:

    socket.error
    error: [Errno 10061] ����������� �� �����������,

    Traceback (most recent call last)
    File "c:\Python27\myproject\venv\lib\site-packages\flask\app.py", line 1701, in call
    return self.wsgi_app(environ, start_response)
    File "c:\Python27\myproject\venv\lib\site-packages\flask\app.py", line 1689, in wsgi_app
    response = self.make_response(self.handle_exception(e))
    File "c:\Python27\myproject\venv\lib\site-packages\flask\app.py", line 1687, in wsgi_app
    response = self.full_dispatch_request()
    File "c:\Python27\myproject\venv\lib\site-packages\flask\app.py", line 1360, in full_dispatch_request
    rv = self.handle_user_exception(e)
    File "c:\Python27\myproject\venv\lib\site-packages\flask\app.py", line 1358, in full_dispatch_request
    rv = self.dispatch_request()
    File "c:\Python27\myproject\venv\lib\site-packages\flask\app.py", line 1344, in dispatch_request
    return self.view_functionsrule.endpoint
    File "c:\Python27\myproject\venv\lib\site-packages\flask_login.py", line 496, in decorated_view
    return fn(args, *kwargs)
    File "C:\Python27\myproject\petsreviews\views.py", line 201, in follow
    follower_notification(user, g.user)
    File "C:\Python27\myproject\petsreviews\emails.py", line 24, in follower_notification
    user = followed, follower = follower))
    File "C:\Python27\myproject\petsreviews\emails.py", line 14, in send_email
    mail.send(msg)
    File "c:\Python27\myproject\venv\lib\site-packages\flask_mail.py", line 400, in send
    with self.connect() as connection:
    File "c:\Python27\myproject\venv\lib\site-packages\flask_mail.py", line 102, in enter
    self.host = self.configure_host()
    File "c:\Python27\myproject\venv\lib\site-packages\flask_mail.py", line 116, in configure_host
    host = smtplib.SMTP(self.mail.server, self.mail.port)
    File "C:\Python27\Lib\smtplib.py", line 250, in init
    (code, msg) = self.connect(host, port)
    File "C:\Python27\Lib\smtplib.py", line 310, in connect
    self.sock = self._get_socket(host, port, self.timeout)
    File "C:\Python27\Lib\smtplib.py", line 285, in _get_socket
    return socket.create_connection((host, port), timeout)
    File "C:\Python27\Lib\socket.py", line 571, in create_connection
    raise err
    error: [Errno 10061] ����������� �� �����������,

    Do you have any idea what might be the issue?

  • #8 Miguel Grinberg said

    @Olga: the server is trying to send an email, but found that the email server you provided in the configuration is not accepting connections, maybe because there isn't an email server listening on that location.

  • #9 Olga said

    Thank's! it seems I take settings from tutorial (MAIL_SERVER = 'smtp.googlemail.com' ), but it's need 'smtp.gmail.com' .

    I got the same error as previously got Zach. Also email goes through.

    Full traceback:

    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "c:\Python27\myproject\venv\lib\site-packages\flask_mail.py", line 401, in send
    message.send(connection)
    File "c:\Python27\myproject\venv\lib\site-packages\flask_mail.py", line 335, in send
    connection.send(self)
    File "c:\Python27\myproject\venv\lib\site-packages\flask_mail.py", line 141, in send
    email_dispatched.send(message, app=current_app._get_current_object())
    File "c:\Python27\myproject\venv\lib\site-packages\werkzeug\local.py", line 295, in _get_current_o
    bject
    return self.__local()
    File "c:\Python27\myproject\venv\lib\site-packages\flask\globals.py", line 26, in _find_app
    raise RuntimeError('working outside of application context')
    RuntimeError: working outside of application context

    Thank you for your attention and sorry for big amount of questions! :)
    It would be very interesting if you write a tutorial about comments and tags. Also knowledge from the earlier articles were given a huge jolt in the flask's study.

  • #10 Miguel Grinberg said

    @Olga: Looks like Flask-Mail versions 0.8 and up break my example that sends an email from the command line. I have now updated it to work with the current releases.

  • #11 Olga said

    Thank's, it works! But after some changes I've got another issue.
    send.mail() works from shell, but displays error: [Errno 10061] (Traceback above) from app. Have no idea where to look this bug. Why server accepting connections from shell, but not from app?

  • #12 Olga said

    I'm sorry, I fix my problem. Thanks for all!

  • #13 Veronica said

    Hi Miguel, I have the same errors as Olga got, can you give me an advice about it, how to solve ? Thank you so much, all your tutorials rocks !

  • #14 Carlos said

    Miguel,

    I'm having a similar issue to Olga. Getting a runtime error but the email is sending nonetheless when a user is followed.
    Running on windows 7

    Traceback (most recent call last):
    File "C:\Python27\Lib\threading.py", line 808, in __bootstrap_inner self.run()
    File "C:\Python27\Lib\threading.py", line 761, in run self.__target(self.__args, *self.__kwargs)
    File "C:\Users\Owner\Desktop\DERI\microblog\flask\lib\site-packages\flask_mail.py", line 416, in send message.send(connection)
    File "C:\Users\Owner\Desktop\DERI\microblog\flask\lib\site-packages\flask_mail.py", line 351, in send connection.send(self)
    File "C:\Users\Owner\Desktop\DERI\microblog\flask\lib\site-packages\flask_mail.py", line 170, in send email_dispatched.send(message, app=current_app._get_current_object())
    File "C:\Users\Owner\Desktop\DERI\microblog\flask\lib\site-packages\werkzeug\locla.py", line 297, in _get_current_object return self.__local()
    File "C:\Users\Owner\Desktop\DERI\microblog\flask\lib\site-packages\flask\globals.py", line 34, in _find_app raise RuntimeError('working outside of application context')

    thanks.

  • #15 saurshaz said

    CHeck the smtp server settings -

    as per your gmail settings '@gmail.com' might need to be there in configuration instead of '*googlemail.com'.

    The other thing,
    make sure that before the line email.py -
    mail = Mail(app)

    you have read the config by doing

    app.config.from_object('config')

  • #16 Tim said

    Miguel,
    First of all, this has been a really good set of tutorials. However, I did stumble on getting my email to work. I had the mail object created before the app object in my init.py. This would silently not do anything.

    Tim

  • #17 dowlf said

    First thanks so much for such a detailed, and easy to follow tutorial.

    It seems that there is an issue with Flask-Mail 0.9.0 with this example as well. I get the same error as Zach and Olga above with 0.9.0, but it works with 0.8.0. I have tried to figure out how to make it work with 0.9.0 so I could post a solution, but I am stumped. Not surprising though because I just started learning Python a few weeks ago and Flask this week. I have been working with .net before this, I am enjoying working with Python and Flask much more than asp.

    Thanks again!

  • #18 Jeff Peck said

    Miguel,
    I also ran into the 'working outside of application context' error. I'm not sure if this is the best way to fix this, but I got it working by modifying my async method like this:

    from app import app
    def async(f):
    def wrapper(args, kwargs):
    def inner(
    args, kwargs):
    with app.app_context():
    f(*args,
    kwargs)
    t = Thread(target = inner, args = args, kwargs = kwargs)
    t.start()
    return wrapper

  • #19 Miguel Grinberg said

    @Jeff: I see two problems with your solution:
    1. the async decorator is supposed to be general purpose, not tied to a specific need/limitation of Flask-Mail.
    2. During normal use there is an application context, it is only when testing from the shell that the app context is missing. The "with app.app_context()" should be issued in the shell, so that the rest of the code does not need to change.

  • #20 Aarthi said

    Hi Miguel, Thank you for these tutorials, they are extremely helpful. I am totally new to python, flask and I am trying to create an app that would need to send an email or a reminder on a future date. Is that possible through flask-mail or any other way? I currently have it sending an email as soon as an action is don, but I need it to send out the reminder at a date calculated based on some parameters.

  • #21 Miguel Grinberg said

    @Aarthi: you need to have a background job sending the emails at the proper time, your Flask server will only run when someone sends a request, it is not running all the time.

  • #22 Dominic Monroe said

    Also seeing the RuntimeError on Flask-mail on 0.9.0 when being called from a view. Not sure what to do? Obviously not officially supported by you. But struggling to find a solution of my own accord.

  • #23 Dwayne Boothe said

    So I am back on the wheel again... Miguel another great tut. like how gave some insight on the alternative approach.

    Doing some research and I found this "http://stackoverflow.com/a/18407455/2047815" where the person used the @copy_current_request_context decorator for the sending function but decorator is only available in Flask 0.10 -_-.

    I am getting the same RuntimeError: working outside of application context but the email is still sent so guest its ok.

  • #24 Miguel Grinberg said

    Hmm. I think I finally understand what's going on. Flask-Mail changed since I wrote this article. It did not use to need the app context, but it does now. You can workaround it by installing Flask-Mail 0.8.2, or by removing the async stuff. I'll think about how to address this.

  • #25 Jordan said

    Hey Miguel, do you use Flask-Mail or a third party solution like SendGrid to send your transactional emails on your web apps?

    I'm a bit worried about using Flask-Mail on my own web app... seems like trying to get things delivered is quite the task - http://www.codinghorror.com/blog/2010/04/so-youd-like-to-send-some-email-through-code.html - but as this is the first time I've been building a web app, I'd like to here your thoughts.

Leave a Comment