2019-06-04T15:49:57Z

The Ultimate Guide to Python Decorators, Part I: Function Registration

One of the signatures of the Flask framework is its clever use of decorators for common application tasks such as defining routes and error handlers. Decorators give a very concise and readable structure to your code, so much that most Flask extensions and many other Python packages follow the same pattern and expose core parts of their functionality through decorators.

Today I'm starting a series of in-depth posts about different ways in which you can incorporate custom decorators into your Python applications. In this first part I'm going to show you how to create simple decorators that register functions as callbacks for application-specific events.

For your reference, here is a list of all the posts in this series:

Registering Functions With Decorators

The most basic type of decorator is the one used to register a function as a handler for an event. This pattern is very common in Python applications, as it allows two or more subsystems to communicate without knowing much about each other, what is formally known as a "decoupled" design.

A function registration decorator that takes no arguments has the following structure:

def request_logger(f):
    # ...
    # insert custom decorator actions here
    # ...
    return f

In this example, request_logger is the name of the decorator, and is defined as a standard Python function. I said above that this decorator takes no arguments, so you may be confused about seeing that in fact there is an argument that I called f. When working with decorators you have to separate the implementation of the decorator (shown above) versus its usage. To help you get the whole picture, below you can see how this decorator would be used:

@request_logger
def log_a_request(request):
    pass

So as you see, the decorator has no arguments when it is used, but the function that implements the decorator does take an argument. This argument is actually required, because the decorator function is invoked indirectly by Python each time the decorator is used, and Python passes the decorated function (the log_a_request function in the example above) as this argument.

The effect that the decorator has on the decorated function can be better explained with a short snippet of Python code that achieves the same effect without using decorators:

def log_a_request(request):
    pass

log_a_request = request_logger(log_a_request)

Using this snippet as a guide, you can see that the decorator function is invoked with the decorated function as its argument, and the return value is a function that takes the place of the original function that was decorated. This allows advanced decorators to provide an alternative function to replace the decorated function. The simpler decorators that I'm covering in this chapter do not need to do any of that, so they just end with return f to keep the original function untouched.

The common pattern for this type of function registration decorator is to save a reference to the decorated function to invoke it later, when the event represented by the decorator occurs. Below you can see a complete implementation of the request_logger decorator. This is a decorator that registers one or more functions to act as a request loggers. The only thing that the decorator does is to add the decorated function to a list:

all_request_loggers = []

def request_logger(f):
    all_request_loggers.append(f)
    return f

Here is an example of how this decorator can be applied to a function, which can be located in another part of the application:

@request_logger
def log_a_request(request):
    print(request.method, request.path)

This usage of the decorator is going to cause the request_logger() decorator function defined above to execute with the log_a_request function passed as the f argument. The decorator function will then store a reference to this function in the all_request_loggers global list. If there are more functions decorated with this decorator, they are all going to be added to the all_request_loggers list.

The last part of this implementation is to invoke all the functions registered as request loggers when the event (in this case the arrival of a request) occurs. This can be done with a simple for-loop in a separate function:

def invoke_request_loggers(request):
    for logger in all_request_loggers:
        logger(request)

Here is the complete implementation of this example, using a simple Flask application:

from flask import Flask, request

app = Flask(__name__)
all_request_loggers = []

def request_logger(f):
    all_request_loggers.append(f)
    return f

def invoke_request_loggers(request):
    for logger in all_request_loggers:
        logger(request)

@request_logger
def log_a_request(request):
    print(request.method, request.path)

@app.route('/')
def index():
    invoke_request_loggers(request)
    return 'Hello World!'

You may wonder why all this effort if it is much easier to call the log_a_request() function directly from the index() view function. In some cases, particularly in simpler applications, that might be an acceptable solution, but using the decorator allows the application to decouple the view function from the function or functions registered to log requests, which is a good design practice.

In the above example the index() view function does not need to know what is the request logging function or if there is zero, one or many of them. You may also want to register different logging functions on a production deployment versus when you run locally for development. This design keeps your application code clean and independent of the decision of how to log a request. In a real-world application the decorator functions would be in their own Python module, separate from the other modules or packages of the application. Any module that needs to register a request logger would import the request_logger decorator and use it. Likewise, in any part of the application where a request needs to be logged, the invoke_request_loggers() function can be imported and called.

The Observer Pattern

You may have noticed that the ideas that I'm presenting in this article are suspiciously similar to the Observer Pattern. In fact, Python decorators used in the way I showed above are nothing more than a really nice way to implement this pattern!

The observer pattern can be used in a variety of situations. Some examples:

  • In a game, you could register collision handlers to be invoked when two game sprites touch.
  • In a desktop application, you could register background update handlers that are invoked while the application is idle.
  • In a command line application, you could register error handler functions that are invoked to clean up after an unexpected error occurs.
  • In a web application, well, I can just give you some examples from Flask. The route, before_request, after_request and teardown_request decorators along with a few others all use this pattern! These decorators are a bit more complex than the one I showed above, as some take arguments while others alter the behavior of the application based on what the decorated function returns. These are more advanced features that I'll discuss in future articles in this series.

Other Decorator Types

As I hinted above, function registration decorators with no arguments like the one I presented in this article are the simplest type of Python decorators. They are excellent as an introduction to the topic, but they just scratch the surface of the power decorators can bring to your application. In the next part of this series I'm going to discuss decorators that override or replace the decorated function!

9 comments

  • #1 Jason said 2019-06-05T12:29:01Z

    I see that in your code above, for logger in all_request_loggers: logger(request) # this line is where the @request_logger def log_a_request(request): print(request.method, request.path) is getting called... how is logger(request) associated with log_a_request?

  • #2 Miguel Grinberg said 2019-06-05T12:56:57Z

    @Jason: the reference to the log_a_request() function is stored in the all_request_loggers list. This is what the decorator does.

  • #3 Fisher said 2019-06-06T03:58:36Z

    I do test the code snippet, it works as expected. But I have something doubted, what's called in the index request is - invoke_request_loggers(request) It just take elements from the list named all_request_loggers and run them one by one. I don't think the request_logger or log_a_request has been explicitly or implicitly called, then how the log_a_request was put into the list all_request_loggers? And when? At the application was run or the request occured?

  • #4 Fisher said 2019-06-06T04:07:58Z

    with a simple test, I saw that when the application ran, the decorator registered the f into the list all_request_loggers. I don't know why this happens since I thought the decorator should work at least when a decorated function is called - what I saw is weird.

  • #5 Miguel Grinberg said 2019-06-06T08:21:16Z

    @Fisher: The decorator function runs at the time the decorated function is imported, not when it is called. So just by importing a module that has a function decorated with the @request_logger decorator you end up with the decorated function added to the list of loggers.

  • #6 Rushi said 2019-06-06T15:36:37Z

    @Miguel as you replied to @Fisher comment : "just by importing a module that has a function decorated with the @request_logger decorator you end up with the decorated function added to the list of loggers" but it seems that func "log_a_request" is not imported or call anywhere.

  • #7 Miguel Grinberg said 2019-06-06T21:51:58Z

    @Rushi: log_a_request is the decorated function. The decorator takes care of registering the function so that it is indirectly invoked when the invoke_request_loggers function is called. This is the same as in Flask with the @app.route decorator. The view functions are never invoked directly, they are just registered with the decorator. and then called indirectly by Flask when appropriate.

  • #8 Fisher said 2019-06-07T10:33:53Z

    Thanks Miguel, I use simple decorators often, some have celery inside to log requests without influence(response time) to the decorated flask api. But this knowledge is fresh to know, appreciate that.

  • #9 Brandon said 2019-10-29T14:34:20Z

    Thanks Miguel! I had no idea that decorator functions run when they are imported. That was confusing at first because I was expecting log_a_request to be called explicitly, but it makes sense now.

Leave a Comment