Writing a Javascript REST client

Posted by
on under

Last month I published an article on writing RESTful web services in Python, in which I developed a small web service.

Today I'm putting my "front-end" hat to show you how to write a Javascript client application that uses the Python service.

Source code for this tutorial

The source code for this tutorial is available on my REST-tutorial project on github. The incremental versions are tagged and linked in the appropriate sections of the article to save you from having to type or copy/paste the code yourself.

Download the initial version: zip | github

This version includes just the REST server. To execute it under Python's development web server you have to run the following command:

  • For Linux, OS X and Cygwin: ./rest-server.py
  • For Windows: python rest-server.py

The development server will be listening for requests on port 5000 of the local host, so you'll access it as http://localhost:5000.

The REST web service

In the aforementioned article I developed a simple web service that maintains a to-do list. The REST API endpoints for this service are:

HTTP MethodURIAction
GEThttp://[hostname]/todo/api/v1.0/tasksRetrieve list of tasks
GEThttp://[hostname]/todo/api/v1.0/tasks/[task_id]Retrieve a task
POSThttp://[hostname]/todo/api/v1.0/tasksCreate a new task
PUThttp://[hostname]/todo/api/v1.0/tasks/[task_id]Update an existing task
DELETEhttp://[hostname]/todo/api/v1.0/tasks/[task_id]Delete a task

All the endpoints in this service are secured by HTTP basic authentication.

The only resource exposed by this service is a "task", which is composed of the following data fields:

  • uri: unique URI for the task. String type.
  • title: short task description. String type.
  • description: long task description. Text type.
  • done: task completion state. Boolean type.

The REST server article shows how to place calls into this server using a command line utility called curl. Please see that article for details on this if you are interested in learning about the server side.

We will not discuss any more server related matters today. Are you ready to cross to the client-side? Let's go!

Choosing the stack

We will develop a client application that will run on web browsers, so we need to decide what tools and/or frameworks we will use.

For the base stack we don't really have much of a choice: The layout will be done in HTML, styling will be done in CSS and scripting in Javascript. No surprises here. While there are other choices, these guarantee that most modern browsers will run our application.

But these technologies alone would make for a rough development experience. For example, while Javascript and CSS work in all browsers, implementations differ, many times in subtle or obscure ways. There are three areas in which we would benefit from higher level cross-browser frameworks:

  • Presentation and styling
  • REST request management
  • Templates and event handling

Let's review each and evaluate what options there are.

Presentation and styling

We don't really have the patience nor the interest to test several browsers to make sure our HTML, CSS and Javascript works. There are a few frameworks that provide already tested CSS styles and Javascript functions to build websites with a modern look and user interface.

We will use Twitter Bootstrap, the most popular CSS/Javascript layout framework.

REST request management

Our client application running inside the web browser will need to issue requests to the REST web server. The browser's Javascript interpreter provides an API for this called XMLHttpRequest, but the actual implementation varies from browser to browser, so we would need to write browser specific code if we wanted to code against this API directly.

Lucky for us there are several cross-browser Javascript frameworks that hide these little differences and provide a uniform API. Once again we will pick the leading framework in this category, jQuery, which not only provides a uniform Ajax API but a large number of cross-browser helper functions.

Templates and event handling

Finally, our client application will need to generate dynamic content that will be inserted into an existing HTML document, and also update this content as a response to the actions issued by the user. We could use jQuery for all this, but jQuery's support in this area is pretty basic.

Many application frameworks adopt a pattern called Model View Controller to write maintainable applications that separate data from behavior. In MVC frameworks models store the data, views render the data and controllers update views and models according to user actions. The separation between data, presentation and behavior is very clear. There are some variations of MVC, like MVP (Model View Presenter) or MVVM (Model View ViewModel) that are popular as well. All these patterns are usually referred together as MV* frameworks.

MV* frameworks in the client side is a very hot topic these days, there are several projects that compete in this area without a clear winner. Here are the ones that I have evaluated:

AngularJS and Ember.js are the ones with the most features, in particular both have two-way data binding, which allows you to associate Javascript variables to elements in the HTML page so that when one changes the other updates automatically. They both look pretty good, but the learning curve is steep, and their documentation is pretty bad.

Backbone is the oldest framework of the four. It has no automatic data binding, instead you have to set up event handlers and write the code to perform the updates inside them.

Knockout was a surprise. This is a framework that is smaller than the other three, its main feature is two-way data binding. The documentation is excellent, very detailed and complete. There is a fun interactive tutorial followed by reference documentation and several examples, all integrated into an easy to navigate site.

Based on the above analysis, I've decided to give Knockout a chance.

Basic page layout

Let's begin by creating the main page skeleton:

<!DOCTYPE html>
<html>
<head>
<title>ToDo API Client Demo</title>
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<link href="http://netdna.bootstrapcdn.com/twitter-bootstrap/2.3.2/css/bootstrap-combined.min.css" rel="stylesheet">
<script src="http://ajax.aspnetcdn.com/ajax/jquery/jquery-1.9.0.js"></script>
<script src="http://netdna.bootstrapcdn.com/twitter-bootstrap/2.3.2/js/bootstrap.min.js"></script>
<script src="http://ajax.aspnetcdn.com/ajax/knockout/knockout-2.2.1.js"></script>
</head>
<body>
    <div class="navbar">
        <div class="navbar-inner">
            <a class="brand" href="#">ToDo API Client Demo</a>
        </div>
    </div>
    <div id="main" class="container">

        Main content here!

    </div>
    <script type="text/javascript">

        // application code here!

    </script>
</body>
</html>

This is a pretty standard HTML5 document.

The meta name="viewport" comes from Bootstrap. It makes the page scale according to the browser dimensions, be it a browser on a desktop PC or one in a smartphone. What follows is the Bootstrap CSS file, which we import from a CDN (content delivery network) so that we don't have to host it ourselves. Next come the Javascript files for jQuery, Bootstrap and Knockout, also imported from CDNs.

In the body of the page we first have a top bar with the application title, created using Bootstrap's CSS styles. Then we have the main content area, where we will insert the application data. And finally, we will have our application code at the bottom, keeping in mind that for larger projects the application code should likely go into one or more independent Javascript source files.

Download the project at this stage: zip | github

To see how the page looks start the Python server and then navigate to http://localhost:5000/index.html in your browser.

Content area layout

Using Bootstrap styles we can create a mock up of the content area of our application:

TaskOptions
Done

Task title

Task description

In Progress

Task title

Task description

The HTML code that achieves the above look is:

<table class="table table-striped">
    <tr><td style="width: 1px;"></td><td><b>Task</b></td><td><b>Options</b></td></tr>
    <tr>
        <td>
            <span class="label label-success">Done</span>
        </td>
        <td><p><b>Task title</b></p><p>Task description</p></td>
        <td>
            <button class="btn">Edit</button>
            <button class="btn">Delete</button>
            <span><button class="btn">Mark In Progress</button></span>
        </td>
    </tr>
    <tr>
        <td>
            <span class="label label-important">In Progress</span>
        </td>
        <td><p><b>Task title</b></p><p>Task description</p></td>
        <td>
            <button class="btn">Edit</button>
            <button class="btn">Delete</button>
            <span><button class="btn">Mark Done</button></span>
        </td>
    </tr>
</table>
<button class="btn">Add Task</button>

Download the project at this stage: zip | github

Once again, at this point you can start the Python server and then navigate to http://localhost:5000/index.html in your browser to see the page.

Two-way data binding from the HTML side

The above HTML code looks great, but it is static, so it is of no use other than to prototype the look of our application. We now need to convert it into a Knockout view that can show actual data and respond to user clicks. For this we will use some constructs provided by Knockout:

<table class="table table-striped">
    <tr><td style="width: 1px;"></td><td><b>Task</b></td><td><b>Options</b></td></tr>
    <!-- ko foreach: tasks -->
    <tr>
        <td>
            <span data-bind="visible: done" class="label label-success">Done</span>
            <span data-bind="visible: !done()" class="label label-important">In Progress</span>
        </td>
        <td><p><b data-bind="text: title"></b></p><p data-bind="text: description"></p></td>
        <td>
            <button data-bind="click: $parent.beginEdit" class="btn">Edit</button>
            <button data-bind="click: $parent.remove" class="btn">Delete</button>
            <span data-bind="visible: done">
                <button data-bind="click: $parent.markInProgress" class="btn">Mark In Progress</button>
            </span>
            <span data-bind="visible: !done()">
                <button data-bind="click: $parent.markDone" class="btn">Mark Done</button>
            </span>
        </td>
    </tr>
    <!-- /ko -->
</table>
<button data-bind="click: beginAdd" class="btn">Add Task</button>

The ko foreach: tasks directive is inside an HTML comment to avoid making the HTML document illegal. Even as a comment, Knockout will recognize it and interpret it, causing all the HTML elements that appear after it and until the closing /ko comment to repeat once per item in the tasks collection (that we haven't defined yet).

Note that the HTML element that will display the task title got a data-bind="text: title" attribute added. This directive tells Knockout to bind the title variable to the text of the element. The title variable is obtained from the current context. Since we are inside a foreach loop the context is the current task in the tasks collection.

Knockout provides binding constructs that can connect to attributes of an element instead of its text. For example, the In progress and Done labels get their visible attribute bound to the done field of the task, so that only the label that reflects the current state of each task is visible while the other one is hidden. The same binding method is used for the buttons that change the task's done state.

To respond to user input Knockout provides the click binding, which takes a method name. For example, at the very bottom we have the Add Task button, which has a click binding to method beginAdd. Like variables, methods are searched in the current context. The button is outside the foreach loop, so it is in the global context, which as we will see soon is the ViewModel instance that controls the element.

The buttons that are inside the foreach are more tricky, because the context at the level is each task in the tasks collection. We could add methods to the task objects, but that is a complication since task objects will be returned by the REST server. Instead, we use the $parent prefix to indicate that the parent context should be used.

With the above changes our page is depending on a few things that don't exist yet:

  • a tasks collection that has all the tasks to display, each with fields called title, description, and done.
  • a method called beginEdit that opens a dialog box to edit the selected task.
  • a method called remove that deletes the selected task.
  • a method called markDone that changes that done state of the selected task to true.
  • a method called markInProgress that changes that done state of the selected task to false.
  • a method called beginAdd that opens a dialog box to enter a new task.

Two-way data binding from the Javascript side

Knockout uses the MVVM pattern. So far we have worked on the V part, which is the View. The M (Model) will come as JSON data delivered by the REST server. The last component of the triangle is the VM, or ViewModel. This is a Javascript object that will hold the bindings between the models and the views, and will also include the event handlers for user generated actions.

Let's write a mock ViewModel that can interact with our view. This code will be inserted at the bottom of the page, in the <script> area:

    function TasksViewModel() {
        var self = this;
        self.tasks = ko.observableArray();

        self.tasks([
            {
                title: ko.observable('title #1'),
                description: ko.observable('description #1'),
                done: ko.observable(false)
            },
            {
                title: ko.observable('title #2'),
                description: ko.observable('description #2'),
                done: ko.observable(true)
            }
        ]);

        self.beginAdd = function() {
            alert("Add");
        }
        self.beginEdit = function(task) {
            alert("Edit: " + task.title());
        }
        self.remove = function(task) {
            alert("Remove: " + task.title());
        }
        self.markInProgress = function(task) {
            task.done(false);
        }
        self.markDone = function(task) {
            task.done(true);
        }
    }
    ko.applyBindings(new TasksViewModel(), $('#main')[0]);

The var self = this; idiom the begins this function is common in Javascript functions that have callback functions, as it saves the original value of this so that the callbacks, which may have a different this, can use it.

The ko.observableArray() and ko.observable() are the magic that makes Knockout incredibly easy to use. Observables are special Javascript objects that connect a value from a Model to one or more data-bind attributes that refer to it in the View. Once the connection is established any changes to the value will automatically update the view, and any changes to the view will automatically update the value.

Knockout provides the ko.observable() object for single values and the ko.observableArray() for a collection of values. The array version is particularly interesting, because adding or removing array elements will also automatically update what appears in a ko.foreach section.

Something to keep in mind when working with observables is that they look like regular variables, but they are objects. One can be tempted to write this type of code to update an observable:

var myValue = ko.observable(5);
alert(myValue); // this does not work
myValue = 6; // this does not work

At first sight this appears to create an observable with the value 5, display the value in an alert window and then update it to 6. In reality, the alert will print an obscure string that represents the observable object itself, and the assignment will just replace the observable object with a regular value of 6. The correct way to access and modify and observable is to invoke it as a function:

var myValue = ko.observable(5);
alert(myValue()); // display value
myValue(6); // update value to 6

This is even more error prone on observable arrays, where the function call parenthesis sometimes have to be inserted before the square brackets:

var myArray = ko.observableArray([1, 2, 3]);
alert(myArray()[1]); // show second element

For our TasksViewModel object we have defined an observable array called tasks that will get connected to the ko.foreach: tasks that we have in our view.

We initialize our tasks collection with a mock Model that includes two task objects, each having observables for its properties, so that there are also automatic updates for the data-bind attributes that point to the fields of each task.

We then define the event handlers that will respond to click events. For now the beginAdd, beginEdit and remove just show an alert dialog. Note that Knockout sends the task object for the events that were generated from inside the ko.foreach.

The last two events are interesting, because instead of just alerting we are actually modifying our model. Since the task.done field is an observable the simple act of changing the value will trigger updates to the appropriate parts of the web page.

The last statement in the example calls ko.applyBindings to activate the ViewModel. The first argument is the created ViewModel object and the second argument is the root element in the HTML document to associate with it.

Download the project at this stage: zip | github

This is, again, a good time to check how the application is progressing. Go ahead and start the Python server and then open http://localhost:5000/index.html in your browser to see Knockout in action. You will see that the two example tasks in our Model were automagically rendered to the page, and that all the buttons are active. In particular try the "Mark Done" and "Mark In Progress" buttons, to see how simple it is to update the page when data changes.

Connecting to the REST server

Now that we have the client rendering a mock model we are ready to switch to the real thing.

To make an HTTP request to our server we will use jQuery's $.ajax() function. Here is a second version of our TasksViewModel, modified to send a request to the server for the list of tasks:

    function TasksViewModel() {
        var self = this;
        self.tasksURI = 'http://localhost:5000/todo/api/v1.0/tasks';
        self.username = "miguel";
        self.password = "python";
        self.tasks = ko.observableArray();

        self.ajax = function(uri, method, data) {
            var request = {
                url: uri,
                type: method,
                contentType: "application/json",
                accepts: "application/json",
                cache: false,
                dataType: 'json',
                data: JSON.stringify(data),
                beforeSend: function (xhr) {
                    xhr.setRequestHeader("Authorization", 
                        "Basic " + btoa(self.username + ":" + self.password));
                },
                error: function(jqXHR) {
                    console.log("ajax error " + jqXHR.status);
                }
            };
            return $.ajax(request);
        }

        self.beginAdd = function() {
            alert("Add");
        }
        self.beginEdit = function(task) {
            alert("Edit: " + task.title());
        }
        self.remove = function(task) {
            alert("Remove: " + task.title());
        }
        self.markInProgress = function(task) {
            task.done(false);
        }
        self.markDone = function(task) {
            task.done(true);
        }

        self.ajax(self.tasksURI, 'GET').done(function(data) {
            for (var i = 0; i < data.tasks.length; i++) {
                self.tasks.push({
                    uri: ko.observable(data.tasks[i].uri),
                    title: ko.observable(data.tasks[i].title),
                    description: ko.observable(data.tasks[i].description),
                    done: ko.observable(data.tasks[i].done)
                });
            }
        });
    }
    ko.applyBindings(new TasksViewModel(), $('#main')[0]);

Let's go over the changes from top to bottom.

At the top of the TasksViewModel class we define a few new member variables. We will use tasksURI as our root URI to access the REST server. If you recall, this is the URI that when queried with the GET HTTP method returns the list of tasks. We also have two variables that record the login information, which for now we are setting to values that are known to work (we will improve on this later).

The tasks observable array is still there, but we do not initialize it with mock data anymore.

Then we have a new method, simply called ajax. This is a helper function that wraps jQuery's $.ajax() call and makes it more convenient. The function takes three arguments, the URI to connect to, the HTTP method to use and optionally the data to send in the body of the request.

Our ajax function doesn't really do much, it just creates a request object and sends it over to $.ajax(). In addition to the data given in the function arguments, the request object include the mime type for the request and the response, it disables caching and sets a couple of mysterious callback functions.

The first callback function is called beforeSend, and jQuery will invoke it after it has created the jqXHR object that will carry over the request. We need to use this callback to insert the HTTP Basic authentication credentials, because without that the server will not accept our request. For now we just encode the hardcoded username and password. We use the btoa function to encode the credentials in base64 format, as required by the HTTP protocol.

The second callback function will be invoked by jQuery if the request comes back with an error code. This can happen if the username and/or password are incorrect, for example. In this case we just log the error.

Our ajax wrapper function returns the return value from $.ajax, which is a promise object. A promise acts as a proxy for a result of an asynchronous function. When $.ajax() is invoked a request is sent to the server, but the function returns immediately without waiting for the response. The promise object represents that unknown response.

The last change is at the bottom of the class definition, where the actual request to the REST server is made. Here is that snippet of code copied again:

        self.ajax(self.tasksURI, 'GET').done(function(data) {
            for (var i = 0; i < data.tasks.length; i++) {
                self.tasks.push({
                    uri: ko.observable(data.tasks[i].uri),
                    title: ko.observable(data.tasks[i].title),
                    description: ko.observable(data.tasks[i].description),
                    done: ko.observable(data.tasks[i].done)
                });
            }
        });

We call the ajax() wrapper function with the arguments to send a GET request to the server on the main URI. Then on the return promise we call done(). All jQuery promises provide a done method that takes a callback. The callback will be invoked once the asynchronous function associated with the promise ends, with the result of the function passed as an argument.

Our done callback will receive the response data, which is a JSON object that contains the array of tasks returned by the server. We then take this array and for each element create a similar object to add to our tasks observable array. Note that we can't add the task returned by the server directly because we need some of the fields in the tasks to be observables, so that they update the page automatically when they change.

Download the project at this stage: zip | github

If you try the application with these changes you will see the task list from the server rendered in the page.

Adding a new task

Adding a new task is fun because it requires us to create a dialog box where the user can enter the details.

The Boostrap documentation provides all the information required to create modal dialog boxes. Below is the code that creates a dialog box with a form inside to enter task information:

<div id="add" class="modal hide fade" tabindex="=1" role="dialog" aria-labelledby="addDialogLabel" aria-hidden="true">
    <div class="modal-header">
        <button type="button" class="close" data-dismiss="modal" aria-hidden="true">×</button>
        <h3 id="addDialogLabel">Add Task</h3>
    </div>
    <div class="modal-body">
        <form class="form-horizontal">
            <div class="control-group">
                <label class="control-label" for="inputTask">Task</label>
                <div class="controls">
                    <input data-bind="value: title" type="text" id="inputTask" placeholder="Task title" style="width: 150px;">
                </div>
            </div>
            <div class="control-group">
                <label class="control-label" for="inputDescription">Description</label>
                <div class="controls">
                    <input data-bind="value: description" type="text" id="inputDescription" placeholder="Description" style="width: 300px;">
                </div>
            </div>
        </form>
    </div>
    <div class="modal-footer">
        <button data-bind="click: addTask" class="btn btn-primary">Add Task</button>
        <button class="btn" data-dismiss="modal" aria-hidden="true">Cancel</button>
    </div>
</div>

This dialog box will be hidden initially. The code can be inserted anywhere in the page, for example a good place is right after the main div.

If you look carefully you will see that the <input> elements have data-bind attributes in them that connect to their values. This is so that we can bind variables to them. Likewise, the Add Task button at the bottom is bound to its click event, so that we can act when the user accepts the dialog box.

According to the Bootstrap documentation to display the dialog box we have to do this:

$('add').modal('show');

Since we want to display this dialog box when the user clicks the Add Task button we can now replace our alert with the real thing:

        self.beginAdd = function()
        {
            $('#add').modal('show');
        }

This dialog box is effectively our second View, it is a different entity than our task list. Since this is a different View, we also need a different Model and ViewModel:

    function TasksViewModel() {

        // ... same contents and before

        self.add = function(task)
        {
            self.ajax(self.tasksURI, 'POST', task).done(function(data) {
                self.tasks.push({
                    uri: ko.observable(data.task.uri),
                    title: ko.observable(data.task.title),
                    description: ko.observable(data.task.description),
                    done: ko.observable(data.task.done)
                });
            });
        }
    }
    function AddTaskViewModel() {
        var self = this;
        self.title = ko.observable();
        self.description = ko.observable();

        self.addTask = function() {
            $('#add').modal('hide');
            tasksViewModel.add({
                title: self.title(),
                description: self.description()
            });
            self.title("");
            self.description("");
        }
    }

    var tasksViewModel = new TasksViewModel();
    var addTaskViewModel = new AddTaskViewModel();
    ko.applyBindings(tasksViewModel, $('#main')[0]);
    ko.applyBindings(addTaskViewModel, $('#add')[0]);

The new AddTaskViewModel is extremely simple. It has two observables, which connect to the two <input> tags in the dialog box. It also has one event handler that connects to the Add Task button.

When the button is clicked the event handler simply calls the add() method of the tasksViewModel, passing the title and description values for the new task. The function finally resets the values of the two fields, so that the next time the user wants to add a task the fields appear empty again.

The add() method of tasksViewModel invokes the ajax() function again, but this time it issues a POST request, which according to the REST standard practices corresponds to a request to add a new resource. The response from the request will be the added task, so when our done() callback is invoked we just push a new task into our tasks array. Due to the data binding that exists between the array and the ko.foreach in the HTML portion of the document as soon as we add an element to the array that element is rendered to the page.

The changes to edit an existing tasks are a bit more complex, but are largely similar.

A new dialog box is added to the document (we cannot use the same as the Add Task dialog because for editing we have the "done" checkbox that we don't have when adding a task).

We also have to add a new ViewModel:

    function EditTaskViewModel() {
        var self = this;
        self.title = ko.observable();
        self.description = ko.observable();
        self.done = ko.observable();

        self.setTask = function(task) {
            self.task = task;
            self.title(task.title());
            self.description(task.description());
            self.done(task.done());
            $('edit').modal('show');
        }
        self.editTask = function() {
            $('#edit').modal('hide');
            tasksViewModel.edit(self.task, {
                title: self.title(),
                description: self.description() ,
                done: self.done()
            });
        }
    }

And here are the new and updated methods in the TasksViewModel class to support task editing:

        self.beginEdit = function(task) {
            editTaskViewModel.setTask(task);
            $('#edit').modal('show');
        }
        self.edit = function(task, data) {
            self.ajax(task.uri(), 'PUT', data).done(function(res) {
                self.updateTask(task, res.task);
            });
        }
        self.updateTask = function(task, newTask) {
            var i = self.tasks.indexOf(task);
            self.tasks()[i].uri(newTask.uri);
            self.tasks()[i].title(newTask.title);
            self.tasks()[i].description(newTask.description);
            self.tasks()[i].done(newTask.done);
        }

The additional complication with the edit dialog box is that prior to showing it we have to fill out the form fields with the current data for the selected task. This is done in the setTask() method of the EditTaskViewModel class.

Once the dialog box is accepted the handler in the EditTaskViewModel class calls the edit() method of the TasksViewModel, which sends a PUT request into the server to update the task. Note how the URI for this request comes from the task itself, since each task has a unique URI. Finally, the callback for the ajax request calls a new helper functon updateTask() to refresh the observables associated with the task that was edited.

Another method that we can complete is the one that deletes a task. This one is easy, as it does not require any HTML changes to the document:

        self.remove = function(task) {
            self.ajax(task.uri(), 'DELETE').done(function() {
                self.tasks.remove(task);
            });
        }

And even though the "Mark Done" and "Mark In Progress" buttons appear to work, they aren't really working, all they are doing right now is change the model, but they do not communicate these changes to the server. Here are the updated versions that talk to the server:

        self.markInProgress = function(task) {
            self.ajax(task.uri(), 'PUT', { done: false }).done(function(res) {
                self.updateTask(task, res.task);
            });
        }
        self.markDone = function(task) {
            self.ajax(task.uri(), 'PUT', { done: true }).done(function(res) {
                self.updateTask(task, res.task);
            });
        }

Download the project at this stage: zip | github

The client application is nearing completion now. You can take a break and spend some time playing with the different options, since all of them are working now.

Authentication

And we now arrive to our final battle. Up to now the authentication credentials that we were sending to the REST server are hardcoded into the client application. We happen to know that these work, but in a real life situation we have to prompt the user to provide his own credentials.

Let's start by removing the hardcoded credentials:

        self.username = "";
        self.password = "";

We will also have to add a login dialog box, which I'm not going to show here since it is similar to the previous dialog boxes we created.

We need a ViewModel for our dialog box:

    function LoginViewModel() {
        var self = this;
        self.username = ko.observable();
        self.password = ko.observable();

        self.login = function() {
            $('#login').modal('hide');
            tasksViewModel.login(self.username(), self.password());
        }
    }

And finally we need to add a few support methods to our TasksViewModel class:

    function TasksViewModel() {

        // ...  no changes here

        self.beginLogin = function() {
            $('#login').modal('show');
        }
        self.login = function(username, password) {
            self.username = username;
            self.password = password;
            self.ajax(self.tasksURI, 'GET').done(function(data) {
                for (var i = 0; i < data.tasks.length; i++) {
                    self.tasks.push({
                        uri: ko.observable(data.tasks[i].uri),
                        title: ko.observable(data.tasks[i].title),
                        description: ko.observable(data.tasks[i].description),
                        done: ko.observable(data.tasks[i].done)
                    });
                }
            }).fail(function(jqXHR) {
                if (jqXHR.status == 403)
                    setTimeout(self.beginLogin, 500);
            });
        }

        self.beginLogin();
    }

Let's see how this works. At the bottom of TasksViewModel we had the code that issued the request to get the task list. We now replaced that with a call to beginLogin() which starts the login process by displaying the login dialog box.

When the user accepts the dialog box method login() will be called, with the entered credentials. Only now we are ready to talk to the server, so we issue the request for the task list after we update our login variables. If the request succeeds then we populate our tasks array as we did before.

If the request fails we now have a second callback attached to the ajax request promise. The fail callback in a promise executes if the asynchronous function returns an error. What we do in this case is check the error code from the ajax request object and if found to be 403 we just call beginLogin() again to ask for new credentials. Note we don't show the dialog immediately, instead we wait 0.5 sec with a timeout. This is just because the dialog box has animated events to appear and disappear, so we need to make sure the "show" animation does not collide with the "hide" animation for the previous dialog instance.

I just said that login errors return a 403 code. Those that are familiar with HTTP error codes will jump to correct me. Error 403 is the error code for the "Forbidden" error. The correct HTTP code for the "Not Authorized" error is 401. The problem with error code 401 is that browsers display their own login dialog box when they receive this error, even if the error came from an ajax request. We don't want the browser to show its login dialog since we have our own, so we trick the browser by having the server send error 403 instead of 401.

Download the completed project: zip | github

Security

If we were to deploy this application on a real web server for real users we have to take some security measures.

The REST server takes user credentials according to the HTTP Basic Authentication protocol, which sends usernames and passwords in clear text. Of course this is not acceptable, we should not risk leaking confidential information from our users to third parties.

The proper way to address this problem is by putting the server on secure HTTP, which ensures that communication between client and server is encrypted.

The recommended way to implement secure HTTP is with a proxy server. Our server remains the same, when it starts it listens, for example, on http://localhost:5000, which is only reachable to other processes in the same machine. Then another web server is installed and configured to listen on the secure HTTP port and do all the proper encryption procedures. This server is then configured to act as a proxy for our server, so to us nothing changes, we just receive requests from the proxy server, which in turn receives requests from the user over an encrypted channel.

There are other measures to take if we were to deploy this application on a real server, at the very least the Flask development web server should be replaced with a more robust server. I recommend that you see the deployment chapters of my Flask Mega-Tutorial to learn about this topic.

Final words

Phew! Can you believe we are done?

The application is now fully functional and exercises all the functions of our REST server. A nice enhancement that I will leave as an exercise for those interested would be to save the login credentials in a browser cookie, so that the user does not need to enter them every time. You get bonus points if you add a "Remember Me" checkbox to the login dialog.

Writing rich client side applications in Javascript can be fun. Javascript is an interesting language that at first may seem odd to those of us used to more traditional languages, but once you familiarize with it you find it can do many cool things.

I hope this article and example code along with the previous one serve as a good introduction to the use of REST APIs. If you have any questions or suggestions you are welcome to write below in the comments area.

Miguel

Become a Patron!

Hello, and thank you for visiting my blog! If you enjoyed this article, please consider supporting my work on this blog on Patreon!

122 comments
  • #51 Miguel Grinberg said

    @Simon: the client has the responsibility to send login credentials with every request, the server should not and will not keep user information. But the client application is free to store the login information, it does not need to ask the user for it if that is not desired and instead can store it in local storage or a cookie.

  • #52 Andres said

    Hi Miguel, just one observation. It looks like there is an error in the .fail function call of the security part:

    Instead of

    .fail(function(err) {
    if (jqXHR.status == 403)
    setTimeout(self.beginLogin, 500);
    });

    It looks like it should be something like

    .fail(function(jqXHR) { // to resolve the variable name.
    if (jqXHR.status == 403)
    setTimeout(self.beginLogin, 500);
    });

    By the way, thank you very much for the tutorial. It really help me!

  • #53 Ashwin said

    @Miguel Grinberg - Nice post. I have a question regarding authentication against the api. In your TaskViewModel you are setting the header - with username/password which is exposed in Javascript and anyone has access to that from the browser.

    What is the best way to implement authorization for REST Api's for Single Page Apps or from JS in normal web apps ?

  • #54 Miguel Grinberg said

    Ashwin: Regarding "Anyone has access to that from the browser", what do you mean? The credentials should be sent to the server over HTTPS, so they will not be visible to anyone except the client and the server. The only potential breach can occur in the client, if an attacker manages to get an application running that can access the memory of the Javascript process. Typically the way to address this risk is by exchanging username/password for a token, which is used to authenticate requests. Tokens can have expiration and can be revoked, so they are easier to handle if there is a breach.

  • #55 Sandeep Kumar said

    I learned a lot from your article. But my question to you is that how can I access my switch functionalities using REST API . I am using HP-2920 switch and want ot operate it through REST API.

  • #56 Julio Guzman said

    Miguel, Awesome tutorial!!!

    Maybe it would be really helpful to add a API + Frontend section to the Mega-Flask Tutorial using Angular?

    I would love to see something like that.

    Again, Thank you very much

  • #57 Henning said

    Hi Miguel,
    very great tutorials on creating restful flask api. Thank you for that. But now im struggling with my jQuery client and it drives me crazy:

    The problem is: My flask server and jQuery-client are on the same ip-address with different ports (client: 80, api: 8080). I used the apache server and created two virtual hosts running on those ports. My Apache api-server has enabled:

    WSGIPassAuthorization On
    Header set Access-Control-Allow-Origin "*"
    Header set Access-Control-Allow-Credentials true

    Now everything without (!) authentication works fine. I receive my JSON-data in my jQuery-client. But of course there are some functions on my server that need authentication. I used the basic HTTPAuthentication from your flask restful tutorial.

    But when I run my client and try to authenticate I receive the server response with status "200 OK" .. but there's no data delivered. Also my console shows an "error" after receiving the 200-package. In addition to that my server responses with a 200-status when I type in the wrong credentials.

    When I use cURL to check the functionality of my api everything works fine - also the authentication. Whats going on there??

    My clients function:

    $.ajax({
    url: server + "user/" + request,
    beforeSend: function (xhr) {
    xhr.setRequestHeader ("Content-Type", "application/json");
    xhr.setRequestHeader ("Authorization", btoa(username + ":" + password));
    },
    type: "GET",
    success: function(html){
    console.log(html);
    },
    error: function(html){
    console.log('error');
    }
    });

  • #58 Miguel Grinberg said

    @Henning: nothing comes to mind. I think you need to debug your server and figure out if it is your application that is responding with those 200s, or if these responses are somehow generated by apache.

  • #59 Alex White said

    Great tutorial. Well written and very detailed. Just what I was looking for. Thank you

  • #60 Binay said

    My question is how secure your web service is ? Since url are visible to outside,

  • #61 Miguel Grinberg said

    @Binay: read the remaining articles on this series, there is one on authentication.

  • #62 Dan said

    Hi Miguel

    Thanks for this. You are totally my gateway drug into web development. Another amazing tutorial. I'm getting your book for Christmas!

    I have a few questions about how these techniques relate to those from your mega tutorial:
    Is the use of the g session variable totally out if you want to be ReST compliant? I am using g.user everywhere in my code and find it really useful. It seems a shame to give it up.
    Also, does jinja2 just get forgotten about if you are using these MV* frontend templating techniques? This seemed to be another major part of what Flask is about.
    If so, what is left of Flask (and the mega tutorial) after you take these things out? If being ReST compliant is so important why do these other approaches exist in Flask in the first place?
    The trade-off seems to be all or nothing rather than mixing the best bits of both. What are the problems if you try and combine the 'old' server driven app approach with some of these front end javascript libraries like Knockout.js?

    I think this relates to comments #31 and #32 where you seem to advocate a blended solution. Could you elaborate on where you would draw the line between the two paradigms? That is, what features would you use from each?

    Once again, thank you for this great tutorial series. It's great to see this first-principals approach to client side development.

    Dan

  • #63 Miguel Grinberg said

    @Dan: the "g" variable is not a session variable, it's life span is one request, so it's totally fine to use in REST APIs. The one that would not be a good idea to use is "session".
    If you write a rich client application, with a supporting REST API in the server, then most template rendering needs occur in the client, so typically you will not use Jinja2, or at least it would not be used as widely as in a traditional web application, assuming some parts of the application have server participation besides the API.

    If so, what is left of Flask (and the mega tutorial) after you take these things out?
    You still have the RESTful routing, along with the before/after requests. These help make API writing a pleasant task. And you may still benefit from blueprints if that makes sense for your API.

    If being ReST compliant is so important why do these other approaches exist in Flask in the first place?
    Aha, this is where I think you are not thinking about this in the right way. Being complain with REST is not always important, that is a misconception. If you are building an API, then the REST approach is the one that most people prefer these days. If you are building a web application, then you do not need to use a rich client and an API server, it's perfectly acceptable to create a rich server app, or a hybrid. In fact, these days a hybrid model is, in my opinion, the best choice. So you still get to use Jinja2, and sessions to render pages in the server, but within each page you have small client-side features that you can implement the REST way.

    What are the problems if you try and combine the 'old' server driven app approach with some of these front end javascript libraries like Knockout.js?
    I don't believe there are any problems. It works for me. :-)

    what features would you use from each?
    I think what makes more sense is to implement the larger chunks of functionality as server-side pages. Then within each page you can add dynamic behaviors that do not require a page reload. For example, if I were to recode this blog as a hybrid, I would implement adding a comment as a client-side feature with an API in the server. I would keep each article as a separate page, rendered in the server. I could also add a pop-up notification while you are reading an article if someone posts a comment, that would a good fit for a rich client paradigm.

    Hope this clarifies things a bit!

  • #64 Tweedle said

    Thanks for your great help Miguel. Any angular version in the pipe ?
    (learning CORS with Flask (API, RestFull, Models,WTForms,Auth) and Angular (Factory,Services, Ressources/RestAngular) to bootstrap further Flask-Angular app, maybe with Yeoman)

  • #65 Miguel Grinberg said

    @Tweedle: I'm not sure how I feel about Angular to be honest. Knowing that they plan to change everything for 2.0, I don't want to write an example that will become outdated soon.

  • #66 Hicham said

    Hi Miguel,
    thanks for this nice tutorial. I have developed an Client using AngularJS.
    If you think it will be useful for others , I can share with in Github.

    Let me know.

    Thank you.

  • #67 Miguel Grinberg said

    @Hicham: Yes, I think some will find it useful, so I think putting it on Github is a good idea!

  • #68 Dan said

    Hi Miguel
    This is a follow on from comments 62 and 63. Thanks for your quick reply there. I now have your book (excellent – would recommend to anyone reading this blog) and am trying to work through that as well.

    the "g" variable is not a session variable, it's life span is one request, so it's totally fine to use in REST APIs.
    In your megatutorial you attach the current_user object from the flask-login extension to the g variable in the before_request function. This tracking of the current_user from one request to the next is what threw me. It is this that feels un-ReST, rather than the use of g itself. Is the use of server-side user authentication out in a ReST application?
    The one that would not be a good idea to use is "session".
    In your book you describe the post/redirect/get paradigm to avoid form resubmission issues on page refresh. This relies on the session variable to keep track various data through the redirect. Is this approach now out if you want to be ReST compliant?
    If you are building a web application, then you do not need to use a rich client and an API server, it's perfectly acceptable to create a rich server app, or a hybrid. In fact, these days a hybrid model is, in my opinion, the best choice.
    Primarily it is a web app. It feels like a client-agnostic api is still a good thing to have if you want to support mobile apps accessing the same data and functionality? In this case would you build a traditional server-side/hybrid web app and then stick on an api, perhaps in a blueprint, to support these other client types? If you are going down this route then isn’t it better to develop the whole thing through an api from the start to avoid duplication?
    One of my stumbling blocks when thinking about building an api is that I have some fat view functions; checking complex conditions, updating multiple data items etc. It seems difficult to capture all of the same logic in an api. I am trying to make my view functions thinner. The technique in your book of using custom validators on wtf-forms is a great way to do this; and adding event watching on the database side also seems like a good way to spread the logic out. Am I thinking about this in the right way? Can apis actually be equally complex or should view functions be kept very simple?
    I have a couple of other questions more related to your book:
    When you use the bootstrap quick_form() macro is there a way to include the novalidate attribute? Otherwise I get the native browser validation firing rather than my nice formatted one.
    I’m finding the flask-script/flask-manager/flask-migrate database approach in the book a bit less idiot friendly than the simple migrate.py script you use in the megatutorial. What do you do if you get errors when you try and run ‘manage.py db upgrade’? Do you need to delete the migration file created by the ‘db migrate’ stage, edit the database, then try again? Also, do you have to manage migrations on you dev and test databases separately and manually?
    Sorry some of this isn’t related to the immediate blog.
    Thanks again for all your help. The book is fantastic and really helping me to refactor my code so it looks more professional and maintainable.
    Dan

  • #69 Miguel Grinberg said

    Server-side authentication is necessary in both REST and traditional web apps. The difference is that for a REST API the authentication is done for every request. The server authenticates, handles the request, then forgets about the client. In a web app you create a user session to remember the user across requests.

    Post/redirect/get is a technique to deal with a web browser idiosyncrasy around the refresh button when submitting forms. A REST API is much lower level than that, there is no concept of forms, and the client does not necessarily is a browser (and when it is, it is not sending regular requests, it is sending background requests all within the context of a single page, so refresh does not apply).

    In this case would you build a traditional server-side/hybrid web app and then stick on an api, perhaps in a blueprint, to support these other client types? If you are going down this route then isn’t it better to develop the whole thing through an api from the start to avoid duplication?

    You can structure your code so that there is no duplication. Implement the functionality in a module or package, then have the web app and the API both invoke functions from there.

    One of my stumbling blocks when thinking about building an api is that I have some fat view functions

    Not a good idea. Keep your view functions simple, push code that does the work into reusable modules/packages. Then it can be used for the web app and the api.

    When you use the bootstrap quick_form() macro is there a way to include the novalidate attribute? Otherwise I get the native browser validation firing rather than my nice formatted one.

    Not that I know of. You can steal the quick_form implementation and put a modified version in your code though. Or you can use regular HTML widgets, not the fancier ones from HTML5.

    What do you do if you get errors when you try and run ‘manage.py db upgrade’?

    If you get errors then your database was not changed in any way. So fix the migration and try again.

    Also, do you have to manage migrations on you dev and test databases separately and manually?

    Yes. Each database that you use needs to be managed separately. The test database usually is not a problem, because it is created during the tests. Production you only upgrade when you deploy, so really the one you work with most of the time is the development database.

  • #70 Satish said

    Hi Miguel,
    i followed your all blogs and very good tutorial for flask freshers like me.
    Thanks for great blogs.

    Now i have some issues:
    our Server side is Flask restful, which returns token after successful user authentication (followed your token based authentication https://github.com/miguelgrinberg/REST-auth).. every method works well when i use curl test case provided by you.

    My client side is AngularJs. After successful authentication i set the token in the local storage in java script. but now when i tried to access other restful service with the token it pop ups asking for username and password.

    Can you advice me how to set the token, so that authorisation should happen from token, as it happens with curl

    Here is my client code

    Service class:
    resource: function(success, error) {
    $http.get(baseUrl + 'api/resource').success(success).error(error)
    }

    A common header is set for every request

    $httpProvider.interceptors.push(['$q', '$location', '$localStorage', function($q, $location, $localStorage) {
    return {
    'request': function (config) {
    config.headers = config.headers || {};
    if ($localStorage.token) {
    config.headers.Authorization = $localStorage.token;
    }
    return config;
    },
    'responseError': function(response) {
    if(response.status === 401 || response.status === 403) {
    $location.path('/signin');
    }
    return $q.reject(response);
    }
    };
    }]);

    }
    
  • #71 Miguel Grinberg said

    @Satish: the Authorization header has certain rules that you need to follow. See http://en.wikipedia.org/wiki/Basic_access_authentication#Client_side.

  • #72 Anish said

    Hi Miguel,

    How to do provide logout ( Browser forgetting the credentials ) ? Since its HTTP Basic Auth, the broswer may cache the credentials.
    Is there any uniform way to do logout.

  • #73 Miguel Grinberg said

    @Anish: if you implement tokens, then you can use a mechanism that allows you to revoke a token from the server. With username/password this is not possible, obviously.

  • #74 Cole said

    Thanks for the tutorial Miguel. Extremely helpful in learning knockout.

    There is a typo in your last LoginViewModel
    }).fail(function(err) {
    if (jqXHR.status == 403)
    setTimeout(self.beginLogin, 500);
    });
    Should be
    }).fail(function(jqXHR) {
    if (jqXHR.status == 403)
    setTimeout(self.beginLogin, 500);
    });

  • #75 Miguel Grinberg said

    @code: Thanks, I have updated the article.

Leave a Comment