The Flask Mega-Tutorial, Part XV: Ajax (2012)

Posted by
on under

(Great news! There is a new version of this tutorial!)

This is the fifteenth article in the series in which I document my experience writing web applications in Python using the Flask microframework.

The goal of the tutorial series is to develop a decently featured microblogging application that demonstrating total lack of originality I have decided to call microblog.

NOTE: This article was revised in September 2014 to be in sync with current versions of Python and Flask.

Here is an index of all the articles in the series that have been published to date:

This will be the last of the I18n and L10n themed articles, as we complete our effort to make our microblog application accessible and friendly to non-English speakers.

In this article we will take a departure from the "safe zone" of server-side development we've come to love and will work on a feature that has equally important server and client-side components. Have you seen the "Translate" links that some sites show next to user generated content? These are links that trigger a real time automated translation of content that is not in the user's native language. The translated content is typically inserted below the original version. Google shows it for search results in foreign languages. Facebook shows it for posts. Today we are adding that very same feature to microblog!

Server-side vs. Client-side

In the traditional server-side model that we've followed so far we have a client (the user's web browser) making requests to our server. A request can simply ask for a web page, like when you click the "Your Profile" link, or it can make us perform an action, like when the user edits his or her profile information and clicks the Submit button. In both types of requests the server completes the request by sending a new web page to the client, either directly or by issuing a redirect. The client then replaces the current page with the new one. This cycle repeats for as long as the user stays on our web site. We call this model server-side because the server does all the work while the client just displays the web pages as it receives them.

In the client-side model we have a web browser that again, issues a request to the server. The server responds with a web page like in server-side, but not all the page data is HTML, there is also code, typically written in Javascript. Once the client receives the page it will display it and then execute the code that came with it. From then on you have an active client that can do work on its own without reaching out to the server. In a strict client-side application the entire application is downloaded to the client with the initial page request, and then the application runs on the client without ever refreshing the page, only contacting the server to retrieve or store data. This type of applications are called Single Page Applications or SPAs.

Most applications are a hybrid between the two models and combine techniques of both worlds. Our microblog application is clearly a server-side application, but today we will be adding a little bit of client-side action to the mix. To do real time translations of user posts the client browser will send requests to the server, but the server will respond with translated texts without causing a page refresh. The client will then insert the translations into the current page dynamically. This technique is known as Ajax, which is short for Asynchronous Javascript and XML (even though these days XML is often replaced with JSON).

Translating user generated content

We now have pretty good support for foreign languages thanks to Flask-Babel. Assuming we can find translators willing to help us, we could publish our application in as many languages as we want.

But there is one element missing. We made the application very friendly and inviting to people of all places, so now we are going to get content in a variety of languages, so our users are likely to come across blog posts that are written in languages they don't understand. Wouldn't it be nice if we could offer an automated translation service? The quality of automated translations isn't great, but in most cases it is good enough to get an idea of what someone said in a language we don't speak, so all our users (even ourselves!) can benefit from such a feature.

This is an ideal feature to implement as an Ajax service. Consider that our index page could be showing several posts, some of which might be in several different foreign languages. If we implemented the translation using traditional server-side techniques a request for a translation would cause the original page to get replaced with a new page showing just the translation for the selected post. After reading the translation the user would have to hit the back button to go back to the post listing. The fact is that requesting a translation isn't a big enough action to require a full page update, this feature works much better if the translated text is dynamically inserted below the original text while leaving the rest of the page untouched. So today we are implementing our first Ajax service!

Implementing live automated translations requires a few steps. First we need to identify the source language for the text we want to translate. Once we know the language we also know if a translation is needed for a given user, because we also know what language the user has chosen. When a translation is offered and the user wishes to see it we invoke the Ajax translation service, which will run on our server. In the final step the client-side javascript code will dynamically insert the translated text into the page.

Identifying the language of a post

Our first problem is identifying what language a post was written in. This isn't an exact science, it will not always be possible to detect a language, so we will consider it a "best effort" kind of thing. We are going to use the guess-language Python module for this. Unfortunately if you are using Python 3 the package guess-language will have issues, but a fork exists that can be used in its place:

flask/bin/pip uninstall guess-language
flask/bin/pip install https://bitbucket.org/spirit/guess_language/downloads/guess_language-spirit-0.5a4.tar.bz2

With this module we will scan the text of each blog post to try to guess its language. Since we don't want to scan the same posts over and over again we will do this once for each post, at the time the post is submitted by the user. We will then store the detected language along with the post in our database.

So let's begin by adding a language field to our Posts table:

class Post(db.Model):
    __searchable__ = ['body']

    id = db.Column(db.Integer, primary_key=True)
    body = db.Column(db.String(140))
    timestamp = db.Column(db.DateTime)
    user_id = db.Column(db.Integer, db.ForeignKey('user.id'))
    language = db.Column(db.String(5))

Each time we modify the database we also need to do a migration:

$ ./db_migrate.py
New migration saved as microblog/db_repository/versions/005_migration.py
Current database version: 5

Now we have a place to store the language of each post, so let's detect the language of each added blog post:

from guess_language import guessLanguage

@app.route('/', methods=['GET', 'POST'])
@app.route('/index', methods=['GET', 'POST'])
@app.route('/index/<int:page>', methods=['GET', 'POST'])
@login_required
def index(page=1):
    form = PostForm()
    if form.validate_on_submit():
        language = guessLanguage(form.post.data)
        if language == 'UNKNOWN' or len(language) > 5:
            language = ''
        post = Post(body=form.post.data, 
                    timestamp=datetime.utcnow(), 
                    author=g.user, 
                    language=language)
        db.session.add(post)
        db.session.commit()
        flash(gettext('Your post is now live!'))
        return redirect(url_for('index'))
    posts = g.user.followed_posts().paginate(page, POSTS_PER_PAGE, False)
    return render_template('index.html',
                           title='Home',
                           form=form,
                           posts=posts)

If the language guessing doesn't work or returns an unexpectedly long result we play it safe and save an empty string to the database, a convention that we will use to signal that a post has an unknown language.

Displaying the "Translate" link

The next step is to show a "Translate" link next to any posts that are not in the language used by the user (file app/templates/posts.html):

{% if post.language != None and post.language != '' and post.language != g.locale %}
<div><a href="#">{{ _('Translate') }}</a></div>
{% endif %}

We are doing this in the post.html sub-template, so any page that displays blog posts will get this functionality. The Translate link will only be visible if we could detect the language of the blog post and the language we detected is different than the language selected by the localeselector function from Flask-Babel, which based on the work we did in the previous article we can now access at g.locale.

This link required us to add a new text, the word "Translate" which needs to be included in our translation files, so we entered it with the translation wrapper to make it visible to Flask-Babel. To get it translated we have to refresh our language catalog (tr_update.py), translate in poedit and recompile (tr_compile.py), as seen in the I18n article.

We don't really know what we will be doing to trigger a translation yet, so for now the link does nothing.

Translation services

Before we can proceed with our application we need to find a translation service that we can use.

There are many translation services available, but unfortunately most are either paid services or free with big limitations.

The two major translation services are Google Translate and Microsoft Translator. Both are paid services, but the Microsoft offering has an entry level option for low volume of translations that is free. Google offered a free translation service in the past but that isn't available anymore. That makes our choice of translation service very easy.

Using the Microsoft Translator service

To use the Microsoft Translator there are a couple of requirements that need to be met:

  • The application developer needs to register with the Microsoft Translator app at the Azure Marketplace. Here the level of service can be selected (the free option is at the bottom).
  • Then the developer needs to register an application. The registered application will get Client ID and Client Secret codes to be used as part of the requests sent to this service.

Once the registration part is complete the process to request a translation is as follows:

  • Get an access token, passing in the client id and secret.
  • Call the desired translation method, either Ajax, HTTP or SOAP, providing the access token and the text to translate.

This really sounds more complicated than it really is, so without going into details here is a function that does all the dirty work and translates a text to another language (file app/translate.py):

try:
    import httplib  # Python 2
except ImportError:
    import http.client as httplib  # Python 3
try:
    from urllib import urlencode  # Python 2
except ImportError:
    from urllib.parse import urlencode  # Python 3
import json
from flask_babel import gettext
from config import MS_TRANSLATOR_CLIENT_ID, MS_TRANSLATOR_CLIENT_SECRET

def microsoft_translate(text, sourceLang, destLang):
    if MS_TRANSLATOR_CLIENT_ID == "" or MS_TRANSLATOR_CLIENT_SECRET == "":
        return gettext('Error: translation service not configured.')
    try:
        # get access token
        params = urlencode({
            'client_id': MS_TRANSLATOR_CLIENT_ID,
            'client_secret': MS_TRANSLATOR_CLIENT_SECRET,
            'scope': 'http://api.microsofttranslator.com', 
            'grant_type': 'client_credentials'})
        conn = httplib.HTTPSConnection("datamarket.accesscontrol.windows.net")
        conn.request("POST", "/v2/OAuth2-13", params)
        response = json.loads (conn.getresponse().read())
        token = response[u'access_token']

        # translate
        conn = httplib.HTTPConnection('api.microsofttranslator.com')
        params = {'appId': 'Bearer ' + token,
                  'from': sourceLang,
                  'to': destLang,
                  'text': text.encode("utf-8")}
        conn.request("GET", '/V2/Ajax.svc/Translate?' + urlencode(params))
        response = json.loads("{\"response\":" + conn.getresponse().read().decode('utf-8') + "}")
        return response["response"]
    except:
        return gettext('Error: Unexpected error.')

This function imports two new items from our configuration file, the id and secret codes assigned by Microsoft (file config.py):

# microsoft translation service
MS_TRANSLATOR_CLIENT_ID = '' # enter your MS translator app id here
MS_TRANSLATOR_CLIENT_SECRET = '' # enter your MS translator app secret here

To use the service you will need to register yourself and the application to receive the codes that go in the above configuration variables. Even if you just intend to test this application you can register with the service for free.

We have added more texts, this time a few error messages. These need translations, so we have to re-run tr_update.py, poedit and tr_compile.py to update our translation files.

Let's translate some text!

So how do we use the translator service? It's actually very simple. Here is an example:

$ flask/bin/python
Python 2.6.8 (unknown, Jun  9 2012, 11:30:32)
>>> from app import translate
>>> translate.microsoft_translate('Hi, how are you today?', 'en', 'es')
u'¿Hola, cómo estás hoy?'

Ajax in the server

Now we can translate texts between languages, so we are ready to integrate this functionality into our application.

When the user clicks the Translate link in a post there will be an Ajax call issued to our server. We'll see how this call is made in a bit, for now let's concentrate on implementing the server side of the Ajax call.

An Ajax service in the server is like a regular view function with the only difference that instead of returning an HTML page or a redirect it returns data, typically formatted as either XML or JSON. Since JSON is much more friendly to Javascript we will go with that format (file app/views.py):

from flask import jsonify
from .translate import microsoft_translate

@app.route('/translate', methods=['POST'])
@login_required
def translate():
    return jsonify({ 
        'text': microsoft_translate(
            request.form['text'], 
            request.form['sourceLang'], 
            request.form['destLang']) })

There is not much new here. This route handles a POST request that should come with the text to translate and the source and destination language codes. Since this is a POST request we access these items as if this was data entered in an HTML form, using the request.form dictionary. We call one of our translation functions with this data and once we get the translated text we just convert it to JSON using Flask's jsonify function. The data that the client will see as a response to this request will have this format:

{ "text": "<translated text goes here>" }

Ajax in the client

Now we need to call the Ajax view function from the web browser, so we need to go back to the post.html sub-template to complete the work we started earlier.

We start by wrapping the post text in a span element with a unique id, so that we can later find it in the DOM and replace it with the translated text (file app/templates/post.html):

<p><strong><span id="post{{ post.id }}">{{ post.body }}</span></strong></p>

Note how we construct the unique id using the post's id number. If a given post has an id = 3 then the id of the <span> element that wraps it will be post3.

We are also going to wrap the Translate link in a span with a unique id, in this case so that we can hide it once the translation is shown:

<div><span id="translation{{post.id}}"><a href="#">{{ _('Translate') }}</a></span></div>

Continuing with the example above, the translation link would get the id translation3.

And to make it a nice and user friendly feature we are also going to throw in a spinning wheel animation that starts hidden and will only appears while the translation service is running on the server, also with a unique id:

<img id="loading{{post.id}}" style="display: none" src="/static/img/loading.gif">

So now we have an element called post<id> that contains the text to translate, an element called translation<id> that contains the Translate link but will later be replaced with the translated text and an image with id loading<id> that will be shown while the translation service is working. The <id> suffix is what makes these elements unique, we can have many posts in a page and each will get its own set of three elements.

We now need to trigger the Ajax call when the Translate link is clicked. Instead of triggering the call directly from the link we'll create a Javascript function that does all the work, since we have a few things to do there and we don't want to duplicate code in every post. Let's start by adding a call to this function when the translate link is clicked:

<a href="javascript:translate('{{ post.language }}', '{{ g.locale }}', '#post{{ post.id }}', '#translation{{ post.id }}', '#loading{{ post.id }}');">{{ _('Translate') }}</a>

The template variables obscure this code a bit, but really the function that we are calling is simple. Assuming a post with id = 23 that is written in Spanish and is viewed by a user that uses English then this function will be called as follows:

translate('es', 'en', '#post23', '#translation23', '#loading23')

So this function will receive the source and destination languages and the three DOM elements associated with the post to translate.

We can't write this function in the post.html sub-template because its contents are repeated for each post. We'll implement the function in our base template, so that there is a single copy of it that is accessible in all pages (file app/templates/base.html):

<script>
function translate(sourceLang, destLang, sourceId, destId, loadingId) {
    $(destId).hide();
    $(loadingId).show();
    $.post('/translate', {
        text: $(sourceId).text(),
        sourceLang: sourceLang,
        destLang: destLang
    }).done(function(translated) {
        $(destId).text(translated['text'])
        $(loadingId).hide();
        $(destId).show();
    }).fail(function() {
        $(destId).text("{{ _('Error: Could not contact server.') }}");
        $(loadingId).hide();
        $(destId).show();
    });
}
</script>

We rely on jQuery for this functionality. Recall that we included jQuery back when we styled the application using Twitter's Bootstrap framework.

We start by hiding the element that contains the translate link and showing the spinning wheel image.

Then we use jQuery's $.post() function to send the Ajax request to our server. The $.post function issues a POST request that to the server will be identical to a request issued by the browser when a form is submitted. The difference is that in the client this request happens in the background without causing a page reload. When a response is received from the server the function given as an argument to the done() call will execute and will have a chance to insert the data received into the page. This function receives the response as an argument, so we just overwrite the translate link in the DOM with the translated text, then hide the spinning wheel and finally display the translated text for the user to see. Neat!

If there is any error that prevents the client from getting a response from the server then the function inside the fail() call will be invoked. In this case we just show a generic error message, which also needs to be translated into all languages and thus requires a refresh of our translation database.

Unit testing

Remember our unit test framework? It is always a good idea to evaluate if it makes sense to write tests every time we add a new feature. Invoking the text translation service could be something we could inadvertently break while we work on our code, or it could break due to updates in the service. Let's write a quick test that makes sure we can contact the service and obtain translations:

from app.translate import microsoft_translate

class TestCase(unittest.TestCase):
    #...
    def test_translation(self):
        assert microsoft_translate(u'English', 'en', 'es') == u'Inglés'
        assert microsoft_translate(u'Español', 'es', 'en') == u'Spanish'

We do not have a client-side testing framework at this time, so we will not test the end-to-end Ajax procedure.

Running our testing framework should give us no errors:

$ ./tests.py
.....
----------------------------------------------------------------------
Ran 5 tests in 5.932s

OK

But if you run the test framework without having done the proper configuration for the Microsoft Translator you would get:

$ ./tests.py
....F
======================================================================
FAIL: test_translation (__main__.TestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "./tests.py", line 120, in test_translation
    assert microsoft_translate(u'English', 'en', 'es') == u'Inglés'
AssertionError

----------------------------------------------------------------------
Ran 5 tests in 3.877s

FAILED (failures=1)

Final words

And with this we conclude this article. I hope it was as fun to read for you as it was for me to write!

I have recently been alerted of some issues with the database when Flask-WhooshAlchemy is used for the full text search. In the next chapter I will use this problem as an excuse to go over some of my debugging techniques for Flask applications. Stay tuned for episode XVI!

The code for the updated application is available on github, here. For those that do not use github below is a friendly download link:

Download microblog-0.15.zip.

I look forward to see you in the next article!

Miguel

Become a Patron!

Hello, and thank you for visiting my blog! If you enjoyed this article, please consider supporting my work on this blog on Patreon!

55 comments
  • #26 Miguel Grinberg said

    @Lahan: Thanks, you are correct. I have updated the article and the code now.

  • #27 Christian Banik said

    Dear Miguel,

    first of all, thanks a lot for this great tutorial! I have found a problem with Microsoft Translation service - it always gave me "Error: Unexpected error."

    The reason for this was that the response was encoded in UTF-8 with BOM which json.loads could not deal with.

    This line fixed it:
    response = json.loads("{\"response\":" + conn.getresponse().read().decode('utf-8-sig') + "}")

  • #28 Ahmad said

    I LOVE YOU MIGUEL..!!! Amazing tutorial , two thumbs and two toes up..! learning so much about flask and python

  • #29 Mark Jeghers said

    Hi Miguel,

    I am getting an error trying to JSON-ize the returned data, cannot get pas it. I added more try/except blocks to narrow it down:

    def microsoft_translate(text, sourceLang, destLang):
    if MS_TRANSLATOR_CLIENT_ID == "" or MS_TRANSLATOR_CLIENT_SECRET == "":
    return gettext('Error: translation service not configured.')
    try:
    # get access token
    params = urlencode({
    'client_id': MS_TRANSLATOR_CLIENT_ID,
    'client_secret': MS_TRANSLATOR_CLIENT_SECRET,
    'scope': 'http://api.microsofttranslator.com',
    'grant_type': 'client_credentials'})
    conn = httplib.HTTPSConnection("datamarket.accesscontrol.windows.net")
    conn.request("POST", "/v2/OAuth2-13", params)
    print ('sent connection request')
    except Exception, e:
    return gettext('Error: Unexpected error:' + repr(e))
    try:
    response = json.loads (conn.getresponse().read())
    token = response[u'access_token']
    print ('got token ' + repr(token))
    except Exception, e:
    return gettext('Error: Unexpected error:' + repr(e))

    try:
        # translate
        conn = httplib.HTTPConnection('api.microsofttranslator.com')
        params = {'appId': 'Bearer ' + token,
                  'from': sourceLang,
                  'to': destLang,
                  'text': text.encode("utf-8")}
        conn.request("GET", '/V2/Ajax.svc/Translate?' + urlencode(params))
        print ('sent translate request')
    except Exception, e:
        return gettext('Error: Unexpected error:' + repr(e))
    try:
        # seems to choke here
        response = json.loads('{\"response\":' + conn.getresponse().read().decode('utf-8') + '}')
        return response["response"]
    except Exception, e:
        return gettext('Error: Unexpected error:' + repr(e))
    

    from app import translate
    translate.microsoft_translate('Hi, how are you today?', 'en', 'es')
    sent connection request
    got token u'http%3a%2f%2fschemas.xmlsoap.org%2fws%2f2005%2f05%2fidentity%2fclaim
    s%2fnameidentifier=CrimsonBluesBlog&http%3a%2f%2fschemas.microsoft.com%2faccessc
    ontrolservice%2f2010%2f07%2fclaims%2fidentityprovider=https%3a%2f%2fdatamarket.a
    ccesscontrol.windows.net%2f&Audience=http%3a%2f%2fapi.microsofttranslator.com&Ex
    piresOn=1418948188&Issuer=https%3a%2f%2fdatamarket.accesscontrol.windows.net%2f&
    HMACSHA256=q%2f3cyftAp8OitWPzEBkQUPC8T0sYTkIMLxt0heb5LbU%3d'
    sent translate request
    "Error: Unexpected error:ValueError('No JSON object could be decoded',)"

    With a print statement, I did see some translated text at one point, but am unsure how to properly unpackage it. ANy ideas?

    /Mark

  • #30 Mark Jeghers said

    Also, tried this diagnostic code:

        rsp = conn.getresponse().read().decode('utf-8')
        print (rsp)
    

    ...and got this:

    from app import translate
    translate.microsoft_translate('Hi, how are you today?', 'en', 'es')
    sent connection request
    got token u'http%3a%2f%2fschemas.xmlsoap.org%2fws%2f2005%2f05%2fidentity%2fclaim
    s%2fnameidentifier=CrimsonBluesBlog&http%3a%2f%2fschemas.microsoft.com%2faccessc
    ontrolservice%2f2010%2f07%2fclaims%2fidentityprovider=https%3a%2f%2fdatamarket.a
    ccesscontrol.windows.net%2f&Audience=http%3a%2f%2fapi.microsofttranslator.com&Ex
    piresOn=1418948711&Issuer=https%3a%2f%2fdatamarket.accesscontrol.windows.net%2f&
    HMACSHA256=aEnmH%2fARgMgWdBH8iQG7Z9OIrkmP2fKDa%2bwSA8w1BeY%3d'
    sent translate request
    'Error: Unexpected error:UnicodeEncodeError(\'charmap\', u\'\ufeff"Hola, \xbfc
    \xf3mo est\xe1s hoy?"\', 0, 1, \'character maps to <undefined>\')'

  • #31 Miguel Grinberg said

    @Mark: it appears a utf-8 BOM character was left there, so the decoding of the unicode string isn't working well. Which Python version are you using?

  • #32 Mark Jeghers said

    Using 2.7 since too many thing weren't working quite right yet in 3.4 Plz clarify: where is the problem character and what is wrong about it?

  • #33 Miguel Grinberg said

    @Mark: I am not able to reproduce this problem, but from your dump I believe the problem is the first character in your string, '\ufeff', which is the byte order mark. This character should have been removed during the translation to utf-8. See http://stackoverflow.com/questions/17912307/u-ufeff-in-python-string for a related problem.

  • #34 steve shepard said

    For what it's worth: guess-language available at:

    https://code.google.com/p/guess-language/source/checkout

    (The server I use does not like compressed files ftp'd over)

  • #35 Miguel Grinberg said

    @steve: there is really no need to download anything, you can let pip do this for you.

  • #36 Theo said

    Hi Miguel,

    To implement Christian's fix and, I believe, the issue Mark is having you need to apply the utf-8 decoding when you get the access token as well as when you get the translation. So it should be:

    response = json.loads(conn.getresponse().read().decode('utf-8-sig'))

    instead of

    response = json.loads(conn.getresponse().read())

    -Theo

  • #37 bossip said

    Hello Miguel,
    I had the same error (ValueError: No JSON object could be decoded), using Python 2.7.9.

    response = json.loads(conn.getresponse().read().decode('utf-8-sig')) solved the problem.

    Regards

  • #38 bossip said

    My mistake:
    the correct code is:

        response = json.loads("{\"response\":" + conn.getresponse().read().decode('utf-8-sig') + "}")
    

    not
    response = json.loads(conn.getresponse().read().decode('utf-8-sig')).

  • #39 Aaron Deadman said

    Hi Miguel,

    Thanks for your work. It's been a real help and inspiration in building my own web app using Flask. Do you have an example of submitting WTForms using AJAX (to get the benefit of CSRF protection and form validation)? In this example, it would seem that your translation view is susceptible to it (although it doesn't touch the database, so maybe that's OK).

  • #40 Miguel Grinberg said

    @Aaron: I don't have an example, but the mechanism requires you putting the token as a javascript variable in the page, so that then you can do a jquery form submission via ajax that includes this token.

  • #41 Satish Viswanathan said

    Absolutely awesome tutorial.
    I completed in the second attempt. Really appreciate the time, effort and energy put in by you for this epic tutorial, salute you for that.
    Now for the reason for the second attempt. In the first attempt, i did not have enough Python under my belt to understand what was going around. Once I got that this article was simply awe-inspiring.
    Have a small suggestion though. Can we have the additional code to be written, inside of your code, highlighted so that a person who does not have adequate experience with Python can follow without missing and getting errors.
    But anyone with some decent Python exposure will gain hugely through your posts. Of course Python is one of the best languages to learn programming anyway. This tutorial and your books/videos will be their bible along the web development way.
    My personal thanks.

  • #42 Norbert Stüken said

    Awesome! Just a some correction from my side as a '.' got lost ;):

    It must be
    from .translate import microsoft_translate
    intead of
    from translate import microsoft_translate

    Greets from Berlin!

  • #43 Josh said

    Awesome tutorial, I have learned so much am very grateful!

    Thinking something may have broke on the microsofttranslator api.
    Getting an error when I run this:

    $ flask/bin/python
    Python 2.6.8 (unknown, Jun 9 2012, 11:30:32)

    from app import translate
    translate.microsoft_translate('Hi, how are you today?', 'en', 'es')
    u'¿Hola, cómo estás hoy?'

    End up getting:

    Traceback (most recent call last):
    File "<stdin>", line 1, in <module>
    File "app/translate.py", line 40, in microsoft_translate
    conn.getresponse().read().decode('utf-8') + '}')
    File "/usr/local/Cellar/python/2.7.10_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/init.py", line 338, in loads
    return _default_decoder.decode(s)
    File "/usr/local/Cellar/python/2.7.10_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 366, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    File "/usr/local/Cellar/python/2.7.10_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py", line 384, in raw_decode
    raise ValueError("No JSON object could be decoded")
    ValueError: No JSON object could be decoded

    As a side note there is a microsofttranslator package I just tried out with my credentials and works.

    Thanks again Miguel, I have really expanded my knowledge of not only flask but python by working thru these tutorials.

  • #44 Miguel Grinberg said

    @Norbert: thanks, I corrected the import statement.

  • #45 Miguel Grinberg said

    @Josh: print the result of conn.getresponse().read().decode('utf-8') to find out what's coming back as a response from the translation server.

  • #46 Gavin Elie said

    Hi Miguel

    Thanks for this excellent tutorial !! It is very well written and I am really learning quite a bit about Flask.

    I really appreciate your effort.

    PS: I have slightly modified my code for the AJAX chapter using https://pypi.python.org/pypi/microsofttranslator/0.7. Hope you dont mind!!

    translate.py:

    from flask.ext.babel import gettext
    from config import MS_TRANSLATOR_CLIENT_ID, MS_TRANSLATOR_CLIENT_SECRET
    from microsofttranslator import Translator

    def microsoft_translate(text, destLang):
    if MS_TRANSLATOR_CLIENT_ID == "" or MS_TRANSLATOR_CLIENT_SECRET == "":
    return gettext('Error: translation service not configured.')
    try:
    translator = Translator(MS_TRANSLATOR_CLIENT_ID, MS_TRANSLATOR_CLIENT_SECRET)
    return translator.translate(text, destLang)
    except:
    return gettext('Error: Unexpected error.')

  • #47 Matthieu said

    Hello,

    Thanks for the tuto ;)

    I had some issues with the encoding in the 'translate.py' file. I wonder if Microsoft hasn't changed something since you write the tuto.

    I use CPython 3.4 and I translate to French

    1. When getting the token, I had issues with errors in json. Adding a .decode('utf8') solved the problem.
      conn.request("POST", "/v2/OAuth2-13", binary_params)
      response = json.loads(conn.getresponse().readall().decode("utf8"))

    2. The second one was more tricky ;

      conn.request("GET", '/V2/Ajax.svc/Translate?' + urlencode(params))
      response = json.loads("{\"response\":" + conn.getresponse().read().decode('utf-8') + "}")
      

    Doesn't work, I had strange issues with JSON. When I looked at the string, I noticed some strange characters : ":\ufeff"
    => '{"response":\ufeff"Salut, comment allez-vous aujourd\'hui ?"}'
    In fact, these characters are encoding in UTF8 with BOM (http://stackoverflow.com/questions/17912307/u-ufeff-in-python-string)

    So something like this solved the problem
    conn.request("GET", '/V2/Ajax.svc/Translate?' + urlencode(params))
    res = "{\"response\":" + conn.getresponse().readall().decode("utf-8-sig") + "}"

    Hope it is reproducible and can save some frustration to people :)

  • #48 Payam said

    Hi,

    I have been doing this tutorial from the first section till now. It is an amazing resource. Of course there have been changes from 2013 till now with some of the services. Today I struggled for 6 hours to get this section up and running. There have been significant changes to Microsoft Translator API and urllib in Python 3. Just in case this is translate.py script:
    import json
    from flask_babel import gettext
    import requests
    from xml.etree import ElementTree

    def microsoft_translate(text, sourceLang, destLang):

    # get access token
    token_parameter = {
        'Subscription-Key': '' #Here go to the Keys section of your app in Azure and copy one of the two keys
    }
    r = requests.post("https://api.cognitive.microsoft.com/sts/v1.0/issueToken", params=token_parameter)
    
    
    # translate
    params = {'appid': 'Bearer ' + r.text,
              'from': sourceLang,
              'to': destLang,
              'text': text.encode("utf-8")}
    t = requests.get("https://api.microsofttranslator.com/v2/http.svc/Translate", params=params)
    tree = ElementTree.fromstring(t.content)
    translated_text = tree.text
    response = json.dumps("{\"response\":" + translated_text +"}")
    return response
    

    I hope this will be useful.
    And special thanks to Miguel for this awesome material.

  • #49 Vladislav Veselov said

    @Payam thank you, man!
    It is awesome working script, and I'm so glad!

    But..
    I think, that " response = json.dumps("{\"response\":" + translated_text +"}") " is redundant, coz i was getting escaped string, and spent 2 hours figuring out, why it's so (I have scarce knowledges about JSON)

    But without you I would spent much more time!

  • #50 Nick said

    Miguel,
    Thanks so much for these tutorials. I've never found a resource as helpful as this one has proven to be.

    I'm having some difficulty all of a sudden when I try the ./db_migrate.py command. It's worked for me in the past, and I've checked over all of my code and it's all identical to that within the .zip file at the end of this lesson. Here's the traceback I'm getting:

    /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/flask_sqlalchemy/init.py:839: FSADeprecationWarning: SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and will be disabled by default in the future. Set it to True or False to suppress this warning.
    'SQLALCHEMY_TRACK_MODIFICATIONS adds significant overhead and '
    /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/flask_whooshalchemy.py:18: ExtDeprecationWarning: Importing flask.ext.sqlalchemy is deprecated, use flask_sqlalchemy instead.
    import flask.ext.sqlalchemy as flask_sqlalchemy
    Traceback (most recent call last):
    File "./db_migrate.py", line 4, in <module>
    from app import db
    File "/Users/nicktulli/Desktop/Code/Python/Personal_Projects/microblog/app/init.py", line 65, in <module>
    from app import views, models
    File "/Users/nicktulli/Desktop/Code/Python/Personal_Projects/microblog/app/views.py", line 6, in <module>
    from .forms import LoginForm, EditForm, PostForm, SearchForm
    File "/Users/nicktulli/Desktop/Code/Python/Personal_Projects/microblog/app/forms.py", line 5, in <module>
    from .models import User
    File "/Users/nicktulli/Desktop/Code/Python/Personal_Projects/microblog/app/models.py", line 111, in <module>
    whooshalchemy.whoosh_index(app, Post)
    NameError: name 'whooshalchemy' is not defined

    I have no idea why whooshalchemy is undefined now. I've never run into this problem before when modifying the database. Do you have any idea why this might be the case?

    Thanks once again for the work you've done.

Leave a Comment