2017-08-01T17:41:59Z

Using Headless Chrome with Selenium

While working on the second edition of my flask book, I was reviewing my Selenium tests, which allow me to automate a web browser and do end-to-end testing. In the current version of the book I recommend running these tests against Firefox. I thought this was a great opportunity to see how Headless Chrome works, as that eliminates the annoying browser window that pops out every time you run the tests.

The results are encouraging. This super short article describes what you need to do to set up Selenium to use the Headless Chrome browser.

Unit Test Structure

In case you haven't seen my book, the way I configure the tests that use Selenium within Python's unittest framework is shown below:

from selenium import webdriver

class SeleniumTestCase(unittest.TestCase):
    client = None

    @classmethod
    def setUpClass(cls):
        # start Firefox
        try:
            cls.client = webdriver.Firefox()
        except:
            pass

        # skip these tests if the browser could not be started
        if cls.client:
            # create the application
            cls.app = create_app('testing')
            cls.app_context = cls.app.app_context()
            cls.app_context.push()

            # create the database and populate with some fake data
            db.create_all()
            Role.insert_roles()
            User.generate_fake(10)
            Post.generate_fake(10)

            # start the Flask server in a thread
            threading.Thread(target=cls.app.run).start()

            # give the server a second to ensure it is up
            time.sleep(1)

    @classmethod
    def tearDownClass(cls):
        if cls.client:
            # stop the flask server and the browser
            cls.client.get('http://localhost:5000/shutdown')
            cls.client.close()

            # destroy database
            db.drop_all()
            db.session.remove()

            # remove application context
            cls.app_context.pop()

    def setUp(self):
        if not self.client:
            self.skipTest('Web browser not available')

    def tearDown(self):
        pass

    # tests go here!

The setUpClass() method creates a Selenium client, which is stored in the client class variable, and also creates an application context and a database. Then a real Flask server is started in a background thread. You can't use the Flask test client for this type of test because the browser controlled by Selenium needs a real server it can connect to. The tearDownClass() just destroys all the resources created in setUpClass(). The setUp() method checks that a client instance exists, and if it doesn't, it tells the unit testing framework that the test needs to be skipped. This can happen if, for example, you did not have Firefox installed.

Replacing Firefox with Chrome Headless

The first thing you need to do to switch to Chrome is to install Chrome if you don't have it yet (obviously!), and then you need to install ChromeDriver, which is the little bit of glue that allows Selenium to send commands to Chrome and automate it. If you are on a Mac, then brew install chromedriver is all you need to do. On other platforms, download an installer from the ChromeDriver site: https://sites.google.com/a/chromium.org/chromedriver/downloads.

To switch to Chrome, you just need to change the initialization of the client class variable:

        # start Chrome
        try:
            cls.client = webdriver.Chrome()
        except:
            pass

But this will still use a regular Chrome window. If you want to use the headless option, you have to add options:

        # start Chrome
        options = webdriver.ChromeOptions()
        options.add_argument('headless')
        try:
            cls.client = webdriver.Chrome(chrome_options=options)
        except:
            pass

And that's it! Now you get your tests running in the same way as before, but on an invisible Chrome window that is not going to disrupt your other windows. If you want to switch back to the regular mode, just comment out the options.add_argument('headless') line and you'll get a visible window that you can watch while the tests run.

So far my experience with Headless Chrome is positive. Have you tried it? Let me know how it works for you.

13 comments

  • #1 Mark ten Brinke said 2017-08-13T10:41:54Z

    Hi Miguel, Great work on your Flask Mega-Tutorial and the second edition of your Flask book. Is there a release date set for the second edition of the book? Mark.

  • #2 Miguel Grinberg said 2017-08-13T18:28:39Z

    @Mark: I don't have much control on the release date of the book because of all the stuff that needs to happen after I deliver my manuscript, but I can tell you that O'Reilly estimates it will be out January 25th 2018, as shown in the pre-order Amazon page here: https://www.amazon.com/Flask-Web-Development-Developing-Applications/dp/1491991739

  • #3 Jose Luis Lopez Pino said 2017-08-20T22:33:42Z

    Hi Miguel, I also like to end-to-end testing with Selenium. In my case, I've used PhantomJS for a while and the experience was good. Do you find any reason to use Headless Chrome over PhantomJS?

  • #4 Miguel Grinberg said 2017-08-20T23:09:39Z

    @Jose: If PhantomJS works for you, then I would not switch. One thing I like with the Chrome driver is that you can switch between a visible window and headless very easily, so if I need to look at a test run I can with minimal effort.

  • #5 Angel Talavera said 2017-08-22T16:56:14Z

    Hi there, Do you read minds? How come every time I'm working on a new idea, you come out with a great related tutorial? Thanks a lot!

  • #6 German Munoz said 2017-08-29T16:19:23Z

    @Jose: Not an expert but I understand that the former maintainer of PhantomJS said that "people will switch to it, eventually. Chrome is faster and more stable than PhantomJS. And it doesn’t eat memory like crazy.". This according to https://intoli.com/blog/running-selenium-with-headless-chrome/ I don´t know why but in my case it always opens up the browser, even if using this technique of the headless option (chrome version 60.0.3, chromedriver 2.30 installed...will check 2.31...).

  • #7 Miguel Grinberg said 2017-08-29T16:50:44Z

    @German: The versions that I have here are selenium 3.4.3, chrome 60.0.3112.113 and chromedriver 2.31.488774, and with these everything works well.

  • #8 Dmitry said 2017-08-31T12:44:47Z

    Good work, Miguel! Can you write here about ChromeOptions detailed? 1. I see the `options.add_argument('headless')` call, but i don't see full` options list. I know that exists `headless`, `start-maximized`, `user-data-dir`, but is there other ChromeOptions ? 2. Is there other options setting way? Now, I know only a way with `add_argument()`.

  • #9 Miguel Grinberg said 2017-09-01T19:55:52Z

    @Dmitry: Here is the complete list of options: https://peter.sh/experiments/chromium-command-line-switches/

  • #10 Leandro Garcia said 2017-09-24T17:20:05Z

    As always, great job. Thanks. Just one question: Did you tried to run Selenium on Heroku?

  • #11 Miguel Grinberg said 2017-09-25T05:34:00Z

    @Leandro: No, I did not. Why would you want to run Selenium on Heroku?

  • #12 metulburr said 2017-10-11T14:50:30Z

    i get this error Message: unknown error: cannot get automation extension from unknown error: page could not be found: chrome-extension://aapnijgdinlhnhlmodcfapnahmbfebeb/_generated_background_page.html (Session info: headless chrome=58.0.3029.110) (Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.4.0-96-generic x86_64)

  • #13 Miguel Grinberg said 2017-10-11T21:54:52Z

    @metulburr: I believe you need to install a newer chrome driver.

Leave a Comment