Using Headless Chrome with Selenium

Posted by
on under

While working on the second edition of my flask book, I was reviewing my Selenium tests, which allow me to automate a web browser and do end-to-end testing. In the current version of the book I recommend running these tests against Firefox. I thought this was a great opportunity to see how Headless Chrome works, as that eliminates the annoying browser window that pops out every time you run the tests.

The results are encouraging. This super short article describes what you need to do to set up Selenium to use the Headless Chrome browser.

Unit Test Structure

In case you haven't seen my book, the way I configure the tests that use Selenium within Python's unittest framework is shown below:

from selenium import webdriver

class SeleniumTestCase(unittest.TestCase):
    client = None

    @classmethod
    def setUpClass(cls):
        # start Firefox
        try:
            cls.client = webdriver.Firefox()
        except:
            pass

        # skip these tests if the browser could not be started
        if cls.client:
            # create the application
            cls.app = create_app('testing')
            cls.app_context = cls.app.app_context()
            cls.app_context.push()

            # create the database and populate with some fake data
            db.create_all()
            Role.insert_roles()
            User.generate_fake(10)
            Post.generate_fake(10)

            # start the Flask server in a thread
            threading.Thread(target=cls.app.run).start()

            # give the server a second to ensure it is up
            time.sleep(1)

    @classmethod
    def tearDownClass(cls):
        if cls.client:
            # stop the flask server and the browser
            cls.client.get('http://localhost:5000/shutdown')
            cls.client.close()

            # destroy database
            db.drop_all()
            db.session.remove()

            # remove application context
            cls.app_context.pop()

    def setUp(self):
        if not self.client:
            self.skipTest('Web browser not available')

    def tearDown(self):
        pass

    # tests go here!

The setUpClass() method creates a Selenium client, which is stored in the client class variable, and also creates an application context and a database. Then a real Flask server is started in a background thread. You can't use the Flask test client for this type of test because the browser controlled by Selenium needs a real server it can connect to. The tearDownClass() just destroys all the resources created in setUpClass(). The setUp() method checks that a client instance exists, and if it doesn't, it tells the unit testing framework that the test needs to be skipped. This can happen if, for example, you did not have Firefox installed.

Replacing Firefox with Chrome Headless

The first thing you need to do to switch to Chrome is to install Chrome if you don't have it yet (obviously!), and then you need to install ChromeDriver, which is the little bit of glue that allows Selenium to send commands to Chrome and automate it. If you are on a Mac, then brew cask install chromedriver is all you need to do. On other platforms, download an installer from the ChromeDriver site: https://sites.google.com/a/chromium.org/chromedriver/downloads.

To switch to Chrome, you just need to change the initialization of the client class variable:

        # start Chrome
        try:
            cls.client = webdriver.Chrome()
        except:
            pass

But this will still use a regular Chrome window. If you want to use the headless option, you have to add options:

        # start Chrome
        options = webdriver.ChromeOptions()
        options.add_argument('headless')
        try:
            cls.client = webdriver.Chrome(chrome_options=options)
        except:
            pass

And that's it! Now you get your tests running in the same way as before, but on an invisible Chrome window that is not going to disrupt your other windows. If you want to switch back to the regular mode, just comment out the options.add_argument('headless') line and you'll get a visible window that you can watch while the tests run.

So far my experience with Headless Chrome is positive. Have you tried it? Let me know how it works for you.

Become a Patron!

Hello, and thank you for visiting my blog! If you enjoyed this article, please consider supporting my work on this blog on Patreon!

17 comments
  • #1 Mark ten Brinke said

    Hi Miguel,

    Great work on your Flask Mega-Tutorial and the second edition of your Flask book.
    Is there a release date set for the second edition of the book?

    Mark.

  • #2 Miguel Grinberg said

    @Mark: I don't have much control on the release date of the book because of all the stuff that needs to happen after I deliver my manuscript, but I can tell you that O'Reilly estimates it will be out January 25th 2018, as shown in the pre-order Amazon page here: https://www.amazon.com/Flask-Web-Development-Developing-Applications/dp/1491991739

  • #3 Jose Luis Lopez Pino said

    Hi Miguel,

    I also like to end-to-end testing with Selenium. In my case, I've used PhantomJS for a while and the experience was good. Do you find any reason to use Headless Chrome over PhantomJS?

  • #4 Miguel Grinberg said

    @Jose: If PhantomJS works for you, then I would not switch. One thing I like with the Chrome driver is that you can switch between a visible window and headless very easily, so if I need to look at a test run I can with minimal effort.

  • #5 Angel Talavera said

    Hi there,

    Do you read minds? How come every time I'm working on a new idea, you come out with a great related
    tutorial?

    Thanks a lot!

  • #6 German Munoz said

    @Jose: Not an expert but I understand that the former maintainer of PhantomJS said that "people will switch to it, eventually. Chrome is faster and more stable than PhantomJS. And it doesn’t eat memory like crazy.".
    This according to https://intoli.com/blog/running-selenium-with-headless-chrome/
    I don´t know why but in my case it always opens up the browser, even if using this technique of the headless option (chrome version 60.0.3, chromedriver 2.30 installed...will check 2.31...).

  • #7 Miguel Grinberg said

    @German: The versions that I have here are selenium 3.4.3, chrome 60.0.3112.113 and chromedriver 2.31.488774, and with these everything works well.

  • #8 Dmitry said

    Good work, Miguel!

    Can you write here about ChromeOptions detailed?

    1.
    I see the options.add_argument('headless') call, but i don't see fulloptions list. I know that existsheadless,start-maximized,user-data-dir`, but is there other ChromeOptions ?

    2.
    Is there other options setting way? Now, I know only a way with add_argument().

  • #9 Miguel Grinberg said

    @Dmitry: Here is the complete list of options: https://peter.sh/experiments/chromium-command-line-switches/

  • #10 Leandro Garcia said

    As always, great job. Thanks.

    Just one question: Did you tried to run Selenium on Heroku?

  • #11 Miguel Grinberg said

    @Leandro: No, I did not. Why would you want to run Selenium on Heroku?

  • #12 metulburr said

    i get this error

    Message: unknown error: cannot get automation extension
    from unknown error: page could not be found: chrome-extension://aapnijgdinlhnhlmodcfapnahmbfebeb/_generated_background_page.html
    (Session info: headless chrome=58.0.3029.110)
    (Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.4.0-96-generic x86_64)

  • #13 Miguel Grinberg said

    @metulburr: I believe you need to install a newer chrome driver.

  • #14 Lesha Pak said

    One use case for Selenium on Heroku is scraping on a schedule (something with JavaScript, so that BeautifulSoup wouldn't be enough).

  • #15 Grey Li said

    Hi Miguel, you can enable Firefox headless mode in the same way now with Firefox 56.

  • #16 Kyle Lawlor said

    Hiya, friends. Here's a lil' snippet for running Firefox headless, thanks for the guidance as always Miguel!

    from selenium import webdriver
    
    options = webdriver.FirefoxOptions()
    options.set_headless()
    browser = webdriver.Firefox(firefox_options=options)
    browser.get('http://localhost:8000')
    
    assert 'Title' in browser.title
    
    browser.quit()
    
  • #17 Anurag Choudhary said

    Hi Miguel,

    Thanks for the article. Can we pass the headless mode via command line to pytest ?

    Currently, I have been handling that in conftest.py

    Thanks

Leave a Comment