Using Headless Chrome with Selenium
Posted by
on underWhile working on the second edition of my flask book, I was reviewing my Selenium tests, which allow me to automate a web browser and do end-to-end testing. In the current version of the book I recommend running these tests against Firefox. I thought this was a great opportunity to see how Headless Chrome works, as that eliminates the annoying browser window that pops out every time you run the tests.
The results are encouraging. This super short article describes what you need to do to set up Selenium to use the Headless Chrome browser.
Unit Test Structure
In case you haven't seen my book, the way I configure the tests that use Selenium within Python's unittest
framework is shown below:
from selenium import webdriver
class SeleniumTestCase(unittest.TestCase):
client = None
@classmethod
def setUpClass(cls):
# start Firefox
try:
cls.client = webdriver.Firefox()
except:
pass
# skip these tests if the browser could not be started
if cls.client:
# create the application
cls.app = create_app('testing')
cls.app_context = cls.app.app_context()
cls.app_context.push()
# create the database and populate with some fake data
db.create_all()
Role.insert_roles()
User.generate_fake(10)
Post.generate_fake(10)
# start the Flask server in a thread
threading.Thread(target=cls.app.run).start()
# give the server a second to ensure it is up
time.sleep(1)
@classmethod
def tearDownClass(cls):
if cls.client:
# stop the flask server and the browser
cls.client.get('http://localhost:5000/shutdown')
cls.client.close()
# destroy database
db.drop_all()
db.session.remove()
# remove application context
cls.app_context.pop()
def setUp(self):
if not self.client:
self.skipTest('Web browser not available')
def tearDown(self):
pass
# tests go here!
The setUpClass()
method creates a Selenium client, which is stored in the client
class variable, and also creates an application context and a database. Then a real Flask server is started in a background thread. You can't use the Flask test client for this type of test because the browser controlled by Selenium needs a real server it can connect to. The tearDownClass()
just destroys all the resources created in setUpClass()
. The setUp()
method checks that a client
instance exists, and if it doesn't, it tells the unit testing framework that the test needs to be skipped. This can happen if, for example, you did not have Firefox installed.
Replacing Firefox with Chrome Headless
The first thing you need to do to switch to Chrome is to install Chrome if you don't have it yet (obviously!), and then you need to install ChromeDriver, which is the little bit of glue that allows Selenium to send commands to Chrome and automate it. If you are on a Mac, then brew cask install chromedriver
is all you need to do. On other platforms, download an installer from the ChromeDriver site: https://sites.google.com/a/chromium.org/chromedriver/downloads.
To switch to Chrome, you just need to change the initialization of the client
class variable:
# start Chrome
try:
cls.client = webdriver.Chrome()
except:
pass
But this will still use a regular Chrome window. If you want to use the headless option, you have to add options:
# start Chrome
options = webdriver.ChromeOptions()
options.add_argument('headless')
try:
cls.client = webdriver.Chrome(chrome_options=options)
except:
pass
And that's it! Now you get your tests running in the same way as before, but on an invisible Chrome window that is not going to disrupt your other windows. If you want to switch back to the regular mode, just comment out the options.add_argument('headless')
line and you'll get a visible window that you can watch while the tests run.
So far my experience with Headless Chrome is positive. Have you tried it? Let me know how it works for you.
-
#1 Mark ten Brinke said
Hi Miguel,
Great work on your Flask Mega-Tutorial and the second edition of your Flask book.
Is there a release date set for the second edition of the book?Mark.
-
#2 Miguel Grinberg said
@Mark: I don't have much control on the release date of the book because of all the stuff that needs to happen after I deliver my manuscript, but I can tell you that O'Reilly estimates it will be out January 25th 2018, as shown in the pre-order Amazon page here: https://www.amazon.com/Flask-Web-Development-Developing-Applications/dp/1491991739
-
#3 Jose Luis Lopez Pino said
Hi Miguel,
I also like to end-to-end testing with Selenium. In my case, I've used PhantomJS for a while and the experience was good. Do you find any reason to use Headless Chrome over PhantomJS?
-
#4 Miguel Grinberg said
@Jose: If PhantomJS works for you, then I would not switch. One thing I like with the Chrome driver is that you can switch between a visible window and headless very easily, so if I need to look at a test run I can with minimal effort.
-
#5 Angel Talavera said
Hi there,
Do you read minds? How come every time I'm working on a new idea, you come out with a great related
tutorial?Thanks a lot!
-
#6 German Munoz said
@Jose: Not an expert but I understand that the former maintainer of PhantomJS said that "people will switch to it, eventually. Chrome is faster and more stable than PhantomJS. And it doesn’t eat memory like crazy.".
This according to https://intoli.com/blog/running-selenium-with-headless-chrome/
I don´t know why but in my case it always opens up the browser, even if using this technique of the headless option (chrome version 60.0.3, chromedriver 2.30 installed...will check 2.31...). -
#7 Miguel Grinberg said
@German: The versions that I have here are selenium 3.4.3, chrome 60.0.3112.113 and chromedriver 2.31.488774, and with these everything works well.
-
#8 Dmitry said
Good work, Miguel!
Can you write here about ChromeOptions detailed?
1.
I see theoptions.add_argument('headless')
call, but i don't see fulloptions list. I know that exists
headless,
start-maximized,
user-data-dir`, but is there other ChromeOptions ?2.
Is there other options setting way? Now, I know only a way withadd_argument()
. -
#9 Miguel Grinberg said
@Dmitry: Here is the complete list of options: https://peter.sh/experiments/chromium-command-line-switches/
-
#10 Leandro Garcia said
As always, great job. Thanks.
Just one question: Did you tried to run Selenium on Heroku?
-
#11 Miguel Grinberg said
@Leandro: No, I did not. Why would you want to run Selenium on Heroku?
-
#12 metulburr said
i get this error
Message: unknown error: cannot get automation extension
from unknown error: page could not be found: chrome-extension://aapnijgdinlhnhlmodcfapnahmbfebeb/_generated_background_page.html
(Session info: headless chrome=58.0.3029.110)
(Driver info: chromedriver=2.29.461571 (8a88bbe0775e2a23afda0ceaf2ef7ee74e822cc5),platform=Linux 4.4.0-96-generic x86_64) -
#13 Miguel Grinberg said
@metulburr: I believe you need to install a newer chrome driver.
-
#14 Lesha Pak said
One use case for Selenium on Heroku is scraping on a schedule (something with JavaScript, so that BeautifulSoup wouldn't be enough).
-
#15 Grey Li said
Hi Miguel, you can enable Firefox headless mode in the same way now with Firefox 56.
-
#16 Kyle Lawlor said
Hiya, friends. Here's a lil' snippet for running Firefox headless, thanks for the guidance as always Miguel!
from selenium import webdriver options = webdriver.FirefoxOptions() options.set_headless() browser = webdriver.Firefox(firefox_options=options) browser.get('http://localhost:8000') assert 'Title' in browser.title browser.quit()
-
#17 Anurag Choudhary said
Hi Miguel,
Thanks for the article. Can we pass the headless mode via command line to pytest ?
Currently, I have been handling that in conftest.py
Thanks