The last few days I’ve been working with implementing testing a web application using Selenium. The tests I wrote are relatively simple; what made them complex is the fact that I am tasked with load testing an application through UI testing using a real browser instance (for now limited to Firefox and Chrome, though Selenium supports IE as well). The next few blog posts will detail my experience with writing Selenium tests to simulate a load. The blog posts will not deal with regression testing.
Preface: I fully realize that Selenium is not meant to be used as a load testing framework. An enormous amount of resources are required in order to simulate even a couple hundred concurrent users that makes it impractical. There are actual load-testing frameworks, such as JMeter, that should be used instead; however in the current instance, placing a load on the UI via a real browser was necessary.
Selenium supports many different environments, the simplest of which is a Firefox plugin. I’ve decided to settle on the Python bindings, as I had slight familiarity with it, and I knew it was a powerful language that I can develop quickly in. .NET and Java are two major alternatives, however the rapid work I can accomplish with Python cannot be beat by either.
During my investigation of Selenium I’ve written tests to run from a single machine, many machines (running the same script on multiple machines), and multiple machines via a client/controller mechanism, including deployment in Amazon EC2. The first blog post will deal with the simplest solution – run tests from a single machine. The simplest solution can be extended to any number of machines, however the test must be started manually on each whenever it needs to be run, meaning it ceases to be practical once more than a handful of machines is required.
Installing and setting up Selenium, including additional libraries, is relatively quick:
1. Download and install Python 2.7: http://www.python.org/download/ After installing Python, add the Python installation directory and the Scripts subdirectory to the PATH environment variable (c:\Python27;c:\Python27\Scripts). Restart the command prompt if open after altering the PATH variable.
2. Download and save Distribute and Pip to a desired directory (c:\Selenium for purposes of this post)
3. Start a command prompt (as Administrator if UAC is enabled) and cd to c:\selenium
4. Install distribute: python distribute_setup.py
5. Install pip: python get-pip.py
6. Install Selenium: pip install selenium
7. If you want to use Chrome, download chromedriver (making sure to download the appropriate win_22 or win_23 version depending on version of Chrome installed on your machine), and place it in a directory that is located in the PATH environment variable.
Now we’re ready to run a Selenium test.
Before I dig into the actual code, here is the main reference to consult: Google
And to save some searching:
Save the following code as selenium_test.py (note: do not name the file selenium.py, it will conflict with the selenium API), and run it with python selenium_test.py
from selenium import webdriver driver = webdriver.Firefox() driver.get('http://www.google.com') print 'Page Title: ' + driver.title driver.close()
If the install went smoothly, the above code should start a new Firefox instance, open Google, print the title of the page in the console window, and close Firefox. If no arguments are supplied to the webdriver.Firefox() constructor, a new profile will be created each time Firefox is launched. This is beneficial if we want to have a clean environment with every test. If we want to preserve data such as cookies, or set certain profile parameters, we need to create a webdriver.FirefoxProfile() object, supply a path to the profile, then supply the profile object to the webdriver.Firefox() constructor.
Finding page elements (WebElement)
Once a page is loaded, we need to find one or more elements on the page (links, textboxes, divs, etc.), and do something with them. Continuing from the above example, once we load google we want to find the search box, enter some text in it, then perform a search.
We need an additional import in order to get access to the RETURN (and other) key:
from selenium.webdriver.common.keys import Keys
Then we locate the element in the page based on the element name, type in our search query, and simulate pressing the Return key in the search box:
q = driver.find_element_by_name('q') q.send_keys('Selenium') q.send_keys(Keys.RETURN)
There are a number of ways to search for an element on a page:
by_id – quickest and easiest way to find, however the element we are interested in has to have a unique id that does not change. Many web frameworks assign auto-generated IDs, which cannot be used.
by_link_text/by_partial_link_text – finds elements by link text, or by a partial link text.
by_tag_name – uses tag names to find elements
by_class_name – uses CSS class names to find elements
by_css_selector – uses CSS selector queries to find elements
by_xpath – most useful as it uses standard xpath queries to locate elements, but can be somewhat slow, especially on large pages.
Elements on a page may not be loaded at the same time the page is loaded. For pages such as these (which is a majority of web applications today), Selenium provides two methods of waiting for an element to be loaded
The easier of the two methods is known as an implicit wait. A driver can be forced to wait on every find element query for up to a specified number of seconds before a timeout is reached and an exception is thrown. To enable implicit waits, simply call the implicitly_wait method after creating the driver:
Now the driver will wait up to 30 seconds on every find_element query if the element is not immediately available before a timeout exception is thrown.
The second method, known as an explicit wait, is slightly more verbose, however it allows for controlling the timeout length, and allows for finding elements with and without waits:
from selenium.webdriver.support.ui import WebDriverWait elem = WebDriverWait(driver, 30).until(lambda driver : driver.find_element_by_id(id))
The WebDriverWait object will wait up to the specified timeout for the specified function to succeed, after which a timeout exception will be thrown.
Some web application go a step further, and delay-load elements as they become available. As an example, a search may sporadically load results over the course of 30 seconds. In such cases, both an implicit and an explicit wait will return the results that are immediately available; usually being only the first result. In such cases, it is necessary to sleep for a period of time to allow for results to populate before querying the dom for results.
So far, we’ve only managed to launch a single browser instance, and run a single test. As we are trying to simulate a load, we need to launch multiple browsers to run multiple tests at once. There are a number of ways to accomplish this, simplest perhaps being launching several instances of command prompts, and running the same script many times. Python provides an easy thread pool we can use however:
First, we need a new import statement:
import multiprocessing as mp[/sourceode] Then we need to group our earlier code into a method: def run_test(params): driver = webdriver.Firefox() ... [/sourecocde] Finally, we need to add an entry point to our code, and create a pool of threads: if __name__ == '__main__': parameters = range(3) pool = mp.Pool(len(parameters)) pool.map(run_test, parameters)
The final piece of code here creates a dummy parameter list [0,1,2], creates a pool of threads, and maps the parameters to the pool of threads – in our case, one thread per parameter.
If we want to supply something more useful than a dummy parameter, we can. We can pass an list of arguments to our function by creating a nested list:
parameters = [ ['Selenium', 'Windows'], ['Protegra', 'Software Development'], ['Winnipeg', 'Snow'] ]
And then unpacking it in our run_test method:
def run_test(params): search1, search2 = params ... q.send_keys(search1 + ' ' + search2)
There are of course better ways to pass parameters, but passing a list is quick and easy.
Putting it all Together
Below is the complete source of the test script. The test will spawn as many threads as we have parameters in our list, and each thread will open Firefox, navigate to Google, and search using the specified search parameters.
import multiprocessing as mp from selenium import webdriver from selenium.webdriver.common.keys import Keys def run_test(params): search1, search2 = params driver = webdriver.Firefox() driver.get('http://www.google.com') q = driver.find_element_by_name('q') q.send_keys(search1 + ' ' + search2) q.send_keys(Keys.RETURN) driver.close() if __name__ == '__main__': parameters = [ ['Selenium', 'Windows'], ['Protegra', 'Software Development'], ['Winnipeg', 'Snow'] ] pool = mp.Pool(len(parameters)) pool.map(run_test, parameters)
As indicated in the introduction, the above is the first step in setting up Selenium to run tests using the Python WebDriver API. With a little bit of creativity through running concurrent Chrome and Firefox tests, we can reliably run 2-4 instances of each browser per machine, or possibly more depending on the machine, test, and web application that is being tested. Given a handful of machines or VMs (I have 4 old laptops sitting on my desk), we can simulate a load of 20-30 simultaneous users on a web application. Once we get past 4-6 machines, configuration and running of tests becomes cumbersome.
To partially alleviate the disadvantage of having to manually start test on each individual machine, next post will deal with writing a simple HTTP server to listen for incoming requests, and spawn tests when a requests is made. Once again, Python makes this task easy by providing a high-performance multi-threaded HTTP server (CherryPy WSGI HTTP Server).