Coding

Web UI Testing with Selenium, Pt. 2

Last post dealt with the basics of getting up and running with web UI testing using Selenium Python WebDriver API. This post will deal with using Selenium Server to receive test commands, and writing a multi-threaded controller to run many tests in parallel, and thus simulate a load on a server.

Installation

In addition to having Selenium Web Driver API and Python set up, this post also required Selenium Server to be set up:

1. Download latest selenium-server-standalone-<version>.jar (current version 2.25.0) from the Selenium’s Download Page
2. Optionally for interoperability with IE and Chrome, download the appropriate IEDriverServer from the above download page, and chromedriver as explained in the previous blog post.
3. Add the directory Selenium Server, IEDriverServer, and chromedriver were saved in to the PATH environment variable.

Note: Selenium Server is only required on machines you plan on running the server on. Installation of Selenium Server is not required on the machine the controller will be running. Additionally, if a host machine is only running Selenium Server, it will not require Python, WebDriver API, or any other libraries installed on it – only Selenium Server, Java to run it, and the appropriate browsers/browser drivers need to be installed.

Starting Selenium Server

To start the selenium server with default options, run java –jar <path to>\selenium-server-standalone-<version>.jar

By default, Selenium Server will listen on port 4444. To alter the listening port, use the –port 8080 switch. Ensure any firewalls allow incoming TCP connections on the specified port. In addition, if multiple instances are required (as they will be for the purposes of this blog post), ensure two ports are open per individual instance starting at port 7054. So, if we want to run 5 instances in parallel per machine, we will need to unblock ports 7054 through 7063. If all went well, you should see output similar to the following:

C:\Projects\Selenium\selenium-2.25.0>java -jar selenium-server-standalone-2.25.0.jar -port 8080
Oct 17, 2012 11:44:02 AM org.openqa.grid.selenium.GridLauncher main
INFO: Launching a standalone server
11:44:02.372 INFO – Java: Oracle Corporation 23.1-b03
11:44:02.388 INFO – OS: Windows 7 6.1 amd64
11:44:02.388 INFO – v2.25.0, with Core v2.25.0. Built from revision 17482
11:44:02.497 INFO – RemoteWebDriver instances should connect to:
http://127.0.0.1:8080/wd/hub
11:44:02.497 INFO – Version Jetty/5.1.x
. . . . .

The most important part is line displaying the URL we need to connect to:

11:44:02.497 INFO – RemoteWebDriver instances should connect to: http://127.0.0.1:8080/wd/hub

Executing Tests Against Selenium Server

In the first blog post, we executed a test against our local machine using the webdriver.Firefox/Chrome/etc. objects, avoiding the use of Selenium Server. In order to execute tests against a Selenium Server, we only need to alter the object we use to create the driver. If we started an instance of Selenium Server on our local machine, listening on port 8080, we would create our driver as follows:

driver = webdriver.Remote(url, webdriver.DesiredCapabilities.FIREFOX)

The rest of the code remains identical. The complete code for a simple test is available below:

from selenium import webdriver

def main():
driver = webdriver.Remote("http://localhost:8080/wd/hub", webdriver.DesiredCapabilities.FIREFOX)
driver.get("http://www.google.com")
print "Page Title: " + driver.title
driver.close()

if __name__== "__main__":
main()

Save the above code as controller.py and run by executing python controller.py. You should see a Firefox browser open, Google home page load, and the console print the title of the page.

Running Tests In Parallel

Now that we have a working Selenium Server, we can extend our controller program to run tests in parallel on multiple servers, supplying different (or identical if necessary) test data, and therefore simulating a load. To get a real idea of how the test will behave, we need to run multiple instances of Selenium Server on different machines (virtual or physical). Make sure to record the IP addresses or hostnames of the servers we have running, as we will need them in the test.

As always, we specify the imports at the top. In addition to Selenium imports, we need threading and traceback (for formatted exception printing) libraries:

import time, threading, traceback, sys
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

As a part of our test, we will print a result to the console from each thread, meaning if we would like to have our console output readable, we need to limit console printing to a single thread at a time using a lock. As a part of the threading library, Python provides a Lock object, which can be used to acquire() and release() a lock. We declare the Lock object as a global variable which we can access from each thread:

threadLock = threading.Lock()

Next, we define our run method that will be called by each thread in order to execute our Selenium test. As before, our test will load google.com, execute a search, and to print the number of results returned by the search. Here is the complete code of our run method:

def run(url, query):
try:
	driver = webdriver.Remote(url, webdriver.DesiredCapabilities.FIREFOX)
	driver.implicitly_wait(30)
	driver.get("http://www.google.com")
	q = driver.find_element_by_name("q")
	q.send_keys(query)
	q.send_keys(Keys.RETURN)
	resultsDiv = driver.find_element_by_id("resultStats")
	numResults = resultsDiv.text
	driver.close()

	threadLock.acquire()
	print "Test complete, number of Google results: %s" % numResults
	threadLock.release()
except Exception, e:
	threadLock.acquire()
	print "Unexpected error encountered while running test."
	traceback.print_exc(file=sys.stdout)
	threadLock.release()

The code is essentially the same code as before, however we have added a few new concepts to make the program useable. First, we must pass a URL to Selenium Server which we use to create the Remote driver, and a query to search Google for.

Next, we wrap the entire method in a try/except block. If we allow exceptions to bubble through, our entire program will crash every time an exception happens (such as a web page timeout, etc). Instead, we catch all exceptions, and print them to the console. The traceback_print_exc() method prints the latest caught exception in a pretty manner to the console (stdout). As mentioned earlier, we acquire a lock before writing anything to the console to avoid the likely case of multiple threads writing to the console at the same time.

The code to create a driver, open google.com and execute a search does not differ from our previous tests. As we may want to print some results from our test, once the search executes successfully, we pull the number of results returned by grabbing the text of the resultStats element. We close the browser and print the results, acquiring and releasing the lock in the process.

Creating and Issuing Requests

Now that we have a working test method, we need to somehow create and issue multiple requests in parallel. In the previous test example we used the multiprocessing library to create a thread pool. In the interest of learning something new, we will switch the code up to use the threading library.

First, we define the number of requests we want to issue per host, all available hosts, and our search queries:

maxRequestsPerHost = 1
hosts = [ "localhost:8080", "selenium:8080"]
queries = [ "Selenium", "Protegra", "Winnipeg", "Canada" ]

Next, we build our list of requests by iterating over the available hosts, and creating up to maximum number of requests for each host:

reqs = []
for idx, host in enumerate(hosts):
	first = idx*maxRequestsPerHost
	last = idx*maxRequestsPerHost+maxRequestsPerHost
	for query in queries[first:last]:
		url = str.format("http://{0}/wd/hub", host)
		reqs.append([url, query])

Once we have our list of requests for each host, we need to start a thread for each host, and call the run method supplying the url and query parameters:

maxRequestsPerHost = 1
hosts = [ "localhost:8080", "selenium:8080"]
queries = [ "Selenium", "Protegra", "Winnipeg", "Canada" ]

Now that all our threads are running and executing a test, we wait for all threads to complete before exiting (i.e. join() the threads):

for t in threads:
	t.join()

The above code will issue 2 requests – one on localhost and one on the SELENIUM host. If we increase the maxRequestPerHost to 2, we will issue 4 requests – 2 per host. We could increase the maximum further yet, however we need to add more queries to the list, or we need to alter our code to use query strings more than once.

Next Steps

At this point we have a fully functioning python script that will launch multiple Selenium tests against one or more hosts running Selenium Server. For smaller deployments of several hosts, above method will work just fine. One major disadvantage of the above approach however is trying to manage many Selenium Server hosts. As each test requires a real browser instance, we must physically log onto each Windows machine (RDC is acceptable), and start Selenium Server in a real desktop environment. With the exception of writing a windows service and granting it interaction with the desktop, I do not know of any easy way to automate the steps required to have a fully working Selenium Server in Windows OS. Additionally, setting up individual machines (even if they are VMs) takes time, and requires physical resources which may not be readily available. Next blog post will solve the issue of easily managing as many VMs as necessary for a stress test using the Amazon EC2 service.

Complete Code

Here is the complete code for controller.py

import time, threading, traceback, sys
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

threadLock = threading.Lock()

def run(url, query):
  try:
    driver = webdriver.Remote(url, webdriver.DesiredCapabilities.FIREFOX)
    driver.implicitly_wait(30)
    driver.get("http://www.google.com")
    q = driver.find_element_by_name("q")
    q.send_keys(query)
    q.send_keys(Keys.RETURN)
    resultsDiv = driver.find_element_by_id("resultStats")
    numResults = resultsDiv.text
    driver.close()

    threadLock.acquire()
    print "Test complete, number of Google results: %s" % numResults
    threadLock.release()
  except Exception, e:
    threadLock.acquire()
    print "Unexpected error encountered while running test."
    traceback.print_exc(file=sys.stdout)
    threadLock.release()

def main(): 
  maxRequestsPerHost = 1
  hosts = [ "localhost:8080",  "selenium:8080"]
  queries = [ "Selenium", "Protegra", "Winnipeg", "Canada" ]

  if len(hosts) == 0:
    return

  reqs = []
  for idx, host in enumerate(hosts):
    first = idx*maxRequestsPerHost
    last = idx*maxRequestsPerHost+maxRequestsPerHost
    for query in queries[first:last]:
      url = str.format("http://{0}/wd/hub", host)
      reqs.append([url, query])

  print "Sending %s Requests" % len(reqs)

  threads = []
  for req in reqs:
    thread = threading.Thread(target=run, args=[req[0], req[1]])
    thread.start()
    threads.append(thread)

  for t in threads:
    t.join()

  print "All requests completed processing."

if __name__== "__main__":
    main()

Discussion

No comments yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: