Summary
In today’s post we will discuss the steps necessary to enhance an XLT-based test suite with multi-browser support. We will show how to tag your test cases to conveniently run them in different environments and execute the test suite in a local or remote fashion.
Introduction
Xceptance maintains a MIT licensed test suite at GitHub which demonstrates functional testing for large scale projects. With the suite we’ve put an emphasis on clear structures, naming and test case organization. Targeting Demandware’s SiteGenesis storefront at heart, the underlying concepts and mechanisms are valuable for everyone building test suites for comparable web applications with XLT. Next to being a template for test automation and best practices in test suite design, it can be a starting point ready to pick up in your very own projects. We regularly utilize it and want to encourage you to explore, employ and contribute.
A regular challenge in testing ecommerce applications is the variety of different browsers and platforms that are available today. As you probably know XLT, the test automation and load testing framework from Xceptance, is based on Selenium browser automation and the Webdriver API. Supporting multiple browsers therefore comes naturally. This blog post will demonstrate how XLT is able to streamline different testing environments directly in your test suite. You will learn how to execute your tests locally and remotely with the help of Sauce Labs and similar automated testing platforms. Along the way you will pick up some details about XLT as well as Script Developer and quickly find yourself equipped with a ready to use multi-browser test suite example.
Tools
To setup your multi-browser testing adventures you will need a few basics first. We will assume that JDK 8 or later is installed on your machine. The latest version of XLT framework is required as well. Furthermore, we recommend some tools, but other alternatives will do nicely as well.
- Browser(s) of your choice, e.g. Chrome, Firefox or Internet Explorer
- Java development environment, e.g. Eclipse
- Optionally, Apache Ant or Maven as execution frameworks
- Optionally, a Sauce Labs account (read below for more details)
Project Setup
Once everything from the list is installed and you have downloaded and extracted the XLT framework, it is time to retrieve the example test suite. Clone the project from its GitHub repository into a folder on your local drive.
1 |
$ git clone https://github.com/Xceptance/multi-browser-suite.git |
Next you will need to import the cloned repository as an existing project into Eclipse. Lastly, add the XLT framework to your newly created Eclipse project as a user library.
Add all .jar
files contained in XLT’s /lib
folder to your newly created user library and confirm. This will reference the XLT framework in your test suite project.
XLT Configuration
If you want to use Ant to execute the test suite, be sure to specify the proper path to where you extracted the XLT framework in the xlt.home.dir
property of the build.properties
file in the multi-browser test suite in Eclipse.
1 2 3 |
# multi-browser-suite/build.properties xlt.home.dir = local/path/to/xlt |
Since the Firefox Webdriver is integrated, its installation is not required. If you want to use the Firefox Webdriver, tell XLT to do so by configuring it in your test suite.
1 2 3 |
# multi-browser-suite/config/default.properties xlt.webDriver = firefox |
If you would like to use the Chrome Webdriver, be sure to download and extract it to your local drive. Similar to Firefox, you need to instruct XLT to use it. Additionally, you need to specify the path to the locally downloaded Chrome driver.
1 2 3 4 |
# multi-browser-suite/config/default.properties xlt.webDriver = chrome xlt.webDriver.chrome.pathToDriverServer = path/to/webDriver/chromedriver.exe |
Another choice of Webdriver can be Internet Explorer, which is available on Microsoft operating systems only. For more information visit this site. A download of IEWebdriver is available here. Unpack the driver and configure it in XLT as follows.
1 2 3 4 |
# multi-browser-suite/config/default.properties xlt.webDriver = ie xlt.webDriver.ie.pathToDriverServer = path/to/webDriver/IEDriverServer.exe |
Configuring Test Execution
After setting up our project and configuring XLT regarding choice of web driver, we will turn to specifying different execution environments for our test cases. We want to be able to easily switch between different browsers and test environments later on. Furthermore, we would like to run our test cases either locally or remotely. Creating so called browser profiles allows us to do all of this effortlessly.
XLT allows remote execution of our tests with automated testing platforms like Sauce Labs, Browser Stack or Selenium Grid. Configure access to one of the grid services in file multi-browser-suite/config/browser.properties
with the help of the following properties template:
1 2 3 |
browserprofile.testEnvironment.<gridname>.url = <url> browserprofile.testEnvironment.<gridname>.username = <username> browserprofile.testEnvironment.<gridname>.password = <password> |
The
constitutes the name of the service you are using. We will reference it later on. The .url
property determines the hub of the testing platform provider. The account details .username
and .password
will be provided by the service as well.
For Sauce Labs the configuration could look as follows. Please note, the password is your sauce labs access key which can be retrieved here.
1 2 3 |
browserprofile.testEnvironment.saucelabs.url = http://ondemand.saucelabs.com/wd/hub browserprofile.testEnvironment.saucelabs.username = <sauce labs="" username=""> browserprofile.testEnvironment.saucelabs.password = <sauce labs="" access="" key=""> |
Next to configuring a grid service, the browser.properties
also host the profiles of browsers and platforms you want your test cases to execute in. A good number of properties are possible, so be sure to have a look into the respective file existing in the provided example suite. We will explain mandatory browser properties and show some common profiles next.
1 2 |
browserprofile.<unique profile="" identifier="">.name = browserprofile.<unique profile="" identifier="">.browser = |
The
is a string that makes the unique browser profiles accessible. Like the grid name we will reference this identifier. The .name
property allows for assigning a detailed name to the browser definition. Property .browser
is used to specify the actual browser, like chrome
, firefox
or internetexplorer
. Optionally, you can define the resolution of your browser window with the .browserResolution
property.
1 |
browserprofile.<unique profile="" identifier="">.browserResolution = |
To execute the profile remotely, we can reference the previously defined grid name with the help of our .testEnvironment
property. For local execution XLT reserves the value local
.
1 2 |
# run on local machine browserprofile.<unique profile="" identifier="">.testEnvironment = local |
1 2 |
# run on sauce labs, with account details given in test environment settings browserprofile.<unique profile="" identifier="">.testEnvironment = saucelabs |
With Sauce Labs you can choose between different (emulated) platform and device parameters. The following table lists the respective properties you can use:
browserprofile. |
Determines which version of the browser should be used or defines the version of the OS of an emulated device. By default the version property references the browser version, but in case of saucelabs device emulation, it may be used for the OS version instead. |
browserprofile. |
Defines on which (possibly emulated) platform the test should execute. |
browserprofile. |
Defines detailed name of the device. |
browserprofile. |
Defines the orientation of the device. |
A few examples of common profiles are shown next. You will find most of the properties utilized which were discussed in preceding sections.
Local Chrome
1 2 3 |
browserprofile.Chrome_local.name = Latest local Chrome browserprofile.Chrome_local.browser = chrome browserprofile.Chrome_local.testEnvironment = local |
Samsung Galaxy Note 3 run as local Chrome Emulation
1 2 3 |
browserprofile.Galaxy_Note3_Emulation.name = Samsung Galaxy Note Chrome Emulation browserprofile.Galaxy_Note3_Emulation.browser = chrome browserprofile.Galaxy_Note3_Emulation.chromeEmulationProfile = Samsung Galaxy Note |
Chrome browser resolution to 1280×900 and Test-Environment to Sauce Labs
1 2 3 4 |
browserprofile.Chrome_1280x900_sl.name = Chrome on SauceLabs browserprofile.Chrome_1280x900_sl.browser = chrome browserprofile.Chrome_1280x900_sl.browserResolution = 1280x900 browserprofile.Chrome_1280x900_sl.testEnvironment = saucelabs |
iPhone 5s on Sauce Labs
1 2 3 4 5 6 7 |
browserprofile.iPhone5s.name = iPhone5s on SauceLabs browserprofile.iPhone5s.browser = iphone browserprofile.iPhone5s.platform = OS X 10.10 browserprofile.iPhone5s.version = 9.2 browserprofile.iPhone5s.deviceName = iPhone 5s browserprofile.iPhone5s.deviceOrientation = portrait browserprofile.iPhone5s.testEnvironment = saucelabs |
Internet Explorer on your own Selenium grid
1 2 3 4 5 |
browserprofile.testEnvironment.myLocalGrid.url = http://localhost:4444/wd/hub browserprofile.ie11.name = IE11 on my own Selenium grid browserprofile.ie11.browser = internetexplorer browserprofile.ie11.version = 11 browserprofile.ie11.testEnvironment = myLocalGrid |
Test Case Configuration
Test scripts created in XLT Script Developer can be executed in Eclipse or other Java execution environments (e.g. Ant). Enable the Script Developers checkbox shown in the following screenshot and a Java wrapper class for each of your test scripts will be generated. These Java classes can be executed directly in Eclipse (or similar).
By default, automatically generated wrapper classes will inherit from class AbstractScriptTestCase
and function as a proxy class during execution. Upon running the classes, the associated test scripts will be instrumented by XLT. Typically you would refrain from editing the generated wrapper classes by hand, because Script Developer may overwrite them at any time.
Now with our multi-browser suite we want to go a step further and execute our test scripts in a number of different execution environments. For this purpose we will link our Java wrapper classes to the browser profiles we did configure earlier. This can be conveniently done with the help of class AbstractAnnotatedScriptTestCase
, which is provided by our multi-browser test suite in multi-browser-suite/src/xltutil/
.
We copy the existing test script Java wrappers generated by Script Developer to some other location in our suite. Next we will modify the wrapper classes, so they inherit from AbstractAnnotatedScriptTestCase
. Now we are in the position to simply use the @TestTarget
annotation to tag our test cases with the unique identifiers of our browser profiles. The following example shows an exemplary Search test case annotated with a local Chrome browser profile.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 |
package tests.search; import com.xceptance.xlt.api.engine.scripting.ScriptName; import xltutil.AbstractAnnotatedScriptTestCase; import xltutil.annotation.TestTargets; @ScriptName("tests.search.TSearch") // the referenced test script @TestTargets({ "Chrome_local" }) public class TSearch extends AbstractAnnotatedScriptTestCase { } |
The multi-browser suite which you initially cloned already includes a number of exemplary test cases in folder /src
that demonstrate the concept. The browser.properties
also have a number of execution environments preconfigured. Be sure to have a look!
Of course it is possible to specify more than one browser profile in a @TestTarget
annotation. XLT will run all different setups that are specified.
1 2 3 |
@TestTargets({ "Chrome_1280x900", "iPhone5s", "Galaxy_Note3_Emulation" }) |
Test Case Execution
Finally we are ready to execute our annotated test cases. You can employ the Eclipse JUnit runner or rely on Apache Ant.
The multi-browser test suite offers some pre-configured Ant targets (e.g. test.java
). Use the browserdefinition
option to run test cases with a specific browser annotation only.
1 |
$ ant test -Dbrowserdefinition=Chrome_1280x900, Chrome_1500x1000 |
Summary
The article described the steps necessary to adapt a test suite created with XLT Script Developer to conveniently support multiple browsers during local or remote execution. The multi-browser test suite accompanying the article backs the demonstrated concepts in more detail and includes all the necessary source code enhancements. Together with the structuring and naming approach demonstrated in our SiteGenesis Community Edition test suite you have a ready to use solution for your own XLT-based test projects at your disposal. Let us know how they went!
Hi Markus. Your perspective on Multi-Browser Support for Test Automation with XLT! The article was absolutely compelling. We have recently posted a blog focusing on Accelerating Time to Market through Next-Gen Test Automation.We would like to get your views on the article and our perspective. This is the link for your reference :http://bit.ly/2bhDIcE