Archive for category 'Java'

Use XLT with Sauce Labs and BrowserStack

Wednesday, 12. March 2014 18:13

Sauce Labs and BrowserStack – What Are They and Why Use Them?

Sauce Labs and BrowserStack allow you to run automated test cases on different browsers and operating systems. Both provide more than 200 mobile and desktop browsers on different operating systems. The benefit? You can focus on coding instead of having to maintain different devices. You can easily run your test cases written on iOS on an Internet Explorer without actually buying a Windows device; and last not least, you don’t need to worry about drivers or maintenance.

By the way, Internet Explorer even seems to run faster at Sauce Labs than on a desktop machine. Also note that Sauce Labs supports Maven builds.

What You Need

Not much, really. Only a Sauce Lab or BrowserStack account, a Java IDE (e.g., Eclipse), and XLT 4.3.2.

How to Do it

First, import your XLT test suite into the IDE (Eclipse). Then import all .jar files from the XLT lib directory into your project. In the XLT Script Developer options menu, check “Generate JUnit wrapper class for test cases”.

You then need to “Export…” a test case (select “XLT Scripting API” as API) to generate a new wrapper class so that you can place your new code for the WebDriver in it.

Add the statements below to the exported Java code. Here you can choose your browser (Android, Safari, Firefox, Chrome, Internet Explorer):

DesiredCapabilities capabilities = DesiredCapabilities.firefox();

Now set the desired version of the browser:

capabilities.setCapability("version", "19.0");

Set the operating system on which you want the test case to run:

capabilities.setCapability("platform", Platform.LINUX);

Give your test case a name so that you can refer to it later:

capabilities.setCapability("name", "XLT Firefox Test");

Now you can set up a new WebDriver. Replace “xxx” by your user name and “yyy” by your access key for Sauce Labs:

Webdriver driver = new RemoteWebDriver(
    new URL("http://xxx:yyy@ondemand.saucelabs.com:80/wd/hub"), capabilities);

With this piece of code you should be able to run your test cases on the Sauce Lab servers. This also works to for manually written classes. Just use the same WebDriver settings.

For BrowserStack, you can use the following commands:

DesiredCapabilities caps = DesiredCapabilities.firefox();
caps.setCapability("browser", "FireFox");
caps.setCapability("browser_version", "22.0");
caps.setCapability("os", "Windows");
caps.setCapability("os_version", "7");
caps.setCapability("browserstack.debug", "true");
driver = new RemoteWebDriver( 
    new URL("http://" + USERNAME + ":" + 
        AUTOMATE_KEY + "@hub.browserstack.com/wd/hub”, caps);

Some Final Notes

Selenium works best with Chrome and Firefox. Other browsers, like Internet Explorer or Safari, may fail on test cases that, in fact, do successfully run in Chrome or Firefox. Some websites have a different layout for mobile than for desktop. So you probably can’t use your default tests. If you want to write tests for mobile devices with XLT Script Developer, install user agent changer and change your user agent to iPhone 3.

Sauce Labs provides a detailed tutorial which describes all functions of the service, like parallel testing and other cool stuff. We definitely recommend you read it in case you have any questions.

Category: Automation, Java, Software Development, Testing, XLT | Comments (1) | Author:

Simplify Your Life With a Little UI Automation

Wednesday, 12. June 2013 10:10

Are you familiar with someone asking you to do things such as putting all of your clients into the new CRM or inserting 80 new users in a software product x? If so and if your software has no import feature, you probably found yourself copying and pasting all day long. A simple way to save you from this torture is using a UI automation tool like Sikuli Script.

What is Sikuli?

Sikuli automates UI functionalities like mouse, keyboard, and clipboard actions. It works with UI recognition to identify UI components such as buttons, input fields, or menus. To use this kind of recognition, Sikuli compares the actual screen with screenshots from the user.

You can use Sikuli via the Sikuli IDE, Sikuli Slides, or directly from Java code. The latter is our focus for now.

How to Use Sikuli in Your Java Program

To install Sikuli, visit www.sikuli.org, download the version for your OS, and unzip it. In the lib folder, you can find the sikuli-script.jar that you’ll need in your classpath.

Note: You may have to to unzip the whole downloaded file and install some requirements like OpenCV2.1 (Linux). Also you may need to add some environment variables.

Adding variables to the run config in Eclipse

Adding variables to the run config in Eclipse

As soon as everything is setup, you can start coding.

Example

Let’s go for a simple example: you have a list of URLs and for each you want to open a new tab in Firefox.

Read the list with simple Java I/O operations:

BufferedReader br = new BufferedReader(
                        new InputStreamReader(
                            new FileInputStream("urls.txt"), "UTF-8"));

String string = br.readLine();
while (string != null)
{
    // TODO do something with this line
    string = br.readLine();
}

br.close();

The next step is the automation. Start with a screen:

Screen s = new Screen();

To automate the mouse usage, you need a screenshot of the button you want to press. To use this button, tell Sikuli to find it on your screen:

Pattern add = new Pattern("button.png");
Match match = s.find(add);

Now you’re able to use this button the exact same way as your mouse:

s.click(match);

Keep in mind that, by default, you can just click in the center of the screenshot.

To insert data into an UI, you may also revert to your keyboard. With the type(String s) method from the screen you can type any string using the keyboard. If you decide to use the keyboard, keep in mind that you’ll have to deal with different keyboard layouts. To avoid this, you can simply use the clipboard:

s.paste(string);

Note: It’s always a good idea to give the software some time to react, so add some sleep time:

Thread.sleep(500);

Since Firefox automatically places the cursor into the address bar, we don’t need to add another screenshot to click on. But of course we have to confirm the URL by pressing enter:

s.type(Key.ENTER);

Hint: If you have to fill out multiple fields, use s.type(Key.TAB); instead of clicking into the fields. It’s faster and more reliable.

Running Sikuli from Java has one disadvantage: you have to minimize your IDE after starting the program because Sikuli can only use what it actually sees. To be sure you have enough time to do so, it’s recommended to add some sleep time. Three seconds should do.

Thread.sleep(3000);

Now the program is completed.

import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.InputStreamReader;

import org.sikuli.script.Key;
import org.sikuli.script.Match;
import org.sikuli.script.Pattern;
import org.sikuli.script.Screen;

public class TestSikuli
{
	public static void main(String[] args) throws Exception
	{
		BufferedReader br = new BufferedReader(new InputStreamReader(
								new FileInputStream("urls.txt"), "UTF-8"));

		Thread.sleep(3000);
		Screen s = new Screen();

		Pattern add = new Pattern("button.png");
		Match match = s.find(add);

		String string = br.readLine();
		while (string != null)
		{
			s.click(match);
			Thread.sleep(500);
			s.paste(string);
			s.type(Key.ENTER);

			string = br.readLine();
		}

		br.close();
	}
}

Just run it, place Firefox in the top of your screen, and enjoy the show. Of course, these are only a few of the many possibilities Sikuli offers. However, to solve such simple but annoying tasks as we’ve outlined here, this is all you need.

Note: If you get some .dll errors in Windows, try to run it in debug mode.

Category: Automation, Java, Testing | Comments (0) | Author:

Web Drivers in XLT: Basic Access Authentication

Thursday, 30. May 2013 11:36

Today’s article of our WebDrivers series deals with HTTP authentication – a topic that, at first sight, seems to be very specific and of minor relevance. However, in the world of software testing it’s way more important than you’d think. Often you will have an additional testing instance of a website to be tested. These instances are protected from abuse which is why they require credentials before you can access them. See below for an example in Internet Explorer:

HTTP Authentification IE

HTTP Authentication IE

This browser dialog appears just once. If you’ve entered the right credentials, you can access the related website as often as you like without further authentication – as long as you don’t reopen the browser. The latter is a critical issue for automated WebDriver testing.

Two circumstances are responsible for this:

  1. You can’t perform automated tests using WebDrivers without reopening the browser
  2. Browser dialogs like the one above can’t be accessed during test automation without using other tools

Is there a way to solve this problem? Yes, there is! Go on reading to learn how to achieve basic access authentication during automated testing for each WebDriver.

XltDriver: As a headless browser, the XltDriver lets you transmit the authentication credentials via a property setting. Just look in the XLT default-properties file for the following two properties and adapt them (replace placeholders):

com.xceptance.xlt.auth.userName = USERNAME
com.xceptance.xlt.auth.password = PASSWORD

As for the third-party WebDrivers, you unfortunately have to do without this option.

Yet there’s still another way to achieve authentication: passing the credentials via the URL. This is not recommended for private use, but a useful option in the context of development and testing. The URL has to match the following pattern:

http://username:password@example.com

ChromeDriver/OperaDriver: For these WebDrivers, you can go for the credentials-via-URL-option without any additional expenses. Just adapt the respective base URL in your test cases appropriately and basic access authentication will no longer be a hurdle.

FirefoxDriver: When you enter the URL, the Firefox browser displays such a confirmation dialogue:

HTTP Authentification Firefox

HTTP Authentication Firefox

Remember that browser dialogues can’t be accessed during automated testing? Thus, we have to dismiss this message. In Firefox, you just need to customize this configuration put about:config into the Firefox address bar and search on the config page for the property

network.http.phishy-userpass-length

If such a property doesn’t exist, create it. Right-click and select “New”.
Set the type to Integer and the value really high, maybe 1024, and you end up with the following:

httpAuth_3

From now on, the dialogue won’t come up anymore and you can use the URL access method for test automation.

InternetExplorerDriver: Ever since the release of Internet Explorer 7, the use of URL patterns with embedded credentials like shown above has been disabled due to security concerns. Granted, a good thing for “normal” users, but you are a tester and definitely need this ability. Luckily, you can overcome the disabling by modifying the registry. Open the registry explorer by typing regedit into the Windows command line. Search for the registry key

HKEY_LOCAL_MACHINE\Software\Microsoft\Internet Explorer\Main\FeatureControl\FEATURE_HTTP_USERNAME_PASSWORD_DISABLE

Inside this folder, create two new DWORD-Values (if they don’t already exist). Right-click, select “New” > “DWORD-Value (32-Bit)”, and have them named iexplore.exe and explorer.exe. Their values have to be set to 0 (double-click) as shown below:

Windows regedit http auth disabled

Windows regedit http auth disabled

As we all know that playing around in the registry is a terrible idea when you don’t know what you’re doing, you should be absolutely careful and make sure to select the correct registry key and value names. Registry changes normally require a restart. If you’ve done everything the right way, the authentication via URL should work and allow automated testing with the InternetExplorerDriver.

Stay tuned for our next blog entry that will talk about mobile WebDrivers and how to use them.

Category: Automation, Java, Testing, XLT | Comments (0) | Author:

Handle authentication during WebDriver testing

Saturday, 25. February 2012 20:03

Sometimes authentication is necessary before a test case can be executed. While HtmlUnit based tests can easily enter and confirm authentication requests, most browser based tests, cannot workaround the dialog. This is a browser security measure to prevent automated data capture and/or data entering. WebDriver for Firefox delivers a solution for that problem, but IE and Chrome rely on a manual interaction with the browser before the test automation can run.

The following steps describe a solution for the authentication problem and how to run a script test case as WebDriver based test. The key to this solution is the usage of Sikuli, an image based testing tool that directly interacts with the screen to find the right elements by using the screen.

Or in other words, the test automation controls the browser, not the operating system, while Sikuli controls the display of the operating system and uses the keyboard and mouse to control any program.

  • Download Sikuli tool for your platform (http://sikuli.org)
  • Extract the zip file
  • Add the path to Sikuli lib directory to your environment settings
  • Add sikuli-script.jar to your project
  • Implement a Java class as shown below
  • Take the necessary screenshots of the elements to use
import org.junit.Test;
import org.openqa.selenium.ie.InternetExplorerDriver;
import com.xceptance.xlt.api.engine.scripting.AbstractWebDriverScriptTestCase;
import com.xceptance.xlt.engine.scripting.XlteniumScriptInterpreter;
import org.sikuli.script.*;

public class TfoobarIE extends AbstractWebDriverScriptTestCase
{
    /**
     * Constructor starts internet explorer 
     */
    public TfoobarIE()
    {
        super(new InternetExplorerDriver(), "http://xlt.xceptance.com");
    }
    
    /**
     * Main test method solves authentication problem and executes xlt script developer script
     */
    @Test
    public void test() throws Throwable
    {   
    	// create Sikuli screen object
    	Screen s = new Screen();

    	// open page with xlt
    	open("/demo/");

    	// wait 5 seconds for the image with the username input field 
    	s.wait("./img/username_input_field.jpg",5);

    	// type username "demo" into the input field
    	s.type("demo");    

    	// type password into the password field
    	s.type("./img/password_input_field.jpg","demoomed!");

    	// click ok button
    	s.click("./img/ok_button.jpg");
    	
    	// execute xlt script developer script
    	XlteniumScriptInterpreter interpreter = new XlteniumScriptInterpreter(getWebDriver());
    	interpreter.executeScript("Tfoobar");   	
    }
}

Do not forget to make the appropriate screenshots of the buttons and input fields (see code) and of course, adjust the path and login data. Have fun and watch the magic happening. For more information how Sikuli works, please visit sikuli.org.

Category: Automation, Java, Testing, XLT | Comments (2) | Author:

Review Of Cross-Browser Testing Tools

Sunday, 7. August 2011 17:37

Smashing Magazine lists a couple of free and commercial tools to cover cross-browser testing:

Good news: very powerful free testing tools are available for Web designers today. Some are more user-friendly than others, and some have significantly better user interfaces. Don’t expect much (if any) support with these tools. But if you’d rather not spend extra money on testing, some great options are here as well.

Read the full article…

By the way, our own tool Xceptance LoadTest (XLT) offers a way to run cross-browser functional tests. XLT leverages WebDriver, a multi-browser API for automation. WebDriver does not support all browser and does not equally support all browser well, but we tried to iron out as much as possible. On top of it, you can use the XLT Script Developer to easily create automation scripts and run them either using our own scripting language or export them to Java to directly run them on the WebDriver-API.

You can download Xceptance LoadTest for free with no strings attached from our web site: www.xceptance-loadtest.com.

Category: Java, Testing, XLT | Comments (0) | Author:

Spurious wakeup – the rare event

Friday, 6. May 2011 2:12

After hunting for quite some for a strange application behavior, I finally found the reason.

The Problem

The Java application was behaving strangely in 4 out of 10 runs. It did not process all data available and assumed that the data input already ended. The application features several producer-consumer patterns, where one thread offers preprocessed data to the next one, passing it into a buffer where the next thread reads it from.

The consumer or producer fall into a wait state in case no data is available or the buffer is full. In case of a state change, the active threads notifies all waiting threads about the new data or the fact that all data is consumed.

On 2-core and 8-core machines, the application was running fine but when we moved it to 24-cores, it suddenly started to act in an unpredictable manner.

The Cause

After a lot of debugging I found out that threads wake up without having been notified by their partner thread. In this case the consumer was woken up despite the fact that data was unavailable aka the producer has not delivered and therefore not notified anyone. But the consumer was awake…

The debugging nightmare finally revealed a rare behavior of any POSIX based application. This is the quote from the JDK6 doc:

A thread can also wake up without being notified, interrupted, or timing out, a so-called spurious wakeup. While this will rarely occur in practice, applications must guard against it by testing for the condition that should have caused the thread to be awakened, and continuing to wait if the condition is not satisfied. In other words, waits should always occur in loops.

The Verdict

So never trust your notification chains. Threads might wake up even though nobody directly notified it. Additionally you should never exclude the possibility that the move from a small box to a big box does not influence the application behavior. More cores mean more trouble.

Category: Java, Software Development | Comments (0) | Author:

How does Garbage Collection work?

Monday, 11. April 2011 13:46

Just found two nice blog entries by Chaotic Java which explain nicely how Java Garbage Collection works. Might be still too much if you have never dealt with the topic before, but good reading for the others.

Enjoy reading.

Category: Java | Comments (0) | Author:

The Argument about the Curly Brackets

Thursday, 3. March 2011 8:00

When you talk about code styleguides, you often talk about basic formatting. This means you probably already fought the holy war over the curly brackets {} and where to put them.

Of course, the next line is the only right place. A curly bracket is a hermit and does not like to be put next to any other character…  :)

What is your opinion?

Cartoon courtesy of Geek and Poke under CC-BY-ND-2.0

Category: Java, Software Development | Comments (0) | Author:

XLT – Garbage Collector details visualized

Thursday, 17. February 2011 5:00

Today we want to give you a small preview of an upcoming XLT feature. Most of you probably know that XLT features agent statistics. These statistics help you to keep an eye on the health of the test execution engines (agents) to ensure that you do not influence the test results by providing insufficient hardware or by applying no or incorrect settings.

Most modern programming languages are virtual machine based and these machines have knobs you can turn to adjust their behavior according to your requirements. XLT runs on Java and so all the things you might have already learnt from tuning your Java-based servers apply to XLT as well. If you do not have experience in tuning your Java-based servers, you will learn a lot that can be applied to your servers and help you to increase performance.

So, let’s take a look at the upcoming XLT feature that provides you with more details about garbage collection (GC) without the need to use any external tool to monitor or analyze logs or JVMs runtime behavior.

Perfect GC behavior

This is a chart displays nice memory statistics that indicate absolutely no problems with the garbage collection. As you can see the memory usage (first chart in the image) is alternating but never touching the upper limit. The second chart gives you the times when full garbage collection occurred, basically these are the bad events. Events that block the virtual machine from continuing to run. We should avoid these at any cost… nearly, because a full GC is costly. There was only one full GC right at the beginning at the test. This is a normal JVM behavior. So we seem to have done a pretty good job. No major collection of garbage. Sweat!

There are still the minor collections. Minor collections occur frequently and they reclaim small portions of memory with short living objects. The third charts shows us, that minor collection took about 10 to 30 ms, some spikes up to 70 ms. This is acceptable.

GC with Issues

Ok, let’s have a bad example in comparison. It runs with almost default VM settings. This means just min and max heap were set (Xms256MB/Xmx512MB). You can see clearly that a lot of major collections were running for up to 800 ms each. Also the minor collections were longer.

Clearly, the new charts help you see and fix misbehavior quickly. “What about the settings?” you might ask now. Yes, this example is not very helpful without revealing the difference between both test runs. So, we will give you the settings of the good one. The bad run had only Xms and Xmx set, as mentioned before.

## Set minimum memory to use
-Xms512m

## Set maximum permitted memory
-Xmx512m

## Enable concurrent old GC to avoid sudden long pauses
-XX:+UseConcMarkSweepGC

## When to start GC for the tenured/old area of the memory.
## This has to be low enough to avoid that threads need
## memory and can not get any before the GC has finished.
## This will lower the wait time.
-XX:CMSInitiatingOccupancyFraction=70

That’s really all. The use of the Concurrent-Mark-Sweep-Collector (CMS) is the most important setting. The CMS does not block the main VM worker threads, but collects the garbage concurrently. Because there might be situations were the CMS collector is not fast enough in cleaning up, this means the old space runs out of free memory, we give the CMS a hint with the last line. This line explicitly tells the CMS to start early with the clean up. In this case at a 70% fill level. The default is 96% and way too high.

One last thing. Setting Xms and Xmx to the same number helps the VM to relax because it will not try to get back to the minimal memory limit (Xms) and so it cleans less aggressively.

If you want to lean more about Java Garbage Collection, take a look at this Oracle documents: GC Tuning.

Category: Java, Performance, Testing, XLT | Comments (0) | Author:

What is the state of the G1 Garbage Collector?

Saturday, 31. July 2010 20:39

Because I do not know what is the current state of the Java G1 Garbage Collector, I decided to try G1 with JDK6u20. Somehow I was disappointed because after a short moment of predictable GC performance, the entire VM stopped and some major collection was running. You can easily see that in the charts of that run. Right around 20:09:45, the threads were stopped and the entire VM behaved ugly.

The G1 stop the world

So, the G1 is not yet ready for production, of course nobody stated that it is ready for production. If I read the release notes of JDK6u21 correctly, it delivers plenty of G1 changes, so I might try that soon.

Category: Java, Performance, XLT | Comments (0) | Author: