Xceptance, Inc. is hiring a Quality Assurance Test Lead in the Greater Boston Area. Take a look at our job posting.

The German computer magazine c’t recently published the results of their recurring income study among IT personnel.

The hourly rates for knowledge and skills list XLT – Xceptance LoadTest – in second place, right after DB2 and before Windows Embedded.

Yes, we were a little surprised to see XLT skills in such high demand. It encourages and motivates us, because we know that we are on the right track with XLT to deliver an flexible but simple and efficient tool for load and performance testing, as well as regression testing.

What does this mean for you as developer, tester, or IT decider? Get yourself a copy of XLT, take a closer look at it, train your developers, or learn it yourself, because others already have and they are enjoying a competitive advantage.

Source: Zahltag – c’t 06/2012, page 94

Sometimes authentication is necessary before a test case can be executed. While HtmlUnit based tests can easily enter and confirm authentication requests, most browser based tests, cannot workaround the dialog. This is a browser security measure to prevent automated data capture and/or data entering. WebDriver for Firefox delivers a solution for that problem, but IE and Chrome rely on a manual interaction with the browser before the test automation can run.

The following steps describe a solution for the authentication problem and how to run a script test case as WebDriver based test. The key to this solution is the usage of Sikuli, an image based testing tool that directly interacts with the screen to find the right elements by using the screen.

Or in other words, the test automation controls the browser, not the operating system, while Sikuli controls the display of the operating system and uses the keyboard and mouse to control any program.

• Download Sikuli tool for your platform (http://sikuli.org)
• Extract the zip file
• Add the path to Sikuli lib directory to your environment settings
• Add sikuli-script.jar to your project
• Implement a Java class as shown below
• Take the necessary screenshots of the elements to use

import org.junit.Test;
import org.openqa.selenium.ie.InternetExplorerDriver;
import com.xceptance.xlt.api.engine.scripting.AbstractWebDriverScriptTestCase;
import com.xceptance.xlt.engine.scripting.XlteniumScriptInterpreter;
import org.sikuli.script.*;

public class TfoobarIE extends AbstractWebDriverScriptTestCase
{
    /**
     * Constructor starts internet explorer
     */
    public TfoobarIE()
    {
        super(new InternetExplorerDriver(), "http://xlt.xceptance.com");
    }

    /**
     * Main test method solves authentication problem and executes xlt script developer script
     */
    @Test
    public void test() throws Throwable
    {
    	// create Sikuli screen object
    	Screen s = new Screen();

    	// open page with xlt
    	open("/demo/");

    	// wait 5 seconds for the image with the username input field
    	s.wait("./img/username_input_field.jpg",5);

    	// type username "demo" into the input field
    	s.type("demo");    

    	// type password into the password field
    	s.type("./img/password_input_field.jpg","demoomed!");

    	// click ok button
    	s.click("./img/ok_button.jpg");

    	// execute xlt script developer script
    	XlteniumScriptInterpreter interpreter = new XlteniumScriptInterpreter(getWebDriver());
    	interpreter.executeScript("Tfoobar");
    }
}

Do not forget to make the appropriate screenshots of the buttons and input fields (see code) and of course, adjust the path and login data. Have fun and watch the magic happening. For more information how Sikuli works, please visit sikuli.org.

The upcoming Xceptance LoadTest 4.2 will discontinue the direct support of JRuby as a programming language for writing test cases. This will not change the way Java and JRuby interact and therefore you can still use all Java components and integrate them into your JRuby-based test suite. You can also still write test cases in Ruby and use XLT-APIs.

You can read about this feature, in case you are not sure whether or not you will be affected by this change: XLT 3.3.0 JRuby support release note.

Version 4.2 of Xceptance LoadTest will not continue the JRuby package anymore (jruby.jar). The test execution scripts for JRuby-based tests and all demo test cases will be removed as well.

If you have any questions regarding this change, please do not hesitate to contact our support team directly.

XLT 4.1.8 has been released. It contains a couple of improvements and bug fixes. You can get the latest version here: https://lab.xceptance.de/releases/xlt/4.1.8/.

These are the two most important changes.

Optimized scanning CSS rules

XLT simulates the download behavior of a browser as close as possible. With com.xceptance.xlt.css.download.images set to onDemand, XLT carefully considers the download of image resources referred to in CSS files. It checks all CSS rules if they apply to all elements in the DOM tree and, if so, extracts the resource information for later download. This process was previously very expensive and ran for up to several seconds when a page was complex and a lot of CSS rules had to be matched. This has been optimized heavily.

Configurable retry behavior for keep-alive connections

If persistent connections (keep-alive) are enabled, test cases might failed sporadically with an exception such as:
java.lang.RuntimeException: org.apache.http.NoHttpResponseException: The target server failed to respond

This exception occurs only when the request was sent while the connection was about to be closed on the server side at the same time.

The underlying HttpClient retries requests if the connection was closed by the server. However, by default it retries only idempotent operations such as GET and HEAD, but not POST and PUT. In order to avoid these connection errors, the user can now configure whether non-idempotent operations are to be retried as well. Common browsers seem to use the same behavior and most certainly assume that the server did not start to process the request when it closes the connection immediately without responding with a proper HTTP status code.

A new boolean property com.xceptance.xlt.http.retry.nonIdempotentRequests in config/default.properties controls whether or not operations such as PUT and POST are retried in case of certain networking problems. Note that idempotent operations are always retried.

Xceptance is announcing the availability of Xceptance LoadTest (XLT) version 4.1.7. This is an update release and includes smaller improvements and defect fixes. The update is recommended for everyone.

Improvements

• Script Developer: Breakpoints can be set at a module call now. They have been moved to the first command of the module before.
• Runtime Engine: WebDriver has been updated to version 2.16.1.
• Script Developer: Now it supports new HTML 5 input elements, such as email and date, during record and playback.

Fixes

• Runtime Engine: When running multiple test cases at once from Eclipse or with Ant, the values measured for each test case were always stored in the timers.csv file of the first test case. This was a development mode issue only.
• Master controller: Agent controller urls are now checked to ensure that they are unique.
• Script Developer: The base url text box was not correctly updated when the url was changed via the Edit Dialog before.
• Script Developer: WaitFor commands included a small pause before checking the condition even though the condition was already fulfilled. This pause has been removed.
• Script Developer: Using a mouse wheel for scrolling now works in Firefox 9.
• Runtime Engine: When getting the value of an option element, the HtmlUnitDriver does not fall back to the option’s text content in case the option does not specify a value attribute. This behavior is mandatory according to the HTML specification. Fixed in the XltDriver until a fix is provided by the underlying project.
• Runtime Engine: For input elements of type hidden, HtmlUnit’s API method isDisplayed() returned true although hidden inputs are never visible to the user.

All details and the full download can be found here: https://lab.xceptance.de/releases/xlt/4.1.7/

We just released Xceptance LoadTest version 4.1.6. This upgrade release delivers small bugfixes and support for Firefox 9. Additionally you can specify an individual temporary download and upload directory for master and agents now. For the first time, you  also get an XLT-AMI for the new US-West Oregon data center of Amazon.

All details and the full download can be found here: https://lab.xceptance.de/releases/xlt/4.1.6/

This is a nice summary of web application security related technologies, processes, and development patterns: Design Guidelines for Secure Web Applications. A little .NET heavy, but most stuff is generally applicable.

If you read and like the above information, you should not miss the OWASP web security guidelines. This is a must read for every tester and developer. OWASP Guide Project:

Web application security is an essential component of any successful project, whether open source PHP applications, web services such as straight through processing, or proprietary business web sites. Hosters (rightly) shun insecure code, and users shun insecure services that lead to fraud. The aim of this Development Guide is to allow businesses, developers, designers and solution architects to produce secure web applications. If done from the earliest stages, secure applications cost about the same to develop as insecure applications, but are far more cost effective in the long run.

Unlike other forms of security (such as firewalls and secure lockdowns), web applications have the ability to make a skilled attacker rich, or make the life of a victim a complete misery. At this highest level of the OSI software map, traditional firewalls and other controls simply do not help. The application itself must be self-defending. The Development Guide can help you get there. The Development Guide has been written to cover all forms of web application security issues, from old hoary chestnuts such as SQL Injection, through modern concerns such as AJAX, phishing, credit card handling, session fixation, cross-site request forgeries, compliance, and privacy issues…

As you may know, there aren’t only Western character encoded websites on the web. There are also websites in Chinese, Japanese, Arabic, etc., and we wanted to know how XLT would perform if we use it for testing a non-Latin website. Non-Latin does not necessarily mean non-UTF-8, but in this case it especially means non-Western characters and right-to-left (RTL). Or in other words, things a normal European or American programmer is not used to.

So, for the first time we tested a non-Latin website, precisely an Arabic one. We created a test case with the Script Developer to test an Arabic news website. The test entered Arabic words in a search field and validated the response to check the correctness of the website’s content. Afterwards, we ran the script as a JUnit Test in Eclipse. It was successful. The test was short, but provisionally it proved that XLT  also works for non-Latin websites.

Furthermore, we ran the same test in a real browser and it worked as well. We used the FirefoxDriver to simulate user-like actions in Firefox to see if the WebDriver also works with non-Latin input.

When we continued testing, we observed some facts that may interest people who don’t often use Arabic as an input language. As you might know, Arabic is written from right to left. We noticed that there’s a difference between what appears and what actually happens technically. The following example illustrates that.

Placement of the Asterisk

We inserted a text assertion command in the test case. It was an ends-with validation, i.e. the asterisk should be at the first position if the input language is Latin. So, theoretically, if the input language is Arabic, then the asterisk should be at the first position of the Arabic text (the far right), but that wasn’t the case. The asterisk was at the first position from the Latin point of view (the far left).

Then, we tried inserting the asterisk at the first position of the Arabic text (the far right), but as you can see in the figure below the evaluation failed because the asterisk’s encoding is Latin-1, and accordingly the first position is then the far left according to the computer. So, we tried to insert the asterisk in Arabic (as input language), but the evaluation failed, as well.

The next picture shows how it should be.

It was really hard trying to switch our way of thinking to the “right way”.  We spent so much time trying to figure out how to insert the asterisks in the test cases and trying to understand the Arabic point of view.

Not all Asterisks are the same

So, in this case the asterisk must be inserted in Latin. Firstly, the asterisk will be inserted at the last position of the Arabic text and secondly, it has another code. As it turns out, the Arabic asterisk does not have the same code as the Latin asterisk. The Latin asterisk’s code in Unicode code points is U+002A and the Arabic one’s is U+066D, and it even has another name. It is called “Arabic Five Pointed Star” and it actually looks differently. You might be wondering why that is. We asked ourselves the same thing, and we couldn’t find any plausible answer.  Of course, there are different characters in Arabic such as the comma (Arabic comma: “،”), but the asterisk is pretty much an asterisk everywhere and we wondered why it is a different character code.

In the following example we wrote a very simple website as an example to check if the position of the asterisks is always on the wrong side (seen from the Arabic point of view).

At the beginning it is important to set the “charset” to Unicode or you’ll just get question marks as output. As you can see, we set in the head tag the direction of the text alignment to “rtl”- this means “from right to left”- so that the output on the website would appear from right to left. This also applies to the input field in line 13. As you may notice, there’s an exclamation mark added in Latin that’s why it’s shown at the beginning of the Arabic word because it’s technically the last position in Latin. The following picture shows how it appears in the browser.

We decided to write the text in different HTML elements (here: div and span elements) to see if it makes a difference or not if the tested text is over multiple elements. As it turned out, it has no affect if the text is over multiple elements or not and it will be output in the right order, but the problem that the position of the asterisk is wrong to the eyes of an Arab still remains, which may cause great confusion.

Parameters in Western encoding preferred

There’s also a small disadvantage for Arabic developers. Parameter names cannot be put in Arabic. Because our tool only accepts characters “A”-”Z”, “a”-”z”, “0″-”9″ and “_” for parameter names. That goes for test case names, as well. If you export your test cases with the Script Developer and you change the parameter names to Arabic in Eclipse, your test will fail unless you change the parameters in the data files as well, and then it will work just fine. The Script Developer will also show the modified Arabic parameter names and it will replay without any trouble. But if you check the parameter names in the Script Developer, you’ll notice that the name field is empty.

Programming in Arabic

Unfortunately, Arabic isn’t really supported in Eclipse on Windows, even if you run Eclipse in an Arabic version because it’s just a translated version of the platform and the operating system’s encoding is set to ISO Latin-1, i.e. any output in the console that is non-Latin will only be displayed as question marks. But in Linux it works because it supports UTF-8.

We were surprised that the Arabic version could align the Arabic text to the (far) right where it actually should begin, which is not supported by the Latin version of Eclipse. So, if you’re planning on developing a non-Latin website in Eclipse and you stumble upon a version of Eclipse in your language and decide to give it a try don’t get your hopes up because these versions are just translated, and might even be incomplete.

To sum up, we expanded our horizon by proving that our tool could also do test automation problem-free on non-Latin websites.

Smashing Magazine lists a couple of free and commercial tools to cover cross-browser testing:

Good news: very powerful free testing tools are available for Web designers today. Some are more user-friendly than others, and some have significantly better user interfaces. Don’t expect much (if any) support with these tools. But if you’d rather not spend extra money on testing, some great options are here as well.

Read the full article…

By the way, our own tool Xceptance LoadTest (XLT) offers a way to run cross-browser functional tests. XLT leverages WebDriver, a multi-browser API for automation. WebDriver does not support all browser and does not equally support all browser well, but we tried to iron out as much as possible. On top of it, you can use the XLT Script Developer to easily create automation scripts and run them either using our own scripting language or export them to Java to directly run them on the WebDriver-API.

You can download Xceptance LoadTest for free with no strings attached from our web site: www.xceptance-loadtest.com.