Category Archives: XLT

XLT 4.3 Preview

We are looking forward to release XLT 4.3 soon. A lot of work has been invested to improve XLT and to provide a much easier way to gather all information from our powerful report engine. Have a look at some of the new features below.

Highlights

New summary statistics directly on the overview page

It´s easier than before to get a quick overview of the performance of a system in general. Right on the overview page you will find information about summary statistics calculated over all transactions, actions, requests and custom timers. In addition to these statistics you´ll find a summary graph on the respecting category pages:

Trend reports will also contain summary charts, summary statistics will be available in comparison reports.

Arrival rate charts

We have already explained in detail why using the arrival rate model is best suited to simulate realistic system behavior. It was about time that we are offering dedicated charts to observe the arrival rate of a test run.

New error overview table

In addition to the detailed error table there will be a brand new reduced overview table which provides a quick overview about occurred errors:

Better readability for charts with high peaks

Charts with peaks (for example, caused by timeouts) can be hard to interpret as the relevant value area is squeezed together. We will offer two ways to improve this:

  • The report generator can produce charts with a logarithmic scale instead of a linear scale.
  • The maximum displayed y-value can be limited to a certain upper value, using a configurable factor multiplied by the mean value.

Both ways can be controlled within XLT settings and can be combined as well.

More report improvements

  • It will be possible to switch to the same chart tab for all shown timers
  • Added sorting feature to certain report tables such as the error and the response code table
  • Overview about all responses grouped by their content type

Under the hood

  • XLT 4.3 will use HtmlUnit v2.11
  • It will be possible to ping agent controllers to briefly check whether all configured agent controllers are alive and run the same version of XLT.
  • Improved error messages
  • Added include feature to configuration files which makes it much easier to create flexible test suite configurations for different environments and with different load profiles.

There are many more improvements and several bugfixes. Stay tuned!

HPQC and XLT – Integration Example

You have to work with HP Quality Center (HPQC), but you don’t want to execute all the test cases manually. You automated some tests using XLT Script Developer and like the outcome. You want to use the Script developer much more but you face one last problem: You still have to enter the test results manually into HPQC. This renders some of the test automation advantages useless.

The following example can mitigate that problem. HPQC offers an API called Quality Center Open Test Architecture API (OTA API).
Using this interface, you can set test results automatically.
Continue reading HPQC and XLT – Integration Example

Test Automation for Demandware SiteGenesis with XLT

Demandware, Inc., a leading on-demand ecommerce platform provider, offers a fast and ready to use storefront implementation called SiteGenesis. Implementation partners are using that as a template to implement customized storefronts. This means that testing is part of the effort of course.

Testing a storefront can be a tedious task. It often consists of regression testing to ensure that the already properly tested functionality is still doing well after a bug fix. Additionally a lot of development processes are agile nowadays. You do not want to go back and start testing again, when you get a new iteration, do you? So what to do?

Test automation to the rescue! We are going to provide you an initial test suite that covers most bases and can act as a template for your test automation effort. The test suite runs with Xceptance LoadTest Script Developer and is well documented.

Watch this short introductory video and get an idea what you can achieve with proper test automation.

The test automation suite can be downloaded right here: XLT SiteGenesis Test Automation Suite.

If you are taking part in the development of SiteGenesis because you joined Demandware’s Open Collaboration Model and got access to GitHub, you can also get the latest version of the test suite straight from the repository. We are of course always interested to get your feedback, so fork the suite and let us know.

You need Xceptance LoadTest (XLT) and Mozilla Firefox to get the test suite running. Windows, Linux, MacOS? Any OS is fine. XLT is free of charge. No strings attached.

If you are interested in your own version of the test suite, you want achieve a broader coverage, or your simple need a hint how to proceed, let us know. If you need support for XLT, you can purchase it right here: XLT Service Portal.

P.S. We do not provide you an instance of SiteGenesis for testing. You have to be a signed up partner or customer of Demandware to get your own test instance.

XLT Script Developer and Firefox 16

There is an open regression in Mozilla Firefox 16 (Bug 799348) that can prevent the XLT Script Developer from working correctly. The extension might crash as a result. This mostly happens when you work with multiple tabs or windows. Mozilla does not plan to fix this defect with version 16. The next version 17 will fix it.

To continue working with Script Developer and Firefox you have to downgrade to version 15 and disable the auto-update functionality. Once Mozilla Firefox 17 is released, you can re-enable the auto-update and continue as usual.

Your favorite Firefox extension Firebug might also be affected by that defect.

We apologize for any inconvenience this may cause.

Availability of Xceptance LoadTest 4.2.0

XLT 4.2.0 delivers a large set of new and improved functionality.

The Script Developer has seen a lot of enhancements. Most noticeably is the support of variables now. You can store data from the web page, run calculations in JavaScript and store the result, and reuse all later. There is now the notion of global and local test data. So any data that applies to multiple test cases at the same time can be managed more easily now. Besides the already mentioned store commands, the Script Developer also permits assertion and wait commands for visibility, select and check boxes, as well as value attributes of an element.

Further usability improvements permit easier switching between projects and the execution of a single command in a test case for faster development. Test cases can now be run directly from the package tree without the need to open them first. The test data management has been completely reworked to simplify and speed up editing of data.

The load test engine supports a fully variable load test profile now. You can define any load pattern you want, such as a 24 hour test with peaks around noon and early afternoon, low traffic during night hours, slow ramp up in the morning and multiple load changes over the course of the day. Additionally the ramp up of arrival rates is now supported as well.

To speed up things and fully utilize the available connectivity, the upload to and the download from agents has been fully parallelized. Optionally you can adjust the desired concurrency. Also the timeout against agents can now be set.

The documentation layout as well as the report design have been redesigned to improve usability, make them visually more pleasant, and unify their appearance. Third party libraries used to deliver enhanced functionality have been upgraded to cover the latest browsers as well as UI trends.

Please see the detailed release notes for more information about new features, enhancements, and defect fixes.

XLT skills are in demand

XLT skills in demandThe German computer magazine c’t recently published the results of their recurring income study among IT personnel.

The hourly rates for knowledge and skills list XLT – Xceptance LoadTest – in second place, right after DB2 and before Windows Embedded.

Yes, we were a little surprised to see XLT skills in such high demand. It encourages and motivates us, because we know that we are on the right track with XLT to deliver an flexible but simple and efficient tool for load and performance testing, as well as regression testing.

What does this mean for you as developer, tester, or IT decider? Get yourself a copy of XLT, take a closer look at it, train your developers, or learn it yourself, because others already have and they are enjoying a competitive advantage.

Source: Zahltag – c’t 06/2012, page 94

Handle authentication during WebDriver testing

Sometimes authentication is necessary before a test case can be executed. While HtmlUnit based tests can easily enter and confirm authentication requests, most browser based tests, cannot workaround the dialog. This is a browser security measure to prevent automated data capture and/or data entering. WebDriver for Firefox delivers a solution for that problem, but IE and Chrome rely on a manual interaction with the browser before the test automation can run.

The following steps describe a solution for the authentication problem and how to run a script test case as WebDriver based test. The key to this solution is the usage of Sikuli, an image based testing tool that directly interacts with the screen to find the right elements by using the screen.
Continue reading Handle authentication during WebDriver testing

Discontinued support of JRuby

The upcoming Xceptance LoadTest 4.2 will discontinue the direct support of JRuby as a programming language for writing test cases. This will not change the way Java and JRuby interact and therefore you can still use all Java components and integrate them into your JRuby-based test suite. You can also still write test cases in Ruby and use XLT-APIs.

You can read about this feature, in case you are not sure whether or not you will be affected by this change: XLT 3.3.0 JRuby support release note.

Version 4.2 of Xceptance LoadTest will not continue the JRuby package anymore (jruby.jar). The test execution scripts for JRuby-based tests and all demo test cases will be removed as well.

If you have any questions regarding this change, please do not hesitate to contact our support team directly.

XLT 4.1.8 Update Release

XLT 4.1.8 has been released. It contains a couple of improvements and bug fixes. You can get the latest version here: https://lab.xceptance.de/releases/xlt/4.1.8/.

These are the two most important changes.

Optimized scanning CSS rules

XLT simulates the download behavior of a browser as close as possible. With com.xceptance.xlt.css.download.images set to onDemand, XLT carefully considers the download of image resources referred to in CSS files. It checks all CSS rules if they apply to all elements in the DOM tree and, if so, extracts the resource information for later download. This process was previously very expensive and ran for up to several seconds when a page was complex and a lot of CSS rules had to be matched. This has been optimized heavily.

Configurable retry behavior for keep-alive connections

If persistent connections (keep-alive) are enabled, test cases might failed sporadically with an exception such as:
java.lang.RuntimeException: org.apache.http.NoHttpResponseException: The target server failed to respond

This exception occurs only when the request was sent while the connection was about to be closed on the server side at the same time.

The underlying HttpClient retries requests if the connection was closed by the server. However, by default it retries only idempotent operations such as GET and HEAD, but not POST and PUT. In order to avoid these connection errors, the user can now configure whether non-idempotent operations are to be retried as well. Common browsers seem to use the same behavior and most certainly assume that the server did not start to process the request when it closes the connection immediately without responding with a proper HTTP status code.

A new boolean property com.xceptance.xlt.http.retry.nonIdempotentRequests in config/default.properties controls whether or not operations such as PUT and POST are retried in case of certain networking problems. Note that idempotent operations are always retried.

Availability of XLT 4.1.7

Xceptance is announcing the availability of Xceptance LoadTest (XLT) version 4.1.7. This is an update release and includes smaller improvements and defect fixes. The update is recommended for everyone.

Improvements

  • Script Developer: Breakpoints can be set at a module call now. They have been moved to the first command of the module before.
  • Runtime Engine: WebDriver has been updated to version 2.16.1.
  • Script Developer: Now it supports new HTML 5 input elements, such as email and date, during record and playback.

Fixes

  • Runtime Engine: When running multiple test cases at once from Eclipse or with Ant, the values measured for each test case were always stored in the timers.csv file of the first test case. This was a development mode issue only.
  • Master controller: Agent controller urls are now checked to ensure that they are unique.
  • Script Developer: The base url text box was not correctly updated when the url was changed via the Edit Dialog before.
  • Script Developer: WaitFor commands included a small pause before checking the condition even though the condition was already fulfilled. This pause has been removed.
  • Script Developer: Using a mouse wheel for scrolling now works in Firefox 9.
  • Runtime Engine: When getting the value of an option element, the HtmlUnitDriver does not fall back to the option’s text content in case the option does not specify a value attribute. This behavior is mandatory according to the HTML specification. Fixed in the XltDriver until a fix is provided by the underlying project.
  • Runtime Engine: For input elements of type hidden, HtmlUnit’s API method isDisplayed() returned true although hidden inputs are never visible to the user.

All details and the full download can be found here: https://lab.xceptance.de/releases/xlt/4.1.7/