Videos of TEDxFSU Jena

Friday, 20. March 2015 20:12

About a month ago, we proudly sponsored the TEDxFSU event in Jena. Today, the videos of all talks came online and of course we want to share them with you, because the TEDx mission is free access to education. Enjoy: Videos of TEDxFSU Jena.

This is one of the talks in which Peter Perceval discusses the connection between humor and innovation.

Category: Xceptance | Comments (0)

Our Demandware SiteGenesis Community Test Suite

Saturday, 14. March 2015 18:03

dw-community-test-suite-homepageToday, we are happy to announce the release of our new Demandware SiteGenesis Community Test Suite! It is a test suite for automated storefront testing of the Demandware SiteGenesis reference e-commerce storefront.

The test suite’s intention is to share experiences, transfer knowledge, and to demonstrate best practices in test automation. The test suite is built using XLT of course. XLT is freely available and extends concepts you already know from Selenium.


  • Over a dozen test cases and more than 200 modules you can build on
  • Test cases for the search, cart, checkout, and account area, as well as catalog navigation
  • Extensive use of test data for test cases and test packages
  • Ready to run via Script Developer or Firefox and Chrome WebDriver
  • Extensive inline as well as exportable documentation for test cases and code
  • Extensive documentation for the structure and style of the test suite
  • Open Source under MIT license
  • Completely free to share, fork, and derive

If you’re a Demandware customer, this test suite will be a great starting point for your own test automation projects. It lays a foundation you can build on. We want to make this project a hub for exchanging ideas, test cases, defects, and much more around test automation. We will add more test cases over time and regularly adjust the tests to new versions of SiteGenesis.

The Community Test Suite at GitHubWe will also continuously update the documentation with information such as Jenkins integration, Sauce Labs support, and the like.

If you are not a Demandware customer, you can still study the design, code, and concepts and derive ideas for your own e-commerce test suite.

The test suite can be downloaded and forked on GitHub (Open Source, MIT License). The installation is a short straightforward process described in the project’s readme.

Please feel free to use it for your own purposes and let us know what you think about it. Any feedback is appreciated!

Category: Automation, Open Source, Testing, XLT | Comments (0)

Xceptance released XLT 4.5

Wednesday, 25. February 2015 21:38

Xceptance released version 4.5 of its load testing and test automation product Xceptance LoadTest (XLT). This release enables you to write better test documentation, delivers more scripting capabilities, and shows really cool charts.

XLT 4.5.1 ScriptdocScript Developer has been enhanced to let you maintain the documentation of your script library directly in Script Developer. This way you can create and maintain your test documentation together with the code and don’t need to be able to read the code to understand the test cases.

Load testing is even more fun now. XLT can send performance metrics to Graphite, a data collection and graphing tool. From now on you can thus watch the most important performance results in real time while a load test is still running.

We also did some fine-tuning of our reports. Most notably, the runtime charts now visualize the distribution of values and show how this distribution varies over time.

Users of Amazon’s EC2 cloud will be happy to hear that the ec2_admin tool can now also be used in scripted processes, making it possible to fully automate the starting and stopping of Amazon machines.

The scripting capabilities in Script Developer and the XLT framework have been extended. There are new commands to operate an application’s context menu. Package-level test data, a feature unique to XLT, which was previously accessible to test cases only, can now be used from script modules as well.

As usual for a major release, the technology stack the XLT framework is built upon has been updated and now provides the latest WebDriver support. Last but not least, the XLT Jenkins plug-in can now automatically start and stop agent machines in Amazon’s compute cloud.

As always, this update is free of charge for every user. You can use XLT freely for your daily automation work, regression testing, and performance validation. If you are an eligible Open Source project, you can get a full XLT license for your load and performance testing needs for free.

Download it now!

Category: Software, XLT | Comments (0)

We Take Web Privacy Seriously

Tuesday, 17. February 2015 22:58

Internet SurveillanceWhile privacy laws are still a work in progress, the EU and Germany have made several statements about what is right or wrong. But the current state of regulation is unclear about what actually is permitted or has to be provided.

Xceptance takes the privacy of our users seriously. We decided to implement tracking and privacy for our website differently than what is usually seen on the market: we provide users with more choices, and we’re stricter about respecting your decisions, yet without pestering you with lots of button clicking.

Our Approach in a Nutshell

There are no clear rules in Germany and across Europe that define if the user has to actively agree to cookies (opt-in) or if he or she has the choice to decline cookie usage (opt-out). Cookies are just one issue of course, because analytics works even without cookies.

On we decided to use an intermediate approach that does not require actions when you agree, but still gives you the time to make a decision.

Visiting our page initially doesn’t include any analytics. You have the possibility to choose to be part of tracking or not. We offer an opt-out option to the user but also ensure we don’t track anything before you make a decision.

Everyone is Welcome

We’ve seen several versions of tracking agreements on websites. A common approach is to inform the user with a pop-up, overlay, or info box that the site uses cookies and tracking solutions. If the visitors don’t agree with this they are free to leave. In other words, accessing the website is only possible if you agree to be tracked. Additionally your first pageview was already tracked in most cases, so you do not have really a choice after all.

We decided to let people visit our site even if they don’t want to be tracked. If visitors don’t agree to be part of our analytics we do not gather any data, but they can still browse and enjoy our entire site. […]

Category: Open Source, Security and Privacy, Xceptance | Comments (0) Goes Open Source

Monday, 9. February 2015 18:33

GitHubLike most companies we use and benefit from open source projects from all around the world. To actively participate and give something back, we started our own open source project that others can hopefully benefit from as well: Our website has been open sourced.

In building our new website, we faced different requirements and challenges, like easy to maintain pages, mostly static content, a modern design, multiple languages, multiple domains, and so on.

We decided to build a website which is statically generated, so we can avoid annoying software updates and security issues, get the fastest delivery possible, and have everything versioned in Git. The best solution for static content websites we found back then was Jekyll. There are so many solutions out now, but Jekyll is still the most popular.

Jekyll is a great tool, but it has its limitations, such as language-specific content and multi-domain usage. So you will find a couple of plugins we used in the code base, as well as some code in the templates to achieve a great menu.

For a clean, modern design we used other projects such as Bootstrap, Less, and FontAwesome.

On GitHub you can see the result of our work. This is a version that only deviates a little from our actual website. We watermarked all images and blurred the employee photos to make the copyright handling easier.

All the code is licensed under the MIT license. All content and the design is licensed under Creative Commons by Attribution-ShareAlike 4.0.

We invite you to learn from it, contribute to it, or simply use it as starting point for your own projects. But please obey the license.

Category: Open Source, Software, Xceptance | Comments (0)

Inaugural XLT Summit in Jena

Monday, 2. February 2015 16:45

Presentation XLT Summit 2015On January 19-20, 2015, Xceptance held its first XLT Summit. About thirty representatives of well-known companies from a variety of fields, including finance, retail and eCommerce, all of whom use XLT to help them in developing web-based software applications, accepted our invitation to meet at Xceptance’s headquarters in Jena, Germany. The goal of the conference was to give XLT users the opportunity to discuss their experiences using our tool both with each other and with the internal XLT development team.

To kick things off, on January 19 we enjoyed a casual dinner in a local restaurant. The next day there was a varied program consisting of presentations contributed both by Xceptance colleagues and external users. Our customers reported on their practical experience using XLT, and had a chance to discuss their results with the other users. The many different applications of XLT–from load testing to test automation to application monitoring–were thus illustrated in light of actual day-to-day practice. A selection of these presentations can be found here:, and parts of the presentation provided by our guests from OTTO.DE can be read here:

In addition, the XLT development team presented the newest XLT features. Our guests had the opportunity to speak directly with the developers about their specific feature requests, some of which we were able to immediately agree to implement.

In short, the conference was a great success. In response to the overwhelmingly positive feedback from the participants, the XLT Summit will now be a regular annual event.

Last but not least, we would like to thank all the attendees, without whose engagement, interest and suggestions the event would not have been possible!

Category: Xceptance, XLT | Comments (0)

Should Load Tests Validate Functionality?

Friday, 30. January 2015 10:56

My answer to this question is a very strong “yes”. You might want to limit yourself a little in the overall validation, but checking response codes only is a strong fail in my opinion. Additionally, just checking the result by checking a single phrase or word is not enough.

Reasons and Examples

  • Modern web implementations often incorrectly return application status pages with response code 200.
  • How do you ensure that you got the entire page back and not only the first 75%?
  • Imagine an e-commerce search that breaks under load and instead of saying “I found 200 matches”, it returns a page saying “no matches found, did you mean …”. The latter is still a valid page but your load test will not discover the flaw.
  • If you implement real page flows, meaning you don’t have hard-coded URLs that you fire against the server and instead take the data from the page and apply randomness to get a broad sample, you might find pages that are incorrect or inconsistent, but which are hidden outside normal functional-testing click paths.
  • How can you distinguish a “login succeeded” page from a “login failed” page when you do not validate for content? Both are valid scenarios and “login failed” might not return an HTTP code different from 200. Even when it is designed to return “get lost”, aka HTTP code 401, it might break under load and incorrectly return 200.
  • Always assume the worst case: always assume that the application will behave differently under load. Not only timing-wise but also behaviorally. This is one reason for running load and performance tests.

A Tale from the Past

In my life before Xceptance, a colleague was testing something for a new web component. He said that the new implementation was better, because it got faster when he applied more load.

What? Well, he did not check the content of the returned page. Under higher load, he got error pages back and incorrectly assumed great performance instead of a broken application.

How We Handle Validation

Because I am with a web testing company (Xceptance) specialized in load and performance testing, here is what we generally do for all our load and performance testing:

  • Validate the response code
  • Validate the basic structure of the page: check that the main components are there, such as header, footer, search box, mini cart, user account link, etc.
  • Validate the completeness of the page: in other words, the closing HTML tag should be there or the JSON or XML should be complete (hopefully also valid)
  • Validate the most important points of the current activity, e.g:
    • if you search for an existing product, you have to find it
    • if you expect to get no search results, check that you get the correct message
    • if you add something to the cart, validate that the quantity changes correctly
    • if you check out, validate the order number format (under perfect conditions even the uniqueness or at least the change between checkouts); validate that all price information is a valid format (not necessarily calculated correctly, although that could be required under certain test conditions); check that the (random) user name used is applied during that visit (for billing, shipping, order, account, etc.)
    • if you log on, verify that from then on, you always see the name of the user on the page (the famous “Welcome Mr. Foo” or something like that)
    • verify that you do not see a “Welcome Mr. Foo” message before logging in, that you do not have a cart when you first arrive, and so on

We have so often found problems that are load related or often also functionality related that every single validation step really pays off again and again.
Only when applications return bad stuff and the implementation crew does not want to change that and cannot be convinced or forced to clean up the mess, do we apply exceptions and reduce the validation.

These are just examples, of course.


You should extensively validate during load and performance testing. Always assume both that the application will break under load, and also that the failure is mostly not reflected in a status code 500. Status codes 404 and 500 are for the beginner; the real professionals find problems beyond that.

We usually find functional problems during our load and performance testing that would have stayed undetected without proper validation. Of course, don’t forget the classical erroneous performance behavior such as inconsistent response times, spikes, and everything else imaginable.

Or in other words: You do load and performance testing to find the unusual and unexpected. This is often not reflected in increased response times and error codes.

Of course there are tests when you really are hunting for something specific, and when you can easily skip most of the validations, but take it from us: never strip them out completely.

This article was created as response to a question on StackExchange: Should load test validate functionality?

Category: Performance, Testing, Testing Culture, Xceptance | Comments (0)

It is just HTTPS now

Tuesday, 2. December 2014 22:46

The Blog is HTTPS onlyThe recent events of security problems with WordPress motivated us to speed up the effort to move our blog into the HTTPS only mode. This does not fix WordPress security problems, but makes it easier to spot possible problems later and might protect us against the negative outcome from time to time.

So we set the Strict-Transport-Security header to tell the browser that HTTP is off-limits for this site once and for all. We also turned all links to our blog from our website and internally in the blog including resources into HTTPS links.

Also we set .htaccess redirect rules to force all users to use HTTPS and only hit the domain that has the proper certificate.

# Send .de to .com
RewriteCond %{HTTP_HOST} xceptance\.de$ [NC]
RewriteRule ^(.*)$$1 [L,R=301]

# Force https
RewriteCond %{HTTPS} !=on
RewriteRule ^ https://%{HTTP_HOST}%{REQUEST_URI} [L,R=301]

# Set the correct header to tell the browser secure only
Header set Strict-Transport-Security "max-age=31536000" env=HTTPS

The only thing open is to get our hoster convinced to properly support TLS 1.2.

Category: Security, Xceptance | Comments (0)

Our New Website is Live!

Monday, 11. August 2014 13:59

New Xceptance WebsiteWe proudly announce that Xceptance has a new website. Our 10th anniversary made us look back on where we are coming from, what we have been doing and what experiences we gained throughout the past ten years. It was time to have a new web presence reflect all that!

We took advantage of Bootstrap, Less, Jekyll, Git, Font Awesome, and Jenkins to create a website that primarily wants to help our visitors quickly learn about Xceptance, our services and our product. We wanted it to be modern but plain so that we can communicate what we do in the most comprehensive and user-friendly way possible. No boasting, no bragging, and just a little bit about ourselves. To have it all look nice and work smoothly for the mobile users as well, we used Bootstrap.

Since we’re always looking for new people that want to join us, we added a comprehensive jobs page which lists current open positions in both our offices, Cambridge, MA, USA and Jena, Germany.

Go check it out for yourselves! As always, we appreciate any kind of feedback!

Category: Misc, Software, Xceptance | Comments (0)

An Xceptional Storefront!

Tuesday, 10. June 2014 19:08

Xceptance Logo at the B59The B59, that’s the 16-story office building that accommodates the headquarters of Xceptance Germany, now has a new addition to its front. The red and black neon writing with Xceptance’s name and signature X was put up a week ago and thereby joins the logotypes of the Postbank and Allianz insurance.

The workers who installed the sign definitely had to be free from giddiness, since the sign overlooks the city of Jena from a height of approximately 98 ft. (30m). Several curious bystanders watched the installation process and were able to witness how, piece by piece, the name emerged on the wall.

So now, three years after Xceptance moved to the B59 office space, our flag is finally set. We think it’s been worth the wait!


Category: Xceptance | Comments (0)