Motivation

Did you ever have to create multiple versions of your test cases to accommodate small differences of your test objects? Looking for a trade-off between good testing practice and minimizing project complexity. The following blog post reflects on this challenge and introduces you to a potential solution: Conditional Expressions.

Introduction

Xceptance introduced its test automation and load testing tool XLT 4.6 in February 2016 and with it we brought you conditionals. Today we want to shed some light on this new feature, the discussion that came along with it and why we finally decided to introduce it. This blog post will equip you with everything required to employ conditional expressions in your test scripts.

In computer programming, a condition or conditional expression performs an action depending on whether a given statement (the condition) evaluates to true or false. The programmer has the possibility to execute a part of the program only if certain circumstances are met. Now don’t worry, you do not need to become a full-fledged programmer to create your test cases with XLT Script Developer. But you should not skip on the possibilities this new feature is offering.

The Challenge

In testing typically you want your flow of execution to be linear, deterministic and transparent. The individual execution steps of your test case should be well-defined and yield the same results in a constant environment. If one execution step fails – e.g. an assertion does not check out – the whole test case always breaks and evaluates to failed. Run, rinse and repeat.

On the contrary often enough your real world test cases need to cater various scenarios. Think multi-region support of your page for example. Area specific content and functionality can quickly bring you into a catch-22 situation. Follow good practice in test case design, but deal with complexity and organizational nightmares in your test suite. Tiny differences in your test objects force you to keep multiple versions of your (already lengthy) test cases. Farewell maintainability!

The Solution

Weighting good practice in test case creation with the complexity of today’s quality assurance projects, we have decided to include conditional execution into our newest version of XLT. Being pragmatic, we want to offer you the advantages of an IF condition, but hope you will apply this potential when appropriate and required.

Starting with version 4.6.0, Script Developer supports conditional module calls. In short this means separating the variable parts of your test cases into individual modules and executing these modules when needed. You do not need to duplicate the whole test case anymore just because this little difference deep inside your test script. Now you can conveniently skip execution of parts of your test cases. No need to copy your test cases and maintain multiple versions of them any longer. Factor out the execution steps that are specific and conditionally call them when needed.

How-To Use

By specifying a condition to your module, you can simulate a simple IF condition. Adding one or more conditional module calls gives you the possibility of an IF/ELSE or even a SWITCH statement. In any case, you need to separate the unique functionality into a module and in the next step call it in your (more general) test case script. The module call’s editor window enables you to assign a condition which needs to be met for the module to be executed.

The condition itself is an arbitrary JavaScript expression. If the expression evaluates to true, your module is executed. You can reference test data or runtime variables via ${…} or @{…} as usual. Script Developer commands are not permitted in the expression. If you have ever used the assertEval or storeEval function before, you will feel at home with conditional execution expressions.

Use the checkbox Temporarily Disabled to put the conditional evaluation on hold. This will call your module without any checks and might come in handy when debugging your test case. Click the red minus icon to completely remove the condition from your module call. Of cause this will leave the module itself unaltered.

Conclusion

Even though incorporating many conditional calls with (arbitrary complex) expressions might result in a hard to grasp and maintain test execution flow, we think the risks do not outweigh the benefits. We have certainly put it to good use in our own projects and are convinced you will find the right measure as well.

Of course conditional execution is not the only improvement that comes with XLT 4.6. There are many other new and helpful features to discover. We will highlight some of them in upcoming blog posts and urge you to unconditionally try them out.