Image © Juja Schneider
Early in 2013, we compiled a personal list of relevant software testing trends. Having received great feedback on this, we would like to add a couple of other interesting aspects today.
We believe that the domain of software testing is not and should not be subject to short-lived trends and fashions since new approaches always need to prove worthwhile first before becoming custom. So when we’re talking about trends here, we refer to issues that didn’t just come up recently but that have been around and widely discussed for a while now. The subjects we’re going to introduce in the following are subjects we feel our customers care about and that are of relevance in our own daily work. Please keep in mind that the list below is thus once again a very personal one, deriving from our experience in web and e-commerce as well as small and midsize test projects.
The cloud enhances testing in that it offers new tools that help broaden the view, cover more, set up faster, and deal with data as well as traffic amounts never seen before. We have now access to cheap hardware, we can set up test environments quickly, hibernate them, we can emulate a larger number of configurations, easily clone setups, or add machines.
Cloud testing is especially suited for scenarios such as:
- load testing where the load drivers are in the cloud and dynamically scaled
- load testing where your system under test is in the cloud probably because the live system is or will be cloud based as well
- functional testing where you have plenty of systems available to try configurations or give every tester their own hardware to avoid the famous all-on-one-system test problem
- several system running automation around the clock, just to test stability, for instance
- preserving copies of data or installations because it might be necessary to compare to older versions from time to time
- having different operation systems available and switching quickly between them
Of course, the cloud has also disadvantages. It can turn into a mess because more systems mean more management and more management can mean more errors, more mistakes, and more resources just to keep it running.
The end user is the one that matters. If you create an application for a big number of end users – why let it test by a single person or a small test office? Crowd testing delivers results from the people who are actually going to work with your application.
You are creating a mobile application and can’t cope with the diversity of devices and operating systems? You feel that problems don’t occur in simulators but in the real world? Crowd testing can be the perfect solution for such problems.
Keep in mind that crowd testing is not the answer to everything though. Secretive and sensitive stuff won’t work that way. Additionally, you will have to find a good way to get valuable feedback from strangers. So crowd testing might only be a small component of your overall testing strategy.
Web Security Testing
Hijacked accounts, hacked applications, stolen user data: it’s in the headlines again and again. Customers are scared, sensitive data get in the hands of competitors and prove a powerful tool for criminals – you really want to avoid that.
Security testing will become increasingly important in the data-driven world. Information and data are money you don’t want to lose. Thus, dealing with authentication, session management cross site scripting, etc. should be part of your priorities if you’re providing a web application.
Keep in mind that security is something that cannot be tested into an application. Security has to be a design decision.
Data-driven testing is all about variation of data. Get rid off a small set of test data, make it wide, make it random, make it flexible.
As for test automation, data-driven testing has already been a relevant topic. Keeping your test data separated from the actual test scripts leads to more flexibility and easier maintenance of the scripts. Data can be put into databases, spreadsheets, data pools, etc.
Have you ever had to update all of your test scripts just for testing with a different set of data? You might be interested in data-driven testing then.
Reliability, Stability, HA… Testing
In a world where everything has to be available 24/7, where your customers are anywhere and not just in one country, keeping your applications running reliably is a key to success. Reliability covers a lot of topics here: stability, performance, high availability, backups… you name it.
So testing is not just about users facing functionality, it’s about the system behind it as well. You don’t take it for granted that it will just run. What about a failing database? What about a failing application server? What about full disks? What about failing hardware of any kind? What about faulty software that requires a rollback of some sort?
You might think this is all DevOps… think again: if you haven’t tested it, it probably won’t work. Question everything, but ask the right questions. Additionally, the lines between departments are fading and so you have to know how the deployment works, what the hardware setup is, and so on.
Of course, this can be easily subsumed under reliability testing but it is often viewed separately because it’s a difficult and error-prone area. How do you deploy new software while keeping downtimes minimal? How do you deal with deployment failures? How can you quickly rollback to older versions?
This all has to be answered and tested. So once again the lines between the test department, engineering, and operations or DevOps are fading. This might be the reason why big players, such as Google, did away with the classical three tier approach in software development: development, test, operations. It just doesn’t work anymore.
Feel free to add additional ideas!