This is Just a Data Issue


Image © Juja Schneider

Many projects have their closed or “won’t fix” bugs commented with remarks such as:

  • “Content-related issue”,
  • “Won’t happen on live system”, or
  • “This is just a data issue”.

Both the development of new features and data maintenance may be accomplished in different environments. For testers, it’s thus difficult to decide on the cause for a certain issue: is a required text missing because pages aren’t implemented the right way or is it simply not there because the corresponding product data aren’t available in this particular test system? Very often testers work on development systems without having access to latest data. The following is a typical scenario:

  1. Testers report a bug.
  2. The development resolves this bug as content-related and won’t fix.
  3. Testers report further bugs of that sort.
  4. The development gets irritated: “We’ve told them that these are content issues!”
  5. The relationship between testers and developers might really go downhill from there, seriously threatening the project’s success: the credibility of the test team is damaged, developers  don’t take reported bugs seriously anymore, testers get overly cautious and stop reporting content-related problems…

Inevitably, potential bugs don’t get reported and reported bugs never get retested before the project goes live. This is even more true in poorly managed projects because no one really feels responsible to solve this problem.

Here are a couple of guidelines we advise you to keep in mind if the descriptions above somehow sound familiar to you:

For testers:

  • A bug is a bug is a bug. When you see it, report it. Let your test or project manager know that you’ll report these kind of issues even if they are potentially content-related. Only this way, you can be sure not to miss anything important.
  • Draw conclusions from previously reported issues and comments.
  • Try to get a feeling for your project and make sure not to address problems you’re actually causing yourself.

For project and test managers:

  • Don’t dismiss these issues as minor troubles you don’t want to bother with. They can easily turn into big ones at any stage of your project. Before go-live, you want everything to be fixed, not only functional problems.
  • Problems that appear to be data-related at first sight might in fact be functional. Handling them within the regular test process facilitates their correct fixing.
  • Don’t hesitate to tell your testers if they are causing the problem themselves. Granted, this might be an uncomfortable situation at first, but it will definitely trigger an Aha! effect everyone in the team can benefit from.
  • Give content-related issues a low severity. This way, you can focus on troubles that are more pressing and, at the same time, ensure that nothing gets lost and QA will still retest everything.
  • Inform your test team about the system’s architecture and how content and date influence its behavior.
  • Also let your test team know when certain data aren’t available yet and when you don’t want the resulting problems reported. Testing is way easier when having such information in advance.