Archive for category 'Software Development'

Use XLT with Sauce Labs and BrowserStack

Wednesday, 12. March 2014 18:13

Sauce Labs and BrowserStack – What Are They and Why Use Them?

Sauce Labs and BrowserStack allow you to run automated test cases on different browsers and operating systems. Both provide more than 200 mobile and desktop browsers on different operating systems. The benefit? You can focus on coding instead of having to maintain different devices. You can easily run your test cases written on iOS on an Internet Explorer without actually buying a Windows device; and last not least, you don’t need to worry about drivers or maintenance.

By the way, Internet Explorer even seems to run faster at Sauce Labs than on a desktop machine. Also note that Sauce Labs supports Maven builds.

What You Need

Not much, really. Only a Sauce Lab or BrowserStack account, a Java IDE (e.g., Eclipse), and XLT 4.3.2.

How to Do it

First, import your XLT test suite into the IDE (Eclipse). Then import all .jar files from the XLT lib directory into your project. In the XLT Script Developer options menu, check “Generate JUnit wrapper class for test cases”.

You then need to “Export…” a test case (select “XLT Scripting API” as API) to generate a new wrapper class so that you can place your new code for the WebDriver in it.

Add the statements below to the exported Java code. Here you can choose your browser (Android, Safari, Firefox, Chrome, Internet Explorer):

DesiredCapabilities capabilities = DesiredCapabilities.firefox();

Now set the desired version of the browser:

capabilities.setCapability("version", "19.0");

Set the operating system on which you want the test case to run:

capabilities.setCapability("platform", Platform.LINUX);

Give your test case a name so that you can refer to it later:

capabilities.setCapability("name", "XLT Firefox Test");

Now you can set up a new WebDriver. Replace “xxx” by your user name and “yyy” by your access key for Sauce Labs:

Webdriver driver = new RemoteWebDriver(
    new URL("http://xxx:yyy@ondemand.saucelabs.com:80/wd/hub"), capabilities);

With this piece of code you should be able to run your test cases on the Sauce Lab servers. This also works to for manually written classes. Just use the same WebDriver settings.

For BrowserStack, you can use the following commands:

DesiredCapabilities caps = DesiredCapabilities.firefox();
caps.setCapability("browser", "FireFox");
caps.setCapability("browser_version", "22.0");
caps.setCapability("os", "Windows");
caps.setCapability("os_version", "7");
caps.setCapability("browserstack.debug", "true");
driver = new RemoteWebDriver( 
    new URL("http://" + USERNAME + ":" + 
        AUTOMATE_KEY + "@hub.browserstack.com/wd/hub”, caps);

Some Final Notes

Selenium works best with Chrome and Firefox. Other browsers, like Internet Explorer or Safari, may fail on test cases that, in fact, do successfully run in Chrome or Firefox. Some websites have a different layout for mobile than for desktop. So you probably can’t use your default tests. If you want to write tests for mobile devices with XLT Script Developer, install user agent changer and change your user agent to iPhone 3.

Sauce Labs provides a detailed tutorial which describes all functions of the service, like parallel testing and other cool stuff. We definitely recommend you read it in case you have any questions.

Category: Automation, Java, Software Development, Testing, XLT | Comments (1) | Author:

Tutorial: Git – The Incomplete Introduction

Tuesday, 1. October 2013 18:29

Software Testing is part of software development. So you need a form of revision control for your source aka test code, and documents. You also need it to be able to review code, compare the history of code… or maybe simply to help others to master it.

We recently started our migration from Subversion to Git. Not because we have been unsatisfied with SVN, mostly because we want to use what our customers use. Additionally we want to profit from the different functionality Git offers, such as local commits and cheap branching.

But Git is different and just changing the tool does not change anything, it might even turns things worse. Because you cannot run Git like SVN. Well, you can, but that still requires you to know the basics of Git to understand what it will do to your work and how a typical workflow looks like. The commands are different too.

So we created this tutorial to get used to Git, understand, and learn it. It starts slow with the basics, explains some terms and concepts, and later picks up speed when it talks about rebasing and merging. Everything is explained with commands you can execute quickly in your own test project. Command line rulez!

Not everything is explained of course. Basically what you don’t need to know to master Git has been intentionally left out. Google it or immerse in the official documentation, if you need to know.

So enjoy the tutorial, use it, copy it, share it. The entire presentation is licensed under Creative Commons by Attribution Share-Alike. Our way of saying “Thank you” to the Git open source community.

You can download the Libreoffice document if you want to modify it, take things out, or simply go with a different style: GIT-the-incomplete-overview.odp. Fonts are embedded, so that hopefully works ok.

If you prefer the easy way aka the PDF, you can download it right here as well: GIT-the-incomplete-overview.pdf.

Category: Interesting Reading, Open Source, Software Development | Comments (0) | Author:

HPQC and XLT – Integration Example

Friday, 14. December 2012 15:23

You have to work with HP Quality Center (HPQC), but you don’t want to execute all the test cases manually. You automated some tests using XLT Script Developer and like the outcome. You want to use the Script developer much more but you face one last problem: You still have to enter the test results manually into HPQC. This renders some of the test automation advantages useless.

The following example can mitigate that problem. HPQC offers an API called Quality Center Open Test Architecture API (OTA API).
Using this interface, you can set test results automatically.

This is the way to do it: Run the test cases as a Script Developer Batch Test, export the results, parse them and submit them to HPQC. Completely without touching the HPQC web interface at all.

The following Python command line script does the parsing and submitting:

# -*- coding: utf-8 -*-
import win32com
from win32com.client import Dispatch
from datetime import datetime
from bs4 import BeautifulSoup
import re

# global vars
result_file_name = 'X:\\path\\to\\xlt-result.html'

# the regular expression for the id that can be found in test case names, 
# the id has to be enclosed in brackets so that the regex can contain things 
# that are not part of the id
# examples: XLT id: r'(XLT[0-9]{4})[^0-9]', whole test case name: r'(^.*$)', r'(T[0-9]{2})[^0-9]'
tc_id_regex = re.compile(r'(T[0-9]{2})[^0-9]')

# value for strftime to generate the name for the run
run_name_pattern = "Run_%Y-%m-%d-%H-%M-%S"

# hpqc credentials
qcServer = "https://hpqc.server"
qcUser = "username"
qcPassword = "password"
qcDomain = "domain"
qcProject = "project_name"
qcTestCasePath = "Path\to\test\case"
qcTestCaseNode = "test case node name"

# test case status names in hpqc
hpqc_status_pass = 'Passed'
hpqc_status_fail = 'Failed'
hpqc_status_no_run = 'No Run'
hpqc_status_not_completed = 'Not Completed'
hpqc_status_na = 'N/A'

# status names in the XLT result file
xlt_result_pass = 'OK'
xlt_result_fail = 'FAILED'


# parse the result file
# the processed html structure looks like this:
#<html>
#<body>
#<table></table>
#<h2>Details</h2>
#<table>
    #<thead></thead>
    #<tbody>
        #<tr>
            #<td>Tests.XLT0001_First_Test_Case_Name</td>
            #<td class="success">OK</td>
            #<td class="number">0.355</td>
            #<td></td>
        #</tr>
        #<tr>
            #<td>Tests.XLT0002_Second_Test_Case_Name</td>
            #<td class="error">FAILED</td>
            #<td class="number">2.255</td>
            #<td>Assertion &apos;assertElementPresent(provoke error)&apos; at line 1 has failed. =&gt;</td>
        #</tr>
    #</tbody>
#</table>
#</body>
#</html>
result = BeautifulSoup(open(result_file_name))
resultArr = {}
for tr in result.select("table")[1].select("tbody tr"):
    tc_name = tr.select("td")[0].string[6:]
    tc_res = tr.select("td")[1].string
    tc_id_mo = tc_id_regex.search(tc_name)
    
    if tc_id_mo:
        tc_id = tc_id_mo.group(1)
        if tc_id in resultArr:
            print("ERROR: there has already been a result for tc_id " + tc_id + " tc_name " + tc_name + " tc_res: " + tc_res)
        else:
            if tc_res == xlt_result_pass:
                resultArr[tc_id] = hpqc_status_pass
            else:
                resultArr[tc_id] = hpqc_status_fail
            print(tc_id + " - " + tc_name + " tc_res: " + tc_res)
    else:
        print("ERROR: no valid id found in tc_name " + tc_name + " tc_res: " + tc_res)

# login to hpqc
td = win32com.client.Dispatch("TDApiOle80.TDConnection.1")
td.InitConnectionEx(qcServer)
td.Login(qcUser,qcPassword)
td.Connect(qcDomain,qcProject)
if td.Connected:
    print("Logged in to " + qcProject)
else:
    print("ERROR: Connect failed to " + qcProject)

# navigate through the test set tree
tsTreeMgr = td.TestSetTreeManager
tsFolder = tsTreeMgr.NodeByPath(qcTestCasePath)
tsList = tsFolder.FindTestSets(qcTestCaseNode)

for tsItem in tsList:
    tsTestList = tsItem.TSTestFactory.NewList("")
    
    # loop through all test cases in this list
    for tsTest in tsTestList:
        print("Test case name: " + tsTest.TestName)
        runFactory = tsTest.RunFactory
        run_name = datetime.today().strftime(run_name_pattern)
        run = runFactory.AddItem(run_name)
        run.CopyDesignSteps() # copies the test steps into this run
        
        # extract the test case id from the test name
        tc_id_mo = tc_id_regex.search(tsTest.TestName)
        if tc_id_mo:
            tc_id = tc_id_mo.group(1)
        else:
            tc_id = ''
            print("ERROR: no valid id found in name " + tsTest.TestName)
        
        # get the results from resultArr and set the test result accordingly (identical for test case and all test steps)
        if tc_id in resultArr:
            status = resultArr[tc_id]
        else:
            status = hpqc_status_no_run
        
        run.Status = status
        
        # set the status for each test step
        tsStepList = run.StepFactory.NewList("")
        for tsStep in tsStepList:
            print("    Test step name: " + tsStep.Name)
            tsStep.Status = status
            tsStep.Post()
        
        # submit this run to hpqc
        run.Post()
        print("Run added: status: " + status + " run_name: " + run_name)

# close the connection to hpqc
if td.Connected:
    td.Disconnect
    td.Logout
    print("Logged out from " + qcProject)

td = None

The processed part of the Script Developer result file looks like this:

<html>
<body>
<table></table>
<h2>Details</h2>
<table>
    <thead></thead>
    <tbody>
        <tr>
            <td>Tests.XLT0001_First_Test_Case_Name</td>
            <td class="success">OK</td>
            <td class="number">0.355</td>
            <td></td>
        </tr>
        <tr>
            <td>Tests.XLT0002_Second_Test_Case_Name</td>
            <td class="error">FAILED</td>
            <td class="number">2.255</td>
            <td>Assertion &apos;assertElementPresent(provoke error)&apos; at line 1 has failed.</td>
        </tr>
    </tbody>
</table>
</body>
</html>

It needs to run in a Windows environment because it is importing win32com. You have to install:

  • Python (version 3 is needed)
  • pywin32 (Python for Windows extensions)
  • Beautiful Soup 4 (a HTML parser for Python) (after extracting the archive, cd into it and run ‘setup.py install’ to install it)

Now adjust the variables in the script and you are ready to go. The following screenshot depicts the parts of HPQC that the script is using.

In the first td there is the name of the test case (always starting with “Tests.”) and the second td contains the status of the test case.

Disclaimer: This is an example only. Use at your own risk.

The script has been inspired by Generating Custom Reports with Quality Center OTA using Python.

Category: Automation, Software Development, XLT | Comments (0) | Author:

Test Automation Community on Google+

Tuesday, 11. December 2012 14:29

When Google+ brought the community feature online, we immediately knew, that could be it to get testers together and discuss test automation, learn from each other, and share knowledge. Google+ gathered a more technical crowd compared to Facebook and so we will give it a try.

Feel invited and we hope to see you soon: Test Automation Community at Google+.

Category: Automation, Software Development, Testing | Comments (0) | Author:

Spurious wakeup – the rare event

Friday, 6. May 2011 2:12

After hunting for quite some for a strange application behavior, I finally found the reason.

The Problem

The Java application was behaving strangely in 4 out of 10 runs. It did not process all data available and assumed that the data input already ended. The application features several producer-consumer patterns, where one thread offers preprocessed data to the next one, passing it into a buffer where the next thread reads it from.

The consumer or producer fall into a wait state in case no data is available or the buffer is full. In case of a state change, the active threads notifies all waiting threads about the new data or the fact that all data is consumed.

On 2-core and 8-core machines, the application was running fine but when we moved it to 24-cores, it suddenly started to act in an unpredictable manner.

The Cause

After a lot of debugging I found out that threads wake up without having been notified by their partner thread. In this case the consumer was woken up despite the fact that data was unavailable aka the producer has not delivered and therefore not notified anyone. But the consumer was awake…

The debugging nightmare finally revealed a rare behavior of any POSIX based application. This is the quote from the JDK6 doc:

A thread can also wake up without being notified, interrupted, or timing out, a so-called spurious wakeup. While this will rarely occur in practice, applications must guard against it by testing for the condition that should have caused the thread to be awakened, and continuing to wait if the condition is not satisfied. In other words, waits should always occur in loops.

The Verdict

So never trust your notification chains. Threads might wake up even though nobody directly notified it. Additionally you should never exclude the possibility that the move from a small box to a big box does not influence the application behavior. More cores mean more trouble.

Category: Java, Software Development | Comments (0) | Author:

The Argument about the Curly Brackets

Thursday, 3. March 2011 8:00

When you talk about code styleguides, you often talk about basic formatting. This means you probably already fought the holy war over the curly brackets {} and where to put them.

Of course, the next line is the only right place. A curly bracket is a hermit and does not like to be put next to any other character…  :)

What is your opinion?

Cartoon courtesy of Geek and Poke under CC-BY-ND-2.0

Category: Java, Software Development | Comments (0) | Author:

One digit version numbers only, please!

Sunday, 16. May 2010 17:02

Just read about a nice small software problem at Opera. Their latest browser is version 10, but they couldn’t continue to use the version number in the user agent string, because some web sites try to identify the agent version and fail with 2 digit version numbers. Seems to be similar to the famous Y2K problem, but now it is a BVN problem – a browser version number problem.

“…It appears that a considerable amount of browser sniffing scripts are not quite ready for this change to double digits, as they detect only the first digit of the user agent string: in such a scenario, Opera 10 is interpreted as Opera 1. This results in sites mistakenly identifying Opera 10 as an unsupported browser, thereby breaking server, as well as client-side scripts…”

Read more at Dev.Opera.

Category: Bugs in the Wild, Software Development | Comments (0) | Author:

Some nice reading about HBase

Tuesday, 16. March 2010 21:35

HBase LogoIf you want to stay in touch with cutting-edge technology in terms of scalability of databases, high traffic sites, and large storage volumes, you should read these two articles on the new hstack.org blog.

Cosmin Lehene wrote two excellent articles on Adobe’s experiences with HBase: Why we’re using HBase: Part 1 and Why we’re using HBase: Part 2. Adobe needed a generic, real-time, structured data storage and processing system that could handle any data volume, with access times under 50ms, with no downtime and no data loss. The article goes into great detail about their experiences with HBase and their evaluation process, providing a “well reasoned impartial use case from a commercial user”. It talks about failure handling, availability, write performance, read performance, random reads, sequential scans, and consistency.

(via High Scalability)

Category: Java, Software Development | Comments (0) | Author:

5 Gründe, warum Webdesigner coden können sollten

Friday, 19. February 2010 15:33

Mike Kus hat einen kurzen Artikel geschrieben, warum Webdesigner unbedingt coden können sollten. Gerade aus Sicht von QA kann ich ihm nur zustimmen.

Via T3N

Category: Software Development | Comments (0) | Author:

Eclipse und Ubuntu 9.10

Tuesday, 5. January 2010 12:39

Wer seine eigene Eclipse-Installation unter Ubuntu 9.10 betreibt bzw. ältere Versionen von Eclipse im Einsatz hat, der kennt evenutell Probleme mit Buttons. Diese lassen sich oft mit der Maus nicht klicken oder anwählen. Nur mit Hife der Tastatur kann man noch etwas ausrichten.

Das Ganze ist ein bekanntes Problem seit Ubuntu 9.10 und sollte mit Eclipse 3.5.1 weg sein. Wenn das aber keine Lösung ist, dann muss man seine Umgebung mit diesem Parameter anpassen:

GDK_NATIVE_WINDOWS=true

Danach funktioniert es wieder. Die Lösung habe ich hier gefunden: Widdix – Eclipse unter Ubuntu 9.10 und hier gibt es mehr dazu in Englisch.

Category: Bugs in the Wild, Linux, Software Development, XLT | Comments (0) | Author: