Accessibility, Automation, Java, Open Source

Automated Accessibility Testing with Google Lighthouse and Neodymium

TL;DR: Accessibility is a crucial aspect of web development that ensures everyone, regardless of their abilities, can access and use your website. Web Content Accessibility Guidelines (WCAG) provide a standard for web accessibility, and adhering to them is not only ethically important but also increasingly becoming a legal requirement.

One valuable tool for checking WCAG compliance is Google Lighthouse. It can help identify accessibility issues during development. And when integrated into your test automation project with tools like Neodymium, it can significantly streamline your accessibility testing and monitoring process.

What is WCAG and Why is it Important?

WCAG, or Web Content Accessibility Guidelines, are a set of standards developed to make web content more accessible to people with disabilities. These guidelines cover a wide range of disabilities, including visual, auditory, motor, and cognitive impairments. By following WCAG, you ensure that your website is usable by a larger audience. Approximately 15% of the world’s population has some form of disability.

WCAG compliance is also gaining traction legally, with governments in the US and EU enforcing accessibility standards. Non-compliance can lead to lawsuits and of course to less revenue and visitors.

The Role of Test Automation

Automating WCAG checks using tools like Lighthouse can help find and fix accessibility flaws early in the development cycle, saving time and resources. It also reduces the burden on manual testers by automatically generating accessibility reports.

Most importantly, test automation maintains the achieved level of accessibility of the application through regression testing.

If you have an existing or planned test automation project, it is crucial to incorporate WCAG compliance into your testing strategy. This is why we have integrated Lighthouse into Neodymium, allowing you to easily extend your test cases with accessibility checks.

How to Use Lighthouse with Neodymium

Here’s a simplified example of how to integrate Lighthouse with Neodymium:

  • Prerequisites: Ensure you have Neodymium set up and Lighthouse CLI installed. Lighthouse only works with Chrome or Chromium-based browsers.
  • Triggering Lighthouse: Neodymium provides the LighthouseUtils.createLightHouseReport(name) method. Call this method at the desired point in your test to generate a Lighthouse report.
  • Analyzing Results: After the test run, the Lighthouse report will be attached to the final test report (e.g., Allure report).
Showing how a Lighthouse report in Neodymium looks like. It shows some test case step descriptions and result data such as performance 90%, Accessability 85% and more
Lighthouse report in Neodymium’s Allure report

Important: Creating a report requires Lighthouse to refresh the current web page. That approach doesn’t work with some types of pages or states.

Lighthouse Validation

Neodymium allows you to validate Lighthouse scores and specific audits programmatically.

Scores

Lighthouse provides scores for Performance, Accessibility, Best Practices, and SEO. You can set thresholds in Neodymium to automatically fail tests if these scores fall below a certain value.

To properly verify those scores, we implemented configuration properties in the neodymium.properties file.

Audits

You can also validate specific Lighthouse audits. For example:

This property is used to identify which audit IDs should be evaluated. The example checks for errors in the Lighthouse audits for aria-roles and aria-text. If either or both of these audits find an error, an exception will be thrown. For a complete list of existing audit IDs and their corresponding titles, please refer to our documentation.

Limitations of Automated Accessibility Testing

While automated tools like Lighthouse are incredibly helpful, they can’t catch all accessibility issues. Some aspects, like the meaningfulness of alt text or the logical flow of keyboard navigation, still require human judgment.

It’s important to note that it is possible to build an inaccessible website that still gets a high Lighthouse score. Automated tools provide a good starting point, but they should be complemented by manual testing and user testing with people with disabilities.

Conclusion

Integrating Google Lighthouse with Neodymium into your test automation project is a great way to improve your website’s accessibility. It helps catch issues early, automates report generation, and enables validation of accessibility scores and audits. While not a complete solution, it is a powerful tool in making the web more accessible to everyone.

More information on how to integrate Lighthouse in Neodymium are available at https://github.com/Xceptance/neodymium/wiki/Accessibility

Accessibility, Testing, Xceptance

WCAG Testing in E-Commerce: Compliance, Accessibility, and Business Impact

TL;DR: The EU’s upcoming legislation makes WCAG (Web Content Accessibility Guidelines) compliance non-negotiable. With 87 million EU citizens living with disabilities, accessibility isn’t just ethical—it’s smart business. Xceptance, as experts in testing, offers comprehensive WCAG testing to help businesses meet legal requirements while improving user experience and inclusivity. Compliance isn’t just about avoiding fines—it’s about embracing inclusivity, enhancing user trust, and unlocking new market potential.

The Growing Need

Digital accessibility is no longer a nice-to-have — it’s a fundamental necessity. The European Commission estimates that over 87 million people in the EU experience some form of disability – that’s nearly 20% of the population. In Germany alone, that number translates to roughly 7.8 million individuals with severe disabilities. These numbers highlight the importance of making websites, apps, and digital services accessible to all users. The European Accessibility Act (EAA), which will come into effect in June 2025, mandates that e-commerce businesses comply with WCAG 2.2 Level AA standards. This means that online stores, banking services, and digital platforms must remove barriers that hinder people with disabilities from using their services. Non-compliance doesn’t just mean legal repercussions — it risks alienating a significant customer base and damaging brand reputation.

Challenges in E-Commerce

E-commerce businesses face unique challenges in ensuring accessibility compliance:

  • Complex User Interfaces: Online shops often use dynamic elements, pop-ups, carousels, and multimedia content, which can be difficult for assistive technologies to interpret.
  • Checkout and Payment Issues: Forms, captchas, and payment gateways must be accessible to users relying on screen readers or keyboard navigation.
  • Mobile Responsiveness: Accessibility must be maintained across devices, including smartphones and tablets.
  • SEO and Usability Overlap: Many accessibility improvements, such as proper heading structures and alt text, also enhance SEO and user experience.

Without structured WCAG testing, businesses risk excluding a significant customer base and facing compliance issues.

Xceptance: Your Partner in WCAG Compliance

Xceptance specializes in comprehensive software testing, including in-depth WCAG compliance verification. We go beyond simply checking boxes; we empower you to create an inclusive and user-friendly experience.

  • Automated and Manual Testing: We use automated tools to identify accessibility gaps and complement this with manual testing by experts using screen readers and keyboard navigation.
  • Evaluation Against WCAG 2.2 Guidelines: Our tests cover perceivability, operability, understandability, and robustness, ensuring compliance with Level AA standards.
  • Real-User Simulations: We test e-commerce platforms with real users to gain real-world insights into accessibility challenges.
  • Detailed Reporting and Actionable Recommendations: We provide clear, structured reports outlining issues, their impact, and practical solutions.
  • Ongoing Compliance Support: Accessibility is an ongoing process. We help businesses continuously monitor and improve their accessibility standards.

Business Benefits Beyond Compliance

Accessibility isn’t just about checking a legal box—it’s a competitive advantage:

  • Expand Your Customer Base: An accessible site welcomes millions of potential customers.
  • SEO Boost: Many accessibility best practices align with improved search engine rankings.
  • Stronger Brand Trust: Inclusive brands build deeper connections and customer loyalty.
  • Future-Proof Your Business: Early compliance with WCAG standards means staying ahead of regulations and market shifts.

Don’t Wait – Embrace Accessibility Today

The 2025 WCAG compliance deadline is fast approaching. Xceptance provides the expertise and support you need to ensure your online store meets the legal requirements and delivers an exceptional experience for every user.

Ready to unlock the full potential of an accessible e-commerce platform? Contact Xceptance today for a consultation.

A visual call to action for web accessibility, featuring the text "Embrace Accessibility Today" in a dynamic and eye-catching design.
Automation, Open Source, Testing

Neodymium 5.1 – Get more done faster

TL;DR: Neodymium 5.1 has been released with a host of impressive new features. These include full-page screenshots in reports, enhanced control over JSON assertions, accessibility testing via Google Lighthouse, and simplified session handling and URL validation.

The newest enhancements primarily focus on these key areas to boost testing efficiency and user experience.

Enhanced Reporting: We’ve significantly improved the quality and usability of test reports. You now get more informative and insightful results with features like full-page screenshots, enhanced JSON assertions for easier data comparison, and a streamlined report structure for improved readability.

Accessibility Testing: Recognizing the growing importance of web accessibility, we’ve integrated Google Lighthouse to automate accessibility checks. This allows you to easily identify and address accessibility issues within your web applications, ensuring compliance with WCAG standards.

Streamlined Workflows: We’ve introduced several features to streamline your testing processes. These include improved session handling for cleaner test environments, enhanced configuration options for greater flexibility, and robust URL validation to prevent unintended access to sensitive systems and ensure test stability.

These improvements are all about making your testing smoother and more dependable. You’ll get better results and learn more from them, so you can build even better software! So, let’s get into the details, shall we?

Enhanced Reporting

Neodymium 5.1 brings significant improvements to its reporting capabilities, providing you with more insightful and informative test results.

Improved Screenshots

You can now capture full-page screenshots with highlighted viewports for better visibility of the test context. Additionally, the last element, the test has interacted with, will be highlighted. This makes it easier to pinpoint the exact location of an interaction and understand the state of the application at that time.

Shows a screenshot of a report with an embedded screenshot showing the marking of the viewpport
Screenshot within Viewport Marking

Test Data Overview

Gain a deeper understanding of your test data and streamline the debugging process by accessing test data in JSON format for each test case. This detailed data analysis capability allows you to quickly identify and resolve potential issues.

Screenshot showing how test data is shown in an Allure report
Test Data within the Allure Report

JSON Assertions

The JSON file comparison feature has been enhanced. It now provides improved accuracy in identifying differences between expected and actual results.

A screenshot with test results on the left and a JSON comparison example on the right
JSON Comparison Within the Allure Report

Environment Information

We’ve added the browser used for testing to the report’s environment section to give you more context about the test results. Also, you can now add custom values to tell the difference between environments or profiles in your test runs, making it easier to figure out if any setup or deployment problems are causing issues.

Step Information and URLs

The clickable URLs included in each step of the report simplify and streamline the process to reproduce test failures and navigate to specific application pages at various points during test execution.

Reduced Report Clutter

The report now features fewer INFO sections, providing a cleaner and more concise overview with less visual noise, allowing for easier identification of critical information.

Accessibility Testing

Neodymium 5.1 now includes Google Lighthouse to help you identify and fix accessibility issues in your web applications. This integration supports your compliance with WCAG standards, and shows that we understand how important web accessibility is.

Shows a screenshot of a report with an embedded Google Lighthouse report
Lighthouse Report Embedded in Neodymium’s Allure Report

Additionally, you can assert against Lighthouse report data to track changes in state or outcome.

Furthermore, this enables WebVitals measurement and verification capabilities.

Workflow Enhancements

Several features help to streamline the testing process and improve overall test automation efficiency.

Popup Blocker

The improved popup blocker boosts test stability by minimizing disruptions from unexpected popups, which can lead to inconsistent test results.

Session Handling

Neodymium now utilizes a dedicated annotation to run @BeforeEach and @AfterEach methods in individual, fresh browser sessions. This guarantees a consistent environment for setup and tear down, and minimizes any influence from prior test steps.

Properties

Environment variables can now be used to override Neodymium properties, providing greater flexibility and simpler integration with CI/CD pipelines. This allows for more dynamic test adjustments based on the specific environment or build.

URL Validation

Restrict test access to certain sites only by defining include and exclude lists. This prevents accidental access to live sites or sensitive data, mitigating the risk of unintended modifications. Any attempt to access a URL not included in the defined list will result in an immediate test failure, providing early feedback and preventing further interactions.

Driver Command-Line Arguments

The configuration now allows for direct pass-through of driver command-line arguments, which streamlines driver setup, facilitates customization, and enables fine-tuning of driver behavior and test execution to meet your specific requirements.

Dependencies

We also updated numerous dependencies to keep them up-to-date and secure. Examples are Selenide, Allure, Log4J, and JUnit.

Summary

Neodymium 5.1 is now available! We value your feedback on new features and any defects encountered. Your input helps ensure Neodymium remains a powerful tool for your needs. You might even find a newer version of Neodymium when you read this article, depending on when you read it.

Automation, Open Source, Testing

A new magnetic force: Neodymium 5.0.0

TL;DR: We proudly announce the release of Neodymium 5.0.0. Neo is a Java-based test library for web automation that utilizes existing libraries (Selenide, WebDriver, Allure, JUnit, Maven) and concepts (localization, test multiplication, page objects) and adds missing components such as test data handling, starter templates, multi-device handling, and other small but useful everyday helpers.

Our R&D team has been busy brewing! So today we finally get back on the major release train and present you the new Neodymium 5.0.0. It comes with a lot of new little convenience features, giving you more control and possibilities on your test automation. A better browser control, improved configuration possibilities and a bunch of new annotations like @DataItem, @WorkInProgress, @RandomBrowser, will help you set up and configure your test automation to your specific needs. Even if a picture says more than 1000 words sometimes it’s a pain to see on a single screenshot, why the automation journey broke, to help you in such cases, we introduced video recording, to exactly see what happened during the whole user journey.

See below for what you can find inside the big box of updated code:

Continue reading A new magnetic force: Neodymium 5.0.0 →
Interesting Reading, Misc, Xceptance

Xceptance Supports Young Explorers with Donation to Witelo e.V.

The little Ozobot speeds around the corner, flashing colorfully, winds its way along a spiral, collides briefly with one of its colleagues and finally finds its way through the maze. It is navigated to its destination by color sensors on its underside, which allow it to follow a defined route. The miniature robot was not programmed by a high-tech engineer, but by Magnus, 10 years old, and the son of our co-founder Simone Drill.

During a programming workshop, he learned to give commands to the Ozobot and to control it with the help of color codes.  This was made possible by witelo e.V. from Jena, who offer working groups and experimentation courses at schools, as well as extracurricular learning formats. These scientific and technical learning venues were founded to promote so-called STEM education, with computer science as one of its components. Students learn the basics about coding, robotics and algorithms in research clubs, on hands-on days and during a wide range of vacation activities. In this context, Magnus also had the opportunity to playfully gain his first programming experience and took his enthusiasm from the workshop home with him.

Source: Ozobot
Continue reading Xceptance Supports Young Explorers with Donation to Witelo e.V. →
Conferences, Interesting Reading, Performance, XLT

Data Science Day 2022

After a two-year break, on 18 May 2022, the 4th Data Science Day Jena took place. The Friedrich-Schiller University is hosting this mini-conference format annually.

Xceptance presented this year a closer look at the data that is collected and processed during load and performance testing. René Schwietzke, Managing Director of Xceptance, talked about the challenges to capture the right data as well as translate the collected data into meaningful results. 

A load test simulates millions of user interactions with a website and therefore is capturing huge amounts of data points. These have to be transformed into a few numbers to make the result of the test easy to communicate but still preserve important details. The talk started with typical business requirements and expectations of the target groups of a load test. It showed the data XLT captures and the dimensions which later drive the data reduction. A few example data series demonstrated the challenges behind the data reduction and what numbers are finally used to satisfy the requirements.

An example load test result illustrated the talk with real data. That example test run created about 17,500 data rows per second which contain about 293,000 data points. The entire test result consists of 3.2 billion data points. This massive data set is turned into a consumable report by XLT in less than six minutes. 

For everyone with an interest in data science, this presentation also offers ideas for research in regard to unsolved data challenges. There might be even some Master’s and Bachelor’s theses topics waiting for you. 

You can find a recording of the presentation below (courtesy of Thüringer Universitäts- und Landesbibliothek Jena).

This is the accompanying slide deck. It is a Reveal.js-based. You can navigate with the spacebar and the arrow keys.

Picture of the presentation


Misc

2020 – One Year, One Picture

One Picture Says It All

If we can only use one picture to symbolize 2020, this might just be it. Working from home and video conferences are now the new normal.

Happy Holidays, Merry Christmas, and a Happy and Healthy 2021, because we all need that.

Open Source, Performance, Software, Xceptance, XLT

Xceptance LoadTest Goes Open Source

We have often been asked why XLT (Xceptance LoadTest) is free but not open source. Now, after considering our customers’ business needs and the advantages of using open source as opposed to free software, Xceptance has decided to open source XLT under the Apache License 2.0.

Within the next 30 days, we will restructure the XLT code base and move subprojects into independent open repositories. We will continue to expand and improve the documentation. All code will be published on GitHub.

Xceptance will continue to maintain and extend XLT as before. Releases will be built and published by Xceptance on a regular schedule. Libraries will be published to Maven Central as part of the open-sourcing process. The latest releases will be available from the XLT GitHub repository as well. Of course, the GitHub defect tracker will start to be your source for feature development and defect fixes.

If you are interested in training, projects, or support for XLT and load testing in general, we of course also offer those services, and will expand this offering further in upcoming months.

By open sourcing, we are opening a new chapter for XLT and we invite everyone to join us. More to come!

Bugs in the Wild, Testing

BOM – Byte Order Mark

Because we encountered another hidden encoding issue as part of test data, here are some information about BOM and why this might be interesting in general for everyone working with a computer beyond Excel and Word.

Before you educate yourself, here is the tool to own to see such a problem easily. Most of all editors hide that information and so you might scratch your head why some data is failing with strange error messages. Get xxd and you will see with other eyes:

The first marked bytes are the magic and now head over to the Wikipedia to read more about BOM: https://en.wikipedia.org/wiki/Byte_order_mark

Interesting Reading, Java, Software Development

Java Training Sessions

Today we are going to publish four of our Java training sessions so you can use the material and benefit from it.

Let’s get started with four direct links to extensive material that might help you to understand Java or code quality better or just help you to reflect on topics you already know.

  • The Java Memory Model: Why you have to know the JMM to understand Java and write stable, correct, and fast code.
  • Java Memory Management: Know more about the size of objects and how Java does garbage collection.
  • High Performance Java: All about the smart Java internals that turn your code into fast code and how you can leverage that knowledge.
  • High Quality Code: The anatomy of high quality code that supports longevity, cross-team usage, and correctness. This is not just about Java, this is about good code in general.

Show a little patience when loading the training, these are all large reveal.js based slide sets. Use the arrow keys or space to navigate. Because the slide sets are designed to be interactive sessions, in many cases, not the entire slide context is revealed at once but block by block.

We publish these training sessions because they are also based on openly shared material, it greatly helped us to advance and understand, as well of course advertise a little what Xceptance might be able to do for you.

We will release more of our material in the next weeks and month, so everybody can browse and learn. This won’t be limited to Java and also cover material about approaching load testing, how to come up with test cases, and more about the modern web and its quality and performance challenges. Of course there will be more Java material too. You can get a glimpse of it when you just follow this link and page through the slides: The Infinite Java Training. Please remember, not all material is complete yet.

If you like the material and you need an audio track aka a real presentation, please talk to us. If you see other training needs in the area of quality assurance, testing, and Java, please contact to us.

More to come.