Signagelive speeds up testing, catches more bugs, and frees up time with Rainforest QA

Francisco José Arráez Segura

QA Manager

at

Signagelive

Employees

10-50

Industry

Digital signage

Key results

The situation

Signagelive struggled with slow and unreliable manual testing — customers found bugs before the QA team did. They needed a faster solution that could give them confidence in the accuracy of the app’s user interface.

The result

Signagelive’s non-technical QA team automated their tests in Rainforest’s intuitive, no-code platform. This sped up testing, drastically reduced the number of critical bugs reaching production, and freed up QA personnel for strategic work.

"Yesterday I reported eleven bugs in four minutes. I managed to double check that they were resolved in two minutes. That’s impossible with manual testing."

Facing several challenges with manual testing

Signagelive is a global leader in the digital signage industry powering tens of thousands of screens deployed across 46 countries, with a cloud-based platform available in multiple languages.

In early 2016, Signagelive had an informal approach to QA. When developers had time, they performed a few manual tests ahead of each software release and hoped for the best. This allowed the company to focus on growth and building new features, but the number of bugs clients found was unacceptable.

Signagelive’s first step toward building a QA program was to pull Francisco José Arráez Segura from the customer support team to head up QA efforts. When Fran first took over QA, he worked with a team of QA testers doing manual software testing. They wrote instructions for testers, sent those instructions, and waited for results. It often took a few days to get all the results back for their regression test suite.

After receiving the results, the most time-consuming task began: categorizing the failures.

Ideally, each failure would represent an actual bug, but not all bugs were equally important. Fran had to re-run each test to determine the bug's priority before sending it to developers.

As Signagelive’s test suite grew, these errors happened more often, slowing down testing and undermining the reliability of test results.

In an effort to increase test coverage, Signagelive added two more people to the QA Team. Even with more people, manual testing was too slow to achieve the level of coverage desired.

Getting time back while improving product quality with Rainforest

Rainforest automated testing helped Signagelive get fast and consistent test results across their entire platform. They eliminated human error, ran all tests in parallel, and quickly categorized test results to identify bugs.

Rainforest QA’s automation tool allowed anyone to create and maintain automated tests without programming skills.

Fran and Lewis were able to get started immediately without hiring a QA engineer or learning a framework. When needing to speed up test creation, they borrowed team members who could get up to speed with only a short tutorial.

Fran automated much of their regression test suite. The team saw benefits immediately after converting manual tests to automated tests. Test runs took significantly less time, and Fran no longer worried about human error. If a test failed, it was either due to a bug or a test step needing an update.

Before using automated testing, it took Fran and Lewis two or three weeks to complete a full regression test suite. With Rainforest, running an entire suite only takes as long as the longest test, averaging less than four minutes.

With automation, Signagelive now completes their entire regression suite of 500 tests in half a day.

Evaluating test results with Rainforest’s automation was much faster than manual results evaluating, facilitating bug categorization and reporting to developers.

Thanks to video recordings of tests and other platform information, the QA team can quickly determine whether a failure was caused by a bug or a test problem.

"Now I’m confident if a test fails on step X, it’s because that feature or button in the app has failed."

Rainforest integrates seamlessly with Jira, automating ticket creation for any found bugs. Verifying fixes takes just a few minutes.

With automation, they accomplish more testing in less time, allowing Fran to focus on the app's user experience and design tests for how features interact. The rest of the QA team can devote more time to hardware testing. The number of critical bugs reaching production has decreased dramatically.

“I’m more confident than ever that our platform’s quality is OK."

Needing an automated test solution that verifies visual elements, not underlying code

In late 2020, a client requested a feature called granular user permissions (GUP), allowing network admins to control user visibility of certain features. Testing GUP requires confidence in verifying what users see.

GUP wasn’t suitable for manual testing due to potential human error from misreading complex instructions. Fran sought an automated testing solution.

Selenium was not a good fit for testing GUP, as it checks the underlying code rather than the visual interface. Testing with Selenium may yield unreliable results if elements are blocked or misplaced.

Rainforest was the only automation service that visually identifies elements on the page and met Signagelive’s needs.

Find more bugs, faster, without adding headcount

Rainforest empowers teams to enhance QA, allowing more focus on product building and reducing bugs.