Observations of an Autonomous Testing Company

As we work with more and more customers of various sizes and scale, we consistently hear that companies aren’t satisfied with their ability to have high velocity sprints without sacrificing quality. Though automation is nothing new, the stakes have never been this high, nor the IT landscape this complicated. As we continue to bring autonomous testing to new customers in different verticals, the same problems persist across customer environments, and we wanted to reflect upon two recurring themes we see in customer discussions.

People Have Burnt by AI

Over the past few years, IT has been induced by AI solutions that have promised the world but delivered minimal value. Companies have been burnt by vendors trying to deliver value with immature technology who frankly don’t understand the daily headaches of managing large IT teams. At the same time, AI enabled technology solutions are rapidly becoming a reality due to the rise in ecosystems like Tensorflow, the cheapness and scalability of cloud computing, and the apparent bottomless pockets of large technology companies and the venture capital community, making it harder to ignore the writing on the wall. We often find that customers are keenly aware of this reality and know they need to take action but find it’s not simple to separate fact from fiction. Moreover, teams are just now adopting the right vocabulary to assess the answers to their tough questions.

So how can companies cope with the divide between the need to innovate without leaning of the tips of their skis? By going into each project with eyes wide open on the value these new solutions bring while being keenly aware of their limitations, companies can understand what they’re getting (and not getting) from vendors, while understanding what’s needed for the solution to add value in their environment. Remembering to look for the skill set gaps needed to use the product, the scalability and maintainability of the solution, and the ease to bring it into (and take it out of) the existing tool stack are good starting points.

The Rationality of a Low Test Automation Environment

When we talk with customers, we talk to three main types of companies: companies without automation, companies just starting to fidget with automation, and behemoth enterprises who have bought everything under the sun. The most interesting group to talk to is the large companies who have no automation, especially since test automation is nothing new.
So why do companies consistently not adopt automation? We found that companies aren’t afraid to invest the capital to automation projects, but rather have come to the realization that it isn’t maintainable, forcing them to spend more time reworking the automation than testing, and draining resources from precious development time. Moreover, they can’t afford to put their value added resources on constant maintenance, leading them to keep those teams as productive as possible while piling up on manual QA people. With this context, the decision to not adopt automation is perfectly rationale. Furthermore, the only way to actually solve this problem is to find a lateral solution to scale in their environments.

As we continue to work with companies across various scales, industries, and technological maturities, we find autonomous testing gets companies over these hurdles, though the work is far from over. With an increasing number of customers to learn from, we hope to drill down deeper beyond the ailments of poor quality to apply the concepts behind autonomous testing across the software development cycle. For now, we hope companies can use autonomous to eliminate their quality bottleneck to focus on what matters most – developing great solutions to their customer needs.