Many companies deploy software-as-a-service solutions to support their business activities.While solutions like these are often “out of the box”, the companies still struggle to address test automation, as usual approaches to test automation are hard to fit. New tools are breaking ground in this space and have some clear benefits, yet also have to apply some good-old automation practices.
The Business Challenges
We rarely see software teams building their own test case management or bug tracker. Solutions like those are readily available on the Internet. Similarly very few companies build their own HR systems, payroll or financing systems, they apply standard enterprise solutions like SAP or what previously was known as Microsoft Dynamics AX or C5.
It becomes critical for the business operations and financing that the systems are both updated consistently on one hand and operates consistently on the other. Readers of the DevOps handbook by (Gene Kim et.al. 2016) will recognize the core chronic conflict of pursuing both: responding to the rapidly changing competitive landscape and providing stable, reliable and secure services.
The challenge faced by many companies are that while they have opted for standard enterprise tools and software-as-a-service solutions, those tools still need to be tested. They need to be tested either because they have been customized or the recurring releases need to be understood from a business perspective.
There are at least three driving forces in the business need for testing in the space of customisable and out-of-the-box solutions:
- the more standardized, the less testing
- the more customized, the more testing
- the more risky business-wise the more testing
Examples of Enterprise Business Solutions
If a company deals with logistics of receiving physical orders, handling parts, processing and shipping finished products – this does not run off someone’s local Excel file. These companies rely so heavily on these processes to run smoothly and the systems are loaded with data about the company’s products and processes.
Standard solutions exist for hospitals, yet every hospital in the country is so different that while best-practises are built into the hospital systems they need to be heavily configured to apply. American hospital solutions have been implemented in the Nordics where health-care is free and have very little co-pay, so with every release on the standard package the local IT team needs to go through the solution to apply local rules and customisations. And this is where testing comes to play, the hospital needs to iron out the issues and learn the new release before setting it live for the doctors and nurses in the field.
Another example could be ServiceNow, a strong tool in the service management and task management field. ServiceNow needs to be tailored and customized to the specific business needs. These changes need to be tested with every new release of the basic commercial of the shelf system.
Similarly, for more out of the box systems, where only master data and simple configurations are added, like Microsoft Dynamics 365 for Finance and Operations, the users companies might prefer to retest every release not to have any surprises for their end users.
Adding Test Automation
There are two solutions given the current state of tooling: You either develop and code automation in Selenium, Webdriver, or robot framework. Some libraries exist to ease this task, the Robot Framework for instance has an SAP library. That is if you have developers in your organization. Many small- or medium-sized businesses have not. Neither have organizations that have outsourced everything outside their core business.
The challenge with business solutions like the above is, that while most of the SaaS are web-based, there’s no access to making the source code testable. I know. I have tried to add automation on top of a tool that has guid-like id’s in the HTML, that re-renders with every page refresh.
Within the recent years a new class of automation tools has emerged, where automation is done without code. Among the most relevant for testing purposes are Testim.io, Mabl and Leapwork. They all excel in their visual style of putting together automation of web solutions – it seems so easy that even end-users of systems quickly can set up advanced automation of tedious tasks (login, test data creation) and set up recurring continuous testing.
There are some pitfalls though for these no-code tools too. One of them is that you can as easily end up with a mess of unmaintainable scripts after a bit of time. And even in the context of no-code tools have to apply some coding best practices: version control, comments and building reusable components.
While it might not be for the end users and business experts to learn concepts of high cohesion and low coupling, they need guidance from someone who does… as with all automation projects really.
As we have seen with previous test automation projects for custom built software solutions, we also have to consider maintaining the test suite as consistently as we update the systems under test.
Lessons Learned in SaaS Test Automation
Initially, the low-code approach to test automation for software-as-a-service and out-of-the-box solutions seems like a novel and completely new approach. On the tooling side, the technical need is still there to implement specific tools to hook into what is otherwise challenging to automate.
No matter the fancy moniker “low-code”, code is still code and needs to be treated as such to avoid previous pitfalls of automation and achieve the business needs of both responding to the rapidly changing competitive landscape and providing stable, reliable and secure services.
About the Author:
Jesper Ottosen | Senior Test Manager
Jesper usually leads testing of all kinds of IT – from application development to implementing commercial standard applications, deploying infrastructure and operational technologies and large transition programs.
Jesper works as a Senior Test Manager at NNIT A/S, an IT services company that provides IT consultancy and IT services to the Danish public sector, Danish companies and international life science customers.
