Five tips for a quality process using continuous testing
We increasingly hear about continuous testing. Continuous integration, continuous testing, and continuous delivery are increasingly talked-about activities, but they are not always understood. In this article, we try to shed some light on the continuous testing process. At the end of the day, what should my software quality process include for my testing to be considered “continuous?”
Everyone agrees on a few things. There is a consensus that continuous testing involves a substantial investment in automated testing, which is done to ensure quality at delivery. But is that all? The automation of testing is nothing new, and it is not unusual to make a significant investment in it. So, what’s new here?
In brief, continuous testing mainly involves making the execution of automatic testing a fundamental part of the software delivery lifecycle. Instead of running automated tests at the end of the delivery cycle, we use the feedback offered by automatic regression tests and automatic testing of new functionalities with every small delivery. Yes, even new functionalities! That is the main challenge in this approach to testing, to bring it to fruition, a few techniques are required to make the testing automation process more efficient. Some of these techniques lie directly within the automation, others in enabling the management of small functional deliverables and of code quality. Our purpose is not to outline the details of these techniques, but here are the most important for your research:
· Automation of the software lifecycle
· Unit testing
· Test doubles (mocks)
· Service virtualization
Each of these techniques can be useful in a different context, but the automation of the software lifecycle is 100% necessary if we are to work with continuous testing, because they are intrinsically connected. An automation server is a fundamental basis for the automatic execution of automatized tests.
The automation of the software lifecycle normally begins with the continuous integration of changes. Using an automation server (the Jenkins, for example), the process of bringing in new versions of source code and generating a new version of the executable software package is automated, normally delivered within the execution environment that makes sense at the time (for example, development, integration, homologation, or even production).
Beginning continuous testing is the next step in pursuing a mature software functionality delivery process; if you already create your new software version automatically, it makes sense to conduct your testing automatically, and validate the expected behavior of the software without human interaction. To do so, we add a delivery automation step after the build and deploy of the new version of the software, so software tests are conducted automatically.
So, is that it? Yes and no. If we conduct a large volume of automated tests, which are an integral part of the software delivery process, beginning with the first builds of a functionality, then we could say yes, we already have a continuous testing process underway. But the simple fact is that such a process exists does not guarantee it is efficient. I would like to share five tips that I consider fundamental to the proper function of a quality process using continuous testing:
1) A single multidisciplinary team
We do not have a development team, a testing team, and an operations team. We have one, single team, 100% committed to the quality of the deliverable. Yes, we have specialists in architecture, development, testing, and infrastructure, but they all work together to create the desired result. The strategy of development, testing, and deployment is discussed from the beginning of the current development cycle by the whole team, and tests development begins in parallel to the functionality code, allowing for the shortest possible development feedback cycle.
2) Have the necessary infrastructure available
Ideally, tests reflect the needs of a well-structured infrastructure architecture, without any demands that don’t make sense. For example, if we have a business layer that is shared by a web application and a mobile application, tests for this business layer should be conducted only once, because there is no replication of functionalities. On the other hand, it is important to ensure the specific behaviors for each client platform, evaluated according to the business risk involved. It is your business risk, and not the availability of an environment for execution, that should determine whether a test should be used to ensure quality. What is the cost of not having the necessary infrastructure?
3) Testing in the software development life cycle
Testing automation should be launched as early as possible in the software delivery automation process. This ensures there is no hand-off between the development team and the testing team. The test takes place on pace with each delivery, on the smallest scale possible.
Tests are conducted beginning in the development environment, in intermediary environments, and even in the production environment. Of course, some tests cannot be conducted indiscriminately in production; however, testing should be conducted in every environment, depending on feasibility and need.
4) High degree of automation
The biggest benefit of the process as a whole is the ability to conduct tests without human interaction, and get secure feedback about the quality of deliverables. The lower the level of manual interventions in a process, the greater the benefit, and the faster the ROI. If activities are done manually, are there ways to overcome roadblocks that prevent automation? That breaks the automation flow. Market leaders are reaching 80% automation; that is the level of functionality automation necessary to provide a fast enough feedback cycle.
5) A clear view of every platform
Ideally, we want to have a dashboard with all of the information: the tests performed, what happened, what failed, the functionalities we delivered, the tests involved in the delivery. It is important to integrate the results from every platform (for example, mobile, web browsers and back end) in a clear view, so new deliverables and their results can be identified.
Conclusion: What sets an automated testing process apart, so it can be called “continuous,” is more quantitative than qualitative, considering that we should consider quantitative measures in automated tests. Even so, agile management techniques, along with automated testing and techniques to automate the software life cycle, is what allows much faster feedback on quality. This speed significantly increases the ROI of the overall development process, with cases of estimated savings of 20% in total development time using these techniques. That is definitely an improvement worth the investment.