Tools and Techniques DevOps Continuous Integration and Testing

For nearly two decades, agile development processes are considered a vehicle for faster software releases and reduced software development risks. Organizations are adopting a mix of different software practices, which emphasize iterative software development, continuous and frequent integration, requirements documentation and prioritization, automated testing and software verification, as well as team development. In recent years, organizations have also acknowledged a need for converging development activities with operations, which have given rise to a new set of practices known as DevOps. DevOps practices are focused on continuous integration and continuous delivery, based on a holistic approach that considers not only how software is developed, but also how it is deployed and operated in modern computing infrastructures such as elastic cloud infrastructures. DevOps stresses the importance of communication, collaboration, and integration between different stakeholders of the software development and deployment processes, including developers, quality assurance teams, test engineers and IT administrators. It’s not a single methodology or tool, but rather a new concept about stakeholders’ interaction, which is backed up by a range of software tools.

In order to build an effective DevOps culture, the team must be able to rapidly discover and alleviate problems. To this end, there is a need for integrating automated testing disciplines in the DevOps Continuous Integration (CI) and Continuous Delivery (CD) environments. In order to understand this integration, one has to explore not only the continuous integration and continuous release processes, but also leverage modern tools that support these processes as part of the DevOps cycle.

Supporting Continuous Release in the DevOps Cycle

The continuous software release is a very challenging task, as it has many prerequisites. Continuous release means being able and ready to build, test and deploy the entire product based on all the latest changes to the codebase. It usually requires multiple environments for continuous development and low-level testing, while making available the entire product for system/product level testing as well. In particular, environments for development, continuous integration, quality assurance (QA), staging and production should be available to a team that is releasing the software continually.

The development environment is the one that is continuously updated with the latest developments/changes in the code. The code in the development environment has typically passed all static analysis and unit testing processes. On the other hand, the quality assurance environment is usually updated once a day (nightly), with all the new code in order to allow for execution of the module, sub-system and system level testing i.e. the QA environment runs tests once per day rather than continually. The staging environment contains a stable, almost production-ready code, which is destined to facilitate end-to-end testing and pilot operations that require production-like environments. Last, but not least, the production environment contains product versions that are ready to be used or already used by the client in production operations. The production versions are stable and fully-tested.

DevOps Continuous Integration (CI) Processes

Continuous Integration is an integral element of DevOps and the above-mentioned continuous release processes. It enables the integration of software, datasets, and tests from different teams towards producing a single, robust and consistent software product. In order for CI processes to be effective and to be run frequently, they have to be automated. Automation is supported within build, deployment and testing processes based on automated build tools, automated deployment tools, and automated testing tools respectively.  In the scope of a DevOps approach, the use of these tools is orchestrated in order to coordinate developers’ efforts, while at the same time identifying and resolving problems automatically. In this way, automation helps to execute many tests in a short time and processing successful builds at a higher rate. Moreover, it helps project stakeholders in gaining more visibility into the various builds, and also in acquiring and sharing knowledge, which is a key element of DevOps collaborative processes.

The first step in an automated CI workflow involves one or more developers that commit their code to a branch within a CI server. Once this is done, the server is capable of detecting the changes and of initiating a build of the codebase based on the new code. The build entails static analysis on the code, as well as the execution of all unit tests. In the case where the build is successful (i.e. no test fails), the branch is merged to the master, while applications that depend on it are also built and prepared for quality assurance testing. On the other hand, if the build is unsuccessful (e.g., some tests or quality checks fail), the team is promptly and automatically notified to fix the errors. The automation of the entire process has some clear benefits: First, errors are discovered as early as possible, along with the member of the team that is in charge of fixing them i.e. typically the developer that performed the latest commit that resulted in the unsuccessful build. Second, for any new piece of code, all dependencies are tested and validated, which facilitates all related continuous release processes. Last but not least, the entire process can be repeated and completed rapidly, as it does not involve manual and time-consuming checks. Overall, the close monitoring of the results of the automated build process provides prompt insights on the failures and their root causes, which shortens the time needed to fix issues and to achieve a build of high quality.

Automation Tools

As outlined, automated CI is not a matter of using a single tool, but rather a combination of automated tools that accelerate the entire CI workflow. Specifically, the following types of tools are key to automating the entire workflow:

• Source and version control tools: Source control tools facilitate teams to work on a single codebase while tracking changes and being able to view the changes of other team members. They also facilitate the tracking of versions, as well as the association of bugs and defects with the proper version of a module or component. Moreover, they enable teams to easily rollback to stable and safe versions of the code, whenever a new commit is proven problematic and has to be debugged. Popular source control tools include Git, CVS, and Subversion.
• Build Automation: All software developers engage in automated build tasks, which involve extracting source code from a repository, defining and validating library dependencies needed for the code to compile, compiling source code, as well as executing appropriate unit tests after ensuring the appropriateness of test dependencies. Build automation tools facilitate the management of the above steps, as well as the management of the artifacts of this process. Popular build automation tools include Maven, Ant and make.
• CI Servers: These servers boost the management of the CI process. They serve as central repositories, where all code is analyzed, compiled and executed. Moreover, they provide GUI tools for viewing the status of the various modules, visualizing the history of the builds and providing details on the build status such as information about failures, causes of failures, dependencies entailed in the build and more. Jenkins, Bamboo, and TeamCity are some of the most popular CI Servers, which have a central role in DevOps.
• Test Tools: A wide range of test tools are used as part of the CI process. For example, each time some code is checked in, a style checker checks for layout issues, comments, and documentation. Likewise, code checkers can scrutinize the code for issues like unused variables, empty catch blocks, unnecessary object creation, duplicate code fragments and more. Another part of the testing involves bug checking i.e. checking the code for known and/or common bugs. The above-listed testing processes (i.e. code checking, bug checking, style checking) falls in the realm of static code analysis and is supported by tools like Java Stack and .NET Stack. There are also unit testing tools, which ensure that the code exhibits deterministic behavior and complies with the contract (e.g., API) that it is supposed to support. Popular unit testing tools include JUnit, TestNG, and NUnit.

The modern DevOps world is full of outstanding and unique open source tools. Finding the best DevOps tools takes some testing and experimentation. By combining the above-listed tools in proper pipelines, one can really get multiplicative benefits.