CI/CD is a best practice for devops and agile development. Here's how software development teams automate continuous integration and delivery all the way through the CI/CD pipeline.
By Isaac Sacolick
Contributing Editor, InfoWorld|
doguhakan / robertiez / Getty Images
Table of Contents
Continuous integration (CI) and continuous delivery (CD), also known as CI/CD, embodies a culture, operating principles, and a set of practices that application development teams use to deliver code changes more frequently and reliably.
CI/CD is a best practice for devops teams. It is also a best practice in agile methodology. By automating integration and delivery, CI/CD lets software development teams focus on meeting business requirements while ensuring code quality and software security.
Continuous integration is a coding philosophy and set of practices that drive development teams to frequently implement small code changes and check them in to a version control repository. Most modern applications require developing code using a variety of platforms and tools, so teams need a consistent mechanism to integrate and validate changes. Continuous integration establishes an automated way to build, package, and test their applications. Having a consistent integration process encourages developers to commit code changes more frequently, which leads to better collaboration and code quality.
Continuous delivery picks up where continuous integration ends, and automates application delivery to selected environments, including production, development, and testing environments. Continuous delivery is an automated way to push code changes to these environments.
Automating the CI/CD pipeline
CI/CD tools help store the environment-specific parameters that must be packaged with each delivery. CI/CD automation then makes any necessary service calls to web servers, databases, and other services that need restarting. It can also execute other procedures following deployment.
Because the objective is to deliver quality code and applications, CI/CD also requires continuous testing. In continuous testing, a set of automated regression, performance, and other tests are executed in the CI/CD pipeline.
A mature devops team with a robust CI/CD pipeline can also implement continuous deployment, where application changes run through the CI/CD pipeline and passing builds are deployed directly to the production environment. Some teams practicing continuous deployment elect to deploy daily or even hourly to production, though continuous deployment isn’t optimal for every business application.
Organizations that implement a CI/CD pipeline often have several devops best practices in place, including microservices development, serverless architecture, continuous testing, infrastructure as code, and deployment containers. Each of these practices improves process automation and increases the robustness of cloud computing environments. Together, these practices provide a strong foundation to support continuous deployment.
How continuous integration improves collaboration and code quality
Continuous integration is a development philosophy backed by process mechanics and automation. When practicing continuous integration, developers commit their code into the version control repository frequently; most teams have a standard of committing code at least daily. The rationale is that it’s easier to identify defects and other software quality issues on smaller code differentials than on larger ones developed over an extensive period. In addition, when developers work on shorter commit cycles, it is less likely that multiple developers will edit the same code and require a merge when committing.
Teams implementing continuous integration often start with the version control configuration and practice definitions. Although checking in code is done frequently, agile teams develop features and fixes on shorter and longer timeframes. Development teams practicing continuous integration use different techniques to control what features and code are ready for production.
Many teams use feature flags, a configuration mechanism to turn features and code on or off at runtime. Features that are still under development are wrapped with feature flags in the code, deployed with the main branch to production, and turned off until they are ready to be used. In recent research, devops teams using feature flags had a ninefold increase in development frequency. Feature flagging tools such as CloudBees, Optimizely Rollouts, and LaunchDarkly integrate with CI/CD tools to support feature-level configurations.
Continuous integration not only packages all the software and database components, but the automation will also execute unit tests and other types of tests. Testing provides vital feedback to developers that their code changes didn’t break anything.
Most CI/CD tools let developers kick off builds on demand, triggered by code commits in the version control repository, or on a defined schedule. Teams need to determine the build schedule that works best for the size of the team, the number of daily commits expected, and other application considerations. A best practice is to ensure that commits and builds are fast; otherwise, these processes may impede teams trying to code quickly and commit frequently.
Continuous testing and security automation
Automated testing frameworks help quality assurance engineers define, execute, and automate various types of tests that can help development teams know whether a software build passes or fails. They include functionality tests developed at the end of every sprint and aggregated into a regression test for the entire application. The regression test informs the team whether a code change failed one or more of the tests developed across the functional areas of the application where there is test coverage.
A best practice is to enable and require developers to run all or a subset of regression tests in their local environments. This step ensures developers only commit code to version control after code changes have passed regression tests.
Regression tests are just the beginning, however. Devops teams also automate performance, API, browser, and device testing. Today, teams can also embed static code analysis and security testing in the CI/CD pipeline for shift-left testing. Agile teams can also test interactions with third-party APIs, SaaS, and other systems outside of their control using service virtualization. The key is being able to trigger these tests through the command line, a webhook, or a web service, and get a success or failure response.
Continuous testing implies that the CI/CD pipeline integrates test automation. Some unit and functionality tests will flag issues before or during the continuous integration process. Tests that require a full delivery environment, such as performance and security testing, are often integrated into continuous delivery and done after a build is delivered to its target environments.
Stages in the continuous delivery pipeline
Continuous delivery is the automation that pushes applications to one or more delivery environments. Development teams typically have several environments to stage application changes for testing and review. A devops engineer uses a CI/CD tool such as Jenkins, CircleCI, AWS CodeBuild, Azure DevOps, Atlassian Bamboo, Argo CD, Buddy, Drone, or Travis CI to automate the steps and provide reporting.
For example, Jenkins users define their pipelines in a Jenkinsfile that describes different stages such as build, test, and deploy. Environment variables, options, secret keys, certifications, and other parameters are declared in the file and then referenced in stages. The post section handles error conditions and notifications.
A typical continuous delivery pipeline has build, test, and deploy stages. The following activities could be included at different stages:
A more sophisticated continuous delivery pipeline might have additional steps such as synchronizing data, archiving information resources, or patching applications and libraries.
Teams using continuous deployment to deliver to production may use different cutover practices to minimize downtime and manage deployment risks. One option is configuring canary deployments with an orchestrated shift of traffic usage from the older software version to the newer one.
CI/CD tools and plugins
CI/CD tools typically support a marketplace of plugins. For example, Jenkins lists more than 1,800 plugins that support integration with third-party platforms, user interface, administration, source code management, and build management.
Once the development team has selected a CI/CD tool, it must ensure that all environment variables are configured outside the application. CI/CD tools allow development teams to set these variables, mask variables such as passwords and account keys, and configure them at the time of deployment for the target environment.
Continuous delivery tools also provide dashboard and reporting functions, which are enhanced when devops teams implement observable CI/CD pipelines. Developers are alerted if a build or delivery fails. The dashboard and reporting functions integrate with version control and agile tools to help developers determine what code changes and user stories made up the build.
Measuring CI/CD success with devops KPIs
The impact of implementing CI/CD pipelines can be measured as a devops key performance indicator (KPI). Indicators such as deployment frequency, change lead time, and incident meantime to recovery (MTTR) are often improved by implementing CI/CD with continuous testing. However, CI/CD is just one process that can drive these improvements, and there are other prerequisites to improving deployment frequencies.
CI/CD with Kubernetes and serverless architectures
Many teams operating CI/CD pipelines in cloud environments also use containers such as Docker and orchestration systems such as Kubernetes. Containers allow for packaging and shipping applications in a standard, portable way. Containers make it easy to scale up or tear down environments with variable workloads.
There are many approaches to using containers, infrastructure as code (IaC), and CI/CD pipelines together. Free tutorials such as Kubernetes with Jenkins or Kubernetes with Azure DevOps can help you explore your options.
Another option is to use a serverless architecture to deploy and scale your applications. In a serverless environment, the cloud service provider manages the infrastructure, and the application consumes resources as needed based on its configuration. On AWS, for example, serverless applications run as Lambda functions and deployments can be integrated into a Jenkins CI/CD pipeline with a plugin. and GPS serverless computing are similar services.
Next generation CI/CD applications
You may be wondering about some of the more advanced areas for CI/CD pipeline development and management. Here are a few notable ones:
To recap, continuous integration packages and tests software builds and alerts developers if their changes fail any unit tests. Continuous delivery is the automation that delivers applications, services, and other technology deployments to the runtime infrastructure and may execute additional tests.
Developing a CI/CD pipeline is a standard practice for businesses that frequently improve applications and require a reliable delivery process. Once in place, the CI/CD pipeline lets the team focus more on enhancing applications and less on the details of delivering it to various environments.
Getting started with CI/CD requires devops teams to collaborate on technologies, practices, and priorities. Teams need to develop consensus on the right approach for their business and technologies. Once a pipeline is in place, the team should follow CI/CD practices consistently.
Isaac Sacolick is president of StarCIO and the author of the Amazon bestseller Driving Digital: The Leader’s Guide to Business Transformation through Technology and Digital Trailblazer: Essential Lessons to Jumpstart Transformation and Accelerate Your Technology Leadership. He covers agile planning, devops, data science, product management, and other digital transformation best practices. Sacolick is a recognized top social CIO and digital transformation influencer. He has published more than 800 articles at InfoWorld.com, CIO.com, his blog Social, Agile, and Transformation, and other sites.
Which service lacks features to assign individual developers tasks to work on?
Azure Pipelines is a CI/CD tool for building an automated toolchain. It lacks features to assign tasks for individual developers to work on. However, it can automate other tools to assign tasks to users.
Which of the following choices is not a cloud computing category?
Which of the following choices isn't a cloud computing category: NAAS, PAAS, SAAS, IAAS, DAAS? - NaaS isn't a cloud computing category.
Which of the following options can you use to link virtual networks?
Through VNet Peering: You can connect virtual networks to each other, enabling resources in either virtual network to communicate with each other, using virtual network peering. The virtual networks you connect can be in the same, or different, Azure regions. To learn more, see Virtual network peering.
Which is the best way for tailwind traders to ensure that the team deploys only cost effective virtual machine SKU sizes?
Which is the best way for Tailwind Traders to ensure that the team deploys only cost- effective virtual machine SKU sizes? Create a policy in Azure Policy that specifies the allowed SKU sizes.