CI/CD Overview
Continuous integration (CI) and continuous delivery (CD) are essential terms that are used in DevOps and encompass a set of practices that enable modern development teams to deliver code changes more frequently and quickly.
This is done by introducing automation when it comes to building, deploying, and releasing applications.
CI ensures that code changes are regularly tested and released after merging them into a shared repository (version control system) to ensure their stability while CD allows the quick and smooth delivery of these changes, where they can then be deployed into a live production environment.
Both CI and CD facilitate the efficient release of software to get new, high-quality products out into the market faster than ever before.
Continuous Integration
First, we will take a deeper look at the concept of continuous integration.
In modern software development, developers are usually simultaneously working on different features.
Without continuous integration, while developers attempt to merge their code changes made in separate branches, there’s a high chance that these changes may conflict with changes made by other developers, which may result in what is known as ‘merge conflict hell’. This usually happens when developers try to merge multiple feature branches at the same time.
- Keep reading: Pros and cons of different Git branching strategies
First step: Continuous Integration
Continuous integration (CI) allows developers to merge their code changes into a shared trunk. This is known as trunk-based development, which is a key enabler of continuous integration. With this method, developers can integrate their changes, or commit code in small increments, much more frequently, perhaps even several times a day. Each commit will trigger an automated build and test.
In other words, once these changes are merged, a series of automated tests will verify the build to detect any errors so that any bugs can be quickly fixed without disruption to the software.
Benefits of continuous integration
Because developers are integrating small changes frequently, this allows for faster deployment. It also allows for faster feedback so that developers can fix bugs almost immediately.
In trunk-based development, the master branch is the only long-lived branch while all other branches have a limited life span. This is unlike feature branching where developers make a copy of the codebase and then developers can work on their features separately. This usually leads to merge conflict, and in extreme cases merge hell, as developers are simultaneously merging numerous branches.
In this case, since developers are not integrating their changes frequently, they will not get quick feedback. Instead, they will not be able to see any new changes or release their own features until all the other developers’ changes are ready. Afterward, they will attempt to merge several long-lived branches that may contain significant rather than small changes (and hence major bugs) which could have been avoided had they merged to the trunk more often.
Continuous integration, then, results in higher quality releases as bugs can be detected and fixed quickly leading to increased efficiency and productivity since developers no longer have to wait for everyone else to be finished with their own changes.
Next step: Continuous Delivery
Continuous delivery is a software release approach where teams release quality products frequently through a series of automated tests. For an efficient continuous delivery process, continuous deployment will also need to be built into your pipeline. More on that in a bit.
Therefore, the goal behind continuous delivery is to have software that is always ready for deployment to a production environment from the source repository. In other words, it makes sure that the code is always in a deployable state even as multiple developers are making daily changes through continuous integration. Though this is usually an automated process, the actual release into the production environment may be done manually by teams.
The benefits of continuous delivery are clear
Quicker time to market: Perhaps the most obvious benefit is the quicker time to market, as code is always ready to be deployed to users.
Constant feedback loop: A constant feedback loop allows teams to receive constant feedback on products from their end-users and then incorporate this feedback into the next release.
Enhances productivity: Teams no longer have to deal with tedious tasks, which can be performed by pipelines instead, allowing these teams to focus on building better products resulting in increased customer satisfaction.
Reduces risk: When changes are released more frequently in small increments, errors can be easily and quickly spotted and fixed, thereby reducing the typical risks associated with releases.
Continuous Integration vs Continuous Delivery vs Continuous Deployment
In software development, the process starts with continuous integration then continuous delivery builds on this process to release the changes that have been merged into the shared trunk during continuous integration. This means that continuous delivery enables the automated deployment of the code from development to the production stage.
Consequently, CI/CD represents the process of continuous development, testing, and delivery of new releases.
Often confused with continuous delivery, continuous deployment, in fact, goes one step further than continuous delivery.
During this stage, all changes are automatically released without any human intervention, whereas in continuous delivery, the changes are prepared for delivery but when they are released is determined by the team manually.
In other words, continuous delivery is a partly manual process while continuous deployment is all about automating the entire release process. If an automated test fails at this stage then the change will not be released but if the changes pass the test then they are automatically deployed.
Therefore, continuous deployment is an efficient means to accelerate the feedback loop with your end users. However, all these processes must follow each other, with continuous integration representing the foundation for the other two to take place.
Let’s sum up these three concepts:
- Continuous Integration
- Short-lived branches that are merged into a shared trunk several times a day where a series of automated tests give feedback about the changes introduced.
- Continuous Delivery
- After continuous integration, continuous delivery prepares the software for delivery; deployment to production is usually manual.
- Continuous Deployment
- After CI and CD, changes are automatically deployed into production; fully automated process.
Continuous Testing
We’ve already mentioned that during CI/CD, software goes through a series of automated tests. A CI/CD process, thus, may include the following types of tests:
- Unit tests – to verify single parts of the application. This isolated part of the code base is referred to as a unit.
- Integration tests – since unit tests focus on an individual component and thus may be insufficient by itself, integration tests ensure that multiple components work together correctly and test how parts of the application work together as a whole.
- Functional tests – these tests make sure that the feature is working as it should.
- End-to-end tests – these tests simulate a user experience to ensure that real users have a smooth, bug-free experience.
- Acceptance tests – these verify the behavior of the software under significant load to ensure its stability and reliability
The testing pyramid below depicts the different types of tests you can run. In some cases, you may not need to run all these tests, especially if you’re just getting started.
Since unit tests are the easiest to implement, requiring fewer resources, then they generally make a good foundation for a fast build and get feedback much more quickly to developers.
Meanwhile, UI tests, which ensure that an application works correctly from a user perspective, are much slower and more complex to run. To sum up, not every CI/CD process will have all these tests but it’s worth remembering that continuous testing through automation is a key component of continuous integration and continuous delivery.
What is a CI/CD pipeline?
A CI/CD pipeline is a series of automated tests that follows a software through its delivery lifecycle by taking the source code through to production.
Thus, a typical pipeline builds the code, runs tests, and then deploys the new software into production in a true replica of the software development lifecycle.
Incorporating a CI/CD pipeline is an essential factor in maintaining a DevOps culture as it ensures the fast and efficient release of software with minimal risk. Building a CI/CD pipeline, thus, puts DevOps ideals into practice as it allows developers to commit their changes frequently to gain fast feedback leading to the emergence of a culture of collaboration, increased productivity, and transparency among teams. These fast feedback loops help fulfill the main goal behind building an efficient CI/CD pipeline, which is reducing the risk usually associated with new releases.
Thus, such a pipeline will include the following elements:
- Building, merging then testing the code-continuous integration
- Preparing the code for delivery- continuous delivery
- Deploying the code automatically- continuous deployment
Example of a typical CI/CD Pipeline
Thus, we can deduce that the stages of the CI/CD pipeline include:
- Source: the CI/CD pipeline is triggered when a new code is committed to the repository.
- Build: this is where developers put their new code changes and compile them so they may pass through the initial testing phase
- Test: this is when the new code is tested through automated tests (for example, running unit tests through continuous integration). Depending on the size and complexity of the software, this step could last from seconds to hours. This stage will provide the feedback necessary for developers to fix any issues.
- Deploy: this is when the code is deployed to a testing or staging environment to prepare it for final release i.e. continuous delivery. Usually, the build will automatically deploy once it passes through a series of automated tests.
- Deploy to production: here the code is released into a live production environment to reach end-users, either manually or automatically
It is important to have such a pipeline within modern software development teams as such processes allow teams to direct their energy and time to writing code and improving products while more tedious tasks become automated.
This ties into the idea behind a true DevOps culture, which is reducing manual processes by introducing automation. Without CI/CD, integrating changes and then testing and deploying them would require separate processes which require significant time and effort.
Such automated processes, thus, ensure fewer errors and increased collaboration and efficiency through the software development life cycle. Indeed, implementing a CI/CD pipeline promotes an environment of collaboration as development, IT, and operations teams work together to deliver higher quality software more frequently.
CI/CD best practices
Among the best practices for an efficient CI/CD pipeline include the following:
1. Commit early and often
Keep in mind that the more often you commit changes, the more quickly you will receive feedback on the changes introduced into the trunk. This ensures enhanced collaboration and productivity among your team, reducing risk of errors and merge conflicts when integrating major changes instead of smaller ones more frequently.
The general rule is to commit at least once a day so that others within the team remain up-to-date on all changes taking place. Even if the features are not yet complete, any unfinished changes can be hidden from the end-user with the help of feature flags. More on this later.
2. Make it the only way to deploy to production
After building and implementing a reliable and fast pipeline, do not waste your efforts by bypassing the process.
In other words, you need to make sure that any changes introduced go through the pipeline and that this pipeline is the only way for code to be released into the production environment.
3. Continuously review your automation processes
In modern software development, new technologies are constantly evolving and new processes are always being introduced. Thus, it is important to continuously evaluate which processes and tests can be integrated into your pipeline to increase efficiency.
Also, keep in mind that not everything needs to be automated, at least not all at once. It’s sometimes preferable to start out manual to review what actually needs to be automated.
4. Speed up your pipeline
The whole point of designing a CI/CD pipeline is to speed things up to get software out faster than before through automation.
As such, a general rule of thumb is to run the fastest tests first before letting your pipeline tackle the more demanding, time-consuming tests. For example, executing unit tests first then integration followed by functional tests. That way, with these simpler, quicker tests you’ll be able to detect errors fast and fix them in the time it takes to run a more demanding test.
Thus, it is all about knowing how to prioritize the tests in your test suite.
5. Monitor your pipeline
You should also be on the lookout for any areas of improvement to understand whether there are stages within your pipeline that need to be optimized.
Monitor the metrics collected by your CI/CD tool to identify any issues that need to be addressed to ensure the reliability and performance of your infrastructure.
CI/CD tools to design an efficient pipeline
CI/CD can help teams automate the processes of development, testing and deployment. Some tools will focus on handling the continuous integration side of things while other tools will be more focused on continuous delivery.
In this section, we will highlight some of the common tools used to automate these processes as choosing the right tools is key in implementing an efficient CI/CD pipeline that is most suitable for your organization.
Some popular tools include:
1. Jenkins
This is one of the most well-known open-source tools for CI/CD. As an extensible automation server, it can be used as a CI server and can be turned into a continuous delivery hub.
2. CircleCI
A tool that offers flexible environments and thousands of pre-built integrations; CI/CD orchestration in the cloud or option to use self-hosted runners for added flexibility and control.
3. GitLab CI/CD
Streamline and automate your release process offering safe and flexible deployment options. GitLab also acts as the single source of truth for CI/CD and so you can build, test, deploy and monitor your code from a single application.
4. Travis CI
This is an open-source CI/CD platform to help developers quickly and easily develop, test and deploy code. This tool is quick to set up and supports over 30 languages offering great flexibility.
5. Semaphore
this tool supports a number of languages and platforms including iOS apps. Thus, it can be used to accelerate your releases and deploy across web, desktop and mobile apps.
6. Spinnaker
This is an open-source continuous delivery platform that works with a variety of cloud providers with the aim of offering fast, safe, and repeatable deployments.
CI/CD + feature flags: The magic formula for even faster deployments
As we’ve seen, continuous integration and continuous delivery are two essential practices to help you deliver quality software faster.
Implementing feature flags onto these processes provides further value and reduced risk when it comes to integrating new changes and then deploying them.
What are feature flags?
Feature flags are a software development tool whose purpose is to turn certain functionalities on or off to safely test in production by decoupling code deployment from feature release.
Let’s imagine this scenario: there are multiple developers working on several changes over various timelines. What happens when there are developers who have finished their changes while others have not yet finished? Before, this meant that developers would need to wait till everyone on the team was done with their changes before they could finally integrate and deploy the changes.
This may result in dissatisfied customers who would need to wait longer for new releases and in a disruption in the feedback loop as changes are not being merged frequently enough. With feature flags, developers can push their changes without waiting for other developers by simply turning off the portions of the code that are incomplete.
In other words, these incomplete changes can be hidden behind a feature flag while the finished changes can be released. Once they are finished, they can be turned on to become visible to end-users.
This is important as the whole aim of continuous integration is to integrate changes at least once a day and so feature flags help maintain the momentum of continuous integration.
Much in the same way, feature flags help to deliver on the promise of continuous delivery as developers can still proceed with a release while keeping the unfinished changes hidden behind a flag so they don’t affect the user experience.
This means faster time to market as well as the ability to gather continuous feedback to keep improving your products resulting in increased customer satisfaction.
Feature flags are also helpful as kill switches, which means if any bug made it past automated testing, it can be easily turned off or rolled back using feature flags. This way, you disable the feature with the bug and not the entire feature release.
The main takeaway here is that with feature flags, you can deliver releases faster and more safely to end-users.
Conclusion
To sum up, continuous integration and continuous delivery are essential staples in modern software development but with feature flags, they become even better and more powerful by offering significant value to your CI/CD pipeline and eventually your customers.
