At Go2Scale, we answer developers’ issues, which are always evolving! It is why we always try to define standards that can be easily reused and improved upon, such as pipeline’s stages.
At the Continuous Integration (CI) and Continuous Deployment (CD) level, the jobs that we use have to be fast, so we can use them in parallel whenever possible, to provide efficient and continuous delivery. But we want them in a specific order that will ensure we do not, for example, try to deploy a new release of an application before the tests are even finished. That’s why DevOps need to carefully construct their pipeline’s stages to categorize them, and more importantly, order them.
From that observation, we tried to define a norm of pipeline’s stages that would encompass every part of a CI and CD pipeline, for every kind of application and production environment. This config is composed of 5 stages:
- Static tests
- Dynamic test
You want to know what compose each of those stages? Some of them are self-explanatory, and you are probably already using them similarly. To be sure, the answer is this way! 👇
The basic stages of every DevOps’ pipelines
Static tests stage
This first stage of your DevOps’ pipeline is really about what you want to do using only the code of your application. Of course, you will find unit testing in this stage. But everything from linters to static analysis tools should be run here. It’s also where you want to run some vulnerability detection tools, like Static Analysis Security Tool or secret detection.
📌 You can find this kind of job in this pipeline’s stage:
python_tests: tests checking the behavior of functions with pytest.
gitleaks verifies you didn’t push a secret key or a password in your commits.
With ShiftLeftSecurity, a Static Application Security Testing (SAST) tool, your code will be analyzed to detect vulnerabilities.
This stage is pretty much self-explanatory: you want to build a new version of your application, and/or the documentation of your application, so you can use it in the next stages and maybe release it and deploy it.
📌 Here are some job’s examples:
docker_build builds a containerized version of your project to ensure its portability and reproducibility.
With rpm_build, you can create a rpm release package for your project, to install it on RHEL systems.
mkdocs will build an HTML version of your project documentation using markdown files.
doxygen can build an HTML version of your project documentation using comments inside your code.
Dynamic test stage
Contrary to the static tests stage, this one is really about testing the version of the application you built just before, dynamically. That means testing the behavior of the different inputs your application can have, instead of the functions.
📌 Job examples:
python_tests runs tests checking the behavior of your API using queries as API users.
zaproxy is a dynamic vulnerability detection tool, that will mimic attackers trying to abuse your project and show the vulnerabilities that may be in it.
Review stage, to create an environment to test manually
The one that may be unknown to you is probably the review stage, because that is not something you typically see in all development environments. However, it’s an essential and powerful tool to optimize your development workflow and improve the confidence in your product.
The review stage is supposed to give you a way to create a full working ephemeral environment, to test it manually. It’s also a perfect way to show new changes to a client, before merging them in your master branch.
That environment is dynamically produced but, contrary to the dynamic test one, the destruction is triggered manually. It gives you all the time you need to verify some things that may not be tested yet automatically.
Indeed, you may have your production deployed to my-webapp.com, but this stage will allow you to review the new features at review-32-my-issue.my-webapp.com.
📌 For example, helm_review creates the review environment on kubernetes.
Deployment, the last of your pipeline’s stages
Also self-explanatory, if every job in every stage was completed, you should be able to deploy this version of your project and release it. You can also deploy other parts of the project like documentation. Or anything you created during the build stage, using for example the docker images built with a helm chart to deploy, or Ansible to update every server you have.
📌 Some job examples:
helm_deploy uses Helm to deploy application on your Kubernetes environments.
ansible_deploy is used to deploy your application with Ansible, using your Ansible roles and playbooks.
pages can deploy the HTML version of your documentation on Gitlab pages server, to make it accessible by your users.