An in-depth explanation of Go2Scale’s default pipeline’s stages

An in-depth explanation of Go2Scale’s default pipeline's stages
+1
4 min read

At Go2Scale, we answer developers’ issues, which are always evolving! It is why we always try to define standards that can be easily reused and improved upon, such as pipeline’s stages.

At the Continuous Integration (CI) and Continuous Deployment (CD) level, the jobs that we use have to be fast, so we can use them in parallel whenever possible, to provide efficient and continuous delivery. But we want them in a specific order that will ensure we do not, for example, try to deploy a new release of an application before the tests are even finished. That’s why DevOps need to carefully construct their pipeline’s stages to categorize them, and more importantly, order them. 

From that observation, we tried to define a norm of pipeline’s stages that would encompass every part of a CI and CD pipeline, for every kind of application and production environment. This config is composed of 5 stages:

  • Static tests
  • Build
  • Dynamic test
  • Review
  • Deployment
The pipeline's stages in Go2Scale

You want to know what compose each of those stages? Some of them are self-explanatory, and you are probably already using them similarly. To be sure, the answer is this way!  👇

efficiency

The basic stages of every DevOps’ pipelines

Static tests stage

This first stage of your DevOps’ pipeline is really about what you want to do using only the code of your application. Of course, you will find unit testing in this stage. But everything from linters to static analysis tools should be run here. It’s also where you want to run some vulnerability detection tools, like Static Analysis Security Tool or secret detection. 

📌 You can find this kind of job in this pipeline’s stage:

python_tests: tests checking the behavior of functions with pytest.

gitleaks verifies you didn’t push a secret key or a password in your commits.

With ShiftLeftSecurity, a Static Application Security Testing (SAST) tool, your code will be analyzed to detect vulnerabilities.

Build stage

This stage is pretty much self-explanatory: you want to build a new version of your application, and/or the documentation of your application, so you can use it in the next stages and maybe release it and deploy it. 

 

📌 Here are some job’s examples:

docker_build builds a containerized version of your project to ensure its portability and reproducibility.

With rpm_build, you can create a rpm release package for your project, to install it on RHEL systems. 

mkdocs will build an HTML version of your project documentation using markdown files. 

doxygen can build an HTML version of your project documentation using comments inside your code. 

Dynamic test stage

Contrary to the static tests stage, this one is really about testing the version of the application you built just before, dynamically. That means testing the behavior of the different inputs your application can have, instead of the functions. 

📌 Job examples:

python_tests runs tests checking the behavior of your API using queries as API users. 

zaproxy is a dynamic vulnerability detection tool, that will mimic attackers trying to abuse your project and show the vulnerabilities that may be in it.

efficiency

Review stage, to create an environment to test manually

The one that may be unknown to you is probably the review stage, because that is not something you typically see in all development environments. However, it’s an essential and powerful tool to optimize your development workflow and improve the confidence in your product.

The review stage is supposed to give you a way to create a full working ephemeral environment, to test it manually. It’s also a perfect way to show new changes to a client, before merging them in your master branch. 

That environment is dynamically produced but, contrary to the dynamic test one, the destruction is triggered manually. It gives you all the time you need to verify some things that may not be tested yet automatically.

Indeed, you may have your production deployed to my-webapp.com, but this stage will allow you to review the new features at review-32-my-issue.my-webapp.com.

📌 For example, helm_review creates the review environment on kubernetes.

efficiency

Deployment,  the last of your pipeline’s stages

Also self-explanatory, if every job in every stage was completed, you should be able to deploy this version of your project and release it. You can also deploy other parts of the project like documentation. Or anything you created during the build stage, using for example the docker images built with a helm chart to deploy, or Ansible to update every server you have.

📌 Some job examples:

helm_deploy uses Helm to deploy application on your Kubernetes environments.

ansible_deploy is used to deploy your application with Ansible, using your Ansible roles and playbooks.

pages can deploy the HTML version of your documentation on Gitlab pages server, to make it accessible by your users.

efficiency

Build your pipeline – An example

Here is a tutorial where we build a pipeline using all the stages to its advantages, to reduce development time of an application. In this video, we are using Go2Scale’s product, R2Devops

If you want to learn a bit more about this platform and see how it can help you optimize your pipeline’s creation, you can read our article and join our community!

+1

Leave a Reply

Your email address will not be published. Required fields are marked *