We all want to be better. That applies in our personal life, but also in our professional life. As software developers, we want to develop faster, and offer a more efficient, secure and complete product to our users. But how do you know that you’ve become better? That’s where metrics step in!
Metrics in DevOps
DevOps is the culture of continuous anything and automation. This culture was created to support you in your development processes, and to help you optimize them. Metrics intervene in this goal of endless improvement. Tracking data on specific input, you’ll get feedback on the action you implemented and can measure the impact of the work your team provides.
Some metrics useful for DevOps development
First, we explained why it’s important to measure precise characteristics of your project. Now, let’s see which one particularly apply to DevOps development and can help you grow in competence and improve your activity.
The deployment frequency helps you track the number of time you deploy in a predefined lapse of time. Tracking this measure will help you figure out if you deploy enough times during your project.
In DevOps, with CI/CD practices, it’s better and easier to deploy often small portion of code. You can test them faster thanks to continuous testing.
The deployment time regards the time your team makes to deploy. This metric can be impacted by the job contained in your CI/CD pipeline. To reduce it, you can improve your process of pipeline creation. You can focus on its speed and efficiency, by selecting light images or personalizing the launch of some tools.
Percentage of code coverage by automated tests
We talk about test in the deployment time, saying they could influence in a good way this point. Adding to this metric, you can track the percentage of your code that has been tested before been release. As you can imagine, the higher this percentage is, the better!
Be careful: coverage is a good indicator of test. But having a test coverage to 100% doesn’t mean that you’ve tested all the possible case and that your code will be flawless.
Lead time can be an important metric for your team. It measures the time between the moment your team suggest an idea, and the moment this feature is accessible to your users. With this metric, you can track if your development follow the timeline defined at the beginning of the project. It can also help you identify the blocking points.
By tracking this metric, you will get important information such as the steps that involve a bottleneck and the next effort you have to do to reduce this Lead time.
Support ticket volumes
Tracking the number of tickets opened by your customer declaring a bug and the time of resolution is a good indicator of your product quality and your team efficiency. You’ll see in a blink of an eye if your product answers your target needs or if they saw many improvements you should work on.
Flow escape rate
The flow escape rate will help you track the number of bug that went into production and have succeeded your CI/CD pipeline. Yes, of course, in a perfect world this rate will be equal to zero. But as this world is located in a parallel universe, we need to be able to track these escapes, to see where our automated tests might be failing. A good philosophy can be: a specific bug can only be missed once! Then, your pipeline should catch it.
I don’t think this one needs an explanation. But let me tell you that the whole point of DevOps is to overcome this issue, and make failed deployments a distant nightmare. But in order to be really proactive and nailed all your deployment, you must track the one’s that might not succeed and learn from your mistakes.
Mean Time To Detection (MTTD)
This metric corresponds to the average time passed between the moment an issue appeared in your project in production and the moment your development team know about it. No need to say that this slot time should always be the smallest possible!
Mean Time To Recovery (MTTR)
The MTTR completes the MTTD. Once an error has pop up and is known by your DevOps team, you can calculate the average time they spend to correct it. This helps reveal the efficiency of an organization to fix an error.
Uninterrupted app availability
This one is quite transparent. You need to be able to tell for how long people have been able to use your app or software. As soon as your product is unavailable, this metric goes back to zero.
App usage, traffic and performance
In order to track if your product answers your customer’s needs, you can also measure the usage of your app and the traffic registered on it. Grafana, combined with ElasticSearch, can help you see how many people are using your apps.
Regarding the performance of your app or software, you need to make sure that it’s always available and can answer the number of requests.
Tracking metrics, what for?
Tracking metrics in your project will help you and your project (or development) team improve in several areas. How can you say that your time to market has reduced or at the contrary is increasing if you don’t measure it week from week? Spoiler alert, you can’t (unless you are a seer).
It’s why tracking your metrics is really important: it will help you take decisions based on data and not on feeling. And this advice is true in any project you might lead, not only development’s ones.
Measure what is important to you
Before defining the metrics you want to track, you should decide what the goal of your project is. Do you want to improve your time to market? Do you focus on customer experience, and will assess the number of tickets open and the resolution time?
They are no right answers! It’s up to your team to decide what they want to focus on and what metrics they’ll have to measure. Furthermore, you don’t have to choose only one objective. It’s just easier to follow our metrics according to one goal. If you want to improve on several aspects of your project at the same time, you can create several tracking systems!
See and share the result of your effort
As well as helping your team improve, following the evolution of the mains numbers of your project can emphasis your team successes and accomplishments! It’s important to highlight those and to congratulate your team for the effort they already put in to develop in a record time your app.
Where to begin?
The first step to gather the right metrics for your DevOps project is to determine what you want to improve. You can plan a meeting with the whole team and check with them what is possible to do and who will be in charge to track and measure the progress.