DORA Metrics: Four Keys to Measure DevOps Success

What Are DORA Metrics?

DORA stands for DevOps Research and Assessment, a research program that examines how organizations utilize DevOps to improve software development and delivery processes. DORA collects and analyzes data to identify the factors contributing to the success of DevOps teams. Based on this research, DORA provides frameworks, best practices, and guidance to help organizations enhance their software delivery performance.

The key outcome of DORA’s research is the identification of four specific metrics, known as the Four Keys, which are widely used to measure software delivery performance in the industry.

The four specific metrics are mentioned below:

• Deployment Frequency (DF)
• Lead Time for Changes (LT)
• Time to Restore Service (formerly Mean Time to Recover, or MTTR)
• Change Failure Rate (CFR)

DORA has since added a fifth metric, Reliability, which measures operational performance rather than delivery throughput. Together, the five metrics are often referred to as the DORA metrics, and you will see the terms “Four Keys” and “DORA metrics” used interchangeably.

These metrics help evaluate the effectiveness and efficiency of an organization’s DevOps practices. They reflect the team’s ability to release software quickly while maintaining stability. That tension is the point: the metrics split cleanly into two groups that pull against each other, and a healthy team improves one without sacrificing the other.

Reading the metrics as a pair of columns stops the most common misuse of DORA. Shipping more often is trivial if you ignore the right-hand column, and a team can drive its Change Failure Rate to zero by shipping nothing at all. Elite performers move both columns at once.

In simple terms, DORA metrics reflect the performance of teams and software delivery within an organization, ranging from low to high performers. They also help answer the question, “Are we performing more effectively compared to the previous year?”

DORA metrics are an excellent tool for comparing your organization’s performance with others in the software industry. Here’s how DORA metrics help:

• Better Decision-Making: Make informed decisions on process improvements, resource allocation, and task prioritization by analyzing the metrics.
• Ongoing Improvement: Track progress over time and measure the impact of process changes.
• Team Collaboration: Provide shared data for teams to work together more effectively.
• Improved User Satisfaction: Improve software delivery speed, and reliability, and reduce failures, enhancing the user experience.

Note: Boost your DevOps performance and take software delivery to the next level. Try TestMu AI Today!

Types of DORA Metrics and Their Calculation

As you are already aware, the four key performance indicators of DORA metrics are used to measure the effectiveness of DevOps practices in software delivery, with Reliability added later as a fifth.

The first four are calculated to provide insights into the team’s delivery speed, stability, and responsiveness. Reliability works differently, and is covered in its own section below.

• Deployment Frequency (DF): It measures how often new code is deployed to production, reflecting the speed of updates and improvements. A low frequency often indicates reliance on manual processes or delays in resolving errors, while a higher frequency suggests a faster and more agile deployment process. Note that a deployment here means a release to production, which is why the difference between continuous delivery and continuous deployment changes what this number counts.

  How to Calculate It:

  
  

  Deployment Frequency = Number of Deployments/ Time Period

  Improving DF:

  • Implement automated testing to ensure faster validation.
  • Automate code validation processes to streamline the workflow.
  • Break changes into smaller, manageable updates to reduce deployment complexity.

  DF Benchmarks:

  Benchmark release frequency across teams using these categories:

  • Elite: Multiple deployments daily
  • High: Weekly to monthly deployments
  • Medium: Monthly to every six months
  • Low: Fewer than one deployment every six months
• Lead Time for Changes (LT)

  Lead Time for Changes measures the time from code commit to production deployment, highlighting delays in development or CI/CD pipelines. You will also see this called Mean Lead Time for Changes (MLT), though DORA’s own reporting favors the median over the mean for the same reason it avoids “Mean” in Time to Restore Service: a handful of stalled changes can drag an average far away from the typical experience.

  How to Calculate It:

  
  

  Lead Time for Changes= Sum of Lead Times / Number of Deployments

  Improving MLT:

  • Assess the efficiency of the DevOps pipeline.
  • Identify bottlenecks using visual tools like Value Stream Analytics.
  • Break work into smaller chunks of updates.
  • Automate repetitive tasks using automation tools.

  Lead Time Benchmarks:

  • Elite: Less than one hour
  • High: One day to one week
  • Medium: One month to six months
  • Low: More than six months
• Time to Restore Service (formerly MTTR)

  Time to Restore Service is the time taken to resolve production issues and restore service to users after a failure. A low Time to Restore Service indicates quick recovery and efficient incident handling. Most teams still call this metric MTTR (Mean Time to Recover) out of habit, but DORA retired that name deliberately, and the reasons are worth understanding before you start reporting the number.

  How to Calculate Time to Restore Service:

  Time to Restore Service = Total Downtime​ / Number of Incidents

  Improving Time to Restore Service:

  • Track how quickly the team detects, responds to, and resolves outages.
  • Use DevOps monitoring tools to gain real-time insights into system health.
  • Create clear processes, assign roles, and automate repetitive tasks.
  • Set up prioritized alerts for immediate issue detection.

  Time to Restore Service Benchmarks:

  • Elite: Under 1 hour
  • High: Less than 1 day
  • Medium: 1 day to 1 week
  • Low: Over 6 months
• Change Failure Rate (CFR)

  CFR measures the percentage of code changes that cause issues after deployment. It reflects code quality and helps assess whether updates improve the user experience.

  How to Calculate CFR:

  CFR = (Number of Failed Changes / Total Number of Changes) × 100

  Improving CFR:

  • Benchmark stability and quality across teams.
  • Balance speed and stability in releases.
  • Improve code reviews and adopt continuous testing so regressions surface before release.
  • Promote collaboration among developers, operations, and stakeholders.

  CFR Benchmarks:

  • Elite: 0–15%
  • High: 16–30%
  • Medium: 16–30%
  • Low: 16–30%
• Reliability (the fifth metric)

  Reliability is the fifth metric DORA added to the original Four Keys. It measures operational performance, that is, how well your team meets the user-facing commitments it has made, rather than how fast it delivers.

  How to Calculate It:

  Unlike the other four, Reliability has no universal formula and no Elite-to-Low benchmark bands, because the target is one you set yourself. It is measured as attainment against your own SLOs, so the calculation depends on the indicator you choose and the threshold you commit to. See The Fifth DORA Metric below for how to define and track it.

Two of these definitions have shifted since the Four Keys were first published, so before you implement anything, it is worth being precise about what DORA measures today.

Why DORA Replaced MTTR With Time to Restore Service

Search for DORA metrics and you will find MTTR everywhere, including in dashboards sold as DORA-compliant. DORA itself no longer uses the term. Its reports, and the documentation from Atlassian, GitLab, and Google Cloud, all say Time to Restore Service. The rename was not cosmetic. Both halves of “MTTR” were saying something DORA did not want to say.

The problem with “Mean.” An average is a poor summary of incident data because incident durations are not evenly distributed. Most incidents resolve quickly and a rare few run for hours or days, and those outliers drag the mean somewhere no real incident lives. Nine incidents resolved in 10 minutes and one that took 20 hours produce a mean of roughly two hours, which describes none of the ten. Reporting the median, or a percentile such as the 90th, tells you what recovery actually looks like and where the tail is. This is why DORA drops “Mean” from the name and stops implying a particular statistic.

The problem with “Recover.” “Recovery” invites you to time the fix, that is, the moment the server came back, the pod rescheduled, or the bad deploy was rolled back. But users do not experience your infrastructure, they experience your service. Restoring service means the user-facing failure is over, which may be earlier than the fix if a feature flag or a failover cleared the impact while engineers kept debugging, and may be later if the box is healthy while a queue still drains. Timing service restoration instead of repair keeps the metric anchored to the only thing that matters, which is whether the product worked.

Practically, this changes what you instrument. Rather than reading a timestamp from a deployment or ticketing system, you measure from the start of user-facing impact to the point where your monitoring shows normal service, which is usually derived from the same SLO signals discussed in the next section. This is where test observability earns its keep, since you cannot time the start of impact you never detected. If your organization still calls the metric MTTR, there is no need to fight over the label. What matters is agreeing on the two timestamps and reporting a median rather than a mean.

The Fifth DORA Metric: Reliability and Operational Performance

The gap Reliability fills is straightforward. The original four describe how well you deliver software, and none of them describe whether what you delivered is holding up for the people using it. A team can post elite numbers across all four while its users are having a bad week.

Reliability measures how well your team meets its user-facing commitments: availability, latency, correctness, and whatever else you have promised. Because the commitment is yours to define, that definition comes from your SLOs (Service Level Objectives), which are the internal targets you hold yourselves to, and your SLAs (Service Level Agreements), which are the contractual promises made to customers, usually with financial consequences attached. Your SLOs should be stricter than your SLAs, so that you find out you are drifting before your customers do.

Measuring Reliability, then, means tracking performance against those targets over a window:

• Define the SLI: Pick the Service Level Indicator that reflects user experience, such as the percentage of requests served successfully under 300 ms.
• Set the SLO target: Decide the threshold, for example 99.9% of requests over a rolling 30-day window.
• Track the error budget: That 99.9% target permits roughly 43 minutes of failure per month. Spending the budget is expected, and burning through it early is the signal to slow down and stabilize.
• Report attainment: Reliability is the share of the period in which you met the SLO, which is what makes it comparable across teams that measure very different things.

Reliability also acts as a brake on gaming the other four. A team can inflate Deployment Frequency and shorten Lead Time by shipping small, careless changes, and the damage will not always show up in Change Failure Rate if nothing technically fails. It will show up as an eroding error budget. Guarding against that is largely a testing problem, and continuous testing improves DevOps efficiency precisely because it catches the regressions that quietly spend the budget.

Deployment Rework Rate is an emerging companion metric that elite teams are starting to track alongside these five. It measures the share of deployments that required unplanned follow-up work, such as a hotfix, a patch, or a rollback, to reach a working state. Where Change Failure Rate counts outright failures, Deployment Rework Rate catches the quieter cost of changes that shipped, technically worked, and still had to be redone.

Note: Catch failures before they burn your error budget. Try TestMu AI Today!

How to Implement DORA Metrics?

To implement the DORA metrics, you can follow these steps:

1. Set Up Tracking Systems: Ensure your Version Control System (VCS) and CI/CD pipelines are correctly set up. Tools like GitHub or GitLab should be integrated with the best CI/CD tools like Jenkins or CircleCI to capture data on commit times, deployment frequency, and release times

Use DevOps monitoring tools like Prometheus, Datadog, or New Relic for real-time system performance tracking to measure Time to Restore Service and Change Failure Rate. The same tools supply the SLI data behind the Reliability metric. Incident management tools such as Jira or ServiceNow are key for tracking downtime and recovery efforts.

2. Automate Data Collection: Once your systems are set up, automate the collection of key metrics. Your CI/CD tool should log every production deployment to track Deployment Frequency. For Lead Time for Changes, calculate the time between code commits and when the changes are deployed to production.

Track Change Failure Rate by marking deployments as either successful or failed, with failures requiring a rollback or fix. To calculate Time to Restore Service, use monitoring and incident management tools to track how long it takes to restore service after an outage, measuring from the start of user-facing impact rather than from the moment the ticket was opened.

3. Define a Baseline: Set a baseline for each of the DORA metrics before making improvements. Measure your current lead time or deployment frequency over a month or sprint to establish a starting point. This will help you compare against improvements.

4. Analyze and Act on the Metrics: Regularly analyze the metrics to identify bottlenecks or areas for improvement. If the lead time is too long, pinpoint where delays occur in the CI/CD pipeline, such as during testing or code review.

A high Change Failure Rate may require investigating testing, deployment procedures, or code quality. Collaborate with your team to find solutions and improvements, using automation and better tools to optimize the metrics.

5. Monitor and Iterate: DORA metrics should be monitored continuously to track progress and make necessary changes. Evaluate whether improvements have had the desired impact. Adjust and refine processes based on the metrics. For instance, if lead time improves but Time to Restore Service does not, focus on optimizing incident management next.

If you would rather not build this plumbing from scratch, Google publishes a reference implementation that does most of it for you.

Implementing Google’s Four Keys Open-Source Pipeline

Google maintains an open-source project called Four Keys that collects the four original DORA metrics from your existing tooling and renders them on a dashboard. It is worth knowing about before you buy a DORA product or write your own collectors, because it answers the hardest question for you, which is where the numbers come from.

The Four Keys pipeline is a straightforward extract, transform, and visualize chain built on Google Cloud services, and it will feel familiar if you have built a Google Cloud CI/CD pipeline before:

• Data extraction: Webhook events from your version control and CI/CD systems, such as GitHub or GitLab pushes, deployments, and incidents, are sent to a Cloud Run endpoint and published to a Pub/Sub topic. Pub/Sub decouples ingestion from processing, so a burst of events or a downstream outage does not lose data.
• Data transformation: Events land as raw JSON in BigQuery, where scheduled queries parse each source’s payload into a normalized schema and derive the four metrics. Keeping the raw events means you can change a definition, such as what counts as an incident, and recompute history rather than starting your dataset over.
• The dashboard: A Looker Studio dashboard reads the BigQuery tables and reports all four metrics against DORA’s Elite, High, Medium, and Low benchmark bands, so a team sees its performance category rather than a raw number that means little on its own.
• Cloud Build: Deployment events come from Cloud Build, and the project uses it to deploy the pipeline itself. If you build elsewhere, on Jenkins or GitLab CI for instance, you add a webhook and a parser for your system.

Two caveats are worth setting expectations on. Four Keys is a reference implementation, not a supported product, so you own it once it is running and it will bill against your Google Cloud project. It also covers only the original four metrics, which means Reliability still comes from your SLO tooling. Even if you never run it in production, the source is a useful specification of how each metric can be derived from raw events.

DORA Metrics and Value Stream Management (VSM)

DORA metrics have a blind spot, and it is a big one. They start at the commit. Lead Time for Changes begins when a developer commits code, which means every DORA metric measures only the part of the process that engineering already controls. Teams that automate their pipeline hard often hit a wall here: the metrics look good, and the business still waits months for features.

Value Stream Management (VSM) is the practice of mapping the full path from initial idea to production value, including everything upstream of that first commit. A VSM map covers the wait time in intake and prioritization, product and design approvals, security and compliance reviews, and the handoffs between the teams that own each step.

The two work well together because they measure different segments of the same line. Consider a team with an elite Lead Time for Changes of under an hour whose features still take four months to reach customers. DORA cannot explain that, because the delay is not in the pipeline. A value stream map makes it obvious: the idea sat in a backlog for eight weeks and waited three more on a quarterly architecture review board. No amount of CI/CD tuning touches eleven weeks of queue time, which is why agile release planning often moves the needle more than pipeline work at this stage.

Used together, DORA tells you how well the engine runs, and VSM tells you whether the engine is the bottleneck at all. The practical move is to map the value stream first to find where time is actually lost, then use DORA to measure and improve the delivery segment once you have confirmed that segment is worth optimizing. If the map does point at delivery, automation testing in the CI/CD pipeline is usually where the time goes.

Now that you understand how to implement DORA metrics to speed up your software delivery process let’s explore some real-world use cases for a better understanding.

Real-World Use Cases of DORA Metrics

Here are the real-world use cases of DORA metrics, which will help you get through its practical use:

• Improving Deployment Pipelines: A tech company developing a mobile app uses DORA metrics to identify delays in their deployment process. They discover that manual testing is creating a bottleneck by analyzing Lead Time for Changes. By optimizing QA in their CI/CD pipeline with automated testing, they reduce lead time from 3 days to 8 hours.
• Reducing Downtime for Critical Services: A banking company tracks Time to Restore Service (formerly MTTR) to improve incident handling. They notice that recovery from production outages is slow and introduce better monitoring tools, along with regular incident drills. Over time, they cut their median restore time from 6 hours to 1 hour, helping meet regulatory standards and maintain customer confidence.
• Scaling DevOps in a Growing SaaS Company: A SaaS company expanding from one team to five uses DF to monitor consistency. They find one team falling behind due to CI/CD integration challenges. After providing training and standardizing on shared test automation platforms, all teams align on deployment schedules, improving overall delivery speed.
• Benchmarking DevOps Performance for Planning: A large enterprise compares its DevOps team’s performance to industry benchmarks using DORA metrics. Upon noticing their Deployment Frequency and Lead Time for Changes lagging behind top performers, they focus on automating processes and fostering collaboration to close the gap.

Challenges and Solutions in Implementing DORA Metrics

Implementing DORA metrics has the potential to change DevOps procedures, although there are challenges involved, and several of them overlap with the CI/CD pipeline challenges teams already know well.

Here are a few typical obstacles and effective strategies to overcome them:

• Challenge: Missing Tools and Automation Tracking metrics like Time to Restore Service or deployment frequency manually is time-consuming and error-prone.

  Solution: Invest in DevOps automation tools such as Jenkins, GitLab CI/CD, Datadog, or Prometheus to automate data collection. Automation ensures accurate, real-time insights with minimal effort.

• Challenge: Resistance to ChangeTeams may fear being micromanaged or judged based on metrics.

  Solution: Foster a culture focused on improvement and collaboration. Emphasize that DORA metrics aim to enhance processes, not evaluate individuals. Involve teams in decision-making and highlight workflow improvements.

• Challenge: Scattered DataMetrics are difficult to calculate when data is spread across multiple tools and teams.

  Solution: Integrate systems across development, testing, and operations. Use centralized dashboards like Grafana or Splunk for better data visibility.

• Challenge: Inconsistent Data CollectionDifferent teams may track and interpret data inconsistently.

  Solution: Establish clear standards for data collection and interpretation. Define what constitutes a “failure” or “deployment” and document these standards for uniform application. Teams running multiple testing environments should also agree on which ones count as production for DF purposes.

• Challenge: Over-Focus on MetricsFocusing solely on metrics can lead to rushed work and compromised quality.

  Solution: Treat DORA metrics as indicators of progress, not end goals. Balance them with practices like thorough testing and code reviews to maintain quality.

• Challenge: Hard-to-Understand MetricsTeams may struggle to interpret metrics or connect them to their work.

  Solution: Train teams on the meaning and implications of each metric. Provide actionable insights, such as analyzing patterns in recurring outages to reduce Time to Restore Service.

• Challenge: High Start-Up EffortSetting up systems to measure DORA metrics can be overwhelming.

  Solution: Start with one or two metrics that address immediate priorities, such as Deployment Frequency or Time to Restore Service. Gradually expand as you observe benefits, reducing the initial burden.

• Challenge: Lack of Leadership SupportWithout leadership buy-in, DORA metric initiatives may stall.

  Solution: Demonstrate the business value of DORA metrics, such as faster delivery and improved reliability. Use industry benchmarks to show how they align with company goals.

• Challenge: Measuring Across Complex SystemsTracking metrics in distributed systems or microservices is complex.

  Solution: Use tools designed for observability in distributed systems, like Honeycomb or Lightstep, to gain visibility across teams and architectures.

• Challenge: Unrealistic ExpectationsExpecting immediate results can lead to frustration.

  Solution: Set realistic expectations, emphasizing that improvements take time. Celebrate small wins, like resolving bottlenecks, to keep teams motivated while pursuing long-term goals.

Implementing DORA metrics can undoubtedly bring improvements, but as we’ve seen, there are several challenges that teams often face, from tool integration and data consistency to resistance to change and unrealistic expectations. By addressing these obstacles with the right strategies, teams can better track and optimize metrics like Time to Restore Service, deployment frequency, and change failure rate.

To help overcome some of these challenges and streamline the DevOps process, leveraging a cloud-based platform like TestMu AI can make a significant difference. TestMu AI is an AI-native test execution platform that enables teams to run automated tests at scale across various environments, eliminating the need for complex setup or maintenance.

It seamlessly integrates with popular DevOps testing tools and CI/CD pipelines, allowing teams to optimize their testing workflows and focus on improving key DORA metrics like deployment frequency and Time to Restore Service. With real-time insights and the ability to test in distributed systems and microservices, TestMu AI provides comprehensive visibility across applications, helping teams stay on top of progress and focus on high-priority tasks.

Conclusion

DORA metrics are crucial for improving software delivery. The four original keys measure deployment frequency, change speed, and stability, while the fifth metric, Reliability, keeps you honest about whether what you shipped is actually working for users. By tracking all five, you can identify strengths and areas for improvement, helping you consistently deliver high-quality software.

These metrics serve as practical guides for teams aiming to enhance performance and meet modern development demands. They focus on progress, not perfection, helping you achieve greater efficiency and reliability over time. Pair them with solid DevOps best practices and the numbers tend to follow.

Citations

A Framework for Automating the Measurement of DevOps Research and Assessment (DORA) Metrics: https://www.researchgate.net/

DevOps Research and Assessment (DORA) metrics: https://docs.gitlab.com/

Take the DORA Quick Check: https://dora.dev/quickcheck/

DORA’s software delivery performance and reliability metrics: https://dora.dev/

Four Keys, Google’s open-source DORA metrics pipeline: https://github.com/dora-team/fourkeys