What Are DORA Metrics?

DORA metrics are four engineering performance measures developed by the DevOps Research and Assessment team at Google. They come from the Accelerate research program, which analyzed thousands of engineering teams to identify which practices distinguish elite software delivery performers from low performers. The four metrics are:

Deployment Frequency: How often your team successfully deploys to production. Elite teams deploy multiple times per day. Low performers deploy less than once per month.
Lead Time for Changes: The elapsed time from a code commit entering your main branch to that code being live in production. Elite teams achieve this in under an hour. Low performers take more than six months.
Change Failure Rate: The percentage of production deployments that result in a degradation requiring a hotfix, rollback, or incident. Elite teams keep this below 5%. Low performers see failure rates above 15%.
Mean Time to Restore (MTTR): How long it takes to restore service after a production incident. Elite teams restore in under an hour. Low performers take more than six months.

The DORA research shows that these four metrics cluster together — teams that score well on deployment frequency also tend to have shorter lead times, lower change failure rates, and faster recovery times. Together, they form a predictive model of software delivery capability that correlates strongly with organizational outcomes like profitability, market share, and employee satisfaction.

Why Azure DevOps Does Not Expose DORA Metrics Natively

Azure DevOps is genuinely well-suited to DORA measurement in theory. It bundles Azure Repos (source control), Azure Pipelines (CI/CD), and Azure Boards (work item tracking) under a single authentication boundary with a unified REST API. All the underlying data exists in one system.

The problem is that Azure DevOps Analytics — the built-in analytics layer that powers Power BI connectors and the Analytics widgets in dashboards — exposes each service area through separate OData entity sets. The PipelineRuns entity does not join to WorkItems. The GitCommits entity does not join to Releases. Each service area has its own analytics view with no cross-service correlation built in.

This means that calculating accurate DORA metrics from Azure DevOps requires either:

Building a custom data pipeline that calls the Azure DevOps REST APIs directly across Repos, Pipelines, and Boards, handles commit-to-release SHA matching, implements rollback detection, and maintains the correlation logic as your pipeline topology changes.
Using a dedicated engineering metrics platform that handles the cross-service correlation for you.

Most teams that try option 1 spend weeks building the pipeline, then spend ongoing time maintaining it as release definition IDs change, environments get renamed, or incident tracking moves to a different tool. Most eventually switch to option 2.

How to Calculate Each DORA Metric from Azure DevOps

Deployment Frequency

Deployment Frequency requires the Azure Pipelines Releases API, not the Builds API. The critical distinction: the Builds API returns every CI run including feature branch builds, PR validation builds, and failed builds that never deployed anywhere. If you count CI builds as deployments, your Deployment Frequency will be inflated 10-50x depending on how active your CI is.

The correct approach is to query the deployments endpoint of the Releases API, filtered to your production environment ID and a deploymentStatus=succeeded filter. Count the returned deployments over your measurement window and divide by the number of days. Use a rolling 28-day window rather than calendar months to avoid variance from different month lengths.

Lead Time for Changes

Lead Time for Changes is the most technically complex DORA metric to calculate from Azure DevOps. It requires correlating two data sources: commit timestamps from Azure Repos and release completion timestamps from Azure Pipelines.

The correlation key is the commit SHA. Azure Pipelines release artifacts contain the commit range that was built — specifically, the last commit SHA in the build. Fetch recent commits merged to your main branch from the Repos Commits API, then match each commit to the first successful production release that included it by matching commit SHAs to release artifact references.

Lead Time for a commit equals the release completedOntimestamp minus the commit's committer.date timestamp. Calculate the median across all matched commits for your measurement window — not the mean. Lead time distributions are right-skewed, and a single large refactor with a slow review can inflate the mean while the median stays representative.

Change Failure Rate

Change Failure Rate requires two queries: one for total successful production deployments (your denominator) and one for failed or rollback deployments (your numerator). The Releases API's deploymentStatus=failed filter catches pipeline-level failures. It does not catch deployments that technically succeeded at the pipeline level but caused a production incident.

For complete CFR accuracy, supplement pipeline failure counts with rollback release detection (filtering releases by naming conventions like "rollback" or "hotfix" in the release name) and, if your team logs incidents in Azure Boards or a dedicated incident tool, correlate each deployment with incidents opened in the hour following the deployment.

MTTR

MTTR measures how long it takes to restore service after a production incident. The calculation is the delta between when an incident was detected and when service was restored. In Azure DevOps, the most common implementation uses Azure Boards work items tagged as production incidents — taking the work item creation timestamp as incident start and the next successful production release timestamp as restoration time.

This Azure Boards approach works, but it has accuracy problems. Work item creation timestamps lag behind actual incident start (someone has to create the work item). Resolution timestamps depend on work item state transitions, which different team members handle inconsistently. If your team uses a dedicated incident management tool — PagerDuty, OpsGenie, or incident.io — use that tool's incident open and resolve timestamps for MTTR. They are more accurate and do not require Azure Boards discipline.

Common Mistakes Azure DevOps Teams Make When Measuring DORA

The most damaging measurement errors make your numbers look better than they are. Here are the four mistakes that corrupt DORA data most often in Azure DevOps setups:

Using build counts instead of deployment counts. The Builds API returns every CI run. Always query the Releases API filtered to your production environment.
Not filtering by environment stage. Release pipelines typically have Dev, QA, Staging, and Production stages. Without explicit environment filtering, every release to any stage inflates Deployment Frequency and contaminates Lead Time distributions with pre-production deployments.
Using work item cycle time as a Lead Time proxy. Azure DevOps Analytics exposes work item cycle time — time from Active to Closed state. This ends when the work item is closed, which may happen before or after the code reaches production. DORA Lead Time ends at the production release timestamp.
Counting failed pipeline runs as Change Failure Rate. Pipeline failures include transient infrastructure issues, flaky tests, and configuration errors that have nothing to do with the quality of the code change. CFR should count deployments that caused production degradations, not every failed pipeline run.

How Koalr Automates DORA Metrics for Azure DevOps

Koalr connects to Azure Repos, Azure Pipelines, and Azure Boards through a single OAuth connection and handles all the correlation logic described above automatically. The connection takes about five minutes and backfills 90 days of history from your first login — so your DORA dashboard has trend data from day one, not just a current snapshot.

Koalr's Azure DevOps integration handles the specific calculation problems that trip up manual implementations:

Production environment detection: You map your production stage or YAML environment once during onboarding. Koalr filters all Deployment Frequency and Lead Time calculations to production only — not dev, staging, or QA deployments.
Commit-to-release correlation: Koalr matches commit SHAs from Azure Repos to release artifact references automatically and calculates median Lead Time across all commits in the measurement window.
Change Failure Rate with rollback detection: Koalr combines failed deployment statuses, rollback release name patterns, and optional Azure Boards incident tag correlation to produce accurate CFR without counting transient pipeline failures.
MTTR from your incident tool: Koalr integrates with PagerDuty, OpsGenie, and incident.io directly — using accurate incident timestamps rather than Azure Boards work item state transitions.

Beyond DORA metrics, Koalr adds deploy risk scoring — a 0-100 pre-merge risk score on every PR based on change entropy, file churn, author expertise, and DDL migration detection. This gives your team predictive signal before a deployment reaches the production stage gate, not only retrospective DORA numbers after an incident.

For teams evaluating how Koalr compares with building a custom Azure DevOps DORA implementation, see the Azure DevOps DORA metrics integration page for a detailed breakdown of what native Azure Analytics covers and what requires external tooling.

DORA Metrics for Azure DevOps: A Complete Setup Guide