Change Failure Rate
Change Failure Rate (CFR) measures the percentage of deployments to production that result in a degraded service requiring remediation — whether through a hotfix, rollback, patch, or incident response. It is a DORA stability metric and the primary indicator of deployment quality.
Formula
Change Failure Rate = (Deployments causing incidents ÷ Total deployments) × 100 A deployment "causes" an incident if: - An incident is opened within a configurable window (default: 2 hours) after the deployment - The deployment is explicitly linked to an incident via postmortem or manual tagging - A rollback or hotfix deployment follows within the window Example: 3 incidents linked to deployments out of 60 total deployments = 5% CFR Target: Elite teams maintain CFR below 5%; most teams should target below 15%.
Industry Benchmarks
| Performance Level | Benchmark (percentage of deployments) |
|---|---|
| Elite | 0–5% |
| High | 5–10% |
| Medium | 10–15% |
| Low | Over 15% |
Benchmarks from the 2023 State of DevOps Report (DORA). High deployment frequency and low change failure rate are not in tension — elite teams achieve both simultaneously by deploying smaller, more tested changes.
Data Sources
- Deployment events from Vercel, Railway, Netlify, Render, Fly.io, AWS CodeDeploy
- Incident data from PagerDuty or incident.io (incident open/resolve timestamps)
- GitHub: rollback commits and revert PRs following a deployment
- Koalr deploy risk scores — high-risk deploys that result in incidents tighten the outcome correlation model
Why Change Failure Rate matters
Change Failure Rate is the quality check on deployment frequency. A team deploying 20 times per day with a 30% CFR is creating more production incidents than a team deploying once per week with a 2% CFR. CFR reveals whether your deployment pipeline has adequate test coverage, review processes, and rollback capability. It is also a leading indicator of engineering morale: teams with chronically high CFR spend significant time firefighting rather than building, which compounds into burnout and attrition. Improving CFR requires identifying which types of changes (large diffs, low coverage, untested paths) are most correlated with failures.
Common measurement mistakes
Only counting rollbacks and missing hotfixes. A hotfix deployment is a change failure — it is remediation for a broken change. Including hotfixes and patches alongside rollbacks gives a more accurate CFR than rollbacks alone.
Using too short a correlation window. An incident triggered 4 hours after a deployment is often caused by that deployment, especially for async processing systems, scheduled jobs, or services with gradual traffic ramp. Koalr defaults to 2 hours but recommends adjusting this to your typical incident detection latency.
Not excluding deployments that failed before reaching production. A deployment that fails in CI or fails to start in production is not a "change failure" in the DORA sense — the change never reached users. Only successfully deployed changes that subsequently cause incidents count.
How Koalr measures Change Failure Rate
Koalr correlates deployment events with incident timelines from PagerDuty or incident.io. If an incident opens within 2 hours of a production deployment (configurable), Koalr flags that deployment as a contributing factor and increments the CFR numerator. Koalr also detects rollback deployments (commits that revert a previous merge) via GitHub and counts those automatically. Each incident-deployment correlation is reviewable in the deploy risk dashboard, where engineers can confirm or dismiss the correlation.
Related DORA metrics
See your Change Failure Rate in Koalr
Connect GitHub and your deployment platform to start tracking all four DORA metrics automatically. No manual data entry, no spreadsheets.