GitHub Actions and DORA Metrics: The Complete Integration Guide

The core problem: not all workflow runs are deployments

The instinct when first integrating GitHub Actions with a DORA metrics tool is to count workflow runs. It is the obvious data source — every pipeline execution is recorded, timestamped, and associated with a commit SHA. The problem is that the vast majority of workflow runs are not deployments, and treating them as such produces numbers that bear no resemblance to your actual delivery cadence.

Consider what a typical repository contains in its GitHub Actions history:

• Test workflow runs triggered on every pull request branch push — not deployments.
• Lint and typecheck runs on every commit — not deployments.
• A deployment workflow run that targets staging — not a production deployment.
• A deployment workflow run to production that failed halfway through — not a successful deployment.
• A scheduled workflow that runs database cleanup every night — not a deployment at all.

Of all the workflow runs in a repository on a given day, the subset that represents a production deployment is typically small — often one or two runs out of dozens or hundreds. If you count all successful workflow runs as deployments, your deployment frequency metric becomes meaningless. Teams that make this mistake consistently report deployment frequency numbers that are 5–20× higher than their actual production deployment rate.

The correct filter requires two conditions to both be true: the workflow run must have conclusion: success, and it must target a production environment. Getting that second condition right requires understanding how GitHub models deployments.

GitHub's deployment data model

GitHub has a distinct concept for deployments that is separate from workflow runs. Most teams do not use it explicitly — they just write a deploy.yml workflow and consider it done. But understanding the formal model is what makes accurate DORA measurement possible.

GitHub Deployments API

The GitHub Deployments API (POST /repos/{owner}/{repo}/deployments) creates a formal deployment record associated with a specific commit SHA. Deployment records carry an environment field — typically production, staging, or development. For DORA purposes, only deployments with environment: production count.

When a GitHub Actions workflow job has environment: production in its configuration, GitHub automatically creates a Deployment record via this API when the job runs. This is the cleanest integration path — the deployment record is created as a first-class object, queryable independently of the workflow run that created it.

Deployment statuses

Every deployment record has an associated status stream. Status updates are posted via POST /repos/{owner}/{repo}/deployments/{deployment_id}/statuses as the deployment progresses. The relevant states are:

• in_progress — deployment is running
• success — deployment completed successfully
• failure — deployment failed
• error — deployment encountered an unexpected error

For deployment frequency, you count deployments where the final status is success. A deployment record with a final status of failure or error does not count — it represents an attempted deployment that did not reach production.

Key API endpoints

These are the four GitHub API endpoints that power DORA metric calculation from GitHub Actions data:

The on: push distinction

Workflows triggered by on: push are not the same as deployments, even if they happen to include a deployment step. A push event fires on every branch push — including feature branches, draft PRs, and automated commits. To determine whether a push-triggered workflow run actually deployed to production, you need to check either whether the workflow job had an environment: production declaration (which creates a Deployment record) or whether the workflow name matches a deployment-specific naming convention.

The 3 GitHub Actions deployment patterns

GitHub teams implement deployment pipelines in meaningfully different ways. Koalr recognises three distinct patterns and handles each differently to ensure accurate DORA data regardless of how your pipeline is structured.

Pattern 1: GitHub Environments + Deployment API (recommended)

The cleanest integration. The workflow job configuration includes environment: production:

When the job completes, GitHub automatically creates a Deployment record with environment: production and a final status of success or failure. Koalr reads GET /repos/{owner}/{repo}/deployments?environment=production to retrieve these records directly, without any workflow name matching.

Lead time calculation (Pattern 1): The deployment record includes the triggering commit SHA in the deployment.sha field. Koalr uses GET /repos/{owner}/{repo}/compare/{previous_deploy_sha}...{current_deploy_sha} to retrieve all commits between consecutive deployments, then takes the earliestcommitter.date as the start of lead time and deployment.created_at as the end. For PR-based workflows, pull_request.created_at (or the first commit timestamp) is used instead of the raw committer date.

Pattern 2: Workflow-based deployment (no Environments)

Many teams deploy via GitHub Actions without using the formal Environments feature. The workflow exists and runs, but no Deployment record is created automatically. In this case, Koalr detects deployments by matching workflow names against configurable patterns.

Common production deployment workflow name patterns Koalr recognises by default: deploy-production, production-deploy, deploy.yml, release.yml, deploy-prod. These are configurable per repository in Koalr's settings.

For matched workflows, Koalr reads GET /repos/{owner}/{repo}/actions/workflows/{workflow_id}/runs?status=success to retrieve successful runs. The deployment timestamp is taken from the updated_at field on the run object (the completion time), not created_at (the trigger time). Using start time instead of end time is one of the most common DORA timestamp mistakes — it understates lead time by the duration of the deployment itself.

This pattern is less reliable than Pattern 1. It requires discipline around workflow naming, and it produces false positives if a staging workflow happens to match a production naming pattern. If you are setting up a new repository, using GitHub Environments (Pattern 1) is always preferable.

Pattern 3: External deployment tool with GitHub Actions trigger

The third pattern is common in teams that use Vercel, Railway, ArgoCD, or similar platform-layer deployment tools that handle the actual deployment mechanics. In this setup, GitHub Actions provides the CI pipeline (build, test, push image) but the deployment event itself is fired by the external platform.

For these teams, Koalr reads deployment events from the target platform rather than from GitHub Actions. Vercel's deployment webhook, Railway's deployment API, and ArgoCD's sync events all provide a production deployment timestamp independently of the GitHub Actions workflow. GitHub Actions data still contributes to lead time calculation — specifically the CI duration from trigger to build completion — but it is not the source of the deployment event itself.

Computing each DORA metric from GitHub Actions

Deployment Frequency

Deployment frequency is the count of successful production deployments per day or per week. The query is straightforward once you have the right data source:

• Pattern 1 teams: paginate GET /repos/{owner}/{repo}/deployments?environment=production&per_page=100, filter to records where the latest deployment status is success, group by date.
• Pattern 2 teams: paginate successful runs of the matched deployment workflow, group by date of updated_at.

For monorepos where one workflow deploys multiple services, each service deployment is counted independently (see the monorepo section below). The service key is the combination of repository and deployment environment name (e.g., production-api, production-web).

DORA elite performance benchmark: four or more deployments per day per service. High performance: one deployment per day to one per week. Medium: one per week to one per month. Low: fewer than one per month.

Lead Time for Changes

Lead time measures how long it takes for a committed change to reach production. The calculation requires resolving the set of commits included in a given deployment:

1. Identify deployment.sha (the HEAD commit of the deployment) and the SHA of the previous successful production deployment.
2. Call GET /repos/{owner}/{repo}/compare/{prev_sha}...{current_sha} — this returns the full list of commits between the two deployment points.
3. For each commit, take commit.committer.date (or pull_request.created_at for PR-based workflows) as the "change started" timestamp.
4. The lead time for a given deployment is: deployment.created_at minus the committer.date of the earliest commit in the range.

For teams that work through pull requests, using pull_request.created_at as the start point is more semantically accurate than committer.date. A commit authored on Monday but not opened as a PR until Thursday has a different lead time story than one opened immediately. Koalr uses the PR creation timestamp when PR data is available and falls back to commit timestamp when it is not.

Change Failure Rate

Change failure rate (CFR) is the percentage of deployments that result in a production incident, rollback, or hotfix. GitHub Actions alone cannot compute CFR in isolation — you need to connect deployment events to failure signals. Koalr uses two approaches:

• Deployment failure pattern: A DeploymentStatus.state == 'failure' followed within 2 hours by a new success deployment (rollback pattern) is flagged as a change failure.
• Incident correlation: If PagerDuty or incident.io is connected, incidents that open within 2 hours of a deployment.created_at timestamp are attributed to that deployment. Attributed incidents count as change failures.
• Hotfix pattern: A revert commit (matching Revert "..." in the commit message) merged within 4 hours of a deployment is treated as a change failure.

MTTR

Mean time to restore cannot be computed from GitHub Actions data alone. MTTR measures the time from when an incident starts to when it is resolved — and GitHub has no concept of an incident. Koalr computes MTTR by connecting GitHub deployment timestamps with incident lifecycle data from PagerDuty or incident.io:

MTTR = incident.resolved_at - incident.created_at for incidents attributed to a deployment.

If neither PagerDuty nor incident.io is connected, Koalr can use the rollback detection pattern as a proxy — the time from the failing deployment to the succeeding rollback deployment — but this is a rough approximation and understates MTTR for incidents that required diagnosis time before rollback.

The GitHub Actions check run signal

Beyond the four DORA metrics, GitHub Actions provides a particularly valuable deploy risk signal through check runs. GET /repos/{owner}/{repo}/commits/{ref}/check-runs returns all check runs for a given commit — each CI job that ran, its conclusion, and its duration.

Koalr uses check run data to compute two signals that feed into the deploy risk model:

• CI flakiness rate: A check run that alternates between success and failure conclusions across retries on the same commit is classified as flaky. High flakiness rates are a leading indicator of test suite quality degradation and, empirically, correlate with higher change failure rates. Koalr surfaces CI flakiness rate as a deploy risk input (Signal #28 in the 32-signal model).
• CI duration trend: If average CI duration increases 20% or more week-over-week, it signals test debt accumulation — the suite is growing slower without a corresponding increase in coverage or reliability. This is tracked as a leading indicator rather than a direct risk signal.

Both signals are visible in Koalr's GitHub integration dashboard alongside the DORA metrics, giving engineering managers a fuller picture of CI health than deployment counts alone provide.

Setting up GitHub Actions in Koalr

GitHub is the core integration in Koalr — required for PR metrics, CODEOWNERS sync, and deploy risk scoring. DORA metrics from GitHub Actions require no additional authentication beyond the GitHub App connection used for PR data.

Once GitHub is connected, Koalr automatically begins reading the Deployments API for all connected repositories. No additional workflow configuration is required if you are already using GitHub Environments (Pattern 1). Configuration steps:

1. Connect GitHub via Settings → Integrations → GitHub using the Koalr GitHub App.
2. Select the repositories to track — Koalr requests read access to deployments, check-runs, and workflow runs.
3. Set the environment name that represents production. The default mapping is production. If your team uses a non-standard name (prod, live, main-env, etc.), configure the mapping in Settings → GitHub → Environment Mapping.
4. If you use Pattern 2 (workflow name matching), set the production workflow name patterns in Settings → GitHub → Deployment Workflows.

Historical backfill covers the previous 90 days of deployment data. DORA metrics are available in the dashboard within minutes of completing the setup.

Common GitHub Actions DORA mistakes

These are the errors Koalr most frequently encounters when teams connect GitHub Actions data for the first time and find their metrics look wrong:

Monorepo handling

Monorepos present a specific challenge: a single GitHub Actions workflow run may deploy multiple independent services simultaneously, using a matrix strategy. If you count that workflow run as a single deployment, you lose per-service DORA visibility entirely.

Koalr detects per-service deployments from monorepo workflows using two approaches:

• Matrix strategy variable: If the workflow uses strategy: matrix: service: [api, web, worker], each matrix job creates a separate deployment record (when using GitHub Environments) or a separate workflow run job. Koalr reads the environment field per job rather than per workflow run to identify the service.
• Environment name suffix: Teams that name their environments production-api, production-web, and production-worker get automatic service separation in Koalr — each environment is tracked as a distinct service for deployment frequency, lead time, and CFR purposes.

For teams using Pattern 2 (workflow name matching without GitHub Environments), Koalr looks for the service identifier in the workflow file name or in a configurable job output variable. This requires more explicit configuration than the environment-based approach but supports the common monorepo pattern of a single deploy.yml with a service parameter.

AI Chat queries on GitHub Actions data

Once GitHub Actions data is connected, Koalr's AI chat panel can answer natural language questions against the live dataset. Some of the most useful queries engineering managers run:

The AI chat panel has access to the full deployment history, PR timeline, and check run data for all connected repositories. Queries that would require custom SQL or a dashboard build in other tools are answered in seconds against live data.

Putting it all together

Accurate DORA metrics from GitHub Actions require getting four things right: using the Deployments API rather than raw workflow runs as your data source; filtering to the production environment specifically; using workflow completion time rather than start time as the deployment timestamp; and resolving the commit-to-deployment range correctly for lead time calculation.

The most common failure mode is skipping step one — counting all workflow runs and then wondering why deployment frequency is reporting 47 deploys per day for a team that ships once or twice a week. The Deployments API, combined with GitHub Environments, gives you a clean, authoritative record of production deployments that does not require any heuristic filtering or workflow name matching.

GitHub Actions also provides signal beyond the four DORA metrics. CI flakiness rates, build duration trends, and check run failure patterns are all available from the same integration and feed into Koalr's deploy risk model. A team with high DORA performance but a flaky CI suite is accumulating hidden risk — and that risk shows up in the deploy risk scores before it shows up in the change failure rate.