fourkeys

fourkeys

Let's measure the performance of your team's software delivery.

DORA's Four Keys

Deployment Frequency

• The days of deployments to production per week.
• Elite: Over the last 3 months, the median number is greater than 3.

Lead Time for Changes

• The time for a commit to be deployed into production.
• Elite: Over the last 3 months, the median amount is less than 1 day.

Time to Restore Services

• The time between the which caused the failure and the remediation.
• Elite: Over the last 3 months, the median amount is less than 1 day.

Change Failure Rate

• The number of failures per the number of deployments.
• Elite: Over the last 3 months, the median amount is less than 15 %.

Required

• Google Cloud Run
• Google BigQuery
• Google Secret Manager
• Google Cloud Logging
• Google Cloud Trace
• (optional) Google Cloud Build
• (optional) Google Cloud Scheduler or GitHub Actions

Configuration

• Set GitHub Personal Access Token to Secret Manager as PAT.
  • scope: repo

Build and Deploy

export IMAGE=gcr.io/${PROJECT_ID}/fourkeys
export ENV_VARS=PROJECT_ID=${PROJECT_ID},DATASET_LOCATION=${REGION},DATASET_NAME=fourkeys,GIN_MODE=release

gcloud builds submit --tag ${IMAGE}
gcloud run deploy --image ${IMAGE} --set-env-vars=${ENV_VARS} --update-secrets=PAT=PAT:latest fourkeys

How to fetch activity

export SERVICE_URL=$(gcloud run services describe fourkeys --format 'value(status.url)')
export AUTH_HEADER="Authorization: Bearer $(gcloud auth print-identity-token)"

curl -s -H "${AUTH_HEADER}" ${SERVICE_URL}/_fetch/${OWNER}/${REPOSITORY}/_init
curl -s -H "${AUTH_HEADER}" ${SERVICE_URL}/_fetch/${OWNER}/${REPOSITORY}/pullreqs/update
curl -s -H "${AUTH_HEADER}" ${SERVICE_URL}/_fetch/${OWNER}/${REPOSITORY}/commits
curl -s -H "${AUTH_HEADER}" ${SERVICE_URL}/_fetch/${OWNER}/${REPOSITORY}/events
curl -s -H "${AUTH_HEADER}" ${SERVICE_URL}/_fetch/${OWNER}/${REPOSITORY}/releases
curl -s -H "${AUTH_HEADER}" ${SERVICE_URL}/_fetch/${OWNER}/${REPOSITORY}/pullreqs
curl -s -H "${AUTH_HEADER}" ${SERVICE_URL}/_fetch/${OWNER}/${REPOSITORY}/pullreqs/commits
curl -s -H "${AUTH_HEADER}" ${SERVICE_URL}/_fetch/${OWNER}/${REPOSITORY}/actions/runs
curl -s -H "${AUTH_HEADER}" ${SERVICE_URL}/_fetch/${OWNER}/${REPOSITORY}/actions/jobs

• Google Cloud Scheduler
• GitHub Actions

Query

WITH A AS (
  SELECT
    PERCENTILE_CONT(lead_time, 0.5) OVER() AS lead_time_median
  FROM
    `${PROJECT_ID}.fourkeys._pullreqs_leadtime`
  WHERE
    DATE(updated_at) > DATE_SUB(CURRENT_DATE(), INTERVAL 3 MONTH)
    AND login IN ("itsubaki")
    AND (owner = "itsubaki" AND repository = "fourkeys" AND workflow_name = "deploy")
)

SELECT MAX(lead_time_median)/60 AS hours FROM A

WITH A AS (
  SELECT
    DATE_ADD(DATE(date), INTERVAL - EXTRACT(DAYOFWEEK FROM DATE_ADD(DATE(date), INTERVAL -0 DAY)) +1 DAY) as week
  FROM
    UNNEST(GENERATE_DATE_ARRAY(DATE_SUB(CURRENT_DATE(), INTERVAL 3 MONTH), CURRENT_DATE(), INTERVAL 7 DAY)) date
), B AS (
  SELECT
    DATE_ADD(DATE(date), INTERVAL - EXTRACT(DAYOFWEEK FROM DATE_ADD(DATE(date), INTERVAL -0 DAY)) +1 DAY) as week,
    COUNT(date) as days_per_week
  FROM
    `${PROJECT_ID}.fourkeys._pullreqs_frequency`
  WHERE
    date > DATE_SUB(CURRENT_DATE(), INTERVAL 3 MONTH)
    AND login IN ("itsubaki")
    AND (owner = "itsubaki" AND repository = "fourkeys" AND workflow_name = "deploy")
  GROUP BY week
), C AS (
  SELECT A.week, IFNULL(B.days_per_week, 0) as days_per_week FROM A LEFT JOIN B ON A.week = B.week
), D AS (
  SELECT
    PERCENTILE_CONT(days_per_week, 0.5) OVER() as days_per_week_median
  FROM C
)

SELECT MAX(days_per_week_median) AS days_per_week FROM D

WITH A AS (
  SELECT
    PERCENTILE_CONT(ttr, 0.5) OVER() as ttr_median
  FROM `${PROJECT_ID}.fourkeys._pullreqs_ttr`
  WHERE
    DATE(updated_at) > DATE_SUB(CURRENT_DATE(), INTERVAL 3 MONTH)
    AND login IN ("itsubaki")
    AND (owner = "itsubaki" AND repository = "fourkeys" AND workflow_name = "deploy")
)

SELECT IFNULL(MAX(ttr_median), 0)/60 as hours FROM A

SELECT
  sum(failure) as failure,
  sum(runs) as runs,
  sum(failure) / sum(runs) * 100 as rate
FROM `${PROJECT_ID}.fourkeys._pullreqs_failure_rate`
WHERE
  date > DATE_SUB(CURRENT_DATE(), INTERVAL 3 MONTH)
  AND login IN ("itsubaki")
  AND (owner = "itsubaki" AND repository = "fourkeys" AND workflow_name = "deploy")

Reference

• Are you an Elite DevOps performer? Find out with the Four Keys Project
• GoogleCloudPlatform/fourkeys