github-action-metrics

GitHub Action Metrics

Analyze your GitHub Actions!

Do you wonder

• why PRs take so long to be checked by your GitHub actions?
• how often PRs fail to pass a certain workflow, job or step?
• if caching dependencies did make a job run faster?

These are just a fraction of questions you can find answers to using this project 😊

Architecture

This project provides a CLI which is a small wrapper around GitHub's Octokit library. It will fetch GitHub action data from https://docs.github.com/en/rest/reference/actions and store it in a place of your choice. You can for example store it in a Git repository on GitHub itself.

You can then index the data into Elasticsearch. Create visualizations (graphs, metrics, dashboards, ...) to answer a lot of questions you have about your use of GitHub actions.

Elasticsearch and Kibana

Initial Setup

Copy the kibana config

cp ./elk/config/kib01.example.yml ./elk/config/kib01.yml

Start Elasticsearch and Kibana

docker compose up

Follow Create passwords for built-in users by executing

docker exec -it es01 ./bin/elasticsearch-setup-passwords auto --batch

Store passwords somewhere safe and change the elasticsearch.password,

Run

docker exec -it kib01 ./bin/kibana-keystore create
docker exec -it kib01 ./bin/kibana-keystore add elasticsearch.password

Add the kibana_system password when prompted.

Restart

docker compose restart

You should now be able to login using user elastic and the password that was auto-generated.

Index Data

Once you fetched GitHub workflow data and setup Elasticsearch and Kibana you can index it using

gham index all \
    --url http://localhost:9200 \
    --workflow-id 10954 \
    --source ~/metrics/data \
    --user elastic --password $(password-manager get elasticsearch-password)

Create kibana index patterns

./elk/kibana_index_pattern.sh elastic --password $(password-manager get elasticsearch-password)

Example Project

I started this project to analyze the test workflow we use at DHIS2. I wanted to know where time was spent, why some test runs took 15min while others took 23min to finish. How could we get faster feedback on PRs and reduce this variation in test duration?

Using this project I

• fetched GitHub action data every day using a scheduled GitHub action and stored it in a GitHub repository
• indexed and analyzed the data using Elasticsearch and Kibana