README
¶
= Udash
An Updatecli Dashboard
IMPORTANT: This project is still experimental, feel free to share feedback
== Description
Udash is an Updatecli backend application. Its mission is to visualize Updatecli pipeline report and to provide various insights such as Git repositories "update" state.
**Pipeline Report**
Updatecli can be configured to upload pipeline reports after each execution. The goal is to have a central place to visualize all pipelines for a specific project.
**Insight**
Insight reuse pipeline report to provide a visualization per theme such as the state for all dependencies for a specific git repository.
__Still work in progress__
== Demo
The best way to discover Udash is to try it yourself. There is a docker compose example available in the demo directory.
Deploy Udash with the following steps:
1. Make sure you have Docker and Docker Compose installed.
2. Run `docker compose up -d` in the directory `demo`.
3. Configure your browser to access Udash at `http://localhost`. Traefik serves the frontend on
port 80 and the API on `http://localhost/api`, its own dashboard is the one on port 8080.
4. Run `updatecli udash login "http://localhost" --experimental` to configure Updatecli to upload reports to Udash.
5. Then you can run any updatecli command (apply/diff) to start publishing reports to Udash
The demo runs with authentication disabled. `udash login` notices this, skips the token prompt,
and simply records the endpoint in the Updatecli configuration file.
Please be aware that the UI is designed to visualize pipelines per git repository, so without an
`scmid` pipelines will be hard to discover.
Don't worry if you don't have any Updatecli manifest yet, you can still try the demo by using the Updatecli policies used by this Git repository.
For that you need Updatecli and GitHub access token to interact with the GitHub API.
You can run the following commands to configure Updatecli to use the policies defined in this repository:
1. `export GITHUB_TOKEN=<your-github-token>`
2. `export GITHUB_ACTOR=<your-github-username>`
3. `docker login ghcr.io` to authenticate on the GitHub Container Registry for pullling Updatecli policies
4. `updatecli compose diff --experimental` to execute Updatecli policies defined in the file `updatecli-compose.yaml`
== Architecture
=== Requirements
Udash requires a postgresql database to store the various pipeline reports. An oauth provider is
only required when authentication is enabled, which it is not by default.
**Postgresql Database**
**OAuth provider**
The oauth provider **must** allow the PKCE flow.
* link:https://zitadel.com[Zitadel] (the reference deployment, and the provider behind the dedicated `zitadel` mode)
* Auth0 (tested)
* GitHub link:https://docs.github.com/en/apps/oauth-apps/building-oauth-apps/authorizing-oauth-apps#1-request-a-users-github-identity[Not supported yet]
* link:https://docs.gitlab.com/ee/api/oauth2.html[GitLab]
* link:https://github.com/dexidp/dex/issues/2244[Dex]
* link:https://www.keycloak.org/docs/latest/securing_apps/#_installed_adapter[Keycloak]
=== Udash API
==== Documentation
The api documentation is available at `/swagger/index.html` or in the docs directory.
==== Option
Udash must be configured via a configuration file, and some settings can be overridden by environment variables
**Config File**
The configuration file is named `config.yaml` and is looked up, in order, in the working directory,
then `$HOME/.udash/`, then `/etc/udash/`. The first one found wins. `--config` takes the path of a
different file instead, which then must exist.
```yaml
server:
auth:
# mode selects how incoming tokens are validated.
# Accepted values are "oidc", "zitadel", and "none".
# Unset or "none" disables authentication entirely.
# An unrecognised value stops the server rather than serving an open API.
mode: "oidc"
# visibility controls which endpoints require a token.
# "public" (the default) leaves the read endpoints open and requires
# authentication for anything that writes.
# "private" requires authentication everywhere.
visibility: "public"
# oidc settings, used when mode is "oidc".
# Tokens are verified locally against the issuer signing keys, so this mode
# only accepts JWT access tokens.
oidc:
# issuer is compared to the "iss" claim of the token, verbatim.
# A scheme is optional, https is assumed when it is omitted, but the
# trailing slash is significant: Auth0 issues one, Zitadel and Keycloak
# do not. A mismatch rejects every token.
issuer: "https://example.eu.auth0.com/"
# audience is a list, and every entry is accepted.
audience:
- "https://udash.example/api"
# zitadel settings, used when mode is "zitadel".
# Tokens are validated by introspection, which also accepts opaque ones such
# as Zitadel personal access tokens.
zitadel:
domain: "xxx.region.zitadel.cloud"
# keyfile is the path to a service account key file
keyfile: "/etc/udash/zitadel-key.json"
# roles maps the roles carried by a token onto Udash permissions.
roles:
# claim is the token claim holding the identity provider roles. It defaults
# to Zitadel's claim in "zitadel" mode and must be set otherwise.
# Both shapes are accepted: an object keyed by role name, as Zitadel emits,
# and an array of strings, as Keycloak and Auth0 emit.
# Zitadel: "urn:zitadel:iam:org:project:roles"
# Keycloak: "realm_access.roles"
# Auth0: "https://udash/roles"
claim: "realm_access.roles"
# mapping lists, per permission, the provider roles granting it.
mapping:
admin: ["udash.admin"]
publisher: ["udash.publisher"]
viewer: ["udash.viewer"]
# default is granted to an authenticated identity matching no role at all.
# It is deliberately the least privileged one: without it, everybody who can
# sign in could publish reports and mint API tokens.
default: "viewer"
# resolver decides how the permission behind an Udash API token is resolved,
# since such a request carries no provider token to read roles from.
# "zitadel" asks Zitadel for the current grants, so revoking a role takes
# effect on tokens created before it. It requires mode "zitadel", and the
# service user behind keyfile must be allowed to read user grants.
# "snapshot" trusts the permission recorded when the token was created, and
# is the only option for other providers. Offboarding somebody then means
# deleting their tokens.
resolver: "snapshot"
# cacheTTL is how long a resolved permission is reused.
cacheTTL: "60s"
database:
# uri defines the postgresql URI used to connect with its database
uri: "postgres://udash:password@db:5432/udash?sslmode=disable"
# migrationdisabled skips the schema migrations run at startup
migrationdisabled: false
# connecttimeout is how long to wait for the database at startup
connecttimeout: 2m
gc:
# maxHistoryDays is how many days of reports are kept, see "Data retention".
# 0, the default, keeps everything.
maxHistoryDays: 0
# interval is the time between two garbage collections run by the server.
interval: "24h"
# batchSize is the maximum number of reports deleted by a single statement.
batchSize: 5000
```
**Environment**
Each variable below is only a fallback: it is read when the matching key is absent from the
configuration file, so the file always wins.
* **UDASH_AUTH_MODE**: Authentication mode. Accepted values are ["", "none", "oidc", "zitadel"]
* **UDASH_AUTH_OIDC_ISSUER**: OIDC issuer URL, requires `UDASH_AUTH_MODE` set to "oidc"
* **UDASH_AUTH_OIDC_AUDIENCE**: OIDC audience, requires `UDASH_AUTH_MODE` set to "oidc"
* **UDASH_AUTH_ZITADEL_DOMAIN**: Zitadel domain, requires `UDASH_AUTH_MODE` set to "zitadel"
* **UDASH_AUTH_ZITADEL_KEYFILE**: Path to the Zitadel service account key file, requires `UDASH_AUTH_MODE` set to "zitadel"
* **UDASH_AUTH_ROLES_CLAIM**: Token claim holding the identity provider roles
* **UDASH_AUTH_ROLES_DEFAULT**: Permission granted to an identity matching no role
* **UDASH_AUTH_ROLES_RESOLVER**: How an API token's permission is resolved ["zitadel", "snapshot"]
* **UDASH_DB_URI**: Define the postgresql URI
* **UDASH_GC_MAX_HISTORY_DAYS**: Days of reports to keep, `0` keeps everything
* **UDASH_GC_INTERVAL**: Time between two garbage collections run by the server, such as `24h`
* **UDASH_GC_BATCH_SIZE**: Maximum number of reports deleted by a single statement
==== Data retention
By default Udash keeps every report forever. Setting `gc.maxHistoryDays` enables a garbage
collector, run by the server a minute after it starts and then every `gc.interval`, which deletes:
* every report older than `gc.maxHistoryDays`, including the last report of a pipeline which
stopped running, so that pipeline disappears from the dashboard;
* then the scms, labels, and source, condition and target configs which no remaining report
references anymore, and which were not used within the retention period either.
When several Udash instances share a database, a postgres advisory lock ensures only one of them
collects at a time.
The same garbage collection can be run once from the command line, for instance from a scheduled
job, with `--dry-run` to only count what would be deleted. It reads the same configuration file and
environment variables, and `--max-history-days` overrides the retention. Unlike the server, it never
migrates the database schema, so it must run against a database the server already migrated:
```shell
udash gc --config config.yaml --max-history-days 90 --dry-run
```
The frontend `MAX_HISTORY_DAYS` should not exceed `gc.maxHistoryDays`, or the date filter would
offer a range with no report left in it.
==== Permissions
Authorization has two axes: what a *person* may do, and what a given *token* may do.
Permissions come from the identity provider roles, mapped by `server.auth.roles.mapping`:
[cols="1,3"]
|===
| Permission | Grants
| `viewer` | read pipeline reports
| `publisher` | publish pipeline reports, and create API tokens
| `admin` | everything, plus managing any identity's tokens
|===
Token scopes are chosen when a token is created and can never exceed what its creator is
allowed to do: `reports:read` and `reports:write`. There is deliberately no scope for
managing tokens, so a token can never mint another one.
==== API tokens
An access token from an identity provider always expires, while an unattended pipeline needs
a credential it can keep. Udash therefore issues its own tokens, validates them itself, and
lets them live forever unless an expiry is set.
They are created from **Profile ▸ Tokens** in the frontend, by anybody with the `publisher`
permission, and shown exactly once — only a sha256 of the token is stored. They are prefixed
`udash_pat_` so they can be told apart from a provider token, and recognised by secret
scanners if one ever leaks.
Point Updatecli at one with either:
```bash
updatecli udash login --experimental https://udash.example # prompts for the token
export UPDATECLI_UDASH_ACCESS_TOKEN="udash_pat_..." # for CI
```
==== Non-expiring tokens without the frontend
In `zitadel` mode, tokens are validated by introspection, which accepts opaque tokens. A
Zitadel **personal access token** on a machine user therefore works as a permanent
credential with no Udash-side setup at all:
1. In Zitadel, create a *service user*, grant it the project role mapped to `publisher`
(`udash.publisher` by default), and create a personal access token leaving the expiration
field empty.
2. Set `UPDATECLI_UDASH_ACCESS_TOKEN` to it in CI.
The trade-offs against an Udash API token: every request costs an introspection round-trip to
Zitadel, and minting one needs Zitadel administrator rights, so it does not scale to letting
each team issue their own.
=== Udash Frontend
==== Option
Even though the Udash frontend is a client-side javascript application, it is configured entirely
at runtime through a single `config.json`, served next to the application at
`/usr/share/nginx/html/config.json`. The page fetches it before loading the bundle, so the same
image serves an open deployment and an authenticated one without a rebuild.
```json
{
"AUTH_ENABLED": false,
"OAUTH_DOMAIN": "https://your-instance.zitadel.cloud",
"OAUTH_CLIENTID": "86FVLxxxxxxxxxxxxxxxxxx",
"OAUTH_SCOPE": "openid profile email offline_access urn:zitadel:iam:org:project:id:PROJECT_ID:aud",
"OAUTH_AUDIENCE": "https://app.updatecli.io/api",
"API_BASE_URL": "/api",
"APP_BASE_PATH": "/",
"MAX_HISTORY_DAYS": 30
}
```
* **AUTH_ENABLED**: Require authentication. Defaults to `false`.
* **OAUTH_DOMAIN**: The provider issuer URL.
* **OAUTH_CLIENTID**: The client ID of the SPA application.
* **OAUTH_SCOPE**: Requested scopes. Defaults to `openid profile email offline_access`, where
`offline_access` is what enables silent token renewal. Zitadel additionally requires the project
audience scope `urn:zitadel:iam:org:project:id:<PROJECT_ID>:aud`.
* **OAUTH_AUDIENCE**: Not used by the frontend itself. It is read by Updatecli, see below.
* **API_BASE_URL**: Where the browser reaches the API. Relative (`/api`) for same-host routing, or
an absolute URL when the API lives on its own domain. Defaults to `/api`.
* **APP_BASE_PATH**: Base path of the SPA, for mounting it below a subpath such as `/udash/`.
Defaults to `/`.
* **MAX_HISTORY_DAYS**: How far back the date filter and the activity chart may reach. Defaults to
`30` and is capped at the API's own maximum of `366`.
`config.json` is also what Updatecli reads to discover the oauth settings when running
`updatecli udash login`. It fetches `<url>/config.json` and takes `OAUTH_DOMAIN`, `OAUTH_CLIENTID`,
and `OAUTH_AUDIENCE` from it, so an authenticated deployment has to publish `OAUTH_AUDIENCE` there
even though the frontend never reads it.
=== Updatecli
Updatecli is expected to run as usual from CI environment.
But it must know where to publish before uploading any reports, by running:
`updatecli udash login "https://app.updatecli.io" --experimental`
Then any apply/diff command will upload pipeline reports, as long as it is also run with
`--experimental`. Without that flag the upload is skipped silently.
Against a deployment with authentication enabled, pass at least one oauth flag so the command runs
the PKCE flow rather than just recording the endpoint:
`updatecli udash login --oauth-clientId "<client id>" "https://app.updatecli.io" --experimental`
The remaining oauth settings are then discovered from `<url>/config.json`.
`--api-url` sets the API endpoint the CLI publishes to, defaulting to `<url>/api/v1beta1`, the
base every pipeline route is served under. Set it when the API is not served next to the frontend,
or set `UPDATECLI_UDASH_API_URL`.
=== Links
* https://github.com/updatecli/updatecli[Updatecli]
* https://github.com/updatecli/udash-front[Udash Frontend]
* https://github.com/updatecli/charts[Helm Chart]
Documentation
¶
There is no documentation for this package.
Click to show internal directories.
Click to hide internal directories.