eodhp-workspace-services

EO DataHub Workspace Services

An API gateway to the EO DataHub to manage workspace related API requests and process events for workspace management.

Getting Started

Requisites

• Go 1.24 or higher
• Access to a PostgresSQL database
• Apache Pulsar installed locally or access to a Pulsar cluster

Installation

Clone the repository:

git clone git@github.com:EO-DataHub/eodhp-workspace-services.git
cd eodhp-workspace-services

Usage

Configuration File

The config is in the proposed format:

host: {{ENV}}.eodatahub.org.uk
basePath: /api
docsPath: /api/docs/workspace-services
accounts:
  serviceAccountEmail: platform@account-verification.{{ENV}}.eodatahub.org.uk
  helpdeskEmail: enquiries@eodatahub.org.uk
database:
  driver: pgx
  source: postgres://{{.SQL_USER}}:{{.SQL_PASSWORD}}@{{.SQL_HOST}}:{{.SQL_PORT}}/{{.SQL_DATABASE}}?search_path={{.SQL_SCHEMA}}
pulsar:
  url: pulsar://<<REPLACE>>
  topicProducer: persistent://public/default/workspace-settings
  topicConsumer: persistent://public/default/workspace-status
  subscription: workspace-status-sub
keycloak:
  url: "https://{{ENV}}.eodatahub.org.uk/keycloak"
  realm: eodhp
  clientId: eodh-workspaces
aws:
  account: {{AWS_ACCOUNT_ID}}
  cluster_prefix: eodhp-{{ENV}}
  region: eu-west-2
  workspace_domain: workspaces.{{ENV}}.eodhp.eco-ke-staging.com
  s3:
    bucket: workspaces-eodhp-{{ENV}}
    host: s3-accesspoint.eu-west-2.amazonaws.com
    roleArn: arn:aws:iam::{{AWS_ACCOUNT_ID}}:role/WorkspaceServices-{{AWS_CLUSTER_NAME}}
files:
  responseTimeFormat: "2006-01-02T15:04:05Z"
  maxUploadFormMemoryMB: 32
  blockBaseUrl: "http://efs-nginx:80"
  blockTimeoutSeconds: 30
providers:
  airbus:
    access_token_url: https://authenticate.foundation.api.oneatlas.airbus.com/auth/realms/IDP/protocol/openid-connect/token
    optical_contracts_url: https://order.api.oneatlas.airbus.com/api/v1/contracts
    sar_contracts_url: https://sar.api.oneatlas.airbus.com/v1/user/whoami

The config map is defined in eodhp-argocd-deployment app/workspace-services/base/config.yaml

Files configuration:

• files.maxUploadFormMemoryMB: Maximum multipart form memory (in MB) used when parsing upload requests.
• files.responseTimeFormat: Go time layout used to format file timestamps in API responses.
• files.blockBaseUrl: Base URL of the block-store nginx endpoint used for block file operations.
• files.blockTimeoutSeconds: HTTP timeout (in seconds) for block-store requests.

CLI Options

The service has three primary CLI functions:

• API Server (serve)
• Workspace Status Updater (consume)
• Database Reconciler (reconcile)

API Server

This hosts the API endpoints for billing accounts and workspaces. The API documentation can be viewed at https://staging.eodatahub.org.uk/api/docs/workspace-services/index.html

Run this with:

go run main.go serve --config {path-to-config.yaml}

Workspace Status Updater

This listens for workspace status updates from pulsar topic persistent://public/default/workspace-status. It will update the database accordingly.

Run this with:

go run main.go consume --config {path-to-config.yaml}

Database Reconciler

This reconciles workspaces that exist in the database against what exists in the cluster, making sure that the database serves as the source of truth against these resources

Run this with:

go run main.go reconcile --config {path-to-config.yaml}

Local Setup

Docker Development Environment

This repository includes a local Docker Compose setup that runs:

• Postgres
• Pulsar (standalone)
• MinIO (S3-compatible)
• Keycloak (dev realm)
• Workspace Services API

Start the environment:

docker compose up --build

Key files:

• docker-compose.yml
• config/local-docker.yaml
• config/keycloak/realm-eodhp.json
• config/db/seed.sql

Default local endpoints:

• API: http://localhost:8080
• Keycloak: http://localhost:8081
• MinIO Console: http://localhost:9001 (user: minio, pass: minio123)

Expected container state:

• Running: eodhp-workspace-services, eodhp-postgres, eodhp-pulsar, eodhp-minio, eodhp-keycloak
• Exited (one-off jobs): workspace-migrate-1, db-seed-1, minio-init-1, pulsar-init-1

Keycloak dev user:

• username: dev-user
• password: devpass
• client: eodh-workspaces
• client secret: eodh-workspaces-secret

Obtain a token:

curl -X POST http://localhost:8081/realms/eodhp/protocol/openid-connect/token \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=password&client_id=eodh-workspaces&client_secret=eodh-workspaces-secret&username=dev-user&password=devpass"

MinIO bucket (created automatically): workspaces-eodhp-local

Database

To connect to the database, setup an SSH tunnel:

ssh -i <PATH-TO-PRIVATE-KEY> -L 8443:<REMOTE-HOST>:5432 <SSH-USER>@<SSH-HOST> -N

A database proxy EC2 instance is used to connect.

Pulsar

Make sure pulsar is installed. If you run ./pulsar standalone and amend the config file to your localhost, then the app will attach to it.

Deployment

make VERSION={VERSION} dockerpush

Docs

This project uses swaggo/swag to generate API documentation from inline comments in Go code.

Use the following Makefile target to generate and convert the documentation:

make docs

This produces:

• ./docs/swagger.json: API spec in Swagger 2.0 (OpenAPI v2)
• ./docs/swagger.yaml: YAML version of the same
• ./docs/openapi.yaml: OpenAPI v3-compliant version (converted)

The swaggo/swag tool only supports Swagger 2.0 (OpenAPI v2) and does not generate OpenAPI v3 specs natively. If your tooling or downstream consumers require OpenAPI v3, you must use a converter like swagger2openapi to upgrade the spec.

This is handled automatically in the make docs step, but keep in mind that certain OpenAPI v3 features will only be available after conversion.

swagger2openapi must be installed on your system. You can install it via:

npm install -g swagger2openapi