tibuild

module
v0.0.0-...-63c78a4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 20, 2026 License: MIT

README

TiBuild

It provide the CD functions for PingCAP, Welcome bros!

  • trigger Dev-build runs for custom requirement.
  • publish/deliver artifacts.

Publish domain

TiUP package publishing and the TiDB Cloud ops/TCMS integrations live here now (formerly in publisher). The queue is publish_unit + publish_step, guarded by a single-row lane that keeps the two TiUP mirrors from publishing at the same time; PARTIAL publications need an operator and are resumed explicitly.

See docs/publish.md for the model, targets, failure handling and the legacy /tiup/* contract, and docs/nightly.md for the nightly desired-state schedule (what maintainers should expect, and the operator runbook).

Technologies

Quick Start

Design the backend server

go to internal/service/design/design.go and update it.

Design the database

go to internal/database/schema/ and update them.

Generate code from design

we need the goa tool

run:

go generate -x ./...
Run it

Please prepare a config.yaml file before start or you can passing the --config option when run the server.

go run ./cmd/server

After waiting a few seconds, application is available and can be visited in the browser:localhost:8080

Worker deployment model

TiBuild v2 runs two binaries:

Binary Role State
cmd/tibuild HTTP API (Goa), stateless 2+ replicas
cmd/tibuild-worker image-sync executor, lease-based 1+ replicas per target set

The API no longer executes image sync tasks. Every environment runs the standalone image exp-tibuild-worker (built from ./cmd/tibuild-worker); the chart defaults worker.enabled=true. Rollback is a chart/image rollback, not an in-process fallback.

Queue semantics

image_sync_tasks is the queue. A claim sets claimed_by, lease_until, available_at, and increments attempts in one guarded update. The worker renews the lease with a heartbeat; every result write requires the same claimed_by token, so an expired owner cannot overwrite the next owner's result. A reaper returns rows whose lease_until has passed to PENDING.

Failures retry with exponential backoff (10s, 20s, ... capped at 5min, ±20% jitter) and become FAILED after worker.max_attempts.

Configuration
worker:
  id: ""                    # defaults to the hostname
  targets: ["*"]            # registry hosts this worker claims
  concurrency: 4            # polling loops
  poll_interval: "10s"
  lease: "90s"
  heartbeat: "30s"
  max_attempts: 3
  kinds: ["image_copy"]
  metrics_interval: "15s"   # task stats / alert evaluation period
  reap_interval: "30s"      # expired-lease reclaim period
  backoff_base: "10s"       # first retry delay (doubles per attempt)
  backoff_max: "5m"         # retry delay cap
  alerts:
    enabled: true           # false keeps metrics but stops alert delivery
    backlog_after: "10m"
    oldest_pending_after: "30m"
    lease_reclaim_window: "5m"
    dedupe: "1h"

The observability keys default to the values that used to be hard-coded, so an untouched config behaves exactly as before. metrics_interval, reap_interval, the retry backoff and the alert thresholds are re-read on every tick, so a config reload applies them without a restart; the claim loop interval (poll_interval), concurrency, kinds, targets and the lease settings need a rollout. docs/config-reload.md lists every hot key, the restart-only keys, the reload metrics and the rollback procedure.

worker.targets must be a subset of the global target catalogue; an unknown target fails fast at startup. Until the target catalogue lands in pkg/delivery, only "*" is accepted.

Metrics and alerts

The worker serves /metrics on its health port (8081):

  • publish_task_backlog{target}
  • publish_task_oldest_pending_seconds{target}
  • publish_task_inflight{target}
  • publish_task_lease_expired_total
  • publish_task_failed_total{target,reason}

The config watcher adds config_reload_total{result}, config_generation, config_last_reload_timestamp_seconds and config_restart_required_total{field} on the same endpoint; see docs/config-reload.md.

Alerts reuse the Lark notifier and fire when the backlog has been non-empty for worker.alerts.backlog_after (default 10m), when the oldest PENDING task is older than oldest_pending_after (default 30m), or when a lease was reclaimed within lease_reclaim_window (default 5m). Repeats of the same condition are suppressed for dedupe (default 1h). worker.alerts.enabled: false stops alert delivery but keeps the metrics above refreshing. Configure lark.channels to give the alerts recipients.

Alert evaluation itself is elected through the alert_leases table: the workers of all clusters share one queue, so only the lease holder calls the notifier. The lease TTL is twice metrics_interval, and an evicted holder loses the election on the next tick instead of double-sending. A worker without lark.channels does not compete for the lease, so a cluster that has no recipients configured cannot take the lease and silence the others.

Changing recipients, forcing an External Secrets sync and verifying delivery is documented in docs/worker-alerts.md; scraping the metrics per cluster is in docs/worker-metrics.md (prod2 uses the chart's worker.podMonitor, tke/gcp have no CRDs).

Testing

go test ./...

Integration tests (e.g. internal/service/tests/artifact_test.go) spin up a real registry via testcontainers, so they require a running Docker daemon and are skipped with go test -short ./....

The multi-worker zero-duplicate gate uses a MySQL container and is opt-in:

TESTCONTAINERS_RYUK_DISABLED=true go test -tags=integration \
  ./internal/service/impl -run TestWorkerMultiReplicaAndLeaseRecoveryMySQL -timeout=600s

When using colima as the Docker backend on macOS, set the following env vars so testcontainers can reach the daemon and mount its socket into containers:

DOCKER_HOST=unix://$HOME/.colima/default/docker.sock \
TESTCONTAINERS_DOCKER_SOCKET_OVERRIDE=/var/run/docker.sock \
go test ./...

File Structure

WIP

Lark build notifications

Notifications use Card JSON 2.0 (Lark client 7.20 or newer). Image results are grouped by repository, then platform, then tag. Multi-architecture tags appear first within each repository. Each platform shows its tags inline as code-styled text. Binary results are grouped by platform and filename; each file and its SHA256 download link share a row, indented beneath the platform.

Card text is uniformly English and maintained in internal/service/impl/lark_templates/devbuild-notify.yaml.tmpl. Artifact identifiers and pipeline status values remain unchanged.

Binary links use tekton.oci_file_download_url, with the same repository, tag and file query parameters as the DevBuild API. Without a download service URL, notifications display filenames without a download hyperlink.

Build Report uses nested Markdown list items: artifact type, repository or platform, and platform/tag or binary/checksum entries. Image repositories and tags use inline code spans so underscores and other identifier characters stay literal. Tags remain on the platform line; each binary filename and SHA256 link share a list item. Lists wrap naturally on narrow clients.

Binary filename and SHA256 hyperlinks can be right-clicked to copy download addresses. Status and Request retain their original field/list layout and horizontal-rule section separators. Report text uses the default font size, with bold labels and no shaded background blocks.

Directories

Path Synopsis
cmd
tibuild command
tibuild-cli command
tibuild-worker command
Command tibuild-worker runs the tibuild background task executor as a standalone process.
Command tibuild-worker runs the tibuild background task executor as a standalone process.
hack
image-delivery-golden command
Command image-delivery-golden records the expansion of the image delivery rules with the *current* implementation (the deno script gen-delivery-image-commands.ts in PingCAP-QE/artifacts) and writes it as the golden dataset consumed by pkg/delivery tests.
Command image-delivery-golden records the expansion of the image delivery rules with the *current* implementation (the deno script gen-delivery-image-commands.ts in PingCAP-QE/artifacts) and writes it as the golden dataset consumed by pkg/delivery tests.
service/impl
Package impl implements the tibuild-v2 service endpoints, including the broker-backed GitHub client used for repository lookups and hotfix tags.
Package impl implements the tibuild-v2 service endpoints, including the broker-backed GitHub client used for repository lookups and hotfix tags.
worker
Package worker contains the reusable polling loop used by the in-process tibuild worker and the standalone tibuild-worker binary.
Package worker contains the reusable polling loop used by the in-process tibuild worker and the standalone tibuild-worker binary.
pkg
authz
Package authz decides whether a request may perform an action.
Package authz decides whether a request may perform an action.
delivery
Package delivery interprets the delivery rules maintained in PingCAP-QE/artifacts (packages/delivery.yaml).
Package delivery interprets the delivery rules maintained in PingCAP-QE/artifacts (packages/delivery.yaml).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL