Documentation
¶
Index ¶
- func SocketPath(dataDir string) string
- type Config
- type DispatchClient
- func (d *DispatchClient) Close() error
- func (d *DispatchClient) ConsumeContainerStats(ctx context.Context, intervalSeconds uint32, runtimeLabel string, ...) error
- func (d *DispatchClient) Create(ctx context.Context, id, image, jitConfig, provider, repo string) error
- func (d *DispatchClient) Destroy(ctx context.Context, id string) error
- func (d *DispatchClient) Wait(ctx context.Context, id string) (uint32, error)
- type DispatchServer
- func (s *DispatchServer) CreateJob(ctx context.Context, req *apiv1.CreateJobRequest) (*apiv1.CreateJobResponse, error)
- func (s *DispatchServer) DestroyJob(ctx context.Context, req *apiv1.DestroyJobRequest) (*apiv1.DestroyJobResponse, error)
- func (s *DispatchServer) RegisterSampler(id, repo string, sampler metrics.Sampler)
- func (s *DispatchServer) StreamContainerStats(req *apiv1.StreamContainerStatsRequest, ...) error
- func (s *DispatchServer) UnregisterSampler(id string)
- func (s *DispatchServer) WaitJob(ctx context.Context, req *apiv1.WaitJobRequest) (*apiv1.WaitJobResponse, error)
- type DispatchServerConfig
- type OrphanSweepConfig
- type RetryConfig
- type Scheduler
- type VMSSHInfo
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func SocketPath ¶
SocketPath returns the path to the gRPC control socket for a given data dir.
Types ¶
type Config ¶
type Config struct {
Runtime *runtime.Runtime
Providers []providers.Provider
Artifacts *artifacts.Extractor // OCI image layer extractor for macOS VM jobs (nil if not available)
LinuxDispatcher *DispatchClient // if non-nil, Linux jobs are dispatched to a Linux VM worker via gRPC
MacOSVMConfig *vm.MacOSVMConfig // if non-nil, macOS-native jobs are enabled (darwin only)
DataDir string // ephemerd data directory (used for artifact extraction paths)
MaxConcurrent int
MaxMacOSVMs int // max concurrent macOS VMs (Vz limit; default auto-detected)
Labels []string
PollInterval time.Duration // if >0, use polling mode (default)
WebhookPort int // listen port for health/webhook server
WebhookSecret string // webhook signature secret
TLSCert string // TLS certificate path
TLSKey string // TLS private key path
Tunnel tunnel.Provider // if non-nil, creates a public tunnel for webhooks
TunnelMaxRetries int // max consecutive reconnect failures before fallback to polling (0 = default 5)
// ExternalURL is the public base URL of an externally-managed tunnel
// (tunnel = "external"). When set alongside webhook mode and NO managed
// Tunnel, the scheduler registers each webhook-capable provider's hook to
// <ExternalURL>/webhook/<provider> on startup, so the operator doesn't
// have to hand-add a hook per repo. External hooks are operator-owned and
// are NOT deregistered on shutdown. Empty means "receiver only, don't
// touch the platform's webhooks".
ExternalURL string
JobTimeout time.Duration
ShutdownTimeout time.Duration
LogRetention time.Duration // max age for job log files (default 7d)
// Retry configures the claim/provision retry queue. When the initial
// attempt to claim a queued job fails with a retryable error
// (rate-limit exhausted, transient 5xx, network), the job is
// enqueued and re-attempted on a backoff ladder rather than lost.
// GitHub does not re-deliver workflow_job webhooks. Leave zero-valued
// (Enabled=false) to keep the pre-existing "log and drop" behavior.
Retry RetryConfig
// OrphanSweep configures teardown of dispatched runners that were
// never observed picking up a job. GitHub schedules JIT runners onto
// ANY queued job with matching labels, so the runner dispatched "for"
// a job may end up running a different one — leaving the runner that
// was dispatched for THAT job idle with no job-completion event ever
// pointing at it. The sweep destroys such runners once they have been
// idle-unbound for Grace. Only active in webhook mode and only for
// runners dispatched via providers that report runner assignments
// (providers.RunnerNameReporter) — otherwise "never observed bound"
// would just mean "we had no way to observe it".
OrphanSweep OrphanSweepConfig
// RunnerImageForRepo resolves the per-repo, per-OS image override
// configured under [runner.images]. Returns "" when no override is
// set; the scheduler then falls back to the provider per-OS default
// and finally the runtime's host-aware default. Nil-safe.
RunnerImageForRepo func(repo, os string) string
MaxNativeMac int // max concurrent native macOS jobs (default 4)
MacOSModeForRepo func(repo string) string // returns "native" or "vm" per repo (nil = always VM)
NativeMacUser string // non-root user for native macOS runner processes
NativeMacStrict bool // opt-in deny-by-default sandbox for native macOS jobs
NativeMacMaxProcs int // ulimit -u for native macOS jobs (0 = unlimited; default 2048)
RunnerDir string // path to extracted GHA runner binary dir (runner.Manager.Dir())
PrivateKeyPath string // GitHub App private_key_path, denied read access in the native sandbox (empty for PAT auth)
Log *slog.Logger
}
Config for the scheduler.
type DispatchClient ¶
type DispatchClient struct {
// contains filtered or unexported fields
}
DispatchClient dispatches Linux jobs to the WSL worker via gRPC.
func NewDispatchClient ¶
func NewDispatchClient(addr string) (*DispatchClient, error)
NewDispatchClient connects to the dispatch gRPC server at the given address.
func (*DispatchClient) Close ¶
func (d *DispatchClient) Close() error
Close closes the gRPC connection.
func (*DispatchClient) ConsumeContainerStats ¶
func (d *DispatchClient) ConsumeContainerStats(ctx context.Context, intervalSeconds uint32, runtimeLabel string, log *slog.Logger) error
ConsumeContainerStats opens the StreamContainerStats stream, asks the in-VM dispatch server for samples at the given interval, and feeds each batch into the host metrics registry under the supplied runtime label (typically metrics.RuntimeLinuxVM). The call returns when ctx is cancelled. On non-fatal stream errors the consumer reconnects with backoff so a transient network blip in the VM doesn't lose metrics for the rest of the daemon's lifetime.
Series are deleted via metrics.DeleteContainerSeries when the consumer sees a container id stop appearing in batches (the in-VM UnregisterSampler removes it from the stream).
func (*DispatchClient) Create ¶
func (d *DispatchClient) Create(ctx context.Context, id, image, jitConfig, provider, repo string) error
Create dispatches a container create to the WSL worker. provider + repo are passed through so the VM-side dind server can scope its per-repo image cache namespace to (provider, repo) and not leak private images across forges or repos.
type DispatchServer ¶
type DispatchServer struct {
apiv1.UnimplementedDispatchServer
// contains filtered or unexported fields
}
DispatchServer implements the Dispatch gRPC service. It runs inside the WSL containerd-only worker and proxies Create/Wait/Destroy calls to the local Linux Runtime. It also serves StreamContainerStats so the host can scrape per-container resource metrics on its own /metrics endpoint without exposing a second listener inside the VM. See docs/arch/container-metrics.md.
func NewDispatchServer ¶
func NewDispatchServer(rt *runtime.Runtime, log *slog.Logger, defaultInterval time.Duration, linuxCPULimit, linuxMemLimitBytes uint64) *DispatchServer
NewDispatchServer constructs a Dispatch service handler. defaultInterval is used when the StreamContainerStats client passes interval_seconds=0; linuxCPULimit / linuxMemLimitBytes are baked into each per-container sampler as the configured cap.
func StartDispatchServer ¶
func StartDispatchServer(cfg DispatchServerConfig) (*DispatchServer, func())
StartDispatchServer starts the dispatch gRPC server on the given TCP port and returns the running server instance plus a cleanup function that gracefully stops it. The returned *DispatchServer exposes RegisterSampler / UnregisterSampler so the local runtime can plumb its OnTaskStarted / OnTaskDestroy hooks into the stats stream surface area.
Binds to 0.0.0.0 so the host (outside the VM) can reach it. WSL on Windows shares localhost with the host, so this used to be 127.0.0.1, but the same process is now invoked from inside an Apple Vz VM where the host lives on the NAT side and needs the listener exposed on the VM's external interface.
func (*DispatchServer) CreateJob ¶
func (s *DispatchServer) CreateJob(ctx context.Context, req *apiv1.CreateJobRequest) (*apiv1.CreateJobResponse, error)
func (*DispatchServer) DestroyJob ¶
func (s *DispatchServer) DestroyJob(ctx context.Context, req *apiv1.DestroyJobRequest) (*apiv1.DestroyJobResponse, error)
func (*DispatchServer) RegisterSampler ¶
func (s *DispatchServer) RegisterSampler(id, repo string, sampler metrics.Sampler)
RegisterSampler is called by the in-VM runtime's OnTaskStarted hook to expose a container's sampler to StreamContainerStats subscribers.
func (*DispatchServer) StreamContainerStats ¶
func (s *DispatchServer) StreamContainerStats(req *apiv1.StreamContainerStatsRequest, stream grpc.ServerStreamingServer[apiv1.ContainerStatsBatch]) error
StreamContainerStats serves the long-lived sampling stream that the host uses to surface per-container resource series. The handler ticks at the client-requested cadence and sends one batch per tick covering every registered sampler. Returns when the client cancels the context, when the underlying connection drops, or when Send fails for any reason.
func (*DispatchServer) UnregisterSampler ¶
func (s *DispatchServer) UnregisterSampler(id string)
UnregisterSampler removes a container's sampler from the stream set, called by the runtime's OnTaskDestroy hook.
func (*DispatchServer) WaitJob ¶
func (s *DispatchServer) WaitJob(ctx context.Context, req *apiv1.WaitJobRequest) (*apiv1.WaitJobResponse, error)
type DispatchServerConfig ¶
type DispatchServerConfig struct {
Port int
Runtime *runtime.Runtime
Log *slog.Logger
StatsInterval time.Duration // default 10s when zero
LinuxCPULimit uint64 // 0 = unlimited
LinuxMemLimitBytes uint64 // 0 = unlimited
}
DispatchServerConfig configures the in-VM dispatch gRPC server.
type OrphanSweepConfig ¶
type OrphanSweepConfig struct {
// Enabled toggles the sweep.
Enabled bool
// Grace is how long a dispatched runner may remain unbound (never
// seen in an in_progress event) before it is destroyed. Defaults to
// 10 minutes when zero.
Grace time.Duration
}
OrphanSweepConfig tunes the orphaned-runner sweep. Zero-valued = disabled (matching pre-existing behavior); the CLI enables it by default with a 10-minute grace window.
type RetryConfig ¶
type RetryConfig struct {
// Enabled toggles the entire retry queue. When false, failures still
// log and drop as before (no behavior change).
Enabled bool
// Schedule is the ordered backoff ladder. Each entry is the base
// delay for that attempt; jitter is applied on top. If nil, defaults
// to {30s, 1m, 2m, 5m, 10m}.
Schedule []time.Duration
// MaxAge is the wall-clock budget from first failure to giving up.
// Once (now - firstFailure) > MaxAge, we log a WARN and drop. Default 90m.
MaxAge time.Duration
// Jitter is the fraction (0-1) of a delay that's randomized +/- around
// the base value. Set to a NEGATIVE value (e.g. -1) to request the
// default (0.2 = +/-20%). Literal 0 is honored - tests use it for
// deterministic scheduling.
Jitter float64
// RateHint returns the last-observed GitHub rate-limit state. When
// remaining == 0 and now < reset and updated is fresh (<5m old),
// the next attempt is snapped to reset + a small jitter instead of
// the Schedule entry. Nil-safe: nil means "no rate awareness".
RateHint func() (remaining int64, reset time.Time, updated time.Time)
// Now is the clock function. Defaults to time.Now. Tests inject
// a fake clock so backoff scheduling is deterministic.
Now func() time.Time
}
RetryConfig tunes the claim retry queue.
A zero-valued RetryConfig disables retries entirely, matching the pre-existing "log-and-drop" behavior. In practice New() applies sensible defaults so callers get retries by default.
type Scheduler ¶
type Scheduler struct {
// contains filtered or unexported fields
}
Scheduler ties CI provider job events to container lifecycle. When a job is queued, it provisions a runner environment. When the job completes, it destroys the environment.
func (*Scheduler) Run ¶
Run starts the scheduler. It discovers jobs via polling (default) or webhooks (when TLS certs are configured), and manages runner lifecycle.
func (*Scheduler) SetMacOSVMConfig ¶
func (s *Scheduler) SetMacOSVMConfig(cfg *vm.MacOSVMConfig)
SetMacOSVMConfig enables macOS job support after startup. This is used when the macOS disk image is being provisioned in the background — the scheduler starts immediately for Linux jobs and picks up macOS jobs once the install finishes.
func (*Scheduler) StartVMSSHServer ¶
StartVMSSHServer starts a small HTTP server on the unix control socket for the VM SSH info endpoint. Called after the gRPC server is set up.