deploy

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 15, 2026 License: Apache-2.0 Imports: 45 Imported by: 0

Documentation

Overview

Package deploy implements the deploy gateway protocol (SPEC §7): the server side (Gateway, mountable inside the main server, as `backyard gateway`, or as an AWS Lambda function), the Go client used by the CLI and MCP server, and the local file collection/ignore rules shared by every deploy method.

Upload integrity

A deploy is planned (create), uploaded file by file, then committed. The plan validates every declared size against the per-file and per-site limits, and each upload is bound to its declared size and SHA-256:

  • The gateway's own endpoint (PUT {G}/uploads/{token}; fs storage or deploy.presign: false) streams the body through a verifier, so a body of another size or digest is never stored.
  • Presigned storage URLs bind what the backend can enforce (see blob.PresignPutOptions): on S3 the size is signed (Content-Length, or the Content-MD5 of the empty body) and, on AWS, the SHA-256 as x-amz-checksum-sha256, so S3 rejects any other body; on GCS x-goog-content-length-range pins the size (GCS cannot check SHA-256); an Azure SAS URL cannot restrict the body at all.

The commit is the second line of defence: every listed file must exist with its declared size, or the commit fails with 409 conflict naming the paths. A stored object of the wrong size is left alone where every upload URL binds the size (the gateway's endpoint, S3, GCS): it can only be another deploy's verified upload or a direct sync, which rsync semantics keep. Only where presigned uploads cannot bind the size (Azure SAS) is a wrong-size object that is not the version the previous manifest recorded deleted, so no unverified object of the wrong size outlives a commit attempt there.

Overlapping deploys

Deploys of a site may overlap (two CI runs): their uploads write the same paths and the last write wins per file, as with rsync. The commit therefore never trusts a declared digest for a stored object the previous manifest does not describe (this deploy's upload, or whatever replaced it since): it records the digest of the bytes stored now with their etag, taken from the store where it verified one itself (S3 checksums of presigned uploads, blob.DigestReporter) and otherwise hashed from the object, which costs a read of the uploaded bytes. A later deploy of content that was overwritten is then never wrongly "unchanged".

Uncommitted uploads

Uploads write directly to sites/<site>/<path> (rsync semantics; deploys are not atomic), so an upload that is never committed stays in storage:

  • A path that is not in the current manifest (a new file) becomes an orphan object: it is served like any object under the site but is not in the manifest, never counts as "unchanged", and is deleted by the next successful commit (which deletes everything it does not list) or with the site; a refresh after a direct sync adopts it instead.
  • A path in the current manifest is replaced in place, as rsync would.

This is accepted and bounded rather than cleaned up by a janitor: upload URLs expire with their deploy id (DeployTTL), are bound to sizes the plan checked against the limits (except on Azure, see above), and are only issued to the site's authorized deployers by a rate-limited create whose audit record ("deploy.create") counts the uploads, their bytes and the new paths among them.

Index

Constants

View Source
const (
	CodeBadRequest      = "bad_request"
	CodeUnauthenticated = "unauthenticated"
	CodeForbidden       = "forbidden"
	CodeNotFound        = "not_found"
	CodeConflict        = "conflict"
	CodeTooLarge        = "too_large"
	CodeRateLimited     = "rate_limited"
	CodeUnsupported     = "unsupported"
	CodeUpstream        = "upstream_error"
	CodeInternal        = "internal"
)

Error codes shared with the site API (SPEC §5).

View Source
const (
	// MaxPathBytes is the longest site-relative path accepted (SPEC §7.3).
	MaxPathBytes = 1024
	// ReservedPathPrefix can never be deployed: it is the platform namespace.
	ReservedPathPrefix = "__backyard/"
)
View Source
const DefaultConcurrency = 8

DefaultConcurrency is the default number of parallel uploads.

View Source
const (
	// DefaultReadFileMax caps Client.ReadFile when maxBytes <= 0.
	DefaultReadFileMax = 32 << 20
)
View Source
const DeployTTL = time.Hour

DeployTTL is how long a deploy id (and its upload URLs) stays valid.

View Source
const EmbeddedPath = "/__backyard/api/deploy"

EmbeddedPath is where the server mounts the embedded gateway.

View Source
const IgnoreFile = ".backyardignore"

IgnoreFile is the per-directory ignore file read by CollectFiles.

View Source
const (
	// MaxPublishList bounds the files and the delete paths of one publish.
	MaxPublishList = 1000
)

Variables

View Source
var ErrFileTooLarge = errors.New("deploy: file exceeds the read limit")

ErrFileTooLarge is returned by Client.ReadFile when a file exceeds maxBytes.

Functions

func DefaultIgnore

func DefaultIgnore() []string

DefaultIgnore returns the built-in ignore patterns alone: the overridable defaults followed by ProtectedIgnore. It is what a folder without any ignore configuration gets (WithDefaultIgnore(nil)).

func FormatBytes added in v0.2.0

func FormatBytes(n int64) string

FormatBytes renders a byte count for messages ("812 B", "1.2 MiB").

func HasGitSegment

func HasGitSegment(p string) bool

HasGitSegment reports whether a "/"-separated segment of p is ".git" in any letter case.

func IgnorePatterns

func IgnorePatterns(dir string, extra []string) ([]string, error)

IgnorePatterns returns the complete pattern list CollectFiles applies to dir: WithDefaultIgnore of the lines of dir/.backyardignore (if present) followed by extra — so those may re-include the overridable defaults but never ProtectedIgnore. Direct-sync methods use it to build exclude flags.

func Ignored

func Ignored(rel string, isDir bool, patterns []string) bool

Ignored reports whether a site-relative path is excluded by patterns (used by direct-sync methods to build tool-specific exclude flags).

func LambdaHandler

LambdaHandler adapts an http.Handler to AWS Lambda function URL (and API Gateway HTTP API v2) events, including base64 bodies.

The request gets the event's method, raw path and query, headers, cookies and body (base64-decoded when flagged); RemoteAddr is the caller's source IP. Responses carry the status, headers (multi-valued ones comma-joined), Set-Cookie values as cookies, and a body that is base64-encoded unless it is valid UTF-8 text. A text body of at least 32 KiB is gzipped when the request accepts it (Go clients do by default), so that e.g. the upload list of a deploy of thousands of files fits Lambda's 6 MB response payload. A response whose payload would still exceed that limit (measured after base64 and JSON encoding) becomes a JSON 413 that points to the server's embedded gateway, which has no such limit.

func MaxPublishBytes added in v0.2.0

func MaxPublishBytes(cfg *config.Config) int64

MaxPublishBytes returns the decoded file bytes one publish may carry (mcp.max_publish_bytes).

func ProtectedIgnore

func ProtectedIgnore() []string

ProtectedIgnore returns the default patterns no configuration can override: ".git" (directory or file) and the project files, as case-insensitive globs (".[gG][iI][tT]" …).

func ValidatePath

func ValidatePath(p string) string

ValidatePath reports why a site-relative path is unacceptable for a deploy or a gateway read, or "" when it is valid: ValidatePathSyntax, and no segment may be ".git" in any letter case (HasGitSegment).

The ".git" rule is defense in depth: deploys never publish repository metadata (the CLI's ignore rules cannot re-include .git/, see ProtectedIgnore), so a client sending such a path is broken or hostile, and a gateway never serves one back to a "backyard pull" (a crafted .git/config runs commands on the next git command in the pulled folder). Case-insensitive file systems (macOS, Windows) resolve ".GIT" to ".git". The Cloudflare edition's validatePath applies the same rule.

func ValidatePathSyntax

func ValidatePathSyntax(p string) string

ValidatePathSyntax reports why p is not a well-formed site-relative path, or "" when it is: relative, "/"-separated, no "." / ".." / empty segments, at most MaxPathBytes, valid UTF-8 without control characters or backslashes, and not under "__backyard/". It does not apply ValidatePath's ".git" rule, so clients use it for paths a site may already hold (stored before the rule, or by a direct sync) that they only need to name safely, e.g. to skip or delete them.

func WithDefaultIgnore

func WithDefaultIgnore(patterns []string) []string

WithDefaultIgnore returns the complete, ordered pattern list for the configured patterns: the overridable defaults, then patterns (which may re-include some of them with "!" lines), then ProtectedIgnore (which nothing can re-include). Every consumer — CollectFiles, the direct-sync exclude translation, the prune of ignored site files — evaluates this list as one gitignore sequence.

Types

type APIError

type APIError struct {
	Status   int
	Code     string
	Message  string
	LoginURL string
}

APIError is returned for non-2xx gateway responses.

func (*APIError) Error

func (e *APIError) Error() string

type AuthInfo

type AuthInfo struct {
	Mode        string      `json:"mode"`
	CLILoginURL string      `json:"cli_login_url,omitempty"`
	Device      *DeviceInfo `json:"device,omitempty"`
	// AnonymousDeploy is true when deploys need no token (auth.mode=none + allow_anonymous).
	AnonymousDeploy bool `json:"anonymous_deploy"`
}

AuthInfo tells clients how to obtain a token.

type Client

type Client struct {
	BaseURL   string // gateway base URL G
	Token     string // bearer token ("" for anonymous)
	HTTP      *http.Client
	UserAgent string
}

Client talks to a deploy gateway.

A Client is safe for concurrent use once configured.

func NewClient

func NewClient(baseURL, token string) *Client

NewClient returns a client with sane timeouts: 30 s to connect, 15 s for TLS handshakes and 15 min for response headers (what the gateway allows a create, commit or refresh of a large site), no overall deadline (uploads may be long; cancel via the context). Cookies are kept for the client's lifetime so a guest identity stays stable between the create and commit calls of an anonymous deploy.

func (*Client) DeleteSite

func (c *Client) DeleteSite(ctx context.Context, site string) error

DeleteSite calls DELETE /sites/{site}.

func (*Client) Deploy

func (c *Client) Deploy(ctx context.Context, site, dir string, opts DeployOptions) (*DeployResult, error)

Deploy collects files from dir (CollectFiles), creates a deploy, uploads changed files concurrently and commits.

Uploads are retried (3 times, with backoff) on network errors, throttling and 5xx responses. Bytes in the result counts uploaded bytes (for DryRun: bytes that would be uploaded). DryRun never changes anything on the server: it compares the local files with GET /sites/{site}/files when the gateway is reachable, and otherwise reports every file as an upload.

func (*Client) GetSite

func (c *Client) GetSite(ctx context.Context, site string) (*sites.Meta, error)

GetSite fetches GET /sites/{site}.

func (*Client) Info

func (c *Client) Info(ctx context.Context) (*Info, error)

Info fetches GET /info.

func (*Client) ListSites

func (c *Client) ListSites(ctx context.Context) ([]sites.Meta, error)

ListSites fetches GET /sites.

func (*Client) Me

func (c *Client) Me(ctx context.Context) (*identity.User, error)

Me fetches GET /me: the identity behind the client's token (anonymous guests in auth mode none without a token).

func (*Client) ReadFile

func (c *Client) ReadFile(ctx context.Context, site, path string, maxBytes int64) ([]byte, error)

ReadFile fetches GET /sites/{site}/files/{path} (maxBytes caps the read; <=0 = 32 MiB).

func (*Client) Refresh

func (c *Client) Refresh(ctx context.Context, site string) (*sites.Meta, error)

Refresh calls POST /sites/{site}/refresh.

func (*Client) SetOwners

func (c *Client) SetOwners(ctx context.Context, site string, add, remove []string) (*sites.Meta, error)

SetOwners calls POST /sites/{site}/owners.

func (*Client) SiteFiles

func (c *Client) SiteFiles(ctx context.Context, site string) ([]SiteFile, error)

SiteFiles fetches GET /sites/{site}/files.

type CommitRequest

type CommitRequest struct {
	Files []FileEntry `json:"files"`
}

CommitRequest is the body of POST {G}/sites/{site}/deploys/{id}/commit.

type CommitResponse

type CommitResponse struct {
	Site    sites.Meta `json:"site"`
	URL     string     `json:"url"`
	Deleted int        `json:"deleted"`
}

CommitResponse is returned after a successful commit.

type CreateDeployRequest

type CreateDeployRequest struct {
	Files   []FileEntry `json:"files"`
	Message string      `json:"message,omitempty"`
}

CreateDeployRequest is the body of POST {G}/sites/{site}/deploys.

type CreateDeployResponse

type CreateDeployResponse struct {
	DeployID  string    `json:"deploy_id"`
	Uploads   []Upload  `json:"uploads"`
	Unchanged []string  `json:"unchanged"`
	ExpiresAt time.Time `json:"expires_at"`
}

CreateDeployResponse lists uploads to perform.

type DeployOptions

type DeployOptions struct {
	Ignore      []string // extra gitignore-style patterns
	Message     string
	Concurrency int // parallel uploads (default 8)
	DryRun      bool
	// Progress receives progress events. Calls are serialized, so the
	// callback need not be safe for concurrent use.
	Progress func(ProgressEvent)
}

DeployOptions tune Client.Deploy.

type DeployResult

type DeployResult struct {
	Site      sites.Meta
	URL       string
	Uploaded  int
	Unchanged int
	Deleted   int
	Bytes     int64
	Files     []LocalFile // the collected files (also filled for DryRun)
}

DeployResult summarizes a deploy.

type DeviceInfo

type DeviceInfo struct {
	Issuer   string   `json:"issuer"`
	ClientID string   `json:"client_id"`
	Scopes   []string `json:"scopes"`
}

DeviceInfo enables the OAuth device authorization grant against the IdP.

type ErrorBody

type ErrorBody struct {
	Error struct {
		Code     string `json:"code"`
		Message  string `json:"message"`
		LoginURL string `json:"login_url,omitempty"`
	} `json:"error"`
}

ErrorBody is the JSON error envelope shared with the site API.

type FileEntry

type FileEntry struct {
	Path        string `json:"path"`
	Size        int64  `json:"size"`
	SHA256      string `json:"sha256"`
	ContentType string `json:"content_type,omitempty"`
}

FileEntry describes a local file in a deploy request.

type Gateway

type Gateway struct {
	// contains filtered or unexported fields
}

Gateway serves the deploy protocol. Mount it with the base path stripped: requests arrive as "/info", "/sites/…", "/uploads/…".

func NewGateway

func NewGateway(opts GatewayOptions) (*Gateway, error)

NewGateway validates options and returns a Gateway.

When BaseURL is empty, deploy.gateway_url is used; when that is empty too, the gateway derives its URL from each request's Host (useful behind an AWS Lambda function URL whose address is not known in advance).

func (*Gateway) CheckPublish added in v0.2.0

func (g *Gateway) CheckPublish(req *PublishRequest, decode func(i int) ([]byte, *PublishError)) *PublishError

CheckPublish runs the checks of Publish that need no storage, in their normative order (SPEC §14.1, backyard_publish steps 1–4), and fills in default content types. When decode is not nil it is called for Files[i] once the file's path checks passed and sets its Data, so an encoding error (E7) is reported in its place in the order; with a nil decode, Data is used as is.

func (*Gateway) Info

func (g *Gateway) Info() Info

Info returns the /info document.

func (*Gateway) Publish added in v0.2.0

func (g *Gateway) Publish(ctx context.Context, req PublishRequest) (*PublishResult, *PublishError)

Publish writes req to the site: by default the listed files are added or replaced and the others kept; Delete removes paths; Replace makes the site exactly Files. It runs SPEC §14.1 backyard_publish steps 1–4 and 6–10 (the caller holds the connector's lease, step 5): CheckPublish, the ownership policy, the overwrite guard, the deploys_per_hour quota (keyed user:<id>, guests included), the limits of the resulting site, then the writes, the manifest and meta switch, and deletions last.

func (*Gateway) ServeHTTP

func (g *Gateway) ServeHTTP(w http.ResponseWriter, r *http.Request)

ServeHTTP implements http.Handler.

type GatewayOptions

type GatewayOptions struct {
	Config   *config.Config
	Store    blob.Store
	Registry *sites.Registry
	// Auth authenticates requests (the server's Authenticator when embedded,
	// auth.NewBearer for the standalone gateway / Lambda).
	Auth auth.Authenticator
	// BaseURL is the absolute URL the gateway is reachable at (used to build
	// self-hosted upload URLs and Info.GatewayURL), e.g.
	// https://backyard.acme.com/__backyard/api/deploy
	BaseURL string
	// OnDeleteSite is called after a site is deleted (e.g. to drop its
	// database documents). Optional.
	OnDeleteSite func(ctx context.Context, site string) error
	Logger       *slog.Logger
	// Version reported in Info.
	Version string
	// MCPURL is the remote MCP endpoint of the hub this gateway is embedded
	// in (SPEC §14.1), reported as Info.MCPURL; "" omits it.
	MCPURL string
}

GatewayOptions configures a Gateway.

type Info

type Info struct {
	Name        string            `json:"name"`
	Version     string            `json:"version"`
	PublicURL   string            `json:"public_url"`
	BaseDomain  string            `json:"base_domain"`
	URLMode     string            `json:"url_mode"`
	GatewayURL  string            `json:"gateway_url"`
	Auth        AuthInfo          `json:"auth"`
	Storage     *blob.Description `json:"storage,omitempty"`
	Methods     []string          `json:"methods"`
	RsyncTarget string            `json:"rsync_target,omitempty"`
	Limits      Limits            `json:"limits"`
	// MCPURL is the hub's remote MCP endpoint (SPEC §7.1, §14.1): present
	// while the Claude connector is enabled, on the hub's embedded gateway
	// only.
	MCPURL string `json:"mcp_url,omitempty"`
}

Info is returned by GET {gateway}/info (no auth).

type Limits

type Limits struct {
	MaxSiteBytes int64 `json:"max_site_bytes"`
	MaxFiles     int   `json:"max_files"`
	MaxFileBytes int64 `json:"max_file_bytes"`
}

Limits advertised to clients.

type LocalFile

type LocalFile struct {
	Path        string // site-relative, "/"-separated
	Abs         string // absolute path on disk
	Size        int64
	SHA256      string // hex
	ContentType string
}

LocalFile is a file selected for deployment.

func CollectFiles

func CollectFiles(dir string, extra []string) ([]LocalFile, error)

CollectFiles walks dir applying the default ignore rules, dir/.backyardignore and extra patterns (gitignore syntax, see IgnorePatterns), hashing every selected file. Symlinks to files are followed; symlinked directories are not.

Symlinks whose target lies outside dir, broken symlinks and special files (sockets, devices, pipes) are skipped. A selected file whose path the gateway would reject (e.g. under "__backyard/") is an error. The result is sorted by Path.

type Matcher

type Matcher struct {
	// contains filtered or unexported fields
}

Matcher evaluates gitignore-style patterns against site-relative paths.

Supported syntax: blank lines and "#" comments, "!" negation (the last matching pattern wins), a trailing "/" for directory-only patterns, a leading or inner "/" to anchor a pattern at the root (otherwise it matches a name at any depth), "**" for any number of directories ("**/x", "x/**", "a/**/b"), and path.Match globs ("*", "?", "[…]", "\" escapes) within a segment. As in git, a path inside an excluded directory cannot be re-included. Invalid globs are ignored.

func NewMatcher

func NewMatcher(patterns []string) *Matcher

NewMatcher compiles patterns (one gitignore line each).

func (*Matcher) Ignored

func (m *Matcher) Ignored(rel string, isDir bool) bool

Ignored reports whether rel ("/"-separated, relative to the site root) is excluded, either itself or through an excluded parent directory.

type OwnersRequest

type OwnersRequest struct {
	Add    []string `json:"add,omitempty"`
	Remove []string `json:"remove,omitempty"`
}

OwnersRequest is the body of POST {G}/sites/{site}/owners.

type ProgressEvent

type ProgressEvent struct {
	Phase string // "scan" | "plan" | "upload" | "commit" | "done"
	Path  string
	Done  int
	Total int
	Bytes int64
}

ProgressEvent reports deploy progress.

type PublishError added in v0.2.0

type PublishError struct {
	ID, Message string
	RetryAfter  time.Duration
}

PublishError is a refusal or failure with its catalog id ("E2" … "E27", the tool errors of SPEC §14.1) and the tool error text.

func (*PublishError) Error added in v0.2.0

func (e *PublishError) Error() string

type PublishFile added in v0.2.0

type PublishFile struct {
	Path        string
	Data        []byte
	ContentType string
}

PublishFile is one file of a publish call, already decoded.

type PublishRequest added in v0.2.0

type PublishRequest struct {
	User      *identity.User
	Site      string
	Files     []PublishFile
	Delete    []string
	Replace   bool
	Overwrite bool
	// ClientIP, ClientName and GrantID go into the deploy.commit audit record.
	ClientIP, ClientName, GrantID string
}

PublishRequest is one backyard_publish call.

type PublishResult added in v0.2.0

type PublishResult struct {
	Site, URL                           string
	Created                             bool
	Files, Uploaded, Unchanged, Deleted int
	Bytes                               int64
	Warnings                            []string
}

PublishResult describes the site after a publish.

type SiteFile

type SiteFile struct {
	Path      string    `json:"path"`
	Size      int64     `json:"size"`
	SHA256    string    `json:"sha256,omitempty"`
	UpdatedAt time.Time `json:"updated_at"`
}

SiteFile is one entry of GET {G}/sites/{site}/files.

type Upload

type Upload struct {
	Path    string            `json:"path"`
	URL     string            `json:"url"`
	Method  string            `json:"method"`
	Headers map[string]string `json:"headers,omitempty"`
}

Upload is one file the client must PUT.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL