Documentation
¶
Overview ¶
Package deploy implements the deploy gateway protocol (SPEC §7): the server side (Gateway, mountable inside the main server, as `backyard gateway`, or as an AWS Lambda function), the Go client used by the CLI and MCP server, and the local file collection/ignore rules shared by every deploy method.
Upload integrity ¶
A deploy is planned (create), uploaded file by file, then committed. The plan validates every declared size against the per-file and per-site limits, and each upload is bound to its declared size and SHA-256:
- The gateway's own endpoint (PUT {G}/uploads/{token}; fs storage or deploy.presign: false) streams the body through a verifier, so a body of another size or digest is never stored.
- Presigned storage URLs bind what the backend can enforce (see blob.PresignPutOptions): on S3 the size is signed (Content-Length, or the Content-MD5 of the empty body) and, on AWS, the SHA-256 as x-amz-checksum-sha256, so S3 rejects any other body; on GCS x-goog-content-length-range pins the size (GCS cannot check SHA-256); an Azure SAS URL cannot restrict the body at all.
The commit is the second line of defence: every listed file must exist with its declared size, or the commit fails with 409 conflict naming the paths. A stored object of the wrong size is left alone where every upload URL binds the size (the gateway's endpoint, S3, GCS): it can only be another deploy's verified upload or a direct sync, which rsync semantics keep. Only where presigned uploads cannot bind the size (Azure SAS) is a wrong-size object that is not the version the previous manifest recorded deleted, so no unverified object of the wrong size outlives a commit attempt there.
Overlapping deploys ¶
Deploys of a site may overlap (two CI runs): their uploads write the same paths and the last write wins per file, as with rsync. The commit therefore never trusts a declared digest for a stored object the previous manifest does not describe (this deploy's upload, or whatever replaced it since): it records the digest of the bytes stored now with their etag, taken from the store where it verified one itself (S3 checksums of presigned uploads, blob.DigestReporter) and otherwise hashed from the object, which costs a read of the uploaded bytes. A later deploy of content that was overwritten is then never wrongly "unchanged".
Uncommitted uploads ¶
Uploads write directly to sites/<site>/<path> (rsync semantics; deploys are not atomic), so an upload that is never committed stays in storage:
- A path that is not in the current manifest (a new file) becomes an orphan object: it is served like any object under the site but is not in the manifest, never counts as "unchanged", and is deleted by the next successful commit (which deletes everything it does not list) or with the site; a refresh after a direct sync adopts it instead.
- A path in the current manifest is replaced in place, as rsync would.
This is accepted and bounded rather than cleaned up by a janitor: upload URLs expire with their deploy id (DeployTTL), are bound to sizes the plan checked against the limits (except on Azure, see above), and are only issued to the site's authorized deployers by a rate-limited create whose audit record ("deploy.create") counts the uploads, their bytes and the new paths among them.
Index ¶
- Constants
- Variables
- func DefaultIgnore() []string
- func FormatBytes(n int64) string
- func HasGitSegment(p string) bool
- func IgnorePatterns(dir string, extra []string) ([]string, error)
- func Ignored(rel string, isDir bool, patterns []string) bool
- func LambdaHandler(h http.Handler) ...
- func MaxPublishBytes(cfg *config.Config) int64
- func ProtectedIgnore() []string
- func ValidatePath(p string) string
- func ValidatePathSyntax(p string) string
- func WithDefaultIgnore(patterns []string) []string
- type APIError
- type AuthInfo
- type Client
- func (c *Client) DeleteSite(ctx context.Context, site string) error
- func (c *Client) Deploy(ctx context.Context, site, dir string, opts DeployOptions) (*DeployResult, error)
- func (c *Client) GetSite(ctx context.Context, site string) (*sites.Meta, error)
- func (c *Client) Info(ctx context.Context) (*Info, error)
- func (c *Client) ListSites(ctx context.Context) ([]sites.Meta, error)
- func (c *Client) Me(ctx context.Context) (*identity.User, error)
- func (c *Client) ReadFile(ctx context.Context, site, path string, maxBytes int64) ([]byte, error)
- func (c *Client) Refresh(ctx context.Context, site string) (*sites.Meta, error)
- func (c *Client) SetOwners(ctx context.Context, site string, add, remove []string) (*sites.Meta, error)
- func (c *Client) SiteFiles(ctx context.Context, site string) ([]SiteFile, error)
- type CommitRequest
- type CommitResponse
- type CreateDeployRequest
- type CreateDeployResponse
- type DeployOptions
- type DeployResult
- type DeviceInfo
- type ErrorBody
- type FileEntry
- type Gateway
- func (g *Gateway) CheckPublish(req *PublishRequest, decode func(i int) ([]byte, *PublishError)) *PublishError
- func (g *Gateway) Info() Info
- func (g *Gateway) Publish(ctx context.Context, req PublishRequest) (*PublishResult, *PublishError)
- func (g *Gateway) ServeHTTP(w http.ResponseWriter, r *http.Request)
- type GatewayOptions
- type Info
- type Limits
- type LocalFile
- type Matcher
- type OwnersRequest
- type ProgressEvent
- type PublishError
- type PublishFile
- type PublishRequest
- type PublishResult
- type SiteFile
- type Upload
Constants ¶
const ( CodeBadRequest = "bad_request" CodeUnauthenticated = "unauthenticated" CodeForbidden = "forbidden" CodeNotFound = "not_found" CodeConflict = "conflict" CodeTooLarge = "too_large" CodeRateLimited = "rate_limited" CodeUnsupported = "unsupported" CodeUpstream = "upstream_error" CodeInternal = "internal" )
Error codes shared with the site API (SPEC §5).
const ( // MaxPathBytes is the longest site-relative path accepted (SPEC §7.3). MaxPathBytes = 1024 // ReservedPathPrefix can never be deployed: it is the platform namespace. ReservedPathPrefix = "__backyard/" )
const DefaultConcurrency = 8
DefaultConcurrency is the default number of parallel uploads.
const (
// DefaultReadFileMax caps Client.ReadFile when maxBytes <= 0.
DefaultReadFileMax = 32 << 20
)
const DeployTTL = time.Hour
DeployTTL is how long a deploy id (and its upload URLs) stays valid.
const EmbeddedPath = "/__backyard/api/deploy"
EmbeddedPath is where the server mounts the embedded gateway.
const IgnoreFile = ".backyardignore"
IgnoreFile is the per-directory ignore file read by CollectFiles.
const (
// MaxPublishList bounds the files and the delete paths of one publish.
MaxPublishList = 1000
)
Variables ¶
var ErrFileTooLarge = errors.New("deploy: file exceeds the read limit")
ErrFileTooLarge is returned by Client.ReadFile when a file exceeds maxBytes.
Functions ¶
func DefaultIgnore ¶
func DefaultIgnore() []string
DefaultIgnore returns the built-in ignore patterns alone: the overridable defaults followed by ProtectedIgnore. It is what a folder without any ignore configuration gets (WithDefaultIgnore(nil)).
func FormatBytes ¶ added in v0.2.0
FormatBytes renders a byte count for messages ("812 B", "1.2 MiB").
func HasGitSegment ¶
HasGitSegment reports whether a "/"-separated segment of p is ".git" in any letter case.
func IgnorePatterns ¶
IgnorePatterns returns the complete pattern list CollectFiles applies to dir: WithDefaultIgnore of the lines of dir/.backyardignore (if present) followed by extra — so those may re-include the overridable defaults but never ProtectedIgnore. Direct-sync methods use it to build exclude flags.
func Ignored ¶
Ignored reports whether a site-relative path is excluded by patterns (used by direct-sync methods to build tool-specific exclude flags).
func LambdaHandler ¶
func LambdaHandler(h http.Handler) func(ctx context.Context, req events.LambdaFunctionURLRequest) (events.LambdaFunctionURLResponse, error)
LambdaHandler adapts an http.Handler to AWS Lambda function URL (and API Gateway HTTP API v2) events, including base64 bodies.
The request gets the event's method, raw path and query, headers, cookies and body (base64-decoded when flagged); RemoteAddr is the caller's source IP. Responses carry the status, headers (multi-valued ones comma-joined), Set-Cookie values as cookies, and a body that is base64-encoded unless it is valid UTF-8 text. A text body of at least 32 KiB is gzipped when the request accepts it (Go clients do by default), so that e.g. the upload list of a deploy of thousands of files fits Lambda's 6 MB response payload. A response whose payload would still exceed that limit (measured after base64 and JSON encoding) becomes a JSON 413 that points to the server's embedded gateway, which has no such limit.
func MaxPublishBytes ¶ added in v0.2.0
MaxPublishBytes returns the decoded file bytes one publish may carry (mcp.max_publish_bytes).
func ProtectedIgnore ¶
func ProtectedIgnore() []string
ProtectedIgnore returns the default patterns no configuration can override: ".git" (directory or file) and the project files, as case-insensitive globs (".[gG][iI][tT]" …).
func ValidatePath ¶
ValidatePath reports why a site-relative path is unacceptable for a deploy or a gateway read, or "" when it is valid: ValidatePathSyntax, and no segment may be ".git" in any letter case (HasGitSegment).
The ".git" rule is defense in depth: deploys never publish repository metadata (the CLI's ignore rules cannot re-include .git/, see ProtectedIgnore), so a client sending such a path is broken or hostile, and a gateway never serves one back to a "backyard pull" (a crafted .git/config runs commands on the next git command in the pulled folder). Case-insensitive file systems (macOS, Windows) resolve ".GIT" to ".git". The Cloudflare edition's validatePath applies the same rule.
func ValidatePathSyntax ¶
ValidatePathSyntax reports why p is not a well-formed site-relative path, or "" when it is: relative, "/"-separated, no "." / ".." / empty segments, at most MaxPathBytes, valid UTF-8 without control characters or backslashes, and not under "__backyard/". It does not apply ValidatePath's ".git" rule, so clients use it for paths a site may already hold (stored before the rule, or by a direct sync) that they only need to name safely, e.g. to skip or delete them.
func WithDefaultIgnore ¶
WithDefaultIgnore returns the complete, ordered pattern list for the configured patterns: the overridable defaults, then patterns (which may re-include some of them with "!" lines), then ProtectedIgnore (which nothing can re-include). Every consumer — CollectFiles, the direct-sync exclude translation, the prune of ignored site files — evaluates this list as one gitignore sequence.
Types ¶
type AuthInfo ¶
type AuthInfo struct {
Mode string `json:"mode"`
CLILoginURL string `json:"cli_login_url,omitempty"`
Device *DeviceInfo `json:"device,omitempty"`
// AnonymousDeploy is true when deploys need no token (auth.mode=none + allow_anonymous).
AnonymousDeploy bool `json:"anonymous_deploy"`
}
AuthInfo tells clients how to obtain a token.
type Client ¶
type Client struct {
BaseURL string // gateway base URL G
Token string // bearer token ("" for anonymous)
HTTP *http.Client
UserAgent string
}
Client talks to a deploy gateway.
A Client is safe for concurrent use once configured.
func NewClient ¶
NewClient returns a client with sane timeouts: 30 s to connect, 15 s for TLS handshakes and 15 min for response headers (what the gateway allows a create, commit or refresh of a large site), no overall deadline (uploads may be long; cancel via the context). Cookies are kept for the client's lifetime so a guest identity stays stable between the create and commit calls of an anonymous deploy.
func (*Client) DeleteSite ¶
DeleteSite calls DELETE /sites/{site}.
func (*Client) Deploy ¶
func (c *Client) Deploy(ctx context.Context, site, dir string, opts DeployOptions) (*DeployResult, error)
Deploy collects files from dir (CollectFiles), creates a deploy, uploads changed files concurrently and commits.
Uploads are retried (3 times, with backoff) on network errors, throttling and 5xx responses. Bytes in the result counts uploaded bytes (for DryRun: bytes that would be uploaded). DryRun never changes anything on the server: it compares the local files with GET /sites/{site}/files when the gateway is reachable, and otherwise reports every file as an upload.
func (*Client) Me ¶
Me fetches GET /me: the identity behind the client's token (anonymous guests in auth mode none without a token).
func (*Client) ReadFile ¶
ReadFile fetches GET /sites/{site}/files/{path} (maxBytes caps the read; <=0 = 32 MiB).
type CommitRequest ¶
type CommitRequest struct {
Files []FileEntry `json:"files"`
}
CommitRequest is the body of POST {G}/sites/{site}/deploys/{id}/commit.
type CommitResponse ¶
type CommitResponse struct {
Site sites.Meta `json:"site"`
URL string `json:"url"`
Deleted int `json:"deleted"`
}
CommitResponse is returned after a successful commit.
type CreateDeployRequest ¶
type CreateDeployRequest struct {
Files []FileEntry `json:"files"`
Message string `json:"message,omitempty"`
}
CreateDeployRequest is the body of POST {G}/sites/{site}/deploys.
type CreateDeployResponse ¶
type CreateDeployResponse struct {
DeployID string `json:"deploy_id"`
Uploads []Upload `json:"uploads"`
Unchanged []string `json:"unchanged"`
ExpiresAt time.Time `json:"expires_at"`
}
CreateDeployResponse lists uploads to perform.
type DeployOptions ¶
type DeployOptions struct {
Ignore []string // extra gitignore-style patterns
Message string
Concurrency int // parallel uploads (default 8)
DryRun bool
// Progress receives progress events. Calls are serialized, so the
// callback need not be safe for concurrent use.
Progress func(ProgressEvent)
}
DeployOptions tune Client.Deploy.
type DeployResult ¶
type DeployResult struct {
Site sites.Meta
URL string
Uploaded int
Unchanged int
Deleted int
Bytes int64
Files []LocalFile // the collected files (also filled for DryRun)
}
DeployResult summarizes a deploy.
type DeviceInfo ¶
type DeviceInfo struct {
Issuer string `json:"issuer"`
ClientID string `json:"client_id"`
Scopes []string `json:"scopes"`
}
DeviceInfo enables the OAuth device authorization grant against the IdP.
type ErrorBody ¶
type ErrorBody struct {
Error struct {
Code string `json:"code"`
Message string `json:"message"`
LoginURL string `json:"login_url,omitempty"`
} `json:"error"`
}
ErrorBody is the JSON error envelope shared with the site API.
type FileEntry ¶
type FileEntry struct {
Path string `json:"path"`
Size int64 `json:"size"`
SHA256 string `json:"sha256"`
ContentType string `json:"content_type,omitempty"`
}
FileEntry describes a local file in a deploy request.
type Gateway ¶
type Gateway struct {
// contains filtered or unexported fields
}
Gateway serves the deploy protocol. Mount it with the base path stripped: requests arrive as "/info", "/sites/…", "/uploads/…".
func NewGateway ¶
func NewGateway(opts GatewayOptions) (*Gateway, error)
NewGateway validates options and returns a Gateway.
When BaseURL is empty, deploy.gateway_url is used; when that is empty too, the gateway derives its URL from each request's Host (useful behind an AWS Lambda function URL whose address is not known in advance).
func (*Gateway) CheckPublish ¶ added in v0.2.0
func (g *Gateway) CheckPublish(req *PublishRequest, decode func(i int) ([]byte, *PublishError)) *PublishError
CheckPublish runs the checks of Publish that need no storage, in their normative order (SPEC §14.1, backyard_publish steps 1–4), and fills in default content types. When decode is not nil it is called for Files[i] once the file's path checks passed and sets its Data, so an encoding error (E7) is reported in its place in the order; with a nil decode, Data is used as is.
func (*Gateway) Publish ¶ added in v0.2.0
func (g *Gateway) Publish(ctx context.Context, req PublishRequest) (*PublishResult, *PublishError)
Publish writes req to the site: by default the listed files are added or replaced and the others kept; Delete removes paths; Replace makes the site exactly Files. It runs SPEC §14.1 backyard_publish steps 1–4 and 6–10 (the caller holds the connector's lease, step 5): CheckPublish, the ownership policy, the overwrite guard, the deploys_per_hour quota (keyed user:<id>, guests included), the limits of the resulting site, then the writes, the manifest and meta switch, and deletions last.
type GatewayOptions ¶
type GatewayOptions struct {
Config *config.Config
Store blob.Store
Registry *sites.Registry
// Auth authenticates requests (the server's Authenticator when embedded,
// auth.NewBearer for the standalone gateway / Lambda).
Auth auth.Authenticator
// BaseURL is the absolute URL the gateway is reachable at (used to build
// self-hosted upload URLs and Info.GatewayURL), e.g.
// https://backyard.acme.com/__backyard/api/deploy
BaseURL string
// OnDeleteSite is called after a site is deleted (e.g. to drop its
// database documents). Optional.
OnDeleteSite func(ctx context.Context, site string) error
Logger *slog.Logger
// Version reported in Info.
Version string
// MCPURL is the remote MCP endpoint of the hub this gateway is embedded
// in (SPEC §14.1), reported as Info.MCPURL; "" omits it.
MCPURL string
}
GatewayOptions configures a Gateway.
type Info ¶
type Info struct {
Name string `json:"name"`
Version string `json:"version"`
PublicURL string `json:"public_url"`
BaseDomain string `json:"base_domain"`
URLMode string `json:"url_mode"`
GatewayURL string `json:"gateway_url"`
Auth AuthInfo `json:"auth"`
Storage *blob.Description `json:"storage,omitempty"`
Methods []string `json:"methods"`
RsyncTarget string `json:"rsync_target,omitempty"`
Limits Limits `json:"limits"`
// MCPURL is the hub's remote MCP endpoint (SPEC §7.1, §14.1): present
// while the Claude connector is enabled, on the hub's embedded gateway
// only.
MCPURL string `json:"mcp_url,omitempty"`
}
Info is returned by GET {gateway}/info (no auth).
type Limits ¶
type Limits struct {
MaxSiteBytes int64 `json:"max_site_bytes"`
MaxFiles int `json:"max_files"`
MaxFileBytes int64 `json:"max_file_bytes"`
}
Limits advertised to clients.
type LocalFile ¶
type LocalFile struct {
Path string // site-relative, "/"-separated
Abs string // absolute path on disk
Size int64
SHA256 string // hex
ContentType string
}
LocalFile is a file selected for deployment.
func CollectFiles ¶
CollectFiles walks dir applying the default ignore rules, dir/.backyardignore and extra patterns (gitignore syntax, see IgnorePatterns), hashing every selected file. Symlinks to files are followed; symlinked directories are not.
Symlinks whose target lies outside dir, broken symlinks and special files (sockets, devices, pipes) are skipped. A selected file whose path the gateway would reject (e.g. under "__backyard/") is an error. The result is sorted by Path.
type Matcher ¶
type Matcher struct {
// contains filtered or unexported fields
}
Matcher evaluates gitignore-style patterns against site-relative paths.
Supported syntax: blank lines and "#" comments, "!" negation (the last matching pattern wins), a trailing "/" for directory-only patterns, a leading or inner "/" to anchor a pattern at the root (otherwise it matches a name at any depth), "**" for any number of directories ("**/x", "x/**", "a/**/b"), and path.Match globs ("*", "?", "[…]", "\" escapes) within a segment. As in git, a path inside an excluded directory cannot be re-included. Invalid globs are ignored.
func NewMatcher ¶
NewMatcher compiles patterns (one gitignore line each).
type OwnersRequest ¶
type OwnersRequest struct {
Add []string `json:"add,omitempty"`
Remove []string `json:"remove,omitempty"`
}
OwnersRequest is the body of POST {G}/sites/{site}/owners.
type ProgressEvent ¶
type ProgressEvent struct {
Phase string // "scan" | "plan" | "upload" | "commit" | "done"
Path string
Done int
Total int
Bytes int64
}
ProgressEvent reports deploy progress.
type PublishError ¶ added in v0.2.0
PublishError is a refusal or failure with its catalog id ("E2" … "E27", the tool errors of SPEC §14.1) and the tool error text.
func (*PublishError) Error ¶ added in v0.2.0
func (e *PublishError) Error() string
type PublishFile ¶ added in v0.2.0
PublishFile is one file of a publish call, already decoded.
type PublishRequest ¶ added in v0.2.0
type PublishRequest struct {
User *identity.User
Site string
Files []PublishFile
Delete []string
Replace bool
Overwrite bool
// ClientIP, ClientName and GrantID go into the deploy.commit audit record.
ClientIP, ClientName, GrantID string
}
PublishRequest is one backyard_publish call.
type PublishResult ¶ added in v0.2.0
type PublishResult struct {
Site, URL string
Created bool
Files, Uploaded, Unchanged, Deleted int
Bytes int64
Warnings []string
}
PublishResult describes the site after a publish.