Documentation
¶
Overview ¶
Package source is the runner image's `source` role: the fetch step that brings a run's input source in before the run, as plain files under one directory. The repository kind is a depth-1 fetch of one commit over git's smart HTTP protocol, spoken by this package itself (git.go, pack.go) rather than by a git binary or a third-party library: the image builds with no module download, and no shell or git runs in the fetch container. The checkout writes regular files only (tree.go): no link, no submodule, no .git, LFS pointer files as they are. The url kind (url.go) is one https GET of a file, saved as one file under a portable name.
The caller runs this in the files container on a network whose only route is a CONNECT proxy to the source's host on port 443, with the access token, when there is one, on stdin: never in the environment or an argument. The token is sent to that host alone: as HTTP basic auth to a repository, as a bearer token for a file.
The output directory is mounted into that container alone, the container's one process is this package, and this package starts no other. So no process can write the output directory, or swap a folder of it for a link, while a call here writes it, and it is held with os.Root, which refuses a link out of the directory where the runtime applies O_NOFOLLOW to a folder, rather than with beneath.Dir, which guards a directory another process can write.
This package is part of the embedded runner-image build context and so imports nothing outside the standard library.
Index ¶
Constants ¶
const ( Schema = "source/v1" ReportFile = "source.json" // KindRepository is a git repository at a ref (KindURL is the other). KindRepository = "repository" )
Schema is the report's format; ReportFile is where Fetch writes it, in the work directory, beside nothing the agent reads.
const ( DefaultMaxFiles = 20000 DefaultMaxBytes = 1 << 30 )
The default caps: a local repository.
const ( // CapFiles and CapBytes: Options.MaxFiles and Options.MaxBytes. CapFiles = "files" CapBytes = "bytes" // OfFiles and OfBytes: the tree's files and their bytes, or a url // source's one file. OfFiles = "files" OfBytes = "bytes" // OfObjects: the objects a pack says it holds, more than a repository // within the file cap can. OfObjects = "objects" // OfEntries: the tree's files, folders and links, bounded at a // multiple of the file cap. OfEntries = "entries" // OfPackBytes and OfInflatedBytes: the pack as sent and as inflated, // each bounded past what a repository within the byte cap makes. OfPackBytes = "pack-bytes" OfInflatedBytes = "inflated-bytes" )
The caps an Over names, and what it counts: the words of agentry-cli's receipt.ReasonCount, which this package cannot import (the engine's TestTheFetchStepCountsInTheReceiptsWords holds them to it).
const ( SkipLink = "link" SkipSubmodule = "submodule" SkipGit = "git" SkipName = "name" SkipCase = "case" SkipDeep = "deep" )
The reasons an entry is left out of the checkout.
const ( // DefaultURLMaxBytes is a url source's byte cap locally: the most of // one document the host reads back from a decode // (pkg/runner/decode.MaxDocumentBytes, held equal by a test in // pkg/runner/sources). DefaultURLMaxBytes = 256 << 20 )
const KindURL = "url"
KindURL is the url kind.
Variables ¶
This section is empty.
Functions ¶
func CheckFileURL ¶
CheckFileURL holds a url source's address to what a source may be: https, a plain host, no credentials, no port but 443, and no fragment (which is never sent; the caller drops it). A query is kept: a download address often needs one.
func CheckURL ¶
CheckURL holds a repository address to what a source may be: https, a host, no credentials, no query or fragment (the ref travels apart), and no port but 443.
func PortableFileName ¶
PortableFileName makes a name the host or the address gave a plain file name on Linux, macOS and Windows alike, or answers "" when nothing of it is left: its last path segment alone; controls, the marks that reorder or hide text and Windows' reserved characters as "_"; no trailing dot or space; a device name prefixed with "_"; not git's own folder; within maxFileName bytes, its extension kept. The result passes portableName and the member-name rule (decode.PlainMemberPath).
func TokenForms ¶
TokenForms are the spellings of a token a host could echo back: the token, its password half, and the basic-auth header's value. Each is scrubbed from what the fetch reports.
Types ¶
type Options ¶
type Options struct {
// Kind is the source kind: KindRepository or KindURL.
Kind string
// URL is the repository's or the file's https address, with no
// credentials.
URL string
// Ref is the branch, tag or full commit name; empty is the default
// branch. A url source has none.
Ref string
// Token is the access token, empty for anonymous. A repository sends
// it as basic auth (a token holding a colon is user:token; otherwise
// the user is x-access-token), a url source as a bearer token.
Token string
// Proxy is the CONNECT proxy every request goes through; nil is a
// direct connection (tests only).
Proxy *url.URL
// Out is the empty directory the files are written to; Work the
// directory the pack and the report are written to.
Out, Work string
// MaxFiles and MaxBytes are the caps; over either the fetch is
// refused and nothing is written to Out.
MaxFiles int
MaxBytes int64
// Transport replaces the HTTP transport (tests only).
Transport http.RoundTripper
// contains filtered or unexported fields
}
Options is one fetch.
type Over ¶
type Over struct {
Cap string `json:"cap"`
Of string `json:"of"`
Count int64 `json:"count"`
More bool `json:"more,omitempty"`
}
Over is a refusal's count: the cap that refused it (CapFiles or CapBytes; its value is the caller's, who set it), what the step counted (Of*) and how many, and More when it stopped counting there.
type Report ¶
type Report struct {
Schema string `json:"schema"`
Kind string `json:"kind"`
// Ref is the ref resolved (a full ref name, or the commit asked for);
// Commit the commit checked out.
Ref string `json:"ref,omitempty"`
Commit string `json:"commit,omitempty"`
// Files and Bytes are what was written.
Files int `json:"files"`
Bytes int64 `json:"bytes"`
// Name is the one file a url source saved, under Out.
Name string `json:"name,omitempty"`
// LFSPointers counts the files that are Git LFS pointers.
LFSPointers int `json:"lfsPointers,omitempty"`
// Skipped counts what was left out, by reason; SkippedPaths names the
// first few of each.
Skipped map[string]int `json:"skipped,omitempty"`
SkippedPaths map[string][]string `json:"skippedPaths,omitempty"`
// Refusal says why nothing was written, when a cap refused the source:
// its count against the cap. Over is the same count in typed fields,
// which a driver that keeps no text can keep.
Refusal string `json:"refusal,omitempty"`
Over *Over `json:"over,omitempty"`
// Error says why the fetch failed, in words for the person.
Error string `json:"error,omitempty"`
// Protocol is the git protocol the host spoke.
Protocol string `json:"protocol,omitempty"`
}
Report is what one fetch did.