source

package
v0.7.22-dev.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 4, 2026 License: MIT Imports: 25 Imported by: 0

Documentation

Overview

Package source is the runner image's `source` role: the fetch step that brings a run's input source in before the run, as plain files under one directory. The repository kind is a depth-1 fetch of one commit over git's smart HTTP protocol, spoken by this package itself (git.go, pack.go) rather than by a git binary or a third-party library: the image builds with no module download, and no shell or git runs in the fetch container. The checkout writes regular files only (tree.go): no link, no submodule, no .git, LFS pointer files as they are. The url kind (url.go) is one https GET of a file, saved as one file under a portable name.

The caller runs this in the files container on a network whose only route is a CONNECT proxy to the source's host on port 443, with the access token, when there is one, on stdin: never in the environment or an argument. The token is sent to that host alone: as HTTP basic auth to a repository, as a bearer token for a file.

The output directory is mounted into that container alone, the container's one process is this package, and this package starts no other. So no process can write the output directory, or swap a folder of it for a link, while a call here writes it, and it is held with os.Root, which refuses a link out of the directory where the runtime applies O_NOFOLLOW to a folder, rather than with beneath.Dir, which guards a directory another process can write.

This package is part of the embedded runner-image build context and so imports nothing outside the standard library.

Index

Constants

View Source
const (
	Schema     = "source/v1"
	ReportFile = "source.json"
	// KindRepository is a git repository at a ref (KindURL is the other).
	KindRepository = "repository"
)

Schema is the report's format; ReportFile is where Fetch writes it, in the work directory, beside nothing the agent reads.

View Source
const (
	DefaultMaxFiles = 20000
	DefaultMaxBytes = 1 << 30
)

The default caps: a local repository.

View Source
const (
	// CapFiles and CapBytes: Options.MaxFiles and Options.MaxBytes.
	CapFiles = "files"
	CapBytes = "bytes"
	// OfFiles and OfBytes: the tree's files and their bytes, or a url
	// source's one file.
	OfFiles = "files"
	OfBytes = "bytes"
	// OfObjects: the objects a pack says it holds, more than a repository
	// within the file cap can.
	OfObjects = "objects"
	// OfEntries: the tree's files, folders and links, bounded at a
	// multiple of the file cap.
	OfEntries = "entries"
	// OfPackBytes and OfInflatedBytes: the pack as sent and as inflated,
	// each bounded past what a repository within the byte cap makes.
	OfPackBytes     = "pack-bytes"
	OfInflatedBytes = "inflated-bytes"
)

The caps an Over names, and what it counts: the words of agentry-cli's receipt.ReasonCount, which this package cannot import (the engine's TestTheFetchStepCountsInTheReceiptsWords holds them to it).

View Source
const (
	SkipLink      = "link"
	SkipSubmodule = "submodule"
	SkipGit       = "git"
	SkipName      = "name"
	SkipCase      = "case"
	SkipDeep      = "deep"
)

The reasons an entry is left out of the checkout.

View Source
const (
	// DefaultURLMaxBytes is a url source's byte cap locally: the most of
	// one document the host reads back from a decode
	// (pkg/runner/decode.MaxDocumentBytes, held equal by a test in
	// pkg/runner/sources).
	DefaultURLMaxBytes = 256 << 20
)
View Source
const KindURL = "url"

KindURL is the url kind.

Variables

This section is empty.

Functions

func CheckFileURL

func CheckFileURL(raw string) (*url.URL, error)

CheckFileURL holds a url source's address to what a source may be: https, a plain host, no credentials, no port but 443, and no fragment (which is never sent; the caller drops it). A query is kept: a download address often needs one.

func CheckURL

func CheckURL(raw string) (*url.URL, error)

CheckURL holds a repository address to what a source may be: https, a host, no credentials, no query or fragment (the ref travels apart), and no port but 443.

func PortableFileName

func PortableFileName(raw string) string

PortableFileName makes a name the host or the address gave a plain file name on Linux, macOS and Windows alike, or answers "" when nothing of it is left: its last path segment alone; controls, the marks that reorder or hide text and Windows' reserved characters as "_"; no trailing dot or space; a device name prefixed with "_"; not git's own folder; within maxFileName bytes, its extension kept. The result passes portableName and the member-name rule (decode.PlainMemberPath).

func TokenForms

func TokenForms(token string) []string

TokenForms are the spellings of a token a host could echo back: the token, its password half, and the basic-auth header's value. Each is scrubbed from what the fetch reports.

Types

type Options

type Options struct {
	// Kind is the source kind: KindRepository or KindURL.
	Kind string
	// URL is the repository's or the file's https address, with no
	// credentials.
	URL string
	// Ref is the branch, tag or full commit name; empty is the default
	// branch. A url source has none.
	Ref string
	// Token is the access token, empty for anonymous. A repository sends
	// it as basic auth (a token holding a colon is user:token; otherwise
	// the user is x-access-token), a url source as a bearer token.
	Token string
	// Proxy is the CONNECT proxy every request goes through; nil is a
	// direct connection (tests only).
	Proxy *url.URL
	// Out is the empty directory the files are written to; Work the
	// directory the pack and the report are written to.
	Out, Work string
	// MaxFiles and MaxBytes are the caps; over either the fetch is
	// refused and nothing is written to Out.
	MaxFiles int
	MaxBytes int64
	// Transport replaces the HTTP transport (tests only).
	Transport http.RoundTripper
	// contains filtered or unexported fields
}

Options is one fetch.

type Over

type Over struct {
	Cap   string `json:"cap"`
	Of    string `json:"of"`
	Count int64  `json:"count"`
	More  bool   `json:"more,omitempty"`
}

Over is a refusal's count: the cap that refused it (CapFiles or CapBytes; its value is the caller's, who set it), what the step counted (Of*) and how many, and More when it stopped counting there.

type Report

type Report struct {
	Schema string `json:"schema"`
	Kind   string `json:"kind"`
	// Ref is the ref resolved (a full ref name, or the commit asked for);
	// Commit the commit checked out.
	Ref    string `json:"ref,omitempty"`
	Commit string `json:"commit,omitempty"`
	// Files and Bytes are what was written.
	Files int   `json:"files"`
	Bytes int64 `json:"bytes"`
	// Name is the one file a url source saved, under Out.
	Name string `json:"name,omitempty"`
	// LFSPointers counts the files that are Git LFS pointers.
	LFSPointers int `json:"lfsPointers,omitempty"`
	// Skipped counts what was left out, by reason; SkippedPaths names the
	// first few of each.
	Skipped      map[string]int      `json:"skipped,omitempty"`
	SkippedPaths map[string][]string `json:"skippedPaths,omitempty"`
	// Refusal says why nothing was written, when a cap refused the source:
	// its count against the cap. Over is the same count in typed fields,
	// which a driver that keeps no text can keep.
	Refusal string `json:"refusal,omitempty"`
	Over    *Over  `json:"over,omitempty"`
	// Error says why the fetch failed, in words for the person.
	Error string `json:"error,omitempty"`
	// Protocol is the git protocol the host spoke.
	Protocol string `json:"protocol,omitempty"`
}

Report is what one fetch did.

func Fetch

func Fetch(ctx context.Context, o Options) (Report, error)

Fetch fetches one source and writes its report to Work. The error is for a fetch that could not be made; a refusal is a report.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL