sourceupload

package
v0.8.6 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 27, 2026 License: Apache-2.0 Imports: 20 Imported by: 0

Documentation

Overview

Package sourceupload packs a local directory into a deterministic archive, stores uploaded archives for a bounded time and extracts them safely.

Index

Constants

This section is empty.

Variables

View Source
var DefaultIgnores = []string{
	".git", ".svn", ".hg", "node_modules", ".next", ".nuxt", ".turbo", ".venv", "venv",
	"__pycache__", "target", "coverage", ".terraform", ".DS_Store",
	".env", ".env.*", "*.pem", "*.key", "id_rsa*", "id_ed25519*",
}

DefaultIgnores are never uploaded unless named in Options.Include. Secrets and dependency or cache directories stay on the laptop by default.

View Source
var ErrEmptyArchive = errors.New("sourceupload: archive has no files")

ErrEmptyArchive is returned when a zip holds no files once OS metadata is ignored.

View Source
var ErrInvalidName = errors.New("sourceupload: invalid app name or hash")

ErrInvalidName is returned for an app name or hash that is not a safe path segment.

View Source
var ErrNotFound = errors.New("sourceupload: upload not found")

ErrNotFound is returned when an upload is unknown or already expired.

View Source
var ErrTooLarge = errors.New("sourceupload: source exceeds the size limit")

ErrTooLarge is returned when a directory or archive exceeds its size cap.

View Source
var ErrUnsafeArchive = errors.New("sourceupload: unsafe archive entry")

ErrUnsafeArchive is returned for an entry that could escape the extraction root or is not a plain file.

Functions

func ValidHash

func ValidHash(s string) bool

ValidHash reports whether s is a lowercase hex sha256.

Types

type File

type File struct {
	Path string
	Size int64
	Mode fs.FileMode
}

File is one entry that will be uploaded.

type Limits

type Limits struct {
	MaxFiles     int
	MaxBytes     int64
	MaxFileBytes int64
}

Limits cap what an archive may expand to. Zero fields use the defaults.

type Meta

type Meta struct {
	Hash      string    `json:"hash"`
	App       string    `json:"app"`
	Size      int64     `json:"size"`
	Files     int       `json:"file_count"`
	Expanded  int64     `json:"extracted_bytes"`
	CreatedAt time.Time `json:"created_at"`
}

Meta describes one stored upload.

type Options

type Options struct {
	// Include re-adds paths the default and .gitignore rules would exclude, for example ".env".
	Include []string
	// MaxBytes caps the total uncompressed size of the files; 0 means no cap.
	MaxBytes int64
}

Options tune Pack.

type Plan

type Plan struct {
	Root  string
	Files []File
	Total int64
}

Plan is the resolved list of files, shown to the user before anything is sent.

func Scan

func Scan(ctx context.Context, root string, opts Options) (Plan, error)

Scan walks root applying the ignore rules. Symlinks and special files are skipped.

func (Plan) Largest

func (p Plan) Largest(n int) []File

Largest returns up to n files, biggest first.

func (Plan) WriteArchive

func (p Plan) WriteArchive(ctx context.Context, w io.Writer) error

WriteArchive writes the plan as a gzip tar. Entries are sorted with zeroed times, owners and normalised modes so the same tree always hashes the same.

type Stats

type Stats struct {
	Files int
	Bytes int64
}

Stats describes an archive's contents.

func Extract

func Extract(r io.Reader, dest string, lim Limits) (Stats, error)

Extract unpacks r under dest, refusing anything but regular files and directories.

func Inspect

func Inspect(r io.Reader, lim Limits) (Stats, error)

Inspect validates every entry and counts files and bytes without writing anything.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store keeps archives as <root>/<app>/<hash>.tar.gz with a .json sidecar.

func NewStore

func NewStore(dir string, lim Limits) *Store

NewStore returns a Store rooted at dir. The directory is created on first Put.

func (*Store) Cleanup

func (s *Store) Cleanup(retention time.Duration) (int, error)

Cleanup removes uploads last stored or re-sent more than retention ago, plus abandoned partial files, and returns how many uploads it removed.

func (*Store) Extract

func (s *Store) Extract(app, hash, dest string) (Stats, error)

Extract unpacks a stored upload under dest.

func (*Store) Get

func (s *Store) Get(app, hash string) (Meta, error)

Get returns the metadata of a stored upload, or ErrNotFound.

func (*Store) List

func (s *Store) List(app string) ([]Meta, error)

List returns an app's stored uploads, newest first.

func (*Store) Open

func (s *Store) Open(app, hash string) (*os.File, error)

Open returns the stored archive for reading.

func (*Store) Put

func (s *Store) Put(app string, r io.Reader, maxBytes int64) (meta Meta, deduped bool, err error)

Put streams r into the store, enforcing maxBytes on the compressed size and validating the archive before it becomes visible. deduped is true when the same content already existed for the app.

func (*Store) Usage

func (s *Store) Usage() (int64, error)

Usage returns the total bytes of stored archives.

type ZipResult

type ZipResult struct {
	Stats
	// HasIndex reports whether index.html sits at the site root after wrapper folders are stripped.
	HasIndex bool
	// StrippedDir is the wrapper folder path removed so the site root is the archive root.
	StrippedDir string
}

ZipResult describes a converted zip.

func ConvertZip

func ConvertZip(src io.ReaderAt, size int64, w io.Writer, lim Limits) (ZipResult, error)

ConvertZip validates a zip and rewrites it as the gzip tar the store keeps. Wrapper folders holding the whole site are stripped; symlinks and escaping paths are refused.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL