lfs

package module
v0.0.0-...-f7d8164 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 11, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Overview

Package lfs implements the server side of the Git LFS Batch API and the basic transfer adapter on top of pluggable metadata and object storage.

It leaves HTTP routing, authentication, authorization, and the encoding of responses to the caller, so that servers with different URL layouts, credentials, and databases can share the transfer logic.

Spec: https://github.com/git-lfs/git-lfs/blob/main/docs/api/batch.md

Index

Constants

View Source
const (
	OperationUpload   = "upload"
	OperationDownload = "download"
	TransferBasic     = "basic"
)
View Source
const (
	// MediaType is the content type of LFS API requests and responses.
	MediaType = "application/vnd.git-lfs+json"
)

Variables

This section is empty.

Functions

func ValidOID

func ValidOID(oid string) bool

ValidOID reports whether oid is a valid LFS object ID.

func WriteError

func WriteError(w http.ResponseWriter, code int, msg string)

WriteError writes an error response in the format required by the LFS API: a JSON body of {"message": "..."} with the MediaType content type. Callers use it for the errors of their own middleware, e.g., authentication, so clients get a parseable body on every error.

Types

type Access

type Access struct {
	// Repo identifies the repository to the Store.
	Repo string
	// Href is the URL that the transfer endpoints of the repository are served
	// under, without a trailing slash: objects at Href/{oid} and verifications
	// at Href/verify.
	Href string
	// CanRead allows downloads.
	CanRead bool
	// CanWrite allows uploads and verifications.
	CanWrite bool
}

Access is what the caller resolved for a request after authenticating it.

type Action

type Action struct {
	Href   string            `json:"href"`
	Header map[string]string `json:"header,omitempty"`
}

Action tells the client where and how to transfer an object.

type BatchObject

type BatchObject struct {
	OID  string `json:"oid"`
	Size int64  `json:"size"`
}

BatchObject is an object requested in a BatchRequest.

type BatchRequest

type BatchRequest struct {
	Operation string        `json:"operation"`
	Transfers []string      `json:"transfers,omitempty"`
	Objects   []BatchObject `json:"objects"`
}

BatchRequest is the request payload of the batch endpoint.

type BatchResponse

type BatchResponse struct {
	Transfer string                `json:"transfer"`
	Objects  []BatchResponseObject `json:"objects"`
}

BatchResponse is the response payload of the batch endpoint.

type BatchResponseObject

type BatchResponseObject struct {
	OID           string            `json:"oid"`
	Size          int64             `json:"size"`
	Authenticated bool              `json:"authenticated,omitempty"`
	Actions       map[string]Action `json:"actions,omitempty"`
	Error         *ObjectError      `json:"error,omitempty"`
}

BatchResponseObject is the result for one object of a BatchRequest. Either Actions or Error is set, unless the object needs no transfer.

type DownloadResult

type DownloadResult struct {
	// Redirect is set when the client must download the content from another
	// location.
	Redirect *Redirect
	// Content is the object content, which the caller must close.
	Content io.ReadCloser
	// Size is the size of Content in bytes.
	Size int64
}

DownloadResult is the outcome of a successful Service.Download. Exactly one of Redirect and Content is set.

type Endpoint

type Endpoint struct {
	Href string
	// Header is included in every action, e.g., to forward the credentials of
	// the batch request.
	Header map[string]string
	// Authenticated tells the client that Header authenticates the actions, so
	// it does not look up credentials for them. It is required when Storage is a
	// storage.Presigner: otherwise the Git LFS client looks up credentials for
	// the host of the presigned URL it is redirected to, and fails.
	Authenticated bool
}

Endpoint describes how the client reaches the transfer endpoints of a repository: objects at Href/{oid} and verifications at Href/verify.

type Error

type Error struct {
	Code    int
	Message string
}

Error is a client error that the caller should respond with, using Code as the HTTP status code.

func (*Error) Error

func (e *Error) Error() string

type Object

type Object struct {
	OID  string
	Size int64
	// URI is the location of the object content, as returned by the storage
	// backend that stores it.
	URI string
}

Object is a verified LFS object.

type ObjectError

type ObjectError struct {
	Code    int    `json:"code"`
	Message string `json:"message"`
}

ObjectError is the error of one object in a BatchResponse.

type Redirect

type Redirect struct {
	URL string
	// Header must be sent by the client verbatim.
	Header map[string]string
}

Redirect sends the client to another location to transfer the content.

type Service

type Service struct {
	Store Store
	// Storage receives uploads and decides the transfer mode: content is
	// proxied through the caller for a storage.Proxier, and transferred
	// directly by the client through redirects for a storage.Presigner.
	Storage storage.Backend
	// PreviousStorages are consulted for downloads of objects stored before
	// Storage was changed. An object is served by the first backend, starting
	// with Storage, whose scheme prefixes the object's URI.
	PreviousStorages []storage.Backend
	// MaxObjectSize is the maximum size in bytes of an uploaded object. Zero or
	// negative means no limit.
	MaxObjectSize int64
	// CrossRepoDedup allows an upload to be satisfied by an object that is
	// already stored for another repository, without the client sending the
	// content again. Without it, a repository only gains an object by uploading
	// its full content, so knowing an OID and its size is not enough to read the
	// object through a repository the client has write access to.
	CrossRepoDedup bool
}

Service serves LFS operations for repositories. Callers authenticate and authorize each request, then either hand it to Serve, or call the operation methods and respond with the Code and Message of a returned *Error. Any other error returned by an operation method is an internal error.

func (*Service) Batch

func (s *Service) Batch(ctx context.Context, logger *slog.Logger, repo string, endpoint Endpoint, req BatchRequest) (*BatchResponse, error)

Batch answers a batch request for repo. Callers must check that the actor is allowed to perform req.Operation before calling it.

func (*Service) Download

func (s *Service) Download(ctx context.Context, repo, oid string) (*DownloadResult, error)

Download handles a download of oid from repo.

func (*Service) Serve

func (s *Service) Serve(logger *slog.Logger, w http.ResponseWriter, r *http.Request, access Access)

Serve handles an LFS API request to the repository in access. It dispatches on the method and the last segment of the URL path, so it serves any URL layout whose endpoints end in "batch", "verify", and the OID:

  • POST .../batch
  • GET .../{oid}
  • PUT .../{oid}
  • POST .../verify

The request's Authorization header is forwarded in the actions of batch responses, so the client authenticates transfers with the same credentials.

func (*Service) Upload

func (s *Service) Upload(ctx context.Context, logger *slog.Logger, repo, oid string, size int64, body io.Reader) (*UploadResult, error)

Upload handles an upload of size bytes for oid to repo. For a storage.Proxier, it stores the content read from body after verifying its size and hash. For a storage.Presigner, it does not read body.

func (*Service) Verify

func (s *Service) Verify(ctx context.Context, logger *slog.Logger, repo string, req VerifyRequest) error

Verify handles the verification of an upload to repo. A presigned upload becomes downloadable only after it is verified.

type Store

type Store interface {
	// GetRepoObjects returns the verified objects among oids that are linked to
	// repo, keyed by OID. Missing OIDs are omitted.
	GetRepoObjects(ctx context.Context, repo string, oids []string) (map[string]Object, error)
	// GetObjects returns the verified objects among oids regardless of the
	// repositories they are linked to, keyed by OID. Missing OIDs are omitted.
	// It is only called when Service.CrossRepoDedup is true.
	GetObjects(ctx context.Context, oids []string) (map[string]Object, error)
	// LinkObject records a verified object whose content is stored at uri and
	// links it to repo. It returns the URI the object is recorded with, and when
	// that differs from uri, the caller deletes the content at uri as no longer
	// referenced.
	LinkObject(ctx context.Context, repo, oid string, size int64, uri string) (storedURI string, err error)
	// InsertPendingObject records a pending object for oid, unless the object is
	// already recorded, so that abandoned presigned uploads can be cleaned up.
	InsertPendingObject(ctx context.Context, oid string) error
	// VerifyObject marks the pending object for oid as verified with content
	// stored at uri and links it to repo. It returns the URI the object is
	// recorded with, like LinkObject. It fails when no object is recorded for
	// oid.
	VerifyObject(ctx context.Context, repo, oid string, size int64, uri string) (storedURI string, err error)
}

Store is the metadata layer of LFS objects. An object is "pending" from the start of a presigned upload until it is verified, and only verified objects are served. A repository is identified by an opaque string chosen by the caller.

type UploadResult

type UploadResult struct {
	// Redirect is set when the client must upload the content to another
	// location and then call the verify endpoint.
	Redirect *Redirect
	// Existed reports that the object was already stored and the content was not
	// read.
	Existed bool
}

UploadResult is the outcome of a successful Service.Upload.

type VerifyRequest

type VerifyRequest struct {
	OID  string `json:"oid"`
	Size int64  `json:"size"`
}

VerifyRequest is the request payload of the verify endpoint.

Directories

Path Synopsis
internal
iox

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL