Documentation
¶
Overview ¶
Package pktlineeditor provides a streaming editor for manipulating Git pkt-line protocol responses.
The editor allows manipulation of pkt-line headers without buffering multi-gigabyte packfile bodies. It uses a two-phase streaming approach:
- Buffer and transform the pkt-line header section (typically <1 MB)
- Stream the binary body directly without buffering (potentially gigabytes)
Usage ¶
Create a transformed reader:
reader, delta, err := pktlineeditor.TransformReader(
responseBody,
pktlineeditor.InjectCapability("packfile-uris"),
nil, // use default options
)
if err != nil {
log.Printf("transformation failed, using fallback: %v", err)
}
// Adjust Content-Length header if needed: newLength = originalLength + delta
io.Copy(output, reader) // always safe to read, even on error
Transform functions can be composed:
transform := pktlineeditor.Compose(
pktlineeditor.InjectCapability("packfile-uris"),
pktlineeditor.FilterLines(func(line []byte) bool {
return bytes.HasPrefix(line, []byte("debug:"))
}),
)
reader, delta, err := pktlineeditor.TransformReader(responseBody, transform, nil)
Index ¶
- Variables
- func BuildURIFetchResponse(headerSections []Section, uri PackfileURI, format ObjectFormat) ([]byte, error)
- func EmptyPack(format ObjectFormat) []byte
- func SplitFetchResponse(src io.Reader, format ObjectFormat) ([]Section, *PackReader, error)
- func TransformReader(source io.Reader, transformer TransformFunc, opts *Options) (io.Reader, int64, error)
- func WriteInlineFetchResponse(dst io.Writer, headerSections []Section, pack io.Reader, format ObjectFormat) (string, error)
- type ObjectFormat
- type Options
- type PackReader
- type PackfileURI
- type Section
- type TransformFunc
Constants ¶
This section is empty.
Variables ¶
var ( ObjectFormatSHA1 = ObjectFormat{/* contains filtered or unexported fields */} ObjectFormatSHA256 = ObjectFormat{/* contains filtered or unexported fields */} )
The two object formats git supports. hashLen is the object hash size in bytes, which is also the length of a pack's checksum trailer.
var ErrInvalidPktLineLength = errors.New("invalid pkt-line length")
ErrInvalidPktLineLength is returned when a pkt-line length is invalid.
var ErrInvalidPosition = errors.New("invalid position for line injection")
ErrInvalidPosition is returned when attempting to inject lines at an invalid position.
var ErrMaxSizeExceeded = errors.New("peek size exceeds maximum buffer size")
ErrMaxSizeExceeded is returned when a peek operation requests more bytes than the maximum allowed size.
var ErrNoPackfileSection = errors.New("no packfile section in response")
ErrNoPackfileSection is returned when a git-upload-pack result has no packfile section to extract a pack from.
Functions ¶
func BuildURIFetchResponse ¶ added in v1.42.0
func BuildURIFetchResponse(headerSections []Section, uri PackfileURI, format ObjectFormat) ([]byte, error)
BuildURIFetchResponse builds a git-upload-pack v2 response that sends the pack out of band. The client gets every object from the packfile-uri and an empty inline pack.
The empty packfile section is always band-1 framed: a protocol-v2 packfile section is always sideband-multiplexed, which is the only protocol Packhorse speaks. format selects the empty pack's checksum trailer so it matches the client's repository object format.
func EmptyPack ¶ added in v1.42.0
func EmptyPack(format ObjectFormat) []byte
EmptyPack returns a copy of the zero-object packfile for the object format.
func SplitFetchResponse ¶ added in v1.42.0
func SplitFetchResponse(src io.Reader, format ObjectFormat) ([]Section, *PackReader, error)
SplitFetchResponse separates a git-upload-pack v2 result into the sections that come before the packfile (for example shallow-info) and a reader over the raw pack bytes. The pack is streamed rather than buffered.
func TransformReader ¶
func TransformReader(source io.Reader, transformer TransformFunc, opts *Options) (io.Reader, int64, error)
TransformReader creates a new reader that transforms pkt-line headers while streaming the body. It parses the header section, applies the transformation, then returns a reader that streams the transformed header followed by the unchanged body.
The delta return value indicates the change in total stream size caused by the transformation, calculated as (newHeaderSize - originalHeaderSize). This can be used to adjust Content-Length headers in HTTP responses. On error or fallback to original stream, delta will be 0.
IMPORTANT: This function has non-standard behavior - it ALWAYS returns a valid io.Reader, even when an error occurs. On error, the returned reader will stream the original data (including any bytes already buffered by the parser). This allows graceful fallback to the original stream without data loss. Callers should check the error for logging/testing purposes, but can always safely read from the returned reader.
func WriteInlineFetchResponse ¶ added in v1.43.0
func WriteInlineFetchResponse(dst io.Writer, headerSections []Section, pack io.Reader, format ObjectFormat) (string, error)
WriteInlineFetchResponse writes a git-upload-pack v2 result that carries the whole pack inline. It replays headerSections, then frames pack into band-1 pkt-lines, and returns the hex-encoded checksum of the pack it wrote.
This is the inverse of SplitFetchResponse, so it turns a bare pack held in object storage back into the response upstream sent when that pack was cached. The packfile section is always band-1 framed because a protocol-v2 packfile section is always sideband-multiplexed. format sets the length of the checksum trailer the returned hash is taken from.
Types ¶
type ObjectFormat ¶ added in v1.42.0
type ObjectFormat struct {
// contains filtered or unexported fields
}
ObjectFormat is a git object hash format. It sets the pack checksum trailer length and the framing of an empty pack, both of which differ between SHA1 and SHA256 repositories.
func ObjectFormatByName ¶ added in v1.42.0
func ObjectFormatByName(name string) ObjectFormat
ObjectFormatByName maps a git "object-format" capability value to its ObjectFormat. An empty or unrecognised value is treated as SHA1, which is git's default when the capability is absent.
func (ObjectFormat) Name ¶ added in v1.42.0
func (f ObjectFormat) Name() string
Name returns the git "object-format" capability value for the format, for example "sha1" or "sha256".
type Options ¶
type Options struct {
MaxHeaderSize int64 // Maximum header size (default: 10 MB)
}
Options for configuring the pkt-line editor.
type PackReader ¶ added in v1.42.0
type PackReader struct {
// contains filtered or unexported fields
}
PackReader is a reader over the raw pack bytes of a git-upload-pack packfile section. It removes the sideband framing so callers get the pack as git wrote it.
func (*PackReader) PackHash ¶ added in v1.42.0
func (pr *PackReader) PackHash() string
PackHash returns the hex-encoded pack checksum. It is only meaningful after the reader has been read to EOF.
func (*PackReader) Read ¶ added in v1.42.0
func (pr *PackReader) Read(p []byte) (int, error)
Read implements io.Reader over the de-sidebanded pack bytes.
func (*PackReader) ValidPack ¶ added in v1.42.0
func (pr *PackReader) ValidPack() bool
ValidPack reports whether a well-formed pack was read: it starts with the "PACK" signature and is at least the 12-byte header plus the checksum trailer (20 bytes for SHA1, 32 for SHA256). It is only meaningful after the reader has been read to EOF.
type PackfileURI ¶
type PackfileURI struct {
URI string // The URI where the packfile can be fetched
Hash string // The 40-character hexadecimal SHA-1 hash of the packfile
}
PackfileURI represents a packfile URI with its corresponding hash.
type Section ¶
type Section struct {
Name string `json:"name"` // Section name (e.g., "shallow-info", "acknowledgments", "wanted-refs", "packfile-uris") - stored without newline
Lines [][]byte `json:"lines"` // Lines in the section (WITH trailing newline if present in original data)
}
Section represents a Git protocol v2 response section. Each section has a name (e.g., "shallow-info", "acknowledgments") and lines of data.
type TransformFunc ¶
TransformFunc receives Git protocol v2 sections and returns transformed sections. Lines are represented as raw payloads WITHOUT pkt-line encoding. IMPORTANT: Lines preserve trailing newlines if present in the original data. When creating new lines, transformers should include trailing newlines to match Git protocol conventions.
The hasPackfileSection parameter indicates whether the response includes a packfile section. During negotiation, the server may not send a packfile (negotiation ongoing), so transforms should check this flag before injecting sections like packfile-uris.
func Compose ¶
func Compose(transforms ...TransformFunc) TransformFunc
Compose combines multiple TransformFunc into a single function that applies them in sequence. Transformations are applied left-to-right. If any transformation returns an error, the composition stops and returns that error.
func FilterLines ¶
func FilterLines(sectionName string, shouldRemove func([]byte) bool) TransformFunc
FilterLines returns a TransformFunc that removes lines matching a predicate from a specific section. The sectionName specifies which section to filter. The shouldRemove function returns true for lines that should be removed.
func InjectCapability ¶
func InjectCapability(capability string) TransformFunc
InjectCapability returns a TransformFunc that adds a capability to info/refs response. The capability is injected into the first ref advertisement line that contains capabilities. Format: "<hash> <ref>\x00<capabilities>". nolint:gocognit
func InjectPackfileURIs ¶
func InjectPackfileURIs(uris ...PackfileURI) TransformFunc
InjectPackfileURIs returns a TransformFunc that adds a packfile-uris section. The section is appended after existing sections and before the packfile section. If a packfile-uris section already exists, the new URIs are appended to it. If hasPackfileSection is false (negotiation ongoing), no injection occurs. Protocol v2 format:
- Section header: "packfile-uris"
- Each URI line: "<hash> <uri>" where hash is a 40-character hexadecimal SHA-1
- Delimiter packet: 0001 (added automatically by serializeSections)
func RemovePackfileURIsSection ¶
func RemovePackfileURIsSection() TransformFunc
RemovePackfileURIsSection returns a TransformFunc that removes the packfile-uris section if present.
Source Files