Documentation
¶
Overview ¶
Package podlogs provides generic pod log fetching utilities.
Index ¶
- Constants
- Variables
- func LineTimestamp(line string) (time.Time, bool)
- func OpenStream(ctx context.Context, req *rest.Request) (io.ReadCloser, error)
- func ReadAll(ctx context.Context, fetcher PodLogFetcher, namespace, podName string, ...) error
- func ScanLines(reader io.Reader) ([]string, error)
- type LogOptions
- type Page
- type PageFetcher
- type PodLogFetcher
- type ReadAllOptions
Constants ¶
const DefaultMaxLogBytes int64 = 8 * 1024 * 1024
DefaultMaxLogBytes caps a single log fetch at 8 MB. Sized well above a normal sampling window (a chatty trainer emits a few hundred KB per minute) while staying far below the controller's memory limit.
const ( // DefaultMaxPages bounds how many reads ReadAll issues for one log: 64 // pages of DefaultMaxLogBytes is 512 MB, far past any NCCL test log. DefaultMaxPages = 64 )
const DefaultStreamTimeout = 2 * time.Minute
DefaultStreamTimeout bounds the complete lifecycle of a pod log request, including opening the stream and consuming its response body.
Variables ¶
var ErrLogUnpageable = errors.New("log cannot be read in full")
ErrLogUnpageable is returned when a log cannot be read in full: it outgrew the page bound, one second of output outgrew the largest page, a line is longer than the scanner accepts, or its lines carry no timestamps to resume from. Retrying fails the same way.
Functions ¶
func LineTimestamp ¶ added in v0.6.0
LineTimestamp parses the RFC3339Nano timestamp the kubelet prefixes to each log line when timestamps are requested.
func OpenStream ¶
OpenStream opens a pod log request bounded by DefaultStreamTimeout or the parent context's deadline, whichever is earlier. The returned stream keeps that deadline active until it is closed, so stalled response-body reads are bounded as well as the initial request.
func ReadAll ¶ added in v0.6.0
func ReadAll(ctx context.Context, fetcher PodLogFetcher, namespace, podName string, opts ReadAllOptions, visit func(line string)) error
ReadAll reads a container's current log from its first line to its last and passes each complete line to visit exactly once, in log order.
A single read is capped server-side and keeps the head of the log, so a log larger than one page is read in pages. Each page after the first resumes from the timestamp of the last complete line. SinceTime is second-precise on the wire, so the kubelet returns every line at or after the start of that second; the lines already visited from that second are exactly the first ones of the new page, in the same order, and are skipped by count. A page cut mid-line drops the partial line so it is read whole next time.
Built for logs that are no longer written to, such as a finished workload. A log that rotates between pages can lose or repeat lines.
Types ¶
type LogOptions ¶
type LogOptions struct {
Container string
SinceTime *metav1.Time
// LimitBytes caps how many bytes the API server returns for this fetch.
// Zero means DefaultMaxLogBytes; reads are never unbounded.
LimitBytes int64
}
LogOptions configures log reading.
type Page ¶ added in v0.6.0
type Page struct {
Lines []string
// Truncated reports that the read stopped at LimitBytes, so the log may
// continue past Lines.
Truncated bool
// PartialTail reports that the byte limit cut the last element of Lines
// short. It is only ever true together with Truncated.
PartialTail bool
}
Page is one bounded read of a container log.
type PageFetcher ¶ added in v0.6.0
type PageFetcher interface {
FetchLogsPage(ctx context.Context, namespace, podName string, opts LogOptions) (Page, error)
}
PageFetcher is a PodLogFetcher that can also report whether a read was cut off at LimitBytes. ReadAll needs that to page through a log larger than one read; a fetcher without it is read once.
type PodLogFetcher ¶
type PodLogFetcher interface {
FetchLogs(ctx context.Context, namespace, podName string, opts LogOptions) ([]string, error)
}
PodLogFetcher abstracts fetching raw log lines from a pod.
func NewKubernetesLogFetcher ¶
func NewKubernetesLogFetcher(clientset *kubernetes.Clientset) PodLogFetcher
NewKubernetesLogFetcher creates a PodLogFetcher backed by a Kubernetes clientset.
type ReadAllOptions ¶ added in v0.6.0
type ReadAllOptions struct {
Container string
// PageBytes caps each read. Zero means DefaultMaxLogBytes.
PageBytes int64
// MaxPages bounds the number of reads. Zero means DefaultMaxPages.
MaxPages int
}
ReadAllOptions configures ReadAll.
Source Files