s3oplog

package
v0.20.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 12, 2026 License: Apache-2.0 Imports: 12 Imported by: 0

README

internal/s3oplog

Overview

Opt-in per-process S3 round-trip log for diagnostics. When OPENSTORAGE_S3_OP_LOG names an absolute file, a smithy Deserialize middleware appends one line per HTTP attempt, tagged with the process role and an optional scope. When the variable is unset, nothing is installed.

Interface

  • Env — env var name OPENSTORAGE_S3_OP_LOG [s3oplog.go:Env]
  • APIOption — returns the smithy API option that adds the openstorageS3OpLog middleware, plus an enabled flag; nil, false when disabled [s3oplog.go:APIOption]
  • WithScope — tags a context so its S3 round trips log as role:scope [s3oplog.go:WithScope]
  • Scopes — returns a copy of the registered scope list [s3oplog.go:Scopes]
  • ScopeSchedule, ScopeKeyRefresh — scope constants schedule and keyrefresh [s3oplog.go:ScopeSchedule]

Invariants

  • WithScope panics on an unknown scope [s3oplog.go:WithScope]
  • A relative log path is refused and logging stays disabled; an open failure is reported to stderr and also leaves logging disabled [s3oplog.go:setup]
  • setup runs at most once per process through sync.Once [s3oplog.go:APIOption]
  • logOp drops write errors, logs status 0 when there is no HTTP response, and - when there is no URL [s3oplog.go:logOp]
  • procRole stops at the first flag argument and after three parts, and squashes whitespace so lines stay field-splittable [s3oplog.go:procRole]

Side Effects

  • Reads OPENSTORAGE_S3_OP_LOG through envconfig.Getenv [s3oplog.go:setup]
  • Opens the log file with O_CREATE|O_WRONLY|O_APPEND, mode 0644, and keeps it for the process lifetime [s3oplog.go:setup]
  • Appends one line per S3 attempt: timestamp, pid, role, operation, status, duration, path [s3oplog.go:logOp]
  • Reads os.Args [s3oplog.go:procRole] and os.Getpid [s3oplog.go:logOp]

Verification

  • Run: go test ./internal/s3oplog/... -tags=openstorage_viewcheck -count=1 -race -timeout=30m

Documentation

Overview

Package s3oplog is the per-request S3 round-trip log seam (issue #182): when OPENSTORAGE_S3_OP_LOG names a file, every S3 HTTP attempt made through the openstorage SDK clients — the s3wal conditional client (ref and meta authority) and the blob client — appends one whitespace- separated line:

<unixnano> <pid> <role> <operation> <status> <dur_us> <path>

role is derived from the process argv (serve vs hook subprocesses), so a log shared by one push's processes attributes each round trip to its origin. Background work tags its context with WithScope, and its round trips log as role:scope, so a census can tell them from request work. The seam exists to census and regression-guard the per-push S3 round-trip count; it is diagnostics, not telemetry — unset (the default) it installs nothing and costs nothing.

The middleware sits in the Deserialize step (the captureStoreDate precedent in internal/ref/s3wal): it wraps only the wire exchange, runs once per HTTP attempt (inside the retry loop), and sees the raw response status uniformly for every operation.

Index

Constants

View Source
const (
	// ScopeSchedule tags the sync schedule's sweeps and runs (internal/sync).
	ScopeSchedule = "schedule"
	// ScopeKeyRefresh tags the S3WAL org-key refresher (internal/meta).
	ScopeKeyRefresh = "keyrefresh"
)

The op-log scopes of background work. Every scope is listed here, so a census (TestS3WAL_PushOpCensus) can drop all background round trips with Scopes, and a renamed or new scope cannot drift from its list.

View Source
const Env = "OPENSTORAGE_S3_OP_LOG"

Env names the op-log file. Empty (the default) disables the seam.

Variables

This section is empty.

Functions

func APIOption

func APIOption() (func(*middleware.Stack) error, bool)

APIOption returns the smithy API option installing the op-log middleware and whether the seam is enabled. Callers append it to s3.Options .APIOptions exactly like the expected-bucket-owner option.

func Scopes added in v0.19.0

func Scopes() []string

Scopes returns every scope WithScope accepts.

func WithScope added in v0.19.0

func WithScope(ctx context.Context, scope string) context.Context

WithScope returns ctx tagged with scope: S3 round trips made under it log their role as role:scope. scope must be one of the Scope constants: a scope missing from Scopes would escape the census filter.

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL