serviceutil

package
v0.0.1-alpha.30 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 5, 2026 License: MIT Imports: 24 Imported by: 0

Documentation

Overview

Package serviceutil provides shared, generic utilities used across all service handlers. The goal is to ensure every service follows the same patterns for request parsing, response writing, error handling, and logging — so that reading any service handler feels immediately familiar.

TypeScript analogy: this is the equivalent of a shared utils/ or lib/ directory that every route handler imports from.

Usage pattern in a handler: 

func (h *Handler) CreateQueue(w http.ResponseWriter, r *http.Request) {
    var req createQueueRequest
    if !serviceutil.DecodeJSON(w, r, &req) {
        return // error already written
    }
    if !serviceutil.RequireFields(w, r, req.QueueName, "QueueName") {
        return
    }
    // ... implementation
}

Index

Constants

View Source 
const TraceLevel = logging.TraceLevel

── Trace level ──────────────────────────────────────────────────────────────

TraceLevel, ParseLevel, and WrapLevelEncoder are defined in the leaf package internal/logging (see its doc comment for why: internal/state's background maintenance/flush loops need TraceLevel directly, and internal/serviceutil → internal/protocol → internal/state would otherwise be a cycle). Re-exported here so existing callers of this package don't need to know about that split.

Variables

View Source 
var ErrInvalidPageToken = errors.New("serviceutil: invalid pagination token")

ErrInvalidPageToken is returned by Paginate when the caller-supplied continuation token cannot be decoded, or decodes to a start position that is no longer valid (negative, or past the end of the item set).

Silently treating an invalid/garbled token as "start from page 1" is the most common AWS-fidelity divergence in this codebase (see docs/plans/pagination-plan.md, items H1/G3): a client polling with a stale or corrupted token receives the full item set again instead of an error, which looks like — and is handled by SDK retry logic as — a legitimate page, producing duplicate delivery.

Callers MUST check this error (errors.Is) and map it to their own service's AWS error type/code instead of falling through to the returned (zero-value) Page. For example: 

page, err := serviceutil.Paginate(items, maxResults, req.NextToken, opts)
if err != nil {
    // SSM: https://docs.aws.amazon.com/systems-manager/latest/APIReference/API_DescribeParameters.html#API_DescribeParameters_Errors
    return nil, &protocol.AWSError{Code: "InvalidNextToken", Message: "The specified token isn't valid.", HTTPStatus: http.StatusBadRequest}
}
View Source 
var (

	// GraphQLIdentifierPattern matches AppSync GraphQL identifiers documented for
	// data source, function, type, and field names.
	// https://docs.aws.amazon.com/appsync/latest/APIReference/API_CreateDataSource.html
	GraphQLIdentifierPattern = regexp.MustCompile(`^[_A-Za-z][_0-9A-Za-z]*$`)
)

Functions

func ARNRegion 

func ARNRegion(arn string) string

ARNRegion extracts the region component from an AWS ARN (the 4th colon-separated field, e.g. "us-east-1"). Returns an empty string for non-ARN inputs or ARNs without a region field.

func AllowProtocolDrift 

func AllowProtocolDrift(cfg *config.Config, log *ServiceLogger, operation string, claimed codec.Codec, declared []codec.Codec) bool

AllowProtocolDrift decides what a service's Dispatch/DispatchQuery method should do when the wire protocol identified for a request (claimed) is not one of the protocols the service declares support for (declared, from its SupportedProtocols()).

This implements the "claimed-but-undeclared" reactive posture from docs/plans/level2-codegen.md Track 1.2: since April 2026, AWS SDKs may switch a service's wire protocol without notice (the CloudWatch metrics protocol switch is the precedent). Hard-rejecting an undeclared protocol turns a still-decodable request into an unnecessary 415 the moment AWS ships that kind of change. The default, lenient posture instead logs a loud "protocol drift" warning and tells the caller to attempt the decode/dispatch anyway — a silent SDK protocol switch becomes a working request plus a signal, not a mystery outage.

Strict mode (cfg.ProtocolStrict, env OVERCAST_PROTOCOL_STRICT) restores the old reject-on-mismatch behaviour for environments that want hard protocol-fidelity gating (e.g. a CI job asserting no service has drifted).

Returns true when the caller should proceed with normal dispatch (either claimed is already declared — the common case, no log — or lenient mode allowed the drift through). Returns false only in strict mode with an undeclared claim; the caller is expected to write its existing UnsupportedProtocol error in that case.

log may be nil (drift is simply not logged); cfg may be nil (treated as lenient — the safer default).

func AlphaNumericHyphenUnderscorePeriod 

func AlphaNumericHyphenUnderscorePeriod(c rune) bool

AlphaNumericHyphenUnderscorePeriod matches the common AWS identifier alphabet used by SQS queue names and DynamoDB table names.

func AppSyncDataSourceName 

func AppSyncDataSourceName(name string) *protocol.AWSError

func AppSyncFieldName 

func AppSyncFieldName(name string) *protocol.AWSError

func AppSyncFunctionName 

func AppSyncFunctionName(name string) *protocol.AWSError

func AppSyncGraphQLAPIName 

func AppSyncGraphQLAPIName(name string) *protocol.AWSError

AppSyncGraphQLAPIName validates the required GraphQL API name. AWS documents this field as required without publishing a stricter length or pattern.

func AppSyncIdentifierName 

func AppSyncIdentifierName(name, field string) *protocol.AWSError

AppSyncIdentifierName validates AppSync names documented with the [_A-Za-z][_0-9A-Za-z]* pattern and 1..65536 length constraint.

func AppSyncTypeName 

func AppSyncTypeName(name string) *protocol.AWSError

func BucketName 

func BucketName(name string) *protocol.AWSError

BucketName validates an S3 bucket name against AWS naming rules. Returns nil if valid, or a *protocol.AWSError with code "InvalidBucketName".

func CallerIsSiblingContainer 

func CallerIsSiblingContainer(addr string) bool

CallerIsSiblingContainer reports whether an address belongs to a container on one of Overcast's Docker networks rather than to the host.

It exists for the one class of value the URL-minting rule cannot settle on its own: an endpoint that names a *container Overcast started* — an RDS instance, an ElastiCache node — rather than Overcast itself. Such a container answers on its engine port (3306) inside the Docker network and on a published port on the host, and no single pair is dialable from both sides. The hostname is no help in choosing, since a split-horizon name is used by both; the source address is, because a sibling container reaches Overcast over a Docker bridge and so arrives on a private address, while the host arrives on loopback.

Loopback is the host. So is a Docker bridge *gateway* address, which is the one that catches people out: when Overcast itself runs in a container, a request from the host arrives through the userland proxy and is seen as coming from the gateway (172.17.0.1 for the default bridge), not from loopback. Docker reserves the first address of every bridge subnet for that gateway and allocates containers from the second upwards, so a final octet of 1 identifies the host side without enumerating interfaces — which would have to be re-read as VPC networks come and go.

A caller on the machine's LAN address is classified as a sibling. Overcast binds container ports on the host, not on the LAN, so neither view is dialable for them and the mistake costs nothing that was working. See docs/networking.md § Data-plane endpoints.

func ClampInt 

func ClampInt(v, min, max int) int

ClampInt returns v clamped to [min, max]. 

maxMessages := serviceutil.ClampInt(req.MaxNumberOfMessages, 1, 10)

func ClientBaseURL 

func ClientBaseURL(cfg *config.Config, r *http.Request) string

ClientBaseURL returns the base URL services embed in client-facing responses: the configured OVERCAST_HOSTNAME (when set) on the *caller's* port. It is the single implementation of the URL-minting rule — see docs/plans/client-facing-url-minting.md for the rule, the per-service analysis behind it, and the deliberate divergences (SQS's wire echo).

Why each part:

  • Host: a configured hostname is the operator's assertion that one name resolves for every party — the split-horizon defaults do, from the host and from inside containers alike. A caller's own host (an IP, a compose alias) may resolve nowhere else, so the configured name wins (#351).
  • Port: always the caller's. Their request is the only proof of a dialable port — Overcast cannot see its own port mapping, and with the API port remapped (`docker run -p 4652:4566`) a URL carrying cfg.Port is undialable by every host-side caller. cfg.Port fills in only when the request carries no port at all (background work, synthetic internal dispatch), and those values are re-rendered per caller at read time.
  • Scheme: https if Overcast itself serves TLS or the caller arrived over https; never a downgrade. A synthetic internal request carries no TLS state, which is why config must be able to assert the scheme.

With no hostname configured, the caller's origin is returned verbatim — their own host is the only name known to resolve for them.

func ClientBaseURLFromOrigin 

func ClientBaseURLFromOrigin(cfg *config.Config, origin string) string

ClientBaseURLFromOrigin is ClientBaseURL for callers holding a middleware-stamped origin (middleware.ClientEndpointFromContext) rather than an *http.Request — CloudFormation's provisioner paths and Cognito's typed (Smithy CBOR) dispatch. One function for both shapes is what stops the hand-kept copies this replaced from drifting apart again: before it, the repo had four base-URL precedences and two of them disagreed on the port, which was exactly the bug.

func ClientIP 

func ClientIP(r *http.Request) string

ClientIP returns the caller's IP address: the first entry of X-Forwarded-For when present and valid, otherwise the host portion of r.RemoteAddr. Used to populate AWS-shaped request contexts (API Gateway "identity.sourceIp", Lambda function URL "requestContext.http.sourceIp").

func DecodeJSON 

func DecodeJSON(w http.ResponseWriter, r *http.Request, dst any) bool

DecodeJSON decodes the request body as JSON into dst. On failure, writes a well-formed AWS JSON error response and returns false. The caller should return immediately when false is returned — the response has already been written.

This replaces the repetitive pattern: 

if err := json.NewDecoder(r.Body).Decode(&req); err != nil {
    protocol.WriteJSONError(w, r, protocol.ErrSerialization("invalid request body"))
    return
}

func DefaultInt 

func DefaultInt(v, defaultVal int) int

DefaultInt returns v if v > 0, otherwise defaultVal. Useful for optional integer request fields that default to a non-zero value. 

timeout := serviceutil.DefaultInt(req.VisibilityTimeout, 30)

func DomainPrefix 

func DomainPrefix(host string) string

DomainPrefix returns the first label of host (e.g. "api" for "api.example.com"), stripping any port first. Falls back to "localhost" for an empty host. Used to populate AWS-shaped request contexts (API Gateway v2 / Lambda function URL "requestContext.domainPrefix").

func FindRegioned 

func FindRegioned[T any](ctx context.Context, st state.Store, ns, id, defaultRegion string) (value *T, region string, found bool, err error)

FindRegioned locates the record stored under id in any region of a region-keyed namespace and returns it with its region. Event callbacks that only carry a resource ID (Docker container events) use it because they cannot resolve the region the resource was created under. Returns found == false when no region holds the id.

func FoldHostname 

func FoldHostname(hostname string) string

FoldHostname lowercases an ASCII hostname (a trailing :port is unaffected, being digits). A hostname is case-insensitive — RFC 4343 for DNS, RFC 3986 §3.2.2 for the URI authority — so folding loses nothing, and lowercase is the canonical form both that section and RFC 3986 §6.2.2.1 tell producers to emit.

Overcast needs this on both sides of a request. Inbound, middleware folds the Host before deciding who owns it, so casing cannot change which service answers. Outbound, the URLs and request-context fields minted here are folded, so what Overcast hands back does not carry the case of whichever caller happened to create the resource.

Folding is pay-per-use: an already-lowercase host is returned unchanged with no copy, which is what keeps inbound classification allocation-free on every real request. ASCII-only by construction — DNS names are ASCII, an IDN arrives punycode-encoded, and Unicode case folding carries locale hazards a hostname comparison must not inherit. strings.ToLower has the same no-copy property but measured ~2.4x slower here, because its scan also tracks non-ASCII and cannot stop at the first upper-case byte.

func HasQueryParam 

func HasQueryParam(r *http.Request, param string) bool

HasQueryParam returns true if the named query parameter is present in the URL, regardless of its value. Useful for S3-style action parameters like ?location. 

if serviceutil.HasQueryParam(r, "location") { ... }

func HeaderPrefix 

func HeaderPrefix(r *http.Request, prefix string) map[string]string

HeaderPrefix extracts all headers whose names start with prefix and returns them as a map with the prefix stripped from each key name. Keys are lowercased for consistency.

Used by S3 to extract x-amz-meta-* user metadata headers: 

meta := serviceutil.HeaderPrefix(r, "X-Amz-Meta-")
// {"author": "alice", "version": "1.0"}

func HostRoutedHostname 

func HostRoutedHostname(cfg *config.Config, r *http.Request, label, id, region string) string

HostRoutedHostname is HostRoutedURL reduced to the bare DNS name, with no scheme, port or path — the shape AWS uses for fields that carry a hostname rather than a URL, such as AppSync's GraphqlApi.dns map.

It is derived from HostRoutedURL rather than rebuilt, so the two forms cannot disagree about the grammar.

func HostRoutedHostnameFromBase 

func HostRoutedHostnameFromBase(baseURL, label, id, region string) string

HostRoutedHostnameFromBase is HostRoutedHostname for callers that already hold a resolved base URL.

func HostRoutedURL 

func HostRoutedURL(cfg *config.Config, r *http.Request, label, id, region, path string) string

HostRoutedURL mints the canonical AWS Host-routed URL for a resource: 

{scheme}://{id}.{label}.{region}.{host}[:{port}]{path}

It is the single place any service builds one, so a URL Overcast hands back is by construction one its own router can resolve — label must be a key of middleware's host-route table (use the LabelExecuteAPI / LabelLambdaURL / LabelAppSyncAPI constants, which that table is built from). serviceutil cannot import middleware, since middleware imports serviceutil, so the guarantee is enforced by round-trip tests that feed a minted URL back in as a Host header rather than by the type system.

It builds on ClientBaseURL, not config.ExternalBaseURL: only ClientBaseURL falls back to the request port when cfg.Port is unset and honours cfg.TLSEnabled(). See docs/plans/harness-representativeness-audit.md finding 2.

path is appended verbatim and may be empty (API Gateway v2's apiEndpoint has no path), "/" (Lambda function URLs), or a fixed route ("/graphql").

func HostRoutedURLFromBase 

func HostRoutedURLFromBase(baseURL, label, id, region, path string) string

HostRoutedURLFromBase is HostRoutedURL for callers that have already resolved the client-facing base URL and have no *http.Request to hand — AppSync's typed handlers carry it on the context instead.

func ParseIntDefault 

func ParseIntDefault(s string, defaultVal int) int

ParseIntDefault parses a string as an integer. Returns defaultVal if the string is empty or unparseable. 

timeout := serviceutil.ParseIntDefault(q.Attributes["VisibilityTimeout"], 30)

func ParseLevel 

func ParseLevel(s string) (zapcore.Level, error)

ParseLevel parses a log-level string into a zapcore.Level, extending zapcore.ParseLevel with Overcast's "trace" level. See internal/logging.ParseLevel.

func ParseLevelOrDefault 

func ParseLevelOrDefault(s string) (level zapcore.Level, ok bool)

ParseLevelOrDefault parses s as a log level, falling back to InfoLevel — Overcast's default — when s is empty or invalid. ok reports whether s parsed cleanly, so callers can warn about the fallback (naming the level actually in effect) instead of either failing startup over an observability typo or silently ignoring it.

func QueryInt 

func QueryInt(r *http.Request, param string, defaultVal int) int

QueryInt extracts an integer query parameter. Returns defaultVal if the parameter is absent or unparseable. Does not write an error — call RequireQueryInt if the parameter is mandatory. 

maxKeys := serviceutil.QueryInt(r, "max-keys", 1000)

func QueryString 

func QueryString(r *http.Request, param, defaultVal string) string

QueryString extracts a string query parameter, returning defaultVal if absent. 

prefix := serviceutil.QueryString(r, "prefix", "")

func QueueName 

func QueueName(name string) *protocol.AWSError

QueueName validates an SQS queue name. Standard queues: alphanumeric + hyphens + underscores, 1–80 chars. FIFO queues: same rules + must end in .fifo. https://docs.aws.amazon.com/AWSSimpleQueueService/latest/SQSDeveloperGuide/sqs-queue-message-identifiers.html

func RegionKey 

func RegionKey(region, key string) string

RegionKey builds a region-scoped store key by prepending region + "/" to key. If region is empty the key is returned unchanged, enabling "scan all regions" queries when used as a List/Scan prefix.

func RequestBaseURL 

func RequestBaseURL(r *http.Request) string

RequestBaseURL derives a base URL from proxy headers or the request host.

The host is folded to its canonical lowercase form: this is the single funnel every client-facing URL is minted through (via ClientBaseURL), so folding here is what stops a mixed-case request baking its casing into a queue URL, an invoke endpoint or a stack output.

func RequestProtocol 

func RequestProtocol(r *http.Request) string

RequestProtocol returns r.Proto, falling back to "HTTP/1.1". Used to populate AWS-shaped request contexts (API Gateway "identity.protocol", Lambda function URL "requestContext.http.protocol").

func RequireString 

func RequireString(w http.ResponseWriter, r *http.Request, value, paramName string) bool

RequireString checks that a string field is non-empty. On failure, writes a MissingParameter error and returns false. 

if !serviceutil.RequireString(w, r, req.QueueName, "QueueName") {
    return
}

func ResourceName 

func ResourceName(name string, rule NameRule) *protocol.AWSError

ResourceName validates name with a reusable rule and returns the configured AWS-style error. Service-specific validators should wrap this helper rather than exposing generic resource-name policy from handlers.

func SplitRegionKey 

func SplitRegionKey(key string) (region, rest string)

SplitRegionKey extracts the region prefix and the remaining key from a region-scoped store key. Returns ("", key) if the key has no "/" separator.

func SupportsHostRouting 

func SupportsHostRouting(baseURL string) bool

SupportsHostRouting reports whether a host-routed URL minted on baseURL would actually resolve for the client holding it.

Host routing needs the base to carry an arbitrary subdomain. An IP literal cannot, and a bare single-label name — "localhost", a Docker service alias — only resolves under wildcard DNS that Windows and macOS do not provide for *.localhost. Every public domain in config.WildcardDNSDomains is multi-label and so passes, as does any multi-label OVERCAST_HOSTNAME an operator set, which is the operator asserting that their own name resolves.

It is the gate for the fields that have a path-style alternative Overcast also serves (AppSync's uris; the console's REST v1 invoke URLs), so those degrade to a URL that works rather than advertising one that will not resolve. Fields with no path-style form — Lambda FunctionUrl, API Gateway v2 apiEndpoint — are host-routed unconditionally, because there is nothing to fall back to.

web/src/lib/host-routed-url.ts states the same rule for the console.

func TableName 

func TableName(name string) *protocol.AWSError

TableName validates a DynamoDB table name. Rules: 3–255 chars, alphanumeric + hyphens + underscores + periods. https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.NamingRulesDataTypes.html

func ValidateAndRespond 

func ValidateAndRespond(w http.ResponseWriter, r *http.Request, aerr *protocol.AWSError) bool

ValidateAndRespond is a convenience helper that writes the error and returns false if aerr is non-nil, otherwise returns true. Reduces boilerplate in handlers that call multiple validators in sequence. 

if !serviceutil.ValidateAndRespond(w, r, serviceutil.BucketName(bucket)) {
    return
}

func WrapLevelEncoder 

func WrapLevelEncoder(base zapcore.LevelEncoder, label string) zapcore.LevelEncoder

WrapLevelEncoder wraps a zapcore.LevelEncoder so TraceLevel renders as label instead of the base encoder's zero-value fallback. See internal/logging.WrapLevelEncoder.

Types

type LazyInit 

type LazyInit struct {
	// contains filtered or unexported fields
}

LazyInit provides a generic lazy-initialisation wrapper using sync.Once.

In Go, "lazy loading" of a service means: register routes immediately (so the router is ready), but defer any expensive initialisation — store setup, background goroutines, connection pools — until the first actual request arrives.

This is important for:

  • Lambda: don't start Node.js process supervisors until the first Invoke
  • DynamoDB: don't start the expression evaluator goroutines until first use
  • Any service with expensive startup: only pay the cost if the service is called

TypeScript analogy: a module-level singleton initialised on first import, but with explicit control over when "first import" happens.

Usage in a service handler: 

type Handler struct {
    init   serviceutil.LazyInit
    // ... other fields
}

func (h *Handler) ensureInitialised(cfg *config.Config) error {
    return h.init.Do(func() error {
        // expensive setup: extract bootstrap.js, start supervisor, etc.
        return h.startNodeSupervisor(cfg)
    })
}

func (h *Handler) Invoke(w http.ResponseWriter, r *http.Request) {
    if err := h.ensureInitialised(h.cfg); err != nil {
        protocol.WriteJSONError(w, r, protocol.Wrap(protocol.ErrInternalError, err))
        return
    }
    // ... handle request
}

func (*LazyInit) Do 

func (l *LazyInit) Do(fn func() error) error

Do runs fn exactly once, the first time Do is called. Subsequent calls return the same error (or nil) without calling fn again.

If fn returns an error, the LazyInit is NOT marked as successfully done — the next call to Do will attempt fn again. This allows transient failures (e.g. "node not in PATH") to be retried without restarting the server.

If fn succeeds (returns nil), all future calls return nil immediately without calling fn again.

func (*LazyInit) Done 

func (l *LazyInit) Done() bool

Done reports whether the lazy initialisation has completed successfully.

func (*LazyInit) Reset 

func (l *LazyInit) Reset()

Reset forces re-initialisation on the next Do call. Primarily useful in tests that need a clean state between runs.

type NameRule 

type NameRule struct {
	MinLength      int
	MaxLength      int
	Allowed        func(rune) bool
	Pattern        *regexp.Regexp
	ErrorCode      string
	LengthMessage  string
	AllowedMessage string
	PatternMessage string
	HTTPStatus     int
}

NameRule describes one AWS resource-name validation rule. It intentionally carries service-specific error details because AWS does not use one global validation error shape across services.

type Page 

type Page[T any] struct {
	// Items contains the items on this page.
	Items []T
	// NextToken is the opaque continuation token to retrieve the next page.
	// Empty string means this is the last page.
	NextToken string
	// IsTruncated is true when there are more items after this page.
	// It mirrors the AWS convention (used by S3's IsTruncated field).
	IsTruncated bool
}

Page represents a single page of results from a paginated list operation. T is the item type (e.g. *s3.Object, *sqs.Queue).

func Paginate 

func Paginate[T any](items []T, requestedLimit int, continuationToken string, opts PaginateOptions) (Page[T], error)

Paginate applies limit and continuation-token logic to a full item slice, returning a Page with at most the effective limit's items.

The continuation token encodes the start index opaquely (base64 of a JSON integer) so that callers cannot make assumptions about item ordering.

requestedLimit is the raw value the client asked for (e.g. req.MaxResults); pass 0 or a negative number when the client omitted it. opts supplies the operation's AWS-documented default and cap.

Returns ErrInvalidPageToken when continuationToken is non-empty but doesn't decode to a valid start position — see that error's doc comment for the caller contract this establishes.

Example: 

allObjects, _ := h.store.listObjects(ctx, bucket, prefix)
page, err := serviceutil.Paginate(allObjects, maxKeys, req.ContinuationToken,
    serviceutil.PaginateOptions{DefaultLimit: 1000})
if err != nil {
    // map to the service's AWS invalid-token error and return
}
// page.Items       — items for this page
// page.NextToken    — pass back to client as NextContinuationToken
// page.IsTruncated  — set in the response envelope

type PaginateOptions 

type PaginateOptions struct {
	// DefaultLimit is used when the caller-requested limit is <= 0 (i.e.
	// the client omitted MaxResults/MaxItems/Limit entirely). If left at
	// the zero value, Paginate falls back to 1000 (S3 ListObjects'
	// documented default) so callers that don't set this field keep the
	// pre-H1 behavior.
	DefaultLimit int
	// MaxLimit caps the effective limit even when the caller requests (or
	// DefaultLimit specifies) more than AWS allows for this operation.
	// Zero means no cap is applied.
	MaxLimit int
}

PaginateOptions configures the effective per-call page size. AWS documents a different default and cap for nearly every List/Describe operation (see the pagination-plan's per-op citations), so callers supply their own rather than relying on a single package-wide default.

type RecordLocks 

type RecordLocks struct {
	// contains filtered or unexported fields
}

RecordLocks serialises the read-modify-write of individual stored records.

A record is stored as one blob, so every writer reads the whole record, edits it and writes the whole record back: two of them overlapping means the second silently discards the first's edit. In an emulator that is not hypothetical — every service with a background lifecycle transition has a second writer running against the same records its API handlers write, and neither knows about the other. What gets discarded is whichever edit landed first: a property change applied while a health check was dialling, or the status write that a modify then rolled back.

Holding one of these across the read and the write is what makes them one step. It has to cover both: releasing between them is the race.

Striped rather than a mutex per record, because record identity churns — a long-lived process replaces its instances, clusters and tasks many times over — so a map keyed by record would grow for the life of the process. Two records sharing a stripe serialise needlessly, which costs nothing here: a critical section is one read and one write of a single record, with anything slow (a Docker call, a dial) deliberately left outside it.

The zero value is ready to use, so a handler assembled without a constructor, as several tests do, is safe to lock.

func (*RecordLocks) Lock 

func (l *RecordLocks) Lock(key string) func()

Lock takes the lock covering key and returns the function that releases it, for use as: 

defer h.instanceLocks.Lock(id)()

Callers that key records by region must include the region in key: the same name in two regions is two records.

type Regioned 

type Regioned[T any] struct {
	Region string
	Value  *T
}

Regioned pairs a decoded store record with the region it is stored under.

func ScanRegions 

func ScanRegions[T any](ctx context.Context, st state.Store, ns, defaultRegion string) ([]Regioned[T], error)

ScanRegions decodes every record in a region-keyed namespace across all regions. Background paths that have no request context (startup reconciliation, sweepers) use it instead of a region-scoped list, which would silently cover only the default region. Records whose region prefix is empty are attributed to defaultRegion. Malformed records are skipped, matching the per-service list helpers.

type ServiceLogger 

type ServiceLogger struct {
	// contains filtered or unexported fields
}

ServiceLogger wraps a zap.Logger with service-scoped context. All log calls automatically include the service name as a structured field, so log lines are filterable by service without adding the field at every call.

Create one per service in service.go and pass it to the handler: 

type Service struct {
    log *serviceutil.ServiceLogger
    ...
}

func New(cfg *config.Config, store state.Store, logger *zap.Logger) *Service {
    return &Service{
        log: serviceutil.NewServiceLogger(logger, "s3"),
        ...
    }
}

func NewServiceLogger 

func NewServiceLogger(logger *zap.Logger, service string) *ServiceLogger

NewServiceLogger returns a ServiceLogger scoped to the named service. When stdout is an interactive terminal, log messages are prefixed with a coloured [SERVICE] tag so different services are instantly distinguishable when running locally. In non-interactive environments (CI, Docker) the tag is omitted and the structured "service" field carries the same information.

func (*ServiceLogger) Debug 

func (l *ServiceLogger) Debug(msg string, fields ...zap.Field)

Debug logs at DEBUG level — fine-grained detail useful during development. Use for: parsed request parameters, individual state reads/writes. Do NOT use for: every middleware step (already logged by Logger middleware).

func (*ServiceLogger) Error 

func (l *ServiceLogger) Error(msg string, fields ...zap.Field)

Error logs at ERROR level — failures producing 5xx responses. Use for: state backend failures, serialisation errors, panics. Always include zap.Error(err) — never log just the message without the error.

func (*ServiceLogger) Info 

func (l *ServiceLogger) Info(msg string, fields ...zap.Field)

Info logs at INFO level — significant lifecycle events. Use for: resource created/deleted, queue purged, bucket emptied. Do NOT use for: individual requests (already logged by Logger middleware).

func (*ServiceLogger) LogStateError 

func (l *ServiceLogger) LogStateError(r *http.Request, op string, aerr *protocol.AWSError, fields ...zap.Field)

LogStateError logs a state backend failure at ERROR level with standard fields. This is the canonical way to log storage failures in service store.go files: 

if aerr := s.store.putObject(ctx, obj); aerr != nil {
    s.log.LogStateError(r, "put object", aerr, zap.String("bucket", bucket), zap.String("key", key))
    protocol.WriteXMLError(w, r, aerr)
    return
}

func (*ServiceLogger) Logger 

func (l *ServiceLogger) Logger() *zap.Logger

Logger returns the underlying zap.Logger for cases where raw zap is needed.

func (*ServiceLogger) Trace 

func (l *ServiceLogger) Trace(msg string, fields ...zap.Field)

Trace logs at TRACE level (below DEBUG) — periodic machine-generated chatter that even a debugging session rarely wants: health/readiness probe and /_debug/* request logs, flush/checkpoint/maintenance/sweep cycle logs, buffer/pool internals. See the TraceLevel doc comment and CONTRIBUTING.md § Log levels for the full trace-vs-debug distinction. Do NOT use for anything a human debugging Overcast would actually want to see — that's DEBUG.

func (*ServiceLogger) Warn 

func (l *ServiceLogger) Warn(msg string, fields ...zap.Field)

Warn logs at WARN level — handled but unexpected conditions. Use for: oversized payloads, deprecated parameters, known limitations hit.

func (*ServiceLogger) With 

func (l *ServiceLogger) With(fields ...zap.Field) *ServiceLogger

With returns a new ServiceLogger with the additional fields attached to every subsequent log call. Useful for adding operation-scoped context: 

log := h.log.With(zap.String("bucket", bucket), zap.String("key", key))
log.Debug("fetching object")

func (*ServiceLogger) WithOperation 

func (l *ServiceLogger) WithOperation(op string) *ServiceLogger

WithOperation returns a child ServiceLogger scoped to the named operation (e.g. "CreateQueue", "PutObject"). All log calls on the returned logger automatically include an "operation" structured field.

In console mode, messages are also prefixed with a dim [Operation] tag so it is easy to trace which handler produced each log line: 

10:42:03  DEBUG  [S3] [PutObject]  stored object  {bucket=foo key=bar}

Typical usage at the top of a handler: 

log := h.log.WithOperation("PutObject")
log.Debug("decoded request", zap.String("bucket", bucket))

func (*ServiceLogger) ZapLogger 

func (l *ServiceLogger) ZapLogger() *zap.Logger

ZapLogger returns the underlying *zap.Logger so callers that require the raw logger (e.g. Docker GC) can create named children from the service-scoped logger.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.