Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrServiceAlreadyStarted = errors.New("connector/service: Service.Serve called more than once (Service is single-use; construct a new Service instance to re-start)")
ErrServiceAlreadyStarted is returned by Serve() when it is invoked more than once on the same Service instance. Serve is single-use by contract; after the first call (whether still running, returned normally, or returned with an error), subsequent calls are refused without side effects. Callers who need to re-start a service should construct a fresh Service instance. See SVC-F3 for the full rationale and the hazards of double-Serve (duplicate hook fires, competing signal handlers, leaked goroutines, double Cloud Map registration).
Functions ¶
This section is empty.
Types ¶
type Service ¶
type Service struct {
// AppName is the logical name used to discover this service's config
// file (service.yaml or {AppName}.yaml depending on layout). Required.
AppName string
// ConfigFileName overrides the config file basename. Optional —
// defaults to "service" when empty.
ConfigFileName string
// CustomConfigPath overrides the config file directory. Optional —
// defaults to the standard config search path when empty.
CustomConfigPath string
// WebServerConfig optionally enables a sidecar HTTP server (gin)
// for non-gRPC routes. When non-nil, Serve also starts the web
// server on its own port and shuts it down during Service teardown.
WebServerConfig *WebServerConfig
// RegisterServiceHandlers is the caller-provided callback that
// registers protobuf-generated server handlers on the gRPC server
// instance. Required — Serve fails fast with an error if nil.
RegisterServiceHandlers func(grpcServer *grpc.Server)
// DefaultHealthCheckHandler returns the SERVING status for the
// default (empty-string) health-check service name. Optional — if
// nil, the embedded grpc_health_v1 server reports SERVING
// unconditionally.
DefaultHealthCheckHandler func(ctx context.Context) grpc_health_v1.HealthCheckResponse_ServingStatus
// ServiceHealthCheckHandlers maps named gRPC services to their
// per-service health-check status function. Optional. Use this to
// expose per-service liveness independent of the default handler.
ServiceHealthCheckHandlers map[string]func(ctx context.Context) grpc_health_v1.HealthCheckResponse_ServingStatus
// RateLimit, if non-nil, gates inbound RPCs through the supplied
// rate limiter. Note: the rate-limit reject path was added in P1-1
// (deep-review-2026-04-12); prior to that fix the limiter recorded
// hits but did not actually reject excess traffic.
RateLimit ratelimiter.RateLimiterIFace
// UnaryServerInterceptors is the unary interceptor chain. Order
// matters: interceptors run in slice order on the way in (request)
// and reverse order on the way out (response). The recommended
// ordering is: tracing -> metrics -> auth -> tenancy -> logger ->
// recovery -> handler. Wire metrics and logger via the
// adapters/metrics and adapters/logger constructors.
UnaryServerInterceptors []grpc.UnaryServerInterceptor
// StreamServerInterceptors is the stream interceptor chain.
// Same ordering rules as UnaryServerInterceptors.
StreamServerInterceptors []grpc.StreamServerInterceptor
// StatsHandler is the optional grpc/stats.Handler for connection
// and RPC lifecycle events. Used by tracing/metrics integrations
// that need lower-level events than interceptors expose.
StatsHandler stats.Handler
// UnknownStreamHandler handles RPCs whose method is not registered
// on the server. Default behavior (nil) returns Unimplemented;
// override to implement proxying or graceful degradation.
UnknownStreamHandler grpc.StreamHandler
// BeforeServerStart runs synchronously immediately before the gRPC
// listener is created. Use it for late configuration (mutating
// other Service fields) or for warm-up actions that must complete
// before the server accepts traffic. nil is a no-op.
BeforeServerStart func(svc *Service)
// AfterServerStart runs synchronously immediately after the gRPC
// server is accepting connections AND the Cloud Map register call
// has succeeded. nil is a no-op.
AfterServerStart func(svc *Service)
// BeforeServerShutdown runs synchronously at the start of the
// shutdown sequence, before deregister or grpc.GracefulStop. nil
// is a no-op. Use this to drain caller-managed background work.
BeforeServerShutdown func(svc *Service)
// AfterServerShutdown runs synchronously after the gRPC server has
// fully stopped and Cloud Map deregistration has completed. nil
// is a no-op. This is the last hook to fire before Serve returns.
AfterServerShutdown func(svc *Service)
// GracefulStopTimeout bounds how long the default signal-driven
// shutdown path will wait for in-flight RPCs to drain via
// grpc.Server.GracefulStop before escalating to grpc.Server.Stop
// (hard-close of all connections).
//
// Zero (the default) means 30 seconds. Set this to match your
// deployment's terminationGracePeriodSeconds (k8s) / task SIGKILL
// grace window (ECS) minus a safety margin, so the fallback Stop
// always runs before the orchestrator SIGKILLs the process.
//
// Fix: SVC-F1 (deep-review-2026-04-13-contrarian). Prior to this
// field, the default signal path called grpc.Server.Stop directly,
// contradicting the lifecycle godoc and killing in-flight RPCs.
GracefulStopTimeout time.Duration
// ShutdownCancel enables the opt-in ShutdownCtx capability. When
// false (the default), ShutdownCtx() returns nil and no shutdown
// context is allocated — existing consumers see ZERO behavior
// change. When true, Serve() allocates a dedicated context that is
// cancelled at the phase named by ShutdownCancelPhase (or
// ShutdownPhaseImmediate if the phase is ShutdownPhaseNone), giving
// cooperating handlers an in-band signal to stop new work and
// return early.
//
// Set this BEFORE calling Serve. Mutating it after Serve has been
// called has no effect and is racy with respect to the internal
// allocation path.
//
// Fix: SVC-F5 (2026-04-14). Resolves the gap where
// grpc.GracefulStop does not cancel the handler's ctx and
// long-running handlers have no way to learn the server is
// draining.
ShutdownCancel bool
// ShutdownCancelPhase names the shutdown phase at which
// ShutdownCtx() fires when ShutdownCancel is true. See
// ShutdownPhase for the per-phase semantics. When ShutdownCancel
// is true but this is the zero value (ShutdownPhaseNone), the
// phase defaults to ShutdownPhaseImmediate — enabling the
// capability without picking a phase gives the fastest-shutdown
// mental model.
//
// Ignored when ShutdownCancel is false.
ShutdownCancelPhase ShutdownPhase
// contains filtered or unexported fields
}
Service is the gRPC server lifecycle owner for a connector-managed service instance. A Service binds together: configuration loading (service.yaml), gRPC server setup, AWS Cloud Map service discovery registration/heartbeat/deregistration, optional health-check handlers, optional rate limiting and TLS, optional companion HTTP web server, and a set of caller-defined lifecycle hooks (BeforeServerStart, AfterServerStart, BeforeServerShutdown, AfterServerShutdown).
Construct one with NewService, then optionally configure exported fields BEFORE calling Serve. Serve blocks until an OS signal (SIGTERM/SIGINT) or an explicit GracefulStop/ImmediateStop call. After Serve returns the Service has fully shut down — both the gRPC server and the Cloud Map deregistration. A given Service is single-use; do not call Serve more than once.
Concurrency contract:
- Exported fields (AppName, hooks, interceptors, RateLimit, WebServerConfig, etc.) MUST be set before Serve is called and MUST NOT be mutated after. The Service does not synchronize reads of these fields against caller-side writes. Use Frozen() to check whether the Service has entered serving state; Frozen returns true once Serve has been called and the configuration is locked.
- GracefulStop, ImmediateStop, and the internal signal handler may be invoked concurrently from any goroutine — they are guarded by _mu plus an atomic.Bool one-shot CAS (_deregFired) on the Cloud Map deregister path that ensures exactly-once dereg without blocking concurrent callers.
Lifecycle (see also _src/docs/repos/connector/architecture.md):
NewService -> (configure fields) -> Serve
Serve: readConfig -> (BeforeServerStart) -> startServer
startServer: bind -> register-with-cloudmap -> serve gRPC
-> (AfterServerStart) -> awaitOsSigExit
-> (BeforeServerShutdown) -> deregister-with-cloudmap ->
grpc.GracefulStop (bounded by GracefulStopTimeout;
falls back to grpc.Stop on timeout)
-> (AfterServerShutdown) -> Serve returns
GracefulStopTimeout caps how long the default signal path will wait for in-flight RPCs to drain before forcing termination — see that field's godoc for details.
func NewService ¶
func NewService(appName string, configFileName string, customConfigPath string, registerServiceHandlers func(grpcServer *grpc.Server)) *Service
NewService constructs a Service ready for further configuration. appName is the logical service name used to locate the config file and to seed Cloud Map registration. configFileName overrides the default basename ("service") when the YAML lives at a non-default path. customConfigPath overrides the default config search directory. registerServiceHandlers is the caller's protobuf handler-registration callback — typically a one-line wrapper around the generated pb.RegisterFooServiceServer call. It is REQUIRED; passing nil here causes Serve to fail.
The returned Service has only the four constructor parameters set; configure all other fields directly on the returned struct before calling Serve.
func (*Service) CurrentlyServing ¶
CurrentlyServing indicates if this service health status indicates currently serving mode or not
func (*Service) Frozen ¶ added in v1.8.4
Frozen reports whether the Service's configuration is locked. It returns true once Serve() has been called (even if Serve subsequently failed or returned), and false before that point.
Callers that programmatically build a Service and conditionally mutate exported fields can use Frozen as a pre-check:
if !svc.Frozen() {
svc.GracefulStopTimeout = 10 * time.Second
}
The method is safe to call concurrently from any goroutine; it reads the same monotonic _started flag that guards Serve re-entry (SVC-F3).
AD-3: lifecycle guardrail for post-Serve mutation.
func (*Service) GracefulStop ¶
func (s *Service) GracefulStop()
GracefulStop initiates an orderly shutdown of the Service.
Behavior depends on whether Serve() has been called yet:
In the normal "Serve is running" case, GracefulStop signals the unified quit handler (the same one the OS-signal path wakes) and BLOCKS until that handler completes the full teardown: BeforeServerShutdown is NOT re-invoked from this path (Serve owns BeforeServerShutdown), but the quit handler publishes the offline notification, unsubscribes from SNS, deletes the health report, runs WebServerConfig.CleanUp, deregisters from Cloud Map, fires the SVC-F5 PreDrain / PostGraceExpiry phases, and runs the bounded grpc.Server.GracefulStop -> Stop escalation. By the time GracefulStop returns, the gRPC server has fully stopped and Serve()'s blocking awaitOsSigExit + quit-wait have unblocked (Serve will run AfterServerShutdown next and then return nil).
Serve()'s awaitOsSigExit only wakes on an OS signal delivered via signal.Notify, so this method first sends SIGTERM to its own pid (SVC-F8) to unblock the sig-demux goroutine; the Go runtime routes that signal to awaitOsSigExit's local channel rather than the process default handler, so the process is NOT terminated by the signal itself. This mirrors the self-SIGTERM pattern already used on the grpc-server error path (FIX #13, service.go:1138-1149).
In the rare pre-Serve case (the Service was constructed but Serve was never called), GracefulStop falls through to a legacy idempotent teardown body that publishes the offline notification, runs CleanUp, deregisters, and calls grpc.Server.GracefulStop directly. This path is provided as a safety net for callers that perform partial setup outside Serve. Most callers will never hit it.
GracefulStop is safe to call concurrently with the internal signal-driven shutdown path and with ImmediateStop. The unified quit channel (SVC-F7) routes both paths into one quit-handler invocation; the buffer-1 send on _quit is idempotent (a second concurrent caller's send drops via select default but the caller still blocks on _quitDone alongside the winner). The Cloud Map deregister step inside the quit handler is additionally guarded by a one-shot atomic.Bool CAS (_deregFired, see SVC-F4) so concurrent programmatic + signal callers cannot double-fire the AWS API.
Re-entry from a gRPC handler: calling GracefulStop from inside a gRPC unary/stream handler (e.g. an admin RPC that triggers shutdown) STALLS for up to GracefulStopTimeout (default 30s) and then returns. The reason is that the quit handler's teardown invokes grpc.Server.GracefulStop, which blocks on ALL in-flight RPCs — including the handler that called this method, which is parked inside GracefulStop waiting on _quitDone. The bounded drain helper escalates to grpc.Server.Stop after GracefulStopTimeout, severing the transport; the handler's response is not delivered and its client observes an Unavailable / Canceled error.
If you must trigger shutdown from a gRPC handler, spawn a goroutine so the handler can return normally before teardown begins:
go s.GracefulStop()
return &AdminResponse{Ok: true}, nil
GracefulStop is safe to call from a goroutine — the unified quit channel (SVC-F7) serializes all shutdown callers.
SVC-F5 / rule #13: this routing makes the ShutdownCtx() promise (every fire-site phase runs before Serve returns) hold on the programmatic-stop path, not just the signal path. A consumer that opted in to ShutdownCancel=true and triggers shutdown via GracefulStop() will see the configured phase fire before this method returns (and the safety-net Final fire from the Serve() defer fires before Serve unblocks).
Fix: SVC-F7 (C2-001 + C2-002) established the unified quit routing. SVC-F8 (F1, deep-review-2026-04-14-contrarian-pass3) added the self-SIGTERM so the unified routing actually wakes Serve()'s awaitOsSigExit. Prior to SVC-F8 the unified routing completed teardown correctly but Serve()'s main goroutine stayed parked in awaitOsSigExit until a real OS signal arrived — a zombie-process hazard on programmatic-only stop paths.
func (*Service) ImmediateStop ¶
func (s *Service) ImmediateStop()
ImmediateStop forcibly tears down the Service without waiting for in-flight RPCs to complete.
Behavior depends on whether Serve() has been called yet:
In the normal "Serve is running" case, ImmediateStop sets the internal "_immediateStopRequested" atomic, signals the unified quit handler, and BLOCKS until the handler completes teardown. The quit handler observes _immediateStopRequested and bypasses the bounded grpc.GracefulStop drain entirely, calling grpc.Server.Stop directly. The SVC-F5 PostGraceExpiry hook still fires synchronously before gs.Stop, so handlers observing ShutdownCtx see the cancel happens-before the transport is severed.
As with GracefulStop, ImmediateStop sends SIGTERM to its own pid (SVC-F8) so Serve()'s awaitOsSigExit releases; the Go runtime routes the signal to the signal.Notify channel rather than the process default handler, so this self-signal does NOT terminate the process.
In the rare pre-Serve case, ImmediateStop falls through to a legacy idempotent teardown body that ends with grpc.Server.Stop directly. As with GracefulStop, this is a safety net for callers that performed partial setup outside Serve.
Use this for hard shutdown paths (panic recovery, fatal config error, shutdown deadline exceeded). Prefer GracefulStop in the normal case.
Like GracefulStop, this method is safe to invoke concurrently with the signal-driven shutdown path; the unified quit channel (SVC-F7) guarantees the quit handler runs exactly once regardless of how many call sites raced. ImmediateStop's "immediate" semantics are PRESERVED under concurrent shutdown: setting _immediateStopRequested before the quit handler reads it forces the handler down the gs.Stop()-direct path even if a concurrent GracefulStop call won the buffer-1 send. The Cloud Map deregister inside the quit handler is guarded by a one-shot atomic.Bool CAS (_deregFired, see SVC-F4) so concurrent callers cannot stall on each other's AWS poll loop.
Re-entry from a gRPC handler: calling ImmediateStop from inside a gRPC handler does NOT stall 30s the way GracefulStop does — the quit handler observes _immediateStopRequested and skips the bounded grpc.GracefulStop drain entirely, calling gs.Stop() directly, which severs the transport without waiting for in-flight RPCs. The calling handler's response is not delivered and its client observes an Unavailable / Canceled error immediately. If the handler must return a response to its client before shutdown severs the transport, spawn a goroutine:
go s.ImmediateStop()
return &AdminResponse{Ok: true}, nil
ImmediateStop is safe to call from a goroutine — the unified quit channel (SVC-F7) serializes all shutdown callers and _immediateStopRequested is a one-way latch.
SVC-F5 / rule #13: this routing makes the ShutdownCtx() promise hold on the programmatic ImmediateStop path — the configured fire site runs before this method returns even when shutdown is triggered without an OS signal. This promise applies ONLY to the normal "Serve is running" case. In the pre-Serve fallback no ShutdownCancelPhase fire runs, because Serve() is the only site that allocates _shutdownCtx; consumers observing ShutdownCtx() before Serve() is entered will see a nil context regardless of ShutdownCancelPhase, which is the expected contract for any consumer opting in via ShutdownCancel=true.
Fix: SVC-F7 (C2-001 + C2-002) established the unified quit routing. SVC-F8 (F1, deep-review-2026-04-14-contrarian-pass3) added the self-SIGTERM so the unified routing actually wakes Serve()'s awaitOsSigExit. Godoc precision: C2-007 (pass-2), F3 re-entry contract (pass-3).
func (*Service) LocalAddress ¶
LocalAddress returns the service server's address and port
func (*Service) Serve ¶
Serve runs the full service lifecycle and blocks until the gRPC server has shut down. The sequence is:
- Read config from service.yaml.
- Bind the gRPC listener and (optionally) the sidecar HTTP server.
- Run BeforeServerStart hook.
- Register with AWS Cloud Map (if Cloud Map is configured).
- Start the gRPC server in a worker goroutine.
- Run AfterServerStart hook.
- Block on awaitOsSigExit waiting for SIGTERM/SIGINT, an internal panic from the server goroutine, or an explicit GracefulStop / ImmediateStop call.
- Run BeforeServerShutdown hook.
- Deregister from Cloud Map (one-shot atomic.Bool CAS-guarded — see _deregFired).
- grpc.Server.GracefulStop (or Stop, depending on shutdown path).
- Run AfterServerShutdown hook.
Returns the first error encountered along the way. A successful shutdown returns nil. Serve is single-use: calling Serve more than once on the same Service instance returns ErrServiceAlreadyStarted without side effects. The single-use contract is enforced atomically via a compare-and-swap on s._started, so concurrent Serve calls from two goroutines are safe: exactly one proceeds, the other receives the sentinel error (SVC-F3).
Terminal-failure contract: any error returned by Serve — including readConfig / setupServer failures that occur BEFORE the gRPC server binds a port — puts this *Service into a terminal state. The CAS guard on s._started has already claimed the instance, so a second Serve() call on the same value returns ErrServiceAlreadyStarted even though no gRPC work ever ran. Consumers MUST construct a new *Service via NewService(...) to retry; they CANNOT reuse this one. The single-use CAS (SVC-F3) exists to protect one-shot resources (shutdown signal context, sync.Once-guarded hooks, Cloud Map registration ledger) from re-entry, and that protection is exactly what makes the instance unrecoverable after an early failure.
All exported fields on Service must be set before calling Serve; post-Serve mutation is not synchronized. Use Frozen() to check whether the Service has entered the frozen (serving/served) state.
func (*Service) ShutdownCtx ¶ added in v1.8.0
ShutdownCtx returns the opt-in handler shutdown-signal context, or nil if the capability is disabled (ShutdownCancel is false, or Serve() has not yet allocated it).
Handlers that want to cooperate with graceful shutdown should observe Done() on the returned context in addition to their normal RPC ctx:
func (s *Handler) LongRunning(ctx context.Context, req *pb.Req) (*pb.Resp, error) {
shut := s.svc.ShutdownCtx()
for chunk := range work {
if shut != nil {
select {
case <-shut.Done():
return nil, status.Error(codes.Unavailable, "server draining")
case <-ctx.Done():
return nil, ctx.Err()
default:
}
}
processChunk(chunk)
}
return &pb.Resp{}, nil
}
Returning nil when the capability is disabled is intentional — it lets handlers use a nil-check fast path that compiles away when the consumer hasn't opted in, rather than always allocating a context just to have something non-nil to return.
The returned context is also cancelled as a safety net at the end of Serve, so a handler observing ShutdownCtx.Done() is guaranteed to see it fire before Serve returns — even if no configured phase matched (e.g., a future phase constant is added and not all fire sites are updated).
SVC-F7 (2026-04-14): the safety net applies symmetrically on the programmatic shutdown path. GracefulStop() and ImmediateStop() route through the same unified quit handler the signal path wakes, so every fireShutdownCancelIfPhase site (Immediate, PreDrain, PostGraceExpiry) AND the deferred Final fire run before the exported stop method returns to the caller. Prior to SVC-F7 these methods bypassed every phase fire and Serve() remained blocked waiting on an OS signal; project rule #13 programmatic-stop semantics are now enforceable end to end.
Fix: SVC-F5 (2026-04-14), extended by SVC-F7 (2026-04-14).
type ShutdownPhase ¶ added in v1.8.0
type ShutdownPhase int
ShutdownPhase controls when Service.ShutdownCtx() fires during the shutdown sequence. It is used with Service.ShutdownCancelPhase and takes effect only when Service.ShutdownCancel is true.
Design: SVC-F5 (see _src/docs/repos/connector/plans/ 2026-04-14__svc-f5-shutdown-ctx-design.md).
The capability is strictly opt-in. Existing consumers that do not set ShutdownCancel see ZERO behavior change — ShutdownCtx() returns nil, no context is allocated, and no fire site is taken.
const ( // ShutdownPhaseNone disables the ShutdownCtx capability. ShutdownCtx() // returns nil. This is the zero value so the default Service has no // shutdown-cancel behavior unless the consumer explicitly opts in. ShutdownPhaseNone ShutdownPhase = iota // ShutdownPhaseImmediate fires the cancel the instant the signal // handler receives SIGTERM/SIGINT/SIGQUIT, before BeforeServerShutdown // runs. Use for short-API services where handlers should abort // immediately rather than consume any grace time. The fire happens // inside the signal-demux goroutine, before it notifies Serve that a // signal was received, so handlers observing ShutdownCtx.Done() see // the cancel happens-before any subsequent Serve shutdown step. ShutdownPhaseImmediate // ShutdownPhasePreDrain fires the cancel inside the unified quit // handler, AFTER all non-gRPC teardown has completed and // IMMEDIATELY BEFORE the bounded graceful-stop attempt (or the // ImmediateStop bypass) hands control to the gRPC server. By the // time handlers observe ShutdownCtx.Done() on this phase, the // following have already run in the quit handler, in order: // SNS offline-discovery publish, SNS topic unsubscribe, health // report data-store delete, stopHealthReportService tick, // WebServerConfig.CleanUp, clear local address, setServing(false), // Cloud Map deregisterInstance (one-shot CAS), and Disconnect on // the SD/SQS/SNS AWS clients. What has NOT yet run: the bounded // grpc.GracefulStop drain (or, on the ImmediateStop bypass, the // direct grpc.Server.Stop call) and the listener close. The // service-discovery layer has already stopped routing to us — // handlers use this phase to stop cooperating before the grace // window begins, which is the good middle-ground default for // typical gRPC services. // // Note: BeforeServerShutdown runs BEFORE this fire site only on // the signal path (Serve owns that hook and calls it after // awaitOsSigExit unblocks, before sending on the quit channel). // On the programmatic GracefulStop/ImmediateStop path the // BeforeServerShutdown hook is not re-invoked from the quit // handler — Serve owns BeforeServerShutdown — so handlers should // not assume BeforeServerShutdown has executed when PreDrain // fires via a programmatic stop. ShutdownPhasePreDrain // ShutdownPhasePostGraceExpiry fires the cancel only after // grpc.Server.GracefulStop's own bounded timeout (GracefulStopTimeout) // elapses, as a "last warning" before grpc.Server.Stop severs the // transport. Gives handlers maximum grace; useful for long-running // imports where interruption is expensive. ShutdownPhasePostGraceExpiry )
func (ShutdownPhase) String ¶ added in v1.8.0
func (p ShutdownPhase) String() string
String returns a human-readable name for the phase (used in log lines).
type WebServerConfig ¶ added in v1.0.6
type WebServerConfig struct {
AppName string
ConfigFileName string
CustomConfigPath string
// define web server router info
WebServerRoutes map[string]*ginw.RouteDefinition
// getter only
WebServerLocalAddress string
// clean up func
CleanUp func()
}
func (*WebServerConfig) GetWebServerLocalAddress ¶ added in v1.0.7
func (w *WebServerConfig) GetWebServerLocalAddress() string
GetWebServerLocalAddress returns FQDN url to the local web server
Source Files
Directories