dcontext

package
v1.1.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Dec 9, 2020 License: Apache-2.0 Imports: 2 Imported by: 8

Documentation

Overview

Package dcontext provides tools for dealing with separate hard/soft cancellation of Contexts.

Given

softCtx := WithSoftness(hardCtx)

then

// The soft Context being done signals the end of "normal
// operation", and the program should initiate a graceful
// shutdown; a "soft shutdown".  In other words, it means, "You
// should start shutting down now."
<-softCtx.Done()

// The hard Context being done signals that the time for a
// graceful shutdown has passed and that the program should
// terminate *right now*, not-so-gracefully; a "hard shutdown".
// In other words, it means, "If you haven't finished shutting
// down yet, then you should hurry it up."
<-HardContext(softCtx).Done()

When writing code that makes use of a Context, which Context should you use, the soft Context or the hard Context?

- For most normal-operation code, you should use the soft Context (since this is most code, I recommend that you name it just `ctx`, not `softCtx`).

- For shutdown/cleanup code, you should use the hard Context (`dcontext.HardContext(ctx)`).

- For normal-operation code that explicitly may persist in to the post-shutdown-initiated grace-period, it may be appropriate to use the hard Context.

Design principles

- The lifetimes of the various stages (normal operation, shutdown) should be signaled with Contexts, rather than with bare channels. Because each stage may want to call a function that takes a Context, there should be a Context whose lifetime is scoped to the lifetime of that stage. If things were signaled with bare channels, things taking a Context might shut down too late or too early (depending on whether the Context represented hard shutdown (as it does for pkg/supervisor) or soft shutdown).

- A soft graceful shutdown is enough fully signal a shutdown, and if everything is well-behaved will perform a full shutdown; analogous to how clicking the "X" button in the upper corner of a window *should* be enough to quit the program. The harder not-so-graceful is the fallback for when something isn't well-behaved (whether that be local code or a remote network service) and isn't shutting down in an acceptable time; analogous to the window manager prompting you "the program is not responding, would you like to force-kill it?"

- There should only be one thing to pass around. For much of amb-sidecar's life (2019-02 to 2020-08), it used two separate Contexts for hard and soft cancellation, both explicitly passed around. This turned out to be clunky: (1) it was far too easy to accidentally use the wrong one; (2) the hard Context wouldn't have Values attached to the soft Context or vice-versa, so if you cared about both Values and cancellation, there were situations where *both* were the wrong choice.

- It should be simple and safe to interoperate with dcontext-unaware code; code needn't be dcontext-aware if it doesn't have fancy shutdown logic. This is one of the reasons why (in conjunction with "A soft shutdown is enough to fully signal a shutdown") the main Context that gets passed around is the soft Context and you need to call `dcontext.HardContext(ctx)` to get the hard Context; the hard-shutdown case is opt-in to facilitate code that has shutdown logic that might not be instantaneous and might need to be cut short if it takes too long (such as a server waiting for client connections to drain). Simple code with simple roughly instantaneous shutdown logic need not be concerned about hard Contexts and shutdown getting cut short.

Interfacing dcontext-aware code with dcontext-unaware code

When dcontext-aware code passes the soft Context to dcontext-unaware code, then that callee code will shutdown at the beginning of the shutdown grace period. This is correct, because the beginning of that grace period means "start shutting down" (on the above principles); if the callee code is dcontext-unaware, then shutting down when told to start shutting down is tautologically the right thing. If it isn't the right thing, then the code is code that needs to be made dcontext-aware (or adapted to be dcontext-aware, as in the HTTP server example).

When dcontext-unaware code passes a hard (normal) Context to dcontext-aware code, then that callee code will observe the <-ctx.Done() and <-HardContext(ctx).Done() occurring at the same instant. This is correct, because the caller code doesn't allow any grace period between "start shutting down" and "you need to finish shutting down now", so both of those are in the same instant.

Because of these two properties, it is the correct thing for...

- dcontext-aware caller code to just always pass the soft Context to things, regardless of whether the code being called it is dcontext-aware or not, and for

- dcontext-aware callee code to just always assume that the Context it has received is a soft Context (if for whatever reason it really cares, it can check if `ctx == dcontext.HardContext(ctx)`).

Example (Callee)

This example shows a simple 'exampleCallee' that is a worker function that takes a Context and uses it to support graceful shutdown.

//nolint:deadcode
package main

import (
	"context"

	"github.com/datawire/dlib/dcontext"
)

// This example shows a simple 'exampleCallee' that is a worker function that
// takes a Context and uses it to support graceful shutdown.
func main() {
	// Ignore this function, it's just here because godoc won't let let you
	// define an example function with arguments.
}

// This is the real example function that you should be paying attention to.
func exampleCallee(ctx context.Context, datasource <-chan Data) (err error) {
	// We assume that ctx is a soft Context

	defer func() {
		// We use the hard Context as the Context for shutdown logic.
		ctx := dcontext.HardContext(ctx)
		_err := DoShutdown(ctx)
		// Don't hide an error returned by the main part of the work.
		if err == nil {
			err = _err
		}
	}()

	// Run the main "normal-operation" part of the code until ctx is done.
	// We use the passed-in soft Context as the context for normal
	// operation.
	for {
		select {
		case dat := <-datasource:
			if err := DoWorkOnData(ctx, dat); err != nil {
				return err
			}
		case <-ctx.Done():
			return
		}
	}
}
Example (Caller)

This should be a very simple example of a parent caller function, showing how to manage a hard/soft Context and how to call code that is dcontext-aware.

package main

import (
	"context"
	"net"
	"net/http"
	"time"

	"github.com/datawire/dlib/dcontext"
)

// This should be a very simple example of a parent caller function, showing how
// to manage a hard/soft Context and how to call code that is dcontext-aware.
func main() error {
	ctx := context.Background()                       // Context is hard by default
	ctx, timeToDie := context.WithCancel(ctx)         // hard Context => hard cancel
	ctx = dcontext.WithSoftness(ctx)                  // make it soft
	ctx, startShuttingDown := context.WithCancel(ctx) // soft Context => soft cancel

	retCh := make(chan error)
	go func() {
		retCh <- ListenAndServeHTTPWithContext(ctx, &http.Server{
			// ...
		})
	}()

	// Run for a while.
	time.Sleep(10 * time.Second)

	// Shut down.
	startShuttingDown() // Soft shutdown; start draining connections.
	select {
	case err := <-retCh:
		// It shut down fine with just the soft shutdown; everything was
		// well-behaved.  It isn't necessary to cut shutdown short by
		// triggering a hard shutdown with timeToDie() in this case.
		return err
	case <-time.After(5 * time.Second): // shutdown grace period
		// It's taking too long to shut down--it seems that some clients
		// are refusing to hang up.  So now we trigger a hard shutdown
		// and forcefully close the connections.  This will cause errors
		// for those clients.
		timeToDie() // Hard shutdown; cause errors for clients
		return <-retCh
	}
}

// ListenAndServeHTTPWithContext runs server.ListenAndServe() on an http.Server,
// but properly calls server.Shutdown when the Context is canceled.
//
// It obeys hard/soft cancellation as implemented by dcontext.WithSoftness; it
// calls server.Shutdown() when the soft Context is canceled, and the hard
// Context being canceled causes the .Shutdown() to hurry along and kill any
// live requests and return, instead of waiting for them to be completed
// gracefully.
//
// PS: Since this example function is actually useful, it's published as part of
// the github.com/datawire/dlib/dutil package.
func ListenAndServeHTTPWithContext(ctx context.Context, server *http.Server) error {
	// An HTTP server is a bit of a complex example; for two reasons:
	//
	//  1. Like all network servers, it is a thing that manages multiple
	//     worker goroutines.  Because of this, it is an exception to a
	//     usual rule of Contexts:
	//
	//      > Do not store Contexts inside a struct type; instead, pass a
	//      > Context explicitly to each function that needs it.
	//      >
	//      > -- the "context" package documentation
	//
	//  2. http.Server has its own clunky soft/hard shutdown mechanism, and
	//     a large part of what this function is doing is adapting that to
	//     the less-clunky dcontext mechanism.
	//
	// For those reasons, this isn't necessarily a good instructive example
	// of how to use dcontext, but it is a *real* example.

	// Regardless of if you use dcontext, you should always set
	// `.BaseContext` on your `http.Server`s so that your HTTP Handler
	// receives a request object that has `Request.Context()` set correctly.
	server.BaseContext = func(_ net.Listener) context.Context {
		// We use the hard Context here instead of the soft Context so
		// that in-progress requests don't get interrupted when we enter
		// the shutdown grace period.
		return dcontext.HardContext(ctx)
	}

	serverCh := make(chan error)
	go func() {
		serverCh <- server.ListenAndServe()
	}()
	select {
	case err := <-serverCh:
		// The server quit on its own.
		return err
	case <-ctx.Done():
		// A soft shutdown has been initiated; call server.Shutdown().
		return server.Shutdown(dcontext.HardContext(ctx))
	}
}
Example (PollingCallee)

This example shows a simple 'examplePollingCallee' that is a worker function that takes a Context and uses it to support graceful shutdown.

Unlike the plain "Callee" example, instead of using the <-ctx.Done() channel to select when to shut down, it polls ctx.Err() in a loop to decide when to shut down.

//nolint:deadcode,errcheck
package main

import (
	"context"

	"github.com/datawire/dlib/dcontext"
)

// This example shows a simple 'examplePollingCallee' that is a worker function that
// takes a Context and uses it to support graceful shutdown.
//
// Unlike the plain "Callee" example, instead of using the <-ctx.Done() channel
// to select when to shut down, it polls ctx.Err() in a loop to decide when to
// shut down.
func main() {
	// Ignore this function, it's just here because godoc won't let let you
	// define an example function with arguments.
}

// This is the real example function that you should be paying attention to.
func examplePollingCallee(ctx context.Context) {
	// We assume that ctx is a soft Context

	// Run the main "normal-operation" part of the code until ctx is done.
	// We use the passed-in soft Context as the context for normal
	// operation.
	for ctx.Err() == nil { // ctx.Err() returns nil iff ctx is not done
		DoWork(ctx)
	}

	// Once the soft ctx is done, we use the hard Context as the context for
	// shutdown logic.
	ctx = dcontext.HardContext(ctx)
	DoShutdown(ctx)
}

Index

Examples

Constants

This section is empty.

Variables

This section is empty.

Functions

func HardContext

func HardContext(softCtx context.Context) context.Context

HardContext takes a child Context that is canceled sooner (a "soft" cancellation) and returns a Context with the same values, but with the cancellation of a parent Context that is canceled later (a "hard" cancellation).

Such a "soft" cancellation Context is created by WithSoftness(hardCtx). If the passed-in Context doesn't have softness (WithSoftness isn't somewhere in its ancestry), then it is returned unmodified, because it is already hard.

func WithSoftness

func WithSoftness(hardCtx context.Context) (softCtx context.Context)

WithSoftness returns a copy of the parent "hard" Context with a way of getting the parent's Done channel. This allows the child to have an earlier cancellation, triggering a "soft" shutdown, while allowing hard/soft-aware functions to use HardContext() to get the parent's Done channel, for a "hard" shutdown.

Types

This section is empty.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL