mx

package module
v0.0.0-...-972f8a3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 19, 2025 License: Apache-2.0 Imports: 27 Imported by: 0

README ΒΆ

🌿 mx β€” A Continuation of Service Weaver

mx is a community-maintained fork of the original Service Weaver project by Google, which has been archived and is no longer under active development.

This fork aims to keep the project alive, maintain it, and continue evolving its capabilities for modern distributed systems in Go.


πŸ“¦ Installation

Ensure you have Go installed, version 1.24 or higher. Then, run the following to install the weaver command:

go install github.com/sh3lk/mx/cmd/mx@latest

go install installs the weaver command to $GOBIN, which defaults to $HOME/go/bin. Make sure this directory is included in your PATH. You can accomplish this, for example, by adding the following to your .bashrc and running source ~/.bashrc:

export PATH="$PATH:$HOME/go/bin"

πŸ“„ Documentation

Much of the documentation remains compatible with https://serviceweaver.dev, but updates specific to mx will be published here as the project evolves.


πŸ’¬ Community and Contributions

We welcome contributions of all kinds! Feel free to:

  • Open issues and suggest features
  • Submit pull requests
  • Help test and improve mx

πŸ“ License

This project is based on the original Service Weaver and follows the same license. See LICENSE for details.


mx is not affiliated with Google. It is an independent continuation of an open-source project.

Documentation ΒΆ

Overview ΒΆ

Package mx provides the interface for building single-image distributed programs.

A program is composed of a set of Go interfaces called components. Components are recognized by "mx generate" (typically invoked via "go generate"). "mx generate" generates code that allows a component to be invoked over the network. This flexibility allows MX to decompose the program execution across many processes and machines.

Index ΒΆ

Examples ΒΆ

Constants ΒΆ

View Source
const HealthzURL = "/debug/mx/healthz"

HealthzURL is the URL path on which MX performs health checks. Every application HTTP server must register a handler for this URL path, e.g.:

mux := http.NewServeMux()
mux.HandleFunc(mx.HealthzURL, func(http.ResponseWriter, *http.Request) {
    // ...
})

As a convenience, MX registers HealthzHandler under this URL path in the default ServerMux, i.e.:

http.HandleFunc(mx.HealthzURL, mx.HealthzHandler)

Variables ΒΆ

View Source
var HealthzHandler = func(w http.ResponseWriter, _ *http.Request) {
	fmt.Fprintf(w, "OK")
}

HealthzHandler is a health-check handler that returns an OK status for all incoming HTTP requests.

View Source
var RemoteCallError = errors.New("MX remote call error")

RemoteCallError indicates that a remote component method call failed to execute properly. This can happen, for example, because of a failed machine or a network partition. Here's an illustrative example:

// Call the foo.Foo method.
err := foo.Foo(ctx)
if errors.Is(err, mx.RemoteCallError) {
    // foo.Foo did not execute properly.
} else if err != nil {
    // foo.Foo executed properly, but returned an error.
} else {
    // foo.Foo executed properly and did not return an error.
}

Note that if a method call returns an error with an embedded RemoteCallError, it does NOT mean that the method never executed. The method may have executed partially or fully. Thus, you must be careful retrying method calls that result in a RemoteCallError. Ensuring that all methods are either read-only or idempotent is one way to ensure safe retries, for example.

Functions ΒΆ

func InstrumentHandler ΒΆ

func InstrumentHandler(label string, handler http.Handler) http.Handler

InstrumentHandler instruments the provided HTTP handler to collect sampled traces and metrics of HTTP request executions. Each trace and metric is labelled with the supplied label. The following metrics are collected:

  • mx_http_request_count: Total number of requests.
  • mx_http_error_count: Total number of 4XX and 5XX replies.
  • mx_http_request_latency_micros: Execution latency in microseconds.
  • mx_http_request_bytes_received: Total number of request bytes.
  • mx_http_request_bytes_returned: Total number of response bytes
Example ΒΆ
var mux http.ServeMux
mux.Handle("/foo", InstrumentHandler("foo", http.HandlerFunc(func(http.ResponseWriter, *http.Request) { /*...*/ })))
mux.Handle("/bar", InstrumentHandler("bar", http.HandlerFunc(func(http.ResponseWriter, *http.Request) { /*...*/ })))
http.ListenAndServe(":9000", &mux)

func InstrumentHandlerFunc ΒΆ

func InstrumentHandlerFunc(label string, f func(http.ResponseWriter, *http.Request)) http.Handler

InstrumentHandlerFunc is identical to InstrumentHandler but takes a function instead of an http.Handler.

func Run ΒΆ

func Run[T any, P PointerToMain[T]](ctx context.Context, app func(context.Context, *T) error) error

Run runs app as a MX application.

The application is composed of a set of components that include mx.Main as well as any components transitively needed by mx.Main. An instance that implement mx.Main is automatically created by mx.Run and passed to app. Note: other replicas in which mx.Run is called may also create instances of mx.Main.

The type T must be a struct type that contains an embedded `mx.Implements[mx.Main]` field. A value of type T is created, initialized (by calling its Init method if any), and a pointer to the value is passed to app. app contains the main body of the application; it will typically run HTTP servers, etc.

If this process is hosting the `mx.Main` component, Run will call app and will return when app returns. If this process is hosting other components, Run will start those components and never return. Most callers of Run will not do anything (other than possibly logging any returned error) after Run returns.

func main() {
    if err := mx.Run(context.Background(), app); err != nil {
        log.Fatal(err)
    }
}

Types ΒΆ

type AutoMarshal ΒΆ

type AutoMarshal struct{}

AutoMarshal is a type that can be embedded within a struct to indicate that "mx generate" should generate serialization methods for the struct.

Named struct types are not serializable by default. However, they can trivially be made serializable by embedding AutoMarshal. For example:

type Pair struct {
    mx.AutoMarshal
    x, y int
}

The AutoMarshal embedding instructs "mx generate" to generate serialization methods for the struct, Pair in this example.

Note, however, that AutoMarshal cannot magically make any type serializable. For example, "mx generate" will raise an error for the following code because the NotSerializable struct is fundamentally not serializable.

// ERROR: NotSerializable cannot be made serializable.
type NotSerializable struct {
    mx.AutoMarshal
    f func()   // functions are not serializable
    c chan int // chans are not serializable
}

func (AutoMarshal) MXMarshal ΒΆ

func (AutoMarshal) MXMarshal(*codegen.Encoder)

func (AutoMarshal) MXUnmarshal ΒΆ

func (AutoMarshal) MXUnmarshal(*codegen.Decoder)

type Implements ΒΆ

type Implements[T any] struct {
	// contains filtered or unexported fields
}

Implements[T] is a type that is be embedded inside a component implementation struct to indicate that the struct implements a component of type T. For example, consider a Cache component.

type Cache interface {
    Get(ctx context.Context, key string) (string, error)
    Put(ctx context.Context, key, value string) error
}

A concrete type that implements the Cache component is written as follows:

type lruCache struct {
    mx.Implements[Cache]
    ...
}

Because Implements is embedded inside the component implementation, methods of Implements are available as methods of the component implementation type and can be invoked directly. For example, given an instance c of type lruCache, we can call c.Logger().

func (Implements[T]) Logger ΒΆ

func (i Implements[T]) Logger(ctx context.Context) *slog.Logger

Logger returns a logger that associates its log entries with this component. Log entries are labeled with any OpenTelemetry trace id and span id in the provided context.

func (Implements[T]) MX ΒΆ

func (i Implements[T]) MX() MXInfo

MX returns runtime information about the deployed application.

type InstanceOf ΒΆ

type InstanceOf[T any] interface {
	// contains filtered or unexported methods
}

InstanceOf[T] is the interface implemented by a struct that embeds mx.Implements[T].

type Listener ΒΆ

type Listener struct {
	net.Listener // underlying listener
	// contains filtered or unexported fields
}

Listener is a network listener that can be placed as a field inside a component implementation struct. Once placed, MX automatically initializes the Listener and makes it suitable for receiving network traffic. For example:

type myComponentImpl struct {
    mx.Implements[MyComponent]
    myListener      mx.Listener
    myOtherListener mx.Listener
}

By default, all listeners listen on address ":0". This behavior can be modified by passing options for individual listeners in the application config. For example, to specify local addresses for the above two listeners, the user can add the following lines to the application config file:

[listeners]
myListener      = {local_address = "localhost:9000"}
myOtherListener = {local_address = "localhost:9001"}

Listeners are identified by their field names in the component implementation structs (e.g., myListener and myOtherListener). If the user wishes to assign different names to their listeners, they may do so by adding a `mx:"name"` struct tag to their listener fields, e.g.:

type myComponentImpl struct {
    mx.Implements[MyComponent]
    myListener      mx.Listener
    myOtherListener mx.Listener `mx:"mylistener2"`
}

Listener names must be valid Go identifier names. Listener names must be unique inside a given application binary, regardless of which components they are specified in. For example, it is illegal to declare a Listener field "foo" in two different component implementation structs, unless one is renamed using the `mx:"name"` struct tag.

HTTP servers constructed using this listener are expected to perform health checks on the reserved HealthzURL path. (Note that this URL path is configured to never receive any user traffic.)

func (*Listener) ProxyAddr ΒΆ

func (l *Listener) ProxyAddr() string

ProxyAddr returns the dialable address of the proxy that forwards traffic to this listener, or returns the empty string if there is no such proxy.

func (Listener) String ΒΆ

func (l Listener) String() string

String returns the address clients should dial to connect to the listener; this will be the proxy address if available, otherwise the <host>:<port> for this listener.

type MXInfo ΒΆ

type MXInfo struct {
	// Unique identifier for the application deployment.
	DeploymentID string
}

MXInfo holds runtime information about a deployed application.

type Main ΒΆ

type Main interface{}

Main is the interface implemented by an application's main component.

type NotRetriable ΒΆ

type NotRetriable interface{}

type PointerToMain ΒΆ

type PointerToMain[T any] interface {
	*T
	InstanceOf[Main]
}

PointerToMain is a type constraint that asserts *T is an instance of Main (i.e. T is a struct that embeds mx.Implements[mx.Main]).

type Ref ΒΆ

type Ref[T any] struct {
	// contains filtered or unexported fields
}

Ref[T] is a field that can be placed inside a component implementation struct. T must be a component type. MX will automatically fill such a field with a handle to the corresponding component.

func (Ref[T]) Get ΒΆ

func (r Ref[T]) Get() T

Get returns a handle to the component of type T.

type RoutedBy ΒΆ

type RoutedBy[T any] interface {
	// contains filtered or unexported methods
}

RoutedBy[T] is the interface implemented by a struct that embeds mx.RoutedBy[T].

type Unrouted ΒΆ

type Unrouted interface {
	// contains filtered or unexported methods
}

Unrouted is the interface implemented by instances that don't embed mx.WithRouter[T].

type WithConfig ΒΆ

type WithConfig[T any] struct {
	// contains filtered or unexported fields
}

WithConfig[T] is a type that can be embedded inside a component implementation. The MX runtime will take per-component configuration information found in the application config file and use it to initialize the contents of T.

Example ΒΆ

Consider a cache component where the cache size should be configurable. Define a struct that includes the size, associate it with the component implementation, and use it inside the component methods.

type cacheConfig struct
    Size int
}

type cache struct {
    mx.Implements[Cache]
    mx.WithConfig[cacheConfig]
    // ...
}

func (c *cache) Init(context.Context) error {
    // Use c.Config().Size...
    return nil
}

The application config file can specify these values as keys under the full component path.

["example.com/mypkg/Cache"]
Size = 1000

Field Names ΒΆ

You can use `toml` struct tags to specify the name that should be used for a field in a config file. For example, we can change the cacheConfig struct to the following:

type cacheConfig struct
    Size int `toml:"my_custom_name"`
}

And change the config file accordingly:

["example.com/mypkg/Cache"]
my_custom_name = 1000

func (*WithConfig[T]) Config ΒΆ

func (wc *WithConfig[T]) Config() *T

Config returns the configuration information for the component that embeds this mx.WithConfig.

Any fields in T that were not present in the application config file will have their default values.

Any fields in the application config file that are not present in T will be flagged as an error at application startup.

type WithRouter ΒΆ

type WithRouter[T any] struct{}

WithRouter[T] is a type that can be embedded inside a component implementation struct to indicate that calls to a method M on the component must be routed according to the the value returned by T.M().

Example ΒΆ

For example, consider a Cache component that maintains an in-memory cache.

type Cache interface {
    Get(ctx context.Context, key string) (string, error)
    Put(ctx context.Context, key, value string) error
}

We can create a router for the Cache component like this.

type cacheRouter struct{}
func (cacheRouter) Get(_ context.Context, key string) string { return key }
func (cacheRouter) Put(_ context.Context, key, value string) string { return key }

To associate a router with its component, embed mx.WithRouter in the component implementation.

type lruCache struct {
	mx.Implements[Cache]
	mx.WithRouter[cacheRouter]
}

For every component method that needs to be routed (e.g., Get and Put), the associated router should implement an equivalent method (i.e., same name and argument types) whose return type is the routing key. When a component's routed method is invoked, its corresponding router method is invoked to produce a routing key. Method invocations that produce the same key are routed to the same replica.

Routing Keys ΒΆ

A routing key can be any integer (e.g., int, int32), float (i.e. float32, float64), or string; or a struct where all fields are integers, floats, or strings. A struct may also embed AutoMarshal. For example, the following are valid routing keys.

int
int32
float32
float63
string
struct{}
struct{x int}
struct{x int; y string}
struct{mx.AutoMarshal; x int; y string}

Every router method must return the same routing key type. The following, for example, is invalid:

// ERROR: Get returns a string, but Put returns an int.
func (cacheRouter) Get(_ context.Context, key string) string { return key }
func (cacheRouter) Put(_ context.Context, key, value string) int { return 42 }

Semantics ΒΆ

NOTE that routing is done on a best-effort basis. MX will try to route method invocations with the same key to the same replica, but this is not guaranteed. As a corollary, you should never depend on routing for correctness. Only use routing to increase performance in the common case.

Directories ΒΆ

Path Synopsis
cmd
mx command
MX deploys and manages MX applications.
MX deploys and manages MX applications.
dev
docgen command
docgen generates a static web site from markdown and other files.
docgen generates a static web site from markdown and other files.
examples
bankofanthos command
Package main implements a demo banking application called Bank of Anthos.
Package main implements a demo banking application called Bank of Anthos.
chat command
collatz command
Package main implements a service that explores the Collatz conjecture.
Package main implements a service that explores the Collatz conjecture.
factors command
Package main implements a service to compute the factors of an integer.
Package main implements a service to compute the factors of an integer.
hello command
helloworld command
reverser command
internal
cond
Package cond implements a context-aware condition variable.
Package cond implements a context-aware condition variable.
config
Package config contains config related utilities.
Package config contains config related utilities.
env
Package env implements helper functions for dealing with environment variables.
Package env implements helper functions for dealing with environment variables.
files
Package files contains file-related utilities.
Package files contains file-related utilities.
heap
Package heap provide a min-heap implementation called Heap.
Package heap provide a min-heap implementation called Heap.
must
Package must provides a generic Must function.
Package must provides a generic Must function.
mx
net/call
Package call implements an RPC mechanism.
Package call implements an RPC mechanism.
pipe
Package pipe extends os.exec, making it easier to create pipes to subcommands.
Package pipe extends os.exec, making it easier to create pipes to subcommands.
reflection
Package reflection implements helpers for reflection code.
Package reflection implements helpers for reflection code.
register
Package register implements a write-once register.
Package register implements a write-once register.
routing
Package routing includes utilities for routing and assignments.
Package routing includes utilities for routing and assignments.
status
Package status contains code for implementing status related commands like "mx multi status" and "mx single dashboard".
Package status contains code for implementing status related commands like "mx multi status" and "mx single dashboard".
testdeployer
Package testdeployer includes unit tests for a remote mxn spawned by a test deployer.
Package testdeployer includes unit tests for a remote mxn spawned by a test deployer.
tool/callgraph
Package callgraph contains code to create visualizations of component call graphs stored inside a MX binary.
Package callgraph contains code to create visualizations of component call graphs stored inside a MX binary.
Package metadata provides support for the propagation of metadata information from a component method caller to the callee.
Package metadata provides support for the propagation of metadata information from a component method caller to the callee.
Package metrics provides an API for counters, gauges, and histograms.
Package metrics provides an API for counters, gauges, and histograms.
Package mxtest provides a way to test MX components.
Package mxtest provides a way to test MX components.
internal/chain
Package chain defines a chain of components A -> B -> C with methods to propagate a value from the head of the chain to the tail of the chain.
Package chain defines a chain of components A -> B -> C with methods to propagate a value from the head of the chain to the tail of the chain.
internal/generate
Package generate tests that "mx generate"-generated code *executes* correctly.
Package generate tests that "mx generate"-generated code *executes* correctly.
internal/simple
Package simple is used for a trivial test of code that uses MX.
Package simple is used for a trivial test of code that uses MX.
Package runtime contains code suitable for deployer implementers but not MX application developers.
Package runtime contains code suitable for deployer implementers but not MX application developers.
bin
Package bin contains code to extract data from a MX binary.
Package bin contains code to extract data from a MX binary.
bin/testprogram command
testprogram is used by bin tests.
testprogram is used by bin tests.
codegen
Package codegen contains functions and types used by the mx_gen.go files generated by "mx generate".
Package codegen contains functions and types used by the mx_gen.go files generated by "mx generate".
colors
Package colors contains color-related utilities.
Package colors contains color-related utilities.
deployers
Deployers provides useful utilities for implementing MX deployers.
Deployers provides useful utilities for implementing MX deployers.
envelope
Package envelope implements a sidecar-like process that connects a mxn to its environment.
Package envelope implements a sidecar-like process that connects a mxn to its environment.
logging
Package logging contains logging related utilities.
Package logging contains logging related utilities.
metrics
Package metrics implements MX metrics.
Package metrics implements MX metrics.
perfetto
Package perfetto provides utilities for encoding trace spans in a format that can be read by the Perfetto UI.
Package perfetto provides utilities for encoding trace spans in a format that can be read by the Perfetto UI.
protomsg
Package protomsg contains protobuf-related utilities.
Package protomsg contains protobuf-related utilities.
retry
Package retry contains code to perform retries with exponential backoff.
Package retry contains code to perform retries with exponential backoff.
tool
Package tool contains utilities for creating MX tools similar to mx-multi, mx-gke, and mx-gke-local.
Package tool contains utilities for creating MX tools similar to mx-multi, mx-gke, and mx-gke-local.
traces
Package perfetto contains libraries for displaying trace information in the Perfetto UI.
Package perfetto contains libraries for displaying trace information in the Perfetto UI.
version
Package version contains the version of the mx module and its constituent APIs (e.g., the pipe API, the codegen API).
Package version contains the version of the mx module and its constituent APIs (e.g., the pipe API, the codegen API).
sim
Package sim implements deterministic simulation.
Package sim implements deterministic simulation.
internal/bank
Package bank includes an example of how to test MX applications using the sim package.
Package bank includes an example of how to test MX applications using the sim package.
website
blog/deployers/multi command
Package main implements a simple multiprocess deployer.
Package main implements a simple multiprocess deployer.
blog/deployers/single command
Package main implements a simple singleprocess deployer.
Package main implements a simple singleprocess deployer.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL