blackstart

package module
v0.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 19, 2025 License: Apache-2.0 Imports: 14 Imported by: 0

README

Blackstart

Lint and Test

Starting a system from scratch can require a lot of manual work. Blackstart provides an ordered workflow for bootstrapping cloud infrastructure and application deployments after the initial compute and other infrastructure is deployed. It is designed to be idempotent, implementing a "check then set" approach. This means that it can be run on a periodic basis to ensure that the system is kept in the desired state.

[!WARNING]
Blackstart is still being developed and does not currently have an initial release. Artifacts including helm charts and container images are not yet published.

To avoid race conditions, Blackstart operates on eventual consistency. It builds a directed acyclic graph of operations, processing them in a topological order. This ensures that operations are run only after their dependencies have been met. If an operation fails, Blackstart logs the information and tries again during the next run. Failure is expected during the initial setup and the expectation is that the system will eventually converge to the desired state.

Components

Workflow

A workflow is a set of operations that are related and depend on each other. Each workflow is made up of multiple operations and may make use of many different types of modules in various orders. As independent groups of operations, multiple workflows may be run in parallel.

Operation

An operation is a configuration for a module to implement and is a single step in the overall workflow. Each operation carries metadata about the module it uses, its provided inputs, and the dependencies it requires.

Module

A module implements the interface that allows for extensibility within Blackstart. The core set of logic in Blackstart is separate from module operations. This allows a concrete module implementation to only worry about the basic set of operations needed, without worrying about how the module is used.

Each module interfaces with Blackstart by implementing the following:

  1. Validate: A method that validates the inputs to the module. This method is called before the Check and Set methods.
  2. Check: A method that checks the current state of the system to determine if the operation needs to be run. It also must return correct output values if the module has outputs.
  3. Set: A method that sets the desired state of the system. This method is only called if the check method returns false.

Logic

Topological Sorting

The dependencies between operations in a workflow must create a directed acyclic graph (DAG). This DAG is currently used to topologically sort into a linear set of operations which may be executed in an order where an operation is not run until all of its dependencies have completed. Because of this, there is no depth-first execution of operations, or parallel execution of independent branches within the DAG of operations. Additionally, the topological sorting, while guaranteeing dependencies are executed first, does not provide a deterministic ordering of operations executed.

Basic Workflow

Basic Workflow Basic Workflow

Example - Bootstrapping a Database

The following is an example of a workflow that bootstraps a database. The challenge is that tools to create infrastructure such as Terraform will need to connect to the database after creation to assign table-level grants. This is a good implementation of declarative security as code, but it requires the connecting to the database via a public IP, and it has a race condition on the table creation that will cause the infrastructure creation to partially fail.

Architecture

For infrastructure creation, the DevOps process or CICD pipeline runs in a separate network such as a management network or a CICD service.

Architecture Architecture

Terraform Only

When deploying this infrastructure with only Terraform, the apply fails because the application has not been deployed, and the application manages the database schema. It is possible to converge the infrastructure and application releases, allowing Terraform to manage application releases. However, this is not ideal as it creates a tight coupling between infrastructure and application releases, adding to the complexity in larger systems.

Terraform Only Terraform Only

Terraform and Blackstart

When deploying this infrastructure with Terraform and Blackstart, the core infrastructure is created by Terraform, and Blackstart handles creating the database grants from inside the VPC.

Terraform and Blackstart Terraform and Blackstart

Additional Concepts

When Blackstart runs, it processes a set of operations to configure an environment. Each operation is performed by a module that implements the "check then set" interface. Blackstart and its modules are stateless and perform idempotent operations. However, because it is stateless, there are a couple concepts to understand:

  1. Idempotency requires authoritative control. Each module must specify what it manages authoritatively. For example, a database role may be given 4 permissions on a table. The module to do this may exercise authority over the role permissions on that table. If the role already exists, that would mean existing permissions would be removed and replaced with what is configured in the operation for the module.
  2. Deleting an operation may orphan resources or settings. For example, we can assume a database user was created using a module. At a later time, the operation to create that user is removed from the configuration. The user previously created will be orphaned. To fix this, replace the create user operation with an operation to delete the user. The idempotent state of a deleted user is simply a user that should not exist.

Individual operations may have dependencies on other operations. Using Blackstart, each operation requires a unique identifier - other operations use this identifier to specify what they depend on to run. Using this information, Blackstart builds a directed acyclic graph and processes operations in a topological order. Relying on the dependency graph, Blackstart operates using an eventually consistent model. If a dependency is not met, Blackstart will log this information and then try again during the next run. Failures are expected and are okay. For example, an operation may grant access to a database, and it may depend on an operation to create the database. If the database creation operation did not complete successfully because the database instance was missing, the operation to grant access to the database will not run. Once the database instance is up, and the database is created, the operation to grant access to the database will run.

Secure by Default

Blackstart is designed to be secure by default and help automate the setup of secure systems. That said, there is no intent to support insecure configurations or resources. For example, for any database type that supports workload identity or other dynamic authentication, Blackstart does and will not support creating a database user with a static password. Instead, Blackstart only supports workload identity or other secure methods of authentication.

Documentation

Index

Constants

View Source 
const (
	UserAgent     = "blackstart/1.0"
	LoggerKey key = "logger"
	ConfigKey key = "config"
	SchemeKey key = "scheme"
)

Variables

View Source 
var ErrInputDoesNotExist = errors.New("input does not exist")
View Source 
var ErrOperationCycle = errors.New("operation cycle detected")
View Source 
var K8sNamespaceEnv = getConfigEnv("KubeNamespace")
View Source 
var LogOutputEnv = getConfigEnv("LogOutput")

Functions

func GetRegisteredModules 

func GetRegisteredModules() map[string]func() Module

GetRegisteredModules returns a map of all registered modules and their factory functions.

func GetRegisteredPathNames 

func GetRegisteredPathNames() map[string]string

GetRegisteredPathNames returns a map of all registered path names.

func NewLogger 

func NewLogger(config *RuntimeConfig) *slog.Logger

func NewTestingLogger 

func NewTestingLogger() *slog.Logger

func NewTextHandler 

func NewTextHandler(o io.Writer, opts *slog.HandlerOptions) slog.Handler

func RegisterModule 

func RegisterModule(module string, factory func() Module)

RegisterModule is used by modules to register themselves with the global module registry. Each module must provide its ID and a factory function that will be used to create new instances of the module. The module factory function should only create the module instance and not perform any setup or validation. It should also not need to return any error, setup and verification of the module will be done in the setup method.

func RegisterPathName 

func RegisterPathName(path, name string)

RegisterPathName is used by modules to register a friendly name for a path segment.

Types

type Input 

type Input interface {
	IsStatic() bool
	String() string
	Bool() bool
	Number() int64
	Float() float64
	Any() any
	Auto() (any, error)
	DependencyId() string
	OutputKey() string
}

--8<-- [start:Input]

func NewInputFromDep 

func NewInputFromDep(id string, output string) Input

NewInputFromDep creates a new module input from a dependency output. This is used to reference the output of another operation as the input to this operation. These inputs are only available at runtime.

func NewInputFromValue 

func NewInputFromValue(value interface{}) Input

NewInputFromValue creates a new module input from a static value. It detects the type and assigns it to the correct field.

type InputValue 

type InputValue struct {
	// Description is a short description of the value. This is used to provide context about what
	// the value is and how it should be used.
	Description string

	// Type is the type of the value. This is used to provide context about what type of value is
	// expected.
	Type reflect.Type

	// Required indicates whether the value is required. When used as an input, this indicates
	// that the value must be provided for the module to function correctly.
	Required bool

	// Default is an optional default value for the input parameter.
	Default any
}

InputValue is a structure that describes a value used in a module's inputs.

type Module 

type Module interface {
	// Info returns a ModuleInfo structure that provides information about the module. This is used
	// to provide context about the module and its capabilities.
	Info() ModuleInfo

	// Validate is used by the module to validate that the Operation settings and parameters are
	// valid. If the Operation is invalid, then an error should be returned.
	Validate(op Operation) error

	// Check should be a safe, non-destructive method to ensure the expected state exists. If it
	// does not exist, Check must return false. If an alternate error is encountered while checking
	// the state, then an error should also be returned.
	Check(ctx ModuleContext) (bool, error)

	// Set configures the expected state if Check returns false or if tainted.
	Set(ctx ModuleContext) error
}

Module is the interface that all modules must implement. Modules are used to configure resources in various systems, but they all provide the same "check then set" interface. When running a job, blackstart will orchestrate the execution of operations and the modules they use. --8<-- [start:Module]

func NewModule 

func NewModule(op *Operation) (Module, error)

NewModule creates a new module instance based on the provided operation. The operation has the module ID and this is used to create an instance of the Module interface.

type ModuleContext 

type ModuleContext interface {
	context.Context
	Input(key string) (Input, error)
	Output(key string, value interface{}) error
	DoesNotExist() bool
	Tainted() bool
}

--8<-- [start:ModuleContext]

func InputsToContext 

func InputsToContext(ctx context.Context, si map[string]Input, flags ...ModuleContextFlag) (
	mctx ModuleContext,
)

InputsToContext is a helper function that takes a map of inputs and returns module context with the inputs set as if they were passed at runtime. This is available as a helper for module testing.

func OpContext 

func OpContext(ctx context.Context, op *Operation) ModuleContext

OpContext creates a ModuleContext from an Operation. This is available as an exported helper for testing and special cases where a ModuleContext needs to be created directly outside the normal execution flow.

Use caution when using this function, as it will panic if attempting to access a module which has not been registered.

type ModuleContextFlag 

type ModuleContextFlag int
const (
	TaintedFlag ModuleContextFlag = iota
	DoesNotExistFlag
)

type ModuleInfo 

type ModuleInfo struct {
	// Id is the identifier of the module. This is used to identify the module in the
	// configuration and in the logs.
	Id string

	// Name is the name of the module. This is a human-readable identifier.
	Name string

	// Description is a short description of the module. This is used to provide context about
	// what the module does and how it should be used.
	Description string

	// Inputs is a map of input names to their configuration. This is used to provide context
	// about what inputs the module requires and how they should be used.
	Inputs map[string]InputValue

	// Outputs is a map of output names to their descriptions. This is used to provide context
	// about what outputs the module provides and how they should be used.
	Outputs map[string]OutputValue

	// Examples is a map of example titles to their YAML implementations. This is used to provide
	// users with a quick way to understand how to use the module.
	Examples map[string]string
}

ModuleInfo is a static structure that provides information about a module. It is used to provide metadata about the module, such as its name, description, and the inputs it requires and outputs it provides.

type Operation 

type Operation struct {
	// Module is the identifier value of the module that will be used to configure the resource.
	Module string

	// Id is a unique identifier for the operation. This is used to reference the operation in
	// other operations.
	Id string

	// Name is a human-readable name for the operation.
	Name string

	// Description is a human-readable description of the operation. Use this to provide more
	// context about the operation.
	Description string

	// DependsOn is a list of operation IDs that this operation depends on. The operations that
	// this operation depends on will be executed before this operation.
	DependsOn []string

	// Inputs are used to configure the module. The moduleContext are specific to each module and
	// are used to configure the module's behavior.
	Inputs map[string]Input

	// DoesNotExist is a special parameter that can be used to indicate that the resource should
	// not exist. This is useful for resources that are changed from a previous state and now
	// should be deleted if they still exist.
	DoesNotExist bool

	// Tainted is a special parameter that can be used to indicate that the resource is tainted and
	// should be replaced. This is useful for resources that always must be updated so that
	// attributes / output values are known by Blackstart. This should not be configured by users,
	// and should only be used explicitly by modules.
	Tainted bool
}

Operation represents a single operation in a Workflow. Each operation uses a specific module to configure resources. In an imperative Workflow, operations could be a step to perform in the Workflow. In Blackstart's declarative Workflow, the order of the operations is determined at runtime based on the dependencies between operations. --8<-- [start:Operation]

type OutputValue 

type OutputValue struct {
	// Description is a short description of the value. This is used to provide context
	// about what the value is and how it should be used.
	Description string

	// Type is the type of the value. This is used to provide context about what type
	// of value is expected.
	Type reflect.Type
}

OutputValue is a structure that describes a value used in a module's outputs.

type RuntimeConfig 

type RuntimeConfig struct {
	Version       bool   `short:"v" long:"version" description:"Show version information"`
	LogOutput     string `long:"log-output" env:"BLACKSTART_LOG_OUTPUT" description:"Logging output file name" default:""`
	LogFormat     string `long:"log-format" env:"BLACKSTART_LOG_FORMAT" description:"Logging format (json, text)" default:"text"`
	LogLevel      string `long:"log-level" env:"BLACKSTART_LOG_LEVEL" description:"Logging level" default:"info"`
	WorkflowFile  string `short:"f" long:"workflow-file" env:"BLACKSTART_WORKFLOW_FILE" description:"Path to the workflow file" required:"false"`
	KubeNamespace string `` /* 136-byte string literal not displayed */
}

func ReadConfig 

func ReadConfig() (*RuntimeConfig, error)

type Workflow 

type Workflow struct {
	// Name is a simple Name or identifier for the Workflow.
	Name string `yaml:"name"`

	// Description is an optional field to describe the Workflow in greater detail.
	Description string `yaml:"description,omitempty"`

	// Operations is a map of operations that will be executed in the Workflow.
	Operations []Operation `yaml:"operations"`

	// Source is the original source of the workflow definition, if available.
	Source any
}

Workflow represents a series of operations to be executed. Each operation may depend on the outputs of other operations, forming a directed acyclic graph (DAG) of operations. The Workflow will be executed in an order that respects these dependencies. --8<-- [start:Workflow]

func (*Workflow) Run 

func (w *Workflow) Run(ctx context.Context) WorkflowResult

Run will execute the Workflow using the provided context.

type WorkflowResult 

type WorkflowResult struct {
	Phase               string
	Op                  *Operation
	Err                 error
	TotalOperations     int
	CompletedOperations int
}

WorkflowResult represents the result of executing an operation. It contains the operation that was executed and any error that occurred during execution.

Source Files

View all Source files

Directories

Path Synopsis
api
v1alpha1
Package v1alpha1 +groupName=blackstart.pezops.github.io
 Package v1alpha1 +groupName=blackstart.pezops.github.io
cmd
blackstart command
internal
all_modules
module_docs command
modules
google/cloud
google/cloudsql
kubernetes
mock
postgres

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.