ibk

package module
v0.0.4 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jan 31, 2025 License: Apache-2.0 Imports: 17 Imported by: 0

README

Image Builder Packer plugin

HashiCorp Packer plugin for image-builder-cli and bootc-image-builder. Builds are happening on a remote linux machine over SSH.

Preparing the environment

All building is done remotely over SSH connection, you need to create an instance or VM with a user dedicated to image building and sudo permission to start podman (or docker) without password or root account.

adduser -m builder

Either setup a password

passwd builder

or preferably deploy a public SSH key (execute from machine with packer)

ssh-copy-id builder@host

Make sure the container runtime can be executed without password.

cat <<EOF >/etc/sudoers.d/builder
builder ALL=(ALL) NOPASSWD: /usr/bin/podman, /usr/bin/docker
EOF

Finally, install podman or docker:

dnf -y install podman

Cross-architecture building is currently not supported so make sure the builder host architecture is correct.

Install packer

Install packer on your machine not on the builder instance/VM, for example on Fedora:

sudo dnf install -y dnf-plugins-core
sudo dnf config-manager addrepo --from-repofile=https://rpm.releases.hashicorp.com/fedora/hashicorp.repo
sudo dnf -y install packer

On MacOS:

brew tap hashicorp/tap
brew install hashicorp/tap/packer

Building using image-builder-cli

Create a packer template named template.pkr.hcl:

packer {
  required_plugins {
    image-builder = {
      source = "github.com/lzap/image-builder"
      version = ">= 0.0.2"
    }
  }
}

source "image-builder" "example" {
    build_host {
        hostname = "buildhost.example.com"
        username = "builder"
    }

    distro = "centos-9"

    blueprint = <<BLUEPRINT
[[customizations.user]]
name = "user"
password = "changeme"
groups = ["wheel"]
BLUEPRINT

    image_type = "minimal-raw"
}

build {
    sources = [ "source.image-builder.example" ]
}

See osbuild blueprint reference for more info about blueprint format.

Perform the build via:

  packer init template.pkr.hcl
  packer build template.pkr.hcl

The image builder plugin will print last several lines from the image builder output as an artifact. To see more detailed output:

  PACKER_LOG=1 packer build template.pkr.hcl
Config mapping

For more info: https://github.com/osbuild/image-builder-cli

  • build_host.hostname - IP or hostname with optional SSH port (required)
  • build_host.username - either root or username with sudo permissions (required)
  • build_host.password - SSH password when SSH keys are not available
  • distro - maps to --distro
  • blueprint - maps to --blueprint
  • image_type - maps to image type argument

If there is an option missing, file an issue for us.

Building using bootc-image-builder

Create a packer template named template.pkr.hcl:

packer {
  required_plugins {
    image-builder = {
      source = "github.com/lzap/image-builder"
      version = ">= 0.0.2"
    }
  }
}

source "image-builder" "example" {
    build_host {
        hostname = "buildhost.example.com"
        username = "builder"
    }

    container_repository = "quay.io/centos-bootc/centos-bootc:stream9"

    blueprint = <<BLUEPRINT
[[customizations.user]]
name = "user"
password = "changeme"
groups = ["wheel"]
BLUEPRINT

    image_type = "raw"
}

build {
    sources = [ "source.image-builder.example" ]
}

See osbuild blueprint reference for more info about blueprint format.

Perform the build via:

  packer init template.pkr.hcl
  packer build template.pkr.hcl

The image builder plugin will print last several lines from the image builder output as an artifact. To see more detailed output:

  PACKER_LOG=1 packer build template.pkr.hcl
Config mapping

For more info: https://github.com/osbuild/bootc-image-builder

  • build_host.hostname - IP or hostname with optional SSH port (required)
  • build_host.username - either root or username with sudo permissions (required)
  • build_host.password - SSH password when SSH keys are not available
  • container_repository - maps to container repository argument
  • blueprint - maps to --blueprint
  • image_type - maps to --type
  • rootfs - maps to --rootfs
  • aws_upload.ami_name - maps to AMI cloud uploader configuration
  • aws_upload.s3_bucket - maps to AMI cloud uploader configuration
  • aws_upload.region - maps to AMI cloud uploader configuration
  • aws_upload.access_key_id - maps to AMI cloud uploader configuration
  • aws_upload.secret_access_key - maps to AMI cloud uploader configuration

If there is an option missing, file an issue for us.

Dry run

If you want to perform, for any reason, a dry run where the main build command is echoed to the console rather than executed, just set IMAGE_BUILDER_DRY_RUN=1 environment variable when executing packer. Good for demos or testing the integration.

Building without Packer

To test this library directly without packer, do:

go run github.com/lzap/packer-plugin-image-builder/cmd/ibpacker/ -help

Use options to initiate a build:

Usage of ibpacker:
  -arch string
        architecture (default "x86_64")
  -blueprint string
        path to blueprint file
  -distro string
        distribution name (fedora, centos, rhel, ...) (default "fedora")
  -dry-run
        dry run
  -hostname string
        SSH hostname or IP with optional port (e.g. example.com:22)
  -type string
        image type (minimal-raw, qcow2, ...) (default "minimal-raw")
  -username string
        SSH username

For example:

git clone github.com/lzap/packer-plugin-image-builder
go run ./cmd/ibpacker/ \
    -hostname example.com \
    -username builder \
    -distro centos-9 \
    -type minimal-raw \
    -blueprint ./cmd/ibpacker/blueprint_example.toml

Testing

To run unit and integration test against mock SSH server running on localhost:

make test

Do not invoke tests directly via go test command because some tests require packer binary to be present in the ./build directory and will be skipped if it is not present.

Keys in internal/sshtest/keys.go are just dummy (test only) keys, you may receive false positives from security scanners about leaked keys when cloning the repo.

LICENSE

Apache Version 2.0

Documentation

Index

Constants

This section is empty.

Variables

View Source
var ErrCommand = errors.New("command error")
View Source
var ErrConfigure = errors.New("error while configuring")

ErrConfigure is returned when an error occurs during configuration.

View Source
var ErrContainerPull = errors.New("error while pulling container")
View Source
var ErrCopy = errors.New("copy error")
View Source
var ErrHostnameEmpty = errors.New("hostname is empty")
View Source
var ErrKnownHosts = errors.New("known hosts error")
View Source
var ErrNoContainerRuntime = errors.New("no container runtime found")
View Source
var ErrPush = errors.New("error while pushing")

ErrPush is returned when an error occurs during pushing.

View Source
var ErrSSHDial = errors.New("ssh dial error")
View Source
var ErrSSHNewSession = errors.New("ssh new session error")

Functions

func ApplyCommand

func ApplyCommand(ctx context.Context, c Command, t Transport) error

ApplyCommand configures, pushes, and executes a command.

func ApplyCommandPrint

func ApplyCommandPrint(ctx context.Context, c Command, t Transport, log PrintFunc) error

ApplyCommand configures, pushes, and executes a command. It logs the command to the provided PrintFunc.

func RandomString

func RandomString(n int) string

RandomString generates a random string of length n that is not cryptographically safe

func ReadPrivateKey

func ReadPrivateKey(path string) (*bytes.Buffer, error)

ReadPrivateKey reads a private key from the specified path.

func ReadPrivateKeys

func ReadPrivateKeys() ([]*bytes.Buffer, error)

ReadPrivateKeys reads the default private keys from the known paths.

func Wait

func Wait(ctx context.Context, f func() error) error

Wait waits for the context to be done or for the function to return

Types

type AWSUploadConfig

type AWSUploadConfig struct {
	// AWSAccessKeyID credential. Maps to the AWS_ACCESS_KEY_ID environment variable.
	AWSAccessKeyID string

	// AWSSecretAccessKey credential. Maps to the AWS_SECRET_ACCESS_KEY environment variable.
	AWSSecretAccessKey string

	// AMIName is the name of the AMI to register.
	AMIName string

	// S3Bucket is the name of a temporary S3 bucket to upload the image to.
	S3Bucket string

	// S3Region is the region of the S3 bucket and the resulting AMI.
	Region string
}

AWSUploadCommand uploads the image to an S3 bucket and registers it as an AMI.

type Closer

type Closer interface {
	Close(ctx context.Context) error
}

type CombinedWriter

type CombinedWriter struct {
	// contains filtered or unexported fields
}

CombinedWriter is a writer that writes to a buffer and is safe for concurrent use.

func (*CombinedWriter) FirstLine

func (w *CombinedWriter) FirstLine() string

FirstLine returns the first line of the output trimmed of leading and trailing whitespace

func (*CombinedWriter) Reset

func (w *CombinedWriter) Reset()

func (*CombinedWriter) String

func (w *CombinedWriter) String() string

func (*CombinedWriter) Write

func (w *CombinedWriter) Write(p []byte) (int, error)

type Command

type Command interface {
	// Configure configures the command and prepares the environment.
	Configure(ctx context.Context, exec Executor) error

	// Push uploads one or more files to the remote host. Files are stored as temporary
	// files and are automatically cleaned up after the command is executed.
	Push(ctx context.Context, pusher Pusher) error

	// Build returns the command to execute. This command is executed in the remote environment.
	// The command should be a single string that can be passed to a shell. The command should
	// not include the shell invocation and must be protected against shell injection.
	Build() string
}

Command is an interface for a command that can be executed.

type CommonArgs

type CommonArgs struct {
	// DryRun is a flag to print the command instead of executing it. Blueprint is still pushed
	// to the remote machine and then cleaned up.
	DryRun bool

	// Interactive passes the --interactive flag to the container runtime.
	Interactive bool

	// TTY passes the --tty flag to the container runtime.
	TTY bool

	// TeeLog is a flag to tee the output of the command to a file named build.log for later use.
	TeeLog bool
}

type ContainerBootCommand

type ContainerBootCommand struct {
	// Source OCI (podman, docker) image repository URL.
	Repository string

	// Type is the image type: ami, anaconda-iso, gce, iso, qcow2, raw, vhd, vmdk (default [qcow2]
	// Maps to the argument named --type.
	Type string

	// Arch is the architecture, must be set to the architecture of the remote machine
	// since cross-compilation is not supported yet.
	// Maps to the argument named --target-arch.
	Arch string

	// RootFS is the root filesystem to use. Supported values: ext4, xfs, btrfs.
	// Maps to the argument named --rootfs.
	RootFS string

	// Blueprint is the full contents of a blueprint.
	Blueprint string

	// OutputDir is the directory where the output image is saved. When unset, a new directory will
	// be created in the remote machine home directory. The caller must cleanup the directory.
	OutputDir string

	// Common arguments for all container commands.
	Common CommonArgs

	// AWSUploadConfig is the configuration for uploading the image to AWS. Must be set when the
	// Type is set to "ami".
	AWSUploadConfig *AWSUploadConfig
	// contains filtered or unexported fields
}

ContainerBootCommand builds a bootc-image-builder command line via podman or docker which builds an image from a blueprint. The blueprint is pushed to the remote machine via SSH and the command is executed there. The output image is saved to the specified directory which must be created and cleanuped up beforehand.

For more information see https://github.com/osbuild/bootc-image-builder

func (*ContainerBootCommand) Build

func (c *ContainerBootCommand) Build() string

func (*ContainerBootCommand) Configure

func (c *ContainerBootCommand) Configure(ctx context.Context, t Executor) error

func (*ContainerBootCommand) Push

func (c *ContainerBootCommand) Push(ctx context.Context, pusher Pusher) error

type ContainerCliCommand

type ContainerCliCommand struct {
	// Distro is the distribution name
	Distro string

	// Type is the image type
	Type string

	// Arch is the architecture, must be set to the architecture of the remote machine
	// since cross-compilation is not supported yet.
	Arch string

	// Blueprint is the full contents of a blueprint.
	Blueprint string

	// OutputDir is the directory where the output image is saved. When unset, a new directory will
	// be created in the remote machine home directory. The caller must cleanup the directory.
	OutputDir string

	// Common arguments for all container commands.
	Common CommonArgs
	// contains filtered or unexported fields
}

ContainerCliCommand builds an image-builder-cli command line via podman or docker which builds an image from a blueprint. The blueprint is pushed to the remote machine via SSH and the command is executed there. The output image is saved to the specified directory which must be created and cleanuped up beforehand.

For more information see https://github.com/osbuild/image-builder-cli

func (*ContainerCliCommand) Build

func (c *ContainerCliCommand) Build() string

func (*ContainerCliCommand) Configure

func (c *ContainerCliCommand) Configure(ctx context.Context, t Executor) error

func (*ContainerCliCommand) Push

func (c *ContainerCliCommand) Push(ctx context.Context, pusher Pusher) error

type ExecuteOpt

type ExecuteOpt func(*ssh.Session)

ExecuteOpt is a function that configures the SSH session for a command execution

func WithCombinedWriter

func WithCombinedWriter(w *CombinedWriter) ExecuteOpt

WithStdinWriter configures the SSH session to use the specified writer for standard input.

func WithInputOutput

func WithInputOutput(stdin io.Reader, stdout, stderr io.Writer) ExecuteOpt

WithInputOutput configures the SSH session to use the specified reader and writers for standard input, output, and error.

type Executor

type Executor interface {
	Execute(ctx context.Context, cmd Command, opts ...ExecuteOpt) error
}

type PrintFunc

type PrintFunc func(string)

type Pusher

type Pusher interface {
	Push(ctx context.Context, contents, extension string) (string, error)
}

type SSHTransport

type SSHTransport struct {
	// contains filtered or unexported fields
}

SSHTransport is a struct that represents an SSH connection to a remote machine.

func NewSSHTransport

func NewSSHTransport(cfg SSHTransportConfig) (*SSHTransport, error)

NewSSHTransport creates a new SSHTransport with the given configuration. It immediatelly establishes a connection to the remote machine. Use Close to close the connection.

func (*SSHTransport) Close

func (t *SSHTransport) Close(ctx context.Context) error

Close closes the SSH connection. Additionally, it deletes the temporary files created during the session.

func (*SSHTransport) Execute

func (t *SSHTransport) Execute(ctx context.Context, cmd Command, opts ...ExecuteOpt) error

Execute performs a command remotely via SSH session with standard input, output, and error configured as specified in the SSHTransportConfig. The command is executed in the remote machine. An optional arguments can be provided to override Stdout and Stderr config.

func (*SSHTransport) Push

func (t *SSHTransport) Push(ctx context.Context, contents, extension string) (string, error)

Push copies the contents of a reader to a temporary file on the remote machine. Returns the path of the temporary file. The file(s) will be deleted when the SSH connection is closed.

type SSHTransportConfig

type SSHTransportConfig struct {
	// Host is the hostname or IP address of the remote machine including optional port (e.g. "example.com:22").
	Host string

	// Username is the username to use for authentication.
	Username string

	// Password is the password to use for authentication. It is optional if private keys are provided.
	Password string

	// PrivateKeys is a list of private keys to use for authentication. If not provided, the default private keys will be used.
	PrivateKeys []*bytes.Buffer

	// KnownHosts is the path to the known hosts file. If not provided, the default known hosts file will be used.
	KnownHosts string

	// Timeout is the maximum amount of time a dial will wait for a connect to complete. The default is 10 seconds.
	Timeout time.Duration

	// Stdin is the standard input for the SSH session. The default is os.Stdin.
	Stdin io.Reader

	// Stdout is the standard output for the SSH session. The default is os.Stdout.
	Stdout io.Writer

	// Stderr is the standard error for the SSH session. The default is os.Stderr.
	Stderr io.Writer
}

SSHTransportConfig is a configuration struct for creating a new SSHTransport.

type StringCommand

type StringCommand string

StringCommand is a command that is just a string. Useful for testing purposes. The caller is responsible for escaping the command properly.

func (StringCommand) Build

func (c StringCommand) Build() string

func (StringCommand) Configure

func (c StringCommand) Configure(ctx context.Context, exec Executor) error

func (StringCommand) Push

func (c StringCommand) Push(ctx context.Context, pusher Pusher) error

type Transport

type Transport interface {
	Pusher
	Executor
	Closer
}

Directories

Path Synopsis
cmd
ibpacker command
plugin command
internal

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL