serializer

package
v0.9.8 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Mar 7, 2026 License: Apache-2.0 Imports: 26 Imported by: 0

Documentation

Overview

Package serializer provides encoding and decoding of measurement data in multiple formats.

Overview

The serializer package handles conversion between measurement data structures and various output formats including JSON, YAML, and human-readable tables. It supports both encoding (writing data) and decoding (reading data) operations with automatic format detection.

Supported Formats

JSON:

  • Machine-parseable, compact representation
  • Suitable for API responses and programmatic consumption
  • Standard encoding/json package

YAML:

  • Human-readable with preserved structure
  • Suitable for configuration files and version control
  • gopkg.in/yaml.v3 package

Table:

  • Hierarchical text representation
  • Suitable for terminal/console viewing
  • Custom tree-style formatting
  • Read-only (no deserialization support)

Core Types

Format: Enum representing output formats (JSON, YAML, Table)

Serializer: Interface for encoding data to output

type Serializer interface {
    Serialize(ctx context.Context, snapshot any) error
}

Reader: Handles decoding data from input sources

type Reader struct {
    format Format
    input  io.Reader
    closer io.Closer
}

Usage - Encoding

Write to stdout (YAML):

w := serializer.NewStdoutWriter(serializer.FormatYAML)

data := map[string]any{"version": "1.0.0", "status": "ok"}
if err := w.Serialize(context.Background(), data); err != nil {
    log.Fatal(err)
}

Write to file with automatic format detection:

w, err := serializer.NewFileWriterOrStdout(serializer.FormatJSON, "config.json")
if err != nil {
    log.Fatal(err)
}
defer w.Close()

if err := w.Serialize(context.Background(), data); err != nil {
    log.Fatal(err)
}

Write with explicit format:

w := serializer.NewWriter(serializer.FormatTable, output)
defer w.Close()

snapshot := // ... measurement data
if err := w.Serialize(context.Background(), snapshot); err != nil {
    log.Fatal(err)
}

Usage - Decoding

Read from file with automatic format detection:

reader, err := serializer.NewFileReaderAuto("snapshot.yaml")
if err != nil {
    log.Fatal(err)
}
defer reader.Close()

var snapshot snapshotter.Snapshot
if err := reader.Deserialize(&snapshot); err != nil {
    log.Fatal(err)
}

Read from file with explicit format:

reader, err := serializer.NewFileReader(serializer.FormatJSON, "data.json")
if err != nil {
    log.Fatal(err)
}
defer reader.Close()

var data map[string]any
if err := reader.Deserialize(&data); err != nil {
    log.Fatal(err)
}

Read with custom io.Reader:

reader, err := serializer.NewReader(serializer.FormatYAML, strings.NewReader(yamlData))
if err != nil {
    log.Fatal(err)
}

var config Config
if err := reader.Deserialize(&config); err != nil {
    log.Fatal(err)
}

Format Detection

File extension-based detection:

  • .json → JSON
  • .yaml, .yml → YAML
  • .table, .txt → Table
  • Other → JSON (default)

Format detection is automatic when using:

  • NewFileWriterOrStdout(format, path)
  • NewFileReaderAuto(path)

Table Format

The table format provides hierarchical visualization:

Snapshot
├─ version: v1.0.0
├─ measurements:
│  ├─ K8s
│  │  ├─ server
│  │  │  ├─ version: 1.33.5
│  │  │  └─ platform: linux/amd64
│  │  └─ node
│  │     ├─ provider: eks
│  │     └─ kernel: 6.8.0
│  └─ GPU
│     ├─ driver: 570.158.01
│     └─ model: H100

Table format:

  • Does not support deserialization (read-only)
  • Best for human viewing in terminals
  • Preserves structure with tree-style indentation

Resource Management

Always close serializers and readers that manage files:

w, err := serializer.NewFileWriterOrStdout(serializer.FormatJSON, "output.json")
if err != nil {
    return err
}
defer w.Close()  // Required for file resources

Stdout writers don't require closing but Close() is safe to call.

Error Handling

Errors are returned when:

  • Format is unknown or unsupported
  • File cannot be opened or created
  • Data cannot be marshaled/unmarshaled
  • Table format used for deserialization

All errors include context for debugging.

Integration

Used throughout AICR for data I/O:

  • pkg/cli - Command output formatting
  • pkg/snapshotter - Snapshot serialization
  • pkg/api - HTTP response encoding
  • pkg/recipe - Recipe output

Package serializer provides utilities for serializing data to various formats.

The package supports three main output formats:

  • JSON: Machine-readable structured data with proper indentation
  • YAML: Human-readable configuration format
  • Table: Human-readable tabular output with flattened keys

Usage:

writer := serializer.NewWriter(serializer.FormatJSON, os.Stdout)
defer writer.Close() // Important: close to release file handles
if err := writer.Serialize(ctx, data); err != nil {
	log.Fatal(err)
}

For HTTP responses:

serializer.RespondJSON(w, http.StatusOK, data)

The package automatically handles:

  • Proper content-type headers for HTTP responses
  • Buffering to prevent partial responses on errors
  • Flattening nested structures for table format
  • Resource cleanup via Close() method

Index

Examples

Constants

View Source
const (
	// ConfigMapURIScheme is the URI scheme for Kubernetes ConfigMap destinations.
	// Format: cm://namespace/configmap-name
	ConfigMapURIScheme = "cm://"

	// StdoutURI is the special URI indicating output should be written to stdout.
	StdoutURI = "-"
)

URI scheme constants for output destinations

View Source
const (
	HTTPReaderUserAgent = "AICR-Serializer/1.0"
)

Variables

View Source
var (
	HTTPReaderDefaultTimeout               = defaults.HTTPClientTimeout
	HTTPReaderDefaultKeepAlive             = defaults.HTTPKeepAlive
	HTTPReaderDefaultConnectTimeout        = defaults.HTTPConnectTimeout
	HTTPReaderDefaultTLSHandshakeTimeout   = defaults.HTTPTLSHandshakeTimeout
	HTTPReaderDefaultResponseHeaderTimeout = defaults.HTTPResponseHeaderTimeout
	HTTPReaderDefaultIdleConnTimeout       = defaults.HTTPIdleConnTimeout
	HTTPReaderDefaultMaxIdleConns          = 100
	HTTPReaderDefaultMaxIdleConnsPerHost   = 10
	HTTPReaderDefaultMaxConnsPerHost       = 0
)

Functions

func ExecuteTemplateToBytes

func ExecuteTemplateToBytes(ctx context.Context, templatePath string, data any) ([]byte, error)

ExecuteTemplateToBytes executes a template with the given data and returns the result as bytes. The template can be loaded from a local file path or HTTP/HTTPS URL. This is useful for agent mode where we need to retrieve snapshot data and transform it.

func FromFile

func FromFile[T any](path string) (*T, error)

FromFile is a generic convenience function that loads and deserializes a file in one call. The file format is automatically detected from the file extension.

Type Parameter:

  • T: The target type (struct, slice, map, etc.) compatible with JSON/YAML unmarshaling

Parameters:

  • path: File path or HTTP/HTTPS URL

Returns:

  • Pointer to populated instance of type T
  • Error if file cannot be read or deserialized

Resource Management:

  • Automatically handles Reader creation and cleanup (Close is called internally)
  • No need to manually close the reader

Example:

type Config struct { Name string; Port int }
config, err := FromFile[Config]("config.yaml")
if err != nil { panic(err) }
fmt.Println(config.Name) // Use config directly

Note: This is a higher-level API. Use NewFileReader directly if you need more control over the Reader lifecycle or want to reuse it. FromFile reads and deserializes data from a file path, URL, or ConfigMap URI into type T.

Supported input sources:

Format detection:

  • File paths: Determined by extension (.json, .yaml, .yml)
  • URLs: Determined by URL path extension or response Content-Type
  • ConfigMap: Always YAML format (ConfigMaps store data as YAML)

Returns:

  • Pointer to deserialized object of type T
  • Error if file/URL/ConfigMap not found, network error, or deserialization fails

ConfigMap Format:

  • Reads from ConfigMap data field "snapshot.{json|yaml}"
  • Falls back to "snapshot.yaml" if specific format field not found
  • Requires Kubernetes cluster access (kubeconfig)

Example:

snap, err := FromFile[Snapshot]("cm://gpu-operator/aicr-snapshot")

func FromFileWithKubeconfig

func FromFileWithKubeconfig[T any](path, kubeconfig string) (*T, error)

FromFileWithKubeconfig reads and deserializes data from a file path, HTTP URL, or ConfigMap URI with custom kubeconfig.

This is identical to FromFile but allows specifying a custom kubeconfig path for ConfigMap URIs. The kubeconfig parameter is only used when path is a ConfigMap URI (cm://namespace/name).

Parameters:

  • path: File path, HTTP/HTTPS URL, or ConfigMap URI (cm://namespace/name)
  • kubeconfig: Path to kubeconfig file (only used for ConfigMap URIs, empty string uses default discovery)

Example:

snap, err := FromFileWithKubeconfig[Snapshot]("cm://gpu-operator/aicr-snapshot", "/custom/kubeconfig")

func RespondJSON

func RespondJSON(w http.ResponseWriter, statusCode int, data any)

RespondJSON writes a JSON response with the given status code and data. It buffers the JSON encoding before writing headers to prevent partial responses.

func SupportedFormats

func SupportedFormats() []string

SupportedFormats returns a list of all supported output formats for serialization.

func ValidateTemplateFile

func ValidateTemplateFile(path string) error

ValidateTemplateFile checks if the template file exists and is readable. For local files, validates existence and that it's not a directory. For URLs (http:// or https://), skips validation as we can't check without fetching. Returns nil if the file is valid, or an error describing the problem.

func WriteToFile

func WriteToFile(path string, data []byte) (err error)

WriteToFile writes data to a file at the specified path. This is a convenience function for writing raw byte data to a file. The file is created with 0644 permissions.

Types

type Closer

type Closer interface {
	Close() error
}

Closer is an optional interface that Serializers can implement if they need to release resources (e.g., close file handles).

type ConfigMapWriter

type ConfigMapWriter struct {
	// contains filtered or unexported fields
}

ConfigMapWriter writes serialized data to a Kubernetes ConfigMap. The ConfigMap is created if it doesn't exist, or updated if it does.

func NewConfigMapWriter

func NewConfigMapWriter(namespace, name string, format Format) *ConfigMapWriter

NewConfigMapWriter creates a new ConfigMapWriter that writes to the specified namespace and ConfigMap name in the given format.

func (*ConfigMapWriter) Close

func (w *ConfigMapWriter) Close() error

Close is a no-op for ConfigMapWriter as there are no resources to release. This method exists to satisfy the Closer interface.

func (*ConfigMapWriter) Serialize

func (w *ConfigMapWriter) Serialize(ctx context.Context, snapshot any) error

Serialize writes the snapshot data to a ConfigMap. The ConfigMap will have: - data.snapshot.{yaml|json}: The serialized snapshot content - data.format: The format used (yaml or json) - data.timestamp: ISO 8601 timestamp of when the snapshot was created

type Format

type Format string

Format represents the output format type

const (
	// FormatJSON outputs data in JSON format
	FormatJSON Format = "json"
	// FormatYAML outputs data in YAML format
	FormatYAML Format = "yaml"
	// FormatTable outputs data in table format
	FormatTable Format = "table"
)

func FormatFromPath

func FormatFromPath(filePath string) Format

FormatFromPath determines the serialization format based on file extension. Supported extensions:

  • .json → FormatJSON
  • .yaml, .yml → FormatYAML
  • .table, .txt → FormatTable

Returns FormatJSON as default for unknown extensions. Extension matching is case-insensitive.

func (Format) IsUnknown

func (f Format) IsUnknown() bool

type HTTPReader

type HTTPReader struct {
	UserAgent             string
	TotalTimeout          *time.Duration
	ConnectTimeout        *time.Duration
	TLSHandshakeTimeout   *time.Duration
	ResponseHeaderTimeout *time.Duration
	IdleConnTimeout       *time.Duration
	MaxIdleConns          *int
	MaxIdleConnsPerHost   *int
	MaxConnsPerHost       *int
	InsecureSkipVerify    *bool
	Client                *http.Client
	// contains filtered or unexported fields
}

HTTPReader handles fetching data over HTTP with configurable options.

func NewHTTPReader

func NewHTTPReader(options ...HTTPReaderOption) *HTTPReader

NewHTTPReader creates a new HTTPReader with the specified options.

func (*HTTPReader) Download

func (r *HTTPReader) Download(url, filePath string) error

Download reads data from the specified URL and writes it to the given file path. The request is bounded by the HTTPReader's TotalTimeout. Use DownloadWithContext for caller-controlled cancellation.

func (*HTTPReader) DownloadWithContext

func (r *HTTPReader) DownloadWithContext(ctx context.Context, url, filePath string) error

DownloadWithContext reads data from the specified URL and writes it to the given file path. The request is bound to the provided context for cancellation and deadlines.

func (*HTTPReader) Read

func (r *HTTPReader) Read(url string) ([]byte, error)

Read fetches data from the specified URL and returns it as a byte slice. The request is bounded by the HTTPReader's TotalTimeout. Use ReadWithContext for caller-controlled cancellation.

func (*HTTPReader) ReadWithContext

func (r *HTTPReader) ReadWithContext(ctx context.Context, url string) ([]byte, error)

ReadWithContext fetches data from the specified URL and returns it as a byte slice. The request is bound to the provided context for cancellation and deadlines. Callers must provide a non-nil context.

type HTTPReaderOption

type HTTPReaderOption func(*HTTPReader)

HTTPReaderOption defines a configuration option for HTTPReader.

func WithClient

func WithClient(client *http.Client) HTTPReaderOption

func WithConnectTimeout

func WithConnectTimeout(timeout time.Duration) HTTPReaderOption

func WithIdleConnTimeout

func WithIdleConnTimeout(timeout time.Duration) HTTPReaderOption

func WithInsecureSkipVerify

func WithInsecureSkipVerify(skip bool) HTTPReaderOption

func WithMaxConnsPerHost

func WithMaxConnsPerHost(max int) HTTPReaderOption

func WithMaxIdleConns

func WithMaxIdleConns(max int) HTTPReaderOption

func WithMaxIdleConnsPerHost

func WithMaxIdleConnsPerHost(max int) HTTPReaderOption

func WithResponseHeaderTimeout

func WithResponseHeaderTimeout(timeout time.Duration) HTTPReaderOption

func WithTLSHandshakeTimeout

func WithTLSHandshakeTimeout(timeout time.Duration) HTTPReaderOption

func WithTotalTimeout

func WithTotalTimeout(timeout time.Duration) HTTPReaderOption

func WithUserAgent

func WithUserAgent(userAgent string) HTTPReaderOption

type Reader

type Reader struct {
	// contains filtered or unexported fields
}

Reader handles deserialization of structured data from various formats (JSON, YAML). It supports reading from any io.Reader source including files, strings, and HTTP responses.

Resource Management:

  • Close must be called to release resources when using NewFileReader or NewFileReaderAuto
  • Safe to call Close multiple times (idempotent)
  • Close is a no-op for readers created with NewReader from non-closeable sources

Supported formats: JSON, YAML (Table format is write-only)

Example

Example usage

// Create a reader from a string
jsonData := `{"name":"example","value":42}`
reader, err := NewReader(FormatJSON, strings.NewReader(jsonData))
if err != nil {
	panic(err)
}

// Deserialize into a struct
var config testConfig
if err := reader.Deserialize(&config); err != nil {
	panic(err)
}

// Use the data
_ = config.Name  // "example"
_ = config.Value // 42

func NewFileReader

func NewFileReader(format Format, filePath string) (*Reader, error)

NewFileReader creates a new Reader that reads from a file path or URL.

Parameters:

  • format: The serialization format (FormatJSON or FormatYAML)
  • filePath: Local file path or HTTP/HTTPS URL

URL Support:

  • Supports http:// and https:// URLs
  • Downloads remote files to temporary directory
  • Temporary files are managed by Reader.Close()

Returns error if:

  • format is unknown or unsupported
  • format is FormatTable (table format does not support deserialization)
  • file cannot be opened or URL cannot be downloaded

Resource Management:

  • Close must be called to release the file handle
  • For remote URLs, Close also removes the temporary downloaded file

Example:

reader, err := NewFileReader(FormatJSON, "/path/to/config.json")
if err != nil { panic(err) }
defer reader.Close()

func NewFileReaderAuto

func NewFileReaderAuto(filePath string) (*Reader, error)

NewFileReaderAuto creates a new Reader with automatic format detection. The format is determined from the file extension using FormatFromPath.

This is a convenience wrapper around NewFileReader that auto-detects the format. See NewFileReader for full documentation on supported paths, URLs, and resource management.

Example:

reader, err := NewFileReaderAuto("config.yaml") // Auto-detects YAML format
if err != nil { panic(err) }
defer reader.Close()
var config MyConfig
err = reader.Deserialize(&config)
Example
// Create a temporary file for this example
tmpfile, _ := os.CreateTemp("", "example*.json")
defer os.Remove(tmpfile.Name())
tmpfile.WriteString(`{"name":"example","value":42}`)
tmpfile.Close()

// Auto-detect format from file extension
reader, err := NewFileReaderAuto(tmpfile.Name())
if err != nil {
	panic(err)
}
defer reader.Close()

var config testConfig
if err := reader.Deserialize(&config); err != nil {
	panic(err)
}

// Use the data
_ = config.Name // "example"

func NewReader

func NewReader(format Format, input io.Reader) (*Reader, error)

NewReader creates a new Reader for deserializing data from an io.Reader source.

Parameters:

  • format: The serialization format (FormatJSON or FormatYAML)
  • input: Any io.Reader implementation (e.g., strings.Reader, bytes.Buffer, *os.File)

Returns error if:

  • format is unknown or unsupported
  • format is FormatTable (table format does not support deserialization)

Resource Management:

  • If input implements io.Closer, it will be stored and closed by Reader.Close()
  • Otherwise, Close() is a no-op

Example:

reader, err := NewReader(FormatJSON, strings.NewReader(`{"key":"value"}`})
if err != nil { panic(err) }
var data map[string]string
err = reader.Deserialize(&data)

func (*Reader) Close

func (r *Reader) Close() error

Close releases any resources held by the Reader.

Behavior:

  • If Reader was created from a file (NewFileReader), closes the file handle
  • If Reader was created from a non-closeable source (NewReader), this is a no-op
  • Sets internal closer to nil after first close to prevent double-close errors
  • Safe to call on nil Reader

Idempotency:

  • Safe to call multiple times (subsequent calls are no-ops)
  • Returns nil on subsequent calls after successful first close

Best Practice:

  • Always defer Close() immediately after creating a Reader from files
  • Example: defer reader.Close()

func (*Reader) Deserialize

func (r *Reader) Deserialize(v any) error

Deserialize reads data from the input source and unmarshals it into v.

Parameters:

  • v: A pointer to the target structure or variable

Type Requirements:

  • v must be a pointer (e.g., &myStruct, &mySlice, &myMap)
  • The underlying type must be compatible with the format (JSON or YAML)

Returns error if:

  • Reader is nil
  • Input source is nil
  • Data cannot be decoded (invalid format, type mismatch)
  • Format is FormatTable (not supported for deserialization)

Example:

var config struct { Name string; Value int }
err := reader.Deserialize(&config)

type Serializer

type Serializer interface {
	Serialize(ctx context.Context, snapshot any) error
}

Serializer is an interface for serializing snapshot data. Implementations of this interface can serialize data to various formats such as JSON, YAML, or plain text.

The context parameter is used for cancellation and timeouts, particularly important for implementations that perform I/O operations (e.g., ConfigMap writes).

func NewFileWriterOrStdout

func NewFileWriterOrStdout(format Format, path string) (Serializer, error)

NewFileWriterOrStdout creates a new Writer that outputs to the specified file path in the given format. If path is empty or "-", writes to stdout. Returns an error if the path is invalid or the file cannot be created. Remember to call Close() on the returned Writer to ensure the file is properly closed.

Supports ConfigMap URIs in the format cm://namespace/name for Kubernetes ConfigMap output.

type TemplateWriter

type TemplateWriter struct {
	// contains filtered or unexported fields
}

TemplateWriter executes Go templates with snapshot data. It supports sprig template functions for rich formatting capabilities.

func NewTemplateFileWriter

func NewTemplateFileWriter(templatePath, outputPath string) (*TemplateWriter, error)

NewTemplateFileWriter creates a writer that outputs to a file. If outputPath is empty or "-", writes to stdout.

func NewTemplateWriter

func NewTemplateWriter(templatePath string, output io.Writer) *TemplateWriter

NewTemplateWriter creates a writer that outputs to the given io.Writer. The templatePath must point to a valid Go template file.

func (*TemplateWriter) Close

func (t *TemplateWriter) Close() error

Close releases any resources associated with the TemplateWriter. It should be called when done writing, especially for file-based writers. It's safe to call Close multiple times or on stdout-based writers.

func (*TemplateWriter) Serialize

func (t *TemplateWriter) Serialize(ctx context.Context, data any) error

Serialize executes the template with the provided data and writes to output. The data is passed directly to the template, which can access all exported fields. The template can be loaded from a local file path or HTTP/HTTPS URL.

type Writer

type Writer struct {
	// contains filtered or unexported fields
}

Writer handles serialization of configuration data to various formats. Close must be called to release file handles when using NewFileWriterOrStdout.

func NewStdoutWriter

func NewStdoutWriter(format Format) *Writer

NewStdoutWriter creates a new Writer that outputs to stdout in the specified format.

func NewWriter

func NewWriter(format Format, output io.Writer) *Writer

NewWriter creates a new Writer with the specified format and output destination. If output is nil, os.Stdout will be used. If format is unknown, defaults to JSON format.

func (*Writer) Close

func (w *Writer) Close() error

Close releases any resources associated with the Writer. It should be called when done writing, especially for file-based writers. It's safe to call Close multiple times or on stdout-based writers.

func (*Writer) Serialize

func (w *Writer) Serialize(ctx context.Context, config any) error

Serialize outputs the given configuration data in the configured format. Serialize writes the configuration data in the specified format. Context is provided for consistency with the Serializer interface, but is not actively used for file/stdout writes (which are fast and blocking).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL