Documentation
¶
Overview ¶
Package serializer provides encoding and decoding of measurement data in multiple formats.
Overview ¶
The serializer package handles conversion between measurement data structures and various output formats including JSON, YAML, and human-readable tables. It supports both encoding (writing data) and decoding (reading data) operations with automatic format detection.
Supported Formats ¶
JSON:
- Machine-parseable, compact representation
- Suitable for API responses and programmatic consumption
- Standard encoding/json package
YAML:
- Human-readable with preserved structure
- Suitable for configuration files and version control
- gopkg.in/yaml.v3 package
Table:
- Hierarchical text representation
- Suitable for terminal/console viewing
- Custom tree-style formatting
- Read-only (no deserialization support)
Core Types ¶
Format: Enum representing output formats (JSON, YAML, Table)
Serializer: Interface for encoding data to output
type Serializer interface {
Serialize(ctx context.Context, snapshot any) error
}
Reader: Handles decoding data from input sources
type Reader struct {
format Format
input io.Reader
closer io.Closer
}
Usage - Encoding ¶
Write to stdout (YAML):
w := serializer.NewStdoutWriter(serializer.FormatYAML)
data := map[string]any{"version": "1.0.0", "status": "ok"}
if err := w.Serialize(context.Background(), data); err != nil {
log.Fatal(err)
}
Write to file with automatic format detection:
w, err := serializer.NewFileWriterOrStdout(serializer.FormatJSON, "config.json")
if err != nil {
log.Fatal(err)
}
defer w.Close()
if err := w.Serialize(context.Background(), data); err != nil {
log.Fatal(err)
}
Write with explicit format:
w := serializer.NewWriter(serializer.FormatTable, output)
defer w.Close()
snapshot := // ... measurement data
if err := w.Serialize(context.Background(), snapshot); err != nil {
log.Fatal(err)
}
Usage - Decoding ¶
Read from file with automatic format detection:
reader, err := serializer.NewFileReaderAuto("snapshot.yaml")
if err != nil {
log.Fatal(err)
}
defer reader.Close()
var snapshot snapshotter.Snapshot
if err := reader.Deserialize(&snapshot); err != nil {
log.Fatal(err)
}
Read from file with explicit format:
reader, err := serializer.NewFileReader(serializer.FormatJSON, "data.json")
if err != nil {
log.Fatal(err)
}
defer reader.Close()
var data map[string]any
if err := reader.Deserialize(&data); err != nil {
log.Fatal(err)
}
Read with custom io.Reader:
reader, err := serializer.NewReader(serializer.FormatYAML, strings.NewReader(yamlData))
if err != nil {
log.Fatal(err)
}
var config Config
if err := reader.Deserialize(&config); err != nil {
log.Fatal(err)
}
Format Detection ¶
File extension-based detection:
- .json → JSON
- .yaml, .yml → YAML
- .table, .txt → Table
- Other → JSON (default)
Format detection is automatic when using:
- NewFileWriterOrStdout(format, path)
- NewFileReaderAuto(path)
Table Format ¶
The table format provides hierarchical visualization:
Snapshot ├─ version: v1.0.0 ├─ measurements: │ ├─ K8s │ │ ├─ server │ │ │ ├─ version: 1.33.5 │ │ │ └─ platform: linux/amd64 │ │ └─ node │ │ ├─ provider: eks │ │ └─ kernel: 6.8.0 │ └─ GPU │ ├─ driver: 570.158.01 │ └─ model: H100
Table format:
- Does not support deserialization (read-only)
- Best for human viewing in terminals
- Preserves structure with tree-style indentation
Resource Management ¶
Always close serializers and readers that manage files:
w, err := serializer.NewFileWriterOrStdout(serializer.FormatJSON, "output.json")
if err != nil {
return err
}
defer w.Close() // Required for file resources
Stdout writers don't require closing but Close() is safe to call.
Error Handling ¶
Errors are returned when:
- Format is unknown or unsupported
- File cannot be opened or created
- Data cannot be marshaled/unmarshaled
- Table format used for deserialization
All errors include context for debugging.
Integration ¶
Used throughout AICR for data I/O:
- pkg/cli - Command output formatting
- pkg/snapshotter - Snapshot serialization
- pkg/api - HTTP response encoding
- pkg/recipe - Recipe output
Package serializer provides utilities for serializing data to various formats.
The package supports three main output formats:
- JSON: Machine-readable structured data with proper indentation
- YAML: Human-readable configuration format
- Table: Human-readable tabular output with flattened keys
Usage:
writer := serializer.NewWriter(serializer.FormatJSON, os.Stdout)
defer writer.Close() // Important: close to release file handles
if err := writer.Serialize(ctx, data); err != nil {
log.Fatal(err)
}
For HTTP responses:
serializer.RespondJSON(w, http.StatusOK, data)
The package automatically handles:
- Proper content-type headers for HTTP responses
- Buffering to prevent partial responses on errors
- Flattening nested structures for table format
- Resource cleanup via Close() method
Index ¶
- Constants
- Variables
- func ExecuteTemplateToBytes(ctx context.Context, templatePath string, data any) ([]byte, error)
- func FromFile[T any](path string) (*T, error)
- func FromFileWithKubeconfig[T any](path, kubeconfig string) (*T, error)
- func RespondJSON(w http.ResponseWriter, statusCode int, data any)
- func SupportedFormats() []string
- func ValidateTemplateFile(path string) error
- func WriteToFile(path string, data []byte) (err error)
- type Closer
- type ConfigMapWriter
- type Format
- type HTTPReader
- type HTTPReaderOption
- func WithClient(client *http.Client) HTTPReaderOption
- func WithConnectTimeout(timeout time.Duration) HTTPReaderOption
- func WithIdleConnTimeout(timeout time.Duration) HTTPReaderOption
- func WithInsecureSkipVerify(skip bool) HTTPReaderOption
- func WithMaxConnsPerHost(max int) HTTPReaderOption
- func WithMaxIdleConns(max int) HTTPReaderOption
- func WithMaxIdleConnsPerHost(max int) HTTPReaderOption
- func WithResponseHeaderTimeout(timeout time.Duration) HTTPReaderOption
- func WithTLSHandshakeTimeout(timeout time.Duration) HTTPReaderOption
- func WithTotalTimeout(timeout time.Duration) HTTPReaderOption
- func WithUserAgent(userAgent string) HTTPReaderOption
- type Reader
- type Serializer
- type TemplateWriter
- type Writer
Examples ¶
Constants ¶
const ( // ConfigMapURIScheme is the URI scheme for Kubernetes ConfigMap destinations. // Format: cm://namespace/configmap-name ConfigMapURIScheme = "cm://" // StdoutURI is the special URI indicating output should be written to stdout. StdoutURI = "-" )
URI scheme constants for output destinations
const (
HTTPReaderUserAgent = "AICR-Serializer/1.0"
)
Variables ¶
var ( HTTPReaderDefaultTimeout = defaults.HTTPClientTimeout HTTPReaderDefaultKeepAlive = defaults.HTTPKeepAlive HTTPReaderDefaultConnectTimeout = defaults.HTTPConnectTimeout HTTPReaderDefaultTLSHandshakeTimeout = defaults.HTTPTLSHandshakeTimeout HTTPReaderDefaultResponseHeaderTimeout = defaults.HTTPResponseHeaderTimeout HTTPReaderDefaultIdleConnTimeout = defaults.HTTPIdleConnTimeout HTTPReaderDefaultMaxIdleConns = 100 HTTPReaderDefaultMaxIdleConnsPerHost = 10 HTTPReaderDefaultMaxConnsPerHost = 0 )
Functions ¶
func ExecuteTemplateToBytes ¶
ExecuteTemplateToBytes executes a template with the given data and returns the result as bytes. The template can be loaded from a local file path or HTTP/HTTPS URL. This is useful for agent mode where we need to retrieve snapshot data and transform it.
func FromFile ¶
FromFile is a generic convenience function that loads and deserializes a file in one call. The file format is automatically detected from the file extension.
Type Parameter:
- T: The target type (struct, slice, map, etc.) compatible with JSON/YAML unmarshaling
Parameters:
- path: File path or HTTP/HTTPS URL
Returns:
- Pointer to populated instance of type T
- Error if file cannot be read or deserialized
Resource Management:
- Automatically handles Reader creation and cleanup (Close is called internally)
- No need to manually close the reader
Example:
type Config struct { Name string; Port int }
config, err := FromFile[Config]("config.yaml")
if err != nil { panic(err) }
fmt.Println(config.Name) // Use config directly
Note: This is a higher-level API. Use NewFileReader directly if you need more control over the Reader lifecycle or want to reuse it. FromFile reads and deserializes data from a file path, URL, or ConfigMap URI into type T.
Supported input sources:
- Local file paths: /path/to/file.json, ./config.yaml
- HTTP URLs: http://example.com/data.json, https://api.example.com/config.yaml
- ConfigMap URIs: cm://namespace/configmap-name
Format detection:
- File paths: Determined by extension (.json, .yaml, .yml)
- URLs: Determined by URL path extension or response Content-Type
- ConfigMap: Always YAML format (ConfigMaps store data as YAML)
Returns:
- Pointer to deserialized object of type T
- Error if file/URL/ConfigMap not found, network error, or deserialization fails
ConfigMap Format:
- Reads from ConfigMap data field "snapshot.{json|yaml}"
- Falls back to "snapshot.yaml" if specific format field not found
- Requires Kubernetes cluster access (kubeconfig)
Example:
snap, err := FromFile[Snapshot]("cm://gpu-operator/aicr-snapshot")
func FromFileWithKubeconfig ¶
FromFileWithKubeconfig reads and deserializes data from a file path, HTTP URL, or ConfigMap URI with custom kubeconfig.
This is identical to FromFile but allows specifying a custom kubeconfig path for ConfigMap URIs. The kubeconfig parameter is only used when path is a ConfigMap URI (cm://namespace/name).
Parameters:
- path: File path, HTTP/HTTPS URL, or ConfigMap URI (cm://namespace/name)
- kubeconfig: Path to kubeconfig file (only used for ConfigMap URIs, empty string uses default discovery)
Example:
snap, err := FromFileWithKubeconfig[Snapshot]("cm://gpu-operator/aicr-snapshot", "/custom/kubeconfig")
func RespondJSON ¶
func RespondJSON(w http.ResponseWriter, statusCode int, data any)
RespondJSON writes a JSON response with the given status code and data. It buffers the JSON encoding before writing headers to prevent partial responses.
func SupportedFormats ¶
func SupportedFormats() []string
SupportedFormats returns a list of all supported output formats for serialization.
func ValidateTemplateFile ¶
ValidateTemplateFile checks if the template file exists and is readable. For local files, validates existence and that it's not a directory. For URLs (http:// or https://), skips validation as we can't check without fetching. Returns nil if the file is valid, or an error describing the problem.
func WriteToFile ¶
WriteToFile writes data to a file at the specified path. This is a convenience function for writing raw byte data to a file. The file is created with 0644 permissions.
Types ¶
type Closer ¶
type Closer interface {
Close() error
}
Closer is an optional interface that Serializers can implement if they need to release resources (e.g., close file handles).
type ConfigMapWriter ¶
type ConfigMapWriter struct {
// contains filtered or unexported fields
}
ConfigMapWriter writes serialized data to a Kubernetes ConfigMap. The ConfigMap is created if it doesn't exist, or updated if it does.
func NewConfigMapWriter ¶
func NewConfigMapWriter(namespace, name string, format Format) *ConfigMapWriter
NewConfigMapWriter creates a new ConfigMapWriter that writes to the specified namespace and ConfigMap name in the given format.
func (*ConfigMapWriter) Close ¶
func (w *ConfigMapWriter) Close() error
Close is a no-op for ConfigMapWriter as there are no resources to release. This method exists to satisfy the Closer interface.
func (*ConfigMapWriter) Serialize ¶
func (w *ConfigMapWriter) Serialize(ctx context.Context, snapshot any) error
Serialize writes the snapshot data to a ConfigMap. The ConfigMap will have: - data.snapshot.{yaml|json}: The serialized snapshot content - data.format: The format used (yaml or json) - data.timestamp: ISO 8601 timestamp of when the snapshot was created
type Format ¶
type Format string
Format represents the output format type
func FormatFromPath ¶
FormatFromPath determines the serialization format based on file extension. Supported extensions:
- .json → FormatJSON
- .yaml, .yml → FormatYAML
- .table, .txt → FormatTable
Returns FormatJSON as default for unknown extensions. Extension matching is case-insensitive.
type HTTPReader ¶
type HTTPReader struct {
UserAgent string
TotalTimeout *time.Duration
ConnectTimeout *time.Duration
TLSHandshakeTimeout *time.Duration
ResponseHeaderTimeout *time.Duration
IdleConnTimeout *time.Duration
MaxIdleConns *int
MaxIdleConnsPerHost *int
MaxConnsPerHost *int
InsecureSkipVerify *bool
Client *http.Client
// contains filtered or unexported fields
}
HTTPReader handles fetching data over HTTP with configurable options.
func NewHTTPReader ¶
func NewHTTPReader(options ...HTTPReaderOption) *HTTPReader
NewHTTPReader creates a new HTTPReader with the specified options.
func (*HTTPReader) Download ¶
func (r *HTTPReader) Download(url, filePath string) error
Download reads data from the specified URL and writes it to the given file path. The request is bounded by the HTTPReader's TotalTimeout. Use DownloadWithContext for caller-controlled cancellation.
func (*HTTPReader) DownloadWithContext ¶
func (r *HTTPReader) DownloadWithContext(ctx context.Context, url, filePath string) error
DownloadWithContext reads data from the specified URL and writes it to the given file path. The request is bound to the provided context for cancellation and deadlines.
func (*HTTPReader) Read ¶
func (r *HTTPReader) Read(url string) ([]byte, error)
Read fetches data from the specified URL and returns it as a byte slice. The request is bounded by the HTTPReader's TotalTimeout. Use ReadWithContext for caller-controlled cancellation.
func (*HTTPReader) ReadWithContext ¶
ReadWithContext fetches data from the specified URL and returns it as a byte slice. The request is bound to the provided context for cancellation and deadlines. Callers must provide a non-nil context.
type HTTPReaderOption ¶
type HTTPReaderOption func(*HTTPReader)
HTTPReaderOption defines a configuration option for HTTPReader.
func WithClient ¶
func WithClient(client *http.Client) HTTPReaderOption
func WithConnectTimeout ¶
func WithConnectTimeout(timeout time.Duration) HTTPReaderOption
func WithIdleConnTimeout ¶
func WithIdleConnTimeout(timeout time.Duration) HTTPReaderOption
func WithInsecureSkipVerify ¶
func WithInsecureSkipVerify(skip bool) HTTPReaderOption
func WithMaxConnsPerHost ¶
func WithMaxConnsPerHost(max int) HTTPReaderOption
func WithMaxIdleConns ¶
func WithMaxIdleConns(max int) HTTPReaderOption
func WithMaxIdleConnsPerHost ¶
func WithMaxIdleConnsPerHost(max int) HTTPReaderOption
func WithResponseHeaderTimeout ¶
func WithResponseHeaderTimeout(timeout time.Duration) HTTPReaderOption
func WithTLSHandshakeTimeout ¶
func WithTLSHandshakeTimeout(timeout time.Duration) HTTPReaderOption
func WithTotalTimeout ¶
func WithTotalTimeout(timeout time.Duration) HTTPReaderOption
func WithUserAgent ¶
func WithUserAgent(userAgent string) HTTPReaderOption
type Reader ¶
type Reader struct {
// contains filtered or unexported fields
}
Reader handles deserialization of structured data from various formats (JSON, YAML). It supports reading from any io.Reader source including files, strings, and HTTP responses.
Resource Management:
- Close must be called to release resources when using NewFileReader or NewFileReaderAuto
- Safe to call Close multiple times (idempotent)
- Close is a no-op for readers created with NewReader from non-closeable sources
Supported formats: JSON, YAML (Table format is write-only)
Example ¶
Example usage
// Create a reader from a string
jsonData := `{"name":"example","value":42}`
reader, err := NewReader(FormatJSON, strings.NewReader(jsonData))
if err != nil {
panic(err)
}
// Deserialize into a struct
var config testConfig
if err := reader.Deserialize(&config); err != nil {
panic(err)
}
// Use the data
_ = config.Name // "example"
_ = config.Value // 42
func NewFileReader ¶
NewFileReader creates a new Reader that reads from a file path or URL.
Parameters:
- format: The serialization format (FormatJSON or FormatYAML)
- filePath: Local file path or HTTP/HTTPS URL
URL Support:
- Supports http:// and https:// URLs
- Downloads remote files to temporary directory
- Temporary files are managed by Reader.Close()
Returns error if:
- format is unknown or unsupported
- format is FormatTable (table format does not support deserialization)
- file cannot be opened or URL cannot be downloaded
Resource Management:
- Close must be called to release the file handle
- For remote URLs, Close also removes the temporary downloaded file
Example:
reader, err := NewFileReader(FormatJSON, "/path/to/config.json")
if err != nil { panic(err) }
defer reader.Close()
func NewFileReaderAuto ¶
NewFileReaderAuto creates a new Reader with automatic format detection. The format is determined from the file extension using FormatFromPath.
This is a convenience wrapper around NewFileReader that auto-detects the format. See NewFileReader for full documentation on supported paths, URLs, and resource management.
Example:
reader, err := NewFileReaderAuto("config.yaml") // Auto-detects YAML format
if err != nil { panic(err) }
defer reader.Close()
var config MyConfig
err = reader.Deserialize(&config)
Example ¶
// Create a temporary file for this example
tmpfile, _ := os.CreateTemp("", "example*.json")
defer os.Remove(tmpfile.Name())
tmpfile.WriteString(`{"name":"example","value":42}`)
tmpfile.Close()
// Auto-detect format from file extension
reader, err := NewFileReaderAuto(tmpfile.Name())
if err != nil {
panic(err)
}
defer reader.Close()
var config testConfig
if err := reader.Deserialize(&config); err != nil {
panic(err)
}
// Use the data
_ = config.Name // "example"
func NewReader ¶
NewReader creates a new Reader for deserializing data from an io.Reader source.
Parameters:
- format: The serialization format (FormatJSON or FormatYAML)
- input: Any io.Reader implementation (e.g., strings.Reader, bytes.Buffer, *os.File)
Returns error if:
- format is unknown or unsupported
- format is FormatTable (table format does not support deserialization)
Resource Management:
- If input implements io.Closer, it will be stored and closed by Reader.Close()
- Otherwise, Close() is a no-op
Example:
reader, err := NewReader(FormatJSON, strings.NewReader(`{"key":"value"}`})
if err != nil { panic(err) }
var data map[string]string
err = reader.Deserialize(&data)
func (*Reader) Close ¶
Close releases any resources held by the Reader.
Behavior:
- If Reader was created from a file (NewFileReader), closes the file handle
- If Reader was created from a non-closeable source (NewReader), this is a no-op
- Sets internal closer to nil after first close to prevent double-close errors
- Safe to call on nil Reader
Idempotency:
- Safe to call multiple times (subsequent calls are no-ops)
- Returns nil on subsequent calls after successful first close
Best Practice:
- Always defer Close() immediately after creating a Reader from files
- Example: defer reader.Close()
func (*Reader) Deserialize ¶
Deserialize reads data from the input source and unmarshals it into v.
Parameters:
- v: A pointer to the target structure or variable
Type Requirements:
- v must be a pointer (e.g., &myStruct, &mySlice, &myMap)
- The underlying type must be compatible with the format (JSON or YAML)
Returns error if:
- Reader is nil
- Input source is nil
- Data cannot be decoded (invalid format, type mismatch)
- Format is FormatTable (not supported for deserialization)
Example:
var config struct { Name string; Value int }
err := reader.Deserialize(&config)
type Serializer ¶
Serializer is an interface for serializing snapshot data. Implementations of this interface can serialize data to various formats such as JSON, YAML, or plain text.
The context parameter is used for cancellation and timeouts, particularly important for implementations that perform I/O operations (e.g., ConfigMap writes).
func NewFileWriterOrStdout ¶
func NewFileWriterOrStdout(format Format, path string) (Serializer, error)
NewFileWriterOrStdout creates a new Writer that outputs to the specified file path in the given format. If path is empty or "-", writes to stdout. Returns an error if the path is invalid or the file cannot be created. Remember to call Close() on the returned Writer to ensure the file is properly closed.
Supports ConfigMap URIs in the format cm://namespace/name for Kubernetes ConfigMap output.
type TemplateWriter ¶
type TemplateWriter struct {
// contains filtered or unexported fields
}
TemplateWriter executes Go templates with snapshot data. It supports sprig template functions for rich formatting capabilities.
func NewTemplateFileWriter ¶
func NewTemplateFileWriter(templatePath, outputPath string) (*TemplateWriter, error)
NewTemplateFileWriter creates a writer that outputs to a file. If outputPath is empty or "-", writes to stdout.
func NewTemplateWriter ¶
func NewTemplateWriter(templatePath string, output io.Writer) *TemplateWriter
NewTemplateWriter creates a writer that outputs to the given io.Writer. The templatePath must point to a valid Go template file.
func (*TemplateWriter) Close ¶
func (t *TemplateWriter) Close() error
Close releases any resources associated with the TemplateWriter. It should be called when done writing, especially for file-based writers. It's safe to call Close multiple times or on stdout-based writers.
func (*TemplateWriter) Serialize ¶
func (t *TemplateWriter) Serialize(ctx context.Context, data any) error
Serialize executes the template with the provided data and writes to output. The data is passed directly to the template, which can access all exported fields. The template can be loaded from a local file path or HTTP/HTTPS URL.
type Writer ¶
type Writer struct {
// contains filtered or unexported fields
}
Writer handles serialization of configuration data to various formats. Close must be called to release file handles when using NewFileWriterOrStdout.
func NewStdoutWriter ¶
NewStdoutWriter creates a new Writer that outputs to stdout in the specified format.
func NewWriter ¶
NewWriter creates a new Writer with the specified format and output destination. If output is nil, os.Stdout will be used. If format is unknown, defaults to JSON format.
func (*Writer) Close ¶
Close releases any resources associated with the Writer. It should be called when done writing, especially for file-based writers. It's safe to call Close multiple times or on stdout-based writers.
func (*Writer) Serialize ¶
Serialize outputs the given configuration data in the configured format. Serialize writes the configuration data in the specified format. Context is provided for consistency with the Serializer interface, but is not actively used for file/stdout writes (which are fast and blocking).