ion

Ion

Ion is an enterprise-grade observability client for Go services. It unifies structured logging (Zap) and distributed tracing (OpenTelemetry) into a single, cohesive API designed for high-throughput, long-running infrastructure.

Status: v0.2 Release Candidate
Target: Microservices, Blockchain Nodes, Distributed Systems

Guarantees & Design Invariants

Ion is built on strict operational guarantees. Operators can rely on these invariants in production:

1. No Process Termination: Ion will never call os.Exit, panic, or log.Fatal. Even Critical level logs are strictly informational (mapped to FATAL severity) and guarantee control flow returns to the caller.
2. Thread Safety: All public APIs on Logger and Tracer are safe for concurrent use by multiple goroutines.
3. Non-Blocking Telemetry: Trace export is asynchronous and decoupled from application logic. A slow OTEL collector will never block your business logic (logs are synchronous to properly handle crash reporting, but rely on high-performance buffered writes).
4. Failure Isolation: Telemetry backend failures (e.g., Collector down) are isolated. They may result in data loss (dropped spans) but will never crash the service.

Non-Goals

To maintain focus and stability, Ion explicitly avoids:

• Metrics: Use the Prometheus or OpenTelemetry Metrics SDKs directly.
• Alerting: Ion emits signals; it does not manage thresholds or paging.
• Framework Magic: Ion does not auto-inject into HTTP handlers without explicit middleware usage.

Operational Model

How Ion Works

• Logs: Emitted synchronously to the configured cores (Console/File/Memory). This ensures that if your application crashes immediately after a log statement, the log is persisted (up to OS buffering).
• Traces: Buffered and exported asynchronously. Spans are batched in memory and sent to the configured OTEL, endpoint on a timer or size threshold.
• Correlation: trace_id and span_id are extracted from context.Context at the moment of logging and injected as fields.

When to Use Logs vs Traces

• Logs: Use for state changes, errors, and high-cardinality events (e.g., specific transaction failure reasons). Logs must be reliable and available immediately.
• Traces: Use for latency analysis, causality (who called whom), and request flows. Traces are sampled and statistically significant, but individual traces may be dropped under load.

Installation

go get github.com/JupiterMetaLabs/ion

Requires Go 1.21+.

Quick Start

A minimal, correct example for a production service.

package main

import (
    "context"
    "log"
    "time"

    "github.com/JupiterMetaLabs/ion"
)

func main() {
    ctx := context.Background()

    // 1. Initialize with Service Identity
    // Returns warning slice for non-fatal config issues (e.g. invalid OTEL url)
    app, warnings, err := ion.New(ion.Default().WithService("payment-node"))
    if err != nil {
        log.Fatalf("Fatal: failed to init observability: %v", err)
    }
    for _, w := range warnings {
        log.Printf("Ion Startup Warning: %v", w)
    }

    // 2. Establishing the Lifecycle Contract
    // Ensure logs/traces flush before exit.
    defer func() {
        shutdownCtx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
        defer cancel()
        // Errors here mean data loss, not application failure.
        if err := app.Shutdown(shutdownCtx); err != nil {
            log.Printf("Shutdown data loss: %v", err)
        }
    }()

    // 3. Application Logic
    app.Info(ctx, "node started", ion.String("version", "1.0.0"))
    
    // Simulate work
    doWork(ctx, app)
}

func doWork(ctx context.Context, logger ion.Logger) {
    // Context is mandatory for correlation
    logger.Info(ctx, "processing block", ion.Uint64("height", 100))
}

Configuration Reference

Ion uses a comprehensive configuration struct for behavior control. This maps 1:1 with ion.Config.

Root Configuration (ion.Config)

Console Configuration (ion.ConsoleConfig)

File Configuration (ion.FileConfig)

OTEL Configuration (ion.OTELConfig)

Controls the OpenTelemetry Logs Exporter.

Tracing Configuration (ion.TracingConfig)

Controls the OpenTelemetry Trace Provider.

Detailed Initialization

For full control, initialize the struct directly rather than using builders.

cfg := ion.Config{
    Level:       "info",
    ServiceName: "payment-service",
    Version:     "v1.2.3",
    
    Console: ion.ConsoleConfig{
        Enabled:        true,
        Format:         "json",
        ErrorsToStderr: true,
    },
    
    // File rotation
    File: ion.FileConfig{
        Enabled:    true,
        Path:       "/var/log/payment-service.log",
        MaxSizeMB:  500,
        MaxBackups: 3,
        Compress:   true,
    },
    
    // Remote Telemetry (OTEL)
    OTEL: ion.OTELConfig{
        Enabled:  true,
        Endpoint: "otel-collector.prod:4317",
        Protocol: "grpc",
        // Attributes added to every log
        Attributes: map[string]string{
            "cluster": "us-east-1",
            "env":     "production",
        },
    },
    
    // Distributed Tracing
    Tracing: ion.TracingConfig{
        Enabled: true,
        // Fallback to OTEL.Endpoint is automatic if this is empty
        Sampler: "ratio:0.05", // Sample 5% of traces
    },
}

// Initialize
app, warnings, err := ion.New(cfg)
if err != nil {
    panic(err)
}
// Handle warnings (e.g., invalid sampler string fallback)
for _, w := range warnings {
    log.Println("Ion config warning:", w)
}
defer app.Shutdown(context.Background())

Proper Usage Guide

1. The Logger

Use the Logger for human-readable events, state changes, and errors.

• Always pass context.Context (even if generic).
• Prefer typed fields (ion.String) over generic ion.F.
• Do not log sensitive data (PII/Secrets).

// INFO: Operational state changes
app.Info(ctx, "transaction processed", 
    ion.String("tx_id", "0x123"),
    ion.Duration("latency", 50 * time.Millisecond),
)

// ERROR: Actionable failures. Does not interrupt flow.
if err != nil {
    // Automatically adds "error": err.Error() field
    app.Error(ctx, "database connection failed", err, ion.String("db_host", "primary"))
}

// CRITICAL: Invariant violations (e.g. data corruption).
// Use this for "wake up the on-call" events.
// GUARANTEE: Does NOT call os.Exit(). Safe to use in libraries.
app.Critical(ctx, "memory corruption detected", nil)

2. The Tracer

Use the Tracer for latency measurement and causal chains.

• Start/End: Every Start MUST have a corresponding End().
• Defer: Use defer span.End() immediately after checking err isn't nil (or just immediately if function is simple).
• Attributes: Add attributes to spans only if they are valuable for querying latency/filtering (e.g., "http.status_code"). High cardinality data belongs in Logs, not Spans attributes (usually).

func ProcessOrder(ctx context.Context, orderID string) error {
    // 1. Get Named Tracer
    tracer := app.Tracer("order.processor")
    
    // 2. Start Span
    ctx, span := tracer.Start(ctx, "ProcessOrder")
    // 3. Ensure End
    defer span.End()
    
    // 4. Enrich Span
    span.SetAttributes(attribute.String("order.id", orderID))
    
    // ... work ...
    
    if err := validate(ctx); err != nil {
        // 5. Record Errors in Span
        span.RecordError(err)
        span.SetStatus(codes.Error, "validation failed")
        return err
    }
    
    return nil
}

Common Configurations

Recipes for standard deployment scenarios.

1. Local Development

Goal: readable logs, no external dependencies.

// Pretty print to console, Debug level enabled
cfg := ion.Development()

2. Production Node

Goal: machine-readable JSON, structured errors, persistent file logs.

cfg := ion.Default()
cfg.Level = "info"
cfg.Console.Format = "json"
cfg.File.Enabled = true
cfg.File.Path = "/var/log/ion/service.log"

3. Distributed Cluster

Goal: centralized tracing and high-volume logging.

cfg := ion.Default()
cfg.OTEL.Enabled = true
cfg.OTEL.Endpoint = "otel-collector.infra.svc:4317"
cfg.Tracing.Enabled = true
cfg.Tracing.Sampler = "ratio:0.1" // Sample 10% of traffic

HTTP & gRPC Integration

Ion provides specialized middleware/interceptors to automate context propagation.

HTTP Middleware (middleware/ionhttp)

import "github.com/JupiterMetaLabs/ion/middleware/ionhttp"

mux := http.NewServeMux()
handler := ionhttp.Handler(mux, "payment-api") 
http.ListenAndServe(":8080", handler)

gRPC Interceptors (middleware/iongrpc)

import "github.com/JupiterMetaLabs/ion/middleware/iongrpc"

// Server
s := grpc.NewServer(
    grpc.StatsHandler(iongrpc.ServerHandler()),
)

// Client
conn, err := grpc.Dial(addr, 
    grpc.WithStatsHandler(iongrpc.ClientHandler()),
)

Production Failure Modes

Operators must understand how Ion behaves under stress:

• OTEL Collector Down: The internal exporter will retry with exponential backoff. If buffers fill, new traces will be dropped. Application performance is preserved (failure is isolated).
• Disk Full (File Logging): lumberjack rotation will attempt to write. If the write syscall fails, Zap internal error handling catches it. The application continues, but logs are lost (written to stderr fallback if possible).
• High Load: Tracing uses a batch processor. Under extreme load, if the export rate lags generation, spans are dropped to prevent memory leaks (bounded buffer).

Globals & Dependency Injection

⚠️ WARNING: Global state is strictly for migration and legacy support.

Do not use ion.SetGlobal in new libraries or microservices. Do inject ion.Logger via struct constructors.

// CORRECT: Dependency Injection
type Server struct {
    log ion.Logger
}

// INCORRECT: Hidden dependency
func (s *Server) Handle() {
    ion.Info(...) // Relies on global state
}

If ion.GetGlobal() is called without initialization, it returns a safe no-op logger. It will not panic, but your logs will be silently discarded to protect the runtime.

Best Practices

1. Pass Context Everywhere: context.Background() breaks the trace chain. Only use it in main or background worker roots.
2. Shutdown is Mandatory: Failing to call Shutdown guarantees data loss (buffered traces/logs) on deployment.
3. Structured Keys: Use consistent key names (e.g., user_id, not userID or uid) to make logs queryable.
4. No Dynamic Keys: ion.String(userInput, "value") is a security risk and breaks indexing. Keys must be static constants.

Versioning

• Public API: ion.go, logger.go, config.go. Stable v0.2.
• Internal: internal/*. No stability guarantees.
• Behavior: Log format changes or configuration defaults are considered breaking changes.

License

MIT © 2025 JupiterMeta Labs