synth

package

v0.6.1 Latest Latest Go to latest Published: Nov 3, 2026 License: Apache-2.0 Imports: 11 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/dshakes/halos

Links

Open Source Insights

Documentation ¶

Overview ¶

Package synth emits synthetic OTLP/HTTP (JSON) telemetry shaped like the real harnesses, for control vs treatment cohorts with a known injected effect. It exists to prove the collector -> ClickHouse -> analyze path (test/e2e/obs_test.go) and is a test helper, not a production dependency.

Shapes follow the vendors' docs: Claude Code sends claude_code.* delta sums with user.id/session.id/model/type/decision attributes; Codex and Gemini send the gen_ai.client.token.usage histogram. halo.harness is left unset for Claude Code and Gemini so the collector's transform must stamp it; Codex sets it.

Log events follow the vendors' documented names and attributes: Claude Code claude_code.api_request / api_error / tool_result (unprefixed event.name, full name in the body), Gemini gemini_cli.api_response / api_error / tool_call and Codex codex.api_request / tool_result (full name in event.name). Each session makes CallsPerSession API calls and tool calls with the cohort's latency and error-rate effects. Events draw from their own RNG stream so adding them did not change the metric data of a given seed.

Index ¶

Constants
func Generate(cfg Config, cohorts []Cohort) (metrics, logs []kv)
func Send(ctx context.Context, endpoint string, metrics, logs []kv) error
type Cohort
type Config

Constants ¶

View Source

const CallsPerSession = 10

CallsPerSession is the number of API calls (and of tool calls) per session in the log events.

Variables ¶

This section is empty.

Functions ¶

func Generate ¶

func Generate(cfg Config, cohorts []Cohort) (metrics, logs []kv)

Generate returns one OTLP metrics request body per user-session for every cohort and harness, plus one logs body per session per harness.

func Send ¶

func Send(ctx context.Context, endpoint string, metrics, logs []kv) error

Send POSTs every body to the collector's OTLP/HTTP endpoint (e.g. http://127.0.0.1:14318).

Types ¶

type Cohort ¶

type Cohort struct {
	Experiment, Variant, Ring, Release string
	CostMult                           float64 // multiplies per-session cost (1.3 = +30%)
	AcceptRate                         float64 // P(edit accepted)
	LatencyMult                        float64 // multiplies API call latency; 0 = 1
	APIErrorRate                       float64 // P(API call fails)
	ToolErrorRate                      float64 // P(tool call fails)
}

Cohort is one experiment arm.

type Config ¶

type Config struct {
	Seed            int64
	Now             time.Time
	UsersPerArm     int
	SessionsPerUser int // sessions per user
	BaseCostUSD     float64
}

Config controls the volume and determinism of a run.

Source Files ¶

View all Source files

synth.go

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL