Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func ExportPath ¶
ExportPath returns the filesystem path for an exported evaluation JSONL file. Exports are stored under ~/.aimux/exports/.
func WriteExport ¶
func WriteExport(path string, turns []ExportTurn, sessionMeta *ExportSessionMeta) error
WriteExport writes all enriched turns as JSONL to the given path. The parent directory is created if it does not exist. If sessionMeta is non-nil, it is written as the first line (type: "session_meta"). Writes are performed atomically via a temp file and rename.
Types ¶
type Annotation ¶
type Annotation struct {
Turn int `json:"turn"`
Label string `json:"label"`
Note string `json:"note,omitempty"`
Timestamp time.Time `json:"timestamp"`
}
Annotation represents a human evaluation label applied to a specific conversation turn. Labels classify turns as "good", "bad", "wasteful", or "error" with an optional free-text note.
type ExportAction ¶
type ExportAction struct {
Tool string `json:"tool"`
Input string `json:"input"`
Success bool `json:"success"`
Error string `json:"error,omitempty"`
}
ExportAction describes a single tool invocation within a conversation turn.
type ExportSessionMeta ¶ added in v0.6.0
type ExportSessionMeta struct {
Type string `json:"type"` // always "session_meta"
SessionID string `json:"session_id"`
Annotation string `json:"annotation,omitempty"` // achieved/partial/failed/abandoned
FailureModes []string `json:"failure_modes,omitempty"` // failure-mode tags
Note string `json:"note,omitempty"` // free-text rationale
Title string `json:"title,omitempty"` // LLM-generated title
}
ExportSessionMeta holds session-level evaluation metadata written as the first line of a JSONL export file (type: "session_meta").
type ExportTurn ¶
type ExportTurn struct {
Turn int `json:"turn"`
Timestamp string `json:"timestamp"`
Input string `json:"input"`
Output string `json:"output"`
Actions []ExportAction `json:"actions"`
TokensIn int64 `json:"tokens_in"`
TokensOut int64 `json:"tokens_out"`
CostUSD float64 `json:"cost_usd"`
DurationMs int64 `json:"duration_ms"`
Label string `json:"label,omitempty"`
Note string `json:"note,omitempty"`
}
ExportTurn is the full enriched representation of a conversation turn, combining raw trace data with evaluation annotations. Used for serializing evaluated sessions to JSONL for downstream analysis.
type Store ¶
type Store struct {
// contains filtered or unexported fields
}
Store manages annotation persistence for a single agent session. Annotations are stored as JSONL files under ~/.aimux/evaluations/.
func NewStore ¶
NewStore creates a Store for the given session ID. The backing directory (~/.aimux/evaluations/) is created lazily on the first write.
func (*Store) GetForTurn ¶
func (s *Store) GetForTurn(turn int) *Annotation
GetForTurn returns the latest annotation for the given turn number, or nil if no annotation exists for that turn.
func (*Store) Load ¶
func (s *Store) Load() ([]Annotation, error)
Load reads all annotations for this session from the JSONL file. Returns an empty slice (not an error) if the file does not exist.
func (*Store) Remove ¶
Remove deletes all annotations for the given turn by rewriting the JSONL file without that turn's entries. The rewrite is performed atomically: data is written to a temporary file first, then renamed over the original.
func (*Store) Save ¶
func (s *Store) Save(a Annotation) error
Save appends an annotation as a single JSONL line. The directory is created on the first call. Writes are performed atomically by appending to the file with O_APPEND to avoid partial writes on concurrent access.