Documentation
¶
Overview ¶
Package annotations provides out-of-band annotations for session recordings. Annotations allow layering evaluations, feedback, and metadata on recorded sessions without modifying the authoritative event record.
Index ¶
- type Annotation
- type AnnotationType
- type AnnotationValue
- func NewAssertionValue(passed bool, message string) AnnotationValue
- func NewCommentValue(text string) AnnotationValue
- func NewFlagValue(flag bool) AnnotationValue
- func NewLabelValue(label string) AnnotationValue
- func NewLabelsValue(labels ...string) AnnotationValue
- func NewMetricValue(value float64, unit string) AnnotationValue
- func NewScoreValue(score float64) AnnotationValue
- type FileStore
- func (s *FileStore) Add(ctx context.Context, ann *Annotation) error
- func (s *FileStore) Close() error
- func (s *FileStore) Delete(ctx context.Context, id string) error
- func (s *FileStore) Get(ctx context.Context, id string) (*Annotation, error)
- func (s *FileStore) Query(ctx context.Context, filter *Filter) ([]*Annotation, error)
- func (s *FileStore) Update(ctx context.Context, previousID string, ann *Annotation) error
- type Filter
- type Store
- type Target
- type TargetType
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Annotation ¶
type Annotation struct {
// ID is a unique identifier for this annotation.
ID string `json:"id"`
// Type identifies the kind of annotation.
Type AnnotationType `json:"type"`
// SessionID is the session this annotation belongs to.
SessionID string `json:"session_id"`
// Target specifies what this annotation targets.
Target Target `json:"target"`
// Key is the annotation key (e.g., "quality", "category", "safety").
Key string `json:"key"`
// Value holds the annotation value (type depends on annotation type).
Value AnnotationValue `json:"value"`
// Metadata contains additional structured data.
Metadata map[string]interface{} `json:"metadata,omitempty"`
// CreatedAt is when this annotation was created.
CreatedAt time.Time `json:"created_at"`
// CreatedBy identifies who created this annotation.
CreatedBy string `json:"created_by,omitempty"`
// Version is the annotation version (for corrections/updates).
Version int `json:"version"`
// PreviousID references the previous version if this is an update.
PreviousID string `json:"previous_id,omitempty"`
}
Annotation represents an out-of-band annotation on a session or event.
type AnnotationType ¶
type AnnotationType string
AnnotationType identifies the kind of annotation.
const ( // TypeScore represents a numeric evaluation score. TypeScore AnnotationType = "score" // TypeLabel represents a categorical label. TypeLabel AnnotationType = "label" // TypeComment represents a textual comment or note. TypeComment AnnotationType = "comment" // TypeFlag represents a binary flag (e.g., safety, policy). TypeFlag AnnotationType = "flag" // TypeMetric represents a performance or quality metric. TypeMetric AnnotationType = "metric" // TypeAssertion represents an assertion result (pass/fail). TypeAssertion AnnotationType = "assertion" // TypeGroundTruth represents ground truth labels for training. TypeGroundTruth AnnotationType = "ground_truth" )
Annotation types.
type AnnotationValue ¶
type AnnotationValue struct {
// Score is the numeric value (for TypeScore, TypeMetric).
Score *float64 `json:"score,omitempty"`
// Label is the categorical value (for TypeLabel, TypeGroundTruth).
Label string `json:"label,omitempty"`
// Labels is a list of categorical values (for multi-label scenarios).
Labels []string `json:"labels,omitempty"`
// Text is the textual value (for TypeComment).
Text string `json:"text,omitempty"`
// Flag is the boolean value (for TypeFlag).
Flag *bool `json:"flag,omitempty"`
// Passed indicates assertion result (for TypeAssertion).
Passed *bool `json:"passed,omitempty"`
// Message is an optional message (for TypeAssertion, TypeComment).
Message string `json:"message,omitempty"`
// Unit is the unit of measurement (for TypeMetric).
Unit string `json:"unit,omitempty"`
}
AnnotationValue holds the value of an annotation. The actual type depends on the annotation type.
func NewAssertionValue ¶
func NewAssertionValue(passed bool, message string) AnnotationValue
NewAssertionValue creates an assertion annotation value.
func NewCommentValue ¶
func NewCommentValue(text string) AnnotationValue
NewCommentValue creates a comment annotation value.
func NewFlagValue ¶
func NewFlagValue(flag bool) AnnotationValue
NewFlagValue creates a flag annotation value.
func NewLabelValue ¶
func NewLabelValue(label string) AnnotationValue
NewLabelValue creates a label annotation value.
func NewLabelsValue ¶
func NewLabelsValue(labels ...string) AnnotationValue
NewLabelsValue creates a multi-label annotation value.
func NewMetricValue ¶
func NewMetricValue(value float64, unit string) AnnotationValue
NewMetricValue creates a metric annotation value with optional unit.
func NewScoreValue ¶
func NewScoreValue(score float64) AnnotationValue
NewScoreValue creates a score annotation value.
type FileStore ¶
type FileStore struct {
// contains filtered or unexported fields
}
FileStore implements Store using JSON Lines files. Annotations for each session are stored in a separate file.
func NewFileStore ¶
NewFileStore creates a file-based annotation store.
func (*FileStore) Add ¶
func (s *FileStore) Add(ctx context.Context, ann *Annotation) error
Add creates a new annotation.
type Filter ¶
type Filter struct {
// SessionID filters by session.
SessionID string
// Types filters by annotation type.
Types []AnnotationType
// Keys filters by annotation key.
Keys []string
// TargetTypes filters by target type.
TargetTypes []TargetType
// EventID filters by target event ID.
EventID string
// TurnIndex filters by target turn index.
TurnIndex *int
// CreatedBy filters by creator.
CreatedBy string
// Since filters by creation time (inclusive).
Since time.Time
// Until filters by creation time (exclusive).
Until time.Time
// IncludeDeleted includes deleted annotations.
IncludeDeleted bool
// LatestVersionOnly returns only the latest version of each annotation.
LatestVersionOnly bool
// Limit limits the number of results.
Limit int
}
Filter specifies criteria for querying annotations.
type Store ¶
type Store interface {
// Add creates a new annotation.
Add(ctx context.Context, ann *Annotation) error
// Update creates a new version of an existing annotation.
// The new annotation will reference the previous version.
Update(ctx context.Context, previousID string, ann *Annotation) error
// Get retrieves an annotation by ID.
Get(ctx context.Context, id string) (*Annotation, error)
// Query returns annotations matching the filter.
Query(ctx context.Context, filter *Filter) ([]*Annotation, error)
// Delete removes an annotation by ID.
// Note: This is a soft delete - the annotation is marked as deleted but preserved.
Delete(ctx context.Context, id string) error
// Close releases resources held by the store.
Close() error
}
Store persists annotations separately from the event stream.
type Target ¶
type Target struct {
// Type identifies the target type.
Type TargetType `json:"type"`
// EventID is the target event ID (for TargetEvent).
EventID string `json:"event_id,omitempty"`
// EventSequence is the target event sequence number (alternative to EventID).
EventSequence int64 `json:"event_sequence,omitempty"`
// TurnIndex is the target turn index (for TargetTurn).
TurnIndex int `json:"turn_index,omitempty"`
// MessageIndex is the target message index (for TargetMessage).
MessageIndex int `json:"message_index,omitempty"`
// StartTime is the start of the time range (for TargetTimeRange).
StartTime time.Time `json:"start_time,omitempty"`
// EndTime is the end of the time range (for TargetTimeRange).
EndTime time.Time `json:"end_time,omitempty"`
}
Target specifies what an annotation targets.
func AtEventSequence ¶
AtEventSequence creates a target for an event by sequence number.
func InTimeRange ¶
InTimeRange creates a target for a time range.
type TargetType ¶
type TargetType string
TargetType identifies what the annotation targets.
const ( // TargetSession targets the entire session. TargetSession TargetType = "session" // TargetEvent targets a specific event. TargetEvent TargetType = "event" // TargetTimeRange targets a time range within the session. TargetTimeRange TargetType = "time_range" // TargetTurn targets a specific conversation turn. TargetTurn TargetType = "turn" // TargetMessage targets a specific message. TargetMessage TargetType = "message" )
Target types.