Documentation
¶
Overview ¶
Package locomo implements the LOCOMO benchmark runner for Cortex.
LOCOMO evaluates memory systems on 1,986 questions across 10 long-term conversations with 5 category types (1-5).
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
Types ¶
type Config ¶
type Config struct {
DataPath string
Limit int // Max questions (0 = all)
JudgeCfg *common.JudgeConfig
GraphBoost bool
EmbeddingCfg *embedding.Config // If set, enables vector search
}
Config controls the benchmark run.
type Conversation ¶
type Conversation struct {
SampleID string `json:"sample_id"`
Conversation ConversationData `json:"conversation"`
QA []QA `json:"qa"`
Observation json.RawMessage `json:"observation"` // Complex nested structure, parsed manually
}
Conversation represents a LOCOMO conversation sample.
type ConversationData ¶
type ConversationData struct {
SpeakerA string `json:"speaker_a"`
SpeakerB string `json:"speaker_b"`
Sessions map[string][]Turn // Parsed from session_N keys
Dates map[string]string // Parsed from session_N_date_time keys
}
ConversationData holds the dialogue sessions and metadata.
func (*ConversationData) UnmarshalJSON ¶
func (cd *ConversationData) UnmarshalJSON(data []byte) error
UnmarshalJSON custom-parses the conversation dict with dynamic session keys.
type QA ¶
type QA struct {
Question string `json:"question"`
Answer json.RawMessage `json:"answer"` // Can be string or number
Evidence []string `json:"evidence"`
Category int `json:"category"`
}
QA represents a question-answer pair.
func (QA) AnswerString ¶
AnswerString returns the answer as a string regardless of JSON type.
Click to show internal directories.
Click to hide internal directories.