Documentation
¶
Overview ¶
Package meeting lets an agent attend a meeting as a note taker: it announces itself, transcribes the meeting live, answers when addressed by name, and summarizes the meeting with decisions and action items when it ends.
The Attendant works on any omnimeet-core attendee.Session that delivers audio, such as one from the dial-in joiner:
sess, err := dialin.NewJoiner(gw).Join(ctx, target)
if err != nil {
return err
}
record, err := meeting.New(meeting.Config{
Name: "OmniAgent",
Transcriber: deepgram,
Speaker: elevenlabs,
Responder: meeting.LLMResponder{Client: llm, Model: "claude-sonnet-5"},
Summarizer: meeting.LLMSummarizer{Client: llm, Model: "claude-sonnet-5"},
}).Attend(ctx, sess)
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ErrNoAudio = errors.New("meeting session has no audio to transcribe")
ErrNoAudio is returned when the session does not deliver meeting audio.
Functions ¶
func DefaultAnnouncement ¶
DefaultAnnouncement returns the disclosure spoken when the agent joins: who it is, whom it attends for, and that it is transcribing.
Types ¶
type ActionItem ¶
type ActionItem struct {
// Task is what needs to be done.
Task string `json:"task"`
// Owner is who took it on, when the transcript says so.
Owner string `json:"owner,omitempty"`
// Due is the deadline as stated in the meeting, when there is one.
Due string `json:"due,omitempty"`
}
ActionItem is a task from the meeting.
type Attendant ¶
type Attendant struct {
// contains filtered or unexported fields
}
Attendant attends meetings as a note taker.
type ChatCompleter ¶
type ChatCompleter interface {
CreateChatCompletion(ctx context.Context, req *provider.ChatCompletionRequest) (*provider.ChatCompletionResponse, error)
}
ChatCompleter is the LLM call LLMSummarizer needs. omnillm's ChatClient and omnillm-core providers satisfy it.
type Config ¶
type Config struct {
// Name is how the agent introduces itself, for example "OmniAgent".
Name string
// OnBehalfOf names the person the agent attends for, if any.
OnBehalfOf string
// Names are other names the agent answers to, besides Name, such as a
// spelling speech to text tends to produce.
Names []string
// Announcement is spoken once when the agent is admitted. Empty uses
// DefaultAnnouncement, followed by an invitation to ask questions when
// there is a Responder. It is skipped when Speaker is nil or the session
// cannot send audio.
Announcement string
// Transcriber turns the meeting audio into text. Required.
Transcriber Transcriber
// TranscriptionConfig sets the model and language. The encoding and
// sample rate are taken from the session's audio format.
TranscriptionConfig stt.TranscriptionConfig
// Speaker synthesizes the announcement and answers. Optional.
Speaker tts.Provider
// SynthesisConfig sets the voice. With no OutputFormat, the meeting's
// own audio format is requested, such as mu-law at 8 kHz on a phone
// call, which Deepgram, ElevenLabs, and OpenAI produce. Otherwise request
// raw PCM or mu-law output, since compressed formats cannot be played on
// a call.
SynthesisConfig tts.SynthesisConfig
// Responder answers when someone addresses the agent by name, after
// it is admitted. Optional: without one, or without a Speaker, the agent
// stays silent after its announcement. It answers one question at a
// time; a question asked while it is answering is not answered.
Responder Responder
// Summarizer summarizes the transcript when the meeting ends. Optional.
Summarizer Summarizer
// OnSegment is called for each finalized transcript segment, for
// example to print a live transcript. Optional.
OnSegment func(Segment)
// Logger receives problems that do not stop attendance. Defaults to
// slog.Default().
Logger *slog.Logger
}
Config configures an Attendant.
type Exchange ¶ added in v0.22.0
type Exchange struct {
// At is when the answer was spoken, as an offset from when
// transcription began.
At time.Duration `json:"at"`
// Question is what was said to the agent.
Question string `json:"question"`
// Answer is what the agent said.
Answer string `json:"answer"`
}
Exchange is a question the agent answered in the meeting.
type LLMResponder ¶ added in v0.22.0
type LLMResponder struct {
// Client sends the chat completion request.
Client ChatCompleter
// Model is the model name, for example "claude-sonnet-5".
Model string
}
LLMResponder answers questions from the meeting transcript with a chat model. It sends no tools, so a question cannot make it act.
type LLMSummarizer ¶
type LLMSummarizer struct {
// Client sends the chat completion request.
Client ChatCompleter
// Model is the model name, for example "claude-sonnet-5".
Model string
}
LLMSummarizer summarizes transcripts with a chat model.
func (LLMSummarizer) Summarize ¶
func (s LLMSummarizer) Summarize(ctx context.Context, transcript Transcript) (*Summary, error)
Summarize asks the model for a JSON summary of the transcript.
type Record ¶
type Record struct {
// Target is the meeting attended.
Target attendee.JoinTarget `json:"target"`
// Started and Ended bound the attendance.
Started time.Time `json:"started"`
Ended time.Time `json:"ended"`
// FinalState is how the session ended: left, ended, removed, or failed.
FinalState attendee.State `json:"final_state"`
// Announced reports whether the announcement was played.
Announced bool `json:"announced"`
// Transcript is the meeting's speech. The agent's answers are included
// with the agent's name as the speaker.
Transcript Transcript `json:"transcript"`
// Exchanges are the questions the agent answered.
Exchanges []Exchange `json:"exchanges,omitempty"`
// Summary is nil when there is no Summarizer or nothing was said.
Summary *Summary `json:"summary,omitempty"`
}
Record is what the Attendant captured from a meeting.
type Request ¶ added in v0.22.0
type Request struct {
// Name is the agent's name.
Name string
// Question is what was said to the agent, including its name.
Question string
// Transcript is the recent meeting speech, oldest first, ending with
// the question.
Transcript Transcript
}
Request is a question put to the agent by name during a meeting.
type Responder ¶ added in v0.22.0
Responder answers a question put to the agent during a meeting. The answer is spoken into the meeting; an empty answer says nothing.
Meeting speech is untrusted input: anyone in the meeting chooses the question. A Responder should not have tools that act outside the meeting, such as sending messages or changing data. LLMResponder has none.
type Segment ¶
type Segment struct {
// Start and End are offsets from when transcription began.
Start time.Duration `json:"start"`
End time.Duration `json:"end"`
// Speaker is the speaker label, when the connection provides one.
// Phone dial-in carries one mixed stream, so it is empty.
Speaker string `json:"speaker,omitempty"`
// Text is what was said.
Text string `json:"text"`
}
Segment is one finalized stretch of transcribed speech.
type Summarizer ¶
type Summarizer interface {
Summarize(ctx context.Context, transcript Transcript) (*Summary, error)
}
Summarizer turns a meeting transcript into a Summary.
type Summary ¶
type Summary struct {
// Overview is a short neutral recap of the discussion.
Overview string `json:"overview"`
// Decisions are what the participants agreed on.
Decisions []string `json:"decisions"`
// ActionItems are tasks someone took on.
ActionItems []ActionItem `json:"action_items"`
// OpenQuestions are questions raised but not resolved.
OpenQuestions []string `json:"open_questions"`
}
Summary is the outcome of a meeting.
type Transcriber ¶
type Transcriber interface {
TranscribeStream(ctx context.Context, config stt.TranscriptionConfig) (io.WriteCloser, <-chan stt.StreamEvent, error)
}
Transcriber streams speech to text. omnivoice-core's stt.StreamingProvider and stt.Client satisfy it.
type Transcript ¶
type Transcript []Segment
Transcript is the meeting's speech in order.
func (Transcript) String ¶
func (t Transcript) String() string
String renders the transcript one segment per line with timestamps, for example "[00:01:05] Let's move the launch to Thursday."