driver

package
v0.10.3 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: MIT Imports: 27 Imported by: 0

Documentation

Overview

Package driver is the compiled engine an agent runs on: the LLM HTTP client (Host, a host.Host implementation), TTS synthesis + playback, the serial speech queue, and health/Twilio HTTP servers. An agent supplies its particulars — tiers, TTS transport, and a system-prompt provider — through Config and stands the driver up with New. It bakes in no agent identity: prompts and policy come from the caller.

The engine driver serves HTTP (design/aa-server-status.md §7.1, §10): aa-server-status needs a mandatory GET /healthz on :9730 to manage it as a "source" server (§6.1 — health probe is the authoritative "serving" signal). This file is intentionally minimal: the health listener and the Twilio-facing listener, nothing more. A local HTTP tester for the driver (app-cli) is separate, deferred work (design §11) and does not belong here.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func EnvBool

func EnvBool(k string) bool

EnvBool reads a boolean-ish env var: "1"/"true"/"yes"/"on" (any case) is true.

func EnvFloatOr

func EnvFloatOr(k string, def float64) float64

EnvFloatOr reads a float env var, falling back to def when unset or unparseable.

func EnvOr

func EnvOr(k, def string) string

func FilePrompt

func FilePrompt(path, defaultText string) func() string

FilePrompt returns a system-prompt provider (Config.Prompt) that reads path and hot-reloads it when the file's mtime changes, falling back to defaultText (or the last good load) if the file can't be read. Agents that don't want a file can pass any func() string to Config.Prompt instead.

func ParseStreamScheme

func ParseStreamScheme(args []string) (string, error)

ParseStreamScheme parses the -stream-scheme flag, defaulting to "wss" and rejecting any value other than "ws" or "wss". The flag itself is validated so an invalid scheme never reaches twilio.Server.StreamScheme. It uses a fresh FlagSet (rather than the package-global flag.CommandLine) so it can be called repeatedly and in isolation from tests.

func StartHealthServer

func StartHealthServer(addr string) *http.Server

StartHealthServer launches the health listener in the background and returns the *http.Server handle so the caller can Shutdown it on exit. A bind failure is reported loudly to stderr but does not crash the driver — aa-server-status will simply see the health probe fail, which is itself the correct signal (design §6.5: runtime errors abort loudly, they don't take the whole program down).

func StartTwilioServer

func StartTwilioServer(addr string, s *twilio.Server, opts ...TwilioServerOption) *http.Server

StartTwilioServer launches the Twilio-facing HTTP listener in the background, serving the routes built by newTwilioMux — which is where they are listed, so adding one is not a two-comment edit. opts are passed straight through to newTwilioMux; with none, the server is exactly what it was before they existed. Bind failures are reported to stderr but do not crash the process.

opts was added in AATK-159. Existing calls compile unchanged, but the function's type changed, so code holding StartTwilioServer in a variable of the old func type must update that type.

Types

type Config

type Config struct {
	Tiers  map[string]Tier
	TTS    TTSConfig
	Prompt func() string

	// UserContext supplies an optional user-context block, injected as a
	// system message after the system prompt and before the history.
	// queryText is the text the policy passed to Context or ContextFor. It may
	// do I/O. It runs outside the history lock, and the driver adds no
	// deadline, cancellation or error path of its own: bound your own I/O and
	// return "" on failure. It takes no ctx, so a ctx passed to ContextFor or
	// bound by WithContext never reaches it; set UserContextCtx for that. If
	// both are set, UserContextCtx wins and UserContext is never called.
	UserContext func(queryText string) string

	// UserContextCtx is the ctx-taking form of UserContext: it supplies the
	// same optional user-context block, injected after the system prompt and
	// before the history, from the queryText the policy passed to Context or
	// ContextFor. ctx is context.Background() from Context(q), the caller's
	// ctx from ContextFor(ctx, q), and the bound ctx from a WithContext view's
	// Context. The driver passes ctx on unchanged and adds no deadline,
	// cancellation or error path of its own: ctx carries only what the caller
	// set, and its deadline and cancellation stop the provider's I/O only
	// where the provider honours them. It may do I/O and runs outside the
	// history lock: bound your own I/O and return "" on failure. If both are
	// set, UserContextCtx wins and UserContext is never called.
	UserContextCtx func(ctx context.Context, queryText string) string

	// RealtimeURL selects the voice transport. Empty — the default — runs the
	// existing VAD/STT/TTS sidecar path and constructs no realtime client. Set
	// to a backend websocket URL, calls run over that backend instead, which
	// also hands it the turn-taking; see twilio.NewStreamHandler for what that
	// bypasses. The engine supplies no default URL: like every other field
	// here, it is the consumer's to inject.
	RealtimeURL string

	// TracerProvider supplies the tracer for the span each model call
	// produces (AATK-159): one CLIENT span, named "chat <Tier.Model>", per
	// call whose tier is found and whose window encodes. That includes a call
	// that sends nothing because its context is already done or its tier URL
	// cannot be requested; its span records why it failed. Nil, the default,
	// means otel.GetTracerProvider() is resolved at each call, so a consumer
	// that installs its SDK globally, before or after New, gets spans. With
	// nothing installed, the no-op provider records and sends nothing. The
	// engine never installs a provider, exporter or propagator, and never adds
	// trace headers to the upstream request.
	//
	// Message content stays off the span by default: it is personal data (the
	// system prompt, the user-context block, the history and the user's own
	// text). The environment variable AATOOLKIT_CAPTURE_MESSAGE_CONTENT turns
	// it on when strconv.ParseBool reads its value as true ("1", "true"):
	// each request's messages and the model's output are then recorded as
	// gen_ai.input.messages and gen_ai.output.messages. Unset, empty, false or
	// unparsable leaves it off. It is read per request, and only for a span
	// that is recording. Values are not truncated; bound them with the SDK's
	// span limits (OTEL_ATTRIBUTE_VALUE_LENGTH_LIMIT).
	TracerProvider trace.TracerProvider
}

Config is what an agent supplies to stand up a driver: its LLM tiers, TTS transport, and a system-prompt provider. The engine holds none of this by default — a different agent (with its own prompt, tiers, policy) constructs its own Config. Prompt is called every turn; use FilePrompt for a hot-reloading file or supply any func() string (embed, remote, constant).

func (Config) StreamHandler added in v0.3.0

func (c Config) StreamHandler() twilio.StreamHandler

StreamHandler returns the Media Streams handler for the configured transport, for a consumer to install on twilio.Server.HandleStream. It is the one place the transport switch is read.

type Host

type Host struct {
	// contains filtered or unexported fields
}

Host is the driver's concrete host.Host: it turns a (messages, tier) pair into an HTTP call against the tier's llama-server.

func New

func New(cfg Config) *Host

New builds a driver Host from cfg, wiring the serial speech queue internally.

func (*Host) CancelQueued

func (h *Host) CancelQueued()

CancelQueued implements host.Host: drops all pending clips from the speech queue without rendering them. Call alongside Forget() on stream errors so orphaned segments don't play from a rolled-back turn.

func (*Host) Clear

func (h *Host) Clear()

func (*Host) Context

func (h *Host) Context(queryText string) []byte

Context implements host.Host: ContextFor under context.Background().

func (*Host) ContextFor added in v0.10.0

func (h *Host) ContextFor(ctx context.Context, queryText string) []byte

ContextFor implements host.ContextBuilder. It assembles [current system prompt] + optional user context + history into a JSON messages array ready for Send. queryText is handed to the user-context provider, so its block can depend on the current turn, and ctx is handed on unchanged to a Config.UserContextCtx provider; a plain Config.UserContext never sees it. ContextFor adds no deadline or cancellation of its own and never checks ctx beyond rejecting nil: a nil ctx panics, as WithContext's does, whichever provider form is configured.

func (*Host) Forget

func (h *Host) Forget()

func (*Host) LastAnswer

func (h *Host) LastAnswer() []byte

func (*Host) Remember

func (h *Host) Remember(role string, content []byte)

func (*Host) Send

func (h *Host) Send(contextWindow []byte, tier string) ([]byte, []byte, error)

Send implements host.Host. contextWindow is the already serialized `messages` array; tier is "fast" | "deep". Returns content, reasoning (empty if the model/flags don't emit it), error. It is SendContext under context.Background(), so sendTimeout (300 s) alone bounds it.

func (*Host) SendContext added in v0.9.0

func (h *Host) SendContext(ctx context.Context, contextWindow []byte, tier string) ([]byte, []byte, error)

SendContext implements host.ContextSender: Send under ctx. It's SendStreamContext without the per-segment callback — the full assembled text is all the caller wants.

func (*Host) SendStream

func (h *Host) SendStream(contextWindow []byte, tier string, onSegment func(string)) ([]byte, []byte, error)

SendStream is like Send but streams the response, calling onSegment each time a punctuation boundary is reached with enough accumulated text. The returned slices are the fully assembled (content, reasoning). Segments are flushed when the buffer crosses ~30 chars and a delimiter (. ? ! , \n) is seen; the delimiter is kept at the end of the segment so the TTS has the punctuation it needs for natural cadence. It is SendStreamContext under context.Background(), so sendTimeout (300 s) alone bounds it.

func (*Host) SendStreamContext added in v0.9.0

func (h *Host) SendStreamContext(ctx context.Context, contextWindow []byte, tier string, onSegment func(string)) (content, reasoning []byte, err error)

SendStreamContext implements host.ContextSender: SendStream under ctx. The request runs under ctx bounded by sendTimeout (300 s), so ctx's values reach it and its deadline applies when it is the earlier one. Once ctx is done the call fails at the next read, line or segment, even when the rest of the stream is already buffered: no onSegment call starts after that, though one already running is not interrupted. Wherever the call stops once ctx is done (sending, waiting for the response headers, reading a non-200 status body or the stream, or between lines), and over HTTP/1 or HTTP/2 alike, the error matches both ctx.Err() and context.Cause(ctx) under errors.Is: for a context cancelled or timed out with a cause (context.WithCancelCause, WithTimeoutCause, WithDeadlineCause), it matches that cause as well as context.Canceled or DeadlineExceeded. It is the one implementation of a model request: Send, SendStream and SendContext all end here.

Once its tier is found and its window encodes, a call gets one CLIENT span, a child of ctx's span, from Config.TracerProvider or the global provider. An unknown tier, or a window that does not encode (bytes that are not JSON, or an empty non-nil slice), gets none and sends nothing; a nil window encodes as null and is sent, as it always was. A call that then sends nothing, because ctx is already done or the tier URL cannot be requested, still gets its span, which records why it failed. The span covers the whole call, onSegment's time included, and ends after the last segment. It is ended by a defer, so a panicking onSegment still ends it, as an error of type _OTHER; the panic is not recovered.

func (*Host) Speak

func (h *Host) Speak(text []byte, voice string, speed float64) error

Speak implements host.Host. It queues synthesis+playback on the serial speech worker and returns immediately, so the turn's text prints without waiting for audio; a failure is logged to stderr and never breaks the turn. No cgo — the ONNX runtime lives in the separate `supertonic serve` process; the driver only does an HTTP POST and shells out to afplay.

func (*Host) SpeakSync

func (h *Host) SpeakSync(text []byte, voice string, speed float64) error

SpeakSync is Speak that blocks until playback finishes, so callers can play clips in sequence (used by /voicetest).

func (*Host) SynthesizeWAV added in v0.3.0

func (h *Host) SynthesizeWAV(ctx context.Context, text []byte, voice string, speed float64) ([]byte, error)

SynthesizeWAV fetches synthesized WAV bytes from the TTS server without playing them.

func (*Host) SystemPrompt

func (h *Host) SystemPrompt() string

SystemPrompt implements host.Host: the current system prompt, reloaded from its file only when the file changes.

func (*Host) WithContext added in v0.9.0

func (h *Host) WithContext(ctx context.Context) host.Host

WithContext returns a host.Host whose Send and SendStream run under ctx: their spans are children of ctx's span, and ctx's deadline and cancellation bound them. Its Context hands ctx to a Config.UserContextCtx provider. Every other method is the receiver's own, and ContextFor, SendContext and SendStreamContext keep their explicit ctx. Make one per inbound request and do not keep it beyond that request. For work that outlives an HTTP handler, pass context.WithoutCancel(ctx). A nil ctx panics, as http.Request.WithContext does.

type TTSConfig

type TTSConfig struct {
	URL    string // .../v1/tts
	Lang   string // ISO code, e.g. "en"
	Format string // wav / flac / ogg — wav plays via afplay
}

TTSConfig is the driver's fixed TTS transport config for Supertonic's native /v1/tts endpoint (the OpenAI-compatible /v1/audio/speech has no speed knob). The tunable per-call settings — voice and speed — are owned by the interpreted policy and passed into Speak, so /voice and /speed change them live.

type Tier

type Tier struct {
	URL, Model string
	Reasoning  bool
	MaxTokens  int
	// APIKey, when non-empty, is sent as "Authorization: Bearer <APIKey>" on
	// this tier's chat request (AATK-112) -- what a hosted OpenAI-compatible
	// API needs and a local llama-server ignores. Empty sends no header.
	APIKey string
	// NoTools, when true, declares an empty tools list on this tier's chat
	// request (AATK-113). This driver is text-only -- it reads content and
	// reasoning and drops tool_calls -- and a tool-trained hosted model that
	// is merely TOLD about capabilities in the prompt will invent a function
	// call and answer nothing. An explicit "tools": [] is what stops it;
	// tool_choice "none" was measured not to. Off by default: the local
	// servers have not been exercised against an empty tools array.
	NoTools bool
	// IncludeUsage, when true, sends "stream_options": {"include_usage": true}
	// on this tier's chat request (AATK-159), asking the upstream to report
	// the request's token usage at the end of the stream. Off by default,
	// because not every OpenAI-compatible upstream accepts the key; with it
	// off the request body is byte-identical to before the field existed.
	// Measured on one hosted OpenAI-compatible provider: it accepts the key,
	// and it also sends a final usage chunk without it. A local mlx-serve has
	// not been measured, so keep IncludeUsage off on local tiers until it is.
	// Usage is recorded on the span whenever the stream carries it, whatever
	// this flag says.
	IncludeUsage bool
}

Tier describes one tier's llama-server. reasoning=false asks the model to skip its thinking trace; maxTokens caps a single turn so a runaway generation can't burn the whole sendTimeout (300 s).

type Turn

type Turn struct {
	Text    string
	Speaker string
}

Turn is one unit of user input. Text is the typed line or the transcript; Speaker is who said it (empty until speaker-ID lands).

type TwilioServerOption added in v0.9.0

type TwilioServerOption func(*twilioServerConfig)

TwilioServerOption configures StartTwilioServer (AATK-159). It mirrors twilio.RealtimeOption: the zero set of options is today's server exactly, and every option is additive, so a caller that passes none gets what a caller got before options existed.

func WithHandlerWrapper added in v0.9.0

func WithHandlerWrapper(wrap func(route string, h http.Handler) http.Handler) TwilioServerOption

WithHandlerWrapper wraps each Twilio route's handler before it is mounted. At construction, wrap is called once per route with that route's path ("/webhook", "/streams", "/sms/inbound", "/sms/status") and its handler, and the mux serves what it returns on that path. It exists so a consumer can put request middleware in front of these routes (typically an OpenTelemetry server span, so model calls made while handling an inbound message nest under it) without rebuilding the route table on a mux of its own.

A nil wrap leaves the routes unwrapped. Given more than once, the last one wins, a nil one included. A wrap that returns a nil handler panics at construction, as http.ServeMux.Handle does.

Caveats:

  • /streams is the Media Streams WebSocket upgrade, which takes over the connection through the ResponseWriter. The writer the wrapper passes on must implement http.Hijacker, or have an Unwrap() http.ResponseWriter method that leads to one; otherwise the upgrade is answered 501 and every call fails. otelhttp.NewHandler keeps it. A span around /streams lasts the whole call, so a wrapper that wants per-request spans can return h unchanged for "/streams".
  • The wrapper runs before signature validation, so forged POSTs pass through it too (and get a span). It must not read r.Body or rewrite r.Host or r.URL: validation parses the body itself and reconstructs the signed URL from the host and path.
  • The ctx HandleSMS receives is the request's context: it carries what the wrapper put there, and it is cancelled when the request has been handled, moments after HandleSMS returns. Work that outlives the handler, such as a model call started in a goroutine, must run under context.WithoutCancel(ctx), which keeps the values (the span parent among them) and drops the cancellation. The same holds for the ctx HandleSMSStatus, RejectCall and AnswerCall receive.
  • With otelhttp, pass otelhttp.WithPublicEndpointFn returning true (current otelhttp releases no longer have WithPublicEndpoint): these requests come from the provider, not from a traced caller, so a trace context they carry is linked rather than adopted as the parent. The engine adds no span attributes of its own here, From and To included.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL