answer

package
v0.15.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 19, 2026 License: MIT Imports: 15 Imported by: 0

Documentation

Overview

Package answer turns a question into a grounded answer, or into an honest refusal to answer.

Declining is a first-class outcome rather than an error path. Calibration against the curated corpus showed that a retrieval score cannot separate a question the corpus answers from one whose subject it merely covers: the bands overlap, and excluding every unanswerable question would cost 40% of the answerable ones. BM25 measures term overlap, not whether a passage answers anything.

So the judgement lives here, and it is made against the retrieved passages. A component that can only produce answers produces one for every question, including the questions whose correct answer is that we do not know — and those answers arrive carrying a real citation to a page that does not say it, which is worse than silence.

Index

Constants

View Source
const (
	ClarifyName  = "name"
	ClarifySplit = "split"
)

Clarification reasons.

View Source
const (
	ThreadDelimiter = "-----BEGIN THREAD MESSAGE"
	ThreadEnd       = "-----END THREAD MESSAGE"
)

ThreadDelimiter and ThreadEnd open the lines that fence a thread message in the prompt. Each line also carries an id drawn afresh for every prompt, so a message cannot end its fence by writing one: it cannot know the id.

View Source
const DefaultLimit = 8

DefaultLimit is how many passages are retrieved for one question.

View Source
const PassageDelimiter = "-----BEGIN PASSAGE-----"

PassageDelimiter fences retrieved content in the prompt.

Passage text is untrusted: it comes from public repositories anyone may open a merge request against, and in later phases the question arrives from a public Discord channel. Delimiting is what keeps content from reading as instruction — it does not make injection impossible, but it makes the boundary explicit and it is the part a prompt can actually enforce.

Variables

View Source
var (
	// ErrUngrounded means the composer returned prose citing nothing. Whatever
	// it says is not supported by anything retrieved.
	ErrUngrounded = errors.NewSentinel("answer.ungrounded", "answer: no passage was cited")

	// ErrFabricatedCitation means a citation pointed outside the retrieved set.
	ErrFabricatedCitation = errors.NewSentinel("answer.fabricated_citation",
		"answer: cited a passage that was not retrieved")
)

Reasons an answer was refused after the composer produced one.

View Source
var DefaultClarify = ClarifyPolicy{SplitShare: defaultSplitShare, NamePassages: defaultNamePassages}

DefaultClarify is the policy with spec 0031 D5's defaults.

Functions

func ClassifierPrompt added in v0.15.0

func ClassifierPrompt() string

ClassifierPrompt is the system prompt the classifier's client is built with.

func ClassifierSchema added in v0.15.0

func ClassifierSchema() *jsonschema.Schema

ClassifierSchema is the structured-output schema the classifier expects.

func RenumberMarkers added in v0.12.1

func RenumberMarkers(text string, cs []Citation, listed int) string

RenumberMarkers rewrites the composer's passage numbers in text to positions in a Sources list showing the first listed citations, and drops markers for passages that are not listed (spec 0030 D1). Code and quoted lines are left alone: their brackets are not markers.

func ResponseSchema

func ResponseSchema() *jsonschema.Schema

ResponseSchema is the structured-output schema the composer expects.

Reflected from composeReply through chat.GenerateSchema rather than written by hand. The hand-written map worked against claude-local, which marshals whatever it is given, and was silently mangled by the Anthropic provider — that provider reads a *jsonschema.Schema for its properties and required fields, and treats any other value as the properties map itself. Reflecting the struct keeps the schema and the type it decodes into from drifting apart as well.

Returns the concrete *jsonschema.Schema rather than any. chat.Config.Bind types the field concretely from v0.9.0, so the interface return now needs a type assertion at every call site — and the whole reason this exists is that handing that field the wrong shape fails silently rather than loudly.

func SystemPrompt

func SystemPrompt() string

SystemPrompt is the instruction the composer relies on, exported so a caller constructing the client can set it.

Types

type Answer

type Answer struct {
	// Outcome is the composer's own judgement, and the only record of it.
	// Callers select how to present an answer from this and the citation count
	// (spec 0010 D4), so "answered" and "refuted" must stay distinguishable all
	// the way out: a refutation hedged for having one citation reads as
	// uncertainty about the one thing the documentation is certain of.
	Outcome Outcome

	// Text is the prose, empty when declining.
	Text string

	// Cites indexes the passages the answer is grounded in, as positions in the
	// slice the composer was given.
	Cites []int

	// DeclineReason says why, in terms a person can read.
	DeclineReason string

	// Title is the model's thread title, for every outcome (spec 0026 D1).
	Title string
}

Answer is what a Composer produces.

func (Answer) Declined

func (a Answer) Declined() bool

Declined reports whether the composer refused to answer.

A method rather than a field so the outcome stays the single source of truth. The two carried the same fact when both were fields, and a struct literal setting one and not the other is a lie no test would catch.

type ChatClassifier added in v0.15.0

type ChatClassifier struct {
	// contains filtered or unexported fields
}

ChatClassifier classifies thread messages through a chat provider.

func NewClassifier added in v0.15.0

func NewClassifier(client chat.ChatClient) *ChatClassifier

NewClassifier builds a classifier over a client constructed with ClassifierPrompt and ClassifierSchema.

func (*ChatClassifier) Classify added in v0.15.0

func (c *ChatClassifier) Classify(ctx context.Context, message string) (Label, error)

Classify labels one message. Anything but an exact label is LabelUnsure, and so is a failed call, whose error is returned for logging.

type ChatComposer

type ChatComposer struct {
	// contains filtered or unexported fields
}

ChatComposer composes answers through a chat provider.

func NewComposer

func NewComposer(client chat.ChatClient) *ChatComposer

NewComposer builds a composer over a chat client.

func (*ChatComposer) Compose

func (c *ChatComposer) Compose(
	ctx context.Context, question string, results []index.Result,
) (Answer, error)

Compose asks the model to answer from the passages, or to decline.

func (*ChatComposer) ComposeInThread added in v0.15.0

func (c *ChatComposer) ComposeInThread(
	ctx context.Context, question string, thread []Turn, results []index.Result,
) (Answer, error)

ComposeInThread is Compose with the thread the question was asked in, as untrusted context (spec 0031 D8).

type Citation

type Citation struct {
	// Passage is the number the composer cited, which is what the answer's
	// inline markers refer to (spec 0030 D1).
	Passage int

	URL         string
	HeadingPath string
	SourcePath  string
	RepoPath    string
	CommitSHA   string

	index.PageInfo
}

Citation is a resolved reference to the passage supporting an answer.

func (Citation) Label added in v0.15.0

func (c Citation) Label() string

Label is how the citation is named to a reader: its heading, and for a published site's page its lane and date or that it is a guide.

type Clarification added in v0.15.0

type Clarification struct {
	Reason string
	Offer  []string
}

Clarification is the decision to ask back, and what to offer. A zero Reason means answer.

type ClarifyPolicy added in v0.15.0

type ClarifyPolicy struct {
	// SplitShare is the share of the passage limit two readings must each hold
	// for a question to be asked back.
	SplitShare float64
	// NamePassages is how many passages must come from the modules carrying an
	// unresolved name for the bot to ask whether the name meant them.
	NamePassages int
}

ClarifyPolicy holds the two thresholds that decide when the bot asks rather than answers (spec 0031 D5). Both are per-server settings.

func (ClarifyPolicy) Decide added in v0.15.0

func (p ClarifyPolicy) Decide(r index.Readings, limit int) Clarification

Decide applies the policy to a question's readings, where limit is how many passages were asked for.

type Composer

type Composer interface {
	Compose(ctx context.Context, question string, results []index.Result) (Answer, error)
}

Composer turns a question and its retrieved passages into an answer, or declines.

The contract is deliberately able to say no. Sufficiency is judged against the passages rather than against the question or the model's own knowledge: an answer that cannot be grounded in what was retrieved must not be returned however plausible it is.

type Evidence added in v0.15.0

type Evidence struct {
	Counts map[string]int
	// TopShares is the two largest readings' shares of the passage limit,
	// largest first, with fewer when fewer readings were found.
	TopShares []float64
	Names     map[string][]string
	Policy    ClarifyPolicy

	Clarification
}

Evidence is what a clarification decision was made on, kept so a threshold can be judged against real traffic. It holds module names and counts, never text.

type Label added in v0.15.0

type Label string

Label is the classifier's reading of a message in a conversation thread (spec 0031 D4).

const (
	LabelFollowUp      Label = "follow_up"
	LabelCommentary    Label = "commentary"
	LabelToSomeoneElse Label = "to_someone_else"
	LabelUnsure        Label = "unsure"
)

The labels a classifier may return. Only LabelFollowUp is a reason to reply.

func (Label) FollowUp added in v0.15.0

func (l Label) FollowUp() bool

FollowUp reports whether the message should be answered.

type Option

type Option func(*Service)

Option configures a Service.

func WithClarify added in v0.15.0

func WithClarify(p ClarifyPolicy) Option

WithClarify judges every question's passages by p, so a caller can ask back rather than answer (spec 0031 D5). Without it nothing is judged.

func WithLimit

func WithLimit(n int) Option

WithLimit sets how many passages are retrieved.

func WithScoreFloor

func WithScoreFloor(f float64) Option

WithScoreFloor discards passages scoring below f before composing.

A floor discards noise. It is deliberately not the gate: calibration showed no score separates answerable questions from unanswerable ones, so a floor set high enough to exclude the latter takes 40% of the former with it.

func WithSignposts added in v0.15.0

func WithSignposts(s Signposter) Option

WithSignposts adds upstream signposts to declines and partial answers.

type Outcome added in v0.2.0

type Outcome string

Outcome is the composer's judgement about the passages it was given.

It is the confidence signal. Spec 0010 D3 replaced a numeric one derived from retrieval scores, because measurement showed the bands overlap: a question about something that does not exist matches the page saying so, and scores highly for exactly the right reason. A retrieval score measures how well the query matched text, which is a different question from whether the text answers it. The composer judges sufficiency first-hand, so it is asked.

const (
	// OutcomeAnswered means the passages answer the question.
	OutcomeAnswered Outcome = "answered"

	// OutcomeRefuted means the passages CONTRADICT the question's premise.
	// That is an answer, not a failure, and it is stated plainly rather than
	// hedged — the documentation is what makes it certain.
	OutcomeRefuted Outcome = "refuted"

	// OutcomeDeclined means the passages do not answer the question.
	OutcomeDeclined Outcome = "declined"

	// OutcomePartial means the passages answer part of the question. It is an
	// answer and a documentation gap at once, and is recorded as both (spec
	// 0030 D3).
	OutcomePartial Outcome = "partial"
)

The outcomes a composer may report.

type ReadingSource added in v0.15.0

type ReadingSource interface {
	Readings(ctx context.Context, question string, results []index.Result) (index.Readings, error)
}

ReadingSource is a retriever that can say which readings its results hold. index.Targeted is one.

type Response

type Response struct {
	// Outcome is the composer's judgement, carried through unchanged, except
	// that an answer this package refuses to publish becomes a decline.
	//
	// A decline is an outcome, not a failure: a declined question is a
	// knowledge gap, which is the product's highest-value output (R-ANS-6c).
	Outcome Outcome

	Text      string
	Citations []Citation

	DeclineReason string

	// Title is the composer's thread title, unguarded. respond.ThreadTitle
	// decides what is published (spec 0026).
	Title string

	// Cause carries why an answer was refused after being produced, when it was
	// refused here rather than declined by the composer.
	Cause error

	// Retrieved is what the question found, for reporting and for measuring
	// retrieval independently of composition.
	Retrieved []index.Result

	// Signposts point a declined or partly answered question at the upstream
	// it belongs to (spec 0036).
	Signposts []Signpost

	// Evidence is what clarification judged the passages by, nil when nothing
	// was judged (spec 0031 D5).
	Evidence *Evidence
}

Response is the outcome of asking a question.

func (Response) Declined

func (r Response) Declined() bool

Declined reports whether the question went unanswered.

type Retrieval added in v0.15.0

type Retrieval struct {
	Question string
	Results  []index.Result

	// Evidence is nil when the service has no clarify policy, its retriever
	// reports no readings, or nothing was retrieved.
	Evidence *Evidence

	// Thread is the thread the question was asked in, set by the caller
	// between Retrieve and Compose. Context for the composer, never retrieved
	// on (spec 0031 D8).
	Thread []Turn
}

Retrieval is what a question found, before anything is composed from it.

func (Retrieval) Clarification added in v0.15.0

func (r Retrieval) Clarification() Clarification

Clarification is what the policy decided, zero when nothing was judged.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service answers questions from the index.

func New

func New(r index.Retriever, c Composer, opts ...Option) *Service

New builds an answering service.

func (*Service) Ask

func (s *Service) Ask(ctx context.Context, question string) (Response, error)

Ask answers a question, or declines, whatever clarification would say.

An error means something broke — the index was unreachable, the provider failed. It never means "we do not know", which is a Response with Declined set. Conflating them would lose the distinction between a knowledge gap and an incident, and the gap backlog depends on it.

func (*Service) Compose added in v0.15.0

func (s *Service) Compose(ctx context.Context, r Retrieval) (Response, error)

Compose answers from what Retrieve found, or declines. A retrieval that found nothing declines without a model call.

func (*Service) Retrieve added in v0.15.0

func (s *Service) Retrieve(ctx context.Context, question string) (Retrieval, error)

Retrieve finds a question's passages and judges whether it should be asked back. It never calls the composer.

type Signpost added in v0.15.0

type Signpost struct {
	Upstream string
	Wrapper  string

	// Version is what the wrapper builds or requires, empty when unknown.
	Version string
	Docs    string
}

Signpost points at an upstream project's own documentation, for a question the bot cannot answer from this estate's (spec 0036).

func (Signpost) Line added in v0.15.0

func (s Signpost) Line() string

Line is the signpost in one sentence. A fixed template, never the model's words, so the bot cannot be talked into answering for an upstream (D3).

type Signposter added in v0.15.0

type Signposter interface {
	Signposts(ctx context.Context, question string) []Signpost
}

Signposter finds the signposts a question warrants.

type Speaker added in v0.15.0

type Speaker int

Speaker is who wrote a thread message.

const (
	SpeakerMember Speaker = iota
	// SpeakerBot is another bot or a webhook.
	SpeakerBot
	// SpeakerSelf is this bot's own earlier reply.
	SpeakerSelf
)

The speakers a thread message is labelled with.

type ThreadComposer added in v0.15.0

type ThreadComposer interface {
	ComposeInThread(ctx context.Context, question string, thread []Turn, results []index.Result) (Answer, error)
}

ThreadComposer is a Composer that can take the thread a question was asked in. ChatComposer is one; a Composer that is not composes without it.

type Turn added in v0.15.0

type Turn struct {
	Speaker Speaker
	Text    string
}

Turn is one earlier message of the thread a question was asked in. It is untrusted context for the composer, never a source (spec 0031 D8).

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL