Documentation
¶
Overview ¶
Package answer turns a question into a grounded answer, or into an honest refusal to answer.
Declining is a first-class outcome rather than an error path. Calibration against the curated corpus showed that a retrieval score cannot separate a question the corpus answers from one whose subject it merely covers: the bands overlap, and excluding every unanswerable question would cost 40% of the answerable ones. BM25 measures term overlap, not whether a passage answers anything.
So the judgement lives here, and it is made against the retrieved passages. A component that can only produce answers produces one for every question, including the questions whose correct answer is that we do not know — and those answers arrive carrying a real citation to a page that does not say it, which is worse than silence.
Index ¶
- Constants
- Variables
- func ClassifierPrompt() string
- func ClassifierSchema() *jsonschema.Schema
- func RenumberMarkers(text string, cs []Citation, listed int) string
- func ResponseSchema() *jsonschema.Schema
- func SystemPrompt() string
- type Answer
- type ChatClassifier
- type ChatComposer
- type Citation
- type Clarification
- type ClarifyPolicy
- type Composer
- type Evidence
- type Label
- type Option
- type Outcome
- type ReadingSource
- type Response
- type Retrieval
- type Service
- type Signpost
- type Signposter
- type Speaker
- type ThreadComposer
- type Turn
Constants ¶
const ( ClarifyName = "name" ClarifySplit = "split" )
Clarification reasons.
const ( ThreadDelimiter = "-----BEGIN THREAD MESSAGE" ThreadEnd = "-----END THREAD MESSAGE" )
ThreadDelimiter and ThreadEnd open the lines that fence a thread message in the prompt. Each line also carries an id drawn afresh for every prompt, so a message cannot end its fence by writing one: it cannot know the id.
const DefaultLimit = 8
DefaultLimit is how many passages are retrieved for one question.
const PassageDelimiter = "-----BEGIN PASSAGE-----"
PassageDelimiter fences retrieved content in the prompt.
Passage text is untrusted: it comes from public repositories anyone may open a merge request against, and in later phases the question arrives from a public Discord channel. Delimiting is what keeps content from reading as instruction — it does not make injection impossible, but it makes the boundary explicit and it is the part a prompt can actually enforce.
Variables ¶
var ( // ErrUngrounded means the composer returned prose citing nothing. Whatever // it says is not supported by anything retrieved. ErrUngrounded = errors.NewSentinel("answer.ungrounded", "answer: no passage was cited") // ErrFabricatedCitation means a citation pointed outside the retrieved set. ErrFabricatedCitation = errors.NewSentinel("answer.fabricated_citation", "answer: cited a passage that was not retrieved") )
Reasons an answer was refused after the composer produced one.
var DefaultClarify = ClarifyPolicy{SplitShare: defaultSplitShare, NamePassages: defaultNamePassages}
DefaultClarify is the policy with spec 0031 D5's defaults.
Functions ¶
func ClassifierPrompt ¶ added in v0.15.0
func ClassifierPrompt() string
ClassifierPrompt is the system prompt the classifier's client is built with.
func ClassifierSchema ¶ added in v0.15.0
func ClassifierSchema() *jsonschema.Schema
ClassifierSchema is the structured-output schema the classifier expects.
func RenumberMarkers ¶ added in v0.12.1
RenumberMarkers rewrites the composer's passage numbers in text to positions in a Sources list showing the first listed citations, and drops markers for passages that are not listed (spec 0030 D1). Code and quoted lines are left alone: their brackets are not markers.
func ResponseSchema ¶
func ResponseSchema() *jsonschema.Schema
ResponseSchema is the structured-output schema the composer expects.
Reflected from composeReply through chat.GenerateSchema rather than written by hand. The hand-written map worked against claude-local, which marshals whatever it is given, and was silently mangled by the Anthropic provider — that provider reads a *jsonschema.Schema for its properties and required fields, and treats any other value as the properties map itself. Reflecting the struct keeps the schema and the type it decodes into from drifting apart as well.
Returns the concrete *jsonschema.Schema rather than any. chat.Config.Bind types the field concretely from v0.9.0, so the interface return now needs a type assertion at every call site — and the whole reason this exists is that handing that field the wrong shape fails silently rather than loudly.
func SystemPrompt ¶
func SystemPrompt() string
SystemPrompt is the instruction the composer relies on, exported so a caller constructing the client can set it.
Types ¶
type Answer ¶
type Answer struct {
// Outcome is the composer's own judgement, and the only record of it.
// Callers select how to present an answer from this and the citation count
// (spec 0010 D4), so "answered" and "refuted" must stay distinguishable all
// the way out: a refutation hedged for having one citation reads as
// uncertainty about the one thing the documentation is certain of.
Outcome Outcome
// Text is the prose, empty when declining.
Text string
// Cites indexes the passages the answer is grounded in, as positions in the
// slice the composer was given.
Cites []int
// DeclineReason says why, in terms a person can read.
DeclineReason string
// Title is the model's thread title, for every outcome (spec 0026 D1).
Title string
}
Answer is what a Composer produces.
type ChatClassifier ¶ added in v0.15.0
type ChatClassifier struct {
// contains filtered or unexported fields
}
ChatClassifier classifies thread messages through a chat provider.
func NewClassifier ¶ added in v0.15.0
func NewClassifier(client chat.ChatClient) *ChatClassifier
NewClassifier builds a classifier over a client constructed with ClassifierPrompt and ClassifierSchema.
type ChatComposer ¶
type ChatComposer struct {
// contains filtered or unexported fields
}
ChatComposer composes answers through a chat provider.
func NewComposer ¶
func NewComposer(client chat.ChatClient) *ChatComposer
NewComposer builds a composer over a chat client.
type Citation ¶
type Citation struct {
// Passage is the number the composer cited, which is what the answer's
// inline markers refer to (spec 0030 D1).
Passage int
URL string
HeadingPath string
SourcePath string
RepoPath string
CommitSHA string
index.PageInfo
}
Citation is a resolved reference to the passage supporting an answer.
type Clarification ¶ added in v0.15.0
Clarification is the decision to ask back, and what to offer. A zero Reason means answer.
type ClarifyPolicy ¶ added in v0.15.0
type ClarifyPolicy struct {
// for a question to be asked back.
SplitShare float64
// NamePassages is how many passages must come from the modules carrying an
// unresolved name for the bot to ask whether the name meant them.
NamePassages int
}
ClarifyPolicy holds the two thresholds that decide when the bot asks rather than answers (spec 0031 D5). Both are per-server settings.
func (ClarifyPolicy) Decide ¶ added in v0.15.0
func (p ClarifyPolicy) Decide(r index.Readings, limit int) Clarification
Decide applies the policy to a question's readings, where limit is how many passages were asked for.
type Composer ¶
type Composer interface {
Compose(ctx context.Context, question string, results []index.Result) (Answer, error)
}
Composer turns a question and its retrieved passages into an answer, or declines.
The contract is deliberately able to say no. Sufficiency is judged against the passages rather than against the question or the model's own knowledge: an answer that cannot be grounded in what was retrieved must not be returned however plausible it is.
type Evidence ¶ added in v0.15.0
type Evidence struct {
Counts map[string]int
// TopShares is the two largest readings' shares of the passage limit,
// largest first, with fewer when fewer readings were found.
Names map[string][]string
Policy ClarifyPolicy
Clarification
}
Evidence is what a clarification decision was made on, kept so a threshold can be judged against real traffic. It holds module names and counts, never text.
type Label ¶ added in v0.15.0
type Label string
Label is the classifier's reading of a message in a conversation thread (spec 0031 D4).
type Option ¶
type Option func(*Service)
Option configures a Service.
func WithClarify ¶ added in v0.15.0
func WithClarify(p ClarifyPolicy) Option
WithClarify judges every question's passages by p, so a caller can ask back rather than answer (spec 0031 D5). Without it nothing is judged.
func WithScoreFloor ¶
WithScoreFloor discards passages scoring below f before composing.
A floor discards noise. It is deliberately not the gate: calibration showed no score separates answerable questions from unanswerable ones, so a floor set high enough to exclude the latter takes 40% of the former with it.
func WithSignposts ¶ added in v0.15.0
func WithSignposts(s Signposter) Option
WithSignposts adds upstream signposts to declines and partial answers.
type Outcome ¶ added in v0.2.0
type Outcome string
Outcome is the composer's judgement about the passages it was given.
It is the confidence signal. Spec 0010 D3 replaced a numeric one derived from retrieval scores, because measurement showed the bands overlap: a question about something that does not exist matches the page saying so, and scores highly for exactly the right reason. A retrieval score measures how well the query matched text, which is a different question from whether the text answers it. The composer judges sufficiency first-hand, so it is asked.
const ( // OutcomeAnswered means the passages answer the question. OutcomeAnswered Outcome = "answered" // OutcomeRefuted means the passages CONTRADICT the question's premise. // That is an answer, not a failure, and it is stated plainly rather than // hedged — the documentation is what makes it certain. OutcomeRefuted Outcome = "refuted" // OutcomeDeclined means the passages do not answer the question. OutcomeDeclined Outcome = "declined" // OutcomePartial means the passages answer part of the question. It is an // answer and a documentation gap at once, and is recorded as both (spec // 0030 D3). OutcomePartial Outcome = "partial" )
The outcomes a composer may report.
type ReadingSource ¶ added in v0.15.0
type ReadingSource interface {
Readings(ctx context.Context, question string, results []index.Result) (index.Readings, error)
}
ReadingSource is a retriever that can say which readings its results hold. index.Targeted is one.
type Response ¶
type Response struct {
// Outcome is the composer's judgement, carried through unchanged, except
// that an answer this package refuses to publish becomes a decline.
//
// A decline is an outcome, not a failure: a declined question is a
// knowledge gap, which is the product's highest-value output (R-ANS-6c).
Outcome Outcome
Text string
Citations []Citation
DeclineReason string
// Title is the composer's thread title, unguarded. respond.ThreadTitle
// decides what is published (spec 0026).
Title string
// Cause carries why an answer was refused after being produced, when it was
// refused here rather than declined by the composer.
Cause error
// Retrieved is what the question found, for reporting and for measuring
// retrieval independently of composition.
Retrieved []index.Result
// Signposts point a declined or partly answered question at the upstream
// it belongs to (spec 0036).
Signposts []Signpost
// Evidence is what clarification judged the passages by, nil when nothing
// was judged (spec 0031 D5).
Evidence *Evidence
}
Response is the outcome of asking a question.
type Retrieval ¶ added in v0.15.0
type Retrieval struct {
Question string
Results []index.Result
// Evidence is nil when the service has no clarify policy, its retriever
// reports no readings, or nothing was retrieved.
Evidence *Evidence
// Thread is the thread the question was asked in, set by the caller
// between Retrieve and Compose. Context for the composer, never retrieved
// on (spec 0031 D8).
Thread []Turn
}
Retrieval is what a question found, before anything is composed from it.
func (Retrieval) Clarification ¶ added in v0.15.0
func (r Retrieval) Clarification() Clarification
Clarification is what the policy decided, zero when nothing was judged.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service answers questions from the index.
func (*Service) Ask ¶
Ask answers a question, or declines, whatever clarification would say.
An error means something broke — the index was unreachable, the provider failed. It never means "we do not know", which is a Response with Declined set. Conflating them would lose the distinction between a knowledge gap and an incident, and the gap backlog depends on it.
type Signpost ¶ added in v0.15.0
type Signpost struct {
Upstream string
Wrapper string
// Version is what the wrapper builds or requires, empty when unknown.
Version string
Docs string
}
Signpost points at an upstream project's own documentation, for a question the bot cannot answer from this estate's (spec 0036).
type Signposter ¶ added in v0.15.0
Signposter finds the signposts a question warrants.
type ThreadComposer ¶ added in v0.15.0
type ThreadComposer interface {
ComposeInThread(ctx context.Context, question string, thread []Turn, results []index.Result) (Answer, error)
}
ThreadComposer is a Composer that can take the thread a question was asked in. ChatComposer is one; a Composer that is not composes without it.