Documentation
¶
Overview ¶
Package extraction provides automated extraction and synthesis of observations, entities, and knowledge-graph relationships from raw text, code notes, and session logs.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var ( // ErrUnsafeDestination marks a destination rejected by the outbound // policy at the URL, DNS-resolution/dial, or redirect layer. It aborts // extraction instead of falling back to heuristics. ErrUnsafeDestination = errors.New("extraction: outbound destination rejected by policy") // ErrResponseTooLarge marks a provider response that exceeded the // configured size bound before decoding. It also aborts. ErrResponseTooLarge = errors.New("extraction: provider response exceeded the size limit") ErrProviderUnavailable = errors.New("extraction: provider request failed") // ErrProviderRejected marks a non-2xx provider response. ErrProviderRejected = errors.New("extraction: provider returned an error status") // ErrInvalidProviderResponse marks an unparseable provider payload. ErrInvalidProviderResponse = errors.New("extraction: provider response could not be parsed") )
Stable sentinel outbound errors (SEC-02). Messages are static: they never embed URLs, hostnames, addresses, credentials, or upstream bodies.
Functions ¶
func ProviderDefaultBaseURL ¶
func ProviderDefaultBaseURL(provider LLMProvider) string
ProviderDefaultBaseURL returns the canonical HTTPS endpoint for a provider preset (empty string for empty or unknown providers).
Types ¶
type Config ¶
type Config struct {
Provider LLMProvider `json:"provider"`
BaseURL string `json:"base_url"`
APIKey string `json:"api_key"`
Model string `json:"model"`
Temperature float64 `json:"temperature"`
Timeout time.Duration `json:"timeout"`
}
Config holds LLM client parameters for server-side extraction.
type EdgeDraft ¶
type EdgeDraft struct {
FromTitle string `json:"from_title"`
ToTitle string `json:"to_title"`
RelationType string `json:"relation_type"` // references, relates_to, follows, supersedes, contradicts
Reasoning string `json:"reasoning"`
Confidence float64 `json:"confidence"`
}
EdgeDraft represents a candidate knowledge graph relationship.
type ExtractionRequest ¶
type ExtractionRequest struct {
Text string `json:"text"`
Project string `json:"project"`
SessionID string `json:"session_id,omitempty"`
Source string `json:"source,omitempty"`
LLMConfig *Config `json:"llm_config,omitempty"`
}
ExtractionRequest represents a payload submitted for knowledge extraction.
type ExtractionResult ¶
type ExtractionResult struct {
Observations []*ObservationDraft `json:"observations"`
Edges []*EdgeDraft `json:"edges"`
Summary string `json:"summary"`
SourceMethod string `json:"source_method"` // "llm" or "heuristic"
ExtractedAt time.Time `json:"extracted_at"`
}
ExtractionResult is the output of an extraction operation.
type LLMProvider ¶
type LLMProvider string
LLMProvider represents the supported LLM provider types.
const ( ProviderOpenAI LLMProvider = "openai" ProviderAnthropic LLMProvider = "anthropic" ProviderGoogle LLMProvider = "google" ProviderGemini LLMProvider = "gemini" ProviderOllama LLMProvider = "ollama" ProviderGeneric LLMProvider = "generic" )
type ObservationDraft ¶
type ObservationDraft struct {
Title string `json:"title"`
Content string `json:"content"`
Type string `json:"type"` // decision, bugfix, pattern, architecture, discovery, manual
Project string `json:"project"`
Scope string `json:"scope"`
Confidence float64 `json:"confidence"`
Tags []string `json:"tags"`
Entities []string `json:"entities,omitempty"`
}
ObservationDraft represents an observation extracted from raw input.
type OutboundPolicy ¶
type OutboundPolicy struct {
// AllowedHosts is the admin-approved destination allowlist (exact,
// case-insensitive hostnames without port; IP literals additionally pass
// the address-class rules). Empty means no outbound destination is
// approved.
AllowedHosts []string
// AllowedPorts lists approved TCP ports. Default: 443 only.
AllowedPorts []int
// AllowLoopback is an explicit local-only development switch that
// permits loopback addresses for approved hosts (still HTTPS-only).
AllowLoopback bool
// AllowInsecureLoopbackHTTP is an explicit local-only development switch
// that permits plain HTTP to strict loopback hosts (see
// transportpolicy.IsStrictLoopbackHost).
AllowInsecureLoopbackHTTP bool
// MaxRedirects caps the redirect chain length. Default: 3.
MaxRedirects int
// MaxResponseBodyBytes bounds the provider success body. Default: 4 MiB.
MaxResponseBodyBytes int64
// MaxErrorBodyBytes bounds how much of an error body is drained. Default: 4 KiB.
MaxErrorBodyBytes int64
// MaxConcurrent bounds concurrent outbound provider requests. Default: 4.
MaxConcurrent int
// TLSConfig is an optional admin TLS configuration (e.g. private CA
// roots). It never relaxes destination validation.
TLSConfig *tls.Config
// contains filtered or unexported fields
}
OutboundPolicy is the reusable destination policy (SEC-02) injected into the extraction Service. It is enforced at three layers:
- URL layer: ValidateURL runs before any request is built or a credential is attached. It requires HTTPS (plain HTTP only under an explicit local-only development switch), rejects userinfo, requires an approved port, requires the host to be admin-approved, and denies IP literals in loopback/private/link-local/metadata/multicast/unspecified/broadcast/ shared (CGNAT) ranges for both IPv4 and IPv6 (including IPv4-mapped spellings).
- Dial layer: a custom DialContext resolves the host, filters every resolved address through the same address-class rules, and dials only an approved resolved address directly — a DNS rebinding between validation and dial cannot reach a denied address.
- Redirect layer: CheckRedirect revalidates every redirect target URL through ValidateURL and caps the hop count.
All fields are admin/server configuration. Request data can never widen the policy.
func DefaultOutboundPolicy ¶
func DefaultOutboundPolicy() OutboundPolicy
DefaultOutboundPolicy returns the strict server policy: HTTPS on port 443 only, no approved hosts (outbound LLM disabled until an administrator approves a destination), loopback denied.
func (*OutboundPolicy) ApproveDestination ¶
func (p *OutboundPolicy) ApproveDestination(rawURL string) error
ApproveDestination adds the host and explicit port of a trusted (administrator-configured) HTTP(S) destination URL to the allowlists. It rejects userinfo and non-HTTP(S) schemes, and must be called before NewServiceWithPolicy normalizes the policy. Approval only widens the host/port allowlist: every request is still validated against the full policy (scheme, port, address class, redirects, dial targets).
func (*OutboundPolicy) ValidateURL ¶
func (p *OutboundPolicy) ValidateURL(u *url.URL) error
ValidateURL enforces the URL layer of the outbound policy. Rejection errors wrap ErrUnsafeDestination with static, redacted messages.
type Service ¶
type Service struct {
// contains filtered or unexported fields
}
Service provides extraction and synthesis capabilities.
func NewService ¶
NewService creates a new extraction service. The outbound policy is derived from the (trusted, admin-provided) configuration: its destination host (and explicit port) is approved into the default policy. The URL-layer policy still validates scheme, port, and address class at request time, so plain HTTP or loopback/private destinations remain rejected unless the policy carries explicit development switches — use NewServiceWithPolicy for those.
func NewServiceWithPolicy ¶
func NewServiceWithPolicy(cfg Config, policy OutboundPolicy) *Service
NewServiceWithPolicy creates a new extraction service with an explicit outbound destination policy. This is the server-mode constructor: the policy is composed by the server, never from request data. It performs NO implicit approval — the composition must approve the trusted configured destination itself via ApproveDestination so an explicit policy approves exactly what its author intended.
func (*Service) Extract ¶
func (s *Service) Extract(ctx context.Context, req ExtractionRequest) (*ExtractionResult, error)
Extract processes raw text and extracts structured observations and graph edges.
func (*Service) Synthesize ¶
func (s *Service) Synthesize(ctx context.Context, req SynthesisRequest) (*SynthesisResult, error)
Synthesize consolidates observations into an overarching architectural summary.
type SynthesisRequest ¶
type SynthesisRequest struct {
Project string `json:"project"`
Observations []*domain.Observation `json:"observations"`
LLMConfig *Config `json:"llm_config,omitempty"`
}
SynthesisRequest represents a request to consolidate multiple observations.
type SynthesisResult ¶
type SynthesisResult struct {
Project string `json:"project"`
Summary string `json:"summary"`
KeyDecisions []string `json:"key_decisions"`
Patterns []string `json:"patterns"`
OpenIssues []string `json:"open_issues"`
SynthesizedAt time.Time `json:"synthesized_at"`
}
SynthesisResult represents the consolidated knowledge summary.
Source Files