Documentation
¶
Overview ¶
Package document defines the canonical serializable Document content value shared by extraction, retrieval, and model-facing components.
NewDocument requires text, media, or both. Metadata is JSON-safe and belongs to the document itself; query-specific relevance belongs to the vector-store match value. Clone provides an independent snapshot when ownership crosses a component boundary. Extraction, formatting, splitting, identifier assignment, batching, and loading policy live in the separate etl module.
Example ¶
package main
import (
"fmt"
"github.com/Tangerg/scope/core/document"
)
func main() {
doc, err := document.NewDocument("Scope are wild cats.", nil)
if err != nil {
panic(err)
}
doc.ID = "doc-1"
if setErr := doc.Metadata.Set("source", "field-guide"); setErr != nil {
panic(setErr)
}
source, _, err := doc.Metadata.Decode[string]("source")
if err != nil {
panic(err)
}
fmt.Println(doc.ID, source)
}
Output: doc-1 field-guide
Index ¶
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrInvalidDocument = errors.New("document: invalid document")
Functions ¶
This section is empty.
Types ¶
type Document ¶
type Document struct {
ID string `json:"id,omitempty"`
// Text is the textual content. May be empty if Media is set.
Text string `json:"text,omitempty"`
Media *media.Media `json:"media,omitempty"`
Metadata metadata.Map `json:"metadata,omitzero"`
}
Document is the canonical content carrier. It holds identity, content, and metadata; query-specific relationships and runtime policies live outside this value. Clone recursively snapshots media and metadata so indexing and retrieval boundaries never retain caller-owned mutable buffers.
func NewDocument ¶
NewDocument requires at least one of text or media. A document with neither carries nothing to rank or read, and would travel through splitting, embedding, and retrieval before failing somewhere that no longer identifies where it was created. Whether the two describe the same thing is the caller's responsibility; only their presence is checked here.
Source Files