Documentation
¶
Overview ¶
Package seeder provides library documentation seeding for Trace.
The seeder extracts documentation from project dependencies and indexes them into Weaviate for context assembly. It supports Go modules initially, with extensibility for Python and TypeScript.
Index ¶
- Constants
- Variables
- func DeleteByDataSpace(ctx context.Context, client *weaviate.Client, dataSpace string) error
- func EnsureSchema(ctx context.Context, client *weaviate.Client) error
- func GenerateDocID(library, version, symbolPath string) string
- func GetLibraryDocSchema() *models.Class
- func IndexDocs(ctx context.Context, client *weaviate.Client, docs []LibraryDoc) (int, error)
- func ResolveLocalPath(dep Dependency) (string, error)
- func ValidateProjectRoot(projectRoot string) error
- type Dependency
- type LibraryDoc
- type SeedResult
- type Seeder
- type SeederConfig
Constants ¶
const BatchSize = 100
BatchSize is the number of documents to batch import at once.
const LibraryDocClassName = "LibraryDoc"
LibraryDocClassName is the Weaviate class name for library documentation.
const MaxFilesPerModule = 500
MaxFilesPerModule limits how many files to parse per module.
Variables ¶
var ( // ErrNoGoMod indicates no go.mod file was found. ErrNoGoMod = errors.New("go.mod not found") // ErrModuleNotCached indicates the module is not in the local cache. ErrModuleNotCached = errors.New("module not in local cache") // ErrUnsupportedLanguage indicates an unsupported language. ErrUnsupportedLanguage = errors.New("unsupported language") // ErrEmptyProjectRoot indicates projectRoot is empty. ErrEmptyProjectRoot = errors.New("projectRoot must not be empty") // ErrRelativeProjectRoot indicates projectRoot is not absolute. ErrRelativeProjectRoot = errors.New("projectRoot must be absolute path") // ErrPathTraversal indicates projectRoot contains path traversal. ErrPathTraversal = errors.New("projectRoot must not contain path traversal") // ErrNilClient indicates the Weaviate client is nil. ErrNilClient = errors.New("weaviate client must not be nil") )
Sentinel errors for the seeder package.
Functions ¶
func DeleteByDataSpace ¶
DeleteByDataSpace removes all library docs for a data space.
Description:
Deletes all LibraryDoc objects matching the given data space. Used for cleanup when a project is removed.
Inputs:
ctx - Context for cancellation client - Weaviate client dataSpace - Data space to delete
Outputs:
error - Non-nil if deletion fails
func EnsureSchema ¶
EnsureSchema creates the LibraryDoc class if it doesn't exist.
Description:
Checks if the LibraryDoc class exists in Weaviate and creates it if not. This operation is idempotent.
Inputs:
ctx - Context for cancellation client - Weaviate client
Outputs:
error - Non-nil if schema creation fails
func GenerateDocID ¶
GenerateDocID creates a unique ID for a library doc.
func GetLibraryDocSchema ¶
GetLibraryDocSchema returns the Weaviate schema for LibraryDoc class.
func IndexDocs ¶
IndexDocs batch imports library documentation into Weaviate.
Description:
Imports documents in batches for efficiency. Handles duplicates gracefully by using the docId as a deterministic ID.
Inputs:
ctx - Context for cancellation client - Weaviate client docs - Documents to index
Outputs:
int - Number of documents successfully indexed error - Non-nil if batch import fails
func ResolveLocalPath ¶
func ResolveLocalPath(dep Dependency) (string, error)
ResolveLocalPath finds the local cache path for a Go module.
Description:
Locates the module in the Go module cache. Handles path escaping for modules with capital letters or special characters.
Inputs:
dep - The dependency to resolve
Outputs:
string - Absolute path to the cached module error - ErrModuleNotCached if not found
func ValidateProjectRoot ¶
ValidateProjectRoot validates that a project root path is safe to use.
Description:
Validates the project root path for security. Rejects empty paths, relative paths, and paths containing traversal sequences.
Inputs:
projectRoot - Path to validate
Outputs:
error - Non-nil if validation fails
Types ¶
type Dependency ¶
type Dependency struct {
// ModulePath is the module path (e.g., "github.com/gin-gonic/gin").
ModulePath string `json:"module_path"`
// Version is the module version (e.g., "v1.9.1").
Version string `json:"version"`
// Language is the dependency language ("go", "python", "typescript").
Language string `json:"language"`
// LocalPath is the resolved local cache path.
LocalPath string `json:"local_path"`
// Indirect indicates if this is an indirect dependency.
Indirect bool `json:"indirect"`
}
Dependency represents a project dependency.
func FilterDirectDependencies ¶
func FilterDirectDependencies(deps []Dependency) []Dependency
FilterDirectDependencies returns only direct (non-indirect) dependencies.
func ParseGoMod ¶
func ParseGoMod(content []byte) ([]Dependency, error)
ParseGoMod parses a go.mod file and returns dependencies.
Description:
Uses the official Go module parser to extract require statements. Skips indirect dependencies to keep scope manageable.
Inputs:
content - Raw go.mod file content
Outputs:
[]Dependency - List of direct dependencies error - Non-nil if parsing fails
func ResolveDependencies ¶
func ResolveDependencies(ctx context.Context, projectRoot string) ([]Dependency, error)
ResolveDependencies finds all dependencies for a project.
Description:
Parses the project's dependency manifest (go.mod for Go) and returns a list of dependencies with their resolved local cache paths.
Inputs:
ctx - Context for cancellation projectRoot - Absolute path to the project root
Outputs:
[]Dependency - List of resolved dependencies error - Non-nil if parsing fails
Errors:
ErrNoGoMod - No go.mod file found
type LibraryDoc ¶
type LibraryDoc struct {
// DocID is a unique identifier for this doc entry.
// Format: "{library}@{version}:{symbol_path}"
DocID string `json:"doc_id"`
// Library is the library/module path (e.g., "github.com/gin-gonic/gin").
Library string `json:"library"`
// Version is the library version (e.g., "v1.9.1").
Version string `json:"version"`
// SymbolPath is the fully qualified symbol path.
// Format: "package.SymbolName" (e.g., "gin.Context" or "gin.Context.JSON")
SymbolPath string `json:"symbol_path"`
// SymbolKind is the type of symbol ("function", "method", "type", "constant", "variable").
SymbolKind string `json:"symbol_kind"`
// Signature is the type signature (e.g., "func(code int, obj interface{})").
Signature string `json:"signature"`
// DocContent is the documentation text.
DocContent string `json:"doc_content"`
// Example is an optional code example.
Example string `json:"example,omitempty"`
// DataSpace is the Weaviate data space for isolation.
DataSpace string `json:"data_space"`
}
LibraryDoc represents documentation for a library symbol.
func ExtractDocs ¶
func ExtractDocs(ctx context.Context, dep Dependency, dataSpace string) ([]LibraryDoc, error)
ExtractDocs extracts documentation from a dependency's source files.
Description:
Parses Go source files to extract documentation for exported symbols. Uses the standard library's go/doc package for accurate extraction.
Inputs:
ctx - Context for cancellation dep - The dependency to extract docs from dataSpace - Weaviate data space for isolation
Outputs:
[]LibraryDoc - Extracted documentation entries error - Non-nil if extraction fails completely
func SearchDocs ¶
func SearchDocs(ctx context.Context, client *weaviate.Client, query, library, dataSpace string, limit int) ([]LibraryDoc, error)
SearchDocs searches library documentation in Weaviate.
Description:
Searches for library documentation matching the query. Supports filtering by library name and data space.
Inputs:
ctx - Context for cancellation client - Weaviate client query - Search query library - Optional library filter (empty = all libraries) dataSpace - Data space filter limit - Maximum number of results
Outputs:
[]LibraryDoc - Matching documents error - Non-nil if search fails
type SeedResult ¶
type SeedResult struct {
// DependenciesFound is the number of dependencies discovered.
DependenciesFound int `json:"dependencies_found"`
// DocsIndexed is the number of documentation entries indexed.
DocsIndexed int `json:"docs_indexed"`
// Errors contains non-fatal errors encountered during seeding.
Errors []string `json:"errors,omitempty"`
}
SeedResult contains the result of a seeding operation.
type Seeder ¶
type Seeder struct {
// contains filtered or unexported fields
}
Seeder handles library documentation seeding.
func NewSeeder ¶
func NewSeeder(client *weaviate.Client, config SeederConfig) (*Seeder, error)
NewSeeder creates a new library seeder.
Description:
Creates a Seeder configured for library documentation extraction and indexing.
Inputs:
client - Weaviate client. Must not be nil. config - Seeder configuration.
Outputs:
*Seeder - The configured seeder error - Non-nil if client is nil
Thread Safety: Seed() is safe for concurrent use. Multiple projects can be seeded concurrently as they use isolated data spaces.
func (*Seeder) Seed ¶
Seed extracts and indexes library documentation for a project.
Description:
Discovers dependencies from the project, extracts documentation from each, and indexes into Weaviate. Uses a deterministic data space based on the project root hash for multi-tenant isolation.
Inputs:
ctx - Context for cancellation projectRoot - Absolute path to the project root dataSpace - Optional override for data space (uses hash if empty)
Outputs:
*SeedResult - Seeding statistics error - Non-nil if seeding fails completely
type SeederConfig ¶
type SeederConfig struct {
// MaxConcurrent is the max concurrent doc extractions.
MaxConcurrent int
// Timeout is the overall seeding timeout.
Timeout time.Duration
// SkipIndirect skips indirect dependencies.
SkipIndirect bool
}
SeederConfig configures the seeder.
func DefaultSeederConfig ¶
func DefaultSeederConfig() SeederConfig
DefaultSeederConfig returns sensible defaults.
Source Files