ingest

package
v0.12.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jul 19, 2026 License: MIT Imports: 3 Imported by: 0

Documentation

Overview

@index Consumer-owned transaction ports for graph ingest application workflows.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func FilePackagesFromContext

func FilePackagesFromContext(ctx context.Context) map[string]string

FilePackagesFromContext loads canonical file package hints for parser adapters. @intent let parser adapters seed qualified names from application-owned file context.

func ImportPackagesFromContext

func ImportPackagesFromContext(ctx context.Context) map[string]string

ImportPackagesFromContext loads repository import-path normalization hints. @intent let parser adapters consume application-owned package context without reversing dependencies.

func WithFilePackages

func WithFilePackages(ctx context.Context, packages map[string]string) context.Context

WithFilePackages stores canonical import paths keyed by repository-relative file path. @intent provide deterministic package prefixes for languages without package declarations.

func WithImportPackages

func WithImportPackages(ctx context.Context, packages map[string]string) context.Context

WithImportPackages stores repository import-path normalization hints for parser adapters. @intent thread parser-neutral package names through build and update calls without adapter-specific APIs.

Types

type AnnotatingParser

type AnnotatingParser interface {
	Parser
	ParseWithComments(ctx context.Context, filePath string, content []byte) ([]graph.Node, []graph.Edge, []CommentBlock, error)
	Language() string
}

AnnotatingParser adds comments and language identity for annotation restoration. @intent make comment-aware parsing an optional ingest capability.

type BatchIncrementalSyncer

type BatchIncrementalSyncer interface {
	SyncBatchesWithExisting(ctx context.Context, source FileBatchSource, deletedFiles []string) (*SyncStats, error)
}

BatchIncrementalSyncer supports a single staged reconciliation across multiple input batches. @intent prevent batch order from affecting cross-file edge resolution during large updates. @domainRule deletedFiles must contain only paths absent from the supplied source batches.

type CommentBlock

type CommentBlock struct {
	StartLine      int
	EndLine        int
	Text           string
	IsDocstring    bool
	OwnerStartLine int
}

CommentBlock is parser-neutral documentation text associated with source lines. @intent carry comments and docstring ownership from parser adapters into ingest binding policy.

type FileBatchSource

type FileBatchSource func(visitor FileBatchVisitor) error

FileBatchSource replays the current source snapshot in bounded batches. @intent let workflow retain source spooling while incremental reconciliation controls node and edge phases.

type FileBatchVisitor

type FileBatchVisitor func(files map[string]FileInfo) error

FileBatchVisitor receives one bounded source batch during staged incremental reconciliation. @intent keep bulk update input streaming while allowing a syncer to own cross-batch ordering.

type FileInfo

type FileInfo struct {
	Hash    string
	Content []byte
	Force   bool
}

FileInfo carries one incremental source snapshot and its change-detection metadata. @intent keep incremental update inputs owned by ingest rather than a concrete sync implementation.

type GraphStore

type GraphStore interface {
	GetNodesByIDs(ctx context.Context, ids []uint) ([]graph.Node, error)
	GetNodesByFile(ctx context.Context, filePath string) ([]graph.Node, error)
	GetNodesByFiles(ctx context.Context, filePaths []string) (map[string][]graph.Node, error)
	GetNodesByQualifiedNames(ctx context.Context, names []string) (map[string][]graph.Node, error)
	ListFileNodes(ctx context.Context) ([]graph.Node, error)
	ListImportFileNodes(ctx context.Context) ([]graph.Node, error)
	GetFileNodesByPathSuffix(ctx context.Context, suffix string) ([]graph.Node, error)
	GetEdgesFromNodes(ctx context.Context, nodeIDs []uint) ([]graph.Edge, error)
	GetEdgesToNodes(ctx context.Context, nodeIDs []uint) ([]graph.Edge, error)
	UpsertNodes(ctx context.Context, nodes []graph.Node) error
	UpsertEdges(ctx context.Context, edges []graph.Edge) error
	UpsertAnnotations(ctx context.Context, annotations []*graph.Annotation) error
	DeleteNodesByFiles(ctx context.Context, filePaths []string) error
	DeleteEdgesByFile(ctx context.Context, filePath string) error
	DeletePackageSemanticEdges(ctx context.Context, anchorFiles []string) error
	DeleteGraph(ctx context.Context) error
}

GraphStore is the transaction-scoped graph persistence surface required by build and update workflows. @intent keep ingest graph reads and writes inside the unit-of-work boundary without exposing a persistence implementation.

type IncrementalSyncer

type IncrementalSyncer interface {
	SyncWithExisting(ctx context.Context, files map[string]FileInfo, existingFiles []string) (*SyncStats, error)
}

IncrementalSyncer reconciles one current source snapshot against known files. @intent let ingest orchestrate batching and deletion policy through an implementation-neutral sync seam.

type MetadataParser

type MetadataParser interface {
	AnnotatingParser
	ParseWithCommentsAndMetadata(ctx context.Context, filePath string, content []byte) ([]graph.Node, []graph.Edge, []CommentBlock, ParseMetadata, error)
}

MetadataParser adds package-level parse metadata used by full and incremental semantic refreshes. @intent expose package/interface metadata without coupling ingest to parser adapter structs.

type PackageContext

type PackageContext struct {
	Package        string
	Language       string
	Files          []string
	Nodes          []graph.Node
	Interfaces     []PackageInterfaceInfo
	ImportPackages map[string]string
}

PackageContext contains the package-wide graph state used to derive semantic edges. @intent let parser adapters enrich multi-file packages without leaking AST types into ingest.

type PackageDiscoverer

type PackageDiscoverer interface {
	DiscoverPackages(ctx context.Context, opts PackageDiscoveryOptions) (map[string]PackageInfo, error)
}

PackageDiscoverer is an optional parser capability for repository-level package discovery. @intent delegate language-specific package discovery while ingest owns traversal policy.

type PackageDiscoveryOptions

type PackageDiscoveryOptions struct {
	RootDir   string
	WalkFiles func(func(path, relPath string) error) error
	HasParser func(ext string) bool
}

PackageDiscoveryOptions supplies traversal policy to parser package discovery. @intent reuse ingest include/exclude and parser-registration policy during language-specific package discovery.

type PackageEdgeBuilder

type PackageEdgeBuilder interface {
	PackageEdges(ctx PackageContext) []graph.Edge
}

PackageEdgeBuilder is an optional parser capability for multi-file semantic relationships. @intent derive language-specific package edges through a parser-neutral ingest contract.

type PackageInfo

type PackageInfo struct {
	ImportPath string
	Name       string
	Dir        string
	Language   string
	Files      []string
}

PackageInfo describes one source package discovered by a parser adapter. @intent let ingest create package nodes and membership edges without knowing language-specific discovery details.

type PackageInterfaceInfo

type PackageInterfaceInfo struct {
	Name    string
	Methods []string
}

PackageInterfaceInfo describes one parsed interface and its declared methods. @intent preserve package-level implementation inference without exposing parser implementation types.

type ParseCache

type ParseCache interface {
	LoadParseResult(ctx context.Context, key ParseCacheKey) ([]byte, bool, error)
	StoreParseResult(ctx context.Context, key ParseCacheKey, payload []byte) error
}

ParseCache stores opaque serialized parse results outside the graph transaction. @intent make repeated full builds skip Tree-sitter work without coupling ingest to one cache backend. @domainRule cache failures are optimization misses and must not fail graph builds.

type ParseCacheKey

type ParseCacheKey struct {
	FilePath      string
	SourceHash    string
	ParserVersion string
	ContextHash   string
}

ParseCacheKey identifies one correctness-equivalent parsed source result. @intent include every input known to affect parser output instead of trusting source content alone.

type ParseMetadata

type ParseMetadata struct {
	Package    string
	Interfaces []PackageInterfaceInfo
}

ParseMetadata carries package-level enrichment inputs produced by a parser. @intent let ingest coordinate package semantics through parser-owned metadata.

type Parser

type Parser interface {
	Parse(filePath string, content []byte) ([]graph.Node, []graph.Edge, error)
	ParseWithContext(ctx context.Context, filePath string, content []byte) ([]graph.Node, []graph.Edge, error)
}

Parser is the minimum source parser consumed by build and update workflows. @intent parse source into domain graph values without exposing Tree-sitter or another parser implementation. @domainRule full builds may invoke one Parser instance concurrently, so ParseWithContext implementations must isolate mutable parser state.

type SearchWriter

type SearchWriter interface {
	RebuildAll(ctx context.Context) error
	RebuildNodes(ctx context.Context, nodeIDs []uint) error
}

SearchWriter updates derived search state inside the same transaction as graph mutations. @intent expose full and scoped search rebuilds as indivisible application operations. @domainRule implementations must combine search-document refresh and backend index rebuild so callers cannot commit only one half.

type SyncStats

type SyncStats struct {
	Added      int
	Modified   int
	Skipped    int
	Deleted    int
	Unresolved resolve.FilterResolvedDiagnostics
}

SyncStats reports incremental file outcomes and unresolved-edge diagnostics. @intent expose update results without coupling callers to the incremental implementation package.

type Transaction

type Transaction interface {
	Graph() GraphStore
	Search() SearchWriter
}

Transaction exposes the graph and search capabilities participating in one atomic ingest operation. @intent give an ingest callback transaction-scoped capabilities without exposing a raw database handle.

type TransactionalBatchIncrementalSyncer

type TransactionalBatchIncrementalSyncer interface {
	SyncBatchesWithExistingStore(ctx context.Context, graphStore GraphStore, source FileBatchSource, deletedFiles []string) (*SyncStats, error)
}

TransactionalBatchIncrementalSyncer runs staged reconciliation against the active graph transaction. @intent preserve one atomic graph and search transaction while reconciling streamed update batches.

type TransactionalIncrementalSyncer

type TransactionalIncrementalSyncer interface {
	SyncWithExistingStore(ctx context.Context, graphStore GraphStore, files map[string]FileInfo, existingFiles []string) (*SyncStats, error)
}

TransactionalIncrementalSyncer runs incremental reconciliation against the active graph transaction. @intent keep incremental graph mutations inside the same unit of work as package and search updates.

type UnitOfWork

type UnitOfWork interface {
	WithinTransaction(ctx context.Context, fn func(Transaction) error) error
}

UnitOfWork owns the atomic boundary for graph persistence and derived search updates. @intent commit graph and search changes together only when the callback succeeds. @ensures a callback error rolls back both graph and search changes.

type UnresolvedEdgeStore

type UnresolvedEdgeStore interface {
	UpsertUnresolvedEdges(ctx context.Context, candidates []graph.UnresolvedEdgeCandidate) error
	FindUnresolvedEdgesByLookupKeys(ctx context.Context, keys []string) ([]graph.Edge, error)
	FindUnresolvedEdgesByFiles(ctx context.Context, filePaths []string) ([]graph.Edge, error)
	DeleteUnresolvedEdgesByFingerprints(ctx context.Context, fingerprints []string) error
	UnresolvedIndexReady(ctx context.Context, version string) (bool, error)
	MarkUnresolvedIndexReady(ctx context.Context, version string) error
}

UnresolvedEdgeStore persists and replays unresolved syntax relationships for semi-naive updates. @intent select unchanged source edges affected by newly added symbols without exposing persistence details.

type VersionedParser

type VersionedParser interface {
	Parser
	ParseCacheVersion() string
}

VersionedParser exposes the semantic parser/query identity used for parse-result cache invalidation. @intent let ingest reuse parsed output only while parser behavior and embedded queries remain compatible.

Directories

Path Synopsis
@index Session-local parsed-edge spool for staged incremental reconciliation.
@index Session-local parsed-edge spool for staged incremental reconciliation.
@index Language-specific dispatch hooks used by call edge resolution.
@index Language-specific dispatch hooks used by call edge resolution.
@index Full graph build pipeline: spool, batch persistence, and edge resolution.
@index Full graph build pipeline: spool, batch persistence, and edge resolution.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL