ingest

package
v0.0.10 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 27, 2026 License: MIT Imports: 16 Imported by: 0

Documentation

Overview

Package ingest orchestrates multi-source content ingestion into knowledge base topics, handling frontmatter assembly, raw file writes, and operation log entries.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ExistingYouTubeVideoIDs added in v0.0.10 

func ExistingYouTubeVideoIDs(vaultPath, topicRef string) (map[string]struct{}, error)

ExistingYouTubeVideoIDs returns the set of YouTube video IDs already ingested under the topic's raw/youtube directory. It powers resumable channel ingestion by reading each raw document's video_id frontmatter field and the id embedded in its source_url. A missing raw/youtube directory yields an empty set.

func Ingest 

func Ingest(ctx context.Context, options Options) (models.IngestResult, error)

Ingest validates the target topic, optionally converts the source, writes the raw markdown document, and appends a log entry.

Types

type Options 

type Options struct {
	VaultPath        string
	Topic            string
	SourceKind       models.SourceKind
	SourcePath       string
	SourceURL        string
	Title            string
	Markdown         string
	ExtraFrontmatter map[string]any
	ConvertOptions   map[string]any
	Registry         Registry
	ScrapedAt        time.Time
}

Options configures a single-source ingest run.

type Registry 

type Registry interface {
	Convert(ctx context.Context, input models.ConvertInput) (*models.ConvertResult, error)
}

Registry converts file-backed inputs into markdown content.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.