sync

package
v0.2.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Apr 27, 2026 License: MIT Imports: 7 Imported by: 0

Documentation

Overview

Package sync handles repository ingestion from GitHub.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func CalculateMaxCharsFromTokens added in v0.2.0 

func CalculateMaxCharsFromTokens(maxTokens int) int

CalculateMaxCharsFromTokens calculates the maximum characters based on token limit. Uses a conservative ratio of ~3.3 characters per token to stay safely under limits. This accounts for worst-case scenarios with special characters and encoding overhead.

func IngestRepos 

func IngestRepos(ctx context.Context, opts IngestOptions) <-chan IngestEvent

IngestRepos runs the full ingestion pipeline. Events are sent to the returned channel as repos are processed. The channel is closed when ingestion completes. Never panics — all errors are sent as IngestEvent{Type: "error"}.

Types

type IngestEvent 

type IngestEvent struct {
	Type    string // "repo", "skip", "error", "done"
	Repo    string
	Status  string // "new" or "updated" for type=="repo"
	Reason  string // for type=="skip" or "error"
	Total   int
	Skipped int
	Errors  int
}

IngestEvent represents a progress event from the ingestion pipeline.

type IngestOptions 

type IngestOptions struct {
	IncludeOwned   bool
	IncludeStarred bool
	FullTree       bool
	MaxChunkSize   int // Maximum characters per chunk (calculated from embedding model token limit)
	DB             *sql.DB
	GitHubPAT      string
}

IngestOptions configures the ingestion pipeline.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.