sources

package
v0.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Sep 5, 2025 License: AGPL-3.0 Imports: 6 Imported by: 0

README

sources

Package sources provides abstractions for fetching AI model catalog data from various external sources including provider APIs and community repositories.

sources

import "github.com/agentstation/starmap/pkg/sources"

Package sources provides public APIs for working with AI model data sources.

Package sources defines interfaces and types for catalog data sources. Sources are responsible for fetching and synchronizing model data from various providers including local files, provider APIs, and external repositories.

The package provides a unified interface for different data sources while supporting merge strategies, authorities for data precedence, and flexible configuration options.

Example usage:

// Create a provider fetcher
fetcher := NewProviderFetcher()

// Fetch models from a provider
models, err := fetcher.FetchModels(ctx, provider)
if err != nil {
    log.Fatal(err)
}

// Check if a provider is supported
if fetcher.HasClient(providerID) {
    // Provider has a client implementation
}

Index

type ProviderFetcher

ProviderFetcher provides operations for fetching models from provider APIs. This is the public API for external packages to interact with provider data.

type ProviderFetcher struct {
    // contains filtered or unexported fields
}

func NewProviderFetcher
func NewProviderFetcher(opts ...ProviderOption) *ProviderFetcher

NewProviderFetcher creates a new provider fetcher for interacting with provider APIs. It provides a clean public interface for external packages.

func (*ProviderFetcher) FetchModels
func (pf *ProviderFetcher) FetchModels(ctx context.Context, provider *catalogs.Provider, opts ...ProviderOption) ([]catalogs.Model, error)

FetchModels fetches available models from a single provider's API. It handles credential loading, client creation, and API communication.

Example:

fetcher := NewProviderFetcher()
models, err := fetcher.FetchModels(ctx, provider)

With options:

fetcher := NewProviderFetcher(WithTimeout(30 * time.Second))
models, err := fetcher.FetchModels(ctx, provider, WithAllowMissingAPIKey())

func (*ProviderFetcher) FetchRawResponse
func (pf *ProviderFetcher) FetchRawResponse(ctx context.Context, provider *catalogs.Provider, endpoint string, opts ...ProviderOption) ([]byte, error)

FetchRawResponse fetches the raw API response from a provider's endpoint. This is useful for testing, debugging, or saving raw responses as testdata.

The endpoint parameter should be the full URL to the API endpoint. The response is returned as raw bytes (JSON) without any parsing.

func (*ProviderFetcher) HasClient
func (pf *ProviderFetcher) HasClient(id catalogs.ProviderID) bool

HasClient checks if a provider has a client implementation available. This can be used to determine which providers are supported.

func (*ProviderFetcher) List
func (pf *ProviderFetcher) List() []catalogs.ProviderID

List returns all provider IDs that have client implementations. This is useful for discovering which providers can be used with FetchModels.

type ProviderOption

ProviderOption configures ProviderFetcher behavior

type ProviderOption func(*providerOptions)

func WithAllowMissingAPIKey
func WithAllowMissingAPIKey() ProviderOption

WithAllowMissingAPIKey allows operations even when API key is not configured. Useful for checking provider support without credentials.

func WithTimeout
func WithTimeout(d time.Duration) ProviderOption

WithTimeout sets a timeout for provider operations. The timeout applies to the context passed to FetchModels.

func WithoutCredentialLoading
func WithoutCredentialLoading() ProviderOption

WithoutCredentialLoading disables automatic credential loading from environment. Use this when credentials are already loaded or when testing.

type Source

Source represents a data source for catalog information

type Source interface {
    // Name returns the name of this source
    Name() SourceName

    // Setup initializes the source with dependencies (called once before Fetch)
    Setup(providers *catalogs.Providers) error

    // Fetch retrieves data from this source
    // Sources handle their own concurrency internally
    Fetch(ctx context.Context, opts ...SourceOption) (catalogs.Catalog, error)

    // Cleanup releases any resources (called after all Fetch operations)
    Cleanup() error
}

type SourceName

SourceName represents the name/type of a data source

type SourceName string

Common source names

const (
    ProviderAPI   SourceName = "Provider APIs"
    ModelsDevGit  SourceName = "models.dev (git)"
    ModelsDevHTTP SourceName = "models.dev (http)"
    LocalCatalog  SourceName = "Local Catalog"
)

func (SourceName) String
func (sn SourceName) String() string

String returns the string representation of a source name

type SourceOption

SourceOption is a function that configures sourceOptions

type SourceOption func(*sourceOptions)

func WithFresh
func WithFresh(fresh bool) SourceOption

WithFresh configures fresh sync mode for sources

func WithProviderFilter
func WithProviderFilter(providerID catalogs.ProviderID) SourceOption

WithProviderFilter configures filtering for a specific provider

func WithSafeMode
func WithSafeMode(safeMode bool) SourceOption

WithSafeMode configures safe mode for sources

func WithSourceContext
func WithSourceContext(key string, value any) SourceOption

WithSourceContext adds source-specific context data

Generated by gomarkdoc

Documentation

Overview

Package sources provides public APIs for working with AI model data sources.

Package sources defines interfaces and types for catalog data sources. Sources are responsible for fetching and synchronizing model data from various providers including local files, provider APIs, and external repositories.

The package provides a unified interface for different data sources while supporting merge strategies, authorities for data precedence, and flexible configuration options.

Example usage: 

// Create a provider fetcher
fetcher := NewProviderFetcher()

// Fetch models from a provider
models, err := fetcher.FetchModels(ctx, provider)
if err != nil {
    log.Fatal(err)
}

// Check if a provider is supported
if fetcher.HasClient(providerID) {
    // Provider has a client implementation
}

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Option 

type Option func(*Options)

Option is a function that configures options.

func WithCleanupRepo 

func WithCleanupRepo(cleanup bool) Option

WithCleanupRepo configures whether to clean up temporary repositories after fetch.

func WithFresh 

func WithFresh(fresh bool) Option

WithFresh configures fresh sync mode for sources.

func WithProviderFilter 

func WithProviderFilter(providerID catalogs.ProviderID) Option

WithProviderFilter configures filtering for a specific provider.

func WithReformat 

func WithReformat(reformat bool) Option

WithReformat configures whether to reformat output files.

func WithSafeMode 

func WithSafeMode(safeMode bool) Option

WithSafeMode configures safe mode for sources.

type Options 

type Options struct {
	// Provider filtering (needed by provider source)
	ProviderID *catalogs.ProviderID

	// Behavior flags (needed by various sources)
	Fresh    bool // Fresh sync (delete existing before adding)
	SafeMode bool // Don't delete models, only add/update

	// Typed source-specific options
	CleanupRepo bool // For models.dev git source - remove repo after fetch
	Reformat    bool // For file-based sources - reformat output files
}

Options is the configuration for sources.

func ApplyOptions 

func ApplyOptions(opts ...Option) *Options

ApplyOptions applies a set of options to create configured sourceOptions This is a helper for sources to use internally.

type ProviderFetcher 

type ProviderFetcher struct {
	// contains filtered or unexported fields
}

ProviderFetcher provides operations for fetching models from provider APIs. This is the public API for external packages to interact with provider data.

func NewProviderFetcher 

func NewProviderFetcher(opts ...ProviderOption) *ProviderFetcher

NewProviderFetcher creates a new provider fetcher for interacting with provider APIs. It provides a clean public interface for external packages.

func (*ProviderFetcher) FetchModels 

func (pf *ProviderFetcher) FetchModels(ctx context.Context, provider *catalogs.Provider, opts ...ProviderOption) ([]catalogs.Model, error)

FetchModels fetches available models from a single provider's API. It handles credential loading, client creation, and API communication.

Example: 

fetcher := NewProviderFetcher()
models, err := fetcher.FetchModels(ctx, provider)

With options: 

fetcher := NewProviderFetcher(WithTimeout(30 * time.Second))
models, err := fetcher.FetchModels(ctx, provider, WithAllowMissingAPIKey())

func (*ProviderFetcher) FetchRawResponse 

func (pf *ProviderFetcher) FetchRawResponse(ctx context.Context, provider *catalogs.Provider, endpoint string, opts ...ProviderOption) ([]byte, error)

FetchRawResponse fetches the raw API response from a provider's endpoint. This is useful for testing, debugging, or saving raw responses as testdata.

The endpoint parameter should be the full URL to the API endpoint. The response is returned as raw bytes (JSON) without any parsing.

func (*ProviderFetcher) HasClient 

func (pf *ProviderFetcher) HasClient(id catalogs.ProviderID) bool

HasClient checks if a provider has a client implementation available. This can be used to determine which providers are supported.

func (*ProviderFetcher) List 

func (pf *ProviderFetcher) List() []catalogs.ProviderID

List returns all provider IDs that have client implementations. This is useful for discovering which providers can be used with FetchModels.

type ProviderOption 

type ProviderOption func(*providerOptions)

ProviderOption configures ProviderFetcher behavior.

func WithAllowMissingAPIKey 

func WithAllowMissingAPIKey() ProviderOption

WithAllowMissingAPIKey allows operations even when API key is not configured. Useful for checking provider support without credentials.

func WithTimeout 

func WithTimeout(d time.Duration) ProviderOption

WithTimeout sets a timeout for provider operations. The timeout applies to the context passed to FetchModels.

func WithoutCredentialLoading 

func WithoutCredentialLoading() ProviderOption

WithoutCredentialLoading disables automatic credential loading from environment. Use this when credentials are already loaded or when testing.

type ResourceType 

type ResourceType string

ResourceType identifies the type of resource being merged. 

const (
	// ResourceTypeModel represents a model resource.
	ResourceTypeModel ResourceType = "model"
	// ResourceTypeProvider represents a provider resource.
	ResourceTypeProvider ResourceType = "provider"
	// ResourceTypeAuthor represents an author resource.
	ResourceTypeAuthor ResourceType = "author"
)

type Source 

type Source interface {
	// Type returns the type of this source
	Type() Type

	// Setup initializes the source with dependencies (called once before Fetch)
	Setup(providers *catalogs.Providers) error

	// Fetch retrieves data from this source
	// Sources handle their own concurrency internally
	Fetch(ctx context.Context, opts ...Option) error

	// Catalog returns the catalog of this source
	Catalog() catalogs.Catalog

	// Cleanup releases any resources (called after all Fetch operations)
	Cleanup() error
}

Source represents a data source for catalog information.

type Type 

type Type string

Type represents the type/name of a data source. 

const (
	ProviderAPI   Type = "Provider APIs"
	ModelsDevGit  Type = "models.dev (git)"
	ModelsDevHTTP Type = "models.dev (http)"
	LocalCatalog  Type = "Local Catalog"
)

Common source names.

func (Type) String 

func (sn Type) String() string

String returns the string representation of a source name.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.