sources

sources

Package sources provides abstractions for fetching AI model catalog data from various external sources including provider APIs and community repositories.

sources

import "github.com/agentstation/starmap/pkg/sources"

Package sources provides public APIs for working with AI model data sources.

Package sources defines interfaces and types for catalog data sources. Sources are responsible for fetching and synchronizing model data from various providers including local files, provider APIs, and external repositories.

The package provides a unified interface for different data sources while supporting merge strategies, authorities for data precedence, and flexible configuration options.

Example usage:

// Create a provider fetcher
fetcher := NewProviderFetcher()

// Fetch models from a provider
models, err := fetcher.FetchModels(ctx, provider)
if err != nil {
    log.Fatal(err)
}

// Check if a provider is supported
if fetcher.HasClient(providerID) {
    // Provider has a client implementation
}

Index

• type Option
  • func WithCleanupRepo(cleanup bool) Option
  • func WithFresh(fresh bool) Option
  • func WithProviderFilter(providerID catalogs.ProviderID) Option
  • func WithReformat(reformat bool) Option
  • func WithSafeMode(safeMode bool) Option
• type Options
  • func ApplyOptions(opts ...Option) *Options
• type ProviderFetcher
  • func NewProviderFetcher(opts ...ProviderOption) *ProviderFetcher
  • func (pf *ProviderFetcher) FetchModels(ctx context.Context, provider *catalogs.Provider, opts ...ProviderOption) ([]catalogs.Model, error)
  • func (pf *ProviderFetcher) FetchRawResponse(ctx context.Context, provider *catalogs.Provider, endpoint string, opts ...ProviderOption) ([]byte, error)
  • func (pf *ProviderFetcher) HasClient(id catalogs.ProviderID) bool
  • func (pf *ProviderFetcher) List() []catalogs.ProviderID
• type ProviderOption
  • func WithAllowMissingAPIKey() ProviderOption
  • func WithTimeout(d time.Duration) ProviderOption
  • func WithoutCredentialLoading() ProviderOption
• type ResourceType
• type Source
• type Type
  • func (sn Type) String() string

type Option

Option is a function that configures options.

type Option func(*Options)

func WithCleanupRepo

func WithCleanupRepo(cleanup bool) Option

WithCleanupRepo configures whether to clean up temporary repositories after fetch.

func WithFresh

func WithFresh(fresh bool) Option

WithFresh configures fresh sync mode for sources.

func WithProviderFilter

func WithProviderFilter(providerID catalogs.ProviderID) Option

WithProviderFilter configures filtering for a specific provider.

func WithReformat

func WithReformat(reformat bool) Option

WithReformat configures whether to reformat output files.

func WithSafeMode

func WithSafeMode(safeMode bool) Option

WithSafeMode configures safe mode for sources.

type Options

Options is the configuration for sources.

type Options struct {
    // Provider filtering (needed by provider source)
    ProviderID *catalogs.ProviderID

    // Behavior flags (needed by various sources)
    Fresh    bool // Fresh sync (delete existing before adding)
    SafeMode bool // Don't delete models, only add/update

    // Typed source-specific options
    CleanupRepo bool // For models.dev git source - remove repo after fetch
    Reformat    bool // For file-based sources - reformat output files
}

func ApplyOptions

func ApplyOptions(opts ...Option) *Options

ApplyOptions applies a set of options to create configured sourceOptions This is a helper for sources to use internally.

type ProviderFetcher

ProviderFetcher provides operations for fetching models from provider APIs. This is the public API for external packages to interact with provider data.

type ProviderFetcher struct {
    // contains filtered or unexported fields
}

func NewProviderFetcher

func NewProviderFetcher(opts ...ProviderOption) *ProviderFetcher

NewProviderFetcher creates a new provider fetcher for interacting with provider APIs. It provides a clean public interface for external packages.

func (*ProviderFetcher) FetchModels

func (pf *ProviderFetcher) FetchModels(ctx context.Context, provider *catalogs.Provider, opts ...ProviderOption) ([]catalogs.Model, error)

FetchModels fetches available models from a single provider's API. It handles credential loading, client creation, and API communication.

Example:

fetcher := NewProviderFetcher()
models, err := fetcher.FetchModels(ctx, provider)

With options:

fetcher := NewProviderFetcher(WithTimeout(30 * time.Second))
models, err := fetcher.FetchModels(ctx, provider, WithAllowMissingAPIKey())

func (*ProviderFetcher) FetchRawResponse

func (pf *ProviderFetcher) FetchRawResponse(ctx context.Context, provider *catalogs.Provider, endpoint string, opts ...ProviderOption) ([]byte, error)

FetchRawResponse fetches the raw API response from a provider's endpoint. This is useful for testing, debugging, or saving raw responses as testdata.

The endpoint parameter should be the full URL to the API endpoint. The response is returned as raw bytes (JSON) without any parsing.

func (*ProviderFetcher) HasClient

func (pf *ProviderFetcher) HasClient(id catalogs.ProviderID) bool

HasClient checks if a provider has a client implementation available. This can be used to determine which providers are supported.

func (*ProviderFetcher) List

func (pf *ProviderFetcher) List() []catalogs.ProviderID

List returns all provider IDs that have client implementations. This is useful for discovering which providers can be used with FetchModels.

type ProviderOption

ProviderOption configures ProviderFetcher behavior.

type ProviderOption func(*providerOptions)

func WithAllowMissingAPIKey

func WithAllowMissingAPIKey() ProviderOption

WithAllowMissingAPIKey allows operations even when API key is not configured. Useful for checking provider support without credentials.

func WithTimeout

func WithTimeout(d time.Duration) ProviderOption

WithTimeout sets a timeout for provider operations. The timeout applies to the context passed to FetchModels.

func WithoutCredentialLoading

func WithoutCredentialLoading() ProviderOption

WithoutCredentialLoading disables automatic credential loading from environment. Use this when credentials are already loaded or when testing.

type ResourceType

ResourceType identifies the type of resource being merged.

type ResourceType string

const (
    // ResourceTypeModel represents a model resource.
    ResourceTypeModel ResourceType = "model"
    // ResourceTypeProvider represents a provider resource.
    ResourceTypeProvider ResourceType = "provider"
    // ResourceTypeAuthor represents an author resource.
    ResourceTypeAuthor ResourceType = "author"
)

type Source

Source represents a data source for catalog information.

type Source interface {
    // Type returns the type of this source
    Type() Type

    // Setup initializes the source with dependencies (called once before Fetch)
    Setup(providers *catalogs.Providers) error

    // Fetch retrieves data from this source
    // Sources handle their own concurrency internally
    Fetch(ctx context.Context, opts ...Option) error

    // Catalog returns the catalog of this source
    Catalog() catalogs.Catalog

    // Cleanup releases any resources (called after all Fetch operations)
    Cleanup() error
}

type Type

Type represents the type/name of a data source.

type Type string

Common source names.

const (
    ProviderAPI   Type = "Provider APIs"
    ModelsDevGit  Type = "models.dev (git)"
    ModelsDevHTTP Type = "models.dev (http)"
    LocalCatalog  Type = "Local Catalog"
)

func (Type) String

func (sn Type) String() string

String returns the string representation of a source name.

Generated by gomarkdoc