Help us shape the future of Go! Take the 2026 Go Developer Survey

datasource

package
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 12, 2026 License: Apache-2.0 Imports: 16 Imported by: 0

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func DeserializeInputFromJSON

func DeserializeInputFromJSON(key string) (*service.DataSourceInput, error)

DeserializeInputFromJSON deserializes a JSON cache key back into a DataSourceInput. This is used by groupcache when fetching on a remote server.

func NewDistributedCachingDataSource

func NewDistributedCachingDataSource(source service.DataSource, config DistributedCachingConfig) service.DataSource

NewDistributedCachingDataSource wraps a data source with distributed caching using groupcache. Returns the original source if it doesn't implement Cacheable.

Note: groupcache requires that you set up the peer pool before creating caching data sources. See groupcache documentation for details on setting up peers.

func NewInMemoryCachingDataSource

func NewInMemoryCachingDataSource(source service.DataSource, obs CacheObserver, opts ...InMemoryCachingDataSourceOption) service.DataSource

NewInMemoryCachingDataSource wraps a data source with in-memory caching if it implements Cacheable. obs is required when the source is cacheable; it is used on every Fetch. Returns the original source if it doesn't implement Cacheable (obs is unused in that case).

func NewStaticDataSource added in v0.2.0

func NewStaticDataSource(name string, data map[string]any) (service.DataSource, error)

NewStaticDataSource creates a data source that always returns data.

func SerializeInputToJSON

func SerializeInputToJSON(input *service.DataSourceInput) (string, error)

SerializeInputToJSON serializes a DataSourceInput to JSON (reversible). This is used for distributed caching where the key must be deserializable.

Types

type CacheFetchProbe

type CacheFetchProbe interface {
	CacheHit()
	CacheMiss()
	CacheExpired()
	FetchFailed(err error)
	End()
}

CacheFetchProbe tracks a single cache fetch invocation. Implementations should embed NoOpCacheFetchProbe for forward compatibility.

type CacheObserver

type CacheObserver interface {
	// CacheFetchStarted is called when a cache fetch begins.
	// Returns a potentially modified context and a probe to track the operation.
	CacheFetchStarted(ctx context.Context, dataSourceName string) (context.Context, CacheFetchProbe)
}

CacheObserver is called at key points during data source cache operations. Implementations should embed NoOpCacheObserver for forward compatibility with new methods added to this interface.

type CacheableLuaDataSource

type CacheableLuaDataSource struct {
	*LuaDataSource
}

CacheableLuaDataSource is a Lua data source that implements the Cacheable interface.

func NewCacheableLuaDataSource

func NewCacheableLuaDataSource(config CacheableLuaDataSourceConfig) (*CacheableLuaDataSource, error)

NewCacheableLuaDataSource creates a new cacheable Lua data source.

func (*CacheableLuaDataSource) CacheKey

CacheKey implements the Cacheable interface. A Lua nil (or non-table) from fetch_cache_key means skip the cache entirely.

type CacheableLuaDataSourceConfig

type CacheableLuaDataSourceConfig struct {
	// Name identifies this data source
	Name string

	// Script is the Lua script to execute
	//
	// The script should define a function called 'fetch' that takes an input table
	// and returns a result table with 'data' and 'content_type' fields
	//
	// The script must define fetch_cache_key(input), which returns a modified input
	// table with only the fields relevant for caching. Return nil to skip the cache
	// (no read, no write) for that request.
	//
	// Example:
	//   function fetch_cache_key(input)
	//     return {subject = {subject = input.subject.subject}}
	//   end
	Script string

	// ConfigSource provides configuration values available to the script via config.get()
	// If nil, an empty MapConfigSource will be used
	ConfigSource luaservices.ConfigSource

	// HTTP bundles the client and optional base URL for relative Lua URLs.
	HTTP httpclient.LuaClient

	// RequestOptions is a programmatic hook that augments outgoing HTTP requests. Optional.
	RequestOptions luaservices.RequestOptions

	// Observer for Lua-specific execution events on the inner Lua data source.
	// If nil, NewLuaDataSource substitutes NoOpDataSourceObserver{}.
	Observer LuaObserver
}

CacheableLuaDataSourceConfig configures a cacheable Lua data source

type DataSourceObserver

type DataSourceObserver interface {
	CacheObserver
	LuaObserver
}

DataSourceObserver is the per-package aggregate for all datasource observer interfaces.

type DistributedCachingConfig

type DistributedCachingConfig struct {
	// GroupName is the name for this groupcache group
	// Should be unique per data source type
	GroupName string

	// CacheSizeBytes is the maximum size of the cache in bytes
	// Default: 64MB
	CacheSizeBytes int64

	// Clock provides the current time for TTL bucketing.
	// Default: system clock.
	Clock clock.Clock

	// CacheTTL is the time-to-live for cached entries.
	// 0 means no TTL-based expiration (cache indefinitely).
	CacheTTL time.Duration
}

DistributedCachingConfig configures the distributed caching data source

type DistributedCachingDataSource

type DistributedCachingDataSource struct {
	// contains filtered or unexported fields
}

DistributedCachingDataSource wraps a cacheable data source with groupcache for distributed caching across multiple servers

func (*DistributedCachingDataSource) Fetch

Fetch checks the distributed cache first, then fetches from source on miss

func (*DistributedCachingDataSource) Name

Name forwards to the underlying data source

type InMemoryCachingDataSource

type InMemoryCachingDataSource struct {
	// contains filtered or unexported fields
}

InMemoryCachingDataSource wraps a cacheable data source with simple in-memory caching. It implements issuer.DataSource but not Cacheable (it does the caching itself).

func (*InMemoryCachingDataSource) Cleanup

func (c *InMemoryCachingDataSource) Cleanup()

Cleanup removes expired entries from the cache This should be called periodically to prevent memory leaks

func (*InMemoryCachingDataSource) Fetch

Fetch checks the cache first, then fetches from source on miss

func (*InMemoryCachingDataSource) Name

Name forwards to the underlying data source

func (*InMemoryCachingDataSource) Size

func (c *InMemoryCachingDataSource) Size() int

Size returns the number of entries in the cache (for debugging/monitoring)

type InMemoryCachingDataSourceOption

type InMemoryCachingDataSourceOption func(*InMemoryCachingDataSource)

InMemoryCachingDataSourceOption is a functional option for configuring InMemoryCachingDataSource

func WithCacheTTL added in v0.2.0

WithCacheTTL sets the time-to-live for cached entries. 0 means no TTL-based expiration (cache indefinitely).

func WithClock

WithClock sets the clock for the caching data source.

type LuaDataSource

type LuaDataSource struct {
	// contains filtered or unexported fields
}

LuaDataSource executes a Lua script to fetch data. The script has access to http, config, and json services.

The Lua script is compiled once at construction time; each call to LuaDataSource.Fetch loads the pre-compiled bytecode into a fresh LState, avoiding repeated parsing and compilation.

func NewLuaDataSource

func NewLuaDataSource(config LuaDataSourceConfig) (*LuaDataSource, error)

NewLuaDataSource creates a new Lua data source.

The script is compiled once during construction; see LuaDataSource for the runtime lifecycle.

func (*LuaDataSource) Fetch

Fetch executes the Lua script to fetch data

func (*LuaDataSource) Name

func (ds *LuaDataSource) Name() string

Name returns the data source name

type LuaDataSourceConfig

type LuaDataSourceConfig struct {
	// Name identifies this data source
	Name string

	// Script is the Lua script to execute
	// The script should define a function called 'fetch' that takes an input table
	// and returns a result table with 'data' and 'content_type' fields
	//
	// Example:
	//   function fetch(input)
	//     local response = http.get("https://api.example.com/user/" .. input.subject.subject)
	//     if response.status == 200 then
	//       return {data = response.body, content_type = "application/json"}
	//     end
	//     return nil
	//   end
	Script string

	// ConfigSource provides configuration values available to the script via config.get()
	// If nil, an empty MapConfigSource will be used
	ConfigSource luaservices.ConfigSource

	// HTTP bundles the client and optional base URL for relative Lua URLs.
	// Zero value uses [http.DefaultClient] with absolute URLs only.
	HTTP httpclient.LuaClient

	// RequestOptions is a programmatic hook that augments outgoing HTTP requests
	// (e.g. injecting headers or query params from Go context). Optional.
	RequestOptions luaservices.RequestOptions

	// Observer for Lua-specific execution events. If nil, defaults to NoOpObserver.
	Observer LuaObserver
}

LuaDataSourceConfig configures a Lua data source

type LuaFetchProbe

type LuaFetchProbe interface {
	ScriptLoadFailed(err error)
	ScriptExecutionFailed(err error)
	InvalidReturnType(got string)
	FetchCompleted()
	FetchCompletedNil()
	// FetchAudit is called when a script attaches optional audit metadata.
	FetchAudit(fields map[string]string)
	ResultConversionFailed(err error)
	End()
}

LuaFetchProbe tracks a single Lua fetch invocation. Implementations should embed NoOpLuaFetchProbe for forward compatibility.

type LuaObserver

type LuaObserver interface {
	// LuaFetchStarted is called when a Lua fetch begins.
	// Returns a potentially modified context and a probe to track the operation.
	LuaFetchStarted(ctx context.Context, dataSourceName string) (context.Context, LuaFetchProbe)
}

LuaObserver is called at key points during Lua data source operations. Implementations should embed NoOpLuaObserver for forward compatibility with new methods added to this interface.

type NoOpCacheFetchProbe

type NoOpCacheFetchProbe struct{}

NoOpCacheFetchProbe is a no-op implementation of CacheFetchProbe. Embed this in concrete probe types for forward compatibility.

func (NoOpCacheFetchProbe) CacheExpired

func (NoOpCacheFetchProbe) CacheExpired()

func (NoOpCacheFetchProbe) CacheHit

func (NoOpCacheFetchProbe) CacheHit()

func (NoOpCacheFetchProbe) CacheMiss

func (NoOpCacheFetchProbe) CacheMiss()

func (NoOpCacheFetchProbe) End

func (NoOpCacheFetchProbe) End()

func (NoOpCacheFetchProbe) FetchFailed

func (NoOpCacheFetchProbe) FetchFailed(error)

type NoOpCacheObserver

type NoOpCacheObserver struct{}

NoOpCacheObserver is a no-op implementation of CacheObserver. Embed this in concrete observer types for forward compatibility.

func (NoOpCacheObserver) CacheFetchStarted

func (NoOpCacheObserver) CacheFetchStarted(ctx context.Context, _ string) (context.Context, CacheFetchProbe)

type NoOpDataSourceObserver

type NoOpDataSourceObserver struct {
	NoOpCacheObserver
	NoOpLuaObserver
}

NoOpDataSourceObserver satisfies both datasource observer interfaces with empty probes.

type NoOpLuaFetchProbe

type NoOpLuaFetchProbe struct{}

NoOpLuaFetchProbe is a no-op implementation of LuaFetchProbe. Embed this in concrete probe types for forward compatibility.

func (NoOpLuaFetchProbe) End

func (NoOpLuaFetchProbe) End()

func (NoOpLuaFetchProbe) FetchAudit added in v0.10.0

func (NoOpLuaFetchProbe) FetchAudit(map[string]string)

func (NoOpLuaFetchProbe) FetchCompleted

func (NoOpLuaFetchProbe) FetchCompleted()

func (NoOpLuaFetchProbe) FetchCompletedNil

func (NoOpLuaFetchProbe) FetchCompletedNil()

func (NoOpLuaFetchProbe) InvalidReturnType

func (NoOpLuaFetchProbe) InvalidReturnType(string)

func (NoOpLuaFetchProbe) ResultConversionFailed

func (NoOpLuaFetchProbe) ResultConversionFailed(error)

func (NoOpLuaFetchProbe) ScriptExecutionFailed

func (NoOpLuaFetchProbe) ScriptExecutionFailed(error)

func (NoOpLuaFetchProbe) ScriptLoadFailed

func (NoOpLuaFetchProbe) ScriptLoadFailed(error)

type NoOpLuaObserver

type NoOpLuaObserver struct{}

NoOpLuaObserver is a no-op implementation of LuaObserver. Embed this in concrete observer types for forward compatibility.

func (NoOpLuaObserver) LuaFetchStarted

func (NoOpLuaObserver) LuaFetchStarted(ctx context.Context, _ string) (context.Context, LuaFetchProbe)

type StaticDataSource added in v0.2.0

type StaticDataSource struct {
	// contains filtered or unexported fields
}

StaticDataSource returns fixed JSON data on every fetch.

func (*StaticDataSource) Fetch added in v0.2.0

func (*StaticDataSource) Name added in v0.2.0

func (s *StaticDataSource) Name() string

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL