ldai

package module
v0.9.3 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 11, 2026 License: Apache-2.0 Imports: 13 Imported by: 0

README

LaunchDarkly Server-side AI SDK for Go

Actions Status

[!CAUTION] This AI SDK is in pre-release and not subject to backwards compatibility guarantees. The API may change based on feedback.

Pin to a specific minor version and review the changelog before upgrading.

Active feature development is ongoing in the Python and Node.js AI SDKs, so this SDK will receive new features at a slower pace. Refer to those for the latest capabilities.

LaunchDarkly overview

LaunchDarkly is a feature management platform that serves trillions of feature flags daily to help teams build better software, faster. Get started using LaunchDarkly today!

Twitter Follow

Getting started

Import the module:

import (
	ld "github.com/launchdarkly/go-server-sdk/v7"
	"github.com/launchdarkly/go-server-sdk/ldai"
)

Configure the base LaunchDarkly Server SDK:

sdkClient, _ = ld.MakeClient("your-sdk-key", 5*time.Second)

Instantiate the AI client, passing in the base Server SDK:

aiClient, err := ldai.NewClient(sdkClient)

Fetch a model configuration for a specific LaunchDarkly context:

// The default value 'ldai.Disabled()' be returned if LaunchDarkly is unavailable or the config
// cannot be fetched. To customize the default value, use ldai.NewConfig().
config := aiClient.CompletionConfig("your-model-key", ldcontext.New("user-key"), ldai.Disabled(), nil)

// Access the methods on config. To generate analytic events related to usage of the model config,
// obtain a tracker by calling config.CreateTracker().
tracker := config.CreateTracker()

Learn more

Read our documentation for in-depth instructions on configuring and using LaunchDarkly. You can also head straight to the complete reference guide for this SDK.

Contributing

We encourage pull requests and other contributions from the community. Check out our contributing guidelines for instructions on how to contribute to this library.

About LaunchDarkly

  • LaunchDarkly is a continuous delivery platform that provides feature flags as a service and allows developers to iterate quickly and safely. We allow you to easily flag your features and manage them from the LaunchDarkly dashboard. With LaunchDarkly, you can:
    • Roll out a new feature to a subset of your users (like a group of users who opt-in to a beta tester group), gathering feedback and bug reports from real-world use cases.
    • Gradually roll out a feature to an increasing percentage of users, and track the effect that the feature has on key metrics (for instance, how likely is a user to complete a purchase if they have feature A versus feature B?).
    • Turn off a feature that you realize is causing performance problems in production, without needing to re-deploy, or even restart the application with a changed configuration file.
    • Grant access to certain features based on user attributes, like payment plan (eg: users on the ‘gold’ plan get access to more features than users in the ‘silver’ plan). Disable parts of your application to facilitate maintenance, without taking everything offline.
  • LaunchDarkly provides feature flag SDKs for a wide variety of languages and technologies. Read our documentation for a complete list.
  • Explore LaunchDarkly

Documentation

Overview

Package ldai contains an AI SDK suitable for usage with generative AI applications.

Index

Constants

View Source 
const (
	JudgePlaceholderMessageHistory     = "{{message_history}}"
	JudgePlaceholderResponseToEvaluate = "{{response_to_evaluate}}"
)

JudgePlaceholderMessageHistory and JudgePlaceholderResponseToEvaluate are the literal placeholder strings shared between JudgeConfig (pass 1) and Judge.buildMessages (pass 2). Both must use the same values or substitution silently fails.

View Source 
const (
	// Version is the current version string of the ldai package. This is updated by our release scripts.
	Version = "0.9.3" // {{ x-release-please-version }}

	// SDKName is the canonical name of this AI SDK package.
	SDKName = "go-server-sdk/ldai"

	// SDKLanguage is the programming language of this AI SDK.
	SDKLanguage = "go"
)

Variables

This section is empty.

Functions

This section is empty.

Types

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client is the main entrypoint for the AI SDK. A client can be used to obtain an AI Config from LaunchDarkly. Unless otherwise noted, the Client's method are not safe for concurrent use.

func NewClient 

func NewClient(sdk ServerSDK) (*Client, error)

NewClient creates a new AI Client. The provided SDK interface must not be nil. The client will use the provided SDK's loggers to log warnings and errors.

func (*Client) CompletionConfig added in v0.8.1 

func (c *Client) CompletionConfig(
	key string,
	context ldcontext.Context,
	defaultValue Config,
	variables map[string]interface{},
) Config

CompletionConfig retrieves an AI Config and interpolates its message templates using the provided variables. Returns the default value if the config cannot be evaluated. Template interpolation is not applied to the default value's messages.

To send analytic events to LaunchDarkly, call CreateTracker on the returned Config to obtain a Tracker.

func (*Client) CreateTracker added in v0.9.0 

func (c *Client) CreateTracker(token string, context ldcontext.Context) (*Tracker, error)

CreateTracker reconstructs a Tracker from a resumption token and the given context. This delegates to TrackerFromResumptionToken. See that function for details.

func (*Client) JudgeConfig added in v0.8.0 

func (c *Client) JudgeConfig(
	key string,
	context ldcontext.Context,
	defaultValue Config,
	variables map[string]interface{},
) Config

JudgeConfig retrieves a Judge AI Config and interpolates its message templates. The reserved variables message_history and response_to_evaluate are preserved as literal placeholders for substitution by Judge.buildMessages during evaluation.

To send analytic events to LaunchDarkly, call CreateTracker on the returned Config to obtain a Tracker.

type Config 

type Config struct {
	// contains filtered or unexported fields
}

Config represents an AI Config.

func Disabled 

func Disabled() Config

Disabled is a helper that returns a built Config that is disabled and contains no messages.

func (*Config) AsLdValue 

func (c *Config) AsLdValue() ldvalue.Value

AsLdValue is used internally.

func (*Config) CreateTracker added in v0.9.0 

func (c *Config) CreateTracker() *Tracker

CreateTracker returns a new Tracker for a fresh AI run. Each call mints a new runId (a UUIDv4) that LaunchDarkly uses to correlate the run's events in metrics views. Call this once per AI run; metrics from different runIds cannot be combined.

Returns nil if the config was not obtained via the Client.

func (*Config) CustomModelParam 

func (c *Config) CustomModelParam(key string) (ldvalue.Value, bool)

CustomModelParam returns the custom model parameter named by key. The second parameter is true if the key exists.

func (*Config) Enabled 

func (c *Config) Enabled() bool

Enabled returns whether the config is enabled.

func (*Config) EvaluationMetricKey added in v0.8.0 

func (c *Config) EvaluationMetricKey() string

EvaluationMetricKey returns the evaluation metric key for judge mode configs.

func (*Config) EvaluationMetricKeys added in v0.8.0 

func (c *Config) EvaluationMetricKeys() []string

EvaluationMetricKeys returns the deprecated array of evaluation metric keys. Use EvaluationMetricKey instead.

func (*Config) JudgeConfiguration added in v0.8.0 

func (c *Config) JudgeConfiguration() *datamodel.JudgeConfiguration

JudgeConfiguration returns the judge configuration attached to this config, if any. Returns a defensive copy to prevent mutations.

func (*Config) Messages 

func (c *Config) Messages() []datamodel.Message

Messages returns the messages defined by the config. The series of messages may be passed to an AI model provider.

func (*Config) Mode added in v0.8.0 

func (c *Config) Mode() string

Mode returns the AI Config mode (e.g., "completion", "agent", "judge").

func (*Config) ModelName added in v0.2.0 

func (c *Config) ModelName() string

ModelName returns the model name associated with the config.

func (*Config) ModelParam 

func (c *Config) ModelParam(key string) (ldvalue.Value, bool)

ModelParam returns the model parameter named by key. The second parameter is true if the key exists.

func (*Config) ProviderName added in v0.2.0 

func (c *Config) ProviderName() string

ProviderName returns the provider name associated with the config.

func (*Config) VariationKey added in v0.3.0 

func (c *Config) VariationKey() string

VariationKey is used internally by LaunchDarkly.

func (*Config) Version added in v0.6.0 

func (c *Config) Version() int

Version is used internally by LaunchDarkly.

type ConfigBuilder 

type ConfigBuilder struct {
	// contains filtered or unexported fields
}

ConfigBuilder is used to define a default AI Config, returned when LaunchDarkly is unreachable or there is an error evaluating the Config.

func NewConfig 

func NewConfig() *ConfigBuilder

NewConfig returns a new ConfigBuilder. By default, the Config is disabled.

func (*ConfigBuilder) Build 

func (cb *ConfigBuilder) Build() Config

Build creates a Config from the current builder state.

func (*ConfigBuilder) Disable 

func (cb *ConfigBuilder) Disable() *ConfigBuilder

Disable disables the config.

func (*ConfigBuilder) Enable 

func (cb *ConfigBuilder) Enable() *ConfigBuilder

Enable enables the config.

func (*ConfigBuilder) WithCustomModelParam 

func (cb *ConfigBuilder) WithCustomModelParam(key string, value ldvalue.Value) *ConfigBuilder

WithCustomModelParam sets a custom model parameter named by key to the given value. If the key already exists, it will be overwritten.

func (*ConfigBuilder) WithEnabled 

func (cb *ConfigBuilder) WithEnabled(enabled bool) *ConfigBuilder

WithEnabled sets whether the config is enabled. See also Enable and Disable.

func (*ConfigBuilder) WithEvaluationMetricKey added in v0.8.0 

func (cb *ConfigBuilder) WithEvaluationMetricKey(key string) *ConfigBuilder

WithEvaluationMetricKey sets the evaluation metric key for judge mode configs.

func (*ConfigBuilder) WithEvaluationMetricKeys added in v0.8.0 

func (cb *ConfigBuilder) WithEvaluationMetricKeys(keys []string) *ConfigBuilder

WithEvaluationMetricKeys sets the deprecated array of evaluation metric keys. Use WithEvaluationMetricKey instead.

func (*ConfigBuilder) WithJudgeConfiguration added in v0.8.0 

func (cb *ConfigBuilder) WithJudgeConfiguration(judgeConfig *datamodel.JudgeConfiguration) *ConfigBuilder

WithJudgeConfiguration sets the judge configuration for this config. The provided judgeConfig is defensively copied.

func (*ConfigBuilder) WithMessage 

func (cb *ConfigBuilder) WithMessage(content string, role datamodel.Role) *ConfigBuilder

WithMessage appends a message to the config with the given role.

func (*ConfigBuilder) WithMode added in v0.8.0 

func (cb *ConfigBuilder) WithMode(mode string) *ConfigBuilder

WithMode sets the AI Config mode (e.g., "completion", "agent", "judge").

func (*ConfigBuilder) WithModelName added in v0.2.0 

func (cb *ConfigBuilder) WithModelName(modelName string) *ConfigBuilder

WithModelName sets the model name associated with the config.

func (*ConfigBuilder) WithModelParam 

func (cb *ConfigBuilder) WithModelParam(key string, value ldvalue.Value) *ConfigBuilder

WithModelParam sets a model parameter named by key to the given value. If the key already exists, it will be overwritten. Model parameters are generally set by LaunchDarkly; for custom parameters not recognized by LaunchDarkly, use WithModelCustomParam.

func (*ConfigBuilder) WithProviderName added in v0.2.0 

func (cb *ConfigBuilder) WithProviderName(providerName string) *ConfigBuilder

WithProviderName sets the provider name associated with the config.

type EventSink 

type EventSink interface {
	// TrackMetric sends a named analytic event to LaunchDarkly relevant to a particular context, and containing a
	// metric value and additional data.
	TrackMetric(
		eventName string,
		context ldcontext.Context,
		metricValue float64,
		data ldvalue.Value,
	) error
}

EventSink represents the Tracker's requirements for delivering analytic events. This is generally satisfied by the LaunchDarkly SDK's TrackMetric method.

type Feedback 

type Feedback string

Feedback represents the feedback provided by a user for a model evaluation. 

const (
	// FeedbackPositive is positive feedback.
	FeedbackPositive Feedback = "positive"
	// FeedbackNegative is negative feedback.
	FeedbackNegative Feedback = "negative"
)

type MetricSummary added in v0.7.0 

type MetricSummary struct {
	// Duration is the tracked duration in milliseconds.
	Duration ldcommon.Option[time.Duration]
	// Feedback is the tracked user feedback (positive or negative).
	Feedback ldcommon.Option[Feedback]
	// Tokens contains information about token usage.
	Tokens ldcommon.Option[TokenUsage]
	// Success indicates whether the operation was successful.
	Success ldcommon.Option[bool]
	// TimeToFirstToken is the time to the first token in milliseconds.
	TimeToFirstToken ldcommon.Option[time.Duration]
}

MetricSummary represents a summary of metrics tracked by the tracker.

type Metrics 

type Metrics struct {
	// Latency is the latency of the request.
	Latency time.Duration
	// TimeToFirstToken is the time to the first token of the streamed response.
	TimeToFirstToken time.Duration
}

Metrics represents the metrics returned by a model provider for a specific request.

type ProviderResponse 

type ProviderResponse struct {
	// Usage is the token usage.
	Usage TokenUsage
	// Metrics is the request metrics.
	Metrics Metrics
}

ProviderResponse represents the response from a model provider for a specific request.

type ServerSDK 

type ServerSDK interface {
	JSONVariation(
		key string,
		context ldcontext.Context,
		defaultVal ldvalue.Value,
	) (ldvalue.Value, error)
	Loggers() interfaces.LDLoggers
	TrackMetric(
		eventName string,
		context ldcontext.Context,
		metricValue float64,
		data ldvalue.Value,
	) error
}

ServerSDK defines the required methods for the AI SDK to interact with LaunchDarkly. These methods are satisfied by the LaunchDarkly Go Server SDK.

type Stopwatch 

type Stopwatch interface {
	// Start starts the stopwatch.
	Start()
	// Stop stops the stopwatch and returns the duration since Start was called.
	Stop() time.Duration
}

Stopwatch is used to measure the duration of a task. Start will always be called before Stop. If an implementation is not provided, the Tracker uses a default implementation that delegates to time.Now and time.Since.

type TokenUsage 

type TokenUsage struct {
	// Total is the total number of tokens used.
	Total int
	// Input is the number of input tokens used.
	Input int

	// Output is the number of output tokens used.
	Output int
}

TokenUsage represents the token usage returned by a model provider for a specific request.

func (TokenUsage) Set 

func (t TokenUsage) Set() bool

Set returns true if any of the fields are non-zero.

type Tracker 

type Tracker struct {
	// contains filtered or unexported fields
}

Tracker records metrics for a single AI run. Unless otherwise noted, the Tracker's methods are not safe for concurrent use.

All events a Tracker emits share a runId (a UUIDv4) so LaunchDarkly can correlate them in metrics views. See individual track methods for their specific semantics. Call CreateTracker on the AI Config to start a new run. A ResumptionToken preserves the runId, so events emitted by a Tracker reconstructed in another process correlate with the original run.

func TrackerFromResumptionToken added in v0.9.0 

func TrackerFromResumptionToken(token string, sdk ServerSDK, context ldcontext.Context) (*Tracker, error)

TrackerFromResumptionToken reconstructs a Tracker from a resumption token and the given context. This is used for cross-process scenarios (e.g., deferred feedback) where the original tracker is no longer available but its runId must be reused. The token is obtained from Tracker.ResumptionToken(). The reconstructed tracker will have empty modelName and providerName since these are not included in the token.

func (*Tracker) GetSummary added in v0.7.0 

func (t *Tracker) GetSummary() MetricSummary

GetSummary returns a summary of all metrics that have been tracked using this tracker.

func (*Tracker) ResumptionToken added in v0.9.0 

func (t *Tracker) ResumptionToken() string

ResumptionToken returns a URL-safe Base64-encoded token that can be used to reconstruct a tracker in a different process (e.g., for deferred feedback). The token contains the runId, configKey, variationKey, and version. It does not contain modelName or providerName.

func (*Tracker) TrackDuration 

func (t *Tracker) TrackDuration(dur time.Duration) error

TrackDuration tracks the duration of a task. For example, the duration of a model evaluation request may be tracked here. See also TrackRequest. The duration in milliseconds must fit within a float64.

Records at most once per Tracker; further calls are ignored.

func (*Tracker) TrackError added in v0.4.0 

func (t *Tracker) TrackError() error

TrackError tracks an unsuccessful model evaluation.

Records at most once per Tracker. TrackSuccess and TrackError share state; only one of the two can record per Tracker, and subsequent calls are ignored.

func (*Tracker) TrackFeedback 

func (t *Tracker) TrackFeedback(feedback Feedback) error

TrackFeedback tracks the feedback provided by a user for a model evaluation. If the feedback is not FeedbackPositive or FeedbackNegative, returns an error and does not track anything.

Records at most once per Tracker; further calls are ignored.

func (*Tracker) TrackJudgeResponse added in v0.8.0 

func (t *Tracker) TrackJudgeResponse(response datamodel.JudgeResponse) error

TrackJudgeResponse tracks the evaluation scores from a judge response.

May be called multiple times per Tracker; each call records the scores from the given response.

func (*Tracker) TrackRequest 

func (t *Tracker) TrackRequest(task func(c *Config) (ProviderResponse, error)) (ProviderResponse, error)

TrackRequest tracks metrics for a model evaluation request. The task function should return a ProviderResponse which can be used to specify request metrics and token usage. All fields of the returned ProviderResponse are optional.

The task function will be passed the current AI Config, which can be used to obtain any parameters or messages relevant to the request.

If the task returns an error, then the request is not considered successful and no metrics are tracked. Otherwise, the following metrics are tracked:

  1. Successful model evaluation.
  2. Any metrics that were that set in the ProviderResponse 2a) If Latency was not set in the ProviderResponse's Metrics field, an automatically measured duration.
  3. Any token usage that was set in the ProviderResponse.

Subsequent calls re-run the task but emit only metrics not already recorded on this Tracker. Call CreateTracker on the AI Config to start a new run.

func (*Tracker) TrackSuccess 

func (t *Tracker) TrackSuccess() error

TrackSuccess tracks a successful model evaluation.

Records at most once per Tracker. TrackSuccess and TrackError share state; only one of the two can record per Tracker, and subsequent calls are ignored.

func (*Tracker) TrackTimeToFirstToken added in v0.5.0 

func (t *Tracker) TrackTimeToFirstToken(dur time.Duration) error

TrackTimeToFirstToken tracks the time to the first token of the streamed response.

Records at most once per Tracker; further calls are ignored.

func (*Tracker) TrackTokens added in v0.9.0 

func (t *Tracker) TrackTokens(usage TokenUsage) error

TrackTokens tracks the token usage for a model evaluation.

Records at most once per Tracker; further calls are ignored.

func (*Tracker) TrackUsage deprecated 

func (t *Tracker) TrackUsage(usage TokenUsage) error

TrackUsage tracks token usage.

Deprecated: Use TrackTokens instead.

Source Files

View all Source files

Directories

Path Synopsis

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.