Documentation
¶
Overview ¶
Package bedrock provides AWS Bedrock LLM integration for LangGraph-Go.
Overview ¶
This package implements the ChatModel interface for AWS Bedrock, Amazon's managed service for foundation models. It supports multiple model families including Claude (Anthropic), Llama (Meta), Titan (Amazon), and Mistral.
Key features:
- Multi-model support with automatic schema translation
- Regional failover for high availability
- Automatic retry with exponential backoff
- Streaming response support (model-dependent)
- Tool/function calling (Claude models)
- Comprehensive error handling with retry classification
Quick Start ¶
Basic usage with Claude model:
import (
"context"
"github.com/dshills/langgraph-go/graph/model/bedrock"
)
config := bedrock.Config{
Region: "us-east-1",
ModelID: "anthropic.claude-3-5-sonnet-20241022-v2:0",
MaxTokens: 4096,
}
adapter, err := bedrock.NewAdapter(context.Background(), config)
if err != nil {
log.Fatal(err)
}
messages := []model.Message{
{Role: model.RoleUser, Content: "What is the capital of France?"},
}
response, err := adapter.Chat(context.Background(), messages, nil)
if err != nil {
log.Fatal(err)
}
fmt.Println(response.Text) // "The capital of France is Paris."
Regional Failover ¶
Configure automatic failover to backup regions for high availability:
config := bedrock.Config{
Region: "us-east-1",
FallbackRegions: []string{"us-west-2", "eu-west-1"},
ModelID: "anthropic.claude-3-5-sonnet-20241022-v2:0",
}
On throttling or service errors in us-east-1, requests automatically retry in us-west-2, then eu-west-1. The actual region used is tracked in ChatOut.Meta["region"] for observability.
Tool Calling ¶
Claude models support tool/function calling:
tools := []model.ToolSpec{
{
Name: "get_weather",
Description: "Get current weather for a location",
Schema: map[string]interface{}{
"type": "object",
"properties": map[string]interface{}{
"location": map[string]interface{}{
"type": "string",
"description": "City name",
},
},
"required": []string{"location"},
},
},
}
response, err := adapter.Chat(ctx, messages, tools)
for _, call := range response.ToolCalls {
fmt.Printf("Tool: %s, Input: %v\n", call.Name, call.Input)
}
Error Handling ¶
Errors are wrapped as BedrockError with retry classification:
response, err := adapter.Chat(ctx, messages, nil)
if err != nil {
var bedrockErr *bedrock.BedrockError
if errors.As(err, &bedrockErr) {
if bedrockErr.Retryable {
// Transient error - already retried automatically
fmt.Printf("Failed after retries: %s\n", bedrockErr.Code)
} else {
// Permanent error - needs user action
fmt.Printf("Non-retryable error: %s\n", bedrockErr.Code)
}
}
}
Retryable errors: ThrottlingException, ModelTimeoutException, InternalServerException Non-retryable errors: ValidationException, AccessDeniedException, ResourceNotFoundException
Supported Models ¶
Claude (Anthropic):
- anthropic.claude-3-5-sonnet-20241022-v2:0 (recommended)
- anthropic.claude-3-sonnet-20240229-v1:0
- anthropic.claude-3-haiku-20240307-v1:0
- Features: Tools, streaming, system messages, long context
Llama (Meta) - Phase 5:
- meta.llama3-2-90b-instruct-v1:0
- meta.llama3-1-70b-instruct-v1:0
- Features: Text generation only
Titan (Amazon) - Phase 5:
- amazon.titan-text-premier-v1:0
- amazon.titan-text-express-v1:0
- Features: Text generation only
Mistral - Phase 5:
- mistral.mistral-large-2402-v1:0
- mistral.mistral-7b-instruct-v0:2
- Features: Text generation
AWS Credentials ¶
The adapter uses the AWS SDK default credential chain:
- Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
- Shared credentials file (~/.aws/credentials)
- IAM role (for EC2/ECS/Lambda)
Or provide credentials explicitly:
config := bedrock.Config{
Region: "us-east-1",
ModelID: "anthropic.claude-3-5-sonnet-20241022-v2:0",
CredentialsProvider: aws.NewCredentialsCache(
credentials.NewStaticCredentialsProvider(accessKey, secretKey, ""),
),
}
Architecture ¶
The package uses a translator pattern to support multiple model families:
Adapter (stateless)
├─ Config (immutable)
├─ AWS SDK Client (bedrockruntime)
└─ SchemaTranslator (model family specific)
├─ ClaudeSchemaTranslator
├─ LlamaSchemaTranslator (Phase 5)
├─ TitanSchemaTranslator (Phase 5)
└─ MistralSchemaTranslator (Phase 5)
Each translator converts between LangGraph message format and the model-specific request/response format required by Bedrock.
Implementation Status ¶
Completed:
- Claude model support (full features)
- Regional failover
- Retry logic with exponential backoff
- Error classification
- Metadata tracking
In Progress (Phase 5-10):
- Streaming support
- Llama, Titan, Mistral models
- Integration tests
- Examples
See specs/008-bedrock-llm-support/tasks.md for detailed roadmap.
Index ¶
- type Adapter
- type BedrockError
- type ClaudeSchemaTranslator
- func (t ClaudeSchemaTranslator) SupportsStreaming() bool
- func (t ClaudeSchemaTranslator) SupportsTools() bool
- func (t ClaudeSchemaTranslator) TranslateRequest(messages []model.Message, tools []model.ToolSpec, config *Config) (json.RawMessage, error)
- func (t ClaudeSchemaTranslator) TranslateResponse(response json.RawMessage) (model.ChatOut, error)
- func (t ClaudeSchemaTranslator) TranslateStreamEvent(event json.RawMessage) (StreamChunk, error)
- type Config
- type ModelFamily
- type NovaSchemaTranslator
- func (t NovaSchemaTranslator) SupportsStreaming() bool
- func (t NovaSchemaTranslator) SupportsTools() bool
- func (t NovaSchemaTranslator) TranslateRequest(messages []model.Message, tools []model.ToolSpec, config *Config) (json.RawMessage, error)
- func (t NovaSchemaTranslator) TranslateResponse(rawResp json.RawMessage) (model.ChatOut, error)
- func (t NovaSchemaTranslator) TranslateStreamEvent(rawEvent json.RawMessage) (StreamChunk, error)
- type SchemaTranslator
- type StreamCallback
- type StreamChunk
- type ToolCallDelta
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
This section is empty.
Types ¶
type Adapter ¶
type Adapter struct {
// contains filtered or unexported fields
}
Adapter implements the ChatModel interface for AWS Bedrock.
The adapter is stateless - all configuration is immutable after initialization. State management is handled by the LangGraph-Go Engine via the reducer pattern.
Supports multiple model families: - Claude (Anthropic): Full features including tools and streaming - Llama (Meta): Text generation, no tools - Titan (Amazon): Text generation, no tools - Mistral: Text generation
Example:
config := bedrock.Config{
Region: "us-east-1",
ModelID: "anthropic.claude-3-5-sonnet-20241022-v2:0",
}
adapter, err := bedrock.NewAdapter(ctx, config)
if err != nil {
log.Fatal(err)
}
messages := []model.Message{
{Role: model.RoleUser, Content: "Hello!"},
}
response, err := adapter.Chat(ctx, messages, nil)
func NewAdapter ¶
NewAdapter creates a new Bedrock adapter with the given configuration.
Initializes AWS SDK client, validates configuration, and selects appropriate schema translator based on model family.
Returns error if: - Configuration validation fails - AWS credentials cannot be loaded - Model family is unsupported
The returned adapter is safe for concurrent use.
func (*Adapter) Chat ¶
func (a *Adapter) Chat(ctx context.Context, messages []model.Message, tools []model.ToolSpec) (model.ChatOut, error)
Chat sends messages to the Bedrock model and returns the response.
Implements the ChatModel interface from graph/model.
Process: 1. Translate LangGraph messages to Bedrock-specific request format 2. Call Bedrock InvokeModel API 3. Translate Bedrock response back to LangGraph ChatOut format 4. Handle errors with retry logic and regional fallback
Parameters: - ctx: Context for cancellation and timeout control - messages: Conversation history (system, user, assistant messages) - tools: Optional tool specifications (only supported by Claude models)
Returns: - ChatOut with response text and/or tool calls - Error if request fails after retries
Respects context cancellation and enforces timeouts.
func (*Adapter) ChatStream ¶
func (a *Adapter) ChatStream(ctx context.Context, messages []model.Message, tools []model.ToolSpec, callback StreamCallback) (model.ChatOut, error)
ChatStream sends messages to Bedrock and streams the response token-by-token.
This method uses AWS Bedrock's InvokeModelWithResponseStream API to receive incremental responses. Each chunk is parsed and delivered via the callback.
Process:
- Validate messages and check model supports streaming
- Translate LangGraph messages to Bedrock request format
- Call InvokeModelWithResponseStream API
- Process event stream, calling callback for each chunk
- Accumulate final response and return ChatOut
Parameters:
- ctx: Context for cancellation and timeout control
- messages: Conversation history (system, user, assistant messages)
- tools: Optional tool specifications (only supported by Claude models)
- callback: Function called for each streaming chunk
Returns:
- ChatOut: Complete accumulated response
- Error: Stream errors, translation errors, or callback errors
The callback receives chunks in this order:
- message_start: Initial metadata (request_id, model, input_tokens)
- content_block_start: Start of text or tool block
- content_block_delta: Multiple incremental content chunks
- content_block_stop: End of content block
- message_delta: Final metadata (stop_reason, output_tokens)
- message_stop: Stream complete
Example:
var fullText string
callback := func(chunk bedrock.StreamChunk) error {
fullText += chunk.Delta
fmt.Print(chunk.Delta) // Print each token immediately
return nil
}
response, err := adapter.ChatStream(ctx, messages, nil, callback)
if err != nil {
log.Fatal(err)
}
fmt.Printf("\nFinal response: %s\n", response.Text)
type BedrockError ¶
type BedrockError struct {
// Code is the AWS error code (e.g., "ThrottlingException", "AccessDeniedException").
Code string
// Message is a human-readable error description.
Message string
// RequestID is the AWS request ID for debugging with AWS support.
RequestID string
// Region is the AWS region where the error occurred.
Region string
// Retryable indicates if this error is transient and safe to retry.
// True for: ThrottlingException, ModelTimeoutException, InternalServerException
// False for: AccessDeniedException, ValidationException, ModelNotReadyException
Retryable bool
// OriginalError is the underlying AWS SDK error.
OriginalError error
}
BedrockError wraps AWS Bedrock-specific errors with actionable context.
Provides retry classification and regional context for error handling. Used to determine whether to retry requests and which fallback strategies to use.
func (*BedrockError) Error ¶
func (e *BedrockError) Error() string
Error implements the error interface.
func (*BedrockError) Unwrap ¶
func (e *BedrockError) Unwrap() error
Unwrap returns the original error for errors.Is/As compatibility.
type ClaudeSchemaTranslator ¶
type ClaudeSchemaTranslator struct{}
ClaudeSchemaTranslator implements SchemaTranslator for Anthropic Claude models.
Uses the Anthropic Messages API format as documented in: specs/008-bedrock-llm-support/contracts/bedrock-claude-request.json
Features: - Full tool/function calling support - Streaming responses - System message extraction - Multi-turn conversations
func (ClaudeSchemaTranslator) SupportsStreaming ¶
func (t ClaudeSchemaTranslator) SupportsStreaming() bool
SupportsStreaming returns true (Claude supports streaming).
func (ClaudeSchemaTranslator) SupportsTools ¶
func (t ClaudeSchemaTranslator) SupportsTools() bool
SupportsTools returns true (Claude supports tool calling).
func (ClaudeSchemaTranslator) TranslateRequest ¶
func (t ClaudeSchemaTranslator) TranslateRequest(messages []model.Message, tools []model.ToolSpec, config *Config) (json.RawMessage, error)
TranslateRequest converts messages to Claude Messages API format.
func (ClaudeSchemaTranslator) TranslateResponse ¶
func (t ClaudeSchemaTranslator) TranslateResponse(response json.RawMessage) (model.ChatOut, error)
TranslateResponse parses Claude response to ChatOut.
func (ClaudeSchemaTranslator) TranslateStreamEvent ¶
func (t ClaudeSchemaTranslator) TranslateStreamEvent(event json.RawMessage) (StreamChunk, error)
TranslateStreamEvent parses Claude streaming events.
Handles 7 Claude streaming event types:
- message_start: Initial metadata (request_id, model, input_tokens)
- content_block_start: Start of text or tool_use block
- content_block_delta: Incremental content (text_delta or input_json_delta)
- content_block_stop: End of content block (no-op)
- message_delta: Final metadata with stop_reason and output_tokens
- message_stop: Stream complete (no-op)
- error: Error event (returns error)
Returns StreamChunk with populated fields based on event type.
type Config ¶
type Config struct {
// Region is the AWS region for Bedrock endpoint (e.g., "us-east-1").
// Required. Must be a valid AWS region where Bedrock is available.
Region string
// ModelID is the Bedrock model identifier (e.g., "anthropic.claude-3-5-sonnet-20241022-v2:0").
// Required. Format: "provider.model-name[:version]"
ModelID string
// CredentialsProvider explicitly sets AWS credentials.
// Optional. If nil, uses AWS SDK default credential chain:
// 1. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
// 2. Shared credentials file (~/.aws/credentials)
// 3. IAM role (for EC2/ECS/Lambda)
CredentialsProvider aws.CredentialsProvider
// EndpointURL overrides the default Bedrock endpoint.
// Optional. Used for VPC endpoints or testing with localstack.
EndpointURL string
// FallbackRegions provides ordered list of backup regions for automatic failover.
// Optional. On throttling or service errors, adapter retries in fallback regions.
FallbackRegions []string
// MaxRetries sets maximum retry attempts for transient errors.
// Optional. Default: 3. Range: 0-10.
// Retries use exponential backoff with jitter.
MaxRetries int
// Temperature controls randomness in model output.
// Optional. Range: 0.0 (deterministic) to 1.0 (very random).
// Default: model-specific (typically 1.0 for Claude, 0.5 for Llama).
Temperature float64
// MaxTokens limits the maximum tokens to generate.
// Optional. Range: 1 to model maximum (4096 for Claude, 8192 for Llama).
// Default: model-specific.
MaxTokens int
// TopP controls nucleus sampling diversity.
// Optional. Range: 0.0 to 1.0.
// Default: model-specific (typically 0.9).
TopP float64
// StopSequences defines sequences that stop generation.
// Optional. Maximum 4 sequences (Bedrock API limit).
StopSequences []string
// StreamingEnabled enables streaming responses token-by-token.
// Optional. Default: false.
// Requires model support (Claude supports, Llama 4 Instruct does not).
StreamingEnabled bool
}
Config holds configuration for the Bedrock adapter.
Bedrock is AWS's managed service for foundation models (Claude, Llama, Titan, Mistral). This config specifies which model to use, AWS credentials, region, and generation parameters.
Example:
config := bedrock.Config{
Region: "us-east-1",
ModelID: "anthropic.claude-3-5-sonnet-20241022-v2:0",
MaxTokens: 4096,
Temperature: 0.7,
}
func (*Config) Validate ¶
Validate checks if the configuration is valid.
Validates: - Region is a known AWS region - ModelID follows expected format - Temperature is in range [0.0, 1.0] - MaxTokens is positive and within model limits - TopP is in range [0.0, 1.0] - FallbackRegions are valid and don't duplicate primary Region - MaxRetries is in range [0, 10]
Returns error describing first validation failure, or nil if valid.
type ModelFamily ¶
type ModelFamily int
ModelFamily identifies the Bedrock model family for schema selection.
Different model families use different request/response formats. The adapter selects the appropriate SchemaTranslator based on ModelFamily.
const ( // ModelFamilyUnknown indicates an unsupported or unrecognized model. ModelFamilyUnknown ModelFamily = iota // ModelFamilyClaude represents Anthropic Claude models. // Format: Anthropic Messages API // Features: Full tool support, streaming, system messages // Examples: anthropic.claude-3-5-sonnet-20241022-v2:0 ModelFamilyClaude // ModelFamilyLlama represents Meta Llama models. // Format: Llama instruction template // Features: Text generation only, no tools // Examples: meta.llama3-2-90b-instruct-v1:0 ModelFamilyLlama // ModelFamilyTitan represents Amazon Titan models. // Format: Titan text generation API // Features: Text generation only, no tools, no structured messages // Examples: amazon.titan-text-premier-v1:0 ModelFamilyTitan // ModelFamilyMistral represents Mistral models. // Format: Mistral instruction template // Features: Text generation, limited tool support // Examples: mistral.mistral-large-2402-v1:0 ModelFamilyMistral // ModelFamilyNova represents Amazon Nova models. // Format: Amazon Nova Messages API (Converse API compatible) // Features: Text generation, multimodal support, tool calling // Examples: amazon.nova-lite-v1:0, amazon.nova-pro-v1:0, amazon.nova-micro-v1:0 ModelFamilyNova )
func (ModelFamily) String ¶
func (mf ModelFamily) String() string
String returns the string representation of the ModelFamily.
type NovaSchemaTranslator ¶
type NovaSchemaTranslator struct{}
NovaSchemaTranslator implements SchemaTranslator for Amazon Nova models.
Amazon Nova models use a simplified Messages API format compatible with AWS Bedrock's standard format. Unlike Claude models, Nova does not require anthropic-specific fields like "anthropic_version" or "max_tokens".
Features: - Full tool/function calling support - Streaming responses - System message extraction - Multi-turn conversations - Multimodal support (text, images)
func (NovaSchemaTranslator) SupportsStreaming ¶
func (t NovaSchemaTranslator) SupportsStreaming() bool
SupportsStreaming returns false as Nova streaming is not yet implemented. TODO: Implement Nova streaming support and verify event format against AWS API documentation.
func (NovaSchemaTranslator) SupportsTools ¶
func (t NovaSchemaTranslator) SupportsTools() bool
SupportsTools returns true as Nova models support tool calling
func (NovaSchemaTranslator) TranslateRequest ¶
func (t NovaSchemaTranslator) TranslateRequest(messages []model.Message, tools []model.ToolSpec, config *Config) (json.RawMessage, error)
TranslateRequest converts messages to Amazon Nova Messages API format.
func (NovaSchemaTranslator) TranslateResponse ¶
func (t NovaSchemaTranslator) TranslateResponse(rawResp json.RawMessage) (model.ChatOut, error)
TranslateResponse converts Nova response to ChatOut
Nova responses use a similar format to Claude, so we can reuse the Claude logic
func (NovaSchemaTranslator) TranslateStreamEvent ¶
func (t NovaSchemaTranslator) TranslateStreamEvent(rawEvent json.RawMessage) (StreamChunk, error)
TranslateStreamEvent converts Nova streaming events to StreamChunk
Nova streaming uses a similar format to Claude
type SchemaTranslator ¶
type SchemaTranslator interface {
// TranslateRequest converts LangGraph messages to Bedrock request format.
//
// Parameters:
// - messages: LangGraph conversation history
// - tools: Optional tool specifications
// - config: Bedrock configuration for generation parameters
//
// Returns:
// - JSON-encoded request body for Bedrock InvokeModel API
// - Error if translation fails (e.g., unsupported features)
TranslateRequest(messages []model.Message, tools []model.ToolSpec, config *Config) (json.RawMessage, error)
// TranslateResponse converts Bedrock response to LangGraph ChatOut format.
//
// Parameters:
// - response: JSON response from Bedrock InvokeModel API
//
// Returns:
// - ChatOut with extracted text and tool calls
// - Error if parsing fails
TranslateResponse(response json.RawMessage) (model.ChatOut, error)
// TranslateStreamEvent converts a Bedrock streaming event to StreamChunk.
//
// Only applicable for models that support streaming (e.g., Claude).
//
// Parameters:
// - event: JSON streaming event from Bedrock InvokeModelWithResponseStream
//
// Returns:
// - StreamChunk with incremental content
// - Error if parsing fails
TranslateStreamEvent(event json.RawMessage) (StreamChunk, error)
// SupportsStreaming returns true if this model family supports streaming responses.
SupportsStreaming() bool
// SupportsTools returns true if this model family supports tool/function calling.
SupportsTools() bool
}
SchemaTranslator defines the interface for translating between LangGraph message format and Bedrock model-specific schemas.
Each model family has its own implementation (Claude, Llama, Titan, Mistral). Translators are stateless and safe for concurrent use.
type StreamCallback ¶
type StreamCallback func(chunk StreamChunk) error
StreamCallback is called for each streaming chunk received from Bedrock.
Parameters:
- chunk: Incremental content with Delta (text), ToolCallDelta (tool input), FinishReason (when stream ends), and Metadata (token counts, etc.)
The callback should return an error to abort streaming. Returning nil continues processing the stream.
Example:
callback := func(chunk bedrock.StreamChunk) error {
if chunk.Delta != "" {
fmt.Print(chunk.Delta) // Print tokens as they arrive
}
if chunk.FinishReason != "" {
fmt.Printf("\nFinished: %s\n", chunk.FinishReason)
}
return nil
}
type StreamChunk ¶
type StreamChunk struct {
// Delta contains incremental text content.
// Empty for non-text events (e.g., metadata-only events).
Delta string
// ToolCallDelta contains incremental tool call data (if applicable).
// Only used for models that support tools and streaming tool calls.
ToolCallDelta *ToolCallDelta
// FinishReason indicates why generation stopped (if this is the final chunk).
// Values: "end_turn", "max_tokens", "stop_sequence", "tool_use"
// Empty for intermediate chunks.
FinishReason string
// Metadata contains model-specific metadata (token counts, request IDs, etc.).
Metadata map[string]interface{}
}
StreamChunk represents an incremental piece of a streaming response.
Used to assemble streaming responses token-by-token.
type ToolCallDelta ¶
type ToolCallDelta struct {
// Index identifies which tool call this delta belongs to.
// Multiple tool calls may be streamed in parallel.
Index int
// Name is the tool name (only present in first delta for this tool call).
Name string
// PartialJSON contains a partial JSON fragment for tool input.
// Must be accumulated across deltas and parsed when complete.
PartialJSON string
}
ToolCallDelta represents an incremental piece of a tool call.
Tool calls may be streamed in parts, requiring accumulation before parsing.