bedrock

package
v0.4.0-beta Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 18, 2025 License: MIT Imports: 11 Imported by: 0

Documentation

Overview

Package bedrock provides AWS Bedrock LLM integration for LangGraph-Go.

Overview

This package implements the ChatModel interface for AWS Bedrock, Amazon's managed service for foundation models. It supports multiple model families including Claude (Anthropic), Llama (Meta), Titan (Amazon), and Mistral.

Key features:

  • Multi-model support with automatic schema translation
  • Regional failover for high availability
  • Automatic retry with exponential backoff
  • Streaming response support (model-dependent)
  • Tool/function calling (Claude models)
  • Comprehensive error handling with retry classification

Quick Start

Basic usage with Claude model:

import (
    "context"
    "github.com/dshills/langgraph-go/graph/model/bedrock"
)

config := bedrock.Config{
    Region:  "us-east-1",
    ModelID: "anthropic.claude-3-5-sonnet-20241022-v2:0",
    MaxTokens: 4096,
}

adapter, err := bedrock.NewAdapter(context.Background(), config)
if err != nil {
    log.Fatal(err)
}

messages := []model.Message{
    {Role: model.RoleUser, Content: "What is the capital of France?"},
}

response, err := adapter.Chat(context.Background(), messages, nil)
if err != nil {
    log.Fatal(err)
}
fmt.Println(response.Text) // "The capital of France is Paris."

Regional Failover

Configure automatic failover to backup regions for high availability:

config := bedrock.Config{
    Region: "us-east-1",
    FallbackRegions: []string{"us-west-2", "eu-west-1"},
    ModelID: "anthropic.claude-3-5-sonnet-20241022-v2:0",
}

On throttling or service errors in us-east-1, requests automatically retry in us-west-2, then eu-west-1. The actual region used is tracked in ChatOut.Meta["region"] for observability.

Tool Calling

Claude models support tool/function calling:

tools := []model.ToolSpec{
    {
        Name: "get_weather",
        Description: "Get current weather for a location",
        Schema: map[string]interface{}{
            "type": "object",
            "properties": map[string]interface{}{
                "location": map[string]interface{}{
                    "type": "string",
                    "description": "City name",
                },
            },
            "required": []string{"location"},
        },
    },
}

response, err := adapter.Chat(ctx, messages, tools)
for _, call := range response.ToolCalls {
    fmt.Printf("Tool: %s, Input: %v\n", call.Name, call.Input)
}

Error Handling

Errors are wrapped as BedrockError with retry classification:

response, err := adapter.Chat(ctx, messages, nil)
if err != nil {
    var bedrockErr *bedrock.BedrockError
    if errors.As(err, &bedrockErr) {
        if bedrockErr.Retryable {
            // Transient error - already retried automatically
            fmt.Printf("Failed after retries: %s\n", bedrockErr.Code)
        } else {
            // Permanent error - needs user action
            fmt.Printf("Non-retryable error: %s\n", bedrockErr.Code)
        }
    }
}

Retryable errors: ThrottlingException, ModelTimeoutException, InternalServerException Non-retryable errors: ValidationException, AccessDeniedException, ResourceNotFoundException

Supported Models

Claude (Anthropic):

  • anthropic.claude-3-5-sonnet-20241022-v2:0 (recommended)
  • anthropic.claude-3-sonnet-20240229-v1:0
  • anthropic.claude-3-haiku-20240307-v1:0
  • Features: Tools, streaming, system messages, long context

Llama (Meta) - Phase 5:

  • meta.llama3-2-90b-instruct-v1:0
  • meta.llama3-1-70b-instruct-v1:0
  • Features: Text generation only

Titan (Amazon) - Phase 5:

  • amazon.titan-text-premier-v1:0
  • amazon.titan-text-express-v1:0
  • Features: Text generation only

Mistral - Phase 5:

  • mistral.mistral-large-2402-v1:0
  • mistral.mistral-7b-instruct-v0:2
  • Features: Text generation

AWS Credentials

The adapter uses the AWS SDK default credential chain:

  1. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
  2. Shared credentials file (~/.aws/credentials)
  3. IAM role (for EC2/ECS/Lambda)

Or provide credentials explicitly:

config := bedrock.Config{
    Region: "us-east-1",
    ModelID: "anthropic.claude-3-5-sonnet-20241022-v2:0",
    CredentialsProvider: aws.NewCredentialsCache(
        credentials.NewStaticCredentialsProvider(accessKey, secretKey, ""),
    ),
}

Architecture

The package uses a translator pattern to support multiple model families:

Adapter (stateless)
  ├─ Config (immutable)
  ├─ AWS SDK Client (bedrockruntime)
  └─ SchemaTranslator (model family specific)
     ├─ ClaudeSchemaTranslator
     ├─ LlamaSchemaTranslator (Phase 5)
     ├─ TitanSchemaTranslator (Phase 5)
     └─ MistralSchemaTranslator (Phase 5)

Each translator converts between LangGraph message format and the model-specific request/response format required by Bedrock.

Implementation Status

Completed:

  • Claude model support (full features)
  • Regional failover
  • Retry logic with exponential backoff
  • Error classification
  • Metadata tracking

In Progress (Phase 5-10):

  • Streaming support
  • Llama, Titan, Mistral models
  • Integration tests
  • Examples

See specs/008-bedrock-llm-support/tasks.md for detailed roadmap.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Adapter

type Adapter struct {
	// contains filtered or unexported fields
}

Adapter implements the ChatModel interface for AWS Bedrock.

The adapter is stateless - all configuration is immutable after initialization. State management is handled by the LangGraph-Go Engine via the reducer pattern.

Supports multiple model families: - Claude (Anthropic): Full features including tools and streaming - Llama (Meta): Text generation, no tools - Titan (Amazon): Text generation, no tools - Mistral: Text generation

Example:

config := bedrock.Config{
    Region:  "us-east-1",
    ModelID: "anthropic.claude-3-5-sonnet-20241022-v2:0",
}
adapter, err := bedrock.NewAdapter(ctx, config)
if err != nil {
    log.Fatal(err)
}

messages := []model.Message{
    {Role: model.RoleUser, Content: "Hello!"},
}
response, err := adapter.Chat(ctx, messages, nil)

func NewAdapter

func NewAdapter(ctx context.Context, config Config) (*Adapter, error)

NewAdapter creates a new Bedrock adapter with the given configuration.

Initializes AWS SDK client, validates configuration, and selects appropriate schema translator based on model family.

Returns error if: - Configuration validation fails - AWS credentials cannot be loaded - Model family is unsupported

The returned adapter is safe for concurrent use.

func (*Adapter) Chat

func (a *Adapter) Chat(ctx context.Context, messages []model.Message, tools []model.ToolSpec) (model.ChatOut, error)

Chat sends messages to the Bedrock model and returns the response.

Implements the ChatModel interface from graph/model.

Process: 1. Translate LangGraph messages to Bedrock-specific request format 2. Call Bedrock InvokeModel API 3. Translate Bedrock response back to LangGraph ChatOut format 4. Handle errors with retry logic and regional fallback

Parameters: - ctx: Context for cancellation and timeout control - messages: Conversation history (system, user, assistant messages) - tools: Optional tool specifications (only supported by Claude models)

Returns: - ChatOut with response text and/or tool calls - Error if request fails after retries

Respects context cancellation and enforces timeouts.

func (*Adapter) ChatStream

func (a *Adapter) ChatStream(ctx context.Context, messages []model.Message, tools []model.ToolSpec, callback StreamCallback) (model.ChatOut, error)

ChatStream sends messages to Bedrock and streams the response token-by-token.

This method uses AWS Bedrock's InvokeModelWithResponseStream API to receive incremental responses. Each chunk is parsed and delivered via the callback.

Process:

  1. Validate messages and check model supports streaming
  2. Translate LangGraph messages to Bedrock request format
  3. Call InvokeModelWithResponseStream API
  4. Process event stream, calling callback for each chunk
  5. Accumulate final response and return ChatOut

Parameters:

  • ctx: Context for cancellation and timeout control
  • messages: Conversation history (system, user, assistant messages)
  • tools: Optional tool specifications (only supported by Claude models)
  • callback: Function called for each streaming chunk

Returns:

  • ChatOut: Complete accumulated response
  • Error: Stream errors, translation errors, or callback errors

The callback receives chunks in this order:

  1. message_start: Initial metadata (request_id, model, input_tokens)
  2. content_block_start: Start of text or tool block
  3. content_block_delta: Multiple incremental content chunks
  4. content_block_stop: End of content block
  5. message_delta: Final metadata (stop_reason, output_tokens)
  6. message_stop: Stream complete

Example:

var fullText string
callback := func(chunk bedrock.StreamChunk) error {
    fullText += chunk.Delta
    fmt.Print(chunk.Delta)  // Print each token immediately
    return nil
}

response, err := adapter.ChatStream(ctx, messages, nil, callback)
if err != nil {
    log.Fatal(err)
}
fmt.Printf("\nFinal response: %s\n", response.Text)

type BedrockError

type BedrockError struct {
	// Code is the AWS error code (e.g., "ThrottlingException", "AccessDeniedException").
	Code string

	// Message is a human-readable error description.
	Message string

	// RequestID is the AWS request ID for debugging with AWS support.
	RequestID string

	// Region is the AWS region where the error occurred.
	Region string

	// Retryable indicates if this error is transient and safe to retry.
	// True for: ThrottlingException, ModelTimeoutException, InternalServerException
	// False for: AccessDeniedException, ValidationException, ModelNotReadyException
	Retryable bool

	// OriginalError is the underlying AWS SDK error.
	OriginalError error
}

BedrockError wraps AWS Bedrock-specific errors with actionable context.

Provides retry classification and regional context for error handling. Used to determine whether to retry requests and which fallback strategies to use.

func (*BedrockError) Error

func (e *BedrockError) Error() string

Error implements the error interface.

func (*BedrockError) Unwrap

func (e *BedrockError) Unwrap() error

Unwrap returns the original error for errors.Is/As compatibility.

type ClaudeSchemaTranslator

type ClaudeSchemaTranslator struct{}

ClaudeSchemaTranslator implements SchemaTranslator for Anthropic Claude models.

Uses the Anthropic Messages API format as documented in: specs/008-bedrock-llm-support/contracts/bedrock-claude-request.json

Features: - Full tool/function calling support - Streaming responses - System message extraction - Multi-turn conversations

func (ClaudeSchemaTranslator) SupportsStreaming

func (t ClaudeSchemaTranslator) SupportsStreaming() bool

SupportsStreaming returns true (Claude supports streaming).

func (ClaudeSchemaTranslator) SupportsTools

func (t ClaudeSchemaTranslator) SupportsTools() bool

SupportsTools returns true (Claude supports tool calling).

func (ClaudeSchemaTranslator) TranslateRequest

func (t ClaudeSchemaTranslator) TranslateRequest(messages []model.Message, tools []model.ToolSpec, config *Config) (json.RawMessage, error)

TranslateRequest converts messages to Claude Messages API format.

func (ClaudeSchemaTranslator) TranslateResponse

func (t ClaudeSchemaTranslator) TranslateResponse(response json.RawMessage) (model.ChatOut, error)

TranslateResponse parses Claude response to ChatOut.

func (ClaudeSchemaTranslator) TranslateStreamEvent

func (t ClaudeSchemaTranslator) TranslateStreamEvent(event json.RawMessage) (StreamChunk, error)

TranslateStreamEvent parses Claude streaming events.

Handles 7 Claude streaming event types:

  • message_start: Initial metadata (request_id, model, input_tokens)
  • content_block_start: Start of text or tool_use block
  • content_block_delta: Incremental content (text_delta or input_json_delta)
  • content_block_stop: End of content block (no-op)
  • message_delta: Final metadata with stop_reason and output_tokens
  • message_stop: Stream complete (no-op)
  • error: Error event (returns error)

Returns StreamChunk with populated fields based on event type.

type Config

type Config struct {
	// Region is the AWS region for Bedrock endpoint (e.g., "us-east-1").
	// Required. Must be a valid AWS region where Bedrock is available.
	Region string

	// ModelID is the Bedrock model identifier (e.g., "anthropic.claude-3-5-sonnet-20241022-v2:0").
	// Required. Format: "provider.model-name[:version]"
	ModelID string

	// CredentialsProvider explicitly sets AWS credentials.
	// Optional. If nil, uses AWS SDK default credential chain:
	// 1. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
	// 2. Shared credentials file (~/.aws/credentials)
	// 3. IAM role (for EC2/ECS/Lambda)
	CredentialsProvider aws.CredentialsProvider

	// EndpointURL overrides the default Bedrock endpoint.
	// Optional. Used for VPC endpoints or testing with localstack.
	EndpointURL string

	// FallbackRegions provides ordered list of backup regions for automatic failover.
	// Optional. On throttling or service errors, adapter retries in fallback regions.
	FallbackRegions []string

	// MaxRetries sets maximum retry attempts for transient errors.
	// Optional. Default: 3. Range: 0-10.
	// Retries use exponential backoff with jitter.
	MaxRetries int

	// Temperature controls randomness in model output.
	// Optional. Range: 0.0 (deterministic) to 1.0 (very random).
	// Default: model-specific (typically 1.0 for Claude, 0.5 for Llama).
	Temperature float64

	// MaxTokens limits the maximum tokens to generate.
	// Optional. Range: 1 to model maximum (4096 for Claude, 8192 for Llama).
	// Default: model-specific.
	MaxTokens int

	// TopP controls nucleus sampling diversity.
	// Optional. Range: 0.0 to 1.0.
	// Default: model-specific (typically 0.9).
	TopP float64

	// StopSequences defines sequences that stop generation.
	// Optional. Maximum 4 sequences (Bedrock API limit).
	StopSequences []string

	// StreamingEnabled enables streaming responses token-by-token.
	// Optional. Default: false.
	// Requires model support (Claude supports, Llama 4 Instruct does not).
	StreamingEnabled bool
}

Config holds configuration for the Bedrock adapter.

Bedrock is AWS's managed service for foundation models (Claude, Llama, Titan, Mistral). This config specifies which model to use, AWS credentials, region, and generation parameters.

Example:

config := bedrock.Config{
    Region:      "us-east-1",
    ModelID:     "anthropic.claude-3-5-sonnet-20241022-v2:0",
    MaxTokens:   4096,
    Temperature: 0.7,
}

func (*Config) Validate

func (c *Config) Validate() error

Validate checks if the configuration is valid.

Validates: - Region is a known AWS region - ModelID follows expected format - Temperature is in range [0.0, 1.0] - MaxTokens is positive and within model limits - TopP is in range [0.0, 1.0] - FallbackRegions are valid and don't duplicate primary Region - MaxRetries is in range [0, 10]

Returns error describing first validation failure, or nil if valid.

type ModelFamily

type ModelFamily int

ModelFamily identifies the Bedrock model family for schema selection.

Different model families use different request/response formats. The adapter selects the appropriate SchemaTranslator based on ModelFamily.

const (
	// ModelFamilyUnknown indicates an unsupported or unrecognized model.
	ModelFamilyUnknown ModelFamily = iota

	// ModelFamilyClaude represents Anthropic Claude models.
	// Format: Anthropic Messages API
	// Features: Full tool support, streaming, system messages
	// Examples: anthropic.claude-3-5-sonnet-20241022-v2:0
	ModelFamilyClaude

	// ModelFamilyLlama represents Meta Llama models.
	// Format: Llama instruction template
	// Features: Text generation only, no tools
	// Examples: meta.llama3-2-90b-instruct-v1:0
	ModelFamilyLlama

	// ModelFamilyTitan represents Amazon Titan models.
	// Format: Titan text generation API
	// Features: Text generation only, no tools, no structured messages
	// Examples: amazon.titan-text-premier-v1:0
	ModelFamilyTitan

	// ModelFamilyMistral represents Mistral models.
	// Format: Mistral instruction template
	// Features: Text generation, limited tool support
	// Examples: mistral.mistral-large-2402-v1:0
	ModelFamilyMistral

	// ModelFamilyNova represents Amazon Nova models.
	// Format: Amazon Nova Messages API (Converse API compatible)
	// Features: Text generation, multimodal support, tool calling
	// Examples: amazon.nova-lite-v1:0, amazon.nova-pro-v1:0, amazon.nova-micro-v1:0
	ModelFamilyNova
)

func (ModelFamily) String

func (mf ModelFamily) String() string

String returns the string representation of the ModelFamily.

type NovaSchemaTranslator

type NovaSchemaTranslator struct{}

NovaSchemaTranslator implements SchemaTranslator for Amazon Nova models.

Amazon Nova models use a simplified Messages API format compatible with AWS Bedrock's standard format. Unlike Claude models, Nova does not require anthropic-specific fields like "anthropic_version" or "max_tokens".

Features: - Full tool/function calling support - Streaming responses - System message extraction - Multi-turn conversations - Multimodal support (text, images)

func (NovaSchemaTranslator) SupportsStreaming

func (t NovaSchemaTranslator) SupportsStreaming() bool

SupportsStreaming returns false as Nova streaming is not yet implemented. TODO: Implement Nova streaming support and verify event format against AWS API documentation.

func (NovaSchemaTranslator) SupportsTools

func (t NovaSchemaTranslator) SupportsTools() bool

SupportsTools returns true as Nova models support tool calling

func (NovaSchemaTranslator) TranslateRequest

func (t NovaSchemaTranslator) TranslateRequest(messages []model.Message, tools []model.ToolSpec, config *Config) (json.RawMessage, error)

TranslateRequest converts messages to Amazon Nova Messages API format.

func (NovaSchemaTranslator) TranslateResponse

func (t NovaSchemaTranslator) TranslateResponse(rawResp json.RawMessage) (model.ChatOut, error)

TranslateResponse converts Nova response to ChatOut

Nova responses use a similar format to Claude, so we can reuse the Claude logic

func (NovaSchemaTranslator) TranslateStreamEvent

func (t NovaSchemaTranslator) TranslateStreamEvent(rawEvent json.RawMessage) (StreamChunk, error)

TranslateStreamEvent converts Nova streaming events to StreamChunk

Nova streaming uses a similar format to Claude

type SchemaTranslator

type SchemaTranslator interface {
	// TranslateRequest converts LangGraph messages to Bedrock request format.
	//
	// Parameters:
	// - messages: LangGraph conversation history
	// - tools: Optional tool specifications
	// - config: Bedrock configuration for generation parameters
	//
	// Returns:
	// - JSON-encoded request body for Bedrock InvokeModel API
	// - Error if translation fails (e.g., unsupported features)
	TranslateRequest(messages []model.Message, tools []model.ToolSpec, config *Config) (json.RawMessage, error)

	// TranslateResponse converts Bedrock response to LangGraph ChatOut format.
	//
	// Parameters:
	// - response: JSON response from Bedrock InvokeModel API
	//
	// Returns:
	// - ChatOut with extracted text and tool calls
	// - Error if parsing fails
	TranslateResponse(response json.RawMessage) (model.ChatOut, error)

	// TranslateStreamEvent converts a Bedrock streaming event to StreamChunk.
	//
	// Only applicable for models that support streaming (e.g., Claude).
	//
	// Parameters:
	// - event: JSON streaming event from Bedrock InvokeModelWithResponseStream
	//
	// Returns:
	// - StreamChunk with incremental content
	// - Error if parsing fails
	TranslateStreamEvent(event json.RawMessage) (StreamChunk, error)

	// SupportsStreaming returns true if this model family supports streaming responses.
	SupportsStreaming() bool

	// SupportsTools returns true if this model family supports tool/function calling.
	SupportsTools() bool
}

SchemaTranslator defines the interface for translating between LangGraph message format and Bedrock model-specific schemas.

Each model family has its own implementation (Claude, Llama, Titan, Mistral). Translators are stateless and safe for concurrent use.

type StreamCallback

type StreamCallback func(chunk StreamChunk) error

StreamCallback is called for each streaming chunk received from Bedrock.

Parameters:

  • chunk: Incremental content with Delta (text), ToolCallDelta (tool input), FinishReason (when stream ends), and Metadata (token counts, etc.)

The callback should return an error to abort streaming. Returning nil continues processing the stream.

Example:

callback := func(chunk bedrock.StreamChunk) error {
    if chunk.Delta != "" {
        fmt.Print(chunk.Delta)  // Print tokens as they arrive
    }
    if chunk.FinishReason != "" {
        fmt.Printf("\nFinished: %s\n", chunk.FinishReason)
    }
    return nil
}

type StreamChunk

type StreamChunk struct {
	// Delta contains incremental text content.
	// Empty for non-text events (e.g., metadata-only events).
	Delta string

	// ToolCallDelta contains incremental tool call data (if applicable).
	// Only used for models that support tools and streaming tool calls.
	ToolCallDelta *ToolCallDelta

	// FinishReason indicates why generation stopped (if this is the final chunk).
	// Values: "end_turn", "max_tokens", "stop_sequence", "tool_use"
	// Empty for intermediate chunks.
	FinishReason string

	// Metadata contains model-specific metadata (token counts, request IDs, etc.).
	Metadata map[string]interface{}
}

StreamChunk represents an incremental piece of a streaming response.

Used to assemble streaming responses token-by-token.

type ToolCallDelta

type ToolCallDelta struct {
	// Index identifies which tool call this delta belongs to.
	// Multiple tool calls may be streamed in parallel.
	Index int

	// Name is the tool name (only present in first delta for this tool call).
	Name string

	// PartialJSON contains a partial JSON fragment for tool input.
	// Must be accumulated across deltas and parsed when complete.
	PartialJSON string
}

ToolCallDelta represents an incremental piece of a tool call.

Tool calls may be streamed in parts, requiring accumulation before parsing.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL