README
¶
BedrockBatchSfn
---
All classes are under active development and subject to non-backward compatible changes or removal in any future version. These are not subject to the Semantic Versioning model. This means that while you may use them, you may need to update your source code when upgrading to a newer version of this package.
| Language | Package |
|---|---|
 TypeScript |
@cdklabs/generative-ai-cdk-constructs |
 Python |
cdklabs.generative_ai_cdk_constructs |
Table of contents
Overview
The BedrockBatchSFN CDK construct simplifies the implementation of batch inference workflows with Amazon Bedrock by providing a pattern for processing large volumes of data asynchronously. It helps developers efficiently orchestrate batch processing tasks using Step Functions and Lambda, automatically handling job creation, status monitoring, and result collection. The construct is particularly valuable for cost-sensitive workloads like bulk text analysis, embeddings generation, and document summarization, taking advantage of Bedrock's 50% pricing discount for batch operations. By abstracting away the complexity of managing asynchronous model invocations and state management, developers can focus on their application logic while the construct handles the infrastructure and workflow orchestration needed for reliable batch processing at scale.
Usage
This construct implements an AWS Step Functions StateMachineFragment which can be used in your state machines to manage Bedrock batch inference jobs.
It requires Amazon Simple Storage Service(S3) buckets for input and output manifests and an AWS Identity and Access Management(IAM) managed policy that allows inference. You can use a single bucket for both input and output. The policy must have the following permissions for the models and inference profiles you plan to use:
- bedrock:InvokeModel
- bedrock:CreateModelInvocationJob
Here is a minimal deployable pattern definition:
batchBucket := s3.NewBucket(this, jsii.String("BedrockBatchBucket"))
batchPolicy := iam.NewManagedPolicy(this, jsii.String("BatchPolicy"), &ManagedPolicyProps{
})
batchPolicy.AddStatements(
iam.NewPolicyStatement(&PolicyStatementProps{
Sid: jsii.String("Inference"),
Actions: []*string{
jsii.String("bedrock:InvokeModel"),
jsii.String("bedrock:CreateModelInvocationJob"),
},
Resources: []*string{
fmt.Sprintf("arn:aws:bedrock:%v::foundation-model/*", this.Region),
},
}))
bedrockBatchSfnFragment := bedrock_batch_stepfn.NewBedrockBatchSfn(this, jsii.String("AwsBedrockBatchSfn"), &BedrockBatchSfnProps{
BedrockBatchInputBucket: batchBucket,
BedrockBatchOutputBucket: batchBucket,
BedrockBatchPolicy: batchPolicy,
Timeout: cdk.Duration_Hours(jsii.Number(48)),
})
inputState := sfn.NewPass(this, jsii.String("InputState"), &PassProps{
Parameters: map[string]interface{}{
"job_name": jsii.String("test_job"),
"manifest_keys": []interface{}{
jsii.String("test_key.jsonl"),
},
"model_id": jsii.String("test.model-v1"),
},
})
outputState := sfn.NewPass(this, jsii.String("OutputState"))
failState := sfn.NewFail(this, jsii.String("FailState"), &FailProps{
CausePath: sfn.JsonPath_StringAt(jsii.String("$.cause")),
ErrorPath: sfn.JsonPath_*StringAt(jsii.String("$.error")),
})
chain := inputState.Next(bedrockBatchSfnFragment).Next(outputState)
for _, endState := range bedrockBatchSfnFragment.EndStates {
if endState instanceof sfn.TaskStateBase {
endState.AddCatch(failState)
}
}
stateMachine := sfn.NewStateMachine(this, jsii.String("StateMachine"), &StateMachineProps{
DefinitionBody: sfn.DefinitionBody_FromChainable(chain),
})
See the API documentation.
Architecture
Cost
Please note that you will be responsible for the costs associated with the AWS services used during the execution of this construct. The cost of using this construct varies heavily according to model selection and the size of model inference jobs. As a reference point, we will assume a workload that uses Amazon Nova Pro with 10,000 input tokens and 1,000 output tokens per invocation, 100 records per invocation job and 300 invocation jobs per month.
We recommend creating a budget through AWS Cost Explorer to help manage costs. Prices are subject to change. For full details, refer to the pricing webpage for each AWS service used in this solution.
The following table provides a sample cost breakdown for deploying this solution with the default parameters in the US East (N. Virginia) Region for one month.
| AWS Service | Dimensions | Cost [USD] |
|---|---|---|
| Amazon Bedrock | Amazon Nova Pro with 10,000 input tokens and 1,000 output tokens per invocation, 100 records per invocation job and 300 invocation jobs per month | $168.00 |
| AWS Lambda | 6000 invocation requests, 128 MB, arm64 arch, 1 sec duration of each request | $0.01 |
| AWS Step Functions | 300 workflow requests with 1 state transition each | $0.00 |
| Amazon Simple Storage Service | Temporary Storage of Bedrock input and output manifests - 900 PUT requests, 600 GET requests, 1 GB data storage | $0.03 |
| AWS CloudWatch | < 1 GB logs ingested | $0.50 |
| Total | $168.54 |
For comparison, with on-demand inference, the Amazon Bedrock usage would cost $336.00.
Security
When you build systems on AWS infrastructure, security responsibilities are shared between you and AWS. This shared responsibility model reduces your operational burden because AWS operates, manages, and controls the components including the host operating system, virtualization layer, and physical security of the facilities in which the services operate. For more information about AWS security, visit AWS Cloud Security.
Supported AWS Regions
This solution uses the Amazon Bedrock service, which is not currently available in all AWS Regions. You must launch this construct in an AWS Region where these services are available. For the most current availability of AWS services by Region, see the AWS Regional Services List.
Note You need to explicity enable access to models before they are available for use in the Amazon Bedrock service. Please follow the Amazon Bedrock User Guide for steps related to enabling model access.
Quotas
Service quotas, also referred to as limits, are the maximum number of service resources or operations for your AWS account.
Make sure you have sufficient quota for each of the services implemented in this solution. For more information, refer to AWS service quotas.
To view the service quotas for all AWS services in the documentation without switching pages, view the information in the Service endpoints and quotas page in the PDF instead.
Clean up
When deleting your stack which uses this construct, do not forget to go over the following instructions to avoid unexpected charges:
- delete all the associated logs created by the different services in Amazon CloudWatch logs
© Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
Documentation
¶
Index ¶
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func BedrockBatchSfn_IsConstruct ¶
func BedrockBatchSfn_IsConstruct(x interface{}) *bool
Checks if `x` is a construct.
Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked.
Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead.
Returns: true if `x` is an object created from a class which extends `Construct`. Experimental.
func NewBedrockBatchSfn_Override ¶
func NewBedrockBatchSfn_Override(b BedrockBatchSfn, parent constructs.Construct, id *string, props *BedrockBatchSfnProps)
Experimental.
Types ¶
type BedrockBatchSfn ¶
type BedrockBatchSfn interface {
awsstepfunctions.StateMachineFragment
// The states to chain onto if this fragment is used.
// Experimental.
EndStates() *[]awsstepfunctions.INextable
// Descriptive identifier for this chainable.
// Experimental.
Id() *string
// The tree node.
// Experimental.
Node() constructs.Node
// The start state of this state machine fragment.
// Experimental.
StartState() awsstepfunctions.State
// Continue normal execution with the given state.
// Experimental.
Next(next awsstepfunctions.IChainable) awsstepfunctions.Chain
// Prefix the IDs of all states in this state machine fragment.
//
// Use this to avoid multiple copies of the state machine all having the
// same state IDs.
// Experimental.
PrefixStates(prefix *string) awsstepfunctions.StateMachineFragment
// Wrap all states in this state machine fragment up into a single state.
//
// This can be used to add retry or error handling onto this state
// machine fragment.
//
// Be aware that this changes the result of the inner state machine
// to be an array with the result of the state machine in it. Adjust
// your paths accordingly. For example, change 'outputPath' to
// '$[0]'.
// Experimental.
ToSingleState(options *awsstepfunctions.SingleStateOptions) awsstepfunctions.Parallel
// Returns a string representation of this construct.
// Experimental.
ToString() *string
}
A state machine fragment that creates a Bedrock Batch Inference Job and waits for its completion.
Input schema: ```
{
"job_name": string, // Required. Name of the batch inference job
"manifest_keys": string[], // Required. List of S3 keys of the input manifest files
"model_id": string // Required. Model ID to use.
}
```
Output schema: ```
{
"status": string, // Required. Status of the batch job. One of: "Completed" or "PartiallyCompleted"
"bucket": string, // Required. S3 bucket where the output is stored
"keys": string[] // Required. Array of S3 keys for the output files
}
```
Error schema: ```
{
"status": string, // Required. Status will be one of: "Failed", "Stopped", or "Expired"
"error": string, // Required. Error code
"cause": string // Required. Error message
}
```. Experimental.
func NewBedrockBatchSfn ¶
func NewBedrockBatchSfn(parent constructs.Construct, id *string, props *BedrockBatchSfnProps) BedrockBatchSfn
Experimental.
type BedrockBatchSfnProps ¶
type BedrockBatchSfnProps struct {
// The S3 bucket where the Bedrock Batch Inference Job gets the input manifests.
// Experimental.
BedrockBatchInputBucket awss3.IBucket `field:"required" json:"bedrockBatchInputBucket" yaml:"bedrockBatchInputBucket"`
// The S3 bucket where the Bedrock Batch Inference Job stores the output.
// Experimental.
BedrockBatchOutputBucket awss3.IBucket `field:"required" json:"bedrockBatchOutputBucket" yaml:"bedrockBatchOutputBucket"`
// IAM policy used for Bedrock batch processing.
//
// The policy must have the following permissions for the models and inference profiles you plan to use:
// - bedrock:InvokeModel
// - bedrock:CreateModelInvocationJob.
// Default: const bedrockBatchPolicy = new iam.ManagedPolicy(this, 'BedrockBatchPolicy', {
// statements: [
// new iam.PolicyStatement({
// sid: 'Inference',
// actions: ['bedrock:InvokeModel', 'bedrock:CreateModelInvocationJob'],
// resources: [
// 'arn:aws:bedrock:*::foundation-model/*',
// Stack.of(this).formatArn({
// service: 'bedrock',
// resource: 'inference-profile',
// resourceName: '*',
// }),
// ],
// }),
// ],
// });.
//
// Experimental.
BedrockBatchPolicy awsiam.IManagedPolicy `field:"optional" json:"bedrockBatchPolicy" yaml:"bedrockBatchPolicy"`
// JSONPath expression to select part of the state to be the input to this state.
//
// May also be the special value JsonPath. DISCARD, which will cause the effective input to be the empty object {}.
//
// Input schema:
// “`
// {
// "job_name": string, // Required. Name of the batch inference job
// "manifest_keys": string[], // Required. List of S3 keys of the input manifest files
// "model_id": string // Required. Model ID to use.
// }
// “`.
// Default: The entire task input (JSON path '$').
//
// Experimental.
InputPath *string `field:"optional" json:"inputPath" yaml:"inputPath"`
// JSONPath expression to indicate where to inject the state's output May also be the special value JsonPath.
//
// DISCARD, which will cause the state's input to become its output.
//
// Output schema:
// “`
// {
// "status": string, // Required. Status of the batch job. One of: "Completed" or "PartiallyCompleted"
// "bucket": string, // Required. S3 bucket where the output is stored
// "keys": string[] // Required. Array of S3 keys for the output files
// }
// “`.
// Default: Replaces the entire input with the result (JSON path '$').
//
// Experimental.
ResultPath *string `field:"optional" json:"resultPath" yaml:"resultPath"`
// The timeout duration for the batch inference job.
//
// Must be between 24 hours and 1 week (168 hours).
// Default: Duration.hours(48)
//
// Experimental.
Timeout awscdk.Duration `field:"optional" json:"timeout" yaml:"timeout"`
}
Experimental.
Source Files
Directories