comprehend

package
v1.8.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 20, 2026 License: MIT Imports: 36 Imported by: 0

README

Comprehend

Parity grade: A · SDK aws-sdk-go-v2/service/comprehend@v1.43.4 · last audited 2026-10-07 (a2084957b)

Coverage

Metric Value
PARITY entries audited 28 (28 ok)
Feature families 1 (1 ok)
Known gaps 1
Structural gaps (can't be emulated) 1
Deferred items 1
Resource leaks clean
Known gaps
  • FlywheelIterationProperties.EvaluationManifestS3Prefix: the pinned SDK documents the member with an empty doc string and the data-lake object layout is not specified, so no value can be derived without inventing a path.
Structural gaps

These do not block an A grade — no implementation could produce real data here because the underlying data source cannot exist in an emulator.

  • ClassifyDocument/DetectEntities Bytes of PDF, Word and image documents (DocumentReaderConfig effect): text extraction needs Amazon Textract; plain-text Bytes are decoded and DocumentReaderConfig is validated, binary documents are rejected.
Deferred
  • ALREADY COVERED BY CHAOS (verified gopherstack-sw2q): ResourceLimitExceededException/ResourceUnavailableException/TooManyRequestsException/ConcurrentModificationException are real modeled errors for several ops here (confirmed against deserializers.go's per-op error-case switches) but have no non-fabricated deterministic backend-state trigger in this emulator: no rate limiting is implemented anywhere in gopherstack per-service, no fixed per-account resource quota is documented precisely enough to emulate without risking false failures on legitimate high-volume test/integration usage, and ConcurrentModificationException describes a real-AWS eventual-consistency race that cannot occur under this backend's single coarse lock. Concretely verified this pass: comprehend.Handler implements ChaosServiceName() -> "comprehend" and ChaosOperations() -> h.GetSupportedOperations() (handler.go), and pkgs/chaos.Middleware is wired globally via registry.Use(chaos.Middleware(faultStore)) in internal/app/cli.go, matching purely on the request's SigV4 service name + X-Amz-Target operation + region and injecting an arbitrary caller-specified FaultError{Code, StatusCode} without touching backend state. A fault rule such as {"service":"comprehend","error":{"code":"TooManyRequestsException","statusCode":429}} deterministically returns that exact typed error to a real aws-sdk-go-v2 client on any operation, with zero backend code changes -- proven end-to-end against a real containerized client in test/integration/chaos_test.go. Error-code wiring in errors.go/handler.go intentionally does not include backend sentinels for these four; the chaos mechanism is the correct, non-fabricated way to exercise them, not a backend-state workaround.

More

Documentation

Index

Constants

This section is empty.

Variables

View Source 
var (
	// ErrNotFound is returned when a requested Comprehend resource is absent.
	ErrNotFound = errors.New("ResourceNotFoundException")
	// ErrConflict is returned when a named Comprehend resource already exists.
	ErrConflict = errors.New("ResourceInUseException")
	// ErrValidation is returned for invalid request values.
	ErrValidation = errors.New("InvalidRequestException")
	// ErrJobNotFound is returned when an async job ID does not exist. Real AWS
	// uses this distinct code (not ResourceNotFoundException) for every
	// Describe*Job/Stop*Job operation -- confirmed against every
	// awsAwsjson11_deserializeOpErrorDescribe*Job/Stop*Job case in the SDK's
	// generated deserializers.go.
	ErrJobNotFound = errors.New("JobNotFoundException")
	// ErrTooManyTags is returned when a resource would carry more than the
	// 50-tag-per-resource limit (both existing and newly requested tags count).
	ErrTooManyTags = errors.New("TooManyTagsException")
	// ErrBatchSizeLimitExceeded is returned when a Batch* request's TextList
	// carries more than 25 documents.
	ErrBatchSizeLimitExceeded = errors.New("BatchSizeLimitExceededException")
	// ErrTextSizeLimitExceeded is returned when a single document's text
	// exceeds the per-operation byte limit.
	ErrTextSizeLimitExceeded = errors.New("TextSizeLimitExceededException")
	// ErrUnsupportedLanguage is returned when a request's LanguageCode is not
	// one of the languages the target operation supports.
	ErrUnsupportedLanguage = errors.New("UnsupportedLanguageException")
	// ErrKmsKeyValidation is returned when a supplied KMS key ID/ARN does not
	// match a valid KMS key ID or ARN shape.
	ErrKmsKeyValidation = errors.New("KmsKeyValidationException")
)

Sentinel errors map 1:1 to Comprehend's modeled exception shapes (aws-sdk-go-v2/service/comprehend/types/errors.go). handleError in handler.go matches each with errors.Is and emits the exact "__type" wire code with HTTP 400 (every exception here has smithy.FaultClient; only InternalServerException is FaultServer, so it stays the unmapped 500 default rather than getting its own sentinel).

Not every exception the real SDK models for these operations has a sentinel here: ResourceLimitExceededException, ResourceUnavailableException, TooManyRequestsException, and ConcurrentModificationException are real modeled errors for several ops in this service, but none has a non-fabricated deterministic trigger in a synchronous, single-lock, unbounded in-memory emulator (no rate limiting, no enforced per-account resource quotas, no real concurrent-write races -- see PARITY.md gaps). Generic throttling/5xx injection for any operation is available instead through the chaos fault-injection system (ChaosOperations/ChaosServiceName).

Functions

This section is empty.

Types

type FlywheelIteration 

type FlywheelIteration struct {
	CreationTime            time.Time
	EndTime                 time.Time
	FlywheelArn             string
	FlywheelIterationID     string
	FlywheelIterationStatus string
	Message                 string
	// contains filtered or unexported fields
}

FlywheelIteration represents one model training iteration.

type Handler 

type Handler struct {
	Backend *InMemoryBackend
	// contains filtered or unexported fields
}

Handler serves Amazon Comprehend JSON operations.

func NewHandler 

func NewHandler(backend *InMemoryBackend) *Handler

NewHandler creates Comprehend handler backed by in-memory state.

func (*Handler) BackendFor 

func (h *Handler) BackendFor(region string) *InMemoryBackend

BackendFor returns the backend serving region: the home backend, or the sibling built on first use.

func (*Handler) ChaosOperations 

func (h *Handler) ChaosOperations() []string

ChaosOperations returns fault-injectable operations.

func (*Handler) ChaosRegions 

func (h *Handler) ChaosRegions() []string

ChaosRegions returns configured service region.

func (*Handler) ChaosServiceName 

func (h *Handler) ChaosServiceName() string

ChaosServiceName returns service key for fault matching.

func (*Handler) EnableRegions 

func (h *Handler) EnableRegions()

EnableRegions makes h serve every other region through lazily built per-region siblings.

func (*Handler) ExtractOperation 

func (h *Handler) ExtractOperation(c *echo.Context) string

ExtractOperation returns operation in request target.

func (*Handler) ExtractResource 

func (h *Handler) ExtractResource(c *echo.Context) string

ExtractResource retrieves common ARN and job identifier fields.

func (*Handler) GetSupportedOperations 

func (h *Handler) GetSupportedOperations() []string

GetSupportedOperations reports implemented operations.

func (*Handler) Handler 

func (h *Handler) Handler() echo.HandlerFunc

Handler returns Echo JSON target dispatcher.

func (*Handler) MatchPriority 

func (h *Handler) MatchPriority() int

MatchPriority returns header matching priority.

func (*Handler) Name 

func (h *Handler) Name() string

Name returns service name.

func (*Handler) RegionBackends 

func (h *Handler) RegionBackends() []*InMemoryBackend

RegionBackends returns the home backend followed by every sibling built so far.

func (*Handler) RegionHandler 

func (h *Handler) RegionHandler(region string) *Handler

RegionHandler returns the handler serving region: h itself for the home region, else its sibling.

func (*Handler) Reset 

func (h *Handler) Reset()

Reset clears backend state.

func (*Handler) Restore 

func (h *Handler) Restore(ctx context.Context, data []byte) error

Restore implements persistence.Persistable.

func (*Handler) RouteMatcher 

func (h *Handler) RouteMatcher() service.Matcher

RouteMatcher matches Comprehend X-Amz-Target headers.

func (*Handler) Snapshot 

func (h *Handler) Snapshot(ctx context.Context) []byte

Snapshot implements persistence.Persistable; other regions ride in an additive "regions" key.

type InMemoryBackend 

type InMemoryBackend struct {
	// contains filtered or unexported fields
}

InMemoryBackend stores Comprehend state safely for concurrent requests.

jobs, resources, and iterations are *store.Table[T] (see store_setup.go). Each keys off a real, non-json:"-" identity field the value type already carries (Job.JobID, Resource.Arn, FlywheelIteration.FlywheelIterationID), so all three register directly on registry -- no DTO indirection is needed. Neither table needs a secondary store.Index: every filtered listing (ListJobs by JobType, ListResources by Type, ListFlywheelIterations by FlywheelArn) scans the table's full contents, exactly as the original map-range loops did, and none of the fields mutated in place by advanceJob/advanceTrainingResource/etc. are ever used as an index key. tags, policies, and policyRevisions are left as plain maps: their values are not *T (map[string]string / string), so they do not fit store.Table's keyed-by-identity-value shape.

func NewInMemoryBackend 

func NewInMemoryBackend(accountID, region string) *InMemoryBackend

NewInMemoryBackend creates a configured Comprehend backend.

func (*InMemoryBackend) CreateResource 

func (b *InMemoryBackend) CreateResource(
	resourceType, name, versionName string,
	values map[string]any,
	tags []Tag,
) (*Resource, error)

CreateResource creates a stateful Comprehend resource and optionally attaches tags.

func (*InMemoryBackend) DeleteResource 

func (b *InMemoryBackend) DeleteResource(resourceArn, resourceType string) error

DeleteResource removes a stored resource and its tags. AWS returns ResourceInUseException when deleting a classifier or recognizer that is still training (status SUBMITTED or IN_PROGRESS), and also -- per DeleteDocumentClassifierInput/DeleteEntityRecognizerInput's own doc comments ("If an active inference job is using the model, a ResourceInUseException will be returned") -- when an async detection job still referencing the model is itself SUBMITTED or IN_PROGRESS.

func (*InMemoryBackend) DeleteResourcePolicy 

func (b *InMemoryBackend) DeleteResourcePolicy(resourceArn, expectedRevision string) error

DeleteResourcePolicy removes a resource policy.

func (*InMemoryBackend) DescribeJob 

func (b *InMemoryBackend) DescribeJob(id, jobType string) (*Job, error)

DescribeJob retrieves and advances a submitted job through its lifecycle.

func (*InMemoryBackend) EndpointCustomEntityTypes 

func (b *InMemoryBackend) EndpointCustomEntityTypes(endpointArn string) ([]string, error)

EndpointCustomEntityTypes resolves endpointArn (DetectEntitiesInput's EndpointArn) to its backing entity recognizer's configured entity type names (resourceEntityTypeNames, handler_resources.go), for DetectEntities' custom-model detection path. Returns ErrNotFound if endpointArn does not name a known endpoint (matching DetectEntities' declared error set); a nil, no-error slice when the endpoint's model is not an entity recognizer or the recognizer has no configured types, since real AWS only branches detection behavior for a genuine custom recognizer endpoint.

func (*InMemoryBackend) GetFlywheelIteration 

func (b *InMemoryBackend) GetFlywheelIteration(id string) (*FlywheelIteration, error)

GetFlywheelIteration returns and advances an iteration. Real FlywheelIterationStatus values are TRAINING -> EVALUATING -> COMPLETED (types/enums.go:325-334) -- not the JobStatus-style SUBMITTED/IN_PROGRESS vocabulary used elsewhere in this file, which FlywheelIterationStatus does not share.

func (*InMemoryBackend) GetResource 

func (b *InMemoryBackend) GetResource(resourceArn, resourceType string) (*Resource, error)

GetResource finds resource by ARN. For classifier and recognizer types, each Describe call advances training lifecycle: SUBMITTED → IN_PROGRESS → TRAINED (or FAILED when the name contains "[fail]"). This mirrors AWS async training.

func (*InMemoryBackend) GetResourcePolicy 

func (b *InMemoryBackend) GetResourcePolicy(resourceArn string) (string, string, time.Time, time.Time, error)

GetResourcePolicy retrieves a resource policy along with its creation and last-modified times, which real DescribeResourcePolicy tracks per policy rather than stamping at read time.

func (*InMemoryBackend) IterationModelArn 

func (b *InMemoryBackend) IterationModelArn(iteration *FlywheelIteration) string

IterationModelArn returns the ARN of the model trained by iteration, or "" while the iteration has not yet produced one.

func (*InMemoryBackend) ListFlywheelIterations 

func (b *InMemoryBackend) ListFlywheelIterations(flywheelArn string) []*FlywheelIteration

ListFlywheelIterations lists iterations belonging to one flywheel.

func (*InMemoryBackend) ListJobs 

func (b *InMemoryBackend) ListJobs(jobType string) []*Job

ListJobs returns jobs for one operation family in stable submission order.

func (*InMemoryBackend) ListResources 

func (b *InMemoryBackend) ListResources(resourceType string) []*Resource

ListResources returns resources of one type. For classifier and recognizer types, listing advances the async training lifecycle one step (mirroring a status poll), consistent with how Describe advances it. This lets a create→describe→list→delete flow reach a deletable (TRAINED) state.

func (*InMemoryBackend) ListTags 

func (b *InMemoryBackend) ListTags(resourceArn string) ([]Tag, error)

ListTags returns sorted resource tags.

func (*InMemoryBackend) PutResourcePolicy 

func (b *InMemoryBackend) PutResourcePolicy(resourceArn, policy, expectedRevision string) (string, error)

PutResourcePolicy saves a resource policy.

func (*InMemoryBackend) Region 

func (b *InMemoryBackend) Region() string

Region returns configured AWS region.

func (*InMemoryBackend) Reset 

func (b *InMemoryBackend) Reset()

Reset clears all stored Comprehend resources.

func (*InMemoryBackend) Restore 

func (b *InMemoryBackend) Restore(ctx context.Context, data []byte) error

Restore deserializes backend state from a snapshot. It implements persistence.Persistable.

func (*InMemoryBackend) SetAppConfig 

func (b *InMemoryBackend) SetAppConfig(cfg any)

SetAppConfig records the service.AppContext.Config for lazy S3 lookup.

func (*InMemoryBackend) SetLifecycleDelay 

func (b *InMemoryBackend) SetLifecycleDelay(d time.Duration)

SetLifecycleDelay sets the dwell per stage for async jobs (SUBMITTED, then IN_PROGRESS, then terminal) and classifier/recognizer training (SUBMITTED, TRAINING, TRAINED). Zero (the default) advances one stage per Describe and trains instantly.

func (*InMemoryBackend) SetS3Writer 

func (b *InMemoryBackend) SetS3Writer(w S3Writer)

SetS3Writer overrides the S3 destination for job output.

func (*InMemoryBackend) Snapshot 

func (b *InMemoryBackend) Snapshot(ctx context.Context) []byte

Snapshot serializes the backend state to JSON. It implements persistence.Persistable.

func (*InMemoryBackend) StartFlywheelIteration 

func (b *InMemoryBackend) StartFlywheelIteration(flywheelArn string) (*FlywheelIteration, error)

StartFlywheelIteration creates an asynchronous flywheel iteration.

func (*InMemoryBackend) StartJob 

func (b *InMemoryBackend) StartJob(jobType, name string, values map[string]any, tags []Tag) (*Job, error)

StartJob submits an analysis job with AWS-style initial status. tags are associated with the job's ARN in the same tag store Create* resources use, so ListTagsForResource/TagResource/UntagResource work against job ARNs too (Start*DetectionJob requests all accept an optional Tags field in the real API).

func (*InMemoryBackend) StopJob 

func (b *InMemoryBackend) StopJob(id, jobType string) (*Job, error)

StopJob starts cancellation of an active job. AWS returns InvalidRequestException when the job is already in a terminal state (COMPLETED, FAILED, STOPPED, STOP_REQUESTED).

func (*InMemoryBackend) StopTrainingResource 

func (b *InMemoryBackend) StopTrainingResource(resourceArn, resourceType string) error

StopTrainingResource sets a trainable resource's status to STOPPED.

func (*InMemoryBackend) TagResource 

func (b *InMemoryBackend) TagResource(resourceArn string, tags []Tag) error

TagResource adds or replaces tags on an existing resource. AWS returns TooManyTagsException when the resulting tag set (existing keys plus newly requested ones) would exceed the 50-tag-per-resource limit.

func (*InMemoryBackend) TaggedResources added in v1.3.1 

func (b *InMemoryBackend) TaggedResources() []TaggedEntry

TaggedResources returns every Comprehend resource ARN that currently has at least one tag applied via TagResource.

func (*InMemoryBackend) UntagResource 

func (b *InMemoryBackend) UntagResource(resourceArn string, keys []string) error

UntagResource removes keys from an existing resource.

func (*InMemoryBackend) UpdateResource 

func (b *InMemoryBackend) UpdateResource(resourceArn, resourceType string, values map[string]any) (*Resource, error)

UpdateResource changes stored configuration of a mutable resource.

type Job 

type Job struct {
	SubmitTime time.Time
	EndTime    time.Time

	Configuration         map[string]any
	OutputDataConfig      map[string]any
	InputDataConfig       map[string]any
	JobArn                string
	EntityRecognizerArn   string
	JobType               string
	JobName               string
	DocumentClassifierArn string
	DataAccessRoleArn     string
	JobStatus             string
	LanguageCode          string
	JobID                 string
	FailureReason         string
	TargetEventTypes      []string
	// contains filtered or unexported fields
}

Job represents an asynchronous document-analysis job.

type Provider 

type Provider struct{}

Provider implements service.Provider for Amazon Comprehend.

func (*Provider) Init 

func (p *Provider) Init(ctx *service.AppContext) (service.Registerable, error)

Init creates a Comprehend backend and handler.

func (*Provider) Name 

func (p *Provider) Name() string

Name returns provider name.

type Resource 

type Resource struct {
	CreatedAt time.Time
	UpdatedAt time.Time
	// TrainingStartTime/TrainingEndTime back DocumentClassifierProperties/
	// EntityRecognizerProperties' TrainingStartTime/TrainingEndTime fields
	// (distinct from SubmitTime/EndTime: real AWS bills the interval between
	// these two separately from the submit-to-completion interval). Left
	// zero for resource types that don't train (endpoint/flywheel/dataset).
	TrainingStartTime time.Time
	TrainingEndTime   time.Time
	Name              string
	Arn               string
	Type              string
	Status            string
	VersionName       string
	ModelArn          string
	FlywheelArn       string
	EndpointArn       string
	DatasetArn        string
	Configuration     map[string]any
	FailureReason     string
}

Resource stores a Comprehend trainable or hosted resource.

type S3Writer 

type S3Writer interface {
	PutObject(ctx context.Context, bucket, key string, data []byte) error
}

S3Writer stores a finished job's output archive.

type Tag 

type Tag struct {
	Key   string `json:"Key"`
	Value string `json:"Value"`
}

Tag is a Comprehend resource tag.

type TaggedEntry added in v1.3.1 

type TaggedEntry struct {
	Tags map[string]string
	ARN  string
}

TaggedEntry pairs a resource ARN with its tags.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.