s3

package
v1.162.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 16, 2026 License: Apache-2.0 Imports: 27 Imported by: 0

Documentation

Overview

Package s3 provides an S3 toolkit adapter for the MCP data platform.

Index

Constants

View Source
const (
	// DefaultTimeout is the default HTTP client timeout for S3 operations.
	DefaultTimeout = 30 * time.Second

	// DefaultMaxGetSize is the default maximum size for S3 GET operations (10MB).
	DefaultMaxGetSize = 10 * 1024 * 1024

	// DefaultMaxPutSize is the default maximum size for S3 PUT operations (100MB).
	DefaultMaxPutSize = 100 * 1024 * 1024
)
View Source
const ConfigSchemaJSON = `` /* 2201-byte string literal not displayed */

ConfigSchemaJSON declares what an s3 connection's config takes. See the same constant on the trino kind for why a kind declares this at all (#1805).

Variables

View Source
var SecretKeys = []string{"access_key_id", "secret_access_key", "session_token"}

SecretKeys are the configuration keys an s3 connection may name a stored secret in as {{secret:<name>}} (#1806). The value is read as the connection's client is built.

Functions

func PlatformBuckets added in v1.159.0

func PlatformBuckets(portalBucket, resourceBucket string) func() []ReservedBucket

PlatformBuckets is the platform's two stores as reserved buckets: the portal bucket (assets, exports, script outputs and site releases) and the managed-resource bucket (the resource library, webhook events and the scratch catalog's tables, and street maps unless their settings name a bucket of their own). An empty name reserves nothing for that store.

func SetReservedBuckets added in v1.159.0

func SetReservedBuckets(fn func() []ReservedBucket)

SetReservedBuckets installs the buckets every S3 toolkit refuses; nil reserves none.

Types

type AnnotationConfig added in v0.20.0

type AnnotationConfig = toolkit.AnnotationConfig

AnnotationConfig holds tool annotation overrides from configuration.

type Config

type Config struct {
	Region          string                      `yaml:"region"`
	Endpoint        string                      `yaml:"endpoint"`
	PublicEndpoint  string                      `yaml:"public_endpoint"`
	AccessKeyID     string                      `yaml:"access_key_id"`
	SecretAccessKey string                      `yaml:"secret_access_key"`
	SessionToken    string                      `yaml:"session_token"` // #nosec G117 -- S3 session token from admin YAML config
	Profile         string                      `yaml:"profile"`
	UsePathStyle    bool                        `yaml:"use_path_style"`
	Timeout         time.Duration               `yaml:"timeout"`
	DisableSSL      bool                        `yaml:"disable_ssl"`
	ReadOnly        bool                        `yaml:"read_only"`
	MaxGetSize      int64                       `yaml:"max_get_size"`
	MaxPutSize      int64                       `yaml:"max_put_size"`
	ConnectionName  string                      `yaml:"connection_name"`
	Description     string                      `yaml:"description"`
	BucketPrefix    string                      `yaml:"bucket_prefix"`
	Titles          map[string]string           `yaml:"titles"`
	Descriptions    map[string]string           `yaml:"descriptions"`
	Annotations     map[string]AnnotationConfig `yaml:"annotations"`
}

Config holds S3 toolkit configuration.

func ParseConfig

func ParseConfig(cfg map[string]any) (Config, error)

ParseConfig parses an S3 toolkit configuration from a map.

type MultiConfig added in v1.129.0

type MultiConfig struct {
	DefaultConnection string
	Instances         map[string]Config
}

MultiConfig is every S3 instance a deployment declares, and which of them a call that names no connection means.

func ParseMultiConfig added in v1.129.0

func ParseMultiConfig(defaultName string, instances map[string]map[string]any) (MultiConfig, error)

ParseMultiConfig parses every instance of the s3 kind. An instance whose config does not parse is logged and skipped rather than blocking startup.

type ReservedBucket added in v1.159.0

type ReservedBucket struct {
	// Name is the bucket as configured.
	Name string
	// Holds says what the platform keeps there.
	Holds string
	// Use names the platform surface that reads it instead.
	Use string
}

ReservedBucket is a bucket the platform keeps its own data in (#2616). No S3 tool action and no bucket source reaches it, on any connection and for any persona, administrators included: what it holds has an owner and a scope (a personal resource, an unshared asset, a private site release, a webhook payload) that only the platform's own surfaces check.

func IsReserved added in v1.159.0

func IsReserved(bucket string) (ReservedBucket, bool)

IsReserved reports whether a bucket is one the platform keeps its own data in, and which.

type Toolkit

type Toolkit struct {
	// contains filtered or unexported fields
}

Toolkit is the one S3 toolkit of a deployment: every instance is a connection of it, routed by the `connection` argument, so two instances never register the same tool name twice (the SDK keeps the last registration, which left every earlier instance unreachable).

func New

func New(name string, cfg Config) (*Toolkit, error)

New creates an S3 toolkit over one instance.

func NewMulti added in v1.129.0

func NewMulti(cfg MultiConfig) (*Toolkit, error)

NewMulti creates the toolkit over every instance. Each instance is bound by its connection_name (its instance name when it sets none), which is the name a call's `connection` argument carries, an audit row records, and a persona's connection rules match. The default instance serves a call naming none; with no default declared the alphabetically first instance is it.

func (*Toolkit) AddConnection added in v1.48.0

func (t *Toolkit) AddConnection(name string, config map[string]any) error

AddConnection adds a named S3 connection at runtime.

func (*Toolkit) BucketNames added in v1.159.0

func (t *Toolkit) BucketNames(ctx context.Context, connection string) ([]string, error)

BucketNames is every bucket a connection's credentials list, with no limit of the toolkit's applied: what the boot check reads to warn of a granted connection that reaches the platform's buckets or a backup (#2616). It is never answered to a caller.

func (*Toolkit) Client

func (t *Toolkit) Client() *s3client.Client

Client returns the underlying S3 client for direct use.

func (*Toolkit) Close

func (t *Toolkit) Close() error

Close releases every connection's client.

func (*Toolkit) Config

func (t *Toolkit) Config() Config

Config returns the toolkit configuration.

func (*Toolkit) Connection added in v0.12.1

func (t *Toolkit) Connection() string

Connection returns the name a call that names no connection binds: the default connection's bound name.

func (*Toolkit) HasConnection added in v1.48.0

func (t *Toolkit) HasConnection(name string) bool

HasConnection returns true if a connection with the given name exists.

func (*Toolkit) Kind

func (*Toolkit) Kind() string

Kind returns the toolkit kind.

func (*Toolkit) ListConnections added in v1.129.0

func (t *Toolkit) ListConnections() []toolkit.ConnectionDetail

ListConnections reports every connection this toolkit serves, by bound name, for list_connections and the connection resolver.

func (*Toolkit) Name

func (t *Toolkit) Name() string

Name returns the toolkit instance name.

func (*Toolkit) OutsideBucketPrefix added in v1.158.0

func (t *Toolkit) OutsideBucketPrefix(connection, bucket string) (prefix string, outside bool)

OutsideBucketPrefix reports whether a bucket lies outside the bucket_prefix of the connection named (the default connection for ""), and the prefix it was compared with: the limit refuseOutsidePrefix enforces on every call, asked ahead of one by the boot check over script destinations (#2584).

func (*Toolkit) ProbeConnection added in v1.133.4

func (t *Toolkit) ProbeConnection(ctx context.Context, name string) connprobe.Result

ProbeConnection opens the named connection and lists its buckets, which is the smallest request that proves the endpoint is reachable and the credential is accepted (#1805).

A connection carrying a bucket prefix lists what the prefix admits, so the count reported is the buckets this connection can actually see rather than the account's.

Implements connprobe.Prober.

func (*Toolkit) ReadClient added in v1.159.0

func (t *Toolkit) ReadClient(connection, bucket string) (s3tools.S3Client, error)

ReadClient is the client a reader of bucket on the named connection uses outside a tool call (#2593: a site or a resource folder loaded from a bucket prefix), refused for an unknown connection and for a bucket outside the connection's bucket_prefix, the limit s3_list and s3_object apply (#2584). A connection is always named: the default connection is never chosen for a caller who named none.

func (*Toolkit) RegisterTools

func (t *Toolkit) RegisterTools(s *mcp.Server)

RegisterTools registers s3_list and s3_object with the MCP server. The platform's unified list_connections stands in for upstream's s3_list_connections, so nothing else from mcp-s3 is registered.

func (*Toolkit) RemoveConnection added in v1.48.0

func (t *Toolkit) RemoveConnection(name string) error

RemoveConnection removes a named S3 connection at runtime.

func (*Toolkit) SetMetrics added in v1.70.0

func (t *Toolkit) SetMetrics(m *observability.Metrics)

SetMetrics installs the recorder the S3 handlers report to. The platform calls this before RegisterTools, when metrics OR tracing is enabled; the handlers record nil-safely (no-op when m is disabled) and emit spans only inside an active trace, which is what makes a tracing-only deployment (m nil) still produce S3 spans.

func (*Toolkit) SetQueryProvider

func (t *Toolkit) SetQueryProvider(provider query.Provider)

SetQueryProvider sets the query execution provider for enrichment.

func (*Toolkit) SetSemanticProvider

func (t *Toolkit) SetSemanticProvider(provider semantic.Provider)

SetSemanticProvider sets the semantic metadata provider for enrichment.

func (*Toolkit) Tools

func (*Toolkit) Tools() []string

Tools returns the tool names this toolkit registers. Both are registered whatever read_only says: s3_object carries the read actions too, and a writing action on a read-only connection is refused by the handler, naming the connection.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL