bucketsource

package
v1.164.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 18, 2026 License: Apache-2.0 Imports: 14 Imported by: 0

Documentation

Overview

Package bucketsource reads the objects under a bucket prefix on an S3 connection, as the caller (#2593): a site's source tree or a media folder of tens of thousands of files arrives from where it already sits, without passing through tool calls.

Threat model.

  • Boundary: the caller (a person in the portal, or an agent's tool call, identified by the PlatformContext on the request) reaching a bucket on one named S3 connection. The objects then cross a second boundary into a store the caller writes: a site's release (manage_site publish) or a folder of the managed-resource library (manage_resource extract), each under the caller's own authority there, checked by that store.
  • What a weaker persona can make it do: nothing beyond s3_list and s3_object get on the same connection. The caller's persona must hold both tools and the connection, by the persona rules every tool call is authorized under (pkg/persona: a persona denied s3_object, or granted s3_list alone, reads no object this way either); the bucket must be inside the connection's bucket_prefix (#2584), by the S3 toolkit's own check; and the connection is always named, so the default connection is never read for a caller who named none (an empty connection name is "platform-level" to the persona rules, and must not reach a bucket).
  • What is stored: the bytes of each object, copied into the site's store or the resource library at its path under the prefix. A name no folder or release can hold (a parent reference, an empty or dotted segment, a backslash, a control character) is refused before anything is read.
  • Secrets: the connection's credentials stay inside its client. No response, log, audit row or error carries them: a failure from the object store is reported by its code (AccessDenied, NoSuchBucket) and the bucket and prefix the caller named, never by the client's error text, which can name the endpoint it was configured with.
  • Size: every limit is checked from the listing, before a body is read. An object can be replaced between the listing and the read, so each is read as a range of at most its limit and one byte more, never whole, and the bytes read are added up against the total limit as they arrive: a bucket someone else writes cannot make a read hold more than the limits allow.

Index

Constants

This section is empty.

Variables

View Source
var ErrRefused = errors.New("bucket source refused")

ErrRefused is a source the caller can act on: an unknown connection, one the persona may not use, a bucket outside the connection's limit, a name no store can hold, or a limit passed. The message says which.

Functions

This section is empty.

Types

type Access

type Access struct {
	Clients Clients
	// Allowed reports whether a persona may use a connection; nil allows
	// every connection, as a deployment with no personas allows every
	// connection to its tool calls.
	Allowed func(persona, connection string) bool
	// ToolAllowed reports whether a persona holds a tool; nil allows every
	// tool, as a deployment with no personas does.
	ToolAllowed func(persona, tool string) bool
}

Access opens sources as the caller.

func FromToolkits

func FromToolkits(reg *registry.Registry, personas *persona.Registry) *Access

FromToolkits is the access a deployment grants: a connection's reading client from its S3 toolkit, read on every open so a connection added at run time is the one used, and the persona connection and tool rules from the live registry. No rule is restated here: ReadClient is the S3 toolkit's own bucket_prefix check, and Allowed and ToolAllowed are the persona filter a tool call is authorized under.

func (*Access) Open

func (a *Access) Open(ctx context.Context, src Source) (*Bucket, error)

Open checks the caller may read the source and returns it. The caller's persona comes from the request's PlatformContext.

type Bucket

type Bucket struct {
	// contains filtered or unexported fields
}

Bucket is one source opened for the caller.

func (*Bucket) List

func (b *Bucket) List(ctx context.Context, lim Limits) ([]Object, error)

List is every object under the prefix, page by page, its limits checked from the listing before any body is read. A key ending in a slash is a folder marker and is passed over.

func (*Bucket) Read

func (b *Bucket) Read(ctx context.Context, o Object) (body []byte, contentType string, err error)

Read is one object's bytes and the content type it was stored with, held to the limits the listing was checked against: the object can have been replaced since, so it is read as a range of at most its limit and one byte more, refused past the limit, and the bytes every read takes are added up against the total limit. Reads may run at once. The type is what its writer declared, which a reader treats as a create's declaration: a specific one is kept, a generic one is detected from the bytes. A zero-byte object has no byte a range can start at, and is read as empty with no type.

func (*Bucket) Source

func (b *Bucket) Source() Source

Source is what the bucket was opened on.

type Client

type Client interface {
	ListObjects(ctx context.Context, bucket, prefix, delimiter string, maxKeys int32, continueToken string) (*s3client.ListObjectsOutput, error)
	GetObjectRange(ctx context.Context, bucket, key string, offset, length int64) (*s3client.ObjectContent, error)
}

Client is what a reader of one bucket uses: s3_list's listing, and a read of a byte range, so an object is never held whole past its limit.

type Clients

type Clients interface {
	ReadClient(connection, bucket string) (Client, error)
}

Clients finds the client reading a bucket on a connection, refusing an unknown connection or a bucket outside the connection's bucket_prefix.

type Limits

type Limits struct {
	MaxObjects     int
	MaxObjectBytes int64
	MaxTotalBytes  int64

	ObjectsKey, ObjectBytesKey, TotalBytesKey string
}

Limits bound a listing: how many objects, how large one may be, and how much they may add up to. The *Key fields are the configuration keys that lift each limit, which a refusal names. A non-positive bound is unbounded.

type Object

type Object struct {
	Key  string
	Path string
	Size int64
}

Object is one object under the prefix: its key, its path relative to the prefix, and the size the listing reported.

type Source

type Source struct {
	Connection string `json:"connection"`
	Bucket     string `json:"bucket"`
	Prefix     string `json:"prefix,omitempty"`
}

Source names the objects under a prefix of a bucket on a connection.

func (Source) String

func (s Source) String() string

String is the source as a reader names it: s3://bucket/prefix on connection.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL