blob

package
v1.19.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 30, 2026 License: MIT Imports: 31 Imported by: 0

Documentation

Index

Constants

View Source
const (

	// MaxOriginalImageSizeBytes bounds the declared size of an ORIGINAL image upload.
	// It is pinned into the upload policy, so storage rejects anything larger before a
	// single byte lands.
	MaxOriginalImageSizeBytes = 8 * 1024 * 1024 // 8 MiB

)

Variables

View Source
var (
	// ErrImageCorrupt means the bytes could not be read or decoded as an image.
	ErrImageCorrupt = errors.New("image is corrupt or undecodable")

	// ErrImageUnsupportedType means the bytes are a kind the service does not
	// accept — an unsupported format, or an animated image.
	ErrImageUnsupportedType = errors.New("unsupported image type")

	// ErrImageTooLarge means the image's pixel dimensions exceed the limits.
	ErrImageTooLarge = errors.New("image exceeds dimension limits")
)

InspectImage failure categories. They are wrapped into the descriptive error it returns so finalization can classify a rejection into the right RejectionReason without re-deriving why the bytes were unacceptable. A failure carrying none of these is an internal processing fault.

View Source
var (
	// ErrNotFound is returned when no blob exists for the given id.
	ErrNotFound = errors.New("blob not found")

	// ErrExists is returned when a blob with the given id already exists.
	ErrExists = errors.New("blob already exists")

	// ErrCannotAdvanceToRejected is returned by Store.Advance when StateRejected is
	// passed as the target. Rejection is terminal and carries metadata, so it is
	// reached only through Store.Reject, never Advance.
	ErrCannotAdvanceToRejected = errors.New("cannot advance to rejected state; use Reject")
)
View Source
var ErrInvalidGrant = errors.New("invalid grant")

ErrInvalidGrant is returned by AccessStore methods when a grant — or the key used to look one up — is not well-formed: a missing blob id, an unknown principal type, an empty principal id, or an unknown permission.

View Source
var ErrObjectNotFound = errors.New("object not found")

ErrObjectNotFound is returned by ObjectStorage.GetUploaded when no bytes have been uploaded for the key yet (the upload never happened or is incomplete).

View Source
var SupportedImageMimeTypes = func() map[string]bool {
	set := make(map[string]bool, len(imageFormatToMimeType))
	for _, mimeType := range imageFormatToMimeType {
		set[mimeType] = true
	}
	return set
}()

SupportedImageMimeTypes is the set of MIME types a client may declare for an upload. It is derived from the decodable formats, so the type pinned into the upload policy is always one the server can re-derive and validate from the stored bytes. It is the single source of truth for "is this an image we accept".

Functions

func StorageKey

func StorageKey(id *blobpb.BlobId, mimeType string) (string, error)

StorageKey derives the object key for an image blob's bytes from its id and mime type. Images can have server-derived renditions (display, thumbnail, ...), so an image's bytes live under a per-media-item directory keyed by its id, leaving room to group its renditions under the same prefix:

images/<uuid>/original.jpg

Only images are supported today, so a non-image mime type is rejected outright rather than being silently forced into the image layout. Other kinds (videos, files) may want a different prefix and layout — keyed off the kind, not merely a resolvable extension — so adding one has to make a deliberate decision here. The extension is derived from the (immutable) mime type, and the same key is used in both the upload and origin stores.

Types

type AccessStore

type AccessStore interface {
	// Grant records that the grant's principal may exercise its permission on its
	// blob. It is idempotent: re-granting the same (blob, principal, permission)
	// is a no-op. It returns ErrInvalidGrant if the grant is not well-formed.
	Grant(ctx context.Context, g *Grant) error

	// HasGrant reports whether a grant exists for the exact (blob, principal,
	// permission) triple. A missing grant is (false, nil), not an error. It
	// returns ErrInvalidGrant if the lookup key is not well-formed.
	HasGrant(ctx context.Context, blobID *blobpb.BlobId, p Principal, perm Permission) (bool, error)

	// Revoke removes a grant. It is idempotent: revoking a grant that does not
	// exist is a no-op. It returns ErrInvalidGrant if the key is not well-formed.
	Revoke(ctx context.Context, blobID *blobpb.BlobId, p Principal, perm Permission) error
}

AccessStore persists blob ACL grants. A grant's existence authorizes its principal to exercise its permission on the blob; there is no other state.

The store resolves a grant by its exact (blob, principal, permission) key and performs no membership resolution, so it never depends on the chat — or any other principal — subsystem. Authorizing a concrete user against a non-user principal (e.g. checking chat membership for a PrincipalTypeChat grant) is the caller's responsibility.

type Blob

type Blob struct {
	ID *blobpb.BlobId

	// Rendition is which rendition of its media this blob holds. An ORIGINAL has
	// a nil ParentID; any other rendition type is a server-derived variant with
	// ParentID pointing at the original.
	Rendition RenditionType

	// ParentID is set when this blob is a server-derived rendition of another
	// blob; it points at the ORIGINAL the client uploaded. It is nil for an
	// ORIGINAL. Renditions are never uploaded by clients — the server derives
	// them from the original's bytes.
	ParentID *blobpb.BlobId

	Owner *commonpb.UserId

	// State is the blob's internal lifecycle state — the source of truth for how
	// far processing has progressed. The public blobpb.BlobStatus is derived from
	// it via State.ToBlobStatus.
	State State

	// StorageKey is the object key the bytes live under in the backing store. It
	// is derived from the ID and never leaves the server.
	StorageKey string

	// MimeType is the declared MIME type, pinned at reservation and immutable.
	MimeType string

	// SizeBytes is the declared size, pinned at reservation and immutable.
	SizeBytes uint64

	// Image is the derived IMAGE metadata, set only when this blob is an image
	// and READY. It is the image variant of the blob's kind-specific metadata;
	// as additional content kinds are supported they will be carried by their
	// own sibling fields here (e.g. Video, Audio), one per blobpb.BlobMetadata
	// kind variant. Only images exist today.
	Image *ImageMetadata

	// Rejection records why this blob was rejected, set only when State is
	// StateRejected; it is nil for any non-rejected blob.
	Rejection *RejectionMetadata
}

Blob is the server-authoritative record for a stored blob. It is the durable identity behind a BlobId and tracks the blob through its lifecycle.

The MimeType and SizeBytes are declared by the client on reservation and pinned into the signed upload policy, so storage rejects any upload that does not match them. They are immutable for the life of the blob: finalization re-validates the stored bytes against them and REJECTs the blob on any mismatch rather than overwriting them. Only the derived kind-specific metadata is filled in at finalization.

func (*Blob) Clone

func (b *Blob) Clone() *Blob

Clone returns a deep copy of the blob, so stores can hand out values callers cannot mutate in place.

type ChatResolver

type ChatResolver struct {
	// contains filtered or unexported fields
}

ChatResolver is the PrincipalResolver for chat-scoped grants: a user is covered by a PrincipalTypeChat principal iff they are a member of the chat the principal identifies. It resolves membership directly against the chat store.

func (*ChatResolver) Covers

func (r *ChatResolver) Covers(ctx context.Context, principal Principal, user *commonpb.UserId) (bool, error)

Covers reports whether user is covered by principal. A PrincipalTypeChat principal is covered iff user is a member of the chat identified by the principal id. Any other principal type is outside this resolver's scope — it is chat-only — so it reports not-covered rather than guessing; supporting another scope is a matter of layering a different resolver, not changing this one.

type CompositeResolver

type CompositeResolver struct {
	// contains filtered or unexported fields
}

CompositeResolver is the top-level PrincipalResolver: it routes each principal to the domain resolver registered for its type. The server holds one of these, so adding an access surface is a matter of registering its resolver here — the read path does not change. A principal whose type has no registered resolver is not covered, mirroring how an unknown access scope is treated.

func (*CompositeResolver) Covers

func (r *CompositeResolver) Covers(ctx context.Context, principal Principal, user *commonpb.UserId) (bool, error)

Covers routes principal to the resolver registered for its type and returns that resolver's decision. A principal whose type has no registered resolver is not covered (false, nil) — an unroutable scope authorizes nothing.

type Grant

type Grant struct {
	BlobID     *blobpb.BlobId
	Principal  Principal
	Permission Permission
}

Grant authorizes a Principal to exercise a Permission on a blob. It is the ACL entry: its existence is the authorization and it carries no other state. A blob's Owner holds every permission implicitly and needs no grant. Grants are made against the ORIGINAL blob id; server-derived renditions inherit their original's grants.

func (*Grant) Validate

func (g *Grant) Validate() error

Validate reports whether the grant is well-formed. Stores call it before persisting or looking up a grant so a malformed principal or permission can never be written or silently miss.

type ImageInspection

type ImageInspection struct {
	MimeType string
	Metadata *ImageMetadata

	// Decoded is the decoded image, retained so callers can derive other
	// renderings (e.g. the moderation payload) without decoding the bytes again.
	Decoded image.Image
}

ImageInspection is the result of decoding image bytes: the MIME type authoritatively derived from the bytes plus the intrinsic image metadata.

func InspectImage

func InspectImage(data []byte) (*ImageInspection, error)

InspectImage decodes the bytes as an image and derives their authoritative MIME type, pixel dimensions, and BlurHash. It returns an error if the bytes are not a decodable image of a supported format, or if the image's pixel count exceeds maxImagePixels; callers treat either as a rejection.

type ImageMetadata

type ImageMetadata struct {
	Width    uint32
	Height   uint32
	Blurhash string
}

ImageMetadata holds the server-derived, intrinsic descriptors of a still image. Every field is derived once from the stored bytes and is immutable.

This is the IMAGE variant of a blob's kind-specific metadata. It is populated only for blobs whose bytes are an image; other content kinds (video, audio, ...) will each carry their own distinct metadata type, mirroring the blobpb.BlobMetadata.kind oneof. Images are simply the only kind supported today.

type ObjectStorage

type ObjectStorage interface {
	// PresignUpload mints a short-lived, presigned target the client uploads the
	// bytes to directly, into the UPLOAD store under the given key. The target
	// pins the declared content type and exact size, so storage rejects any
	// upload that does not match.
	PresignUpload(ctx context.Context, key, mimeType string, sizeBytes uint64) (*blobpb.UploadTarget, error)

	// GetUploaded returns the bytes a client uploaded under the key in the UPLOAD
	// store, or ErrObjectNotFound if no bytes are present yet. Finalization reads
	// them back to validate the content before promoting it.
	GetUploaded(ctx context.Context, key string) ([]byte, error)

	// CopyToOrigin copies a validated object from the UPLOAD store to the ORIGIN
	// store under the same key, making it servable through the CDN. It overwrites
	// any object already served under the key, so it is safe to call more than
	// once. The upload copy is intentionally left in place — the caller removes it
	// with DeleteUpload only after the blob's terminal state is durably recorded,
	// so an interrupted finalization is always replayable from the upload bytes.
	CopyToOrigin(ctx context.Context, key string) error

	// DeleteUpload removes an object from the UPLOAD store. It is best-effort
	// cleanup run after a blob reaches a terminal state, and is idempotent:
	// deleting an absent key is not an error.
	DeleteUpload(ctx context.Context, key string) error

	// SignDownloadURL mints a fresh, short-lived CDN URL for fetching a promoted
	// object's bytes from the ORIGIN store, paired with the instant it expires. It
	// is authorized at mint time and expires on its own, so callers mint a new one
	// rather than persisting it.
	SignDownloadURL(ctx context.Context, key string) (*blobpb.DownloadUrl, error)
}

ObjectStorage is the bytes backend behind blobs. It spans two object stores: an UPLOAD store clients write to directly, and an ORIGIN store — fronted by a CDN — that only validated bytes are promoted into. Quarantining untrusted uploads this way means nothing is ever served until the server has read it back, validated it, and promoted it.

The server never proxies blob bytes: it presigns upload targets, reads uploaded bytes back to validate them, promotes the good ones, and mints signed CDN URLs for serving. It is deliberately provider-agnostic. The production implementation is two S3 buckets (presigned PUT into the upload bucket, GetObject to read it back, a server-side copy into the origin bucket) with a CloudFront CDN in front of the origin bucket; an in-memory implementation backs tests.

type Permission

type Permission int

Permission is what a grant authorizes a principal to do with a blob. READ is the only permission today; WRITE, DELETE, and SHARE can be added without changing the store. Persisted by value, so the numbering is significant.

const (
	PermissionUnknown Permission = iota

	// PermissionRead authorizes resolving the blob and minting a download URL for
	// it — the access GetBlobs grants.
	PermissionRead
)

type Principal

type Principal struct {
	Type PrincipalType
	ID   []byte
}

Principal is the typed subject a grant is made to. ID is the type-specific identifier bytes: a user id for PrincipalTypeUser, a chat id for PrincipalTypeChat.

func PrincipalForChat

func PrincipalForChat(chatID *commonpb.ChatId) Principal

PrincipalForChat returns the principal for the members of a chat.

func PrincipalForUser

func PrincipalForUser(userID *commonpb.UserId) Principal

PrincipalForUser returns the principal for a single user.

type PrincipalResolver

type PrincipalResolver interface {
	Covers(ctx context.Context, principal Principal, user *commonpb.UserId) (bool, error)
}

PrincipalResolver reports whether a concrete user is covered by a principal — that is, whether a grant made to that principal authorizes the user. A PrincipalTypeUser principal is covered by identity; a group principal such as PrincipalTypeChat is covered by live membership.

It is the single extension point the read path consults to resolve a grant's principal, so supporting a new access surface is a matter of adding a PrincipalType and teaching the resolver about it — the server does not change. Defining it here, rather than depending on the chat (or any future scope) subsystem, keeps blob decoupled from what backs each principal type; the wiring supplies a resolver that knows how to resolve them (e.g. mapping a PrincipalTypeChat to chat membership).

func NewChatResolver

func NewChatResolver(chats chat.Store) PrincipalResolver

NewChatResolver returns a ChatResolver backed by the given chat store.

func NewCompositeResolver

func NewCompositeResolver(byType map[PrincipalType]PrincipalResolver) PrincipalResolver

NewCompositeResolver returns a CompositeResolver that dispatches a principal to the resolver registered for its PrincipalType. The routing table is copied, so later mutation of the passed map does not affect the resolver.

type PrincipalType

type PrincipalType int

PrincipalType identifies the kind of subject a grant is made to. It is the extension point for "where a blob can be accessed from": each access surface (a chat today; a feed, a profile, a public link later) is a principal type with its own server-side membership check. The constants are persisted by value, so their numbering is significant and must not be reordered.

const (
	PrincipalTypeUnknown PrincipalType = iota

	// PrincipalTypeUser is a single user, identified by their user id. A user
	// principal is resolved by direct identity, with no membership lookup.
	PrincipalTypeUser

	// PrincipalTypeChat is every current member of a chat, identified by the chat
	// id. A chat principal is resolved against live chat membership, so a single
	// grant covers the whole chat and stays correct as membership changes.
	PrincipalTypeChat
)

type RejectionMetadata

type RejectionMetadata struct {
	Reason RejectionReason

	// FlaggedCategory is the moderation category that tripped, set only when
	// Reason is RejectionReasonModeration; it is NONE (the zero value) otherwise.
	FlaggedCategory moderationpb.FlaggedCategory
}

RejectionMetadata records why a blob was rejected during finalization. It is set only on a StateRejected blob and is immutable thereafter.

func (*RejectionMetadata) ToProto

ToProto renders the rejection metadata for the wire. A nil receiver renders to nil, so a non-rejected blob simply carries no rejection.

type RejectionReason

type RejectionReason int

RejectionReason is the internal mirror of blobpb.RejectionReason: why a blob's uploaded bytes failed finalization. It is set only on a StateRejected blob, is immutable thereafter, and is its own type (persisted as its own value) so the stored representation does not depend on the wire enum's numbering.

const (
	RejectionReasonUnknown RejectionReason = iota
	RejectionReasonModeration
	RejectionReasonUnsupportedType
	RejectionReasonMismatchedType
	RejectionReasonTooLarge
	RejectionReasonCorrupt
	RejectionReasonInternal
)

func (RejectionReason) ToProto

ToProto maps the internal reason onto the public blobpb.RejectionReason.

type RenditionType

type RenditionType int

RenditionType identifies which rendition of a piece of media a blob holds. The ORIGINAL is the exact bytes the client uploaded; every other type is a variant the server derives from that original. It is a server-internal concept: clients only ever upload and reference ORIGINALs, and the server derives and serves the rest.

const (
	RenditionUnknown RenditionType = iota

	// RenditionOriginal is the exact bytes the client uploaded.
	RenditionOriginal

	// RenditionDisplay is a server-derived variant sized and optimized for
	// inline display — e.g. rendering the image inline within a chat message or
	// feed — rather than serving the full-resolution original.
	RenditionDisplay

	// RenditionThumbnail is a small, server-derived preview image — e.g. a
	// grid/list thumbnail — smaller than the display rendition.
	RenditionThumbnail
)

type Server

type Server struct {
	blobpb.UnimplementedBlobStorageServer
	// contains filtered or unexported fields
}

func NewServer

func NewServer(
	log *zap.Logger,
	authz auth.Authorizer,
	accounts account.Store,
	blobs Store,
	storage ObjectStorage,
	access AccessStore,
	resolver PrincipalResolver,
	moderator moderation.Client,
	requireStaff bool,
) *Server

func (*Server) GetBlobs

func (*Server) GetUploadPolicy

GetUploadPolicy returns the upload constraints in force for the caller. It is advisory and cacheable: a client uses it to validate and resize before reserving an upload, but InitiateExternalUpload remains authoritative. Access is gated identically to initiating an upload, so a caller who could not upload does not receive a policy.

type State

type State int

State is the blob's internal, fine-grained lifecycle state. It records how far processing has progressed so an interrupted finalize can resume from the last completed checkpoint instead of repeating expensive steps — re-reading the bytes, re-deriving metadata, re-moderating, re-copying. It is deliberately finer-grained than the public blobpb.BlobStatus, which it maps onto, so the proto enum stays a derived view and this state is the source of truth.

The success path advances strictly forward — Pending → Uploaded → Inspected → Promoted → GeneratingRenditions → Ready — with Rejected an alternative terminal. The ordering of the constants is significant: a blob is only ever advanced to a higher-ranked state.

const (
	// StatePending is a freshly reserved blob awaiting the client's upload.
	StatePending State = iota

	// StateUploaded means the client's upload is complete and the bytes are
	// present in the upload store. This is the signal a processing worker keys off
	// of to begin deriving metadata, moderating, and promoting the blob.
	StateUploaded

	// StateInspected means the uploaded bytes were validated against the declared
	// type/size, the metadata was derived, and moderation passed. The derived
	// metadata is persisted at this checkpoint, so resuming skips re-moderation.
	StateInspected

	// StatePromoted means the original's bytes were copied into the origin (CDN)
	// store. The blob is NOT client-ready yet — its renditions have not been
	// generated. Resuming skips the copy.
	StatePromoted

	// StateGeneratingRenditions means the original is in the origin store and the
	// server is deriving its renditions (display, thumbnail) from it. The blob is
	// still not client-ready, so clients do not see READY until this completes.
	// Reserved for when rendition generation is implemented; nothing enters it yet.
	StateGeneratingRenditions

	// StateReady means processing is complete — the renditions are generated and
	// the upload-store bytes have been cleaned up — so the blob is client-ready.
	// Terminal.
	StateReady

	// StateRejected means the bytes failed validation or moderation. Terminal.
	StateRejected
)

func (State) Terminal

func (s State) Terminal() bool

Terminal reports whether no further processing is possible from this state.

func (State) ToBlobStatus

func (s State) ToBlobStatus() blobpb.BlobStatus

ToBlobStatus maps the internal state onto the public lifecycle status. A blob is reported READY only once it is fully processed — its renditions generated — so a client never references it (e.g. in a message) before the renditions it will use exist.

type Store

type Store interface {
	// CreatePending inserts a freshly reserved blob in the PENDING state. The
	// blob may be an ORIGINAL (nil ParentID) or a server-created rendition of an
	// existing original (ParentID set).
	//
	// ErrExists is returned if a blob with the same id already exists.
	CreatePending(ctx context.Context, blob *Blob) error

	// GetByID returns the blob with the given id, or ErrNotFound. Ownership is
	// not enforced here; callers that act on behalf of the uploader (e.g.
	// finalization) compare Owner themselves.
	GetByID(ctx context.Context, id *blobpb.BlobId) (*Blob, error)

	// GetByIDs returns the blobs among the given ids that exist, in unspecified
	// order. A BlobId is an opaque capability, so resolution is not scoped to an
	// owner; ids that do not exist are simply omitted. A BlobId resolves to at
	// most one record, so duplicate ids collapse to a single result.
	GetByIDs(ctx context.Context, ids []*blobpb.BlobId) ([]*Blob, error)

	// GetRenditions returns every blob whose ParentID is parentID — i.e. all
	// server-derived renditions of the given original — in unspecified order. It
	// returns an empty slice (not ErrNotFound) when there are none.
	GetRenditions(ctx context.Context, parentID *blobpb.BlobId) ([]*Blob, error)

	// Advance moves a blob forward along the success path to a later lifecycle
	// state, persisting derived metadata when provided (image is set only on the
	// transition into StateInspected). It advances strictly forward and never out
	// of a terminal state, so a replayed or concurrent finalize is idempotent:
	// advancing to a state the blob is already at or past is a no-op. The declared
	// MimeType and SizeBytes are never changed.
	//
	// StateRejected is not a valid target — rejection is terminal and carries
	// metadata, so it is reached only through Reject. Passing it returns
	// ErrCannotAdvanceToRejected.
	//
	// It reports whether this call actually performed the transition. A false
	// return with a nil error means the blob was already at or past the target
	// (or terminal) — a concurrent or replayed finalize lost the race — so the
	// caller can stop instead of applying further side effects on a stale view.
	//
	// ErrNotFound is returned if no blob exists for the given id.
	Advance(ctx context.Context, id *blobpb.BlobId, to State, image *ImageMetadata) (bool, error)

	// Reject moves a non-terminal blob to the terminal StateRejected, recording
	// why. Like Advance it transitions only out of a non-terminal state and is
	// idempotent: it reports whether it performed the transition, and a false with
	// a nil error means the blob was already terminal — a concurrent or replayed
	// finalize won the race, so the committed rejection (or readiness) stands and
	// must not be overwritten.
	//
	// ErrNotFound is returned if no blob exists for the given id.
	Reject(ctx context.Context, id *blobpb.BlobId, rejection *RejectionMetadata) (bool, error)
}

Store persists server-authoritative blob metadata. The bytes themselves live in an ObjectStorage; this store only tracks the lifecycle and derived metadata keyed by BlobId.

Directories

Path Synopsis
Package s3 implements blob.ObjectStorage on top of Amazon S3 for storage and a CloudFront CDN for delivery.
Package s3 implements blob.ObjectStorage on top of Amazon S3 for storage and a CloudFront CDN for delivery.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL