mediastore

package
v0.1.96 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Sep 22, 2026 License: MIT Imports: 17 Imported by: 0

Documentation

Overview

Package mediastore records the media objects (audio, images, video) the gateway keeps outside the database and pairs each record with its bytes in a blob store. See docs/adr/0013-media-storage.md.

Index

Constants

View Source
const (
	StorageFilesystem = "filesystem"
	StorageMemory     = "memory"
)

Blob storage types an operator can choose with MEDIA_STORAGE_TYPE.

View Source
const (
	// SweepInterval is how often expired objects are removed.
	SweepInterval = time.Hour
)

Variables

View Source
var ErrNotFound = errors.New("media object not found")

ErrNotFound reports a media object that does not exist or has expired.

Functions

func Inline

func Inline(contentType string) bool

Inline reports whether a stored content type is safe to serve inline. It is the same allowlist SafeContentType applies, so an object stored before a rule changed is still judged at serve time.

func OpenBlobStore

func OpenBlobStore(storageType, path string) (blobstore.Store, error)

OpenBlobStore builds the blob store named by storageType: a directory for "filesystem", process memory for "memory".

func SafeContentType

func SafeContentType(kind Kind, contentType string) string

SafeContentType returns the content type an object of kind may be stored under. A type is trusted only when it matches the kind (audio/* for audio, a passive raster type for images, video/* for video); anything else, including a client-declared text/html on an upload, is stored as application/octet-stream so it can never be served as active content.

func ValidID

func ValidID(id string) bool

ValidID reports whether id has the shape the service issues. Callers that take ids from clients check it first, so a malformed id never reaches a store query or a path.

Types

type Descriptor

type Descriptor struct {
	Kind        Kind
	Source      Source
	ContentType string
	RequestID   string
	UserPath    string
	// TTL is how long the object is kept; zero keeps it forever.
	TTL time.Duration
}

Descriptor says what an object holds and who it belongs to.

type Kind

type Kind string

Kind classifies a media object by what it holds.

const (
	KindAudio Kind = "audio"
	KindImage Kind = "image"
	KindVideo Kind = "video"
)

type MemoryStore

type MemoryStore struct {
	// contains filtered or unexported fields
}

MemoryStore keeps media records in process memory.

func NewMemoryStore

func NewMemoryStore() *MemoryStore

NewMemoryStore returns an empty in-memory record store.

func (*MemoryStore) Close

func (s *MemoryStore) Close() error

Close releases nothing.

func (*MemoryStore) Delete

func (s *MemoryStore) Delete(_ context.Context, id string) error

Delete removes the record.

func (*MemoryStore) Expired

func (s *MemoryStore) Expired(_ context.Context, now time.Time, limit int) ([]*Object, error)

Expired returns expired records, oldest expiry first.

func (*MemoryStore) Get

func (s *MemoryStore) Get(_ context.Context, id string) (*Object, error)

Get returns a copy of the record.

func (*MemoryStore) Insert

func (s *MemoryStore) Insert(_ context.Context, object *Object) error

Insert stores a record; an existing id is replaced.

type MongoDBStore

type MongoDBStore struct {
	// contains filtered or unexported fields
}

MongoDBStore keeps media records in MongoDB.

func NewMongoDBStore

func NewMongoDBStore(database *mongo.Database) (*MongoDBStore, error)

NewMongoDBStore creates collection indexes if needed.

func (*MongoDBStore) Close

func (s *MongoDBStore) Close() error

Close is a no-op; client lifecycle is managed by the storage layer.

func (*MongoDBStore) Delete

func (s *MongoDBStore) Delete(ctx context.Context, id string) error

Delete removes one record by id.

func (*MongoDBStore) Expired

func (s *MongoDBStore) Expired(ctx context.Context, now time.Time, limit int) ([]*Object, error)

Expired returns expired records, oldest expiry first.

func (*MongoDBStore) Get

func (s *MongoDBStore) Get(ctx context.Context, id string) (*Object, error)

Get returns one record by id.

func (*MongoDBStore) Insert

func (s *MongoDBStore) Insert(ctx context.Context, object *Object) error

Insert stores a record; an existing id is replaced.

type Object

type Object struct {
	ID          string    `json:"id" bson:"_id"`
	Kind        Kind      `json:"kind" bson:"kind"`
	Source      Source    `json:"source" bson:"source"`
	ContentType string    `json:"content_type" bson:"content_type"`
	Bytes       int64     `json:"bytes" bson:"bytes"`
	StorageKey  string    `json:"storage_key" bson:"storage_key"`
	RequestID   string    `json:"request_id,omitempty" bson:"request_id,omitempty"`
	UserPath    string    `json:"user_path,omitempty" bson:"user_path,omitempty"`
	CreatedAt   time.Time `json:"created_at" bson:"created_at"`
	// ExpiresAt is when the retention sweep removes the object; the zero
	// time keeps it forever.
	ExpiresAt time.Time `json:"expires_at,omitzero" bson:"expires_at,omitempty"`
}

Object is one stored media payload.

func (*Object) Expired

func (o *Object) Expired(now time.Time) bool

Expired reports whether the object's retention has run out at now.

type Result

type Result struct {
	Service *Service
}

Result holds the initialized media service.

func New

func New(ctx context.Context, shared storage.Storage, blobs blobstore.Store) (*Result, error)

New creates the media service: records on the shared storage connection, bytes in blobs.

func (*Result) Close

func (r *Result) Close() error

Close releases the service and both stores behind it.

type SQLStore

type SQLStore struct {
	// contains filtered or unexported fields
}

SQLStore keeps media records in SQLite or PostgreSQL.

func NewSQLStore

func NewSQLStore(ctx context.Context, db sqlx.DB) (*SQLStore, error)

NewSQLStore creates the media_objects table and indexes if needed.

func (*SQLStore) Close

func (s *SQLStore) Close() error

Close is a no-op; connection lifecycle is managed by the storage layer.

func (*SQLStore) Delete

func (s *SQLStore) Delete(ctx context.Context, id string) error

Delete removes one record by id.

func (*SQLStore) Expired

func (s *SQLStore) Expired(ctx context.Context, now time.Time, limit int) ([]*Object, error)

Expired returns expired records, oldest expiry first.

func (*SQLStore) Get

func (s *SQLStore) Get(ctx context.Context, id string) (*Object, error)

Get returns one record by id.

func (*SQLStore) Insert

func (s *SQLStore) Insert(ctx context.Context, object *Object) error

Insert stores a record; an existing id is replaced.

type Service

type Service struct {
	// contains filtered or unexported fields
}

Service stores media: a record per object in a Store and the bytes in a blobstore.Store. It runs the retention sweep until closed.

func NewService

func NewService(objects Store, blobs blobstore.Store) *Service

NewService pairs a record store with a blob store and starts the hourly retention sweep.

func (*Service) Begin

func (s *Service) Begin(ctx context.Context, d Descriptor) (*Upload, error)

Begin opens an upload for an object described by d.

func (*Service) Close

func (s *Service) Close() error

Close stops the sweep and closes both stores.

func (*Service) Delete

func (s *Service) Delete(ctx context.Context, id string) error

Delete removes an object's bytes and record.

func (*Service) Get

func (s *Service) Get(ctx context.Context, id string) (*Object, error)

Get returns an object's record, or ErrNotFound once it has expired. A malformed id is not found without touching the store.

func (*Service) Open

func (s *Service) Open(ctx context.Context, id string) (*Object, io.ReadSeekCloser, error)

Open returns an object's record and a reader over its bytes. The caller closes the reader.

func (*Service) Put

func (s *Service) Put(ctx context.Context, d Descriptor, r io.Reader) (*Object, error)

Put stores everything r yields as one object.

func (*Service) Sweep

func (s *Service) Sweep(ctx context.Context) (int, error)

Sweep removes every object whose retention has run out and reports how many it removed. An object that cannot be removed does not stop the others: the sweep works through the batch, then reports the failures together. It stops after a batch with failures rather than reselecting the same objects forever; the next hourly run tries them again. The hourly loop calls it; tests call it directly.

type Source

type Source string

Source records who stored an object, which decides what may read it and how it is retained.

const SourceAudit Source = "audit"

SourceAudit marks media captured for the audit log: the payload of an audio or image request whose body logging is enabled.

type Store

type Store interface {
	Insert(ctx context.Context, object *Object) error
	// Get returns the record, or ErrNotFound. Expiry is not applied here;
	// the service checks it so a swept-but-not-yet-deleted row still reads
	// as gone.
	Get(ctx context.Context, id string) (*Object, error)
	// Delete removes the record; a missing record is ErrNotFound.
	Delete(ctx context.Context, id string) error
	// Expired returns up to limit records whose expiry passed before now,
	// oldest expiry first.
	Expired(ctx context.Context, now time.Time, limit int) ([]*Object, error)
	Close() error
}

Store persists media object records.

type Upload

type Upload struct {
	// contains filtered or unexported fields
}

Upload is an object being written. Bytes go straight to the blob store; the record is written on Commit, so a discarded upload leaves nothing.

func (*Upload) Close

func (u *Upload) Close() error

Close discards an uncommitted upload; after Commit it is a no-op.

func (*Upload) Commit

func (u *Upload) Commit(ctx context.Context) (*Object, error)

Commit publishes the blob and writes its record. A record that cannot be written takes the blob with it, so the two never disagree.

func (*Upload) Write

func (u *Upload) Write(p []byte) (int, error)

Write appends to the blob.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL