files

package
v0.11.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Aug 17, 2026 License: MIT Imports: 25 Imported by: 0

Documentation

Overview

Package files provides file processing utilities for Telegram messages.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func IsFileSizeAllowed added in v0.6.0 

func IsFileSizeAllowed(size int64) bool

IsFileSizeAllowed checks if a file size is within Telegram Bot API limits.

func IsGeminiSupported added in v0.6.0 

func IsGeminiSupported(mimeType string) bool

IsGeminiSupported checks if a MIME type is supported by Gemini API.

func MaxFileSize added in v0.6.0 

func MaxFileSize() int64

MaxFileSize returns the maximum file size allowed for download.

func NormalizeMimeForGemini added in v0.6.2 

func NormalizeMimeForGemini(mimeType string) string

NormalizeMimeForGemini normalizes a MIME type for Gemini API consumption. For text types not explicitly supported by Gemini, returns text/plain. For other types, returns the original MIME type unchanged.

func RecordFileDownload 

func RecordFileDownload(userID storage.ScopeID, fileType FileType, durationSeconds float64, sizeBytes int64, success bool)

RecordFileDownload records metrics for a file download operation.

Types

type AggregateBudgetExceededError added in v0.11.0 

type AggregateBudgetExceededError struct {
	Size      int64
	Remaining int64
	FileName  string
}

AggregateBudgetExceededError is returned for an otherwise valid occurrence that does not fit into the bounded aggregate download budget for this turn.

func (*AggregateBudgetExceededError) Error added in v0.11.0 

func (e *AggregateBudgetExceededError) Error() string

type DeclaredFileSizeExceededError added in v0.11.0 

type DeclaredFileSizeExceededError struct {
	Declared int64
	Actual   int64
	FileName string
}

DeclaredFileSizeExceededError means a transport returned more bytes than it declared while the scheduler was reserving the aggregate budget. The bytes are discarded so a misleading declaration cannot violate the memory bound.

func (*DeclaredFileSizeExceededError) Error added in v0.11.0 

func (e *DeclaredFileSizeExceededError) Error() string

type FileSaver added in v0.6.0 

type FileSaver interface {
	// SaveFile saves a file and returns the artifact ID (existing on dedup, new on creation).
	// messageText is the text content of the message (msg.Text or msg.Caption) for context (v0.6.0).
	SaveFile(ctx context.Context, userID storage.ScopeID, messageID int64, fileType string, originalName string, mimeType string, reader io.Reader, messageText string, skipExtraction bool) (*int64, error)
}

FileSaver is the interface for saving artifacts (to avoid circular dependency).

type FileStorage added in v0.6.0 

type FileStorage struct {
	// contains filtered or unexported fields
}

FileStorage handles saving files to disk with SHA256 calculation.

func NewFileStorage added in v0.6.0 

func NewFileStorage(basePath string, logger *slog.Logger) *FileStorage

NewFileStorage creates a new file storage service.

func (*FileStorage) DeleteFile added in v0.6.0 

func (fs *FileStorage) DeleteFile(_ context.Context, relativePath string) error

DeleteFile deletes a file from disk. Routed through the same containment guard as ReadFile so a malformed/traversing key can never escape the base.

func (*FileStorage) GetFullPath added in v0.6.0 

func (fs *FileStorage) GetFullPath(relativePath string) string

GetFullPath returns the absolute path for a relative artifact path. Local-only helper (not part of Storage) — used by DeleteFile and any disk-specific code.

func (*FileStorage) ReadFile added in v0.10.0 

func (fs *FileStorage) ReadFile(_ context.Context, key string) ([]byte, error)

ReadFile returns the contents of the artifact stored at key (relative path).

func (*FileStorage) SaveFile added in v0.6.0 

func (fs *FileStorage) SaveFile(
	ctx context.Context,
	userID storage.ScopeID,
	reader io.Reader,
	filename string,
) (*SavedFile, error)

SaveFile saves a file to disk with streaming SHA256 calculation. Returns relative path, hash, and size.

type FileTooLargeError added in v0.6.0 

type FileTooLargeError struct {
	Size     int64
	FileName string
}

FileTooLargeError is returned when a file exceeds the size limit.

func (*FileTooLargeError) Error added in v0.6.0 

func (e *FileTooLargeError) Error() string

type FileType 

type FileType string

FileType represents the type of file being processed. 

const (
	FileTypePhoto     FileType = "photo"
	FileTypeImage     FileType = "image" // image sent as document
	FileTypePDF       FileType = "pdf"
	FileTypeVoice     FileType = "voice"
	FileTypeAudio     FileType = "audio"      // audio files (MP3, etc.)
	FileTypeVideo     FileType = "video"      // video files (MP4, etc.)
	FileTypeAnimation FileType = "animation"  // rich-message animation (MP4 when supported)
	FileTypeVideoNote FileType = "video_note" // video messages (circles)
	FileTypeDocument  FileType = "document"   // text files
)

type IncomingFile added in v0.10.0 

type IncomingFile struct {
	Kind         FileType
	SourceID     string // transport file id (Telegram file_id); used for ProcessedFile.FileID + logging
	FileUniqueID string // stable transport identity; non-empty values coalesce downloads within one call
	FetchKey     string // transport reference useful for diagnostics/markers; never used as a dedupe key
	Origin       string // low-cardinality source such as "telegram_rich" or "mattermost"
	Ordinal      int    // semantic occurrence ordinal assigned by the transport/projector
	BlockPath    string // semantic source path assigned by a recursive rich projector
	Marker       string // stable textual marker emitted by the rich projector
	FileName     string // declared filename where the kind carries one; "" otherwise
	MIME         string // declared/resolved MIME type
	Size         int64  // declared size in bytes (pre-download), for validation; 0 if unknown
	Duration     int    // duration in seconds (voice gating); 0 otherwise
	// Fetch must honor maxBytes when it is positive and reject an oversized
	// source before returning a buffer larger than that limit. maxBytes==0 asks
	// for the transport's ordinary per-file policy (the legacy single-file path).
	Fetch func(ctx context.Context, maxBytes int64) ([]byte, error)
}

IncomingFile is a transport-neutral description of a single attachment plus a Fetch closure that lazily pulls its bytes (encapsulating transport-specific download + retry). Telegram populates these via ExtractFiles/ExtractRichMedia; other transports build them directly.

Validation metadata (MIME, Size, Duration) is carried so ProcessFiles can reject unsupported/oversized files BEFORE invoking Fetch — preserving the pre-download validation the Telegram path always did.

type ProcessFileResult added in v0.11.0 

type ProcessFileResult struct {
	Incoming       IncomingFile
	Processed      *ProcessedFile
	Status         ProcessFileStatus
	Err            error
	DuplicateOf    int
	DownloadedSize int64
}

ProcessFileResult preserves one result for every incoming occurrence and in the same order. DuplicateOf is a result-slice index, or -1 when this is not a duplicate. A duplicate_ref deliberately has no Processed value: its binary payload is represented by the referenced available occurrence while its own semantic metadata remains available in Incoming.

type ProcessFileStatus added in v0.11.0 

type ProcessFileStatus string

ProcessFileStatus is the per-occurrence disposition returned by ProcessFilesDetailed. It is intentionally low-cardinality so callers can safely reuse it as an observability label. 

const (
	ProcessFileAvailable    ProcessFileStatus = "available"
	ProcessFileOmittedLimit ProcessFileStatus = "omitted_limit"
	ProcessFileUnsupported  ProcessFileStatus = "unsupported"
	ProcessFileFailed       ProcessFileStatus = "failed"
	ProcessFileDuplicateRef ProcessFileStatus = "duplicate_ref"
)

type ProcessedFile 

type ProcessedFile struct {
	// LLMParts contains the file data formatted for the LLM API
	// (FilePart, TextPart from llm package) (v0.6.0: unified on FilePart)
	LLMParts []interface{}

	// Instruction is the localized LLM instruction for this file type
	// (e.g., "Quote the transcription..." for voice messages)
	Instruction string

	// FileType indicates the type of file
	FileType FileType

	// FileID is the Telegram file ID
	FileID string

	// FileName is the original file name (if available)
	FileName string

	// MimeType is the MIME type of the file
	MimeType string

	// Size is the file size in bytes
	Size int64

	// Duration is the download time (for metrics)
	Duration time.Duration

	// ArtifactID is the ID of the saved artifact (nil if not saved or disabled)
	ArtifactID *int64

	// Rich-ingress occurrence metadata. Legacy attachments leave these at their
	// zero values unless their adapter supplies the corresponding identity.
	Ordinal      int
	BlockPath    string
	Origin       string
	FileUniqueID string
}

ProcessedFile represents a processed file ready for LLM consumption.

type Processor 

type Processor struct {
	// contains filtered or unexported fields
}

Processor handles file downloads and processing from messages.

func NewProcessor 

func NewProcessor(
	downloader telegram.FileDownloader,
	translator *i18n.Translator,
	language string,
	logger *slog.Logger,
) *Processor

NewProcessor creates a new file processor.

func (*Processor) ExtractFiles added in v0.10.0 

func (p *Processor) ExtractFiles(msg *telegram.Message, userID storage.ScopeID) []IncomingFile

ExtractFiles maps a Telegram message's single attachment (if any) to a transport-neutral IncomingFile, with a Fetch closure that downloads on demand via downloadWithRetry. Returns an empty slice when the message has no file. Priority matches the legacy dispatch: photo, document, voice, audio, video note.

func (*Processor) ExtractRichMedia added in v0.11.0 

func (p *Processor) ExtractRichMedia(media []telegram.RichMediaOccurrence, userID storage.ScopeID) []IncomingFile

ExtractRichMedia adapts the ordered semantic occurrences produced by the Telegram rich projector to the neutral lazy-download pipeline. It preserves one IncomingFile for every occurrence, including malformed occurrences that lack a downloadable object; ProcessFilesDetailed reports those as failed instead of silently losing their marker and position.

func (*Processor) ProcessFiles added in v0.10.0 

func (p *Processor) ProcessFiles(ctx context.Context, incoming []IncomingFile, userID storage.ScopeID, groupText string) ([]*ProcessedFile, error)

ProcessFiles turns transport-neutral IncomingFiles into ProcessedFiles (LLM parts + artifact rows). It is the neutral core shared by all transports; the Telegram entry point ProcessMessage delegates here via ExtractFiles.

Behavior matches the legacy per-type Telegram pipeline: validation errors (unsupported format, too large) are returned to the caller; download failures are logged and the file is skipped (not fatal). A Telegram message carries at most one file, so for that path the slice has 0 or 1 element.

func (*Processor) ProcessFilesDetailed added in v0.11.0 

func (p *Processor) ProcessFilesDetailed(
	ctx context.Context,
	incoming []IncomingFile,
	userID storage.ScopeID,
	groupText string,
) []ProcessFileResult

ProcessFilesDetailed processes a multi-occurrence rich attachment set under a single 20 MiB aggregate budget. Downloads are coalesced by a non-empty FileUniqueID, run with at most three concurrent fetches, and results always correspond one-for-one with incoming in input order.

Unlike the legacy ProcessFiles wrapper, validation and download failures are per-occurrence outcomes and never abort later siblings.

func (*Processor) ProcessMessage 

func (p *Processor) ProcessMessage(ctx context.Context, msg *telegram.Message, userID storage.ScopeID, groupText string) ([]*ProcessedFile, error)

ProcessMessage extracts and processes the file from a Telegram message. It is the Telegram adapter over the transport-neutral pipeline: it maps the message to IncomingFiles (ExtractFiles) and runs them through ProcessFiles. A Telegram message contains at most one file, so the result has 0 or 1 entry. groupText is the full text of all messages in the current MessageGroup (v0.6.0).

func (*Processor) SetFileHandler added in v0.6.0 

func (p *Processor) SetFileHandler(handler FileSaver)

SetFileHandler sets the optional file handler for saving artifacts.

func (*Processor) SetImageInputFormat added in v0.10.0 

func (p *Processor) SetImageInputFormat(format string)

SetImageInputFormat selects how image/video attachments are encoded as LLM content parts (llm.ImageInputFormatFile | ImageInputFormatOpenAI).

func (*Processor) SetMinVoiceDurationSec added in v0.6.0 

func (p *Processor) SetMinVoiceDurationSec(seconds int)

SetMinVoiceDurationSec sets the minimum voice duration for saving as artifact. 0 = save all voices, -1 = disable voice artifacts, N = only save voices >= N seconds.

type S3Options added in v0.10.0 

type S3Options struct {
	Endpoint  string // e.g. https://storage.yandexcloud.net
	Region    string // e.g. ru-central1
	Bucket    string // e.g. laplaced-dev
	AccessKey string
	SecretKey string
}

S3Options carries the connection parameters for an S3-compatible bucket (Yandex Object Storage in this deployment). Kept free of the config package so the files layer stays decoupled — the wiring translates config into these.

type S3Storage added in v0.10.0 

type S3Storage struct {
	// contains filtered or unexported fields
}

S3Storage persists artifacts in an S3-compatible bucket. Object keys match the relative keys used by the local backend (user_{scope}/YYYY-MM/{uuid}{ext}), so artifact.FilePath is portable across backends with no migration.

func NewS3Storage added in v0.10.0 

func NewS3Storage(_ context.Context, opts S3Options, logger *slog.Logger) (*S3Storage, error)

NewS3Storage builds an S3-backed store with static credentials and a custom endpoint (virtual-hosted addressing, which Yandex Object Storage supports).

func (*S3Storage) DeleteFile added in v0.10.0 

func (s *S3Storage) DeleteFile(ctx context.Context, key string) error

DeleteFile removes the object at key. A missing key is not an error (S3 DeleteObject is idempotent); a NoSuchKey is treated as success defensively.

func (*S3Storage) ReadFile added in v0.10.0 

func (s *S3Storage) ReadFile(ctx context.Context, key string) ([]byte, error)

ReadFile downloads the object at key and returns its full contents.

func (*S3Storage) SaveFile added in v0.10.0 

func (s *S3Storage) SaveFile(
	ctx context.Context,
	userID storage.ScopeID,
	reader io.Reader,
	filename string,
) (*SavedFile, error)

SaveFile buffers the reader (artifacts are size-capped, ≤20 MB), computes the SHA256, and uploads under a freshly generated key. The returned key is the same relative form the local backend uses.

type SavedFile added in v0.6.0 

type SavedFile struct {
	Path        string // Relative path from base
	ContentHash string // SHA256 hex
	Size        int64  // Bytes
}

SavedFile contains metadata about a saved file.

type Storage added in v0.10.0 

type Storage interface {
	// SaveFile persists the reader's contents under a freshly generated key and
	// returns the relative key, content hash, and size.
	SaveFile(ctx context.Context, userID storage.ScopeID, reader io.Reader, filename string) (*SavedFile, error)
	// ReadFile returns the full contents of the object at key (the relative path
	// stored as artifact.FilePath).
	ReadFile(ctx context.Context, key string) ([]byte, error)
	// DeleteFile removes the object at key. Missing objects are not an error.
	DeleteFile(ctx context.Context, key string) error
}

Storage abstracts the artifact blob store so the bot can persist files on a local disk (FileStorage) or in an S3-compatible bucket (S3Storage) chosen by config. The DB only ever stores the relative key returned in SavedFile.Path; that key is backend-agnostic (same object key on disk and in S3), so reads go through ReadFile(key) and never need a filesystem path.

GetFullPath is intentionally NOT part of this interface: an absolute path has no meaning for an object store. Callers that need the bytes use ReadFile.

type UnsupportedFormatError added in v0.6.0 

type UnsupportedFormatError struct {
	MimeType string
	FileName string
}

UnsupportedFormatError is returned when a file format is not supported by Gemini.

func (*UnsupportedFormatError) Error added in v0.6.0 

func (e *UnsupportedFormatError) Error() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.