Documentation
¶
Overview ¶
Package storage provides an abstraction layer for storing snapshots and resources.
This package defines the Storage interface which can be implemented by various backends (filesystem, S3, etc.). Currently, only filesystem storage is implemented.
Snapshots are compressed HTML archives of bookmarked web pages, while resources are embedded assets like images, stylesheets, and scripts extracted from pages.
All stored content is compressed with gzip to save disk space. Files are organized in a two-character prefix directory structure based on their hash to avoid filesystem limitations with too many files in a single directory.
The package provides a global storage instance that is initialized at startup and used throughout the application. All functions panic if storage is accessed before initialization.
Example usage:
err := storage.Init(cfg.Storage)
if err != nil {
log.Fatal(err)
}
// Save a snapshot
key := storage.Hash(htmlContent) + ".html"
err = storage.SaveSnapshot(key, htmlContent)
// Retrieve a snapshot
reader, err := storage.GetSnapshot(key)
if err != nil {
return err
}
defer reader.Close()
Index ¶
- Variables
- func FS() iofs.FS
- func GetResource(key string) (io.ReadCloser, error)
- func GetResourceSize(key string) uint
- func GetResourceURL(key string) string
- func GetSnapshot(key string) (io.ReadCloser, error)
- func GetSnapshotSize(key string) uint
- func GetStream(key string) (io.ReadCloser, error)
- func GetStreamSize(key string) uint
- func GetStreamURL(key string) string
- func Hash(x []byte) string
- func Init(sCfg config.Storage) error
- func SaveResource(ext string, resource io.Reader) (string, error)
- func SaveSnapshot(key string, snapshot []byte) error
- func SaveStream(ext string, resource io.Reader) (string, error)
- type Storage
Constants ¶
This section is empty.
Variables ¶
var ErrResourceNotFound = errors.New("resource not found")
ErrResourceNotFound is returned when a resource cannot be found.
var ErrSnapshotNotFound = errors.New("snapshot not found")
ErrSnapshotNotFound is returned when a snapshot cannot be found.
var ErrStreamNotFound = errors.New("streamable content not found")
ErrStreamNotFound is returned when a streamable content cannot be found.
var ErrUninitialized = errors.New("uninitialized storage")
ErrUninitialized is returned when storage is accessed before initialization.
var ErrUnknownStorage = errors.New("unknown storage type")
ErrUnknownStorage is returned when an unknown storage type is configured.
Functions ¶
func FS ¶ added in v0.5.0
FS returns the storage backend as an io/fs.FS interface for file system operations. This allows reading stored files using the standard library's fs.FS methods. Panics if storage has not been initialized.
func GetResource ¶ added in v0.2.0
func GetResource(key string) (io.ReadCloser, error)
GetResource retrieves a resource by its key and returns a reader for its contents. Resources are typically images, stylesheets, or other embedded assets from web pages. The returned reader must be closed by the caller when done. Returns ErrUninitialized if storage is not initialized, or ErrResourceNotFound if the resource doesn't exist.
func GetResourceSize ¶ added in v0.2.0
GetResourceSize returns the size in bytes of a stored resource. Returns 0 if the resource doesn't exist. Panics if storage has not been initialized.
func GetResourceURL ¶ added in v0.3.0
GetResourceURL returns the URL path for accessing a resource via HTTP. This is typically used to generate URLs for embedded images and assets in HTML. Panics if storage has not been initialized.
func GetSnapshot ¶
func GetSnapshot(key string) (io.ReadCloser, error)
GetSnapshot retrieves a snapshot by its key and returns a reader for its contents. The returned reader must be closed by the caller when done. Returns ErrUninitialized if storage is not initialized, or ErrSnapshotNotFound if the snapshot doesn't exist.
func GetSnapshotSize ¶ added in v0.2.0
GetSnapshotSize returns the size in bytes of a stored snapshot. Returns 0 if the snapshot doesn't exist. Panics if storage has not been initialized.
func GetStream ¶ added in v0.9.0
func GetStream(key string) (io.ReadCloser, error)
GetStream retrieves a streamable content by its key and returns a reader for its contents. The returned reader must be closed by the caller when done. Returns ErrUninitialized if storage is not initialized, or ErrStreamNotFound if the resource doesn't exist.
func GetStreamSize ¶ added in v0.9.0
GetStreamSize returns the size in bytes of a stored streamable content. Returns 0 if the resource doesn't exist. Panics if storage has not been initialized.
func GetStreamURL ¶ added in v0.9.0
GetStreamURL returns the URL path for accessing a streamable content via HTTP. Panics if storage has not been initialized.
func Hash ¶
Hash computes a SHA256 hash of the given bytes and returns it as a hexadecimal string. This is used to generate unique keys for storing snapshots and resources.
func Init ¶
Init initializes the storage backend with the given configuration. This must be called before using any other storage functions. Returns an error if the storage configuration is invalid or initialization fails.
func SaveResource ¶ added in v0.2.0
SaveResource stores a resource with the given file extension. Resources are typically images, stylesheets, or other embedded assets. The resource data is compressed before storage to save disk space. Returns with the key required to access the resource on success. Returns ErrUninitialized if storage is not initialized, or an error if saving fails.
func SaveSnapshot ¶
SaveSnapshot stores a snapshot with the given key. The snapshot data is compressed before storage to save disk space. Returns ErrUninitialized if storage is not initialized, or an error if saving fails.
func SaveStream ¶ added in v0.9.0
SaveResource stores a streamable content with the given file extension. Returns with the key required to access the resource on success. Returns ErrUninitialized if storage is not initialized, or an error if saving fails.
Types ¶
type Storage ¶
type Storage interface {
FS() (iofs.FS, error)
GetSnapshot(string) io.ReadCloser
GetSnapshotSize(string) uint
SaveSnapshot(string, []byte) error
SaveResource(string, io.Reader) (string, error)
SaveStream(string, io.Reader) (string, error)
GetResource(string) io.ReadCloser
GetStream(string) io.ReadCloser
GetResourceSize(string) uint
GetStreamSize(string) uint
GetResourceURL(string) string
GetStreamURL(string) string
}
Storage defines the interface for snapshot and resource storage backends.
Source Files
Directories