filestore

package
v0.41.1 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 22, 2026 License: Apache-2.0 Imports: 8 Imported by: 0

README

Filestore Service

The Filestore service provides file storage for things and groups. File contents are written to the local filesystem, while file metadata is persisted to a database. Files can be scoped to an individual thing (authenticated with a thing key) or to a group (authenticated with a user bearer token at editor level or above).

Files

Each stored file is described by the following metadata fields:

Field Description
name File name; used as the unique identifier within the thing or group scope
class Logical file class (e.g. image, document, bim, pointcloud, binary)
format File format / MIME subtype (e.g. csv, pdf, png, ifc)
time Unix timestamp (floating-point seconds) associated with the file
metadata Arbitrary key-value pairs for custom attributes

Scopes

Scope Auth header Description
Thing files Authorization: Thing <thing_key> Files private to a specific thing
Group files Authorization: Bearer <token> Files shared across a group; require editor-level access

A thing can also retrieve its own group's files directly using its thing key via GET /groupfiles/{name}.

Configuration

The service is configured using the environment variables presented in the following table. Note that any unset variables will be replaced with their default values.

Variable Description Default
MF_FILESTORE_LOG_LEVEL Log level for the Filestore service (debug, info, warn, error) error
MF_FILESTORE_HTTP_PORT Filestore service HTTP port 9024
MF_JAEGER_URL Jaeger server URL for distributed tracing. Leave empty to disable tracing.
MF_FILESTORE_DB_HOST Database host address localhost
MF_FILESTORE_DB_PORT Database host port 5432
MF_FILESTORE_DB_USER Database user mainflux
MF_FILESTORE_DB_PASS Database password mainflux
MF_FILESTORE_DB Name of the database used by the service filestore
MF_FILESTORE_DB_SSL_MODE Database connection SSL mode (disable, require, verify-ca, verify-full) disable
MF_FILESTORE_DB_SSL_CERT Path to the PEM encoded certificate file
MF_FILESTORE_DB_SSL_KEY Path to the PEM encoded key file
MF_FILESTORE_DB_SSL_ROOT_CERT Path to the PEM encoded root certificate file
MF_FILESTORE_TLS Flag that indicates if TLS should be turned on false
MF_FILESTORE_CA_CERTS Path to trusted CAs in PEM format
MF_FILESTORE_SERVER_CERT Path to server certificate in PEM format
MF_FILESTORE_SERVER_KEY Path to server key in PEM format
MF_THINGS_AUTH_GRPC_URL Things service Auth gRPC URL localhost:8183
MF_THINGS_AUTH_GRPC_TIMEOUT Things service Auth gRPC request timeout in seconds 1s
MF_FILESTORE_ES_URL Event store URL redis://localhost:6379/0
MF_FILESTORE_EVENT_CONSUMER Event store consumer name filestore

Deployment

The service itself is distributed as Docker container. Check the filestore service section in docker-compose to see how service is deployed.

To start the service, execute the following shell script:

# Download the latest version of the service
git clone https://github.com/MainfluxLabs/mainflux

cd mainflux

# compile the filestore service
make filestore

# Copy binary to bin
make install

# Set the environment variables and run the service
MF_FILESTORE_LOG_LEVEL=[Filestore log level] \
MF_FILESTORE_HTTP_PORT=[Filestore service HTTP port] \
MF_FILESTORE_DB_HOST=[Database host address] \
MF_FILESTORE_DB_PORT=[Database host port] \
MF_FILESTORE_DB_USER=[Database user] \
MF_FILESTORE_DB_PASS=[Database password] \
MF_FILESTORE_DB=[Filestore database name] \
MF_THINGS_AUTH_GRPC_URL=[Things service Auth gRPC URL] \
MF_THINGS_AUTH_GRPC_TIMEOUT=[Things service Auth gRPC request timeout] \
$GOBIN/mainfluxlabs-filestore

Usage

For the full HTTP API reference, see the OpenAPI specification.

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type FileGroupsPage 

type FileGroupsPage struct {
	PageMetadata
	Files []FileInfo
}

FileGroupsPage contains page related metadata as well as list of files that belong to this page.

type FileInfo 

type FileInfo struct {
	Name     string         `json:"name"`
	Class    string         `json:"class"`
	Format   string         `json:"format"`
	Metadata map[string]any `json:"metadata,omitempty"`
	Time     float64        `json:"time,omitempty"`
}

FileInfo contains information about the file

type FileThingsPage 

type FileThingsPage struct {
	PageMetadata
	Files []FileInfo
}

FileThingsPage contains page related metadata as well as list of files that belong to this page.

type GroupsRepository 

type GroupsRepository interface {
	// Save stores path in filestore
	Save(ctx context.Context, groupID string, fi FileInfo) error
	// Update updates path in filestore
	Update(ctx context.Context, groupID string, fi FileInfo) error
	// Retrieve retrieves path from filestore
	Retrieve(ctx context.Context, groupID string, fi FileInfo) (FileInfo, error)
	// RetrieveByGroup retrieves files from filestore by group
	RetrieveByGroup(ctx context.Context, groupID string, fi FileInfo, pm PageMetadata) (FileGroupsPage, error)
	// Remove removes path from filestore
	Remove(ctx context.Context, groupID string, fi FileInfo) error
	// RemoveByGroup removes all paths from filestore by group ID
	RemoveByGroup(ctx context.Context, groupID string) error
}

GroupsRepository represents filestore interface.

type PageMetadata 

type PageMetadata struct {
	Total    uint64
	Offset   uint64         `json:"offset,omitempty"`
	Limit    uint64         `json:"limit,omitempty"`
	Name     string         `json:"name,omitempty"`
	Order    string         `json:"order,omitempty"`
	Dir      string         `json:"dir,omitempty"`
	Metadata map[string]any `json:"metadata,omitempty"`
}

PageMetadata contains page metadata that helps navigation.

type Service 

type Service interface {
	// SaveFile stores files in filestore
	SaveFile(ctx context.Context, file io.Reader, key string, fi FileInfo) error
	// UpdateFile updates file from filestore
	UpdateFile(ctx context.Context, key string, fi FileInfo) error
	// ViewFile views file from filestore
	ViewFile(ctx context.Context, key string, fi FileInfo) ([]byte, error)
	// ListFiles retrieves files from filestore by thing
	ListFiles(ctx context.Context, key string, fi FileInfo, pm PageMetadata) (FileThingsPage, error)
	// RemoveFile removes file from filestore
	RemoveFile(ctx context.Context, key string, fi FileInfo) error
	// RemoveFiles removes files from filestore by thing ID
	RemoveFiles(ctx context.Context, thingID string) error

	// SaveGroupFile stores group files in filestore
	SaveGroupFile(ctx context.Context, file io.Reader, token, groupID string, fi FileInfo) error
	// UpdateGroupFile updates group file from filestore
	UpdateGroupFile(ctx context.Context, token, groupID string, fi FileInfo) error
	// ViewGroupFile views group file from filestore
	ViewGroupFile(ctx context.Context, token, groupID string, fi FileInfo) ([]byte, error)
	// ListGroupFiles retrieves files from filestore by group
	ListGroupFiles(ctx context.Context, token, groupID string, fi FileInfo, pm PageMetadata) (FileGroupsPage, error)
	// RemoveGroupFile removes group file from filestore
	RemoveGroupFile(ctx context.Context, token, groupID string, fi FileInfo) error

	// RemoveAllFilesByGroup removes group files and
	// all files belonging to things related to the given group
	RemoveAllFilesByGroup(ctx context.Context, groupID string) error

	// ViewGroupFileByKey views group file from filestore using Thing Key
	ViewGroupFileByKey(ctx context.Context, thingKey string, fi FileInfo) ([]byte, error)
}

Service specifies an API that must be fulfilled by the domain service implementation, and all of its decorators (e.g. logging & metrics).

func New 

func New(tc domain.ThingsClient, thingsRepo ThingsRepository, groupsRepo GroupsRepository) Service

New instantiates the filestore service implementation.

type ThingsRepository 

type ThingsRepository interface {
	// Save stores path in filestore
	Save(ctx context.Context, thingID, groupID string, fi FileInfo) error
	// Update updates path in filestore
	Update(ctx context.Context, thingID string, fi FileInfo) error
	// Retrieve retrieves path from filestore
	Retrieve(ctx context.Context, thingID string, fi FileInfo) (FileInfo, error)
	// RetrieveByThing retrieves files from filestore by thing
	RetrieveByThing(ctx context.Context, thingID string, fi FileInfo, pm PageMetadata) (FileThingsPage, error)
	// Remove removes path from filestore
	Remove(ctx context.Context, thingID string, fi FileInfo) error
	// RemoveByThing removes paths from filestore by thing ID
	RemoveByThing(ctx context.Context, thingID string) error
	// RemoveByGroup removes paths from filestore by group ID
	RemoveByGroup(ctx context.Context, groupID string) error

	// RetrieveThingIDsByGroup retrieves thing IDs from filestore by group ID
	RetrieveThingIDsByGroup(ctx context.Context, groupID string) ([]string, error)
}

ThingsRepository represents filestore interface.

Source Files

View all Source files

Directories

Path Synopsis
api
Package api contains API-related concerns: endpoint definitions, middlewares and all resource representations.
 Package api contains API-related concerns: endpoint definitions, middlewares and all resource representations.
http
Package http contains implementation of kit service HTTP API.
 Package http contains implementation of kit service HTTP API.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.