fs

package module
v0.0.5 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 15, 2026 License: Apache-2.0 Imports: 4 Imported by: 0

README

fs Go Reference codecov experimental

Simple S3-compatible storage server for development and testing.

Features

S3-Compatible Storage Server

A lightweight S3-compatible storage server for development and testing.

Quick Start:

# Install
go install github.com/go-faster/fs/cmd/fs@latest

# Start the server
fs s3

# Or with custom configuration
fs s3 --addr :9000 --root /data/s3

Features:

  • Bucket operations (create, delete, list)
  • Object operations (put, get, delete, list)
  • File system-based storage
  • Compatible with AWS CLI, MinIO client, and other S3 clients
  • Health check endpoint

Example Usage:

# Using AWS CLI
export AWS_ENDPOINT_URL=http://localhost:8080
aws s3 mb s3://mybucket --endpoint-url $AWS_ENDPOINT_URL
aws s3 cp file.txt s3://mybucket/ --endpoint-url $AWS_ENDPOINT_URL

# Using cURL
curl -X PUT http://localhost:8080/mybucket
curl -X PUT -d "Hello!" http://localhost:8080/mybucket/hello.txt
curl http://localhost:8080/mybucket/hello.txt

See S3_README.md for detailed documentation.

Installation

go install github.com/go-faster/fs/cmd/fs@latest

Or build from source:

git clone https://github.com/go-faster/fs
cd fs
go build -o bin/fs ./cmd/fs

Usage

Quick Start
# Start S3 server with defaults
fs s3

# Show help
fs s3 --help
Configuration

The server supports both YAML configuration files and command-line flags:

# Using YAML configuration
fs s3 --config config.yaml

# Using command-line flags
fs s3 --addr :9000 --root /var/lib/s3data

# Mix both (flags override config file)
fs s3 --config config.yaml --addr :9000

# Generate example configuration
fs s3 --generate-config > my-config.yaml

See CONFIGURATION.md for detailed configuration documentation.

Example Configuration
server:
  addr: ":8080"
  read_timeout: 30s
  write_timeout: 30s
  idle_timeout: 120s
  health_path: "/health"

storage:
  root: ".s3data"
  type: "filesystem"

observability:
  service_name: "go-faster/fs"
  enable_request_logging: true
  enable_metrics: true
  enable_tracing: true

Development

# Run tests
go test ./...

# Build
go build ./cmd/fs

# Run with coverage
./go.coverage.sh

License

Apache 2.0

Documentation

Overview

Package fs is a S3-compatible storage server implementation.

Index

Constants

This section is empty.

Variables

View Source
var (
	ErrBucketNotFound       = errors.New("bucket not found")
	ErrObjectNotFound       = errors.New("object not found")
	ErrUploadNotFound       = errors.New("upload not found")
	ErrInvalidBucketName    = errors.New("invalid bucket name")
	ErrUnsupportedOperation = errors.New("unsupported operation")
)

Functions

This section is empty.

Types

type Bucket

type Bucket struct {
	Name         string
	CreationDate time.Time
}

Bucket represents an S3 bucket.

type CompleteMultipartUploadRequest added in v0.0.4

type CompleteMultipartUploadRequest struct {
	Bucket   string
	Key      string
	UploadID string
	Parts    []CompletedPart
}

CompleteMultipartUploadRequest represents a request to complete multipart upload.

type CompleteMultipartUploadResponse added in v0.0.4

type CompleteMultipartUploadResponse struct {
	Location string
	Bucket   string
	Key      string
	ETag     string
}

CompleteMultipartUploadResponse represents the response for completing multipart upload.

type CompletedPart added in v0.0.4

type CompletedPart struct {
	PartNumber int
	ETag       string
}

CompletedPart represents a completed part for completing multipart upload.

type GetObjectResponse

type GetObjectResponse struct {
	Reader       io.ReadCloser
	Size         int64
	LastModified time.Time
	ETag         string
	ContentType  string
}

GetObjectResponse represents the response for GetObject operation.

type MultipartUpload added in v0.0.4

type MultipartUpload struct {
	UploadID  string
	Bucket    string
	Key       string
	Initiated time.Time
}

MultipartUpload represents an in-progress multipart upload.

type Object

type Object struct {
	Key          string
	Size         int64
	LastModified time.Time
	ETag         string
}

Object represents an S3 object.

type Part added in v0.0.4

type Part struct {
	PartNumber int
	ETag       string
	Size       int64
}

Part represents a part of a multipart upload.

type PutObjectRequest

type PutObjectRequest struct {
	Reader io.Reader
	Bucket string
	Key    string
	Size   int64
}

type Service

type Service interface {
	ListBuckets(ctx context.Context) ([]Bucket, error)
	CreateBucket(ctx context.Context, bucket string) error
	DeleteBucket(ctx context.Context, bucket string) error
	BucketExists(ctx context.Context, bucket string) (bool, error)
	ListObjects(ctx context.Context, bucket, prefix string) ([]Object, error)
	PutObject(ctx context.Context, req *PutObjectRequest) error
	GetObject(ctx context.Context, bucket, key string) (*GetObjectResponse, error)
	DeleteObject(ctx context.Context, bucket, key string) error

	CreateMultipartUpload(ctx context.Context, bucket, key string) (*MultipartUpload, error)
	UploadPart(ctx context.Context, req *UploadPartRequest) (*Part, error)
	CompleteMultipartUpload(ctx context.Context, req *CompleteMultipartUploadRequest) (*CompleteMultipartUploadResponse, error)
	AbortMultipartUpload(ctx context.Context, bucket, key, uploadID string) error
}

Service defines the interface for S3-compatible storage operations.

type Storage

type Storage interface {
	ListBuckets(ctx context.Context) ([]Bucket, error)
	CreateBucket(ctx context.Context, bucket string) error
	DeleteBucket(ctx context.Context, bucket string) error
	BucketExists(ctx context.Context, bucket string) (bool, error)
	ListObjects(ctx context.Context, bucket, prefix string) ([]Object, error)
	PutObject(ctx context.Context, req *PutObjectRequest) error
	GetObject(ctx context.Context, bucket, key string) (*GetObjectResponse, error)
	DeleteObject(ctx context.Context, bucket, key string) error

	CreateMultipartUpload(ctx context.Context, bucket, key string) (*MultipartUpload, error)
	UploadPart(ctx context.Context, req *UploadPartRequest) (*Part, error)
	CompleteMultipartUpload(ctx context.Context, req *CompleteMultipartUploadRequest) (*CompleteMultipartUploadResponse, error)
	AbortMultipartUpload(ctx context.Context, bucket, key, uploadID string) error
}

Storage defines the interface for S3-compatible storage operations.

type UploadPartRequest added in v0.0.4

type UploadPartRequest struct {
	Bucket     string
	Key        string
	UploadID   string
	PartNumber int
	Reader     io.Reader
	Size       int64
}

UploadPartRequest represents a request to upload a part.

Directories

Path Synopsis
cmd
fs command
internal
core/storage/storagefs
Package storagefs implements fs.Storage.
Package storagefs implements fs.Storage.
core/storage/storagemem
Package storagemem implements fs.Storage using in-memory storage.
Package storagemem implements fs.Storage using in-memory storage.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL