stor

package module
v0.0.0-...-c286f04 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Feb 4, 2025 License: Apache-2.0 Imports: 0 Imported by: 0

README

stor

codecov e2e

Toy file storage. Not for production use.

https://github.com/user-attachments/assets/c61de5dc-f518-4c28-b4e3-695c06838fbf

stor.svg

sequenceDiagram
    Note over node, front: register
    node->>front: register
    front->>ydb: register
    ydb-->>front: ok
    front-->>node: ok

    actor user
    Note over user, front: upload
    user->>front: upload
    front-->front: split file into chunks
    front->>ydb: get nodes
    ydb-->>front: nodes
    front->>ydb: create file
    front->>ydb: create chunks
    front->>node: upload chunks
    node-->>front: ok
    ydb-->>front: ok
    front-->>user: upload link

    Note over user, front: download
    user->>front: download
    front->>ydb: get chunks
    ydb-->>front: chunks
    front->>node: download chunks
    node-->>front: chunks
    front-->front: assemble file
    front-->>user: file

Running

Run with observability stack:

docker compose --profile full up -d

Grafana dashboard is available at http://localhost:3000/d/stor/stor.

Run with minimal setup:

docker compose --profile app up -d
Checking

See ./cmd/stor-upload.

stor-upload --help
Usage of stor-upload:
  -check
    	download and check file checksum
  -file string
    	file to upload
  -gen
    	generate random file to temp dir
  -gen-size string
    	generate file of given size (default "100M")
  -name string
    	name of the file (defaults to file base name)
  -rnd
    	use random prefix for the file name
  -server-url string
    	server URL (default "http://localhost:8080")
$ go run ./cmd/stor-upload -gen -gen-size 1GB --check
uploading 100% |██████████████████████████████████████████| (1.0/1.0 GB, 850 MB/s)
uploaded link: http://localhost:8080/download/stor-upload-501649240.bin
computing original sha256 100% |██████████████████████████| (1.0/1.0 GB, 2.0 GB/s)
original sha256: 3e6d6d836b6298d3540df54da03bf6c4d980a749890abcf9af6f199f58ff0d18
downloading 100% |████████████████████████████████████████| (1.0/1.0 GB, 1.6 GB/s)
downloaded sha256: 3e6d6d836b6298d3540df54da03bf6c4d980a749890abcf9af6f199f58ff0d18
checksum match

Cleanup

docker compose --profile full down --timeout 1 --volumes

Documentation

Overview

Package stor implements file storage.

Directories

Path Synopsis
cmd
stor-front command
stor-node command
stor-upload command
internal
front
Package front implements frontend service.
Package front implements frontend service.
integration
Package integration wraps integration test utilities.
Package integration wraps integration test utilities.
node
Package node holds storage node implementation.
Package node holds storage node implementation.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL