Documentation
¶
Overview ¶
Package blob is where the bytes live.
A rig_file row is metadata: who owns it, what it is called, how big it is, whether it has been uploaded yet. The object itself is somewhere else, and this is the seam between them. Memory is the implementation that ships here, for tests and `go run`. A backend needing a cloud SDK belongs in a module of its own, so a project that never calls it does not carry the dependency — which is why this package's own are a UUID and nothing else.
Signer and Marker are separate interfaces on purpose. A backend that cannot mint a URL, or has nowhere to record that an object is deleted, simply does not have the method — which a caller can ask about — rather than having one that always returns an error, which a caller finds out about in production.
Example ¶
The whole seam in one pass: write bytes, learn what they weighed and hashed to, read them back. The checksum comes back from the write rather than being handed to it, which is why a rig_file row is finalized after the upload rather than before — until the last byte has gone past there is nothing to record.
package main
import (
"context"
"fmt"
"io"
"strings"
"github.com/simonjanss/rig/files/blob"
)
func main() {
ctx := context.Background()
store := blob.NewMemory()
info, err := store.Put(ctx, "files/ab/cd/abcd", strings.NewReader("hello"), blob.PutOptions{
ContentType: "text/plain",
})
if err != nil {
panic(err)
}
fmt.Println(info.Size, info.Checksum)
r, err := store.Get(ctx, "files/ab/cd/abcd")
if err != nil {
panic(err)
}
defer r.Close()
body, err := io.ReadAll(r)
if err != nil {
panic(err)
}
fmt.Printf("%s\n", body)
}
Output: 5 2cf24dba5fb0a30e26e83b2ac5b9e29e1b161e5c1fa7425e73043362938b9824 hello
Index ¶
- Variables
- func Key(id uuid.UUID) string
- type Info
- type Marker
- type Memory
- func (m *Memory) Delete(_ context.Context, key string) error
- func (m *Memory) Get(_ context.Context, key string) (io.ReadSeekCloser, error)
- func (m *Memory) Keys() []string
- func (m *Memory) Mark(_ context.Context, key string, state State, at time.Time) error
- func (m *Memory) Put(_ context.Context, key string, r io.Reader, _ PutOptions) (Info, error)
- func (m *Memory) Stat(_ context.Context, key string) (Info, error)
- func (m *Memory) State(key string) (State, time.Time, bool)
- type PutOptions
- type Signer
- type State
- type Store
Examples ¶
Constants ¶
This section is empty.
Variables ¶
var ErrNotFound = errors.New("blob: no object with that key")
ErrNotFound is what a Get or a Stat answers for a key that is not there.
It is a sentinel rather than a typed error because the only thing a caller ever needs to know is which of the two cases it has, and errors.Is is how the rest of the standard library asks that question.
Functions ¶
func Key ¶
Key is the storage key for a file, derived from its identifier and nothing else.
The name a client uploaded never reaches the store. It goes in the row, where it is data; letting it into the key makes it a path, and "../../etc/passwd" or an absolute path or a name that differs only by case are all then somebody else's problem to get right. A uuid has none of those questions.
The two-level prefix is for the backends that shard by key prefix, and costs nothing on the ones that do not.
Example ¶
The key is derived from the file's identifier and nothing else. The name a client uploaded stays in the row, where it is data: the moment it is part of a path, traversal and casing and absolute paths are all somebody's problem to get right everywhere the key is used.
The two-level prefix is for the backends that shard by key prefix, and costs nothing on the ones that do not.
package main
import (
"fmt"
"github.com/google/uuid"
"github.com/simonjanss/rig/files/blob"
)
func main() {
id := uuid.MustParse("0192a3b4-c5d6-7e8f-9012-3456789abcde")
fmt.Println(blob.Key(id))
}
Output: files/01/92/0192a3b4-c5d6-7e8f-9012-3456789abcde
Types ¶
type Info ¶
type Info struct {
Size int64
// Checksum is the hex-encoded SHA-256 of the content.
//
// Not an ETag: an ETag is whatever the backend felt like, and S3 makes it
// something other than the MD5 as soon as an upload is multipart. A checksum
// rig computes as the bytes go past means the same thing everywhere.
Checksum string
ModTime time.Time
}
Info is what a store knows about an object.
type Marker ¶
Marker is the optional half of a store that can record a deletion on the object itself, rather than only in the database.
Without it, the bucket cannot answer "what here is deleted?" — so a bucket audit, or a bucket restored from one snapshot beside a database restored from another, has no way to reconcile. And when somebody asks whether a file was deleted, the object is usually the thing they mean; the row is rig's bookkeeping.
The row leads and the mark follows: set deleted_at, commit, then mark, and treat the mark as best-effort. A failed mark leaves a deleted row beside an unmarked object, which is the safe direction — the sweeper still knows from the row and re-marks anything out of step. Marking first would produce an object tagged deleted that the database says is live, and nothing reconciles that back.
Example ¶
A backend says what it can do by having the method, so a caller asks rather than calling and handling an error in production. Memory can record a deletion on the object; it cannot mint a URL, and there is no URL it could usefully send a client to.
package main
import (
"fmt"
"github.com/simonjanss/rig/files/blob"
)
func main() {
var store blob.Store = blob.NewMemory()
_, marks := store.(blob.Marker)
_, signs := store.(blob.Signer)
fmt.Println(marks, signs)
}
Output: true false
type Memory ¶
type Memory struct {
// contains filtered or unexported fields
}
Memory keeps objects in a map.
It is for tests and for `go run`, and it is not durable — which is worth saying in the type rather than only in a README, because the failure mode of discovering it in production is losing every upload on a restart.
It implements Marker as well as Store, so a test can assert that a delete marked the object and a restore unmarked it. It deliberately does not implement Signer: there is no URL a client could usefully be sent to, and a method that returns an error would make "does this backend sign?" a question with two answers.
func (*Memory) Delete ¶
Delete implements Store, and says nothing about a key already gone.
Example ¶
Delete says nothing about a key that was already gone. That is what lets the sweeper be a simple loop over whatever the table says is expired: a second pass after a crash is not an error anybody has to tell apart from a real one.
package main
import (
"context"
"fmt"
"strings"
"github.com/simonjanss/rig/files/blob"
)
func main() {
ctx := context.Background()
store := blob.NewMemory()
if _, err := store.Put(ctx, "k", strings.NewReader("x"), blob.PutOptions{}); err != nil {
panic(err)
}
fmt.Println(store.Delete(ctx, "k"))
fmt.Println(store.Delete(ctx, "k"))
fmt.Println(store.Delete(ctx, "never existed"))
}
Output: <nil> <nil> <nil>
func (*Memory) Mark ¶
Mark implements Marker.
Example ¶
The row leads and the mark follows: set deleted_at, commit, then mark. A failed mark leaves a deleted row beside an unmarked object, which is the safe direction — the sweeper still knows from the row. Marking first would produce an object tagged deleted that the database says is live, and nothing reconciles that back.
package main
import (
"context"
"fmt"
"strings"
"time"
"github.com/simonjanss/rig/files/blob"
)
func main() {
ctx := context.Background()
store := blob.NewMemory()
at := time.Date(2026, 8, 19, 12, 0, 0, 0, time.UTC)
if _, err := store.Put(ctx, "k", strings.NewReader("x"), blob.PutOptions{}); err != nil {
panic(err)
}
state, _, _ := store.State("k")
fmt.Println(state)
if err := store.Mark(ctx, "k", blob.StateDeleted, at); err != nil {
panic(err)
}
state, marked, _ := store.State("k")
fmt.Println(state, marked.Format(time.RFC3339))
}
Output: live deleted 2026-08-19T12:00:00Z
type PutOptions ¶
type PutOptions struct {
// ContentType is stored with the object so a backend that serves bytes
// directly labels them correctly. rig never trusts it on the way back —
// the type a download announces is the one that was sniffed at upload and
// recorded in the row.
ContentType string
}
PutOptions is what the store is told about an object it is being handed.
type Signer ¶
type Signer interface {
SignUpload(ctx context.Context, key string, expires time.Duration) (string, error)
}
Signer is the optional half of a store that can mint a URL for a client to upload to directly.
There is deliberately no signed *download*. A signed URL bypasses the endpoint, and the endpoint is where the permission check lives — so downloads always flow through rig, and the honest cost is bandwidth through the application.
type State ¶
type State string
State is what a Marker records about an object.
const ( // StateLive is an object whose row is not deleted. StateLive State = "live" // StateDeleted is an object whose row has been soft-deleted and which the // sweeper will remove once the restore window closes. StateDeleted State = "deleted" )
The states. Two, and deliberately not more: the object mirrors what the row says, and every question about how a file got here — abandoned upload, soft delete, restore — is answered by the row rather than by a third state nobody could reconcile against it.
type Store ¶
type Store interface {
// Put writes the object and reports how many bytes it took and what they
// hash to.
//
// The checksum comes back from the write rather than being handed to it,
// because it cannot be known until the last byte has gone past — which is
// also why a rig_file row is finalized after the upload rather than before.
Put(ctx context.Context, key string, r io.Reader, opt PutOptions) (Info, error)
// Get opens the object for reading.
//
// The reader is a ReadSeekCloser so a download can answer a range request:
// http.ServeContent needs to seek, and without it a video cannot be scrubbed
// and a resumed download starts from the beginning.
Get(ctx context.Context, key string) (io.ReadSeekCloser, error)
// Stat reports what is known about the object without opening it.
Stat(ctx context.Context, key string) (Info, error)
// Delete removes the object, and says nothing about a key that was already
// gone.
//
// Idempotence is what lets the sweeper be a simple loop: it runs over
// whatever the table says is expired, and a second pass over the same row
// after a crash is not an error to handle.
Delete(ctx context.Context, key string) error
}
Store is the backend the bytes are kept in.
Every method takes a key rather than anything richer, because a store knows nothing about tenants, rows or permissions. Everything that decides who may reach an object happens before a call gets here.
Example (Range) ¶
Get returns a ReadSeekCloser so a download can answer a range request: http.ServeContent needs to seek, and without it a video cannot be scrubbed and a resumed download starts from the beginning.
package main
import (
"context"
"fmt"
"io"
"strings"
"github.com/simonjanss/rig/files/blob"
)
func main() {
ctx := context.Background()
store := blob.NewMemory()
if _, err := store.Put(ctx, "k", strings.NewReader("0123456789"), blob.PutOptions{}); err != nil {
panic(err)
}
r, err := store.Get(ctx, "k")
if err != nil {
panic(err)
}
defer r.Close()
if _, err := r.Seek(4, io.SeekStart); err != nil {
panic(err)
}
rest, err := io.ReadAll(r)
if err != nil {
panic(err)
}
fmt.Printf("%s\n", rest)
}
Output: 456789