objectstoretest

package
v0.31.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 31, 2026 License: MIT Imports: 13 Imported by: 0

Documentation

Overview

Package objectstoretest holds the shared conformance suite for objectstore.Backend, plus Memory - an in-memory backend that passes it.

Why an exported suite

The point of objectstore.Backend is that a backend can live in a different module, written by someone who will never send a PR here. That freedom is only safe if "conformant" means something more precise than the prose on the interface. Prose does not catch a List that silently returns one page, a Hydrate that forgets to strip the prefix, or a prefix match that treats "sessions/sess-1" as covering "sessions/sess-10", and each of those produces a corrupted session rather than an error message.

So the definition of conformant is executable and lives here. Three implementations are held to it: Memory below, the S3 module and the GCS module. An out-of-tree backend runs the same suite with a single call:

func TestContract(t *testing.T) {
    objectstoretest.RunSuite(t, func(t *testing.T) objectstore.Backend {
        return newMyBackend(t) // empty, and cleaned up via t.Cleanup
    })
}

Shape

One exported entry point taking a factory, not a pile of helpers a caller has to assemble. The factory is called once per case so no case can be affected by another's leftovers, and returning a Backend rather than accepting one is what makes that isolation the implementer's to arrange (a temp bucket, a cleared prefix, a fresh emulator namespace) instead of something the suite has to guess at.

Taking *testing.T rather than returning an error - the testing/fstest.TestFS approach - is deliberate: subtests give a third-party implementer the name of the exact semantic they broke, and it matches pkg/testharness/contract, the house precedent for an exported suite.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func RunSuite

func RunSuite(t *testing.T, newBackend NewBackend, opts ...Option)

RunSuite runs every conformance case against backends built by newBackend.

Each case gets its own backend, so cases are independent and a failure in one does not cascade. Nothing here uses t.Parallel: a third-party backend may be pointed at a shared emulator, and a suite that is only correct when run serially is better than one that is intermittently wrong.

Types

type Counts

type Counts struct {
	Hydrates int
	Puts     int
	Deletes  int
	Lists    int
	Flushes  int
	Closes   int
}

Counts records how often each method was called. Exposed so a test can assert on the *shape* of the engine's traffic — "exactly one flush on a clean shutdown", "no hydrate when the local tree was already present" — which is the kind of thing the seam gets wrong in ways the resulting file tree cannot show.

type Memory

type Memory struct {
	// contains filtered or unexported fields
}

Memory is a complete objectstore.Backend held entirely in process memory.

It exists for two jobs. First, it is the reference implementation: it passes the contract suite in this package, which is what makes the suite meaningful to hold the S3 and GCS modules to — a suite no implementation has ever passed proves nothing about the suite. Second, it is the substituted seam for ordinary untagged unit tests, in the spirit of the fake commandRunner in cmd/nexus-broker: a test that needs the engine's object-store path to do something real, without a network, a credential or a build tag, hands the engine one of these.

It is deliberately NOT a driver registered at init. A "memory" backend silently selectable in production config is a footgun — an operator who typo'd their real backend name would get a store that discards everything on exit while reporting success. Tests that need it reachable by name call RegisterMemory, which removes it again on cleanup.

Safe for concurrent use, because the interface requires it and because make test-race runs over this package like any other.

func NewMemory

func NewMemory() *Memory

NewMemory returns an empty in-memory backend.

func RegisterMemory

func RegisterMemory(t *testing.T, name string, backend *Memory) *Memory

RegisterMemory registers backend under name for the duration of one test and returns it, so a test can drive the engine through real config (core.object_store.backend: <name>) rather than reaching past it.

The registry is process-global and Register panics on a duplicate, so the cleanup is not optional — leaving the name behind would make a second run of the same test in the same binary panic. Passing a distinct name per test is still wise if tests within a package might run in parallel.

func (*Memory) Close

func (m *Memory) Close() error

Close satisfies io.Closer, which the engine type-asserts rather than requiring on Backend. Implemented here so a test using this double exercises that optional-release path instead of silently skipping it.

func (*Memory) Counts

func (m *Memory) Counts() Counts

Counts returns a snapshot of the per-method call counters.

func (*Memory) Delete

func (m *Memory) Delete(_ context.Context, key string) error

Delete removes key. A missing key is not an error.

func (*Memory) Flush

func (m *Memory) Flush(context.Context) error

Flush is a no-op: an in-memory object is durable the instant it is stored, which is the whole reason this type is a useful baseline — it can never fail the durability barrier for reasons of its own.

func (*Memory) Get

func (m *Memory) Get(key string) ([]byte, bool)

Get returns a copy of the object stored at key.

func (*Memory) Hydrate

func (m *Memory) Hydrate(ctx context.Context, keyPrefix string, destDir string) error

Hydrate writes every object under keyPrefix beneath destDir, with keyPrefix stripped from each key. Directories are created as needed; entries at destDir with no corresponding object are left alone.

func (*Memory) Keys

func (m *Memory) Keys() []string

Keys returns every stored key in ascending order. Sorted here, unlike List, because this is an assertion helper rather than part of the seam.

func (*Memory) Len

func (m *Memory) Len() int

Len returns the number of stored objects.

func (*Memory) List

func (m *Memory) List(_ context.Context, keyPrefix string) ([]objectstore.Object, error)

List returns every object under keyPrefix.

The result is returned in *descending* key order on purpose. The interface says the order is unspecified, and a double that happens to return sorted output lets a caller — or the contract suite itself — grow an accidental dependency on sorting that a real store would break. Descending is deterministic, so nothing is flaky, and it is wrong for anyone who assumed.

func (*Memory) Put

func (m *Memory) Put(ctx context.Context, key string, localPath string) error

Put copies the contents of localPath to key, replacing whatever was there.

func (*Memory) Seed

func (m *Memory) Seed(key string, data []byte) error

Seed stores an object directly, bypassing the local-file round trip. It is how a test arranges "the store already contains a session" without first having to build that session on disk.

func (*Memory) SeedTree

func (m *Memory) SeedTree(keyPrefix string, dir string) error

SeedTree stores every file under dir, keyed by keyPrefix plus the path relative to dir. The inverse of Hydrate, for tests that build a tree on disk and then want to resume it somewhere else.

func (*Memory) SetDeleteError

func (m *Memory) SetDeleteError(err error)

SetDeleteError makes every subsequent Delete fail with err.

func (*Memory) SetFlushError

func (m *Memory) SetFlushError(err error)

SetFlushError makes every subsequent Flush fail with err. The engine's failure_policy branch is the main consumer: degrade must swallow this and strict must surface it.

func (*Memory) SetFlushErrorTimes

func (m *Memory) SetFlushErrorTimes(err error, n int)

SetFlushErrorTimes fails the next n Flushes with err and heals after that. The counterpart to SetPutErrorTimes for the durability barrier, which is where a snapshot most often dies.

func (*Memory) SetHydrateError

func (m *Memory) SetHydrateError(err error)

SetHydrateError makes every subsequent Hydrate fail with err. Pass nil to stop failing.

func (*Memory) SetListError

func (m *Memory) SetListError(err error)

SetListError makes every subsequent List fail with err. The engine's immutable-skip path is the main consumer: a backend that cannot say what it holds must make the engine upload everything, never assume presence.

func (*Memory) SetPutError

func (m *Memory) SetPutError(err error)

SetPutError makes every subsequent Put fail with err.

func (*Memory) SetPutErrorTimes

func (m *Memory) SetPutErrorTimes(err error, n int)

SetPutErrorTimes fails the next n Puts with err and heals after that.

A bounded outage, which is what the engine's retry-and-recover path can only be tested against: an unbounded one proves the retry happens but never that it stops. n <= 0 clears any injected Put error.

type NewBackend

type NewBackend func(t *testing.T) objectstore.Backend

NewBackend builds a fresh, empty backend for one contract case. Any teardown belongs on t.Cleanup.

type Option

type Option func(*options)

Option tunes the suite for backends that cannot afford a default.

func WithListProbeCount

func WithListProbeCount(n int) Option

WithListProbeCount sets how many objects the pagination case stores. Set it below the backend's natural page size and the case stops proving anything; the default is chosen to exceed the common ones.

func WithoutConcurrency

func WithoutConcurrency() Option

WithoutConcurrency skips the concurrent-use case. Provided for backends whose test double or emulator cannot take parallel traffic, not as a way to opt out of the requirement, which the interface states unconditionally.

func WithoutObjectAtPrefix

func WithoutObjectAtPrefix() Option

WithoutObjectAtPrefix drops the tail of the prefix-matching case that stores an object at exactly the prefix other objects live under - key "sessions/sess-1" alongside "sessions/sess-1/files/a.txt".

S3 and GCS both hold that state: their key spaces are flat, "/" has no meaning beyond being a byte, and a key that happens to be another key's prefix is unremarkable. The suite asserts the state because the *engine's* key scheme can produce it and a backend that lets the object at the prefix leak into a hydration writes a file where a directory belongs.

Some stores cannot represent it at all. MinIO is the measured example: a PUT to "sessions/sess-1" against a store already holding "sessions/sess-1/files/a.txt" returns 200 and makes the child object disappear from every subsequent list, in both single-drive and erasure modes. That is a divergence from S3 in the emulator, not in the backend under test, and the resulting bucket cannot even be emptied by listing it.

So this exists for emulators, and only for emulators. It is not a way for a backend to opt out of the semantic: a backend that needs it is claiming its *store* has a flat-key-space limitation, which is a property worth stating out loud at the call site. Run the rest of the suite unaffected -- every other prefix-matching assertion in the case still runs, including the "sess-1" versus "sess-10" collision that motivates the case.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL