Documentation
¶
Overview ¶
Package objectstoretest holds the shared conformance suite for objectstore.Backend, plus Memory - an in-memory backend that passes it.
Why an exported suite ¶
The point of objectstore.Backend is that a backend can live in a different module, written by someone who will never send a PR here. That freedom is only safe if "conformant" means something more precise than the prose on the interface. Prose does not catch a List that silently returns one page, a Hydrate that forgets to strip the prefix, or a prefix match that treats "sessions/sess-1" as covering "sessions/sess-10", and each of those produces a corrupted session rather than an error message.
So the definition of conformant is executable and lives here. Three implementations are held to it: Memory below, the S3 module and the GCS module. An out-of-tree backend runs the same suite with a single call:
func TestContract(t *testing.T) {
objectstoretest.RunSuite(t, func(t *testing.T) objectstore.Backend {
return newMyBackend(t) // empty, and cleaned up via t.Cleanup
})
}
Shape ¶
One exported entry point taking a factory, not a pile of helpers a caller has to assemble. The factory is called once per case so no case can be affected by another's leftovers, and returning a Backend rather than accepting one is what makes that isolation the implementer's to arrange (a temp bucket, a cleared prefix, a fresh emulator namespace) instead of something the suite has to guess at.
Taking *testing.T rather than returning an error - the testing/fstest.TestFS approach - is deliberate: subtests give a third-party implementer the name of the exact semantic they broke, and it matches pkg/testharness/contract, the house precedent for an exported suite.
Index ¶
- func RunSuite(t *testing.T, newBackend NewBackend, opts ...Option)
- type Counts
- type Memory
- func (m *Memory) Close() error
- func (m *Memory) Counts() Counts
- func (m *Memory) Delete(_ context.Context, key string) error
- func (m *Memory) Flush(context.Context) error
- func (m *Memory) Get(key string) ([]byte, bool)
- func (m *Memory) Hydrate(ctx context.Context, keyPrefix string, destDir string) error
- func (m *Memory) Keys() []string
- func (m *Memory) Len() int
- func (m *Memory) List(_ context.Context, keyPrefix string) ([]objectstore.Object, error)
- func (m *Memory) Put(ctx context.Context, key string, localPath string) error
- func (m *Memory) Seed(key string, data []byte) error
- func (m *Memory) SeedTree(keyPrefix string, dir string) error
- func (m *Memory) SetDeleteError(err error)
- func (m *Memory) SetFlushError(err error)
- func (m *Memory) SetFlushErrorTimes(err error, n int)
- func (m *Memory) SetHydrateError(err error)
- func (m *Memory) SetListError(err error)
- func (m *Memory) SetPutError(err error)
- func (m *Memory) SetPutErrorTimes(err error, n int)
- type NewBackend
- type Option
Constants ¶
This section is empty.
Variables ¶
This section is empty.
Functions ¶
func RunSuite ¶
func RunSuite(t *testing.T, newBackend NewBackend, opts ...Option)
RunSuite runs every conformance case against backends built by newBackend.
Each case gets its own backend, so cases are independent and a failure in one does not cascade. Nothing here uses t.Parallel: a third-party backend may be pointed at a shared emulator, and a suite that is only correct when run serially is better than one that is intermittently wrong.
Types ¶
type Counts ¶
Counts records how often each method was called. Exposed so a test can assert on the *shape* of the engine's traffic — "exactly one flush on a clean shutdown", "no hydrate when the local tree was already present" — which is the kind of thing the seam gets wrong in ways the resulting file tree cannot show.
type Memory ¶
type Memory struct {
// contains filtered or unexported fields
}
Memory is a complete objectstore.Backend held entirely in process memory.
It exists for two jobs. First, it is the reference implementation: it passes the contract suite in this package, which is what makes the suite meaningful to hold the S3 and GCS modules to — a suite no implementation has ever passed proves nothing about the suite. Second, it is the substituted seam for ordinary untagged unit tests, in the spirit of the fake commandRunner in cmd/nexus-broker: a test that needs the engine's object-store path to do something real, without a network, a credential or a build tag, hands the engine one of these.
It is deliberately NOT a driver registered at init. A "memory" backend silently selectable in production config is a footgun — an operator who typo'd their real backend name would get a store that discards everything on exit while reporting success. Tests that need it reachable by name call RegisterMemory, which removes it again on cleanup.
Safe for concurrent use, because the interface requires it and because make test-race runs over this package like any other.
func RegisterMemory ¶
RegisterMemory registers backend under name for the duration of one test and returns it, so a test can drive the engine through real config (core.object_store.backend: <name>) rather than reaching past it.
The registry is process-global and Register panics on a duplicate, so the cleanup is not optional — leaving the name behind would make a second run of the same test in the same binary panic. Passing a distinct name per test is still wise if tests within a package might run in parallel.
func (*Memory) Close ¶
Close satisfies io.Closer, which the engine type-asserts rather than requiring on Backend. Implemented here so a test using this double exercises that optional-release path instead of silently skipping it.
func (*Memory) Flush ¶
Flush is a no-op: an in-memory object is durable the instant it is stored, which is the whole reason this type is a useful baseline — it can never fail the durability barrier for reasons of its own.
func (*Memory) Hydrate ¶
Hydrate writes every object under keyPrefix beneath destDir, with keyPrefix stripped from each key. Directories are created as needed; entries at destDir with no corresponding object are left alone.
func (*Memory) Keys ¶
Keys returns every stored key in ascending order. Sorted here, unlike List, because this is an assertion helper rather than part of the seam.
func (*Memory) List ¶
List returns every object under keyPrefix.
The result is returned in *descending* key order on purpose. The interface says the order is unspecified, and a double that happens to return sorted output lets a caller — or the contract suite itself — grow an accidental dependency on sorting that a real store would break. Descending is deterministic, so nothing is flaky, and it is wrong for anyone who assumed.
func (*Memory) Seed ¶
Seed stores an object directly, bypassing the local-file round trip. It is how a test arranges "the store already contains a session" without first having to build that session on disk.
func (*Memory) SeedTree ¶
SeedTree stores every file under dir, keyed by keyPrefix plus the path relative to dir. The inverse of Hydrate, for tests that build a tree on disk and then want to resume it somewhere else.
func (*Memory) SetDeleteError ¶
SetDeleteError makes every subsequent Delete fail with err.
func (*Memory) SetFlushError ¶
SetFlushError makes every subsequent Flush fail with err. The engine's failure_policy branch is the main consumer: degrade must swallow this and strict must surface it.
func (*Memory) SetFlushErrorTimes ¶
SetFlushErrorTimes fails the next n Flushes with err and heals after that. The counterpart to SetPutErrorTimes for the durability barrier, which is where a snapshot most often dies.
func (*Memory) SetHydrateError ¶
SetHydrateError makes every subsequent Hydrate fail with err. Pass nil to stop failing.
func (*Memory) SetListError ¶
SetListError makes every subsequent List fail with err. The engine's immutable-skip path is the main consumer: a backend that cannot say what it holds must make the engine upload everything, never assume presence.
func (*Memory) SetPutError ¶
SetPutError makes every subsequent Put fail with err.
func (*Memory) SetPutErrorTimes ¶
SetPutErrorTimes fails the next n Puts with err and heals after that.
A bounded outage, which is what the engine's retry-and-recover path can only be tested against: an unbounded one proves the retry happens but never that it stops. n <= 0 clears any injected Put error.
type NewBackend ¶
type NewBackend func(t *testing.T) objectstore.Backend
NewBackend builds a fresh, empty backend for one contract case. Any teardown belongs on t.Cleanup.
type Option ¶
type Option func(*options)
Option tunes the suite for backends that cannot afford a default.
func WithListProbeCount ¶
WithListProbeCount sets how many objects the pagination case stores. Set it below the backend's natural page size and the case stops proving anything; the default is chosen to exceed the common ones.
func WithoutConcurrency ¶
func WithoutConcurrency() Option
WithoutConcurrency skips the concurrent-use case. Provided for backends whose test double or emulator cannot take parallel traffic, not as a way to opt out of the requirement, which the interface states unconditionally.
func WithoutObjectAtPrefix ¶
func WithoutObjectAtPrefix() Option
WithoutObjectAtPrefix drops the tail of the prefix-matching case that stores an object at exactly the prefix other objects live under - key "sessions/sess-1" alongside "sessions/sess-1/files/a.txt".
S3 and GCS both hold that state: their key spaces are flat, "/" has no meaning beyond being a byte, and a key that happens to be another key's prefix is unremarkable. The suite asserts the state because the *engine's* key scheme can produce it and a backend that lets the object at the prefix leak into a hydration writes a file where a directory belongs.
Some stores cannot represent it at all. MinIO is the measured example: a PUT to "sessions/sess-1" against a store already holding "sessions/sess-1/files/a.txt" returns 200 and makes the child object disappear from every subsequent list, in both single-drive and erasure modes. That is a divergence from S3 in the emulator, not in the backend under test, and the resulting bucket cannot even be emptied by listing it.
So this exists for emulators, and only for emulators. It is not a way for a backend to opt out of the semantic: a backend that needs it is claiming its *store* has a flat-key-space limitation, which is a property worth stating out loud at the call site. Run the rest of the suite unaffected -- every other prefix-matching assertion in the case still runs, including the "sess-1" versus "sess-10" collision that motivates the case.