Documentation
¶
Overview ¶
Package testredis is a throwaway redis-server for one test.
Start runs the redis-server on PATH as a child of the test binary and returns its address. What a test can rely on:
WHAT IT LISTENS ON. One TCP listener on 127.0.0.1, the IPv4 loopback, on a port the kernel chose. No other interface, no IPv6, no TLS port and no Unix socket. The address is host:port and not a socket path because every caller hands it to a Redis client's Addr or to a tool's --redis flag, and both dial TCP. An extra argument that would move the listener is refused.
IT IS THIS TEST'S ALONE. The port is taken from the kernel, closed and handed to redis-server, so another process can bind it in between. A server is therefore ready only when its OWN output says so, never because something answered on the port; one that lost its port exits and another port is taken. Many tests may call Start at once.
NOTHING IS KEPT. --save "" and no append-only file, in the test's own t.TempDir(). A test that wants persistence asks for it in the extra arguments, and the files still land in that directory.
IT DOES NOT OUTLIVE ITS TEST. The test's cleanup kills the server and waits until the process is gone, then the testing package removes the directory: after a pass, a failure or a panic in the test. When the test binary itself dies without its cleanups (a timeout, os.Exit, a kill) the sentry kills the server: see sentry.go. A server of a test binary that was killed leaves its directory, as every t.TempDir() of that binary does.
A MISSING redis-server skips the test on a laptop and fails it under NOVA_CI=1, which the CI workflows set: a skip there would let a green run be a run that never executed the store. A redis-server that is found and does not come up fails the test anywhere.
THE IMAGE OF A STORE. Image reads every key under a prefix, with its type and a sum of its content and expiry time, by SCAN and pipelines, from any server; Diff names the keys added, removed and changed between two images, in key order. A test that must show a step wrote nothing takes an image before the step and one after it and expects an empty Diff: see image.go.
A STORE AT A DISTANCE. Far puts a proxy in front of a store a test owns, listening on the loopback, that holds each write of the client back by a delay before it forwards it, so a limit is judged at the distance the real store stands at and not at the loopback's. It stops with the test.
Index ¶
- Constants
- func Absent(t testing.TB, cause error)
- func CommandCounter(t testing.TB) (addr string, count func() int64)
- func Diff(before, after map[string]Entry) []string
- func Far(t testing.TB, target string, delay time.Duration) (addr string)
- func FarLink(t testing.TB, target string, delay time.Duration) *delayproxy.Proxy
- func FreePort(t testing.TB) string
- func Image(ctx context.Context, c redis.UniversalClient, prefix string) (map[string]Entry, error)
- func OnlyFCALL(t testing.TB) redis.Hook
- func Program(t testing.TB) string
- func SocketPath(work, name string) (string, error)
- func Start(t testing.TB, extra ...string) string
- func User(name, password string) []string
- type Counter
- type Entry
- type Server
Constants ¶
const CIEnv = "NOVA_CI"
CIEnv is set to "1" by the CI workflows. A missing redis-server fails the test in that environment. Anywhere else it skips, so a laptop without the binary can still run the rest of the suite.
Variables ¶
This section is empty.
Functions ¶
func Absent ¶
Absent is Start's answer when redis-server is not on PATH. CI fails closed. A skip here would let a green run be a run that never executed the store.
func CommandCounter ¶
CommandCounter listens on 127.0.0.1, on a port the kernel chose, and counts every command a Redis client sends it, handshake included. It is not a redis-server and starts no process. It answers just enough RESP2 for a go-redis client to finish a command: HELLO is refused so the client falls back to RESP2, PING is PONG and anything else is OK. A test calls a package's Open or Dial against the address and asserts the count is still zero: the caller's first batch, not the open, is the first command.
It answers "how many commands reached a listener". To ask "did my verb batch" (how many round trips, however many commands they carried, a connection's handshake not counted), use RoundTrips.
func Diff ¶
Diff is what changed between two images, as one line per key: "+key" for a key the second image has and the first does not, "-key" for a key the first has and the second does not, "~key" for a key both have whose Entry is not the same, that is whose type, content, expiry time or (for a hash) fields' expiry times are not. The lines are in key order, sorted bytewise by the key, each with its mark in front, so the lines of added, removed and changed keys are interleaved as their keys are and a test reads them by key. It is empty, and not nil, when the images are equal, and a nil image is an empty one.
A read and a rewrite of a hash, set or sorted set with the same content are not changes (see Entry). The sum of a string, a list and a stream is of their DUMP, so a write that leaves one of those with the same content in another encoding is a change: a difference a test that expects no write should hear about.
func Far ¶
Far puts a store at a distance. It listens on 127.0.0.1, on a port the kernel chose, and forwards every connection to target, holding each write of the client back by delay before it forwards it. A client that dials the address it returns reaches the target as if the target stood a delay away: the answer is not held, so one command and its reply cost delay once, and a delay of 128ms is a store 128ms away by round trip. It is how a test judges a limit at real distance: a PING through Far(t, store, 100*time.Millisecond) costs one delay, a pipeline of a hundred written in one go costs one, and three commands sent one after the other cost three.
A write is what one read of the client's connection returns, which for a write of up to 16 KiB on the loopback is the write. Reads that arrive together are each sent on at their own arrival plus delay, so they pay it once between them, up to a window of delayproxy.WindowBytes (4 MiB) in flight: a pipeline past it pays the delay once more for each further window.
The delay is held by a timer, so a test that asserts on wall time asserts on its machine's load: a test of another package asserts on FarLink's ledger instead. The wall-clock windows of Far itself (one PING takes the delay, a pipeline takes it once, three commands one after the other take it three times) are asserted in the container by tools/fardelay's functional test, with a slack for a busy machine on the upper bounds.
target is the address of a store this test owns, an IP in 127.0.0.0/8 or ::1 or localhost, and a port, typically the address Start returned: Far is a way to make a local store far and never a way out of the machine, and it fails the test for any other target. It stops in the test's cleanup, with nothing it started left running. See FarLink for the proxy in hand.
func FarLink ¶
FarLink is Far with the proxy in hand, for a test that wants the ledger beside the address, as Start has StartServer. The proxy counts every write it has held and forwarded (Writes) and the least time any of them was held by its own clock (Shortest), so a test asserts the events: a pipeline of a hundred paid the delay once, three commands paid it three times and none was held less than the delay, and no test measures a wall clock a busy machine moves. Stop ends it before the test does.
func FreePort ¶
FreePort is a loopback port that was free a moment ago, as text. The kernel chose it. Redis treats --port 0 as "do not listen", so a test that starts a server itself takes the port here and hands it over.
func Image ¶
Image is every key of the store under prefix, each with its Entry: the whole of what a test needs to prove that a step wrote nothing (Diff of an image before it and one after it is empty) or wrote exactly what it should. The Entry of a key is its type and a content sum that folds in its expiry time, and for a hash each field's (see Entry): a write of a time to live alone is seen, and a read, or a rewrite of a hash, set or sorted set in another order, is not.
THE KEYS ARE FOUND BY SCAN, never KEYS. SCAN MATCH prefix*, COUNT 1000, follows the cursor until the server says it is done, and every key is read once whatever number of times SCAN names it. The prefix is a literal: the glob characters in it (* ? [ ] and the backslash) are escaped, so a prefix such as "a[1]:" matches keys that begin with those five characters and no others. The empty prefix is every key of the database.
THE KEYS ARE READ IN PIPELINES, IN BATCHES OF AT MOST 500. A batch is two round trips: TYPE and PEXPIRETIME of every key, then the content of the keys that are there, by type (HGETALL of a hash, SMEMBERS of a set, ZRANGE WITHSCORES of a sorted set, DUMP of every other type); and a third when the batch holds a hash, HPEXPIRETIME of all the fields of each hash. An image of ten thousand keys is at most sixty pipelines and the SCAN calls, not thirty thousand round trips. A batch is bounded by its count of keys and not by bytes: 500 large values are held together and read under one read deadline, so an image is for keys of the size a test writes. The server needs PEXPIRETIME (Redis 7.0 or later) and, for a hash, HPEXPIRETIME of fields (Redis 7.4 or later).
IT IS AN IMAGE OF A STORE AT REST. The keys are read in several calls, so an image taken while another client writes is not a snapshot. A key that is gone when its turn to be read comes (deleted, or expired on its own) is not in the image. A key that was gone at TYPE and there at PEXPIRETIME, a hash, set or sorted set that is not of the type TYPE said when its content is read (the server answers WRONGTYPE), and a hash that is not of that type, or that has lost a field that HGETALL read, when its fields' expiries are read, were written while the image was taken: Image fails and names the key. A key read by DUMP (a string, list, stream, or another type) that is replaced by a key of another type between the round trips is not found out: its Entry has the type TYPE said and the sum of the DUMP that answered.
The image is of the one database the client reads. Image refuses a cluster client and a ring, whose SCAN reads one node: an image of part of a store proves nothing about the rest. An error is never a partial image: the result is the whole map or the error.
func OnlyFCALL ¶
OnlyFCALL is a go-redis hook for a test whose code writes to the store through FCALL and through nothing else:
client.AddHook(testredis.OnlyFCALL(t))
A command in the fixed list of this file (writeGroups: every write command of Redis 8, by type) fails the test with the command's name, and the command is not sent, so the store never sees the write the test has just failed on. The caller gets an error that names the command, and so does every command of a pipeline or a transaction that held one.
WHAT FAILS. A command that writes and is not FCALL: SET, HSET, XADD, DEL, FLUSHALL and the rest of the list, and the two commands that run a script that may write (EVAL, EVALSHA), which is a way round the loaded library. A command with subcommands is a write by its pair: FUNCTION LOAD and XGROUP CREATE fail, FUNCTION LIST and XGROUP HELP do not. The names are read without regard to case.
WHAT PASSES. FCALL, whatever the function does inside the server (the hook sees the call and not its effects), FCALL_RO, EVAL_RO and EVALSHA_RO, every read, and everything else not in the list: the handshake, MULTI, EXEC, WATCH, PUBLISH and the administrative commands. The list holds the commands that write to the keyspace, not every command that changes something.
THE LIST DECIDES, NEVER THE SERVER. The hook asks the server nothing, sends nothing and works on a client that has no server behind it. The source of the list and the test that holds it to a real server are on writeGroups.
A pipeline or a transaction is judged whole: when one command in it writes, the failure names every write in it, with its place in the batch, and none of the batch is sent. A transaction's MULTI and EXEC pass; the commands between them are judged one by one.
LOADING THE LIBRARY IS A WRITE. FUNCTION LOAD fails on a client that has this hook, so a test loads the library through another client and hands the code under test the one that has the hook.
The failure is reported with Errorf, so a client used from a goroutine other than the test's reports it there too; the test ends failed when the test function returns. Like the rest of the package it panics outside a test binary.
func Program ¶
Program is the redis-server on PATH, or Absent's answer when there is none. A test that launches the server through its own production path (nova-redis serve) takes the program from here, so the missing-binary rule stays one.
func SocketPath ¶
SocketPath is where a unix socket of a test goes: under work when the path fits a Unix socket, else in a fresh private directory under the temp directory, else under /tmp. The bound is 104 bytes, below the shortest sockaddr_un path of the platforms this runs on (104 on macOS, 108 on Linux), as internal/tablemodel's maxSocketPath and cmd/nova-sprint's twinSocketMax. A work directory deep under a long TMPDIR (a CI runner's) cannot hold a socket and redis-server refuses to bind it, so the caller must use this before it starts one. The directory this makes is the caller's to remove.
func Start ¶
Start runs a throwaway redis-server and returns its address, 127.0.0.1 and a port, once the server has said it is ready and has answered a PING there (a NOAUTH answer is a live server: the default user may be off). It listens on that one loopback TCP port and nowhere else, keeps nothing, and is killed by the test's cleanup.
The extra arguments follow the fixed ones, so a test can add users (testredis.User), an ACL file, or persistence inside its own directory. Start refuses the ones that are its own: bind, port, tls-port and unixsocket (where it listens), dir and include (where its files are), logfile and loglevel (the output it reads), daemonize and supervised (the process it holds).
A server that does not come up fails the test with the binary, the arguments and the last of its output. Start waits thirty seconds for one server and takes five ports before it gives up.
Types ¶
type Counter ¶
type Counter struct {
// contains filtered or unexported fields
}
Counter counts the round trips a go-redis client makes: the times it sends something to the server and waits for the answer. RoundTrips makes one.
A round trip is one command, one pipeline or one transaction. A pipeline of a hundred commands is one, and so is a transaction: go-redis hands MULTI, the commands and EXEC to a hook as one batch and sends them together. A pipeline with no commands sends nothing and is none. The setup of a connection is not a round trip the caller made and is not counted (see RoundTrips).
A Counter is safe for use by many goroutines.
func RoundTrips ¶
RoundTrips returns a Counter and the hook that feeds it. The test adds the hook to the client it wants counted:
trips, hook := testredis.RoundTrips(t)
client.AddHook(hook)
trips.Expect(t, 1, func() { verb(ctx, client) })
This is the question "did my verb batch": how many times did the client send something and wait, however many commands that carried. CommandCounter asks another: how many commands reached a listener, every one of them, a connection's handshake included.
WHAT IS COUNTED is every command, pipeline and transaction the caller sends the client, once each, when it comes back from the connection, whether or not it succeeded. A failure names them in that order, which on one connection is the order they were sent. A command go-redis sends again after a network error is one round trip here: the retry happens below the hooks.
SETUP IS NOT COUNTED, BY WHO SENT IT AND NEVER BY NAME. go-redis connects inside the first command that needs a connection, with that command's context, and runs the handshake (on go-redis v9 HELLO, then CLIENT SETINFO in a pipeline) through the hooks; every new connection of a pool does it again. The hook marks the context of each round trip it counts, so a command that arrives carrying the mark is a connection's setup and is not counted, whatever its name, and a command the caller sends is counted, whatever its name: the caller's own HELLO or CLIENT SETNAME is one round trip. So a cold client's first PING is one round trip, and the count of work done by many goroutines does not depend on how many connections the pool opens to do it. This is the definition pkg/redisconn.Trips gives a receipt.
ADD IT LAST. go-redis runs the hook added first outermost and the hook added last nearest the connection. A hook added before this one that sends a pipeline again (a library loader that retries) makes two round trips, and this one, added last, counts two. Added first, it would count one: the second sending would happen below it, and a command a hook below it sends with the context it was given would carry the mark and be taken for setup.
Only the names of commands are written down, never their arguments, so a failure message carries no key, value or password.
RoundTrips panics outside a test binary, as the rest of the package does.
func (*Counter) Expect ¶
Expect runs do and fails the test unless the client made exactly n round trips while it ran. The failure says how many were expected and how many were counted, and names the ones counted, so a test that expects one round trip and counted a handshake and a verb reads it at once. Only the round trips that happen while do runs are counted: those before Expect and those after it are not. Round trips made by other goroutines while do runs are counted as well.
A failure ends the test with t.Fatalf. If do itself fails the test or panics, that is the failure and the count is not read.
type Entry ¶
type Entry struct {
Type string // the key's TYPE: string, list, set, zset, hash or stream
Sum [32]byte // the content sum described above
}
Entry is what an image holds of one key: the key's TYPE and a sum of its content and its expiry times. Two entries are equal when the key has the same type, the same content and the same expiry times: the key's, and for a hash each field's.
THE SUM IS A CONTENT SUM, NOT THE SHA256 OF THE KEY'S DUMP ALONE. The specification of this helper said the SHA256 of DUMP; it is amended to this. A DUMP is the server's serialization of a value: it carries the encoding, and for a hash or a set in the hashtable encoding the order of the table. Reading a key was measured to change that order (HGET and SISMEMBER of every field and member of a large hash and set changed the DUMP of both), and a value written in another order, or one that grew and shrank back, has another DUMP with the same content. So the DUMP of a hash, a set or a sorted set is not a fact about what the key holds, and the sum is not of it.
The sum is the SHA256 of these bytes, in this order. Every number is 8 bytes, big endian, and a length is the number of bytes of the string after it.
- The content, by the key's type. A string, a list and a stream, and any type not named here, contribute their DUMP. A hash contributes the number of its fields, then each field, sorted bytewise, as its length, the field, the length of its value and the value. A set contributes the number of its members, then each member, sorted bytewise, as its length and the member. A sorted set contributes the number of its members, then each member, sorted bytewise, as its length, the member and the IEEE 754 bits of its score.
- The expiry of each field of a hash, after the content of the hash and in the same order as its fields: HPEXPIRETIME, the time in milliseconds since the Unix epoch at which the field expires, as a signed number, and -1 for a field with no expiry. The other types have no such part.
- The key's absolute expiry, in every key's sum: PEXPIRETIME, the time in milliseconds since the Unix epoch at which the key expires, as a signed number, and -1 for a key with no expiry.
So the sum of a hash is the SHA256 of: its field count; each field and its value, in field order; the expiry of each field, in field order; and the key's expiry. A field's expiry is in neither HGETALL nor PEXPIRETIME, so it has its own part, and it is read with one HPEXPIRETIME for each hash.
What follows. A read changes no Sum. A hash, set or sorted set rewritten in another order, or in another encoding, with the same content, has the same Sum. A write of the expiry alone (EXPIRE, PEXPIRE, PEXPIREAT to another time, PERSIST; and on a field of a hash HEXPIRE, HPEXPIRE, HPEXPIREAT, HPERSIST) changes the Sum, so a step that wrote nothing but a time to live is seen, whether it is the key's or a field's. The expiry is absolute, so a time to live that runs down by itself is not a change, and the same absolute time set again is not either.
The sum of a string, a list and a stream stays the sum of their DUMP: a write that leaves one of them with the same content in another encoding is a change too, a difference a test that expects no write should hear about.
type Server ¶
type Server struct {
// contains filtered or unexported fields
}
Server is one running throwaway redis-server.
func StartServer ¶
StartServer is Start with the server in hand, for a test that stops it before the test ends.
Source Files