Documentation
¶
Overview ¶
Package redisfn builds, loads and checks a Redis function library from Lua source held in a file system, which for a tool is the Lua it embeds. It knows no library by name: a tool describes its own with a Library value and calls Digest, Registered, Check, Ensure and LoadMissing on it.
The source ¶
One library is one text, built the same way every time. In bytes, it is
"#!lua name=" <name> "\n" <the prelude, as it is; it may be empty>
and then, for every file the glob matches in the order of their names sorted byte by byte, one block:
"\n-- " <file> "\ndo\n" <the file's text, as it is> "\nend -- " <file> "\n"
where <file> is the name the glob matched, with any character that would break or reorder a line escaped (oneline.Escape; a name of printable characters is written as it is). So a block starts with an empty line (unless the text before it does not end in a line break, which that "\n" then ends), its header, "-- <file>", and a line "do"; and a file that ends in a line break is followed by an empty line before its "end -- <file>". These are the bytes of the loader this was lifted from (pkg/nsprint/fn), so a library built by either loader from the same files and prelude has one digest: the digest of a deployed library is a contract, and so is the header, on which that loader's tests split the source ("\n-- lua/").
A file's block is what keeps a library loadable as it grows: Lua refuses a function that holds more than 200 local variables at once, the library's text is one function, and without the blocks every top-level local of every file counts towards the 200 (the library this was lifted from went from 199 to 206 in one merge and the store refused it). Inside its block a file's locals leave scope at the file's end, so the count is the prelude's locals and the largest file's, not the sum. What one file hands to a later one goes through a table the prelude declares. The loader counts them as Lua does and refuses a library over MaxLocals, which leaves room below Lua's 200.
The block holds only when a file cannot reach outside it, so a file is refused when a string or a long comment of it does not close, when its block words do not balance, or when it returns outside every function. Those, and every other refusal, come before the store is touched; each wraps ErrRefused.
Line numbers ¶
Redis names a failing line of the library as user_function:<n>, counted in the text above; the note a failed call adds to an error names the file and the line it came from. A file's header says where the file starts: its line 1 is two lines below the header, after the do, so the same can be done by hand from the library's code as the store returns it (FUNCTION LIST WITHCODE). A carriage return is refused in a file: Lua counts "\r", "\r\n" and "\n\r" each as one line break and an editor may not, and a line number has to mean one thing.
The store ¶
A load replaces the library with one command, FUNCTION LOAD REPLACE, after which the store holds the whole library it held before or the whole of the new one, never a part of either. Check reads the library's code back from the store and compares it with its source: the code is the library's identity, and no key, no counter and no answer of a function stands in for it.
There are two ways to put a library on a store when it is not there, and they differ in what they do to a library that is:
- Ensure is the deployer's: Check, and then a load when the store does not hold this code, so it replaces other code under the name. It is for the one place that deploys.
- LoadMissing is everyone else's: it loads only when the store holds no library of the name, with FUNCTION LOAD and never REPLACE, so it never replaces a library, whatever its code (nova-tools #3620: a verb on its way to an FCALL ran an older binary, whose load replaced the deployed library and took the functions added since). A store that holds a library of the name, a load that loses the race to another loader, a function name another library holds and a caller the store does not let read its libraries are each an answer, not an error.
Each call returns within a bound, whatever the client's timeouts are.
The store under several loaders is modelled in tla/RedisFn.tla: one holder to a function name, a refusal that writes nothing, no moment without the library, a LoadMissing that never replaces, and what two deployers of two builds do to each other. TLC ran every configuration on 2026-09-28 (UTC); tla/README.md has the configurations, what each shows and its state counts. What this package says of Redis is held against a redis-server by its functional tests.
Index ¶
- Constants
- Variables
- func DigestOf(source string) string
- type CollisionError
- type Function
- type Held
- type Library
- func (l Library) Check(ctx context.Context, client redis.UniversalClient) (State, error)
- func (l Library) Digest() (string, error)
- func (l Library) Ensure(ctx context.Context, client redis.UniversalClient) (Receipt, error)
- func (l Library) LoadMissing(ctx context.Context, client redis.UniversalClient) (Receipt, error)
- func (l Library) Query() redis.FunctionListQuery
- func (l Library) Registered() ([]Function, error)
- type MismatchError
- type Origin
- type Outcome
- type Receipt
- type State
Constants ¶
const DefaultBound = 10 * time.Second
DefaultBound is how long one call of Load, Check, Ensure or LoadMissing waits on the store when the Library names no Bound of its own.
const DigestLength = 16
DigestLength is the number of characters of a digest, all lower-case hex.
const MaxLocals = 180
MaxLocals is the most local variables the library's main function may hold at once: the prelude's that are still in scope at its end, and the most any one file holds. A library over it is refused. Lua (and so Redis) refuses a function of more than 200; the 20 between are room for the next merge, so the change that nears the limit is refused in the tests of the tool that carries the library, which build it, and not at a deploy after the change that passes it. The count is Lua 5.1's (scan.go says how), and the functional tests hold it against a redis-server at Lua's limit.
const MaxSourceBytes = 4 << 20
MaxSourceBytes is the largest source a library may have, 4 MiB: a library whose source would be longer is refused. Check reads the whole source back from the store on every call, so the bound is what one Check may cost. The largest library lifted into this form is under 1 MiB.
Variables ¶
var ErrRefused = errors.New("library refused")
ErrRefused is wrapped by every error that refuses a library for what it is (its name, its files, their text, its size) and so before the store was touched. errors.Is(err, ErrRefused) tells a defect of the build, which no retry mends, from a failure of the store.
Functions ¶
Types ¶
type CollisionError ¶
type CollisionError struct {
// Library is the library the store refused.
Library string
// Held is the function the store named, and with it every function name
// read from this library's files that another library registers, sorted
// by name. It is never empty.
Held []Held
// Unread is why the store's libraries could not be listed to find the
// holders, and nil when they were. When it is set, Held is the one
// function the store named, with no holder.
Unread error
}
CollisionError is the load's error when the store refused the library because another library on it registers one of this library's function names. A function name belongs to one library, compared without case, so while two libraries carry the same function one of them cannot be loaded.
func (*CollisionError) Error ¶
func (e *CollisionError) Error() string
Error is one line that names each function with the library that holds it, and the remedy.
func (*CollisionError) Unwrap ¶
func (e *CollisionError) Unwrap() error
Unwrap is Unread, so errors.Is sees why the holders could not be read.
type Function ¶
type Function struct {
// Name is the name as its file spells it.
Name string
// File is the file that registers it, as the glob matched it.
File string
// NoWrites is true when its registration names the flag no-writes, so
// FCALL_RO may call it.
NoWrites bool
}
Function is one function the library registers, as Registered reads it.
type Held ¶
type Held struct {
Function string // the name, as the store's error or this library spells it
Holder string // the library on the store that registers it; "" when none was found
}
Held is one function name and the library that registers it on the store.
type Library ¶
type Library struct {
// Name is the library's name on the store: one or more ASCII letters,
// digits or underscores, which is Redis's own rule. Redis tells names
// apart by case.
Name string
// Files holds the library's Lua files.
Files fs.FS
// Glob names the files of Files that belong to the library, in the
// syntax of path.Match: "lua/*.lua". Every match must be a file that can
// be read and that holds more than whitespace.
Glob string
// Prelude is Lua placed before the first file, outside every file's
// block: its locals are in scope in every file. It may be empty. It is
// held to the rules of a file.
Prelude string
// Remedy is what an operator does to put this library on the store, in
// the tool's own words ("mytool fn load --redis <addr>"). Check's
// error ends with it. When it is empty the error names Ensure.
Remedy string
// Bound is the longest one call of Load, Check, Ensure or LoadMissing
// waits on the store. Zero or less is DefaultBound.
Bound time.Duration
}
Library describes one Redis function library. It is a value: copying it is safe, the methods change nothing in it, and each method reads the files again, so two calls agree exactly when the files did not change between them (an embed.FS never does).
func (Library) Check ¶
Check answers whether the library on the store is the one this binary was built with, and changes nothing. It sends one FUNCTION LIST (Query) and judges the reply (judge): Same with a nil error, Different or Absent with a *MismatchError whose one line names the library, both digests and the remedy, Unknown with the error of a store that could not be read or of a library that the source refuses.
The identity it reads is the library's code as the store holds it. The store keeps that code with the functions it compiled from it, and changes the two together or not at all, so nothing can change what Check reads and leave the library as it was, and nothing can change the library and leave what Check reads. The answer is true of the moment the store replied; another loader may act after it.
It returns within the Bound, whatever the client's own timeouts are, and a context that has ended already sends nothing.
func (Library) Digest ¶
Digest returns the digest of the library's source: the identity of the library this binary was built with, which changes when any byte of the source changes. Its error is the build's refusal.
func (Library) Ensure ¶
Ensure is the deployer's load: it puts this library on the store unless the store holds exactly this code, replacing other code under the name. It reads the store as Check does, and when the state is Absent or Different it loads with one FUNCTION LOAD REPLACE. The receipt says which happened: Unchanged (the store held this code; nothing was written), Loaded (the name was free) or Replaced (other code was under the name, whose digest is Was). Beside an error the outcome is Failed, and the error is Check's for the read or the load's for the load.
The load is one command, FUNCTION LOAD REPLACE, and the store never holds half of the library, which is Redis's doing, not this package's: inside that one command it makes the new library of the whole text and, when anything fails (a syntax error, a Lua error when the library's own text runs, a function name another library holds, a library that registers nothing), has the library it held before in place again when it replies. No other client's command runs in between, so no client sees a store between the two; after a load, error or not, the store holds the whole library it held before or the whole of this one (the functional tests hold each of those refusals against a redis-server; tla/RedisFn.tla, NoGap, shows what FUNCTION DELETE followed by FUNCTION LOAD would lose). A nil outcome says the store answered the load with this library's name; a reply of the store itself (errors.As finds a redis.Error) says the store holds what it held before; any other error, a lost connection or the Bound passing, says neither: the command may have run, whole, after the caller stopped waiting, and Check says which. When the store refuses the library because another library registers one of its function names, the error is a *CollisionError that names the function and the library that holds it, and an error whose text names a line of the library carries the line's file in the note after it.
The read and the load are two commands, and another loader may act between them; the load replaces whatever is there by then, so a receipt that says Loaded may have replaced a library loaded in that moment. Two binaries built with different libraries that both call Ensure on one store replace each other's library for as long as both run (tla/RedisFn.tla, MCRedisFnTwoDeployers). Call it from the one place that deploys; a tool on its way to an FCALL calls LoadMissing, which never replaces, or Check.
It returns within the Bound, the read and the load together, whatever the client's own timeouts are, and a context that has ended already sends nothing.
func (Library) LoadMissing ¶
LoadMissing puts the library on the store only when the store holds no library of its name, and never replaces one, whatever its code. It is for every caller that is not the deployer: a tool on its way to an FCALL runs whatever binary its host has, and an older binary can replace the deployed library and remove every function added since. Putting a newer build over an older one is Ensure's job, in the one place that deploys.
It sends FUNCTION LIST for the name (without the code) and, only when no library of exactly this name is there, FUNCTION LOAD, without REPLACE, so a library another loader puts there between the two is refused by the store and left as it is. The receipt says what happened:
- Unchanged: a library of this name was on the store, at the read or at the load; nothing was written and its code was not read;
- Loaded: the name was free and this library is on the store now;
- Skipped: the store refused the read with NOPERM (the caller is not the deployer), or refused the load because another library holds one of this library's function names; nothing was written, Why is the store's reply, and the caller's FCALL answers for the store.
Each of those comes with a nil error. Any other failure is an error with the outcome Failed: a store that could not be reached or read, or a load the store refused for the library itself (a syntax error, a name registered twice), after which the store holds what it held before.
It returns within the Bound, the read and the load together, whatever the client's own timeouts are, and a context that has ended already sends nothing.
func (Library) Query ¶
func (l Library) Query() redis.FunctionListQuery
Query is the FUNCTION LIST that reads this library's code, the command Check and Ensure send. The store matches the name as a pattern and without case, so the reply may hold other libraries as well; judge takes this one by its exact name.
func (Library) Registered ¶
Registered returns every function the library's files register, with the file that registers it and its no-writes flag, sorted by name. The flag is read from the table form's flags; the string form registers no flags. Its error is the build's refusal.
type MismatchError ¶
type MismatchError struct {
Library string // the library's name
Want string // the digest of this binary's library
Loaded string // the digest of the code the store holds; "" when it holds none
Remedy string // what to do about it
}
MismatchError is Check's error for Different and Absent: the store does not hold the library this binary was built with.
func (*MismatchError) Error ¶
func (e *MismatchError) Error() string
Error is one line that names the library, both digests (loaded=none when the store holds none) and the remedy.
type Origin ¶
type Origin struct {
// File is the file the line belongs to, as the glob matched it. It is
// empty for the library's first line and for the lines of the prelude.
File string
// Line is the line's number in File, or in the prelude, from 1. It is 0
// for a line the loader wrote: the library's first line, and the header,
// the do and the end around File.
Line int
// Prelude is true for the lines of the prelude.
Prelude bool
}
Origin is where one line of a library's source came from.
type Outcome ¶
type Outcome int
Outcome is what Ensure or LoadMissing did to the store.
const ( // Failed is the outcome beside an error: what the store holds is not // known from this call. Failed Outcome = iota // Unchanged: nothing was written, because the store held the library // already. For Ensure that is this code; for LoadMissing it is a library // of this name, whatever its code, and it may have been put there by // another loader between LoadMissing's read and its load. Unchanged // Loaded: the store held no library of this name and holds this one now. Loaded // Replaced (Ensure only): the store held other code under this name and // holds this library now. Replaced // Skipped (LoadMissing only): nothing was written, and the store may not // hold the library, because the store refused this caller the read of // its libraries (NOPERM: the caller is not the deployer) or refused the // load for a function name another library holds. Receipt.Why is the // store's reply. The caller's FCALL answers for the store. Skipped )
type Receipt ¶
type Receipt struct {
Library string // the library's name
Outcome Outcome // what was done
Digest string // the digest of this binary's library
Was string // the digest of the code the store held before; "" when it held none or was not read (LoadMissing never reads it)
Why string // the store's reply when the outcome is Skipped; "" otherwise
}
Receipt is what one Ensure or LoadMissing did.
type State ¶
type State int
State is what Check found on a store.
const ( // Unknown is the state when there is no answer: the store could not be // read, or the library was refused. It comes with the error that says why. Unknown State = iota // Same: the store holds a library of this name whose code is the source, // byte for byte. Same // Different: the store holds a library of this name with other code. Different // Absent: the store holds no library of this name. Absent )
Source Files