Documentation
¶
Overview ¶
Package journal keeps the requests a mock server serves, of any protocol, whatever their outcome, so a test can verify after the fact what the code under test sent: a changelog of JSON lines in segment files, with an index in memory that narrows queries. Queries match with the matchers of mocks, such as those of package github.com/caffeine-projects/vlatte/match. A server embeds a [Verifier], next to its vlatte.Base, to verify its requests.
The servers of github.com/caffeine-projects/vlatte/mockhttp and github.com/caffeine-projects/vlatte/mockgrpc keep a journal when their setup asks for one. A Journal works on its own too, as a changelog of any requests: New opens one.
Entries ¶
An Entry is one request, with the fields every protocol has: its kind, its Outcome, the mock that answered it, its method, path and full target, the status of its answer, its time, headers and body, and, for a request no mock matched, the mocks that came closest. Journal.Append gives each entry a sequence number, which orders them: a query returns the newest first, and Journal.Get finds one by its request id. A server fills in the entry of a request it serves in a Draft, which appends it once the request ends, telling the on_complete events that failed.
Files ¶
The journal never rewrites an entry. Each one is a line of JSON appended to the active segment, a file named journal-NNNNNN.log. A new segment starts once the active one holds a fourth of MaxEntries, 10 000 at most, or 64 MiB, so dropping the oldest entries deletes whole files. In a named directory, a segment that fills gets a sidecar, journal-NNNNNN.idx, with what the index keeps of its entries, as does the active one on Close: the next journal of the directory reads the sidecars instead of the logs, as long as each describes its log as the log is.
A named directory keeps the entries across restarts, and gets a .gitignore that keeps its files out of git. An empty Dir keeps them in a temporary directory, which Close removes. One journal at a time opens a directory, where the system locks files: another fails with ErrInUse.
The index ¶
The journal keeps a row of each entry in memory: its sequence number, request id and time, the fields a query selects by but headers and body, and where its line is on disk. For the indexed fields, kind, outcome, mock id, method, path and status, it also lists, for each value it holds, the entries that have it.
Queries ¶
A Query selects entries by those fields, by the full target, by time, by sequence number, and by headers and body; Filter values, such as the request patterns of a protocol, narrow them further. A query runs in steps, each reading less than the one before:
- It copies the rows, and the values of the indexed fields it selects by, then lets go of the journal.
- It runs each matcher of an indexed field once per value the journal holds, not once per entry, and takes the entries of the values that match: the fields it selects by intersect.
- It checks the rows left against the target and the time, which rows hold too.
- Only then, and only when it selects by headers, body or filters, or returns entries, does it read their lines from disk.
So Journal.Count of a query of indexed fields alone reads no file. Journal.Find returns a Page: the entries, the count of all the query selects, and, for a query with a Limit, where the next page starts. A query's After, such as the Journal.Last of a moment before, selects the entries appended since.
Writes and reads ¶
Append is synchronous: an entry is in the journal once Append returns. The mock servers append a request's entry as they end its answer, so a test checks it right after the request returns. Queries match and read the copy they took without holding the journal, so appends don't wait on them. A journal is safe for concurrent use. Journal.Changed wakes a caller waiting for an entry on the next append, clear or close, so it queries again rather than poll.
Retention and recovery ¶
The journal keeps the newest MaxEntries entries, all of them for 0, and MaxBodySize bytes of each body: an entry whose body lost the rest says so. Dropping the oldest entries deletes the segments that hold none of the newest. Journal.Trimmed tells the newest entry dropped so, and Journal.Cleared the last one before the journal's latest start, a clear or the oldest entry it loaded: a count of the entries after a point is whole when neither reaches past it.
A crash may cut a line short, or leave a file half written: the next journal of the directory leaves the line out, logs it, and removes the file. Journal.Clear writes a tombstone, journal.clear, before it deletes the segments, so a journal opened after a crash, or after a delete that failed, drops them all the same.
Verifying ¶
Package github.com/caffeine-projects/vlatte/verify checks the requests of a server's journal in a test: which came, how many, by when and in what order.
Example ¶
package main
import (
"context"
"fmt"
"net/http"
"github.com/caffeine-projects/vlatte/journal"
"github.com/caffeine-projects/vlatte/match"
)
func main() {
// An empty Dir keeps the files in a temporary directory, which Close removes.
j := journal.MustNew(journal.Options{})
defer j.Close()
for _, e := range []*journal.Entry{
{ID: "1", Kind: "http", Outcome: journal.OutcomeMatched, Method: "POST", Path: "/orders", Status: http.StatusCreated},
{ID: "2", Kind: "http", Outcome: journal.OutcomeMatched, Method: "GET", Path: "/orders/1", Status: http.StatusOK},
{ID: "3", Kind: "http", Outcome: journal.OutcomeUnmatched, Method: "GET", Path: "/users", Status: http.StatusTeapot},
} {
if err := j.Append(e); err != nil {
panic(err)
}
}
page, err := j.Find(context.Background(), journal.Query{Path: match.HasPrefix("/orders")})
if err != nil {
panic(err)
}
// The newest come first.
for _, e := range page.Entries {
fmt.Println(e.Seq, e.Method, e.Path, e.Status)
}
unmatched, err := j.Count(context.Background(), journal.Query{Outcome: journal.OutcomeUnmatched})
if err != nil {
panic(err)
}
fmt.Println(unmatched)
}
Output: 2 GET /orders/1 200 1 POST /orders 201 1
Index ¶
- Constants
- Variables
- func ClipBody(body []byte, n int) []byte
- type Draft
- type Entry
- type Filter
- type Journal
- func (j *Journal) Append(e *Entry) error
- func (j *Journal) Changed() <-chan struct{}
- func (j *Journal) Clear() error
- func (j *Journal) Cleared() uint64
- func (j *Journal) Close() error
- func (j *Journal) Count(ctx context.Context, q Query, filters ...Filter) (int, error)
- func (j *Journal) Dir() string
- func (j *Journal) Draft(entry Entry) *Draft
- func (j *Journal) Find(ctx context.Context, q Query, filters ...Filter) (Page, error)
- func (j *Journal) Get(id string) (*Entry, error)
- func (j *Journal) Last() uint64
- func (j *Journal) Trimmed() uint64
- type Options
- type Outcome
- type Page
- type Query
Examples ¶
Constants ¶
const ( // DefaultMaxEntries is how many entries the journal of a mock server keeps by default, the newest. DefaultMaxEntries = 10_000 // DefaultMaxBodySize is how many bytes of a body the journal of a mock server keeps by default. DefaultMaxBodySize = 1 << 20 )
Variables ¶
var ( // ErrNoJournal is the error of the methods of a nil [Journal], and of a [Verifier] of a server without a journal. ErrNoJournal = errors.New("journal: the server keeps no journal of requests: turn its journal on") // ErrClosed is the error of the methods of a [Journal] after [Journal.Close], but Dir and Close. ErrClosed = errors.New("journal: closed") // ErrInUse is the error of [New] when another journal, of this process or another, keeps its files in the // directory, where the system locks files. ErrInUse = errors.New("journal: another journal keeps its files in the directory") // ErrNotFound is the error of [Journal.Get] when the journal holds no entry with the id, such as one it dropped. ErrNotFound = errors.New("journal: request not found") // ErrBodyCut is the error of a query, or a pattern, that reads the body of an entry the journal kept part of, as // its MaxBodySize says: what was cut may hold what it looks for. It comes with [vlatte.ErrMatching]. ErrBodyCut = errors.New("journal: the journal kept part of the body") )
Functions ¶
Types ¶
type Draft ¶
type Draft struct {
// contains filtered or unexported fields
}
Draft is the entry of a request a mock server is serving, which the server fills in as the request goes and appends once the request ends. It keeps the errors of the request's on_complete events that fail, such as webhooks, for the entry to tell. Its methods are safe from many goroutines, as the events of a request run at the same time.
func (*Draft) Append ¶
Append runs end on the entry, which must not call the methods of d, and appends it to the journal, as Journal.Append does. The entry's Message tells the events that failed, one line each, unless the entry has a message of its own: the request failed, which its message tells. An error of many lines, such as a panic with its stack, is put on one. Only the first Append counts: another appends nothing and returns nil.
func (*Draft) EventFailed ¶
EventFailed keeps err, the error of an on_complete event of the request. An event that fails once the entry is appended is left out of the journal.
type Entry ¶
type Entry struct {
// Seq orders the entries of a journal: a later entry has a greater one. Append sets it.
Seq uint64
// ID is the request's, such as the one the VLatte-Request-Id header names.
ID string
// Time is when the request came.
Time time.Time
// Kind is the protocol of the request, such as "http".
Kind vlatte.Kind
Outcome Outcome
// MockID is the mock that matched the request, if one did.
MockID string
Method string
Path string
// Target is what the request asked for in full, such as its URL.
Target string
// Status is the status of the answer, such as an HTTP status or a gRPC code.
Status int
// Elapsed is how long the request took to serve.
Elapsed time.Duration
// Header holds the request's headers, or its metadata.
Header map[string][]string
// Body holds the request's body, up to the journal's limit: BodyTruncated tells it lost the rest.
Body []byte
BodyTruncated bool
// Message tells what failed: the request, for one that failed, or else its on_complete events that failed, such as
// webhooks, one line each.
Message string
// NearMisses are the mocks that came closest to matching a request no mock matched.
NearMisses []vlatte.NearMiss
// Scenarios are the moves of the scenarios of the mock that matched the request. A scenario moves as its mock is
// picked, so a request that then fails has them too.
Scenarios []matchscenario.Move
}
Entry is a request a mock server served, as its Journal keeps it. The fields every protocol has are its own; a protocol fills those it has: gRPC has no method, and its path is the full method of the call.
func (*Entry) HeaderValue ¶
HeaderValue returns the first value of the entry's header named name, in any case, or "" when it has none: the value a query's Header matcher gets. Of two spellings of a name, the one spelled as name comes first.
func (*Entry) MarshalJSONTo ¶
MarshalJSONTo writes e as its journal's files keep it.
type Filter ¶
Filter tells whether an entry of a journal is one a query wants, such as a protocol's request pattern does.
type Journal ¶
type Journal struct {
// contains filtered or unexported fields
}
Journal keeps the requests a mock server serves, of any protocol, in files: a log of entries, one per line, in segments the journal starts as one fills. It indexes them in memory, so a query reads from disk only the entries its indexed fields leave, and only when it selects by their headers or bodies, or returns them.
Append is synchronous: a request is in the journal once its answer is served, for a test to check right after. A journal is safe for concurrent use. Queries read without holding the journal: appends don't wait on them.
func New ¶
New opens the journal of o.Dir, with the entries it holds, or creates it. One journal at a time keeps its files in a directory: another one fails with ErrInUse, where the system locks files.
func (*Journal) Append ¶
Append adds e to the journal, and gives it its sequence number. A body over the journal's limit loses the rest, and text that isn't UTF-8, which JSON can't hold, gets U+FFFD in place of its bad bytes. A failure is logged, the first and then each time their count doubles, and returned.
func (*Journal) Changed ¶
func (j *Journal) Changed() <-chan struct{}
Changed returns a channel closed once the journal changes: an entry is appended, or the journal is cleared or closed. A caller waiting for an entry queries the journal again once it is closed, and calls Changed again to wait for the next change. A nil or closed journal returns a closed channel: a query tells why.
func (*Journal) Clear ¶
Clear removes every entry. Sequence numbers go on from the last one given while the journal is open: a journal opened on an empty directory starts them over. A file it can't remove yet, such as one a query holds open where the system keeps it, it logs and removes later: its entries are gone either way.
func (*Journal) Cleared ¶
Cleared returns the sequence number of the last entry before the journal's latest start, of which it holds none: the newest one when Journal.Clear last cleared it, or, for a journal that holds none of the entries of its directory before the oldest it loaded, the one before that. It is 0 for a journal that started empty and was never cleared, and for a nil journal.
func (*Journal) Close ¶
Close closes the journal, and removes its directory if it is a temporary one. Closing it again does nothing.
func (*Journal) Count ¶
Count returns how many entries q selects, with filters. It reads no entry when q selects by indexed fields only.
func (*Journal) Draft ¶
Draft starts the draft of entry, which Draft.Append appends to j. A nil j keeps no journal: Append returns ErrNoJournal.
func (*Journal) Find ¶
Find returns the entries q selects, with filters, the newest first, a page of them when q has a Limit.
Example (Pages) ¶
package main
import (
"context"
"fmt"
"github.com/caffeine-projects/vlatte/journal"
)
func main() {
j := journal.MustNew(journal.Options{})
defer j.Close()
for _, id := range []string{"a", "b", "c", "d", "e"} {
if err := j.Append(&journal.Entry{ID: id, Kind: "http", Method: "GET", Path: "/" + id}); err != nil {
panic(err)
}
}
// Set Before to a page's Next to get the next page; Next is 0 on the last one.
q := journal.Query{Limit: 2}
for {
page, err := j.Find(context.Background(), q)
if err != nil {
panic(err)
}
for _, e := range page.Entries {
fmt.Print(e.ID, " ")
}
fmt.Println("of", page.Count)
if page.Next == 0 {
break
}
q.Before = page.Next
}
}
Output: e d of 5 c b of 5 a of 5
func (*Journal) Last ¶
Last returns the sequence number of the newest entry appended, which the next entry's exceeds: a Query with it as After selects the entries appended since. It is 0 for a journal that never kept one, and for a nil journal.
Example ¶
package main
import (
"context"
"fmt"
"github.com/caffeine-projects/vlatte/journal"
"github.com/caffeine-projects/vlatte/match"
)
func main() {
j := journal.MustNew(journal.Options{})
defer j.Close()
// A request from before, such as another test's.
if err := j.Append(&journal.Entry{ID: "1", Kind: "http", Method: "GET", Path: "/health"}); err != nil {
panic(err)
}
mark := j.Last()
if err := j.Append(&journal.Entry{ID: "2", Kind: "http", Method: "GET", Path: "/health"}); err != nil {
panic(err)
}
// After selects only the entries appended since the mark.
n, err := j.Count(context.Background(), journal.Query{Path: match.Equal("/health"), After: mark})
if err != nil {
panic(err)
}
fmt.Println(n)
}
Output: 1
func (*Journal) Trimmed ¶
Trimmed returns the sequence number of the newest entry the journal dropped to keep MaxEntries since its latest start, as Journal.Cleared tells it, or 0 when it dropped none. A query of the entries after one up to it misses those it dropped. It is 0 for a nil journal.
type Options ¶
type Options struct {
// Dir is where the journal keeps its files, across restarts. Empty keeps them in a temporary directory, which
// Close removes.
Dir string
// MaxEntries is how many entries the journal keeps, the newest. 0 keeps them all.
MaxEntries int
// MaxBodySize is how many bytes of a body an entry keeps. 0 keeps whole bodies.
MaxBodySize int64
// Logger logs what the journal fails to keep. Nil logs nothing.
Logger *zerolog.Logger
}
Options configures a Journal.
type Outcome ¶
type Outcome int8
Outcome is what came of a request a mock server served.
const ( // OutcomeUnknown is an outcome that could not be parsed. OutcomeUnknown Outcome = iota // OutcomeMatched is a request a mock answered. OutcomeMatched // OutcomeUnmatched is a request no mock matched, which the server answered as such. OutcomeUnmatched // OutcomeProxied is a request another server answered: the host of a forward proxy, or a proxy's target. OutcomeProxied // OutcomeFailed is a request whose serving failed. OutcomeFailed // OutcomeAbandoned is a request whose client left before it got its answer. OutcomeAbandoned )
func ParseOutcome ¶
ParseOutcome parses an outcome's name in any case, returning OutcomeUnknown for an unrecognized name.
func (Outcome) MarshalText ¶
MarshalText encodes o as its name, keeping outcomes readable in journal files and API responses.
func (*Outcome) UnmarshalText ¶
UnmarshalText parses an outcome's name as ParseOutcome does.
type Page ¶
Page is a page of the entries a query selects, the newest first. Count is the number of entries the query selects, its Limit and Before left out. Next, when not 0, is the Before of the next page.
type Query ¶
type Query struct {
Kind vlatte.Kind
Outcome Outcome
MockID vlatte.Matcher
Method vlatte.Matcher
Path vlatte.Matcher
Target vlatte.Matcher
Status vlatte.Matcher
// Since and Until bound the time of the request: Since includes it, Until doesn't.
Since time.Time
Until time.Time
// Header matches headers by name, in any case: a matcher gets the first value of its header, or "" when the
// request has none.
Header map[string]vlatte.Matcher
// Body gets the body as text. A query the rest of which selects an entry whose body the journal cut fails with
// [vlatte.ErrMatching]: what was cut may hold what Body looks for.
Body vlatte.Matcher
// Limit is the most entries Find returns, the newest first; 0 returns them all. Count leaves it out.
Limit int
// Before selects the entries with a lower sequence number, those that follow a page Find returned, which its Next
// names. Count leaves it out.
Before uint64
// After selects the entries with a greater sequence number, those appended since the entry it names, such as the
// newest one [Journal.Last] returned before a test sent its requests. Count selects by it too.
After uint64
}
Query selects entries of a Journal. Each field set narrows the entries it selects; the zero query selects them all. A matcher gets the value of its field: text, or an int for Status.