Documentation
¶
Overview ¶
Package threadload reads a thread — its entries and, when asked, each entry's message — within limits it states up front, and says exactly what it got.
HEY serves a thread's entries newest first, a page at a time, and keeps each body on the message rather than the entry, so a full read is one page walk and then one request per entry. Every one of those is a request a server of somebody else's choosing answers, so each has a cap: on pages, on entries, on message requests, on bytes kept, on time. Nothing here decides what to do when a cap is hit; the result records it per entry and the caller decides, which is what keeps `hey thread read --count` from reading bodies it will not show and `hey attachment list` reading bodies it cannot do without.
Index ¶
Constants ¶
This section is empty.
Variables ¶
var DefaultLimits = Limits{ MaxPages: 101, MaxEntries: 2000, MaxMessageRequests: 2000, MaxRetries: 1, Concurrency: 8, MaxRetainedBytes: 64 << 20, Deadline: 2 * time.Minute, }
DefaultLimits are what the CLI reads with: a hundred pages past the first, two thousand entries and as many bodies, two tries per body on top of the SDK's own retries, eight at a time, 64 MiB of content kept, two minutes in all.
var ErrOverLimit = errors.New("the message was too large to read")
ErrOverLimit marks an error a Source returns for one message that was too large to read — a response past the transport's cap. It is not retried, since it would be too large again, and not a failed body: the entry is over_limit, like one the byte budget refused.
var ErrSystemic = errors.New("the service is refusing requests")
ErrSystemic marks an error a Source returns that is about the client rather than the one request — a rate limit, an expired credential. A message request that fails with it is not retried and not recorded as one failed body: the whole load stops and returns the error, since every request that follows would meet the same answer.
Functions ¶
This section is empty.
Types ¶
type Entry ¶
type Entry struct {
Entry generated.Entry
Message *generated.Message
State State
// Err is why a failed entry failed.
Err error
}
Entry is one entry of the thread with whatever was read for it.
type Limits ¶
type Limits struct {
// MaxPages is how many pages of the index are read, counting the first.
MaxPages int
// MaxEntries is how many entries are admitted, newest first.
MaxEntries int
// MaxMessageRequests is how many messages are requested, newest first; entries
// past it are over_limit.
MaxMessageRequests int
// MaxRetries is how many more times a failed message request is tried.
MaxRetries int
// Concurrency is how many message requests are in flight at once.
Concurrency int
// MaxRetainedBytes is how much is kept in total — the index entries and then the
// message content; an entry whose content would exceed it is over_limit, and an
// index that would exceed it is truncated.
MaxRetainedBytes int64
// Deadline is how long the whole load may take; entries not yet requested when it
// passes are over_limit.
Deadline time.Duration
}
Limits bounds one load. Each is a hard number rather than a knob: the defaults are what the CLI runs with, and a caller that needs less can lower them.
type Page ¶
Page is one page of the entries index, newest first, and the cursor for the next one — empty on the last page.
type Request ¶
type Request struct {
TopicID int64
// Hydrate asks for each entry's message. Without it every entry is not_requested,
// which is what a count or a list of IDs needs.
Hydrate bool
Limits Limits
}
Request is one thread to read, and whether its bodies are wanted.
type Source ¶
type Source interface {
EntriesPage(ctx context.Context, topicID int64, cursor string) (Page, error)
Message(ctx context.Context, entryID int64) (*generated.Message, error)
}
Source is what a thread is read through: the entries index and each entry's message. The SDK satisfies it through a small adapter; tests satisfy it directly.
func NewSDKSource ¶
NewSDKSource reads a thread through the SDK: the paginated entries index, which carries the cursor `Topics().GetEntries` throws away, and each entry's message, which is where HEY keeps the body. It is the Source both the CLI and the TUI hand Load.
type State ¶
type State string
State is what became of one entry's body.
const ( // StateHydrated: the message was read and carries content. StateHydrated State = "hydrated" // StateBodyless: the message was read and HEY served no content for it. StateBodyless State = "bodyless" // StateNotRequested: the caller did not ask for bodies. StateNotRequested State = "not_requested" // StateOverLimit: a limit — requests, bytes, time — was reached before this entry's // message was read. StateOverLimit State = "over_limit" // StateFailed: the message request failed, retries included. StateFailed State = "failed" )
type Thread ¶
type Thread struct {
Entries []Entry
// IndexTruncated reports that the index was not read to its end — more pages than
// MaxPages, more entries than MaxEntries, or the Deadline passed — so entries older
// than the last one here exist unseen. IndexTruncatedBy names which.
IndexTruncated bool
IndexTruncatedBy Truncation
// Omitted counts the entries whose body was wanted and not read: over_limit or
// failed.
Omitted int
}
Thread is what a load produced: the entries admitted, oldest first, and an account of what is missing.
func Load ¶
Load reads one thread. An error is a thread that could not be read at all — the first page failed, or the index failed part way, since an index with a hole in it is not a thread — or a caller that stopped waiting: its context ending is its decision, not a limit, and is returned as the error it is. A body that could not be read within the limits is an entry in the failed or over_limit state, not an error.
type Truncation ¶
type Truncation string
Truncation is what stopped the index walk short of its end.
const ( TruncatedByPages Truncation = "pages" TruncatedByEntries Truncation = "entries" TruncatedByBytes Truncation = "bytes" TruncatedByDeadline Truncation = "deadline" )
Source Files