Documentation
¶
Overview ¶
Package recommend serves the recommendation models the hermod-ml worker trains (ml/worker/hermod_ml/recommend.py) in-process: it parses a version's artifact once and answers "which items next for this person" and "which items are like this one" from it, without a call to the worker.
The artifact is little-endian throughout:
"HREC" 4 bytes, magic
format version uint32, FormatVersion
header length H uint32
header H bytes of JSON: algorithm, users, items, factors,
neighbors, seen, popular
user factors float32 [users x factors] when factors > 0
item factors float32 [items x factors] when factors > 0
neighbor items uint32 [items x neighbors] when neighbors > 0
neighbor scores float32 [items x neighbors] when neighbors > 0
seen offsets uint32 [users + 1]
seen items uint32 [seen], ascending per person
popular items uint32 [popular], most popular first
popular scores float32 [popular]
Format 2 is a recommendation scenario's (ml/worker/hermod_ml/scenarios.py): the same sections, then what serving the scenario needs. Its header adds scenario, as_of, period_days, customer_neighbors, the names of the customer and item features, customer_attrs and item_attrs (one value or null per customer or item), salesmen, weights, rules, global_gap, global_visit_gap, has_visits and has_margin, and, since customer categories and sizes, has_peers and size_edges; since products' details, new_days and pair_details. Its algorithm may be "none", with neither factors nor neighbors, for a scenario that needs no products. After popular scores come:
customer neighbors uint32 [users x customer_neighbors] when > 0
customer nbr scores float32 [users x customer_neighbors] when > 0
seen last day uint32 [seen], days since 1970-01-01, or empty
seen gap float32 [seen], median days between orders, or NaN
customer features float32 [users x customer features], NaN unknown
item features float32 [items x item features], NaN unknown
seen quantity float32 [seen], median quantity per order, or NaN
(when pair_details)
seen spend float32 [seen], what was spent on the pair
(when pair_details)
An artifact comes from the worker's model store, so it is parsed as input that may be wrong: every count is checked against the bytes that back it before anything is allocated for it, every index against what it indexes, and every float for being finite. A bad artifact is refused with ErrInvalid; nothing in it can make Parse or a later call panic or allocate more than the artifact's own size allows.
Index ¶
- Constants
- Variables
- func CheckTightens(defaults, request []Rule) error
- func ClampK(k int) int
- func ID(v any) (string, bool)
- func ValidateRules(scenario string, rules []Rule) error
- type DueProduct
- type Item
- type Model
- func (m *Model) Algorithm() string
- func (m *Model) Answer(q Query) ([]Result, error)
- func (m *Model) AsOf() time.Time
- func (m *Model) CustomerAttrs(id string) (segment, size string)
- func (m *Model) CustomerIDs() []string
- func (m *Model) DefaultRules() []Rule
- func (m *Model) ForUser(user string, k int, excludeSeen bool) (items []Item, known bool)
- func (m *Model) Items() int
- func (m *Model) PeriodDays() int
- func (m *Model) Popular(k int) []Item
- func (m *Model) ProductAttrs(id string) (brand, subcategory string)
- func (m *Model) ProductIDs() []string
- func (m *Model) Salesmen() []string
- func (m *Model) Scenario() string
- func (m *Model) Similar(item string, k int) (items []Item, known bool)
- func (m *Model) SizeBucket(size string) string
- func (m *Model) SizeEdges() (lo, hi float64, ok bool)
- func (m *Model) Users() int
- type Pick
- type PlanDay
- type Query
- type Reason
- type Result
- type Rule
- type SalesmanInfo
Constants ¶
const ( FormatVersion = 1 FormatVersionScenario = 2 MaxArtifactBytes = 128 << 20 MaxFactors = 1024 MaxNeighbors = 1024 MaxPopular = 1000 MaxCustomerNeighbors = 1024 MaxFeatures = 64 )
Limits of an artifact. MaxArtifactBytes matches the worker's own limit: a scenario's artifact carries per-customer features and neighbours besides the model, so it is 128 MiB, twice format 1's original 64.
const ( AlgorithmALS = "als" AlgorithmItemSimilarity = "item_similarity" AlgorithmNone = "none" )
The algorithms an artifact may hold. AlgorithmNone is a format 2 scenario's that needs no products (a visit plan or reorder dates without a product column).
const ( NewDaysMin = 1 NewDaysMax = 365 MaxStock = MaxBatch * 10 )
NewDaysMin and NewDaysMax bound a model's new_days, as the worker's NEW_DAYS_BOUNDS; MaxStock bounds the stock a request may give.
const ( ReasonBrand = "brand" ReasonNewProduct = "new_product" ReasonSubstitute = "substitute" ReasonSameKind = "same_kind" )
The reason codes of products' details.
const ( DefaultK = 10 MaxK = 100 )
How many items a call returns: DefaultK when it does not say, at most MaxK.
const ( RuleFilter = "filter" RuleExclude = "exclude" RuleBoost = "boost" RuleDiversity = "diversity" RuleCapacity = "capacity" RulePin = "pin" RuleBlock = "block" )
The kinds of Rule.
const ( OpEq = "eq" OpIn = "in" OpNeq = "neq" OpNin = "nin" OpGt = "gt" OpLt = "lt" )
The comparisons a filter or boost makes.
const ( FromSalesman = "salesman" FromTerritory = "territory" )
The request fields a rule's value may be bound to.
const ( MaxRules = 50 MaxRuleIDs = 1000 MaxRuleValues = 1000 MaxExcludeDays = 3650 MaxPerValue = 100 )
Bounds of a rule.
const ( // MaxPlan bounds a visit plan: its items, and visits per day. MaxPlan = 500 // MaxBatch bounds the customers one call may ask for. MaxBatch = 1000 // DefaultVisitsPerDay is a salesman's capacity when nothing says. DefaultVisitsPerDay = 8 // MaxWithinDays bounds a reorder window, as a visit plan's period. MaxWithinDays = 366 )
Bounds of a scenario call.
const ( NearKmMin = 0.1 NearKmMax = 500 NearPeersMin = peerMin NearPeersMax = 200 )
NearKmMin, NearKmMax, NearPeersMin and NearPeersMax bound a model's nearby peers (the training's near_km and near_peers), as the worker's NEAR_KM_BOUNDS and NEAR_PEERS_BOUNDS.
const ( ReasonReorderDue = "reorder_due" ReasonChurnRisk = "churn_risk" ReasonHighValue = "high_value" ReasonWhitespace = "whitespace" ReasonNotVisited = "not_visited" ReasonSimilarCustomers = "similar_customers" ReasonPeerGroup = "peer_group" ReasonBoughtTogether = "bought_together" ReasonBoosted = "boosted" ReasonPinned = "pinned" )
The reason codes of a Pick.
const ( ScenarioVisits = "visits" ScenarioProducts = "products" ScenarioReorder = "reorder" ScenarioAudience = "audience" ScenarioSimilar = "similar" )
The scenarios a format 2 artifact answers. A format 1 artifact answers ScenarioProducts and ScenarioSimilar.
const ( SignalReorderDue = "reorder_due" SignalChurnRisk = "churn_risk" SignalValue = "value" SignalOpportunity = "opportunity" SignalNeglect = "neglect" )
The visit signals, which the weights name.
const AllCustomers = "all_customers"
AllCustomers is the For of a reorder answer for every customer.
const ReasonPopular = "popular"
ReasonPopular marks an item recommended because many people have it, not because of who is asking: for a person or item the model does not know, or to fill a list the model's own scores leave short.
Variables ¶
var DefaultWeights = map[string]float64{ SignalReorderDue: 0.35, SignalChurnRisk: 0.2, SignalValue: 0.2, SignalOpportunity: 0.1, SignalNeglect: 0.15, }
DefaultWeights are a visit plan's weights when its training names none.
var ErrBadQuery = errors.New("invalid recommendation request")
ErrBadQuery marks a scenario call that is wrong in itself or for the model's scenario.
var ErrBadRule = errors.New("invalid rule")
ErrBadRule marks a rule that is wrong in itself, or for its scenario.
var ErrInvalid = errors.New("not a valid recommendation artifact")
ErrInvalid is returned by Parse for bytes that are not a well-formed recommendation artifact. The error says what is wrong.
var ErrLoosens = errors.New("a request rule may only tighten the model's rules")
ErrLoosens marks a request rule that would loosen a rule the model was trained with. A request may only tighten them.
var Scenarios = []string{ScenarioVisits, ScenarioProducts, ScenarioReorder, ScenarioAudience, ScenarioSimilar}
Scenarios lists every scenario, in the order the worker documents them.
Functions ¶
func CheckTightens ¶
CheckTightens refuses request rules that would loosen the defaults, as far as that can be told from the rules alone: a higher capacity or working days the defaults do not allow, a higher diversity cap on the same field, or a pin of what a default blocks. A pin of what a default filter or exclude drops is refused when the call is answered, where the items are known. Filters, excludes, blocks and boosts are added to the defaults, so they can only narrow or reorder what the defaults allow.
func ID ¶
ID is the text a person or item id is matched on, as the worker turned the column's values into text (hermod_ml.datasets.to_text): a string as it is, true or false, and a whole number without ".0". It reports false for anything that cannot be an id: nothing, an empty string, an object or list, or a number that is not finite.
func ValidateRules ¶
ValidateRules reports the first rule that is wrong in itself or for the scenario, and refuses more than MaxRules.
Types ¶
type DueProduct ¶ added in v1.18.3
type DueProduct struct {
ID string `json:"id"`
DueDate string `json:"due_date"`
Overdue bool `json:"overdue,omitempty"`
Fields map[string]any `json:"fields,omitempty"`
}
DueProduct is a product a customer is due to buy again: when, from the customer's own gap between buying it, and whether that day has passed.
type Item ¶
type Item struct {
Item string `json:"item"`
Score float64 `json:"score"`
Reason string `json:"reason,omitempty"`
}
Item is one recommended item. Score is the model's: a dot product for als, a sum of similarities for item_similarity, a cosine similarity for similar items, and the share of the most popular item's count for a popular one.
type Model ¶
type Model struct {
// contains filtered or unexported fields
}
Model is a parsed artifact. It is immutable, so one Model serves every caller at once.
func (*Model) Answer ¶
Answer answers a call of the model's scenario: one Result per subject.
- visits: for a salesman, their customers to visit, ranked and planned across the working days of the period.
- products: for a customer (or a batch of customers), products to offer; for a product, products like it.
- reorder: for a customer, a batch of them, a salesman's customers or every customer, when each will order again (DueDate), most overdue first; for a product, its repeat buyers by when they will buy it again. With WithinDays, only those due before the window ends, overdue ones included, most likely to order first.
- audience: for a product, the customers likely to want it.
- similar: for a product, products like it.
func (*Model) AsOf ¶
AsOf is the day of the latest transaction the model learned from; zero for a format 1 model.
func (*Model) CustomerAttrs ¶ added in v1.18.5
CustomerAttrs is a customer's category (segment) and size; "" for what the model does not know, or a customer it does not.
func (*Model) CustomerIDs ¶
Customer and product ids, for building every result of a model.
func (*Model) DefaultRules ¶
DefaultRules are the rules trained with the model, which every call obeys.
func (*Model) ForUser ¶
ForUser is a person's top k items, best first, leaving out what they already have when excludeSeen is set. A person the model does not know gets the popular items, and known reports false. A list the model's own scores leave short is filled with popular items the person does not have.
func (*Model) PeriodDays ¶
PeriodDays is the model's planning or evaluation period in days; 7 for a format 1 model.
func (*Model) ProductAttrs ¶ added in v1.18.6
ProductAttrs is a product's brand and subcategory; "" for what the model does not know, or a product it does not.
func (*Model) ProductIDs ¶
func (*Model) Salesmen ¶
Salesmen lists the salesmen the model knows, from its salesmen table and its customers' salesman column, sorted.
func (*Model) Scenario ¶
Scenario is the question the model answers: one of Scenarios. A format 1 model answers ScenarioProducts (and similar items).
func (*Model) Similar ¶
Similar is the k items most like item, best first. An item the model does not know gets the popular items, and known reports false; a list its neighbors leave short is filled with popular items.
func (*Model) SizeBucket ¶ added in v1.18.5
SizeBucket is a customer size as the model stores it: small, medium or large in lower case; a number cut at SizeEdges as training cut the customers' sizes (up to the first edge small, up to the second medium, above it large); any other label as given, trimmed.
type Pick ¶
type Pick struct {
ID string `json:"id"`
Score float64 `json:"score"`
Rank int `json:"rank"`
Reasons []Reason `json:"reasons"`
Fields map[string]any `json:"fields"`
DueDate string `json:"due_date,omitempty"`
Overdue bool `json:"overdue,omitempty"`
Products []DueProduct `json:"products,omitempty"`
}
Pick is one result: a customer or a product, its score (higher is stronger; scores compare within one list only), its rank from 1, why, and what the model knows of it. DueDate is set by the reorder scenario, and Overdue when that day has passed with no order since. Products are the products a reorder customer is due to buy again: those due within the window, or the one product asked about.
type Query ¶
type Query struct {
Customer, Product, Salesman, Territory string
Customers []string
AllCustomers bool
From, To time.Time
Today time.Time
K int
Rules []Rule
ExcludeSeen bool
WithinDays int
Segment, Size string
Brand, Subcategory string
Stock map[string]float64
}
Query is one scenario call. Exactly one subject is set: Customer, Customers, Product or Salesman, as the scenario allows. Territory, when set, is what a rule's from_request "territory" binds to; otherwise the territory most of the salesman's customers are in. From and To bound a visit plan (both days included); zero means the model's period from Today. K is how many results, 0 for the default. Rules are the request's, added to the model's own. AllCustomers asks a reorder model about every customer. WithinDays, for reorder, keeps the customers due to order before Today plus that many days, overdue ones included, ranked by how likely they are to order; 0 is no window. Segment and Size, for a list of customers (visits, reorder, audience), keep only the customers of that category and size, either matched in any case; a numeric Size is bucketed as the model's training cut its customers' sizes (SizeBucket). Brand and Subcategory, for a list of products (products, similar), keep only the products of that brand and subcategory, matched in any case. Stock is stock on hand by product id, in place of the model's (as of its training): a product with none is not offered.
type Result ¶
type Result struct {
For string `json:"for"`
Items []Pick `json:"items"`
Plan []PlanDay `json:"plan,omitempty"`
Summary string `json:"summary,omitempty"`
}
Result answers one subject of a call: For is the customer, product or salesman asked about. Summary, for a customer's products, says in a line what the customer's nearby peers buy, such as "Minimarket customers within 5 km buy: A, B, C"; empty otherwise.
type Rule ¶
type Rule struct {
Kind string `json:"kind"`
Field string `json:"field,omitempty"`
Op string `json:"op,omitempty"`
Value any `json:"value,omitempty"`
FromRequest string `json:"from_request,omitempty"`
Days int `json:"days,omitempty"`
Weight float64 `json:"weight,omitempty"`
MaxPer int `json:"max_per,omitempty"`
PerDay int `json:"per_day,omitempty"`
IDs []string `json:"ids,omitempty"`
}
Rule is one business rule, applied when serving: on the model as a default for every call, or on one request, where it is added to the defaults and may only tighten them.
- filter keeps only what matches Field Op Value.
- exclude drops what was interacted with (bought; visited, for a visit plan) in the last Days days, or ever when Days is 0.
- boost adds Weight x the normalised Field to the score, or a flat Weight to what matches Field Op Value or is listed in IDs.
- diversity keeps at most MaxPer results per value of Field.
- capacity (visits only) plans at most PerDay visits a day, on the weekdays Value lists ("mon", ...) when it lists any.
- pin always includes IDs; block never does.
Field is "customers.<name>" or "products.<name>": an attribute the training mapped (salesman, territory, segment, blocked, name, category, active) or a feature (monetary, price, margin, ...), or id. FromRequest binds Value to the request's salesman or territory instead.
type SalesmanInfo ¶
type SalesmanInfo struct {
VisitsPerDay *float64 `json:"visits_per_day"`
WorkingDays []string `json:"working_days"`
}
SalesmanInfo is a salesman's capacity as the salesmen table gave it; a nil or empty field was not given.
Source Files