Documentation
¶
Overview ¶
Package contentcheck finds deliverability problems in an email's content: size against Gmail's 102 KB clipping limit, images without alt text, the image-to-text ratio, links on URI blocklists, URL shorteners and broken links, a missing unsubscribe link or postal address, Liquid errors, spam heuristics in the subject, and the plain-text part.
The composer runs the fast checks inline as people edit; Send Readiness runs the full set (including link probing) before a send.
Index ¶
- Constants
- Variables
- func Blocking(id string) bool
- func BodyHeuristics(text, language string) []string
- func Minify(src string) string
- func PlainText(src string) string
- func PublicOnly(_, address string, _ syscall.RawConn) error
- func RegisterBlocking(ids ...string)
- func RegistrableDomain(host string) string
- func SaveAltLabel(l i18n.Lang) string
- func SectionContainer(doc *html.Node) *html.Node
- func SubjectHeuristicPhrase(subject, language string) string
- func SubjectHeuristics(subject string) []string
- func SubjectHeuristicsIn(subject, language string) []string
- func SuggestSubject(subject string) string
- func TextIn(l i18n.Lang, t i18n.Text) string
- func URILists() []dnscheck.Blocklist
- func WithProgress(ctx context.Context, p Progress) context.Context
- type Blocklist
- type BlocklistChecker
- type BlocklistReport
- type Checker
- type ContentCheckFix
- type ContentCheckImage
- type ContentCheckIssue
- type ContentCheckLink
- type ContentCheckReport
- type ContentCheckSection
- type ContentCheckUnknownDomain
- type Extra
- type HTTPProber
- type Input
- type LiveBlocklist
- type ProbeResult
- type Prober
- type Progress
- type StaticBlocklist
- type StoreBlocklist
Constants ¶
const ( TroubleTimeout = "timeout" // no answer in time TroubleRefused = "refused" // the list refused the query (public resolver, rate limit) TroubleOther = "other" // no usable answer )
Kinds of lookup trouble (BlocklistReport.Unknown).
const ( StageLinks = "links" // requesting each link StageLinkDomains = "link_domains" // looking link domains up on URI blocklists )
Stages a check reports progress on (WithProgress).
const ( Block = "block" Warn = "warn" Info = "info" Pass = "pass" )
Severities, in the order checks are listed.
const ( IssueLiquidSyntax = "personalization.syntax" IssueSubjectMissing = "subject.missing" IssueLinkBlocklist = "links.blocklist" )
The checks whose issues block a send.
const ClipBytes = 102 * 1024
ClipBytes is the size at which Gmail clips a message (102 KB).
ImageShareLimit is the share of the email's area images may cover before the content checks warn (one threshold for the composer and Send Readiness): image-heavy mail scores higher with spam filters.
const MaxProbedLinks = 100
MaxProbedLinks is how many links one run requests: a run is a background job (Send Readiness), so a long digest's links are all requested.
const NearClipBytes = 85 * 1024
NearClipBytes is where the size warning starts.
Variables ¶
var ErrBlockedAddress = errors.New("the link points at an internal address")
ErrBlockedAddress is returned when a link resolves to an internal address.
var URIListNames = []string{"Spamhaus DBL", "SURBL"}
URIListNames are the URI blocklists link domains are looked up on.
Functions ¶
func Blocking ¶
Blocking reports whether a check's issues block a send. Such an issue can't be dismissed: it is fixed or the email isn't sent.
func BodyHeuristics ¶
BodyHeuristics returns the spam heuristics an email's text trips: spam phrases repeated through the body (three or more urgency, money or click-bait phrases), and repeated exclamation marks. A word or two is normal; filters weigh the pile-up.
func Minify ¶
Minify shrinks email HTML without changing how it renders: whitespace between tags and inside style attributes, and comments other than Outlook's conditional comments.
func PlainText ¶
PlainText turns email HTML into the text/plain part: one block per line, headings in capitals, buttons and standalone links as "Label: URL", the footer after a "--" separator, hidden preview text and comments dropped.
func PublicOnly ¶
PublicOnly is a net.Dialer Control function that refuses non-public IPs (netguard's policy, private networks never allowed).
func RegisterBlocking ¶
func RegisterBlocking(ids ...string)
RegisterBlocking names checks another package adds whose issues block a send (a Sponsor block waiting for its creative), so they can't be dismissed either.
func RegistrableDomain ¶
RegistrableDomain returns the domain someone registers ("acme.io" for "news.acme.io").
func SaveAltLabel ¶
SaveAltLabel is the label of the fix that saves an image's alt text ("Save"), in a reader's language, for other checks that offer the same fix (a transactional template's accessibility rows).
func SectionContainer ¶
SectionContainer returns the element whose children are the email's top-level sections in MJML output (body > div > div…), or nil.
func SubjectHeuristicPhrase ¶
SubjectHeuristicPhrase returns the words of a subject that trip a phrase heuristic ("you won't believe"), "" when only its form does (capitals, punctuation).
func SubjectHeuristics ¶
SubjectHeuristics returns the spam heuristics a subject trips: "all caps", "repeated punctuation", "urgency", "money words".
func SubjectHeuristicsIn ¶
SubjectHeuristicsIn returns the heuristics a subject in a language trips: the English phrase lists (urgency, money words) apply to English and to emails without a language; other languages get the language-neutral ones (capitals, punctuation).
func SuggestSubject ¶
SuggestSubject rewrites a shouty subject so that it trips none of the heuristics: sentence case, repeated punctuation collapsed and urgency softened ("FLASH SALE!!! 48 HOURS ONLY" → "48-hour flash sale"). It returns "" when no such rewrite exists.
func TextIn ¶
TextIn writes a message this package kept in a reader's language. Its "section" argument is a section label (mjml.SectionLabel), named in that language too, and "lists" a list of names, joined the way the language joins them. Messages of other packages are written as i18n.Text.In writes them. It returns "" for a message no catalog has.
Types ¶
type Blocklist ¶
type Blocklist interface {
// Listings returns, for each listed domain, the lists it is on. Clean
// and unknown domains are absent.
Listings(ctx context.Context, domains []string) (map[string][]string, error)
}
Blocklist answers which URI blocklists list a domain.
type BlocklistChecker ¶
type BlocklistChecker interface {
Blocklist
Check(ctx context.Context, domains []string) (BlocklistReport, error)
}
BlocklistChecker is a Blocklist that also says which domains it couldn't check: a failed lookup is unknown, never clean.
type BlocklistReport ¶
type BlocklistReport struct {
// Listed maps a listed domain to the lists it is on.
Listed map[string][]string
// Unknown maps a domain that couldn't be checked to why (a Trouble*
// kind): DNS didn't answer, or the list refused the resolver.
Unknown map[string]string
// Lists names the lists the domains were looked up on.
Lists []string
// Domains is how many domains were checked (cached or live).
Domains int
}
BlocklistReport is what a live lookup of link domains found.
type Checker ¶
type ContentCheckFix ¶
type ContentCheckFix struct {
Kind string `json:"kind" enum:"replace_link,edit_link,insert_footer,set_alt,minify,remove_section,use_subject,open"`
Label string `json:"label"`
Value string `json:"value,omitempty"`
Path []int `json:"path,omitempty"`
// LabelText is Label as a message, to write it in a reader's language
// (ContentCheckReport.In).
LabelText i18n.Text `json:"-"`
}
ContentCheckFix is a one-click fix offered with an issue. The composer applies it to the canvas (Path points at the component) or to the campaign settings; an open fix opens the page in Value (a Sponsor block's booking).
type ContentCheckImage ¶
type ContentCheckImage struct {
URL string `json:"url"`
// Host is where the image loads from; empty for the media library's
// images (relative until the send makes them absolute) and for images
// whose address is personalized.
Host string `json:"host,omitempty"`
// Insecure is true for an image loaded over plain HTTP.
Insecure bool `json:"insecure,omitempty"`
Path []int `json:"path"`
// Probed is true when the image was requested (Send Readiness); Status,
// Broken and Bytes (the size the server reported, 0 when it didn't)
// describe the response.
Probed bool `json:"probed,omitempty"`
Status int `json:"status,omitempty"`
Broken bool `json:"broken,omitempty"`
Bytes int64 `json:"bytes,omitempty"`
}
ContentCheckImage is one image of the email.
type ContentCheckIssue ¶
type ContentCheckIssue struct {
ID string `json:"id" example:"links.blocklist"`
Severity string `json:"severity" enum:"block,warn,info,pass"`
Group string `json:"group" enum:"Content,Links,Compliance,Subject,Personalization"`
Title string `json:"title"`
Detail string `json:"detail,omitempty"`
// Tag is the short label shown on the canvas next to the element.
Tag string `json:"tag,omitempty"`
// Path locates the element from mj-body (element indices, comments
// excluded); empty when the issue concerns the whole email.
Path []int `json:"path"`
// Section is the top-level section index, -1 for the whole email.
Section int `json:"section"`
Fixes []ContentCheckFix `json:"fixes"`
// URL is the link a link issue is about (to find it inside text).
URL string `json:"url,omitempty"`
// TitleText, DetailText (one message a sentence) and TagText are the
// issue's words as messages, to write them in a reader's language
// (ContentCheckReport.In, TextIn). Title, Detail and Tag are English.
// Issues another domain adds without them keep their English.
TitleText i18n.Text `json:"-"`
DetailText []i18n.Text `json:"-"`
TagText i18n.Text `json:"-"`
}
ContentCheckIssue is one finding.
func (ContentCheckIssue) In ¶
func (is ContentCheckIssue) In(l i18n.Lang) ContentCheckIssue
In writes one issue's words in a reader's language.
type ContentCheckLink ¶
type ContentCheckLink struct {
URL string `json:"url"`
Host string `json:"host"`
Text string `json:"text"`
Kind string `json:"kind" enum:"button,link,image,nav,social"`
Path []int `json:"path"`
// Probed is true when the link was requested; Status, Broken, Millis
// and Final (the address after redirects) describe the response.
Probed bool `json:"probed"`
Status int `json:"status,omitempty"`
Broken bool `json:"broken,omitempty"`
Millis int `json:"millis,omitempty"`
Final string `json:"final,omitempty"`
// FinalClean is true when Final, the address the link really leads to,
// was looked up on the URI blocklists and found on none: only then is a
// listed or shortened link offered "Replace with destination URL".
FinalClean bool `json:"finalClean,omitempty"`
// BotBlocked is true when the site refused the automated request.
BotBlocked bool `json:"botBlocked,omitempty"`
}
ContentCheckLink is one link of the email.
type ContentCheckReport ¶
type ContentCheckReport struct {
Verdict string `json:"verdict" enum:"ready,warnings,blocked"`
Blocks int `json:"blocks"`
Warnings int `json:"warnings"`
SizeBytes int `json:"sizeBytes"`
ClipBytes int `json:"clipBytes"`
Images int `json:"images"`
ImagesMissingAlt int `json:"imagesMissingAlt"`
Links int `json:"links"`
LinksBlocklisted int `json:"linksBlocklisted"`
LinksBroken int `json:"linksBroken"`
// SubjectHeuristics, PreheaderHeuristics and BodyHeuristics name the
// spam heuristics the subject, the preheader and the body's text trip.
SubjectHeuristics []string `json:"subjectHeuristics"`
PreheaderHeuristics []string `json:"preheaderHeuristics"`
BodyHeuristics []string `json:"bodyHeuristics"`
Issues []ContentCheckIssue `json:"issues"`
Sections []ContentCheckSection `json:"sections"`
PlainText string `json:"plainText"`
CheckedAt time.Time `json:"checkedAt"`
// ImageShare is the estimated share of the email's area covered by
// images (0 to 1; 0 without images). Words counts the plain text.
Words int `json:"words"`
// ImagesInsecure counts images loaded over plain HTTP; ImageList is
// every image with, when images were requested, the response.
ImagesInsecure int `json:"imagesInsecure"`
ImageList []ContentCheckImage `json:"imageList"`
// LinkList is every link with its text and, when links were probed, the
// response (Send Readiness shows the slowest and the link domains).
LinkList []ContentCheckLink `json:"linkList"`
// LinkDomains counts the link domains looked up on the URI blocklists
// (URILists names them); LinkDomainsUnknown are those whose lookup
// failed: unknown, not clean.
LinkDomains int `json:"linkDomains"`
URILists []string `json:"uriLists"`
LinkDomainsUnknown []ContentCheckUnknownDomain `json:"linkDomainsUnknown"`
}
ContentCheckReport is the outcome of a check run.
func (*ContentCheckReport) AddIssues ¶
func (r *ContentCheckReport) AddIssues(issues ...ContentCheckIssue)
AddIssues adds findings of other domains (the Sponsor block's) to a report: they count in its verdict and in their sections.
func (ContentCheckReport) In ¶
func (r ContentCheckReport) In(l i18n.Lang) ContentCheckReport
In writes a report's words in a reader's language: its issues' titles, details, canvas tags and fix labels. Check writes English, and so does In for English readers; issues another domain added without their messages keep their English.
type ContentCheckSection ¶
type ContentCheckSection struct {
Index int `json:"index"`
Label string `json:"label"`
Images int `json:"images"`
ImagesWithAlt int `json:"imagesWithAlt"`
Links int `json:"links"`
LinksOK int `json:"linksOk"`
Issues int `json:"issues"`
}
ContentCheckSection is what the inspector shows for the selected section.
type ContentCheckUnknownDomain ¶
type ContentCheckUnknownDomain struct {
Domain string `json:"domain"`
// Trouble is why: timeout, refused (the list refused the resolver) or other.
Trouble string `json:"trouble" enum:"timeout,refused,other"`
}
ContentCheckUnknownDomain is a link domain that couldn't be looked up.
type Extra ¶
type Extra func(ctx context.Context, in Input) []ContentCheckIssue
Extra is a check another domain adds (X20: a store block naming a product the store removed). It returns its issues; Section and Path say where they are.
type HTTPProber ¶
type HTTPProber struct {
Client *http.Client
// MaxLinks caps how many links one run requests.
MaxLinks int
// Concurrency is how many requests run at once.
Concurrency int
// contains filtered or unexported fields
}
HTTPProber requests links with HEAD (GET when HEAD isn't allowed), with short timeouts and an SSRF guard: it never connects to loopback, private, link-local or otherwise internal addresses, also after redirects.
func NewHTTPProber ¶
func NewHTTPProber() *HTTPProber
NewHTTPProber returns a prober with the SSRF guard on its dialer.
func (*HTTPProber) Probe ¶
func (p *HTTPProber) Probe(ctx context.Context, urls []string) map[string]ProbeResult
Probe implements Prober. Results are cached for ten minutes.
type Input ¶
type Input struct {
Subject string
Preheader string
MJML string
HTML string
// Address is the postal address from the workspace's branding, if set.
Address string
// Fields are the personalization tokens that exist (liquid.Catalog keys).
Fields []string
// ProbeLinks requests each link (Send Readiness); the composer skips it.
ProbeLinks bool
// Language is the language version's code ("de"); the subject's
// English phrase heuristics only apply to English (or no language).
Language string
// WorkspaceID is whose content it is (Extra checks read its data).
WorkspaceID uuid.UUID
}
Input is the content to check.
type LiveBlocklist ¶
type LiveBlocklist struct {
Q store.Querier
DNS dnscheck.Resolver
// Lists default to URILists.
Lists []dnscheck.Blocklist
// MaxAge is how long an answer is reused (default an hour).
MaxAge time.Duration
// Budget bounds the live lookups of one check (default 30 s): the
// composer, which checks as people type, uses a short one.
Budget time.Duration
Now func() time.Time
}
LiveBlocklist looks link domains up on Spamhaus DBL and SURBL over DNS and keeps the answers in link_domain_listings, shared by every workspace: an answer younger than MaxAge is reused, an older one is looked up again. A lookup that fails is unknown and isn't kept. The composer's inline checks and Send Readiness both use it.
func (*LiveBlocklist) Check ¶
func (b *LiveBlocklist) Check(ctx context.Context, domains []string) (BlocklistReport, error)
Check implements BlocklistChecker.
type ProbeResult ¶
type ProbeResult struct {
Status int `json:"status"`
Broken bool `json:"broken"`
// BotBlocked is true when the site refused an automated request (999,
// 429): whether the link works isn't known.
BotBlocked bool `json:"botBlocked,omitempty"`
Error string `json:"error,omitempty"`
// Bytes is the size the server reported (Content-Length), 0 when it
// didn't.
Bytes int64 `json:"bytes,omitempty"`
// Millis is how long the response took; Final is the address after
// redirects (empty when the link didn't redirect).
Millis int `json:"millis,omitempty"`
Final string `json:"final,omitempty"`
}
ProbeResult is the outcome of requesting one link.
type Prober ¶
type Prober interface {
Probe(ctx context.Context, urls []string) map[string]ProbeResult
}
Prober requests links to find broken ones.
type Progress ¶
Progress is told how far a stage of a check got: done of total.
func ProgressFrom ¶
ProgressFrom is the progress observer of ctx, or one that ignores it.
type StaticBlocklist ¶
StaticBlocklist is a fixed map, for tests and installs without lookups.
type StoreBlocklist ¶
StoreBlocklist reads URI blocklist results cached in link_domain_listings. Deliverability lookups (DNS, HostRepute) write that table; content checks only read it, so they stay fast.