imports

package
v1.0.1 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 24, 2026 License: AGPL-3.0 Imports: 41 Imported by: 0

Documentation

Overview

Package imports is the migration importer (spec 5.8, 7.11): CSV and XLSX uploads and API importers for Brevo, Mailchimp, MailerLite, listmonk and Keila. A River job stages the rows; after field and consent mapping and a dry run, a second job imports them in batches. Unsubscribed, bounced and complained contacts become suppressions and are never re-subscribed.

Index

Constants

View Source
const (
	HandleConfirmed         = "confirmed"
	HandlePending           = "pending"
	HandleSkip              = "skip"
	HandleSuppressUnsub     = "suppress_unsubscribe"
	HandleSuppressBounce    = "suppress_bounce"
	HandleSuppressComplaint = "suppress_complaint"
)

Consent handling of each source status.

View Source
const (
	TargetEmail    = "email"
	TargetSkip     = "skip"
	TargetEvidence = "consent_evidence"
	TargetStatus   = "status"
	TargetTags     = "tags"
	TargetName     = "name"
)

Field targets a source field can map to.

View Source
const (
	StatusSubscribed    = "subscribed"
	StatusUnsubscribed  = "unsubscribed"
	StatusCleaned       = "cleaned"
	StatusPending       = "pending"
	StatusTransactional = "transactional"
	StatusComplained    = "complained"
)

Source statuses, normalized.

View Source
const (
	CodeAuthFailed  = "source_auth_failed"
	CodeRateLimited = "source_rate_limited"
	CodeUnavailable = "source_unavailable"
	CodeBadResponse = "source_bad_response"
)

Error codes of SourceError.

View Source
const BatchSize = 500

BatchSize is how many rows one batch imports.

View Source
const FileListID = "file"

FileListID is the pseudo-list of a file import.

View Source
const KeilaAllContacts = "all"

KeilaAllContacts is the pseudo-list id of a Keila import.

View Source
const MaxFileRows = 500000

MaxFileRows bounds an uploaded file ("Up to 500,000 rows").

View Source
const ProblemNoMX = "no MX record"

ProblemNoMX is the invalid reason for domains that can't receive mail.

Variables

View Source
var ErrNoKey = errors.New("paste an API key first")

ErrNoKey is returned when an API importer has no credentials.

View Source
var Sources = []string{"brevo", "mailchimp", "mailerlite", "listmonk", "keila", "csv"}

Sources the importer supports, in the order of the source grid.

Functions

func NormalizeStatus

func NormalizeStatus(v string) string

NormalizeStatus maps a status column's value to a source status.

func Register

func Register(h huma.API, s *Service)

Register adds the importer operations.

func RegisterJobs

func RegisterJobs(reg *jobs.Registry, s *Service)

RegisterJobs adds the staging and import workers.

func SourceName

func SourceName(src string) string

SourceName is the display name of a source.

Types

type Brevo

type Brevo struct {
	// contains filtered or unexported fields
}

Brevo reads lists, contact attributes and the blocklist through the Brevo API v3 (read-only).

func NewBrevo

func NewBrevo(apiKey, base string, hc *http.Client) (*Brevo, error)

NewBrevo builds a client from an API key; base overrides the API URL.

func (*Brevo) Connect

func (b *Brevo) Connect(ctx context.Context) (ImportConnection, error)

Connect implements Source.

func (*Brevo) Fetch

func (b *Brevo) Fetch(ctx context.Context, lists []string, emit func(Row) error) error

Fetch implements Source. The blocklist says why an address is blocked: hard bounces become cleaned, spam reports complained, the rest unsubscribed.

func (*Brevo) Fields

func (b *Brevo) Fields(ctx context.Context, _ []string) ([]SourceField, error)

Fields implements Source: the email plus normal contact attributes.

type Engagement

type Engagement struct {
	LastOpen  *time.Time `json:"last_open,omitempty"`
	LastClick *time.Time `json:"last_click,omitempty"`
	Changed   *time.Time `json:"changed,omitempty"`
	SignupAt  *time.Time `json:"signup_at,omitempty"`
	OptinAt   *time.Time `json:"optin_at,omitempty"`
	OptinIP   string     `json:"optin_ip,omitempty"`
	// Rating is Mailchimp's member rating (1–5).
	Rating int `json:"rating,omitempty"`
}

Engagement is what the source knows about a contact's activity and consent.

type FieldMap

type FieldMap struct {
	Fields []ImportFieldChoice `json:"fields"`
}

FieldMap is imports.field_map.

type FileError

type FileError struct {
	Message string
	// BadRows are rows with the wrong number of columns ("row 1,204 has 9
	// columns, the header has 8"); they can be skipped.
	BadRows []ImportBadRow
	Header  int
}

FileError explains a file that can't be imported as is.

func (*FileError) Error

func (e *FileError) Error() string

type FileSource

type FileSource struct {
	Name string
	Kind string // csv, xlsx
	Data []byte
	// SkipRows are data rows (line numbers) left out because their column
	// count doesn't match the header.
	SkipRows map[int]bool
	// contains filtered or unexported fields
}

FileSource reads a CSV or XLSX upload. The first row holds the headers.

func (*FileSource) Connect

Connect implements Source.

func (*FileSource) Fetch

func (f *FileSource) Fetch(_ context.Context, _ []string, emit func(Row) error) error

Fetch implements Source. Email and status come from the columns that look like them; the mapping step can pick others.

func (*FileSource) Fields

func (f *FileSource) Fields(context.Context, []string) ([]SourceField, error)

Fields implements Source: one field per column, typed from its values.

func (*FileSource) Header

func (f *FileSource) Header() ([]string, error)

Header returns the column names.

func (*FileSource) Rows

func (f *FileSource) Rows() int

Rows returns the number of data rows.

type ImportBadRow

type ImportBadRow struct {
	Line    int    `json:"line"`
	Columns int    `json:"columns"`
	Raw     string `json:"raw"`
}

ImportBadRow is a row whose column count doesn't match the header.

type ImportConnectInput

type ImportConnectInput struct {
	APIKey  string `json:"apiKey" minLength:"1" maxLength:"500" doc:"Write-only: stored encrypted, never returned"`
	User    string `json:"user,omitempty" maxLength:"200" doc:"listmonk API user"`
	BaseURL string `json:"baseUrl,omitempty" maxLength:"500" format:"uri" doc:"listmonk or Keila server address"`
}

ImportConnectInput is the API key (and server for self-hosted tools).

type ImportConnection

type ImportConnection struct {
	Account    string             `json:"account"`
	DataCenter string             `json:"dataCenter,omitempty"`
	BaseURL    string             `json:"baseUrl,omitempty"`
	Lists      []ImportSourceList `json:"lists"`
	CheckedAt  time.Time          `json:"checkedAt"`
}

ImportConnection describes the source account.

type ImportConsentMap

type ImportConsentMap struct {
	Subscribed    string `json:"subscribed" enum:"confirmed,pending,skip"`
	Pending       string `json:"pending" enum:"pending,skip"`
	Transactional string `json:"transactional" enum:"skip,pending"`
}

ImportConsentMap is imports.consent_map: what each source status becomes. Unsubscribed, cleaned and complained are locked to suppressions.

type ImportConsentMapping

type ImportConsentMapping struct {
	Statuses      []ImportConsentStatus `json:"statuses"`
	SuppressCount int64                 `json:"suppressCount"`
	Evidence      string                `json:"evidence" enum:"trust,repermission"`
	// EvidenceSource names what the source's opt-in proof is ("OPTIN_TIME and OPTIN_IP").
	EvidenceSource   string `json:"evidenceSource"`
	DoubleOptIn      bool   `json:"doubleOptIn"`
	PullSuppressions bool   `json:"pullSuppressions"`
	RecordHistory    bool   `json:"recordHistory"`
	// HistoryExample is how the consent history entry will read.
	HistoryExample string `json:"historyExample"`
	// Providers are the providers new suppressions are pushed to.
	Providers []string `json:"providers"`
	// InSync is how many addresses are already shared with the providers.
	InSync int64 `json:"inSync"`
}

ImportConsentMapping is the "Consent" step.

type ImportConsentStatus

type ImportConsentStatus struct {
	Status   string              `json:"status"`
	Label    string              `json:"label" doc:"The source's own word: subscribed, cleaned, archived with \"abuse\" complaint"`
	Note     string              `json:"note,omitempty"`
	Tone     string              `json:"tone" enum:"ok,warn,bad,neutral"`
	Contacts int64               `json:"contacts"`
	Handling string              `json:"handling"`
	Options  []ImportFieldOption `json:"options"`
	Locked   bool                `json:"locked"`
	Effect   string              `json:"effect"`
}

ImportConsentStatus is one row of the consent table.

type ImportCreateInput

type ImportCreateInput struct {
	Source string `json:"source" enum:"brevo,mailchimp,mailerlite,listmonk,keila,csv,xlsx"`
}

ImportCreateInput starts an import.

type ImportDetail

type ImportDetail struct {
	ID             string            `json:"id"`
	Source         string            `json:"source"`
	SourceName     string            `json:"sourceName"`
	Status         string            `json:"status" enum:"draft,staging,mapping,previewing,running,paused,done,failed,cancelled"`
	Step           string            `json:"step" enum:"source,fields,consent,preview,import,done"`
	FileName       string            `json:"fileName"`
	HasCredentials bool              `json:"hasCredentials" doc:"An API key is stored (it is never returned)"`
	Connection     *ImportConnection `json:"connection,omitempty"`
	Options        ImportOptions     `json:"options"`
	StagedRows     int64             `json:"stagedRows"`
	StagedAt       *time.Time        `json:"stagedAt,omitempty"`
	Progress       ImportProgress    `json:"progress"`
	Stats          *ImportStats      `json:"stats,omitempty"`
	Error          *ImportError      `json:"error,omitempty"`
	Log            []ImportLogEntry  `json:"log"`
	CreatedAt      time.Time         `json:"createdAt"`
	StartedAt      *time.Time        `json:"startedAt,omitempty"`
	FinishedAt     *time.Time        `json:"finishedAt,omitempty"`
	CreatedBy      string            `json:"createdBy"`
	// Fields found in the source (merge fields, attributes, columns).
	FieldCount int `json:"fieldCount"`
	// Previous is the latest finished import, for "Previous imports".
	Previous *ImportSummary `json:"previous,omitempty"`
	// ProviderSync summarizes suppressions already pulled from providers
	// ("SES 612, Postmark 188, Postal 41").
	ProviderSync []ImportProviderPulled `json:"providerSync"`
}

ImportDetail is one import for the wizard.

type ImportDuplicateExample

type ImportDuplicateExample struct {
	Email        string `json:"email"`
	InReputeCast string `json:"inReputeCast"`
	// Suppressed rows read as a problem in the "In ReputeCast" column.
	Suppressed bool   `json:"suppressed"`
	InSource   string `json:"inSource"`
	Result     string `json:"result"`
	ResultTone string `json:"resultTone" enum:"ok,bad"`
}

ImportDuplicateExample is one address that is already in ReputeCast.

type ImportError

type ImportError struct {
	Code    string         `json:"code" doc:"source_auth_failed, source_rate_limited, source_unavailable, file_columns, file_unreadable"`
	Message string         `json:"message"`
	Status  int            `json:"status,omitempty" doc:"HTTP status from the source (401, 429)"`
	BadRows []ImportBadRow `json:"badRows,omitempty"`
	Header  int            `json:"header,omitempty"`
	RetryAt *time.Time     `json:"retryAt,omitempty"`
	// Row is where a paused import stopped.
	Row int64 `json:"row,omitempty"`
}

ImportError explains why the source or the file can't be used.

type ImportFieldChoice

type ImportFieldChoice struct {
	Source string `json:"source"`
	// Target: a built-in field, custom:<key>, new:<key> or skip.
	Target string `json:"target" maxLength:"80"`
	// Type of a new field: text, number, date, boolean.
	Type string `json:"type,omitempty" enum:"text,number,date,boolean,"`
}

ImportFieldChoice is one source field and where it goes.

type ImportFieldMapping

type ImportFieldMapping struct {
	Fields  []ImportFieldRow `json:"fields"`
	Mapped  int              `json:"mapped"`
	New     int              `json:"new"`
	Skipped int              `json:"skipped"`
	// Ready is false while rows are still being read from the source.
	Ready bool `json:"ready"`
}

ImportFieldMapping is the "Map fields" step.

type ImportFieldOption

type ImportFieldOption struct {
	Value string `json:"value"`
	Label string `json:"label"`
}

ImportFieldOption is a choice in a mapping select.

type ImportFieldRow

type ImportFieldRow struct {
	Source  string              `json:"source"`
	Label   string              `json:"label"`
	Samples []string            `json:"samples"`
	Target  string              `json:"target"`
	Type    string              `json:"type" doc:"What ReputeCast stores: email, text, number, date, timezone, consent, tags, or – when skipped"`
	Status  string              `json:"status" enum:"mapped,new,skipped"`
	Options []ImportFieldOption `json:"options"`
}

ImportFieldRow is one row of the mapping table.

type ImportFile

type ImportFile struct {
	ContentType        string `header:"Content-Type"`
	ContentDisposition string `header:"Content-Disposition"`
	CacheControl       string `header:"Cache-Control"`
	Body               []byte
}

ImportFile is a CSV download.

type ImportInvalidGroup

type ImportInvalidGroup struct {
	Problem string `json:"problem"`
	Example string `json:"example,omitempty"`
	Rows    int64  `json:"rows"`
}

ImportInvalidGroup is a kind of invalid address and how many rows have it.

type ImportJob

type ImportJob struct {
	ID   string `json:"id"`
	Kind string `json:"kind" enum:"import"`
	// Title: "Importing 13,739 contacts from Mailchimp".
	Title  string `json:"title"`
	Status string `json:"status" enum:"queued,running,done,failed,canceled"`
	// Progress is 0–100 while running.
	Progress *int `json:"progress,omitempty"`
	// Eta: "about 1 min left".
	Eta string `json:"eta,omitempty"`
	// Detail is the result or the failure in words.
	Detail string `json:"detail,omitempty"`
	// Href is the app path of the import's progress or report.
	Href       string     `json:"href"`
	StartedAt  *time.Time `json:"startedAt,omitempty"`
	FinishedAt *time.Time `json:"finishedAt,omitempty"`
}

ImportJob is an import shown in the background-jobs tray: running ones with their progress, and ones that ended in the last two hours.

type ImportJobList

type ImportJobList struct {
	Items []ImportJob `json:"items"`
}

ImportJobList is the jobs tray's list.

type ImportList

type ImportList struct {
	Items []ImportSummary `json:"items"`
	// Active is the draft or running import the wizard resumes.
	Active string `json:"active,omitempty"`
}

ImportList is the import history.

type ImportListAfter

type ImportListAfter struct {
	Name     string `json:"name"`
	Contacts int64  `json:"contacts"`
}

ImportListAfter is a target list and its size after the import.

type ImportListChoice

type ImportListChoice struct {
	SourceID string `json:"sourceId"`
	Target   string `json:"target" doc:"List id, or new:<name>"`
}

ImportListChoice maps a source list to a ReputeCast list.

type ImportLogEntry

type ImportLogEntry struct {
	At      time.Time `json:"at"`
	Kind    string    `json:"kind" enum:"start,running,done,sync,retry,error,info"`
	Message string    `json:"message"`
}

ImportLogEntry is one line of the live log.

type ImportOptions

type ImportOptions struct {
	// Lists are the source lists to import and where each goes: a list id
	// or "new:<name>" to create one.
	Lists            []ImportListChoice `json:"lists"`
	ImportTags       bool               `json:"importTags"`
	ImportEngagement bool               `json:"importEngagement"`
	PullSuppressions bool               `json:"pullSuppressions"`
	RecordHistory    bool               `json:"recordHistory"`
	// Evidence: trust (keep the source's opt-in as evidence) or
	// repermission (import as pending and send a confirmation email).
	Evidence string `json:"evidence" enum:"trust,repermission"`
	// OnConflict for contacts that already exist: fill (empty fields only),
	// overwrite, or keep (ReputeCast values win).
	OnConflict string             `json:"onConflict" enum:"fill,overwrite,keep"`
	Risky      ImportRiskyActions `json:"risky"`
	// SkipRows are file lines left out (rows with the wrong column count).
	SkipRows []int `json:"skipRows"`
	// Pace limits requests to the source, per second (0: as fast as it
	// answers); "Slow down" after a rate limit sets 3.
	Pace int `json:"pace,omitempty" minimum:"0" maximum:"50"`
}

ImportOptions are the choices made across the wizard.

type ImportPathInput

type ImportPathInput struct {
	api.WorkspaceInput
	ID string `path:"id" format:"uuid"`
}

type ImportPreview

type ImportPreview struct {
	Rows        int64 `json:"rows"`
	WillImport  int64 `json:"willImport" doc:"New contacts, subscribed"`
	Merged      int64 `json:"merged" doc:"Already in ReputeCast"`
	Suppressed  int64 `json:"suppressed" doc:"Unsubscribed, bounced or complained in the source"`
	Invalid     int64 `json:"invalid"`
	NotImported int64 `json:"notImported" doc:"Statuses mapped to Skip"`
	Risky       int64 `json:"risky"`
	// ImportCount and SuppressCount label the import button.
	ImportCount   int64                    `json:"importCount"`
	SuppressCount int64                    `json:"suppressCount"`
	RiskyGroups   []ImportRiskyGroup       `json:"riskyGroups"`
	Duplicates    []ImportDuplicateExample `json:"duplicates"`
	InvalidGroups []ImportInvalidGroup     `json:"invalidGroups"`
	OnConflict    string                   `json:"onConflict" enum:"fill,overwrite,keep"`
	ComputedAt    time.Time                `json:"computedAt"`
}

ImportPreview is the dry run: what would happen to every row.

type ImportProgress

type ImportProgress struct {
	Total      int64 `json:"total"`
	Processed  int64 `json:"processed"`
	Batch      int   `json:"batch"`
	Batches    int   `json:"batches"`
	BatchSize  int   `json:"batchSize"`
	Imported   int64 `json:"imported"`
	Merged     int64 `json:"merged"`
	Suppressed int64 `json:"suppressed"`
	Pending    int64 `json:"pending"`
	Skipped    int64 `json:"skipped"`
}

ImportProgress is the state of a running import.

type ImportProviderPulled

type ImportProviderPulled struct {
	Name   string `json:"name"`
	Short  string `json:"short"`
	Pulled int64  `json:"pulled"`
}

ImportProviderPulled is a provider and how many suppressions came from it.

type ImportRiskyActions

type ImportRiskyActions struct {
	Role       string `json:"role" enum:"exclude_broadcasts,skip,import"`
	Disposable string `json:"disposable" enum:"skip,import"`
	Typo       string `json:"typo" enum:"fix,skip,import"`
	Stale      string `json:"stale" enum:"pending,skip,import"`
	Trap       string `json:"trap" enum:"skip,import"`
}

ImportRiskyActions say what happens to each risky group.

type ImportRiskyGroup

type ImportRiskyGroup struct {
	Key      string              `json:"key" enum:"role,disposable,typo,stale,trap"`
	Label    string              `json:"label"`
	Rows     int64               `json:"rows"`
	Why      string              `json:"why"`
	Examples string              `json:"examples"`
	Action   string              `json:"action"`
	Actions  []ImportFieldOption `json:"actions"`
}

ImportRiskyGroup is one group of risky addresses with the chosen action.

type ImportSaveConsentInput

type ImportSaveConsentInput struct {
	Map              ImportConsentMap `json:"map"`
	Evidence         string           `json:"evidence" enum:"trust,repermission"`
	PullSuppressions bool             `json:"pullSuppressions"`
	RecordHistory    bool             `json:"recordHistory"`
}

ImportSaveConsentInput is the consent step's choices.

type ImportSaveFieldsInput

type ImportSaveFieldsInput struct {
	Fields []ImportFieldChoice `json:"fields" maxItems:"500"`
}

ImportSaveFieldsInput is the mapping chosen by the person.

type ImportSourceList

type ImportSourceList struct {
	ID            string   `json:"id"`
	Name          string   `json:"name"`
	Subscribed    int64    `json:"subscribed"`
	Unsubscribed  int64    `json:"unsubscribed"`
	Cleaned       int64    `json:"cleaned" doc:"Hard bounced in the source"`
	Pending       int64    `json:"pending"`
	Transactional int64    `json:"transactional"`
	Complained    int64    `json:"complained"`
	Fields        int      `json:"fields"`
	Tags          int      `json:"tags"`
	TagExamples   []string `json:"tagExamples"`
	DoubleOptIn   bool     `json:"doubleOptIn"`
	// Since is the year the list was created ("double opt-in was on in this audience since 2024").
	Since string `json:"since,omitempty"`
}

ImportSourceList is an audience, list or group in the source.

type ImportStats

type ImportStats struct {
	Rows         int64            `json:"rows"`
	Imported     int64            `json:"imported"`
	Merged       int64            `json:"merged"`
	Suppressed   int64            `json:"suppressed"`
	SuppressedBy map[string]int64 `json:"suppressedBy" doc:"unsubscribe, hard_bounce, complaint"`
	Pending      int64            `json:"pending"`
	Skipped      int64            `json:"skipped"`
	// Reasons counts skipped and changed rows: invalid, disposable, trap,
	// already_suppressed, typo_fixed, role.
	Reasons map[string]int64 `json:"reasons"`
	// Lists are the target lists and their size after the import.
	Lists        []ImportListAfter `json:"lists"`
	DurationMs   int64             `json:"durationMs"`
	PushedTo     []string          `json:"pushedTo" doc:"Providers the new suppressions were pushed to"`
	ConfirmsSent int64             `json:"confirmsSent" doc:"Re-permission emails sent through the System stream"`
}

ImportStats is the summary of a finished import.

type ImportSummary

type ImportSummary struct {
	ID         string     `json:"id"`
	Source     string     `json:"source"`
	SourceName string     `json:"sourceName"`
	Status     string     `json:"status"`
	Step       string     `json:"step"`
	FileName   string     `json:"fileName"`
	Contacts   int64      `json:"contacts" doc:"Contacts imported, merged or imported as pending"`
	CreatedAt  time.Time  `json:"createdAt"`
	FinishedAt *time.Time `json:"finishedAt,omitempty"`
	CreatedBy  string     `json:"createdBy"`
}

ImportSummary is one import in the history.

type ImportUpdateInput

type ImportUpdateInput struct {
	Source  string         `json:"source,omitempty" enum:"brevo,mailchimp,mailerlite,listmonk,keila,csv,xlsx,"`
	Options *ImportOptions `json:"options,omitempty"`
	Step    string         `json:"step,omitempty" enum:"source,fields,consent,preview,"`
}

ImportUpdateInput changes the draft: its source (before connecting), options and the step the person is on.

type Keila

type Keila struct {
	// contains filtered or unexported fields
}

Keila reads contacts through a Keila server's API v1 (read-only). Keila has no lists, so everything comes in as one "All contacts" list.

func NewKeila

func NewKeila(baseURL, key string, hc *http.Client) (*Keila, error)

NewKeila builds a client for a Keila server (https://app.keila.io or self-hosted).

func (*Keila) Connect

func (k *Keila) Connect(ctx context.Context) (ImportConnection, error)

Connect implements Source.

func (*Keila) Fetch

func (k *Keila) Fetch(ctx context.Context, _ []string, emit func(Row) error) error

Fetch implements Source.

func (*Keila) Fields

func (k *Keila) Fields(ctx context.Context, _ []string) ([]SourceField, error)

Fields implements Source: built-in fields plus the custom data keys of the first page.

type Listmonk

type Listmonk struct {
	// contains filtered or unexported fields
}

Listmonk reads lists and subscribers through a listmonk server's API with an API user and token (read-only).

func NewListmonk

func NewListmonk(baseURL, user, token string, hc *http.Client) (*Listmonk, error)

NewListmonk builds a client for a listmonk server.

func (*Listmonk) Connect

func (l *Listmonk) Connect(ctx context.Context) (ImportConnection, error)

Connect implements Source.

func (*Listmonk) Fetch

func (l *Listmonk) Fetch(ctx context.Context, lists []string, emit func(Row) error) error

Fetch implements Source.

func (*Listmonk) Fields

func (l *Listmonk) Fields(ctx context.Context, lists []string) ([]SourceField, error)

Fields implements Source: email, name and the attribute keys found on the first page of subscribers.

type MXChecker

type MXChecker struct {
	Q        *store.Queries
	Resolver Resolver
	TTL      time.Duration
	// Timeout bounds one lookup; Parallel bounds concurrent lookups.
	Timeout  time.Duration
	Parallel int
}

MXChecker tells whether a domain accepts mail, caching answers in mx_cache for a week so repeated imports don't hit DNS again.

func (*MXChecker) Check

func (m *MXChecker) Check(ctx context.Context, domains []string) (map[string]MXResult, error)

Check returns, for each domain, whether it can receive mail. Temporary DNS failures count as "yes": an address is only rejected for a definite answer (NXDOMAIN, or no MX and no address records).

type MXResult

type MXResult struct {
	HasMX bool
	// Hosting names the mailbox hosting behind an "other" domain
	// ("Google Workspace"), from the MX host.
	Hosting string
}

MXResult is what a domain's DNS says.

type Mailchimp

type Mailchimp struct {
	DC string
	// contains filtered or unexported fields
}

Mailchimp reads audiences, merge fields, tags and members through the Marketing API v3 (read-only: nothing is written to Mailchimp).

func NewMailchimp

func NewMailchimp(apiKey, base string, hc *http.Client) (*Mailchimp, error)

NewMailchimp builds a client from an API key ("…-us21"); base overrides the API URL (tests).

func (*Mailchimp) Connect

func (m *Mailchimp) Connect(ctx context.Context) (ImportConnection, error)

Connect implements Source.

func (*Mailchimp) Fetch

func (m *Mailchimp) Fetch(ctx context.Context, lists []string, emit func(Row) error) error

Fetch implements Source. Members who reported abuse come from the abuse reports and are marked complained.

func (*Mailchimp) Fields

func (m *Mailchimp) Fields(ctx context.Context, lists []string) ([]SourceField, error)

Fields implements Source: EMAIL, the merge fields, and the member properties ReputeCast can use (timezone, opt-in evidence, tags).

type MailerLite

type MailerLite struct {
	// contains filtered or unexported fields
}

MailerLite reads groups, fields and subscribers through the MailerLite API (connect.mailerlite.com, read-only).

func NewMailerLite

func NewMailerLite(token, base string, hc *http.Client) (*MailerLite, error)

NewMailerLite builds a client from an API token; base overrides the API URL.

func (*MailerLite) Connect

func (m *MailerLite) Connect(ctx context.Context) (ImportConnection, error)

Connect implements Source.

func (*MailerLite) Fetch

func (m *MailerLite) Fetch(ctx context.Context, lists []string, emit func(Row) error) error

Fetch implements Source.

func (*MailerLite) Fields

func (m *MailerLite) Fields(ctx context.Context, _ []string) ([]SourceField, error)

Fields implements Source.

type Resolver

type Resolver interface {
	LookupMX(ctx context.Context, name string) ([]*net.MX, error)
	LookupHost(ctx context.Context, host string) ([]string, error)
}

Resolver is the part of net.Resolver the MX check uses.

type Row

type Row struct {
	Email      string
	Status     string
	List       string
	Data       map[string]string
	Tags       []string
	Engagement Engagement
}

Row is one contact read from the source.

type RunArgs

type RunArgs struct {
	ImportID uuid.UUID `json:"importId"`
}

RunArgs imports the staged rows of an import.

func (RunArgs) InsertOpts

func (RunArgs) InsertOpts() river.InsertOpts

InsertOpts implements river.JobArgsWithInsertOpts.

func (RunArgs) Kind

func (RunArgs) Kind() string

Kind implements river.JobArgs.

type RunWorker

type RunWorker struct {
	river.WorkerDefaults[RunArgs]
	Svc *Service
}

RunWorker runs RunArgs.

func (*RunWorker) Timeout

func (w *RunWorker) Timeout(*river.Job[RunArgs]) time.Duration

Timeout allows large imports.

func (*RunWorker) Work

func (w *RunWorker) Work(ctx context.Context, job *river.Job[RunArgs]) error

Work implements river.Worker.

type Sealer

type Sealer interface {
	Seal(plaintext []byte, ad string) ([]byte, error)
	Open(ciphertext []byte, ad string) ([]byte, error)
}

Sealer encrypts and decrypts importer credentials.

type Service

type Service struct {
	Pool     *pgxpool.Pool
	Q        *store.Queries
	Audit    *audit.Logger
	Box      Sealer
	Audience *audience.Service
	Mail     mailer.Sender
	// Jobs returns the River client (created after the services are wired).
	Jobs func() *jobs.Client
	MX   *MXChecker
	// HTTP is used by the API importers; BaseURLs overrides their API base
	// per source (tests). Guard decides which addresses a self-hosted
	// server (listmonk, Keila) may have: the client refuses the same ones
	// at connect time.
	HTTP     *http.Client
	Guard    netguard.Policy
	BaseURLs map[string]string
	Now      func() time.Time
	// Inline runs staging and imports in the request instead of River (tests).
	Inline bool
}

func (*Service) BackgroundJobs

func (s *Service) BackgroundJobs(ctx context.Context, ws *store.Workspace) (ImportJobList, error)

BackgroundJobs lists the imports for the jobs tray.

func (*Service) BadRowsCSV

func (s *Service) BadRowsCSV(ctx context.Context, ws *store.Workspace, id uuid.UUID) ([]byte, string, error)

BadRowsCSV returns the rows a file import couldn't read, for download.

func (*Service) Cancel

func (s *Service) Cancel(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportDetail, error)

Cancel stops an import; rows already imported stay.

func (*Service) Connect

Connect checks the key against the source, stores it encrypted and returns the account and its lists.

func (*Service) Consent

Consent returns the consent step: counts per source status and the handling of each.

func (*Service) Create

Create makes a draft import.

func (*Service) Delete

func (s *Service) Delete(ctx context.Context, ws *store.Workspace, id uuid.UUID) error

Delete discards a draft import and its staged rows.

func (*Service) FieldGuesses

func (s *Service) FieldGuesses(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportFieldMapping, error)

FieldGuesses is the mapping step with every target guessed again ("Reset to guesses"); nothing is saved until the mapping is.

func (*Service) Fields

func (s *Service) Fields(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportFieldMapping, error)

Fields returns the mapping step: each source field with sample values, its current or guessed target and the choices.

func (*Service) Get

func (s *Service) Get(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportDetail, error)

Get returns one import with its connection, progress and log.

func (*Service) InvalidCSV

func (s *Service) InvalidCSV(ctx context.Context, ws *store.Workspace, id uuid.UUID) ([]byte, string, error)

InvalidCSV lists the invalid rows of the preview ("Download these rows").

func (*Service) List

func (s *Service) List(ctx context.Context, ws *store.Workspace) (ImportList, error)

List returns the imports of a workspace, newest first.

func (*Service) Preview

func (s *Service) Preview(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportPreview, error)

Preview runs the dry run over every staged row. Nothing is written except the stored preview.

func (*Service) ReportCSV

func (s *Service) ReportCSV(ctx context.Context, ws *store.Workspace, id uuid.UUID, kind string) ([]byte, string, error)

ReportCSV returns one of the import's CSV files.

func (*Service) RequestStage

func (s *Service) RequestStage(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportDetail, error)

RequestStage starts reading rows for the selected lists, unless they are already staged. It moves the draft to the field mapping step.

func (*Service) Resume

func (s *Service) Resume(ctx context.Context, ws *store.Workspace, id uuid.UUID, slow bool) (ImportDetail, error)

Resume restarts a paused import now ("Resume now"). With slow, requests to the source are paced to 3 a second from now on ("Slow down to 3 req/s").

func (*Service) Run

func (s *Service) Run(ctx context.Context, id uuid.UUID) error

Run imports every staged row in batches, with a live log.

func (*Service) SaveConsent

SaveConsent stores the consent mapping and moves on to the preview.

func (*Service) SaveFields

SaveFields stores the mapping and moves on to consent.

func (*Service) SkipBadRows

func (s *Service) SkipBadRows(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportDetail, error)

SkipBadRows leaves out the file rows whose column count doesn't match the header ("Skip the 4 rows and continue").

func (*Service) Stage

func (s *Service) Stage(ctx context.Context, id uuid.UUID) error

Stage reads every row of the selected lists into import_rows.

func (*Service) Start

func (s *Service) Start(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportDetail, error)

Start begins the import of the staged rows.

func (*Service) Update

Update saves changes to a draft.

func (*Service) Upload

func (s *Service) Upload(ctx context.Context, ws *store.Workspace, id uuid.UUID, name string, data []byte) (ImportDetail, error)

Upload stores a CSV or XLSX file on the draft and reads its header.

type Source

type Source interface {
	// Connect checks access and returns the account and its lists with
	// counts per consent status.
	Connect(ctx context.Context) (ImportConnection, error)
	// Fields returns the source's fields for the selected lists.
	Fields(ctx context.Context, lists []string) ([]SourceField, error)
	// Fetch streams every contact of the selected lists, with its status.
	Fetch(ctx context.Context, lists []string, emit func(Row) error) error
}

Source is where contacts are imported from: an email tool's API or a file.

type SourceError

type SourceError struct {
	Status  int
	Code    string // auth_failed, rate_limited, unavailable, bad_response
	Message string
	// Reason is the source's own short words ("API Key Invalid").
	Reason string
	// RetryAfter is set for rate limits.
	RetryAfter time.Duration
}

SourceError is a failure talking to the source, explained for people.

func (*SourceError) Error

func (e *SourceError) Error() string

type SourceField

type SourceField struct {
	Key   string `json:"key"`
	Label string `json:"label"`
	// Type as the source reports it: text, number, date, email, phone,
	// address, birthday, url, boolean.
	Type string `json:"type"`
}

SourceField is a field (merge field, attribute, column) in the source.

type StageArgs

type StageArgs struct {
	ImportID uuid.UUID `json:"importId"`
}

StageArgs reads an import's rows from its source into import_rows.

func (StageArgs) InsertOpts

func (StageArgs) InsertOpts() river.InsertOpts

InsertOpts implements river.JobArgsWithInsertOpts.

func (StageArgs) Kind

func (StageArgs) Kind() string

Kind implements river.JobArgs.

type StageWorker

type StageWorker struct {
	river.WorkerDefaults[StageArgs]
	Svc *Service
}

StageWorker runs StageArgs.

func (*StageWorker) Timeout

func (w *StageWorker) Timeout(*river.Job[StageArgs]) time.Duration

Timeout allows long API reads.

func (*StageWorker) Work

func (w *StageWorker) Work(ctx context.Context, job *river.Job[StageArgs]) error

Work implements river.Worker.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL