imports

package
v1.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Nov 3, 2026 License: AGPL-3.0 Imports: 46 Imported by: 0

Documentation

Overview

Package imports is the migration importer (spec 5.8, 7.11): CSV and XLSX uploads and API importers for Brevo, Mailchimp, MailerLite, listmonk and Keila. A River job stages the rows; after field and consent mapping and a dry run, a second job imports them in batches. Unsubscribed, bounced and complained contacts become suppressions and are never re-subscribed.

Index

Constants

View Source
const (
	HandleConfirmed         = "confirmed"
	HandlePending           = "pending"
	HandleSkip              = "skip"
	HandleSuppressUnsub     = "suppress_unsubscribe"
	HandleSuppressBounce    = "suppress_bounce"
	HandleSuppressComplaint = "suppress_complaint"
)

Consent handling of each source status.

View Source
const (
	TargetEmail    = "email"
	TargetSkip     = "skip"
	TargetEvidence = "consent_evidence"
	TargetStatus   = "status"
	TargetTags     = "tags"
	TargetName     = "name"
)

Field targets a source field can map to.

View Source
const (
	StatusSubscribed    = "subscribed"
	StatusUnsubscribed  = "unsubscribed"
	StatusCleaned       = "cleaned"
	StatusPending       = "pending"
	StatusTransactional = "transactional"
	StatusComplained    = "complained"
)

Source statuses, normalized.

View Source
const (
	CodeAuthFailed  = "source_auth_failed"
	CodeRateLimited = "source_rate_limited"
	CodeUnavailable = "source_unavailable"
	CodeBadResponse = "source_bad_response"
)

Error codes of SourceError.

View Source
const (
	TargetPhone            = "phone"
	TargetSmsConsent       = "sms_consent"
	TargetSmsConsentAt     = "sms_consent_at"
	TargetSmsConsentSource = "sms_consent_source"
	TargetSmsConsentText   = "sms_consent_text"
)

The phone and SMS consent targets of Map fields.

View Source
const BatchSize = 500

BatchSize is how many rows one batch imports.

View Source
const FileListID = "file"

FileListID is the pseudo-list of a file import.

View Source
const KeilaAllContacts = "all"

KeilaAllContacts is the pseudo-list id of a Keila import.

View Source
const MaxFileRows = 500000

MaxFileRows bounds an uploaded file ("Up to 500,000 rows").

View Source
const ProblemNoMX = "no MX record"

ProblemNoMX is the invalid reason for domains that can't receive mail.

View Source
const ProblemService = "rejected by the verification service"

ProblemService is the invalid reason of rows a verification service rejected.

View Source
const ReportKeepDays = 30

ReportKeepDays is how long a finished import keeps its report and skipped-row CSVs, which list every address the import read (review 2026-11 DP-8). Maintenance clears them after that; a done import drops its uploaded file as soon as it finishes.

Variables

View Source
var ErrNoKey = errors.New("paste an API key first")

ErrNoKey is returned when an API importer has no credentials.

View Source
var Sources = []string{"brevo", "mailchimp", "mailerlite", "listmonk", "keila", "notifuse", "csv"}

Sources the importer supports, in the order of the source grid.

Functions

func Erasure

func Erasure() erasure.Source

Erasure is the contact's rows in imports that haven't finished (a done import drops its rows): the address, its fields and tags (review 2026-11 DP-1). An import's kept file and reports are read too (DP-8): a report loses the rows that name an erased address, and an uploaded file that names one is removed, with a line in the import's log saying why.

func NormalizeStatus

func NormalizeStatus(v string) string

NormalizeStatus maps a status column's value to a source status.

func Register

func Register(h huma.API, s *Service)

Register adds the importer operations.

func RegisterJobs

func RegisterJobs(reg *jobs.Registry, s *Service)

RegisterJobs adds the staging and import workers.

func ReportKeptUntil

func ReportKeptUntil(finished time.Time) time.Time

ReportKeptUntil is when a finished import's reports are cleared.

func SourceName

func SourceName(src string) string

SourceName is the display name of a source.

Types

type AddressChecks

type AddressChecks interface {
	ImportAnswers(ctx context.Context, ws *store.Workspace, importID uuid.UUID) (*ImportChecks, error)
}

AddressChecks gives a verification service's answers for an import's new addresses (internal/verification): the preview folds them into its groups and the import follows them. Nil leaves the import as it was.

type Brevo

type Brevo struct {
	// contains filtered or unexported fields
}

Brevo reads lists, contact attributes and the blocklist through the Brevo API v3 (read-only).

func NewBrevo

func NewBrevo(apiKey, base string, hc *http.Client) (*Brevo, error)

NewBrevo builds a client from an API key; base overrides the API URL.

func (*Brevo) Connect

func (b *Brevo) Connect(ctx context.Context) (ImportConnection, error)

Connect implements Source.

func (*Brevo) Fetch

func (b *Brevo) Fetch(ctx context.Context, lists []string, emit func(Row) error) error

Fetch implements Source. The blocklist says why an address is blocked: hard bounces become cleaned, spam reports complained, the rest unsubscribed.

func (*Brevo) Fields

func (b *Brevo) Fields(ctx context.Context, _ []string) ([]SourceField, error)

Fields implements Source: the email plus normal contact attributes.

type Engagement

type Engagement struct {
	LastOpen  *time.Time `json:"last_open,omitempty"`
	LastClick *time.Time `json:"last_click,omitempty"`
	Changed   *time.Time `json:"changed,omitempty"`
	SignupAt  *time.Time `json:"signup_at,omitempty"`
	OptinAt   *time.Time `json:"optin_at,omitempty"`
	OptinIP   string     `json:"optin_ip,omitempty"`
	// Rating is Mailchimp's member rating (1–5).
	Rating int `json:"rating,omitempty"`
}

Engagement is what the source knows about a contact's activity and consent.

type FieldMap

type FieldMap struct {
	Fields []ImportFieldChoice `json:"fields"`
}

FieldMap is imports.field_map.

func (FieldMap) SMSEvidenceMapped

func (fm FieldMap) SMSEvidenceMapped() (consent, evidence bool)

SMSEvidenceMapped reports whether SMS consent can come in: the consent column and its date, source and text are all mapped.

type FileError

type FileError struct {
	Message string
	Text    i18n.Text
	// BadRows are rows with the wrong number of columns ("row 1,204 has 9
	// columns, the header has 8"); they can be skipped.
	BadRows []ImportBadRow
	Header  int
}

FileError explains a file that can't be imported as is: Message in English, Text the message it came from for readers of other languages.

func (*FileError) Error

func (e *FileError) Error() string

type FileSource

type FileSource struct {
	Name string
	Kind string // csv, xlsx
	Data []byte
	// SkipRows are data rows (line numbers) left out because their column
	// count doesn't match the header.
	SkipRows map[int]bool
	// contains filtered or unexported fields
}

FileSource reads a CSV or XLSX upload. The first row holds the headers.

func (*FileSource) Connect

Connect implements Source.

func (*FileSource) Fetch

func (f *FileSource) Fetch(_ context.Context, _ []string, emit func(Row) error) error

Fetch implements Source. Email and status come from the columns that look like them; the mapping step can pick others.

func (*FileSource) Fields

func (f *FileSource) Fields(context.Context, []string) ([]SourceField, error)

Fields implements Source: one field per column, typed from its values.

func (*FileSource) Header

func (f *FileSource) Header() ([]string, error)

Header returns the column names.

func (*FileSource) Rows

func (f *FileSource) Rows() int

Rows returns the number of data rows.

type ImportAnswer

type ImportAnswer struct {
	Result, Reason, Code string
}

ImportAnswer is one address's result: valid, invalid, risky or unknown, with the reason (catch_all, role, disposable, abuse, invalid_address, spam_trap) and the service's own code.

type ImportBadRow

type ImportBadRow struct {
	Line    int    `json:"line"`
	Columns int    `json:"columns"`
	Raw     string `json:"raw"`
}

ImportBadRow is a row whose column count doesn't match the header.

type ImportChecks

type ImportChecks struct {
	JobID       uuid.UUID
	Service     string
	ServiceName string
	// Status is the job's: running while it checks, finished after.
	Status    string
	CheckedAt *time.Time
	Checked   int64
	Cost      float64
	// Answers by lower-cased address.
	Answers map[string]ImportAnswer
}

ImportChecks are a service's answers for one import.

type ImportConnectInput

type ImportConnectInput struct {
	APIKey  string `json:"apiKey" minLength:"1" maxLength:"500" doc:"Write-only: stored encrypted, never returned"`
	User    string `json:"user,omitempty" maxLength:"200" doc:"listmonk API user"`
	BaseURL string `json:"baseUrl,omitempty" maxLength:"500" format:"uri" doc:"listmonk, Keila or Notifuse server address"`
}

ImportConnectInput is the API key (and server for self-hosted tools).

type ImportConnection

type ImportConnection struct {
	Account    string             `json:"account"`
	DataCenter string             `json:"dataCenter,omitempty"`
	BaseURL    string             `json:"baseUrl,omitempty"`
	Lists      []ImportSourceList `json:"lists"`
	CheckedAt  time.Time          `json:"checkedAt"`
	// Templates are the email templates the source can bring along
	// (Notifuse).
	Templates *ImportTemplateCounts `json:"templates,omitempty"`
}

ImportConnection describes the source account.

type ImportConsentMap

type ImportConsentMap struct {
	Subscribed    string `json:"subscribed" enum:"confirmed,pending,skip"`
	Pending       string `json:"pending" enum:"pending,skip"`
	Transactional string `json:"transactional" enum:"skip,pending"`
}

ImportConsentMap is imports.consent_map: what each source status becomes. Unsubscribed, cleaned and complained are locked to suppressions.

type ImportConsentMapping

type ImportConsentMapping struct {
	Statuses      []ImportConsentStatus `json:"statuses"`
	SuppressCount int64                 `json:"suppressCount"`
	Evidence      string                `json:"evidence" enum:"trust,repermission"`
	// EvidenceSource names what the source's opt-in proof is ("OPTIN_TIME and OPTIN_IP").
	EvidenceSource   string `json:"evidenceSource"`
	DoubleOptIn      bool   `json:"doubleOptIn"`
	PullSuppressions bool   `json:"pullSuppressions"`
	RecordHistory    bool   `json:"recordHistory"`
	// Tag goes on every row the import brings in (review 2026-11 AU-10).
	Tag string `json:"tag"`
	// HistoryExample is how the consent history entry will read.
	HistoryExample string `json:"historyExample"`
	// Providers are the providers new suppressions are pushed to.
	Providers []string `json:"providers"`
	// InSync is how many addresses are already shared with the providers.
	InSync int64 `json:"inSync"`
	// Sms is SMS consent and its evidence columns (phase 8), when a column
	// maps to the phone.
	Sms *ImportSmsConsent `json:"sms,omitempty"`
}

ImportConsentMapping is the "Consent" step.

type ImportConsentStatus

type ImportConsentStatus struct {
	Status   string              `json:"status"`
	Label    string              `json:"label" doc:"The source's own word: subscribed, cleaned, archived with \"abuse\" complaint"`
	Note     string              `json:"note,omitempty"`
	Tone     string              `json:"tone" enum:"ok,warn,bad,neutral"`
	Contacts int64               `json:"contacts"`
	Handling string              `json:"handling"`
	Options  []ImportFieldOption `json:"options"`
	Locked   bool                `json:"locked"`
	Effect   string              `json:"effect"`
}

ImportConsentStatus is one row of the consent table.

type ImportCreateInput

type ImportCreateInput struct {
	Source string `json:"source" enum:"brevo,mailchimp,mailerlite,listmonk,keila,notifuse,csv,xlsx"`
}

ImportCreateInput starts an import.

type ImportDetail

type ImportDetail struct {
	ID             string            `json:"id"`
	Source         string            `json:"source"`
	SourceName     string            `json:"sourceName"`
	Status         string            `json:"status" enum:"draft,staging,mapping,previewing,running,paused,done,failed,cancelled"`
	Step           string            `json:"step" enum:"source,fields,consent,preview,import,done"`
	FileName       string            `json:"fileName"`
	HasCredentials bool              `json:"hasCredentials" doc:"An API key is stored (it is never returned)"`
	Connection     *ImportConnection `json:"connection,omitempty"`
	Options        ImportOptions     `json:"options"`
	StagedRows     int64             `json:"stagedRows"`
	StagedAt       *time.Time        `json:"stagedAt,omitempty"`
	Progress       ImportProgress    `json:"progress"`
	Stats          *ImportStats      `json:"stats,omitempty"`
	Error          *ImportError      `json:"error,omitempty"`
	Log            []ImportLogEntry  `json:"log"`
	CreatedAt      time.Time         `json:"createdAt"`
	StartedAt      *time.Time        `json:"startedAt,omitempty"`
	FinishedAt     *time.Time        `json:"finishedAt,omitempty"`
	// ReportsUntil is when a finished import's report and skipped-row
	// files are cleared; ReportsCleared says they were (they list every
	// address read).
	ReportsUntil   *time.Time `json:"reportsUntil,omitempty"`
	ReportsCleared bool       `json:"reportsCleared"`
	CreatedBy      string     `json:"createdBy"`
	// Fields found in the source (merge fields, attributes, columns).
	FieldCount int `json:"fieldCount"`
	// Previous is the latest finished import, for "Previous imports".
	Previous *ImportSummary `json:"previous,omitempty"`
	// ProviderSync summarizes suppressions already pulled from providers
	// ("SES 612, Postmark 188, Postal 41").
	ProviderSync []ImportProviderPulled `json:"providerSync"`
}

ImportDetail is one import for the wizard.

type ImportDuplicateExample

type ImportDuplicateExample struct {
	Email        string `json:"email"`
	InReputeCast string `json:"inReputeCast"`
	// Suppressed rows read as a problem in the "In ReputeCast" column.
	Suppressed bool   `json:"suppressed"`
	InSource   string `json:"inSource"`
	Result     string `json:"result"`
	ResultTone string `json:"resultTone" enum:"ok,bad"`
}

ImportDuplicateExample is one address that is already in ReputeCast.

type ImportError

type ImportError struct {
	Code    string         `json:"code" doc:"source_auth_failed, source_rate_limited, source_unavailable, file_columns, file_unreadable"`
	Message string         `json:"message"`
	Status  int            `json:"status,omitempty" doc:"HTTP status from the source (401, 429)"`
	BadRows []ImportBadRow `json:"badRows,omitempty"`
	Header  int            `json:"header,omitempty"`
	RetryAt *time.Time     `json:"retryAt,omitempty"`
	// Row is where a paused import stopped.
	Row int64 `json:"row,omitempty"`
}

ImportError explains why the source or the file can't be used.

type ImportFieldChoice

type ImportFieldChoice struct {
	Source string `json:"source"`
	// Target: a built-in field, custom:<key>, new:<key> or skip.
	Target string `json:"target" maxLength:"80"`
	// Type of a new field: text, number, date, boolean.
	Type string `json:"type,omitempty" enum:"text,number,date,boolean,"`
}

ImportFieldChoice is one source field and where it goes.

type ImportFieldMapping

type ImportFieldMapping struct {
	Fields  []ImportFieldRow `json:"fields"`
	Mapped  int              `json:"mapped"`
	New     int              `json:"new"`
	Skipped int              `json:"skipped"`
	// Ready is false while rows are still being read from the source.
	Ready bool `json:"ready"`
}

ImportFieldMapping is the "Map fields" step.

type ImportFieldOption

type ImportFieldOption struct {
	Value string `json:"value"`
	Label string `json:"label"`
}

ImportFieldOption is a choice in a mapping select.

type ImportFieldRow

type ImportFieldRow struct {
	Source  string              `json:"source"`
	Label   string              `json:"label"`
	Samples []string            `json:"samples"`
	Target  string              `json:"target"`
	Type    string              `json:"type" doc:"What ReputeCast stores: email, text, number, date, timezone, language, consent, tags, or – when skipped"`
	Status  string              `json:"status" enum:"mapped,new,skipped"`
	Options []ImportFieldOption `json:"options"`
}

ImportFieldRow is one row of the mapping table.

type ImportFile

type ImportFile struct {
	ContentType        string `header:"Content-Type"`
	ContentDisposition string `header:"Content-Disposition"`
	CacheControl       string `header:"Cache-Control"`
	Body               []byte
}

ImportFile is a CSV download.

type ImportInvalidGroup

type ImportInvalidGroup struct {
	Problem string `json:"problem"`
	Example string `json:"example,omitempty"`
	Rows    int64  `json:"rows"`
	// Service names the verification service that rejected these rows.
	Service string `json:"service,omitempty"`
}

ImportInvalidGroup is a kind of invalid address and how many rows have it.

type ImportJob

type ImportJob struct {
	ID   string `json:"id"`
	Kind string `json:"kind" enum:"import,verification,export,optin"`
	// Title: "Importing 13,739 contacts from Mailchimp".
	Title  string `json:"title"`
	Status string `json:"status" enum:"queued,running,done,failed,canceled"`
	// Progress is 0–100 while running.
	Progress *int `json:"progress,omitempty"`
	// Eta: "about 1 min left".
	Eta string `json:"eta,omitempty"`
	// Detail is the result or the failure in words.
	Detail string `json:"detail,omitempty"`
	// Href is the app path of the import's progress or report.
	Href string `json:"href"`
	// DownloadURL fetches a finished export's file (its starter only).
	DownloadURL string     `json:"downloadUrl,omitempty"`
	StartedAt   *time.Time `json:"startedAt,omitempty"`
	FinishedAt  *time.Time `json:"finishedAt,omitempty"`
}

ImportJob is an import shown in the background-jobs tray: running ones with their progress, and ones that ended in the last two hours.

type ImportJobList

type ImportJobList struct {
	Items []ImportJob `json:"items"`
}

ImportJobList is the jobs tray's list.

type ImportLanguageColumn

type ImportLanguageColumn struct {
	// Normalized are values written differently from their code ("Spanish"
	// → es), most rows first; at most five.
	Normalized []ImportLanguageValue `json:"normalized"`
	// Unknown are values that aren't languages, most rows first; at most
	// five, UnknownValues counts them all and UnknownRows their rows.
	Unknown       []ImportLanguageValue `json:"unknown"`
	UnknownValues int                   `json:"unknownValues"`
	UnknownRows   int64                 `json:"unknownRows"`
	// Languages counts the rows per language, by English name, most first.
	Languages []ImportLanguageValue `json:"languages"`
}

ImportLanguageColumn says what importing a column into the contact's language does: the values that change when normalized, and the values that aren't languages, which are left empty.

type ImportLanguageValue

type ImportLanguageValue struct {
	Value string `json:"value"`
	// Code is what the value becomes ("es-MX"); empty for values that
	// aren't languages.
	Code string `json:"code"`
	Name string `json:"name"`
	Rows int64  `json:"rows"`
}

ImportLanguageValue is one value of a language column and its rows.

type ImportList

type ImportList struct {
	Items []ImportSummary `json:"items"`
	// Active is the draft or running import the wizard resumes.
	Active string `json:"active,omitempty"`
}

ImportList is the import history.

type ImportListAfter

type ImportListAfter struct {
	Name     string `json:"name"`
	Contacts int64  `json:"contacts"`
}

ImportListAfter is a target list and its size after the import.

type ImportListChoice

type ImportListChoice struct {
	SourceID string `json:"sourceId"`
	Target   string `json:"target" doc:"List id, or new:<name>"`
}

ImportListChoice maps a source list to a ReputeCast list.

type ImportLogEntry

type ImportLogEntry struct {
	At      time.Time `json:"at"`
	Kind    string    `json:"kind" enum:"start,running,done,sync,retry,error,info"`
	Message string    `json:"message"`
}

ImportLogEntry is one line of the live log.

type ImportOptions

type ImportOptions struct {
	// Lists are the source lists to import and where each goes: a list id
	// or "new:<name>" to create one.
	Lists            []ImportListChoice `json:"lists"`
	ImportTags       bool               `json:"importTags"`
	ImportEngagement bool               `json:"importEngagement"`
	PullSuppressions bool               `json:"pullSuppressions"`
	RecordHistory    bool               `json:"recordHistory"`
	// Evidence: trust (keep the source's opt-in as evidence) or
	// repermission (import as pending and send a confirmation email).
	Evidence string `json:"evidence" enum:"trust,repermission"`
	// OnConflict for contacts that already exist: fill (empty fields only),
	// overwrite, or keep (ReputeCast values win).
	OnConflict string             `json:"onConflict" enum:"fill,overwrite,keep"`
	Risky      ImportRiskyActions `json:"risky"`
	// SkipRows are file lines left out (rows with the wrong column count).
	SkipRows []int `json:"skipRows"`
	// Pace limits requests to the source, per second (0: as fast as it
	// answers); "Slow down" after a rate limit sets 3.
	Pace int `json:"pace,omitempty" minimum:"0" maximum:"50"`
	// ImportTemplates brings the source's email templates along (Notifuse):
	// transactional ones as drafts, the others as saved templates.
	ImportTemplates bool `json:"importTemplates"`
	// Tag goes on every contact the import brings in or updates
	// ("mailchimp-2026-09"), lower-cased (review 2026-11 AU-10).
	Tag string `json:"tag,omitempty" maxLength:"60"`
}

ImportOptions are the choices made across the wizard.

type ImportPathInput

type ImportPathInput struct {
	api.WorkspaceInput
	ID string `path:"id" format:"uuid"`
}

type ImportPreview

type ImportPreview struct {
	Rows        int64 `json:"rows"`
	WillImport  int64 `json:"willImport" doc:"New contacts, subscribed"`
	Merged      int64 `json:"merged" doc:"Already in ReputeCast"`
	Suppressed  int64 `json:"suppressed" doc:"Unsubscribed, bounced or complained in the source"`
	Invalid     int64 `json:"invalid"`
	NotImported int64 `json:"notImported" doc:"Statuses mapped to Skip"`
	Risky       int64 `json:"risky"`
	// ImportCount and SuppressCount label the import button.
	ImportCount   int64                    `json:"importCount"`
	SuppressCount int64                    `json:"suppressCount"`
	RiskyGroups   []ImportRiskyGroup       `json:"riskyGroups"`
	Duplicates    []ImportDuplicateExample `json:"duplicates"`
	InvalidGroups []ImportInvalidGroup     `json:"invalidGroups"`
	OnConflict    string                   `json:"onConflict" enum:"fill,overwrite,keep"`
	ComputedAt    time.Time                `json:"computedAt"`
	// Verification is a verification service's check of the new
	// addresses, when one ran (X14).
	Verification *ImportVerification `json:"verification,omitempty"`
}

ImportPreview is the dry run: what would happen to every row.

type ImportProgress

type ImportProgress struct {
	Total      int64 `json:"total"`
	Processed  int64 `json:"processed"`
	Batch      int   `json:"batch"`
	Batches    int   `json:"batches"`
	BatchSize  int   `json:"batchSize"`
	Imported   int64 `json:"imported"`
	Merged     int64 `json:"merged"`
	Suppressed int64 `json:"suppressed"`
	Pending    int64 `json:"pending"`
	Skipped    int64 `json:"skipped"`
}

ImportProgress is the state of a running import.

type ImportProviderPulled

type ImportProviderPulled struct {
	Name   string `json:"name"`
	Short  string `json:"short"`
	Pulled int64  `json:"pulled"`
}

ImportProviderPulled is a provider and how many suppressions came from it.

type ImportRiskyActions

type ImportRiskyActions struct {
	Role       string `json:"role" enum:"exclude_broadcasts,skip,import"`
	Disposable string `json:"disposable" enum:"skip,import"`
	Typo       string `json:"typo" enum:"fix,skip,import"`
	Stale      string `json:"stale" enum:"pending,skip,import"`
	Trap       string `json:"trap" enum:"skip,import"`
	// A verification service's risky rows (X14); empty imports them, tagged.
	CatchAll string `json:"catch_all,omitempty" enum:"import,skip,"`
	Abuse    string `json:"abuse,omitempty" enum:"import,skip,"`
	Unknown  string `json:"unknown,omitempty" enum:"import,skip,"`
}

ImportRiskyActions say what happens to each risky group.

type ImportRiskyGroup

type ImportRiskyGroup struct {
	Key      string              `json:"key" enum:"role,disposable,typo,stale,trap,catch_all,abuse,unknown"`
	Label    string              `json:"label"`
	Rows     int64               `json:"rows"`
	Why      string              `json:"why"`
	Examples string              `json:"examples"`
	Action   string              `json:"action"`
	Actions  []ImportFieldOption `json:"actions"`
	// After a verification service's check (X14): the rows ReputeCast
	// found (Own), those the service alone found (More) and those where
	// both found the same (Agree, part of Own).
	Own   int64 `json:"own"`
	More  int64 `json:"more"`
	Agree int64 `json:"agree"`
}

ImportRiskyGroup is one group of risky addresses with the chosen action.

type ImportSaveConsentInput

type ImportSaveConsentInput struct {
	Map              ImportConsentMap `json:"map"`
	Evidence         string           `json:"evidence" enum:"trust,repermission"`
	PullSuppressions bool             `json:"pullSuppressions"`
	RecordHistory    bool             `json:"recordHistory"`
	// Tag goes on every contact the import brings in (review 2026-11
	// AU-10); empty for none.
	Tag string `json:"tag,omitempty" maxLength:"60"`
}

ImportSaveConsentInput is the consent step's choices.

type ImportSaveFieldsInput

type ImportSaveFieldsInput struct {
	Fields []ImportFieldChoice `json:"fields" maxItems:"500"`
}

ImportSaveFieldsInput is the mapping chosen by the person.

type ImportSmsConsent

type ImportSmsConsent struct {
	// ConsentMapped: a column maps to SMS consent; EvidenceMapped: its
	// date, source and text columns too.
	ConsentMapped  bool `json:"consentMapped"`
	EvidenceMapped bool `json:"evidenceMapped"`
	// Missing are the evidence columns still to map: date, source, text.
	Missing []string `json:"missing"`
}

ImportSmsConsent is the Consent step's SMS line: SMS consent comes in only with its date, source and text.

type ImportSourceList

type ImportSourceList struct {
	ID            string   `json:"id"`
	Name          string   `json:"name"`
	Subscribed    int64    `json:"subscribed"`
	Unsubscribed  int64    `json:"unsubscribed"`
	Cleaned       int64    `json:"cleaned" doc:"Hard bounced in the source"`
	Pending       int64    `json:"pending"`
	Transactional int64    `json:"transactional"`
	Complained    int64    `json:"complained"`
	Fields        int      `json:"fields"`
	Tags          int      `json:"tags"`
	TagExamples   []string `json:"tagExamples"`
	DoubleOptIn   bool     `json:"doubleOptIn"`
	// Since is the year the list was created ("double opt-in was on in this audience since 2024").
	Since string `json:"since,omitempty"`
}

ImportSourceList is an audience, list or group in the source.

type ImportStats

type ImportStats struct {
	Rows         int64            `json:"rows"`
	Imported     int64            `json:"imported"`
	Merged       int64            `json:"merged"`
	Suppressed   int64            `json:"suppressed"`
	SuppressedBy map[string]int64 `json:"suppressedBy" doc:"unsubscribe, hard_bounce, complaint"`
	Pending      int64            `json:"pending"`
	Skipped      int64            `json:"skipped"`
	// Reasons counts skipped and changed rows: invalid, disposable, trap,
	// already_suppressed, typo_fixed, role.
	Reasons map[string]int64 `json:"reasons"`
	// Lists are the target lists and their size after the import.
	Lists        []ImportListAfter `json:"lists"`
	DurationMs   int64             `json:"durationMs"`
	PushedTo     []string          `json:"pushedTo" doc:"Providers the new suppressions were pushed to"`
	ConfirmsSent int64             `json:"confirmsSent" doc:"Re-permission emails sent through the System stream"`
	// Templates the import brought along (Notifuse).
	Templates *ImportTemplateStats `json:"templates,omitempty"`
	// Texts counts the rows' numbers and SMS consent (phase 8): number,
	// consent, no_evidence (SMS consent without its date, source and text:
	// Not subscribed) and invalid_phone.
	Texts map[string]int64 `json:"texts,omitempty"`
}

ImportStats is the summary of a finished import.

type ImportSummary

type ImportSummary struct {
	ID         string     `json:"id"`
	Source     string     `json:"source"`
	SourceName string     `json:"sourceName"`
	Status     string     `json:"status"`
	Step       string     `json:"step"`
	FileName   string     `json:"fileName"`
	Contacts   int64      `json:"contacts" doc:"Contacts imported, merged or imported as pending"`
	CreatedAt  time.Time  `json:"createdAt"`
	FinishedAt *time.Time `json:"finishedAt,omitempty"`
	CreatedBy  string     `json:"createdBy"`
}

ImportSummary is one import in the history.

type ImportTemplateCounts

type ImportTemplateCounts struct {
	// Transactional templates arrive as transactional drafts.
	Transactional int `json:"transactional"`
	// Saved are marketing and other templates, added to the composer's
	// saved templates.
	Saved int `json:"saved"`
	// Skipped are the source's system emails (double opt-in, unsubscribe,
	// bounce): ReputeCast sends its own.
	Skipped int `json:"skipped"`
}

ImportTemplateCounts are the email templates a source brings along.

func (*ImportTemplateCounts) String

func (c *ImportTemplateCounts) String() string

type ImportTemplateStats

type ImportTemplateStats struct {
	Transactional int `json:"transactional" doc:"Transactional templates imported as drafts"`
	Saved         int `json:"saved" doc:"Marketing and other templates added to the saved templates"`
	// Manual counts the imported templates whose report asks for a manual fix.
	Manual int `json:"manual"`
	// Skipped are the source's system emails (double opt-in, unsubscribe…).
	Skipped int `json:"skipped"`
}

ImportTemplateStats are the templates an import brought along.

type ImportUpdateInput

type ImportUpdateInput struct {
	Source  string         `json:"source,omitempty" enum:"brevo,mailchimp,mailerlite,listmonk,keila,notifuse,csv,xlsx,"`
	Options *ImportOptions `json:"options,omitempty"`
	Step    string         `json:"step,omitempty" enum:"source,fields,consent,preview,"`
}

ImportUpdateInput changes the draft: its source (before connecting), options and the step the person is on.

type ImportVerification

type ImportVerification struct {
	JobID       string     `json:"jobId"`
	Service     string     `json:"service"`
	ServiceName string     `json:"serviceName"`
	Status      string     `json:"status"`
	CheckedAt   *time.Time `json:"checkedAt,omitempty"`
	Checked     int64      `json:"checked"`
	Cost        float64    `json:"cost"`
	// Invalid rows the service rejected (skipped); Risky the rows it alone
	// found risky or couldn't answer; Fewer how many fewer new contacts.
	Invalid int64 `json:"invalid"`
	Risky   int64 `json:"risky"`
	Fewer   int64 `json:"fewer"`
}

ImportVerification is the preview's line about the service's check.

type JobSource

type JobSource func(ctx context.Context, ws *store.Workspace) ([]ImportJob, error)

JobSource adds another domain's jobs to the tray (address verification).

type Keila

type Keila struct {
	// contains filtered or unexported fields
}

Keila reads contacts through a Keila server's API v1 (read-only). Keila has no lists, so everything comes in as one "All contacts" list.

func NewKeila

func NewKeila(baseURL, key string, hc *http.Client) (*Keila, error)

NewKeila builds a client for a Keila server (https://app.keila.io or self-hosted).

func (*Keila) Connect

func (k *Keila) Connect(ctx context.Context) (ImportConnection, error)

Connect implements Source.

func (*Keila) Fetch

func (k *Keila) Fetch(ctx context.Context, _ []string, emit func(Row) error) error

Fetch implements Source.

func (*Keila) Fields

func (k *Keila) Fields(ctx context.Context, _ []string) ([]SourceField, error)

Fields implements Source: built-in fields plus the custom data keys of the first page.

type Listmonk

type Listmonk struct {
	// contains filtered or unexported fields
}

Listmonk reads lists and subscribers through a listmonk server's API with an API user and token (read-only).

func NewListmonk

func NewListmonk(baseURL, user, token string, hc *http.Client) (*Listmonk, error)

NewListmonk builds a client for a listmonk server.

func (*Listmonk) Connect

func (l *Listmonk) Connect(ctx context.Context) (ImportConnection, error)

Connect implements Source.

func (*Listmonk) Fetch

func (l *Listmonk) Fetch(ctx context.Context, lists []string, emit func(Row) error) error

Fetch implements Source.

func (*Listmonk) Fields

func (l *Listmonk) Fields(ctx context.Context, lists []string) ([]SourceField, error)

Fields implements Source: email, name and the attribute keys found on the first page of subscribers.

type MXChecker

type MXChecker = mailbox.MXChecker

MXChecker tells whether a domain accepts mail, caching answers.

type MXResult

type MXResult = mailbox.MXResult

MXResult is what a domain's DNS says.

type Mailchimp

type Mailchimp struct {
	DC string
	// contains filtered or unexported fields
}

Mailchimp reads audiences, merge fields, tags and members through the Marketing API v3 (read-only: nothing is written to Mailchimp).

func NewMailchimp

func NewMailchimp(apiKey, base string, hc *http.Client) (*Mailchimp, error)

NewMailchimp builds a client from an API key ("…-us21"); base overrides the API URL (tests).

func (*Mailchimp) Connect

func (m *Mailchimp) Connect(ctx context.Context) (ImportConnection, error)

Connect implements Source.

func (*Mailchimp) Fetch

func (m *Mailchimp) Fetch(ctx context.Context, lists []string, emit func(Row) error) error

Fetch implements Source. Members who reported abuse come from the abuse reports and are marked complained.

func (*Mailchimp) Fields

func (m *Mailchimp) Fields(ctx context.Context, lists []string) ([]SourceField, error)

Fields implements Source: EMAIL, the merge fields, and the member properties ReputeCast can use (timezone, opt-in evidence, tags).

type MailerLite

type MailerLite struct {
	// contains filtered or unexported fields
}

MailerLite reads groups, fields and subscribers through the MailerLite API (connect.mailerlite.com, read-only).

func NewMailerLite

func NewMailerLite(token, base string, hc *http.Client) (*MailerLite, error)

NewMailerLite builds a client from an API token; base overrides the API URL.

func (*MailerLite) Connect

func (m *MailerLite) Connect(ctx context.Context) (ImportConnection, error)

Connect implements Source.

func (*MailerLite) Fetch

func (m *MailerLite) Fetch(ctx context.Context, lists []string, emit func(Row) error) error

Fetch implements Source.

func (*MailerLite) Fields

func (m *MailerLite) Fields(ctx context.Context, _ []string) ([]SourceField, error)

Fields implements Source.

type Notifuse

type Notifuse struct {
	// contains filtered or unexported fields
}

Notifuse reads contacts and lists from a self-hosted Notifuse instance with a workspace API key (read-only use). The key finds its workspace through workspaces.list. Templates come in with the importer's template step (TemplateImporter): transactional ones as drafts.

func NewNotifuse

func NewNotifuse(baseURL, key string, hc *http.Client) (*Notifuse, error)

NewNotifuse builds a client for the Notifuse instance at baseURL.

func (*Notifuse) Connect

func (n *Notifuse) Connect(ctx context.Context) (ImportConnection, error)

Connect implements Source: the workspace, its lists with counts per status, and its email templates.

func (*Notifuse) Fetch

func (n *Notifuse) Fetch(ctx context.Context, lists []string, emit func(Row) error) error

Fetch implements Source: every contact of each list, a page of 100 at a time, with its status on that list.

func (*Notifuse) Fields

func (n *Notifuse) Fields(ctx context.Context, lists []string) ([]SourceField, error)

Fields implements Source: the built-in fields, then the custom fields set on the first page of each list.

type Resolver

type Resolver = mailbox.Resolver

Resolver is the part of net.Resolver the MX check uses.

type Row

type Row struct {
	Email      string
	Status     string
	List       string
	Data       map[string]string
	Tags       []string
	Engagement Engagement
}

Row is one contact read from the source.

type RunArgs

type RunArgs struct {
	ImportID uuid.UUID `json:"importId"`
}

RunArgs imports the staged rows of an import.

func (RunArgs) InsertOpts

func (RunArgs) InsertOpts() river.InsertOpts

InsertOpts implements river.JobArgsWithInsertOpts.

func (RunArgs) Kind

func (RunArgs) Kind() string

Kind implements river.JobArgs.

type RunWorker

type RunWorker struct {
	river.WorkerDefaults[RunArgs]
	Svc *Service
}

RunWorker runs RunArgs.

func (*RunWorker) Timeout

func (w *RunWorker) Timeout(*river.Job[RunArgs]) time.Duration

Timeout allows large imports.

func (*RunWorker) Work

func (w *RunWorker) Work(ctx context.Context, job *river.Job[RunArgs]) error

Work implements river.Worker.

type Sealer

type Sealer interface {
	Seal(plaintext []byte, ad string) ([]byte, error)
	Open(ciphertext []byte, ad string) ([]byte, error)
}

Sealer encrypts and decrypts importer credentials.

type Service

type Service struct {
	Pool     *pgxpool.Pool
	Q        *store.Queries
	Audit    *audit.Logger
	Box      Sealer
	Audience *audience.Service
	Mail     mailer.Sender
	// Jobs returns the River client (created after the services are wired).
	Jobs func() *jobs.Client
	MX   *MXChecker
	// HTTP is used by the API importers; BaseURLs overrides their API base
	// per source (tests). Guard decides which addresses a self-hosted
	// server (listmonk, Keila) may have: the client refuses the same ones
	// at connect time.
	HTTP     *http.Client
	Guard    netguard.Policy
	BaseURLs map[string]string
	Now      func() time.Time
	// Inline runs staging and imports in the request instead of River (tests).
	Inline bool
	// Templates imports a source's email templates (Notifuse); nil skips
	// the step.
	Templates TemplateImporter
	// Texts records numbers and SMS consent (phase 8); nil imports neither.
	Texts Texts
	// JobSources add other domains' jobs to the background-jobs tray.
	JobSources []JobSource
	// Checks gives a verification service's answers for an import's new
	// addresses (X14); nil skips them.
	Checks AddressChecks
}

func (*Service) BackgroundJobs

func (s *Service) BackgroundJobs(ctx context.Context, ws *store.Workspace) (ImportJobList, error)

BackgroundJobs lists the imports for the jobs tray, in the request's language.

func (*Service) BadRowsCSV

func (s *Service) BadRowsCSV(ctx context.Context, ws *store.Workspace, id uuid.UUID) ([]byte, string, error)

BadRowsCSV returns the rows a file import couldn't read, for download.

func (*Service) Cancel

func (s *Service) Cancel(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportDetail, error)

Cancel stops an import; rows already imported stay.

func (*Service) Connect

Connect checks the key against the source, stores it encrypted and returns the account and its lists.

func (*Service) Consent

Consent returns the consent step: counts per source status and the handling of each, in the request's language. The statuses keep the source's own words (subscribed, cleaned).

func (*Service) Create

Create makes a draft import.

func (*Service) Delete

func (s *Service) Delete(ctx context.Context, ws *store.Workspace, id uuid.UUID) error

Delete discards a draft import and its staged rows.

func (*Service) FieldGuesses

func (s *Service) FieldGuesses(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportFieldMapping, error)

FieldGuesses is the mapping step with every target guessed again ("Reset to guesses"); nothing is saved until the mapping is.

func (*Service) Fields

func (s *Service) Fields(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportFieldMapping, error)

Fields returns the mapping step: each source field with sample values, its current or guessed target and the choices.

func (*Service) Get

func (s *Service) Get(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportDetail, error)

Get returns one import with its connection, progress and log.

func (*Service) InvalidCSV

func (s *Service) InvalidCSV(ctx context.Context, ws *store.Workspace, id uuid.UUID) ([]byte, string, error)

InvalidCSV lists the invalid rows of the preview ("Download these rows").

func (*Service) LanguageColumn

func (s *Service) LanguageColumn(ctx context.Context, ws *store.Workspace, id uuid.UUID, field string) (ImportLanguageColumn, error)

LanguageColumn reads a source field as a language column: what its values become and which aren't languages.

func (*Service) List

func (s *Service) List(ctx context.Context, ws *store.Workspace) (ImportList, error)

List returns the imports of a workspace, newest first.

func (*Service) NewAddresses

func (s *Service) NewAddresses(ctx context.Context, ws *store.Workspace, id uuid.UUID) (string, []string, error)

NewAddresses returns the addresses the import would add as new contacts: the preview's rows to import and its risky rows (with typo fixes), each once; and the import's name as the use reads it (Mailchimp “Acme Customers”).

func (*Service) Preview

func (s *Service) Preview(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportPreview, error)

Preview runs the dry run over every staged row, its words in the request's language. Nothing is written except the stored preview.

func (*Service) ReportCSV

func (s *Service) ReportCSV(ctx context.Context, ws *store.Workspace, id uuid.UUID, kind string) ([]byte, string, error)

ReportCSV returns one of the import's CSV files.

func (*Service) RequestStage

func (s *Service) RequestStage(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportDetail, error)

RequestStage starts reading rows for the selected lists, unless they are already staged. It moves the draft to the field mapping step.

func (*Service) Resume

func (s *Service) Resume(ctx context.Context, ws *store.Workspace, id uuid.UUID, slow bool) (ImportDetail, error)

Resume restarts a paused import now ("Resume now"). With slow, requests to the source are paced to 3 a second from now on ("Slow down to 3 req/s").

func (*Service) Run

func (s *Service) Run(ctx context.Context, id uuid.UUID) error

Run imports every staged row in batches, with a live log.

func (*Service) SaveConsent

SaveConsent stores the consent mapping and moves on to the preview.

func (*Service) SaveFields

SaveFields stores the mapping and moves on to consent.

func (*Service) SkipBadRows

func (s *Service) SkipBadRows(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportDetail, error)

SkipBadRows leaves out the file rows whose column count doesn't match the header ("Skip the 4 rows and continue").

func (*Service) Stage

func (s *Service) Stage(ctx context.Context, id uuid.UUID) error

Stage reads every row of the selected lists into import_rows.

func (*Service) Start

func (s *Service) Start(ctx context.Context, ws *store.Workspace, id uuid.UUID) (ImportDetail, error)

Start begins the import of the staged rows.

func (*Service) Update

Update saves changes to a draft.

func (*Service) Upload

func (s *Service) Upload(ctx context.Context, ws *store.Workspace, id uuid.UUID, name string, data []byte) (ImportDetail, error)

Upload stores a CSV or XLSX file on the draft and reads its header.

type Source

type Source interface {
	// Connect checks access and returns the account and its lists with
	// counts per consent status.
	Connect(ctx context.Context) (ImportConnection, error)
	// Fields returns the source's fields for the selected lists.
	Fields(ctx context.Context, lists []string) ([]SourceField, error)
	// Fetch streams every contact of the selected lists, with its status.
	Fetch(ctx context.Context, lists []string, emit func(Row) error) error
}

Source is where contacts are imported from: an email tool's API or a file.

type SourceError

type SourceError struct {
	Status  int
	Code    string // auth_failed, rate_limited, unavailable, bad_response
	Message string
	Text    i18n.Text
	// Reason is the source's own short words ("API Key Invalid").
	Reason string
	// RetryAfter is set for rate limits.
	RetryAfter time.Duration
}

SourceError is a failure talking to the source, explained for people: Message in English, Text the message it came from for readers of other languages.

func (*SourceError) Error

func (e *SourceError) Error() string

type SourceField

type SourceField struct {
	Key   string `json:"key"`
	Label string `json:"label"`
	// Type as the source reports it: text, number, date, email, phone,
	// address, birthday, url, boolean.
	Type string `json:"type"`
}

SourceField is a field (merge field, attribute, column) in the source.

type StageArgs

type StageArgs struct {
	ImportID uuid.UUID `json:"importId"`
}

StageArgs reads an import's rows from its source into import_rows.

func (StageArgs) InsertOpts

func (StageArgs) InsertOpts() river.InsertOpts

InsertOpts implements river.JobArgsWithInsertOpts.

func (StageArgs) Kind

func (StageArgs) Kind() string

Kind implements river.JobArgs.

type StageWorker

type StageWorker struct {
	river.WorkerDefaults[StageArgs]
	Svc *Service
}

StageWorker runs StageArgs.

func (*StageWorker) Timeout

func (w *StageWorker) Timeout(*river.Job[StageArgs]) time.Duration

Timeout allows long API reads.

func (*StageWorker) Work

func (w *StageWorker) Work(ctx context.Context, job *river.Job[StageArgs]) error

Work implements river.Worker.

type TemplateImporter

type TemplateImporter interface {
	ImportNotifuseTemplates(ctx context.Context, ws *store.Workspace, by *uuid.UUID, baseURL, apiKey string) (ImportTemplateStats, error)
}

TemplateImporter brings a source's email templates along with its contacts (Notifuse, add-on transactional §6): transactional templates arrive as transactional drafts, marketing and other templates as saved templates in the composer. The transactional template importer implements it.

type TextOutcome

type TextOutcome string

TextOutcome is what a row's texts came to.

const (
	TextNumber     TextOutcome = "number"        // the number, no SMS consent asked for
	TextConsent    TextOutcome = "consent"       // the number and SMS consent with its evidence
	TextNoEvidence TextOutcome = "no_evidence"   // SMS consent without its evidence: Not subscribed
	TextInvalid    TextOutcome = "invalid_phone" // not a phone number
)

Text outcomes, counted in ImportStats.Texts.

type TextRow

type TextRow struct {
	Phone   string
	Country string
	// Consent is the SMS consent column; At, Source and Text its evidence.
	Consent bool
	At      *time.Time
	Source  string
	Text    string
	// ImportID and File name the import, kept with the consent record.
	ImportID uuid.UUID
	File     string
	// Overwrite replaces a number the contact has (the conflict rule).
	Overwrite bool
}

TextRow is a row's number and SMS consent columns.

type Texts

type Texts interface {
	// Import records a row's number on the contact and, with its
	// evidence, SMS consent on the list, in the run's transaction.
	Import(ctx context.Context, tx pgx.Tx, ws *store.Workspace, contactID, listID uuid.UUID, row TextRow) (TextOutcome, error)
	// DefaultCountry is the country numbers without a country code are
	// read in when the row has no country column: the workspace's first
	// routed country, or "".
	DefaultCountry(ctx context.Context, ws *store.Workspace) string
}

Texts records imported numbers and SMS consent (internal/texting, which the app wires in); nil imports neither.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL