Help us shape the future of Go! Take the 2026 Go Developer Survey

wpimport

package
v1.0.2 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Oct 28, 2026 License: Apache-2.0 Imports: 42 Imported by: 0

Documentation

Overview

Package wpimport imports a WordPress site into Hatamoto from its REST API or a WXR export: scan report, runner (settings, terms, authors, media, pages, posts, custom post types, menus, redirects, link rewrite) and URL audit. See docs/IMPORT-WORDPRESS.md.

Index

Constants

View Source
const (
	DefaultMaxTotalBytes = 2 << 30
	DefaultMaxPages      = 2000
	// DefaultMaxCollection is the item cap of one REST collection (terms,
	// users, media, menu items).
	DefaultMaxCollection = 100_000
)

Defaults of the bounds on what a source site can make the importer read.

View Source
const (
	KindEntity   = "entity" // posts, pages and custom post types
	KindTerm     = "term"
	KindAuthor   = "author"
	KindMedia    = "media"
	KindMenu     = "menu"
	KindRedirect = "redirect"
)

Kinds of the id map.

View Source
const (
	RunRunning   = "running"
	RunCompleted = "completed"
	RunFailed    = "failed"
	RunScanned   = "scanned"
)

Run statuses.

View Source
const (
	DefaultWXRMaxBytes = 512 << 20
	DefaultMaxItems    = 50000
	// MaxWXRDepth is how deep elements may nest. A real export is five or six
	// levels deep; encoding/xml keeps every open element on a stack with no
	// limit of its own, so a crafted file would otherwise cost memory far
	// beyond its size.
	MaxWXRDepth = 64
	// DefaultMaxWXRElements bounds the total number of elements read.
	DefaultMaxWXRElements = 20_000_000
)

WXR limits.

View Source
const SourceName = "wordpress"

SourceName is the import source label stored in runs, the id map and entity metadata.

Variables

View Source
var ErrBudget = errors.New("the source sent more data than the import allows")

ErrBudget is returned when the source has sent more than the byte budget.

View Source
var ErrImportRunning = errors.New("another import is running")

ErrImportRunning is returned when another import (the wizard's, or the command line's, in this process or another) already holds the site's import lock.

View Source
var ErrPagination = errors.New("pagination of the source does not end")

ErrPagination is returned when a collection does not end or does not advance.

View Source
var ErrTooComplex = errors.New("export is too deeply nested or has too many elements")

ErrTooComplex is returned when an export nests too deeply or has too many elements.

View Source
var ErrTooLarge = errors.New("export file is over the size limit")

ErrTooLarge is returned when an upload goes over the size cap.

Functions

func AcquireImportLock

func AcquireImportLock(ctx context.Context, pool *pgxpool.Pool) (release func(), err error)

AcquireImportLock takes the session-level advisory lock that makes an import exclusive for the target database, without waiting. Hold it for the whole job and call release when it ends. It returns ErrImportRunning when taken.

func DetectBuilders

func DetectBuilders(s string) []string

DetectBuilders returns the page builders whose markers occur in s.

func ImportRunning

func ImportRunning(ctx context.Context, pool *pgxpool.Pool) (bool, error)

ImportRunning reports whether an import currently holds the import lock. It takes the lock without waiting and gives it straight back, so it never blocks an import for longer than the probe itself.

func InferPermalink(link, slug, id string, date time.Time) string

InferPermalink derives the WordPress permalink structure ("/%year%/%monthnum%/%postname%/") from the link of one post, its slug, date and id. It returns "" when the link does not end in the slug (plain "?p=123" permalinks, or a post whose slug was changed after the link).

func IsWPContent

func IsWPContent(s string) bool

IsWPContent reports whether a URL points into wp-content or wp-includes.

func KeyPrefix

func KeyPrefix(siteURL string) string

KeyPrefix is the prefix of the import keys of one source site.

func ParseSitemap

func ParseSitemap(data []byte, limit int) (kind string, locs []string, err error)

ParseSitemap returns "index" or "urlset" and the <loc> of each entry, reading the XML as a stream and stopping at limit entries. A limit of zero or less reads no entries (it is a budget that is used up, never "unlimited").

func PatternFromPermalink(structure string) (pattern string, ok bool)

PatternFromPermalink converts a WordPress structure to a Hatamoto URL pattern ("/{year}/{month}/{slug}"). ok is false when the structure uses something a pattern cannot express (post ids, categories, authors); the importer then keeps the schema's pattern and redirects every old URL.

func WPContentURLs

func WPContentURLs(body string) []string

WPContentURLs returns every wp-content or wp-includes URL left in body, in attributes or text; the importer's "none remain" check.

Types

type AuditFailure

type AuditFailure struct {
	URL      string `json:"url"`
	Status   int    `json:"status"`
	Location string `json:"location,omitempty"`
	Reason   string `json:"reason"`
}

AuditFailure is an old URL that does not resolve.

type AuditResult

type AuditResult struct {
	Checked    int            `json:"checked"`
	OK         int            `json:"ok"`         // answered 200 directly
	Redirected int            `json:"redirected"` // answered one 301 to a 200
	Failures   []AuditFailure `json:"failures,omitempty"`
	// Unresolved are old URLs that answer, but with a page that is not the
	// imported content (see Result.URLConflicts).
	Unresolved []AuditFailure `json:"unresolved,omitempty"`
	// Hidden are old author archives that answer 404 on purpose: the person
	// imported for that author is not public yet. Making the person public
	// gives them a page; a re-run then redirects the old URL to it.
	Hidden []AuditFailure `json:"hidden,omitempty"`
}

AuditResult is the outcome of checking every old URL.

func Audit

func Audit(ctx context.Context, siteURL string, oldURLs []string, p Prober) *AuditResult

Audit checks that every old URL of the site answers 200, or exactly one permanent redirect to a page that answers 200.

type Author

type Author struct {
	Key       string // login or slug: what items refer to
	Name      string
	Slug      string
	Bio       string
	Link      string // old author archive URL
	AvatarURL string // only set for avatars hosted on the site itself
}

Author is a WordPress user shown as a byline.

type Client

type Client struct {
	Base *url.URL
	// contains filtered or unexported fields
}

Client is a rate-limited, retrying, SSRF-guarded HTTP client for the old site.

func NewClient

func NewClient(o ClientOptions) (*Client, error)

NewClient validates the URL and builds the client.

func (*Client) Authed

func (c *Client) Authed() bool

Authed reports whether credentials were given.

func (*Client) Do

func (c *Client) Do(ctx context.Context, rawURL string, maxBytes int64) ([]byte, http.Header, error)

Do GETs rawURL with the limiter, retries and the size cap, and returns the body and headers. Credentials are only sent to the site's own host.

func (*Client) Download

func (c *Client) Download(ctx context.Context, rawURL, dir string, maxBytes int64) (path string, size int64, sum string, err error)

Download fetches a file into a temp file inside dir, enforcing maxBytes, and returns its path, size and sha256. The caller removes the file.

func (*Client) Host

func (c *Client) Host() string

Host is the site's host name without port.

func (*Client) JSON

func (c *Client) JSON(ctx context.Context, route string, q url.Values, out any) (http.Header, error)

JSON GETs a REST route and decodes the answer.

func (*Client) Paged

func (c *Client) Paged(ctx context.Context, route string, q url.Values, max int, fn func(json.RawMessage) error) error

Paged reads every page of a REST collection (100 per page) and calls fn for each element. fn gets the raw element so callers decode what they need. max is the number of items allowed (never "unlimited"): an item past it is an error. A collection that claims too many pages, or returns a page equal to the one before (a cache that ignores ?page=), fails with ErrPagination.

type ClientOptions

type ClientOptions struct {
	// URL is the site's home URL.
	URL string
	// User and AppPassword are WordPress application-password credentials;
	// without them only public content is read (no drafts, no menus).
	User, AppPassword string
	// AllowPrivate lets the importer reach a site on a private or loopback
	// address (an intranet WordPress, the test fixture). Off by default: the
	// guard that protects plugin http.fetch applies here too.
	AllowPrivate bool
	// RatePerSec limits requests to the old site; 0 means 10.
	RatePerSec float64
	// MaxRetries is the number of retries after the first attempt; 0 means 4.
	MaxRetries int
	// Timeout is per request; 0 means 30s.
	Timeout time.Duration
	// MaxTotalBytes bounds everything read through Do (REST pages, sitemaps)
	// over the life of the client; 0 means DefaultMaxTotalBytes.
	MaxTotalBytes int64
	// MaxPages is the hard ceiling of pages of one collection; 0 means DefaultMaxPages.
	MaxPages int
	// Lookup overrides name resolution (tests).
	Lookup safefetch.Lookup
	// Sleep overrides the backoff wait (tests).
	Sleep func(ctx context.Context, d time.Duration) error
}

ClientOptions configures the HTTP client that reads the old site.

type Converted

type Converted struct {
	HTML       string
	Shortcodes []string // names of shortcodes found (and removed)
	Forms      []string // form plugins whose markup or shortcode was removed
	Builders   []string // page builders whose markers were found
	Embeds     int      // YouTube and Vimeo embeds kept
	Dropped    []string // other things removed: unsupported iframes
}

Converted is the result of converting one body.

func Convert

func Convert(in string, raw bool) Converted

Convert turns a WordPress body into HTML the rich-text sanitiser keeps whole: block comments go, YouTube and Vimeo embeds become iframes (also from a bare URL), forms and shortcodes are removed and reported, responsive image attributes are dropped. raw is true for WXR bodies, which are stored as typed (shortcodes unexpanded, paragraphs not yet wrapped).

func (Converted) String

func (c Converted) String() string

String summarises the conversion for logs.

type CustomType

type CustomType struct {
	Slug         string
	Name         string
	RESTBase     string
	Hierarchical bool
	HasArchive   bool
}

CustomType is a public custom post type of the source.

type Deps

type Deps struct {
	DB         *database.DB
	Entities   *entity.Service
	SchemaRepo entity.SchemaRepository
	Media      *media.Service
	Taxonomy   *taxonomy.Service
	TermRepo   taxonomy.Repository
	Menus      *menu.Service
	People     *person.Store
	Redirects  *redirect.Service
	Site       *siteconfig.Store
	Logger     *slog.Logger
}

Deps are the Hatamoto services the importer writes through, built the same way the server builds them so an import and an editor save behave alike.

func NewDeps

func NewDeps(db *database.DB, cfg DepsConfig, logger *slog.Logger) *Deps

NewDeps wires the services over db. Call Close when done.

func (*Deps) Close

func (d *Deps) Close()

Close stops the background work of the redirect service.

type DepsConfig

type DepsConfig struct {
	MediaDir     string
	MediaBaseURL string
	MediaMaxMB   int
}

DepsConfig is the media storage configuration.

type Detection

type Detection struct {
	SEOPlugin     string
	PageBuilders  []string
	Forms         []string
	Multilingual  []string
	DefaultLang   string
	OtherPlugins  []string // REST namespaces or plugin markers nothing maps
	Shortcodes    map[string]int
	Namespaces    []string
	CustomTypes   []CustomType
	PermalinkNote string
}

Detection is what the scan found about the old site's plugins.

type Dump

type Dump struct {
	Site    Site
	Authors []Author
	Terms   []Term
	Media   []Media
	Items   []Item
	Menus   []Menu
	// OldURLs are the public URLs the old site lists in its sitemap (REST), or
	// the links of its published items (WXR); the audit checks each.
	OldURLs []string
	// Detect holds what the scan found out about plugins and builders.
	Detect Detection
	// RawBodies is true when item bodies are as typed (WXR): block comments,
	// unexpanded shortcodes, paragraphs not yet wrapped.
	RawBodies bool
	// Warnings are non-fatal problems met while reading.
	Warnings []string
}

Dump is everything read from the source, in one neutral shape. Both sources (REST and WXR) fill it; the scan report and the runner only read it.

func ParseWXR

func ParseWXR(r io.Reader, o WXROptions) (*Dump, error)

ParseWXR reads a WordPress export as a stream and returns the Dump. The file is never held as a whole, but the parsed items are kept until the Dump is built, so memory grows with the size of the content (at most MaxBytes).

type EntityRef

type EntityRef struct {
	ID   uuid.UUID
	Path string // the new public path
}

EntityRef is what an internal link resolves to.

type Fidelity

type Fidelity struct {
	High    int `json:"high"`
	Medium  int `json:"medium"`
	Low     int `json:"low"`
	Percent int `json:"percent"` // share of items imported without loss
}

Fidelity counts items by how faithfully they import.

type HTTPError

type HTTPError struct {
	Status int
	URL    string
	Body   string
}

HTTPError is a non-2xx answer that is not worth retrying.

func (*HTTPError) Error

func (e *HTTPError) Error() string

type HTTPProber

type HTTPProber struct {
	Base   string
	Client *http.Client
}

HTTPProber probes a running site.

func (HTTPProber) Hosts

func (p HTTPProber) Hosts() []string

Hosts implements HostedProber.

func (HTTPProber) Probe

func (p HTTPProber) Probe(ctx context.Context, path string) (int, string, error)

Probe implements Prober.

type HandlerProber

type HandlerProber struct {
	Handler http.Handler
	// Host is the Host header; empty means example.test.
	Host string
}

HandlerProber probes an http.Handler in process (tests, and the admin wizard, which has the server's handler).

func (HandlerProber) Hosts

func (p HandlerProber) Hosts() []string

Hosts implements HostedProber.

func (HandlerProber) Probe

func (p HandlerProber) Probe(ctx context.Context, path string) (int, string, error)

Probe implements Prober.

type HostedProber

type HostedProber interface {
	Prober
	Hosts() []string
}

HostedProber is a Prober that knows the host of the site it probes, so that a redirect to any other host can be told from one to the new site.

type Item

type Item struct {
	ID       string
	Type     string // post, page or a custom post type slug
	Status   string // WordPress status
	Slug     string
	Title    string
	Content  string // HTML as the reader sees it (REST) or raw (WXR)
	Raw      string // raw content with block comments and shortcodes, when known
	Excerpt  string
	Link     string // old public URL
	Date     time.Time
	Modified time.Time
	ParentID string
	// AuthorKey refers to Author.Key.
	AuthorKey string
	// FeaturedMedia is a Media.ID.
	FeaturedMedia string
	Terms         []TermRef
	Protected     bool   // password protected
	Lang          string // language tag as the source gave it (WPML or Polylang); empty when it gave none
	SEO           SEO
	Meta          map[string]string
}

Item is a post, page or custom post type entry.

type ItemError

type ItemError struct {
	Kind  string `json:"kind"`
	ID    string `json:"id,omitempty"`
	Title string `json:"title,omitempty"`
	Err   string `json:"error"`
}

ItemError is one thing that failed, with the reason.

type ItemNote

type ItemNote struct {
	ID     string `json:"id"`
	Type   string `json:"type"`
	Title  string `json:"title"`
	Link   string `json:"link"`
	Reason string `json:"reason"`
}

ItemNote is an item the report points at, with the reason.

type Media

type Media struct {
	ID      string
	URL     string
	Sizes   []string // URLs of generated sizes, mapped to the same media
	Mime    string
	Title   string
	Alt     string
	Caption string
	Date    time.Time
}

Media is an attachment of the media library.

type Menu struct {
	ID       string
	Name     string
	Slug     string
	Location string
	Items    []MenuItem
}

Menu is a navigation menu.

type MenuItem struct {
	ID       string
	ParentID string
	Title    string
	URL      string
	Type     string // post_type, taxonomy or custom
	Object   string // page, post, category, ...
	ObjectID string // item id, or "taxonomy:slug" for terms
	NewTab   bool
	Order    int
}

MenuItem is one entry of a menu.

type Options

type Options struct {
	// SourceRef identifies the source site; it scopes the id map. Set from the Dump when empty.
	SourceRef string
	// Actor is recorded as the creator of imported rows (nil: none).
	Actor *uuid.UUID
	// AllowPrivate lets media be fetched from a private-network site. It turns
	// the SSRF guard off for everything the source names: set it only from an
	// operator's own command line. Never bind it from request data and never
	// restore it from a stored run's options.
	AllowPrivate bool
	// Client downloads media; built from the site URL when nil (WXR).
	Client *Client
	// MediaConcurrency bounds parallel downloads; 0 means 4.
	MediaConcurrency int
	// KeepPermalinks sets the post and page URL patterns to the WordPress
	// structure. Default true (set NoPermalinks to disable).
	NoPermalinks bool
	// WorkDir holds temporary downloads; empty uses the system temp dir.
	WorkDir string
	// Progress receives one line per step.
	Progress func(phase, msg string)
	// Purge clears rendered-page and sitemap caches after the import. The CLI is a
	// separate process and cannot reach the server's caches: it leaves this nil.
	Purge func(ctx context.Context)
	// Audit checks every old URL against the new site; nil skips it.
	Audit Prober
	// KeepEdited leaves entities alone that were edited in Hatamoto after the
	// last import. By default a re-run overwrites them and lists them in the
	// result.
	KeepEdited bool
	// SkipMedia imports no files: the library is skipped and images and
	// attachments inside bodies are removed or unlinked, as for a file that
	// cannot be fetched (no wp-content URL is left behind).
	SkipMedia bool
	// Options are written to the run (no secrets) so --resume can reuse them.
	Stored map[string]any
}

Options configures a run.

type Prober

type Prober interface {
	Probe(ctx context.Context, path string) (status int, location string, err error)
}

Prober asks the new site for a path without following redirects.

type RESTOptions

type RESTOptions struct {
	ClientOptions
	// MaxItems caps posts, pages and custom post types together (0: 50000).
	MaxItems int
	// MaxCollection caps each of terms, users, media and menu items (0: DefaultMaxCollection).
	MaxCollection int
	// MaxDuration bounds the whole read (0: 4 hours).
	MaxDuration time.Duration
	// Concurrency bounds the per-item requests (SEO data); 0 means 4.
	Concurrency int
	// Progress receives short status lines.
	Progress func(string)
}

RESTOptions configures a REST source.

type RESTSource

type RESTSource struct {
	// contains filtered or unexported fields
}

RESTSource reads a site through its WordPress REST API.

func NewRESTSource

func NewRESTSource(o RESTOptions) (*RESTSource, error)

NewRESTSource builds the source; nothing is fetched yet.

func (*RESTSource) Client

func (s *RESTSource) Client() *Client

Client exposes the HTTP client (media downloads and the audit reuse it).

func (*RESTSource) Load

func (s *RESTSource) Load(ctx context.Context) (*Dump, error)

Load reads the whole site.

func (*RESTSource) Ref

func (s *RESTSource) Ref() string

Ref identifies the source in import runs.

type Report

type Report struct {
	Source string `json:"source"`
	Site   Site   `json:"site"`

	Posts       int            `json:"posts"`
	Pages       int            `json:"pages"`
	CustomTypes map[string]int `json:"custom_types,omitempty"`
	Drafts      int            `json:"drafts"`
	Private     int            `json:"private"`
	Media       int            `json:"media"`
	Terms       int            `json:"terms"`
	Authors     int            `json:"authors"`
	Menus       int            `json:"menus"`
	OldURLs     int            `json:"old_urls"`

	// SiteMode is "empty" or "existing" (see InspectSite); set by ApplySiteState.
	SiteMode        SiteMode `json:"site_mode,omitempty"`
	ExistingContent int      `json:"existing_content,omitempty"`
	ModeNote        string   `json:"mode_note,omitempty"`

	Permalink        string `json:"permalink"`
	PermalinkPattern string `json:"permalink_pattern,omitempty"`
	// PermalinkKept is true when the new URL pattern reproduces the old URLs;
	// otherwise every changed URL gets a redirect.
	PermalinkKept bool `json:"permalink_kept"`

	SEOPlugin          string         `json:"seo_plugin,omitempty"`
	PageBuilders       []string       `json:"page_builders,omitempty"`
	Forms              []string       `json:"forms,omitempty"`
	Multilingual       []string       `json:"multilingual,omitempty"`
	DefaultLang        string         `json:"default_language,omitempty"`
	Languages          map[string]int `json:"languages,omitempty"`
	Shortcodes         map[string]int `json:"shortcodes,omitempty"`
	UnsupportedPlugins []string       `json:"unsupported_plugins,omitempty"`

	LowFidelity []ItemNote `json:"low_fidelity,omitempty"`
	MembersOnly []ItemNote `json:"members_only,omitempty"`
	Embeds      int        `json:"embeds"`
	Fidelity    Fidelity   `json:"fidelity"`
	Warnings    []string   `json:"warnings,omitempty"`
}

Report is the scan of a source: what is there and what will not survive.

func Scan

func Scan(d *Dump, source string) *Report

Scan reads the Dump and reports what the import will do, before anything is written.

func (*Report) ApplySiteState

func (r *Report) ApplySiteState(s *SiteState)

ApplySiteState records the mode on the scan report.

func (*Report) Summary

func (r *Report) Summary() string

Summary is the report as text for the CLI.

type Result

type Result struct {
	RunID uuid.UUID `json:"run_id"`

	Created int `json:"created"`
	Updated int `json:"updated"`
	Failed  int `json:"failed"`

	Terms         int `json:"terms"`
	Authors       int `json:"authors"`
	MediaImported int `json:"media_imported"`
	MediaReused   int `json:"media_reused"`
	MediaFailed   int `json:"media_failed"`
	Menus         int `json:"menus"`
	Redirects     int `json:"redirects"`

	MediaRewritten int      `json:"media_rewritten"`
	LinksRewritten int      `json:"links_rewritten"`
	Unresolved     []string `json:"unresolved,omitempty"`
	// WPContentLeft lists wp-content URLs still present in imported content;
	// the importer's promise is that it is empty.
	WPContentLeft []string `json:"wp_content_left,omitempty"`

	// SiteMode is how the target was treated: "empty" or "existing".
	SiteMode SiteMode `json:"site_mode"`
	// URLConflicts are old URLs that could not be redirected because a live
	// page or a redirect of the owner already answers there.
	URLConflicts []URLConflict `json:"url_conflicts,omitempty"`
	// RedirectsRemoved counts redirects of an earlier run that are no longer needed.
	RedirectsRemoved int `json:"redirects_removed,omitempty"`
	// OverwrittenEdits and KeptEdits list entities edited in Hatamoto since the
	// last import, which a re-run overwrote or (with KeepEdited) left alone.
	OverwrittenEdits []string `json:"overwritten_edits,omitempty"`
	KeptEdits        []string `json:"kept_edits,omitempty"`

	Audit    *AuditResult `json:"audit,omitempty"`
	Errors   []ItemError  `json:"errors,omitempty"`
	Warnings []string     `json:"warnings,omitempty"`
	Report   *Report      `json:"report,omitempty"`
}

Result is what a run did.

type RewriteStats

type RewriteStats struct {
	MediaRewritten int
	LinksRewritten int
	// Unresolved are wp-content URLs that could not be mapped; the element
	// carrying them was removed (an img) or unlinked (an anchor).
	Unresolved []string
}

RewriteStats counts what a rewrite did.

type Rewriter

type Rewriter struct {
	// Hosts are the hostnames of the old site (without port).
	Hosts map[string]bool
	// Media maps a URL of the old site's files to the new media URL. ok is
	// false when the file could not be imported.
	Media func(u *url.URL) (newURL string, ok bool)
	// Entity resolves a path (or ?p= / ?page_id= id) of the old site to an
	// imported entity.
	Entity func(path string, query url.Values) (EntityRef, bool)
	// Path maps any other old-site path to its new one (terms, authors); nil keeps it.
	Path func(path string) (string, bool)
}

Rewriter rewrites the links of imported bodies.

func (*Rewriter) Rewrite

func (r *Rewriter) Rewrite(body string) (string, RewriteStats)

Rewrite returns body with every wp-content URL of the old site replaced by its imported media URL, and internal links turned into <a data-entity-id> links (or plain relative links when the target is not an entity).

type Run

type Run struct {
	ID        uuid.UUID
	Source    string
	SourceRef string
	Status    string
	Phase     string
	Options   json.RawMessage
	Report    json.RawMessage
	Error     string
}

Run is one row of hatamoto_import_runs.

type Runner

type Runner struct {
	// contains filtered or unexported fields
}

Runner writes a Dump into Hatamoto.

func NewRunner

func NewRunner(d *Deps, st *Store, dump *Dump, opts Options) (*Runner, error)

NewRunner prepares a run over dump.

func (*Runner) Run

func (r *Runner) Run(ctx context.Context, resume uuid.UUID) (res *Result, err error)

Run imports the dump. resume names an earlier run to continue (its map rows are reused, so finished work is not repeated); uuid.Nil starts a new one.

type SEO

type SEO struct {
	Title       string
	Description string
	Canonical   string
	OGImage     string
	NoIndex     bool
	Source      string // yoast, rankmath, seopress, aioseo, html
}

SEO is the search metadata the SEO plugin held for an item.

func ParseHead

func ParseHead(page, source string) SEO

ParseHead extracts the SEO fields from a page's HTML head.

type Selection

type Selection struct {
	// Types lists the post types to import ("post", "page", a custom type);
	// nil means all of them.
	Types []string
	// Drafts includes drafts, pending, private and password-protected items
	// (imported as drafts). Off, they are left out.
	Drafts bool
	// Menus includes the navigation menus.
	Menus bool
}

Selection narrows a Dump to what the owner chose to import.

func AllSelected

func AllSelected() Selection

AllSelected imports everything the source has.

func (Selection) Apply

func (s Selection) Apply(d *Dump)

Apply removes what the selection leaves out of d. It never changes the site-level data, authors, terms or the list of old URLs: the audit still asks for every old URL, so what was left out shows up as not covered.

type Site

type Site struct {
	URL         string // home URL, no trailing slash
	Title       string
	Tagline     string
	Language    string // BCP 47-ish ("en-US"); empty when unknown
	Timezone    string
	ShowOnFront string // "posts" or "page"
	FrontPageID int
	PostsPageID int
	// Permalink is the WordPress structure, for example "/%year%/%monthnum%/%postname%/",
	// inferred from the links of posts ("" when unknown).
	Permalink string
}

Site is the site-level data.

type SiteMode

type SiteMode string

SiteMode says how the importer treats the target site.

const (
	// ModeEmpty: the target has no published content besides the starter
	// content of the setup wizard. The import sets the post and page URL
	// patterns to the WordPress structure, the site settings and the home page.
	ModeEmpty SiteMode = "empty"
	// ModeExisting: the target has content of its own. The import never
	// changes URL patterns, site settings or the home page: imported items
	// take the patterns that exist, old URLs get redirects, and what cannot be
	// resolved is reported.
	ModeExisting SiteMode = "existing"
)

type SiteState

type SiteState struct {
	Mode SiteMode `json:"mode"`
	// Existing counts published entities that are neither starter content nor
	// imported from this source.
	Existing int `json:"existing"`
	// Starter counts untouched starter and first-boot entities (ignored).
	Starter int `json:"starter"`
	// Examples are a few slugs of the existing content.
	Examples []string `json:"examples,omitempty"`
}

SiteState is what the target site holds before the import.

func InspectSite

func InspectSite(ctx context.Context, pool *pgxpool.Pool, keyPrefix string) (*SiteState, error)

InspectSite decides the mode. Entities imported from this source (key prefix) do not count, so a re-run of an import into an empty site stays in the mode of the first run.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store persists runs and the source-id to Hatamoto-id map.

func NewStore

func NewStore(pool *pgxpool.Pool, sourceRef string) *Store

NewStore returns a store for one source site.

func (*Store) All

func (s *Store) All(ctx context.Context, kind string) (map[string]uuid.UUID, error)

All returns every mapping of a kind (source id to target id).

func (*Store) Delete

func (s *Store) Delete(ctx context.Context, kind, sourceID string) error

Delete forgets a mapping.

func (*Store) EditedEntities

func (s *Store) EditedEntities(ctx context.Context) (map[string]bool, error)

EditedEntities returns the source ids of imported entities that were changed in Hatamoto after the importer last wrote them: the hash of an entity row is the updated_at the importer left behind.

func (*Store) FinishRun

func (s *Store) FinishRun(ctx context.Context, id uuid.UUID, status string, report any, runErr error) error

FinishRun stores the outcome of a run.

func (*Store) GetRun

func (s *Store) GetRun(ctx context.Context, id uuid.UUID) (*Run, error)

GetRun loads a run; a run of another source site is an error.

func (*Store) Lookup

func (s *Store) Lookup(ctx context.Context, kind, sourceID string) (uuid.UUID, bool, error)

Lookup returns the Hatamoto id a source id was imported as.

func (*Store) LookupHash

func (s *Store) LookupHash(ctx context.Context, kind, hash string) (uuid.UUID, bool, error)

LookupHash returns a target imported with the same content hash (media dedup).

func (*Store) Put

func (s *Store) Put(ctx context.Context, run uuid.UUID, kind, sourceID string, target uuid.UUID, hash string) error

Put records or updates a mapping.

func (*Store) Ref

func (s *Store) Ref() string

Ref is the source reference runs and map rows of this store carry.

func (*Store) ResumeRun

func (s *Store) ResumeRun(ctx context.Context, id uuid.UUID) error

ResumeRun marks a run running again.

func (*Store) SetPhase

func (s *Store) SetPhase(ctx context.Context, id uuid.UUID, phase string) error

SetPhase records the phase a run reached.

func (*Store) StartRun

func (s *Store) StartRun(ctx context.Context, status string, options any) (*Run, error)

StartRun records a new run.

type Term

type Term struct {
	Taxonomy    string // "category", "post_tag" or custom
	Slug        string
	Name        string
	Description string
	ParentSlug  string
	Link        string
}

Term is a category, tag or custom taxonomy term.

type TermRef

type TermRef struct{ Taxonomy, Slug string }

TermRef points at a term of an item.

type URLConflict

type URLConflict struct {
	URL    string `json:"url"`
	Target string `json:"target,omitempty"` // where the imported item lives
	Reason string `json:"reason"`
}

URLConflict is an old URL the importer left alone, with the reason.

type WXROptions

type WXROptions struct {
	MaxBytes int64 // 0: DefaultWXRMaxBytes
	MaxItems int   // 0: DefaultMaxItems
	// MaxElements caps the elements of the whole file (0: DefaultMaxWXRElements).
	MaxElements int
}

WXROptions configures ParseWXR.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL