wiki

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 15, 2026 License: Apache-2.0 Imports: 27 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	// UserAgent identifies the client politely to the Wikimedia hosts. Wikimedia
	// asks every client to send a descriptive User-Agent with contact info; this
	// default carries the project URL. Override with Config.UserAgent.
	UserAgent = "wiki/1.0 (+https://github.com/tamnd/wikipedia-cli)"

	// WikidataSPARQL is the Wikidata Query Service endpoint.
	WikidataSPARQL = "https://query.wikidata.org/sparql"

	// MetricsBase is the analytics/pageviews REST root.
	MetricsBase = "https://wikimedia.org/api/rest_v1/metrics"

	// CoreAPIBase is the cross-wiki REST "core" root.
	CoreAPIBase = "https://api.wikimedia.org/core/v1"

	// DumpsBase is the published XML dumps host.
	DumpsBase = "https://dumps.wikimedia.org"
)

Build/runtime constants for the library.

View Source 
const (
	DefaultTimeout = 60 * time.Second
	DefaultRetries = 4
	DefaultDelay   = 150 * time.Millisecond
	// DefaultMaxlag is sent to the Action API so we back off when replica lag is
	// high. 5 seconds is the value the Wikimedia docs recommend for read clients.
	DefaultMaxlag = 5
)

Defaults for the polite HTTP client.

Variables

View Source 
var ErrNotFound = fmt.Errorf("not found")

ErrNotFound is returned when a page/entity does not exist.

Functions

func ConfigDir 

func ConfigDir() string

ConfigDir returns the directory holding the optional config file.

func ConvertBoth 

func ConvertBoth(src, lang string) (md, txt string)

ConvertBoth renders wikitext to Markdown and plain text sharing one strip pass. Cheaper than calling the two converters separately.

func EdgeHelp added in v0.3.0 

func EdgeHelp() string

EdgeHelp is the one-line catalogue of presets and edges, shared by the flag help and the parse error so the vocabulary lives in exactly one place.

func HTMLToMarkdown 

func HTMLToMarkdown(htmlStr string) string

HTMLToMarkdown renders an HTML fragment into Markdown.

func HTMLToText 

func HTMLToText(htmlStr string) string

HTMLToText renders an HTML fragment into readable plain text. It is pragmatic, not pixel-perfect: headings get blank lines, list items get bullets, links collapse to their text, and citation/edit chrome is dropped.

func NormalizeTitle 

func NormalizeTitle(title string) string

NormalizeTitle turns a human-typed title into the canonical form the APIs use for display: underscores become spaces and surrounding whitespace is trimmed. The server's own normalization remains authoritative; this is for local use.

func NotFound 

func NotFound(err error) bool

NotFound reports whether err is a 404.

func ParseTarget 

func ParseTarget(input string) (title, host string)

ParseTarget accepts a bare title, a "Namespace:Title", or a full Wikipedia URL, and returns the title plus the host it was pasted from (empty when the input was not a URL). A leading/trailing whitespace is trimmed.

func Projects 

func Projects() []string

Projects lists every project name Site understands, for discovery.

func ShortenURI 

func ShortenURI(s string) string

ShortenURI collapses a Wikidata entity or property URI to its bare Q/P id for compact display. The full URI is always kept in the structured output.

func StreamPages 

func StreamPages(path string, namespace int, withText bool, fn func(DumpPage) error) error

StreamPages parses a local pages-articles XML dump (optionally bz2/gz) and calls fn for each page in constant memory. Return an error from fn to stop. namespace >= 0 filters to that namespace; withText includes the body.

For .bz2 dumps it prefers the parallel lbzip2/pbzip2 binaries when they are on PATH (several times faster than Go's single-threaded decompressor), falling back to the standard library otherwise.

func VerifySha1 

func VerifySha1(path, expected string) (bool, error)

VerifySha1 checks a file against an expected sha1 hex digest.

func WikitextToMarkdown 

func WikitextToMarkdown(src, lang string) string

WikitextToMarkdown renders MediaWiki wikitext (the source you get from a dump or prop=wikitext) into clean Markdown. It is pragmatic, not a MediaWiki reimplementation: templates, tables, references and citation chrome are dropped, while headings, lists, bold/italic, code blocks, internal and external links and paragraphs are converted. lang sets the host used for internal-link URLs; an empty lang collapses internal links to their label so the output stays wiki-agnostic.

func WikitextToText 

func WikitextToText(src string) string

WikitextToText renders wikitext into readable plain text, dropping all markup (but keeping the contents of code blocks).

Types

type APIError 

type APIError struct {
	Code string
	Info string
}

APIError is a structured Action API error.

func (*APIError) Error 

func (e *APIError) Error() string

type Cache 

type Cache struct {
	// contains filtered or unexported fields
}

Cache is a tiny on-disk blob cache keyed by an arbitrary string, with a TTL per entry. It is safe for the simple single-process use the CLI makes of it.

func NewCache 

func NewCache(dir string, enabled bool) *Cache

NewCache returns a cache rooted under dir. If dir is empty or enabled is false, all operations are no-ops (cache miss on every Get).

func (*Cache) Clear 

func (c *Cache) Clear() (int, error)

Clear removes every cached entry. It returns the number of files removed.

func (*Cache) Dir 

func (c *Cache) Dir() string

Dir returns the cache directory.

func (*Cache) Get 

func (c *Cache) Get(key string, ttl time.Duration) ([]byte, bool)

Get returns cached bytes for key if present and younger than ttl.

func (*Cache) Info 

func (c *Cache) Info() (count int, bytes int64)

Info returns the number of cache entries and their total size in bytes.

func (*Cache) Put 

func (c *Cache) Put(key string, data []byte)

Put stores data under key.

type Category 

type Category struct {
	Title string `json:"title"`
	NS    int    `json:"ns"`
	URL   string `json:"url"`
}

Category is one category a page belongs to.

type CategoryMember 

type CategoryMember struct {
	Title     string `json:"title"`
	NS        int    `json:"ns"`
	Type      string `json:"type"`
	Timestamp string `json:"timestamp,omitempty"`
	URL       string `json:"url"`
}

CategoryMember is one member of a category.

type CategoryNode added in v0.3.0 

type CategoryNode struct {
	Name  string `json:"name"`
	Title string `json:"title"`
	URL   string `json:"url"`
}

CategoryNode is the category record a walk emits: the bare name, the full "Category:Name" title, and the live URL. A category reached through an edge is pre-built from the edge data, so it costs no extra fetch; a category seed is validated through Info.

type Client 

type Client struct {
	Site Site
	HTTP *HTTPClient
	Cfg  Config
}

Client bundles a resolved Site with the polite HTTP client and config. It is the handle every feature method hangs off. Build one with New.

func New 

func New(cfg Config, cache *Cache) (*Client, error)

New resolves cfg into a Client, sharing one cache and HTTP client. 

func (c *Client) Backlinks(ctx context.Context, title string, namespace, limit int) ([]Link, error)

Backlinks returns pages that link to the given title ("what links here").

func (*Client) Categories 

func (c *Client) Categories(ctx context.Context, title string, limit int) ([]Category, error)

Categories returns the categories a page belongs to.

func (*Client) CategoryMembers 

func (c *Client) CategoryMembers(ctx context.Context, name, memberType string, limit int) ([]CategoryMember, error)

CategoryMembers lists the members of a category. memberType is one of "page", "subcat", "file", or "" for all. The "Category:" prefix is added if missing.

func (*Client) Diff 

func (c *Client) Diff(ctx context.Context, fromRev int, toRev string) ([]DiffLine, error)

Diff compares two revisions. from/to may be numeric revision ids; the toRev may also be "prev"/"cur"/"next" relative to from. It returns the diff lines.

func (*Client) DownloadFile 

func (c *Client) DownloadFile(ctx context.Context, url, outDir string, progress func(n int64)) (string, error)

DownloadFile streams url to outDir, resuming a partial file by byte range and returning the final path. progress, if non-nil, is called with bytes copied.

func (*Client) DumpList 

func (c *Client) DumpList(ctx context.Context, wiki, date string) ([]DumpFile, string, error)

DumpList returns the files of a dump for a wiki and date. date may be a concrete "YYYYMMDD" or "latest"/"" to resolve the most recent dated dump.

func (*Client) EntityByID 

func (c *Client) EntityByID(ctx context.Context, id, lang string, props []string) (*Entity, error)

EntityByID fetches a Wikidata entity (Q… or P… id) in full. props, when set, restricts the claims to those property ids; everything else is always kept.

func (*Client) EntityByTitle 

func (c *Client) EntityByTitle(ctx context.Context, title, lang string, props []string) (*Entity, error)

EntityByTitle resolves a Wikipedia title to its Wikidata entity, then fetches it. Uses pageprops.wikibase_item on the current wiki. 

func (c *Client) ExternalLinks(ctx context.Context, title string, limit int) ([]Link, error)

ExternalLinks returns the external URLs referenced by a page.

func (*Client) Extract 

func (c *Client) Extract(ctx context.Context, title string, intro bool) (string, error)

Extract returns the plain-text extract of a page. When intro is true only the lead section is returned; otherwise the whole article as plain text.

func (*Client) Featured 

func (c *Client) Featured(ctx context.Context, date time.Time) (*FeaturedFeed, error)

Featured fetches the daily featured feed for a date (in the wiki's language).

func (*Client) GeoNear 

func (c *Client) GeoNear(ctx context.Context, title string, radius, limit int) ([]GeoResult, error)

GeoNear returns articles near another article's coordinates.

func (*Client) GeoSearch 

func (c *Client) GeoSearch(ctx context.Context, lat, lon float64, radius, limit int) ([]GeoResult, error)

GeoSearch returns articles within radius metres of (lat, lon).

func (*Client) GetSummary 

func (c *Client) GetSummary(ctx context.Context, title string) (*Summary, error)

GetSummary fetches the REST summary for a title.

func (*Client) HTML 

func (c *Client) HTML(ctx context.Context, title string) ([]byte, error)

HTML returns the rendered HTML of a page (REST page/html).

func (*Client) Info 

func (c *Client) Info(ctx context.Context, title string) (*PageInfo, error)

Info returns page metadata for a title. 

func (c *Client) LangLinks(ctx context.Context, title string, limit int) ([]LangLink, error)

LangLinks returns the same article in other languages. 

func (c *Client) Links(ctx context.Context, title string, namespace, limit int) ([]Link, error)

Links returns the internal wiki links on a page. When namespace >= 0 only that namespace is returned.

func (*Client) Media 

func (c *Client) Media(ctx context.Context, title string, limit int) ([]Media, error)

Media returns the files used on a page with their imageinfo (url, mime, size, dimensions, and license/author where available).

func (*Client) OnThisDayEvents 

func (c *Client) OnThisDayEvents(ctx context.Context, eventType string, month, day int) ([]OnThisDay, error)

OnThisDayEvents fetches historical events of a given type for a month/day. eventType is one of all/selected/births/deaths/holidays/events.

func (*Client) Pageviews 

func (c *Client) Pageviews(ctx context.Context, title, granularity, access, agent string, from, to time.Time) ([]PageviewPoint, error)

Pageviews returns the daily or monthly pageview series for a title between two dates. granularity is "daily" or "monthly"; access/agent default to "all".

func (*Client) Random 

func (c *Client) Random(ctx context.Context, n, namespace int) ([]SearchResult, error)

Random returns random page titles from the main namespace.

func (*Client) Related 

func (c *Client) Related(ctx context.Context, title string) ([]SearchResult, error)

Related returns the REST "related pages" suggestions for a title.

func (*Client) RevisionContent 

func (c *Client) RevisionContent(ctx context.Context, revid int) (string, error)

RevisionContent returns the wikitext of a specific revision id.

func (*Client) Revisions 

func (c *Client) Revisions(ctx context.Context, title string, limit int, user string) ([]Revision, error)

Revisions returns the revision history of a page, newest first.

func (*Client) SPARQL 

func (c *Client) SPARQL(ctx context.Context, query string) (*SPARQLResult, error)

SPARQL runs a query against the Wikidata Query Service and returns the rows keyed by SELECT variable, preserving each binding's type, language and datatype.

func (*Client) Search 

func (c *Client) Search(ctx context.Context, query string, limit int) ([]SearchResult, error)

Search runs a full-text search. It prefers the cross-wiki core endpoint for clean snippets and falls back to the Action API when the core endpoint is not available for the project.

func (*Client) SectionHTML 

func (c *Client) SectionHTML(ctx context.Context, title string, section int) (string, error)

SectionHTML returns the rendered HTML of a single section by index, via the parse API. The caller converts it to text/Markdown with the render helpers.

func (*Client) Sections 

func (c *Client) Sections(ctx context.Context, title string) ([]Section, error)

Sections returns the section table of contents for a page.

func (*Client) Stats 

func (c *Client) Stats(ctx context.Context) (*SiteStats, error)

Stats returns site info and statistics for the selected wiki.

func (*Client) Suggest 

func (c *Client) Suggest(ctx context.Context, prefix string, limit int) ([]Suggestion, error)

Suggest returns opensearch autocomplete results for a prefix.

func (*Client) Top 

func (c *Client) Top(ctx context.Context, year, month, day, limit int) ([]TopArticle, error)

Top returns the most-viewed articles for a day, or for a month when day == 0.

func (*Client) Walk added in v0.3.0 

func (c *Client) Walk(ctx context.Context, seeds []Seed, opts WalkOptions, emit func(*Node) error) error

Walk runs a breadth-first walk from the seeds and calls emit for each node.

func (*Client) Wikitext 

func (c *Client) Wikitext(ctx context.Context, title string) (string, error)

Wikitext returns the raw wikitext source of a page's current revision.

type Config 

type Config struct {
	Lang      string // language subdomain, e.g. "en"
	Project   string // "wikipedia", "wiktionary", "commons", "wikidata", ...
	SiteHost  string // explicit host; overrides Lang/Project when set
	DataDir   string
	CacheDir  string
	Timeout   time.Duration
	Delay     time.Duration
	Retries   int
	Maxlag    int
	UserAgent string
	// AllowAnyHost disables the Wikimedia-host allowlist (SSRF guard). Off by
	// default so a hostile "title"/URL cannot point the client elsewhere.
	AllowAnyHost bool
}

Config controls library behaviour. The zero value is not usable; call DefaultConfig and adjust.

func DefaultConfig 

func DefaultConfig() Config

DefaultConfig returns a Config rooted at ~/data/wiki, reading English Wikipedia, with the polite client defaults.

func (Config) DownloadDir 

func (c Config) DownloadDir() string

DownloadDir is where dump files and downloaded media land.

func (Config) Site 

func (c Config) Site() (Site, error)

Site resolves the configured Lang/Project/SiteHost into a concrete Site.

type Datavalue 

type Datavalue struct {
	Type  string          `json:"type"`
	Value json.RawMessage `json:"value"`
}

Datavalue is the typed value of a snak.

type DiffLine 

type DiffLine struct {
	Op   string `json:"op"` // "add", "del", "context"
	Text string `json:"text"`
}

DiffLine is one line of a unified diff.

type Domain 

type Domain struct{}

Domain is the Wikipedia driver. It carries no state; the per-run client is built by the factory Register hands kit.

func (Domain) Classify 

func (Domain) Classify(input string) (uriType, id string, err error)

Classify turns any accepted input into the canonical (type, id), so `ant resolve` and `ant url` need no network. A Category: title becomes a category; everything else is a page keyed by its title.

func (Domain) Info 

func (Domain) Info() kit.DomainInfo

Info describes the scheme, the hostnames a pasted link is matched against, and the identity a host reuses for help and version.

func (Domain) Locate 

func (Domain) Locate(uriType, id string) (string, error)

Locate is the inverse: the live page URL for a (type, id).

func (Domain) Register 

func (Domain) Register(app *kit.App)

Register installs the client factory and every Wikipedia operation onto app. A resolver op (Single) names its own record type and answers `ant get`; a List op enumerates a parent resource's members and answers `ant ls`.

type DumpFile 

type DumpFile struct {
	Job  string `json:"job"`
	Name string `json:"name"`
	Size int64  `json:"size"`
	Sha1 string `json:"sha1,omitempty"`
	URL  string `json:"url"`
}

DumpFile is one file produced by a dump job.

type DumpPage 

type DumpPage struct {
	ID        int    `json:"id"`
	NS        int    `json:"ns"`
	Title     string `json:"title"`
	RevID     int    `json:"revid"`
	Timestamp string `json:"timestamp"`
	Redirect  bool   `json:"redirect,omitempty"`
	Text      string `json:"text,omitempty"`
}

DumpPage is one page record streamed from a pages-articles XML dump.

type Edge added in v0.3.0 

type Edge string

Edge is the public link vocabulary: what the user types in --follow and what a node reports as the edge it arrived by. 

const (
	EdgeLinks      Edge = "links"      // page -> page (outgoing internal links, ns 0)
	EdgeBacklinks  Edge = "backlinks"  // page -> page (what links here, ns 0)
	EdgeCategories Edge = "categories" // page -> category (the page's categories)
	EdgeMembers    Edge = "members"    // category -> page (articles in the category)
	EdgeSubcats    Edge = "subcats"    // category -> category (subcategories)
)

func (Edge) Target added in v0.3.0 

func (e Edge) Target() NodeKind

Target reports which node kind an edge arrives at.

type EdgeSet added in v0.3.0 

type EdgeSet map[Edge]bool

EdgeSet is a set of edges to follow.

func DefaultEdges added in v0.3.0 

func DefaultEdges() EdgeSet

DefaultEdges is the edge set used when --follow is empty: the content preset, the obvious forward neighbors of whatever was seeded.

func ParseEdges added in v0.3.0 

func ParseEdges(spec string) (EdgeSet, error)

ParseEdges turns a --follow spec into an edge set. An empty spec is the default content preset; otherwise it is a comma-separated mix of preset names and edge names. An unknown token is an error that names the full catalogue.

func (EdgeSet) Has added in v0.3.0 

func (s EdgeSet) Has(e Edge) bool

Has reports whether the set contains e.

func (EdgeSet) List added in v0.3.0 

func (s EdgeSet) List() []Edge

List returns the set's edges in the canonical display order.

func (EdgeSet) String added in v0.3.0 

func (s EdgeSet) String() string

String renders the set as a comma-separated list in canonical order.

type Entity 

type Entity struct {
	ID           string                 `json:"id"`
	PageID       int                    `json:"pageid,omitempty"`
	NS           int                    `json:"ns,omitempty"`
	Title        string                 `json:"title,omitempty"`
	Type         string                 `json:"type,omitempty"`
	Datatype     string                 `json:"datatype,omitempty"`
	Labels       map[string]TermValue   `json:"labels,omitempty"`
	Descriptions map[string]TermValue   `json:"descriptions,omitempty"`
	Aliases      map[string][]TermValue `json:"aliases,omitempty"`
	Claims       map[string][]Statement `json:"claims,omitempty"`
	Sitelinks    map[string]Sitelink    `json:"sitelinks,omitempty"`
	LastRevID    int                    `json:"lastrevid,omitempty"`
	Modified     string                 `json:"modified,omitempty"`
	// contains filtered or unexported fields
}

Entity is a Wikidata entity with its full structure preserved. The JSON encoding mirrors the wbgetentities response, so labels, descriptions, aliases, statements (with qualifiers, references and ranks), sitelinks and the entity datatype all survive a round trip.

func (*Entity) AliasesFor 

func (e *Entity) AliasesFor(lang string) []string

AliasesFor returns the aliases in lang, or none if that language has no set.

func (*Entity) DescriptionFor 

func (e *Entity) DescriptionFor(lang string) string

DescriptionFor returns the description in lang, falling back to any language.

func (*Entity) LabelFor 

func (e *Entity) LabelFor(lang string) string

LabelFor returns the label in lang, falling back to any available language.

func (*Entity) UnmarshalJSON 

func (e *Entity) UnmarshalJSON(b []byte) error

UnmarshalJSON detects the missing marker (wbgetentities emits "missing":"") while decoding the rest of the entity normally.

type FeaturedFeed 

type FeaturedFeed struct {
	TFA       *FeedArticle  `json:"tfa,omitempty"`
	MostRead  []FeedArticle `json:"mostread,omitempty"`
	Image     *FeedImage    `json:"image,omitempty"`
	News      []FeedNews    `json:"news,omitempty"`
	OnThisDay []OnThisDay   `json:"onthisday,omitempty"`
}

FeaturedFeed is the daily feed for a date.

type FeedArticle 

type FeedArticle struct {
	Title       string `json:"title"`
	Description string `json:"description,omitempty"`
	Extract     string `json:"extract,omitempty"`
	URL         string `json:"url"`
	Views       int    `json:"views,omitempty"`
	Rank        int    `json:"rank,omitempty"`
	Thumbnail   string `json:"thumbnail,omitempty"`
}

FeedArticle is an article reference inside a feed.

type FeedImage 

type FeedImage struct {
	Title       string `json:"title"`
	Description string `json:"description,omitempty"`
	URL         string `json:"url"`
}

FeedImage is the picture of the day.

type FeedNews 

type FeedNews struct {
	Story string        `json:"story"`
	Links []FeedArticle `json:"links,omitempty"`
}

FeedNews is an in-the-news story.

type GeoResult 

type GeoResult struct {
	Title string  `json:"title"`
	Lat   float64 `json:"lat"`
	Lon   float64 `json:"lon"`
	Dist  float64 `json:"dist"` // metres
	URL   string  `json:"url"`
}

GeoResult is an article near a coordinate.

type HTTPClient 

type HTTPClient struct {
	// contains filtered or unexported fields
}

HTTPClient is a polite, retrying HTTP client for the Wikimedia hosts. It rate-limits requests, retries on 429/5xx with backoff that honors any Retry-After header, and caches GET-JSON responses on disk with a per-call TTL.

func NewHTTPClient 

func NewHTTPClient(cfg Config, cache *Cache) *HTTPClient

NewHTTPClient builds an HTTPClient from cfg. The cache may be nil.

func (*HTTPClient) GetBytes 

func (h *HTTPClient) GetBytes(ctx context.Context, url string) ([]byte, error)

GetBytes fetches url and returns the whole body, retrying transient failures.

func (*HTTPClient) GetJSON 

func (h *HTTPClient) GetJSON(ctx context.Context, url string, ttl time.Duration, v any) error

GetJSON fetches url and unmarshals the JSON body into v. When ttl > 0 and a cache is configured the response is served from and stored in the cache.

func (*HTTPClient) GetText 

func (h *HTTPClient) GetText(ctx context.Context, url, accept string) ([]byte, error)

GetText fetches url with an explicit Accept header and returns the raw body.

func (*HTTPClient) Open 

func (h *HTTPClient) Open(ctx context.Context, url, rangeHdr string) (*http.Response, error)

Open returns the response body for streaming downloads (no client timeout; the caller closes it and relies on ctx for cancellation). An optional Range header is sent when rangeHdr is non-empty (for resuming a download).

type HTTPError 

type HTTPError struct {
	Status int
	URL    string
	Body   string
}

HTTPError is returned for non-2xx responses.

func (*HTTPError) Error 

func (e *HTTPError) Error() string
type LangLink struct {
	Lang    string `json:"lang"`
	Title   string `json:"title"`
	Autonym string `json:"autonym,omitempty"`
	URL     string `json:"url"`
}

LangLink is an interlanguage link. 

type Link struct {
	Title string `json:"title,omitempty"`
	NS    int    `json:"ns"`
	URL   string `json:"url"`
}

Link is a wiki link (internal) or external link target.

type Media 

type Media struct {
	Title   string `json:"title"`
	URL     string `json:"url"`
	Mime    string `json:"mime,omitempty"`
	Width   int    `json:"width,omitempty"`
	Height  int    `json:"height,omitempty"`
	Size    int    `json:"size,omitempty"`
	License string `json:"license,omitempty"`
	Author  string `json:"author,omitempty"`
}

Media is one file used on a page.

type Node added in v0.3.0 

type Node struct {
	Kind     NodeKind      `json:"kind"`
	Depth    int           `json:"depth"`
	Via      Edge          `json:"via,omitempty"`
	Parent   string        `json:"parent,omitempty"`
	Page     *Summary      `json:"page,omitempty"`
	Category *CategoryNode `json:"category,omitempty"`
}

Node is one object reached by a walk, tagged with how it was reached.

func (*Node) Endpoint added in v0.3.0 

func (n *Node) Endpoint() string

Endpoint is the node's stable identity within a walk: a page's title or a category's bare name.

type NodeKind added in v0.3.0 

type NodeKind string

NodeKind is the kind of object a walk visits. 

const (
	KindPage     NodeKind = "page"
	KindCategory NodeKind = "category"
)

type OnThisDay 

type OnThisDay struct {
	Year  int      `json:"year"`
	Text  string   `json:"text"`
	Pages []string `json:"pages,omitempty"`
}

OnThisDay is a historical event.

type PageInfo 

type PageInfo struct {
	Pageid       int    `json:"pageid"`
	Title        string `json:"title"`
	Length       int    `json:"length"`
	Touched      string `json:"touched"`
	LastRevID    int    `json:"lastrevid"`
	ContentModel string `json:"contentmodel"`
	Lang         string `json:"pagelanguage"`
	Redirect     bool   `json:"redirect"`
	URL          string `json:"url"`
}

PageInfo is page metadata from prop=info.

type PageviewPoint 

type PageviewPoint struct {
	Date  string `json:"date"`
	Views int    `json:"views"`
}

PageviewPoint is one date/value pair in a pageview time series.

type Reference 

type Reference struct {
	Hash       string            `json:"hash,omitempty"`
	Snaks      map[string][]Snak `json:"snaks,omitempty"`
	SnaksOrder []string          `json:"snaks-order,omitempty"`
}

Reference is a citation attached to a statement.

type Revision 

type Revision struct {
	RevID     int    `json:"revid"`
	ParentID  int    `json:"parentid"`
	Timestamp string `json:"timestamp"`
	User      string `json:"user"`
	Size      int    `json:"size"`
	Comment   string `json:"comment"`
	Minor     bool   `json:"minor"`
	Tags      string `json:"tags,omitempty"`
}

Revision is one entry in a page's revision history.

type SPARQLBinding 

type SPARQLBinding struct {
	Type     string `json:"type"`
	Value    string `json:"value"`
	XMLLang  string `json:"xml:lang,omitempty"`
	Datatype string `json:"datatype,omitempty"`
}

SPARQLBinding is one cell of a SPARQL result: its RDF term kind, value, and optional language tag or datatype.

type SPARQLResult 

type SPARQLResult struct {
	Vars []string                   `json:"vars"`
	Rows []map[string]SPARQLBinding `json:"rows"`
}

SPARQLResult holds the column order and rows of a SPARQL response. Each row maps a SELECT variable to its full binding.

type SearchResult 

type SearchResult struct {
	Title       string `json:"title"`
	Description string `json:"description,omitempty"`
	Snippet     string `json:"snippet,omitempty"`
	Size        int    `json:"size,omitempty"`
	WordCount   int    `json:"wordcount,omitempty"`
	Timestamp   string `json:"timestamp,omitempty"`
	URL         string `json:"url"`
}

SearchResult is one full-text search hit.

type Section 

type Section struct {
	Index  string `json:"index"`
	Level  string `json:"level"`
	Line   string `json:"line"`
	Anchor string `json:"anchor"`
	Number string `json:"number"`
}

Section is one parsed section of a page.

type Seed added in v0.3.0 

type Seed struct {
	Kind NodeKind
	Ref  string // page title, or bare category name
}

Seed is a starting point for a walk.

func ParseSeed added in v0.3.0 

func ParseSeed(ref string) (Seed, error)

ParseSeed classifies any accepted reference into a seed, reusing the domain's own Classify so discover reads a string exactly as get and ls do.

type Site 

type Site struct {
	Host    string // e.g. "en.wikipedia.org"
	Project string // e.g. "wikipedia"
	Lang    string // e.g. "en" ("" for non-language projects)
}

Site is a resolved target wiki: its host and the URL roots for the three per-wiki API surfaces. Build one with cfg.Site().

func (Site) APIURL 

func (s Site) APIURL(params url.Values) string

APIURL returns the Action API endpoint with the given query parameters.

func (Site) CoreProject 

func (s Site) CoreProject() (project, lang string, ok bool)

CoreProject returns the project slug used by the cross-wiki core API, or "" when the project is not addressable there.

func (Site) CoreURL 

func (s Site) CoreURL(path string) (string, bool)

CoreURL builds an api.wikimedia.org core endpoint for this wiki.

func (Site) DumpWiki 

func (s Site) DumpWiki() string

DumpWiki returns the dump database name for this site, e.g. "enwiki".

func (Site) PageURL 

func (s Site) PageURL(title string) string

PageURL returns the canonical article URL for a title.

func (Site) RestV1 

func (s Site) RestV1(path string) string

RestV1 returns a per-wiki REST v1 URL for the given path (no leading slash).

type SiteStats 

type SiteStats struct {
	SiteName    string `json:"sitename"`
	Generator   string `json:"generator"`
	Lang        string `json:"lang"`
	MainPage    string `json:"mainpage"`
	Pages       int    `json:"pages"`
	Articles    int    `json:"articles"`
	Edits       int    `json:"edits"`
	Images      int    `json:"images"`
	Users       int    `json:"users"`
	ActiveUsers int    `json:"activeusers"`
	Admins      int    `json:"admins"`
}

SiteStats holds general site info and statistics for a wiki. 

type Sitelink struct {
	Site   string   `json:"site"`
	Title  string   `json:"title"`
	Badges []string `json:"badges,omitempty"`
	URL    string   `json:"url,omitempty"`
}

Sitelink connects an entity to a page on a Wikimedia site.

type Snak 

type Snak struct {
	SnakType  string     `json:"snaktype"`
	Property  string     `json:"property"`
	Hash      string     `json:"hash,omitempty"`
	Datatype  string     `json:"datatype,omitempty"`
	Datavalue *Datavalue `json:"datavalue,omitempty"`
}

Snak is a property-value assertion. Datavalue is kept as raw JSON so the full value structure (time precision, quantity bounds, coordinate globe, monolingual language) survives; ValueString renders a short form on demand.

func (Snak) ValueString 

func (s Snak) ValueString() string

ValueString renders a snak's value into a short string for table output. novalue/somevalue snaks render as "(no value)" / "(unknown value)".

type Statement 

type Statement struct {
	ID              string            `json:"id,omitempty"`
	Type            string            `json:"type,omitempty"`
	Rank            string            `json:"rank,omitempty"`
	Mainsnak        Snak              `json:"mainsnak"`
	Qualifiers      map[string][]Snak `json:"qualifiers,omitempty"`
	QualifiersOrder []string          `json:"qualifiers-order,omitempty"`
	References      []Reference       `json:"references,omitempty"`
}

Statement is one claim on an entity, with its rank, qualifiers and references intact.

type Suggestion 

type Suggestion struct {
	Title       string `json:"title"`
	Description string `json:"description,omitempty"`
	URL         string `json:"url"`
}

Suggestion is one opensearch autocomplete entry.

type Summary 

type Summary struct {
	Title       string  `json:"title" kit:"id"`
	Description string  `json:"description,omitempty"`
	Extract     string  `json:"extract" kit:"body"`
	Type        string  `json:"type,omitempty"`
	Lang        string  `json:"lang,omitempty"`
	URL         string  `json:"url"`
	Thumbnail   string  `json:"thumbnail,omitempty"`
	Lat         float64 `json:"lat,omitempty"`
	Lon         float64 `json:"lon,omitempty"`
	Pageid      int     `json:"pageid,omitempty"`
}

Summary is the short REST page summary (the blob the mobile apps show).

type TermValue 

type TermValue struct {
	Language string `json:"language"`
	Value    string `json:"value"`
}

TermValue is a language-tagged label, description or alias.

type TopArticle 

type TopArticle struct {
	Rank  int    `json:"rank"`
	Title string `json:"title"`
	Views int    `json:"views"`
	URL   string `json:"url"`
}

TopArticle is one entry in a most-viewed list.

type WalkOptions added in v0.3.0 

type WalkOptions struct {
	Depth  int          // hops to follow from each seed (0 = seeds only)
	Max    int          // total node budget, the hard stop
	Fanout int          // max neighbors per edge (0 = bounded only by Max)
	Edges  EdgeSet      // which edges to follow
	Note   func(string) // surface a one-line advisory (never fatal)
}

WalkOptions configures a walk.

type Walker added in v0.3.0 

type Walker struct {
	// contains filtered or unexported fields
}

Walker runs a breadth-first walk over a grapher.

func NewWalker added in v0.3.0 

func NewWalker(g grapher) *Walker

NewWalker builds a walker over any grapher (the client, or a test fake).

func (*Walker) Walk added in v0.3.0 

func (w *Walker) Walk(ctx context.Context, seeds []Seed, opts WalkOptions, emit func(*Node) error) error

Walk is the breadth-first traversal.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.