x

package
v0.4.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 15, 2026 License: AGPL-3.0 Imports: 29 Imported by: 0

Documentation

Overview

Package x is the library beneath the x command line: the normalized X (Twitter) data model and the tiered access clients that fill it.

Three free read tiers feed one set of types: Tier 0 (the public syndication/embed endpoint, no auth), Tier 1 (the opt-in guest-token GraphQL path), and Tier 2 (the GraphQL API spoken with the user's own session cookies). There is no paid API. Every command speaks the types in this file, never a tier's wire shape directly.

Index

Constants

View Source
const UserAgent = "x-cli/" + "dev" + " (+https://github.com/tamnd/x-cli)"

UserAgent is the truthful identifier x sends on every request.

Variables

This section is empty.

Functions

func ConfigPath

func ConfigPath() string

ConfigPath returns the path to the TOML config file.

func EdgeHelp added in v0.4.0

func EdgeHelp() string

EdgeHelp is the one-line catalogue of presets and edges for flag help and usage errors, so the names a user can type live in exactly one place.

func ErrNeedAuth

func ErrNeedAuth(msg string) error

ErrNeedAuth builds a need-auth error (a bearer token unlocks it).

func ErrNeedUser

func ErrNeedUser(msg string) error

ErrNeedUser builds a need-auth error that specifically requires a user session.

func Export

func Export(s *Store, username, outDir string) (int, error)

Export renders a user's stored tweets as browseable Markdown (spec §3.3): one file per month plus an index, media linked. It reads the local store.

func ForgetSession

func ForgetSession() error

ForgetSession removes the persisted session.

func GuestStorePath

func GuestStorePath() string

GuestStorePath returns where a minted guest token is cached between runs. Reusing it keeps repeated invocations from re-minting (and tripping X's per-IP guest-activation rate limit).

func ParseTweetRef

func ParseTweetRef(s string) (string, error)

ParseTweetRef normalizes any of the accepted tweet references (a bare ID, a status URL on x.com/twitter.com/mobile, with or without scheme/query) to the canonical numeric tweet ID.

func ParseUserRef

func ParseUserRef(s string, forceID bool) (ref string, isID bool, err error)

ParseUserRef normalizes any accepted user reference to a handle (without the leading @) or, when forceID is set or the value is a profile URL ending in a numeric id, returns the value as-is. The second return reports whether the result should be treated as a numeric user ID rather than a handle.

func SaveSession

func SaveSession(s Creds) error

SaveSession persists the user's imported credentials (file fallback; 0600).

func SessionStorePath

func SessionStorePath() string

SessionStorePath returns where the imported session is persisted (a file fallback for platforms without a keychain integration).

func TweetURL

func TweetURL(author, id string) string

TweetURL builds the canonical permalink for a tweet.

func UserURL

func UserURL(handle string) string

UserURL builds the canonical permalink for a profile.

Types

type Bucket

type Bucket struct {
	Start time.Time `json:"start"`
	End   time.Time `json:"end"`
	Count int       `json:"count"`
}

Bucket is one time-bucketed tweet count (from x counts).

type Cache

type Cache struct {
	// contains filtered or unexported fields
}

Cache is a tiny content-addressed disk cache for GET responses. Keys are a hash of the canonical request; entries expire by mtime against a per-call TTL.

func NewCache

func NewCache(dir string, enabled bool) *Cache

NewCache returns a cache rooted at dir. When enabled is false every operation is a no-op so callers need not special-case --no-cache.

func (*Cache) Clear

func (c *Cache) Clear() error

Clear removes the whole cache directory.

func (*Cache) Get

func (c *Cache) Get(key string, ttl time.Duration) ([]byte, bool)

Get returns the cached bytes for key if present and younger than ttl.

func (*Cache) Put

func (c *Cache) Put(key string, data []byte)

Put writes bytes for key.

func (*Cache) Size

func (c *Cache) Size() (bytes int64, files int)

Size returns the total bytes and file count of the cache.

type Client

type Client struct {
	// contains filtered or unexported fields
}

Client is the shared HTTP client for every tier: one rate limiter, retry policy, disk cache, and per-endpoint rate-limit accounting (the nitter lesson, spec §13.2 — minus the account pool).

func NewClient

func NewClient(cfg Config) *Client

NewClient builds a Client from a config.

func (*Client) Cache

func (c *Client) Cache() *Cache

Cache exposes the underlying disk cache (for `x cache`).

func (*Client) Config

func (c *Client) Config() Config

Config returns the client's resolved config.

func (*Client) Do

func (c *Client) Do(ctx context.Context, r Req) ([]byte, error)

Do executes a request through the rate limiter, retry policy, and cache, returning the (decompressed) body. Non-2xx returns an *HTTPError.

type Config

type Config struct {
	// Session and access.
	AuthToken  string // the user's own auth_token cookie (Tier 2)
	CT0        string // the user's own ct0 cookie / CSRF token (Tier 2)
	AllowGuest bool   // enable the opt-in free guest-GraphQL tier (Tier 1)
	Tier       string // forced tier: ""|syndication|guest|session

	// Behavior.
	Rate     time.Duration
	Retries  int
	Timeout  time.Duration
	NoCache  bool
	DataDir  string
	CacheDir string
	Store    string // path to the SQLite store (--db)

	// GraphQL overrides (durability against query-id rotation; see spec §13).
	QueryIDs map[string]string // OperationName -> queryId
	Features string            // override features JSON
}

Config is the resolved runtime configuration for a command. It is built from defaults, then the config file, then the environment, then flags. There are no developer-API credentials here by design (spec §0.2): the only secrets x holds are the user's own browser session cookies (Tier 2).

func DefaultConfig

func DefaultConfig() Config

DefaultConfig returns the built-in defaults before any file/env/flag overlay.

func (*Config) FromEnv

func (c *Config) FromEnv()

FromEnv overlays environment variables onto a config.

func (Config) HasSession

func (c Config) HasSession() bool

HasSession reports whether the user's own session cookies are available (Tier 2).

type Creds

type Creds struct {
	AuthToken string `json:"auth_token,omitempty"`
	CT0       string `json:"ct0,omitempty"`
	Handle    string `json:"handle,omitempty"`
}

Creds are the persisted credential fields written by `x auth import`. A zero Creds means guest-only (Tier 1); one carrying the user's cookies is Tier 2.

func LoadSession

func LoadSession() (Creds, bool)

LoadSession reads the persisted credentials written by `x auth import`.

type Domain added in v0.2.0

type Domain struct{}

Domain is the X driver. It carries no state; the per-run Engine is built by the factory Register hands kit.

func (Domain) Classify added in v0.2.0

func (Domain) Classify(input string) (uriType, id string, err error)

Classify turns any accepted input into the canonical (type, id), so `ant resolve` and `ant url` need no network. A bare numeric or a /status/ URL is a tweet; anything else is read as a profile reference.

func (Domain) Info added in v0.2.0

func (Domain) Info() kit.DomainInfo

Info describes the scheme, the hostnames a pasted link is matched against, and the identity a host reuses for help and version.

func (Domain) Locate added in v0.2.0

func (Domain) Locate(uriType, id string) (string, error)

Locate is the inverse: the live page URL for a (type, id).

func (Domain) Register added in v0.2.0

func (Domain) Register(app *kit.App)

Register installs the Engine factory and every X operation onto app. A resolver op (Single) names its own record type and answers `ant get`; a List op enumerates a parent resource's members and answers `ant ls`.

type Edge added in v0.4.0

type Edge string

Edge names a link the walk can follow. The string is the public vocabulary: it is what the user types in --follow, what lands in the store's edges.kind column, and what a discovered node reports as the edge it arrived by.

const (
	// Tier-0 edges: reachable from the object itself, no token needed.
	EdgeAuthor   Edge = "author"   // tweet -> the account that wrote it
	EdgeQuoted   Edge = "quote"    // tweet -> the tweet it quotes
	EdgeRetweet  Edge = "retweet"  // tweet -> the original it retweets
	EdgeReply    Edge = "reply"    // tweet -> the tweet it replies to (the parent)
	EdgeMention  Edge = "mention"  // tweet -> each account it @-mentions
	EdgePinned   Edge = "pinned"   // user  -> their pinned tweet
	EdgeTimeline Edge = "timeline" // user  -> their recent tweets

	// Tier-1/2 edges: need the guest or session GraphQL tier.
	EdgeReplies   Edge = "replies"   // tweet -> the replies under it
	EdgeLiker     Edge = "liker"     // tweet -> accounts that liked it
	EdgeRetweeter Edge = "retweeter" // tweet -> accounts that retweeted it
	EdgeQuotedBy  Edge = "quotedby"  // tweet -> tweets that quote it (search-backed)
	EdgeFollowing Edge = "following" // user  -> accounts they follow
	EdgeFollowers Edge = "followers" // user  -> accounts that follow them
	EdgeLikes     Edge = "likes"     // user  -> tweets they liked
)

func (Edge) Target added in v0.4.0

func (e Edge) Target() NodeKind

Target reports the kind of node an edge leads to.

type EdgeSet added in v0.4.0

type EdgeSet map[Edge]bool

EdgeSet is a chosen set of edges to follow.

func DefaultEdges added in v0.4.0

func DefaultEdges() EdgeSet

DefaultEdges is what a walk follows when --follow is unset: a post's content. It stays entirely on Tier 0, so `x discover <tweet>` works with no token.

func ParseEdges added in v0.4.0

func ParseEdges(spec string) (EdgeSet, error)

ParseEdges turns a --follow spec into an EdgeSet. The spec is a comma list of preset names and/or edge names ("content", "thread,engagement", "author,liker"). An empty spec yields DefaultEdges. An unknown token is a usage error naming the catalogue, so a typo points the user at the real vocabulary.

func (EdgeSet) Has added in v0.4.0

func (s EdgeSet) Has(e Edge) bool

Has reports whether the set contains e (a nil set contains nothing).

func (EdgeSet) List added in v0.4.0

func (s EdgeSet) List() []Edge

List returns the set's edges in stable display order.

func (EdgeSet) String added in v0.4.0

func (s EdgeSet) String() string

String renders the set as a comma-separated, ordered list.

type Engine

type Engine struct {
	// contains filtered or unexported fields
}

Engine wires the three tiers behind one capability surface (spec §4.2). It resolves the cheapest free surface that can answer each call: Tier 0 (syndication, no auth) for single tweets / profiles / recent timelines, then the GraphQL tiers (1 guest, 2 the user's own session) for everything else.

func NewEngine

func NewEngine(cfg Config) *Engine

NewEngine builds an Engine from a resolved config.

func (*Engine) CanGraphQL added in v0.4.0

func (e *Engine) CanGraphQL() bool

CanGraphQL reports whether a GraphQL tier is available. It exports the internal check so the walker (and any other consumer) can read the engine's capability.

func (*Engine) Client

func (e *Engine) Client() *Client

Client exposes the shared HTTP client (for `x cache`, downloads).

func (*Engine) Config

func (e *Engine) Config() Config

Config returns the engine's config.

func (*Engine) Followers

func (e *Engine) Followers(ctx context.Context, ref string, isID bool, limit int, emit func(*User) error) error

Followers / Following / Likers / Retweeters (GraphQL only).

func (*Engine) Following

func (e *Engine) Following(ctx context.Context, ref string, isID bool, limit int, emit func(*User) error) error

func (*Engine) GraphQL

func (e *Engine) GraphQL() *GraphQL

GraphQL returns the GraphQL client (the deeper reads beyond syndication).

func (*Engine) Likers

func (e *Engine) Likers(ctx context.Context, tweetID string, limit int, emit func(*User) error) error

func (*Engine) Likes

func (e *Engine) Likes(ctx context.Context, ref string, isID bool, limit int, emit func(*Tweet) error) error

Likes streams the tweets a user liked (GraphQL only).

func (*Engine) Retweeters

func (e *Engine) Retweeters(ctx context.Context, tweetID string, limit int, emit func(*User) error) error

func (*Engine) Search

func (e *Engine) Search(ctx context.Context, q SearchQuery, emit func(*Tweet) error) error

Search streams search results (GraphQL only).

func (*Engine) Thread

func (e *Engine) Thread(ctx context.Context, id string, limit int, emit func(*Tweet) error) error

Thread streams a conversation, falling back to Tier 0 self-thread.

func (*Engine) Timeline

func (e *Engine) Timeline(ctx context.Context, ref string, isID bool, o TimelineOpts, emit func(*Tweet) error) error

Timeline streams a user's tweets, using Tier 0 for the recent window and the GraphQL tiers to page deeper.

func (*Engine) Tweet

func (e *Engine) Tweet(ctx context.Context, id string) (*Tweet, error)

Tweet resolves one tweet, preferring Tier 0 syndication.

func (*Engine) User

func (e *Engine) User(ctx context.Context, ref string, isID bool) (*User, error)

User resolves a profile, preferring Tier 0 syndication for a handle.

func (*Engine) Walk added in v0.4.0

func (e *Engine) Walk(ctx context.Context, seeds []Seed, opts WalkOptions, emit func(*Node) error) error

Walk runs the engine's traversal. It is the production entry point: it builds a Walker over the engine and walks the seeds. See Walker.Walk.

type Entities

type Entities struct {
	Hashtags []string `json:"hashtags,omitempty"`
	Cashtags []string `json:"cashtags,omitempty"`
	Mentions []string `json:"mentions,omitempty"`
	URLs     []string `json:"urls,omitempty"`
}

Entities are the parsed surface features of a tweet or a bio.

type GraphQL

type GraphQL struct {
	// contains filtered or unexported fields
}

GraphQL is the client for the web-client GraphQL the x.com browser uses (spec §1.6, §13). One type serves both Tier 1 (a minted guest token) and Tier 2 (the user's own imported cookies); the only difference is the headers session.go attaches. The query IDs rotate when X redeploys, so every one is overridable via config (spec §13.3).

func NewGraphQL

func NewGraphQL(c *Client, s *Session, cfg Config) *GraphQL

NewGraphQL builds a GraphQL client over a shared HTTP client and session.

func (*GraphQL) Bookmarks

func (g *GraphQL) Bookmarks(ctx context.Context, limit int, emit func(*Tweet) error) error

Bookmarks streams the user's bookmarks (session only).

func (*GraphQL) Followers

func (g *GraphQL) Followers(ctx context.Context, userID string, limit int, emit func(*User) error) error

Followers / Following / Likers / Retweeters stream User rows.

func (*GraphQL) Following

func (g *GraphQL) Following(ctx context.Context, userID string, limit int, emit func(*User) error) error

func (*GraphQL) Home

func (g *GraphQL) Home(ctx context.Context, limit int, emit func(*Tweet) error) error

Home streams the user's reverse-chron home timeline (session only).

func (*GraphQL) Likers

func (g *GraphQL) Likers(ctx context.Context, tweetID string, limit int, emit func(*User) error) error

func (*GraphQL) Likes

func (g *GraphQL) Likes(ctx context.Context, userID string, limit int, emit func(*Tweet) error) error

Likes streams the tweets a user has liked.

func (*GraphQL) ListTweets

func (g *GraphQL) ListTweets(ctx context.Context, listID string, limit int, emit func(*Tweet) error) error

ListTweets streams a List's timeline via ListLatestTweetsTimeline.

func (*GraphQL) Retweeters

func (g *GraphQL) Retweeters(ctx context.Context, tweetID string, limit int, emit func(*User) error) error

func (*GraphQL) Search

func (g *GraphQL) Search(ctx context.Context, q SearchQuery, emit func(*Tweet) error) error

Search streams search results via SearchTimeline.

func (*GraphQL) Thread

func (g *GraphQL) Thread(ctx context.Context, focalID string, limit int, emit func(*Tweet) error) error

Thread streams a conversation via TweetDetail (focal tweet + replies).

func (*GraphQL) TweetByID

func (g *GraphQL) TweetByID(ctx context.Context, id string) (*Tweet, error)

TweetByID resolves one tweet via TweetResultByRestId.

func (*GraphQL) UserByName

func (g *GraphQL) UserByName(ctx context.Context, handle string) (*User, error)

UserByName resolves a profile via UserByScreenName.

func (*GraphQL) UserByRestID

func (g *GraphQL) UserByRestID(ctx context.Context, id string) (*User, error)

UserByRestID resolves a profile by numeric id via UserByRestId.

func (*GraphQL) UserTweets

func (g *GraphQL) UserTweets(ctx context.Context, userID string, o TimelineOpts, emit func(*Tweet) error) error

UserTweets streams a user's tweets, paging UserTweets/…AndReplies/…Media.

type HTTPError

type HTTPError struct {
	Status     int
	Body       string
	URL        string
	RetryAfter time.Duration
}

HTTPError carries a non-2xx upstream response.

func (*HTTPError) Error

func (e *HTTPError) Error() string

type List

type List struct {
	ID          string    `json:"id"`
	Name        string    `json:"name"`
	Description string    `json:"description,omitempty"`
	Owner       *User     `json:"owner,omitempty"`
	Members     int       `json:"member_count"`
	Followers   int       `json:"follower_count"`
	Private     bool      `json:"private,omitempty"`
	CreatedAt   time.Time `json:"created_at,omitempty"`
}

List is an X List.

type Media

type Media struct {
	Key       string    `json:"key,omitempty"`
	Type      string    `json:"type"` // photo|video|animated_gif
	URL       string    `json:"url,omitempty"`
	Preview   string    `json:"preview_image,omitempty"`
	Width     int       `json:"width,omitempty"`
	Height    int       `json:"height,omitempty"`
	Duration  int       `json:"duration_ms,omitempty"`
	AltText   string    `json:"alt_text,omitempty"`
	Variants  []Variant `json:"variants,omitempty"`
	ViewCount int       `json:"view_count,omitempty"`
}

Media is one attached photo, video, or gif.

type Metrics

type Metrics struct {
	Replies     int `json:"replies"`
	Retweets    int `json:"retweets"`
	Likes       int `json:"likes"`
	Quotes      int `json:"quotes"`
	Bookmarks   int `json:"bookmarks"`
	Impressions int `json:"impressions"`
}

Metrics are the engagement counts on a tweet. The public ones are present on most tiers; impressions/bookmarks may be zero where a tier does not expose them.

type NeedAuthError

type NeedAuthError struct {
	Msg  string
	User bool // true when a user-context session (not just a bearer) is required
}

NeedAuthError marks a capability that needs an API token or user session that is not present (exit code 4).

func (*NeedAuthError) Error

func (e *NeedAuthError) Error() string

type Node added in v0.4.0

type Node struct {
	Kind   NodeKind `json:"kind"`
	Depth  int      `json:"depth"`
	Via    Edge     `json:"via,omitempty"`
	Parent string   `json:"parent,omitempty"`
	Tweet  *Tweet   `json:"tweet,omitempty"`
	User   *User    `json:"user,omitempty"`
}

Node is one object the walk reached, tagged with how it got there: the BFS depth, the edge it arrived by, and the endpoint of the node it came from. Exactly one of Tweet/User is set, matching Kind. Node is what Walk hands to its callback and what the CLI renders.

func (*Node) Endpoint added in v0.4.0

func (n *Node) Endpoint() string

Endpoint is the node's stable identifier inside a walk: a tweet id, or a "@handle" for a user. It is what edges record as src/dst and what the store keys a queue row by.

type NodeKind added in v0.4.0

type NodeKind string

NodeKind is the type of a node the walk visits.

const (
	KindTweet NodeKind = "tweet"
	KindUser  NodeKind = "user"
)

type NotFoundError

type NotFoundError struct {
	Kind string
	Ref  string
}

NotFoundError marks a missing/deleted/protected entity (exit code 6).

func (*NotFoundError) Error

func (e *NotFoundError) Error() string

type Place

type Place struct {
	ID          string `json:"id"`
	FullName    string `json:"full_name"`
	Name        string `json:"name,omitempty"`
	Country     string `json:"country,omitempty"`
	CountryCode string `json:"country_code,omitempty"`
	PlaceType   string `json:"place_type,omitempty"`
}

Place is a geotag.

type Poll

type Poll struct {
	ID           string       `json:"id,omitempty"`
	Options      []PollOption `json:"options"`
	DurationMin  int          `json:"duration_minutes,omitempty"`
	EndDateTime  time.Time    `json:"end_datetime,omitempty"`
	VotingStatus string       `json:"voting_status,omitempty"`
}

Poll is an attached poll.

type PollOption

type PollOption struct {
	Position int    `json:"position"`
	Label    string `json:"label"`
	Votes    int    `json:"votes"`
}

PollOption is one choice in a poll.

type QueueItem

type QueueItem struct {
	Target string
	Kind   string
}

QueueItem is one crawl-queue entry.

type RateLimitedError

type RateLimitedError struct{ Msg string }

RateLimitedError marks an exhausted upstream after retries (exit code 5).

func (*RateLimitedError) Error

func (e *RateLimitedError) Error() string

type Req

type Req struct {
	Method   string
	URL      string
	Endpoint string // a stable label for rate-limit accounting and cache keys
	Header   http.Header
	Body     []byte
	CacheTTL time.Duration // 0 = do not cache
}

Req is one HTTP request to make through the shared policy.

type SearchQuery

type SearchQuery struct {
	Raw     string
	Product string // Top|Latest|People|Photos|Videos
	Limit   int
}

SearchQuery controls a search read.

type Seed added in v0.4.0

type Seed struct {
	Kind NodeKind
	Ref  string // tweet id, or user handle / numeric id
	IsID bool   // for a user seed: Ref is a numeric account id
}

Seed is a parsed starting point for a walk.

func ParseSeed added in v0.4.0

func ParseSeed(ref string) (Seed, error)

ParseSeed classifies a raw reference into a Seed. A bare number or a status URL is a tweet (matching the rest of the CLI, where a bare number is a tweet id); anything else is read as a user handle or profile URL.

type Session

type Session struct {
	AuthToken string
	CT0       string
	Handle    string
	// contains filtered or unexported fields
}

Session is the runtime authentication state shared by the GraphQL tiers. It carries the credentials (if any) plus the cached guest token.

func NewSession

func NewSession(cfg Config) *Session

NewSession builds a runtime session from a config's credentials.

func (*Session) IsUser

func (s *Session) IsUser() bool

IsUser reports whether this session carries the user's own cookies (Tier 2).

type Space

type Space struct {
	ID             string    `json:"id"`
	State          string    `json:"state"` // live|scheduled|ended
	Title          string    `json:"title,omitempty"`
	HostIDs        []string  `json:"host_ids,omitempty"`
	SpeakerIDs     []string  `json:"speaker_ids,omitempty"`
	Participants   int       `json:"participant_count,omitempty"`
	Subscribers    int       `json:"subscriber_count,omitempty"`
	StartedAt      time.Time `json:"started_at,omitempty"`
	ScheduledStart time.Time `json:"scheduled_start,omitempty"`
	EndedAt        time.Time `json:"ended_at,omitempty"`
	Lang           string    `json:"lang,omitempty"`
	Ticketed       bool      `json:"is_ticketed,omitempty"`
	Topics         []string  `json:"topics,omitempty"`
}

Space is an audio Space.

type Store

type Store struct {
	// contains filtered or unexported fields
}

Store is the local SQLite dataset (spec §4.6), pure-Go so the binary stays CGO-free. It holds tweets/users/media/edges and a crawl queue.

func OpenStore

func OpenStore(path string) (*Store, error)

OpenStore opens (creating if needed) the SQLite store at path.

func (*Store) ClearQueue

func (s *Store) ClearQueue() error

ClearQueue empties the crawl queue.

func (*Store) Close

func (s *Store) Close() error

Close closes the store.

func (*Store) DB

func (s *Store) DB() *sql.DB

DB exposes the underlying handle for `x db query`.

func (*Store) Enqueue

func (s *Store) Enqueue(target, kind string, priority int) error

Enqueue adds a crawl target if not present.

func (*Store) MarkDone

func (s *Store) MarkDone(target string) error

MarkDone marks a queue target done.

func (*Store) NextPending

func (s *Store) NextPending() (QueueItem, bool, error)

NextPending returns the next pending target (highest priority first).

func (*Store) QueueCounts

func (s *Store) QueueCounts() (map[string]int, error)

QueueCounts returns counts by state.

func (*Store) Stats

func (s *Store) Stats() (map[string]int, error)

Stats returns row counts per table.

func (*Store) TweetsByAuthor

func (s *Store) TweetsByAuthor(username string) ([]*Tweet, error)

TweetsByAuthor returns stored tweets for a username, oldest first.

func (*Store) UpsertEdge

func (s *Store) UpsertEdge(src, dst, kind string) error

UpsertEdge records a graph edge (follow/like/retweet/quote/reply/mention).

func (*Store) UpsertMedia

func (s *Store) UpsertMedia(tweetID string, m Media) error

UpsertMedia inserts or refreshes a media row.

func (*Store) UpsertNode added in v0.4.0

func (s *Store) UpsertNode(n *Node) error

UpsertNode persists a discovered graph node by dispatching on its kind, so the discover/crawl walkers have one call to store whatever they reached.

func (*Store) UpsertTweet

func (s *Store) UpsertTweet(t *Tweet) error

UpsertTweet inserts or refreshes a tweet (and its author + media).

func (*Store) UpsertUser

func (s *Store) UpsertUser(u *User) error

UpsertUser inserts or refreshes a user.

type TimelineOpts

type TimelineOpts struct {
	Replies bool
	Media   bool
	Limit   int
}

TimelineOpts controls a user-timeline read.

type Trend

type Trend struct {
	Name        string `json:"name"`
	Query       string `json:"query,omitempty"`
	TweetVolume int    `json:"tweet_volume,omitempty"`
	URL         string `json:"url,omitempty"`
	Location    string `json:"location,omitempty"`
}

Trend is one trending topic.

type TrendLocation

type TrendLocation struct {
	WOEID       int    `json:"woeid"`
	Name        string `json:"name"`
	Country     string `json:"country,omitempty"`
	CountryCode string `json:"country_code,omitempty"`
	PlaceType   string `json:"place_type,omitempty"`
	ParentID    int    `json:"parentid,omitempty"`
}

TrendLocation is a place trends can be asked for (a Yahoo! WOEID).

type Tweet

type Tweet struct {
	ID             string    `json:"id" kit:"id"`
	URL            string    `json:"url"`
	Text           string    `json:"text" kit:"body"`
	CreatedAt      time.Time `json:"created_at"`
	Lang           string    `json:"lang,omitempty"`
	Author         *User     `json:"author,omitempty"`
	ConversationID string    `json:"conversation_id,omitempty" kit:"link,kind=x/status,optional"`
	ReplyTo        string    `json:"reply_to,omitempty" kit:"link,kind=x/status,optional"`
	ReplyToUser    string    `json:"reply_to_user,omitempty" kit:"link,kind=x/user,optional"`
	Quoted         *Tweet    `json:"quoted,omitempty"`
	Retweeted      *Tweet    `json:"retweeted,omitempty"`
	Metrics        Metrics   `json:"metrics"`
	Entities       Entities  `json:"entities,omitempty"`
	Media          []Media   `json:"media,omitempty"`
	Poll           *Poll     `json:"poll,omitempty"`
	Place          *Place    `json:"place,omitempty"`
	Source         string    `json:"source,omitempty"`
	Sensitive      bool      `json:"possibly_sensitive,omitempty"`
	ReplySettings  string    `json:"reply_settings,omitempty"`
	Edits          []string  `json:"edits,omitempty"`
	IsRetweet      bool      `json:"is_retweet,omitempty"`
	IsQuote        bool      `json:"is_quote,omitempty"`
	IsReply        bool      `json:"is_reply,omitempty"`
	Provenance     string    `json:"provenance,omitempty"`
}

Tweet (a Post) is the central object. IDs are always strings: an X snowflake does not fit in a JSON number without silent corruption in jq/JavaScript.

func ProfileTimeline

func ProfileTimeline(ctx context.Context, c *Client, handle string, _ int) ([]*Tweet, error)

ProfileTimeline returns a profile's recent public tweets from the timeline-profile widget (Tier 0, no auth). The widget embeds a __NEXT_DATA__ blob whose timeline entries are legacy tweets; one legacy parser maps them.

func TweetByID

func TweetByID(ctx context.Context, c *Client, id string) (*Tweet, error)

TweetByID fetches one tweet via the public syndication endpoint (Tier B).

type User

type User struct {
	ID            string      `json:"id" kit:"id"`
	Username      string      `json:"username"`
	Name          string      `json:"name"`
	CreatedAt     time.Time   `json:"created_at,omitempty"`
	Description   string      `json:"description,omitempty" kit:"body"`
	Location      string      `json:"location,omitempty"`
	URL           string      `json:"url,omitempty"`
	Verified      bool        `json:"verified,omitempty"`
	VerifiedType  string      `json:"verified_type,omitempty"`
	Protected     bool        `json:"protected,omitempty"`
	Metrics       UserMetrics `json:"metrics"`
	ProfileImage  string      `json:"profile_image,omitempty"`
	ProfileBanner string      `json:"profile_banner,omitempty"`
	PinnedTweet   string      `json:"pinned_tweet,omitempty" kit:"link,kind=x/status,optional"`
	Entities      Entities    `json:"entities,omitempty"`
	Kind          string      `json:"kind,omitempty"` // follower|following|liker|retweeter|... when in a list
	Provenance    string      `json:"provenance,omitempty"`
}

User is an account/profile.

func UserByNameSyndication

func UserByNameSyndication(ctx context.Context, c *Client, handle string) (*User, error)

UserByNameSyndication resolves a profile from a single public tweet by the user when possible. The syndication API has no direct profile endpoint, so Tier B user resolution is best-effort: it reads the public profile timeline widget. Returns a NotFoundError if the handle yields nothing.

type UserMetrics

type UserMetrics struct {
	Followers int `json:"followers"`
	Following int `json:"following"`
	Tweets    int `json:"tweets"`
	Listed    int `json:"listed"`
	Likes     int `json:"likes,omitempty"`
	Media     int `json:"media,omitempty"`
}

UserMetrics are the public counters on a profile.

type Variant

type Variant struct {
	Bitrate     int    `json:"bitrate"`
	ContentType string `json:"content_type"`
	URL         string `json:"url"`
}

Variant is one encoding of a video/gif.

type WalkOptions added in v0.4.0

type WalkOptions struct {
	Depth  int     // hops to follow from each seed (0 = seeds only)
	Max    int     // stop after emitting this many nodes (0 = unlimited)
	Fanout int     // per-edge neighbor cap (0 = unlimited)
	Edges  EdgeSet // edges to follow (nil = DefaultEdges)

	// OnEdge, if set, is called for every edge the walk traverses, before the
	// neighbor is visited, with the two endpoints and the edge. The store sink
	// uses it to record the graph; it fires even for an already-visited neighbor
	// so the edge list stays complete.
	OnEdge func(src, dst string, edge Edge)

	// Note, if set, surfaces a one-line advisory (a skipped tier-only edge set, a
	// neighbor that could not be fetched). It never carries a fatal error.
	Note func(string)
}

WalkOptions tunes a traversal.

type Walker added in v0.4.0

type Walker struct {
	// contains filtered or unexported fields
}

Walker performs the breadth-first traversal over a grapher.

func NewWalker added in v0.4.0

func NewWalker(g grapher) *Walker

NewWalker builds a Walker over any grapher (the engine in production, a fake in tests).

func (*Walker) Walk added in v0.4.0

func (w *Walker) Walk(ctx context.Context, seeds []Seed, opts WalkOptions, emit func(*Node) error) error

Walk visits the seeds and their links in breadth-first order, calling emit for each node as it is reached. It returns when the queue drains, the node budget (opts.Max) is hit, emit returns an error, or a seed cannot be fetched. Edges that need a tier are dropped (with a Note) when none is configured, so a Tier-0 walk always produces what it can rather than erroring.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL