linkedin

package
v0.2.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jun 14, 2026 License: Apache-2.0 Imports: 24 Imported by: 0

Documentation

Index

Constants

View Source 
const (
	BaseURL = "https://www.linkedin.com"

	// GuestJobsSearch is the anonymous job-search endpoint. It returns a list of
	// server-rendered <li> job cards and accepts the same query parameters the
	// public jobs page puts on the wire (keywords, location, start, filters).
	GuestJobsSearch = BaseURL + "/jobs-guest/jobs/api/seeMoreJobPostings/search"

	// GuestJobDetail serves one job posting as a clean HTML fragment.
	GuestJobDetail = BaseURL + "/jobs-guest/jobs/api/jobPosting/"

	DefaultDelay    = 2 * time.Second
	DefaultWorkers  = 2
	DefaultTimeout  = 30 * time.Second
	DefaultRetries  = 3
	DefaultCacheTTL = 24 * time.Hour
)

Site constants.

View Source 
const (
	KindProfile = "profile"
	KindCompany = "company"
	KindSchool  = "school"
	KindJob     = "job"
	KindPost    = "post"
	KindUnknown = "unknown"
)

Kinds of LinkedIn entities the tool understands.

Variables

View Source 
var ErrBlocked = errors.New("blocked: LinkedIn refused the page (sign-in wall or bot block)")

ErrBlocked signals that LinkedIn refused the page: an HTTP 999 bot status, a redirect to the sign-in/authwall, or a body that holds no public data where a JSON-LD graph was expected. Callers map it to the "blocked" exit code.

View Source 
var ErrNotFound = errors.New("not found")

ErrNotFound signals a 404. Callers map it to the no-data exit code.

View Source 
var ErrRateLimited = errors.New("rate limited (HTTP 429)")

ErrRateLimited signals a sustained HTTP 429 after retries.

Functions

func CompanyURL 

func CompanyURL(slug string) string

CompanyURL builds the canonical company URL.

func DefaultDataDir 

func DefaultDataDir() string

DefaultDataDir resolves $XDG_DATA_HOME/linkedin (or ~/.local/share/linkedin).

func DocFromBytes 

func DocFromBytes(body []byte) (*goquery.Document, error)

DocFromBytes parses raw HTML bytes into a goquery document.

func JobURL 

func JobURL(id string) string

JobURL builds the canonical job-view URL.

func LoadCookies 

func LoadCookies(path string) ([]*http.Cookie, error)

LoadCookies reads a Netscape/Mozilla cookies.txt file (the format exported by most browser extensions and by curl) into http.Cookie values.

func NormalizeCompanySlug 

func NormalizeCompanySlug(in string) string

NormalizeCompanySlug accepts a slug, a /company/<slug> path, or a full URL and returns the bare slug.

func NormalizeJobID 

func NormalizeJobID(in string) string

NormalizeJobID accepts a numeric id, a urn:li:jobPosting URN, or a /jobs/view/... URL and returns the bare numeric id.

func NormalizeProfileSlug 

func NormalizeProfileSlug(in string) string

NormalizeProfileSlug accepts a slug, an /in/<slug> path, or a full URL and returns the bare slug.

func ProfileURL 

func ProfileURL(slug string) string

ProfileURL builds the canonical profile URL.

func SchoolURL 

func SchoolURL(slug string) string

SchoolURL builds the canonical school URL.

Types

type Affiliation 

type Affiliation struct {
	Name      string `json:"name"`
	URL       string `json:"url"`
	Slug      string `json:"slug"`
	StartDate int    `json:"start_date"`
	EndDate   int    `json:"end_date"`
}

Affiliation is a company or school a person works for or studied at, with the role's year range when LinkedIn exposes it.

type Article 

type Article struct {
	URL       string    `json:"url"`
	Title     string    `json:"title"`
	Author    string    `json:"author"`
	AuthorURL string    `json:"author_url"`
	Published string    `json:"published"`
	Modified  string    `json:"modified"`
	Reactions int64     `json:"reactions"`
	Comments  int64     `json:"comments"`
	ImageURL  string    `json:"image_url"`
	FetchedAt time.Time `json:"fetched_at"`
}

Article is a long-form post a member published, parsed from the Article nodes LinkedIn carries in a profile page's JSON-LD graph.

type Cache 

type Cache struct {
	// contains filtered or unexported fields
}

Cache is a content-addressed, gzipped on-disk page cache. A captured page is stored under cache/<ab>/<sha256>.html.gz, keyed by its URL.

func NewCache 

func NewCache(cfg Config) *Cache

NewCache returns a cache rooted at cfg.CacheDir() (created on first write).

func (*Cache) Clear 

func (c *Cache) Clear() error

Clear removes the entire cache directory.

func (*Cache) Get 

func (c *Cache) Get(url string) ([]byte, bool)

Get returns cached bytes for a URL when present and within the TTL.

func (*Cache) Path 

func (c *Cache) Path(url string) string

Path returns the on-disk cache path for a URL (whether or not it exists).

func (*Cache) Put 

func (c *Cache) Put(url string, body []byte) error

Put writes bytes for a URL into the cache.

func (*Cache) Stats 

func (c *Cache) Stats() (files int, bytes int64, err error)

Stats reports the file count and total bytes held in the cache.

type Client 

type Client struct {
	// contains filtered or unexported fields
}

Client performs polite, retrying HTTP GETs against linkedin.com.

func NewClient 

func NewClient(cfg Config) *Client

NewClient builds an anonymous client.

func NewClientWithCookies 

func NewClientWithCookies(cfg Config, cookies []*http.Cookie) (*Client, error)

NewClientWithCookies builds a client pre-loaded with a lent session.

func (*Client) CachingFetch 

func (c *Client) CachingFetch(ctx context.Context, cache *Cache, cfg Config, url string) ([]byte, error)

CachingFetch returns page bytes, reading the cache unless NoCache/Refresh is set, and populating it on a miss.

func (*Client) CachingFetchHTML 

func (c *Client) CachingFetchHTML(ctx context.Context, cache *Cache, cfg Config, url string) (*goquery.Document, error)

CachingFetchHTML is CachingFetch plus goquery parsing.

func (*Client) Fetch 

func (c *Client) Fetch(ctx context.Context, rawurl string) ([]byte, int, error)

Fetch returns the raw body and status for a URL, retrying transient failures. A 404 returns (nil, 404, nil). A sign-in redirect or HTTP 999 returns ErrBlocked.

func (*Client) FetchCompany 

func (c *Client) FetchCompany(ctx context.Context, cache *Cache, cfg Config, in string) (*Company, error)

FetchCompany fetches and parses a public company page.

func (*Client) FetchCompanyAffiliated 

func (c *Client) FetchCompanyAffiliated(ctx context.Context, cache *Cache, cfg Config, in string) ([]OrgRef, error)

FetchCompanyAffiliated returns the related pages (affiliated and showcase) a company links to from its page.

func (*Client) FetchCompanyLocations 

func (c *Client) FetchCompanyLocations(ctx context.Context, cache *Cache, cfg Config, in string) ([]Location, error)

FetchCompanyLocations returns the offices a company lists on its page.

func (*Client) FetchCompanyPosts 

func (c *Client) FetchCompanyPosts(ctx context.Context, cache *Cache, cfg Config, in string) ([]Post, error)

FetchCompanyPosts returns the recent company posts carried as DiscussionForumPosting nodes in the company page's JSON-LD graph.

func (*Client) FetchHTML 

func (c *Client) FetchHTML(ctx context.Context, rawurl string) (*goquery.Document, int, error)

FetchHTML fetches a URL and parses it into a goquery document.

func (*Client) FetchJob 

func (c *Client) FetchJob(ctx context.Context, cache *Cache, cfg Config, id string) (*Job, error)

FetchJob fetches one job posting through the guest detail fragment.

func (*Client) FetchPost 

func (c *Client) FetchPost(ctx context.Context, cache *Cache, cfg Config, in string) (*Post, error)

FetchPost fetches a public post or article, best effort.

func (*Client) FetchProfile 

func (c *Client) FetchProfile(ctx context.Context, cache *Cache, cfg Config, in string) (*Profile, error)

FetchProfile fetches and parses a public member profile. The input may be a slug, an /in/<slug> path, or a full URL.

func (*Client) FetchProfileArticles 

func (c *Client) FetchProfileArticles(ctx context.Context, cache *Cache, cfg Config, in string) ([]Article, error)

FetchProfileArticles returns the long-form articles carried as Article nodes in a member profile page's JSON-LD graph.

func (*Client) FetchProfilePosts 

func (c *Client) FetchProfilePosts(ctx context.Context, cache *Cache, cfg Config, in string) ([]Post, error)

FetchProfilePosts returns the recent posts carried as DiscussionForumPosting nodes in a member profile page's JSON-LD graph.

func (*Client) SearchJobs 

func (c *Client) SearchJobs(ctx context.Context, cache *Cache, cfg Config, opts JobSearchOptions, limit int) ([]JobStub, error)

SearchJobs walks the guest search endpoint, page by page, until it has limit stubs or the endpoint returns an empty page. A limit of 0 means "one page".

type Company 

type Company struct {
	Slug          string    `json:"slug"`
	URL           string    `json:"url"`
	Name          string    `json:"name"`
	Description   string    `json:"description"`
	Slogan        string    `json:"slogan"`
	Website       string    `json:"website"`
	Followers     int64     `json:"followers"`
	Employees     int64     `json:"employees"`
	Industry      string    `json:"industry"`
	CompanySize   string    `json:"company_size"`
	CompanyType   string    `json:"company_type"`
	Founded       string    `json:"founded"`
	Specialties   string    `json:"specialties"`
	Headquarters  string    `json:"headquarters"`
	Street        string    `json:"street"`
	Locality      string    `json:"locality"`
	Region        string    `json:"region"`
	PostalCode    string    `json:"postal_code"`
	Country       string    `json:"country"`
	LogoURL       string    `json:"logo_url"`
	FundingRounds int       `json:"funding_rounds"`
	FundingURL    string    `json:"funding_url"`
	FetchedAt     time.Time `json:"fetched_at"`
}

Company is a public company page. The core fields come from the Organization JSON-LD; the about-section fields (followers, industry, size band, type, headquarters, founded, specialties) come from the page's about panel, which carries detail the JSON-LD omits.

func ParseCompany 

func ParseCompany(doc *goquery.Document, slug, pageURL string) (*Company, error)

ParseCompany parses a company page into a Company (JSON-LD Organization).

type Config 

type Config struct {
	DataDir    string        // root for cache/store; defaults to XDG data dir
	Workers    int           // concurrency for multi-page/bulk work
	Delay      time.Duration // minimum spacing between requests
	Timeout    time.Duration // per-request timeout
	Retries    int           // retry attempts on 429/5xx
	CacheTTL   time.Duration // on-disk page-cache freshness window
	NoCache    bool          // bypass the on-disk page cache entirely
	Refresh    bool          // force re-fetch and overwrite the cache
	CookiePath string        // optional Netscape cookie jar for lent sessions
}

Config holds runtime configuration for the library.

func DefaultConfig 

func DefaultConfig() Config

DefaultConfig returns sensible, polite defaults rooted at the XDG data dir.

func (Config) CacheDir 

func (c Config) CacheDir() string

CacheDir is where gzipped page captures live.

func (Config) StorePath 

func (c Config) StorePath() string

StorePath is the default path of the optional SQLite store.

type Job 

type Job struct {
	JobID          string    `json:"job_id"`
	URL            string    `json:"url"`
	Title          string    `json:"title"`
	Company        string    `json:"company"`
	CompanyURL     string    `json:"company_url"`
	CompanySlug    string    `json:"company_slug"`
	Location       string    `json:"location"`
	Posted         string    `json:"posted"`
	Applicants     int       `json:"applicants"`
	Seniority      string    `json:"seniority"`
	EmploymentType string    `json:"employment_type"`
	JobFunction    string    `json:"job_function"`
	Industries     string    `json:"industries"`
	Description    string    `json:"description"`
	FetchedAt      time.Time `json:"fetched_at"`
}

Job is a single job posting, parsed from the guest job-detail fragment.

type JobSearchOptions 

type JobSearchOptions struct {
	Keywords   string
	Location   string
	GeoID      string
	Posted     string // f_TPR, e.g. r86400 (24h), r604800 (week), r2592000 (month)
	Remote     string // f_WT: 1 on-site, 2 remote, 3 hybrid
	Experience string // f_E: 1..6
	JobType    string // f_JT: F,P,C,T,I,V,O
	Sort       string // sortBy: R relevance, DD date
}

JobSearchOptions carries the query for the guest job-search endpoint. Every field maps to a parameter the public jobs page puts on the wire (spec §4.1).

type JobStub 

type JobStub struct {
	JobID       string    `json:"job_id"`
	URL         string    `json:"url"`
	Title       string    `json:"title"`
	Company     string    `json:"company"`
	CompanyURL  string    `json:"company_url"`
	CompanySlug string    `json:"company_slug"`
	Location    string    `json:"location"`
	Posted      string    `json:"posted"`
	Benefits    string    `json:"benefits"`
	FetchedAt   time.Time `json:"fetched_at"`
}

JobStub is one job card from the guest search endpoint: enough to list and to fetch the full Job.

type Location 

type Location struct {
	Slug      string    `json:"slug"`
	Primary   bool      `json:"primary"`
	Street    string    `json:"street"`
	Address   string    `json:"address"`
	URL       string    `json:"url"`
	FetchedAt time.Time `json:"fetched_at"`
}

Location is one office a company lists on its page. The full set is read with `company --locations`; the primary office is the registered headquarters.

func ParseCompanyLocations 

func ParseCompanyLocations(doc *goquery.Document, slug, pageURL string, now time.Time) []Location

ParseCompanyLocations reads the office cards a company page lists, one per "Get directions" entry, marking the primary office (the headquarters).

type OrgRef 

type OrgRef struct {
	Slug      string    `json:"slug"`
	Name      string    `json:"name"`
	URL       string    `json:"url"`
	Industry  string    `json:"industry"`
	Location  string    `json:"location"`
	FetchedAt time.Time `json:"fetched_at"`
}

OrgRef is a related page a company links to (an affiliated page or showcase), read with `company --affiliated`.

func ParseCompanyAffiliated 

func ParseCompanyAffiliated(doc *goquery.Document, now time.Time) []OrgRef

ParseCompanyAffiliated reads the related pages a company links to (affiliated pages and showcase pages), each with its name, industry, and location.

type Post 

type Post struct {
	URL       string    `json:"url"`
	Author    string    `json:"author"`
	AuthorURL string    `json:"author_url"`
	Title     string    `json:"title"`
	Text      string    `json:"text"`
	Published string    `json:"published"`
	Modified  string    `json:"modified"`
	Likes     int64     `json:"likes"`
	Comments  int64     `json:"comments"`
	ImageURL  string    `json:"image_url"`
	FetchedAt time.Time `json:"fetched_at"`
}

Post is a best-effort public post or article, parsed from Open Graph tags and any Article/DiscussionForumPosting JSON-LD that serves anonymously.

func ParsePost 

func ParsePost(doc *goquery.Document, pageURL string) (*Post, error)

ParsePost parses a public post or article into a Post, JSON-LD first and Open Graph as a backstop.

type Profile 

type Profile struct {
	Slug        string        `json:"slug"`
	URL         string        `json:"url"`
	Name        string        `json:"name"`
	Headline    string        `json:"headline"`
	Description string        `json:"description"`
	JobTitles   []string      `json:"job_titles"`
	Location    string        `json:"location"`
	Country     string        `json:"country"`
	Followers   int64         `json:"followers"`
	ImageURL    string        `json:"image_url"`
	Languages   []string      `json:"languages"`
	Awards      []string      `json:"awards"`
	WorksFor    []Affiliation `json:"works_for"`
	AlumniOf    []Affiliation `json:"alumni_of"`
	MemberOf    []Affiliation `json:"member_of"`
	SameAs      []string      `json:"same_as"`
	FetchedAt   time.Time     `json:"fetched_at"`
}

Profile is a public member profile, parsed from the page's Person JSON-LD.

func ParseProfile 

func ParseProfile(doc *goquery.Document, slug, pageURL string) (*Profile, error)

ParseProfile parses a member page into a Profile (JSON-LD Person first).

type Ref 

type Ref struct {
	Input string `json:"input"`
	Kind  string `json:"kind"`
	ID    string `json:"id"`
	URL   string `json:"url"`
}

Ref is the classification of a user-supplied input (the `id` command output).

func Classify 

func Classify(input string) Ref

Classify inspects an input and returns its kind, id/slug, and canonical URL.

type Store 

type Store struct {
	// contains filtered or unexported fields
}

Store is a pure-Go SQLite store for parsed records.

func OpenStore 

func OpenStore(path string) (*Store, error)

OpenStore opens (creating if needed) the SQLite store at path and migrates it.

func (*Store) Close 

func (s *Store) Close() error

Close closes the underlying database.

func (*Store) CountsByKind 

func (s *Store) CountsByKind() (map[string]int, error)

CountsByKind returns a map of record kind to count.

func (*Store) Each 

func (s *Store) Each(kind string, fn func(id string, data []byte) error) error

Each streams stored records of a kind to fn as raw JSON ("" for all kinds).

func (*Store) Get 

func (s *Store) Get(kind, id string) ([]byte, error)

Get reads a record's raw JSON by (kind, id).

func (*Store) Info 

func (s *Store) Info() (string, error)

Info returns a human summary of the store's contents.

func (*Store) Put 

func (s *Store) Put(kind, id, url string, record any) error

Put upserts a record keyed by (kind, id).

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.