library

package
v0.3.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 30, 2026 License: Apache-2.0 Imports: 28 Imported by: 0

Documentation

Overview

Package library is the core of Filiation. Every user-visible operation — add, expand, search, ask, path — is a function here.

Everything else is arranged around this package:

  • The front doors (cmd/, internal/mcpsrv, internal/web) are translation layers. Each parses a request, calls one function here, and formats the result. If a front door imports internal/store, that is a bug.
  • This package orchestrates: it calls store, sources, graph, retrieve and jobs. Those packages do not call back into it.
  • It is the composition root: the one place that builds a store and a source side by side. It never moves data between them — that is internal/graph's job — so each stays swappable on its own.

Nothing a front door needs from here is a store type. Results come back as model types or types declared in this package, and errors as errs sentinels, so cmd/ can branch on every outcome without importing store.

The practical test: adding a command to the CLI, an MCP tool and a web endpoint for the same feature should mean one function here and three thin wrappers, not three implementations.

Index

Constants

View Source 
const (
	// DefaultFetchMax is how many PDFs one fetch downloads unless told
	// otherwise. At spike 9's median sizes that is 25–50 MB and a few
	// minutes; run fetch again to continue. Disk and bandwidth are the
	// user's.
	DefaultFetchMax = 50

	// FetchRetryAfter is how long a link that failed is left alone. A host
	// that refused once will usually refuse again tomorrow.
	FetchRetryAfter = 30 * 24 * time.Hour

	// MaxPDFSize stops a runaway download (ADR-013).
	MaxPDFSize = 100 << 20
)
View Source 
const (
	ImportImported      = store.ImportImported
	ImportDuplicate     = store.ImportDuplicate
	ImportUnconfirmed   = store.ImportUnconfirmed
	ImportCandidates    = store.ImportCandidates
	ImportUnidentified  = store.ImportUnidentified
	ImportNotInOpenAlex = store.ImportNotInOpenAlex
	ImportUnreadable    = store.ImportUnreadable
)

Import outcomes, as ImportedFile.Outcome reports them. Only Imported files a PDF; the rest say why not.

View Source 
const (
	// DefaultLocateMax is how many works one lookup of PDF links covers
	// unless told otherwise: 20 requests, 20 credits of 1,000 a day (D11).
	DefaultLocateMax = 2000

	// LocateMaxAge is how old a PDF answer may be before it is asked again.
	// Repositories catch up — a preprint is posted, an embargo ends — so
	// "no open copy" is an answer for a month, not for ever.
	LocateMaxAge = 30 * 24 * time.Hour
)
View Source 
const (
	// ExtractTimeout is the longest one PDF may take to read. A 70-page
	// paper takes about two seconds; a file still going at the limit is
	// pathological, and is recorded as timed out rather than holding up
	// the run (ARCHITECTURE_PHASE3.md §7).
	ExtractTimeout = 2 * time.Minute

	// DefaultSearchLimit is how many passages a search returns, and
	// DefaultPerPaper the most from one paper, so one long paper that uses
	// the words throughout does not crowd out the rest.
	DefaultSearchLimit = 20
	DefaultPerPaper    = 3
)
View Source 
const (
	MarkStart = store.MarkStart
	MarkEnd   = store.MarkEnd
)

MarkStart and MarkEnd surround the matched words in a SearchHit's Snippet, for a front door to show as it likes.

View Source 
const DefaultMaxHops = 6

DefaultMaxHops is how far Path searches by default. Six citation steps reaches across almost any library; a chain longer than that says little about how two papers are related.

View Source 
const DefaultTitleSearches = 20

DefaultTitleSearches is how many title searches one import may spend. Each costs 10 of OpenAlex's 1,000 free daily credits (D11); finding an identifier in the PDF costs nothing.

Variables

View Source 
var ErrNoFolder = errors.New("this library has no folder yet")

ErrNoFolder is returned by what needs a library folder, from a library that is only a database: one made by v0.1 or v0.2, or opened with --db. fil library move gives it a folder.

View Source 
var ErrNoPDF = errors.New("no PDF in the library folder")

ErrNoPDF means the paper has no PDF in papers/ to read.

Functions

func ClearCache 

func ClearCache(dir string) (string, error)

ClearCache deletes every cached HTTP response under dir, or under config.DefaultCacheDir if dir is empty, and returns the directory it cleared. It is `fil cache clear`, and does not need the library open.

func Create added in v0.3.0 

func Create(ctx context.Context, cfg *config.Config, dir string, makeDefault bool) (*config.Config, error)

Create makes a new library folder at dir, with its database, and lists it in the global settings — as the default when makeDefault is set. The library cfg names is left as it is.

func RetryAfter 

func RetryAfter(err error) (time.Duration, bool)

RetryAfter returns how long OpenAlex asked us to wait, when err is a transient failure that carried the request. A front door uses it to say "try again in an hour — today's allowance is spent" rather than just "failed".

func Validate 

func Validate(input string) error

Validate reports whether input is something Add could look up, without touching the network or the library. A front door calls it first, so that a typo is refused before anything is created — including, on a first run, the library itself.

Types

type AddOptions 

type AddOptions struct {
	// AcceptFirst takes the best title-search candidate without asking. It is
	// --accept-first, for scripts, and it is the only way a title seeds a
	// graph unattended (ADR-005).
	AcceptFirst bool

	// SearchLimit is how many title candidates to offer. Zero means the
	// source's default.
	SearchLimit int
}

AddOptions controls Add.

type Added 

type Added struct {
	// Work is the seed as stored.
	Work model.Work

	// AlreadySeed reports the work was a seed before this call, so nothing
	// about it changed. The CLI says "already in your library" rather than
	// printing zeros that look like a failure.
	AlreadySeed bool

	// Refs is how many works the seed cites. NewStubs of them were not yet in
	// the library, and NewEdges citation edges were recorded.
	Refs     int
	NewStubs int
	NewEdges int

	// DeadEnd reports the seed cites nothing OpenAlex could resolve, so the
	// graph cannot grow from it. In arts and humanities that is most works
	// (D12). It is reported, not raised: it is the data, not the tool.
	DeadEnd bool
}

Added reports what Add did.

type AmbiguousError 

type AmbiguousError struct {
	Title      string
	Candidates []Candidate
}

AmbiguousError is returned by Add when a title matched candidates and no choice was made. It matches errs.ErrAmbiguous.

func (*AmbiguousError) Error 

func (e *AmbiguousError) Error() string

func (*AmbiguousError) Is 

func (e *AmbiguousError) Is(target error) bool

Is makes an *AmbiguousError match errs.ErrAmbiguous.

type Attempt added in v0.3.0 

type Attempt = store.Attempt

Attempt is one download attempt: the link, its host, how it ended.

type Candidate 

type Candidate struct {
	Work model.Work

	// TitleMatch flags a candidate whose title is, allowing for case,
	// punctuation and a typo, the one typed. Such candidates come first. It is
	// a hint for the person choosing, never a choice (ADR-005).
	TitleMatch bool
	// contains filtered or unexported fields
}

Candidate is one title-search result offered to the user.

type CitedIn added in v0.3.0 

type CitedIn struct {
	Work model.Work
	Mention
}

CitedIn is a reference and one sentence the paper cites it in.

type Collection added in v0.3.0 

type Collection struct {
	Name       string
	Works      int
	FromFolder bool // mirrors a folder in papers/
}

Collection is one collection and how many works it holds.

type ContextCounts added in v0.3.0 

type ContextCounts = store.ContextCounts

ContextCounts says how far citation context reaches: of the citations made by papers whose PDF has text, how many carry the sentence they were made in.

type DuplicateGroup 

type DuplicateGroup = graph.DuplicateGroup

DuplicateGroup is a set of fetched works that share a title: probably one paper OpenAlex holds as more than one record.

type ExpandOptions 

type ExpandOptions struct {
	MaxNodes int
	MaxDepth int

	// Progress, if set, is called after each batch is committed.
	Progress func(model.ExpansionResult)
}

ExpandOptions bound one expansion. Zero fields take the configured value — max_nodes, max_depth and max_refs_per_work from the config file, or their defaults — so a front door passes only what the user overrode.

type ExportOptions 

type ExportOptions struct {
	// IncludeStubs exports every work, including the many known only by ID.
	// Without it the export holds fetched works and the citations between
	// them: after one expansion that is ~500 readable nodes rather than ~500
	// buried among ~7,000 that have no title, year or type.
	IncludeStubs bool
}

ExportOptions control an export.

type Exported 

type Exported struct {
	Nodes, Edges int
}

Exported reports what an export wrote.

type ExtractOptions added in v0.3.0 

type ExtractOptions struct {
	// Again reads every PDF, or the named paper's, even if already read.
	Again bool
	// Progress, if set, is called after each paper with the running totals.
	Progress func(Extracted)
}

ExtractOptions bound one reading of the library's PDFs.

type Extracted added in v0.3.0 

type Extracted struct {
	Total    int            // PDFs to read
	Done     int            // PDFs read
	Counts   map[string]int // by text.Status
	Passages int
	Mentions int // places the papers cite their references, found in their text
	Duration time.Duration
	Papers   []ExtractedPaper

	// Imported is what was filed from Inbox/ first: PDFs dropped there that
	// identify themselves (ADR-014).
	Imported Imported
}

Extracted is what reading the PDFs did.

type ExtractedPaper added in v0.3.0 

type ExtractedPaper struct {
	Work     model.Work
	File     string // absolute path of the PDF read
	Status   string // text.Status
	Reason   string // why there is no text, when Status is not ok
	Pages    int
	Passages int
	Mentions int // places it cites a reference, found in its text
	Took     time.Duration
	Already  bool // read before, from the same file; nothing was done
}

ExtractedPaper is what reading one paper's PDF gave.

type FetchOptions added in v0.3.0 

type FetchOptions struct {
	// Max is the most PDFs to download; zero means DefaultFetchMax.
	Max int
	// Progress, if set, is called after each paper with the running totals.
	Progress func(Fetched)
	// ReadProgress, if set, is called after each PDF is read, once the
	// downloads are done.
	ReadProgress func(Extracted)
}

FetchOptions bound one fetch.

type Fetched added in v0.3.0 

type Fetched struct {
	Located    Located // links looked up first, for papers never asked about
	Hydrated   int     // papers known only by ID, fetched first so they have a title
	Tried      int     // papers a download was tried for
	Downloaded int     // papers that now have a PDF
	Bytes      int64
	Duration   time.Duration

	// Outcomes counts the failed attempts by why they failed.
	Outcomes map[string]int

	Papers []FetchedPaper

	// Read is the reading of the PDFs that followed the downloads: every PDF
	// in papers/ not yet read, whether downloaded now or before.
	Read Extracted
}

Fetched is what a fetch did.

type FetchedPaper added in v0.3.0 

type FetchedPaper struct {
	Work     model.Work
	File     string // absolute path of the PDF, when one was saved or was already there
	Already  bool   // the PDF was already in the library; nothing was downloaded
	Attempts []Attempt
	Text     *ExtractedPaper // reading the PDF, when there is one
}

FetchedPaper is what happened for one paper.

type File added in v0.3.0 

type File struct {
	Path    string // absolute
	Version string // published | accepted | preprint, or empty
	Size    int64
	Missing bool // gone from disk at the last scan
}

File is a file that belongs to a work.

type ImportCandidate added in v0.3.0 

type ImportCandidate struct {
	Work     model.Work
	Verified bool // its title and first author are in the PDF
	// contains filtered or unexported fields
}

ImportCandidate is a work a PDF might be.

type ImportOptions added in v0.3.0 

type ImportOptions struct {
	// Files are the PDFs to import. Empty means everything in Inbox/ and any
	// PDF in papers/ that is not yet any paper's.
	Files []string
	// Again tries files an earlier import could not identify, unchanged
	// since. fil import does; fetch and extract, picking up Inbox/, do not.
	Again bool
	// TitleSearches is the most title searches to spend; zero means
	// DefaultTitleSearches, and a negative number none.
	TitleSearches int
	// AcceptFirst files a title search's best candidate without asking —
	// but only one whose title and first author are in the PDF.
	AcceptFirst bool
	// Progress, if set, is called after each file.
	Progress func(ImportedFile)
}

ImportOptions shape one import.

type Imported added in v0.3.0 

type Imported struct {
	Files    []ImportedFile
	Counts   map[string]int // by outcome
	Searches int            // title searches spent
	Duration time.Duration
}

Imported is what an import did.

type ImportedFile added in v0.3.0 

type ImportedFile struct {
	File    string // where it was, absolute
	Saved   string // where it is now, absolute, when imported
	Outcome string // Import*
	Reason  string // why, when it was not imported
	Work    model.Work
	Clue    string // where its identifier was found, when one was used
	Skipped bool   // an earlier import found nothing, and the file has not changed

	// Candidates are the works the user may choose from, when Outcome is
	// unconfirmed or candidates. Verified ones have their title and first
	// author in the PDF.
	Candidates []ImportCandidate
	Text       *ExtractedPaper // reading the PDF, once filed
}

ImportedFile is what happened to one PDF.

type Library 

type Library struct {
	// contains filtered or unexported fields
}

Library is an open library: the store, and the sources that grow it.

This is the composition root. It is the one place that builds both a store and a source — but it never moves data between them. That is internal/graph's job, so the two stay swappable independently.

func Open 

func Open(ctx context.Context, cfg *config.Config, opts Options) (*Library, error)

Open opens the library cfg names, creating and migrating it on first use. A library folder gets its papers/, Inbox/ and exports/ folders if they are missing.

func (*Library) Add 

func (l *Library) Add(ctx context.Context, input string, opts AddOptions) (Added, error)

Add adds a paper to the library as a seed: the work, a stub for everything it cites, and the edges between them.

input is whatever the user typed. A DOI, arXiv ID, PMID or OpenAlex ID resolves directly. A title is searched, and unless opts.AcceptFirst is set the candidates come back in an *AmbiguousError and nothing is written — the user chooses and the front door calls AddCandidate. Errors to expect, all compared with errors.Is: 

errs.ErrInvalidInput  the input is not something that can be looked up
errs.ErrUnresolved    OpenAlex has no such work (or no title candidates)
errs.ErrAmbiguous     a title matched candidates; see *AmbiguousError
errs.ErrTransient     the network failed; trying again later may work

func (*Library) AddCandidate 

func (l *Library) AddCandidate(ctx context.Context, c Candidate) (Added, error)

AddCandidate adds the title-search candidate the user chose. The candidate arrived fully fetched, so this makes no request.

func (*Library) Attempts added in v0.3.0 

func (l *Library) Attempts(ctx context.Context, input string) ([]Attempt, error)

Attempts returns the download attempts for the paper input names, newest first, so a paper without a PDF can say exactly why.

func (*Library) Budget 

func (l *Library) Budget() (maxNodes, maxDepth int)

Budget is the configured expansion budget: the default a front door shows when the user has not overridden it.

func (*Library) CitationContexts added in v0.3.0 

func (l *Library) CitationContexts(ctx context.Context, input string) (map[string][]Mention, error)

CitationContexts returns where the paper input names cites each of its references, keyed by the cited work's fil ID, in reading order. A reference cited nowhere fil could find has no entry — no match, no context (ADR-015).

func (*Library) Close 

func (l *Library) Close() error

Close closes the library.

func (*Library) Cloud added in v0.3.0 

func (l *Library) Cloud() string

Cloud names the sync service the library folder is in, or is empty. Its database is then kept outside the folder.

func (*Library) Collections added in v0.3.0 

func (l *Library) Collections(ctx context.Context) ([]Collection, error)

Collections lists the library's collections.

func (*Library) ContextCounts added in v0.3.0 

func (l *Library) ContextCounts(ctx context.Context) (ContextCounts, error)

ContextCounts counts citation context across the library.

func (*Library) Expand 

func (l *Library) Expand(ctx context.Context, opts ExpandOptions) (model.ExpansionResult, error)

Expand grows the graph outward from everything already in the library, best first, until the budget is spent, the frontier is empty, the depth limit is reached, or ctx is cancelled. Each of those is a normal outcome, reported in the result's StoppedBecause with a nil error. Run it again to continue: the frontier is persisted state, so a stopped run resumes where it left off.

An error means something failed — most often the network, wrapping errs.ErrTransient — and the result still reports what was saved before it.

func (*Library) ExportGraphML 

func (l *Library) ExportGraphML(ctx context.Context, w io.Writer, opts ExportOptions) (Exported, error)

ExportGraphML writes the library as GraphML to w, for Gephi, Cytoscape, NetworkX and the rest. It streams, so a library of any size exports in constant memory. An arrow means "cites".

func (*Library) ExtractOne added in v0.3.0 

func (l *Library) ExtractOne(ctx context.Context, input string, again bool) (ExtractedPaper, error)

ExtractOne reads the text of the paper input names. Naming a paper is asking for it: with again, it is read even if it already was.

func (*Library) ExtractText added in v0.3.0 

func (l *Library) ExtractText(ctx context.Context, opts ExtractOptions) (Extracted, error)

ExtractText reads the text of every PDF in papers/ not yet read — new, changed, or read by an older version of fil — so it can be searched. It scans papers/ first, so a PDF you put there yourself is read too.

A paper whose PDF has no usable text is recorded with the reason — a scan, a garbled text layer, a damaged file — and not tried again until the file changes. Cancelling keeps every paper already read.

func (*Library) Fetch added in v0.3.0 

func (l *Library) Fetch(ctx context.Context, opts FetchOptions) (Fetched, error)

Fetch downloads the open-access PDFs of up to Max papers into papers/ — your own papers first, then the ones your library cites most (ADR-013).

It scans papers/ first, so a PDF you added or moved counts and a paper whose PDF you deleted is not downloaded again. It looks up links for papers never asked about. Then, four papers at a time, it tries each paper's links best first — published, accepted, preprint — until one gives a PDF, at most three. Every attempt is recorded; a link that failed is left alone for a month. Cancelling keeps every finished download.

Once the downloads are done, it reads the text of every PDF not yet read, so a search finds them (ExtractText).

func (*Library) FetchOne added in v0.3.0 

func (l *Library) FetchOne(ctx context.Context, input string) (FetchedPaper, error)

FetchOne downloads the PDF of the paper input names. Naming a paper is asking for it: every link is tried, including ones that failed before, and a PDF you deleted is downloaded again.

func (*Library) FetchedOnDisk added in v0.3.0 

func (l *Library) FetchedOnDisk(ctx context.Context) (int, error)

FetchedOnDisk counts the papers that have a PDF in papers/.

func (*Library) Find 

func (l *Library) Find(ctx context.Context, input string) (model.Work, error)

Find resolves what the user typed to a work already in the library. It never asks OpenAlex: a paper that has not been added has no neighbours and no paths, and saying so is more useful than fetching it behind the user's back.

A DOI, arXiv ID, PMID or OpenAlex ID is looked up directly. A title is matched against the titles in the library, allowing for case, punctuation and a typo; one match is used, several come back in an *AmbiguousError. For a read-only question that is safe in a way it is not for Add: nothing is written, and the answer names the work it used.

A work that is not there wraps errs.ErrNotFound.

func (*Library) Folder added in v0.3.0 

func (l *Library) Folder() string

Folder is the library folder, or "" for a library that is only a database.

func (*Library) FolderFor added in v0.3.0 

func (l *Library) FolderFor(ctx context.Context, input string) (WorkFolder, error)

FolderFor returns the folder of the work input names, creating it in papers/ if the work has none.

It scans first, so a folder the researcher moved is found where it is now rather than made again. A folder that still has the name fil gave it is renamed when the name would now be different — a stub has since been fetched, say, and has a title. A folder a person named is never renamed.

func (*Library) Import added in v0.3.0 

func (l *Library) Import(ctx context.Context, opts ImportOptions) (Imported, error)

Import identifies and files PDFs the researcher already has (ADR-014).

For each PDF it collects clues — the arXiv stamp, the file name, a DOI in the page header, the PDF's metadata — and looks each up, free. A clue is trusted only if the work it names has its title and first author's surname in the PDF's first pages: then the paper is added as a seed you uploaded, the file goes into its folder in papers/, and its text is read. A PDF with no clue that checks out gets a title search, within the budget; a search never files anything on its own (ADR-005) — the candidates come back for you to choose from, unless AcceptFirst.

Where a file goes depends on where it was: from Inbox/ it moves into the paper's folder; one already in papers/ stays where you put it; one from anywhere else is copied, and the original is never touched. Bytes the library already holds are reported, not stored twice.

func (*Library) ImportAs added in v0.3.0 

func (l *Library) ImportAs(ctx context.Context, file, input string, force bool) (ImportedFile, error)

ImportAs files the PDF at file as the work input names — a DOI, arXiv ID, OpenAlex ID or PMID. It is still checked against the PDF: a work whose title is not in it is returned unconfirmed, with the work as its one candidate, unless force — the researcher has seen the mismatch and said yes. A PDF with no text cannot be checked, and is filed as named.

func (*Library) ImportChosen added in v0.3.0 

func (l *Library) ImportChosen(ctx context.Context, file string, c ImportCandidate) (ImportedFile, error)

ImportChosen files the PDF at file as a candidate the researcher chose from an earlier import's answer. No further check: they chose.

func (*Library) ImportLocal added in v0.3.0 

func (l *Library) ImportLocal(ctx context.Context, file, title string, year *int) (ImportedFile, error)

ImportLocal files the PDF at file as a local document: a paper OpenAlex does not know, with the title the researcher gives it, a fil ID, its text searchable, and no citations (ADR-014).

func (*Library) InboxWaiting added in v0.3.0 

func (l *Library) InboxWaiting() (int, error)

InboxWaiting counts the PDFs in Inbox/: dropped there and not filed yet.

func (*Library) List added in v0.3.0 

func (l *Library) List(ctx context.Context, opts ListOptions) ([]Listed, error)

List lists the library: seeds first, then the most cited within it.

func (*Library) LocatePDFs added in v0.3.0 

func (l *Library) LocatePDFs(ctx context.Context, opts LocateOptions) (Located, error)

LocatePDFs looks up the open-access PDF links of works whose links are not known yet, or were looked up more than a month ago — your own papers first, then the ones your library cites most (ADR-013). Nothing is downloaded.

func (*Library) Neighbours 

func (l *Library) Neighbours(ctx context.Context, input string) (Neighbourhood, error)

Neighbours returns what a work in the library cites and what cites it.

func (*Library) PDFCounts added in v0.3.0 

func (l *Library) PDFCounts(ctx context.Context) (map[model.PDFAvailability]int, error)

PDFCounts counts the library's works by what is known about their PDF; works never looked up count under model.PDFUnchecked. 

func (l *Library) PDFLinks(ctx context.Context, input string) (PDFLinks, error)

PDFLinks returns what is known about the PDF of the work input names, looking it up first — one request — if it never has been, or if the answer is older than LocateMaxAge, as a library-wide lookup would.

func (*Library) Path 

func (l *Library) Path() string

Path is where the library lives on disk.

func (*Library) PathBetween 

func (l *Library) PathBetween(ctx context.Context, from, to string, opts PathOptions) (Path, error)

PathBetween finds the shortest chain of citations between two works in the library.

By default it looks for lineage — the project's name for it (D6): a chain of references from one work to the other, so that one descends from the other. It tries from the first to the second, then the reverse, and always reports the chain in the order asked. With AnyDirection it finds any connection.

func (*Library) ProbableDuplicates 

func (l *Library) ProbableDuplicates(ctx context.Context) ([]DuplicateGroup, error)

ProbableDuplicates lists fetched works that share a title. It reports and never merges: a matching title is evidence, not proof — a book review carries the title of the book it reviews (D14).

func (*Library) Quota 

func (l *Library) Quota() (Quota, bool)

Quota returns the last OpenAlex allowance seen, and false before any request has reported one — including when every answer came from the cache.

func (*Library) Scan added in v0.3.0 

func (l *Library) Scan(ctx context.Context) (ScanReport, error)

Scan brings the library's record of papers/ up to date with what is on disk (ADR-012). It runs when a command needs it, not continuously.

A work folder is recognised by the fil ID at the end of its name, so a folder the researcher moved or renamed is found wherever it now sits. A folder whose ID was taken off its name is recognised by the hashes of its files. A folder that is gone is marked missing — the paper stays in the graph, because the citation is still true, and fil will not download it again unasked. New files in a work folder are hashed and recorded with it. Nothing on disk is changed.

Hashing happens before the write transaction opens, so the single writer is held only for the update itself.

func (*Library) Search added in v0.3.0 

func (l *Library) Search(ctx context.Context, words string, opts SearchOptions) (Searched, error)

Search finds passages in the library's PDFs by keyword: every word must appear, "quoted phrases" in order, and a word ending in * matches as a prefix. When no passage has every word, it returns those with some of them, and says so (Partial). Words match by stem, so "citing" finds "cited". Reference lists are left out unless asked for.

func (*Library) Show added in v0.3.0 

func (l *Library) Show(ctx context.Context, input string) (Paper, error)

Show gathers everything the library knows about the paper input names: what fil show prints and get_paper returns.

func (*Library) Stats 

func (l *Library) Stats(ctx context.Context) (Stats, error)

Stats reads the library's counts.

func (*Library) Summarise 

func (l *Library) Summarise(ctx context.Context) (Summary, error)

Summarise counts the library.

func (*Library) TextCounts added in v0.3.0 

func (l *Library) TextCounts(ctx context.Context) (map[string]int, error)

TextCounts counts the papers whose PDF has been read, by what reading it gave: ok, no-text, gibberish, unreadable, timed-out.

func (*Library) WithoutText added in v0.3.0 

func (l *Library) WithoutText(ctx context.Context) ([]NoText, error)

WithoutText lists the papers whose PDF gave no searchable text, each with the reason, grouped by status: what fil stats --why prints.

type ListOptions added in v0.3.0 

type ListOptions = store.ListQuery

ListOptions filter and bound a listing of the library.

type Listed added in v0.3.0 

type Listed = store.ListRow

Listed is one paper in a listing, with its state beside it: a PDF on disk, what reading it gave, how many references carry their sentence.

type LocateOptions added in v0.3.0 

type LocateOptions struct {
	// Max is the most works to look up; zero means DefaultLocateMax.
	Max int
	// Progress, if set, is called after each batch of 100.
	Progress func(Located)
}

LocateOptions bound one lookup of PDF links.

type Located added in v0.3.0 

type Located = graph.Located

Located is what a lookup of PDF links found.

type Mention added in v0.3.0 

type Mention struct {
	Page       int
	Sentence   string
	Section    string // the heading above it, or ""
	Marker     string // as printed: "[12]", "Piwowar et al., 2018"
	Method     string // numeric | author-year
	Confidence float64
}

Mention is one place a paper cites a reference, found in its PDF (ADR-015): the sentence, its page and section, and how sure fil is.

type Moved added in v0.3.0 

type Moved struct {
	From, To string // relative to the library folder
	// ByHash means the folder's name no longer carries its fil ID, and it was
	// recognised by the hashes of the files in it.
	ByHash bool
}

Moved is a folder the scan found at a new path.

type Neighbourhood 

type Neighbourhood struct {
	Work model.Work

	// Cites is the work's references, fetched ones first.
	Cites []model.Work

	// CitedBy is the works in this library that cite it — never a global
	// count; see Work.CitedByCount for that.
	CitedBy []model.Work
}

Neighbourhood is a work and the works on either side of it.

type NoText added in v0.3.0 

type NoText struct {
	Work   model.Work
	Status string // no-text, gibberish, unreadable, timed-out
	Reason string
	File   string // absolute
}

NoText is a paper whose PDF gave no searchable text, and why.

type Options 

type Options struct {
	// CacheDir is where HTTP responses are cached. Empty means
	// config.DefaultCacheDir.
	CacheDir string

	// NoCache turns the cache off: --no-cache, which ADR-004 requires from day
	// one because a stale cache can hide a bug.
	NoCache bool

	// Transport replaces the network. Tests replay recorded responses through
	// it; front doors leave it nil.
	Transport http.RoundTripper

	// PDFTransport replaces the network for PDF downloads, which go to
	// publishers and repositories rather than to OpenAlex. Tests point it at
	// a local server; front doors leave it nil.
	PDFTransport http.RoundTripper
}

Options are the choices a front door makes that are not configuration. 

type PDFLinks struct {
	Work model.Work
	model.PDFLinks
	CheckedAt time.Time // zero when never looked up
}

PDFLinks is what is known about one work's open-access PDF.

type Paper added in v0.3.0 

type Paper struct {
	Work    model.Work
	Cites   int // works it cites
	CitedBy int // works in this library that cite it

	Folder string      // its folder in papers/, absolute; "" when it has none
	Files  []PaperFile // its files, present ones first

	// PDF is whether a legal PDF link is known, and the links; Asked is when
	// OpenAlex was asked (zero: never).
	PDF      model.PDFLinks
	Asked    time.Time
	Attempts []Attempt // downloads tried, newest first

	// Text is what reading its PDF gave; nil when it was never read.
	Text *PaperText

	// Contexts is how many of its references carry the sentence they are
	// cited in; Sample is the first few, in reading order.
	Contexts int
	Sample   []CitedIn
}

Paper is everything the library knows about one paper.

type PaperFile added in v0.3.0 

type PaperFile struct {
	Path     string // absolute
	Version  string
	Origin   string // download | import | found
	Missing  bool
	Mismatch bool // its first pages name another paper or edition (fetch's check)
}

PaperFile is one of a paper's files.

type PaperText added in v0.3.0 

type PaperText struct {
	Status string // text.Status
	Reason string
	Pages  int
	File   string // absolute
}

PaperText is what reading a paper's PDF gave.

type Path 

type Path struct {
	// Found reports whether a chain exists within the step limit. Not finding
	// one is an answer, not an error.
	Found bool

	// Works runs from the first work asked about to the second.
	Works []model.Work

	// Cites[i] reports that Works[i] cites Works[i+1]; false means it is the
	// other way round.
	Cites []bool
}

Path is a chain of citations between two works.

type PathOptions 

type PathOptions struct {
	// MaxHops bounds the search. Zero means DefaultMaxHops.
	MaxHops int

	// AnyDirection connects the two works through citations in either
	// direction. Without it, Path looks for lineage: one work reaching the
	// other by following references.
	AnyDirection bool
}

PathOptions control Path.

type Quota 

type Quota = httpx.Quota

Quota is the OpenAlex allowance as last reported: requests left today and how long until it resets.

type Relocated added in v0.3.0 

type Relocated struct {
	Folder   string // the library folder now
	Database string // the database now
	Cloud    string // the sync service the folder is in, if any

	// OldDatabase is the v0.2 database that was copied. It is left where it
	// was, for the person to delete once they are happy.
	OldDatabase string
}

Relocated is what Move did.

func Move added in v0.3.0 

func Move(ctx context.Context, cfg *config.Config, dir string) (Relocated, error)

Move gives the library cfg names a new folder.

A v0.2 library — a database with no folder — gets a library folder at dir: papers/, Inbox/ and exports/ are created, the database is copied into place and checked against the original, and the global settings are updated to open the new one. The old database is left where it was.

A library that already has a folder has it moved to dir, which must be on the same drive: moving gigabytes of PDFs is a job for the file manager, and a rename either happens or does not. If the database belongs in a different place at the new folder — in or out of a cloud sync service — it is copied there and checked, and the old copy removed.

Nothing else may have the library open while it moves.

type ScanReport added in v0.3.0 

type ScanReport struct {
	Folders  int // work folders on disk, found by fil ID or by their files
	Files    int // files of those works on disk
	NewFiles int // files hashed and recorded for the first time

	Moved   []Moved  // folders found somewhere new, moved or renamed by a person
	Missing []string // folders gone since the last scan, as they were recorded

	MissingFiles int      // files gone, in all
	Unknown      []string // folders named for a fil ID this library does not have
	Loose        []string // PDFs in papers/ not matched to a work: for import (ADR-014)

	Collections int      // collections that mirror folders, after the scan
	LongPaths   []string // paths close to Windows' 260-character limit
}

ScanReport is what a scan of papers/ found.

type SearchHit added in v0.3.0 

type SearchHit struct {
	Work    model.Work
	Page    int
	Section string // "references" for a reference list, else ""
	// Snippet is the matching stretch of the passage, with each matched word
	// between MarkStart and MarkEnd.
	Snippet string
	File    string // absolute path of the PDF
}

SearchHit is one passage that matched.

type SearchOptions added in v0.3.0 

type SearchOptions struct {
	Limit      int    // passages; zero means DefaultSearchLimit
	PerPaper   int    // most passages from one paper; zero means DefaultPerPaper
	Paper      string // only this paper, by any identifier
	References bool   // search reference lists too
}

SearchOptions shape one search.

type Searched added in v0.3.0 

type Searched struct {
	Hits []SearchHit
	// Partial reports that no passage had every word, so Hits are the
	// passages with some of them, those with the most ranked first.
	Partial bool
	// Unread counts the PDFs the library knows of but has not read, which a
	// search cannot see: fil extract reads them. Search does not scan
	// papers/, so a PDF put there since the last scan is not counted.
	Unread int
}

Searched is a search's answer.

type Stats 

type Stats struct {
	Works int // every work, stubs included
	Stubs int // works known only by ID so far
	Edges int // citations
}

Stats counts what the library holds.

type Summary 

type Summary struct {
	Works      int // every work, stubs included
	Fetched    int // metadata fetched
	Stubs      int // known only by ID so far
	Unresolved int // of the stubs, those OpenAlex has no record of
	Seeds      int // papers the user added
	Citations  int

	// DeadEnds is how many fetched works have no reference list, so the graph
	// cannot grow through them (D12).
	DeadEnds int
}

Summary is the library at a glance.

type WorkFolder added in v0.3.0 

type WorkFolder struct {
	Work    model.Work
	Path    string // absolute
	Created bool   // made by this call
	Renamed string // the folder's previous name, when this call renamed it
	Files   []File
	// LongPath warns that the folder's path is close to Windows' limit.
	LongPath bool
}

WorkFolder is a work's folder in papers/.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.