Documentation
¶
Overview ¶
Package llmstxt renders /llms.txt and /llms-full.txt (https://llmstxt.org): a Markdown index of a site for language models and, in the full file, the Markdown of the linked pages one after the other. Nothing is served unless the app registers it:
err := llmstxt.Register(r, llmstxt.Doc{
Title: "Shop",
Summary: "A shop that sells games.",
Sections: []llmstxt.Section{{Title: "Docs", Links: []llmstxt.Link{
{Title: "Quick start", URL: "https://shop.example/docs/start", Content: docs.Markdown("start")},
}}},
})
Titles, notes and the summary are single-line text: line breaks become spaces and link text is escaped, so they cannot add headings or links. URLs must be absolute http(s) or root-relative. Details and page content are the app's own Markdown and are written as given.
Index ¶
Constants ¶
const ( // DefaultMaxFullBytes caps llms-full.txt when Doc.MaxFullBytes is 0. DefaultMaxFullBytes = 16 << 20 // TruncatedNote ends an llms-full.txt cut at the size cap. TruncatedNote = "\n\n[llms-full.txt truncated: size limit reached]\n" )
Variables ¶
This section is empty.
Functions ¶
func FullHandler ¶
FullHandler streams llms-full.txt, calling every Content per request. A Content error before the first byte reached the client is a bare 500; after that the connection is aborted, so a cut file never looks whole.
func Handler ¶
Handler serves llms.txt, rendered once, with an ETag, Cache-Control and Last-Modified (when set), answering conditional requests with 304.
func WriteFull ¶
WriteFull writes llms-full.txt: the header of llms.txt, then the Content of every link that has one (sections first, then Optional), each after a "---" rule, its title as H1 and a Source line. Output past MaxFullBytes is cut and ends with TruncatedNote. A Content error stops the writing and is returned.
Types ¶
type Doc ¶
type Doc struct {
// Title is the H1, usually the project or site name. Required.
Title string
// Summary is the blockquote under the title: what the site is.
Summary string
// Details is optional Markdown written after the summary (paragraphs,
// lists; no headings). It is trusted app text, written as given.
Details string
// Sections are the H2 sections, each a list of links.
Sections []Section
// Optional links go under "## Optional": secondary pages a model may
// skip when its context is short.
Optional []Link
// LastModified, when set, is sent as Last-Modified by both handlers.
LastModified time.Time
// MaxFullBytes caps llms-full.txt; past it the file ends with
// TruncatedNote. 0 means DefaultMaxFullBytes.
MaxFullBytes int64
}
Doc describes the site.
type Link ¶
type Link struct {
Title string
// URL is absolute http(s) or root-relative ("/docs/api").
URL string
Notes string
// Content returns the page as Markdown for llms-full.txt; nil leaves
// the page out of it. It runs on every llms-full.txt request.
Content func(ctx context.Context) (string, error)
}
Link is one "- [Title](URL): Notes" entry.
Source Files