parser

package

v0.3.8 Latest Latest Go to latest Published: May 22, 2026 License: MIT Imports: 20 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/radimsem/remindb

Links

Open Source Insights

Documentation ¶

Overview ¶

Package parser turns source files (Markdown, HTML, YAML, JSON, TOON) into a unified ContextNode tree.

Index ¶

Constants
Variables
func FlattenMap(roots []*ContextNode) map[string]*ContextNode
type ContextNode
type HtmlParser
type JsonParser
type NodeType
type ToonParser
type WikilinkRef
- func ExtractWikilinks(text string) (string, []WikilinkRef)
type YamlParser

Constants ¶

View Source

const (
	FormatPlain  = "plain"
	FormatToon   = "toon"
	FormatMathml = "mathml"
	FormatLatex  = "latex"
)

View Source

const MathmlSavingsThreshold = 0.15

Minimum byte-savings fraction LaTeX must beat over the raw MathML XML before it's preferred.

View Source

const MaxInlineFields = 5

Promote a nested map or array to its own ContextNode at or above this many entries; smaller structures stay inlined.

View Source

const ToonSavingsThreshold = 0.15

Minimum byte-savings fraction TOON must beat over plain rendering before it's preferred.

Variables ¶

View Source

var (
	ErrUnsupportedExt = errors.New("unsupported extension")
	ErrInvalidUTF8    = errors.New("invalid UTF-8")
	ErrMalformed      = errors.New("malformed content")
)

Functions ¶

func FlattenMap ¶

func FlattenMap(roots []*ContextNode) map[string]*ContextNode

Types ¶

type ContextNode ¶

type ContextNode struct {
	Children     []*ContextNode
	WikilinkRefs []WikilinkRef
	ID           string
	ParentID     string
	SourceFile   string
	Label        string
	Content      string
	ContentHash  string
	Format       string
	Temperature  *float64
	Depth        int
	TokenCount   int
	SeedPinned   bool
	NodeType
}

func Flatten ¶

func Flatten(roots []*ContextNode) []*ContextNode

func Parse ¶

func Parse(path string, r io.Reader) ([]*ContextNode, error)

Read r and route it to a format-specific parser based on path's extension.

func ParseBytes ¶

func ParseBytes(path string, data []byte) ([]*ContextNode, error)

func ParseFile ¶

func ParseFile(path string) ([]*ContextNode, error)

type HtmlParser ¶ added in v0.2.0

type HtmlParser struct{}

type JsonParser ¶

type JsonParser struct{}

type NodeType ¶

type NodeType string

const (
	NodeHeading  NodeType = "heading"
	NodeList     NodeType = "list"
	NodeTable    NodeType = "table"
	NodeCode     NodeType = "code"
	NodeText     NodeType = "text"
	NodeKV       NodeType = "kv"
	NodePreamble NodeType = "preamble"
	NodeEmbed    NodeType = "embed"
)

type ToonParser ¶

type ToonParser struct{}

type WikilinkRef ¶ added in v0.3.0

type WikilinkRef struct {
	Label      string
	SourceQual string
	IDHint     string
	Weight     float64
}

func ExtractWikilinks ¶ added in v0.3.0

func ExtractWikilinks(text string) (string, []WikilinkRef)

type YamlParser ¶

type YamlParser struct{}

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL