rouge

package module
v0.0.0-...-64efd25 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Jul 17, 2026 License: BSD-3-Clause Imports: 2 Imported by: 0

README

go-ruby-rouge/rouge

rouge — go-ruby-rouge

Docs License Go Coverage

A pure-Go (no cgo) reimplementation of the Ruby rouge syntax highlighter — the state-machine regex lexers, the token hierarchy, and the HTML formatters of the gem, emitting exactly the CSS short-codes (k, s, c, nf, …) rouge emits. It tokenizes source and renders <span class> HTML that matches the gem byte-for-byte on the supported lexers, without any Ruby runtime.

It is a sibling of go-ruby-regexp (the Onigmo engine it lexes with), go-ruby-erb, and go-ruby-yaml, and is bound into go-embedded-ruby to give rbgo a CGO-free Rouge.highlight.

API

import "github.com/go-ruby-rouge/rouge"

// Rouge.highlight(text, lexer, formatter) -> HTML
html, err := rouge.Highlight(`puts "hello"`, "ruby", "html")
// => <span class="nb">puts</span> <span class="s2">"hello"</span>

// Rouge::Lexer.find / find_fancy / guess
lex := rouge.FindLexer("rb")          // by tag or alias (case-insensitive)
lex = rouge.FindFancy("ruby?opt=1")   // "tag" or "tag?opts"
lex = rouge.Guess(src)                // content sniff; never nil (PlainText fallback)

// Token stream + the HTML / HTMLInline formatters directly
toks := lex.Lex(src)                          // []TokenValue{Token, Value}
out := rouge.HTMLFormatter{}.Format(toks)     // <span class="…"> per token
page := rouge.WrapHTML(out)                   // <pre class="highlight"><code>…</code></pre>
inline := rouge.HTMLInlineFormatter{Theme: rouge.Github}.Format(toks) // inline styles
  • Token model — rouge.Token mirrors Rouge::Token::Tokens: a tree with a dotted Qualname() (Literal.String.Double), the CSS Shortname the HTML formatter emits (s2), Matches(ancestor), and TokenByName("Keyword.Constant").
  • Formatters — HTML (<span class="shortcode">, the gem's exact codes; Text/Escape pass through) and HTMLInline (inline style= from a Theme), registered as "html" / "html_inline"; plus WrapHTML for the gem's <pre class="highlight"><code> page wrapper.
  • Themes — Base16, Github, ThankfulEyes: their resolved rendered_rules captured token-by-token, with StyleFor walking the ancestor chain exactly like Rouge::Theme.get_style.

Lexers

Each lexer is a near-mechanical transcription of the gem's state … rule … definition, validated by a differential oracle against the reference rouge 5.0.0 gem: every committed testdata/* corpus file is highlighted and compared byte-for-byte to the gem's HTML (the *.html goldens).

Byte-faithful (gem-identical HTML on the benign corpus):

Lexer Tag(s) Notes
Ruby ruby, rb symbols, %w/%r/%(…) sigils, heredocs (<</<<-/<<~), interpolation, ternary, method-call disambiguation
Go go, golang
Python python, py f-strings, case/match, decorators, doctest
JavaScript javascript, js regex/template/object/ternary state machine
JSON json
YAML yaml, yml indentation state machine, anchors, block scalars, tags
HTML html delegates <script>/<style> to JS/CSS
CSS css property/builtin/color/function sets
Shell shell, bash, zsh, ksh, sh heredocs, $()/${}/$(()), case
Diff diff, patch, udiff with content detector
Markdown markdown, md, mkd delegates fenced code to the named lexer, frontmatter to YAML
SQL sql case-insensitive keyword/type sets
PlainText plaintext, text fallback

Documented simplifications (honest deviations from the gem):

  • Markdown fenced code blocks: the closed-fence form is byte-faithful; the gem's anonymous dynamic state for an unclosed fence is approximated.
  • HTML <script>/<style> delegation is per-chunk (matching the gem); like the gem, a token straddling a bare < inside an embedded string can split (the gem itself emits an Error there).
  • Ruby %-sigil string bodies are scanned in one pass (nesting + interpolation + escapes preserved); the heredoc (?<!\p{Word}) guard is done against the preceding byte (the engine has no variable-width lookbehind).
  • Guess uses each lexer's content detector only (a subset of the gem's multi-signal guesser) — enough for the unambiguous formats (diff, YAML, shebangs, doctype).

Engine

Lexing runs on the sibling go-ruby-regexp Onigmo engine via its MatchAt(src, pos) primitive, which anchors \G at the scan cursor while keeping the whole buffer visible — so a rule's ^ matches only at a true line start and lookbehind sees the real prefix, exactly the StringScanner semantics a RegexLexer needs. Building this port upstreamed a handful of Onigmo-faithfulness fixes to that engine (\<punct> and \f \v \a \e escapes, the \p{Nl}/\p{No}/\p{Cf} property subcategories, and MatchAt itself).

Tests & coverage

Tests are deterministic and Ruby-free: the differential goldens were captured from the rouge gem once and committed, so go test reproduces gem-faithfulness without the gem on PATH (the gem is not stdlib, and the cross-arch / Windows CI lanes have no Ruby). Coverage is 100% including error branches.

go test ./...

CI builds and tests on 3 OSes (linux/macos/windows) and the six 64-bit Go targets (amd64/arm64/riscv64/loong64/ppc64le/s390x).

License

BSD-3-Clause — see LICENSE. Copyright the go-ruby-rouge/rouge authors. The bundled themes' colour tables derive from the BSD-3-licensed rouge gem.

WebAssembly

Being pure Go (CGO=0), this library also compiles to WebAssembly — both GOOS=js GOARCH=wasm (browser / Node.js) and GOOS=wasip1 GOARCH=wasm (WASI). CI builds both targets on every push, alongside the six 64-bit native/qemu arches.

GOOS=js     GOARCH=wasm go build ./...   # browser / Node
GOOS=wasip1 GOARCH=wasm go build ./...   # WASI (wasmtime, wasmer, wasmedge, …)

Documentation

Overview

Package rouge is a pure-Go (CGO=0), MRI-faithful port of the Ruby `rouge` syntax-highlighting gem (https://github.com/rouge-ruby/rouge). It tokenizes source code with state-machine regex lexers and formats the token stream as HTML, emitting exactly the CSS short-codes (k, s, c, nf, ...) the gem emits.

The token hierarchy, the regex-lexer engine, and the HTML formatters mirror the gem's data structures so the rendered HTML matches byte-for-byte on the supported lexers.

Index

Constants

This section is empty.

Variables

View Source 
var (
	// Text is the root token; it has the empty short-code.
	Text           = tok(nil, "Text", "")
	TextWhitespace = tok(Text, "Whitespace", "w")

	Escape = tok(nil, "Escape", "esc")
	Error  = tok(nil, "Error", "err")
	Other  = tok(nil, "Other", "x")

	Keyword            = tok(nil, "Keyword", "k")
	KeywordConstant    = tok(Keyword, "Constant", "kc")
	KeywordDeclaration = tok(Keyword, "Declaration", "kd")
	KeywordNamespace   = tok(Keyword, "Namespace", "kn")
	KeywordPseudo      = tok(Keyword, "Pseudo", "kp")
	KeywordReserved    = tok(Keyword, "Reserved", "kr")
	KeywordType        = tok(Keyword, "Type", "kt")
	KeywordVariable    = tok(Keyword, "Variable", "kv")

	Name                 = tok(nil, "Name", "n")
	NameAttribute        = tok(Name, "Attribute", "na")
	NameBuiltin          = tok(Name, "Builtin", "nb")
	NameBuiltinPseudo    = tok(NameBuiltin, "Pseudo", "bp")
	NameClass            = tok(Name, "Class", "nc")
	NameConstant         = tok(Name, "Constant", "no")
	NameDecorator        = tok(Name, "Decorator", "nd")
	NameEntity           = tok(Name, "Entity", "ni")
	NameException        = tok(Name, "Exception", "ne")
	NameFunction         = tok(Name, "Function", "nf")
	NameFunctionMagic    = tok(NameFunction, "Magic", "fm")
	NameProperty         = tok(Name, "Property", "py")
	NameLabel            = tok(Name, "Label", "nl")
	NameNamespace        = tok(Name, "Namespace", "nn")
	NameOther            = tok(Name, "Other", "nx")
	NameTag              = tok(Name, "Tag", "nt")
	NameVariable         = tok(Name, "Variable", "nv")
	NameVariableClass    = tok(NameVariable, "Class", "vc")
	NameVariableGlobal   = tok(NameVariable, "Global", "vg")
	NameVariableInstance = tok(NameVariable, "Instance", "vi")
	NameVariableMagic    = tok(NameVariable, "Magic", "vm")

	Literal     = tok(nil, "Literal", "l")
	LiteralDate = tok(Literal, "Date", "ld")

	LiteralString          = tok(Literal, "String", "s")
	LiteralStringAffix     = tok(LiteralString, "Affix", "sa")
	LiteralStringBacktick  = tok(LiteralString, "Backtick", "sb")
	LiteralStringChar      = tok(LiteralString, "Char", "sc")
	LiteralStringDelimiter = tok(LiteralString, "Delimiter", "dl")
	LiteralStringDoc       = tok(LiteralString, "Doc", "sd")
	LiteralStringDouble    = tok(LiteralString, "Double", "s2")
	LiteralStringEscape    = tok(LiteralString, "Escape", "se")
	LiteralStringHeredoc   = tok(LiteralString, "Heredoc", "sh")
	LiteralStringInterpol  = tok(LiteralString, "Interpol", "si")
	LiteralStringOther     = tok(LiteralString, "Other", "sx")
	LiteralStringRegex     = tok(LiteralString, "Regex", "sr")
	LiteralStringSingle    = tok(LiteralString, "Single", "s1")
	LiteralStringSymbol    = tok(LiteralString, "Symbol", "ss")

	LiteralNumber            = tok(Literal, "Number", "m")
	LiteralNumberBin         = tok(LiteralNumber, "Bin", "mb")
	LiteralNumberFloat       = tok(LiteralNumber, "Float", "mf")
	LiteralNumberHex         = tok(LiteralNumber, "Hex", "mh")
	LiteralNumberInteger     = tok(LiteralNumber, "Integer", "mi")
	LiteralNumberIntegerLong = tok(LiteralNumberInteger, "Long", "il")
	LiteralNumberOct         = tok(LiteralNumber, "Oct", "mo")
	LiteralNumberOther       = tok(LiteralNumber, "Other", "mx")

	Operator     = tok(nil, "Operator", "o")
	OperatorWord = tok(Operator, "Word", "ow")

	Punctuation          = tok(nil, "Punctuation", "p")
	PunctuationIndicator = tok(Punctuation, "Indicator", "pi")

	Comment            = tok(nil, "Comment", "c")
	CommentHashbang    = tok(Comment, "Hashbang", "ch")
	CommentDoc         = tok(Comment, "Doc", "cd")
	CommentMultiline   = tok(Comment, "Multiline", "cm")
	CommentPreproc     = tok(Comment, "Preproc", "cp")
	CommentPreprocFile = tok(Comment, "PreprocFile", "cpf")
	CommentSingle      = tok(Comment, "Single", "c1")
	CommentSpecial     = tok(Comment, "Special", "cs")

	Generic           = tok(nil, "Generic", "g")
	GenericDeleted    = tok(Generic, "Deleted", "gd")
	GenericEmph       = tok(Generic, "Emph", "ge")
	GenericEmphStrong = tok(Generic, "EmphStrong", "ges")
	GenericError      = tok(Generic, "Error", "gr")
	GenericHeading    = tok(Generic, "Heading", "gh")
	GenericInserted   = tok(Generic, "Inserted", "gi")
	GenericLineno     = tok(Generic, "Lineno", "gl")
	GenericOutput     = tok(Generic, "Output", "go")
	GenericPrompt     = tok(Generic, "Prompt", "gp")
	GenericStrong     = tok(Generic, "Strong", "gs")
	GenericSubheading = tok(Generic, "Subheading", "gu")
	GenericTraceback  = tok(Generic, "Traceback", "gt")
)

The token hierarchy. These mirror Rouge::Token::Tokens one-for-one, including the CSS short-codes, which are kept in sync with pygments STANDARD_TYPES.

View Source 
var Base16 = &Theme{name: "base16", rules: map[string]string{
	"Text":                    "color: #383838",
	"Error":                   "color: #181818;background-color: #ab4642",
	"Comment":                 "color: #585858",
	"Comment.Preproc":         "color: #f7ca88",
	"Name.Tag":                "color: #f7ca88",
	"Operator":                "color: #d8d8d8",
	"Punctuation":             "color: #d8d8d8",
	"Generic.Inserted":        "color: #a1b56c",
	"Generic.Deleted":         "color: #ab4642",
	"Generic.Heading":         "color: #7cafc2;background-color: #181818;font-weight: bold",
	"Generic.Emph":            "font-style: italic",
	"Generic.EmphStrong":      "font-weight: bold;font-style: italic",
	"Generic.Strong":          "font-weight: bold",
	"Keyword":                 "color: #ba8baf",
	"Keyword.Constant":        "color: #dc9656",
	"Keyword.Type":            "color: #dc9656",
	"Keyword.Declaration":     "color: #dc9656",
	"Literal.String":          "color: #a1b56c",
	"Literal.String.Affix":    "color: #ba8baf",
	"Literal.String.Regex":    "color: #86c1b9",
	"Literal.String.Interpol": "color: #a16946",
	"Literal.String.Escape":   "color: #a16946",
	"Name.Namespace":          "color: #f7ca88",
	"Name.Class":              "color: #f7ca88",
	"Name.Constant":           "color: #f7ca88",
	"Name.Attribute":          "color: #7cafc2",
	"Literal.Number":          "color: #a1b56c",
	"Literal.String.Symbol":   "color: #a1b56c",
}}

Base16 is the rouge "base16" theme.

View Source 
var Github = &Theme{name: "github", rules: map[string]string{
	"Text":                    "color: #24292f;background-color: #f6f8fa",
	"Keyword":                 "color: #cf222e",
	"Generic.Error":           "color: #f6f8fa",
	"Generic.Deleted":         "color: #82071e;background-color: #ffebe9",
	"Name.Builtin":            "color: #953800",
	"Name.Class":              "color: #953800",
	"Name.Constant":           "color: #953800",
	"Name.Namespace":          "color: #953800",
	"Literal.String.Regex":    "color: #116329",
	"Name.Attribute":          "color: #116329",
	"Name.Tag":                "color: #116329",
	"Generic.Inserted":        "color: #116329;background-color: #dafbe1",
	"Generic.EmphStrong":      "font-weight: bold;font-style: italic",
	"Keyword.Constant":        "color: #0550ae",
	"Literal":                 "color: #0550ae",
	"Literal.String.Backtick": "color: #0550ae",
	"Name.Builtin.Pseudo":     "color: #0550ae",
	"Name.Exception":          "color: #0550ae",
	"Name.Label":              "color: #0550ae",
	"Name.Property":           "color: #0550ae",
	"Name.Variable":           "color: #0550ae",
	"Operator":                "color: #0550ae",
	"Generic.Heading":         "color: #0550ae;font-weight: bold",
	"Generic.Subheading":      "color: #0550ae;font-weight: bold",
	"Literal.String":          "color: #0a3069",
	"Name.Decorator":          "color: #8250df",
	"Name.Function":           "color: #8250df",
	"Error":                   "color: #f6f8fa;background-color: #82071e",
	"Comment":                 "color: #6e7781",
	"Generic.Lineno":          "color: #6e7781",
	"Generic.Traceback":       "color: #6e7781",
	"Name.Entity":             "color: #24292f",
	"Literal.String.Interpol": "color: #24292f",
	"Generic.Emph":            "color: #24292f;font-style: italic",
	"Generic.Strong":          "color: #24292f;font-weight: bold",
}}

Github is the rouge "github" theme.

View Source 
var ThankfulEyes = &Theme{name: "thankful_eyes", rules: map[string]string{
	"Text":                    "color: #faf6e4;background-color: #122b3b",
	"Generic.Lineno":          "color: #dee5e7;background-color: #4e5d62",
	"Generic.Prompt":          "color: #a8e1fe;font-weight: bold",
	"Comment":                 "color: #6c8b9f;font-style: italic",
	"Comment.Preproc":         "color: #b2fd6d;font-weight: bold",
	"Error":                   "color: #fefeec;background-color: #cc0000",
	"Generic.Error":           "color: #cc0000;font-weight: bold;font-style: italic",
	"Keyword":                 "color: #f6dd62;font-weight: bold",
	"Operator":                "color: #4df4ff;font-weight: bold",
	"Punctuation":             "color: #4df4ff",
	"Generic.Deleted":         "color: #cc0000",
	"Generic.Inserted":        "color: #b2fd6d",
	"Generic.Emph":            "font-style: italic",
	"Generic.EmphStrong":      "font-weight: bold;font-style: italic",
	"Generic.Strong":          "font-weight: bold",
	"Generic.Traceback":       "color: #dee5e7;background-color: #4e5d62",
	"Keyword.Constant":        "color: #f696db;font-weight: bold",
	"Keyword.Namespace":       "color: #ffb000;font-weight: bold",
	"Keyword.Pseudo":          "color: #ffb000;font-weight: bold",
	"Keyword.Reserved":        "color: #ffb000;font-weight: bold",
	"Generic.Heading":         "color: #ffb000;font-weight: bold",
	"Generic.Subheading":      "color: #ffb000;font-weight: bold",
	"Keyword.Type":            "color: #b2fd6d;font-weight: bold",
	"Name.Constant":           "color: #b2fd6d;font-weight: bold",
	"Name.Class":              "color: #b2fd6d;font-weight: bold",
	"Name.Decorator":          "color: #b2fd6d;font-weight: bold",
	"Name.Namespace":          "color: #b2fd6d;font-weight: bold",
	"Name.Builtin.Pseudo":     "color: #b2fd6d;font-weight: bold",
	"Name.Exception":          "color: #b2fd6d;font-weight: bold",
	"Name.Label":              "color: #ffb000;font-weight: bold",
	"Name.Tag":                "color: #ffb000;font-weight: bold",
	"Literal.Number":          "color: #f696db;font-weight: bold",
	"Literal.Date":            "color: #f696db;font-weight: bold",
	"Literal.String.Symbol":   "color: #f696db;font-weight: bold",
	"Literal.String":          "color: #fff0a6;font-weight: bold",
	"Literal.String.Affix":    "color: #f6dd62;font-weight: bold",
	"Literal.String.Escape":   "color: #4df4ff;font-weight: bold",
	"Literal.String.Char":     "color: #4df4ff;font-weight: bold",
	"Literal.String.Interpol": "color: #4df4ff;font-weight: bold",
	"Name.Builtin":            "font-weight: bold",
	"Name.Entity":             "color: #999999;font-weight: bold",
	"Text.Whitespace":         "color: #BBBBBB",
	"Generic.Output":          "color: #BBBBBB",
	"Name.Function":           "color: #a8e1fe",
	"Name.Property":           "color: #a8e1fe",
	"Name.Attribute":          "color: #a8e1fe",
	"Name.Variable":           "color: #a8e1fe;font-weight: bold",
}}

ThankfulEyes is the rouge "thankful_eyes" theme.

Functions

func Highlight 

func Highlight(text, lexerName, formatterName string) (string, error)

Highlight tokenizes text with the named lexer and renders it with the named formatter, returning the formatted string. It mirrors Rouge.highlight. An unknown lexer or formatter name returns an error.

func WrapHTML 

func WrapHTML(inner string) string

WrapHTML wraps formatted HTML in the gem's themed-page container, <pre class="highlight"><code>...</code></pre>, matching the markup Rouge::Formatters::HTMLPygments (and the rougify CLI) produce around the inner spans. The inner string must already be HTML from HTMLFormatter.

Types

type Formatter 

type Formatter interface {
	// Format renders the whole token stream to a string.
	Format(tokens []TokenValue) string
	// Tag returns the formatter's registry tag (e.g. "html").
	Tag() string
}

Formatter renders a token stream into a string (HTML, inline-styled HTML, ...), mirroring Rouge::Formatter.

func FindFormatter 

func FindFormatter(tag string) Formatter

FindFormatter returns the formatter registered under tag, or nil. Mirrors Rouge::Formatter.find.

type HTMLFormatter 

type HTMLFormatter struct{}

HTMLFormatter renders the token stream as <span class="shortcode"> elements, mirroring Rouge::Formatters::HTML. The bare Text token and the Escape token pass their value through without a wrapping span. Highlight does not add a <pre>/<code> wrapper; use WrapHTML for the gem's themed-page wrapper.

func (HTMLFormatter) Format 

func (HTMLFormatter) Format(tokens []TokenValue) string

Format renders the stream. Each token becomes its escaped value wrapped in a span carrying the token's CSS short-code, except Text (passed through) and Escape (passed through unescaped, as in Rouge's escape?-aware path).

func (HTMLFormatter) Tag 

func (HTMLFormatter) Tag() string

Tag returns "html".

type HTMLInlineFormatter 

type HTMLInlineFormatter struct {
	// Theme supplies per-token CSS declarations.
	Theme *Theme
}

HTMLInlineFormatter renders each token with an inline style attribute computed from a Theme, mirroring Rouge::Formatters::HTMLInline. Text passes through without a span.

func (HTMLInlineFormatter) Format 

func (f HTMLInlineFormatter) Format(tokens []TokenValue) string

Format renders the stream with inline styles from the formatter's Theme.

func (HTMLInlineFormatter) Tag 

func (HTMLInlineFormatter) Tag() string

Tag returns "html_inline".

type Lexer 

type Lexer interface {
	// Lex returns the token stream for text. The stream never contains empty
	// values.
	Lex(text string) []TokenValue
	// Tag returns the lexer's primary tag.
	Tag() string
	// Title returns the human-readable title.
	Title() string
	// Aliases returns alternate names.
	Aliases() []string
}

Lexer tokenizes source text into a stream of (token, value) pairs. Every lexer in this package is a *RegexLexer, but the interface lets callers and formatters work against the abstraction, mirroring Rouge::Lexer.

func FindFancy 

func FindFancy(spec string) Lexer

FindFancy resolves a "fancy" lexer spec of the form "tag" or "tag?option=...", mirroring Rouge::Lexer.find_fancy. Options are accepted and ignored (this port has no per-lex options that change tokenization). An empty or "guess" name returns nil so callers can fall back to Guess. An unknown tag returns nil.

func FindLexer 

func FindLexer(name string) Lexer

FindLexer returns the lexer registered under the given tag or alias, or nil if none matches. The lookup is case-insensitive on the tag, mirroring Rouge::Lexer.find.

func Guess 

func Guess(text string) Lexer

Guess picks a lexer for text using each lexer's content sniffer (Rouge's self.detect?). The first lexer (in registration order) that claims the text wins; if none does, PlainText is returned, so Guess never returns nil. This is a deliberately small subset of Rouge's multi-signal guesser: it uses only the content detectors, which is enough for the formats whose detectors are unambiguous (e.g. diff). See README "Simplified" notes.

type RegexLexer 

type RegexLexer struct {
	// contains filtered or unexported fields
}

RegexLexer is a stateful, regex-rule lexer. A concrete lexer registers its states with newRegexLexer / state-builder helpers and exposes itself through the Lexer interface. It is immutable after construction and safe for concurrent use; per-lex mutable state lives in lexState.

func (*RegexLexer) Aliases 

func (rl *RegexLexer) Aliases() []string

Aliases returns the lexer's alternate names (e.g. "rb" for Ruby).

func (*RegexLexer) Lex 

func (rl *RegexLexer) Lex(text string) []TokenValue

Lex tokenizes text and returns the token stream, mirroring Rouge::Lexer#lex for a RegexLexer. The stream never contains empty values.

func (*RegexLexer) Tag 

func (rl *RegexLexer) Tag() string

Tag returns the lexer's primary tag (e.g. "ruby").

func (*RegexLexer) Title 

func (rl *RegexLexer) Title() string

Title returns the lexer's human-readable title.

type Theme 

type Theme struct {
	// contains filtered or unexported fields
}

Theme maps tokens to inline CSS declarations for the HTMLInline formatter, mirroring the resolved output of Rouge::Theme#style_for(tok).rendered_rules. Lookup walks the token's ancestor chain and returns the rules of the most specific ancestor that has a style, falling back to the base (Text) style, exactly like Rouge::Theme.get_style.

func FindTheme 

func FindTheme(name string) *Theme

FindTheme returns the theme registered under name, or nil. Mirrors Rouge::Theme.find.

func (*Theme) Name 

func (t *Theme) Name() string

Name returns the theme's name.

func (*Theme) StyleFor 

func (t *Theme) StyleFor(tok *Token) string

StyleFor returns the ";"-joined inline CSS declarations for tok, resolving up the ancestor chain the way Rouge::Theme.get_style does: the closest ancestor (including tok itself) with a defined style wins; if none is defined the base Text style is used; if even Text is undefined the result is empty.

type Token 

type Token struct {
	// Name is the simple token name, e.g. "Function".
	Name string
	// Shortname is the CSS class the HTML formatter emits, e.g. "nf". The root
	// Text token has the empty short-code.
	Shortname string
	// Parent is the enclosing token, nil for the root Text token.
	Parent *Token
	// contains filtered or unexported fields
}

Token is a syntactic token type, e.g. Keyword or Literal.String. Tokens form a tree mirroring Rouge::Token::Tokens: every token has a Name, a one- or two-letter CSS Shortname the HTML formatter emits, and a Parent. Tokens are compared by identity (pointer), so the package-level token variables below are the canonical instances.

func TokenByName 

func TokenByName(qualname string) *Token

TokenByName returns the canonical token for a dotted qualified name (e.g. "Keyword.Constant"), or nil if no such token exists. It mirrors Rouge::Token::Tokens[qualname].

func (*Token) Matches 

func (t *Token) Matches(other *Token) bool

Matches reports whether t is other or a descendant of other, mirroring Rouge::Token.matches? (other appears in t's ancestor chain).

func (*Token) Qualname 

func (t *Token) Qualname() string

Qualname is the dotted fully-qualified name of the token, e.g. "Literal.String.Double", matching Rouge::Token#qualname.

func (*Token) String 

func (t *Token) String() string

String returns the token's qualified name.

type TokenValue 

type TokenValue struct {
	Token *Token
	Value string
}

TokenValue is a (token, value) pair in the lexed stream.

type UnknownError 

type UnknownError struct {
	// Kind is "lexer" or "formatter".
	Kind string
	// Name is the unrecognized name.
	Name string
}

UnknownError reports an unknown lexer or formatter name from Highlight.

func (*UnknownError) Error 

func (e *UnknownError) Error() string

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.