Documentation
¶
Overview ¶
Package lang provides tokens for possibly multiple languages.
Index ¶
Constants ¶
View Source
const ( CharIllegal = 1 << iota CharOp CharNum CharAlpha CharSep CharLineSep CharGroupSep CharStr CharBlock StrEsc StrNonl ExcludeEnd // exclude end delimiter from content EosValidEnd // end of input string terminates block or string token )
Lexical properties of tokens to allow scanning.
View Source
const ASCIILen = 1 << 7 // 128
ASCIILen is the length of the ASCII characters set.
Variables ¶
View Source
var UnaryOp = map[Token]Token{ Add: Plus, And: Addr, Not: Not, Mul: Deref, Sub: Minus, Tilde: Tilde, Xor: BitComp, }
UnaryOp contains the set of unary operators. TODO: define UnaryOp per language.
Functions ¶
This section is empty.
Types ¶
type Associativity ¶
type Associativity int
Associativity represent the associativity rule of an operator.
const ( Aboth Associativity = iota // both left and right associative Aleft // left associative only Aright // right associative only Anon // non associative )
Associativity kinds for operators.
type Spec ¶
type Spec struct {
CharProp [ASCIILen]uint // special Character properties
End map[string]string // end delimiters, indexed by start
BlockProp map[string]uint // block properties
Tokens map[string]Token // token per string
TokenProps []TokenProp // token properties, indexed by token
DotNum bool // true if a number can start with '.'
IdentASCII bool // true if an identifier can be in ASCII only
NumUnder bool // true if a number can contain _ character
}
Spec represents the language specification for scanning.
type Token ¶
type Token int
Token represents a lexical token.
const ( Illegal Token = iota Comment Ident // Literal values. Char Float Imag Int String // Binary operators (except indicated). // Arithmetic and bitwise binary operators. Add // + Sub // - Mul // * Quo // / Rem // % And // & Or // | Xor // ^ Shl // << Shr // >> AndNot // &^ Period // . // Binary operators returning a boolean. Equal // == Greater // > GreaterEqual // >= Land // && Less // < LessEqual // <= Lor // || NotEqual // != // Assigment operators (arithmetic and bitwise). Define // := Assign // = AddAssign // += SubAssign // -= MulAssign // *= QuoAssign // /= RemAssign // %= AndAssign // &= OrAssign // |= XorAssign // ^= ShlAssign // <<= ShrAssign // >>= AndNotAssign // &^= Inc // ++ Dec // -- IndexAssign // a[i] = DerefAssign // *p = // Unary operations. Plus // unary + Minus // unary - Addr // unary & Deref // unary * BitComp // unary ^ Arrow // unary -> Ellipsis // unary ... Not // unary ! Tilde // unary ~ (underlying type) // Separators (punctuation). Comma // , Semicolon // ; Colon // : // Block tokens. ParenBlock // (..) BracketBlock // [..] BraceBlock // {..} // Reserved keywords. Break Case Chan Const Continue Default Defer Else Fallthrough For Func Go Goto If Import Interface Map Package Range Return Select Struct Switch Type Var // Internal virtual machine tokens (no corresponding keyword). Call ChanSend // ch v -- ; channel send statement Composite EqualSet Grow Index JumpFalse JumpSetFalse JumpSetTrue Label Len New Next Slice Stop TypeAssert TypeSwitchJump PopExpr // discard unused expression-statement return values Drop // pop one value from the stack (used to clean up switch/select dispatch) // This must be the last token value. MaxTok )
All known tokens for the set of supported languages.
func (Token) IsBinaryOp ¶
IsBinaryOp returns true if t is a binary operator (takes 2 operands).
func (Token) IsLogicalOp ¶
IsLogicalOp returns true if t is a logical operator.
func (Token) IsOperator ¶
IsOperator returns true if t is an operator.
Source Files
Directories
Click to show internal directories.
Click to hide internal directories.