token

package

v0.0.0-...-37d2667 Latest Latest Go to latest Published: Dec 13, 2014 License: BSD-3-Clause Imports: 3 Imported by: 0

Details

Valid go.mod file
Redistributable license
Tagged version
Stable version
Learn more about best practices

Repository

github.com/guweigang/php-parser-in-golang

Links

Open Source Insights

Documentation ¶

Index ¶

Variables
type Item
- func Keyword(t Token) Item
- func NewItem(t Token, v string) Item
- func (i Item) Position() Position
- func (i Item) String() string
type ItemList
- func NewList(t ...Item) *ItemList
type Position
type Stream
- func Subset(s Stream, t Type) Stream
type Token
type Type

Constants ¶

This section is empty.

Variables ¶

View Source

var TokenList []string

View Source

var TokenMap = map[string]Token{}/* 133 elements not displayed */

TokenMap maps source code string tokens to types when strings can be represented directly. Not all types will be represented here.

Functions ¶

This section is empty.

Types ¶

type Item ¶

type Item struct {
	Typ        Token
	Begin, End Position
	Val        string
}

Item represents a lexed item.

func Keyword ¶

func Keyword(t Token) Item

func NewItem ¶

func NewItem(t Token, v string) Item

func (Item) Position ¶

func (i Item) Position() Position

func (Item) String ¶

func (i Item) String() string

String renders a string representation of the item.

type ItemList ¶

type ItemList struct {
	// Items contains all the items in the list.
	Items []Item

	// Position is the current position the set is at in the token slice.
	Position int
}

List represents an ordered set of tokens.

func NewList ¶

func NewList(t ...Item) *ItemList

func (*ItemList) Next ¶

func (s *ItemList) Next() (item Item)

func (*ItemList) Peek ¶

func (s *ItemList) Peek() Item

func (*ItemList) Push ¶

func (s *ItemList) Push(i ...Item)

func (*ItemList) PushKeyword ¶

func (s *ItemList) PushKeyword(t Token)

func (*ItemList) PushStream ¶

func (s *ItemList) PushStream(i Stream)

func (*ItemList) Seek ¶

func (s *ItemList) Seek(position int)

type Position ¶

type Position struct {
	Line, Column int // The position relative to other characters in the file
	Position     int // The position in bytes in the file
	File         string
}

type Stream ¶

type Stream interface {
	Next() Item
}

Stream is an ordered set of tokens

func Subset ¶

func Subset(s Stream, t Type) Stream

Subset returns a stream that emits only tokens from s that are of Type t..

type Token ¶

type Token int

const (
	EOF Token = iota
	HTML
	PHP
	PHPBegin
	PHPEnd
	PHPToken
	Error
	Space
	Function
	Static
	Self
	Parent
	Final
	FunctionName
	TypeHint
	VariableOperator
	BlockBegin
	BlockEnd
	Global

	Namespace
	Use

	Comment

	IgnoreErrorOperator

	Return
	ArgumentType
	ArgumentName
	Comma
	StatementEnd
	Echo
	Print

	If
	Else
	ElseIf
	For
	Foreach
	EndIf
	EndFor
	EndForeach
	EndWhile
	EndSwitch
	AsOperator
	While
	Continue
	Break
	Do
	OpenParen
	CloseParen
	Switch
	Case
	Default

	Try
	Catch
	Finally
	Throw

	Class
	Abstract
	Private
	Public
	Protected
	Interface
	Implements
	Extends
	NewOperator
	Const

	Null
	StringLiteral
	NumberLiteral
	BooleanLiteral

	ShellCommand

	Identifier

	AssignmentOperator
	NegationOperator
	AdditionOperator
	SubtractionOperator
	MultOperator
	ConcatenationOperator
	UnaryOperator
	ComparisonOperator
	InstanceofOperator

	AndOperator
	OrOperator
	WrittenAndOperator
	WrittenXorOperator
	WrittenOrOperator

	ObjectOperator
	ScopeResolutionOperator

	CastOperator

	Var
	Array
	ArrayKeyOperator
	ArrayLookupOperatorLeft
	ArrayLookupOperatorRight
	List
	BitwiseShiftOperator
	StrongEqualityOperator
	StrongNotEqualityOperator
	EqualityOperator
	NotEqualityOperator
	AmpersandOperator
	BitwiseXorOperator
	BitwiseOrOperator
	BitwiseNotOperator
	TernaryOperator1
	TernaryOperator2

	Declare

	Include
	Exit
)

func (Token) IsType ¶

func (t Token) IsType(ty Type) bool

func (Token) String ¶

func (i Token) String() string

func (Token) Type ¶

func (t Token) Type() Type

type Type ¶

type Type int

const (
	InvalidType Type = 1 << iota

	KeywordType    // keyword, e.g. "static", "function"
	LiteralType    // literal, e.g. 234, "a string", false
	MarkerType     // marker for code blocks and groupings, e.g. {, (
	OperatorType   // operator, e.g. +, ===, $
	IdentifierType // identifier, e.g. StdClass
	Significant    = KeywordType | LiteralType | MarkerType | IdentifierType

	CommentType
	WhitespaceType
)

Source Files ¶

View all Source files

?	: This menu
/	: Search site
f or F	: Jump to
y or Y	: Canonical URL