protoparse

package
v1.0.0 Latest Latest
Warning

This package is not in the latest version of its module.

Go to latest
Published: Jun 30, 2026 License: MIT Imports: 6 Imported by: 0

Documentation

Overview

Package protoparse implements a recursive descent parser for Protocol Buffer definition files (.proto). It consumes a token stream from the protolex package and produces a typed abstract syntax tree (AST) representing the complete structure of a .proto file, including all declarations, nested constructs, and attached comments.

The two entry points are Parse and ParseTokens. Parse accepts a filename and raw byte input, creates a lexer internally, and returns the parsed AST. ParseTokens accepts a pre-constructed protolex.Lexer, allowing callers to control lexer construction or reuse. Both return a *File (the AST root) and an error. On success the error is nil; on failure the error is an ErrorList containing one or more parse errors, and the returned *File is a partial AST representing everything that was successfully parsed before and between errors.

file, err := protoparse.Parse("example.proto", src)
if err != nil {
    var errs protoparse.ErrorList
    if errors.As(err, &errs) {
        for _, e := range errs {
            fmt.Println(e)
        }
    }
}

The File type is the root of the AST. It contains optional SyntaxDecl or EditionDecl (mutually exclusive), an optional PackageDecl, a slice of ImportDecl entries, top-level OptionDecl entries, and a Decls slice holding all top-level declarations in source order. The Decls slice contains values implementing the sealed Decl interface: *MessageDecl, *EnumDecl, *ServiceDecl, *ExtendDecl, and *OptionDecl.

Message bodies contain elements implementing the sealed MessageElement interface: *FieldDecl, *MapFieldDecl, *OneofDecl, *GroupDecl, *EnumDecl, *MessageDecl, *ExtendDecl, *ExtensionsDecl, *ReservedDecl, and *OptionDecl. All elements within a message body are stored in source order in the Body slice.

The parser supports all three protobuf dialects: proto2 (with required fields, groups, extensions, and extend blocks), proto3 (with label-less fields, maps, oneofs, and services), and editions (edition declaration in place of syntax). It handles string literal concatenation, aggregate (message) literal values in options, fully qualified type names with leading dots, and inline field options in bracket notation.

Comments are preserved in the AST. Leading comments (those immediately preceding a declaration with no intervening blank line) are attached to the declaration's Comments slice. Trailing comments (those on the same line after a semicolon or closing brace) are marked with Comment.Trailing set to true. Detached comments (those separated from any declaration by blank lines) are collected in File.Comments.

The package follows a split error convention. Programmer errors cause panics with a "protoparse: " prefix: Parse panics if input is nil, and ParseTokens panics if the lexer is nil. Data-driven failures (malformed .proto input) are returned as an ErrorList. Each error in the list is formatted as "protoparse: filename:line:col: message" for consistent grep-friendly output.

Error recovery uses synchronization on semicolons, closing braces, and top-level keywords. When the parser encounters an unexpected token, it skips forward to a synchronization point and resumes parsing, accumulating all errors into the ErrorList. This allows a single parse pass to report multiple errors rather than stopping at the first one.

The parser is a pure syntax pass. It does not perform semantic validation such as checking for duplicate field numbers, verifying that enum value zero exists, resolving type references, or enforcing proto2 versus proto3 rules beyond syntactic structure. Those responsibilities belong to a downstream linker or validation pass (see roadmap item 17).

This package depends on protolex from this module and the Go standard library (fmt, math, strconv, strings). It has zero external dependencies.

parse_enum.go handles parsing of enum declarations and enum value declarations in .proto files, and extraction of Go-specific edition features from parsed enum options.

parse_field.go handles parsing of field declarations, map fields, oneof declarations, group declarations, and field-related utilities (type names, field options, ranges) in .proto files.

parse_file.go contains the top-level file parsing logic for .proto files. It handles the file-level declarations: syntax, edition, import, and package. It also provides resolution functions for Go-specific edition features that apply at the file level, such as symbol visibility, naming style, and API mode.

parse_message.go handles parsing of message declarations, message body elements, labeled elements, extend blocks, extensions declarations, and reserved declarations in .proto files.

parse_option.go handles parsing of option declarations, option names, literal values, and aggregate (message) literals in .proto files.

parse_service.go handles parsing of service declarations and RPC method declarations in .proto files.

parser.go contains the core parser infrastructure: the parser struct, public Parse/ParseTokens entry points, token advancement, error recovery, and low-level token consumption helpers.

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func ResolveGoAPIMode

func ResolveGoAPIMode(options []*OptionDecl) descriptor.GoAPIMode

ResolveGoAPIMode examines the parsed file-level options for a Go-specific edition feature extension setting the api_mode behavior. It looks for options with the extension name pattern "(go.features)" and returns the resolved GoAPIMode value. If no matching option is found, it returns the zero value (no override).

func ResolveNamingStyle

func ResolveNamingStyle(options []*OptionDecl) descriptor.NamingStyle

ResolveNamingStyle examines the parsed file-level options for a Go-specific edition feature extension setting the enforce_naming_style behavior. It looks for options with the extension name pattern "(go.features)" and returns the resolved NamingStyle value. If no matching option is found, it returns the zero value (no override).

func ResolveStripEnumPrefix

func ResolveStripEnumPrefix(options []*OptionDecl) descriptor.StripEnumPrefix

ResolveStripEnumPrefix examines the parsed enum options for a Go-specific edition feature extension setting the strip_enum_prefix behavior. It looks for options with the extension name pattern "(go.features)" or "(go.enum).strip_enum_prefix" and returns the resolved StripEnumPrefix value. If no matching option is found, it returns the zero value (no override).

func ResolveSymbolVisibility

func ResolveSymbolVisibility(options []*OptionDecl) descriptor.SymbolVisibility

ResolveSymbolVisibility examines the parsed file-level options for a Go-specific edition feature extension setting the default_symbol_visibility behavior. It looks for options with the extension name pattern "(go.features)" and returns the resolved SymbolVisibility value. If no matching option is found, it returns the zero value (no override).

Types

type AggregateField

type AggregateField struct {
	Pos   Position
	Name  OptionName
	Value Literal
}

AggregateField represents a field: value or field { ... } entry inside an aggregate (message) literal value.

type Comment

type Comment struct {
	Pos      Position
	Text     string
	Trailing bool
}

Comment holds a single comment from the source text. Text includes the delimiters (// or /* */). Trailing is true when the comment appears on the same line after a declaration.

type Decl

type Decl interface {
	// contains filtered or unexported methods
}

Decl is a sealed interface representing a top-level declaration in a .proto file. It is implemented by *MessageDecl, *EnumDecl, *ServiceDecl, and *ExtendDecl.

type EditionDecl

type EditionDecl struct {
	Pos      Position
	Value    string
	Comments []Comment
}

EditionDecl represents an edition declaration (e.g., edition = "2023";). Value holds the string literal value without quotes.

type EnumDecl

type EnumDecl struct {
	Pos      Position
	Name     string
	Values   []*EnumValueDecl
	Options  []*OptionDecl
	Reserved []*ReservedDecl
	Comments []Comment
}

EnumDecl represents an enum declaration. It implements both Decl and MessageElement because enums can appear at the top level and nested inside messages.

type EnumValueDecl

type EnumValueDecl struct {
	Pos      Position
	Name     string
	Number   int32
	Options  []*FieldOption
	Comments []Comment
}

EnumValueDecl represents a single value within an enum declaration.

fieldalignment: fields ordered to match .proto source declaration order

type ErrorList

type ErrorList []error

ErrorList collects multiple parse errors encountered during a single parse operation. It implements the error interface; its Error method joins all contained messages with newlines. The Unwrap method returns the underlying slice for use with errors.As and errors.Is.

func (ErrorList) Error

func (e ErrorList) Error() string

Error returns all error messages joined by newlines.

func (ErrorList) Unwrap

func (e ErrorList) Unwrap() []error

Unwrap returns the underlying error slice, enabling errors.As and errors.Is to match against individual errors within the list.

type ExtendDecl

type ExtendDecl struct {
	Pos      Position
	TypeName TypeName
	Fields   []*FieldDecl
	Groups   []*GroupDecl
	Comments []Comment
}

ExtendDecl represents an extend block. It implements both Decl and MessageElement because extend blocks can appear at the top level and inside messages.

type ExtensionsDecl

type ExtensionsDecl struct {
	Pos      Position
	Ranges   []Range
	Options  []*FieldOption
	Comments []Comment
}

ExtensionsDecl represents an extensions range declaration within a message. It implements MessageElement.

type FieldDecl

type FieldDecl struct {
	Pos      Position
	Label    string
	TypeName TypeName
	Name     string
	Number   int32
	Options  []*FieldOption
	Comments []Comment
}

FieldDecl represents a field declaration within a message or oneof. Label is one of "optional", "required", "repeated", or "" for label-less proto3 fields.

fieldalignment: fields ordered to match .proto source declaration order

type FieldOption

type FieldOption struct {
	Pos   Position
	Name  OptionName
	Value Literal
}

FieldOption represents a single option within the inline option list of a field declaration (the [...] after the field number).

type File

type File struct {
	Pos      Position
	Syntax   *SyntaxDecl
	Edition  *EditionDecl
	Package  *PackageDecl
	Imports  []*ImportDecl
	Options  []*OptionDecl
	Decls    []Decl
	Comments []Comment
}

File is the root AST node representing a parsed .proto file. Exactly one of Syntax or Edition may be non-nil; both nil means the file had no syntax/edition declaration.

func Parse

func Parse(filename string, input []byte) (*File, error)

Parse parses the given .proto source bytes and returns a File AST.

func ParseTokens

func ParseTokens(filename string, lexer *protolex.Lexer) (*File, error)

ParseTokens drives the given lexer to produce tokens and parses them into a File AST.

type GroupDecl

type GroupDecl struct {
	Pos      Position
	Label    string
	Name     string
	Number   int32
	Body     []MessageElement
	Comments []Comment
}

GroupDecl represents a proto2 legacy group declaration.

fieldalignment: fields ordered to match .proto source declaration order

type ImportDecl

type ImportDecl struct {
	Pos      Position
	Path     string
	Modifier ImportModifier
	Comments []Comment
}

ImportDecl represents an import declaration. Path holds the import path without quotes. Modifier indicates whether the import is weak or public.

fieldalignment: fields ordered for semantic clarity, not padding

type ImportModifier

type ImportModifier int8

ImportModifier represents the modifier on an import declaration. The zero value ImportNone means no modifier. ImportWeak and ImportPublic correspond to the weak and public keywords respectively.

const (
	// ImportNone indicates a plain import with no modifier.
	ImportNone ImportModifier = 0

	// ImportWeak indicates a weak import.
	ImportWeak ImportModifier = iota // 1

	// ImportPublic indicates a public import.
	ImportPublic // 2
)

func (ImportModifier) String

func (m ImportModifier) String() string

String returns the name of the import modifier. For valid values (0-2) it returns "none", "weak", or "public". For out-of-range values it returns a formatted string like "ImportModifier(5)".

func (ImportModifier) Valid

func (m ImportModifier) Valid() bool

Valid reports whether m is one of the three defined import modifiers (0 through 2 inclusive).

type Literal

type Literal struct {
	Pos       Position
	Kind      LiteralKind
	Value     string
	Aggregate []*AggregateField
	Sign      string
}

Literal holds a parsed literal value. Kind indicates the type of the literal. Value holds the raw text for scalar literals. Aggregate holds the fields for aggregate literals. Sign holds "-" or "+" if a sign preceded the literal.

type LiteralKind

type LiteralKind int8

LiteralKind represents the type of a literal value in the AST. The zero value is intentionally invalid.

const (
	// LitString represents a string literal value.
	LitString LiteralKind = iota + 1 // 1

	// LitInt represents an integer literal value.
	LitInt // 2

	// LitFloat represents a floating-point literal value.
	LitFloat // 3

	// LitBool represents a boolean literal value (true or false).
	LitBool // 4

	// LitIdentifier represents an identifier literal (e.g., inf, nan, or enum
	// value names).
	LitIdentifier // 5

	// LitAggregate represents an aggregate (message) literal value enclosed
	// in braces.
	LitAggregate // 6
)

func (LiteralKind) String

func (k LiteralKind) String() string

String returns the name of the literal kind. For valid values (1-6) it returns names such as "string" and "aggregate". For out-of-range values it returns a formatted string like "LiteralKind(7)".

func (LiteralKind) Valid

func (k LiteralKind) Valid() bool

Valid reports whether k is one of the six defined literal kinds (1 through 6 inclusive).

type MapFieldDecl

type MapFieldDecl struct {
	Pos       Position
	KeyType   string
	ValueType TypeName
	Name      string
	Number    int32
	Options   []*FieldOption
	Comments  []Comment
}

MapFieldDecl represents a map field declaration (e.g., map<string, int32> name = 1;). KeyType holds the map key type keyword.

fieldalignment: fields ordered to match .proto source declaration order

type MessageDecl

type MessageDecl struct {
	Pos      Position
	Name     string
	Body     []MessageElement
	Comments []Comment
}

MessageDecl represents a message declaration. Body holds all nested elements in source order. It implements both Decl and MessageElement because messages can appear at the top level and nested inside other messages.

type MessageElement

type MessageElement interface {
	// contains filtered or unexported methods
}

MessageElement is a sealed interface representing an element that can appear inside a message body. It is implemented by *FieldDecl, *MapFieldDecl, *OneofDecl, *EnumDecl, *MessageDecl, *ExtendDecl, *ExtensionsDecl, *ReservedDecl, *OptionDecl, and *GroupDecl.

type MethodDecl

type MethodDecl struct {
	Pos             Position
	Name            string
	InputType       TypeName
	OutputType      TypeName
	ClientStreaming bool
	ServerStreaming bool
	Options         []*OptionDecl
	Comments        []Comment
}

MethodDecl represents an RPC method declaration within a service. ClientStreaming and ServerStreaming indicate whether the stream keyword appeared before the input or output type respectively.

fieldalignment: fields ordered to match .proto source declaration order

type OneofDecl

type OneofDecl struct {
	Pos      Position
	Name     string
	Fields   []*FieldDecl
	Options  []*OptionDecl
	Comments []Comment
}

OneofDecl represents a oneof declaration. Fields inside a oneof do not carry a label.

type OptionDecl

type OptionDecl struct {
	Pos      Position
	Name     OptionName
	Value    Literal
	Comments []Comment
}

OptionDecl represents an option declaration. It implements both Decl and MessageElement because options can appear at the top level and inside message bodies.

type OptionName

type OptionName struct {
	Parts []OptionNamePart
}

OptionName represents the full path of an option name, such as (com.example.my_option).field.subfield, as a sequence of parts.

type OptionNamePart

type OptionNamePart struct {
	Name        string
	IsExtension bool
}

OptionNamePart represents a single component of an option name path. When IsExtension is true, the part was parenthesized in the source (e.g., (com.example.my_option)).

type PackageDecl

type PackageDecl struct {
	Pos      Position
	Name     string
	Comments []Comment
}

PackageDecl represents a package declaration. Name holds the full dotted package name (e.g., "foo.bar.baz").

type ParseError

type ParseError struct {
	// Pos is the source location where the error occurred.
	Pos Position
	// Detail is a human-readable description of the failure.
	Detail string
	// Cause is the underlying error, if any.
	Cause error
}

ParseError is a typed error returned when proto file parsing encounters a syntax or semantic error at a known source location. It includes file, line, and column context and wraps an optional underlying cause.

func (*ParseError) Error

func (e *ParseError) Error() string

Error returns a human-readable message with the source location prefix.

func (*ParseError) Unwrap

func (e *ParseError) Unwrap() error

Unwrap returns the underlying cause so that errors.Is and errors.As work.

type Position

type Position struct {
	Filename string
	Line     int
	Column   int
}

Position records the source location of a token or AST node within a .proto file. Line and Column are 1-based.

type Range

type Range struct {
	Start int32
	End   int32
	Max   bool
}

Range represents a numeric range used in extensions and reserved declarations. For single-value ranges Start equals End and Max is false. Max is true when the range endpoint was the keyword max.

type ReservedDecl

type ReservedDecl struct {
	Pos      Position
	Ranges   []Range
	Names    []string
	Comments []Comment
}

ReservedDecl represents a reserved declaration within a message or enum. A single reserved statement contains either ranges or names, never both. It implements MessageElement.

type ServiceDecl

type ServiceDecl struct {
	Pos      Position
	Name     string
	Methods  []*MethodDecl
	Options  []*OptionDecl
	Comments []Comment
}

ServiceDecl represents a service declaration. It implements Decl because services are top-level declarations.

type SyntaxDecl

type SyntaxDecl struct {
	Pos      Position
	Value    string
	Comments []Comment
}

SyntaxDecl represents a syntax declaration (e.g., syntax = "proto3";). Value holds the string literal value without quotes.

type TypeName

type TypeName struct {
	Parts    []string
	Absolute bool
}

TypeName represents a protobuf type reference. Parts holds the dot-separated components (e.g., ["foo", "bar", "Baz"]). Absolute is true when the name had a leading dot in the source.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL