README
¶
protobuf
A pure-Go Protocol Buffers implementation optimized for both standard Go and TinyGo compilation targets, including WebAssembly.
Install
go get github.com/0verkilll/protobuf
Sponsor
If this project is useful to you, please consider supporting its development:
License
Documentation
¶
Overview ¶
Package protobuf is a pure-Go Protocol Buffers implementation optimized for both standard Go (gc) and TinyGo compilation targets, including WebAssembly.
TinyGo Compatibility ¶
The library is organized into tiers with different TinyGo support levels:
Core tier (wire, scalar, field, encode, decode, proto, pool): fully supported under TinyGo. These packages compile to WASM binaries under 1 MB and contain no fmt, reflect, or unsafe imports.
Extended tier (descriptor, dynamicpb, protoreflect, registry, jsoncodec, textcodec): supported under TinyGo with the feature limitations described below.
Excluded tier (protocompile, protoparse, protolex, codegen, protogen, grpccodec, grpcreflect, connectcodec): not expected to compile under TinyGo.
Features Unavailable Under TinyGo ¶
The following features are gated behind //go:build !tinygo and have fallback implementations under TinyGo:
sync.Pool buffer reuse: The pool package uses sync.Pool under standard Go for efficient buffer recycling. Under TinyGo, Get allocates directly via make and Put is a no-op. Functionality is identical; only allocation patterns differ.
String interning via unique.Make: The descriptor package uses unique.Make under standard Go to deduplicate string values across descriptors, reducing heap footprint. Under TinyGo, strings are stored without deduplication, resulting in higher memory usage for large registries.
init()-based auto-registration: Several Extended tier packages (dynamicpb, wkt, protodesc, conformance) register types and descriptors in init() functions under standard Go. Under TinyGo, consumers must call the exported Register() function in each package explicitly before using those packages. Example:
import "github.com/0verkilll/protobuf/dynamicpb" func main() { dynamicpb.Register() // ... use dynamicpb }
Unsafe zero-copy UnmarshalUnsafe: Generated code includes an UnmarshalUnsafe method that uses unsafe.String for zero-copy string aliasing under standard Go. Under TinyGo, UnmarshalUnsafe delegates to the standard Unmarshal method with copying semantics. The API surface is identical; only performance characteristics differ.
TinyGo Build Optimization Flags ¶
Consumers building WASM binaries with TinyGo can use the following flags to optimize binary size:
- -opt=z: Size-optimized compilation (TinyGo default).
- --no-debug: Strip debug symbols for smaller binaries.
- -gc=leaking: Disable garbage collection for short-lived WASM modules.
- -scheduler=none: Disable the goroutine scheduler for single-threaded use.
- -panic=trap: Trap on panic instead of including the full panic runtime.
Example build command:
tinygo build -target=wasm -opt=z --no-debug -o output.wasm ./cmd/wasm/
Source Files
Directories
¶
| Path | Synopsis |
|---|---|
|
cmd
|
|
|
conformance
command
Command conformance implements the protobuf conformance test runner protocol.
|
Command conformance implements the protobuf conformance test runner protocol. |
|
Package codegen -- decode_enum.go contains enum-level descriptor decoding: EnumDescriptorProto and EnumValueDescriptorProto wire parsing, and the buildEnumDescriptor pass-2 builder.
|
Package codegen -- decode_enum.go contains enum-level descriptor decoding: EnumDescriptorProto and EnumValueDescriptorProto wire parsing, and the buildEnumDescriptor pass-2 builder. |
|
Package conformance provides programmatically-built TestAllTypes message descriptors and a dedicated TypeRegistry for conformance testing against the official protobuf conformance test suite.
|
Package conformance provides programmatically-built TestAllTypes message descriptors and a dedicated TypeRegistry for conformance testing against the official protobuf conformance test suite. |
|
Package decode implements a binary unmarshaling engine for Protocol Buffers messages.
|
Package decode implements a binary unmarshaling engine for Protocol Buffers messages. |
|
Package descriptor -- builder_enum.go contains the EnumDescriptorBuilder and EnumValueDescriptorBuilder for constructing immutable enum descriptors through chainable setter APIs.
|
Package descriptor -- builder_enum.go contains the EnumDescriptorBuilder and EnumValueDescriptorBuilder for constructing immutable enum descriptors through chainable setter APIs. |
|
Package dynamicpb provides a dynamic protobuf message implementation that constructs messages from descriptors at runtime without generated code.
|
Package dynamicpb provides a dynamic protobuf message implementation that constructs messages from descriptors at runtime without generated code. |
|
Package encode implements a binary marshaling engine for Protocol Buffers messages.
|
Package encode implements a binary marshaling engine for Protocol Buffers messages. |
|
Package field bridges the wire-level encoder/decoder and concrete message types by providing optional field presence tracking via Go generics, repeated field encoding/decoding with append semantics, and packed repeated encoding for numeric types.
|
Package field bridges the wire-level encoder/decoder and concrete message types by providing optional field presence tracking via Go generics, repeated field encoding/decoding with append semantics, and packed repeated encoding for numeric types. |
|
internal
|
|
|
benchfixture
Package benchfixture provides canonical message fixtures at three size tiers (Small, Medium, Large) for benchmark tests.
|
Package benchfixture provides canonical message fixtures at three size tiers (Small, Medium, Large) for benchmark tests. |
|
Package jsoncodec implements JSON marshaling and unmarshaling for protobuf messages.
|
Package jsoncodec implements JSON marshaling and unmarshaling for protobuf messages. |
|
Package pool provides a sync.Pool-based buffer pool with size-class bucketing for zero-allocation serialization workflows.
|
Package pool provides a sync.Pool-based buffer pool with size-class bucketing for zero-allocation serialization workflows. |
|
Package proto defines the foundational interface hierarchy for protobuf messages, following the Interface Segregation Principle (ISP).
|
Package proto defines the foundational interface hierarchy for protobuf messages, following the Interface Segregation Principle (ISP). |
|
Package protocompile -- build_link.go contains the core descriptor building functions that construct FileDescriptor and MessageDescriptor trees from resolved AST files.
|
Package protocompile -- build_link.go contains the core descriptor building functions that construct FileDescriptor and MessageDescriptor trees from resolved AST files. |
|
Package protodelim implements size-delimited encoding and decoding of Protocol Buffers messages for stream-oriented I/O. Each message is preceded by its size encoded as a base-128 varint, allowing multiple messages to be concatenated on a single io.Writer and read back sequentially from an io.Reader.
|
Package protodelim implements size-delimited encoding and decoding of Protocol Buffers messages for stream-oriented I/O. Each message is preceded by its size encoded as a base-128 varint, allowing multiple messages to be concatenated on a single io.Writer and read back sequentially from an io.Reader. |
|
Package protodesc provides bidirectional conversion between this project's descriptor types and the google.protobuf.FileDescriptorProto wire format.
|
Package protodesc provides bidirectional conversion between this project's descriptor types and the google.protobuf.FileDescriptorProto wire format. |
|
Package protogen provides a framework for writing protoc code generator plugins.
|
Package protogen provides a framework for writing protoc code generator plugins. |
|
Package protolex implements a pull-based lexer for .proto files that converts raw source text into a stream of typed tokens with position tracking.
|
Package protolex implements a pull-based lexer for .proto files that converts raw source text into a stream of typed tokens with position tracking. |
|
Package protoparse implements a recursive descent parser for Protocol Buffer definition files (.proto).
|
Package protoparse implements a recursive descent parser for Protocol Buffer definition files (.proto). |
|
Package protopath provides types for representing paths through a protobuf message tree.
|
Package protopath provides types for representing paths through a protobuf message tree. |
|
Package protorange provides depth-first traversal of protobuf messages.
|
Package protorange provides depth-first traversal of protobuf messages. |
|
Package protoreflect provides the runtime reflection layer that bridges protobuf descriptors (schema) and message instances (data).
|
Package protoreflect provides the runtime reflection layer that bridges protobuf descriptors (schema) and message instances (data). |
|
Package registry provides thread-safe runtime registration and lookup of protobuf file descriptors and type descriptors (messages, enums, and extensions) by fully qualified name or file path.
|
Package registry provides thread-safe runtime registration and lookup of protobuf file descriptors and type descriptors (messages, enums, and extensions) by fully qualified name or file path. |
|
Package scalar provides a type-safe encoding, decoding, and size computation layer for all protobuf scalar types.
|
Package scalar provides a type-safe encoding, decoding, and size computation layer for all protobuf scalar types. |
|
Package textcodec implements protobuf text format marshaling and unmarshaling for protobuf messages.
|
Package textcodec implements protobuf text format marshaling and unmarshaling for protobuf messages. |
|
Package tinygo_test contains integration tests that verify the protobuf library's core packages work correctly under both the standard Go compiler and TinyGo.
|
Package tinygo_test contains integration tests that verify the protobuf library's core packages work correctly under both the standard Go compiler and TinyGo. |
|
Package wire implements the foundational wire format encoding and decoding primitives for a Protocol Buffers library.
|
Package wire implements the foundational wire format encoding and decoding primitives for a Protocol Buffers library. |
|
Package wkt -- descriptors.go builds all Well-Known Type descriptors and registers them into the global registries.
|
Package wkt -- descriptors.go builds all Well-Known Type descriptors and registers them into the global registries. |