Help us shape the future of Go! Take the 2026 Go Developer Survey

protogen

package
v0.36.2 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Oct 19, 2026 License: MIT Imports: 31 Imported by: 1

Documentation

Index

Constants

View Source 
const CacheVersion = 2

CacheVersion is the current cache format version.

View Source 
const (
	// CppBuildTag is the build tag prepended to C++ files.
	CppBuildTag = "//go:build deps_only && cgo"
)
View Source 
const DefaultCacheFile = ".protoc-manifest.json"

DefaultCacheFile is the default cache file name.

View Source 
const DefaultGoLiteFeatures = "marshal+unmarshal+size+equal+json+clone+text"

DefaultGoLiteFeatures is the default set of go-lite features to enable.

View Source 
const DefaultRustInventory = ".protoc-rust-files.txt"

DefaultRustInventory is the default generated-file inventory for Rust output.

View Source 
const TypeIDPluginName = "protoc-gen-go-typeid"

TypeIDPluginName is the protoc plugin executable name served in process.

Variables

This section is empty.

Functions

func DiscoverProtoFiles 

func DiscoverProtoFiles(projectDir string, patterns, excludePatterns []string) ([]string, error)

DiscoverProtoFiles finds proto files matching the given patterns. Uses git ls-files to find tracked proto files. excludePatterns allows excluding files that match certain patterns.

func FindGeneratedFilesForProto 

func FindGeneratedFilesForProto(protoFile, projectDir, vendorDir, modulePath string, langs Languages, rpcs RPCLibraries) ([]string, error)

FindGeneratedFilesForProto finds actual enabled outputs for a proto file.

func FindModuleDir added in v0.32.4 

func FindModuleDir(projectDir string) (string, error)

FindModuleDir finds the nearest ancestor directory containing go.mod.

func GenerateTypeIDs added in v0.36.0 

func GenerateTypeIDs(input []byte) ([]byte, error)

GenerateTypeIDs answers one encoded CodeGeneratorRequest for the typeid generator. For each proto file with typed messages it emits <name>_typeid.pb.go holding a TypeID constant and method per message and the file's []typeid.Type list.

func GetGeneratedFiles 

func GetGeneratedFiles(protoFile, projectDir, modulePath string, hasGo, hasTS bool) []string

GetGeneratedFiles returns the expected generated file paths for a proto file.

func GetGoModule 

func GetGoModule(projectDir string) (string, error)

GetGoModule reads the module path from go.mod in the given directory.

func GetPackageKey 

func GetPackageKey(modulePath, protoFile string) string

GetPackageKey generates a cache key for a proto file. Uses the format: "module/path/to/dir;package_name"

func HashProtocFlags added in v0.34.3 

func HashProtocFlags(flags []string, rootDir string) string

HashProtocFlags computes a checkout-portable hash of the protoc flags. Absolute path prefixes under rootDir are rewritten to rootDir-relative form so two checkouts of identical content at different absolute paths hash the same, while flag semantics (enabled plugins, plugin options, output layout) still change the hash. rootDir must be the absolute Go module root.

Types

type Cache 

type Cache struct {
	// Version is the cache format version.
	Version int `json:"version"`
	// ProtocFlagsHash is the hash of the protoc flags.
	ProtocFlagsHash string `json:"protocFlagsHash"`
	// ToolVersions stores tool version strings for cache invalidation.
	ToolVersions string `json:"toolVersions,omitempty"`
	// Packages maps package identifiers to package info.
	Packages map[string]*PackageInfo `json:"packages"`
}

Cache represents the protoc manifest cache.

func LoadCache 

func LoadCache(path string) (*Cache, error)

LoadCache loads the cache from a file. Returns an empty cache if the file doesn't exist.

func NewCache 

func NewCache() *Cache

NewCache creates a new empty cache.

func (*Cache) CleanOrphanedPackages 

func (c *Cache) CleanOrphanedPackages(currentPackages map[string]struct{}, projectDir string)

CleanOrphanedPackages removes packages from the cache that are not in currentPackages and no longer have any proto file on disk. Packages outside currentPackages whose sources still exist are kept, so a run targeting a subset of the tree does not discard the entries of the rest.

func (*Cache) DropOutsidePackages added in v0.36.2 

func (c *Cache) DropOutsidePackages(currentPackages map[string]struct{})

DropOutsidePackages removes every package not in currentPackages. It is used when the protoc flags or tool versions changed: the outputs of packages that were not regenerated no longer match the new identity recorded in the cache.

func (*Cache) NeedsRegeneration 

func (c *Cache) NeedsRegeneration(packageKey string, protoFiles []string, projectDir string, flagsHash string, toolVersions string, force bool) (bool, error)

NeedsRegeneration checks if a cached package needs regeneration. Returns true if: - The package is not in the cache - The proto file list or content hash has changed - The protoc flags or selected tool versions have changed - Force is true

func (*Cache) Save 

func (c *Cache) Save(path string) error

Save writes the cache to a file.

func (*Cache) SetProtocFlags 

func (c *Cache) SetProtocFlags(flags []string, rootDir string)

SetProtocFlags sets the protoc flags hash, keyed on rootDir-relative paths.

func (*Cache) SetToolVersions 

func (c *Cache) SetToolVersions(versions string)

SetToolVersions sets the tool versions string.

func (*Cache) UpdatePackage 

func (c *Cache) UpdatePackage(packageKey string, protoFiles []string, generatedFiles []string, projectDir string) error

UpdatePackage updates the cache for a package after generation.

type Config 

type Config struct {
	// ProjectDir is the project directory.
	// If empty, uses the current working directory.
	ProjectDir string
	// Targets is the list of proto file glob patterns to process.
	// Default: ["./*.proto"]
	Targets []string
	// Exclude is a list of proto file glob patterns to exclude.
	// Files matching any of these patterns will be skipped.
	Exclude []string
	// Force regenerates all files regardless of cache.
	Force bool
	// CacheFile is the path to the cache file.
	// Default: ".protoc-manifest.json"
	CacheFile string
	// Verbose enables verbose output.
	Verbose bool
	// GoLiteFeatures is the go-lite features to enable.
	// Default: "marshal+unmarshal+size+equal+json+clone+text"
	GoLiteFeatures string
	// ToolsDir is the tools directory containing plugin binaries.
	// Default: ".tools"
	ToolsDir string
	// ExtraArgs contains any additional protoc arguments.
	ExtraArgs []string
	// Languages is the opt-in protobuf output language filter.
	// Empty preserves the Go, TypeScript, C++, and Rust defaults.
	Languages []string
	// RPCLibraries is the opt-in RPC stub generator filter.
	// Empty enables the default RPC library set.
	RPCLibraries []string
	// TsImportBoundaries are module-relative path prefixes where generated
	// TypeScript protobuf imports should switch to @go/... when crossing
	// between boundaries.
	TsImportBoundaries []string
}

Config contains the configuration for proto generation.

func NewConfig 

func NewConfig() *Config

NewConfig returns a new Config with default values.

func (*Config) GetCacheFilePath 

func (c *Config) GetCacheFilePath() (string, error)

GetCacheFilePath returns the absolute path to the cache file.

func (*Config) GetGoModule 

func (c *Config) GetGoModule() (string, error)

GetGoModule returns the effective Go import path for the project directory.

func (*Config) GetLanguages added in v0.34.0 

func (c *Config) GetLanguages() (Languages, error)

GetLanguages returns configured output languages. Explicit config takes precedence; otherwise reads package.json aptre config.

func (*Config) GetModuleDir added in v0.32.4 

func (c *Config) GetModuleDir() (string, error)

GetModuleDir returns the nearest ancestor directory containing go.mod.

func (*Config) GetProjectDir 

func (c *Config) GetProjectDir() (string, error)

GetProjectDir returns the project directory, defaulting to cwd.

func (*Config) GetRPCLibraries added in v0.34.1 

func (c *Config) GetRPCLibraries() (RPCLibraries, error)

GetRPCLibraries returns configured RPC generators. Explicit config takes precedence; otherwise reads package.json aptre config.

func (*Config) GetRust added in v0.36.0 

func (c *Config) GetRust() (*RustConfig, error)

GetRust returns the whole-graph Rust configuration from package.json. It returns nil when the project does not configure aptre.rust.

func (*Config) GetSchemaModule added in v0.36.0 

func (c *Config) GetSchemaModule() (string, error)

GetSchemaModule returns the import path that prefixes this project's schemas. The aptre.module setting in package.json takes precedence, which lets a project without go.mod name its schemas; otherwise it is the Go module path.

func (*Config) GetToolsDir 

func (c *Config) GetToolsDir() (string, error)

GetToolsDir returns the absolute path to the tools directory.

func (*Config) GetTsImportBoundaries added in v0.32.5 

func (c *Config) GetTsImportBoundaries() ([]string, error)

GetTsImportBoundaries returns configured TypeScript import boundaries. Explicit config takes precedence; otherwise reads package.json aptre config.

func (*Config) GetTsTypedDecode added in v0.36.1 

func (c *Config) GetTsTypedDecode() (bool, error)

GetTsTypedDecode reports whether package.json asks protoc-gen-es-lite for a typed decoder per message, which runs faster than the shared field walker in interpreters and ahead-of-time compilers at the cost of larger output.

func (*Config) HasGoMod 

func (c *Config) HasGoMod() (bool, error)

HasGoMod checks if go.mod exists in the project directory.

func (*Config) HasPackageJSON 

func (c *Config) HasPackageJSON() (bool, error)

HasPackageJSON checks if package.json exists in the project directory.

type Generator 

type Generator struct {
	// Config is the generator configuration.
	Config *Config
	// Plugins contains the discovered plugins.
	Plugins *Plugins
	// Cache is the manifest cache.
	Cache *Cache
	// ProjectDir is the resolved project directory.
	ProjectDir string
	// ModuleDir is the resolved Go module root directory.
	ModuleDir string
	// ModulePath is the Go module path.
	ModulePath string
	// VendorDir is the vendor directory.
	VendorDir string
	// TsImportBoundaries are module-relative boundaries that trigger @go/ rewrites.
	TsImportBoundaries []string
	// OutDir is the output directory (same as VendorDir).
	OutDir string
	// Verbose enables verbose output.
	Verbose bool
	// Stdout is where to write standard output.
	Stdout io.Writer
	// Stderr is where to write error output.
	Stderr io.Writer
}

Generator handles protobuf code generation.

func NewGenerator 

func NewGenerator(cfg *Config) (*Generator, error)

NewGenerator creates a new Generator.

func (*Generator) Clean 

func (g *Generator) Clean() error

Clean removes all generated files and the cache.

func (*Generator) Generate 

func (g *Generator) Generate(ctx context.Context) error

Generate runs the proto generation.

type Language added in v0.34.0 

type Language string

Language identifies a protobuf output language. 

const (
	// LanguageGo enables Go protobuf outputs.
	LanguageGo Language = "go"
	// LanguageTypeScript enables TypeScript protobuf outputs.
	LanguageTypeScript Language = "ts"
	// LanguageCpp enables C++ protobuf outputs.
	LanguageCpp Language = "cpp"
	// LanguageRust enables Rust protobuf outputs.
	LanguageRust Language = "rust"
	// LanguageCSharp enables C# protobuf outputs.
	LanguageCSharp Language = "csharp"
	// LanguagePython enables Python protobuf outputs.
	LanguagePython Language = "python"
)

type Languages added in v0.34.0 

type Languages map[Language]struct{}

Languages contains the enabled protobuf output languages.

func NewLanguages added in v0.34.0 

func NewLanguages(names []string) (Languages, error)

NewLanguages validates and normalizes output language names. Empty input preserves the historical default of all languages.

func (Languages) Has added in v0.34.0 

func (l Languages) Has(lang Language) bool

Has returns true when the language is enabled.

func (Languages) Names added in v0.36.0 

func (l Languages) Names() []string

Names returns the enabled language names in sorted order.

func (Languages) Without added in v0.36.0 

func (l Languages) Without(lang Language) Languages

Without returns the languages with lang removed.

type NativePluginHandler 

type NativePluginHandler struct {
	// Plugins is the configured plugins.
	Plugins *Plugins
	// Verbose enables verbose output.
	Verbose bool
	// contains filtered or unexported fields
}

NativePluginHandler implements go-protoc-wasi's PluginHandler interface. It spawns native plugin processes and handles IPC. For protoc-gen-prost, it uses the embedded WASM module instead of a native binary.

func NewNativePluginHandler 

func NewNativePluginHandler(plugins *Plugins, verbose bool) *NativePluginHandler

NewNativePluginHandler creates a new NativePluginHandler.

func (*NativePluginHandler) CloseProstWASM added in v0.30.0 

func (h *NativePluginHandler) CloseProstWASM(ctx context.Context) error

CloseProstWASM closes the prost WASM plugin if initialized.

func (*NativePluginHandler) Communicate 

func (h *NativePluginHandler) Communicate(ctx context.Context, program string, searchPath bool, input []byte) ([]byte, error)

Communicate implements the PluginHandler interface. It spawns a plugin process, sends the CodeGeneratorRequest via stdin, and returns the CodeGeneratorResponse from stdout. For protoc-gen-prost, it uses the embedded WASM module if initialized, and it serves protoc-gen-go-typeid in process.

func (*NativePluginHandler) InitProstWASM added in v0.30.0 

func (h *NativePluginHandler) InitProstWASM(ctx context.Context, runtime wazero.Runtime) error

InitProstWASM initializes the prost WASM plugin with the given wazero runtime. The runtime must already have WASI instantiated (e.g., by protoc). This should be called after protoc.Init() and before running protoc.

type PackageInfo 

type PackageInfo struct {
	// Hash is the content hash of all proto files in this package.
	Hash string `json:"hash"`
	// GeneratedFiles is the list of generated output files.
	GeneratedFiles []string `json:"generatedFiles"`
	// ProtoFiles is the list of source proto file paths.
	ProtoFiles []string `json:"protoFiles"`
}

PackageInfo contains cached information about a proto package.

type Plugin 

type Plugin struct {
	// Name is the plugin name (e.g., "go-lite", "es-lite").
	Name string
	// BinaryName is the executable name (e.g., "protoc-gen-go-lite").
	BinaryName string
	// Path is the full path to the plugin binary.
	Path string
	// Type is the plugin type.
	Type PluginType
	// OutFlag is the output flag name (e.g., "go-lite_out").
	OutFlag string
	// Options are the plugin options.
	Options map[string]string
	// Flags are plugin options forwarded verbatim in order, for options that
	// repeat or take no value.
	Flags []string
}

Plugin represents a protoc plugin configuration.

type PluginType 

type PluginType int

PluginType represents the type of protoc plugin. 

const (
	// PluginTypeGo generates Go declarations.
	PluginTypeGo PluginType = iota
	// PluginTypePython generates Python declarations.
	PluginTypePython
	// PluginTypeTypeScript generates TypeScript declarations.
	PluginTypeTypeScript
	// PluginTypeCpp generates C++ declarations.
	PluginTypeCpp
	// PluginTypeRust generates Rust declarations.
	PluginTypeRust
)

type Plugins 

type Plugins struct {
	// Languages contains the enabled output languages.
	Languages Languages
	// RPCLibraries contains the enabled RPC stub generators.
	RPCLibraries RPCLibraries
	// GoLite is the protoc-gen-go-lite plugin.
	GoLite *Plugin
	// GoStarpc is the protoc-gen-go-starpc plugin.
	GoStarpc *Plugin
	// GoTypeID is the protoc-gen-go-typeid plugin, served in process by
	// GenerateTypeIDs.
	GoTypeID *Plugin
	// ESLite is the protoc-gen-es-lite plugin.
	ESLite *Plugin
	// ESStarpc is the protoc-gen-es-starpc plugin.
	ESStarpc *Plugin
	// CppStarpc is the protoc-gen-starpc-cpp plugin.
	CppStarpc *Plugin
	// RustStarpc is the protoc-gen-starpc-rust plugin.
	RustStarpc *Plugin
	// StarpcPython is the protoc-gen-starpc-python plugin.
	StarpcPython *Plugin
	// RustProst is the protoc-gen-prost plugin for Rust protobuf types.
	// This uses an embedded WASM module, no external binary required.
	RustProst *Plugin
}

Plugins holds the configured plugins for a project.

func DiscoverPlugins 

func DiscoverPlugins(cfg *Config) (*Plugins, error)

DiscoverPlugins finds and configures available plugins.

func (*Plugins) GetProtocArgs 

func (p *Plugins) GetProtocArgs(outDir, csharpOutDir string) []string

GetProtocArgs returns deterministic protoc arguments for configured outputs.

type PostProcessor 

type PostProcessor struct {
	// ProjectDir is the project directory.
	ProjectDir string
	// ModulePath is the Go module path.
	ModulePath string
	// TsImportBoundaries are module-relative prefixes that trigger @go/ rewrites
	// when generated TS protobuf imports cross between them.
	TsImportBoundaries []string
	// VendorDir is the vendor directory path.
	VendorDir string
	// Verbose enables verbose output.
	Verbose bool
}

PostProcessor handles post-processing of generated files.

func NewPostProcessor 

func NewPostProcessor(projectDir, vendorDir, modulePath string, tsImportBoundaries []string, verbose bool) *PostProcessor

NewPostProcessor creates a new PostProcessor.

func (*PostProcessor) ProcessAllCppFiles 

func (p *PostProcessor) ProcessAllCppFiles(dir string) error

ProcessAllCppFiles finds and processes all C++ files in a directory.

func (*PostProcessor) ProcessAllTsFiles 

func (p *PostProcessor) ProcessAllTsFiles(dir string) error

ProcessAllTsFiles finds and processes all TypeScript files in a directory.

func (*PostProcessor) ProcessCppFile 

func (p *PostProcessor) ProcessCppFile(filePath string) error

ProcessCppFile processes a C++ file. - Prepends the Go build tag if not present. - Rewrites include paths from absolute to relative.

func (*PostProcessor) ProcessGeneratedFiles 

func (p *PostProcessor) ProcessGeneratedFiles(protoFile string) error

ProcessGeneratedFiles processes all generated files for a proto file.

func (*PostProcessor) ProcessGoFile added in v0.26.10 

func (p *PostProcessor) ProcessGoFile(filePath string) error

ProcessGoFile processes a Go file. Rewrites standard protobuf imports to protobuf-go-lite equivalents.

func (*PostProcessor) ProcessPythonFile added in v0.35.1 

func (p *PostProcessor) ProcessPythonFile(filePath string) error

ProcessPythonFile rewrites canonical Go module imports to the module-relative packages installed by the current project and its vendored dependencies.

func (*PostProcessor) ProcessRustFiles added in v0.30.0 

func (p *PostProcessor) ProcessRustFiles(protoFile string) error

ProcessRustFiles moves and processes generated Rust .pb.rs files. Prost generates files at vendor/{package_path}/{basename}.pb.rs based on the protobuf package hierarchy, but we need them at vendor/{module_path}/{proto_dir}/ to match other generated files.

func (*PostProcessor) ProcessTsFile 

func (p *PostProcessor) ProcessTsFile(filePath string) error

ProcessTsFile processes a TypeScript file. Rewrites relative import paths to @go/ format for vendor dependencies.

The generated TypeScript files contain relative imports based on the proto file paths. For example, a file generated from "github.com/aperturerobotics/bifrost/daemon/api/api.proto" might import from "../../../controllerbus/bus/api/api.pb.js" which needs to be rewritten to "@go/github.com/aperturerobotics/controllerbus/bus/api/api.pb.js".

type ProtocRun added in v0.36.0 

type ProtocRun struct {
	// Plugins resolves the plugin programs that protoc requests.
	Plugins *Plugins
	// Mounts are host directories that protoc reads or writes, visible at
	// the same path inside the runtime.
	Mounts []string
	// Args are the protoc arguments after the program name.
	Args []string
	// Verbose prints the command line and protoc output to Stdout.
	Verbose bool
	// Stdout receives verbose output. It may be nil when Verbose is false.
	Stdout io.Writer
}

ProtocRun is one protoc invocation inside the embedded WASI runtime.

func (*ProtocRun) Run added in v0.36.0 

func (r *ProtocRun) Run(ctx context.Context) error

Run executes protoc and returns an error when it exits unsuccessfully.

type RPCLibraries added in v0.34.1 

type RPCLibraries map[RPCLibrary]struct{}

RPCLibraries contains the enabled RPC stub generators.

func NewRPCLibraries added in v0.34.1 

func NewRPCLibraries(names []string) (RPCLibraries, error)

NewRPCLibraries validates and normalizes RPC generator names.

func (RPCLibraries) Has added in v0.34.1 

func (r RPCLibraries) Has(lib RPCLibrary) bool

Has returns true when the RPC generator is enabled.

type RPCLibrary added in v0.34.1 

type RPCLibrary string

RPCLibrary identifies an RPC stub generator family. 

const (
	// RPCLibraryStarpc enables StarPC RPC stubs.
	RPCLibraryStarpc RPCLibrary = "starpc"
	// RPCLibraryStarpcPython enables StarPC Python RPC stubs.
	RPCLibraryStarpcPython RPCLibrary = "starpc-python"
)

type RustConfig added in v0.36.0 

type RustConfig struct {
	// ProstOptions are protoc-gen-prost options forwarded verbatim, such as
	// "btree_map=." or "extern_path=.pkg=::crate::pkg".
	ProstOptions []string
	// Exclude lists proto path patterns to skip, in addition to Config.Exclude.
	Exclude []string
	// DescriptorSet is where the encoded FileDescriptorSet of the whole graph,
	// with imports and source info, is written. Empty skips the descriptor set.
	DescriptorSet string
	// ModuleFile is where the module declarations are written. Each generated
	// file is included under the Rust module of its protobuf package. Empty
	// skips the module file.
	ModuleFile string
	// Inventory lists every generated file so that outputs of deleted schemas
	// can be removed. Default: DefaultRustInventory.
	Inventory string
}

RustConfig configures whole-graph Rust generation.

Every path is slash-separated and relative to the project directory, which is also the Rust crate root.

func (*RustConfig) FromJSON added in v0.36.0 

func (c *RustConfig) FromJSON(value *fastjson.Value) error

FromJSON reads the options and relative output paths for whole-graph Rust generation.

type RustGenerator added in v0.36.0 

type RustGenerator struct {
	// Config is the generator configuration.
	Config *Config
	// Rust is the Rust section of the project configuration.
	Rust *RustConfig
	// Plugins contains the Rust plugins.
	Plugins *Plugins
	// ProjectDir is the resolved project directory and crate root.
	ProjectDir string
	// ModulePath is the import path prefix of the project's schemas.
	ModulePath string
	// Stdout is where to write verbose output.
	Stdout io.Writer
}

RustGenerator generates Rust protobuf bindings for the whole schema graph.

Unlike Generator, which caches per directory, it compiles every schema in one protoc run so that type resolution sees the full graph, and it owns the complete output set: messages, service bindings, module declarations, the reflection descriptor set, and the inventory used to remove obsolete files. Output is built in memory and compared with the project before any write, which is what makes a read-only drift check possible.

func NewRustGenerator added in v0.36.0 

func NewRustGenerator(cfg *Config) (*RustGenerator, error)

NewRustGenerator creates a RustGenerator from the aptre.rust configuration.

func (*RustGenerator) Check added in v0.36.0 

func (g *RustGenerator) Check(ctx context.Context) ([]string, error)

Check compares the project with freshly generated output without writing. It returns the project-relative paths that are outdated, missing, or obsolete.

func (*RustGenerator) Generate added in v0.36.0 

func (g *RustGenerator) Generate(ctx context.Context) ([]string, error)

Generate writes the changed outputs and removes obsolete ones. It returns the project-relative paths it wrote or removed.

type RustModules added in v0.36.0 

type RustModules struct {
	// contains filtered or unexported fields
}

RustModules declares generated Rust files under the module of their protobuf package. A package whose files sit in several directories gets one module, so each message is defined once and references between its files resolve. Modules are named as prost names them, so packages that prost merges into one module, such as Upper.case and upper.case, share one here.

func NewRustModules added in v0.36.0 

func NewRustModules() *RustModules

NewRustModules constructs an empty module tree.

func (*RustModules) Add added in v0.36.0 

func (m *RustModules) Add(protoPackage, file string)

Add includes file in the module of the dotted protobuf package. An empty package selects the root module.

func (*RustModules) Render added in v0.36.0 

func (m *RustModules) Render() string

Render returns the module declarations. Includes are relative to the crate root through CARGO_MANIFEST_DIR, so the output does not depend on the checkout path. Output is sorted and so independent of the order of Add.

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.