protobuild

protobuild

A powerful Protocol Buffers build and management tool

中文文档

Features

• 🚀 Unified Build - One command to compile all proto files
• 📦 Multi-Source Dependencies - Support Go modules, Git, HTTP, S3, GCS, and local paths
• 🔌 Plugin Support - Flexible protoc plugin configuration
• 🔍 Linting - Built-in proto file linting with AIP rules
• 📝 Formatting - Auto-format proto files
• ⚙️ Configuration-driven - YAML-based project configuration
• 📊 Progress Display - Visual progress bars and detailed error messages
• 🗑️ Cache Management - Clean and manage dependency cache
• 🌐 Web UI - Visual configuration editor with proto file browser
• 🏥 Environment Check - Doctor command to diagnose development environment
• 🎯 Project Initialization - Quick project setup with templates

Installation

go install github.com/pubgo/protobuild@latest

Quick Start

1. Create a protobuf.yaml configuration file in your project root:

vendor: .proto
root:
  - proto
includes:
  - proto
deps:
  - name: google/protobuf
    url: github.com/protocolbuffers/protobuf
    path: src/google/protobuf
plugins:
  - name: go
    out: pkg
    opt:
      - paths=source_relative

1. Vendor dependencies:

protobuild vendor

1. Generate code:

protobuild gen

Commands

Configuration

Configuration File Structure

# Checksum for tracking changes (auto-generated)
checksum: ""

# Vendor directory for proto dependencies
vendor: .proto

# Base plugin configuration (applied to all plugins)
base:
  out: pkg
  paths: source_relative
  module: github.com/your/module

# Proto source directories
root:
  - proto
  - api

# Include paths for protoc
includes:
  - proto
  - .proto

# Exclude paths from compilation
excludes:
  - proto/internal

# Proto dependencies
deps:
  - name: google/protobuf
    url: github.com/protocolbuffers/protobuf
    path: src/google/protobuf
    version: v21.0
    optional: false

# Protoc plugins configuration
plugins:
  - name: go
    out: pkg
    opt:
      - paths=source_relative
  - name: go-grpc
    out: pkg
    opt:
      - paths=source_relative

# Plugin installers (go install)
installers:
  - google.golang.org/protobuf/cmd/protoc-gen-go@latest
  - google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

# Linter configuration
linter:
  rules:
    enabled_rules:
      - core::0131::http-method
    disabled_rules:
      - all
  format_type: yaml

Plugin Configuration

Each plugin supports the following options:

Dependency Configuration

Supported Dependency Sources

deps:
  # Go Module (default)
  - name: google/protobuf
    url: github.com/protocolbuffers/protobuf
    path: src/google/protobuf

  # Git Repository
  - name: googleapis
    url: https://github.com/googleapis/googleapis.git
    ref: master

  # HTTP Archive
  - name: envoy
    url: https://github.com/envoyproxy/envoy/archive/v1.28.0.tar.gz
    path: api

  # Local Path
  - name: local-protos
    url: ./third_party/protos

  # S3 Bucket
  - name: internal-protos
    url: s3://my-bucket/protos.tar.gz

  # GCS Bucket
  - name: shared-protos
    url: gs://my-bucket/protos.tar.gz

Usage Examples

Custom Config File

protobuild -c protobuf.custom.yaml gen

Lint Proto Files

protobuild lint
protobuild lint --list-rules  # Show available rules
protobuild lint --debug       # Debug mode

Format Proto Files

# Format and preview changes (dry run)
protobuild format

# Format and write changes to files
protobuild format -w

# Show diff of formatting changes
protobuild format --diff

# Exit with error if files need formatting (useful for CI)
protobuild format --exit-code

# Use builtin formatter instead of buf
protobuild format --builtin

# Format specific directories
protobuild format -w proto/ api/

Web Configuration UI

# Start web UI on default port (8080)
protobuild web

# Start web UI on custom port
protobuild web --port 9090

The web interface provides:

• 📝 Visual configuration editor
• 📦 Dependency management
• 🔌 Plugin configuration
• 🚀 One-click build, lint, format operations
• 📄 Real-time YAML preview
• 📊 Project statistics dashboard
• 🔍 Proto file browser with syntax highlighting
• 📚 Configuration examples reference

Initialize New Project

# Interactive initialization
protobuild init

# Use specific template
protobuild init --template basic    # Basic Go + gRPC project
protobuild init --template grpc     # Full gRPC-Gateway project
protobuild init --template minimal  # Minimal configuration

# Specify output directory
protobuild init -o ./my-project

Check Development Environment

# Diagnose environment issues
protobuild doctor

# Auto-install missing Go plugins
protobuild doctor --fix

Example output:

🏥 Protobuild Doctor

  Checking development environment...

  ✅ protoc                 installed (v25.1)
  ✅ protoc-gen-go          installed
  ✅ protoc-gen-go-grpc     installed
  ✅ buf                    installed (v1.28.1)
  ✅ api-linter             installed
  ✅ go                     installed (go1.21.5)
  ✅ Configuration          protobuf.yaml found
  ⚠️  Vendor directory       not found (run 'protobuild vendor')

  ✅ Environment check passed!

Force Vendor Update

protobuild vendor -f      # Force update even if no changes detected
protobuild vendor -u      # Re-download all dependencies (ignore cache)

Show Dependency Status

protobuild deps

Example output:

📦 Dependencies:

  NAME                                SOURCE     VERSION      STATUS
  ----                                ------     -------      ------
  google/protobuf                     Go Module  v21.0        🟢 cached
  googleapis                          Git        master       ⚪ not cached

  Total: 2 dependencies

Clean Dependency Cache

protobuild clean           # Clean all cached dependencies
protobuild clean --dry-run # Preview what will be cleaned

Install Plugins

protobuild install
protobuild install -f  # Force reinstall

Directory-level Configuration

You can place a protobuf.plugin.yaml file in any proto directory to override the root configuration for that directory and its subdirectories.

# proto/api/protobuf.plugin.yaml
plugins:
  - name: go
    out: pkg/api
    opt:
      - paths=source_relative

Supported Protoc Plugins

• google.golang.org/protobuf/cmd/protoc-gen-go@latest
• google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest
• github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-grpc-gateway@latest
• github.com/grpc-ecosystem/grpc-gateway/v2/protoc-gen-openapiv2@latest
• github.com/pseudomuto/protoc-gen-doc/cmd/protoc-gen-doc@latest
• github.com/bufbuild/protoc-gen-validate/cmd/protoc-gen-validate@latest
• And many more...

Error Handling

When dependency resolution fails, protobuild provides detailed error messages with suggestions:

❌ Failed to download dependency: google/protobuf
   Source:  Git
   URL:     git::https://github.com/protocolbuffers/protobuf.git?ref=v99.0
   Ref:     v99.0
   Error:   reference not found

💡 Suggestions:
   • Check if the repository URL is correct and accessible
   • Verify the ref (tag/branch/commit) exists
   • Ensure you have proper authentication (SSH key or token)

Cache Location

Dependencies are cached in:

• macOS/Linux: ~/.cache/protobuild/deps/
• Go modules: Standard Go module cache ($GOPATH/pkg/mod)

Architecture

protobuild
├── cmd/
│   ├── protobuild/          # Main CLI application
│   │   ├── cmd.go           # Entry point and core handlers
│   │   ├── commands.go      # Command factory functions
│   │   ├── config.go        # Configuration structs
│   │   ├── proto_walker.go  # Proto file walking utilities
│   │   ├── protoc_builder.go# Protoc command builder
│   │   ├── vendor_service.go# Dependency vendoring
│   │   ├── util.go          # Shared utilities
│   │   └── yaml_types.go    # YAML type definitions
│   ├── format/              # Proto file formatting (builtin)
│   ├── formatcmd/           # Format command (buf integration)
│   ├── linters/             # AIP linting rules
│   └── webcmd/              # Web configuration UI
│       ├── cmd.go           # Web command entry
│       ├── server.go        # HTTP server and API
│       └── templates/       # HTML templates (Alpine.js + Tailwind)
└── internal/
    ├── depresolver/         # Multi-source dependency resolver
    ├── modutil/             # Go module utilities
    ├── plugin/              # Plugin management
    ├── protoutil/           # Protobuf utilities
    ├── shutil/              # Shell utilities
    └── typex/               # Type extensions

Documentation

• Configuration Examples - Detailed configuration examples for various use cases
• Multi-Source Dependencies - Design document for multi-source dependency resolution
• Design Document - Architecture and design documentation

Roadmap

Upcoming features planned for future releases:

License

MIT License

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.