gotal

README for GoTAL Project

Introduction

Welcome to GoTAL, a Go-based enterprise project. This project utilizes a robust architecture suitable for scalable and efficient web services.

Getting Started

Prerequisites

• Go 1.25 installed and configured (go env GOROOT reflects the 1.25 toolchain)
• A working protoc compiler (3.21+) and the Go plugins provided by make proto-deps
• MySQL 8.0 (or compatible) with network access for the application
• Redis 7 (or compatible)
• OpenSSL for creating local development certificates

Environment Setup

1. Install the protobuf toolchain (Ubuntu example):

   sudo apt update && sudo apt install -y protobuf-compiler
   

2. Install the Go-based protoc plugins:

   make proto-deps
   

   This installs protoc-gen-go and protoc-gen-go-grpc into $(go env GOBIN) (or $GOPATH/bin).
3. Generate all gRPC and protobuf bindings any time the .proto files change:

   make proto
   # or
   go generate ./internal/proto
   

   Override the PROTOC or PROTOC_INCLUDE environment variables if your protoc binary or include directory lives in a non-standard location.
4. Create development TLS certificates (already generated in cert/ for convenience, regenerate when needed):

   openssl req -x509 -nodes -newkey rsa:2048 \
     -keyout cert/apiserver-key.pem \
     -out cert/apiserver.pem \
     -days 365 -subj "/CN=gotal.local"
   

Framework Design

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│   Frontend    │       │   Frontend    │       │   Frontend    │     ...... (Different Roles Of Clients)
│   (Customer)  │       │   (Admin)     │       │   (Robot)     │
└───────┬───────┘       └───────┬───────┘       └───────┬───────┘
        │                       │                       │
        ▼                       ▼                       ▼
┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│    BFF        │       │    BFF        │       │    BFF        │
│  (Controller  │       │  (Controller  │       │  (Controller  │
│   + Service)  │       │   + Service)  │       │   + Service)  │
└───────┬───────┘       └───────┬───────┘       └───────┬───────┘
        │                       │                       │
        ▼                       ▼                       ▼
┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│  Store Layer  │       │  Store Layer  │       │  Store Layer  │
│ (gRPC Client) │       │ (gRPC Client) │       │ (gRPC Client) │
└───────┬───────┘       └───────┬───────┘       └───────┬───────┘
        │                       │                       │
        ▼                       ▼                       ▼
┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│ Backend gRPC  │       │ Backend gRPC  │       │ Backend gRPC  │
│   Services    │       │   Services    │       │   Services    │
│(Service +     │       │(Service +     │       │(Service +     │
│    Store)     │       │    Store)     │       │    Store)     │
└───────────────┘       └───────────────┘       └───────────────┘
        │                       │                       │
        └───────────────────────┴───────────────────────┘
                                │
                                ▼
                           ┌───────────┐
                           │  Database │
                           └───────────┘

Project Structure

├── CHANGELOG
├── CONTRIBUTING.md
├── LICENSE
├── Makefile
├── README.md
├── api
├── build
├── cert
│   ├── apiserver-key.pem
│   ├── apiserver.pem
│   ├── san.cnf
│   ├── server.crt
│   ├── server.csr
│   └── server.key
├── cmd
│   ├── apiserver
│   │   ├──  stdout
│   │   ├── apiserver
│   │   └── apiserver.go
│   ├── authzserver
│   │   ├── authzserver
│   │   └── authzserver.go
│   ├── redis_test
│   │   ├── main
│   │   ├── main.go
│   │   └── redis_test
│   ├── rpc_test
│   │   └── main.go
│   └── user_service
│       ├── user.go
│       └── user_service
├── configs
│   ├── apiserver-client.yaml
│   ├── authzserver.yaml
│   └── user-service.yaml
├── deployments
├── docs
├── githooks
├── go.mod
├── go.sum
├── image.png
├── init
├── scripts
│   ├── init.sh
│   ├── sql
│   │   ├── setup_mysql.sql
│   │   └── setup_mysql_password.sql
│   └── tests
│       ├── test_login.sh
│       └── test_user_registration_login.sh
├── internal
│   ├── apiserver
│   │   ├── app.go
│   │   ├── auth.go
│   │   ├── config
│   │   ├── controller
│   │   ├── grpc.go
│   │   ├── options
│   │   ├── router.go
│   │   ├── run.go
│   │   ├── server.go
│   │   ├── service
│   │   └── store
│   ├── authzserver
│   │   ├── app.go
│   │   ├── config
│   │   ├── options
│   │   ├── run.go
│   │   └── server.go
│   ├── pkg
│   │   ├── code
│   │   ├── errors
│   │   ├── logger
│   │   ├── middleware
│   │   ├── options
│   │   ├── response
│   │   ├── server
│   │   ├── util
│   │   └── validation
│   ├── proto
│   │   ├── options
│   │   └── user
│   └── user_service
│       ├── app.go
│       ├── config
│       ├── doc.go
│       ├── grpc.go
│       ├── options
│       ├── router.go
│       ├── run.go
│       ├── server.go
│       ├── service
│       └── store
├── logs
│   ├── apiserver.error.log
│   ├── apiserver.log
│   └── user_service
│       ├── user.error.log
│       └── user.log
├── pkg
│   ├── app
│   │   ├── app.go
│   │   ├── cmd.go
│   │   ├── config.go
│   │   ├── help.go
│   │   └── option.go
│   ├── cache
│   │   ├── cache.go
│   │   └── redis.go
│   ├── cli
│   ├── db
│   │   ├── mysql.go
│   │   └── plugin.go
│   ├── log
│   │   ├── context.go
│   │   ├── encoder.go
│   │   ├── log.go
│   │   ├── options.go
│   │   └── types.go
│   ├── shutdown
│   │   ├── managers
│   │   └── shutdown.go
│   ├── util
│   │   ├── common
│   │   ├── flag
│   │   └── term
│   └── validator
├── test
│   └── ratelimiter-test.sh
├── third_party
└── tools

61 directories, 64 files

Configuration Files

• configs/apiserver.yaml – runtime configuration for the HTTP gateway. Update the mysql, redis, and jwt sections before running locally.
• configs/user-service.yaml – configuration for the gRPC-based user microservice. Shares the same MySQL, Redis, and TLS settings.

Tip: Both binaries accept the --config (-c) flag. Example:

./cmd/apiserver/apiserver --config configs/apiserver.yaml
./cmd/user_service/user_service --config configs/user-service.yaml

Database Setup

1. Create the application database and user:

   CREATE DATABASE IF NOT EXISTS gotal CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
   CREATE USER IF NOT EXISTS 'gotal'@'%' IDENTIFIED BY 'change-me';
   GRANT ALL PRIVILEGES ON gotal.* TO 'gotal'@'%';
   FLUSH PRIVILEGES;
   

2. Apply the initial schema defined in internal/user_service/store/database/migrations/0001_create_users.sql:

   mysql -u gotal -p gotal < internal/user_service/store/database/migrations/0001_create_users.sql
   

   The script provisions the users table with all fields used by the gorm models.

Regenerating gRPC Stubs

• make proto rebuilds all files under internal/proto. The command enforces the installation of protoc-gen-go and protoc-gen-go-grpc.
• go generate ./internal/proto is wired via internal/proto/generate.go and provides an alternate entry point.
• Generated sources are checked in (*_pb.go) so the build works out of the box; regenerate after any .proto change.

Running the Services

1. Build binaries:

   cd cmd/apiserver && go build
   cd ../user_service && go build
   

2. Launch services with explicit config files:

   ./cmd/user_service/user_service --config ../../configs/user-service.yaml
   ./cmd/apiserver/apiserver --config ../../configs/apiserver.yaml
   

   Ensure MySQL, Redis, and the generated TLS certificates are reachable before starting.

Setup and Running Http REST Server

1. Building the API server: Navigate to the API server directory:

   cd /Users/huanghaitao/gotal/cmd/apiserver
   

   Build the API server:

   go build
   

2. Starting the Server: Run the API server with the specified configuration:

   ./apiserver --config ../../configs/apiserver.yaml
   

   This will initialize the server with various configurations as shown in your provided start-up log.

Full Configuration for the GoTAL API Server

After the section on starting the server, the following detailed configurations are applied:

RESTful Service Configuration

server:
  mode: debug # Modes: release, debug, test. Default is release.
  healthz: true # Enable health check, setting up /healthz route. Default is true.
  middlewares: recovery,logger,secure,nocache,cors,dump # List of gin middlewares.

gRPC Service Configuration

grpc:
  bind-address: 0.0.0.0 # IP address for gRPC. Default is 0.0.0.0.
  bind-port: 8082 # Port for gRPC. Default is 8081.

HTTP Configuration (Insecure)

insecure:
  bind-address: 0.0.0.0 # IP address for insecure binding. Default is 127.0.0.1.
  bind-port: 8884 # Non-secure port. Default is 8080.

HTTPS Configuration (Secure)

secure:
  bind-address: 0.0.0.0 # IP address for HTTPS. Default is 0.0.0.0.
  bind-port: 8445 # Port for HTTPS. Default is 8443.
  tls:
    cert-key:
      cert-file: /Users/huanghaitao/gotal/cert/apiserver.pem # Certificate file path.
      private-key-file: /Users/huanghaitao/gotal/cert/apiserver-key.pem # Private key file path.

MySQL Database Configuration

mysql:
  host: 127.0..0.1 # MySQL server address.
  username: root # MySQL username.
  password:  # MySQL password.
  database: db # Database name.
  max-idle-connections: 100 # Max idle connections.
  max-open-connections: 100 # Max open connections.
  max-connection-life-time: 10s # Connection lifetime.
  log-level: 4 # Log level.

Redis Configuration

redis:
  host: 127.0.0.1 # Redis host.
  port: 6379 # Redis port.
  password:  # Redis password.
  # Additional configuration details can be specified here.

JWT Configuration

jwt:
  realm: JWT # JWT realm identifier.
  key:  # Secret key.
  timeout: 24h # Token expiration time.
  max-refresh: 24h # Token refresh time.

Feature Configuration

feature:
  enable-metrics: true # Enable metrics at /metrics.
  profiling: true # Enable performance analysis at /debug/pprof/.

Rate Limiting Configuration

ratelimit:
  requests-per-second: 1 # Requests per second per user.
  burst-size: 20 # Maximum burst size.
  custom-limits:
    "/test-response":
      requests-per-second: 1.5 # Requests per second for specific endpoint.
      burst-size: 10 # Burst size for specific endpoint.

Logging Configuration

log:
  name: apiserver # Logger name.
  development: true # Development mode.
  level: debug # Log level.
  format: console # Log format.
  enable-color: true # Color output.
  disable-caller: false # Caller information.
  disable-stacktrace: false # Stack trace.
  output-paths: /Users/huanghaitao/gotal/logs/apiserver.log # Output paths.
  error-output-paths: /Users/huanghaitao/gotal/logs/apiserver.error.log # Error log paths.

This detailed configuration will ensure that your GoTAL API server is set up with the specific settings required for its operation. These settings include server modes, service bindings, database connections, logging, and more, ensuring a comprehensive and robust setup for your enterprise-grade application.

Configuration

The server's behavior is controlled by various flags and options. These include but are not limited to:

• gRPC Configurations: Address, port, message size limits.
• MySQL and Redis Configurations: Host, port, authentication, pooling settings.
• JWT Settings: Key, timeout, refresh settings.
• Server Mode: Including debug, test, and release modes.

Detailed flag descriptions are available in the start-up log section of this README.

Architecture Overview

The project is structured into several layers to promote separation of concerns and maintainability:

1. Controller Layer: Handles HTTP requests, invoking the appropriate services.
2. Service Layer: Contains business logic and interacts with the repository layer.
3. Repository Layer: Responsible for data access and storage management.

Key Components

• Cobra & Viper: Used for building CLI and managing configuration files.
• Validator: Ensures that incoming requests meet the defined constraints.
• Middleware: Includes authentication, logging, CORS, rate limiting, etc.
• Logging: Structured logging for tracing and monitoring.
• MySQL & Redis: Used for data storage and caching.

Contribution

Refer to CONTRIBUTING.md for guidelines on how to contribute to this project.

License

This project is licensed under the terms mentioned in LICENSE.

Changelog

For a detailed changelog, see CHANGELOG.

This README provides a basic overview of GoTAL. For detailed documentation, refer to the docs directory.