gendocs

AIStore API Documentation Generator

This tool generates an OpenAPI 3.0 specification from Go source code annotations. Struct schemas are extracted via go/ast.

How It Works

The generator scans Go source files in ais/ for +gen:endpoint annotations, resolves referenced Go packages from the import declarations in those files, parses referenced Go structs into OpenAPI schemas using go/ast, and writes a single openapi.yaml file.

Annotation Syntax

Basic Endpoint Annotation

// +gen:endpoint method /path/to/endpoint
func HandlerFunction() {
    // handler implementation
}

Endpoint with Parameters

// +gen:endpoint method /path/to/endpoint [param1=type,param2=type]
func HandlerFunction() {
    // handler implementation
}

Annotation Components

1. Method

The HTTP method for the endpoint:

• GET - Retrieve data
• POST - Create new resource
• PUT - Update existing resource
• DELETE - Remove resource
• PATCH - Partial update

2. Path

The URL path for the endpoint. Can include:

• Static segments: /buckets/list
• Path parameters: /buckets/{bucket-name}/objects/{object-name}
• Version prefixes: /v1/clusters

3. Parameters (Optional)

Parameters are specified in square brackets [param1=type,param2=type]. Each parameter has:

• Name: The parameter name (should match definitions in ../api/apc/query.go)
• Type: The parameter type (string, int, bool, etc.)

4. Actions vs Models: When to use which

There are two ways to annotate endpoints based on their payload structure:

Action-Based Endpoints: action=[...]

Use when an endpoint supports multiple operations that share the same URL but perform different actions. Each action expects a JSON payload with an action field wrapper.

{
  "action": "selected action",
  "value": "the model",
  "name": "specify if needed for the selected action"
}

Characteristics:

• Multiple actions share the same endpoint
• Each action has its own model for the value field
• JSON payload structure: {"action": "action-name", "value": model-data}
• All actions must have a corresponding model
• Requires +gen:payload annotations for each action

Model-Based Endpoints: model=[...]

Use when an endpoint accepts a single, direct model as the JSON payload without action wrapper.

5. Payload Annotations (Required)

You must provide a +gen:payload annotation for every action or model specified in your endpoint. This defines the JSON payload structure that will appear in the generated curl examples.

For Action-Based Endpoints:

// +gen:endpoint POST /v1/buckets/bucket-name action=[apc.ActCopyBck=apc.TCBMsg|apc.ActETLBck=apc.TCBMsg]
// +gen:payload apc.ActCopyBck={"action": "copy-bck", "value": {"dry_run": false}}
// +gen:payload apc.ActETLBck={"action": "etl-bck", "value": {"id": "ETL_NAME"}}

For Model-Based Endpoints:

// +gen:endpoint GET /v1/ml/moss/{bucket} model=[apc.MossReq]
// +gen:payload apc.MossReq={"in":[{"objname":"file.tar"}],"mime":"tar","coer":true}

Steps:

1. Identify each action or model in your endpoint annotation
2. Write the JSON payload following the appropriate structure (action-wrapped or direct model)
3. Add the annotation using: // +gen:payload

Payload Annotation Placement

For better code maintainability, place payload annotations above the subfunction that implements the action (if it exists):

// +gen:endpoint PUT /v1/cluster action=[apc.ActSetConfig=cmn.ConfigToSet|apc.ActXactStart=apc.ActMsg]
// Administrative cluster operations
func (p *proxy) httpcluput(w http.ResponseWriter, r *http.Request) {
    // Main endpoint handler - clean, no payload clutter
}

// +gen:payload apc.ActSetConfig={"action": "set-config", "value": {"timeout": {"send_file_time": "10m"}}}
func (p *proxy) setCluCfgPersistent(w http.ResponseWriter, r *http.Request, toUpdate *cmn.ConfigToSet, msg *apc.ActMsg) {
    // Subfunction implementation - payload annotation right where the action is implemented
}

// +gen:payload apc.ActXactStart={"action": "start-xaction", "name": "rebalance"}
func (p *proxy) xstart(w http.ResponseWriter, r *http.Request, msg *apc.ActMsg) {
    // Another subfunction with its payload annotation
}

Fallback: If an action doesn't have a dedicated subfunction, place the payload annotation above the main endpoint handler.

These annotations automatically generate the HTTP command examples in the documentation with proper JSON payloads.

For S3-Compatible Endpoints: payload=reference

S3-compatible endpoints require a different approach because they expect AWS S3 XML format, not AIStore JSON action messages. Use the payload= parameter to directly reference XML payload examples.

// +gen:endpoint DELETE /s3/{bucket-name} [s3.QparamMultiDelete=string] payload=s3-delete-multiple
// +gen:payload s3-delete-multiple=<?xml version="1.0" encoding="UTF-8"?><Delete><Object><Key>file1.txt</Key></Object><Object><Key>file2.txt</Key></Object></Delete>
// Delete a list of objects from an S3 bucket
func (p *proxy) delMultipleObjs(w http.ResponseWriter, r *http.Request, bucket string) {
    // S3-compatible implementation
}

// +gen:endpoint PUT /s3/{bucket-name} [s3.QparamVersioning=string] payload=s3-versioning
// +gen:payload s3-versioning=<VersioningConfiguration><Status>Enabled</Status></VersioningConfiguration>
// Configure S3 bucket versioning settings  
func (p *proxy) putBckVersioningS3(w http.ResponseWriter, r *http.Request, bucket string) {
    // S3-compatible implementation
}

Key Differences:

• Content-Type: Automatically generates Content-Type: application/xml headers (not JSON)
• Payload Format: Uses raw XML (not JSON action wrappers)
• Direct Reference: payload=key directly maps to +gen:payload key=xml-content
• Usage: Only for endpoints that expect XML request bodies (e.g., S3 delete multiple objects, versioning configuration)

Automatic Payload Generation

The system automatically generates simple payloads for actions that only require the action field, reducing manual annotation overhead.

Auto-Generated Payloads (Simple Actions)

Actions that only need {"action": "action-name"} are automatically generated and don't require manual +gen:payload annotations:

// +gen:endpoint PUT /v1/cluster action=[apc.ActResetConfig=apc.ActMsg|apc.ActRotateLogs=apc.ActMsg]
// No manual payload needed - these are auto-generated:
// ✓ apc.ActResetConfig → {"action": "reset-config"}  
// ✓ apc.ActRotateLogs → {"action": "rotate-logs"}

Decision Logic

• Auto-generated: Actions that map to apc.ActMsg and only need the action name
• Manual required: Actions with:
  • Complex value objects (configurations, node options, etc.)
  • Additional fields like name (xaction names, provider names, etc.)
  • Custom model structures (non-action based endpoints)

The auto-generation uses gap-filling logic: it only generates payloads for actions that don't already have manual annotations, ensuring developers maintain full control when needed.

Action Constant Mapping

The system automatically maps action constants to their string values:

This mapping ensures consistency between the +gen:endpoint action definitions and the +gen:payload labels.

Data Model Schemas

Structs referenced in action=[...] or model=[...] annotations are automatically parsed into OpenAPI schemas. The generator reads struct fields, JSON tags, and types directly.

All fields are emitted as required unless they carry a // +gen:optional trailing comment.

Parameter Definitions

Parameters must be defined in ../api/apc/query.go with descriptions:

var QueryParameters = map[string]ParameterDefinition{
    "apc.QparamProvider": {
        Name:        "provider",
        Type:        "string",
        Description: "Cloud provider name (e.g., aws, gcp, azure)",
    },
    // ... more parameters
}

Comment Documentation

Add descriptive comments after the annotation to provide endpoint summaries:

// +gen:endpoint GET /v1/buckets [provider=string,namespace=string]
// Lists all buckets for the specified provider and namespace.
// Returns bucket metadata including creation time and storage class.
func ListBuckets() {
    // implementation
}

Generated Tags

The system automatically generates API tags (groupings) based on the endpoint path:

• /v1/buckets/... → Buckets tag
• /v1/objects/... → Objects tag
• /v1/health/... → Health tag
• /v1/etl/... → Etl tag
• /v1/daemon/... → Daemon tag

Operation IDs

Operation IDs are automatically generated from function names:

• Simple case: ListBuckets → ListBuckets
• Multiple endpoints per function: dsortHandler + POST + /sort → dsortHandlerpostsort
• Different methods on same path: dsortHandler + GET + /sort → dsortHandlergetsort

Example 1: GET Endpoint with Query Parameters

// +gen:endpoint GET /v1/clusters/{cluster-id}/buckets [provider=string,namespace=string]
// Retrieves all buckets in the specified cluster.
// Supports filtering by cloud provider and namespace.
// Returns detailed bucket information including size and object count.
func GetClusterBuckets(w http.ResponseWriter, r *http.Request) {
    // Extract cluster ID from path
    clusterID := mux.Vars(r)["cluster-id"]

    // Get query parameters
    provider := r.URL.Query().Get("provider")
    namespace := r.URL.Query().Get("namespace")

    // Implementation...
}

This generates:

• Method: GET
• Path: /v1/clusters/{cluster-id}/buckets
• Parameters: provider (string), namespace (string)
• Tag: Clusters
• Operation ID: GetClusterBuckets
• Summary: "Retrieves all buckets in the specified cluster. Supports filtering by cloud provider and namespace. Returns detailed bucket information including size and object count."

Example 2: POST Endpoint with Action Parameters and Data Models

type PromoteArgs struct {
    DaemonID  string `json:"tid,omitempty"` // target ID
    SrcFQN    string `json:"src,omitempty"` // source file path
    ObjName   string `json:"obj,omitempty"` // destination object name
    Recursive bool   `json:"rcr,omitempty"` // recursively promote nested dirs
}

// +gen:endpoint POST /v1/objects/{bucket-name}/{object-name}[apc.QparamProvider=string] action=[apc.ActPromote=apc.PromoteArgs]
func PromoteObjects(w http.ResponseWriter, r *http.Request) {
    // Implementation...
}

The generator automatically parses apc.PromoteArgs and produces an OpenAPI schema with its fields and types.

Custom Type Handling

Custom types that wrap primitives (e.g., cos.Duration, cos.SizeIEC) are mapped to their OpenAPI types in externalTypeToSchema in schema.go, matching the JSON wire format of each type.

Running the Generator

Locally

# Generate OpenAPI spec and website markdown
make api-docs-website

Via GitHub Actions

The documentation is automatically generated and deployed when changes are pushed to the main branch via the deploy-website.yml workflow.

Output Files

• .docs/openapi.yaml - OpenAPI 3.0 specification (generated then removed by make api-docs-website)
• docs/http-api.md - Final documentation for the Jekyll website