schemas

Contributing

Please do! We're a warm and welcoming community of open source contributors. All types of contributions are welcome. Please read:

• General Contributor Guide - Overview of contribution processes
• Schema Contributor Guide - Schema-specific development workflows and guidelines

🧬 Schema-Driven Development Guide

Meshery follows a Schema-Driven Development (SDD) approach. This means that the structure of data used throughout the system is centrally defined using schemas. These schemas power consistency, validation, and code generation across the Meshery platform.

---

🧾 Schema Definition in Meshery

Meshery uses the OpenAPI v3 specification to define and manage schemas. Given the complexity of the platform, Meshery adopts a modular, versioned, and extensible schema strategy:

• ✅ Versioned schemas for backward compatibility.
• 🧩 Modular constructs for maintainability and reuse.
• 🧪 Schemas are used for validation, API documentation, and automatic code generation.

💡 TIP: When referencing models or other constructs in the schema, always add x-go-type and x-go-import-path to avoid generating redundant Go structs. Refer to existing patterns in the codebase.

---

📁 Schema Directory Structure

All schemas are located in the schemas/ directory at the root of the Meshery repository:

schemas/
  constructs/
    <schema-version>/               # e.g., v1beta1
      <construct>/                  # e.g., model, component
        api.yml                     # Index file: references all subschemas + defines API endpoints
        <construct>.yaml            # Subschema: data model definition for the construct
        <other_subschema>.yaml      # Additional subschemas (optional)
        templates/                  # Manually defined templates directory
          <construct>_template.json # JSON template from schema
          <construct>_template.yaml # YAML template from schema
          <variant>_template.json   # Additional variant templates (optional)
  
  typescript/                       # TypeScript source and generated files
    index.ts                        # Manually maintained - public API surface
    generated/                      # Auto-generated (do NOT commit)
      <schema-version>/
        <construct>/
          <Construct>.d.ts          # TypeScript type definitions
          <Construct>Schema.ts      # OpenAPI schema as JS object
    rtk/                            # RTK Query client configurations
      cloud.ts
      meshery.ts
  
  dist/                             # Built distribution (do NOT commit)
    index.js, index.d.ts
    cloudApi.js, mesheryApi.js
    constructs/                     # Built schema exports (renamed from 'generated')
      <schema-version>/<construct>/<Construct>Schema.js
  
  models/                           # Auto-generated Go code (do NOT commit)
    <schema-version>/
      <construct>/
        <construct>.go

🧠 Explanation

• constructs/ – Holds schemas for various versions.
  • <schema-version>/ – Represents a version (e.g., v1alpha2, v1beta1).
    • <construct>/ – A directory to contain all files for any given construct like pattern, component, etc.
      • api.yml – The index file for the construct. This file:
        1. References all subschemas (via $ref) to bundle them together
        2. Defines all API endpoints (REST operations: GET, POST, PUT, DELETE) for the construct
        3. Serves as the entry point for code generation tools (oapi-codegen, openapi-typescript)
      • <construct>.yaml – A subschema that defines the data model (noun) for the construct. Contains the schema properties, types, and validation rules.
      • Other .yaml files – Additional subschemas can be defined in separate files (e.g., model_core.yml, component_metadata.yml) and referenced from api.yml.
      • templates/ – A subdirectory containing manually defined template files. You can add as many different templates here for different variants, use cases, or configurations. Templates are example instances of the schema with default or sample values.
        • <construct>_template.json / <construct>_template.yaml – Default templates in JSON/YAML format.
        • Additional variant templates can be added (e.g., <construct>_minimal_template.json, <construct>_full_template.yaml) for different use cases.

---

Schema Design Principles: The Dual-Schema Pattern

Every persisted entity in Meshery follows a strict two-schema contract. Violating this contract causes generated Go structs and API clients to be incorrect.

Rule 1 — Entity schema = response schema only

The <construct>.yaml file represents the full server-side object as returned in API responses. It must:

• Include all server-generated fields: id, created_at, updated_at, deleted_at
• List server-generated required fields in required (they are always present in responses)
• Have additionalProperties: false at the top level

# keychain.yaml — response schema ✅
type: object
additionalProperties: false
required:
  - id
  - name
  - owner
  - created_at
  - updated_at
properties:
  id:
    $ref: ../../v1alpha1/core/api.yml#/components/schemas/uuid
  name:
    type: string
  owner:
    $ref: ../../v1alpha1/core/api.yml#/components/schemas/uuid
  created_at:
    $ref: ../../v1alpha1/core/api.yml#/components/schemas/created_at
  updated_at:
    $ref: ../../v1alpha1/core/api.yml#/components/schemas/updated_at
  deleted_at:
    $ref: ../../v1alpha1/core/api.yml#/components/schemas/nullTime

Rule 2 — Write operations use a separate *Payload schema

Every entity that supports POST or PUT must define a dedicated {Entity}Payload schema in api.yml. The payload schema:

• Contains only client-settable fields (no created_at, updated_at, deleted_at)
• Makes id optional with omitempty for upsert patterns, or omits it entirely for create-only
• Is referenced by requestBody in POST/PUT operations
• Is never reused as a response body

# In api.yml — write schema ✅
components:
  schemas:
    KeychainPayload:
      type: object
      description: Payload for creating or updating a keychain.
      required:
        - name
      properties:
        id:
          $ref: ../../v1alpha1/core/api.yml#/components/schemas/uuid
          description: Existing keychain ID for updates; omit on create.
          x-oapi-codegen-extra-tags:
            json: "id,omitempty"
        name:
          type: string
          description: Name of the keychain.
        owner:
          $ref: ../../v1alpha1/core/api.yml#/components/schemas/uuid
          description: Owner UUID; set server-side from auth context if omitted.
          x-oapi-codegen-extra-tags:
            json: "owner,omitempty"

paths:
  /api/auth/keychains:
    post:
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/KeychainPayload"   # ← Payload, not Keychain
      responses:
        "200":
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/Keychain"        # ← Full entity in response

Rule 3 — Never use the entity schema as a POST/PUT request body

Using the full entity schema as a requestBody forces clients to supply server-generated fields (id, created_at, updated_at) and produces incorrect generated client code.

# ❌ Wrong — exposes server-generated required fields to clients
post:
  requestBody:
    content:
      application/json:
        schema:
          $ref: "#/components/schemas/Keychain"

# ✅ Correct — separate payload type for writes
post:
  requestBody:
    content:
      application/json:
        schema:
          $ref: "#/components/schemas/KeychainPayload"

Checklist when adding a new entity

• <construct>.yaml has additionalProperties: false
• <construct>.yaml lists all server-generated fields in properties and required
• api.yml defines a {Construct}Payload schema with only client-settable fields
• All POST/PUT requestBody entries reference {Construct}Payload, not {Construct}
• GET responses reference the full {Construct} entity schema

---

Naming Conventions

• Property names

  • Use camelCase for property fields (e.g., schemaVersion, displayName, componentsCount).
  • Identifier fields use lowerCamelCase with "Id" suffix (e.g., modelId, registrantId, categoryId).
  • Enums use lowercase words (e.g., enabled, ignored, duplicate).
  • Use singular nouns for object names (e.g., model, component, design).

• OpenAPI schema names

  • PascalCase nouns under components/schemas (e.g., Model, Component).
  • Files/folders are lowercase: api.yml (index), <construct>.yaml (subschemas), templates/<construct>_template.(json|yaml).

• Endpoints and operations

  • Paths are under /api with kebab-case , plural nouns (e.g., /api/workspaces, /api/environments).
  • Path params are camelCase (e.g., {subscriptionId}, {connectionId}).
  • Non-CRUD actions append a verb segment (e.g., .../register, .../export, .../cancel); legacy lowerCamelCase may appear (e.g., .../upgradePreview).
  • operationId is camelCase VerbNoun (e.g., registerMeshmodels).

• Versioning

  • schemaVersion uses group/version (e.g., models.meshery.io/v1beta1, components.meshery.io/v1beta1).
  • Version strings follow k8s-style (v1, v1alpha1, v1beta1); semver fields use standard SemVer.

---

🔧 Go Helper Files

While Go structs are auto-generated from schemas, you often need to add custom methods to make these structs compatible with databases, implement interfaces, or add utility functions. This is done through manually created helper files.

When to Create Helper Files

Create a helper file (*_helper.go or helpers.go) in the generated package when you need:

1. SQL Driver Compatibility - Implement database/sql/driver.Scanner and driver.Valuer interfaces
2. Entity Interface Implementation - Implement the entity.Entity interface for database operations
3. GORM Table Names - Define custom table names via TableName() method
4. Utility Methods - Add helper functions for serialization, validation, or business logic
5. Type Conversions - Add methods to convert between related types

Helper File Location

models/
├── core/
│   ├── core.go                    # Auto-generated (do NOT edit)
│   ├── helpers.go                 # Manual: utility functions
│   ├── datatype_map.go            # Manual: Map type with SQL driver methods
│   └── datatype_null_time.go      # Manual: NullTime with SQL driver methods
├── v1beta1/
│   ├── model/
│   │   ├── model.go               # Auto-generated (do NOT edit)
│   │   └── model_helper.go        # Manual: Entity interface, TableName, etc.
│   ├── component/
│   │   ├── component.go           # Auto-generated (do NOT edit)
│   │   └── component_helper.go    # Manual: Entity interface, TableName, etc.
│   └── category/
│       ├── category.go            # Auto-generated (do NOT edit)
│       └── category_helper.go     # Manual: Entity interface, TableName, etc.

SQL Driver Interface Implementation

To store complex types in SQL databases, implement Scan and Value methods:

// helpers.go - This is NOT autogenerated
package mypackage

import (
    "database/sql/driver"
    "encoding/json"
    
    "github.com/meshery/schemas/models/core"
)

// Scan implements sql.Scanner interface for reading from database
func (m *MyComplexType) Scan(value interface{}) error {
    mapVal := core.Map{}
    err := mapVal.Scan(value)
    if err != nil {
        return err
    }
    return core.MapToStruct(mapVal, m)
}

// Value implements driver.Valuer interface for writing to database
func (m MyComplexType) Value() (driver.Value, error) {
    mapVal, err := core.StructToMap(m)
    if err != nil {
        return nil, err
    }
    return core.Map(mapVal).Value()
}

Entity Interface Implementation

For structs that need database CRUD operations, implement the entity.Entity interface:

// component_helper.go - This is NOT autogenerated
package component

import (
    "fmt"
    
    "github.com/gofrs/uuid"
    "github.com/meshery/meshkit/database"
    "github.com/meshery/meshkit/models/meshmodel/entity"
    "gorm.io/gorm/clause"
)

// TableName returns the database table name for GORM
func (c ComponentDefinition) TableName() string {
    return "component_definition_dbs"
}

// Type returns the entity type identifier
func (c ComponentDefinition) Type() entity.EntityType {
    return entity.ComponentDefinition
}

// GenerateID generates a new UUID for the entity
func (c *ComponentDefinition) GenerateID() (uuid.UUID, error) {
    return uuid.NewV4()
}

// GetID returns the entity's ID
func (c ComponentDefinition) GetID() uuid.UUID {
    return c.Id
}

// GetEntityDetail returns a human-readable description
func (c *ComponentDefinition) GetEntityDetail() string {
    return fmt.Sprintf("type: %s, name: %s, model: %s", 
        c.Type(), c.DisplayName, c.Model.Name)
}

// Create inserts the entity into the database
func (c *ComponentDefinition) Create(db *database.Handler, hostID uuid.UUID) (uuid.UUID, error) {
    c.Id, _ = c.GenerateID()
    err := db.Omit(clause.Associations).Create(&c).Error
    return c.Id, err
}

// UpdateStatus updates the entity's status in the database
func (c *ComponentDefinition) UpdateStatus(db *database.Handler, status entity.EntityStatus) error {
    return nil
}

Core Utility Types

The models/core/ package provides reusable types with built-in SQL compatibility:

Type	Purpose	Use Case
core.Map	map[string]any with SQL support	Storing JSON objects in database
core.NullTime	Nullable time with JSON/YAML support	Optional timestamp fields
core.Time	Time wrapper with custom formatting	Required timestamp fields

Example: Using Core Types

// In your helper file
package mypackage

import "github.com/meshery/schemas/models/core"

// For nullable timestamps (e.g., deleted_at)
type MyStruct struct {
    DeletedAt core.NullTime `json:"deleted_at" gorm:"column:deleted_at"`
}

// For JSON metadata stored as blob
type MyStruct struct {
    Metadata core.Map `json:"metadata" gorm:"type:bytes;serializer:json"`
}

Important Notes

1. File Header Comment: Always add // This is not autogenerated. at the top of helper files
2. Same Package: Helper files must be in the same package as the generated code
3. Do Commit Helper Files: Unlike generated .go files, helper files ARE committed to the repository
4. Mutex for Creation: Use sync.Mutex when implementing Create() to prevent race conditions

---

⚙️ Code Generation

Meshery supports automated code generation from schemas for:

• Go: Strongly-typed models for backend → models/<version>/<package>/
• TypeScript Types: Interfaces and type definitions → typescript/generated/<version>/<package>/<Package>.d.ts
• TypeScript Schemas: OpenAPI schemas as const JS objects → typescript/generated/<version>/<package>/<Package>Schema.ts
• RTK Query: Clients generated from OpenAPI for use with Redux → typescript/rtk/
• JSON/YAML: Templates with defaults and resolved references.

TypeScript Schema Exports

Each construct's OpenAPI schema is exported as a const JavaScript object for runtime use:

// Import from main index
import {
  ModelDefinitionV1Beta1OpenApiSchema,
  ComponentDefinitionV1Beta1OpenApiSchema,
  DesignDefinitionV1Beta1OpenApiSchema,
} from "@meshery/schemas";

// Or import individual schemas directly
import ModelSchema from "@meshery/schemas/dist/constructs/v1beta1/model/ModelSchema";
import ComponentSchema from "@meshery/schemas/dist/constructs/v1beta1/component/ComponentSchema";

TypeScript Type Namespaces

Types are organized by version in namespaces:

import { v1beta1, v1alpha1 } from "@meshery/schemas";

const component: v1beta1.Component = { /* ... */ };
const model: v1beta1.Model = { /* ... */ };
const design: v1beta1.Design = { /* ... */ };

---

🚀 Unified Build: One Command for Everything

Use the following command to perform the entire schema-driven generation workflow:

make build
npm run build  # Build TypeScript distribution with tsup

🔧 What make build does:

1. Bundles OpenAPI schemas for:

   • Meshery
   • Remote Providers (e.g. Meshery Cloud)
   • Combined (all constructs)

2. Generates:

   • Golang structs → models/
   • TypeScript type definitions (.d.ts) → typescript/generated/
   • TypeScript schema exports (*Schema.ts) → typescript/generated/
   • RTK Query clients → typescript/rtk/

3. After npm run build:

   • Builds distribution files → dist/
   • Creates CJS and ESM bundles
   • Generates declaration files

⚠️ This is the recommended way to stay in sync with schema changes.

---

🧱 Bundled Schema Outputs

After running make build, three bundled schema files are created:

File	Purpose
merged_schema.yml	All schemas combined (used by Meshery clients)
cloud_schema.yml	Cloud-specific APIs for Remote Providers (e.g. Meshery Cloud)
meshery_schema.yml	Meshery-specific APIs

---

✍️ Annotating OpenAPI Paths

To control which schema paths are included in each bundled output, use the x-internal annotation inside the OpenAPI operations (get, post, etc.).

Example:

paths:
  /api/entitlement/plans:
    get:
      x-internal: ["cloud"]
      operationId: getPlans
      tags:
        - Plans
      summary: Get all plans supported by the system
      responses:
        "200":
          description: Plans fetched successfully
          content:
            application/json:
              schema:
                type: array
                items:
                  $ref: "#/components/schemas/Plan"

• With x-internal: Included only in the respective client (e.g., cloud).
• Without x-internal: Included in all clients.

---

x-generate-db-helpers

Use x-generate-db-helpers: true as a schema-level annotation (not per-property) on a named component under components/schemas to instruct the Go generator to automatically produce SQL driver helper methods (Scan and Value) for that type.

When to use it: The annotated schema type must satisfy both conditions:

1. It has a dedicated OpenAPI schema definition (i.e., it is a named component with explicit properties, not a generic map).
2. It is persisted as a JSON blob in a single database column — not spread across a dedicated table with one column per field.

When NOT to use it: Do not annotate amorphous types that have no fixed schema definition (e.g., a generic metadata object). Those fields should use x-go-type: "core.Map" instead. Also do not annotate types that map to full database tables with individual columns for each property — those are handled by the normal DB tag generation.

Example

components:
  schemas:
    Quiz:
      x-generate-db-helpers: true   # ← schema-level; not on individual properties
      type: object
      required:
        - id
        - title
      properties:
        id:
          $ref: "../../v1alpha1/core/api.yml#/components/schemas/uuid"
        title:
          type: string

The generator produces zz_generated.helpers.go containing:

func (value *Quiz) Scan(src interface{}) error {
    if src == nil {
        *value = Quiz{}
        return nil
    }
    mapVal := core.Map{}
    if err := mapVal.Scan(src); err != nil {
        return err
    }
    return core.MapToStruct(mapVal, value)
}

func (value Quiz) Value() (driver.Value, error) {
    mapVal, err := core.StructToMap(value)
    if err != nil {
        return nil, err
    }
    return core.Map(mapVal).Value()
}

These implement Go's sql.Scanner and driver.Valuer interfaces so the struct is transparently serialized as JSON when reading from or writing to a database column.

Counter-example — metadata: A metadata field stored as JSON in the database is intentionally not annotated with x-generate-db-helpers because it is amorphous — it has no fixed property list. Use x-go-type: "core.Map" for those fields instead:

metadata:
  type: object
  additionalProperties: true
  x-go-type: "core.Map"
  x-go-type-skip-optional-pointer: true
  x-oapi-codegen-extra-tags:
    db: "metadata"

---

🛠️ Advanced Usage (Optional)

📌 Custom Generation in generate.sh

Meshery uses a helper script (generate.sh) to map schema constructs to generated output:

generate_schema_models <construct> <schema-version> [<openapi-file>]

generate_schema_models "capability" "v1alpha1"
generate_schema_models "category" "v1beta1"
generate_schema_models "pattern" "v1beta1" "schemas/constructs/v1beta1/design/api.yml"

This maps to Go packages like:

models/v1alpha1/capability/capability.go

🧩 RTK Query Client Generation

The OpenAPI bundle is passed to a codegen tool to generate RTK Query clients. Include relevant paths using x-internal annotations and define request/response schemas appropriately.

You can control this in generate.sh like:

# Merge relevant constructs for RTK generation
npx @redocly/cli join schemas/base_cloud.yml \
     "${v1beta1}/pattern/${merged_construct}" \
     "${v1beta1}/component/${merged_construct}" \
     "${v1beta1}/model/${merged_construct}" \
     ... \
     -o _openapi_build/merged_openapi.yml \
     --prefix-tags-with-info-prop title \
     --prefix-components-with-info-prop title

Using Generated RTK Query Clients

Prerequisites

Before using the generated RTK clients, ensure you have:

1. Installed the required dependencies:

   • @reduxjs/toolkit
   • @meshery/schemas

2. Set up environment variables:

   • RTK_CLOUD_ENDPOINT_PREFIX: Base URL for Cloud API endpoints
   • RTK_MESHERY_ENDPOINT_PREFIX: Base URL for Meshery API endpoints

Store Configuration

Import API Slices Correctly

To avoid cyclical imports that can break your application, import API slices from their specific exports:

// ✅ Correct: Import from specific API exports
import { cloudApi as cloudBaseApi } from "@meshery/schemas/dist/cloudApi";
import { mesheryApi } from "@meshery/schemas/dist/mesheryApi";

// ❌ Incorrect: Do not import directly from generic API file
// import { api } from "@meshery/schemas/dist/api"; // Can cause cyclical imports

Configure Redux Store

Add the API reducers and middleware to your Redux store configuration:

import { combineReducers, configureStore } from "@reduxjs/toolkit";
import { cloudApi as cloudBaseApi } from "@meshery/schemas/dist/cloudApi";
import catalogReducer from "./slices/catalog";
import connectionReducer from "./slices/connection";
import organizationReducer from "./slices/organization";
import chartReducer from "./slices/charts";
import themeReducer from "./slices/theme";
// Optional: If you have locally defined APIs
import { cloudApi } from "../api";

// Combine reducers
const rootReducer = combineReducers({
  catalog: catalogReducer,
  charts: chartReducer,
  organization: organizationReducer,
  connection: connectionReducer,
  theme: themeReducer,
  // Add generated API reducers
  [cloudBaseApi.reducerPath]: cloudBaseApi.reducer,
  // Optional: Add locally defined API reducers
  [cloudApi.reducerPath]: cloudApi.reducer
});

// Configure store with middleware
export const store = configureStore({
  reducer: reduxPersist.createPersistEnhancedReducer(rootReducer),
  middleware: getDefaultMiddleware =>
    getDefaultMiddleware()
      // Add generated API middleware
      .concat(cloudBaseApi.middleware)
      // Optional: Add locally defined API middleware
      .concat(cloudApi.middleware)
      // Add persistence middleware if needed
      .concat(reduxPersist.persistMiddleware)
});

// Set up listeners for RTK Query cache behaviors like refetchOnFocus/refetchOnReconnect
setupListeners(store.dispatch);

Using API Hooks

After configuring your store, you can import and use the generated hooks:

Cloud API Hooks

import {
  useGetPlansQuery,
  useCreateDesignMutation,
  useGetDesignsQuery,
  // Other cloud API hooks...
} from "@meshery/schemas/dist/cloudApi";

function MyComponent() {
  // Use hooks directly in your components
  const { data: plans, isLoading, error } = useGetPlansQuery();

  // Handle loading states
  if (isLoading) return <div>Loading plans...</div>;

  // Handle errors
  if (error) return <div>Error loading plans</div>;

  // Use data
  return (
    <div>
      {plans.map(plan => (
        <div key={plan.id}>{plan.name}</div>
      ))}
    </div>
  );
}

Meshery API Hooks

import {
  useGetMeshModelsQuery,
  useSubmitMeshConfigMutation,
  // Other Meshery API hooks...
} from "@meshery/schemas/dist/mesheryApi";

function MesheryComponent() {
  const { data: meshModels } = useGetMeshModelsQuery();
  // ...
}

Troubleshooting

Common Issues

1. Stuck Loading States:

   • Verify environment variables are correctly set
   • Check for CORS issues
   • Ensure proper authentication headers are included

2. Cyclical Imports:

   • Always import from specific API files (cloudApi.ts, mesheryApi.ts)
   • Avoid importing from generic api.ts files

3. Multiple RTK Instances:

   • Ensure proper reducer and middleware registration
   • Check for naming conflicts in reducerPaths

Redux DevTools

For better debugging, use Redux DevTools to monitor:

• API request lifecycles
• State changes
• Caching behavior

Best Practices

1. Handle Loading States:

   const { data, isLoading, isFetching, error } = useGetDataQuery();
   

2. Leverage Cache Options:

   const { data } = useGetDataQuery(null, {
     pollingInterval: 30000, // Re-fetch every 30 seconds
     refetchOnMountOrArgChange: true,
     skip: !isReady // Skip query when not ready
   });
   

3. Use Transformations When Needed:

   const transformedData = data?.map(item => ({
     ...item,
     formattedValue: formatValue(item.value)
   }));
   

🧪 Testing & Validating Schemas

Validate your schema updates before committing by running:

make build

Or validate a single file:

npx @redocly/cli lint schemas/constructs/v1beta1/pattern/api.yml

---

✅ Summary

Task	Command
Generate everything	make build
Build TypeScript dist	npm run build
Generate Go code only	make golang-generate
Generate TS types + schemas	make generate-ts
Lint OpenAPI	npx @redocly/cli lint

Importing Schemas

// Via namespaces (types)
import { v1beta1 } from "@meshery/schemas";
const model: v1beta1.Model = { /* ... */ };

// Via schema exports (runtime)
import { ModelDefinitionV1Beta1OpenApiSchema } from "@meshery/schemas";

// Direct schema import
import ModelSchema from "@meshery/schemas/dist/constructs/v1beta1/model/ModelSchema";

---

License

This repository and site are available as open-source under the terms of the Apache 2.0 License.

MESHERY IS A CLOUD NATIVE COMPUTING FOUNDATION PROJECT