extension

NCore Extension System

A flexible and robust extension system that provides dynamic loading, lifecycle management, dependency handling, inter-service communication, and enterprise-grade security features.

Features

• Dynamic Loading: Load extensions from files or built-in registration
• Dependency Management: Strong and weak dependency support with automatic resolution
• Service Discovery: Consul-based service registration and discovery
• Event System: Unified event handling with memory and message queue support
• gRPC Integration: Optional gRPC service support for distributed communication
• Circuit Breaker: Built-in fault tolerance for service calls
• Hot Reload: Runtime plugin loading and unloading
• Cross-Service Calls: Unified local and remote service calling interface
• Security Sandbox: Plugin path validation, signature verification, trusted source validation
• Resource Monitoring: Memory and CPU usage limits, performance metrics collection
• Plugin Configuration: Personalized configuration management for each plugin

Basic Usage

Create Extension

package myext

import (
    "github.com/ncobase/ncore/extension/registry"
    "github.com/ncobase/ncore/extension/types"
)

type MyExtension struct {
    types.OptionalImpl
}

func init() {
    registry.RegisterToGroupWithWeakDeps(New(), "core", []string{"user"})
}

func (m *MyExtension) Name() string { return "my-extension" }
func (m *MyExtension) Version() string { return "1.0.0" }
func (m *MyExtension) Dependencies() []string { return []string{} }

func (m *MyExtension) Init(conf *config.Config, manager types.ManagerInterface) error {
    // Initialize extension
    return nil
}

func (m *MyExtension) GetMetadata() types.Metadata {
    return types.Metadata{
        Name: m.Name(), Version: m.Version(),
        Description: "My extension", Type: "module", Group: "core",
    }
}

Use Manager

func main() {
    mgr, err := manager.NewManager(config)
    if err != nil { panic(err) }
    defer mgr.Cleanup()

    if err := mgr.InitExtensions(); err != nil { panic(err) }
    
    ext, err := mgr.GetExtensionByName("my-extension")
    // Use extension...
}

Management Routes

Health endpoints are mounted by ManageRoutes without requiring host middleware. Inventory, metrics, plugin operations, and system management endpoints must be mounted through ManageRoutesWithGuard so the application owns authentication and authorization policy:

ncore := engine.Group("/ncore")
mgr.ManageRoutesWithGuard(
    ncore,
    AuthenticatedUser(),
    RequirePermission("manage:ncore"),
)

Applications that still compile against types.ManagerInterface can use a small local interface assertion and keep a guarded ManageRoutes fallback for older ncore versions.

Extension Lifecycle

Extensions follow a structured initialization process:

1. Registration - Auto-registered via init() or manually with manager.RegisterExtension()
2. Dependency Resolution - System calculates initialization order based on dependencies
3. Pre-initialization - PreInit() method for early setup
4. Initialization - Init(config, manager) method with full context
5. Post-initialization - PostInit() method for cross-extension communication
6. Runtime - Extension is active and serving requests
7. Cleanup - PreCleanup() and Cleanup() methods for resource cleanup

Dependency Management

Dependency Types

Strong Dependencies - Required and must be initialized first:

func (m *MyExtension) Dependencies() []string {
    return []string{"required-module"}
}

Weak Dependencies - Optional, graceful degradation when missing:

func (m *MyExtension) GetAllDependencies() []types.DependencyEntry {
    return []types.DependencyEntry{
        {Name: "required-module", Type: types.StrongDependency},
        {Name: "optional-module", Type: types.WeakDependency},
    }
}

Dependency Resolution

The system automatically:

• Detects and prevents circular dependencies
• Calculates optimal initialization order
• Handles missing weak dependencies gracefully
• Provides detailed error messages for resolution failures

Extensions with weak dependencies should handle missing services:

func (m *MyExtension) PostInit() error {
    if userService, err := m.manager.GetServiceByName("user"); err == nil {
        m.userService = userService
    } else {
        log.Warn("User service unavailable, limited functionality")
    }
    return nil
}

Service Communication

Service Calling Strategies

// Default local-first strategy
result, err := manager.CallService(ctx, "user-service", "GetUser", userID)

// Explicit strategy
result, err := manager.CallServiceWithOptions(ctx, "user-service", "GetUser", userID, 
    &types.CallOptions{
        Strategy: types.LocalFirst,  // LocalFirst, RemoteFirst, LocalOnly, RemoteOnly
        Timeout:  30 * time.Second,
    })

Strategy Behavior:

• LocalFirst: Try local service, fallback to gRPC
• RemoteFirst: Try gRPC service, fallback to local
• LocalOnly: Local service only, fail if unavailable
• RemoteOnly: gRPC service only, fail if unavailable

Cross-Service Access

// Direct service access
userService, err := manager.GetServiceByName("user")

// Cross-service field access
authService, err := manager.GetCrossService("auth", "TokenManager")

Event System

Event Targets

The event system supports multiple transport mechanisms:

• EventTargetMemory: In-memory, single instance, high performance
• EventTargetQueue: Message queue (RabbitMQ/Kafka), distributed, persistent
• EventTargetAll: Both targets simultaneously

Event Operations

// Subscribe (auto-selects best transport)
manager.SubscribeEvent("user.created", func(data any) {
    eventData := data.(types.EventData)
    // Handle event
})

// Subscribe to specific transport
manager.SubscribeEvent("user.created", handler, types.EventTargetMemory)

// Publish (auto-selects best transport)
manager.PublishEvent("user.created", userData)

// Publish with retry for critical events
manager.PublishEventWithRetry("payment.failed", paymentData, 3)

Event Data Structure

type EventData struct {
    Time      time.Time `json:"time"`
    Source    string    `json:"source"`
    EventType string    `json:"event_type"`
    Data      any       `json:"data"`
}

Security & Performance Features

Security Sandbox

The system provides comprehensive security controls:

// Plugin configuration management
config := map[string]any{
    "cache_ttl": "1h",
    "max_connections": 100,
}
manager.SetPluginConfig("my-plugin", config)

// Get plugin configuration
if cfg, exists := manager.GetPluginConfig("my-plugin"); exists {
    // Use configuration
}

Resource Monitoring

// Get resource usage metrics
metrics := manager.GetResourceMetrics()
for pluginName, metric := range metrics {
    fmt.Printf("Plugin %s: memory=%fMB, cpu=%f%%, load_time=%v\n", 
        pluginName, metric.MemoryUsageMB, metric.CPUUsagePercent, metric.LoadTime)
}

// Get security status
securityStatus := manager.GetSecurityStatus()
fmt.Printf("Security status: %+v\n", securityStatus)

Enhanced Metrics

// Get comprehensive metrics including security, performance, and system info
enhancedMetrics := manager.GetEnhancedMetrics()

Configuration

extension:
  path: "./plugins"          # Plugin directory
  mode: "file"              # "file" or "c2hlbgo" (built-in)
  includes: ["auth", "user"] # Include specific plugins
  excludes: ["debug"]       # Exclude plugins
  hot_reload: true          # Hot reload support
  
  # Advanced configuration
  max_plugins: 50           # Maximum number of plugins
  
  # Security configuration
  security:
    enable_sandbox: true    # Enable security sandbox
    allowed_paths:          # Allowed plugin paths
      - "/opt/plugins"
      - "/usr/local/plugins"
    blocked_extensions:     # Blocked file extensions
      - ".exe"
      - ".bat"
    trusted_sources:        # Trusted plugin sources
      - "company.com"
      - "verified.org"
    require_signature: true # Require plugin signature
  
  # Performance configuration
  performance:
    max_memory_mb: 512      # Maximum memory usage (MB)
    max_cpu_percent: 80     # Maximum CPU usage (%)
    enable_metrics: true    # Enable performance metrics
    metrics_interval: "30s" # Metrics collection interval
    enable_profiling: false # Enable performance profiling
    gc_interval: "5m"       # Garbage collection interval
  
  # Plugin-specific configuration
  plugin_config:
    auth_plugin:
      oauth_providers: ["google", "github"]
    user_plugin:
      cache_ttl: "1h"

consul:
  address: "localhost:8500"  # Consul server
  scheme: "http"
  discovery:
    health_check: true       # Enable health checks
    check_interval: "10s"    # Health check interval
    timeout: "3s"           # Health check timeout

grpc:
  enabled: true             # Enable gRPC support
  host: "localhost"
  port: 9090

Advanced Features

gRPC Integration

Extensions can provide gRPC services:

func (m *MyExtension) RegisterGRPCServices(server *grpc.Server) {
    pb.RegisterMyServiceServer(server, m.grpcService)
}

Service Discovery

Extensions can register with service discovery:

func (m *MyExtension) NeedServiceDiscovery() bool { return true }

func (m *MyExtension) GetServiceInfo() *types.ServiceInfo {
    return &types.ServiceInfo{
        Address: "localhost:8080",
        Tags:    []string{"api", "v1"},
        Meta:    map[string]string{"version": "1.0"},
    }
}

Circuit Breaker

Protect against service failures:

result, err := manager.ExecuteWithCircuitBreaker("external-service", func() (any, error) {
    return callExternalAPI()
})

Plugin Loading Modes

File Mode: Load plugins from filesystem

• Supports .so files on Linux, .dll on Windows
• Hot reload capability
• Include/exclude filtering
• Security sandbox protection

Built-in Mode: Use statically compiled extensions

• Better performance and security
• No filesystem dependencies
• Compile-time dependency resolution

Management API

REST endpoints for runtime management:

• GET /exts - List all extensions with metadata
• GET /exts/status - Get extension status and health
• POST /exts/load?name=plugin - Load specific plugin
• POST /exts/unload?name=plugin - Unload plugin
• POST /exts/reload?name=plugin - Reload plugin
• GET /exts/metrics - System metrics and performance data
• GET /exts/metrics/security - Security status metrics
• GET /exts/metrics/performance - Performance monitoring metrics

Performance Considerations

• Service Discovery: Use appropriate cache TTL (default 30s)
• Event Transport: Choose memory for high-frequency, queue for reliability
• Circuit Breaker: Monitor failure rates and adjust thresholds
• Plugin Loading: Prefer built-in mode for production environments
• Resource Monitoring: Enable performance metrics collection as needed
• Security Checks: Balance security requirements with performance needs

Troubleshooting

Common Issues

Circular Dependencies

Error: cyclic dependency detected in extensions: [module-a, module-b]

Solution: Convert one dependency to weak dependency type

Service Not Found

Error: extension 'user-service' not found

Solution: Check extension registration and initialization order

gRPC Connection Failed

Error: failed to get gRPC connection for service-name  

Solution: Verify service discovery configuration and network connectivity

Security Validation Failed

Error: security validation failed: path /tmp/plugin.so is not in allowed paths

Solution: Check allowed paths configuration in security settings

Resource Limit Exceeded

Error: resource limit check failed: insufficient memory: would exceed limit of 512MB

Solution: Adjust resource limits in performance configuration or optimize plugin memory usage

Plugin Signature Validation Failed

Error: signature validation failed: plugin signature not found

Solution: Ensure plugin file has corresponding .sig signature file or disable signature verification