workflow

package module
v0.0.0-...-044c556 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Feb 11, 2026 License: MIT Imports: 23 Imported by: 0

README

Workflow Engine

A configurable, AI-powered workflow orchestration engine built on CrisisTextLine/modular v1.11.11, featuring a visual builder UI, dynamic component hot-reload, and comprehensive observability.

Overview

This workflow engine lets you create applications by chaining together modular components based on YAML configuration files. You can configure the same codebase to operate as:

  • An API server with authentication middleware
  • An event processing system with state machine workflows
  • A message-driven pipeline with metrics and health monitoring
  • An AI-assisted workflow builder with visual drag-and-drop UI

All without changing code - just by modifying configuration files.

Features

Visual Workflow Builder
  • ReactFlow-based drag-and-drop UI
  • 30 module types across 10 categories
  • Real-time YAML import/export with round-trip fidelity
  • Undo/redo, validation, and property editing
AI-Powered Generation
  • Anthropic Claude direct API integration with tool use
  • GitHub Copilot SDK integration for development workflows
  • Automatic workflow generation from natural language descriptions
  • Component suggestion and validation
Dynamic Component Hot-Reload
  • Yaegi interpreter for runtime Go component loading
  • File watcher for automatic hot-reload
  • Sandboxed execution with stdlib-only imports
  • Component registry with lifecycle management
EventBus Integration
  • Native EventBus bridge for message broker compatibility
  • Workflow lifecycle events (started, completed, failed)
  • Event-driven triggers and subscriptions
Observability
  • Prometheus metrics collection
  • Health check endpoints (/health, /ready, /live)
  • Request ID propagation (X-Request-ID)

Requirements

  • Go 1.25 or later
  • Node.js 18+ (for UI development)

Module Types

The engine supports 30 module types across 10 categories:

Category Module Types
HTTP http.server, http.router, http.handler, http.proxy, api.handler, chimux.router
Middleware http.middleware.auth, http.middleware.logging, http.middleware.ratelimit, http.middleware.cors, http.middleware.requestid
Messaging messaging.broker, messaging.handler, messaging.broker.eventbus
State Machine statemachine.engine, state.tracker, state.connector
Events eventlogger.modular
Integration httpclient.modular, data.transformer, webhook.sender
Scheduling scheduler.modular
Infrastructure auth.modular, eventbus.modular, cache.modular, database.modular, jsonschema.modular
Database database.workflow
Observability metrics.collector, health.checker

Quick Start

Run the Server
go run ./cmd/server -config example/order-processing-pipeline.yaml
Development UI
cd ui && npm install && npm run dev
Configuration

Applications are defined via YAML configuration files:

modules:
  - name: http-server
    type: http.server
    config:
      address: ":8080"
  - name: router
    type: http.router
  - name: hello-handler
    type: http.handler

workflows:
  http:
    routes:
      - method: GET
        path: /hello
        handler: hello-handler

Example Applications

The example/ directory includes several configurations:

  • Order Processing Pipeline: 10-module workflow with HTTP, state machine, messaging, and observability
  • API Server: RESTful API with protected endpoints
  • State Machine Workflow: Order lifecycle with state transitions
  • Event Processor: Message-based event processing
  • Data Pipeline: Data transformation and webhook delivery

Testing

# Go tests
go test ./...

# UI component tests
cd ui && npm test

# E2E Playwright tests
cd ui && npx playwright test

Architecture

The engine is built on these core concepts:

  • Modules: Self-contained components providing specific functionality
  • Workflows: YAML configurations chaining modules together
  • Handlers: Components interpreting and configuring workflow types (HTTP, Messaging, State Machine, Scheduler, Integration)
  • Dynamic Components: Runtime-loaded Go modules via Yaegi interpreter
  • AI Generation: Natural language to workflow YAML via LLM APIs

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Engine 

type Engine interface {
	RegisterWorkflowHandler(handler WorkflowHandler)
	RegisterTrigger(trigger module.Trigger)
	AddModuleType(moduleType string, factory ModuleFactory)
	BuildFromConfig(cfg *config.WorkflowConfig) error
	Start(ctx context.Context) error
	Stop(ctx context.Context) error
	TriggerWorkflow(ctx context.Context, workflowType string, action string, data map[string]interface{}) error
}

type EngineBuilderFunc 

type EngineBuilderFunc func(cfg *config.WorkflowConfig, logger *slog.Logger) (*StdEngine, modular.Application, error)

EngineBuilderFunc is called by the manager to create and configure an engine from a parsed workflow config. The caller is responsible for registering workflow handlers, dynamic components, and other setup. The function must call BuildFromConfig on the engine before returning.

type ManagedEngine 

type ManagedEngine struct {
	WorkflowID uuid.UUID
	Engine     *StdEngine
	App        modular.Application
	Status     string // "running", "stopped", "error"
	StartedAt  time.Time
	Error      error
	// contains filtered or unexported fields
}

ManagedEngine holds a running workflow engine along with its metadata.

func (*ManagedEngine) GetEngine 

func (me *ManagedEngine) GetEngine() module.TriggerWorkflower

GetEngine returns the underlying engine, satisfying the module.triggerableEngine interface so the CrossWorkflowRouter can trigger workflows via duck-typing.

type ModuleFactory 

type ModuleFactory func(name string, config map[string]interface{}) modular.Module

ModuleFactory is a function that creates a module from a name and configuration

type StartStopModule 

type StartStopModule interface {
	modular.Module
	Start(ctx context.Context) error
	Stop(ctx context.Context) error
}

StartStopModule extends the basic Module interface with lifecycle methods

type StdEngine 

type StdEngine struct {
	// contains filtered or unexported fields
}

StdEngine represents the workflow execution engine

func NewStdEngine 

func NewStdEngine(app modular.Application, logger modular.Logger) *StdEngine

NewStdEngine creates a new workflow engine

func (*StdEngine) AddModuleType 

func (e *StdEngine) AddModuleType(moduleType string, factory ModuleFactory)

AddModuleType registers a factory function for a module type

func (*StdEngine) BuildFromConfig 

func (e *StdEngine) BuildFromConfig(cfg *config.WorkflowConfig) error

BuildFromConfig builds a workflow from configuration

func (*StdEngine) GetApp 

func (e *StdEngine) GetApp() modular.Application

GetApp returns the underlying modular Application.

func (*StdEngine) RegisterTrigger 

func (e *StdEngine) RegisterTrigger(trigger module.Trigger)

RegisterTrigger registers a trigger with the engine

func (*StdEngine) RegisterWorkflowHandler 

func (e *StdEngine) RegisterWorkflowHandler(handler WorkflowHandler)

RegisterWorkflowHandler adds a workflow handler to the engine

func (*StdEngine) SetDynamicLoader 

func (e *StdEngine) SetDynamicLoader(loader *dynamic.Loader)

SetDynamicLoader sets the dynamic component loader on the engine. When set, dynamic.component modules can load from source files via the "source" config key.

func (*StdEngine) SetDynamicRegistry 

func (e *StdEngine) SetDynamicRegistry(registry *dynamic.ComponentRegistry)

SetDynamicRegistry sets the dynamic component registry on the engine.

func (*StdEngine) Start 

func (e *StdEngine) Start(ctx context.Context) error

Start starts all modules and triggers

func (*StdEngine) Stop 

func (e *StdEngine) Stop(ctx context.Context) error

Stop stops all modules and triggers

func (*StdEngine) TriggerWorkflow 

func (e *StdEngine) TriggerWorkflow(ctx context.Context, workflowType string, action string, data map[string]interface{}) error

TriggerWorkflow starts a workflow based on a trigger

type WorkflowEngineManager 

type WorkflowEngineManager struct {
	// contains filtered or unexported fields
}

WorkflowEngineManager manages multiple concurrent workflow engine instances.

func NewWorkflowEngineManager 

func NewWorkflowEngineManager(wfStore store.WorkflowStore, linkStore store.CrossWorkflowLinkStore, logger *slog.Logger, engineBuilder EngineBuilderFunc) *WorkflowEngineManager

NewWorkflowEngineManager creates a new manager for workflow engine instances. The engineBuilder function is called to create each new engine instance, allowing the caller to register handlers and configure the dynamic system.

func (*WorkflowEngineManager) DeployWorkflow 

func (m *WorkflowEngineManager) DeployWorkflow(ctx context.Context, workflowID uuid.UUID) error

DeployWorkflow loads config from the store, creates an isolated engine, and starts it.

func (*WorkflowEngineManager) GetStatus 

func (m *WorkflowEngineManager) GetStatus(workflowID uuid.UUID) (*WorkflowStatus, error)

GetStatus returns the runtime status of a workflow.

func (*WorkflowEngineManager) ListActive 

func (m *WorkflowEngineManager) ListActive() []WorkflowStatus

ListActive returns the status of all running workflows.

func (*WorkflowEngineManager) ReloadWorkflow 

func (m *WorkflowEngineManager) ReloadWorkflow(ctx context.Context, workflowID uuid.UUID) error

ReloadWorkflow stops and redeploys a workflow.

func (*WorkflowEngineManager) Router 

func (m *WorkflowEngineManager) Router() *module.CrossWorkflowRouter

Router returns the cross-workflow event router.

func (*WorkflowEngineManager) StopAll 

func (m *WorkflowEngineManager) StopAll(ctx context.Context) error

StopAll gracefully stops all running engines.

func (*WorkflowEngineManager) StopWorkflow 

func (m *WorkflowEngineManager) StopWorkflow(ctx context.Context, workflowID uuid.UUID) error

StopWorkflow gracefully stops a running engine.

type WorkflowHandler 

type WorkflowHandler interface {
	// CanHandle returns true if this handler can process the given workflow type
	CanHandle(workflowType string) bool

	// ConfigureWorkflow sets up the workflow from configuration
	ConfigureWorkflow(app modular.Application, workflowConfig interface{}) error

	// ExecuteWorkflow executes a workflow with the given action and input data
	ExecuteWorkflow(ctx context.Context, workflowType string, action string, data map[string]interface{}) (map[string]interface{}, error)
}

WorkflowHandler interface for handling different workflow types

type WorkflowStatus 

type WorkflowStatus struct {
	WorkflowID  uuid.UUID     `json:"workflow_id"`
	Status      string        `json:"status"`
	StartedAt   time.Time     `json:"started_at"`
	Uptime      time.Duration `json:"uptime"`
	Error       string        `json:"error,omitempty"`
	ModuleCount int           `json:"module_count"`
}

WorkflowStatus describes the current runtime state of a managed workflow.

Source Files

View all Source files

Directories

Path Synopsis
ai
copilot
examples
Package examples contains complete examples of AI-generated workflows.
 Package examples contains complete examples of AI-generated workflows.
llm
cmd
server command
Package handlers provides workflow handling capabilities
 Package handlers provides workflow handling capabilities
Package mock provides common mock implementations for testing
 Package mock provides common mock implementations for testing
Package module defines core interfaces for the workflow engine
 Package module defines core interfaces for the workflow engine

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.