example-queue

Mithril Queue & Scheduler Example

This example demonstrates the Queue System and Task Scheduler features of the Mithril framework.

Features Demonstrated

Queue System

• Job dispatching
• Background job processing
• Job handlers
• Failed job tracking
• Job retry mechanism
• Memory queue (for development)

Task Scheduler

• Interval-based tasks (Every)
• Daily tasks at specific times (Daily)
• Hourly tasks at specific minutes (Hourly)
• Automatic task execution
• Task status tracking

Running the Example

# Start the server
go run example-queue/main.go

The server will start on http://localhost:3003.

API Endpoints

1. Get Server Info

curl http://localhost:3003/

2. Dispatch Email Job

curl -X POST http://localhost:3003/dispatch/email \
  -H "Content-Type: application/json" \
  -d '{"to":"user@example.com","subject":"Hello from Mithril"}'

3. Dispatch Data Processing Job

curl -X POST http://localhost:3003/dispatch/data \
  -H "Content-Type: application/json" \
  -d '{"data_id":"123456"}'

4. List Failed Jobs

curl http://localhost:3003/jobs/failed

5. Retry Failed Job

curl -X POST http://localhost:3003/jobs/retry/JOB_ID

6. List Scheduled Tasks

curl http://localhost:3003/schedule/list

Job Handlers

The example includes two job handlers:

SendEmailJob

• Simulates sending an email
• Takes to and subject from payload
• Processes in 2 seconds

ProcessDataJob

• Simulates data processing
• Takes data_id from payload
• Processes in 3 seconds

Scheduled Tasks

The example includes three scheduled tasks:

1. Health Check - Runs every 10 seconds
2. Daily Report - Runs daily at 09:00
3. Hourly Cleanup - Runs hourly at :00 minutes

Queue Configuration

Configure the queue driver via environment variables:

# Use memory queue (default)
QUEUE_DRIVER=memory

# Use Redis queue
QUEUE_DRIVER=redis
REDIS_HOST=localhost
REDIS_PORT=6379
REDIS_PASSWORD=
REDIS_DB=0

# Queue settings
QUEUE_PREFIX=mithril:queue
QUEUE_TIMEOUT=5m
QUEUE_RETRIES=3

Creating Custom Jobs

// Define a job handler
type MyJob struct{}

func (j *MyJob) Handle(ctx context.Context, job *queue.Job) error {
    data := job.Payload["data"]
    // Process the job
    return nil
}

// Register the handler
queueManager.Register("my_job", &MyJob{})

// Dispatch a job
queueManager.Dispatch(ctx, "my_job", map[string]interface{}{
    "data": "some data",
})

Creating Scheduled Tasks

// Run every X duration
scheduler.Every(5*time.Minute, "task_name", func(ctx context.Context) error {
    // Task logic
    return nil
})

// Run daily at specific time
scheduler.Daily("14:30", "task_name", func(ctx context.Context) error {
    // Task logic
    return nil
})

// Run hourly at specific minute
scheduler.Hourly(15, "task_name", func(ctx context.Context) error {
    // Task logic at :15 of every hour
    return nil
})

Production Usage

For production environments:

1. Use Redis Queue: More reliable than memory queue
2. Deploy Multiple Workers: Scale horizontally
3. Monitor Failed Jobs: Set up alerts for failed jobs
4. Configure Retries: Set appropriate retry limits
5. Use Supervisor: Keep workers running (systemd, PM2, etc.)

Worker Command

In a generated Mithril project, start a queue worker using:

./artisan queue:work default

This will start processing jobs from the "default" queue.

Next Steps

• Implement RabbitMQ driver for high-throughput scenarios
• Add job priority support
• Implement delayed jobs
• Add job progress tracking
• Create dashboard for monitoring jobs and tasks