examples

LightQueue Examples

This directory contains working examples that demonstrate LightQueue's capabilities and real-world usage patterns.

🚀 Quick Start

All examples are self-contained and can be run directly:

# Navigate to the examples directory
cd examples

# Run any example
go run basic_usage.go
go run email_service.go
go run analytics_collector.go
go run durability_demo.go
go run retry_demo.go

📚 Examples Overview

1. Basic Usage (basic_usage.go)

Perfect for: Getting started with LightQueue

Demonstrates:

• ✅ Queue creation and configuration
• ✅ Publishing messages with priorities
• ✅ Consuming and acknowledging messages
• ✅ Basic statistics and monitoring

Key Features:

• Simple message publishing and consumption
• Priority-based message ordering
• Basic error handling
• Statistics display

2. Email Service (email_service.go)

Perfect for: Email notifications, user communications

Demonstrates:

• ✅ Real-world service architecture
• ✅ Multiple worker processes
• ✅ Priority-based email handling
• ✅ Error handling and retries
• ✅ Worker scaling

Key Features:

• Email job queuing with priorities
• Multiple concurrent workers
• Simulated SMTP failures
• Priority-based processing (password resets > newsletters)

3. Analytics Collector (analytics_collector.go)

Perfect for: High-volume event collection, user analytics

Demonstrates:

• ✅ High-volume message processing
• ✅ Batch processing patterns
• ✅ Event prioritization
• ✅ Performance optimization
• ✅ Large buffer configurations

Key Features:

• High-frequency event generation
• Batch processing for efficiency
• Event prioritization (purchases > page views)
• Performance monitoring

4. Durability Demo (durability_demo.go)

Perfect for: Understanding durability trade-offs

Demonstrates:

• ✅ Different durability modes (FULL, NORMAL, OFF)
• ✅ Performance vs safety trade-offs
• ✅ Configuration impact on throughput
• ✅ Persistence verification

Key Features:

• Side-by-side durability comparison
• Performance benchmarking
• Configuration impact analysis
• Use case recommendations

5. Retry Demo (retry_demo.go)

Perfect for: Understanding retry logic and failure handling

Demonstrates:

• ✅ Exponential backoff
• ✅ Retry limits and Dead Letter Queue
• ✅ Jitter for thundering herd prevention
• ✅ Failure rate simulation

Key Features:

• Configurable retry behavior
• Failure simulation
• Retry pattern analysis
• Dead Letter Queue handling

🎯 Use Case Mapping

⚙️ Configuration Examples

High Performance (Analytics)

config := &lightqueue.QueueConfig{
    Name:            "analytics",
    BufferSize:      2000,
    FlushInterval:   10 * time.Second,
    DurabilityMode:  "normal",
    WriteBufferSize: 5000,
    FlushOnPublish:  false,
    EnableFsync:     false,
}

High Reliability (Email)

config := &lightqueue.QueueConfig{
    Name:            "email_service",
    BufferSize:      200,
    FlushInterval:   5 * time.Second,
    MaxRetries:      5,
    DurabilityMode:  "normal",
    WriteBufferSize: 1000,
    FlushOnPublish:  false,
    EnableFsync:     false,
}

Maximum Safety (Financial)

config := &lightqueue.QueueConfig{
    Name:            "financial",
    BufferSize:      50,
    FlushInterval:   1 * time.Second,
    MaxRetries:      10,
    DurabilityMode:  "full",
    WriteBufferSize: 100,
    FlushOnPublish:  true,
    EnableFsync:     true,
}

🔧 Customization

Each example can be customized for your specific needs:

1. Modify Message Types

type CustomMessage struct {
    ID      string                 `json:"id"`
    Type    string                 `json:"type"`
    Data    map[string]interface{} `json:"data"`
    UserID  string                 `json:"user_id"`
}

2. Adjust Priorities

// Custom priority logic
func determinePriority(message CustomMessage) int {
    switch message.Type {
    case "critical": return 10
    case "important": return 7
    case "normal": return 5
    case "low": return 1
    default: return 3
    }
}

3. Scale Workers

// Start multiple workers
numWorkers := runtime.NumCPU() * 2
for i := 0; i < numWorkers; i++ {
    go worker(queue, fmt.Sprintf("worker-%d", i))
}

📊 Performance Tips

1. Buffer Sizing

• Small buffers (100-500): Better for real-time processing
• Large buffers (1000-5000): Better for high-throughput batch processing

2. Flush Intervals

• Short intervals (1-5s): Better durability, lower throughput
• Long intervals (10-30s): Higher throughput, lower durability

3. Worker Scaling

• CPU-bound tasks: 1-2 workers per CPU core
• I/O-bound tasks: 5-10 workers per CPU core

4. Durability Modes

• FULL: Critical data, financial transactions
• NORMAL: Most applications (recommended)
• OFF: High-throughput, non-critical data

🐛 Troubleshooting

Common Issues

1. Messages not being consumed

   • Check if workers are running
   • Verify topic names match
   • Check for dead letter queue entries

2. High memory usage

   • Reduce buffer sizes
   • Increase flush frequency
   • Check for memory leaks in message processing

3. Slow performance

   • Increase buffer sizes
   • Use OFF durability mode
   • Add more workers
   • Check disk I/O performance

4. Data loss concerns

   • Use FULL durability mode
   • Enable FlushOnPublish
   • Enable FSYNC
   • Reduce flush intervals

🚀 Next Steps

After running these examples:

1. Experiment with different configurations
2. Measure performance in your environment
3. Customize for your specific use case
4. Scale based on your requirements
5. Monitor using the built-in statistics

📖 Additional Resources

• Main README - Complete API documentation
• Durability Guide - Detailed durability configuration
• GitHub Repository - Source code and issues

🤝 Contributing

Found a bug or want to add an example? Contributions are welcome!

1. Fork the repository
2. Create your example
3. Add it to this README
4. Submit a pull request

Happy queuing! 🚀