machinery-plugins

machinery-plugins

This project is intended to provide multi broker, lock, backend for the RichardKnop/machinery library. About Machinery is an asynchronous task queue/job queue based on distributed message passing. This plugins library offers additional functionalities and tools to help you better utilize and customize Machinery.

Features

This library provides the following features:

• Brokers: add etcd / mysql
• Locks： add etcd / mysql
• Backends: add etcd / mysql

you can combine whatever you want

Quick Start

Install the library:

go get git.ifooth.com/common/machinery-plugins@latest

Import the library in your code and use it to create tasks:

import (
    "github.com/RichardKnop/machinery/v2"
    redisbackend "github.com/RichardKnop/machinery/v2/backends/redis"
    eagerlock "github.com/RichardKnop/machinery/v2/locks/eager"
    etcdbroker "git.ifooth.com/common/machinery-plugins/brokers/etcd"
)

func main() {
    // Create server instance
	// broker := redisbroker.NewGR(cnf, []string{"localhost:6379"}, 1)
	broker, err := etcdbroker.New(cnf, "http://127.0.0.1:2379")
	if err != nil {
		return nil, err
	}

	lock := eagerlock.New()
	backend := redisbackend.NewGR(cnf, []string{"localhost:6379"}, 3)
	server := machinery.NewServer(cnf, broker, backend, lock)
}

Contributing

We welcome contributions in any form! If you have any issues, suggestions, or wish to add new features, let us know by creating an issue or pull request.

License

machinery-plugins is based on the Apache 2.0 protocol. Please refer to LICENSE for details.

设计

push -> pending_tasks/{queue}/{task_uuid} handle -> 获取keys-only, 判断是否有consumer，list/watch(可用key列表) -> 原子操作 -> handler -> delete(每次全量key N，可以对吧lock)

delay -> delay_tasks/r{xx}-taskuuid handlr -> 按key排序，获取最新的一个(可以大于当前时间) -> 原子put/delete操作 -> done (性能没有问题， 多个服务会有冲突,最多N-1，可以对比Lock)