binlogo

Binlogo

Binlogo is the distributed, visualized application based on MySQL binlog. In short, binlogo is to process the data changes of MySQL into easily understand messages and output to different systems according to the user's configuration. Here are part of advantages:

• Distributed, multi node improves availability of the whole system.
• Visualization, can complete common operations and observe the status of the whole cluster in the control website
• It can be output to multiple queues or other applications, and new outputs are constantly added

Get Started

• Install redis. Binlogo V2 relies on redis, so you must install redis first.

• Install binlogo. Binlogo's download address: Download Address

• Message Format： Data format of binlogo output

• Start binlogo.

  Edit config. ${binlogo-path}/etc/binlogo.yaml

  # ./binlogo server --config ./etc/binlogo.yaml
  

• Open up your browser to http://127.0.0.1:8081/console to view the console website

• Create Pipeline: config.go

Follow the steps.

• Run pipeline.

Click button to run the pipeline instance.

• Operation condition.

You can see the operation condition of pipeline.

• See the output

Insert some into mysql, watch the ouput on stdout.

• Configuration output to Kafka

• High performance, possible data loss.
  • acks=1
  • enable.idempotence=false
  • compression.type=snappy
  • retries=0
• For reliability performance:
  • acks=-1
  • enable.idempotence=true
  • retries=3 or larger one

Docker

Docker Hub

  # docker pull jin06/binlogo
  # docker run -e REDIS_ADDR=127.0.0.1 -e REDIS_PORT=6379 -e REDIS_PASSWORD=yourpassword -e REDIS_DB=0  --name BinlogoNode -it -d -p 8081:8081 jin06/binlogo:latest

Kubernetes

• doc

Other outputs

• HTTP
• RabbitMQ
• Kafka
• Redis
• AliCloud RocketMQ

Docs

• docs link

Questions

• To Report bug: GitHub Issue
• Contact author: jlonmyway@gmail.com