pulsar

package
v0.51.4 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 17, 2023 License: GPL-3.0 Imports: 10 Imported by: 1

README

Apache Pulsar collector

Apache Pulsar is an open-source distributed pub-sub messaging system.

This module will monitor one or more Apache Pulsar instances, depending on your configuration.

It collects broker statistics from the prometheus endpoint.

pulsar module is tested on the following versions:

  • v2.5.0

Metrics

All metrics have "pulsar." prefix.

  • topic_* metrics are available when exposeTopicLevelMetricsInPrometheus is set to true.
  • subscription_* and namespace_subscription metrics are available when exposeTopicLevelMetricsInPrometheus si set to true.
  • replication_* and namespace_replication_* metrics are available when replication is configured and replicationMetricsEnabled is set to true
Metric Scope Dimensions Units
broker_components global namespaces, topics, subscriptions, producers, consumers components
messages_rate global publish, dispatch messages/s
throughput_rate global publish, dispatch KiB/s
storage_size global used KiB
storage_operations_rate global read, write message batches/s
msg_backlog global backlog messages
storage_write_latency global <=0.5ms, <=1ms, <=5ms, =10ms, <=20ms, <=50ms, <=100ms, <=200ms, <=1s, >1s entries/s
entry_size global <=128B, <=512B, <=1KB, <=2KB, <=4KB, <=16KB, <=100KB, <=1MB, >1MB entries/s
subscription_delayed global delayed message bacthes
subscription_msg_rate_redeliver global redelivered messages/s
subscription_blocked_on_unacked_messages global blocked subscriptions
replication_rate global in, out messages/s
replication_throughput_rate global in, out KiB/s
replication_backlog global backlog messages
namespace_broker_components global topics, subscriptions, producers, consumers components
namespace_messages_rate namespace publish, dispatch messages/s
namespace_throughput_rate namespace publish, dispatch KiB/s
namespace_storage_size namespace used KiB
namespace_storage_operations_rate namespace read, write message batches/s
namespace_msg_backlog namespace backlog messages
namespace_storage_write_latency namespace <=0.5ms, <=1ms, <=5ms, =10ms, <=20ms, <=50ms, <=100ms, <=200ms, <=1s, >1s entries/s
namespace_entry_size namespace <=128B, <=512B, <=1KB, <=2KB, <=4KB, <=16KB, <=100KB, <=1MB, >1MB entries/s
namespace_subscription_delayed namespace delayed message bacthes
namespace_subscription_msg_rate_redeliver namespace redelivered messages/s
namespace_subscription_blocked_on_unacked_messages namespace blocked subscriptions
namespace_replication_rate namespace in, out messages/s
namespace_replication_throughput_rate namespace in, out KiB/s
namespace_replication_backlog namespace backlog messages
topic_producers namespace a dimension per topic producers
topic_subscriptions namespace a dimension per topic subscriptions
topic_consumers namespace a dimension per topic consumers
topic_messages_rate_in namespace a dimension per topic publishes/s
topic_messages_rate_out namespace a dimension per topic dispatches/s
topic_throughput_rate_in namespace a dimension per topic KiB/s
topic_throughput_rate_out namespace a dimension per topic KiB/s
topic_storage_size namespace a dimension per topic KiB
topic_storage_read_rate namespace a dimension per topic message batches/s
topic_storage_write_rate namespace a dimension per topic message batches/s
topic_msg_backlog namespace a dimension per topic messages
topic_subscription_delayed namespace a dimension per topic message batches
topic_subscription_msg_rate_redeliver namespace a dimension per topic messages/s
topic_subscription_blocked_on_unacked_messages namespace a dimension per topic blocked subscriptions
topic_replication_rate_in namespace a dimension per topic messages/s
topic_replication_rate_out namespace a dimension per topic messages/s
topic_replication_throughput_rate_in namespace a dimension per topic messages/s
topic_replication_throughput_rate_out namespace a dimension per topic messages/s
topic_replication_backlog namespace a dimension per topic messages
topic_replication_backlog namespace a dimension per topic messages

Configuration

Edit the go.d/pulsar.conf configuration file using edit-config from the Netdata config directory, which is typically at /etc/netdata.

cd /etc/netdata # Replace this path with your Netdata config directory
sudo ./edit-config go.d/pulsar.conf

Needs only url to server's /metrics endpoint. Here is an example for 2 servers:

jobs:
  - name: local
    url: http://127.0.0.1:8080/metrics

  - name: remote
    url: http://203.0.113.10:8080/metrics

For all available options please see module configuration file.

Topic filtering

By default, module collects data for all topics, but it supports topic filtering. Filtering doesn't exclude a topic stats from the summary/namespace stats, it only removes the topic from the topic charts.

To check matcher syntax see matcher documentation.

  - name: local
    url: http://127.0.0.1:8080/metrics
    topic_filter:
      includes:
        - matcher1
        - matcher2
      excludes:
        - matcher1
        - matcher2

Update every

Module default update_every is 60.

Apache Pulsar doesnt expose raw counters, it exposes rate. It counts rates every statsUpdateFrequencyInSecs. Default value is 60 seconds.

Module update_every should be equal to statsUpdateFrequencyInSecs.

Troubleshooting

To troubleshoot issues with the pulsar collector, run the go.d.plugin with the debug option enabled. The output should give you clues as to why the collector isn't working.

  • Navigate to the plugins.d directory, usually at /usr/libexec/netdata/plugins.d/. If that's not the case on your system, open netdata.conf and look for the plugins setting under [directories].

    cd /usr/libexec/netdata/plugins.d/

  • Switch to the netdata user.

    sudo -u netdata -s

  • Run the go.d.plugin to debug the collector:

    ./go.d.plugin -d -m pulsar

Documentation

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

This section is empty.

Types

type Chart 

type Chart = module.Chart

type Charts 

type Charts = module.Charts

type Config 

type Config struct {
	web.HTTP   `yaml:",inline"`
	TopicFiler matcher.SimpleExpr `yaml:"topic_filter"`
}

type Dim 

type Dim = module.Dim

type Dims 

type Dims = module.Dims

type Opts 

type Opts = module.Opts

type Pulsar 

type Pulsar struct {
	module.Base
	Config `yaml:",inline"`
	// contains filtered or unexported fields
}

func New 

func New() *Pulsar

func (*Pulsar) Charts 

func (p *Pulsar) Charts() *Charts

func (*Pulsar) Check 

func (p *Pulsar) Check() bool

func (Pulsar) Cleanup 

func (Pulsar) Cleanup()

func (*Pulsar) Collect 

func (p *Pulsar) Collect() map[string]int64

func (*Pulsar) Init 

func (p *Pulsar) Init() bool

Source Files

View all Source files

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.