README
¶
Beats - The Lightweight Shippers of the Elastic Stack
The Beats are lightweight data shippers, written in Go, that you install on your servers to capture all sorts of operational data (think of logs, metrics, or network packet data). The Beats send the operational data to Elasticsearch, either directly or via Logstash, so it can be visualized with Kibana.
By "lightweight", we mean that Beats have a small installation footprint, use limited system resources, and have no runtime dependencies.
This repository contains libbeat, our Go framework for creating Beats, and all the officially supported Beats:
| Beat | Description |
|---|---|
| Filebeat | Tails and ships log files |
| Heartbeat | Ping remote services for availability |
| Metricbeat | Fetches sets of metrics from the operating system and services |
| Packetbeat | Monitors the network and applications by sniffing packets |
| Winlogbeat | Fetches and ships Windows Event logs |
In addition to the above Beats, which are officially supported by Elastic, the community has created a set of other Beats that make use of libbeat but live outside of this Github repository. We maintain a list of community Beats here.
Documentation and Getting Started
You can find the documentation and getting started guides for each of the Beats on the elastic.co site:
Getting Help
If you need help or hit an issue, please start by opening a topic on our discuss forums. Please note that we reserve GitHub tickets for confirmed bugs and enhancement requests.
Downloads
You can download pre-compiled Beats binaries, as well as packages for the supported platforms, from this page.
Contributing
We'd love working with you! You can help make the Beats better in many ways: report issues, help us reproduce issues, fix bugs, add functionality, or even create your own Beat.
Please start by reading our CONTRIBUTING file.
If you are creating a new Beat, you don't need to submit the code to this repository. You can simply start working in a new repository and make use of the libbeat packages, by following our developer guide. After you have a working prototype, open a pull request to add your Beat to the list of community Beats.
Building Beats from the Source
See our CONTRIBUTING file for information about setting up your dev environment to build Beats from the source.
Directories
¶
| Path | Synopsis |
|---|---|
|
module/audit
Package audit is an Auditbeat module that reports security-relevant information based on data captured from the operating system (OS) or services running on the OS.
|
Package audit is an Auditbeat module that reports security-relevant information based on data captured from the operating system (OS) or services running on the OS. |
|
module/audit/kernel
Package kernel is a metricset that subscribes to the Linux Audit Framework to receive audit events from the the kernel.
|
Package kernel is a metricset that subscribes to the Linux Audit Framework to receive audit events from the the kernel. |
|
scripts
command
|
|
|
dev-tools
|
|
|
cmd/dashboards
command
|
|
|
cmd/index_template
command
|
|
|
harvester/reader
Package reader provides interface and struct to read messages and report them to a harvester
|
Package reader provides interface and struct to read messages and report them to a harvester |
|
prospector/log
Package log harvests different inputs for new information.
|
Package log harvests different inputs for new information. |
|
prospector/redis
Package redis package contains prospector and harvester to read the redis slow log
|
Package redis package contains prospector and harvester to read the redis slow log |
|
look
Package look defines common formatters for fields/types to be used when generating heartbeat events.
|
Package look defines common formatters for fields/types to be used when generating heartbeat events. |
|
beat
Package beat provides the functions required to manage the life-cycle of a Beat.
|
Package beat provides the functions required to manage the life-cycle of a Beat. |
|
common/atomic
Package atomic provides common primitive types with atomic accessors.
|
Package atomic provides common primitive types with atomic accessors. |
|
common/dtfmt
Package dtfmt provides time formatter support with pattern syntax mostly similar to joda DateTimeFormat.
|
Package dtfmt provides time formatter support with pattern syntax mostly similar to joda DateTimeFormat. |
|
common/schema
Package schema contains helper utilities to parse, convert, rename, and reorganize data from the format returned by the various monitored systems to the data model used by Metricbeat.
|
Package schema contains helper utilities to parse, convert, rename, and reorganize data from the format returned by the various monitored systems to the data model used by Metricbeat. |
|
common/schema/mapstriface
Package mapstriface contains utilities for transforming map[string]interface{} objects into metricbeat events.
|
Package mapstriface contains utilities for transforming map[string]interface{} objects into metricbeat events. |
|
common/schema/mapstrstr
Package mapstrstr contains utilities for transforming map[string]string objects into metricbeat events.
|
Package mapstrstr contains utilities for transforming map[string]string objects into metricbeat events. |
|
common/streambuf
Package streambuf provides helpers for buffering multiple packet payloads and some general parsing functions.
|
Package streambuf provides helpers for buffering multiple packet payloads and some general parsing functions. |
|
ml-importer
Package mlimporter contains code for loading Elastic X-Pack Machine Learning job configurations.
|
Package mlimporter contains code for loading Elastic X-Pack Machine Learning job configurations. |
|
paths
Package paths provides a common way to handle paths configuration for all Beats.
|
Package paths provides a common way to handle paths configuration for all Beats. |
|
publisher/broker/brokertest
Package brokertest provides common functionality tests all broker implementations must pass.
|
Package brokertest provides common functionality tests all broker implementations must pass. |
|
publisher/broker/membroker
Package membroker provides an in-memory publisher.Broker implementation for use with the publisher pipeline.
|
Package membroker provides an in-memory publisher.Broker implementation for use with the publisher pipeline. |
|
publisher/pipeline
Package pipeline combines all publisher functionality (processors, broker, outputs) to create instances of complete publisher pipelines, beats can connect to publish events to.
|
Package pipeline combines all publisher functionality (processors, broker, outputs) to create instances of complete publisher pipelines, beats can connect to publish events to. |
|
Package metricbeat contains the entrypoint to Metricbeat which is a lightweight data shipper for operating system and service metrics.
|
Package metricbeat contains the entrypoint to Metricbeat which is a lightweight data shipper for operating system and service metrics. |
|
beater
Package beater provides the implementation of the libbeat Beater interface for Metricbeat and functions for running Metricbeat Modules on their own.
|
Package beater provides the implementation of the libbeat Beater interface for Metricbeat and functions for running Metricbeat Modules on their own. |
|
include
Package include imports all Module and MetricSet packages so that they register their factories with the global registry.
|
Package include imports all Module and MetricSet packages so that they register their factories with the global registry. |
|
mb
Package mb (short for Metricbeat) contains the public interfaces that are used to implement Modules and their associated MetricSets.
|
Package mb (short for Metricbeat) contains the public interfaces that are used to implement Modules and their associated MetricSets. |
|
mb/testing
Package testing provides utility functions for testing Module and MetricSet implementations.
|
Package testing provides utility functions for testing Module and MetricSet implementations. |
|
module
Package module contains Metricbeat modules and their MetricSet implementations.
|
Package module contains Metricbeat modules and their MetricSet implementations. |
|
module/aerospike
Package aerospike is a Metricbeat module that contains MetricSets.
|
Package aerospike is a Metricbeat module that contains MetricSets. |
|
module/apache
Package apache is Metricbeat module for the Apache HTTPD server.
|
Package apache is Metricbeat module for the Apache HTTPD server. |
|
module/apache/status
Package status reads Apache HTTPD server status from the mod_status module.
|
Package status reads Apache HTTPD server status from the mod_status module. |
|
module/ceph
Package ceph is a Metricbeat module that contains MetricSets.
|
Package ceph is a Metricbeat module that contains MetricSets. |
|
module/couchbase
Package couchbase is a Metricbeat module that contains MetricSets.
|
Package couchbase is a Metricbeat module that contains MetricSets. |
|
module/docker
Package docker is a Metricbeat module that contains MetricSets.
|
Package docker is a Metricbeat module that contains MetricSets. |
|
module/dropwizard
Package dropwizard is a Metricbeat module that contains MetricSets.
|
Package dropwizard is a Metricbeat module that contains MetricSets. |
|
module/elasticsearch
Package elasticsearch is a Metricbeat module that contains MetricSets.
|
Package elasticsearch is a Metricbeat module that contains MetricSets. |
|
module/golang
Package golang is a Metricbeat module that contains MetricSets.
|
Package golang is a Metricbeat module that contains MetricSets. |
|
module/haproxy
Package haproxy is a Metricbeat module that contains MetricSets.
|
Package haproxy is a Metricbeat module that contains MetricSets. |
|
module/http
Package http is a Metricbeat module that contains MetricSets.
|
Package http is a Metricbeat module that contains MetricSets. |
|
module/http/_meta/test
command
|
|
|
module/jolokia
Package jolokia is a Metricbeat module that contains MetricSets.
|
Package jolokia is a Metricbeat module that contains MetricSets. |
|
module/kafka
Package kafka is a Metricbeat module that contains MetricSets.
|
Package kafka is a Metricbeat module that contains MetricSets. |
|
module/kibana
Package kibana is a Metricbeat module that contains MetricSets.
|
Package kibana is a Metricbeat module that contains MetricSets. |
|
module/kubernetes
Package kubernetes is a Metricbeat module that contains MetricSets.
|
Package kubernetes is a Metricbeat module that contains MetricSets. |
|
module/memcached
Package memcached is a Metricbeat module that contains MetricSets.
|
Package memcached is a Metricbeat module that contains MetricSets. |
|
module/mongodb
Package mongodb is a Metricbeat module that contains MetricSets.
|
Package mongodb is a Metricbeat module that contains MetricSets. |
|
module/mysql
Package mysql is Metricbeat module for MySQL server.
|
Package mysql is Metricbeat module for MySQL server. |
|
module/mysql/status
Package status fetches MySQL server status metrics.
|
Package status fetches MySQL server status metrics. |
|
module/nginx
Package nginx is Metricbeat module for the Nginx reverse proxy.
|
Package nginx is Metricbeat module for the Nginx reverse proxy. |
|
module/nginx/stubstatus
Package stubstatus reads server status from nginx host under /server-status, ngx_http_stub_status_module is required.
|
Package stubstatus reads server status from nginx host under /server-status, ngx_http_stub_status_module is required. |
|
module/php_fpm
Package php_fpm is a Metricbeat module that contains MetricSets.
|
Package php_fpm is a Metricbeat module that contains MetricSets. |
|
module/postgresql
Package postgresql is a Metricbeat module that contains MetricSets.
|
Package postgresql is a Metricbeat module that contains MetricSets. |
|
module/prometheus
Package prometheus is a Metricbeat module that contains MetricSets.
|
Package prometheus is a Metricbeat module that contains MetricSets. |
|
module/rabbitmq
Package rabbitmq is a Metricbeat module that contains MetricSets.
|
Package rabbitmq is a Metricbeat module that contains MetricSets. |
|
module/redis
Package redis contains shared Redis functionality for the metric sets
|
Package redis contains shared Redis functionality for the metric sets |
|
module/redis/info
Package info fetches Redis server information and statistics using the Redis INFO command.
|
Package info fetches Redis server information and statistics using the Redis INFO command. |
|
module/system
Package system is a Metricbeat module that contains MetricSets that collect system level information like CPU and memory stats.
|
Package system is a Metricbeat module that contains MetricSets that collect system level information like CPU and memory stats. |
|
module/system/_meta/testing
command
|
|
|
module/system/core
Package core collects cpu core metrics from the host OS.
|
Package core collects cpu core metrics from the host OS. |
|
module/system/cpu
Package cpu collects CPU metrics from the host OS.
|
Package cpu collects CPU metrics from the host OS. |
|
module/system/diskio
Package diskio fetches disk IO metrics from the OS.
|
Package diskio fetches disk IO metrics from the OS. |
|
module/system/filesystem
Package filesystem provides a MetricSet implementation that fetches metrics for each of the mounted file systems.
|
Package filesystem provides a MetricSet implementation that fetches metrics for each of the mounted file systems. |
|
module/system/fsstat
Package fsstat provides a MetricSet for fetching aggregated filesystem stats.
|
Package fsstat provides a MetricSet for fetching aggregated filesystem stats. |
|
module/system/load
Package load collects system CPU load metrics from the host OS.
|
Package load collects system CPU load metrics from the host OS. |
|
module/system/memory
Package memory collects memory metrics from the host OS.
|
Package memory collects memory metrics from the host OS. |
|
module/system/network
Package network provides network IO metrics from the OS.
|
Package network provides network IO metrics from the OS. |
|
module/system/process
Package process collects metrics about the running processes using information from the operating system.
|
Package process collects metrics about the running processes using information from the operating system. |
|
module/system/process_summary
Package process_summary collects high level summary metrics about the running processes.
|
Package process_summary collects high level summary metrics about the running processes. |
|
module/system/socket
Package socket captures active sockets and the processes that own them.
|
Package socket captures active sockets and the processes that own them. |
|
module/vsphere
Package vsphere is a Metricbeat module that contains MetricSets.
|
Package vsphere is a Metricbeat module that contains MetricSets. |
|
module/windows
Package windows is a Metricbeat module that contains MetricSets.
|
Package windows is a Metricbeat module that contains MetricSets. |
|
module/windows/perfmon
Package perfmon implements a Metricbeat metricset for reading Windows performance counters.
|
Package perfmon implements a Metricbeat metricset for reading Windows performance counters. |
|
module/zookeeper
Package zookeeper is a Metricbeat module for ZooKeeper servers.
|
Package zookeeper is a Metricbeat module for ZooKeeper servers. |
|
module/zookeeper/mntr
Package mntr fetches metrics from ZooKeeper by using the mntr command which was added to ZooKeeper in version 3.4.0.
|
Package mntr fetches metrics from ZooKeeper by using the mntr command which was added to ZooKeeper in version 3.4.0. |
|
include
Package include imports all protos packages so that they register with the global registry.
|
Package include imports all protos packages so that they register with the global registry. |
|
protos/applayer
Package applayer provides common definitions with common fields for use with application layer protocols among beats.
|
Package applayer provides common definitions with common fields for use with application layer protocols among beats. |
|
protos/dns
Package dns provides support for parsing DNS messages and reporting the results.
|
Package dns provides support for parsing DNS messages and reporting the results. |
|
Package winlogbeat contains the entrypoint to Winlogbeat which is a lightweight data shipper for Windows event logs.
|
Package winlogbeat contains the entrypoint to Winlogbeat which is a lightweight data shipper for Windows event logs. |
|
beater
Package beater provides the implementation of the libbeat Beater interface for Winlogbeat.
|
Package beater provides the implementation of the libbeat Beater interface for Winlogbeat. |
|
checkpoint
Package checkpoint persists event log state information to disk so that event log monitoring can resume from the last read event in the case of a restart or unexpected interruption.
|
Package checkpoint persists event log state information to disk so that event log monitoring can resume from the last read event in the case of a restart or unexpected interruption. |
|
config
Package config provides the winlogbeat specific configuration options.
|
Package config provides the winlogbeat specific configuration options. |
|
eventlog
Package eventlog provides the means for reading event logs from Windows.
|
Package eventlog provides the means for reading event logs from Windows. |
|
sys
Package sys provides common data structures and utilties functions that are used by the subpackages for interfacing with the system level APIs to collect event log records from Windows.
|
Package sys provides common data structures and utilties functions that are used by the subpackages for interfacing with the system level APIs to collect event log records from Windows. |
|
sys/eventlogging
Package eventlogging provides access to the Event Logging API that was designed for applications that run on the Windows Server 2003, Windows XP, or Windows 2000 operating system.
|
Package eventlogging provides access to the Event Logging API that was designed for applications that run on the Windows Server 2003, Windows XP, or Windows 2000 operating system. |
|
sys/wineventlog
Package wineventlog provides access to the Windows Event Log API used in all versions of Windows since Vista (i.e.
|
Package wineventlog provides access to the Windows Event Log API used in all versions of Windows since Vista (i.e. |
Click to show internal directories.
Click to hide internal directories.