Prometheus exporter for Gluster Metrics

These exporters will be run on all Gluster peers, So it makes sense to collect only local metrics and aggregate in Prometheus server when required.

Install

git clone https://github.com/gluster/gluster-prometheus.git
cd gluster-prometheus/gluster_exporter
go get
go build

Usage

Run gluster_exporter with Gluster Peer ID(Current limitation, this will be detected in future releases)

./gluster_exporter -peerid <gluster-peer-id>

For example,

./gluster_exporter -peerid 019042a8-fc13-4abe-88b4-f070905bf78b

Other available options,

-metrics-path string
      Metrics API Path (default "/metrics")
-peerid string
      Gluster Peer ID
-port int
      Exporter Port (default 8080)

Metrics

CPU Usage Percentage of Gluster Processes

Example output:

# HELP gluster_cpu_percentage CPU Percentage used by Gluster processes
# TYPE gluster_cpu_percentage gauge
gluster_cpu_percentage{name="glusterfsd",peerid="019042a8-fc13-4abe-88b4-f070905bf78b",brick_path="/bricks/b1",volume="gv1"} 0
gluster_cpu_percentage{name="glusterfsd",peerid="019042a8-fc13-4abe-88b4-f070905bf78b",brick_path="/bricks/b2",volume="gv1"} 0

Elapsed Time/Uptime of Gluster Processes

Example output:

# HELP gluster_elapsed_time_seconds Elapsed Time of Gluster processes
# TYPE gluster_elapsed_time_seconds gauge
gluster_elapsed_time_seconds{name="glusterfsd",peerid="019042a8-fc13-4abe-88b4-f070905bf78b",brick_path="/bricks/b1",volume="gv1"} 2969
gluster_elapsed_time_seconds{name="glusterfsd",peerid="019042a8-fc13-4abe-88b4-f070905bf78b",brick_path="/bricks/b2",volume="gv1"} 2969

Memory percentage, Resident Memory and Virtual Memory of Gluster Processes

Example output:

# HELP gluster_memory_percentage Memory Percentage used by Gluster processes
# TYPE gluster_memory_percentage gauge
gluster_memory_percentage{name="glusterfsd",peerid="019042a8-fc13-4abe-88b4-f070905bf78b",brick_path="/bricks/b1",volume="gv1"} 0.7
gluster_memory_percentage{name="glusterfsd",peerid="019042a8-fc13-4abe-88b4-f070905bf78b",brick_path="/bricks/b2",volume="gv1"} 0.7

# HELP gluster_resident_memory Resident Memory of Gluster processes
# TYPE gluster_resident_memory gauge
gluster_resident_memory{name="glusterfsd",peerid="019042a8-fc13-4abe-88b4-f070905bf78b",brick_path="/bricks/b1",volume="gv1"} 15392
gluster_resident_memory{name="glusterfsd",peerid="019042a8-fc13-4abe-88b4-f070905bf78b",brick_path="/bricks/b2",volume="gv1"} 14760

# HELP gluster_virtual_memory Virtual Memory of Gluster processes
# TYPE gluster_virtual_memory gauge
gluster_virtual_memory{name="glusterfsd",peerid="019042a8-fc13-4abe-88b4-f070905bf78b",brick_path="/bricks/b1",volume="gv1"} 912260
gluster_virtual_memory{name="glusterfsd",peerid="019042a8-fc13-4abe-88b4-f070905bf78b",brick_path="/bricks/b2",volume="gv1"} 912520

Exporter will automatically detect the running processes which are related to Gluster, and extracts the meta informations like Volume name, Brick path etc from the /proc/<pid>/cmdline. These meta information will be available as labels in Prometheus. Currently it is detecting glusterd, glusterfsd and glusterd2 processes only. Support will be added to more processes like Geo-replication, Self Heal etc.

Adding New metrics

• Add new file under gluster_exporter directory.

• Define Metrics depending on the type of Metric(https://prometheus.io/docs/concepts/metric_types/) For example, "Gauge" Metrics type

glusterCPUPercentage = prometheus.NewGaugeVec(
    prometheus.GaugeOpts{
        Namespace: "gluster",
        Name:      "cpu_percentage",
        Help:      "CPU Percentage used by Gluster processes",
    },
    []string{"volume", "peerid", "brick_path"},
)

• Implement the function to gather data, and register to gather data in required interval

  prometheus.MustRegister(glusterCPUPercentage)

  // Register to update this every 2 seconds
  // Name, Callback Func, Interval Seconds
  registerMetric("gluster_ps", psfunc, 2)

• Thats it! Exporter will run these registered metrics.

TODO

• Build and Install Makefile

• More metrics need to be implemented

• Logging

• Handling failures

• Rules generation

• Consuming Glusterd2 REST APIs

• Detect Gluster Peer ID automatically(Now accepts as Commandline parameter)

• Tutorial for setup, integration with Grafana etc