Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,12 @@ It is also possible to feed the stats from an existing vdbench workload provided
cat console.foo | vdbench_graphite.sh -h server01 -p 2003 -t foo.baz
```

You can additonally specify the hostname of the system where vdbench stats were collected if executing this script from another host.

```
cat console.foo | vdbench_graphite.sh -h server01 -p 2003 -t foo.baz -n host.example.com
```

Installing and configuring graphite and grafana are beyond the scope of this README. As long as there is an appropriately configured storage schema for graphite, the graphs should look good. If you notice gaps in the graphs verify that that the "interval=" parameter in your vdbench config matches the lowest retention period in storage-schemas.conf. The easiest configuration (and highest granularity) is "interval=1" in the vdbench config and a stanza in storage-schemas.conf like:

```
Expand All @@ -57,3 +63,4 @@ Some of the single stats will show as orange or red if thresholds are crossed.
Accurate time on both the client machine (web browser), the graphana server, and workers is appropriate for accurate stats. NTP is recommended. Graph rendering is done by the client browser and if the local time is off on that machine, the graphs will show as skewed in time.

The graphs are ideally setup for a single command vm with multiple workers. It is expected that the command vm will control the execution of vdbench to the workers (ssh keys) and aggregate the stats to be fed to graphite. There's no reason that vdbench running indvidually on hosts (not controlled by a command) wouldn't also work, but some modification of the graphs would help. For example, the table for IOPS could be modified to reflect the worker hostnames so each line can be clearly identified. Also a separate line that totals all of the workers could show aggregate IOPS for all workers.

55 changes: 33 additions & 22 deletions vdbench_graphite.sh
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
#!/bin/bash

# vdbench_graphite.sh is a bash script to feed parsed vdbench output to graphite
#
# Requires netcat (nc), tee, awk, graphite, and vdbench
#
# Requires netcat (nc), tee, awk, graphite, and vdbench
# http://www.oracle.com/technetwork/server-storage/vdbench-downloads-1901681.html
#
# To execute, simply run vdbench and pipe to this script. By default it will send stats to graphite localhost:2003
# vdbench.default.[HOSTNAME].[METRIC] There are no required options, but additional parameters can be passed to
# To execute, simply run vdbench and pipe to this script. By default it will send stats to graphite localhost:2003
# vdbench.default.[HOSTNAME].[METRIC] There are no required options, but additional parameters can be passed to
# change the destination host and/or port as well as a custom tag instead of using the hostname. To implement multiple
# custom tags simply separate the tags with periods.
#
# It is also possible to feed the stats from an existing vdbench workload provided stdout was written to a file.
# It is also possible to feed the stats from an existing vdbench workload provided stdout was written to a file.
# Simply cat the vdbench output file and redirect to vdbench_graphite.sh the same as above. Note: original timestamps
# are used for the metrics so it will be necessary to look at the graph historically.
# are used for the metrics so it will be necessary to look at the graph historically.
#
# ./vdbench -f foo.vdb -o outdir.foo | vdbench_graphite.sh -h [graphite_host] -p [graphite_port] -t [graphite_tag] -o console.foo
# ./vdbench -f foo.vdb -o outdir.foo | vdbench_graphite.sh -h [graphite_host] -p [graphite_port] -t [graphite_tag] -n [vdbench_hostname] -o console.foo


while [[ $# > 1 ]]
Expand All @@ -24,7 +24,7 @@ key="$1"
case $key in
-h|--host)
graphite_host="$2"
shift
shift
;;
-p|--port)
graphite_port="$2"
Expand All @@ -34,43 +34,53 @@ case $key in
tag="$2"
shift
;;
-n|--name)
name=`echo "$2" | sed 's/\./_/g'`
;;
-o|--outfile)
outfile="$2"
;;
*)

;;
esac
shift
done

echo

# Assume graphite is running on localhost if not defined
# Assume graphite is running on localhost if not defined
if [ -z "$graphite_host" ] ; then
echo "Graphite host \"-h\" not defined, assuming localhost."
graphite_host="localhost"
fi

# Addume graphite is running on port 2003 if not defined
# Assume graphite is running on port 2003 if not defined
if [ -z "$graphite_port" ] ; then
echo "Graphite port \"-p\" not defined, assuming 2003."
echo "Graphite port \"-p\" not defined, assuming 2003."
graphite_port="2003"
fi

# Check to make sure we can talk to graphite before continuing
#nc -z $graphite_host $graphite_port
# Check to make sure we can talk to graphite before continuing
#nc -z $graphite_host $graphite_port

#if [ "$?" != "0" ]; then
# echo "Can't communicate with graphite at $graphite_host on TCP port $graphite_port."
# exit
#fi

# Assign a prefix based on tag
# Assign a prefix base using tag
if [ -z "$tag" ]; then
prefix=vdbench.default.`hostname -s`
else
prefix=vdbench.$tag.`hostname -s`
prefix_base=vdbench.default
else
prefix_base=vdbench.$tag
fi

# Assign a prefix based on name
if [ -z "$name" ]; then
prefix=$prefix_base.`hostname -s`
else
prefix=$prefix_base.${name}
fi

# If not outfile specified, kick overything to /dev/null
Expand Down Expand Up @@ -110,7 +120,7 @@ BEGIN {
/^\s*rate/ {
# Replace 1024**2 with sec so we can mash it from a header from the above section to get MB/sec
gsub(/1024\*\*2/,"sec",$0)

# Use gsub to remove an unwanted /
gsub(/\//,"",$0)

Expand All @@ -126,8 +136,8 @@ BEGIN {

title1_column++
title2_column++
title_column++
}
title_column++
}
}

# Search for any lines that do not have letters, assumed to be vdbench metrics
Expand Down Expand Up @@ -157,7 +167,7 @@ BEGIN {
print prefix"."title_array[title_column] " " stats_array[stats_column] " " epochtime
title_column++
stats_column++
}
}
}
}

Expand All @@ -170,3 +180,4 @@ wait

echo
echo "Done"