Cacti is a great tool for time-based visualization of data. Out-of-box functionality can leave something to be desired. Here is a stepwise tutorial for creating custom graphs of web server requests. Most of the instructions can be applied to other scenarios.

From a high-level you need to:

1. Create a source of the (time-based) data which exists independent of Cacti
2. Create a means of obtaining the data (Data Input Method)
3. Assemble outputs of (2) into a Data Template
4. Create Graph Template based on the values available in the Data Template
5. Apply those to a Device, this creates a Data Source and set of graphs

Then after some time passes you will have some nice graphs to ogle at.

A real world example would be capturing requests-per-second statistics from Nginx. This technique would apply to any web server logs with a little massaging I guess. In this case, our Nginx server was not compiled with stub_status ability which might otherwise be used, so instead we fashion a shell script that can run on (each) web server and determine rps from the access log.

statsnginx is a rudimentary script that captures average rps in ranges of 10s, 1m and 5m ago.

Example of running and the output:

$ /usr/local/bin/statsnginx \
/usr/local/nginx/logs/nginx.vhost.access.log
c10s:389 c1m:409 c5m:312


Next step, make this data available to cacti. I choose snmp MIB since snmpd is already running on our Nginx servers.
Just by adding this line to snmpd.conf and restarting can we see the data remotely (wrapped for clarity).
exec 1.3.6.1.4.1.5001.3 statsnginx \
/usr/local/bin/statsnginx \
/usr/local/nginx/logs/nginx.vhost.access.log


(note this exec syntax is broken after net-snmp-5.3.1 on FreeBSD anyway, so YMMV)


$ snmpget -v 1 -On -c public \
192.168.1.117:161 .1.3.6.1.4.1.5001.3.101.1
.1.3.6.1.4.1.5001.3.101.1 = STRING: "c10s:275 c1m:274 c5m:230"

This part can be tricky to figure out the exact OID. Use snmpwalk if necessary, e.g.
$ snmpwalk -v 1 -On -c public \
192.168.1.117:161 .1.3.6.1.4.1.5001.3


Create the cacti script in /opt/cacti/scripts/statsnginx.sh

#!/bin/sh
OUTPUT=`snmpget -Ov -v 2c -c bitpushsnmp $1 \
1.3.6.1.4.1.5001.3.101.1`
echo $OUTPUT | sed -e 's/STRING: //' | sed -e 's/"//g'


Test it from the cacti server.

$ ./statsnginx.sh 192.168.1.117
c10s:190 c1m:192 c5m:152


Notice how the output has three fields of name:value pairs separated by spaces. These are what cacti likes.
Don’t make the mistake like I did of using name=value because it will make cacti think it’s a PARTIAL result..

Now we can move into cacti to make use of the script and data it provides.

Please see this helpful link to Cacti docs.

Go to the Cacti console and create a Data Input Method to tell Cacti how to call the script …

1. Data Input methods
2. Add
3. Name: statsnginx
4. Input Type: Script/Command
5. Input String: <path_cacti>/scripts/statsnginx.sh <hostname>
6. …Save…

Now you have Input Fields and Output Fields

Cacti wants you to provide these.

Since we are gathering data from a remote (to cacti server) host, need to give hostname as input

1. Add an Input Field
2. Name: hostname
3. Field Order: 1
4. Friendly Name: Hostname

Add an Output Field for each of the name:value pairs above (c10s, c1m, c5m)
Example:

1. Field [Output]: c10s
2. Friendly Name: Average requests/sec over last 10 seconds
3. Update RRD File: checked
4. Do this for the other two fields.
5. Then Save once more

Now create the Data Template

1. Add
2. Data Template
3. - Name: nginx - Requests
4. Data Source
   • Name: |host_description| - nginx - Requests
   • Data Input Method: statsnginx (chosen from list)
5. Data Source Items
   • Internal Data Source Name: for each data point above (c60s, c1m, c5m) add all (output fields items from above)

This is a tricky part. You want all three data source items listed with appropriate min/max values (left at 0 in this case) and using GAUGE as the Data Source Type. Also make sure to select the appropriate Output Field from the list for each one.

Now create the Graph Template

1. Add
2. Template Name: nginx - Requests
3. Title: |host_description| - nginx - Requests
4. I changed Upper Limit to 10000 just to be sure.
5. ..Create…

Now add Graph Template Items one by one
e.g Data Source nginx - requests (c10s)
and so on. Give each one an appropriate type like area, stacked or line1.
You can use the Graph Template for ucd/net - Load Average as a reference since it has similar measures (1, 5 and 15m load average).

BitPusher is excited to be sponsoring the Six Hour Startup Conference being held on May 31st at the Columbia City Theatre in Seattle, WA. Six Hour Startup Conference is being put on by Six Hour Startups, an exciting local company dedicated towards social business collaboration and development.

Six Hour Startup Conference’s agenda includes discussions with a variety of startup experts, on topics including writing business plan writing, corporate legal structures, and developing investment oriented pitches. There will also be punch, and pie.

The event is from 9:30 AM until 5:20 PM. Registration is $99 + $2.48 processing fee from EventBrite. I’ll be attending as well, and hope to see you there! PS, I’m going to keep a backpack of t-shirts with me, if you’d like one, email me a size and look me up during lunch!

Come drink our Guinness! From 2:00 PM - 7:00 PM Thursday, May 15th, 2008 BitPusher’s staff will be on-site serving Guinness, and handing out T-Shirts to all of our friends and customers while celebrate our new managed colocation partnership with NetRiver.

NetRiver’s  datacenter is located at  4200 194th Street SW in Lynnwood, WA 98036.

Please RSVP by leaving a comment here. All are welcome!

View Larger Map

Linda, Kaylee, and Daniel at the after party

Originally uploaded by bitpusher

Last weekend we exhibited at LinuxFest NorthWest. It was pretty exciting, since this was the first time we’d actually had a booth and marketing schwag at an event. Hopefully it won’t be the last.

As these things go, the preparation was pretty intense. Between the time we signed up for our booth, and 10 days prior to the conference, we didn’t have much time to plan for the event. At T-10 days, we decided we wanted stickers and T-shirts.

With 8 days left, I called up Robert Kaule over at essensys, who also owns a silk screening business. I asked him to make it happen on time. Being an engineer first, and a screen-printer second, his response was “no problem, I’ll have to bill you for overnight delivery “.

(more…)

IMG_1583

Originally uploaded by bitpusher

Everybody, meet Ben. Ben, meet everybody!

Ben is BitPusher’s newest Sr. Systems Engineer. I just stole him from ClearWire and he starts this coming Monday. We’re all very excited to have him!