Pythian Blog: Technical Track

Exporting custom metrics to influxdb

by Minto Joseph on Jan 5, 2018 12:00:00 AM

In the TICK stack from influxdata , Telegraf is used to collect and send data to influxdb. Metrics in Influxdb can then be viewed using visualization tools like Grafana. There are several input and output plugins for telegraf, which can be included in the /etc/telegraf/telegraf.conf file to support wide range of metric sources and destinations. For example, adding swap plugin like below would export swapping information from the host along with a tag name metrics_source. [code] [[inputs.swap]] [inputs.swap.tags] metrics_source="telegraf_demo" [/code] To demonstrate this, let us use file output plugin with influx as dataformat. [code] [[outputs.file]] files = ["stdout", "/tmp/metrics.out"] data_format = "influx" [/code] In this post we will focus on exporting metrics from a container host using telegraf container. Lets create a Dockerfile to include the telegraf.conf file.. [code] FROM telegraf COPY telegraf.conf /etc/telegraf/telegraf.conf CMD ["telegraf"][/code] build the container.. # docker build -t telegrafdemo .</code? and run it in foreground, so that we can the see the output plugin in work. # docker run --name telegrafdemo telegrafdemo [..] swap,metrics_source=telegraf_demo,host=e9e7086036b8 total=8271163392i,used=0i,free=8271163392i,used_percent=0 1514876555000000000 swap,metrics_source=telegraf_demo,host=e9e7086036b8 out=0i,in=0i 1514876555000000000 swap,host=e9e7086036b8,metrics_source=telegraf_demo used_percent=0,total=8271163392i,used=0i,free=8271163392i 1514876560000000000 swap,metrics_source=telegraf_demo,host=e9e7086036b8 in=0i,out=0i 1514876560000000000 swap,metrics_source=telegraf_demo,host=e9e7086036b8 used=0i,free=8271163392i,used_percent=0,total=8271163392i 1514876565000000000 swap,metrics_source=telegraf_demo,host=e9e7086036b8 out=0i,in=0i 1514876565000000000 If existing plugins are not suitable for the job, a new plugin can be written Another option is to use exec plugin to execute a program to collect metrics in any of the accepted input data formats. For example, following script parse through the output of uptime command and [code language="bash"] #!/bin/sh hostname=`hostname` uptime=`awk '{print $1}' /proc/uptime` if uptime |grep -q user ; then load1=`uptime | grep -ohe 'up .*' | sed 's/,//g' | awk '{ print $7}'` load5=`uptime | grep -ohe 'up .*' | sed 's/,//g' | awk '{ print $8}'` load15=`uptime | grep -ohe 'up .*' | sed 's/,//g' | awk '{ print $9}'` else load1=`uptime | grep -ohe 'up .*' | sed 's/,//g' | awk '{ print $5}'` load5=`uptime | grep -ohe 'up .*' | sed 's/,//g' | awk '{ print $6}'` load15=`uptime | grep -ohe 'up .*' | sed 's/,//g' | awk '{ print $7}'` fi echo "uptime,host=$hostname uptime=$uptime,load1=$load1,load5=$load5,load15=$load15" [/code] The script output value is echoed in InfluxDB’s Line Protocol format. # sh uptime.sh uptime,host=localhost.localdomain uptime=25524.65,load1=1.14,load5=1.06,load15=1.04 Configuration looks like following. In this example influxdb output plugin is used. [code] [agent] interval = "5s" round_interval = true [[inputs.swap]] [inputs.swap.tags] metrics_source="telegraf_demo" [[inputs.exec]] commands = ["/etc/telegraf/uptime.sh"] data_format = "influx" [inputs.exec.tags] metrics_source="telegraf_demo" [[outputs.influxdb]] url = "https://influxdemo:8086" database = "telegraf" [/code] Script file is included in the Dockerfile. [code] FROM telegraf COPY telegraf.conf /etc/telegraf/telegraf.conf COPY uptime.sh /etc/telegraf/uptime.sh RUN chmod +x /etc/telegraf/uptime.sh CMD ["telegraf"] [/code] To ensure connection between the containers lets setup a docker network, # docker network create --driver=bridge mintonw # docker network ls NETWORK ID NAME DRIVER SCOPE 7674c3097332 bridge bridge local d42af7253e0f host host local fd11e0df5eb6 mintonw bridge local Start an influxb container in above network. # docker run --net=mintonw --name influxdemo influxdb In the directory with new telegraf.conf and Dockerfile, build the container and then run the container in docker network, # docker build -t telegrafdemo . # docker run --net=mintonw --name telegrafdemo telegrafdemo Connect to influxdb container, # docker exec -ti 02bb869bf606 /bin/bash bash-4.3# influx host localhost Connected to https://localhost:8086 version 1.2.4 InfluxDB shell version: 1.2.4 and verify that the data stored properly in influxdb. > show databases name: databases name ---- _internal telegraf > use telegraf Using database telegraf > show measurements name: measurements name ---- swap uptime > select * from uptime name: uptime time host load1 load15 load5 metrics_source uptime ---- ---- ----- ------ ----- -------------- ------ 1514874715000000000 3595f01eba01 0.26 0.66 0.54 telegraf_demo 13198.45 1514874720000000000 3595f01eba01 0.24 0.65 0.53 telegraf_demo 13203.45 1514874725000000000 3595f01eba01 0.22 0.65 0.52 telegraf_demo 13208.45 1514874730000000000 3595f01eba01 0.2 0.65 0.52 telegraf_demo 13213.45 1514874735000000000 3595f01eba01 0.18 0.64 0.51 telegraf_demo 13218.45 1514874740000000000 3595f01eba01 0.17 0.64 0.5 telegraf_demo 13223.45 1514874745000000000 3595f01eba01 0.15 0.63 0.49 telegraf_demo 13228.45 As demonstrated above, custom metrics can be exported easily using telegraf exec plugins to influxdb. P.S The functionality explained here is only for demonstration purpose. Metrics from uptime command can be exported using an existing system input plugin, as long as /var/run/utmp file from host is available in the container.
Topics: MySQL Mongodb Technical Track Open Source Replication Oplog

No Comments Yet

Let us know what you think

See all

See all

Subscribe by email
Follow us
Love Your Data
Pythian Logo

© 2024 All rights reserved.