High5! Code

Commit Diff

Commit:: d9a3ff937b28b3aa077047a2a63c829e65b5d3b7
From:: mischa <mischa@rx.high5.nl>
Date:: Sun May 31 14:16:16 2020 UTC
Message:: updated README.html
Actions:: Patch | Tree
commit - 7c02cf83d68d74da6637dec4b5aa5e17dd4c38b2
commit + d9a3ff937b28b3aa077047a2a63c829e65b5d3b7
blob - accc3d4e805c6a62f6326571027815fc8ef17cc7
blob + a22d598bd6d922196619ff31e16cd2b1e78ae359
--- README.html
+++ README.html
@@ -1,14 +1,32 @@
 <!DOCTYPE html>
-<html lang="en" class="black bg-white sans-serif lh-copy">
-<title>RRDtool looks nicer... - High5!</title>
+<html lang="en" class="dark-gray bg-white sans-serif lh-copy">
+<title>RRDtool looks nicer... - OpenBSD Amsterdam</title>
 <meta charset="UTF-8">
-<body>
+<meta name="generator" content="romanzolotarev.com/ssg">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<meta name="discountegg" content="OBSD-EGG-2020">
+<link rel="icon" type="image/png" href="/favicon.png">
+<link rel="stylesheet" href="/style.css">
+<body class="mw7 w-100 center pa3">
+<div class="cf">
+<div class="fl w-100 w-50-l w-50-m"><a href="/" title="OpenBSD Amsterdam Logo"><img class="w3 br-100" src="/images/logo.png" alt="OpenBSD Amsterdam Logo"></a></div>
+<div class="fl w-100 w-50-l w-50-m mt2 tr"><a href="servers.html">364 VMs deployed</a>&nbsp;&#x1F421;</div>
+</div>
 <h1 id="RRDtool%20looks%20nicer...">RRDtool looks nicer...</h1>
 
-<p><a href="https://openbsd.amsterdam/">OpenBSD Amsterdam</a> was in search of a lightweight toolset to keep track of resource usage, at a minimum the CPU load generated by the <a href="https://man.openbsd.org/vmm.4">vmm(4)</a>/<a href="https://man.openbsd.org/vmd.8">vmd(8)</a> hosts and the traffic from and to the hosts. A couple of weeks ago we ended up with a workable [MRTG setup]. While it worked, it didn&#39;t look very pretty.</p>
+<p class="f7">March 14, 2019</p>
 
+<p><a href="https://openbsd.amsterdam/">OpenBSD Amsterdam</a> was in search of a lightweight toolset to keep track of resource usage, at a minimum the CPU load generated by the <a href="https://man.openbsd.org/vmm.4">vmm(4)</a>&#47;<a href="https://man.openbsd.org/vmd.8">vmd(8)</a> hosts and the traffic from and to the hosts. A couple of weeks ago we ended up with a workable <a href="https://openbsd.amsterdam/blog/using-mrtg-again.html">MRTG Setup</a>. While it worked, it didn&#39;t look very pretty.</p>
+
 <p>In a moment of clarity, we thought about using <a href="https://oss.oetiker.ch/rrdtool/">RRDtool</a>. Heck, why shouldn&#39;t we give it a try? From the previous tooling, we already had some required building blocks in place to make <a href="https://oss.oetiker.ch/mrtg/">MRTG</a> understand the CPU Cores and uptime from <a href="https://openbsd.org/">OpenBSD</a>.</p>
 
+<p>Before we start:</p>
+
+<pre>
+# <b>pkg_add rrdtool</b>
+#
+</pre>
+
 <p>We decided to split the collection of the different OIDs (SNMP Object Identifiers) into three different scripts, which <a href="https://man.openbsd.org/cron.1">cron(1)</a> calls, from a wrapper script.</p>
 
 <ul>
@@ -19,170 +37,207 @@
 
 <h3 id="uptime.sh">uptime.sh</h3>
 
-<pre><code>#!/bin/sh
-test -n &quot;$1&quot; || exit 1
-HOST=&quot;$1&quot;
-COMMUNITY=&quot;public&quot;
-UPTIMEINFO=&quot;/tmp/${HOST}-uptime.txt&quot;
+<pre>
+#!/bin/sh
+test -n "$1" || exit 1
+HOST="$1"
+COMMUNITY="public"
+UPTIMEINFO="/tmp/${HOST}-uptime.txt"
 TICKS=$(snmpctl snmp get ${HOST} community ${COMMUNITY} oid hrSystemUptime.0 | cut -d= -f2)
-DAYS=$(echo &quot;${TICKS}/8640000&quot; | bc -l)
-HOURS=$(echo &quot;0.${DAYS##*.} * 24&quot; | bc -l)
-MINUTES=$(echo &quot;0.${HOURS##*.} * 60&quot; | bc -l)
-SECS=$(echo &quot;0.${MINUTES##*.} * 60&quot; | bc -l)
-test -n &quot;$DAYS&quot; &amp;&amp; printf &#39;%s days, &#39; &quot;${DAYS%.*}&quot; &gt; ${UPTIMEINFO}
-printf &#39;%02d\\:%02d\\:%02d\n&#39; &quot;${HOURS%.*}&quot; &quot;${MINUTES%.*}&quot; &quot;${SECS%.*}&quot; &gt;&gt; ${UPTIMEINFO}
-</code></pre>
+DAYS=$(echo "${TICKS}/8640000" | bc -l)
+HOURS=$(echo "0.${DAYS##*.} * 24" | bc -l)
+MINUTES=$(echo "0.${HOURS##*.} * 60" | bc -l)
+SECS=$(echo "0.${MINUTES##*.} * 60" | bc -l)
+test -n "$DAYS" && printf '%s days, ' "${DAYS%.*}" > ${UPTIMEINFO}
+printf '%02d\\:%02d\\:%02d\n' "${HOURS%.*}" "${MINUTES%.*}" "${SECS%.*}" >> ${UPTIMEINFO}
+</pre>
 
 <p>This is a seperate script, due to the uptime usage of both hosts in both graphs. </p>
 
-<p>The origins for this script can be found detailled in our <a href="https://chargen.one/obsdams/using-mrtg-again">MRTG Setup</a>.</p>
+<p>The origins for this script can be found detailled in our <a href="https://openbsd.amsterdam/blog/using-mrtg-again.html">MRTG Setup</a>.</p>
 
 <h3 id="cpu_load.sh">cpu_load.sh</h3>
 
-<pre><code>test -n &quot;$1&quot; || exit 1
-HOST=&quot;$1&quot;
-COMMUNITY=&quot;public&quot;
-RRDFILES=&quot;/var/rrdtool&quot;
-IMAGES=&quot;/var/www/htdocs&quot;
-WATERMARK=&quot;OpenBSD Amsterdam - https://obsda.ms&quot;
-RRDTOOL=&quot;/usr/local/bin/rrdtool&quot;
-CPUINFO=&quot;/tmp/${HOST}-cpu.txt&quot;
+<pre>
+test -n "$1" || exit 1
+HOST="$1"
+COMMUNITY="public"
+RRDFILES="/var/rrdtool"
+IMAGES="/var/www/htdocs"
+WATERMARK="OpenBSD Amsterdam - https://obsda.ms"
+RRDTOOL="/usr/local/bin/rrdtool"
+CPUINFO="/tmp/${HOST}-cpu.txt"
 UPTIME=$(cat /tmp/${HOST}-uptime.txt)
-NOW=$(date &quot;+%Y-%m-%d %H:%M:%S %Z&quot; | sed &#39;s/:/\\:/g&#39;)
+NOW=$(date "+%Y-%m-%d %H:%M:%S %Z" | sed 's/:/\\:/g')
 
-if ! test -f &quot;${RRDFILES}/${HOST}-cpu.rrd&quot;
+if ! test -f "${RRDFILES}/${HOST}-cpu.rrd"
 then
-echo &quot;Creating ${RRDFILES}/${HOST}-cpu.rrd&quot;
+echo "Creating ${RRDFILES}/${HOST}-cpu.rrd"
 ${RRDTOOL} create ${RRDFILES}/${HOST}-cpu.rrd \
-    --step 300 \
-    DS:ds0:GAUGE:600:U:U \
-    RRA:MAX:0.5:1:20000
+        --step 300 \
+        DS:ds0:GAUGE:600:U:U \
+        RRA:MAX:0.5:1:20000
 fi
 
-snmpctl snmp walk ${HOST} community ${COMMUNITY} oid hrProcessorLoad | cut -d= -f2 &gt; ${CPUINFO}
-CORES=$(grep -cv &quot;^0$&quot; ${CPUINFO})
-CPU_LOAD_SUM=$(awk &#39;{sum += $1} END {print sum}&#39; ${CPUINFO})
-CPU_LOAD=$(echo &quot;scale=2; ${CPU_LOAD_SUM}/${CORES}&quot; | bc -l)
+snmpctl snmp walk ${HOST} community ${COMMUNITY} oid hrProcessorLoad | cut -d= -f2 > ${CPUINFO}
+CORES=$(grep -cv "^0$" ${CPUINFO})
+CPU_LOAD_SUM=$(awk '{sum += $1} END {print sum}' ${CPUINFO})
+CPU_LOAD=$(echo "scale=2; ${CPU_LOAD_SUM}/${CORES}" | bc -l)
 
 ${RRDTOOL} update ${RRDFILES}/${HOST}-cpu.rrd N:${CPU_LOAD}
 
 ${RRDTOOL} graph ${IMAGES}/${HOST}-cpu.png \
-    --start -43200 \
-    --title &quot;${HOST} - CPU&quot; \
-    --vertical-label &quot;% CPU Used&quot; \
-    --watermark &quot;${WATERMARK}&quot; \
-    DEF:CPU=${RRDFILES}/${HOST}-cpu.rrd:ds0:AVERAGE \
-    AREA:CPU#FFCC00 \
-    LINE2:CPU#CC0033:&quot;CPU&quot; \
-    GPRINT:CPU:MAX:&quot;Max\:%2.2lf %s&quot; \
-    GPRINT:CPU:AVERAGE:&quot;Average\:%2.2lf %s&quot; \
-    GPRINT:CPU:LAST:&quot; Current\:%2.2lf %s\n&quot; \
-    COMMENT:&quot;\\n&quot; \
-    COMMENT:&quot;  SUM CPU Load / Active Cores = % CPU Used\n&quot; \
-    COMMENT:&quot;  Up for ${UPTIME} at ${NOW}&quot;
-</code></pre>
+        --start -43200 \
+        --title "${HOST} - CPU" \
+        --vertical-label "% CPU Used" \
+        --watermark "${WATERMARK}" \
+        DEF:CPU=${RRDFILES}/${HOST}-cpu.rrd:ds0:AVERAGE \
+        AREA:CPU#FFCC00 \
+        LINE2:CPU#CC0033:"CPU" \
+        GPRINT:CPU:MAX:"Max\:%2.2lf %s" \
+        GPRINT:CPU:AVERAGE:"Average\:%2.2lf %s" \
+        GPRINT:CPU:LAST:" Current\:%2.2lf %s\n" \
+        COMMENT:"\\n" \
+        COMMENT:"  SUM CPU Load / Active Cores = % CPU Used\n" \
+        COMMENT:"  Up for ${UPTIME} at ${NOW}"
+</pre>
 
 <p>On the first run, <a href="https://oss.oetiker.ch/rrdtool/">RRDtool</a> will create the .rrd file. On every subsequent run, it will update the file with the collected values and update the graph.</p>
 
-<p>The origins for this script can be found detailed in our <a href="https://chargen.one/obsdams/using-mrtg-again">MRTG Setup</a>.</p>
+<p>The origins for this script can be found detailled in our <a href="https://openbsd.amsterdam/blog/using-mrtg-again.html">MRTG Setup</a>.</p>
 
 <h3 id="interface.sh">interface.sh</h3>
 
-<pre><code>test -n &quot;$1&quot; || exit 1                                                                             
-test -n &quot;$2&quot; || exit 1                                                                             
-HOST=&quot;$1&quot;                                                                                          
-INTERFACE=&quot;$2&quot;                                                                                     
-COMMUNITY=&quot;public&quot;                                                                                 
-RRDFILES=&quot;/var/rrdtool&quot;
-IMAGES=&quot;/var/www/htdocs&quot;
-WATERMARK=&quot;OpenBSD Amsterdam - https://obsda.ms&quot;
-RRDTOOL=&quot;/usr/local/bin/rrdtool&quot;
+<pre>
+test -n "$1" || exit 1                                                                             
+test -n "$2" || exit 1                                                                             
+HOST="$1"                                                                                          
+INTERFACE="$2"                                                                                     
+COMMUNITY="public"                                                                                 
+RRDFILES="/var/rrdtool"
+IMAGES="/var/www/htdocs"
+WATERMARK="OpenBSD Amsterdam - https://obsda.ms"
+RRDTOOL="/usr/local/bin/rrdtool"
 UPTIME=$(cat /tmp/${HOST}-uptime.txt)
-NOW=$(date &quot;+%Y-%m-%d %H:%M:%S %Z&quot; | sed &#39;s/:/\\:/g&#39;)                                              
+NOW=$(date "+%Y-%m-%d %H:%M:%S %Z" | sed 's/:/\\:/g')                                              
 
-if ! test -f &quot;${RRDFILES}/${HOST}-${INTERFACE}.rrd&quot;                                                
+if ! test -f "${RRDFILES}/${HOST}-${INTERFACE}.rrd"                                                
 then
-echo &quot;Creating ${RRDFILES}/${HOST}-${INTERFACE}.rrd&quot;                                               
+echo "Creating ${RRDFILES}/${HOST}-${INTERFACE}.rrd"                                               
 ${RRDTOOL} create ${RRDFILES}/${HOST}-${INTERFACE}.rrd \                                           
-    --step 300 \
-    DS:ds0:COUNTER:600:0:1250000000 \
-    DS:ds1:COUNTER:600:0:1250000000  \
-    RRA:AVERAGE:0.5:1:600 \
-    RRA:AVERAGE:0.5:6:700 \
-    RRA:AVERAGE:0.5:24:775 \
-    RRA:AVERAGE:0.5:288:797 \
-    RRA:MAX:0.5:1:600 \
-    RRA:MAX:0.5:6:700 \
-    RRA:MAX:0.5:24:775 \
-    RRA:MAX:0.5:288:797
+        --step 300 \
+        DS:ds0:COUNTER:600:0:1250000000 \
+        DS:ds1:COUNTER:600:0:1250000000  \
+        RRA:AVERAGE:0.5:1:600 \
+        RRA:AVERAGE:0.5:6:700 \
+        RRA:AVERAGE:0.5:24:775 \
+        RRA:AVERAGE:0.5:288:797 \
+        RRA:MAX:0.5:1:600 \
+        RRA:MAX:0.5:6:700 \
+        RRA:MAX:0.5:24:775 \
+        RRA:MAX:0.5:288:797
 fi
 
 IN=$(snmpctl snmp get ${HOST} community ${COMMUNITY} oid ifInOctets.${INTERFACE} | cut -d= -f2)    
 OUT=$(snmpctl snmp get ${HOST} community ${COMMUNITY} oid ifOutOctets.${INTERFACE} | cut -d= -f2)  
 DESCR=$(snmpctl snmp get ${HOST} community ${COMMUNITY} oid ifDescr.${INTERFACE} | cut -d= -f2 | tr
--d &#39;&quot;&#39;)
+-d '"')
 
 ${RRDTOOL} update ${RRDFILES}/${HOST}-${INTERFACE}.rrd N:${IN}:${OUT}                              
 
 ${RRDTOOL} graph ${IMAGES}/${HOST}-${INTERFACE}.png \                                              
-    --start -43200 \
-    --title &quot;${HOST} - ${DESCR}&quot; \
-    --vertical-label &quot;Bits per Second&quot; \
-    --watermark &quot;${WATERMARK}&quot; \
-    DEF:IN=${RRDFILES}/${HOST}-${INTERFACE}.rrd:ds0:AVERAGE \                                  
-    DEF:OUT=${RRDFILES}/${HOST}-${INTERFACE}.rrd:ds1:AVERAGE \                                 
-    CDEF:IN_CDEF=&quot;IN,8,*&quot; \
-    CDEF:OUT_CDEF=&quot;OUT,8,*&quot; \
-    AREA:IN_CDEF#00FF00:&quot;In &quot; \
-    GPRINT:IN_CDEF:MAX:&quot;Max\:%5.2lf %s&quot; \
-    GPRINT:IN_CDEF:AVERAGE:&quot;Average\:%5.2lf %s&quot; \                                              
-    GPRINT:IN_CDEF:LAST:&quot; Current\:%5.2lf %s\n&quot; \                                              
-    LINE2:OUT_CDEF#0000FF:&quot;Out&quot; \
-    GPRINT:OUT_CDEF:MAX:&quot;Max\:%5.2lf %s&quot; \
-    GPRINT:OUT_CDEF:AVERAGE:&quot;Average\:%5.2lf %s&quot; \                                             
-    GPRINT:OUT_CDEF:LAST:&quot; Current\:%5.2lf %s\n&quot; \                                             
-    COMMENT:&quot;\\n&quot; \
-    COMMENT:&quot;  Up for ${UPTIME} at ${NOW}&quot;
-</code></pre>
+        --start -43200 \
+        --title "${HOST} - ${DESCR}" \
+        --vertical-label "Bits per Second" \
+        --watermark "${WATERMARK}" \
+        DEF:IN=${RRDFILES}/${HOST}-${INTERFACE}.rrd:ds0:AVERAGE \                                  
+        DEF:OUT=${RRDFILES}/${HOST}-${INTERFACE}.rrd:ds1:AVERAGE \                                 
+        CDEF:IN_CDEF="IN,8,*" \
+        CDEF:OUT_CDEF="OUT,8,*" \
+        AREA:IN_CDEF#00FF00:"In " \
+        GPRINT:IN_CDEF:MAX:"Max\:%5.2lf %s" \
+        GPRINT:IN_CDEF:AVERAGE:"Average\:%5.2lf %s" \                                              
+        GPRINT:IN_CDEF:LAST:" Current\:%5.2lf %s\n" \                                              
+        LINE2:OUT_CDEF#0000FF:"Out" \
+        GPRINT:OUT_CDEF:MAX:"Max\:%5.2lf %s" \
+        GPRINT:OUT_CDEF:AVERAGE:"Average\:%5.2lf %s" \                                             
+        GPRINT:OUT_CDEF:LAST:" Current\:%5.2lf %s\n" \                                             
+        COMMENT:"\\n" \
+        COMMENT:"  Up for ${UPTIME} at ${NOW}"
+</pre>
 
 <p>To pinpoint the network interface you want to measure the bandwith for, this command prints the available interfaces:</p>
 
-<pre><code>snmpctl snmp walk [host] community [community] oid ifDescr
-</code></pre>
+<pre>
+snmpctl snmp walk [host] community [community] oid ifDescr
+</pre>
 
 <p>This will output a list like:</p>
 
-<pre><code>ifDescr.1=&quot;em0&quot;
-ifDescr.2=&quot;em1&quot;
-ifDescr.3=&quot;enc0&quot;
-ifDescr.4=&quot;lo0&quot;
-ifDescr.5=&quot;bridge880&quot;
-ifDescr.6=&quot;vlan880&quot;
-ifDescr.13=&quot;pflog0&quot;
-ifDescr.669=&quot;tap0&quot;
-ifDescr.670=&quot;tap1&quot;
-</code></pre>
+<pre>
+ifDescr.1="em0"
+ifDescr.2="em1"
+ifDescr.3="enc0"
+ifDescr.4="lo0"
+ifDescr.5="bridge880"
+ifDescr.6="vlan880"
+ifDescr.13="pflog0"
+ifDescr.669="tap0"
+ifDescr.670="tap1"
+</pre>
 
 <p>The number behind <code>ifDescr</code> is the one that you need to feed to <strong>interface.sh</strong>, for example:</p>
 
-<pre><code># interface.sh 5
-</code></pre>
+<pre>
+# interface.sh 5
+</pre>
 
 <p>Finally the <strong>wrapper.sh</strong> script calls all the aforementioned scripts:</p>
 
-<pre><code>#!/bin/sh
-SCRIPTS=&quot;/var/rrdtool&quot;
+<pre>
+#!/bin/sh
+SCRIPTS="/var/rrdtool"
 for i in $(jot 2 1); do ${SCRIPTS}/uptime.sh host${i}.domain.tld; done
 for i in $(jot 2 1); do ${SCRIPTS}/cpu_load.sh host${i}.domain.tld; done
 ${SCRIPTS}/interface.sh host1.domain.tld 12
 ${SCRIPTS}/interface.sh host2.domain.tld 11
-</code></pre>
+</pre>
 
 <p>The resulting graphs:</p>
 
 <p><img src="https://ops.high5.nl/obsda.ms/s1.obsda.ms-cpu.png" alt="" />
 <img src="https://ops.high5.nl/obsda.ms/s1.obsda.ms-12.png" alt="" /></p>
 
+<p>To serve the graphs we use <a href="http://man.openbsd.org/httpd.8">httpd(8)</a> with the following config:
+<pre>
+server &#34;default&#34; {
+        listen on * port 80
+        location &#34;&#47;.well-known&#47;acme-challenge&#47;*&#34; {
+                root &#34;&#47;acme&#34;
+                request strip 2
+        }
+        location * {
+                block return 302 &#34;https:&#47;&#47;$HTTP<em>HOST$REQUEST</em>URI&#34;
+        }
+}</p>
+
+<p>server &#34;default&#34; {
+        listen on * tls port 443
+        tls {
+                certificate &#34;&#47;etc&#47;ssl&#47;default-fullchain.pem&#34;
+                key &#34;&#47;etc&#47;ssl&#47;private&#47;default.key&#34;
+        }
+        location &#34;&#47;.well-known&#47;acme-challenge&#47;*&#34; {
+                root &#34;&#47;acme&#34;
+                request strip 2
+        }
+        root &#34;&#47;htdocs&#34;
+}
+</pre>
+All the scripts can be found in our <a href="https://git.high5.nl/rrdtool">Git Repository</a>.</p>
+
 <p>You can follow us on <a href="https://twitter.com/@OpenBSDAms">Twitter</a> and <a href="https://bsd.network/@OpenBSDAms">Mastodon</a>.</p>
+<p class="footer f7 mt4">
+&copy; 2018&ndash;2020 OpenBSD Amsterdam is a project of <a href="https://high5.nl/">High5!</a>
 </body>
 </html>