The last time I upgraded Net-SNMP it wasn’t reporting the hrSystemProcesses OID. I wrote about that here. This time around I’ve upgraded to v5.7 and discovered two issues so far.
I was able to reproduce both of these issues on OpenBSD/i386 5.0.
UPDATE 2011-11-29: Net-SNMP 5.7.1 + fixes has been pushed to the OpenBSD 5.0-stable ports collection as well as -current ports. OpenBSD users should sync their ports tree and rebuild ports/net/net-snmp to alleviate the issues below.
#1 – Values returned by hrStorageUsed are incorrect
The first is with respect to the values returned via hrStorageUsed in the HOST-RESOURCES-MIB. The numbers are way off. For example, prior to the upgrade a disk that reported hrStorageUsed of 1873784 units (3.8GB) now reports 23920757 units (49GB). The Net-SNMP git repo shows there’s been some changes since v5.6.1 in the code that gathers disk stats. Looks like a bug was introduced somewhere.
Solution: Use Net-SNMP v5.7.1. The hrStorageUsed values in v5.7.1 are consistent with the values returned from v5.6.x. OpenBSD users, the net/net-snmp port in CVS HEAD is already updated to 5.7.1. Sync up and go.
#2 – Segfault when querying hrStorageTable
The second issue is a segfault in snmpd when querying the hrStorageTable. The segfault is happening on the memcpy() call(s) in netsnmp_fsys_arch_load(). The segfault is inconsistent. Sometimes you can get a handful of good queries and responses before it happens. Other times it happens on the first query. I don’t know why the segfault is happening, but I know that replacing memcpy() with strncpy() eliminates it. This is pretty hackish but I’m not skilled enough to debug this any further.
Solution: Apply the patch here. On OpenBSD you should be able to save the patch into ports/net/net-snmp/patches/ and rebuild the port. You will need this on Net-SNMP v5.7 and 5.7.1.
Am I the only one?
Are other Net-SNMP users seeing these issues? Does anyone even use Net-SNMP anymore (particularly on OpenBSD)? Please leave a comment and let me know.
I’m crossing my fingers that I won’t have to write another patch file the next time I upgrade Net-SNMP.
I confirm these bugs on OpenBSD 5.0/i386 with net-snmp-5.7p1 package (SNMP monitoring for my OpenBSD firewall @home) :(
I will test Net-SNMP v5.7.1 and apply patch/recompile package ASAP. Thanks for analyze and debug of these issues.
Thanks for commenting. Please follow up with the results of your testing.
i recompiled net-snmp v.5.7.1 package on OpenBSD 5.0/i386 and apply patch for hrStorageTable.
The both issues described are fixed :)
fox@embryo:~$ snmpwalk -v2c -c public 172.16.2.50 hrStorage
HOST-RESOURCES-MIB::hrMemorySize.0 = INTEGER: 261628 KBytes
HOST-RESOURCES-MIB::hrStorageIndex.1 = INTEGER: 1
HOST-RESOURCES-MIB::hrStorageIndex.2 = INTEGER: 2
HOST-RESOURCES-MIB::hrStorageIndex.10 = INTEGER: 10
HOST-RESOURCES-MIB::hrStorageIndex.31 = INTEGER: 31
HOST-RESOURCES-MIB::hrStorageIndex.32 = INTEGER: 32
HOST-RESOURCES-MIB::hrStorageIndex.33 = INTEGER: 33
HOST-RESOURCES-MIB::hrStorageIndex.34 = INTEGER: 34
HOST-RESOURCES-MIB::hrStorageIndex.35 = INTEGER: 35
HOST-RESOURCES-MIB::hrStorageType.1 = OID: HOST-RESOURCES-TYPES::hrStorageRam
HOST-RESOURCES-MIB::hrStorageType.2 = OID: HOST-RESOURCES-TYPES::hrStorageRam
HOST-RESOURCES-MIB::hrStorageType.10 = OID: HOST-RESOURCES-TYPES::hrStorageVirtualMemory
HOST-RESOURCES-MIB::hrStorageType.31 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageType.32 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageType.33 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageType.34 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageType.35 = OID: HOST-RESOURCES-TYPES::hrStorageFixedDisk
HOST-RESOURCES-MIB::hrStorageDescr.1 = STRING: Physical memory
HOST-RESOURCES-MIB::hrStorageDescr.2 = STRING: Real memory
HOST-RESOURCES-MIB::hrStorageDescr.10 = STRING: Swap space
HOST-RESOURCES-MIB::hrStorageDescr.31 = STRING: /
HOST-RESOURCES-MIB::hrStorageDescr.32 = STRING: /home
HOST-RESOURCES-MIB::hrStorageDescr.33 = STRING: /tmp
HOST-RESOURCES-MIB::hrStorageDescr.34 = STRING: /usr
HOST-RESOURCES-MIB::hrStorageDescr.35 = STRING: /var
HOST-RESOURCES-MIB::hrStorageAllocationUnits.1 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.2 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.10 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.31 = INTEGER: 2048 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.32 = INTEGER: 2048 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.33 = INTEGER: 2048 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.34 = INTEGER: 2048 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.35 = INTEGER: 2048 Bytes
HOST-RESOURCES-MIB::hrStorageSize.1 = INTEGER: 65407
HOST-RESOURCES-MIB::hrStorageSize.2 = INTEGER: 65406
HOST-RESOURCES-MIB::hrStorageSize.10 = INTEGER: 132535
HOST-RESOURCES-MIB::hrStorageSize.31 = INTEGER: 252919
HOST-RESOURCES-MIB::hrStorageSize.32 = INTEGER: 1564183
HOST-RESOURCES-MIB::hrStorageSize.33 = INTEGER: 251943
HOST-RESOURCES-MIB::hrStorageSize.34 = INTEGER: 1544103
HOST-RESOURCES-MIB::hrStorageSize.35 = INTEGER: 248983
(…)
=> no segfault !
My firewall SNMP monitoring is functionnal now with the updated Net-SNMP daemon.
Thanks for your work.
Just getting into snmp on OpenBSD and appreciate your continuing documentation work.
Thanks
You are welcome!
Hi Joel,
Do you had any experience configuring OpenBSD snmpd.conf to get external data? I’m trying to get unbound statistic on a OpenBSD server, graphing it into cacti, and I’m trying to avoid installing net-snmp on my unbound server.
Thanks.
Hi, I tried once to forward from snmpd to net-snmp but it didn’t work. I can’t remember any of the details but remember giving up and just running net-snmp directly. Sorry that’s not very helpful.
I should’ve posted this here earlier but forgot: I’ve written a post about getting net-snmp and snmpd to work together. This is working quite well for me in instances where I have to pull in external data.
http://www.packetmischief.ca/2012/02/26/net-snmp-and-snmpd-coexistence-on-openbsd/
There are some problems with the last version of Net-SNMP.
After applying your patch (thanks for it), I have a strange behavior, some queries swap the value between used and free (/home, /var).
Have you experimenting some errors like that ?
Hi Brice,
I’m not seeing that here, no. Is this still on Net-SNMP 5.7.1? What version of OpenBSD? Can you show the actual data vs. what you expect it to be?
Hi Joel,
Sorry for the late response.
For example, the partition /var/www is “empty” but a walk on the disk data, show me
HOST-RESOURCES-MIB::hrStorageSize.36 = INTEGER: 2066049
HOST-RESOURCES-MIB::hrStorageUsed.36 = INTEGER: 2058339
but if I use df, it gives me the following output:
/dev/wd1b 3.9G 15.1M 3.7G 0% /var/www
Any idea ?
— Output —
HOST-RESOURCES-MIB::hrStorageDescr.1 = STRING: Physical memory
HOST-RESOURCES-MIB::hrStorageDescr.2 = STRING: Real memory
HOST-RESOURCES-MIB::hrStorageDescr.10 = STRING: Swap space
HOST-RESOURCES-MIB::hrStorageDescr.11 = STRING: swap /dev/wd0b
HOST-RESOURCES-MIB::hrStorageDescr.12 = STRING: swap /dev/wd1e
HOST-RESOURCES-MIB::hrStorageDescr.31 = STRING: /
HOST-RESOURCES-MIB::hrStorageDescr.32 = STRING: /home
HOST-RESOURCES-MIB::hrStorageDescr.33 = STRING: /tmp
HOST-RESOURCES-MIB::hrStorageDescr.34 = STRING: /usr
HOST-RESOURCES-MIB::hrStorageDescr.35 = STRING: /var
HOST-RESOURCES-MIB::hrStorageDescr.36 = STRING: /var/www
HOST-RESOURCES-MIB::hrStorageAllocationUnits.1 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.2 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.10 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.11 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.12 = INTEGER: 4096 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.31 = INTEGER: 2048 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.32 = INTEGER: 2048 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.33 = INTEGER: 2048 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.34 = INTEGER: 2048 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.35 = INTEGER: 2048 Bytes
HOST-RESOURCES-MIB::hrStorageAllocationUnits.36 = INTEGER: 2048 Bytes
HOST-RESOURCES-MIB::hrStorageSize.1 = INTEGER: 130959
HOST-RESOURCES-MIB::hrStorageSize.2 = INTEGER: 130958
HOST-RESOURCES-MIB::hrStorageSize.10 = INTEGER: 329590
HOST-RESOURCES-MIB::hrStorageSize.11 = INTEGER: 532224
HOST-RESOURCES-MIB::hrStorageSize.12 = INTEGER: 2104515
HOST-RESOURCES-MIB::hrStorageSize.31 = INTEGER: 65103
HOST-RESOURCES-MIB::hrStorageSize.32 = INTEGER: 235414670
HOST-RESOURCES-MIB::hrStorageSize.33 = INTEGER: 259823
HOST-RESOURCES-MIB::hrStorageSize.34 = INTEGER: 1548247
HOST-RESOURCES-MIB::hrStorageSize.35 = INTEGER: 2066034
HOST-RESOURCES-MIB::hrStorageSize.36 = INTEGER: 2066049
HOST-RESOURCES-MIB::hrStorageUsed.1 = INTEGER: 83670
HOST-RESOURCES-MIB::hrStorageUsed.2 = INTEGER: 83669
HOST-RESOURCES-MIB::hrStorageUsed.10 = INTEGER: 0
HOST-RESOURCES-MIB::hrStorageUsed.11 = INTEGER: 0
HOST-RESOURCES-MIB::hrStorageUsed.12 = INTEGER: 0
HOST-RESOURCES-MIB::hrStorageUsed.31 = INTEGER: 31170
HOST-RESOURCES-MIB::hrStorageUsed.32 = INTEGER: 217362547
HOST-RESOURCES-MIB::hrStorageUsed.33 = INTEGER: 259815
HOST-RESOURCES-MIB::hrStorageUsed.34 = INTEGER: 907066
HOST-RESOURCES-MIB::hrStorageUsed.35 = INTEGER: 2050639
HOST-RESOURCES-MIB::hrStorageUsed.36 = INTEGER: 2058339
Filesystem Size Used Avail Capacity Mounted on
/dev/wd0a 127M 66.3M 54.5M 55% /
/dev/wd1f 449G 34.4G 392G 8% /home
/dev/wd1d 508M 16.0K 482M 0% /tmp
/dev/wd0d 3.0G 1.2G 1.6G 44% /usr
/dev/wd1a 3.9G 30.1M 3.7G 1% /var
/dev/wd1b 3.9G 15.1M 3.7G 0% /var/www
visualstation@Charon:ttyp0 ~ % snmpwalk -V
NET-SNMP version: 5.7
Hey Brice, you’re running the version that has issues with disk utilization. Check out the blog post we’re commenting in and you’ll see some examples. Upgrade to Net-SNMP 5.7.1 and you’ll be good to go.
Damn, my bad.
I’ll move to 5.1 and check if the version is the 5.7.1.
Hey Joel
Thanks for this input here.
I’m having some novice trouble.I have managed to use the ports tree and etc.BUT i can’t figure out how to apply the patch available here , could you point me the way please ? Thank You
Hi Marcio,
The easiest way to get the Net-SNMP patches is to upgrade your ports tree to what’s called the -stable branch. There’s instructions in the OpenBSD FAQ for how to do that (http://www.openbsd.org/faq/faq15.html#PortsSecurity). Once you’ve pulled down the -stable version, build and install the port like normal.