Drunken Dionysus the songwriting magus - Feeling like an accomplished geek

Today's work, now that I've finally ironed out problems with $big_company's mail server (it seems an increase in spam to catchalls had caused one of the lower end servers in our network to buckle under load), centred around some of my favourite topics - documentation of automated processes, and new learning. I wrote up a troubleshooting document on using Cygwin binaries on a Windows based mail servers to analyse log files to find mail loops, bounce backs from unpruned mailing lists and the like. That was fun.

The new learning centered around SNMP and the SMI syntax for MIBs. There's virtually zero documentation, outside of RFC's and reading the MIB's themselves, that I've found online for the format, and so I basically had to learn from other people's MIB's and the RFC's. Out of frustration when the MIB didn't work I eventually had to download libsmi, which contains smlint.exe, to verify the MIB, to recognize all my various typos, misunderstandings of the MIB syntax, and so on. I rewrote the OIDs for the postfix queue table which I had added into our vestigial MIB so that it satisfied the rather arbitrary syntax/grammar rules that SMI v2 seems to entail. So now I have my own fully working MIB for $big_company! One which I can add to as I desire as our monitoring techniques evolve. I basically chose a random OID under enterprises for the root node since we still haven't submitted to IANA to have our own enterprise OID.

Now all that remains to do tomorrow is to become more familiar with the format of getnext requests in existing MIB's, and update my shell script that handles the pass_persist to handle the new table structure for the reporting on the various queues. From there, reporting via set requests followed by subsequent get requests can handle figuring out general errors in bounced mail without needing to log into the boxes. This makes it much easier to automate problem discovery without writing an entire chat script to orchestrate data acquisition from several frontend boxes.

This makes me happy. There are a lot of ways in which SNMP and get/set/traps could be useful for us in addition to the existent monitoring system, and homebread automation has always been my sweetspot. I'm looking forward to seeing how this evolves at work.