Forwarding NetApp Event Log Entries via Syslog

Article created 2008-09-22 by Rainer Gerhards

NetApp devices provide diagnostic information via an Windows Event Log like interface. This enables their messages to be browsed by Windows Event viewer and and be automatically processed by tools like EventReporter and MonitorWare Agent.

It goes without saying that there are ample benefits from this capability. For example, instant alerting can be enabled by simply reading the NetApp event logs, checking for some conditions and alerting (for example via email) if something unusual is logged. Another big benefit is that by converting the NetApp Event Logs into syslog, they can be forwarded to a central loghost and thus easily be integrated into a centralized logging and alerting system.

There are two ways to access the NetApp logs. First of all, NetApp exposes them as files which obey the event log file format. These files are dynamically re-written and as such considered to be "live". Adiscon products support this mode since long and are used with great success in that mode (there exists another article on how to do that).

Secondly, NetApp also provides an emulation of the Windows event log API. Under that emulation, a NetApp devices "looks and feels" just like another Windows machine offering its event log. Well ... mostly. There are a couple of subtle differences, with the most important one being that the event log record number is not incremented on NetApp devices. This record number is associated with each event log record. Native Windows always increments this number (up to a very high max value, where it then wraps). The NetApp emulation does not use such a unique record number. Instead, the API always exposes a fixed number of most recent records, but always uses the same recrord number for them. So while the data is being rolled as new records come in, the record number stays the same. Thus, for example, if new data comes in, the data for record number 312 is moved to record number 311 and 312 receives the data from record 313. Under this implementation, event log record numbers do no longer identify a given event log entry.

Adiscon products are optimized for highest throughput and least demand on system ressources. One way to do that is to utilize the uniqueness of record numbers. This optimization leads to problems on NetApp. Namely, new records are never forwarded, because no new record numbers are seen (which leads EventReporter and MonitorWare Agent to beliefe there are no new records). The cure, however, is simple. First of all, checksum verification for the last processed event must be turned on in the configuration of the event log monitor in question. This ensure that an internal checksum is generated as a unique identifier for an event log record. Secondly, the event log monitor must be instructed to always search for new events based on that checksum. This will enable the monitor to properly detect new events. This configuration can be seen in the screenshot below:

Please note that while we turn off some optimizations with these settings, performance is still quite good. Most importantly, typical NetApp event logs are relatively small as the contain only a small subset of recent records. This makes reading through them quite painless. Secondly, we still apply a lot of other optimizations, especially when comparing to other tools which blindly try to re-do everything always. Users are still advised to use a sensible sleep value for the event log monitor (the default should be fine), as without the optimizations additional i/o overhead is unavoidable.

Should you have problems implementing this configuration, please contact support@adiscon.com for assistance. We are glad to help.