Logs, logs, logs

Can you find out what your laptop is doing during its boot with Zabbix? Of course you can! By day, I am a monitoring technical lead in a global cyber security company. By night, I monitor my home with Zabbix & Grafana and do some weird experiments with them. Welcome to my weekly blog about this project.

This week I was originally planning to write about how to monitor a yet another device with Zabbix, but as in today's world deliveries seem to take a long time, I'll blog about something else -- Zabbix and syslog collection.

You might not think Zabbix as a syslog platform, as there are specialised tools for that, providing much more functionality for log management than Zabbix ever could. However, sending syslog information towards Zabbix can be very useful; in bigger environments, sending everything to it might be overkill and too taxing so please filter with care and consider the log retention period, but at home with very low logging rate, this can be handy.

Logs? How and why?

At my home network, a Raspberry Pi 4 is running Zabbix server, among other software. One of the roles for my cool little Raspberry is that it acts as a centralised syslog server. I have configured my home router, MacBook and one more laptop to send their syslog to rsyslogd running on my Raspberry.

Then, on Zabbix, I have an item configured to keep an eye on the centralised log file the events are flowing into.

No alt text provided for this image

 

This way, I can see all kinds of events happening on those devices via Zabbix, and create appropriate triggers if something worth mentioning gets logged. Inspecting the syslog with Zabbix Plain text widget shows absolutely everything, and well, that view probably contains just noise.

No alt text provided for this image

Adding details

However, if you start searching for whatever you would need to know about, then you of course can search for content. Let's see what my MacBook has automatically updated lately:

No alt text provided for this image

I can then add a trigger which would log the time when something got installed or updated. It's no way a replacement for a proper log management solution, but for Super Important Targets something like this could be very useful, as you could catch any looming issues via Zabbix immediately, too.

No alt text provided for this image

Likewise, I can see what my Linux laptop has been doing:

No alt text provided for this image

In my configuration, absolutely everything gets sent to Zabbix, so the syslog entries from the devices are coming in starting from the moment the devices have their network & syslog services up, and they will stop coming when the syslog service stops during shutdown.

Here's the Linux laptop starting up:

No alt text provided for this image

MacBook also sends its events to syslog during OS updates / startup / shutdown, but it's been so long since I last restarted my Mac that my Zabbix does not have the logs for that period of time anymore, and I don't want to restart my MacBook (which I'm using to type this blog entry) just to get a screenshot from its boot sequence.

Hopefully my new gadget will arrive by next week so I can then finally blog about that. :)

I have been working at Forcepoint since 2014 and in addition to monitoring addict, I am a log addict, too.




Add new comment

Restricted HTML

  • Allowed HTML tags: <a href hreflang> <em> <strong> <cite> <blockquote cite> <code> <ul type> <ol start type> <li> <dl> <dt> <dd> <h2 id> <h3 id> <h4 id> <h5 id> <h6 id>
  • Lines and paragraphs break automatically.
  • Web page addresses and email addresses turn into links automatically.
Submitted by whatsuphome on