© August 2005 Rainer Gerhards
Rainer Gerhards (2005-08-04)
In this paper, I describe how to use
(link dead, sorry)
rsyslogd. Php-syslog-ng is a
popular web interface to syslog data. Its name stems from the fact that it
usually picks up its data from a database created by
syslog-ng and some
helper scripts. However, there is nothing syslog-ng specific in the database.
With rsyslogd's high customizability, it is easy to write to a syslog-ng like
schema. I will tell you how to do this, enabling you to use php-syslog-ng as a
front-end for rsyslogd - or save the hassle with syslog-ng database
configuration and simply go ahead and use rsyslogd instead.
The setup is pretty straightforward. Basically, php-syslog-ng's interface to the syslogd is the database. We use the schema that php-syslog-ng expects and make rsyslogd write to it in its format. Because of this, php-syslog-ng does not even know there is no syslog-ng present.
For php-syslog-ng, you can follow its usual setup instructions. Just skip any steps refering to configure syslog-ng. Make sure you create the database schema in MySQL. As of this writing, the expected schema can be created via this script:
Please note that at the time you are reading this paper, the schema might have changed. Check for any differences. As we customize rsyslogd to the schema, it is vital to have the correct one. If this paper is outdated, let me know so that I can fix it.
Once this schema is created, we simply instruct rsyslogd to store received data in it. I wont go into too much detail here. If you are interested in some more details, you might find my paper "Writing syslog messages to MySQL" worth reading. For this article, we simply modify rsyslog.conf so that it writes to the database. That is easy. Just these two lines are needed:
These are just two lines. I have color-coded them so that you see what
belongs together (the colors have no other meaning). The green line is the
actual SQL statement being used to take care of the syslog-ng schema. Rsyslogd
allows you to fully control the statement sent to the database. This allows you
to write to any database format, including your homegrown one (if you so desire).
Please note that there is a small inefficiency in our current usage: the
property is used for both the time and the date (if you wonder about what all
these funny characters mean, see the
property replacer manual) . We could have extracted just the date and time
parts of the respective properties. However, this is more complicated and also
adds processing time to rsyslogd's processing (substrings must be extracted). So we take a full mysql-formatted timestamp and supply it to MySQL. The sql engine in turn
discards the unneeded part. It works pretty well. As of my understanding, the
inefficiency of discarding the unneeded part in MySQL is lower than the
effciency gain from using the full timestamp in rsyslogd. So it is most probably
the best solution.
Please note that rsyslogd knows two different timestamp properties: one is timereported, used here. It is the timestamp from the message itself. Sometimes that is a good choice, in other cases not. It depends on your environment. The other one is the timegenerated property. This is the time when rsyslogd received the message. For obvious reasons, that timestamp is consistent, even when your devices are in multiple time zones or their clocks are off. However, it is not "the real thing". It's your choice which one you prefer. If you prefer timegenerated ... simply use it ;)
The line in red tells rsyslogd which messages to log and where to store it. The "*.*" selects all messages. You can use standard syslog selector line filters here if you do not like to see everything in your database. The ">" tells rsyslogd that a MySQL connection must be established. Then, "mysql-server" is the name or IP address of the server machine, "syslog" is the database name (default from the schema) and "user" and "pass" are the logon credentials. Use a user with low privileges, insert into the logs table is sufficient. "syslog-ng" is the template name and tells rsyslogd to use the SQL statement shown above.
Once you have made the changes, all you need to do is reload (or HUP) rsyslogd. Then, you should see syslog messages flow into your database - and show up in php-syslog-ng.
With minumal effort, you can use php-syslog-ng together with rsyslogd. For those unfamiliar with syslog-ng, this configuration is probably easier to set up then switching to syslog-ng. For existing rsyslogd users, php-syslog-ng might be a nice add-on to their logging infrastructure.
Please note that the MonitorWare family (to which rsyslog belongs) also
offers a web-interface: phpLogCon. At the time of this writing, phpLogCon's code
is by far not as clean as I would like it to be. Also the user-interface is
definitely not as intutive as pp-syslog-ng. From a functionality point of view,
however, I think it already is a bit ahead. So you might
consider using it. I have set up a
(link dead, sorry)
You can have a peek at it
without installing anything.
I would appreciate feedback on this paper. If you have additional ideas, comments or find bugs, please let me know.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license can be viewed at https://www.gnu.org/copyleft/fdl.html.
Got something to add? Send me email.
More Articles by Rainer Gerhards © 2012-07-15 Rainer Gerhards