When network devices run into problems they generate error messages. In a lot of cases, where those error messages go is up to you. Devices like servers (including Windows servers with the utility mentioned below), routers, switches, and even some HP JetDirect print server cards support the use of a "syslog" server. A syslog server is kind of a central repository for log messages as a way for you to centralize your monitoring of network systems and devices. It's a client/server type of setup where the devices are the "clients".
When set up to use a syslog server, devices will send their log messages over the network wire to the syslog server rather than recording them in a local file or displaying them. By default Cisco routers and switches will typically write them to the console screen provided you have a console session open. But since you don't have a console session open most of the time, it's a good idea to change where these messages are sent.
Not only is it up to you to decide where the messages are sent, but you can also decide which messages the client devices send based on the level of severity. These severity levels are standardized and identified by a number and/or a standard abbreviation (shown in parentheses) as so:
Level 7 basically says to send every peep to the syslog server. It's good to use when you want to test your syslog server to make sure it's working.
There are also things called "facilities" which loosely relate to system processes, a way of categorizing messages. When a remote device sends a message to a syslog server it includes one of the standard facility values (along with a severity level). Some of the common facilities are:
local7 is used by Cisco equipment and Windows servers. You can specify different severity levels for different facilities so you can, for example, log all kernel messages but only emergency messages from printers. This is done in the /etc/syslog.conf file using the following format:
facility.severity log-file-name
Using the example we gave above for kernel and printer messages the /etc/syslog.conf file entries would look like this:
I usually set up a large partition with the mount-point name of "logs" just for syslog files. The above /etc/syslog.conf file entries would then look something like this:
There's two things you have to do to set up your Debian system as a log host. Luckily they're simple edits to a couple text files. They are:
The syslog daemon is run at system startup by default because it also handles all the local log files, and there are a bunch of them. If you list the files in the /var/log directory you'll see what I'm talking about.
To take care of the first item above we have to edit the startup script that runs the syslogd daemon when the system boots. Open the script using the command:
nano /etc/init.d/sysklogd
and look for this line near the top:
SYSLOGD=""
and change it to:
SYSLOGD="-r -m0"
(That's a zero after the 'm'.) Then exit the editor saving the file. The -r tells syslogd to listen for remote messages. The -m0 stops syslogd from putting a bunch of annoying -- MARK -- entries in your log files.
To take care of the second item, open the follwing file:
nano /etc/syslog.conf
and add the following lines to it near the top:
Here we are just telling the syslogd daemon to write the messages to the enterprise.log file. If you want to monitor Windows servers and Cisco devices add this line also:
local7.debug /var/log/enterprise.log
We're using the 'debug' level here just for testing to make sure our server is receiving and logging messages. It can tightened up later. Now restart the syslogd daemon with the command:
/etc/init.d/sysklogd restart
Congratulations, you've got yourself a syslog server! You can check it out by listing the files in the /var/log directory again. You should see the enterprise.log file there now.
Now you have to go around to your devices and tell them to use it, and what level of messages to send to it.
Cisco Devices
For Cisco switches running the CatOS you can console or telnet into the switch and enter the following commands to accomplish that:
Note that 3 is where you set the level of severity. For Cisco routers and switches running IOS the commands are:
Note that errors is where you set the level of severity.
Linux Systems
If you want to set up other Linux servers (or even desktops) to be clients (i.e. to send their messages to this Debian log server) you'd add the following line to their/etc/syslog.conf files:
*.* @debianbox
replacing 'debianbox' with whatever the hostname of your Debian system is. The '*.*' specifies that all log messages be sent to the log server. Some devices, like JetDirect cards will not allow you to specify a severity level which is why you want to restrict what's actually logged by the settings you enter in the /etc/syslog.conf file on the syslog server.
Windows Servers
Naturally Microsoft doesn't want to support a long-held standard like syslog so we have to jump through some hoops to monitor Windows servers. A company in Sweden called Datagram has a great free utility called SyslogAgent that runs as a service on Windows servers. It converts the messages in all of the Event Viewer logs (System, Applications, Security, etc.) to a syslog format and sends them to a syslog server. You can even specify a different severity level for each log. And even better, installing it doesn't require a reboot. Go to their download page at:
www.syslogserver.com/download.html
and download just the SyslogAgent file, not the whole suite. It runs on NT, 2000, and 2003.
When set up to use a syslog server, devices will send their log messages over the network wire to the syslog server rather than recording them in a local file or displaying them. By default Cisco routers and switches will typically write them to the console screen provided you have a console session open. But since you don't have a console session open most of the time, it's a good idea to change where these messages are sent.
Not only is it up to you to decide where the messages are sent, but you can also decide which messages the client devices send based on the level of severity. These severity levels are standardized and identified by a number and/or a standard abbreviation (shown in parentheses) as so:
| 0 - Emergency (emerg) 1 - Alerts (alert) 2 - Critical (crit) 3 - Errors (err) 4 - Warnings (warn) 5 - Notification (notice) 6 - Information (info) 7 - Debug (debug) |
Level 7 basically says to send every peep to the syslog server. It's good to use when you want to test your syslog server to make sure it's working.
There are also things called "facilities" which loosely relate to system processes, a way of categorizing messages. When a remote device sends a message to a syslog server it includes one of the standard facility values (along with a severity level). Some of the common facilities are:
| auth - authentication (login) messages cron - messages from the memory-resident scheduler daemon - messages from resident daemons kern - kernel messages lpr - printer messages (used by JetDirect cards) mail - messages from Sendmail user - messages from user-initiated processes/apps local0-local7 - user-defined (see below) syslog - messages from the syslog process itself |
local7 is used by Cisco equipment and Windows servers. You can specify different severity levels for different facilities so you can, for example, log all kernel messages but only emergency messages from printers. This is done in the /etc/syslog.conf file using the following format:
facility.severity log-file-name
Using the example we gave above for kernel and printer messages the /etc/syslog.conf file entries would look like this:
kern.* /var/log/example.log lpr.emerg /var/log/example.log |
I usually set up a large partition with the mount-point name of "logs" just for syslog files. The above /etc/syslog.conf file entries would then look something like this:
kern.* /logs/enterprise.log lpr.emerg /logs/enterprise.log |
*.* /logs/enterprise.log |
| The Syslog Server |
There's two things you have to do to set up your Debian system as a log host. Luckily they're simple edits to a couple text files. They are:
- Tell the syslog daemon to listen for messages from remote devices
- Tell the syslog daemon what to do with those messages
The syslog daemon is run at system startup by default because it also handles all the local log files, and there are a bunch of them. If you list the files in the /var/log directory you'll see what I'm talking about.
To take care of the first item above we have to edit the startup script that runs the syslogd daemon when the system boots. Open the script using the command:
nano /etc/init.d/sysklogd
and look for this line near the top:
SYSLOGD=""
and change it to:
SYSLOGD="-r -m0"
(That's a zero after the 'm'.) Then exit the editor saving the file. The -r tells syslogd to listen for remote messages. The -m0 stops syslogd from putting a bunch of annoying -- MARK -- entries in your log files.
To take care of the second item, open the follwing file:
nano /etc/syslog.conf
and add the following lines to it near the top:
| *.emerg /var/log/enterprise.log *.alert /var/log/enterprise.log *.crit /var/log/enterprise.log |
Here we are just telling the syslogd daemon to write the messages to the enterprise.log file. If you want to monitor Windows servers and Cisco devices add this line also:
local7.debug /var/log/enterprise.log
We're using the 'debug' level here just for testing to make sure our server is receiving and logging messages. It can tightened up later. Now restart the syslogd daemon with the command:
/etc/init.d/sysklogd restart
Congratulations, you've got yourself a syslog server! You can check it out by listing the files in the /var/log directory again. You should see the enterprise.log file there now.
| Syslog Clients |
Now you have to go around to your devices and tell them to use it, and what level of messages to send to it.
Cisco Devices
For Cisco switches running the CatOS you can console or telnet into the switch and enter the following commands to accomplish that:
| set logging server <ip address of your Debian system> set logging server severity 3 set logging timestamp enable set logging server enable |
Note that 3 is where you set the level of severity. For Cisco routers and switches running IOS the commands are:
| config term logging <ip address of your Debian system> logging trap errors service timestamps log datetime logging on |
Note that errors is where you set the level of severity.
Linux Systems
If you want to set up other Linux servers (or even desktops) to be clients (i.e. to send their messages to this Debian log server) you'd add the following line to their/etc/syslog.conf files:
*.* @debianbox
replacing 'debianbox' with whatever the hostname of your Debian system is. The '*.*' specifies that all log messages be sent to the log server. Some devices, like JetDirect cards will not allow you to specify a severity level which is why you want to restrict what's actually logged by the settings you enter in the /etc/syslog.conf file on the syslog server.
Windows Servers
Naturally Microsoft doesn't want to support a long-held standard like syslog so we have to jump through some hoops to monitor Windows servers. A company in Sweden called Datagram has a great free utility called SyslogAgent that runs as a service on Windows servers. It converts the messages in all of the Event Viewer logs (System, Applications, Security, etc.) to a syslog format and sends them to a syslog server. You can even specify a different severity level for each log. And even better, installing it doesn't require a reboot. Go to their download page at:
www.syslogserver.com/download.html
and download just the SyslogAgent file, not the whole suite. It runs on NT, 2000, and 2003.
