LogSentry INSTALL

LogSentry INSTALL

Operation

Logcheck contains several files:
logcheck.sh The main script. This file controls all processing and looks at log files with simple grep commands. This file is executed on a timed basis from cron and reports findings to the sysadmin.

logtail A custom executable that remembers the last position of a text file.
This program is used by logcheck to parse out information from the last time the log was opened, this prevents reviewing old material twice. All log files will be processed with this program and will have a file named "logfile.offset" put in the same directory, where logfile. is the name of the log file checked. This file contains the decimal offset information for logtail to work. If you delete it, logtail will parse the file from the beginning again. Logcheck tracks the size and inode of log files to enable it to tell when a log file has been rotated. If the inode of the log changes, or the file size is smaller than the last run, logtail will reset the counter and parse the entire file.

logcheck.hacking This file contains keywords that are certifiable attacks on your system. I leave this file sparse, unless I know what a certain pattern of attack looks like (The default keywords are almost always generated by Internet Security Scanner attacks, or sendmail(8) if it is being fed illegal syntax in address lines). Any keyword in a log file that matches here will generate a report with a more obnoxious header to grab your attention faster: "ACTIVE SYSTEM ATTACK"

logcheck.violations This file contains keywords of system events that are usually seen as negative. Words such as "denied", "refused", etc. Positive words such as 'su' successes are also put in here. This file is of course not all inclusive and is heavily biased towards FWTK messages and BSDish messages with TCP wrappers installed. Violations here are reported under the heading "Security Violations" in the reports.
logcheck.violations.ignore
This file contains words that are reverse searched against the logcheck.violations file. If these words are found, that entry is not reported. An example of this are the following log entries:
 Feb 28 21:00:08 nemesis sendmail[5475]: 
 VAA05473: to=crowland, ctladdr=root 
 (0/0), delay=00:00:02, xdelay=00:00:01, 
 mailer=local, stat=refused
 
 Feb 28 22:13:53 nemesis rshd: 
 refused connect from hacker@evil.com:1490
The top entry is from sendmail and is a fairly common error, the stat line indicates that the remote host refused connections (stat=refused). This can happen for a variety of reasons and generally is not a problem.
The bottom line however indicates that a person (hacker@evil.com) has tried unsuccessfully to start an rsh session on my machine, this is bad (of course you shouldn't be running rshd to begin with).
The logcheck.violations file will find the word 'refused' and will flag it to be logged, however this will report both instances as being bad and you will get false alarms from sendmail (both had the word refused). By putting the following in the logcheck.violations.ignore file you tell logcheck to ignore the sendmail problem and it will only report to you the bad rsh connection:
 (in logcheck.violations.ignore)
 
 mailer=local, stat=refused
This will prevent reports from any line that contains "refused" but has the rest of the keywords "mailer=local, stat=refused." This is of course pretty basic, and not very intelligent, however remember that by forcing you to be specific in what you ignore, you will not overlook something important. A word of caution though, if you don't pick a long enough string to put in the logcheck.violations.ignore file then you could ignore significant events. Be very very careful what you put in here. The default file has only one entry in it to allow grep to run. Tune it to your system carefully! If the above did not make sense at all, leave the file as it is.
logcheck.ignore This file is the catch-all file for words to look for in the logs and to NOT REPORT. Again be specific with what you want to ignore and go easy on the wildcards. Anything that does not match what is in this file is reported (so you don't risk missing anything) as "Unusual System Activity." The default is again BSDish and biased towards FWTK and TCP Wrappers.
To preserve integrity of the scans the following search order and rules are kept:

Active System Attacks are reported first.
Security Violations are reported second.
Unusual System Events are reported last.

Keyword searches on the logcheck.hacking and logcheck.violations file are CASE INSENSITIVE to ensure we don't miss anything. Keyword searches on the logcheck.violations.ignore and logcheck.ignore files are CASE SENSITIVE to ensure again that we don't miss anything. The *.ignore files REQUIRE you to put in the exact text as part of the contents. The more sensitive logcheck.violations and logcheck.hacking files will report on any word, regardless of case, that is found as a match.
The whole process follows the following structure:
 logcheck.sh executes hourly ----> 
 logcheck.sh executes logtail on log files ----> 
 logtail parses off any text from the last time it was run ---> 
 logcheck greps text for system attack messages ---> 
 logcheck greps text for security violations ---> 
 logcheck greps text for security violations to ignore ---> 
 logcheck greps text for all messages to ignore. ---> 
 any messages found are mailed to system admin.
Overall it's a very simple process and is surprisingly good at telling you information about your system you were never aware of, but probably should have been.

Installation

If you know what a syslog.conf file is, know you have it set up to log as much information as possible, AND HAVE SECURED THE LOGS, go to step TWO, otherwise you should read step ONE.
Step ONE: Configuring syslog daemon and SECURING your log files.
Before setting up logcheck, you should ensure that your system is not only running syslog, but that you have it configured for maximum logging. On most all systems I recommend that you send all syslog messages to ONE file for logcheck to parse through. This configuration ensures that messages will not be missed. On BSDish systems this involves editing the file syslog.conf located in /etc. This file contains parameters for syslogd.
Many syslog.conf files are sensitive to using tabs instead of spaces for your entries and you will mysteriously hose syslogd daemon if you put in spaces so be careful.
In the syslog.conf file you should put in an entry like this:
 *.info                        /var/log/messages
Which will log EVERYTHING to the file "messages" located in /var/log. Obviously you should substitute /var/log to the directory typically found on your system. For BSDI and most variants this is /var/log for Linux this is /var/adm. Your syslog.conf file for your site will have the default in it. Remember this will log everything, if you have a very high volume server (for instance: mail) you may want to cut back on the logging to prevent over running of disk space. You can do this in the following way:
 *.info;mail.none                /var/log/messages
 mail.notice                     /var/log/messages
This will only log non-standard mail messages, I don't recommend this however as it will make it hard to track mail into and out of your system if someone attempts, or succeeds, to gain entry. Many systems have separate log files for different system services, configuring syslog to do this could look like the following
 *.info;mail,ftp,daemon,authpriv.none   /var/log/messages
 mail.info                              /var/log/mail.log
 ftp.info                               /var/log/ftp.log
 daemon.info                            /var/log/daemon.log
 authpriv.*                             /var/log/secure.log
This configuration will have separate logs for the general system messages, mail, ftp, daemon and security messages. Logcheck can be setup to check for all of them. Again please see your syslog.conf man page for more information. Now that you have edited your syslog.conf file you need to re-start syslogd by sending the HUP signal to it. IMPORTANT: You must now go to your log directory (/var/log in the example above) and change the log file to owner root, group wheel and mode 600 on file permissions. First check if the file exists if it doesn't, you should make it. For example if your log files is simply called 'messages' you would do the following:
 touch /var/log/messages
Now ensure that you change the permissions in the following way:
 chown root.wheel /var/log/messages
 chmod 600 /var/log/messages
I also recommend that any other log files have their permissions set in a similar way (at least to mode 600 if you can't change the owner/group). Log files contain very sensitive data about system operations and could contain passwords, system errors, and other data that can reveal vulnerabilites if you are not careful. I personally feel that these files should never be readable by any person other than root.
BSD and FreeBSD: You should go to the /etc directory and edit the /etc/daily, /etc/weekly, and /etc/monthly scripts and change the 'rotate()' script function to change the log permissions on rotation. Simply change the line:
 cp /dev/null "$file"; chmod 644 "$file"
To:
 cp /dev/null "$file"; chmod 600 "$file"
(The above is for BSDI 2.x, BSDI 3.x uses an external rotate function now, just change the mode sent to it from 644 to 600 and you'll be OK. FreeBSD will be similiar to the BSDI 2.x script)
When logs are auto-rotated they will have the permissions set automatically.
Once these steps have been completed you can move onto step TWO:
Step TWO: Logcheck and logtail installation.
Logcheck requires the following files to run:
 logcheck.sh
 logtail.c
 logcheck.hacking
 logcheck.violations
 logcheck.violations.ignore
 logcheck.ignore
Pull logcheck.sh into your favorite editor and find the section entitled: CONFIGURATION SECTION.
Change the name of the SYSADMIN variable to one of your liking. You can use local names (default is root), or e-mail addresses for remote logging.
Go to the section entitled: LOG FILE CONFIGURATION SECTION and either uncomment the log files that apply to you, or add your own. Be sure you know the difference between the > and >> operators before you do this.
If you have one of the default system types (Linux, BSDI, FreeBSD, SunOS, Digital) you can simply type "make <systype>" and it will install for you at this point.
If you are using an alternate path for the files (i.e NOT in /usr/local/whatever), you need to change the path entries for logcheck.hacking, logcheck.violations, logcheck.ignore, logcheck.violations.ignore, and logtail in the main logcheck.sh script. I don't recommend you do this unless you have to.
If you changed the default paths /usr/local/etc and /usr/local/bin in the logcheck.sh file you need to edit the Makefile and change INSTALLDIR and INSTALLDIR_BIN to point to the same directories.
Note that the Makefile will create a directory called /usr/local/etc/tmp by default. This is the scratch area for logcheck to handle it's files. I do NOT recommend that you use /tmp for any reason as it is publically accessible and may pose a danger if a user creates symbolic links to trick the logcheck script into overwriting an important system file. I would also change all automated system scripts to use this directory instead of /tmp which is notoriously unsafe.

Editing Cron

After installing logcheck, you should edit your local crontab file for root and set logcheck to run once per hour (recommended, although you can do it more frequently, or less frequently, although the absolute minimum in my opinion is once every few hours or so). Examples are the following:
 00 * * * * root /bin/sh /usr/local/etc/logcheck.sh
 00,15,30,45 * * * * /usr/local/etc/logcheck.sh
The 15 minute check I would recommend for firewalls that generally don't produce messages unless they are in trouble.
Remember, logcheck does not report anything if it has nothing useful to say (only if the rest of USENET could do this). So running it every 15 minutes will have no impact on your mailbox if the system being watched is quiet. Busier systems can be addressed by less frequent reporting, however longer reports mean spend more time analyzing them, and you may not like this either. Again I recommend hourly.
After you have edited the crontab, send the crond daemon a HUP to reset it.

Final Check and Troubleshooting

You are almost done. I recommend that you run the logcheck.sh script by hand to ensure that it reads the log files without errors and sends mail to the right account. You should check that syslog is in fact logging events and that you generate an event or two (just su to root) for logcheck to report. You should have a report mailed to you. Run logcheck multiple times to ensure that you don't get repeat messages.
If you get repeat messages then something is wrong with the logtail program (not marking the file correctly) and you should check your log directory for files that end in *.offset. If you do not see these files there you have a problem with the logtail binary. Run logtail manually against these logs to see if it correctly marks the file offsets. If it still refuses to do it you should check your file permissions and make sure that you are running all this as root.
Default file permissions for logcheck files:
 logcheck* -- 600 -- Read/Write for root ONLY. Owner root. 
 logtail*  -- 700 -- Read/Write/Execute for root ONLY. Owner root. 
*** NOTE: None of the files should ever be SUID root for any reason.
For the more cautious, you may want to create a special group/user that owns the logfiles and have the logfile scanner run as that user. Configuration to do this should be rather straight forward, just adjust the file permissions on the logcheck files accordingly and ensure they can read and write to the directory where the logs are stored.
If you have any questions you can send e-mail for some limited help... unfortunately my schedule keeps me rather busy so be patient with the response...

logcheck.sh	The main script. This file controls all processing and looks at log files with simple grep commands. This file is executed on a timed basis from cron and reports findings to the sysadmin.
logtail	A custom executable that remembers the last position of a text file. This program is used by logcheck to parse out information from the last time the log was opened, this prevents reviewing old material twice. All log files will be processed with this program and will have a file named "logfile.offset" put in the same directory, where logfile. is the name of the log file checked. This file contains the decimal offset information for logtail to work. If you delete it, logtail will parse the file from the beginning again. Logcheck tracks the size and inode of log files to enable it to tell when a log file has been rotated. If the inode of the log changes, or the file size is smaller than the last run, logtail will reset the counter and parse the entire file.
logcheck.hacking	This file contains keywords that are certifiable attacks on your system. I leave this file sparse, unless I know what a certain pattern of attack looks like (The default keywords are almost always generated by Internet Security Scanner attacks, or sendmail(8) if it is being fed illegal syntax in address lines). Any keyword in a log file that matches here will generate a report with a more obnoxious header to grab your attention faster: "ACTIVE SYSTEM ATTACK"
logcheck.violations	This file contains keywords of system events that are usually seen as negative. Words such as "denied", "refused", etc. Positive words such as 'su' successes are also put in here. This file is of course not all inclusive and is heavily biased towards FWTK messages and BSDish messages with TCP wrappers installed. Violations here are reported under the heading "Security Violations" in the reports.
logcheck.violations.ignore	This file contains words that are reverse searched against the logcheck.violations file. If these words are found, that entry is not reported. An example of this are the following log entries: Feb 28 21:00:08 nemesis sendmail[5475]: VAA05473: to=crowland, ctladdr=root (0/0), delay=00:00:02, xdelay=00:00:01, mailer=local, stat=refused Feb 28 22:13:53 nemesis rshd: refused connect from hacker@evil.com:1490 The top entry is from sendmail and is a fairly common error, the stat line indicates that the remote host refused connections (stat=refused). This can happen for a variety of reasons and generally is not a problem. The bottom line however indicates that a person (hacker@evil.com) has tried unsuccessfully to start an rsh session on my machine, this is bad (of course you shouldn't be running rshd to begin with). The logcheck.violations file will find the word 'refused' and will flag it to be logged, however this will report both instances as being bad and you will get false alarms from sendmail (both had the word refused). By putting the following in the logcheck.violations.ignore file you tell logcheck to ignore the sendmail problem and it will only report to you the bad rsh connection: (in logcheck.violations.ignore) mailer=local, stat=refused This will prevent reports from any line that contains "refused" but has the rest of the keywords "mailer=local, stat=refused." This is of course pretty basic, and not very intelligent, however remember that by forcing you to be specific in what you ignore, you will not overlook something important. A word of caution though, if you don't pick a long enough string to put in the logcheck.violations.ignore file then you could ignore significant events. Be very very careful what you put in here. The default file has only one entry in it to allow grep to run. Tune it to your system carefully! If the above did not make sense at all, leave the file as it is.
logcheck.ignore	This file is the catch-all file for words to look for in the logs and to NOT REPORT. Again be specific with what you want to ignore and go easy on the wildcards. Anything that does not match what is in this file is reported (so you don't risk missing anything) as "Unusual System Activity." The default is again BSDish and biased towards FWTK and TCP Wrappers.