Thursday, December 31, 2009

HOWTO-Customize-LogWatch

SkyHi @ Thursday, December 31, 2009
Logwatch is a customizable log analysis system. 
Logwatch parses through your system's logs for a given period of time and creates a report analyzing areas that you specify, in as much detail as you require. Logwatch is easy to use and will work right out of the package on most systems. 
1. Login to your server as root via SSH. 

2. Load the logwatch configuration file
Type: pico -w /etc/log.d/conf/logwatch.conf 

3. Search for where the log files are mailed too.
Press: CTRL-W
Type: MailTo 
Set the e-mail address to an off server account so incase you get hacked they can not delete the mail without hacking atleast 2 servers. 

4. Now lets change what actions you are alerted of.
CTRL-W
Type: Detail 

5. Detail = Low
Change that to Medium, or High.
I suggest high, because you will get more detailed logs with all actions. 

6. Once you are done lets Exit & Save
CTRL-X then Y then Enter 





LogWatch is a customizable, pluggable log-monitoring system. It will go through your logs for a given period of time and make a report in the areas that you wish with the detail that you wish. Easy to use – works right out of the package on almost all systems. This utility is not a daemon , it is a perl script and usually implemented as a cron job (daily) and reports via email the most important aspects of your log files .
The newcomers to the Linux world can just accept the default configurations , read the messages that are send to their mail box and getting a view of the status of the box ( security alerts , disk space available , failed ssh log-in attempts ….) . It is also possible to run logwatch via the terminal with  parameters and display the result directly on the terminal , so there is no need to wait the report until it is executed from the cron job ( usually ones a day) .The daily report is setup via a symlink in /etc/cron.daily:
0logwatch -> /usr/share/logwatch/scripts/logwatch.pl
If you prefer to customise logwatch , then you have to go through a rather confusing hierarchy of config files and a collection of service-specific filters to cut out logged information about the services you’re not interested in. On CentOs 5.x , logwatch is run daily as a result of a symlink to the main logwatch.pl script in the /etc/cron.daily directory.
Most people will be content to stick with the default configuration of logwatch, but once you’ve mastered its rather labyrinthine assortment of config files and scripts, there’s a lot you can do to configure it, including writing your own filter scripts if you have appropriate programming skills – probably with Perl or PHP.
The most important files of logwatch on CentOs 5.x are :
  • /usr/share/logwatch/scripts/logwatch.pl :  This script is run if  cron  executes the logwatch scheduled job . The /usr/sbin/logwatch command  is a symlink to this script
  • /usr/share/doc/logwatch.xxx/HOWTO-Customize-LogWatch : This is a must read file , not only for those that need to customize logwatch
  • /etc/logwatch/conf/logwatch.con  :This file sets the default values of all the  options (see table below ) . These defaults are used when LogWatch is called without any parameters (i.e. from cron.daily). The file is well-documented, but the explanations below also apply to this config file.
  • /etc/logwatch/scripts/services/* : Actual filter programs for the various services.
  • /etc/logwatch/scripts/shared/* : Filters common to many services and/or logfiles.
  • /etc/logwatch/scripts/logfiles/* : Filters specific to just particular logfiles.
Configuration files priority :
Logwatch can be highly customized through its configuration files , these files are organized on a directory structure .
Actualy logwatch contains 3 directories for the configuration files, all have the same structure but different priority level :
  • /usr/share/logwatch/default.conf /…: The default configuration provided by logwatch
  • /usr/share/logwatch/dist.con/….. : Distribution specific configuration file ( CentOs 5.x doesn’t recommend any configuration at all , so the default configurations will take place if no user specific configurations are available) .
  • /etc/logwatch/….. : The place where the user makes his custom configuration changes
Think of it like the priority level of CSS in HTML .The first level (highest priority ) for the configuration files is the /etc/logwatch/… directory . If the first level is not present then the next level will take place ( distribution specific configuration files ) and if no second level exists then the last level ( default configuration from logwatch ) will take place .
Launching logwatch via the terminal : As mentioned previously , a fresh report can be enabled with the terminal . These are the most useful parameters that can be passed to the perl script .
logwatch terminal options
–usage    or    –helpDisplays usage information
–detail levelThis is the detail level of the report. level can be high, med, low.
–logfile log-file-groupThis will force LogWatch to process only the set of logfiles defined by log-file-group (i.e. messages, xferlog, …). LogWatch will therefore process all services that use those logfiles. This option can be specified more than once to specify multiple logfile-groups.
–service service-nameThis will force LogWatch to process only the service specified in service-name (i.e. login, pam, identd, …). LogWatch will therefore also process any log-file-groups necessary to process these services. This option can be specified more than once to specify multiple services to process. A usefulservice-name is All which will process all services (and logfile-groups) for which you have filters installed.
–printPrint the results to stdout (i.e. the screen).
–mailto addressMail the results to the email address or user specified inaddress.
–archivesEach log-file-group has basic logfiles (i.e. /var/log/messages) as well as archives (i.e. /var/log/messages.? or /var/log/messages.?.gz). This option will make LogWatch search through the archives in addition to the regular logfiles. The entries must still be in the proper date range (see below) to be processed, however.
–range rangeYou can specify a date-range to process. This option is currently limited to only Yesterday, Today and All.
–debug levelFor debugging purposes. level can range from 0 to 100. This will really clutter up your output. You probably don’t want to use this.
–save file-nameSave the output to file-name instead of displaying or mailing it.
–logdir directoryLook in directory for log files instead of the default directory.
–hostname hostnameUse hostname for the reports instead of this system’s hostname. In addition, if HostLimit is set in/etc/log.d/logwatch.conf, then only logs from this hostname will be processed (where appropriate).
Examples
  • logwatch –service ftpd-xferlog –range all –detail high –print –archives 
    This will print out all FTP transfers that are stored in all current and archived xferlogs.
  • logwatch –service pam_pwdb  –range yesterday –detail high –print
    This will print out login information for the previous day…
  • logwatch  –printJust force to execute logwatch immediately and print out results to the terminal
  • logwatch 
    logwatch will be excecuted as it was  enabled through cron , so jour mail box will receive the report
  • logwatch  –detail  high  –logfile secure –print
  • logwatch –detail high –logfile messages –mailtoyourname@domain.com
    Scan the “messages” log and send the report to a custom email






Keep in mind that you can find some Logwatch documentation at /usr/share/doc/logwatch-*/HOWTO-Customize-LogWatch, and it contains a few useful examples.
  1. On RHEL/CentOS/SL, the default logwatch configuration is under/usr/share/logwatch/default.conf/logwatch.conf
    These settings can be overriden by placing your local configuration under/etc/logwatch/conf/logwatch.conf. Place the following in that file to tell logwatch to completely ignore services like 'httpd' and the daily disk usage checks:
    # Don't spam about the following Services
    Service = "-http"
    Service = "-zz-disk_space"
    
  2. Sometimes I don't want to completely disable logwatch for a specific service, I just want to fine tune the results to make them less noisy. /usr/share/logwatch/default.conf/services/*.confcontains the default configuration for the services. These parameters can be overridden by placing your local configuration under /etc/logwatch/conf/services/$SERVICE.conf. Unfortunately, logwatch's ability here is limited, and many of the logwatch executables are full of undocumented Perl. Your choice is to replace the executable with something else, or try to override some settings using /etc/logwatch/conf/services.
    For example, I have a security scanner which runs scans across the network. As the tests run, the security scanner generates many error messages in the application logs. I would like logwatch to ignore errors from my security scanners, but still notify me of attacks from other hosts. This is covered in more detail at Logwatch: Ignore certain IPs for SSH & PAM checks?. To do this, I place the following under /etc/logwatch/conf/services/sshd.conf:
    # Ignore these hosts
    *Remove = 192.168.100.1
    *Remove = X.Y.123.123
    # Ignore these usernames
    *Remove = testuser
    # Ignore other noise. Note that we need to escape the ()
    *Remove = "pam_succeed_if\(sshd:auth\): error retrieving information about user netscan.*
    
    "
  3. logwatch also allows you to strip out output from the logwatch emails by placing regular expressions in /etc/logwatch/conf/ignore.conf. HOWTO-Customize-LogWatch says:
    ignore.conf: This file specifies regular expressions that, when matched by the output of logwatch, will suppress the matching line, regardless of which service is being executed.
    However, I haven't had much luck with this. My requirements need a conditional statement, which is something like 'If there are security warnings due to my security scanner, then don't print the output. But if there are security warnings from my security scanner and from some bad guys, then print the useful parts-- The header which says "Failed logins from:", the IPs of the bad hosts, but not the IPs of scanners.'
  4. Nip it at the source (As suggested by @user48838). These messages are being generated by some application, and then Logwatch is happily spewing the results to you. In these cases, you can modify the application to log less.
    This isn't always desirable, because sometimes you want the full logs to be sent somewhere (to a Central syslog server, central IDS server, Splunk, Nagios, etc.), but you don't want logwatch to email you about this from every server, every day.


REFERENCES
http://www.logwatch.org/tabs/docs/HOWTO-Customize-LogWatch.html
http://tournasdimitrios1.wordpress.com/2011/01/15/logwatch-system-log-analyzer-and-reporter-for-linux/
http://serverfault.com/questions/293226/linux-logwatch8-is-too-noisy-how-can-i-control-the-noise-level