FEHCom
Consulting Qmail TIPN Publications
Spamcontrol QMVC Newanalyse Documentation

Newanalyse - Analysis of Qmail logs

newanalyse

is a tool to post-process and archive the log information produced by qmail-send, qmail-smtpd, and qmail-pop3d written by multilog to disk.
In addition, even in the archived (qmail-send) log files particular emails can be easily found accordingly to the sender (originator) and/or the recipient by means of findmail.

Thus, Qmail together with multilog and newanalyse conforms with the current legislative changes of the European Community requiring archival of the email connection data for a certain period.
For a discussion of the political implications and those concerning the personal rights regarding the required storage of electronic communication data see: "Vorratsspeicherung von Verbindungsdaten in der Telekommunikation" (in German).

Requirements and Features:

Requirements:

  • Qmail (of course)
  • Qmailanalog
  • Daemontools
  • The Korn-shell and PERL
  • Note:

  • Qmail has to 'supervised'
  • newanalyse does not require the common (LWQ) 'qmail-ctl' script; though happily co-exists with it
  • newanalyse is aligned with the current logging of SPAMCONTROL
  • Features:

  • Analysis of qmail-send logfiles by means of Qmailanalog
    • Adaption for particular senders/recipients/conditions
    • Counters for Bounces, (discarded) Double-Bounces, and Nullsender messages
    • Counters for successful remote and local deliveries
    • Counters for local and remote recipients
    • Display of top senders and recipients according to number of messages and volume
    • Display of any (customized) verbose messages in qmail-send log (for error tracking)
  • Analysis of qmail-smtpd and qmail-pop3d log files.
  • Logfile processing
    • Secure and long-haul archival of logfiles and/or the analysis results
    • Customizable purge of old log files after N days
    • Support for several multilog serviced log directories
    • Setup of scratch directories not to be archieved
  • Apart from a generic .newanlyse.profile the following particluar profiles can be used (and customized):
    • [qmail-send]: newanalyse.senders
    • [qmail-send]: newanalyse.recipients
    • [qmail-send]: newanalyse.mtas
    • [qmail-send]: newanalyse.failures
    • [qmail-send]: newanalyse.deferrales
    • [qmail-send]: newanalyse.verbose
    • [qmail-smtpd]: newanalyse.smtpmessages
    • [qmail-smtpd]: greetdelayed sessions (options '-g' or '-d')
  • Reporting
    • Reporting of the results by E-Mail to the sysadmin (newanalyse.logadmin)
    • Verbose output for instant error/failure analysis
  • Logfile parsing
    • Search in the (archived) log files w.r.t. Sender/Recipients by means of findmail/qmFind
    • The algorithm has been improved in speed and precision

    Usage:

  • newanalyse handels any number of logfiles (at least up to 1023 per call)
  • newanalyse is easy customizable and extendable and supports multiple Qmail instances
  • newanalyse can be simply called by cron to perform the daily logfile processing; it is not suited for a multilog !processor directive
  • Typical newanalyse crontab entry:

    59 23 * * * /usr/local/bin/newanalyse -DAZK 1>/dev/null 2>&1

    Download and Installation:

  • newanalyse 1.6.2 (MD5: 114a127a1a97c1e70cd988f553dd361b)
    • Make sure, you meet the installation requirements
    • Note: qmailanaloge is NOT Gentoo compliant
    • Expand the packed tar-archive at an appropriate location; an installation directory will be created
    • Edit conf-pop3dlog, conf-qmail, conf-qmailanalog, conf-sendlog, and conf-smtpdlog to your needs
    • Execute ./install; maybe warnings and errors are displayed - fix them
    • The executables newanalyse, findmail, and qmFind are installed at /var/qmail/bin/ (symlinks provided in /usr/local/bin/) man-pages at /usr/local/man/, and the profiles at /var/qmail/etc/
    • Adjust to newanalyse profiles to your need; verify settings by means of 'newanalyse -h'

    Documentation:

    • newanalyse and findmail include comprehensive man-pages
    • newanalyse and findmail use get-opts style arguments; the current settings are diplayed via '-h'

    [Impressum]

    [FEHCom]

    [top]