Qmail Support
Qmail
is a UNIX-based fast and secure SMTP MTA.
System resources are moderate even under high E-Mail load and volume.
Qmail was developed by
Dan J. Bernstein,
and is well recommended internationally.
In 2008 Dan Bernstein placed Qmail in the public domain.
D.J. Bernstein is Research Professor in the Department of
Mathematics, Statistics, and Computer Science at the University
of Illinois at Chicago and Professor at the Faculteit Wiskunde & Informatica
at the Technical University of Eindhoven. Please have at look at his remarks on:
Qmail 1.03 was released in June 1998 and lacks support for
some ESMTP enhancement which have been published since then,
in particular SMTP Authentication and SMTPS/STARTTLS.
However, some of the ESMTP enhancements are of very little use, in today's
email world even counter-productive, and often weakly documented,
as discussed in my
which was originally planned as part of my
Qmail book.
In addition, I have set up a particular
email discussion
list to deal with the following issues:
- SPAMCONTROL: release announcements, additional usage information,
forthcoming developments
- QMVC: release announcements, usage, additional templates.
- NEWANALYSE: release information, particular customization.
- Other topics: New viruses / WARLORD, SMTP Authentication,
STARTTLS usage, MAV, documentation.
You can participate in those discussions
subscribing to the email list,
which is run by EZMLM, of course.
Support
Since 1998 I install and maintain Qmail and in addition Qmail's
E-Mail list manager EZMLM
on commercial and non-commercial base.
High volume (> 1 mio emails/day), high efficient, and low
latency (< 10 sec/email) mail servers can be achieved with
Qmail and my
recommended
extensions on current hardware.
This allows in addition a high efficient spam and virus
protection with little headache and
administration efforts.
Development
Qmail's home page as maintained by Russel Nelson
Qmail Homepage
is unfortunately out-of-date.
Over the past ten years not only the public recognition of email (and
its abuse) has been changed, but in addition the requirements
for a SMTP MTA. However, by means of my extensions for Qmail, I
not only keep track of current developments, but in addition
try to incorporate the requirements for large email sites.
Currently under investigation & development:
Some major paradigm changes need to be considered:
- Email needs to be considered as private
communication to be transported encrypted.
Exceptions are acceptable for not un-disclosed communication,
i.e. mailing lists.
- Today's hardware is mostly 64 bit capable. While
Qmail is written in ANSI-C, under some OS it won't compile
it any more; or would fail invoking it (FreeBSD).
- Running large and offical/popular sites requires
authentication of messages. Though neither
SPF nor DKIM are in situ qualified for this,
the new approach DMARC has been invented combining both.
- Though probably SMTP email will be the last protocol
to require IPv6 on a larger scale, there is a growing
need to support IPv6. Some current 'defense' mechanisms
- like Relay Black Lists - need to support IPv6 as well.
My plans are to support fully IPv6 and TLS throughout.
ucspi-tcp6 has
already been finished. ucpsi-ssl
shall come next, completed by sqmail
which I consider as successor of vanilla Qmail.
Currently not supported:
- greylisting: postponed
- DKIM: Currently no plans (rblsmtpd ?)
- Bounce Address Tag Validation (BATV): No plans.
Some of the major patches/extensions are available for vanilla Qmail:
Patches, mostly for qmail-smtpd:
SMTP HELO/EHLO Greeting delay
A flexible greetdelay approach is available in
ucspi-tcp6. Within rblsmtpd
a defined waiting delay can be defined after the SMTP client has initiated
the SMTP session and before qmail-smtpd answers with "220
ESMTP". The delay is control as part of the rblsmtpd's options
or via the environment variable GREETDELAY
which is typically set by means of tcpserver's
rule-set file. A value of GREETDELAY="30"
will delay the qmail-smtpd's greeting for 30 seconds.
Time enough to discourage a typical Spambot's SMTP client
from continuing the SMTP session (the session will be aborted
by the client). Setting of GREETDELAY does not harm 'regular'
mail clients, but will (currently) reduce Spam traffic by about
50%.
It has been argued, that the sleeping qmail-smtpd
blocks local resources and it should be better to extend the
algorithm to send the client a 55x reply (in case of
protocol violations). However, one should keep in mind, that
not only the resources on the mail server are used, but
in addition those of the spam mail sender are blocked
for that particular session. Thus, the GREETDELAY will not only
save you for spam mails, but unlike Greylisting and/or
filtering a la SpamAssassin, this is the only mean to really
reduce the overall amount of spam because the timeslot
required for the spam sender to deliver messages (whether successfully
or unsuccessfully) is raised from typically one second to (<=)
GREETDELAY seconds.
On the 2007th "Frühjahrsfachtagung" of the
German Unix User Group (GUUG)
in Berlin, I have given a brief presentation
(in english) about pros'n'cons of GREETDELAY.
Note: In case you run redundant mail servers,
you have to install the patch on every system.
A typical result looks like:
@4000000047c732fd3b15ec54 sslserver: pid 23324 from 24.107.74.16
@4000000047c732fd3b15fbf4 sslserver: ok 23324 mail.fehcom.net:217.172.188.134:25 24-107-74-16.dhcp.stls.mo.charter.com:24.107.74.16::2899
@4000000047c732fd3b160f7c rblsmtpd: 24.107.74.16 pid 23324: greetdelay: 50
In case you are nosey about the efficiency, you can download
the little script greetdelay.ksh.
This script only works in conjunction with the patched rblsmtpd
and SPAMCONTROL > 2.5.
After invoking the script like
./greetdelay.ksh -v /var/log/qmail-smtpd/current
you get the following output (sample from today):
Greetdelay discouraged: @400000004964a20729258414 rblsmtpd: 208.116.174.226 pid 12246: greetdelay: 50 [typicalcocktail.com]
Greetdelay discouraged: @400000004964a49602a67d5c rblsmtpd: 189.70.9.178 pid 12290: greetdelay: 180 [18970009178.user.veloxzone.com.br]
Greetdelay discouraged: @400000004964abec38c07334 rblsmtpd: 81.214.243.141 pid 12596: greetdelay: 80 [dsl.dynamic81214243141.ttnet.net.tr]
Greetdelay discouraged: @400000004964ac0213d02e84 rblsmtpd: 208.116.174.226 pid 12602: greetdelay: 50 [typicalcocktail.com]
Greetdelay discouraged: @400000004964ac7e016cefd4 rblsmtpd: 66.180.215.253 pid 12690: greetdelay: 50 [immortelles.net]
Greetdelay discouraged: @400000004964af750ff89d8c rblsmtpd: 189.18.243.251 pid 12736: greetdelay: 180 [189-18-243-251.dsl.telesp.net.br]
Greetdelay discouraged: @400000004964b6eb1d2c410c rblsmtpd: 208.116.203.40 pid 13067: greetdelay: 50 [bourbonmisto.com]
Greetdelay discouraged: @400000004964bda005330c0c rblsmtpd: 208.116.203.40 pid 13280: greetdelay: 50 [bourbonmisto.com]
Greetdelay discouraged: @400000004964c04517d61aac rblsmtpd: 208.116.174.226 pid 13407: greetdelay: 50 [typicalcocktail.com]
Connections [We Jan 7 16:15:31 CET 2009]: 75 -- Greetdelayed: 39 -- SMTP Sessions: 36 [9 secs]
You may use the options '-v' for verbose, '-t' to be verbose and
to resolve the tai64 timestamp. Further, the input files can be provided
'stacked', e.g. '"@* current"' to analyse all multilog generated files.
The speed of the analysis depends of course on the size of your log files and the
DNS resolution.
RECIPIENTS extension (0.7)
qmail-smtpd accepts messages if the SMTP domain part
of recipient address ("RCPT TO: ") matches
an entry in control/rcpthosts or control/morercpthosts.cdb.
The existence of a mailbox/maildir for the corresponding SMTP
recipient is checked later in the delivery chain. In case no
Mailbox/Maildir exists, the message is bounced back to the SMTP
sender ("MAIL From: "). For normal
SMTP mail traffic thats fine as long as the rate of undeliverable
messages dont exceed 10% and the sender is 'legitmate'; ie. exists.
Todays situation is different: Spam and Virus attacks with
forged/faked sender addresses
to a bunch of random recipient addresses yield a undeliverable rate
up to 90%. Worse: The generated bounces will never reach the
sender and a double-bounce is eventually send to the postmaster.
There is an ongoing need to facilitate the existence of the
Recipient's mailbox (and thus the "RCPT TO:" address)
during the SMTP dialogue. For large sites, this is a complex
task, since often multi-tier systems are involved, and the required
efficiency can only be guaranteed at the borderline (= Internet) MTA.
From the usage and administrative point-of-view, for any email user
one has to setup up:
The different authentication methods may require different
security frameworks, which depend in particular on the client.
Not only that this raises the need for a central repository of
the mail user's attributes for administration purposes, in addition
the email system should be able to use the repository.
While my RECIPIENTS extension makes qmail-smtpd aware
of acceptable recipients, the knowledge of those recipients has
to be kept in a local database.
The RECIPIENTS extension (0.7.2)
(MD5: a39b001c4c049893049cc15677d9dab0) drops this requirement and provides a
checkpassword
compliant API to allow practically any source to be queried,
preferable a LDAP directory (as well as Microsoft's ADS),
though a SQL database may be fine as well (feel free to write your own PAM!).
The following PAMs are supplied:
None-RELAYLCLIENT mode:
The recipient check is done for addresses accepted per
control/rcpthosts or control/morerecpthosts.cdb,
i.e. in a none-RELAYCLIENT case.
Virtual domain support:
The lookup is triggered per domain; allowing
different sources to consult with the additional capability of
redundant severs, wilddomains and fall-thru mode for not-listed
domains.
VERP support: Enabled by default.
Recipient addresses included in the lookup database starting with the
local part local-@domain are automatically
recognized as VERP addresses.
Compatibility:
RECIPIENTS 0.7 is backward-compatible with 0.4 (except for the wilddomain support).
Probably, with some effort, all known recipient verifications methods known
for qmail-smtpd (ie. validrcptto) can be incorporated
easily.
The legacy, fastforward
compatible cdb approach is supported unaltered. Thus, the
contributions of some users, to build the lists of E-Mail addresses
from local information, i.e. vpopmail's accounts is still
valid:
Scripts to generate recipients.cdb:
Note: These scripts are included unaltered without a particular 'fitness' check.
Please check yourself, if they fulfill your requirements.
Warlord (1.4.0)
These days, MTA's are flooded with virus E-Mails in high frequency,
resulting in considerable delays in E-Mail communication. Out-of-band
E-Mail Anti-Virus processing is cost-intensive and bounces are
meaningless and even counter-productive for forged "MAIL
FROM:" addresses.
Based on the idea of Russell Nelson's (and Charles Cazabon's)
so-called "qmail-smtpd-viruscan-1.1.patch" to filter
Windows executables attached as BASE64 encoded MIME parts in
the E-Mail, I created a "Warlord"
(MD5:5f77272038774d1b1a3cf164dcb682a5) enhancement for qmail-smtpd.
"Warlord" enables a wire-speed filtering of E-Mails
containing BASE64 encoded attachments with about 99,5%
efficiency. The Warlord algorithm employs
- a robust MIME type identification. Particular MIME types
can be added on-the-fly in a badmimetypes.cdb and the
badmimetype check is enabled, invoking the environment
variable BADMIMETYPE,
- a high efficient and unique Loader type recognition. Specific
Loader types (i.e. for the Windows OS) can be included in a badloadertypes.cdb
and the badlodertype check is employed, if the environment
variable BADLOADERTYPE is set with a characteristic
pattern (i.e. BADLOADERTYPE='M'),
- additional capability to reject a BASE64 encoded attachment,
in case it includes white spaces (spaces or tabs) which is typically
the case for malware. In order to employ the check, use BADMIMETYPE='!'
.
Note: The WARLORD algorithm has been made "public
domain" in the Unix magazine iX (issue November 2004) and is therefore not
patent-able.
QHPSI (0.2.0)
The Qmail High Performance
Scanner Interface (MD5: Sfabf359926197ccc0e18498ba3a7b6e2)
is a new approach to an old problem:
While typically Virus Scanning is facilitated by means of more
or less complex scripts (i.e. qmail-scanner, AMaViS),
calling in a second or third stage one or more Virus Scanners,
todays requirements are towards more efficiency. With QHPSI,
you can directly invoke a Virus Scanner to scan messages in the
Queue. No need for "staging areas", no need for external
programs like reformime or ripMIME. Simply call
your Virus Scanner for qmail-smtpd from the associated
tcpserver cdb.
The QHPSI works exceptionally well with Clam AV's clamd/clamdscan.
Simply use in your tcpserver's cdb :allow,QHPSI="clamdscan"
and you are done. The entire power of QHPSI is exploited in SPAMCONTROL.
Version 0.9.x.y of ClamAV includes a bug, which
prevents clamscan/clamdscan to log to STDERR. Here,
you find a patch
for clamav-0.9x.y (borrowing some code from 0.7x).
And I've implemented a new approach to cope with email sender
(originator) verification:
'Mail From:' Address Verification (MAV)
There is an ongoing discussion, how to verify that a particular
email sender (originator) is authorized to use a specific MTA
(Mail Transfer Agent):
are the most prominent candidates, though only DKIM is deployed
more commonly.
However, in any case, the burden to verify the sender is put
on the recipient MTA's shoulder.
Reversely, my proposal 'Mail From:'
Address Verification tries to actively label the email to
originate authoritatively from a certain MDA (identified
per IP/Domain name) or an (authenticated) User (or both).
Unlike eg. SPF, there is no heavy-weight patching of Qmail
necessary. By means of the simple plug-in
MAV 0.20 (MD5: 9f1b33a4ab14ea03caa0a1d94d3605fa)
qmail-smtpd can be made
verification aware
for local addresses.
Those extensions in combination are part of SPAMCONTROL:
SPAMCONTROL
SPAMCONTROL is a major extension
for Qmail, in particular for qmail-smtpd to permit an advanced
controlling of incoming
emails with respect to the SMTP envelope and the data.
SPAMCONTROL provides a full featured SMTP AUTH and
ESMTP STARTTLS support for qmail-smtpd
and qmail-remote as well for qmail-pop3d based on
Scott Gifford's extension
and is highly integrated with
ucspi-ssl (0.8x).
A qmail-queue.scan script is available to act as front-end for qmail-queue
allowing to run Spam and Virus checking on a RAM-Disk.
Further, SPAMCONTROL includes (almost) all the Qmail enhancements presented
above.
The current version of SPAMCONTROL is suited for the AMD64 architecture and
compiles with MacOS X/BSD's clang.
Yet more Qmail add-ons:
Any service -- and in particular -- mission critical services like
email shall be setup such, to work in an un-attended, error-free manner.
Qmail together with supervise and multilog provides
this kind of service. However, monitoring and reporting
is important as well.
qmail-mrtg
The log information written by qmail-send, qmail-smtpd,
and qmail-popup/qmail-pop3d (as patched with SPAMCONTROL)
are written by qmail-mrtg in a format to be
used and displayed by MRTG.
qmail-mrtg 3.01
(md5: 7ca5174cae760540c4460e23abb9d074) is now available in version
3.0 and can be used to analyse multilog's log output for:
- qmail-send
- qmail-smtpd
- qmail-popup/qmail-pop3d
qmail-mrtg follows a modular concept and can easily be extended
to include other information. qmail-mrtg allows to customize the
feeding period such, only the most recent logfile - current -
can be read even for very busy servers. qmail-mrtg can read
simultaneously multiple logfiles produced by several instances of
qmail-smtpd, i.e. running on different ports.
Here you find a
living sample.
Newanalyse
Newanalyse is a set of small
programs helping the sysadmin in his task of processing and automatically
rotating multilog generated logfiles.
- In order to use newanalyse
qmailanalog
needs to be installed before.
- Newanalyse includes a small program findmail
to fetch individual (none-) delivered messages from
the qmail-send log.
- Newanalyse can be easily customized for several reports
QMVC
QMVC is a Korn-Shell Plug-In for qmail-local
to enable filtering and Virus scanning of locally delivered E-Mails.
Unlike AMaVis or Qmail-Scanner, it is entirely designed for
Qmail and does not require additional patches.
Documentation
Unfortunately, my forthcoming book about Qmail, Daemontools,
UCSPI, and DJBDNS in German language has been canceled by the
publisher.
However, I will continue in that project (though with less
intensity) and put my material on my
Qmail Documentation web page.
|
