Qmail Support
Qmail
is a UNIX-based fast and secure SMTP MTA.
System resources are moderate even under high E-Mail load and volume.
Qmail was developed by
Dan J. Bernstein,
and is well recommended internationally.
In 2008 Dan Bernstein placed Qmail in the public domain.
D.J. Bernstein is Associate Professor in the Department of
Mathematics, Statistics, and Computer Science at the University
of Illinois at Chicago. Please have at look at his remarks on:
Qmail 1.03 was released in June 1998 and lacks support for
some ESMTP enhancement which have been published since then,
in particular SMTP Authentication and SMTPS/STARTTLS.
However, some of the ESMTP enhancements are of very little use, in today's
email world even counter-productive, and often weakly documented,
as discussed in my
which was originally planned as part of my
Qmail book.
In addition, I have set up a particular
email discussion
list to deal with the following issues:
- SPAMCONTROL: release announcements, additional usage information,
forthcoming developments
- QMVC: release announcements, usage, additional templates.
- NEWANALYSE: release information, particular customization.
- Other topics: New viruses / WARLORD, SMTP Authentication,
STARTTLS usage, MAV, documentation.
You can participate in those discussions
subscribing to the email list,
which is run by EZMLM, of course.
Support
Since 1998 I install and maintain Qmail and in addition Qmail's
E-Mail list manager EZMLM
on commercial and non-commercial base.
High volume (> 1 mio emails/day), high efficient, and low
latency (< 10 sec/email) mail servers can be achieved with
Qmail and my
recommended
extensions on current hardware.
This allows in addition a high efficient spam and virus
protection with little headache and
administration efforts.
Development
An active community supports Qmail, as maintained by Russel
Nelson's excellent
Qmail Homepage,
mentioning in addition my own developments.
Over the past ten years not only the public recognition of email (and
its abuse) has been changed, but in addition the requirements
for a SMTP MTA. By means of my Qmail extension SPAMCONTROL, I
not only keep track of current developments, but in addition
try to incorporate the requirements of large email sites.
Currently not supported:
- STARTTLS with qmail-remote: Due in SPAMCONTROL 2.6 (under development)
- CRAM-MD5 for qmail-remote: Due in SPAMCONTROL 2.7
- Greylisting: Currently no plans yet (rblsmtpd ?)
- DKIM: Currently no plans (rblsmtpd ?)
- IPv6: SPAMCONTROL 3.0.
- Bounce Address Tag Validation (BATV): No plans.
Some of the major patches are available for vanilla Qmail:
Patches, mostly for qmail-smtpd:
SMTP HELO/EHLO Greeting delay
The qmail-smtpd.c_greetdelay
patch adds a defined delay after the SMTP client has initiated
the SMTP session and before qmail-smtpd answers with "220
ESMTP". The delay is control via the environment variable
GREETDELAY, which is typically set by means of tcpserver's
rule-set file. A value of GREETDELAY="30"
will delay the qmail-smtpd's greeting for 30 seconds.
Time enough to discourage a typical Spambot's SMTP client
from continuing the SMTP session (the session will be aborted
by the client). Setting of GREETDELAY does not harm 'regular'
mail clients, but will (currently) reduce Spam traffic by about
50%.
It has been argued, that the sleeping qmail-smtpd
blocks local resources and it should be better to extend the
algorithm to send the client a 55x reply (in case of
protocol violations). However, one should keep in mind, that
not only the resources on the mail server are used, but
in addition those of the spam mail sender are blocked
for that particular session. Thus, the GREETDELAY will not only
save you for spam mails, but unlike Greylisting and/or
filtering a la SpamAssassin, this is the only mean to really
reduce the overall amount of spam because the timeslot
required for the spam sender to deliver messages (whether successfully
or unsuccessfully) is raised from typically one second to (<=)
GREETDELAY seconds.
On the 2007th "Frühjahrsfachtagung" of the
German Unix User Group (GUUG)
in Berlin, I have given a brief presentation
(in english) about pros'n'cons of GREETDELAY.
Note: In case you run redundant mail servers,
you have to install the patch on every system.
Usage hint: As it has been pointed out by Matthew
R. Dempsky, you can achieve the basic GREETDELAY functionality
even without patching qmail-smtpd by means of the following
call:
- tcpserver 0 25 sh -c "sleep $GREETDELAY; exec qmail-smtpd"
rblsmtpd Patches
Dan Bernstein has designed rblsmtpd (as part of it's
ucspi-tcp-0.88)
as front-end to qmail-smtpd. Since vanilla qmail-smtpd
does not include any DNS client capabilities, he decided to include
the RBL DNS lookup (based on the incoming IP of the SMTP client)
into a 'slim' forefront processor: rblsmtpd.
In fact, DJB decouples with this approach a 'heavy-weighted'
SMTP interface with a light-weight DNS front-end (currently about
250 lines of code). One of the bottlenecks of this architecture
is, that any decision -- whether to accept or to deny a SMTP
connection -- is done on a 'binary' base. Currently, there is
no way how the information gathered by rblsmtpd is forwarded
to qmail-smtpd, though rblsmtpd employs the environment
variable RBLSMTPD. However, this kind of information
proliferation is required to allow rblsmtpd not only to
do simple RBL lookup but rather to enable it to serve for DKIM
and SPF queries, which needs some communication with the SMTP
back-end (qmail-smtpd).
Of course, it is a natural choice to include GREETDELAY into
rblsmtpd (Greetdelay
MD5: 39ec9d9290da1fd9921104cb18ee2e73). This patch
- removes the dependency of rblsmtpd w.r.t the default
RBL rbl.maps.vix.com.
- introduces a new flag "-w delay" to include
a default delay -- and --
- includes the new flag "-W" which fetches
the value of 'delay' from the populated GREETDELAY environment
variable (set by tcpserver or sslserver).
- adds a particular line to the rblsmtpd output to tell
the current GREETDELAY value for this connection.
@4000000047c732fd3b15ec54 sslserver: pid 23324 from 24.107.74.16
@4000000047c732fd3b15fbf4 sslserver: ok 23324 mail.fehcom.net:217.172.188.134:25 24-107-74-16.dhcp.stls.mo.charter.com:24.107.74.16::2899
@4000000047c732fd3b160f7c rblsmtpd: 24.107.74.16 pid 23324: greetdelay: 50
In case you are nosey about the efficiency, you can download
the little script greetdelay.ksh.
This script only works in conjunction with the patched rblsmtpd
and SPAMCONTROL 2.5.
After invoking the script like
./greetdelay.ksh -v /var/log/qmail-smtpd/current
you get the following output (sample from today):
Greetdelay discouraged: @400000004964a20729258414 rblsmtpd: 208.116.174.226 pid 12246: greetdelay: 50 [typicalcocktail.com]
Greetdelay discouraged: @400000004964a49602a67d5c rblsmtpd: 189.70.9.178 pid 12290: greetdelay: 180 [18970009178.user.veloxzone.com.br]
Greetdelay discouraged: @400000004964abec38c07334 rblsmtpd: 81.214.243.141 pid 12596: greetdelay: 80 [dsl.dynamic81214243141.ttnet.net.tr]
Greetdelay discouraged: @400000004964ac0213d02e84 rblsmtpd: 208.116.174.226 pid 12602: greetdelay: 50 [typicalcocktail.com]
Greetdelay discouraged: @400000004964ac7e016cefd4 rblsmtpd: 66.180.215.253 pid 12690: greetdelay: 50 [immortelles.net]
Greetdelay discouraged: @400000004964af750ff89d8c rblsmtpd: 189.18.243.251 pid 12736: greetdelay: 180 [189-18-243-251.dsl.telesp.net.br]
Greetdelay discouraged: @400000004964b6eb1d2c410c rblsmtpd: 208.116.203.40 pid 13067: greetdelay: 50 [bourbonmisto.com]
Greetdelay discouraged: @400000004964bda005330c0c rblsmtpd: 208.116.203.40 pid 13280: greetdelay: 50 [bourbonmisto.com]
Greetdelay discouraged: @400000004964c04517d61aac rblsmtpd: 208.116.174.226 pid 13407: greetdelay: 50 [typicalcocktail.com]
Connections [We Jan 7 16:15:31 CET 2009]: 75 -- Greetdelayed: 39 -- SMTP Sessions: 36 [9 secs]
You may use the options '-v' for verbose, '-t' to be verbose and
to resolve the tai64 timestamp. Further, the input files can be provided
'stacked', e.g. '"@* current"' to analyse all multilog generated files.
The speed of the analysis depends of course on the size of your log files and the
DNS resolution.
The sample above shows the reduction after the DNSBL lookup (Spamhaus)
and the efficiency is in my case around 50%.
Note: Some versions of SPAMCONTROL use GREETDELAY
as well, thus employing this patch would result in a 'double delay'.
RECIPIENTS extension (0.6)
qmail-smtpd accepts messages if the SMTP domain part
of recipient address ("RCPT TO: ") matches
an entry in control/rcpthosts or control/morercpthosts.cdb.
The existence of a mailbox/maildir for the corresponding SMTP
recipient is checked later in the delivery chain. In case no
Mailbox/Maildir exists, the message is bounced back to the SMTP
sender ("MAIL From: "). For normal
SMTP mail traffic thats fine as long as the rate of undeliverable
messages dont exceed 10% and the sender is 'legitmate'; ie. exists.
Todays situation is different: Spam and Virus attacks with
forged/faked sender addresses
to a bunch of random recipient addresses yield a undeliverable rate
up to 90%. Worse: The generated bounces will never reach the
sender and a double-bounce is eventually send to the postmaster.
There is an ongoing need to facilitate the existence of the
Recipient's mailbox (and thus the "RCPT TO:" address)
during the SMTP dialogue. For large sites, this is a complex
task, since often multi-tier systems are involved, and the required
efficiency can only be guaranteed at the borderline (= Internet) MTA.
From the usage and administrative point-of-view, for any email user
one has to setup up:
The different authentication methods may require different
security frameworks, which depend in particular on the client.
Not only that this raises the need for a central repository of
the mail user's attributes for administration purposes, in addition
the email system should be able to use the repository.
While my RECIPIENTS extension makes qmail-smtpd aware
of acceptable recipients, the knowledge of those recipients has
to be kept in a local database.
The RECIPIENTS extension (0.6.6)
(MD5: b75db208ac6a327c3fa468064df14e92) drops this requirement and provides a
checkpassword
compliant API to allow practically any source to be queried,
preferable a LDAP directory (as well as Microsoft's ADS),
though a SQL database may be fine as well.
A ldap_pam.pl (here: version 0.9.2)
is included in the patch.
None-RELAYLCLIENT mode:
The recipient check is done for addresses accepted per
control/rcpthosts or control/morerecpthosts.cdb,
i.e. in a none-RELAYCLIENT case.
Virtual domain support:
The lookup is triggered per domain; allowing
different sources to consult with the additional capability of
redundant severs, wilddomains and fall-thru mode for not-listed
domains.
VERP support: Enabled by default.
Version 0.6 has a different (now default) notion of VERP addresses.
Recipient addresses included in the lookup database starting with the
local part local-@domain are automatically
recognized as VERP addresses.
Compatibility:
RECIPIENTS 0.6 is backward-compatible with 0.4 (except for the wilddomain support).
Probably, with some effort, all known recipient verifications methods known
for qmail-smtpd (ie. validrcptto) can be incorporated
easily.
RECIPIENTS 0.5 (0.5.21)
can be used as a lagacy alternative (MD5: a9feb59be62560fd83253d47246ea70).
The legacy, fastforward
compatible cdb approach is still supported unaltered. Thus, the
contributions of some users, to build the lists of E-Mail addresses
from local information, i.e. vpopmail's accounts is still
valid:
Scripts to generate recipients.cdb:
Note: These scripts are included unaltered without a particular 'fitness' check.
Please check yourself, if they fulfill your requirements.
Warlord (1.4.0)
These days, MTA's are flooded with virus E-Mails in high frequency,
resulting in considerable delays in E-Mail communication. Out-of-band
E-Mail Anti-Virus processing is cost-intensive and bounces are
meaningless and even counter-productive for forged "MAIL
FROM:" addresses.
Based on the idea of Russell Nelson's (and Charles Cazabon's)
so-called "qmail-smtpd-viruscan-1.1.patch" to filter
Windows executables attached as BASE64 encoded MIME parts in
the E-Mail, I created a "Warlord"
(MD5:5f77272038774d1b1a3cf164dcb682a5) enhancement for qmail-smtpd.
"Warlord" enables a wire-speed filtering of E-Mails
containing BASE64 encoded attachments with about 99,5%
efficiency. The Warlord algorithm employs
- a robust MIME type identification. Particular MIME types
can be added on-the-fly in a badmimetypes.cdb and the
badmimetype check is enabled, invoking the environment
variable BADMIMETYPE,
- a high efficient and unique Loader type recognition. Specific
Loader types (i.e. for the Windows OS) can be included in a badloadertypes.cdb
and the badlodertype check is employed, if the environment
variable BADLOADERTYPE is set with a characteristic
pattern (i.e. BADLOADERTYPE='M'),
- additional capability to reject a BASE64 encoded attachment,
in case it includes white spaces (spaces or tabs) which is typically
the case for malware. In order to employ the check, use BADMIMETYPE='!'
.
Note: The WARLORD algorithm has been made "public
domain" in the Unix magazine iX (issue November 2004) and is therefore not
patent-able.
QHPSI (0.2.0)
The Qmail High Performance
Scanner Interface (MD5: Sfabf359926197ccc0e18498ba3a7b6e2)
is a new approach to an old problem:
While typically Virus Scanning is facilitated by means of more
or less complex scripts (i.e. qmail-scanner, AMaViS),
calling in a second or third stage one or more Virus Scanners,
todays requirements are towards more efficiency. With QHPSI,
you can directly invoke a Virus Scanner to scan messages in the
Queue. No need for "staging areas", no need for external
programs like reformime or ripMIME. Simply call
your Virus Scanner for qmail-smtpd from the associated
tcpserver cdb.
The QHPSI works exceptionally well with Clam AV's clamd/clamdscan.
Simply use in your tcpserver's cdb :allow,QHPSI="clamdscan"
and you are done. The entire power of QHPSI is exploited in SPAMCONTROL.
Version 0.9.x.y of ClamAV includes a bug, which
prevents clamscan/clamdscan to log to STDERR. Here,
you find a patch
for clamav-0.9x.y (borrowing some code from 0.7x).
And I've implemented a new approach to cope with email sender
(originator) verification:
'Mail From:' Address Verification (MAV)
There is an ongoing discussion, how to verify that a particular
email sender (originator) is authorized to use a specific MTA
(Mail Transfer Agent):
are the most prominent candidates, though only SPF is deployed
more commonly.
However, in any case, the burden to verify the sender is put
on the recipient MTA's shoulder.
Reversely, my proposal 'Mail From:'
Address Verification tries to actively label the email to
originate authoritatively from a certain MDA (identified
per IP/Domain name) or an (authenticated) User (or both).
Unlike eg. SPF there is no heavy-weight patching of Qmail
necessary. By means of the simple plug-in
MAV 0.19 (MD5: c1ef96d37a060024776ab69b87d01296)
qmail-smtpd can be made
verification aware
for local addresses.
Those extensions in combination are part of SPAMCONTROL:
SPAMCONTROL
SPAMCONTROL is an add-on (patch)
for mainly qmail-smtpd for advanced control of incoming
emails with respect to the SMTP envelope and the data.
Version 2.5 includes significant enhancements in particular for
qmail-remote:
SPAMCONTROL provides ESMTP STARTTLS support for qmail-smtpd based on
Scott Gifford's extension
for SuperScript's
sslserver as part of ucspi-ssl.
In addition, a script is available to act as front-end for qmail-queue
allowing to run Spam and Virus checking on a RAM-Disk.
Further, SPAMCONTROL includes all the Qmail enhancements presented
above.
Yet more Qmail add-ons:
QMVC
QMVC is a Korn-Shell Plug-In for qmail-local
to enable filtering and Virus scanning of locally delivered E-Mails.
Unlike AMaVis or Qmail-Scanner, it is entirely designed for
Qmail and does not require additional patches.
Newanalyse
Newanalyse is a set of small
programs helping the sysadmin in his task of processing and automatically
rotating multilog generated logfiles.
Documentation
Unfortunately, my forthcoming book about Qmail, Daemontools,
UCSPI, and DJBDNS in German language has been canceled by the
publisher.
However, I will continue in that project (though with less
intensity) and put my material on my
Qmail Documentation web page.
|
