5. Authentication & Authorization
SMTP was designed as a Host-to-Host protocol with no authentication in mind. In fact, the protocol lacks any kind of credentials which are important today. Many criticism has been raised regarding this fact, but let's try to straighten out the basics.
In general -- regarding email -- it is possible to target anybody. Once you've got an email address as recipient, you are subject of solicitated an unsolicitated emails. You need to control it. However, regarding the sender two scenarios exist:
- The sender is permitted to send emails from the origin domain. This permission may be granted by individual authentication. The credentials need to be checked by the sending MTA.
- The sender is granted permission to send to any domain. This may be called authorization. The credentials can be checked by the receiving MTA.
The permission to send by user is covered by SMTP Authentication. The permission to send per domain can an verified against the sender's IP address of the sending host by means of available SPF records on cryptographic signatures within the email - known as DKIM - and verified by means of the deployed public keys for the domain in the DNS.
| Mechanism | Credentials | Purpose | RFC | Section |
| SMTP Authentication | ... to send emails by (authenticated) user outside the origin domain | allow unrestricted relaying | RFC 4954 | 5.6, 5.9 |
| X.509 Client Certificate | ... to send emails by (trusted) user outside the origin domain | allow unrestricted relaying | [None; proprietary] | 5.5, 5.9 |
| Sending Domain Authorization (SPF) | ... to receive emails from specific IP addresses granted by DNS SPF records | prevent email fraud | RFC 4408 | 5.10 |
| DomainKeys Identified Mails (DKIM) | ... to receive emails for individual cryptographically signed mails verified by the origin domain's public key issued in the DNS | guarantee email integrity and qualify sending MTA | RFC 6376 and RFC 8463 | 5.12 |
s/qmail supports both SMTP Authentication and SPF Authorization together with X.509 client certificates for ESMTP(S) and QMTPS. DKIM signing and verifying for SMTP emails is possible starting with s/qmail release 4.2.
Quick links:
5.1 Qualification and Credentials
Every 'Internet citizen' has to be addressed by a valid email address. An email address consists of a local part - the 'mailbox' name and the domain part - telling to which domain the email has to be forwarded to: typically sender@origindomain.
The 'domain part' is subject of the DNS. A valid SMTP recipient domain needs to have a SMTP host to be responsible and to be able to receive emails for this domain.
The 'local part' simply tells, that the user has a valid 'account' for this domain.
First: Sending an email with SMTP envelope address <sender@origindomain> implies, that originator sender posses a reachable mailbox at origindomain -- which might be subject of a test.
Second: Sending an email to <recipient@targetdomain> will only be successful if recipient has a mailbox at targetdomain -- which might be verified in the SMTP dialogue.
The situation can be modelled in the following way:
Thus, we can identify three scopes -- or domains -- to testify the required qualifications:
- The sending MTA has to testify the credentials of sender. Details are discussed in this chapter.
- Within the Message Transfer System (MTS) the MTAs have to obligation to check whether both sending and receiving MTA do have qualified entries in the DNS In general, different qualification (and anti-qualification - like RBLs) schemes do exist. SPF (Sender Policy Framework) is covered in this chapter.
- Receiving an email for recipient@targetdomain will only be possible if recipient posses a Mailbox (or has at least an Alias entry).
In short:
- The sender of the email is responsible to be reachable via <sender@origindomain>. Once this is verified, he has the credentials to send this email.
- The sending host has the responsibility to verify the existence of the mailbox sender at origindomain prior of sending an email to <recipient@targetdomain>.
- In consequence, once an email is send from <sender@origindomain> to <recipient@targetdomain>, we assume in the current SMTP model that this case is valid: Sender (Originator) and Recipient are Internet citizens.
- In order for <sender@origindomain> to send an email to <recipient@targetdomain> the sending host may require additional qualifications:
- Based on the IP-address of the sender.
- Based on some additional qualification procedure like POP-before-SMTP.
- Based on SMTP-Authentication (this chapter).
- Based on X.509 client certificates (this chapter).
Thus, regarding the sending of email, we have two situations:
- Domain qualified email sender (ie. Web Shops' sending IP).
- User qualified email sender (based on userid/password or X.509 certs).
The question for the recipient of this email is:
Can we differentiate between those sources?
The answer is: No. We neither can trust any information based on (2.) nor
should discriminate emails from source (1.).
5.2 One-factor Authentication
An identity is a subject possessing distinguishable attributes; at least a (user)name to address the subject and a particular attribute, the password. The name is the public part of the identity while the password is used to authenticate the identity among others. We call this a 'One-factor' Authentication.
In this scheme, both the name (Userid) and the password are static and don't depend on the (current) authentication. If, however the current timestamp of the authentication process is used as salt (among other information, like the IP address), one may construct a One-Time Password (OTP) which is used and valid only for this authentication request.
The identity which wants to get authenticated is called the Supplicant; while the system, which has knowledge about the (set of) identities is the Identity Provider (IdP). There might be a third party involved, the Identity Broker which is typically available in terms of a Shibboleth service.
Other schemes depend not only on knowledge but rather include some additional 'hardware' which needs to be uniquely issued by the IdP: a token, an USB stick, a smartcard ... If both, this hardware piece needs to be available during authentication as well together with the password we consider this a Two-factor Authentication.
5.3 Identity Provider (IdP)
The Identity Provider (IdP) is a system able to store the attributes
of an identity persistently and allowing a lookup. Typically, the IdP
can be considered as Authenticator, a server.
The IdP not only holds the user database, but additionally is able
to support different authentication mechanisms.
One important security issue for the IdP is, to protect the user database (and her in particular the passwords) thus an untrusted third party, even in case the user data are proliferated by mistake, is not able to reconstruct the users's passwords. The Unix shadow password and crypt facility are perfect means for those, since the allow (in the running) Unix system access only to a root-owned process and protect the passwords by individual salting.
Other IdPs may use a standard SQL database-access or employ LDAP lookup as back-end.
5.4 Authentication Supplicant qmail-authuser
During qmail's development, several authentication modules have been realized as Pluggable Authentication Module, a PAM. D.J. Bernsten has provided a versatile authentication framework, the checkpassword interface.
Within s/qmail, the qmail-authuser program is available, supporting a file-based authentication, providing access to the Unix user database, and including a call to Vopmail's vcheckpw and VmailMgr's checkvpw in one go.
qmail-authuser replaces:
- checkpassword (natively)
- cmd5checkpw (natively)
- vcheckpw (calling it for vpopmail)
- checkvpw (calling it for vmailmgr)
fallowing the same call syntax.
Thus, qmail-authuser is sqmail's swiss knife for authentication purposes, superseding the older mechanisms; though they still can be used. It supports user, login, plain (which require a 'plaintext' password), and Challenge/Response (CR) authentication; however this depends on the back-end (the IdP) as well. qmail-authuser can be used for (E)SMTP and POP3 authentication.
qmail-authuser depends on a 'tokenized' password scheme in the user database which follows some ideas already outlined for the 'Recipient' extension. Let's explain that by example:
| Userid | Token | Password | Mechanism/Purpose |
| username(@domain) | password | Plaintext or CRAM/MD5 evaluation | |
| username(@domain) | % | hashed password (MD5, SHA1, SHA256) | Plaintext only evaluation |
| username(@domain) || * (any user) | ? | (not required) | Plaintext only against Unix crypt |
| username(@domain) || * (any user) | ! | (not required) | Plaintext only using Unix crypt + return environment settings for username |
| username(@domain) || @domain || @ (any user/any domain) | + | (not required) | Plaintext or C/R evaluation; calling checkvpw as secondary PAM |
| username(@domain) || @domain || @ (any user/any domain) | & | (not required) | Plaintext, calling vchkpw as secondary PAM |
| username(@domain) || @domain || @ (any user/any domain) | = | (not required) | using Dovecot as IdP wrapper/backend for plaintext and C/R passwords |
The 'token' is part of the password and removed prior of authentication and is used to define the authentication mechanism attached to particular userid. The userid may be a single username or may be constructed as username@domain where both parts are considered independently. The settings are tried in most significant order of the userid and in fall-thru mode for all methods. Thus, you can mix 'local' users with virtual domain users for the different virtual domain handlers.
5.4.1 Hashed Password Storage Methodes
qmail-authuser password evaluation and storages methods haven enhanced to support SHA1 and SHA256 alongside with MD5 hashes. Given, complex and unique passwords are used, this allows to protect the user's individual password even in case the server is compromised.
5.4.2 Salted Password Storage Methodes
This mechanism can (and will) be expanded for salted passwords as well, but requires the maintenance of a cdb and a bookkeeping UI module.
5.4.3 Using Dovecot as Authentication Wrapper
qmail-authuser copes now with Dovecot's authentication provisioning system directly, which requires to invoke a local domain socket within Dovecot; typically within the file 10-master.conf:
The location and path of the domain socket can be chosen arbitrarily; however it is necessary to tell qmail-authuser where to find it:
qmail-authuser will now call the doveadm module with the argument auth test -a and 'feeding' this socket with the provided authentication data given on FD 3. The additional argument 'true' is necessary for calls of qmail-authuser.
Note: Potentially, the following configuration is required in the file /etc/pam.d/dovecot:
5.5 X.509 certificate based Authentication
Within s/qmail, qmail-smtpd and qmail-qmtpd support X.509 client certificate based authentication.
What does this mean?
- First, it means to generate X.509 certificates + private keyfiles for individual users, to distribute those to them and to include those to their Mail User Agent (MUA). Of course, the MUA needs to support it. However even very old clients like Mulberry will do. One exception is the Blackberry's mail client, where this is reserved to be used with an Blackberry Enterprise Server; a stupid idea.
- Second, one has to decide whether those X.509 certs are public or private certs.
Private X.509 certs are preferable and can be generated with OpenSSL easily. One important issue is, to include the email address of the cert's owner into the Distinguished Name (DN) as attribute. - Those client X.509 certs need to be signed by a local CA. The local root cert, need to be included for verification purpose in case of s/qmail to sslserver.
- The server has to be advised to request client certificates.
qmail-smtpd and qmail-qmtpd recognize the presence of the X.509 client cert and will accept the MUA as relayclient. Further, the validated X.509 client cert and its DN will be included in the Received header of the email.
Note 1: qmail-smtpd can require the match of the email address in the cert's DN with the used SMTP originator address.
Note 2: SMTP Authentication needs to be enforced by means of the environment variable SMTPAUTH (see below).
5.5.1 sslserver settings
Using sslserver to allow X.509 client cert based authentication the following steps are necessary:
- For each User a X.509 client certificate (+ keyfile in PEM format) needs to generated which is digitally signed by a (local) CA. Those X.509 certs are only useful in your down administrative domain and are solely forseen for email authentication purpose. You may alternatively issue more generic X.509 client certs; however since the (private) keyfile to always bound to a MUA, this is not without risk. In addition, the keyfile can be password protected to prevent abuse.
- The signing CA X.509 root Cert (PEM format) has to included in sslserver key repository: CCAFILE=path_to_CA_signing_root_file(s). It is possible to provide several CA root Certs for different domains/customers (in one common file).
- sslserver has to be advised to request a client cert per connection. The (global) option sslserver -m can be used, or alternatively the environment variable CCAVERIFY="" is provided in sslserver cdb.
5.5.1 Additional qmail-smtpd settings
qmail-smtpd includes the authentication information in the the Received: header and displays the provided Distinguished Name of the user identity. The DN should include the email address of the sender.
In general, there is no connection between the DN and the ESMTP 'Return-Path' address, but this can enforced. For this purpose, the environment variable LOCALMFCHECK="?" can be set; typically as provided in the cdb. Now, only ESMTPA connections with matching conditions are granted.
5.5.2 qmail-smtpd logging with X.509 client certs
In case, the ESMTPA authentication was accepted or rejected, qmail-smtpd displays the DN as user identification in the logs following the token "?~". Further, the authentication method is given as cert.
5.6 qmail-smtpd Authentication
s/qmail includes a rich set of ESMTP Authentication capabilities for qmail-smtpd together with qmail-authuser. For some background, have a look at my SMTP Authentication tutorial; however, this may not be up to date.
Currently, the ESMTP auth mechanisms
- login,
- plain, and
- CRAM/MD5
are supported; but depend on the IdP.
5.6.1 qmail-smtpd Auth environment settings
The environment variable SMTPAUTH can be used to specify the type of SMTP Authentication. Your choices are:
| SMTPAUTH | Meaning |
| "" | Left blank to allow Authentication types "PLAIN" and "LOGIN" |
| "+cram" | Add "CRAM-MD5" support |
| "cram" | Just (secure) "CRAM-MD5" support, no other types offered |
| "!" | Enforcing SMTP Auth (of type "LOGIN" or "PLAIN") |
| "!cram" | Enforcing SMTP Auth of type "CRAM-MD5" |
| "!+cram" | Enforcing SMTP Auth of type "LOGIN", "PLAIN", or "CRAM-MD5" |
| "-" | Disabling SMTP Auth (for a particular connection) |
Unlike the Submission feature, these settings may be realized in the tcprules cdb per connection.
If not using qmail-smtpd in Submission mode on port 587 those settings are provided as variables in sslserver's cdb.
Note: It is required to install ucspi-tcp6 in order to generate the cdb by tcprules.
5.6.2 qmail-smtpd Auth logging
qmail-smtpd provides logging of accepted and failed authentication attempts.
The authentication mechanism together with the username is provided in the logs. Sample:
qmail-smtpd: pid 1167 Accept::AUTH::plain P:ESMTPSA S:1.2.3.4:example.com H:[127.0.0.1] F:me@example.com T:feh@fehcom.de ?~ 'userid'
5.7 qmail-pop3d Authentication
Authentication for Mailbox users is a 'must' and without authentication access to a Mailbox is not permitted. Along side with qmail-pop3d, qmail-popup is required to provide the userid, password, and the environmental settings.
5.7.1 qmail-pop3d Auth environment settings
Unlike qmail-smtpd, for qmail-popup only a slim choice of authentication mechanisms exists.
| POP3AUTH | Meaning | "apop" | Enforcing Auth of type "APOP" |
| "+apop" | Enabling Auth of type "APOP" |
Note: For qmail-popup no particular environment variables are required to enable authentication.
5.8 qmail-qmtpd Authentication
qmail-qmtpd supports authentication solely based on X.509 client certificates employing sslserver. The procedure follows the one discussed in section 5.5, except that the validation of the email address against the provided QMTP originator address is not provided and no specific logging takes place.
5.9 qmail-remote Authentication
qmail-remote is not at personal (E)SMTP client but rather acts on behalf of the (domain) owner. Thus, qmail-remote supports ESMTP authentication under two distinct scenarios:
- As ESMTP client submitting emails from a specific domain. For this purpose, the control file authusers can be employed.
- As ESMTP deliverer for emails to a relayhost as specified by means of the legacy (but now extended) smptroutes control file.
5.9.1 qmail-remote User Authentication
In a setting, where s/qmail has to serve multiple domains, you can set up qmail-remote to respect to act according to the domain's preferences:
Scenario 1 (control/authusers):
Based on the entire SMTP sending address, the domain part; or perhaps in general, one can specify the ESMTP username, the password, and hostname and port to connect to for ESMTP authentication.
However, if s/qmail depends on some remore relayhosts irrespectively of the sender, the following setup will be beneficial:
Scenario 2 (control/smtproutes):
Based on destination, qmail-remote can be advised to send emails on behalf of a (commonly known) username and password to a specific relayhost on a given port.
Note: authsenders settings have precedence over smtproutes.
5.9.2 qmail-remote Client Certificate Authentication
qmail-remote supports (as client) X.509 certificate validation and complementing section 5.5 regarding qmail-smtpd.
Now, the control file domaincerts may be employed to provide domain based X.509 authentication. This requires to include a X.509 + keyfile for a specific domain.
qmail-remote domaincerts provides the following capabilities:
- The PEM encoded X.509 certificate is given per sending domain (syntax):
domain:certificate|keyfile|password
- If domain equals '* this certificate is used as default for all connections.
- The file certificate may include the private key, thus keyfile can be omitted.
- The private key can be protected with a password.
Note: It is certainly a good idea to change the permission of this file only to be readable by qmail-remote running as user qmailr.
5.10 SPF Records and Domain Authorization
Starting with release 3.2 s/qmail supports the Sender Policy Framework SPF. SPF is mainly a mechanism to attach authorization to particular MTAs; their IP address respectively. Based on Jana Saout's code, s/qmail incorporates a native SPF parser with IPv6 capabilities.
SPF is rather complex and provides mechanisms for recursive lookups as given in the SPF record. Here is the layout of qmail-smtpd's SPF implementation:
5.10.1 Checking SPF records
SPF DNS records are common these days for the 'big players' in the Internet, like Gmail and others. However, the may possess a certain level of complexity to parse including text-semantic analysis and delegation to other SPF domain providers.
In order get used to SPF, spfquery may be used to see, how the SPF information retrieval is working. In general, an email supplier (using ESMTP) publishes for this domain a specific DNS TXT record which provides authorization for (mainly) IP addresses of MTAs of the domain example.com. This domain name is taken from the 'Mail From:' SMTP envelope address (in case of bounce, the EHLO/HELO string). The idea is taken from Hadmuth Danisch's RMX proposal, which was sacked in the IETF letter ballot and replaced with the much more complex SPF model.
Within my SPF implementation the entire information retrieval path is provided as output from spfquery.
Here is a (complicated) sample for a FAIL result:
Here, the 'M' denotes the SPF mechanism, 'I' means include. All presented IP addresses are checked against the sending MTA's one. CIDR evaluation is supported.
Once SPF records are published into the DNS, one may specify, if a particular MTA
- is authorized (PASS '+'),
- should be rejected (FAIL '-'),
- is ambiguous (SOFTFAIL '~'), or
- is unknown (NEUTRAL '?')
according to the SPF policy for sending email from example.com as provided as SMTP Return-Path envelope address.
s/qmail provides a complex SPF record parser and supporting all 'mechanisms' as defined in RFC 4408.
5.10.2 SPF evaluation in qmail-smtpd
SPF records requires intensive DNS lookups and this is done at the stage 'RCPT TO:' by qmail-smtpd, thus other anti-Spam means will work before.
Advise: Use a local DNSCache to speed up the lookup and reduce the load of any potential recursive DNS Resolver.
qmail-smtpd provides three tweaks for the SPF setup:
- The environment variable SPF is used as generic switch for the SPF evaluation. s/qmail follows Jana Saout's principals, as shown in the following table. However, no specific control file for qmail-smtpd is provided.
- A local policy (as last resort) can be specified in spflocalrules. The former spfguess is not supported anymore.
- More important, in case of an SPF Fail, the return code of
qmail-smtpd
can be customized to allow a site specific explanation. The default expression build in is:
See http://%{d}/why.html?sender=%{s}&ip=%{i}&receiver=%{R}"
This explanation uses the typical SPF macros, which are expanded at run-time.
SPF values to be specified for individual connections:
| SPF | Meaning | "0" | Never do SPF lookups, don't create Received-SPF headers |
| "1" | Only create Received-SPF headers, never block |
| "2" | Use temporary errors when you have DNS lookup problems |
| "3" | Reject mails when SPF resolves to fail (deny) |
| "4" | Reject mails when SPF resolves to softfail |
| "5" | Reject mails when SPF resolves to neutral |
| "6" | Reject mails when SPF does not resolve to pass |
SPF values 1 to 3 are 'save' and can be used in a standard environment. Setting SPF="1" raises the possibility to allow a Mail Delivery Agent (MDA) to inspect the Received-SPF header for spam and fraud detection.
While SPF record evalution can be triggered by a standard qmail-smtpd environment variable set in the run script, it is advisiable to perform a stronger SPF evaluation only for 'unknown' MTAs. In order to feed downstream Spam detection systems, I recommend:
- Within qmail-smtpd use SPF envaluation for all MTA (in the run script) defining "SPF=1" which means 'annotation' mode only.
- Within tcpserver/sslserver's cdb include "SPF=2/SPF=3" in the last generic 'allow' statement.
Note: In case auf email authentication, no SPF evaluation is used.
5.10.3 SPF Received Header
qmail-smtpd generates a standard Received-SPF header, in case
- the provided SPF environment value is given and not null,
- the domain to lookup provides SPF records to be evaluated.
Thus, the Received-SPF header is only present in case SPF information has been looked up successfully or a particular DNS failure hase been recognized. A standard PASS situation looks like this:
The implementation is able to include all cases of SPF results ('+' pass, '-' fail, '~' softfail, '?' neutral, 'o' none, 't' temperror, 'e' permerror); however depends on a feed of SPF variables.
The (global) variable spfinfo is populated as set of concatenated individual values in the following manner:
- identity carries the information of the retrieval source (i.e. recipient address, Helo greeting ...).
- clientip is the IP of the sending host.
- helo is the Helo greeting of the sender; if this is omitted, qmail-smtpd uses the received E/HELO string.
- envelopefrom is the Return-Path from the envelope.
- receiver is the recipient MTA.
- problem includes a description of encountered errors.
- mechanism is the purported evaluation method.
- result is the result of the evaluation and is expressed in the standard tokens (i.e. '+', '-' ...).
Note 1: Except for result any of these variables may be empty/unsupplied.
Note 2: The provided information follows the guidelines in RFC 6376.
5.10.4 SPF Logging
qmail-smtpd displays the evaluated SPF information under two distinct conditions:
- PASS: In case, the SPF information matches the received criterions, irrespectively of the SPF settings (SPF > 0) the email is labeled als SPF accepted and additional receiving information is added.
- FAIL: In case SPF is setup to reject failed SPF evaluation, a Fail is indicated in the logs.
5.11 Sender Rewriting Scheme SRS
Starting with s/qmail 4.0, the modules srsforward and srsreverse are available taking care of sending and receiving emails with a SRS SMTP envelope addresses, though without requiring to install libsrs2.
Both modules are used within a dot-qmail file and evaluate the control file srsdomains declaring domains to be treated with SRS addresses. A virtual user 'srs' included into the control file virtualdomains may take care of bounces with SRS addresses.
SRS takes care of problems in the SPF scheme:
|
If your MTA at mydomain.org receives emails from user@foo.org the original SMTP address is used unaltered while forwarding the mail. In case the domain foo.org has a SPF record set up and the receiving third party evaluates the SPF DNS TXT record of foo.org, presumable this forwarded email will not be accepted due to violation of the SPF policy at foo.org. With SRS, the forwarder (us at mydomain.org) can re-write the RCPT TO: addresss from foo.org with srsforward prior of forwarding the mail. |
It should be noted, that SRS is not standarised. It is in fact a variant of VERP, comparable with BATV.
Assuming, your domain name is mydomain.org mails received from user@foo.org and is subject for srsforward, it will re-write the SMTP Return-Path address (RCPT TO:) in the following (minimal) way:
user@foo.org ==> SRS0+HHH=TT=foo.org=user@mydomain.org
SRS address ingredients:
- HHH: These three letters perform a cryptographic hash.
- TT: A timestamp, making the delivery unique and mitigates forgery of bounces.
- '+': The chosen delimiting character given in the control file srsdomains; while '=' and '-' are allowed as well. '=' is the default delimiter and this character is alos used in the rest of the local part of the SRS address.
While srsforward and srsreverse are usually called from a dedicated user's dot-qmail file, rather s/qmail allows in addition to facilitate the alias user for this task.
5.11.1 Forwarding mails with SRS
Enabling SRS in s/qmail you can setup a control file srsdomains. Here, you define some recipient domain names which eventually will forward mails via SRS, including cryptographic material to be used and additional configuration parameters. Let's assume, we are a MTA at mydomain.org and additionally serve 2ndomain.com and badadmin.org. We may setup up this srsdomains file:
This results in the following SRS settings:
- The first line defines the default: SRS forwarding is enabled for all received domains, defining 'mysecret' as base for the hash 'HHH' with delimiter "'"and providing the RCPT TO: address as 'srs.mydomain.org': SRS0-HHH=TT=foo.org=user@users.mydomain.org
- The second line defines a particular behaviour for '2ndomain.com' using three secrets, defaulting to the standard delimiter "=", and using srs.mydomain.org as RTCP TO: domain part: SRS0=HHH=TT=foo.org=user@srs.mydomain.org
- The third line shows the possibility to disable particular domains acting as SRS forwarder.
SRS forwarding is possible for recipient domains provisioned in srsdomains with a dedicated secret. If for example foo.org would *not* be present in the srsdomains file given above, the default '*' would be used and any of the given 'secrets' would be valid.
The settings are passive unlike somebody uses srsforward explicitely in a dot-qmail file. Forwarding emails for domains including SRS enhanced SMTP (RCPT TO:) address is triggered upon (local) receipt by a dot-qmail file:
Typically, this is the user's dot-qmail file. However, it could be a 'virtual user' as well, in particular within a vpopmail or vmailmgr setup; thus applicable for Qmail toasters as well. As a result, the email would be forwarded with the SRS enhanced RCPT TO: SMTP envelope address to all domains given in the .qmail file.
It should be noted that the arguments for srsforward - the domain names - can be constructed on-the-fly potentially using the environment variables provided by qmail-local and described in qmail-command.
5.11.2 Receiving bounces with SRS addresses
Having set up SRS and using the scheme, this SRS-enabled domain is responsible for potential bounces, resulting in case the recipient mailbox is not reachable. This is, because the SRS host has substituted the orignal domain-part in the SMTP RCPT TO: envelope with its own.
Quote from libsrs2 regarding SRS reverse:
|
The return-path is used for bounce messages. If the target address is undeliverable, a bounce message is returned to the sender. Under classical forwarding, bounce messages are routed directly to the original sender. Under the new system, bounce messages are relayed back through the forwarding service, to the rewritten sender address. The forwarding service then unwraps the original sender and forwards the bounce.
|
To make it more explicit:
- With SRS enabled, a bounce mail (eg. if the recipient is unvaialable) will always target the forwarding MTA, since the SMTP 'RCPT TO:' has been rewritten and includes in the domain part now the forwarding host's domain name.
- The email needs to be received and thus a matching rcpthosts entry is required to be recognized by qmail-smtpd.
- The SRS address needs to be 'unwrap' and the bounce forwarded to the original sender, which is only possible in case of matching 'secrets'.
- The bounce sender in the SMTP MAIL FROM: needs to be chosen, but defaults to the NULL sender '<>'.
To provide such a service - here srsreverse - needs some care, because accepting unsolicited messages for third-party domains and forwarding those would result in an Open Relay. SRS copes with this, including a hash value and a time-stamp in the local part of the RCPT TO: address for SRS forwarded mails.
Here, we have two choices:
- We use our MTA's standard domain name for SRS forwarding and bounce handling..
- In a multi-tenant setup, we need to raise a subdomain to cope bounces received for any particular domain.
Given these circumstandes, handling of bounce mails with SRS envelope information (RCPT TO:) is simple:
srsreverse reconstructs from the SMTP envelope address of the incoming bounce mail the forseen (remote) recipient while verifying the legitimity of the SRS information; thus checking whether the bounce mail address is in accord with the information provided in srsdomains.
5.11.3 A usable setting for SRS
SRS forwarding shall be an exception. Mostly it happens, when mails need to be dispersed without a mailing listing manager like ezmlm. Here, sending domains might have a SPF record set, and thus need to get treated by SRS.
The handling of bounces is considered to be third order. You might follow these rules:
- Setting up ONE bounce policy for your MTA handling all SRS bounces: You need to setup a .qmail-default file (for the alias user) including the call of a srsreverse and handling bounces.
- More clever, you can raise a SRS subdomain (reachable via DNS MX), which needs to be configured in rcpthosts and locals e.g. as srs.mydomain.org together with a specific dot-qmail file (again for the alias user) .qmail-srs-default following the previous recipe.
- In an advanced multi-tenant environment, you can setup - aligned with your srsdomains definitions - dedicated dot-qmail files for virtual users together with entries in rcpthosts and virtualdomains.
5.11.4 Logging of SRS rewriting actions
Any delivery of srsforward and/or srsreverse will show up in the qmail-send log:
2020-02-25 14:01:34.029809500 delivery 2617: success: srsforward:_SRS0=shBI=4O=example.com=erwin@example.com/did_1+0+1/
5.12 DomainKeys Identified Mails
The task of DomainKeys Identified Mails (DKIM) is to provide autenticity and integrity for emails while adding a digital signature within the email header. This requires a cryptographic public/private key pair, a technical procedure, and a protocol which is defined in mainly
The DKIM protocol includes three major steps:
- The canonicalization of the email, based on the CRLF line format.
- The signing procedure resulting in a DKIM header which includes the required information how the signature was generated, where to find the public key (in DNS), and the signature itself.
- The verification operation while parsing the mail given by the protocol information, generating the signature material to be compared against the signature as unfolded by the public key.
This prodecure is unfortunately quite complex, since it includes several canonicalization schemes, two different hash functions (SHA1 and SHA256), and two signature mechanisms: RSA and Ed25516.
For s/qmail I've refactored the public domain libdkim library from the ALT-N project, straightend the C++ code, removed obsolete implementation details and included Edwards ECC cryptography based on Ed25519.
The resulting module qmail-dkim is however API-compatible with libdkim given the fehQlibs are used additionally in order to provide DNS services.
In order to support DKIM signing and verification the following modules are now part of s/qmail:
- qmail-dkim the main module for signing and verification, which can be used stand-alone.
- qmail-dksign a stub-module called from qmail-rspawn, providing the input message at queue/dkim/X in order to sign it by qmail-dkim and storing it under queue/dkim/Y ready for qmail-remote to be send.
- qmail-dkverify a is stub-module being responsibe for DKIM verification and called via the QMAILQUEUE="bin/qmail-dkverify" meachanism (QMAILQUEUE_EXTRA) invoked by qmail-smtpd. This will provide the message with enhanced CRs at queue/dkim/X to be verified by qmail-dkim. The CR stripped message is scheduled to qmail-queue.
qmail-dkim supports the following signature algorithms:
- RSA with SHA1
- RSA with SHA256
- RSA with SHA1 and SHA256
- Ed25519 (with SHA256)
- Ed25519 with RSA-SHA256
In order to operate and to provide the canonicalization, this version of s/qmail (4.2.x) raises an additional directory structure: queue/dkim/N/, where N is the argumente given in conf-split. These directories are thus staging directories for DKIM. This implies, that any DKIM processed email is read and written several times, which decreases performance. Due to the many operational steps and interdependencies (as seen in the figures above) a lot of concurrent file descripictors are used. This limit may be raised far above 1024 in order to achieve a high qmail-remote concurrency. Otherwise, the following message in qmail-sends's log may be seen:
Under Linux, you need to raise the available file descriptors. Typically the setting is done via /etc/security/limits.conf. Here, you need to include the following:
Now, you raise the FD limit to 4096 for both the users qmails and qmailr. Depending on your throughput, this number shall be even higher! In order to become effective, you need to restart qmail-send.
RSA SHA-256 and Ed25519 keys required:
- A RSA private and public key, where the public key is presented as DER-enhanced and BASE64 encoded key in the DNS as TXT record typically given as default._domainkey.example.com where default is called the 'selector'.
- An Ed25519 private and public key, where the public key is given as 'naked' BASE64 encoded key in the DNS with a distinct selector. Here, I have chosen the default name to be'eddy' resulting in an DNS name eddy._domainkey.example.com.
5.12.1 DKIM sign outgoing messages by qmail-dksign
qmail-dksign is a setuid module:
and is only invoked by qmail-rspawn if the control file
is present. Use the file control/dkimdomains to instruct qmail-dkim how to DKIM sign an outgoing message and provides the following customization:
Apart form the senddomain followed by a colon, all other parameters are optional. By default, a RSA signatature for SHA256 is generated.
- selector is used to prepend the artificial DKIM domain name
selector._domainkey.example.com, where example.com is
the sending domain for RSA signatures.
If not explicitely given, it defaults to 'default'. - selector2 is used for binding of the Ed25519 public key.
If not explicitely given, it defaults to 'eddy'. - SDID (Signing Domain Identifier) defaults usually to senddomain but may be chosen to any other FQDN. This permits to use the same FQDN in the DNS for different sending domains.
- AUID is the Agent/User Identifier and may contain
additional information for some policy to be obeyed. Here,
it has no specific meaning.
In case AUID matches '~' (the 'tilde') automatically the provided 'Mail From:' address is substituded here. - expire is the validity period of the signature in seconds and default to 604800.
- r|s|t|u specifies, how to canonicalize the message:
- s simple/strict,
- t relax on header, simple on body,
- u simple on header, relax on body.
- r is relax on header and body, the default here.
- 1|2|3|4|5 is the signature algorithm:
- RSA-SHA1.
- RSA-SHA256 (default).
- RSA-SHA1 + RSA-SHA256.
- Ed25519.
- Ed25519 + RSA-SHA256.
- l tells to include the length of the message in the DKIM header.
Note: selector and SDID can be chosen freely for any DKIM key but need to reflect the settings in the DNS for the domain name given of the DKIM TXT record.
5.12.2 DKIM key repository: ssl/domainkeys/fqdn
In s/qmail/ the X.509 cerificates, private keys, and other TLS materials are stored in the protected direcory /var/qmail/ssl/. This is enhanced now for the subdirectory domainkeys followed by the sending domain:
In order to work, a private domainkey file needs to exist at:
This structure roughly invertes the DNS name given for the DKIM public key:
Here, fqdn could be either the domain name or the hostname of the MTA.
- The file name <selector> needs to coincide with its setting in control/dkimdomains for the given domain.
- You can avoid the name in control/dkimdomains if the RSA <selector> defaults to 'default' or the Ed25519 <selector2> defaults to 'eddy'.
- Several different <selector>s may exisit in the domain's subdirectory; but only the one explicitely (or implicitely) given by control/dkimdomains becomes active for signing.
5.12.3 Generating DKIM keys
Domainkeys are generated by mkdkimkey; a script available in s/qmail's
installation diretory ./scripts.
In order to enable it, follow these steps:
- Within ./scripts call make. This will customise the shell script and creates mkdkimkey.
- Copy this script to some useful location; potentially the s/qmail bin directory for easy use.
Upon calling the script without arguments, you get the usage information:
mkdkimkey will automatically generate the required keys in the target directory given by domain. Please obey the permissions of the keys. The domainkey private file is automatically raised to be readable only by qmail-dkim operating as qmailq user:
- The generated key files always carry the selector as appendix and thus the matching private and public key file can be identified unambiguously.
- Existing key files with the same selector name, are not automatically overwritten.
- In case of overwriting key files with the same selector name, the old ones are copied with extension .previous.
- The generated private key file is automatically symlinked to its selector name.
- Key generation can be done on-the-fly without side-effects.
5.12.4 Provisioning the DKIM public key in DNS
mkdkimkey will of course also generate the public key for your domain. You can always ask for a BIND compliant output to be used as DKIM TXT record for your DNS authoritive server:
It is important to understand, that only this step defines the binding between the domain name and the public key, which of course needs to be complemented by the private key.
Within the DKIM TXT record you need to specify which signature algorithm is used and you may include constraints on the hash function and on the usage. This is detailed in RFC 6376 Section 3.6. You may modify the generated DNS TXT record manually for your needs and deploy it afterwards in the DNS.
In case, you have a different DNS authoritative server (like tinydns), use their own generic format.
5.12.5 Chosing the DKIM signature procedure
qmail-dksign works multi-tenant, thus you can create individual (or none) keys per sending domain. Apart from that it allows parenting, thus a subdomain inherits the key of its parent, and also a default signing procedure. In any case: More specific settings have precedence over less specific ones.
In the file dkimdomains the following information can be given to be provisioned to qmail-dkim:
- The sending domain before the colon (:). An asterix (*) is used to command signing an email origininating from any domain (even forwarded) with your domainkey.
- The selector which is used to prepend the domainkey to allow key rollover. The default selector is default but can be customized. The private is always referenced as default irrespectable of your choice. Use selector as suffix for the private key.
- The SDID (Sending Domain Information Identitifer). This allows you to use ONE domainkey to be supplied for serveral domains.
- The AUID (Agent User Identity) used to delegate the DKIM signature to another domain.
- The Lifetime of the signature. A receiving MTA should not accept DKIM signed message where the lifetime between sending and receiving is expired.
- The Canonicalization of the email message (relaxed, simple, ...).
- The Signature Algorithm to be used (1 or 2 or combined -3- for RSA or 4 for Ed25519).
- Commands to include the Length of the signed message in the DKIM header (potentially required, if additional footers are added).
Individual domains can be excluded for signing. Here, you can indicate those with a prepending exclamation mark (!). Thus, control/dkimdomains allows you to provide a complete description of your DKIM settings per sending domains. However, there are the following drawbacks:
- You can specify to use concurrent RSA/SHA1 and RSA/SHA256 signatures to be included in the email's DKIM headers (they use the same private key).
- You always have to create a working symlink binding a domain in control/dkimdomains
with the actual available DKIM private key under ssl/domainkeys/<domain>/signkey.
Otherwise, qmail-rspawn
will complain with the following error message:
delivery 106: failure: Unable_to_run_qmail-remote./
5.12.6 DKIM Key-Rollover
It should be a standard procedure to generate new DKIM keys after a certain period, at least each year; depending on the signature strength.
In order to become effective, a new (public) domain key needs to be provisioned in the DNS some time before the private key is used for signing. The time span depends on the TTL of the DKIM key in DNS and should be at least a small multiple of it. Several DKIM DNS records with different selectors per domain may co-exist.
To allow a smooth transition, the old and the new public key need to be present in the DNS. A DKIM validating application, like qmail-dkverify will fetch both and try both for signature verification and succeed, if one key matches.
To support key-rollover, generate new DKIM keys. The new one shall be given a different selector comprising the DNS TXT record newselector._domainkey.example.com to be published in the DNS alongside with the old one. This old one should become retired and removed from the DNS in the next days, after the new selector was used for signing.
mkdimkkey generates new keys while appending the selector information in both the private and the public key. This prevents any ambiguity about the matching of private and public keys.
5.12.7 DKIM Key-Sharing
Another common usage allows to use the same public/private key pair for different domains. Here, you need to synchronize your local data as provisioned to qmail-dksign and the data in the DNS. The basic idea is, that all domains referring to the same public key in DNS share the same private key by a symlinking the source in ssl/domainkeys/.
- If you have raised keys for mydomain.org and would like the use
the some for foreign.net set up the link:
ssl/domainkeys/foreign.net -> ssl/domainkey/myddomain.org. - Include an entry in control/dkimdomains:
mydomain.org:
foreign.net:|mydomain.org|
Thus, the DNS DKIM name - the SDID - to be looked up as provisioned in the DKIM header in the mail, points to an already existing DNS record while allowing you to save raising additional entries in the DNS and thus using the same selector, private and public key for several domains for DKIM signing and verification.
5.12.8 DKIM signature header
The information included in the DKIM header generated by qmail-dksign depends on you settings for the respective domain in control/dkimdomains. Let's give a sample using Ed25519 signatures:
The relevant pieces here are:
- d=exmple.com: The domain name for which the DNS DKIM lookup shall be raised for (SDID) enhanced with
- s=default, the selector,
- i=me@example.com the AUID (not used here),
- q=dns/txt use a DNS query to fetch the public key,
- h=Receveive:Date... the email headers to be included in the signing procedure,
- b=d7... the hash of body (here: SHA256),
- b=d7wD ... the calculated Ed25519 signature covering message headers and including the value of the body hash as well.
Note: Unlike RSA, Ed25519 does not allow to calculate the signature in a 'stream' format; rather it is done in one step only.
5.12.9 Verifying DKIM signed inbound messages by qmail-dkverify
The verification of DKIM signed email is possible upon receipt by qmail-dkverify used queue as plug-in called by qmail-smtpd while setting up: QMAILQUEUE="bin/qmail-dkverify" either in the start script or within the sslserver's cdb, which is the recommended way. This allows to fine-grain the DKIM verification and also to avoid 'false-negatives' for domains with a wrong DKIM DNS setup.
qmail-dkverify is again a setuid module
and works usually in annotation mode only. In case a a DKIM public key could be fetched from the respective DNS DKIM TXT record and the verification was successful, this this shown in the received email header only:
The 'X-Authentication-Results' header is included showing the result 'pass', the sending MTA 'piplus.example.com', and the DKIM verifying MTA: 'bigchief.heaven.com'. This additional header is based on the evaluation of the DKIM header and the DNS DKIM TXT public key for the domain (and selector) indicated in the signature header below, In case of a failure, the reason for this is explained verbose in parenthesis.
As can be seen: The DKIM signature is completely voluntary. Thus it is only provisioned if all boundary conditions are met. Therefore, it ist not useful as a trigger/filter for mail abuse. 'False-negatives' are mainly due to mis-configuration or expired/revoked keys.
However, you can reject emails with failed DKIM signature setting additionally the environment variable DKIM=+ alongside with the QMAILQUEUE="bin/qmail-dkverify" setting.
5.12.10 Addendum: DKIM Settings in DNS TXT Records
A DKIM record in the DNS is a simple TXT record, however with a defined data structure (RFC 6376):
- The data section consists of individual items.
- Each item is introduced by single character (the tag) followed by an equal sign (=): The identifier.
- Each item is terminated by a semi-colon ';' except for the last token, where this can be omitted.
- White spaces in the data section are permitted, but should be avoided to shorten the DNS information.
DKIM items:
| Identifier | Item | Mandatory |
| v= | DKIMv1 | yes |
| p= | public key | yes |
| k= | Signature algorithm (rsa/ed25519) | no |
| h= | Hash algorithm (sha1/sha256) | no |
| s= | Service Type ('email' or '*') | no |
| t= | Test flag ('y'/yes or 's'/special) | no |