ezmlmx 0.68
ezmlmx
Loading...
Searching...
No Matches
ezmlmx FAQ 0.6 - consolidated ezmlm-idx and ezmlm FAQ

Fred Lindberg, lindb.nosp@m.erg@.nosp@m.id.wu.nosp@m.stl..nosp@m.edu & Fred B. Ringel, fredr.nosp@m.@riv.nosp@m.ertow.nosp@m.n.ne.nosp@m.t

22-NOV-1999

This document is a collection of frequently asked questions about ezmlm-idx. Where applicable, ezmlm itself is also covered. This FAQ presumes familiarity with Unix, and with the basic concepts of E-mail and mailing lists. This FAQ is updated for ezmlm-0.53 and ezmlm- idx-0.40.

Erwin Hoffmann, feh@f.nosp@m.ehco.nosp@m.m.de

11-MAY-2025

Since ezmlmx is based on ezmlm-0.53 and ezmlm-idx-0.41, this documetation can be used unaltered here. Only minor changes and corrections were applied apart from the mentioned URLs of course and installations and configuration adjustments. Rather, some explanations about mailing lists are now included.

Note: Since I use this documentation almost unaltered, the term 'ezmlm-idx' used is equivalent with what you get: 'ezmlmx'. Dan Bernstein's original version is called 'ezmlm proper' here. Further, since s/qmail inherits all attributes of qmail and provides the same APIs, the usage of 'qmail' in this FAQ always includes 's/qmail'. Likewise, the original version of qmail is often called 'vanilla qmail' to distinguish it from its forks. Within this document, potentinal differences among 'vanilla qmail' and 's/qmail' are explicitely mentioned.

Table of Contents

  1. General Information.
    1. Acknowledgements
    2. What is this document?
    3. Terminology
    4. What is the difference between ezmlm and ezmlm-idx?
    5. Where can I get all of the ezmlm-related programs?
    6. What are the basic differences among the ezmlm forks?
    7. Where can I find documentation for ezmlm and its different forks/versions?
    8. Where do I send comments on this document?
    9. How to experiment with new versions of ezmlmx.
    10. What is special about ezmlmx and s/qmail?
  2. Quick start.
    1. Requirements.
    2. Slashpackage installation.
    3. Testing.
    4. Making a list and digst list.
  3. Overview of mailing list management and mailing list managers.
    1. How e-mail mailing lists work.
    2. Subsribing to e-mail distibution lists.
    3. The cookie mechanism.
    4. ezmlm subscriber privileges.
  4. Overview of ezmlm function.
    1. The basic setup.
    2. Inventions in ezmlm.
    3. The qmail delivery mechanism.
    4. What the different programs do.
      1. What the directory structure is responsible for.
      2. What the 'dot-qmail' files are responsible for.
      3. What dir/editor is good for.
      4. Why dir/bouncer is needed.
      5. For what dir/manager is used.
      6. For what dir/moderator is responsible for.
      7. Whay ezmlm needs a configuration file for every list.
      8. How ezmlm deals wit email headers.
    5. What the different files in the list directory do.
    6. The paper path for posts.
    7. The ezmlm path for moderation messages.
    8. The ezmlm path for administrative messages.
    9. The ezmlm path for bounces.
    10. Messages to list-owner and list-digest-owner.
    11. Structure of subscriber databases.
    12. Local case in E-mail addresses.
    13. Testing SENDER to allow posts only from list subscribers.
    14. How cookies work.
    15. How moderator E-mail addresses are stored.
    16. How subscription moderation works.
    17. How remote administration works.
    18. How message moderation works.
    19. How QMQP support works
    20. How messages are stored in the archive.
    21. How the message index works.
    22. How threading works.
    23. How digests work.
    24. How ezmlm-tstdig works.
    25. How WWW archive access works.
    26. How sublists work.
    27. How sublisting can be made transparent to the user.
    28. How to service commands in the subject line.
    29. How to support alternative command names.
    30. How to add your own commands.
    31. How remote administrators can retrieve a subscriber list
    32. How remote administrators can determine the number of subscribers
    33. How remote admins can see if an address is a subscriber or not
    34. How remote administrators can search the subscription log
    35. How text file editing works.
    36. How subject line prefixes work.
    37. How bounces are handled.
    38. How the info and faq commands work.
    39. How the global ezmlm list address works.
    40. How ezmlm-cron works.
    41. How ezmlm-make works.
    42. What names can I use for my lists?
    43. Lists in virtual domains
    44. How do I make customization simple for me/my users?
  5. ezmlm support for SQL databases.
    1. Why use an SQL database with ezmlm?
    2. Why not to use an SQL database with ezmlm.
    3. Tables used for (My)SQL support.
      1. Address tables.
      2. Subscriber log tables.
      3. Message logging tables.
    4. How to set up a simple list with SQL support.
      1. Helper programs for SQL-enabled lists.
    5. Manually manipulating the subscribers of a SQL-enabled list.
    6. Converting to and from and SQL database.
    7. Optimizing MySQL for ezmlm.
      1. Address SELECTs, additions, removals.
    8. Maintenance of the MySQL database.
  6. Possible error conditions in ezmlm lists.
    1. What do I do if ezmlm doesn't work?
    2. How do I report ezmlm bugs?
    3. Where do I send suggestions for ezmlm-idx improvements?
    4. Using ezmlm-test to check the ezmlm(-idx) programs.
    5. Using ezmlm-check to find setup errors.
    6. Posts are rejected: Sorry, no mailbox here by that name (#5.1.1).
    7. Post are not sent to subscribers.
    8. ezmlm-make fails: usage: ezmlm-make ...
    9. ezmlm-make fails: Unable to create ...
    10. ezmlm-make fails: ... ezmlmrc does not exist
    11. Index/get/thread requests fail quietly or with errors from ezmlm-manage.
    12. Digest triggering requests fail.
    13. Remote administration (un)subscribe confirm requests go to the user, not the moderator.
    14. (Un)subscribers does not receive a (un)subscribe acknowledgement
    15. Messages posted to a moderated list are sent out without moderation.
    16. Messages posted to a moderated list do not result in moderation requests.
    17. Moderation request replies do not result in the appropriate action.
    18. Moderator comments with moderation request replies are not added to the post/sent to the poster.
    19. Some headers are missing from messages in the digest.
    20. Some Received: headers are missing from messages.
    21. My Mutt users cannot thread their digest messages.
    22. Posts fail: Message already has Mailing-List (#5.7.2).
    23. The last line of a
    24. No CONFIRM requests are sent to moderators.
    25. Deliveries fail 'temporary qmail-queue error'
    26. How to deal with corrupted subscriber lists
    27. Vacation program replies are treated as bounces by ezmlm.
    28. Digests do not come at regular hours.
    29. Preventing loops from misconfigured subscriber addresses.
    30. A user can subscribe and receives warning and probe messages, but no messages from the list.
  7. Customizing ezmlm-make operation via ezmlmrc.
    1. Using ezmlm-make to edit existing lists.
    2. What is ezmlmrc?
    3. Changing defaults for
    4. Changing default moderator directories.
    5. Adapting ezmlm-make for virtual domains.
    6. Setting up ezmlm-make for special situations.
  8. Restricting message posting to the list.
    1. Requiring the list address in To:/Cc: headers.
    2. Rejecting messages sent from other mailing lists.
    3. Restricting posts based on the Subject line.
    4. Restricting the size of posts.
    5. Restricting posts based on MIME content-type.
    6. Restricting posts to list subscribers.
    7. Restricting posts to an arbitrary set of E-mail addresses (higher security option).
    8. Completely restricting posts.
    9. A general solution to restricting posts based on SENDER.
  9. Customizing outgoing messages.
    1. Adding a trailer to outgoing messages.
    2. Adding a subject prefix to outgoing messages.
    3. Adding a header to outgoing messages.
    4. Adding a message number header.
    5. Removing headers from outgoing messages.
    6. Removing MIME parts from messages.
    7. Limiting 'Received:' headers in outgoing messages.
    8. Setting 'Reply-To: list@host'.
    9. Configuring the list so posts are not copied to the original sender.
    10. Customizing ezmlm notification messages.
    11. Specifying character set and content-transfer-encoding for outgoing ezmlm messages.
  10. Customizing archive retrieval.
    1. Specifying the format for retrieved messages.
    2. Specifying the default format for digests and archive retrieval.
    3. Limiting the number of messages per -get/-index request.
  11. Restricting archive retrieval.
    1. Restricting archive access to subscribers.
    2. Restricting available archive retrieval commands.
    3. Restricting archive retrieval to moderators.
    4. Allowing archive retrieval from a non-public list.
  12. Customizing digests.
    1. Setting up a digest list.
    2. Generating daily digests.
    3. Generating the first digest.
    4. Adding standard administrative information to digests.
    5. Controlling the digest format.
    6. Customizing bounce handling.
  13. Remote administration.
    1. How can I remotely add moderators, subscriber aliases, etc?
    2. Moderating posts from a secondary account.
    3. Moderating subscription from a secondary account.
    4. Automatically approving posts or subscriptions.
    5. Allowing remote administrators to get a subscriber list.
    6. Allowing remote administrators to retrieve or search a subscription log.
    7. Allowing users to get a subscriber list.
    8. Changing the timeout for messages in the moderation queue.
    9. Finding out how many messages are waiting for moderation.
    10. Using the same moderators for multiple lists.
    11. Using different moderators for message and subscription moderation.
    12. Setting up moderated lists with the list owner as the 'super moderator' able to add/remove moderators remotely.
    13. Customizing ezmlm administrative messages.
    14. Manually approving a message awaiting moderation.
    15. Manually rejecting a message awaiting moderation.
  14. Sublists.
    1. Sublists of ezmlm lists.
    2. Sublists of non-ezmlm lists.
    3. How to set up a cluster of list and sublists with standard databases.
  15. Migration to Ezmlm from other Mailing List Managers.
    1. Basic Concepts.
    2. Setting up ezmlm to respond to host-centric commands.
    3. Commands of other mailinglist managers recognized by ezmlm.
      1. Listproc/Listserv.
      2. Majordomo.
      3. Smartlist.
  16. Optimizing list performance.
    1. Crond-generated digests for better performance.
    2. Optimizing execution of ezmlm-warn(1).
    3. Decreasing ezmlm-warn time out to increase performance.
    4. Use ezmlm without ezmlm-idx for maximum performance.
    5. Not archiving to maximize performance.
    6. Sublists to maximize performance.
  17. Miscellaneous.
    1. How do I quickly change the properties of my list?
    2. Open archived list with daily digests.
    3. Variations in moderation
    4. Lists that allow remote admin, but not user initiated subscription or archive retrieval.
    5. Lists that allow remote admin, user archive retrieval, but not user-initiated subscription.
    6. Lists that restrict archive retrieval to subscribers.
    7. Lists that do not allow archive retrieval at all.
    8. Lists that do not allow archive retrieval and do not allow digest triggering per mail.
    9. Lists that allow archive retrieval only to moderators, but allow user-initiated subscription.
    10. Lists that do not require user confirmation for (un)subscription.
    11. Announcement lists for a small set of trusted posters
    12. Announcement lists allowing moderated posts from anyone.
    13. Announcement lists with less security and more convenience.
  18. Ezmlm-idx compile time options.
    1. Location of binaries.
    2. Location of man pages.
    3. Base directory of qmail-installation.
    4. Base directory of the qlibs.
    5. Short header texts, etc.
    6. Arbitrary limits.
    7. Command names.
    8. Error messages.
    9. Paths and other odd configuration items.
  19. Multiple language support.
    1. Command names.
    2. Text files.
    3. Multi-byte character code support.
  20. Subscriber notification of moderation events.
    1. General opinions.
    2. Users should know that the list is subscription moderated.
    3. Subscribers should know that posts are moderated.
    4. Senders of posts should be notified of rejections.
  21. Ezmlm-idx security.
    1. General assumptions.
    2. SENDER manipulation.
    3. ezmlm cookies.
    4. Lists without remote admin/subscription moderation.
    5. Message moderation.
    6. Subscription moderation.
    7. Remote administration.
    8. Remote editing of ezmlm text files.
    9. Digest generation and archive retrieval.
    10. Convenience for security: the ezmlm-manage '-S' and '-U' switches.
    11. Denial of service.
    12. Moderator anonymity.
    13. Confidentiality of subscriber E-mail addresses.
    14. Help message for moderators.
    15. Sublists.
    16. SQL databases.
    17. Reporting security problems.

1. General Information.

1.1 Acknowledgements.

Many ezmlm users have contributed to improvements in ezmlm-idx. These are listed in the AUTHOR file in the ezmlm-idx distribution. Others have through questions and suggestions inspired parts in this FAQ, or pointed out errors or omissions. Thanks! Direct contributions are attributed to the respective authors in the text. Thanks again!

While I was preparing ezmlmx, I reached out for Bruce and Frank to ask for a permission and blessing for the the fork. Unfortunately, none of them answered, Fred's email address is gone and Bruce did not reply. However, again a Many Thanks from me to the original authors! I try to keep their spirit and hope to provide an acceptable fork.

1.2 What is this document?

This FAQ contains answers to many questions that arise while installing ezmlm, ezmlm-idx, and while setting up and managing ezmlm mailing lists. See 'README.md' for a brief summary of what is ezmlm and what exmlmx differes from ezmlm-idx.

Many aspects of ezmlm are covered in several places in this FAQ. The early sections explain how ezmlm works. Later sections discuss how to deal with possible errors/problems. Subsequent sections discuss details of customization and list setup in a HOWTO form. Finally, there are sections on information philosophy for moderated lists and on security aspects on ezmlm lists.

This is an evolving document. If you find any errors, or wish to comment, please do so to the authors. This FAQ is currently aimed at system administrators and knowledgeable users, and heavily weighted towards questions specific to the ezmlm-idx add-on.

If you have problems with the ezmlm-idx package, please start by reading the 'man' pages which come with each program, then this document and other ezmlm documentation which is identified here. If you have exhausted these resources, try the ezmlm and qmail mailing lists and their respective mailing list archives. If you have solved a problem not in the documentation, write it up as a proposed section of a FAQ and send it to the authors. This way, it can be added to the next version of this FAQ.

1.3 Terminology.

This document uses a number of terms. Here are the meanings ascribed to them by the authors.

DIR

The base directory of the list.

SENDER

The envelope sender of the message, as passed to ezmlm by qmail via the SENDER environment variable.

LOCAL

The local part of the envelope recipient. For *list-get-1@host*, it is usually list-get-1. If host is a virtual domain, controlled by user-sub then local would be user-sub-list-get-1.

moddir

Base directory for moderators. Moderator E-mail addresses are stored in a hashed database in moddir/subscribers/. By default, 'moddir' is DIR/mod/.

To add or remove moderators: 

% ezmlm-sub DIR/moddir moderator@host.domain
% ezmlm-unsub DIR/moddir moderator@host.domain

dotdir

The second argument of ezmlm-make is the main .qmail file for the list. dotdir is the directory in which this 'dot file' resides, i.e. the directory part of the 'dot' argument. This is usually the home directory of the user controlling the list (but NOT necessarily of the one creating the list). Thus, dotdir is *~alias/* if 'root' creates a list: 

# ezmlm-make ~alias/list ~alias/.qmail-list ...

dotdir is where the .#.ezmlmrc file is expected when the ezmlm-make(1) '-c' switch is used (see 'Customizing ezmlm-make operation').

ezmlm binary directory

The directory where the ezmlm-binaries are normally stored, as defined at compile time in conf-home. This is compiled into the programs and does not change just because you have moved the program.

ezmlm-get(1)

This is a reference to the ezmlm-get.1 man page. Access it with one of the following: 

% man ezmlm-get
% man 1 ezmlm-get

or if you have not yet installed ezmlmx, perhaps modify conf-man to your needs and call

% package/man

basedir

The list directory when referencing the list subscriber address database. For E-mail addresses stored in a set of files within DIR/subscribers/ the 'basedir' is 'DIR'.

address database

A collection of E-mail addresses stored in a set of files within the 'subscribers' subdirectory of the basedir, DIR/subscribers/.

message moderator

An address to which moderation requests for posts to the list are sent. The moderation requests are formatted with From:'-'reject' and a 'To:'-'accept' default headers for moderator replies. A reply to the 'reject' address leads to the rejection of the post. A reply to the 'accept' address leads to the acceptance of the post. Any E-mail address can be a moderator E-mail address. Any number of moderator E-mail addresses can be used. If a post is sent from a moderator E-mail address, the moderation request is sent to that E-mail address only. If a post is sent from an E-mail address that is not a moderator, a moderation request is sent to all moderators.

The first reply to the moderation request determines the fate of the message. Further requests for the action already taken are silently ignored, while a request for the contrary action results in an error message stating the actual fate of the message. Thus, if you want to 'accept' the message and it has already been accepted, you receive no reply, but if you attempt to 'reject' it, you will receive an error message stating that the message already has been accepted.

Most lists are not message moderated. If they are, the owner is usually a 'message moderator', sometimes together with a few other trusted users.

For an announcement list, it is common to make all the 'official announcers' 'message moderators'. This way, they can post securely and 'accept' their own posts, while posts from other users will be sent to this set of 'official announcers' for approval.

subscription moderator

An E-mail address where subscription moderation requests are confirmed her intention to subscribe. The subscription moderation request is sent to all moderators. As soon as a reply to this message is received, the user is subscribed and notified. Any E-mail address can be a subscription moderator and any number of subscription moderators can be used.

Unsubscribe requests are never moderated (except when the ezmlm-manage(1) '-U' flag is used and the sender attempts to remove an address other than the one s/he is sending from). It is hard to imagine a legitimate mailing list that would want to prevent unsubscriptions.

remote administrator

When a remote administrator subscribes or unsubscribes a list member, the 'confirm' request is sent back to the remote administrator, rather than to the subscriber's E-mail address. This allows the remote administrator to (un)subscribe any list member without the cooperation of the subscriber at that address. Any E-mail address can be a remote administrator and any number of E-mail addresses can be remote administrators.

The set of E-mail addresses that are 'remote administrators' and 'subscription moderators' are always the same. This set of E-mail addresses can be 'remote administrators', 'subscription moderators' or both.

For most lists, the owner would be the 'remote administrator', if s/he wishes to moderate messages, the owner would be the 'message moderator' and if s/he wishes to moderate subscriptions the owner would also be the 'subscription moderator'.

The list's 'message moderator(s)' can be the same, but can also be set up to be completely different.

Changing list 'ownership'

Within this FAQ there are references to the need to check or change the list 'ownership.' This is not a reference to the individual user who is the 'list-owner', but a reference to the ownership of the files by your operating system which make up the list and reside in DIR/.

To change the ownership of DIR/ and everything within: 

% chown -R user DIR
% chgrp -R group DIR

Depending on your system/shell, it may be possible to combine these commands into either:

% chown -R user:group DIR % chown -R user:group DIR

1.4 What is the difference between ezmlm, and ezmlm-idx, and ezmlmx?

ezmlm-0.53 is a qmail-based mailing list manager written by Dan J. Bernstein. It has all the basic functionality of a mailing list manager, such as subscriber address management including automated bounce handling as well as message distribution and archiving.

ezmlm-idx is an add-on to ezmlm. It adds multi-message threaded message retrieval from the archive, digests, message and subscription moderation, and a number of remote administration function. It modifies the configuration program ezmlm-make(1) so that it uses a text file template rather than compiled-in texts in list creation. In this manner, ezmlm-idx allows easy setup of lists in different languages and customization of default list setup. ezmlm-idx also adds MIME handling, and other support to streamline use with languages other than English. As an ezmlm add-on, ezmlm-idx does not work without ezmlm and tries to be compatible with ezmlm as much as possible. ezmlm-idx also modifies the ezmlm subscriber database to be case insensitive to avoid many unsubscribe problems.

New in ezmlm-idx-0.40 are better support for announcement lists, support for QMQP to offload message distribution onto external hosts, simplified optional SQL database use (MySQL or PostgreSQL), more flexibility in determining which messages should be moderated, a WWW interface to the list archives, and many small improvements.

ezmlm-idx-0.32 adds improved handling of very large lists with optimized bounce handling, ezmlm-split(1) for forwarding (un)subscribe requests to sublists to allow sublisting transparent to the subscriber, and SQL support to allow sublisting with improved message authentication and monitoring of list function, as well as dynamic addition/removal/reconfiguration of sublists. Also, subscriber 'From:' lines are logged with support for finding a subscription address from a name. The qmail DEFAULT variable is used, if present. Together, these additions eliminate the most common problems making ezmlm use and administration even easier.

This document is a FAQ for ezmlmx based on ezmlm-idx. However, many of the basic items that are discussed also apply to ezmlm per se. Referring to the two paragraphs above, it should be relatively easy to figure out which features require ezmlm-idx and now ezmlmx.

1.5 Where can I get all of the ezmlm-related programs?

We have now registered ezmlm.org to make access to ezmlm-idx and related programs/documentation easier. www.ezmlm.org is currently an alias for Fred B. Ringel's www.rivertown.net/~ezmlm/ and ftp.ezmlm.org an alias for Fred Lindberg's ftp.id.wustl.edu.

In 2025, the following sites are still online:

a) Dan J. Bernstein:

b) Bruce Guenter's and Frank Lindberg's ezmlm-idx:

c) My djbware including s/qmail and ezmlmx:

d) Unmaintained and just for reference:

e) Roberto's fixes for qmail:

1.6 What are the basic differences among the ezmlm forks?

D. J. Bernstein started ezmlm development in 1995 along side with qmail because he was unsatisfied with the performance and security of existing email manager solutions back than. Remember, this was the time, when sendmail was the dominant solution; neiter Postfix nor Exim have been started yet; smail was however already available. ezmlm-0.53 was the last stable solution, though based on 'ANSI-C'.

About 1998 Bruce Guenter, Frank Lindberg, and Frank Riegel started with an enhanced version of ezmlm, developed as patch (given Dan's license requirements) and called it ezmlm-idx. Meanwhile Bruce Guenter and Fred Lindberg updated ezmlm-idx and produced the versions ezmlm-idx-6.xy and ezmlm-idx-7.xy providing better support for internationalization and an enhanced language template scheme for mailing lists. Further, they cared about new RFC 2821 email headers.

In 2025 I decide to provide an updated version of ezmlm-0.53 + ezmlm-idx starting from ezmlm-0.53 + ezmlm-idx-0.44 which I used back in 2000 for a large production site including a Web front end for managing the (many) large mailing list in conjunction with qmail 1.03 and my Spamcontrol patch.

Thus, ezmlmx inherits ezmlm-0.53 and the older version of ezmlm-idx (0.44). The newer language templates of ezmlm-idx won't work in the current ezmlmx version, but may be used in forthcoming releases.

From the original ezmlm-idx document:

ezmlmrc(5) files for different languages

The latest versions at the time of release of a package are included in that package. Thus, this directory will have a file labeled with the current ezmlm-idx version number only if it has been updated later than the package. ezmlmrc(5) files are updated and new ones are added all the time, also with bug fix releases. Therefore, always look at the latest package. Please note that ezmlmrc may change significantly between versions. Thus, do not expect the ezmlm-idx-0.324 ezmlmrc.es to work with ezmlm-idx-0.40.

ezmlmrc(5) files contain some release-specific configurations. Do not use a later file (other than from bug fix releases) with an earlier version of the programs. It is usually OK to use a version from an earlier package (see UPGRADE.idx), but some new functionality may nor be available.

To contribute an ezmlmrc(5) file in a new language, start with the en_US version from the latest package, and send the gzipped file to lindb.nosp@m.erg@.nosp@m.id.wu.nosp@m.stl..nosp@m.edu. Please leave comments intact and in English and do not change the order of items in the file. This will facilitate maintenance.

1.7 Where can I find documentation for ezmlm and its different forks/versions?

a) Relevant Internet 'Request for Comments' (RFC)

b) Inline documenation

man pages

All ezmlm component programs come with their own man pages. Thus, for info on ezmlm-send, type: 

 % man ezmlm-send

or if you have unpacked ezmlm, but not made it or installed it: 

 % cd ezmlmx-V.RR/man
 % man ./ezmlm-send.1

ezmlm(5)

general info on ezmlm and list directories is in ezmlm.5: 

 % man ezmlm

or 

% cd ezmlmx-V.RR/man
% man ./ezmlm.5

Note: Installation of the ezmlmx package updates some existing man pages to reflect changes made by the patch (e.g. ezmlm-send(1), ezmlm(5)).

Text files in the distribution

ezmlmx comes with a README file with general instructions, an INSTALL file with installation instructions and a CHANGES file with information on changes from previous versions.

This FAQ

This FAQ is built from a SGML source by the original authors:

  • Fred Lindberg
  • Bruce Guenter
  • Fred Riegel

and copy-edited to Markdown while updating it for ezmlmx.

d) Online tutorials

e) Mailing lists

Please read other documentation and mailing list archives before posting questions to the lists. It's also useful to 'lurk' on the list for a few days, (i.e. to subscribe and read without posting) before asking your questions on the list.

To subscribe, send mail to the E-mail addresses listed:

1.8 Where do I send comments on this document?

To the follow-up author:

1.9 How to experiment with new versions of ezmlmx.

ezmlm-idx writes DIR/config in a standard format. If ezmlm-make(1) is invoked with the '-e' or '-+' switch and the 'DIR' argument only, ezmlm-make(1) will read other arguments from this file. The difference between the switches is that with '-e' the options used are the ones specified on the command line, whereas with '-+' they are the ones currently active for the list, as overridden by any command line options. Thus, with just: 

% ezmlm-make -+ DIR

you can rebuild the list, without affecting any archives, list state variables, etc. You will lose manual customizatoins to some of your files. However, text files and DIR/headeradd are protected against being overwritten, so that your manual customizations of these files are retained. To override this protection, simply specify the used edit switch twice, e.g. '-ee' and '-++', respectively. This is a feature introduced in ezmlmx.

Since ezmlmx comes in the 'slashpackage' installation format, you can simply install and upgrade to the newest or a previous version without problems following the standard procedures. Upgrade and downgrade the binaries however may require configuration changes and language templates. Make sure to have a copy of settings available for this case.

10. What is special about ezmlmx and s/qmail?

In this document Dan Bernstein's 'original' ezmlm version (0.53) is called 'ezmlm proper'. In the same sense, his original qmail version (1.03) is usually referenced as 'vanilla qmail'.

The qmail forks netqmail, notqmail, and s/qmail possess the same APIs, in particular the VERP mechanism (see below 'Inventions in ezmlm'). However, the particular qmail forks have different installation requirements, and in particular transport layer encryption (using TLS), and further acceptance of mails from remote RECIPIENTS (called SENDER here).

With ezmlmx and s/qmail you get:

  • All programs are based on a common fehQlibs library including DNS lookups.
  • All programs are installed in a uniform way using Dan Bernstein's slashpackage convention.
  • Emails received and send via s/qmail are TLS encrypted.
  • Incoming email may be checked for SPF and DKIM correctness.
  • Outgoing emails (from the mailing lists) may be DKIM signed.
  • s/qmail's RECIPIENTS extension natively supports the VERP mechanism used in ezmlm. Just, simply including the email's list name in the recpient data base is sufficient.

2. Quick start.

2.1. Requirements.

  • fehQlibs installed (version 27 is minimum)
  • A qmail derivative is installed (qmail+patches, notqmail, netqmail, s/qmail, and perhaps indimail).
  • Downloaded ezmlmx-V.RR.tgz (V: version, R: release).

2.2. Slashpackage installation.

  • create /package (if not already available).
  • cd to /package and untar ezmlmx-V.RR here.
  • Move to the generated directory ./mail/ezmlmx/ezmlmx-V.RR.
  • Adjust the conf-* files (qmail home etc).
  • Call package/install.
  • In case you need customzied ezmlmrc language packages, call package/ezmlmrc LANG, where LANG is one of the available ISO language codes.

2.3. Testing.

  • Call package/rts and watch the output.
  • Unlike the more 'modern' versions of ezmlm-test, here (s/)qmail is invoked for delivery. Thus watch the qmail-send logs.

2.4. Make a list and digest list. 

% ezmlm-make -rdugm -5 me@host ~/list ~/.qmail-list me-list host
% ezmlm-sub ~/list me@host
% ezmlm-sub ~/list/digest me@host
% ezmlm-sub ~/list/mod me@host

where 'me' is your user name and 'host' the host your list is on.

Now, you are the owner, remote administrator, and subscriber of both list@host and the accompanying digest list list-digest@host. Only subscribers are allowed to access the archive and to post. To post to the list, mail to list@host. For a user to subscribe, s/he should mail to list-subscribe@host and for help to list-help@host.

When a non-subscriber posts, you will be asked to approve, reject, or ignore the request. If you want to subscriber joe@j.nosp@m.oeho.nosp@m.st.do.nosp@m.m, mail list-subscribe-joe=joehost.dom@host.

Digests are generated about every two days, when 30 messages have arrived since the last digest, or when more than 64 kbytes of message body has arrived. To manage the digest list, use the same commands as the main list, but replace 'list' with 'list-digest'.

The sender restriction on posting used in this setup works, but is not secure. For more info, read the man pages (start with ezmlm(5) and ezmlm-make(1)), this FAQ (FAQ.md in the distribution), README.md, INSTALL.md, and perhaps UPGRADE.md.

3. Overview of mailing list management and mailing list managers.

3.1 How e-mail mailing lists work.

E-mail distribution (or mailing) lists are nothing more than special programs that can automatically generate and manage subscribers. The distribution list is effectively the actor. Therefore, the name of the distribution list is usually the same as the sender. The e-mail distribution list also mechanisms to resend undeliverable e-mails and to react to a set of rules in case of an delivery error.

Once the distribution list has been set up, there are only two tasks to perform:

  1. The distribution list must be supplied with valid recipient e-mail addresses. These recipients are referred to as subscribers. Just as you can subscribe to magazines, for example, you can also subscribe to e-mail distribution lists and unsubscribe from them if necessary. As long as a recipients are reegistered with their e-mail addresses, they will receive messages via the distribution list.
  2. The e-mail has to be send to the distribution list: posting. Now, e-mail messages to the subscribers are generated.
    The distribution list then converts the message as if the e-mail originated from it.

E-mail distribution lists differ in the way subscribers and messages can be added:

  • Open lists allow anyone (provided they have an email address and access) to subscribe to a list and use this list to distribute their messages. For example, the discussion in the email distribution program ezmlm is conducted in such an open list.
  • Closed lists are managed by moderators. Moderators set up subscribers and also ensure that the distribution list is populated with new messages.
    • Semi-open lists typically allow unlimited subscriptions, but messages are moderated. This ensures that only "relatively reasonable" posts are accepted and distributed only.

Another feature of email distribution lists is their ability to collect posts in archives. The archives are freely accessible on open lists. The "Subject:" plays a central role in archiving. Posts can be requested both chronologically and by subject, i.e., by topic often called thread.

Some list archives also allow you to search by sender (author) or sort posts accordingly. The list often also allows you to locate posts based on text passages within the actual message. Lists that support this feature are usually fully indexed for posts.

One weakness of list archives is the handling of attachments. Due to MIME conventions, it is in principle possible to identify the type of attachment (e.g., Word document, image file, or audio file), but their information cannot be indexed or displayed via the list archive, with a few exceptions (plain text documents). One exception is Web access to a list archive under certain circumstances.

3.2 Subscribing to e-mail distribution lists.

Email distribution lists are intended to be practically self-administering. This means that typical tasks such as subscribing to/unsubscribing from an email distribution list, sending messages to a distribution list, and possibly viewing an archive should be essentially self-explanatory and take place without administrator intervention.

To do this, a command is sent via email to the program managing the list, which the program can interpret.

Let's assume we will raise an open email distribution list called "EagleWings@example.com" for a children television program. Current broadcast schedules, as well as topics and games related to individual programs, are presented or discussed in more detail via this distribution list. Furthermore, this list is used for the exchange of personal information ("EagleWings Events"). A typical action would be to subscribe to this list.

Depending on the operating principles of the program managing the list, the potential subscribers have several options:

  • They write to the list and enter the password "Subscribe" in the subject line.
  • They write to the list and enter the password "Subscribe" as the first word in the message.
  • An empty message is send to the recipient "EagleWings-subscribe@example.come".

The last method is that of ezmlm. This offers the following advantages:

  1. It eliminates ambiguities regarding actions that arise from different interpretations.
  2. It makes it very easy to subscribe to the list via a web interface.

Here's a corresponding example. We further assume that "EagleWings" also has its own website. At the bottom of the website, there is the following note: 

> "If you would like to subscribe to the 'EagleWings' 
   information and news list, please click: 
> I subscribe to the 'EagleWings' list."

The following is entered at the crucial point in the source code of the website: 

> <a href="mailto: EagleWings-subscribe@example.com">
> I subscribe to the 'EagleWings' list.</a>

And unsubscribing yields: 

> "If you would like to unsubscribe from the 'EagleWings' 
   information and news list, please click: 
> 'I unsubscribe from the 'EagleWings' list."

The source code of the website is: 

> <a href="mailto: EagleWings-unsubscribe@example.com">
  I unsubscribe from the 'EagleWings' list.</a>

Clicking the corresponding field automatically opens the user's email program with the correct addresses, and by simply hitting the "Send" botton, subscription and unsubscription from the list can be realized.

Further simple options include making the accompanying materials available via the "EagleWings" list archive. In any case, ezmlm can be very efficiently connected to a web interface, and every command can, of course, also be sent via email.

3.2 The cookie mechanism.

The cookie mechanism represents an elegant way to ensure that email actually comes from a specific ("the") sender. The cookie mechanism enables secure authentication.

Let's assume a subscriber wants to subscribe to an email distribution list. To do so, they first send a "Subscribe" request to the list via email. Instead of responding to this request, the email list first sends an email containing a cookie to the sender. The cookie is a non-trivial character string. The sender must send the cookie back to the email list. Since only they and the distribution list know the cookie, this is how authentication occurs.

In the case of the ezmlm list program discussed below, the cookie is generated as part of the email address, e.g., "eaglewings.909434430.bkddhjfkfgbldiljfii0-feh=fehcom.de@example.com".

3.3 ezmlm subscriber privileges.

Email distribution lists offer privileged use of their features. ezmlm offers a multi-level system of subscriber privileges:

a) Subscribers are email addresses (behind which individuals or robot archives are located) that receive information (messages) via email distribution lists. There are essentially two types of email distribution lists:

  • Open lists, to which anyone can subscribe and through which anyone can distribute messages to all other subscribers – without being a subscriber. Open lists generally also allow access to the list archive.
  • Moderated lists, where specific subscribers have moderator rights. These rights include the ability to:
    • Manage subscriber subscriptions,
    • Control the posting of messages to the email distribution list.

b) Moderators regulate the ingress and egress of subscribers and messages from an email distribution list. ezmlm offers three different moderators:

  • Message moderators are responsible for message access. Typically, the message moderator updates the list with new information to be distributed. Messages sent to the list by other subscribers can be accepted or rejected by the moderator. There are two default responses: "FROM: - reject" and "TO: - accept." Messages added to the list by a moderator must be confirmed by that moderator only.

  • Subscriber moderators are responsible for accepting or deleting new subscribers from the list. Typically, a subscriber receives a confirmation request after the first request to be added to the list (subscription). This ensures that

    • the email connection between the list and the subscriber is working, and
    • the subscriber must authenticate themselves via a cookie mechanism.

    If, however, a subscriber moderator is configured, they must also provide a "subscription confirmation."

    Rather, any subscriber can also remove themselves from the list without the assistance of the subscriber moderator.

  • Remote administrators are responsible for subscriber confirmation. This means that the remote administrator, instead of the subscriber, must now confirm the subscription. By design, the "remote administrators" are therefore also "subscriber moderators."

c) Report recipients: Recipients of summaries of the list messages (reports). These are maintained in a special address list.

4. Overview of ezmlm function

4.1. The basic setup.

In designing ezmlm, Dan J. Bernstein has used the Unix philosophy of small component programs with limited and well defined functions. Requests for specific functions can then be met by the addition of new programs.

Thanks to the program execution mechanism Dan built into qmail, it is easy to execute several small programs per delivery in a defined sequence. It is also very easy to add shell scripts for further customization.

4.2 Inventions in ezmlm.

Dan J. Bernstein has written ezmlm in C. It is written for speed and reliability even in the face of power loss and NFS. These features are augmented to a large extent by the ruggedness of the qmail (also by Dan) delivery mechanism (see qmail-command(8)).

However, the most important invetion of Dan was the V.E.R.P. mechanism, also known as Variable Envelope Return Path (VERP). Here, the SENDER address is amendet with some information glued together with typically a dash or a plus sign in a structured way. This mechanism was later picked up by the Sender Rewriting Scheme (SRS) and John Levins's Bounce Tag Address Validation (BATV). Within ezmlm, VERP addresses are the cornerstone for reliable mailings, not only for subscriptions and unsubscriptions, but rather used as command interface to the mailing list.

Using VERP, commands to the mailing list can be encoded in its address, and don't need to be included in the Subject or the body of the mail. Even better: Now, one can define individual commands for ezmlm and include propietary functions to fulfill those tasks.

ezmlm uses some routines and techniques that still are not frequently seen in many mailing list managers. For example, subscriber E-mail addresses are stored in a hash so that searches require reading only, at most, 2% of the E-mail addresses. ezmlm has a optional message archive, where messages are stored 100 per directory, again to allow more efficient storage and retrieval. Important files are written under a new name and, only when safely written, moved in place, to assure that crashes do not leave the list in an undefined state.

In addition, ezmlm has a number of new inventions. One of these is bounce detection, which generates an automatic warning containing information identifying the messages which have bounced, followed by a probe message to the E-mail addresses for which mail has bounced. If the probe bounces, the address is unsubscribed. Thus, the system won't remove E-mail addresses due to temporary bounces: it takes 12 days after the first bounce before a warning is sent, and another 12 days of bounces after the warning bounce before the probe message is set.

Another Dan J. Bernstein invention is the use of cryptographic cookies based on a timestamp, address, and action. These are used to assure that the user sending a request to subscribe or unsubscribe really controls the target address. It is also used to prevent forgery of warning or probe messages to make it exceedingly difficult to subvert the bounce detection mechanism to unsubscribe another user.

4.3 The qmail delivery mechanism.

See sqmail(7), qmail-local(8), qmail-command(8), envelopes(5), and dot-qmail(5). Briefly, qmail having resolved the delivery address delivers it via the .qmail file that most completely matches the address. This file may be a link to another file, as is the case in ezmlm lists. qmail then delivers the message according to successive lines in this file forwarding it to an address, storing it, or piping it to a program. In the latter case, the program is expected to exit 0 leading delivery to proceed to the next line in the .qmail file, or 99 leading to success without delivery to succeeding lines. An exit code of 100 is a permanent error leading to an error message to the SENDER. An exit code of 111 is used for temporary errors, leading to redelivery until successful or until the queue lifetime of the message has been exceeded.

Delivery granularity is the .qmail file and re-deliveries start at the top. Thus, if the message fails temporarily at a later line, the delivery according to an earlier line will be repeated. Similarly, qmail may have made deliveries successfully according to most of the .qmail file and then fail permanently. The SENDER is informed that the delivery failed, but not about at which point.

ezmlm takes advantage of these basic mechanisms to build a fast, efficient, and very configurable mailing list manager from a set of small independent programs.

4.4 What the different programs do.

All information of the ezmlm distribution list is stored in an ezmlm directory tree dir. To set up this directory tree, it is created and populated using the ezmlm-make(1) command. All other commands refer to actions on files or directories in this directory tree:

ezmlm-make

creates the dir directory

ezmlm-sub and ezmlm-unsub

manage the subscriber list stored under dir

ezmlm-manage

represents the command interface for incoming email

ezmlm-send

sends a message to all subscribers in dir and manages the message archive and index, if the list has been configured to do so.

ezmlm-reject

Rejects messages with an empty "Subject:" and if the "Subject:" contains an ezmlm command.

ezmlm-return

Handles bounces and circulating emails.

ezmlm-warn

Warns the sender who sent the circular email and removes the sender from the subscriber list.

ezmlm-idx

Allows the creation of a "Subject:" index for an existing list archive.

ezmlm-get

Manages messages, the index, and "Subject:" archive requests. It also generates the summary (report) for the list.

ezmlm-cron

provides a limited interface for the UNIX cron service to send list reports and messages on a scheduled basis.

ezmlm-store stores messages for moderated lists and sends the moderation request to the moderator.

ezmlm-moderate

processes moderation requests by adding the accepted message to the list via ezmlm-send, or, in the case of a rejection, returning it to the sender.

ezmlm-clean

cleans pending moderation requests or expired messages.

ezmlm-gate

transfers messages from a SENDER to the address database and the actual message to the moderator.

ezmlm-check

is a helper command for diagnosing configuration problems on the ezmlm list.

ezmlm-issub and ezmlm-issubn

determine whether the SENDER is a subscriber or can be found in the address database.

ezmlm-tstdig

determines when a new report needs to be created based on the number of posts received, their volume, and the time elapsed since the last report.

ezmlm-request

translates the commands in the "Subject:" line commonly used by other lists (e.g., Majordomo) into ezmlm commands.

ezmlm-glmake

defines the global list interface.

ezmlm-glconf

creates the configuration file for the global list interface.

4.5. What the different files in the list directory do.

Let's pick up the previous sample of a mailing list, called 'EagleWings' and hosted at 'example.com'. We assume, that the Unix account responsible for that list is simply called 'listmaster' with its home directory at '/home/listmaster'. The problems with the different cases with the names, will be discussed lated.

4.5.1 What the directory structure is responsible for.

Upon invocation ezmlm-make generates the following directories (depending on the options used):

  • DIR (here name 'dir') is the home direcotry of the list: /home/listmaster/EagleWings followed by
  • dir/allow/: database for alias names and subscriber
  • dir/archive/: archive directory holding posts and the list index.
  • dir/bounce/: directory to hold 'bounced' mails.
  • dir/deny/: list of e-mail addresses for banned posters.
  • dir/digest/: subdirectory for digests; with similar structure like the main list
  • dir/mod/: nested subdirectory to hold postings in different states (accepted, pending, rejected) together with the list of moderators organised files with 'hashed' names.
  • dir/subscribers/: subscriber email addresses also distributed in a hash directory.

4.5.2 What the 'dot-qmail' files are responsible for.

The Unix user 'listmaster' for our mailing list 'EagleWings' possesses a set of *~/.qmail-* files which are used by ezmlm to enable the required work flow of emails: 

.qmail-EagleWings -> /home/listmaster/EagleWings/editor
.qmail-EagleWings-accept-default -> /home/listmaster/EagleWings/moderator
.qmail-EagleWings-default -> /home/listmaster/EagleWings/manager
.qmail-EagleWings-digest-owner -> /home/listmaster/EagleWings/owner
.qmail-EagleWings-digest-return-default -> /home/listmaster/EagleWings/digest/bouncer
.qmail-EagleWings-owner -> /home/listmaster/EagleWings/owner
.qmail-EagleWings-reject-default -> /home/listmaster/EagleWings/moderator
.qmail-EagleWings-return-default -> /home/listmaster/EagleWings/bouncer

In order to make qmail aware of this work flow and allowing it to receive mails dedicated to 'EagleWings' by the Unix account 'listmaster' additionly 'dot-qmail' files are raised for the qmail user 'alias' als links (assuming the home directory of 'alias' is '/var/qmail/alias') pointing to 'listmaster':

/var/qmail/alias/.qmail-EagleWings -> /home/listmaster/EagleWings/editor /var/qmail/alias/.qmail-EagleWings-default -> /home/listmaster/EagleWings/manager /var/qmail/alias/.qmail-EagleWings-owner -> /home/listmaster/EagleWings/owner /var/qmail/alias/.qmail-EagleWings-return-default -> /home/listmaser/EagleWings/bouncer

Now let's discuss their content. As shorcut, we use PATH as path to the ezmlm binaries.

4.5.3 What dir/editor is good for.

dir/editor to let user's subscribe to the list: 

|PATH/ezmlm/ezmlm-reject '/home/listmaster/EagleWings'
|PATH/ezmlm/ezmlm-store '/home/listmaster/EagleWings'
|PATH/ezmlm/ezmlm-clean '/home/listmaster/EagleWings' || exit 0
|PATH/ezmlm/ezmlm-warn '/home/listmaster/EagleWings' || exit 0
|PATH/ezmlm/ezmlm-warn -d '/home/listmaster/EagleWings' || exit 0
|PATH/ezmlm/ezmlm-tstdig -m30 -k64 -t48 '/home/listmaster/EagleWings' || exit 99
|PATH/ezmlm/ezmlm-get '/home/listmaster/EagleWings' || exit 0

4.5.4 What dir/owner is used for.

dir/owner is responsibe for positings targeting the list owner (with local address 'master'). Those are stored in a dedicated mailbox and potentially used to setup a digest report. 

&master
|PATH/ezmlm/ezmlm-warn '/home/listmaster/EagleWings' || exit 0
digest format.

4.5.5 Why dir/bouncer is needed.

dir/bouncer is used to process and eliminate bounces. 

|PATH/ezmlm/ezmlm-weed
|PATH/ezmlm/ezmlm-return -D '/home/listmaster/EagleWings'

4.5.6 For what dir/manager is used.

dir/manager is provided to cope with 'Adminstrativia': 

|PATH/ezmlm/ezmlm-weed
|PATH/ezmlm/ezmlm-return -D '/home/listmaster/EagleWings'
|PATH/ezmlm/ezmlm-get -s '/home/listmaster/EagleWings' 1
|PATH/ezmlm/ezmlm-request '/home/listmaster/EagleWings'
|PATH/ezmlm/ezmlm-manage -le '/home/listmaster/EagleWings'
|PATH/ezmlm/ezmlm-warn '/home/listmaster/EagleWings' || exit 0
|PATH/ezmlm/ezmlm-warn -d '/home/listmaster/EagleWings' || exit 0

4.5.7 For what dir/moderator is responsible for.

Generated on for ezmlmx by doxygen 1.14.0