MTA Evaluation on delivery performance



1. Comparison on features

I investigated how DNS queries and SMTP deliveries are done concurrently on each implementations.
See the feature list.

2. Comparison on performance

How to evaluate

  1. A mailing list which contains 341 e-mail addresses was created on the machine for evaluation. These addresses were collected from various domains and messages sent to these addresses were dropped into /dev/null like nobody@domain. Every domain part has different MX RRs.
  2. A message of 2Kbytes were sent to the mailing list.
  3. Traffic on network segment to which the machine is connected was monitored with 'tcpdump' between start and finish time of one delivery through the mailing list.
  4. Graphs were plotted with data from 'tcpdump'. There are two types of graph. One shows number of packets on DNS query and answer. The other shows number of SYN packets sent for SMTP connection request.


It is not so easy to construct pure internet simulated environment for evaluations, so all evaluations were done in real internet environment. As you know, it is better to measure on different MTA implementations at the same time for more precious comparison, but measurements were done in sequence because of lack of equipments. So, it is impossible to reproduce the same environment for each evaluation, it is, therefore, risky to discuss on detailed differences of MTAs. Only rough tendency should be realized. In addition, I tried but failed to set concurrency parameters to 50 for each MTAs (199 for zmailer, 255 for SMTPfeed). I'd like to re-try evaluations with 50s of concurrency for all MTAs.

How to read the graphs


DNS cache is cleared before evaluation.
(evaluation period: 1999/3/22 12:05-13:39)

Results by each implementation:



DNS cache is NOT cleared before evaluation (Fully cached).
(Slow implementations are omitted)
(evaluation period: 1999/3/26 00:08-00:30)

Results by each implementation:


Evaluation-3 (take the Evaluation-2 again)

(evaluation period: 1999/3/27 02:10-02:32)

Results by each implementation:



AMD K6/300Mhz
64MB Main memory
DEC 21140A FastEthernet NIC


Future works ?


I'd like to thank you to who offered 'nobody@domain' addresses for this evaluation.