Re: cyrus-imapd with lmtpd + postfix slow delivery with group email ids

jayesh shinde <jayesh.shinde@xxxxxxxxxxxxx> · Fri, 15 May 2015 11:48:55 +0530



    Thanks Dan for your feedback. 

        I verified the suggested points , its working as require. 
        Please find below inline reply for each. 

        
        I found that  when big group email is getting deliver ( i.e more
        than 500 - 4K members ) at that time other emails are not
        getting deliver simultaneously to user's mailbox. 

        I verified this from live maillog  i.e  by doing grep key words
        like "to mailbox:" /  "Delivered:"  etc ...

        
        My mail queue issue is getting critical in peak hours !!  Kindly
        suggest. 

        
        On 05/14/2015 08:22 PM, Dan White wrote:

      
    On 05/14/15 05:42 +0000,
        jayesh.shinde wrote:
        

      Please
          suggest , for faster delivery  what could be the best config 
          in
          

          postfix + cyrus-imapd .
          

          I am using the cyrus-imapd-2.4.17-6.el6.x86_64 for mailbox
          server and for
          

          smtp server postfix-2.10.0-1.el6.x86_64 Both server have SAS
          hdd , 16 core
          

          , 16 GB RAM .  There is no i/o issues
          

      Mailbox
          server have  21K email ids.  Cyrus running with  tcp socket on
          

          port 24 i.e lmtpd .  cyrus singleinstance is maintained
          

      For
          group id mapping is under virtual_alias_maps of smtp's
          main.cf  i.e no
          

          /etc/aliases. From smtp server emails are getting deliver by
          

          transport_maps to mailbox server .
          

      My
          problem observe :--
          

          --------------------------
          

          1) When HOD send email to 3-5 big group  email ids then , then
          its postfix
          

          delivery get slow. What I observe is postfix getting delivered
          the traffic
          

          per email and not parallel.   Each group contain 3k or 7k
          email ids.
          

        Verify that single instance store is working. If not, you may
        have a
        

        Postfix configuration issue. Find one of these mass emails in a
        mailbox and
        

        stat it to find out:
        

        stat -c %h <file>
        

        A returned value of 1 means single instance store isn't working.
        

        If that's not the issue, you can determine if this is a postfix
        issue or a
        

        cyrus issue by sending a mass email directly to lmtpd, with
        lmtptest.
        

    Cyrus Singleinstance is
      working , I have verified like below. In imapd.conf man page its
      define as default 1 ( i.e default enable ) 

      
      [root@Mailbox ~]# ls -lirth
      /var/spool/imap/domain/v/vdomain/m/user/muser1/ | tail -n 20  

      
      163326970 -rw-------  198 cyrus mail 413K May 15 10:07
      64828.

      163396232 -rw-------  145 cyrus mail  92K May 15 10:13
      64829.

      163379367 -rw-------  151 cyrus mail  96K May 15 10:29
      64830.

      163323993 -rw-------  496 cyrus mail  13K May 15 10:30
      64831.

      163398621 -rw-------  516 cyrus mail 850K May 15 10:42
      64832.

      163398568 -rw-------  523 cyrus mail 455K May 15 10:43
      64833.

      163398933 -rw-------  617 cyrus mail 3.7M May 15 10:47
      64834.

      163398962 -rw-------  522 cyrus mail 1.3M May 15 10:48
      64835.

      163398990 -rw-------  617 cyrus mail  58K May 15 10:49
      64836.

      163396678 -rw-------  465 cyrus mail 207K May 15 10:56
      64837.

      163399529 -rw-------  576 cyrus mail 332K May 15 10:57
      64838.

      163399554 -rw-------    2 cyrus mail  54K May 15 10:57
      64839.

      163346523 -rw-------   36 cyrus mail 767K May 15 11:01
      64840.

      
      [root@Mailbox muser1]# stat -c %h 64829.

      145

      [root@Mailbox muser1]#
      stat -c %h 64828.

      198

      
      In
          peak hours the queue on postfix get high and other normal
          emails also
          

          getting stuck in queue. Once the group email get clear after
          that other
          

          emails also get clear.
          

      in
          mailbox cyrus.conf :-- lmtp  cmd="lmtpd -a" listen="lmtp"
          prefork=0
          

          in main.cf of smtp server :--
          

          lmtp_destination_concurrency_limit = 100
          

          lmtp_destination_recipient_limit = 0
          

        How many lmtp processes do you see spawned in this scenario?
        

    [root@Mailbox ~]# ps
      auxwww| grep lmtp 

      
      cyrus     3224  0.0  0.0  73376  4648 ?        S    10:55   0:00
      lmtpd

      cyrus     4043  0.0  0.0  73376  4156 ?        S    10:59   0:00
      lmtpd

      postfix   6972  0.0  0.0  75708  4336 ?        S    11:10   0:00
      lmtp -t unix -u

      postfix   6974  0.0  0.0  75708  4340 ?        S    11:10   0:00
      lmtp -t unix -u

      postfix   6975  0.0  0.0  75708  4336 ?        S    11:10   0:00
      lmtp -t unix -u

      postfix   6977  0.0  0.0  75708  4344 ?        S    11:10   0:00
      lmtp -t unix -u

      postfix   6979  0.0  0.0  75708  4336 ?        S    11:10   0:00
      lmtp -t unix -u

      postfix   6981  0.0  0.0  75708  4336 ?        S    11:10   0:00
      lmtp -t unix -u

      postfix   6985  0.0  0.0  75708  4348 ?        S    11:10   0:00
      lmtp -t unix -u

      cyrus     6998  0.0  0.0  73376  4052 ?        S    11:10   0:00
      lmtpd

      cyrus     7001  0.0  0.0  73376  4044 ?        S    11:10   0:00
      lmtpd

      cyrus     7234  0.9  0.0  75456  4976 ?        S    11:12   0:04
      lmtpd -a

      cyrus     8758  0.6  0.0  75452  4112 ?        S    11:19   0:00
      lmtpd -a

      root      8898  0.0  0.0 103236   972 pts/1    S+   11:19   0:00
      grep --colour=auto lmtp

      cyrus    46046  0.0  0.0  73376  4648 ?        S    10:28   0:01
      lmtpd

      cyrus    48107  0.0  0.0  73372  4740 ?        S    10:37   0:01
      lmtpd

      cyrus    48108  0.0  0.0  73376  4700 ?        S    10:37   0:01
      lmtpd

      
      lmtpd :-- 7 process

      lmtp -t unix -u :-- 7 process

      lmtpd -a :-- 2 process

      
      Cyrus config is as follows:

      -----------------------------------

      
      SERVICES {

        imap        cmd="imapd" listen="imap" prefork=5

        imaps        cmd="imapd -s" listen="imaps" prefork=1

        pop3        cmd="pop3d" listen="pop3" prefork=3

        pop3s        cmd="pop3d -s" listen="pop3s" prefork=1

        sieve        cmd="timsieved" listen="sieve" prefork=0

      
        lmtp  cmd="lmtpd -a" listen="lmtp" prefork=0

        lmtpunix    cmd="lmtpd" listen="/var/lib/imap/socket/lmtp"
      prefork=1

      }

      
      [root@Mailbox ~]# cat /etc/imapd.conf  

      configdirectory: /var/lib/imap

      partition-default: /var/spool/imap

      admins: cyrus

      sievedir: /var/lib/imap/sieve

      sendmail: /usr/sbin/sendmail

      hashimapspool: true

      sasl_pwcheck_method: saslauthd

      sasl_mech_list: PLAIN

      allowplaintext: yes

      unixhierarchysep: 1

      virtdomains: yes

      altnamespace: on

      defaultdomain: mydomain.com

      expunge_mode: immediate

      delete_mode: immediate

      tls_cert_file: /usr/share/ssl/certs/server.pem

      tls_key_file: /usr/share/ssl/certs/server.key

      tls_ca_file: /etc/pki/tls/certs/ca-bundle.crt

      lmtp_over_quota_perm_failure: 1

      duplicatesuppression: 1

      duplicate_db_path: /var/lib/imap/cyrus-ram-tmpfs/deliver.db

      statuscache_db_path: /var/lib/imap/cyrus-ram-tmpfs/statuscache.db

      tlscache_db_path: /var/lib/imap/cyrus-ram-tmpfs/tls_sessions.db 

      
      Regards

      Jayesh Shinde

    
----
Cyrus Home Page: http://www.cyrusimap.org/
List Archives/Info: http://lists.andrew.cmu.edu/pipermail/info-cyrus/
To Unsubscribe:
https://lists.andrew.cmu.edu/mailman/listinfo/info-cyrus