We have a dovecot server version 2.3.16 with sieve duplicate extension enabled to detect and discard duplicates. We have duplicate routing in Postfix virtual_aliases file which causes duplicate copies with some address lists and is done correctly by Postfix.
However some duplicates are randomly allowed by Dovecot Sieve. Different emails receive duplicates on each iteration of same test. After detailed logging it looks like the duplicates get through when the same Message ID for same mailbox is taken up by two different dovecot processes. If the duplicates with same Message ID for same mailbox is taken up by same process, it discards the duplicate correctly. So if I reduce the process_limit
down to 1 we get zero duplicates in all cases.
Question is why is there a "duplicate mark" synchronization issue? is there a setting I am missing for synchronization? I would prefer NOT to set the lmtp process limit to 1
Dovecot configuration:
service lmtp {
process_limit = 1
}
shutdown_clients = no
Following the direction of serverfault post here I have set sieve configuration as below and sieve script is being executed properly and duplicate is being marked each time
sieve = file:~/sieve;active=~/.dovecot.sieve
sieve_before = /etc/dovecot/conf.d/deduplicate.sieve
sieve_extensions = +duplicate +fileinto +imapflags +editheader
Deduplicate seive script:
require "duplicate";
if duplicate {
discard;
stop;
}
Dovecot log entries for single email:
Oct 12 08:57:59 lmtp([email protected])<230008><IFAiHZclKGV5ggMAfXq90w>: Debug: sieve: msgid=<SA1PR22MB3146CDDB578F27162AAD7C29EFD3A@SA1PR22MB3146.namprd22.prod.outlook.com>: Finish implicit keep action
Oct 12 08:57:59 lmtp([email protected])<230008><IFAiHZclKGV5ggMAfXq90w>: Debug: sieve: msgid=<SA1PR22MB3146CDDB578F27162AAD7C29EFD3A@SA1PR22MB3146.namprd22.prod.outlook.com>: Finishing actions
Oct 12 08:57:59 lmtp([email protected])<230008><IFAiHZclKGV5ggMAfXq90w>: Debug: sieve: msgid=<SA1PR22MB3146CDDB578F27162AAD7C29EFD3A@SA1PR22MB3146.namprd22.prod.outlook.com>: Finish duplicate_mark action
Oct 12 08:57:59 lmtp([email protected])<230008><IFAiHZclKGV5ggMAfXq90w>: Debug: sieve: msgid=<SA1PR22MB3146CDDB578F27162AAD7C29EFD3A@SA1PR22MB3146.namprd22.prod.outlook.com>: Finished executing result (final, status=ok, keep=yes)
journalctl -o short-iso-precise