User Tools

Site Tools


doc:appunti:linux:sa:postfix_spamassassin_clamav_dovecot

Postfix, SpamAssassin, ClamAV and Dovecot

Tested on Debian 9 Stretch.

Postfix on Port 587/TCP (submission)

Uncomment in /etc/postfix/master.cf:

submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=may
  -o smtpd_sasl_auth_enable=yes
  -o smtpd_reject_unlisted_recipient=no
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject

The option smtpd_reject_unlisted_recipient=no causes Postfix to accept non-existant local recipients, thus generating an error message just after queuing it. The default (to be enforced for non-authenticated clients) is to reject non-existant local recipient just after RCPT TO command.

Address Rewrite, Alias, etc.

Addresses without a domain name (both recipient and sender), will be qualified with the value of myorigin configuration, which defaults to the content of /etc/mailname in Debian.

If you need to rewrite the sender of some mails (e.g. locally generated system messages generated by root), you can use the sender_canonical_maps option of Postfix and provide a file like this:

root    user@some_domain.org

Mail for unqualified local users can be diverted to a specific local user by the /etc/aliases file. E.g. many system users do not have an $HOME/Maildir/, so they cannot receive locally generated mail. You can redirect all the messages to a single user with something like (remember to run newaliases after changing the file):

postmaster:     root
www-data:       root
root:           niccolo

Dovecot Authentication

We want a single authentication mechanism for SMTP, POP3 and IMAP, and we want to let the user to authenticate using the Unix username or the full email address and the password. Dovecot authentication is suitable for the task; the default Debian configuration provides authentication against the system usernames via the PAM module. The configuration file is /etc/dovecot/conf.d/auth-system.conf.ext.

To let authentication using the email address, we provide a passwd-like file /etc/dovecot/users, we will leave gecos, shell and extra_fields blank. See PasswdFile wiki page for details:

user:password:uid:gid:(gecos):home:(shell):extra_fields

This file will be used for authentication against the password (i.e. as a Password Database), and it will be used for user's data lookup, like the home directory (i.e. as an User Database). So we need to configure two sections in /etc/dovecot/conf.d/auth-passwdfile.conf.ext

passdb {
  driver = passwd-file
  args = scheme=CRYPT username_format=%u /etc/dovecot/users
}

userdb {
  driver = passwd-file
  args = username_format=%u /etc/dovecot/users
}

The scheme=CRYPT means that libc's crypt() is used for password encryption, so the same password stored in /etc/shadow can be used.

To activate both system PAM and passwd-like authentication, we set into /etc/dovecot/conf.d/10-auth.conf (needed systemctl reload dovecot.service after the change):

# Order matters: we use passwdfile first because it is most likely used
# and because system autentication (PAM) does a 2-seconds delay on fail.
!include auth-passwdfile.conf.ext
!include auth-system.conf.ext

Now we can test both authentication and user's lookup using the doveadm tool:

doveadm auth test username@domain.org
Password: 
passdb: username@domain.org auth succeeded
doveadm user username@domain.org
field   value
uid     1000
gid     1000
home    /home/username
mail    maildir:~/Maildir

The file /etc/dovecot/users should be built e.g. by a cron-job, joining Postfix virtual_alias_maps and /etc/shadow passwords. It is re-read at each lookup. We need also to protect it:

chmod 0640 /etc/dovecot/users
chown root:dovecot /etc/dovecot/users

If you need to generate the hash for a password, you can use the following command line:

openssl passwd -1

When the /etc/dovecot/users is updated, there is no need to reload the Dovecot service. Beside the passwd-file driver, Dovecot can use other database types, check the lookup databases paragraph if you need more performances.

Postfix SASL over Dovecot Auth

See Postfix and Dovecot SASL.

First of all we need to activate the socket used by Posfix for authentication. In /etc/dovecot/conf.d/10-master.conf ensure that into the service auth section there is:

service auth {
  ...
  # Postfix smtp-auth
  unix_listener /var/spool/postfix/private/auth {
    mode = 0660
    user = postfix
    group = postfix
  }
  ...
}

To be compatible with more clients, enable both PLAIN and LOGIN methods. In /etc/dovecot/conf.d/10-auth.conf:

auth_mechanisms = plain login

Once reloaded the dovecot.service, you should see the /var/spool/postfix/private/auth socket, with the requested owner and permission.

For the Postfix part, you must declare in /etc/postfix/main.cf:

# Uses Dovecot Auth socket.
smtpd_sasl_type = dovecot
smtpd_sasl_path = private/auth

After reloading the postfix.service, it should repsond to the EHLO command with:

telnet localhost 25
220 smtp.domain.org ESMTP Postfix (Debian/GNU)
EHLO test
...
250-STARTTLS
250-AUTH PLAIN LOGIN
...

WARNING: The AUTH capability is not announced over an unencrypted connection, if it is enabled only over TLS connections; see the smtpd_tls_auth_only Postfix option. Modern Postfix installations does not offer AUTH over smtp (port 25/tcp), but only over submission (port 587/tcp). Check if the option smtpd_sasl_auth_enable=yes is declared into main.cf or into the submission section of master.cf.

Dovecot IMAP and POP3

Install the Debian packages:

  • dovecot-imapd
  • dovecot-pop3d

Once installed, the basic services IMAP2 on port TCP/143 and POP3 on port TCP/110 are already working, eventually with the Dovecot Authentication on users as seen above.

Enable SSL, imaps and pop3s

See SSL Dovecot Configuration.

We need to enable SSL certificates for encryption, and enable IMAPS on port TCP/993 and POP3S on port TCP/995. We will use the same Let's Encrypt certificates obtained for the web server. So we set into /etc/dovecot/conf.d/10-ssl.conf:

ssl = yes
ssl_cert = </etc/letsencrypt/live/smtp.domain.org/fullchain.pem
ssl_key = </etc/letsencrypt/live/smtp.domain.org/privkey.pem

Dovecot will read the SSL files with root privileges, so no particular settings should be required. If the certificate is renewed, you must reload the dovecot.service to re-read it.

Once restarted the dovecot.service, we will find the services listening on the well known ports, and we can check the SSL certificates.

Client connections are being dropped

You may encounter wanring messages like these into the mail log:

dovecot: master: Warning: service(imap-login): process_limit (100) reached, client connections are being dropped

To increase the number of process that are allowed to spawn, edit the /etc/dovecot/conf.d/10-master.conf file and set

default_process_limit = 200

SpamAssassin Filter

Install the Debian packages spamassassin and spamc. Configure the SpamAssassin filter in /etc/spamassassin/local.cf, we customized a couple of settings:

rewrite_header Subject *****SPAM*****
report_safe 0

We can test the spam daemon using the spamc client:

cat message.txt | spamc

The result should show the added headers X-Spam-Status, etc.

To enable the cron job to automatically update spamassassin's rules on a nightly basis, edit /etc/default/spamassassin and set:

CRON=1

To force a manual update you can run /etc/cron.daily/spamassassin, downloaded rules will be stored into /var/lib/spamassassin/3.004001/. If something goes wrong (e.g. rules with errors), the spamd daemon may stop to work, in this case you should remove the above directory and just run with default, not updated rules.

Sanitizer/ClamAV Filter

For virus scanning of email attachments we use the ClamAV antivirus software. We install scanner, the daemon, the client and the database periodic update utility:

  • clamav
  • clamav-daemon
  • clamdscan
  • clamav-freshclam
  • libclamunrar9 (non-free package to scan inside RAR archives)

Periodic download (update) of viruses database is performed by the clamav-freshclam.service, you can check the /var/log/clamav/freshclam.log.

Beside the plain scan utility, we want a mail filter, which will scan every attachment, removing the infected ones. The sanitizer program will do that.

All the logic will be included into the /etc/sanitizer.cfg configuration file. A Dovecot Sieve filter will do the rest. Testing the Sanitizer/ClamAV filter is quite easy:

cat message.txt | /usr/bin/sanitizer /etc/sanitizer.cfg

Beware that the clamd daemon runs per default with the clamav user, if you want regular users to scan their files (e.g. their mail attachments) you have to use the --fdpass argument when invoking the clamdscan client, otherwise you will get the error message:

eicar-antivirus.com: lstat() failed: Permission denied. ERROR

The --fdpass option works only if clamdscan connects to clamd via the Unix socket.

Dovecot Local Delivery Agent

We apreciate local delivery agents which permits per-user filtering (enable/disable spam or virus filtering on a per-user basis). In the past we used procmail and maildrop, but dovecot-lda was our choiche because it provides all the following features:

  • Maildir support
  • Voluntary quota
  • Per-user filtering (spam, virus, …)
  • Sieve filtering (forward, vacation)

The executable /usr/lib/dovecot/dovecot-lda is provided by the package dovecot-core.

To use the $HOME/.Maildir/ as final recipient (instead of the /var/mail/ Unix mailbox) we have to configure /etc/dovecot/conf.d/10-mail.conf:

mail_location = maildir:~/Maildir

WARNING! It seems that $HOME/Maildir/ will be created if it does not exist, but subfolders will be not! To be safe: make all the required directories and subdirectories with maildirmake.dovecot, before enabling a new mailbox, eventually with system Sieve filters, etc.

To let Postfix use this delivery agent, we have to change /etc/postfix/main.cf:

mailbox_command = /usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"

Error stats-writer Broken pipe

WARNING: At least in Debian 11 Bullseye it seems that the dovecot-lda sometimes (not always!) tries to write to the Dovecot Stats Service using the /run/dovecot/stats-writer socket. Unfortunately the socket is owned by root:dovecot and has mode 0660 (default Debian settings), whereas the LDA is invoked by Postfix with just the user's privileges. So you will find error messages like this into the mail.log:

postfix/local[76866]: 7E67B7D3F4: to=<user@domain.tld>, ...,
    status=bounced (Command died with status 134:
    "/usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"".
    Command output: lda(lica-marilena):
    Error: net_connect_unix(/run/dovecot/stats-writer) failed:
    Permission denied Aborted Unable to flush stdout: Broken pipe )

It seems that the quickest and simplest solution is to make the socket 0666 mode (which will risk the stats service to be abused). Add the following snippet to /etc/dovecot/conf.d/15-lda.conf:

service stats {
  unix_listener stats-writer {
    mode = 0666
  }
}

See the following posts about the problem:

Error Command output: Aborted

Another subtle error with Dovecot LDA can be reported generically into the Postfix log as:

postfix/local[1615363]: E85037D1C6: to=<user@domain.tld>, ...
    status=bounced (Command died with status 134:
    "/usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"".
    Command output: Aborted )

To get some hints about the problem you can enable logging in Dovecot LDA; you can add the following in /etc/dovecot/conf.d/15-lda.conf:

protocol lda {
  ...
  log_path = /var/log/dovecot-lda-errors.log
  info_log_path = /var/log/dovecot-lda.log
}

and create the log files with suitable permissions (in my case the LDA is executed with user's privileges, so I made it world-writable).

lda(username)<1619235><QdsSDJpj72MjtRgACo+hHQ>: Error: program
    exec:/usr/local/lib/dovecot/sieve-filter/spamc-filter.sh (1619240):
    Execution timed out (> 10000 msecs)
lda(username)<1619235><QdsSDJpj72MjtRgACo+hHQ>: Error: program
    exec:/usr/local/lib/dovecot/sieve-filter/spamc-filter.sh (1619240):
    Forcibly terminated with signal 15
lda(username)<1619235><QdsSDJpj72MjtRgACo+hHQ>: Panic:
    output stream (temp iostream in /tmp/dovecot.lda.
    for (program client seekable output)) is missing error handling
lda(username)<1619235><QdsSDJpj72MjtRgACo+hHQ>: Error:
    Raw backtrace: /usr/lib/dovecot/libdovecot.so.0(backtrace_append+0x42)...

There was a problem with the filter program responding too slowly. It is possible to change the exec timeout, in this case for the filter extension, just add the following in /etc/dovecot/conf.d/90-sieve-extprograms.conf (notice that you can configure sieve_pipe_exec_timeout, sieve_filter_exec_timeout and sieve_execute_exec_timeout):

plugin {
  ...
  # Change the default timeout (10 seconds) for the filter extension.
  sieve_filter_exec_timeout = 60s

  # Change the default timeout (10 seconds) for the execute extension.
  sieve_execute_exec_timeout = 60s
}

Error Unable to flush stdout

FIXME This error message has an unwknown origin.

postfix/local[1485291]: 5BDBD7D1C6: to=<user@domain.tld>, ...
    status=bounced (Command died with status 134:
    "/usr/lib/dovecot/dovecot-lda -f "$SENDER" -a "$RECIPIENT"".
    Command output: Aborted Unable to flush stdout: Broken pipe )

Sieve filtering

Sieve is a programming language that can be used for email filtering, filters are usually stored into the mail server as user's files, the dovecot-lda searches that files into the user's $HOME. In that scenario it can act as the procmail or maildrop delivery agents (which uses respectively the .procmailrc and .mailfiter configuration files).

We will use Sieve for:

  • Filtering messages with Sanitizer/ClamAV.
  • Filtering messages with SpamAssassin.
  • Saving SPAM into the Spam folder (or removing it).
  • Creating Vacation auto-responder.

Enable Sieve Filtering in Dovecot LDA

To enable Sieve filtering in Dovecot local delivery agent we have to edit /etc/dovecot/conf.d/15-lda.conf:

protocol lda {
  mail_plugins = $mail_plugins sieve
}

Enable sieve_extprograms plugin

By default a Sieve filter cannot execute an external program or filter, we have to enable the sieve_extprograms plugin, which is however installed by the dovecot-sieve Debian package.

To enable the plugin, just edit /etc/dovecot/conf.d/90-sieve.conf and set into the existing plugin section:

plugin {
  ...
  sieve_extensions = +vnd.dovecot.filter
  ...
  sieve_plugins = sieve_extprograms
  ...
}

and configure the plugin itself, editing the /etc/dovecot/conf.d/90-sieve-extprograms.conf file at the plugin section:

plugin {
  ...
  sieve_filter_socket_dir = sieve-filter
  ...
  sieve_filter_bin_dir = /usr/local/lib/dovecot/sieve-filter
  ...
}

NOTICE: we enabled only the filter capability of the sieve_extprograms plugin, there are also the pipe and execute capabilities.

The actual filters are scripts saved into /usr/local/lib/dovecot/sieve-filter/ directory. We make just two examples: one using Sanitizer/ClamAV and one using SpamAssassin (named clamav-filter.sh and spamc-filter.sh respectively):

/bin/sh
/usr/bin/sanitizer /etc/sanitizer.cfg
#!/bin/sh
/usr/bin/spamc --connect-timeout=5 --timeout=15

Notice that we set two timeouts on spamassassin client, one to connect to the daemon and one to get the result. If the daemon does not respond timely, the message is passed unchanged. Beware that the timeouts should be lower than the one specified into the Dovecot LDA Sieve Extprograms plugin configuration.

How to Test an User's Sieve Filter File

The dovecot-lde searches the user's active filter into $HOME/.dovecot.sieve (configured in /etc/dovecot/conf.d/90-sieve.conf). We can write a simple filter file:

require ["fileinto"];
if header :contains "subject" ["order", "buy"] {
    fileinto "Commercial";
}

When a message is received and handled by the dovecot-lda, it can leave some useful log in /var/log/mail.log:

Jan 19 09:51:58 dovecot: lda(username): sieve: Execution of script /home/username/.dovecot.sieve
    failed, but implicit keep was successful (user logfile /home/username/.dovecot.sieve.log may
    reveal additional details)

As reported in log, an error message is recorded into users' $HOME/.dovecot.sieve.log. In our case the subfolder named Commercial was missing, we have to create it using maildirmake.dovecot ~/Maildir/.Commercial (notice the leading dot in subfolder name, which does not appear in Sieve filter statements).

A more complete Sieve filter example is the following. Each message is filtered for viruses and spam, then the messages marked as spam are saved into a specific folder:

require ["fileinto", "vnd.dovecot.filter"];

# Filter with /usr/local/lib/dovecot/sieve-filter/spamc-filter.sh.
filter "spamc-filter.sh";

# Filter with /usr/local/lib/dovecot/sieve-filter/clamav-filter.sh.
filter "clamav-filter.sh";

if header :contains "X-Spam-Flag" "YES" {
    fileinto "Spam";
}

Every Sieve script will be compiled by Dovecot at the first execution; you will find a file .svbin for each .sieve file. Beware that global Sieve files are usually stored in non-user-writable directories, to the sysadmin must compile them with sievec.

If one of the Sieve filters generate an error, you can see a log entry like this:

dovecot: lda(username)<20922><FIB4Kq5DKmC6UQAACo+hHQ>:
    program `/usr/local/lib/dovecot/sieve-filter/clamav-filter.sh'
    terminated with non-zero exit code 127

FIXME How to handle errors in filter external commands? The default action is to send a non-delivery notification, but you may want to fall-back to deliver the message anyway (e.g. if your antispam filter goes broken).

Multiple Sieve Scripts

In the simplest case there is only one Sieve script per each user, which is $HOME/.dovecot.sieve. The user can create several scripts into $HOME/sieve/ directory, and choose which one to activate by making the proper symlink. The managing of these scripts can be done using the Managesieve daemon on the server and a Managesieve client (available e.g. as a Roundcube plugin).

In some cases it is desiderable to have several Sieve scripts and apply them in sequence, beside the one activated by the user and Managesieve. The most flexible way is to use a sieve_before and/or a sieve_after directory.

Scripts sieve_before and sieve_after

In /etc/dovecot/conf.d/90-sieve.conf we can configure some files or directories to contain Sieve scripts (with the proper .sieve extension), to be executed before or after the user's personal script. If the path starts with ~/ it is relative to the user's home directory (scripts are executed in alphabetical order):

plugin {
    ...
    sieve_before = ~/sieve_before.d/
    sieve_before2 = /var/lib/dovecot/sieve.d/
    #sieve_after =
    ...
}

The Include plugin

Another useful mechanism is to provide some system or user Sieve scripts, and let the users to use them via the include directive. The location for user's scripts is $HOME/sieve by default, for global scripts you have to define sieve_global, e.g.:

plugin {
    sieve = file:~/sieve;active=~/.dovecot.sieve
    ...
    sieve_global = /etc/dovecot/sieve
    ...
}

once defined the location, the user can create a Sieve script like this:

require ["include"];
include :personal "my_script";
include :global "spam_tests";

The files (with the .sieve extension) will be searched into the proper directory.

Using Sieve to decode winmail.dat attachments

Microsoft Outlook uses the infamous winmail.dat attachment to forward emails: it is the proprietary TNEF format. In this article you can find a recipe to filter that attachments using a Sieve filter: Filtering TNEF (winmail.dat) attachments with Sieve.

Roundcube with MySQL

If you missed the database configuration during Roundcube installation, you can re-do it with:

dpkg-reconfigure roundcube-core
  • IMAP server: 127.0.0.1
  • Default language: it_IT
  • Reinstall database for roundcube: Yes
  • Connection method for MySQL database of roundcube: Unix socket
  • MySQL database name for roundcube: roundcube
  • MySQL username for roundcube: roundcube@localhost
  • MySQL application password for roundcube: <blank>
  • Name of the database's administrative user: root
  • Web server(s) to configure automatically: apache2

Roundcube with PostgreSQL

  • IMAP server: 127.0.0.1
  • Default language: it_IT
  • Reinstall database for roundcube: Yes
  • Database type to be used by roundcube: pgsql
  • Connection method for PostgreSQL database of roundcube: Unix socket
  • Method for authenticating the PostgreSQL administrator: ident
  • Method for authenticating PostgreSQL user: password
  • PostgreSQL database name for roundcube: roundcube
  • PostgreSQL username for roundcube: roundcube
  • PostgreSQL application password for roundcube: <blank>
  • Name of the database's administrative user: postgres
  • Web server(s) to configure automatically: apache2

Roundcube: Virtual Users, etc.

/etc/roundcube/config.inc.php

// The default locale setting (leave empty for auto-detection)
$config['language'] = 'it_IT';

Whenever an user login into Roundcube, the webmail attempts an IMAP authentication using the same credentials. If it succeeds, it will create a new record into the users table, filling the username field with the used login name. If the IMAP server accepts both the Unix login name and the full email address as login, you can find yourself having two different Roundcube users for the same IMAP mailbox.

In this scenario is preferable to disable in Roundcube the automatic creation of users and enable the use of email from user's identities as login.

// Set to false if only registered users can use this service
$config['auto_create_user'] = false;
// Enables possibility to log in using email address from user identities
$config['user_aliases'] = true;

The only drawback is that you have to populate the users and identities database tables before the user can login into Roundcube:

INSERT INTO users (username, mail_host, language)
    VALUES ('niccolo', '127.0.0.1', 'it_IT');
SELECT user_id FROM users WHERE username = 'niccolo';
INSERT INTO identities (user_id, standard, name, email)
    VALUES (1234, 1, 'Niccolo Rigacci', 'niccolo@domain.org');

ManageSieve Daemon

Install the dovecot-managesieved Debian package, verify that the daemon is responding on port 4190/TCP. The protocol is enabled by the included snippet /usr/share/dovecot/protocols.d/managesieved.protocol.

Test the ManageSieve Daemon

See ManageSieve Troubleshooting. It is possible to test a ManageSieve daemon talking the proper language over an interactive TCP/IP session. The language is specified by RFC5804.

To authenticate ourself to the server, we'll need a base64 encoded string containing \000login\000password:

echo -en "\000niccolo@mail.domain.org\000MySecret" | base64
AG5pY2NvbG9AbWFpbC5kb21haW4ub3JnAE15U2VjcmV0
telnet localhost 4190

In this session we will authenticate, check if there is space to store a script, create a script and activate it:

CAPABILITY
AUTHENTICATE "PLAIN" "AG5pY2NvbG9AbWFpbC5kb21haW4ub3JnAE15U2VjcmV0"

HAVESPACE "myscript" 200
PUTSCRIPT "myscript" {109+}
require ["copy"];
if header :contains "Subject" "newsletter"
{
    redirect :copy "other@domain.org";
}

LISTSCRIPTS
SETACTIVE "myscript"

LOGOUT

To calculate the size of the script (109 bytes in our case), use cr+lf as newline separator.

Connect Roundcube to the ManageSieve Daemon

Install the Debian package roundcube-plugins, which cointains the managesieve plugin. Edit the /etc/roundcube/plugins/managesieve/config.inc.php file, and set:

$config=array();
$config['managesieve_port'] = 4190;
$config['managesieve_host'] = 'localhost';
$config['managesieve_default'] = '/etc/dovecot/sieve/global';

Enable the plugin adding it to the array in /etc/roundcube/config.inc.php:

$config['plugins'] = array(
    'managesieve'
);

After configuring the two files, login into the Roundcube web interface, click on Settings, you should have a new tab labeled Filters on the left (after Preferences, Folders, etc.)

Roundcube, ManageSieve, sieve_extprograms and Include

It seems that Roundcube with ManageSieve plugin, cannot handle scripts which use the sieve_extprograms plugin or the include directive. The filters are not properly shown into the web interface, and they gets corrupted when the plugin edit/rewrite the script.

Vacation with Roundcube and Sieve/ManageSieve

A Vacation (or out-of-office) automatic responder can ben activated using a Sieve filter. The Roundcube webmail has a plugin which enables a web interface to activate and configure the vacation auto-responder. The Roundcube plugin uses the ManageSieve client/server method to create and activate the Sieve script.

To expose the specific vacation web interface, put into /etc/roundcube/plugins/managesieve/config.inc.php:

$config['managesieve_vacation'] = 1;

With this config, pointing the web interface to the Settings page, you should have a new tab labeled Vacation on the left (after the Filters one). The vacation rules can be managed with the generic Filters tab too, so there is the option to show only the Filter tab (value 0), both the Filter and the Vacation tabs (value 1), or only the Vacation tab (value 2).

This is a Sieve file generated by the Roundcube managesieve/vacation plugin:

require ["date","relational","vacation"];
# rule:[Vacation]
if allof (currentdate :zone "+0100" :value "ge" "iso8601" "2018-01-17T17:00:00+01:00", 
          currentdate :zone "+0100" :value "le" "iso8601" "2018-02-08T09:00:00+01:00")
{
    vacation :subject "Out of Office" "I'm on vacation until February 8th. Be patient.";
}

Information about what addresses were already responded and when, are stored into $HOME/.dovecot.lda-dupes.

The managesieve Roundcube plugin is configured by /etc/roundcube/plugins/managesieve/config.inc.php. Many options are explained into the provided /usr/share/roundcube/plugins/managesieve/config.inc.php.dist file.

Vacation and Virtual Domains

You can use the virtual_alias_maps to handle several virtual domains, e.g. you receive mails for @domain1.org, @domain2.org, etc. while myhostname is mail.domain.org.

This is a problem for Dovecot Vacation, because every local delivery is translated from virtual domain to myhostname, as you can see into Postfix logfile:

postfix/local: ... to=<niccolo@mail.domain.org>, orig_to=<niccolo.rigacci@domain1.org>, ...

in that scenario the filter does not send the automatic reply, because it is implicitly delivered:

discarding vacation response for implicitly delivered message; no known (envelope)
recipient address found in message headers (recipient=<niccolo@mail.domain.org>,
and no additional `:addresses' are specified)

The trick is to enumerate the addresses for which the vacation should respond. In the Roundcube Vacation web interface, Advanced settings, you have the My email addresses field, where you have to add your virtual addresses.

Roundcube can fill automatically that field, using the email address found into the Identities settings, just add the following in /etc/roundcube/plugins/managesieve/config.inc.php:

$config['managesieve_vacation_addresses_init'] = true;

Obviously Roundcube is not aware of aliases and/or virtual addresses configured in Postfix, so you have to add them manually, if any.

DAViCal

We used version 1.1.7 from Debian Stretch Backports, which requires libawl-php 0.59-1 from the same backports. For installation instructions, follow /usr/share/doc/davical/README.Debian.

Add the following to /etc/postgresql/9.6/main/pg_hba.conf (then reload the postgresql service):

local   davical   davical_app   trust
local   davical   davical_dba   trust

As the postgres user, run /usr/share/davical/dba/create-database.sh. Take note of the admin generated password.

Customize /etc/apache2/sites-available/davical.conf. E.g. we used the alias /dav/ instead of /davical/, we forced SSL, etc.

Customize /etc/davical/config.php, we changed:

$c->system_name = "CalDAV Server";
$c->admin_email ='postmaster@domain.org';

Enable the site, the required module, and restart Apache:

a2ensite davical
a2enmod rewrite
systemctl restart apache2

Point the browser to https://host.domain.org/dav/setup.php and login as admin with the above generated password.

Securing the PostgreSQL Access

Once installed, we can make a more secure database connection, setting a password. Change the pg_hba.conf file with:

local   davical    davical_app   md5
local   davical    davical_dba   md5

As the postgres user, connect to the database and issue:

ALTER USER davical_app PASSWORD 'PwdSecret';
ALTER USER davical_dba PASSWORD 'PwdSecret';

In /etc/davical/config.php add the password into the connection string:

$c->pg_connect[] = 'dbname=davical port=5432 user=davical_app password=PwdSecret';

Finally protect the configuration file:

chgrp www-data /etc/davical/config.php
chmod 640 /etc/davical/config.php

Apache RewriteRule

The default base URL for DAViCal resources is http://host.domain.org/davical/caldav.php/, but using alias and rewrite rules you can use more pretty (and well-known) URLs.

Beware that RewriteRule directives must be in VirtualHost section of Apache configuration, so it is likely that you have to replicate the examples found in /etc/apache2/sites-available/davical.conf into your /etc/apache2/sites-available/default-ssl.conf or alike.

Here we used a more generic dav alias instead of davical, and activated the suggested well-known URLs:

# Rewrite rules for DAViCal, must be in VirtualHost section.
RewriteEngine On
#LogLevel info rewrite:trace2
RewriteRule ^/\.well-known/(.*)$ /dav/caldav.php/.well-known/$1 [NC,PT]
RewriteRule ^/principals/users/(.*)$ /dav/caldav.php/$1 [NC,PT]
RewriteRule ^/principals/resources/(.*)$ /dav/caldav.php/$1 [NC,PT]
RewriteRule ^/calendars/__uids__/(.*)$ /dav/caldav.php/$1 [NC,PT]
RewriteRule ^/addressbooks/__uids__/(.*)$ /dav/caldav.php/$1 [NC,PT]

Create an User with a Calendar and an Addressbook

To create a new user, login to the web interface as admin, click on User FunctionsCreate Principal. As Princial Type choose Person. For each person created, DAViCal will create also a calendar and an addressbook, with the following URLs:

  • https://host.domain.org/dav/caldav.php/<username>/calendar
  • https://host.domain.org/dav/caldav.php/<username>/addresses

With the proper Rewrite rules (Apache configuration) you can use also well known URLs, like:

  • http://host.domain.org/principals/users/<username>/calendar
  • http://host.domain.org/principals/users/<username>/addresses

It depends upon the client to be able to discover the proper resource (e.g. the addressbook) when given just the user's URL. May be you have to experiment to discover the capabilities of your client.

  • With Thunderbird 52.5 and the CardBook 26.0 extension, we tested with success the URL
    http://host.domain.org/principals/users/<username>/addresses
  • With Thunderbird and the Lightning 52.5.2 calendar extension, we tested with success the URL
    http://host.domain.org/principals/users/<username>/calendar

There is no automatic way to authenticate in DAViCal using the Dovecot Auth: you must create the user into the DAViCal database. If you prefer, you can use the email address as the login name. When creating the Person you are asked for an email address, which should be unique among the persons, otherwise the scheduling my not work.

Into the davical database, you can browse the table collection to see addressbooks and calendars, browse the table addressbook_resource to see the names, etc.

Quota

See Dovecot Quota.

If Dovecot is started via Systemd, you must disable the PrivateDevices directive (which is enabled e.g. in Debian 10 Buster), otherwise the service does not have access to the /dev/ directory and cannot check filesystem quota. To override the defaults, just run:

systemctl edit dovecot

and insert the following:

[Service]
PrivateDevices=off

That settings will be saved into /etc/systemd/system/dovecot.service.d/override.conf. To enforce the new settings execute:

systemctl daemon-reload
systemctl restart dovecot.service 

Enable overall quota plugin in /etc/dovecot/conf.d/10-mail.conf:

mail_plugins = $mail_plugins quota

Dovecot Quota Backend

Dovecot quota plugin can relay on several backends: dirsize means calculate the actual disk space used every time (very slow!!), maildir means voluntary quota via maildirsize file, etc. We will instead use fs, the quota enforced by the Linux filesystem. In /etc/dovecot/conf.d/90-quota.conf set:

plugin {
  quota = fs:User quota
}

The Dovecot plugin will return mailbox full when user will reach the filesystem quota soft limit. You may want to set the hard limit a bit above the soft one, so that the system can still write mailbox indexes, Spamassassin stats, etc.

Enable Filesystem Quota

Enabling filesystem quota in GNU/Linux can be as simple as running (notice we activated only user quota, not group):

mount -o remount,usrquota /home
quotacheck --user --create-files /home
quotaon /home

You must add the usrquota mount option in /etc/fstab. To check if quota is enabled:

quotaon -p /home
group quota on /home (/dev/sdb) is off
user quota on /home (/dev/sdb) is on
project quota on /home (/dev/sdb) is off

To check quota for each user:

repquota /home
*** Report for user quotas on device /dev/sdb
Block grace time: 7days; Inode grace time: 7days
                        Block limits                File limits
User            used    soft    hard  grace    used  soft  hard  grace
----------------------------------------------------------------------
root      --      24       0       0              3     0     0       
niccolo   --     288   30000   31000             71     0     0       

Checking Dovecot Quota

To query the Dovecot quota backend, use doveadm quota (notice that you can use the Dovecot login name):

doveadm quota get -u username@example.org
Quota name Type     Value  Limit    %
User quota STORAGE   2880  30000    9

NOTICE: The Dovecot's quota limit is the soft limit imposed by the filesystem.

Dovecot quota status service for Postfix

We enable a service that will be checked by Postfix when accepting mail via SMTP. Into /etc/dovecot/conf.d/90-quota.conf add the following:

service quota-status {
  executable = quota-status -p postfix
  inet_listener {
    port = 12340
  }
  client_limit = 1
}

In the same configuration file, set also the responses returned by the service. Notice that if you set a grace quota space of 10%, you must set the filesystem soft quota limit 10% smaller than the hard one; or even more, to be safe:

plugin {
  quota_grace = 10%%
  quota_status_success = DUNNO
  quota_status_nouser = DUNNO
  quota_status_overquota = "554 5.2.2 Quota exceeded (mailbox for user is full)"
}

The Postfix quota-status service may uses also a quota_over_flag provided by the userdb backend. We don't have such a field into the userdb, neverthless we have to configure the parameters quota_over_flag_value and quota_over_script, otherwise the quota check is skipped entirely (don't be fooled if the doveadm tool is working, verify also that the Postfix check is actually working!):

# === WARNING ===
# Both "quota_over_flag_value" and "quota_over_script" are 
# required, otherwise the Postfix quota-status check will always 
# return DUNNO (i.e. user is under quota). This is because 
# without those two parameters the quota check is skipped 
# completely, as seen in the mail_debug = yes log:
# Debug: quota: quota_over_flag check: quota_over_script unset - skipping
#
# Actually we don't have a "quota_over_flag" field in userdb; 
# when quota-status service is called by Postfix, the value 
# quota_over_flag=0(*dummy*) is assumed due the config values 
# below. Current quota status for the user is then checked by 
# querying the filesystem and it will be 1 for overquota, 0 
# otherwise.
#
# As a side effect, the quota-warning script is executed at 
# every check if the user is overquota because 0 mismatches 1.
plugin {
  quota_over_flag_value = FALSE
  quota_over_flag = "*dummy*"
  quota_over_script = quota-warning mismatch %u
}

We have to define also a quota-warning service, which basically is a script called when the user crosses some quota barriers, we can use it to send warning messages. The script needs to be run as root, because we want it to be able to switch to the final user (this is required by our system which uses Maildir and user's filters). The /var/run/dovecot/quota-warning socket must be world-writable so the user can write to it.

service quota-warning {
  executable = script /usr/local/bin/quota-warning.sh
  user = root
  unix_listener quota-warning {
    user = dovecot
    group = dovecot
    mode = 0666
  }
}

To have the script executed when crossing some quota limits, add the following:

plugin {
  quota_warning = storage=90%% quota-warning 90 %u
  quota_warning2 = storage=75%% quota-warning 75 %u
}

So, all after all, here it is the script executed when an user is crossing some quota levels and when the fake quota_over_flag does not match the actual quota status (i.e. it is called when the user is over quota, and we just do nothing in that case!):

/usr/local/bin/quota-warning.sh

#!/bin/sh
ARG1=$1
USER=$2
DATE="$(date -R)"
 
# ==== WARGNING ====
#  * The user is set by Dovecot "service quota-warning", in our
#    configuration is: uid=0(root) gid=0(root) groups=0(root)
#  * Working directory is /run/dovecot/
#  * /tmp directory is /tmp/systemd-private-<...>-dovecot.service-<...>/tmp
 
# Do nothing if called on Dovecot quota_over_flag_value mismatch.
test "$ARG1" = "mismatch" && exit 0
 
# Send the warning message.
#cat << EOF | /usr/lib/dovecot/dovecot-lda -d $USER -o "plugin/quota=fs:User quota:noenforcing"
cat << EOF | /usr/lib/dovecot/dovecot-lda -d $USER
From: postmaster@supermail.texnet.it
Subject: Attenzione: mailbox quasi piena
Date: $DATE
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
 
La tua mailbox è quasi piena; lo spazio occupato ha superato il ${ARG1}%.
Eliminare i messaggi non più necessari ed eventualmente svuotare il cestino.
EOF

Troubleshoting Postfix quota-status service

After reloading dovecot.service, you will have a daemon listening on port 12340/TCP. To check if the service is working, you can telnet to the TCP port and paste some strings of text, the relevant lines are recipient and size in bytes (see more on smtpd-policy language in this example):

telnet localhost 12340

recipient=user@domain.org
size=10000000

action=554 5.2.2 Quota exceeded (mailbox for user is full)

The 554 Quota exceeded error should be returned when the actual user's quota plus the announced size will exceed the filesystem soft quota plus the quota_grace percentage.

Another way to check the service using a single command line is using netcat:

printf "recipient=user@domain.org\nsize=3000000\n\n" | nc localhost 12340

In Postfix configuration /etc/postfix/main.cf add a restriction on smtpd (the order of restrictions matter):

smtpd_recipient_restrictions =
    ...
    check_policy_service inet:127.0.0.1:12340,
    ...

If you use the same Postfix instance for submission and smtp services, it may will perform an useless call to quota-status for emails to be relayed. The check should return the DUNNO status in case of quota_status_nouser, i.e. proceed with the next restriction rule check.

Quota and dovecot-lda

dovecot-lda, on local deliveries, will chek user quota too. Debian default is to include global mail_plugins, so no need to add quota into /etc/dovecot/conf.d/15-lda.conf:

protocol lda {
  mail_plugins = $mail_plugins
}

So also locally generated mails (i.e. not received via SMTP) will be checked against filesystem quota, and eventually bounced with a MAILER-DAEMON error email message.

Dovecot Quota for IMAP

To enable quota support into the IMAP server, add to /etc/dovecot/conf.d/20-imap.conf:

protocol imap {
  mail_plugins = $mail_plugins imap_quota
}

Verify that QUOTA is displayed among CAPABILITY(ies), using telnet to port 143/TCP:

telnet localhost 143
a1 LOGIN username SuperSecret
a1 OK [CAPABILITY IMAP4rev1 ... QUOTA] Logged in
a2 LOGOUT

Checking SSL Certificates

We should periodically check every SSL-enabled service for the certificate validity. The monitoring-plugins-basic Debian package contains some useful scripts that can be used. Here are an example script:

#!/bin/sh
echo "Checking SSL certificate fot SMTP on port 25/TCP..."
/usr/lib/nagios/plugins/check_smtp -H localhost --port=25 --starttls -D 28
echo
echo "Checking SSL certificate fot POP3D on port 995/TCP..."
/usr/lib/nagios/plugins/check_pop -H localhost --ssl --port=995 -D 28
echo
echo "Checking SSL certificate fot IMAP on port 993/TCP..."
/usr/lib/nagios/plugins/check_imap -H localhost --ssl --port=993 -D 28
doc/appunti/linux/sa/postfix_spamassassin_clamav_dovecot.txt · Last modified: 2023/10/30 11:09 by niccolo