doc:appunti:linux:sa:postfix_spamassassin_clamav_dovecot
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
doc:appunti:linux:sa:postfix_spamassassin_clamav_dovecot [2018/01/30 23:33] – [Dovecot Quota Backend] niccolo | doc:appunti:linux:sa:postfix_spamassassin_clamav_dovecot [2023/02/24 16:18] – [Sanitizer/ClamAV Filter] niccolo | ||
---|---|---|---|
Line 39: | Line 39: | ||
We want a **single authentication mechanism** for SMTP, POP3 and IMAP, and we want to let the user to authenticate using the Unix **username** or the full **email address** and the password. Dovecot authentication is suitable for the task; the default Debian configuration provides authentication against the system usernames via the PAM module. The configuration file is **/ | We want a **single authentication mechanism** for SMTP, POP3 and IMAP, and we want to let the user to authenticate using the Unix **username** or the full **email address** and the password. Dovecot authentication is suitable for the task; the default Debian configuration provides authentication against the system usernames via the PAM module. The configuration file is **/ | ||
- | To let authentication using the email address, we provide a passwd-like file **/ | + | To let authentication using the email address, we provide a passwd-like file **/ |
< | < | ||
Line 50: | Line 50: | ||
passdb { | passdb { | ||
driver = passwd-file | driver = passwd-file | ||
- | args = scheme=CRYPT / | + | args = scheme=CRYPT |
} | } | ||
userdb { | userdb { | ||
driver = passwd-file | driver = passwd-file | ||
- | args = / | + | args = username_format=%u |
} | } | ||
</ | </ | ||
Line 64: | Line 64: | ||
< | < | ||
- | !include auth-system.conf.ext | + | # Order matters: we use passwdfile first because it is most likely used |
+ | # and because system autentication (PAM) does a 2-seconds delay on fail. | ||
!include auth-passwdfile.conf.ext | !include auth-passwdfile.conf.ext | ||
+ | !include auth-system.conf.ext | ||
</ | </ | ||
Line 85: | Line 87: | ||
</ | </ | ||
- | The file **/ | + | The file **/ |
< | < | ||
- | chmod 0640 / | + | chmod 0640 / |
- | chown root: | + | chown root: |
</ | </ | ||
+ | |||
+ | If you need to **generate the hash for a password**, you can use the following command line: | ||
+ | |||
+ | < | ||
+ | openssl passwd -1 | ||
+ | </ | ||
+ | |||
+ | When the **/ | ||
+ | |||
+ | |||
===== Postfix SASL over Dovecot Auth ===== | ===== Postfix SASL over Dovecot Auth ===== | ||
- | See **[[https:// | + | See **[[https:// |
First of all we need to activate the socket used by Posfix for authentication. In **/ | First of all we need to activate the socket used by Posfix for authentication. In **/ | ||
Line 137: | Line 149: | ||
... | ... | ||
</ | </ | ||
+ | |||
+ | **WARNING**: | ||
===== Dovecot IMAP and POP3 ===== | ===== Dovecot IMAP and POP3 ===== | ||
Line 145: | Line 159: | ||
* **dovecot-pop3d** | * **dovecot-pop3d** | ||
- | Once installed, the basic services **IMAP2** on port **TCP/143** and **POP3** on port **TCP/110** are already working, eventually with the **Dovecot Authentication on userdb** as seen above. | + | Once installed, the basic services **IMAP2** on port **TCP/143** and **POP3** on port **TCP/110** are already working, eventually with the **Dovecot Authentication on users** as seen above. |
==== Enable SSL, imaps and pop3s ==== | ==== Enable SSL, imaps and pop3s ==== | ||
Line 159: | Line 173: | ||
</ | </ | ||
- | Dovecot will read the SSL files with root privileges, so no particular settings should be required. | + | Dovecot will read the SSL files with root privileges, so no particular settings should be required. If the certificate is **renewed**, |
- | Once restarted the //dovecot.service//, we will find the services listening on the well known ports, and we can check the SSL certificates. | + | Once restarted the **dovecot.service**, we will find the services listening on the well known ports, and we can check the SSL certificates. |
===== SpamAssassin Filter ===== | ===== SpamAssassin Filter ===== | ||
Line 184: | Line 198: | ||
CRON=1 | CRON=1 | ||
</ | </ | ||
+ | |||
+ | To force a manual update you can run **/ | ||
===== Sanitizer/ | ===== Sanitizer/ | ||
- | For virus scanning of email attachments we use the ClamAV antivirus software. We install the daemon, the client and the database periodic update utility: | + | For virus scanning of email attachments we use the ClamAV antivirus software. We install |
+ | * **clamav** | ||
* **clamav-daemon** | * **clamav-daemon** | ||
* **clamdscan** | * **clamdscan** | ||
* **clamav-freshclam** | * **clamav-freshclam** | ||
+ | * **libclamunrar9** (non-free package to scan inside RAR archives) | ||
Periodic download (update) of viruses database is performed by the **clamav-freshclam.service**, | Periodic download (update) of viruses database is performed by the **clamav-freshclam.service**, | ||
Line 202: | Line 220: | ||
cat message.txt | / | cat message.txt | / | ||
</ | </ | ||
+ | |||
+ | Beware that the **clamd** daemon runs per default with the **clamav user**, if you want regular users to scan their files (e.g. their mail attachments) you have to use the **%%--fdpass%%** argument when invoking the **clamdscan** client, otherwise you will get the error message: | ||
+ | |||
+ | < | ||
+ | eicar-antivirus.com: | ||
+ | </ | ||
+ | |||
+ | The '' | ||
===== Dovecot Local Delivery Agent ===== | ===== Dovecot Local Delivery Agent ===== | ||
Line 226: | Line 252: | ||
mailbox_command = / | mailbox_command = / | ||
</ | </ | ||
+ | |||
+ | ==== Error stats-writer Broken pipe ==== | ||
+ | |||
+ | **WARNING**: | ||
+ | |||
+ | < | ||
+ | postfix/ | ||
+ | status=bounced (Command died with status 134: | ||
+ | "/ | ||
+ | Command output: lda(lica-marilena): | ||
+ | Error: net_connect_unix(/ | ||
+ | Permission denied Aborted Unable to flush stdout: Broken pipe ) | ||
+ | </ | ||
+ | |||
+ | It seems that the quickest and simplest solution is to make the socket **0666 mode** (which will risk the stats service to be abused). Add the following snippet to **/ | ||
+ | |||
+ | < | ||
+ | service stats { | ||
+ | unix_listener stats-writer { | ||
+ | mode = 0666 | ||
+ | } | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | See the following posts about the problem: | ||
+ | |||
+ | * [[https:// | ||
+ | * [[https:// | ||
+ | |||
+ | ==== Error Command output: Aborted ==== | ||
+ | |||
+ | Another subtle error with Dovecot LDA can be reported generically into the Postfix log as: | ||
+ | |||
+ | < | ||
+ | postfix/ | ||
+ | status=bounced (Command died with status 134: | ||
+ | "/ | ||
+ | Command output: Aborted ) | ||
+ | </ | ||
+ | |||
+ | To get some hints about the problem you can enable logging in Dovecot LDA; you can add the following in **/ | ||
+ | |||
+ | < | ||
+ | protocol lda { | ||
+ | ... | ||
+ | log_path = / | ||
+ | info_log_path = / | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | and create the log files with suitable permissions (in my case the LDA is executed with user's privileges, so I made it world-writable). | ||
+ | |||
+ | < | ||
+ | lda(username)< | ||
+ | exec:/ | ||
+ | Execution timed out (> 10000 msecs) | ||
+ | lda(username)< | ||
+ | exec:/ | ||
+ | Forcibly terminated with signal 15 | ||
+ | lda(username)< | ||
+ | output stream (temp iostream in / | ||
+ | for (program client seekable output)) is missing error handling | ||
+ | lda(username)< | ||
+ | Raw backtrace: / | ||
+ | </ | ||
+ | |||
+ | There was a problem with **the filter program responding too slowly**. It is possible to change the **exec timeout**, in this case for the **filter** extension, just add the following in **/ | ||
+ | |||
+ | < | ||
+ | plugin { | ||
+ | ... | ||
+ | # Change the default timeout (10 seconds) for the filter extension. | ||
+ | sieve_filter_exec_timeout = 20s | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | ==== Error Unable to flush stdout ==== | ||
+ | |||
+ | FIXME This error message has an unwknown origin. | ||
+ | |||
+ | < | ||
+ | postfix/ | ||
+ | status=bounced (Command died with status 134: | ||
+ | "/ | ||
+ | Command output: Aborted Unable to flush stdout: Broken pipe ) | ||
+ | </ | ||
===== Sieve filtering ===== | ===== Sieve filtering ===== | ||
Line 327: | Line 439: | ||
Every Sieve script will be **compiled** by Dovecot at the first execution; you will find a file **.svbin** for each **.sieve** file. Beware that **global Sieve files** are usually stored in non-user-writable directories, | Every Sieve script will be **compiled** by Dovecot at the first execution; you will find a file **.svbin** for each **.sieve** file. Beware that **global Sieve files** are usually stored in non-user-writable directories, | ||
+ | |||
+ | If one of the Sieve filters generate an error, you can see a log entry like this: | ||
+ | |||
+ | < | ||
+ | dovecot: lda(username)< | ||
+ | program `/ | ||
+ | terminated with non-zero exit code 127 | ||
+ | </ | ||
+ | |||
+ | FIXME How to handle errors in filter external commands? The default action is to send a **non-delivery notification**, | ||
+ | |||
==== Multiple Sieve Scripts ==== | ==== Multiple Sieve Scripts ==== | ||
Line 369: | Line 492: | ||
The files (with the .sieve extension) will be searched into the proper directory. | The files (with the .sieve extension) will be searched into the proper directory. | ||
+ | |||
+ | ==== Using Sieve to decode winmail.dat attachments ==== | ||
+ | |||
+ | Microsoft Outlook uses the infamous **winmail.dat** attachment to forward emails: it is the proprietary **TNEF** format. In this article you can find a recipe to filter that attachments using a Sieve filter: **[[sieve_filtering_tnef]]**. | ||
+ | |||
===== Roundcube with MySQL ===== | ===== Roundcube with MySQL ===== | ||
Line 425: | Line 553: | ||
The only drawback is that you have to populate the **users** and **identities** database tables before the user can login into Roundcube: | The only drawback is that you have to populate the **users** and **identities** database tables before the user can login into Roundcube: | ||
- | < | + | < |
+ | INSERT INTO users (username, mail_host, language) | ||
+ | VALUES (' | ||
+ | SELECT user_id FROM users WHERE username = ' | ||
+ | INSERT INTO identities (user_id, standard, name, email) | ||
+ | VALUES (1234, 1, ' | ||
</ | </ | ||
- | FIXME Add SQL recipes above. | ||
===== ManageSieve Daemon ===== | ===== ManageSieve Daemon ===== | ||
Line 652: | Line 784: | ||
===== Quota ===== | ===== Quota ===== | ||
- | See **[[https:// | + | See **[[https:// |
+ | |||
+ | If Dovecot is started via **Systemd**, | ||
+ | |||
+ | < | ||
+ | systemctl edit dovecot | ||
+ | </ | ||
+ | |||
+ | and insert the following: | ||
+ | |||
+ | < | ||
+ | [Service] | ||
+ | PrivateDevices=off | ||
+ | </ | ||
+ | |||
+ | That settings will be saved into **/ | ||
+ | |||
+ | < | ||
+ | systemctl daemon-reload | ||
+ | systemctl restart dovecot.service | ||
+ | </ | ||
+ | |||
+ | Enable overall quota plugin in **/ | ||
< | < | ||
Line 667: | Line 821: | ||
</ | </ | ||
- | The Dovecot plugin will return mailbox full when user will reach the filesystem quota **soft limit**. You may want to set the **hard limit** a bit above the soft one, so that the system can still write mailbox indexes, Spamassassin stats, etc. | + | The Dovecot plugin will return mailbox full when user will reach the **filesystem quota soft limit**. You may want to set the **hard limit** a bit above the soft one, so that the system can still write mailbox indexes, Spamassassin stats, etc. |
- | ==== Dovecot Quota service for Postfix ==== | + | |
+ | ==== Enable Filesystem Quota ==== | ||
+ | |||
+ | Enabling **filesystem quota** in GNU/Linux can be as simple as running (notice we activated only **user** quota, not **group**): | ||
+ | |||
+ | < | ||
+ | mount -o remount, | ||
+ | quotacheck --user --create-files /home | ||
+ | quotaon /home | ||
+ | </ | ||
+ | |||
+ | You must add the '' | ||
+ | |||
+ | < | ||
+ | quotaon -p /home | ||
+ | group quota on /home (/dev/sdb) is off | ||
+ | user quota on /home (/dev/sdb) is on | ||
+ | project quota on /home (/dev/sdb) is off | ||
+ | </ | ||
+ | |||
+ | To check quota for each user: | ||
+ | |||
+ | < | ||
+ | repquota /home | ||
+ | *** Report for user quotas on device /dev/sdb | ||
+ | Block grace time: 7days; Inode grace time: 7days | ||
+ | Block limits | ||
+ | User used soft hard grace used soft hard grace | ||
+ | ---------------------------------------------------------------------- | ||
+ | root -- 24 | ||
+ | niccolo | ||
+ | </ | ||
+ | |||
+ | ==== Checking | ||
+ | |||
+ | To query the Dovecot quota backend, use **doveadm quota** (notice that you can use the Dovecot login name): | ||
+ | |||
+ | < | ||
+ | doveadm quota get -u username@example.org | ||
+ | Quota name Type | ||
+ | User quota STORAGE | ||
+ | </ | ||
+ | |||
+ | **NOTICE**: The Dovecot' | ||
+ | |||
+ | ==== Dovecot quota status | ||
We enable a service that will be checked by Postfix when accepting mail via SMTP. Into **/ | We enable a service that will be checked by Postfix when accepting mail via SMTP. Into **/ | ||
Line 682: | Line 881: | ||
</ | </ | ||
- | In the same configuration file, select | + | In the same configuration file, set also the responses returned by the service. Notice that if you set a **grace quota space** of 10%, you must set the filesystem |
< | < | ||
plugin { | plugin { | ||
- | | + | |
quota_status_success = DUNNO | quota_status_success = DUNNO | ||
quota_status_nouser = DUNNO | quota_status_nouser = DUNNO | ||
Line 693: | Line 892: | ||
</ | </ | ||
- | After reloading | + | The Postfix quota-status service may uses also a **quota_over_flag** provided by the **userdb** backend. We don't have such a field into the userdb, neverthless we have to configure |
< | < | ||
- | request=smtpd_access_policy | + | # === WARNING === |
- | protocol_state=RCPT | + | # Both " |
- | protocol_name=SMTP | + | # required, otherwise the Postfix quota-status check will always |
- | sender=foo@bar.tld | + | # return DUNNO (i.e. user is under quota). This is because |
+ | # without those two parameters the quota check is skipped | ||
+ | # completely, as seen in the mail_debug | ||
+ | # Debug: quota: quota_over_flag check: quota_over_script unset - skipping | ||
+ | # | ||
+ | # Actually we don't have a " | ||
+ | # when quota-status service is called by Postfix, the value | ||
+ | # quota_over_flag=0(*dummy*) is assumed due the config values | ||
+ | # below. Current quota status for the user is then checked by | ||
+ | # querying the filesystem and it will be 1 for overquota, 0 | ||
+ | # otherwise. | ||
+ | # | ||
+ | # As a side effect, the quota-warning script is executed at | ||
+ | # every check if the user is overquota because 0 mismatches 1. | ||
+ | plugin { | ||
+ | quota_over_flag_value | ||
+ | quota_over_flag = " | ||
+ | quota_over_script = quota-warning mismatch %u | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | We have to define also a **quota-warning service**, which basically is a script called when the user crosses some quota barriers, we can use it to send warning messages. The script needs to be run as root, because we want it to be able to switch to the final user (this is required by our system which uses Maildir and user's filters). The **/ | ||
+ | |||
+ | < | ||
+ | service quota-warning { | ||
+ | executable = script / | ||
+ | user = root | ||
+ | unix_listener quota-warning { | ||
+ | user = dovecot | ||
+ | group = dovecot | ||
+ | mode = 0666 | ||
+ | } | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | To have the script executed when crossing some quota limits, add the following: | ||
+ | |||
+ | < | ||
+ | plugin { | ||
+ | quota_warning = storage=90%% quota-warning 90 %u | ||
+ | quota_warning2 = storage=75%% quota-warning 75 %u | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | So, all after all, here it is the script executed when an user is crossing some quota levels and when the fake **quota_over_flag** does not match the actual quota status (i.e. it is called when the user is over quota, and we just do nothing in that case!): | ||
+ | |||
+ | **/ | ||
+ | |||
+ | <code bash> | ||
+ | #!/bin/sh | ||
+ | ARG1=$1 | ||
+ | USER=$2 | ||
+ | DATE=" | ||
+ | |||
+ | # ==== WARGNING ==== | ||
+ | # * The user is set by Dovecot " | ||
+ | # configuration is: uid=0(root) gid=0(root) groups=0(root) | ||
+ | # * Working directory is / | ||
+ | # * /tmp directory is / | ||
+ | |||
+ | # Do nothing if called on Dovecot quota_over_flag_value mismatch. | ||
+ | test " | ||
+ | |||
+ | # Send the warning message. | ||
+ | #cat << EOF | / | ||
+ | cat << EOF | / | ||
+ | From: postmaster@supermail.texnet.it | ||
+ | Subject: Attenzione: mailbox quasi piena | ||
+ | Date: $DATE | ||
+ | Content-Type: | ||
+ | Content-Transfer-Encoding: | ||
+ | |||
+ | La tua mailbox è quasi piena; lo spazio occupato ha superato il ${ARG1}%. | ||
+ | Eliminare i messaggi non più necessari ed eventualmente svuotare il cestino. | ||
+ | EOF | ||
+ | </ | ||
+ | |||
+ | ==== Troubleshoting Postfix quota-status service ==== | ||
+ | |||
+ | After reloading **dovecot.service**, | ||
+ | |||
+ | < | ||
+ | telnet localhost 12340 | ||
recipient=user@domain.org | recipient=user@domain.org | ||
- | client_address=1.2.3.4 | + | size=10000000 |
- | client_name=another.domain.tld | + | |
- | size=123000 | + | |
action=554 5.2.2 Quota exceeded (mailbox for user is full) | action=554 5.2.2 Quota exceeded (mailbox for user is full) | ||
</ | </ | ||
+ | |||
+ | The **554 Quota exceeded error** should be returned when the **actual user's quota** plus the announced **size** will exceed the filesystem **soft quota** plus the **quota_grace** percentage. | ||
+ | |||
+ | Another way to check the service using a single command line is using **netcat**: | ||
+ | |||
+ | < | ||
+ | printf " | ||
+ | </ | ||
In Postfix configuration **/ | In Postfix configuration **/ | ||
Line 719: | Line 1007: | ||
If you use the same Postfix instance for **submission** and **smtp** services, it may will perform an useless call to **quota-status** for **emails to be relayed**. The check should return the //DUNNO// status in case of // | If you use the same Postfix instance for **submission** and **smtp** services, it may will perform an useless call to **quota-status** for **emails to be relayed**. The check should return the //DUNNO// status in case of // | ||
- | ==== Dovecot-lda Quota ==== | + | ==== Quota and dovecot-lda ==== |
- | Local delivery performed by **dovecot-lda** will chek user quota too. | + | **dovecot-lda**, on local deliveries, |
+ | < | ||
+ | protocol lda { | ||
+ | mail_plugins = $mail_plugins | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | So also locally generated mails (i.e. not received via SMTP) will be checked against filesystem quota, and eventually bounced with a MAILER-DAEMON error email message. | ||
==== Dovecot Quota for IMAP ==== | ==== Dovecot Quota for IMAP ==== | ||
+ | To enable quota support into the IMAP server, add to **/ | ||
+ | |||
+ | < | ||
+ | protocol imap { | ||
+ | mail_plugins = $mail_plugins imap_quota | ||
+ | } | ||
+ | </ | ||
+ | |||
+ | Verify that **QUOTA** is displayed among CAPABILITY(ies), | ||
+ | |||
+ | < | ||
+ | telnet localhost 143 | ||
+ | a1 LOGIN username SuperSecret | ||
+ | a1 OK [CAPABILITY IMAP4rev1 ... QUOTA] Logged in | ||
+ | a2 LOGOUT | ||
+ | </ | ||
===== Checking SSL Certificates ===== | ===== Checking SSL Certificates ===== | ||
doc/appunti/linux/sa/postfix_spamassassin_clamav_dovecot.txt · Last modified: 2023/10/30 11:09 by niccolo