Part 3: How to Set up SPF and DKIM with Postfix on Ubuntu Server
After completing part 1 and part 2, we have a working Postfix SMTP server and Dovecot IMAP server. We can send and receive email using a desktop email client. Although I have correct MX, A and PTR record, my emails were flagged as spam by Gmail and Outlook mail. So in part 3, we are going to look at how to improve email delivery to recipient’s inbox by setting up SPF and DKIM on Ubuntu server.
What are SPF and DKIM Records?
SPF and DKIM are two types of TXT records in DNS that can help prevent email spoofing and ensure legitimate emails are delivered into the recipient’s inbox instead of spam folder. If your domain is abused by email spoofing, then your emails are likely to landed in recipient’s spam folder if they didn’t add you in address book.
SPF (Sender Policy Framework) record specifies which hosts or IP addresses are allowed to send emails on behalf of a domain. You should allow only your own email server or your ISP’s server to send emails for your domain.
DKIM (DomainKeys Identified Mail) uses a private key to add a signature to emails sent from your domain. Receiving SMTP servers verify the signature by using the corresponding public key, which is published in your DNS manager.
Create an SPF Record in DNS
In your DNS management interface, create a new TXT record like below.
TXT @ v=spf1 mx ~all
Explanation:
- TXT indicates this is a TXT record.
- Enter @ in the name field.
- v=spf1 indicates this is a SPF record and the SPF record version is SPF1.
- mx means all hosts listed in the MX records are allowed to send emails for your domain and all other hosts are disallowed.
- ~all indicates that emails from your domain should only come from hosts specified in the SPF record. Emails that are from other hosts will be flagged as forged. Possible alternatives are
+all,-all,?all, but they are rarely used.
Note that some DNS managers require you to wrap the SPF record with quotes like below.
TXT @ "v=spf1 mx ~all"
To check if your SPF record is propagated to the public Internet, you can use the dig utility on your Linux machine like below:
dig your-domain.com txt
The txt option tells dig that we only want to query TXT records.
You can also use online SPF validator such as spf.myisp.ch to see which hosts are allowed to send emails for your domain and debug your SPF record if any error occurs. The dmarcian SPF surveyor can help test your SPF record syntax.
Configuring SPF Policy Agent
We also need to tell our Postfix SMTP server to check for SPF record of incoming emails. This doesn’t help ensure outgoing email delivery but help with detecting forged incoming emails.
First install required packages:
sudo apt install postfix-policyd-spf-python
Then edit the Postfix master process configuration file.
sudo nano /etc/postfix/master.cf
Add the following lines at the end of the file.
policyd-spf unix - n n - 0 spawn
user=policyd-spf argv=/usr/bin/policyd-spf
Save and close the file. Next, edit Postfix main configuration file.
sudo nano /etc/postfix/main.cf
Append the following lines at the end of the file. The first line specifies the Postfix policy agent timeout setting. The following lines will impose restriction on incoming emails by rejecting unauthorized email and checking SPF record.
policyd-spf_time_limit = 3600 smtpd_recipient_restrictions = reject_unauth_destination, check_policy_service unix:private/policyd-spf
Save and close the file. Then restart Postfix.
sudo service postfix restart
or
sudo systemctl restart postfix
Next time, when you receive an email from a domain that has an SPF record, you can see the SPF check results in the raw email header. The following header indicates a successful check against SPF.
Received-SPF: Pass (sender SPF authorized).
Setting up DKIM
First install OpenDKIM which is an open source implementation of the DKIM sender authentication system.
sudo apt install opendkim opendkim-tools
Then add postfix user to opendkim group.
sudo gpasswd -a postfix opendkim
Edit OpenDKIM main configuration file.
sudo nano /etc/opendkim.conf
Uncomment the following lines. Replace simple with relaxed/simple.
Canonicalization simple Mode sv SubDomains no
Then add the following lines below #ADSPAction continue line. If your file doesn’t have #ADSPAction continue line, then just add them below SubDomains no.
AutoRestart yes AutoRestartRate 10/1M Background yes DNSTimeout 5 SignatureAlgorithm rsa-sha256
Add the following lines at the end of this file. (On Ubuntu 18.04, the UserID is already set to opendkim)
#OpenDKIM user # Remember to add user postfix to group opendkim UserID opendkim # Map domains in From addresses to keys used to sign messages KeyTable /etc/opendkim/key.table SigningTable refile:/etc/opendkim/signing.table # Hosts to ignore when verifying signatures ExternalIgnoreList /etc/opendkim/trusted.hosts InternalHosts /etc/opendkim/trusted.hosts
The final configuration file is as follows:
# This is a basic configuration that can easily be adapted to suit a standard # installation. For more advanced options, see opendkim.conf(5) and/or # /usr/share/doc/opendkim/examples/opendkim.conf.sample. # Log to syslog Syslog yes # Required to use local socket with MTAs that access the socket as a non- # privileged user (e.g. Postfix) UMask 002 # Sign for example.com with key in /etc/mail/dkim.key using # selector '2007' (e.g. 2007._domainkey.example.com) #Domain example.com #KeyFile /etc/mail/dkim.key #Selector 2007 # Commonly-used options; the commented-out versions show the defaults. Canonicalization relaxed/simple Mode sv SubDomains no #ADSPAction continue AutoRestart yes AutoRestartRate 10/1M Background yes DNSTimeout 5 SignatureAlgorithm rsa-sha256 # Always oversign From (sign using actual From and a null From to prevent # malicious signatures header fields (From and/or others) between the signer # and the verifier. From is oversigned by default in the Debian pacakge # because it is often the identity key used by reputation systems and thus # somewhat security sensitive. OversignHeaders From # List domains to use for RFC 6541 DKIM Authorized Third-Party Signatures # (ATPS) (experimental) #ATPSDomains example.com #OpenDKIM user # Remember to add user postfix to group opendkim UserID opendkim # Map domains in From addresses to keys used to sign messages KeyTable /etc/opendkim/key.table SigningTable refile:/etc/opendkim/signing.table # Hosts to ignore when verifying signatures ExternalIgnoreList /etc/opendkim/trusted.hosts InternalHosts /etc/opendkim/trusted.hosts
Save and close the file.
Create Signing table, key table and trusted hosts file
Create a directory structure for OpenDKIM
sudo mkdir /etc/opendkim sudo mkdir /etc/opendkim/keys
Change owner from root to opendkim and make sure only opendkim user can read and write to the keys directory.
sudo chown -R opendkim:opendkim /etc/opendkim sudo chmod go-rw /etc/opendkim/keys
Create the signing table.
sudo nano /etc/opendkim/signing.table
Add this line to the file. Note that your-domain.com and your-domain is different. The latter should not contain the top-level domain.
*@your-domain.com default._domainkey.your-domain
Then create the key table.
sudo nano /etc/opendkim/key.table
Add the following line.
default._domainkey.your-domain your-domain.com:default:/etc/opendkim/keys/your-domain.com/default.private
Save and close the file.
Configure Trusted Hosts
Create the file.
sudo nano /etc/opendkim/trusted.hosts
Add the following lines to the newly created file.
127.0.0.1
localhost
*.your-domain.com
The above means that messages coming from the above IP addresses and domains will be trusted and signed.
Generate Private/Public Keypair
Since DKIM is used to sign outgoing messages and verify incoming messages, we need to generate a private key for signing and a public key for remote verifier. Public key will be published in DNS.
Create a separate folder for the domain.
sudo mkdir /etc/opendkim/keys/your-domain.com
Generate keys using opendkim-genkey tool.
sudo opendkim-genkey -b 2048 -d your-domain.com -D /etc/opendkim/keys/your-domain.com -s default -v
The above command will create 2048 bits keys. -d (domain) specifies the domain. -D (directory) specifies the directory where the keys will be stored and we use default as the selector (-s), also known as the name. Once the command is executed, the private key will be default.private and default.txt will be the TXT record that contains public key.
Make opendkim as the owner of the private key.
sudo chown opendkim:opendkim /etc/opendkim/keys/your-domain.com/default.private
Add Public Key in DNS Records
Display the public key
sudo cat /etc/opendkim/keys/your-domain.com/default.txt
The string after the p parameter is the public key.
In you DNS manager, create a TXT record, enter default._domainkey in the name field. Then copy everything in the parentheses and paste it into the value field. Delete all double quotes and white spaces. If you don’t delete them, then the key test in the next step will fail.
Test your configuration
Enter the following command on Ubuntu 16.04 server to test your key.
sudo opendkim-testkey -d your-domain.com -s default -vvv
If everything is OK, you will see
key OK
Connect Postfix to OpenDKIM
Postfix can talk to OpenDKIM via a Unix socket file. The default socket file used by OpenDKIM is /var/run/opendkim/opendkim.sock. But the postfix SMTP daemon shipped with Ubuntu runs in a chroot jail, which means that the SMTP daemon resolves all filenames relative to the Postfix queue directory (/var/spool/postfix). So we need to change the socket file.
Create a directory to hold the OpenDKIM socket file and only allow opendkim user and postfix group to access it.
sudo mkdir /var/spool/postfix/opendkim sudo chown opendkim:postfix /var/spool/postfix/opendkim
Then edit the socket configuration file.
sudo nano /etc/default/opendkim
Find the following line:
SOCKET="local:/var/run/opendkim/opendkim.sock"
Replace it with:
SOCKET="local:/var/spool/postfix/opendkim/opendkim.sock"
Save and close the file.
Note: On Ubuntu 18.04, the opendkim systemd service doesn’t use /etc/default/opendkim file. You need to change the socket file in /etc/opendkim.conf file.
sudo nano /etc/opendkim.conf
Find the following line:
Socket local:/var/run/opendkim/opendkim.sock
Replace it with:
Socket local:/var/spool/postfix/opendkim/opendkim.sock
Next, we need to edit Postfix main configuration file.
sudo nano /etc/postfix/main.cf
Add the following lines after smtpd_recipient_restriction section.
# Milter configuration milter_default_action = accept milter_protocol = 6 smtpd_milters = local:/opendkim/opendkim.sock non_smtpd_milters = $smtpd_milters
Save and close the file. Then restart opendkim and postfix service.
sudo service opendkim restart sudo service postfix restart
or
sudo systemctl restart opendkim sudo systemctl restart postfix
SPF and DKIM Check
Now you can use your desktop email client or webmail client to send a test email to [email protected] and get a free email authentication report. Here’s the report I got from port25.com
You can see that my email passed both SPF and DKIM check. iprev check is used to see if the reverse (IP to hostname) and forward (hostname to IP) lookup results were returned and were in agreement. Ham is a terminology used by Apache SpamAssassin to indicate that this is not spam.
You can also send a test email to your Gmail account to see if SPF and DKIM checks are passed. On the right side of an opened email message in Gmail, if you click the show original button from the drop-down menu, you can see the authentication results.
If your message is not signed and DKIM check failed, you may want to check postfix log (/var/log/mail.log) to see what’s wrong in your configuration.
Your email server will also perform SPF and DKIM check on sender’s domain. You can see the results in the email headers. The following is SPF and DKIM check on a sender using Gmail.
Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::c2d; helo=mail-yw1-xc2d.google.com; [email protected]; receiver=<UNKNOWN> Authentication-Results: email.linuxbabe.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com [email protected] header.b="XWMRd2co"; dkim-atps=neutral
The Struggle with Microsoft Mailboxes
In my test, the email landed in my Gmail inbox. However, it’s stilled labeled as spam in my outlook.com email although both SPF and DKIM are passed. Microsoft seems to be using an internal blacklist that block many legitimate IP addresses. If your emails are rejected by outlook or hotmail, you need to submit the sender information form. After that, your email will be accepted by outlook/hotmail, but may still be labeled as spam.
In part 4, we will see how to create DMARC record to protect your domain from email spoofing. As always, if you found this post useful, please subscribe to our free newsletter or follow us on Google+, Twitter or like our Facebook page
Could you please write about how to forward only email setup with postfix or any other email server.
You mean you want your email server to forward every incoming email to another email service like Gmail? After Postfix is installed, you simply need to create a .forward file in your home dirctory and add the destination address where the email will be forwarded.
Nice, Please add it on a section of this article 🙂
Thank you
I think it will be more appropriate to add this tip in the first part of this tutorial series and I added it in the bonus tip section of this article.
Yes, It will be great 🙂
Thanks for the guide!
I have a forwarding/relay address and I’ve been trying to set up postsrsd, but for the life of me it still fails DKIM checks! Wondering if this series can be expanded to provide Sender Rewriting Scheme (SRS)?
eg:
mydomain.com -> gmail.com DKIM PASS
[email protected] -> [email protected] -> [email protected] DKIM FAIL
Cheers
this tutorial rocks! Got DKIM to work..finally!! …Thanks
Great tutorial cheers!! I have followed all the instructions to setup opendkim on aws ubuntu 14.04. Now, I am using Route 53 for DNS records management. I got KEY OK as described in the tutorial. But I have my mail server pointing to mail.mydomain.com and I don’t see dkim=pass when I test my installation. What should I change to make it work for subdomains?
You might want to suggest to add to opendkm.conf
LogWhy yes
So it will print what the problem is. I spent several hours figuring out permissions issues.
hello, cannot start/restart dkim
I got this error:
opendkim[9027]: opendkim: /etc/opendkim.conf: /etc/opendkim/key.table SigningTable refile:/etc/opendkim/signing.t
able: dkimf_db_open(): Unknown database type
please help. thank you
I have a problem with the file sudo nano /etc/default/opendkim and i can’t check the DKIM
This post saved me
https://serverfault.com/a/847442/438721
OK So I have a question. I followed everything step by step and even double and triple checked everything but every time I go to sudo opendkim-testkey -d your-domain.com -s default -vvv it tells me that ‘default._domainkey.MYDOMAIN.com’ record not found… Is there a reason its doing this and if so is there a way that I can uninstall and reinstall DKIM so that I can start over?
very Nice tutorial. Everything works fine for me but can I use the smtp variable on a mailer. Like gammadyne mailer
You should ask them before buying their product.
OpenDKIM Filter: Unable to bind to port local:/var/run/postfix/opendkim/opendkim.sock: No such file or directory
Hi, the socket should be
local:/var/spool/postfix/opendkim/opendkim.sockHi,
No success , still getting dkim none error while following this tutorial please guide me.
Please paste here any error message you see and the command that generated that error message.
Thanks for this. FYI, after following this guide, my SPF checks failed with SOFTFAIL. I had to update the SPF TXT record with my server’s IP address in order to get the SPF check to pass. Example:
v=spf1 mx ip4:11.22.33.44 ~all
X-Originating-IP: [14.143.226.38]
Authentication-Results: mta4079.mail.sg3.yahoo.com from=talentanytime.com; domainkeys=neutral (no sig); from=talentanytime.com; dkim=neutral (no sig)
i have done all configuration successfully but it is coming while sending mail (dkim=neutral (no sig)
kindly suggest.
Ubuntu 18.04 >>> No matter what config files i edited, the service couldn’t work with /var/spool/postfix/opendkim, so i manually created the folder /run/opendkim or /var/run/opendkim and gave it ownership permissions to opendkim user & group.
The service started successfully, and the opendkim service created it’s PID in the created folder. Everything works normal now.
Never mind, please disregard my above post. It was a mistake done on my own end which resulted in the opendkim PID not working.
Hello,
In signing table (/etc/opendkim/signing.table) it is mentioned to add the line
*@your-domain.com default._domainkey.your-domain
Should I add my domain (after default._domainkey.) with the .com extension or without it ?
Thanks,
Daniel
Without the .com extension.
Hello,
Thank you Xiao for the guide and the support !
I managed to install and configure my mail server.
One comment:
While Copy + Paste the key table line:
default._domainkey.your-domain your-domain.com:default:/etc/opendkim/keys/your-domain.com/default.private
few non ascii chars in between “default._domainkey.your-domain” and “your-domain.com:default:/etc/opendkim/keys/your-domain.com/default.private” somehow appeared and caused port25 DKIM test to fail.
I had to remove them and add one blank instead.
Thanks !
Daniel Orkan
Hi.
Thanks for this awesome guide.
I have done my best to follow the guide, and I get the following error when I check the DKIM by sending a mail by using the following command:
echo “test email” | sendmail [email protected]
The opendkim.sock is empty, and when I try to open it by using Nano, it creates an empty file like if it hasn’t been created. I have no idea on what I’ve messed up.
I have even followed the guide twice to make sure I didn’t messed up anything, but I must have made a small mistake somewhere. Any idea on what I have done wrong?
Best regards SLJ.
Looks like you have duplicate configurations in
/etc/postfix/main.cffile. Don’t worry about the content of opendkim.sock.Hi.
Thanks for your reply. That was exactly the issue. An other guide which didn’t worked told me to put in those lines at an other place in the file.
I keep trying to get SPF and DKIM to work by following the guide, but i’m having the following problems which I have no idea on how to solve:
1. As soon as I have configured the SPF on Postfix, the server won’t send any mails at all. I’m doing exactly all the steps which is explain the section: Configuring SPF Policy Agent.
2. I thought I could avoid setting up SPF and just set up DKIM, but the same happens. Postfix restarts without any errors.
When I test the DKIM by using the following command: echo “test email” | sendmail [email protected]
I don’t get anything at all in the output. It’s just like doing a command which the console accepts without any comments or output.
I’m running a Mailman server. I don’t know if that makes any difference.
When testing the DKIM key, it says it’s not secure and then it says it’s okay. So I expect that part to be fine. But I don’t know why it says it’s not secure.
Do you have any ideas on how to bug fix this, or any clue on what’s wrong?
You won’t see anything on the command line when you send a test email to [email protected] You will receive the report in your mailbox.
But, as explained, the mailserver stops working both when I set up SPF on the server, and when I connect DKIM to Postfix. It doesn’t give me any errors, and no mails comes through Mailman.
Hi.
Do you have any clue on why it won’t work for me? As soon as I set up SPF or DKIM, the Postfix mailserver stops working, and no mails comes through the mailinglists on the Mailman server. Any tips on how to bugfix on this is very much welcome.
You should take a look at Postfix log file (
/var/log/mail.log). If SPF or DKIM is not working, you will see errors in that file.Hi,
Thank you for this detailed tutorial.I have not used ubuntu but Raspian on Raspberry Pi 3B and it worked according to your instructions 🙂
I have 3 observations:
1. In one of your statement there is incorrect phrase “… end of this the file.” but you should really have “… end of this file.” or “… end of the file.”. Not both 😉
2. When testing setting with email using command “echo “test email” | sendmail [email protected]” there is no ‘From’ option. This would mean that email it was sent from would be @your-domain.com (in my case it would be user ‘pi’) and when you do not have such account created, report would not be delivered. Is this correct?
Maybe it will be worth adding i.e. “-aFrom:[email protected]”?
Funny enough, I have not recived report even using that option with specified From value 🙁
3. As per your suggestion, I have used Gmail to verify DKIM record and it looks very much correct, but it has landed in Spam folder anyway. I believe google use some other parameters to mark emails as spam.
This is what I got in the headers from Gmail raw message:
Authentication-Results: mx.google.com; dkim=pass [email protected] header.s=default header.b=b6ARK37a; spf=pass1. Thanks for pointing out the grammar error. Just fixed it.
2. If there’s no From address, then Postfix will append the domain name to the username, so [email protected] is the From address and report will be sent to that address. You can use a command line mail client to see the report.
If you are setting up a email server on a Raspberry Pi in your home, there’s a high chance that your ISP blocks port 25. Ask your ISP if
that’s the case.
3. Yes, there are other factors. You can test your email score at mail-tester.com
Hi.
The log file didn’t gave me much useful information.
The following guide works for me:
https://mladenadamovic.wordpress.com/2018/01/17/how-to-install-and-configure-dkim-with-postfix-on-ubuntu-xenial-16-04-lts/
Setting up SPF and DKIM this way does not crash my server. Not even if the DKIM key do not match. I managed to get it all working by following this guide.
I’m not posting this to say that your guide is bad. It is really awesome, and gives a lot of great information. I’m posting the link to the other guide in case other people are having trouble on getting your guide to work. Maybe you will find it useful.
Feel free to remove this comment if you don’t feel it fits here.
I think the only difference is that I use Unix domain docket. The article you followed uses TCP/IP socket.
No mail from root…. for some reason postfix it’s not receiving any email … please could you help me? thanks in advance
Superb tutorial. Tried many others and kept getting the configuration wrong. This time around it is working! Kudos mate, well done!
You are welcome 🙂
Worked in first attempt, just be careful if you are using bind9, you need to craft the record and split the key. Use below example for bind9. (Do not remove ending dot(.) after your-domain.com
; DKIM policy record
_domainkey.your-domain.com. IN TXT “o=!;”
; DKIM public key record
default._domainkey.your-domain.com. IN TXT “v=DKIM1;k=rsa” “p=MIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAv/ZySexJFmPRz6d8QbRymsNi77yKUACoHevdTSltOhqSHo9YpgjxLA” “52fVm+3jQ3CIn8Z2vxDAclHEFOhnEKE/YnZaAge6BmjhgfdrtyuionhgUmH2hWqRbZeVUl4X36C4XHxvcKUroURxBKwPw474ma” “TrLubDdFq76fXmodhsgsTgBhYYThHJn47odgLynG5uo/LjoBzw8ygqRD6R41jS8ZwMKdovC5rEnoQ+w1PZK6g4rlas4si6yyZN” “bgqmjH4geSCPpfWvuBbjFn0qV79cOttSrpdIiI/3Hceod0GN8sOIUhgtRtghUjhE5yxwyhZix1sfFmNgWHwIDAQAB”
Followed carefully, but I get this error in mail.log
I’m on Ubuntu 12.04
There wasn’t a line with Socket, so I added it
Any suggestion on what I did wrong?
Ubuntu 12.04 is out of support cycle and I haven’t used openDKIM on 12.04. I suggest changing the socket file location in both /etc/default/opendkim and /etc/opendkim.conf file.
Then restart openDKIM and Postfix, cd into /var/spool/postfix/opendkim/ directory to see if the opendkim.sock can be found. If not, it’s likely a permission problem.
Finally! I got it to work on by removing the first slash:
smtpd_milters = local:opendkim/opendkim.sock
instead of
smtpd_milters = local:/opendkim/opendkim.sock
Now the DKIM pass
Díky za pěkný návod….
Thanks for the nice tutorial.
This is an awesome guide. Worked perfectly for me. Thanks so much!
Hello great tutorial but i have two problems:
1. When i am testing the configuration i got this message: “key not secure, key OK”. Why isn`t secure?
2. Even if on mail-test i have 10/10, gmail is getting all my mails in spam. Any ideea?
Thank you for your good job
After following the tutorial, when trying to start opendkim with the command: systemctl start opendkim.service
The following message appears:
Job for opendkim.service failed because the control process exited with error code.
See “systemctl status opendkim.service” and “journalctl -xe” for details.
$ journalctl -xe
After following your instructions everything was OK. The only part you would need to update is to extend the /etc/postfix/main.cf if one is using SASL:
to
otherwise SMTP clients will fail.
Thank you for this AWESOME tutorial!!!
This is a really good tutorial, one of the best I’ve seen.
I experienced the problem that some of my smtp processes were running in a chroot jail and others were not.
This was easily fixed by using a network instead of file sockets.
/etc/default/opendkim
/etc/postfix/main.cf
I now get 10/10, thanks so much for the tutorial!
Hi Thanks for the tutorial. I have been stuck on query timed out for days now. Tried specifying nameservers, turning off firewall. I search the logs and there is nothing. tail -f /var/log/syslog never shows anything happening when I run opendkim-testkey.
I can find the DKIM record of your domain without any problems, using the
digutility.Yeah, that’s what’s so frustrating. The problem is opendkim-testkey times out.
[email protected]:/etc# opendkim-testkey -d mydomain.com -s 20190319 -vvv
opendkim-testkey: using default configfile /etc/opendkim.conf
opendkim-testkey: checking key ‘20190319._domainkey.mydomain.com’
opendkim-testkey: ‘20190319._domainkey.mydomain.com’ query timed out
And with the dig command and in opendkim.conf you can specify the nameserver used. So I used the same dns nameserver for both.
The important thing is that your domain can pass DKIM check and your email server can perform DKIM check on sender’s domain.