Part 4: How to Set up SPF and DKIM with Postfix on Ubuntu Server
After completing part 1 and part 2, we have a working Postfix SMTP server and Dovecot IMAP server. We can send and receive email using a desktop email client. Although I have correct MX, A and PTR records, my emails were flagged as spam by Gmail and Outlook mail. So in this part, we are going to look at how to improve email delivery to the recipient’s inbox by setting up SPF and DKIM on Ubuntu server.
What are SPF and DKIM Records?
SPF and DKIM are two types of TXT records in DNS that allow you to detect email spoofing and help legitimate emails deliver into the recipient’s inbox instead of the spam folder. If your domain is abused by email spoofing, then your emails are likely to land in the recipient’s spam folder if they didn’t add you to the address book.
SPF (Sender Policy Framework) record specifies which hosts or IP addresses are allowed to send emails on behalf of a domain. You should allow only your own email server or your ISP’s server to send emails for your domain.
DKIM (DomainKeys Identified Mail) uses a private key to add a signature to emails sent from your domain. Receiving SMTP servers verify the signature by using the corresponding public key, which is published in your DNS manager.
Create an SPF Record in DNS
In your DNS management interface, create a new TXT record like below.
TXT @ v=spf1 mx ~all
Explanation:
- TXT indicates this is a TXT record.
- Enter @ in the name field.
- v=spf1 indicates this is an SPF record and the SPF record version is SPF1.
- mx means all hosts listed in the MX records are allowed to send emails for your domain and all other hosts are disallowed.
- ~all indicates that emails from your domain should only come from hosts specified in the SPF record. Emails that are from other hosts will be flagged as forged. Possible alternatives are
+all,-all,?all, but they are rarely used.
Note that some DNS managers require you to wrap the SPF record with quotes like below.
TXT @ "v=spf1 mx ~all"
To check if your SPF record is propagated to the public Internet, you can use the dig utility on your Linux box like below. (On Ubuntu, you need to install the bind9-dnsutils package in order to use dig command: sudo apt install bind9-dnsutils)
dig your-domain.com txt
The txt option tells dig that we only want to query TXT records.
You can also use the dmarcian SPF surveyor to test your SPF record syntax.
Configuring SPF Policy Agent
We also need to tell our Postfix SMTP server to check for SPF record of incoming emails. This doesn’t help ensure outgoing email delivery but help with detecting forged incoming emails.
First, install required packages:
sudo apt install postfix-policyd-spf-python
Then edit the Postfix master process configuration file.
sudo nano /etc/postfix/master.cf
Add the following lines at the end of the file, which tells Postfix to start the SPF policy daemon when it’s starting itself.
policyd-spf unix - n n - 0 spawn
user=policyd-spf argv=/usr/bin/policyd-spf
Save and close the file. Next, edit Postfix main configuration file.
sudo nano /etc/postfix/main.cf
Append the following lines at the end of the file. The first line specifies the Postfix policy agent timeout setting. The following lines will impose a restriction on incoming emails by rejecting unauthorized email and checking SPF record.
policyd-spf_time_limit = 3600 smtpd_recipient_restrictions = permit_mynetworks, permit_sasl_authenticated, reject_unauth_destination, check_policy_service unix:private/policyd-spf
Save and close the file. Then restart Postfix.
sudo systemctl restart postfix
Next time, when you receive an email from a domain that has an SPF record, you can see the SPF check results in the raw email header. The following header indicates the sender sent the email from an authorized host.
Received-SPF: Pass (sender SPF authorized).
Setting up DKIM
First, install OpenDKIM which is an open-source implementation of the DKIM sender authentication system.
sudo apt install opendkim opendkim-tools
Then add postfix user to opendkim group.
sudo gpasswd -a postfix opendkim
Edit OpenDKIM main configuration file.
sudo nano /etc/opendkim.conf
Uncomment the following lines. Replace simple with relaxed/simple.
Canonicalization simple Mode sv SubDomains no
Then add the following lines below #ADSPAction continue line. If your file doesn’t have #ADSPAction continue line, then just add them below SubDomains no.
AutoRestart yes AutoRestartRate 10/1M Background yes DNSTimeout 5 SignatureAlgorithm rsa-sha256
Add the following lines at the end of this file. (Note that On Ubuntu 18.04 and 20.04, the UserID is already set to opendkim)
#OpenDKIM user # Remember to add user postfix to group opendkim UserID opendkim # Map domains in From addresses to keys used to sign messages KeyTable refile:/etc/opendkim/key.table SigningTable refile:/etc/opendkim/signing.table # Hosts to ignore when verifying signatures ExternalIgnoreList /etc/opendkim/trusted.hosts # A set of internal hosts whose mail should be signed InternalHosts /etc/opendkim/trusted.hosts
Save and close the file.
Create Signing Table, Key Table and Trusted Hosts File
Create a directory structure for OpenDKIM
sudo mkdir /etc/opendkim sudo mkdir /etc/opendkim/keys
Change the owner from root to opendkim and make sure only opendkim user can read and write to the keys directory.
sudo chown -R opendkim:opendkim /etc/opendkim sudo chmod go-rw /etc/opendkim/keys
Create the signing table.
sudo nano /etc/opendkim/signing.table
Add this line to the file. This tells OpenDKIM that if a sender on your server is using a @your-domain.com address, then it should be signed with the private key identified by default._domainkey.your-domain.com.
*@your-domain.com default._domainkey.your-domain.com
Save and close the file. Then create the key table.
sudo nano /etc/opendkim/key.table
Add the following line, which tells the location of the private key.
default._domainkey.your-domain.com your-domain.com:default:/etc/opendkim/keys/your-domain.com/default.private
Save and close the file. Next, create the trusted hosts file.
sudo nano /etc/opendkim/trusted.hosts
Add the following lines to the newly created file. This tells OpenDKIM that if an email is coming from localhost or from the same domain, then OpenDKIM should not perform DKIM verification on the email.
127.0.0.1
localhost
*.your-domain.com
Save and close the file.
Generate Private/Public Keypair
Since DKIM is used to sign outgoing messages and verify incoming messages, we need to generate a private key for signing and a public key for remote verifier. Public key will be published in DNS.
Create a separate folder for the domain.
sudo mkdir /etc/opendkim/keys/your-domain.com
Generate keys using opendkim-genkey tool.
sudo opendkim-genkey -b 2048 -d your-domain.com -D /etc/opendkim/keys/your-domain.com -s default -v
The above command will create 2048 bits keys. -d (domain) specifies the domain. -D (directory) specifies the directory where the keys will be stored and we use default as the selector (-s), also known as the name. Once the command is executed, the private key will be written to default.private file and the public key will be written to default.txt file.
Make opendkim as the owner of the private key.
sudo chown opendkim:opendkim /etc/opendkim/keys/your-domain.com/default.private
Publish Your Public Key in DNS Records
Display the public key
sudo cat /etc/opendkim/keys/your-domain.com/default.txt
The string after the p parameter is the public key.
In your DNS manager, create a TXT record, enter default._domainkey in the name field. Then go back to the terminal window, copy everything in the parentheses and paste it into the value field of the DNS record. You need to delete all double quotes and white spaces in the value field. If you don’t delete them, then the key test in the next step will probably fail.
Test DKIM Key
Enter the following command on Ubuntu server to test your key.
sudo opendkim-testkey -d your-domain.com -s default -vvv
If everything is OK, you will see Key OK in the command output.
opendkim-testkey: using default configfile /etc/opendkim.conf opendkim-testkey: checking key 'default._domainkey.your-domain.com' opendkim-testkey: key secure opendkim-testkey: key OK
If you see Key not secure in the command output, don’t panic. This is because DNSSEC isn’t enabled on your domain name. DNSSEC is a security standard for secure DNS query. Most domain names haven’t enabled DNSSEC. There’s absolutely no need to worry about Key not secure. You can continue to follow this guide.
Connect Postfix to OpenDKIM
Postfix can talk to OpenDKIM via a Unix socket file. The default socket file used by OpenDKIM is /var/run/opendkim/opendkim.sock, as shown in /etc/opendkim.conf file. But the postfix SMTP daemon shipped with Ubuntu runs in a chroot jail, which means the SMTP daemon resolves all filenames relative to the Postfix queue directory (/var/spool/postfix). So we need to change the OpenDKIM Unix socket file.
Create a directory to hold the OpenDKIM socket file and allow only opendkim user and postfix group to access it.
sudo mkdir /var/spool/postfix/opendkim sudo chown opendkim:postfix /var/spool/postfix/opendkim
Then edit the OpenDKIM main configuration file.
sudo nano /etc/opendkim.conf
Find the following line (Ubuntu 18.04)
Socket local:/var/run/opendkim/opendkim.sock
or (Ubuntu 20.04)
Socket local:/run/opendkim/opendkim.sock
Replace it with the following line. (If you can’t find the above line, then add the following line.)
Socket local:/var/spool/postfix/opendkim/opendkim.sock
Save and close the file.
If you can find the following line in /etc/default/opendkim file.
SOCKET="local:/var/run/opendkim/opendkim.sock"
or
SOCKET=local:$RUNDIR/opendkim.sock
Change it to
SOCKET="local:/var/spool/postfix/opendkim/opendkim.sock"
Save and close the file.
Next, we need to edit the Postfix main configuration file.
sudo nano /etc/postfix/main.cf
Add the following lines at the end of this file, so Postfix will be able to call OpenDKIM via the milter protocol.
# Milter configuration milter_default_action = accept milter_protocol = 6 smtpd_milters = local:opendkim/opendkim.sock non_smtpd_milters = $smtpd_milters
Save and close the file. Then restart opendkim and postfix service.
sudo systemctl restart opendkim postfix
SPF and DKIM Check
You can now send a test email from your mail server to your Gmail account to see if SPF and DKIM checks are passed. On the right side of an opened email message in Gmail, if you click the show original button from the drop-down menu, you can see the authentication results.
If your message is not signed and DKIM check failed, you may want to check postfix log (/var/log/mail.log) to see what’s wrong in your configuration. If you see the following message in the mail log, you may want to check if the opendkim systemd service is actually running.
warning: connect to Milter service local:opendkim/opendkim.sock: No such file or directory
If opendkim is running and you still see the above error, you might need to edit the /etc/postfix/main.cf file, change
smtpd_milters = local:opendkim/opendkim.sock
to
smtpd_milters = local:/opendkim/opendkim.sock
Then restart Postfix.
Your email server will also perform SPF and DKIM checks on the sender’s domain. You can see the results in the email headers. The following is SPF and DKIM check on a sender using Gmail.
Received-SPF: Pass (mailfrom) identity=mailfrom; client-ip=2607:f8b0:4864:20::c2d; helo=mail-yw1-xc2d.google.com; envelope-from=someone@gmail.com; receiver=<UNKNOWN> Authentication-Results: email.linuxbabe.com; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="XWMRd2co"; dkim-atps=neutral
Testing Email Score and Placement
You can also go to https://www.mail-tester.com. You will see a unique email address. Send an email from your domain to this address and then check your score. As you can see, I got a perfect score.
Mail-tester.com can only show you a sender score. There’s another service called GlockApps that allow you to check if your email is placed in the recipient’s inbox or spam folder, or rejected outright. It supports many popular email providers like Gmail, Outlook, Hotmail, YahooMail, iCloud mail, etc
The Struggle with Microsoft Mailboxes
In my test, the email landed in my Gmail inbox. However, it’s stilled labeled as spam in my outlook.com email although both SPF and DKIM are passed. Microsoft seems to be using an internal blacklist that blocks many legitimate IP addresses. If your emails are rejected by outlook or Hotmail, you need to submit the sender information form. After that, your email will be accepted by outlook/hotmail, but may still be labeled as spam.
What if Your Emails Are Still Being Marked as Spam?
I have more tips for you in this article: How to stop your emails from being marked as spam.
Next Step
In part 5, we will see how to create DMARC record to protect your domain from email spoofing. As always, if you found this post useful, please subscribe to our free newsletter or follow us on Twitter or like our Facebook page.
Note
In this article, I used umask 002 in the /etc/opendkim.conf file. A visitor commented below that this solved his problem. However, on Ubuntu 18.04 the default umask in /etc/opendkim.conf file is set to 007, which works fine on my production servers. The umask defines the permission of the OpenDKIM socket file (/var/spool/postfix/opendkim/opendkim.sock)
- 002: Users not in the opendkim group have read and execute permission, but no write permission.
- 007: Users not in the opendkim group have no read, write, or execute permission.
If I remember correctly, the umask 002 value is the default on Ubuntu 14.04 and 16.04, so I used it when I was writing this article for a Ubuntu 16.04 server.
If you added postfix user to the opendkim group, there should be no need to change the umask to 002.
sudo gpasswd -a postfix opendkim
I listed this command in the installing OpenDKIM section, but why some people don’t pay attention? I wonder.
Thanks for the guide!
I have a forwarding/relay address and I’ve been trying to set up postsrsd, but for the life of me it still fails DKIM checks! Wondering if this series can be expanded to provide Sender Rewriting Scheme (SRS)?
eg:
mydomain.com -> gmail.com DKIM PASS
someone@gmail.com -> forward@mydomain.com -> someone@yahoo.com DKIM FAIL
Cheers
this tutorial rocks! Got DKIM to work..finally!! …Thanks
Great tutorial cheers!! I have followed all the instructions to setup opendkim on aws ubuntu 14.04. Now, I am using Route 53 for DNS records management. I got KEY OK as described in the tutorial. But I have my mail server pointing to mail.mydomain.com and I don’t see dkim=pass when I test my installation. What should I change to make it work for subdomains?
You might want to suggest to add to opendkm.conf
LogWhy yes
So it will print what the problem is. I spent several hours figuring out permissions issues.
I have a problem with the file sudo nano /etc/default/opendkim and i can’t check the DKIM
This post saved me
https://serverfault.com/a/847442/438721
OK So I have a question. I followed everything step by step and even double and triple checked everything but every time I go to sudo opendkim-testkey -d your-domain.com -s default -vvv it tells me that ‘default._domainkey.MYDOMAIN.com’ record not found… Is there a reason its doing this and if so is there a way that I can uninstall and reinstall DKIM so that I can start over?
“Record not found” means that it can’t find your DKIM DNS record. DNS record can take some time to propagate to the Internet.
very Nice tutorial. Everything works fine for me but can I use the smtp variable on a mailer. Like gammadyne mailer
You should ask them before buying their product.
OpenDKIM Filter: Unable to bind to port local:/var/run/postfix/opendkim/opendkim.sock: No such file or directory
Hi, the socket should be
local:/var/spool/postfix/opendkim/opendkim.sockThanks for this. FYI, after following this guide, my SPF checks failed with SOFTFAIL. I had to update the SPF TXT record with my server’s IP address in order to get the SPF check to pass. Example:
v=spf1 mx ip4:11.22.33.44 ~all
thank u!
Ubuntu 18.04 >>> No matter what config files i edited, the service couldn’t work with /var/spool/postfix/opendkim, so i manually created the folder /run/opendkim or /var/run/opendkim and gave it ownership permissions to opendkim user & group.
The service started successfully, and the opendkim service created it’s PID in the created folder. Everything works normal now.
Never mind, please disregard my above post. It was a mistake done on my own end which resulted in the opendkim PID not working.
Hi.
Thanks for this awesome guide.
I have done my best to follow the guide, and I get the following error when I check the DKIM by sending a mail by using the following command:
echo “test email” | sendmail check-auth@verifier.port25.com
The opendkim.sock is empty, and when I try to open it by using Nano, it creates an empty file like if it hasn’t been created. I have no idea on what I’ve messed up.
I have even followed the guide twice to make sure I didn’t messed up anything, but I must have made a small mistake somewhere. Any idea on what I have done wrong?
Best regards SLJ.
Looks like you have duplicate configurations in
/etc/postfix/main.cffile. Don’t worry about the content of opendkim.sock.Hi.
Thanks for your reply. That was exactly the issue. An other guide which didn’t worked told me to put in those lines at an other place in the file.
I keep trying to get SPF and DKIM to work by following the guide, but i’m having the following problems which I have no idea on how to solve:
1. As soon as I have configured the SPF on Postfix, the server won’t send any mails at all. I’m doing exactly all the steps which is explain the section: Configuring SPF Policy Agent.
2. I thought I could avoid setting up SPF and just set up DKIM, but the same happens. Postfix restarts without any errors.
When I test the DKIM by using the following command: echo “test email” | sendmail check-auth@verifier.port25.com
I don’t get anything at all in the output. It’s just like doing a command which the console accepts without any comments or output.
I’m running a Mailman server. I don’t know if that makes any difference.
When testing the DKIM key, it says it’s not secure and then it says it’s okay. So I expect that part to be fine. But I don’t know why it says it’s not secure.
Do you have any ideas on how to bug fix this, or any clue on what’s wrong?
You won’t see anything on the command line when you send a test email to check-auth@verifier.port25.com. You will receive the report in your mailbox.
But, as explained, the mailserver stops working both when I set up SPF on the server, and when I connect DKIM to Postfix. It doesn’t give me any errors, and no mails comes through Mailman.
Hi.
Do you have any clue on why it won’t work for me? As soon as I set up SPF or DKIM, the Postfix mailserver stops working, and no mails comes through the mailinglists on the Mailman server. Any tips on how to bugfix on this is very much welcome.
You should take a look at Postfix log file (
/var/log/mail.log). If SPF or DKIM is not working, you will see errors in that file.Hi,
Thank you for this detailed tutorial.I have not used ubuntu but Raspian on Raspberry Pi 3B and it worked according to your instructions 🙂
I have 3 observations:
1. In one of your statement there is incorrect phrase “… end of this the file.” but you should really have “… end of this file.” or “… end of the file.”. Not both 😉
2. When testing setting with email using command “echo “test email” | sendmail check-auth@verifier.port25.com” there is no ‘From’ option. This would mean that email it was sent from would be @your-domain.com (in my case it would be user ‘pi’) and when you do not have such account created, report would not be delivered. Is this correct?
Maybe it will be worth adding i.e. “-aFrom:existing_email_account@your-domain.com”?
Funny enough, I have not recived report even using that option with specified From value 🙁
3. As per your suggestion, I have used Gmail to verify DKIM record and it looks very much correct, but it has landed in Spam folder anyway. I believe google use some other parameters to mark emails as spam.
This is what I got in the headers from Gmail raw message:
Authentication-Results: mx.google.com; dkim=pass header.i=@my-domain.com header.s=default header.b=b6ARK37a; spf=pass1. Thanks for pointing out the grammar error. Just fixed it.
2. If there’s no From address, then Postfix will append the domain name to the username, so pi@your-domain.com is the From address and report will be sent to that address. You can use a command line mail client to see the report.
If you are setting up a email server on a Raspberry Pi in your home, there’s a high chance that your ISP blocks port 25. Ask your ISP if
that’s the case.
3. Yes, there are other factors. You can test your email score at mail-tester.com
Hi.
The log file didn’t gave me much useful information.
The following guide works for me:
https://mladenadamovic.wordpress.com/2018/01/17/how-to-install-and-configure-dkim-with-postfix-on-ubuntu-xenial-16-04-lts/
Setting up SPF and DKIM this way does not crash my server. Not even if the DKIM key do not match. I managed to get it all working by following this guide.
I’m not posting this to say that your guide is bad. It is really awesome, and gives a lot of great information. I’m posting the link to the other guide in case other people are having trouble on getting your guide to work. Maybe you will find it useful.
Feel free to remove this comment if you don’t feel it fits here.
I think the only difference is that I use Unix domain docket. The article you followed uses TCP/IP socket.
Superb tutorial. Tried many others and kept getting the configuration wrong. This time around it is working! Kudos mate, well done!
You are welcome 🙂
Worked in first attempt, just be careful if you are using bind9, you need to craft the record and split the key. Use below example for bind9. (Do not remove ending dot(.) after your-domain.com
The DKIM record output by OpenDKIM is in BIND format.
If you run your own authoritative DNS server with BIND9, simply copy the entire DKIM record output by OpenDKIM, don’t delete anything, and paste it to your BIND zone file.
The following screenshot shows an example DKIM record in a BIND zone file.
Followed carefully, but I get this error in mail.log
I’m on Ubuntu 12.04
There wasn’t a line with Socket, so I added it
Any suggestion on what I did wrong?
Ubuntu 12.04 is out of support cycle and I haven’t used openDKIM on 12.04. I suggest changing the socket file location in both /etc/default/opendkim and /etc/opendkim.conf file.
Then restart openDKIM and Postfix, cd into /var/spool/postfix/opendkim/ directory to see if the opendkim.sock can be found. If not, it’s likely a permission problem.
Finally! I got it to work on by removing the first slash:
instead of
Now the DKIM pass
Díky za pěkný návod….
Thanks for the nice tutorial.
This is an awesome guide. Worked perfectly for me. Thanks so much!
Hello great tutorial but i have two problems:
1. When i am testing the configuration i got this message: “key not secure, key OK”. Why isn`t secure?
2. Even if on mail-test i have 10/10, gmail is getting all my mails in spam. Any ideea?
Thank you for your good job
If you see “Key not secure”, don’t panic. This is because DNSSEC isn’t enabled on your domain name. DNSSEC is a security standard for secure DNS query. Most domain names haven’t enabled DNSSEC.
If your emails get into the spam folder, I have more tips for you in this article: How to stop your emails being marked as spam.
After following your instructions everything was OK. The only part you would need to update is to extend the /etc/postfix/main.cf if one is using SASL:
to
otherwise SMTP clients will fail.
Thank you so much for this comment.
If you have a “reject: RCPT from” in your mail.log when using SMTP this fixes it.
Thank you for this AWESOME tutorial!!!
This is a really good tutorial, one of the best I’ve seen.
I experienced the problem that some of my smtp processes were running in a chroot jail and others were not.
This was easily fixed by using a network instead of file sockets.
/etc/default/opendkim
/etc/postfix/main.cf
I now get 10/10, thanks so much for the tutorial!
Hi Thanks for the tutorial. I have been stuck on query timed out for days now. Tried specifying nameservers, turning off firewall. I search the logs and there is nothing. tail -f /var/log/syslog never shows anything happening when I run opendkim-testkey.
I can find the DKIM record of your domain without any problems, using the
digutility.Yeah, that’s what’s so frustrating. The problem is opendkim-testkey times out.
root@mail:/etc# opendkim-testkey -d mydomain.com -s 20190319 -vvv
opendkim-testkey: using default configfile /etc/opendkim.conf
opendkim-testkey: checking key ‘20190319._domainkey.mydomain.com’
opendkim-testkey: ‘20190319._domainkey.mydomain.com’ query timed out
And with the dig command and in opendkim.conf you can specify the nameserver used. So I used the same dns nameserver for both.
The important thing is that your domain can pass DKIM check and your email server can perform DKIM check on sender’s domain.
Hi,
I get a “warning: connect to private/policyd-spf: No such file or directory”
even though I can open /usr/bin/policyd-spf
my main.cf is check_policy_service unix:private/policyd-spf
and master.cf
policyd-spf unix - n n - 0 spawn user=policyd-spf argv=/usr/bin/policyd-spfCheck if file
/var/spool/postfix/private/policyd-spfexists.the file is not there for me. I get tthsi warning. Ubuntu 18
/usr/bin/policyd-spf is there.
I’m wondering what process is writing /var/spool/postfix/private/policyd-spf becuase the permissions for private are
drwx—— 2 postfix root 4096 Oct 15 15:55 private
where does user policyd-spf come in?
Hi Have setup DLKIM and SPF and DMARC showing pass into Gmail but DKIM showing ‘FAIL’ with domain.
Thanks
Mahendra
Please check the /var/log/mail.log file after sending out an email.
Hi Xiao,
Thank you your reply.
I have checked mail log and found email successfully send to destination email address below i have mention log and google email original message.
Can you please check and help me out this.
May 4 05:24:56 makeabetterday postfix/qmgr[18458]: 0E898100C16: from=, size=420, nrcpt=1 (queue active)
May 4 05:24:56 makeabetterday postfix/smtp[18466]: 0E898100C16: to=, relay=gmail-smtp-in.l.google.com[74.125.195.27]:25, delay=0.48, delays=0.01/0.01/0.13/0.33, dsn=2.0.0, status=sent (250 2.0.0 OK 1556947496 r20si5523976pls.19 – gsmtp)
May 4 05:24:56 makeabetterday postfix/qmgr[18458]: 0E898100C16: removed
SPF: PASS with IP 52.26.165.93
DKIM: ‘FAIL’ with domain makeabetterday.co.uk
DMARC: ‘PASS’
If you can’t find DKIM errors in the mail.log file, check if your DKIM record is entered correctly in DNS.
Awesome tutorial.
I’ve written a self-contained tutorial on SPF/DKIM/DMARC implementation a while ago. Feel free to check it out: https://dmarcly.com/blog/how-to-implement-dmarc-dkim-spf-to-stop-email-spoofing-phishing-the-definitive-guide.
In the /etc/postfix/main.cf, change unix:private/policyd-spf to unix:private/policy-spf
Excellent guide, thank you!
Using this guide as of August 2019 on Ubuntu 18.04 (LTS) I found one issue with the opendkim socket permissions.
I found that changing the UserID in opendkim.conf to opendkim:postfix resolved the issue for me.
Thanks for tutorial.
I did everything on Ubuntu 18.04. And for now I have one error.
The following msg is from verifier.port25.com
———————————————————-
“iprev” check details:
———————————————————-
Result: fail (reverse lookup failed (NXDOMAIN))
ID(s) verified: policy.iprev=
DNS record(s):
x.x.x.x.in-addr.arpa. PTR (NXDOMAIN)
Could you give me some trouble shooting guide or hint?
what host u using?
I found a resolution.
My reverse domain does not be registered.
Hello
Very nice tutorials
Everything works as intended for me. but when am sending to gmail recipient my emals are going to the spam folder even though dkim and spf verification passed
Hi,
I have more tips for getting out of the spam folder in this article: How to stop your emails being marked as spam.
Hi,
Thank you. I gonna try your tips.
I spent some time pondering the DKIM key to be added to the DNS.
A full explanation was not given for this.
The public key comes like:
I was not sure if the second quoted part needs to be included, and if so how.
It needs to be included as the same string, like:
I think I have said in this article that you need to delete all double quotes and white spaces.
Still 5 star rated! As excellent as previous part 1 and part 2. Continue learning next part. Many thanks.
Hi,
Thanks, I am so happy I got my email server to work! But I want it to work even better, and when I test in mail-tester.com I get minus three for “You are not allowed to use one of your sender email addresses”. More specifically, I get these comments (and some more below them):
————————————————————————————————————-
[SPF] nnnn.link does not allow your server 88.888.888.888 to use nnnnnn@nnnn.link
and
What we retained as your current SPF record is:
v=spf1 mx ~allv=spf1 mx ~all
More information about this error:
nnnn.link: Junk encountered in record ‘v=spf1 mx ~allv=spf1 mx ~all’
———————————————————————————————–
Any clues as to what might be the matter?
Your SPF record is incorrect. It should be
v=spf1 mx ~all, notv=spf1 mx ~allv=spf1 mx ~allThank you so much! What a silly mistake!
HI, thank you very much for very well documented tutorial series. Nice work !
I think I found a typo on /etc/postfix/main.cf to allow postfix to connect to Milter service
should be :
Without that I had a warning error on /var/log/mail.log and dkim wasn’t applied.
After playing with Amavis, it turns out that
works with Postfix, Dovecot and OpenKIM. But if Postfix is integrated with Amavis or other software, then we probably need to use
Hi Xiao,
note that Amavis is not installed on my Ubuntu 18.04.03 server working with Postfix 3.3.0 & Opendkim (2.11.0~alpha-11) but I had to remove the starting / from smtp_milters path in order to connect to Milter service.
Ok. I think using
smtpd_milters = local:opendkim/opendkim.sockas default is better.Thank you for this extremely straightforward and simple tutorial. Allow me to add something I learned that wasn’t obvious when first processing through the tutorial:
* If your Postfix is running as a relay for authenticated hosts, you must put the IP addresses of those hosts in the trusted.hosts file
Xiao,
Thank you for your tutorials. I’ve used several of them and they are well written and concise. I really appreciate you taking the time to explain the how’s and why’s of the thing you do instead of just a series of cut and paste commands… it makes it much easier to troubleshoot when you know how the pieces are put together. You my friend are a great teacher.
lol buy her a beer like i did <3
Yikes… I got as far as testing the key and got the following:
opendkim-testkey: tag without value at end of key data
Google searches are finding nothing with opendkim-testkey: “tag without value at end of key data”
any ideas?
I figured it out. I was copying ONLY the information after the P=, and not the v=DKIM, etc.
Hi Team,
First of all, this postfix guide is the best I’ve ever found on the web. Clear, informative and the best thing is that it is workink as described.
So my question is regarding an SPF issue. I’ve implemented the SPF checks and it is working fine. However some domains i.e. amazone.co.uk, hotmail.com, outlook.com are rejected by postfix.
Here are a few logs….
550 5.7.23 : Recipient address rejected: Message rejected due to: SPF fail – not authorized. Please see http://www.openspf.net/Why?s=mfrom;id=20200317125758b8821ba6237a40e0a2b885d2d460p0eu@bounces.amazon.co.uk;ip=10.10.10.251;r=; from= to= proto=ESMTP helo=
550 5.7.23 : Recipient address rejected: Message rejected due to: SPF fail – not authorized. Please see http://www.openspf.net/Why?s=helo;id=eur04-vi1-obe.outbound.protection.outlook.com;ip=10.10.10.251;r=; from= to= proto=ESMTP helo=
Is there a way to whitelist some domains? Or what can be the solution for this?
Thanks in advanced!
10.10.10.251 is a private IP address. Why are you receving Amazon and outlook emails from this address?
My system is a bit complicated. Basically my server is sitting at home. I cannot have a static public IP here. So I’m hosting a MikroTIK EC2 instance and all my email flow is routed through a VPN tunnel. So the 10.10.10.251 is my VPN IP. I need to do like this because of the STATIC IP issue with my ISP.
All other emails are flowing smoothly without any problem at all. i.e. I can receive emails from other domains i.e. yours arriving perfectly.
Now I disabled the service by the time and the hotmail emails started to flow in.
Do you think this is related to the private IP what postfix can see?
Yes. That’s the problem. You should not relay emails from VPN gateway to your mail server. Instead, you should set up a mail proxy so that your mail server can see the SMTP client’s real IP address.
I think I will write about how to set up mail proxy in the following days.
Yes, I was suspecting that as well. I keep you following with that proxy setup. #donationisontheway
Thanks (^-^)
Hi Norbert,
I just published a tutorial for setting up SMTP and IMAP proxy: https://www.linuxbabe.com/mail-server/smtp-imap-proxy-with-haproxy-debian-ubuntu-centos
Ubuntu 18.04-
I followed the tutorial and got no errors, seemed like everything should be fine, but outgoing emails did not have dkim keys included.
After much searching, I discovered, in /var/log/syslog this error:
Again, after much searching I solved the problem with this answer:
https://askubuntu.com/a/1117490/992232
Namely, editing /lib/systemd/system/opendkim.service and removing the line:
Hope this helps someone else.
Oh my god, I had been stuck on that for the past week to no avail. You might be a stranger but I am exceptionally grateful to you for posting this solution. I needed this to work as part of my penultimate year project in uni, and you are such a saviour<3 Thank you x1000
Thanks again – Got through this part too. There were 1 main part I stumbled on here. So I am sharing if other people stumble in the same place.
Putting the DKIM public key into my DNS. (I am with OVH).
Formatting the Output from the:
sudo cat /etc/opendkim/keys/your-domain.com/default.txt command, its output tries to be helpful and splitting it up into 3 small parts. In short, you only really need two.
Trying to follow the author’s instructions, it becomes unhelpful. The author’s instructions, quite rightly, help you compile back up to one TXT string – however for my provider this was too long (An Error)
In short I had to… edit a text version…. of the DNS file and …apply the DKIM over two lines
this is what it looks like:
Line one: default._domainkey IN TXT (“part one of the code in quotes”
Line two: “second part with bracket at end after quote”).
This worked for me.
how on earth did u “edit a text version…. of the DNS file and”?
Thank you for the excellent tutorial! Worked like a charm!
I do have a question in relation to DKIM when it comes to multiple domains… what needs to be changed to accomodate sending and receiving for multiple domains on the same postfix server ?
Any help is greatly appreciated!
You can read the guide here: https://www.linuxbabe.com/mail-server/host-multiple-mail-domains-in-postfixadmin#dkim
Thank you once again! your tutorial was excellent!
Always a good sign if the comment section is longer than the post itself ; ) – excellent article, thanks for that.
A short remark: For AWS Route53 the DNS entry “default._domainkey.your-domain.com must be split into separate strings with a single space in between, eg. “v=DKIM1;k=rsa;” “…MIIBIjAN…” “…qhkiG9w0BA…”.
Great Tutorial, thanks a lot!
I am not sure what this command actually checks for:
I forgot do delete some spaces in the DNS TXT entry and the check with gmail failed. However this command said “Key ok”
I do not think opendkim-genkey generates the correct type of key.
I was seeing DKIM failures with my key on Gmail and other email checking services.
I discovered that opendkim-genkey generates a key that is NOT of type “RSAPublicKey”.
This type is required for DKIM checks to pass.
I was able to generate a proper “RSAPublicKey” DKIM key using the following commands:
# Creating DKIM private key of type RSAPublicKey (aka OpenSSL PEM format)
sudo ssh-keygen -b 2048 -t rsa -m PKCS8 -f ./default.private
# Turning private key into DKIM p= field (public key)
sudo ssh-keygen -y -e -m PKCS8 -f ./default.private
Don’t forget to ‘chown’ the new default.private file:
sudo chown opendkim:opendkim /etc/opendkim/keys//default.private
Set your “P=” data in your DNS DKIM entry to the public key generated by the second command.
This resulted in passing DKIM status on Gmail and other email checkers.
Please let me know if this helps anyone!
Thanks!
I’m not sure this is neccersary :/ but im not sure i had major dkim problems and almost killed a village of inocents in my rage but now its working fine for god knows what reason
Good day, what if for some reason you wanted a setup where your outgoing messages were DKIM-signed, like what seems to be happening here, but you also wanted to:
1) Accept messages from domains not using DKIM
and
2) If the domain is using DKIM, and the key check fails, don’t deliver the message
Where are such policies specified?
1) OpenDKIM won’t reject emails not signed with DKIM.
2.) It’s not ideal to reject emails just because the DKIM check fails. DKIM is more commonly used in conjunction with SPF to form a DMARC policy, which you can use to reject emails that fail DMARC verification.
To understand how DMARC works, read this article: Creating DMARC Record to Protect Your Domain Name From Email Spoofing
To reject emails that fail DMARC verification, read this article: Set Up OpenDMARC with Postfix on Ubuntu to Block Email Spoofing/Spam
Thanks very much for the clarification. I had it conceptualized as all three having commonly used implementations for either direction, that each had separate implementations. But DKIM appears to be a one-stop-shop. Also, perhaps the fact that SPF has policy led me to expect that DKIM had its own in-protocol policy component as well. I did go ahead and set up DMARC for incoming, following your latter article.
SPF: PASS with IP Learn more
DKIM: ‘FAIL’ with domain mydomain.com Learn more
DMARC: ‘PASS’ Learn more
Still getting DKIM failed from gmail…
Hi Sameer,
Check out my comment from a couple weeks ago.
You might need to generate different DKIM keys (public and private) using the commands I mentioned.
This fixed my errors.
opendkim-genkey seems to not generate the proper key.
Hello everyone,
I’ve set up DKIM and passed with ‘sudo opendkim-testkey -d your-domain.com -s default -vvv’ but everytime I send an email there is no DKIM signature, it says it’s missing, is there something else I should be doing?
This is really odd. I installed this awhile ago, and I don’t recall changing anything. I didn’t have this problem, before. I have spamassassin installed, as well – I read, somewhere that, for reasons I didn’t understand, this could cause the problem I am seeing. For some reason, when I check my mail server on this site you suggested it now deducts two points from my score because it says I have two DKIM records. I have no idea why this would be happening. If you have any thoughts, I’d love to hear them.
Another awesome tutorial in the bag! Like the first three chapters, it worked like you’d tested it extensively 😉
While I know you already know this…. I’ve been through the DKIM cat & cut & paste exercise a few times before. If there is any suggestion I could offer, it is this:
Make sure to point out in simple terms (i.e. “break it down Romper Room style”) that… when you issue the ‘sudo cat /etc/opendkim/keys//default.txt’ command… it’s a good idea to copy and paste it into a text file, maximize that text document, and remove all the quotation marks.
When the key is longer than a certain number of characters (2048?) it is spit out as two separate long blocks of character strings. You’ve got to remove the quotes, and drop it as a single block of text into your TXT entry.
You *did* mention it in passing. But some folks are going to miss it, and either put two quoted blocks in the content box at their DNS provider… OR… they’ll try and make two entries. (ask me how I know 😉 )
Great work! On to chapter 5.
Hello everyone,
I have created DKIM at SMTP server and its getting passed when tested from local server.
Now we are are using same SMTP configurations for a server and sender is allowed at SMTP. Its saying DKIM=none.
Sender address is abc@xyz.your-domain.com. Is it because we used top level domain(your-domain) to create DKIM and now sender at server is @xyz.your-domain.com?
Yes. You need to create separate DKIM keys for sub-domains, basically treating them as different domains.
To set up OpenDKIM for multiple domains, you can read the guide here: https://www.linuxbabe.com/mail-server/host-multiple-mail-domains-in-postfixadmin#dkim
THANK YOU FOR THIS! I have been running my own Linux server for years and recently (past few years) email has been a real problem with delivery to other sites. The last straw when Gmail started blocking my server. I finally got around to trying this and got it all working! You made it easy to copy/paste and just get on with my life and not have to become an SME on all these things.
Hi!
I got the key, seems to be there. But when testing it, it says: key not secure. Someone told me that: ” It is an expected consequence of not using DNSSSEC”:
Is this normal? Am I missing something?
THANKS
Yes, the key is not secure when your domain name is not using DNSSEC. You don’t have to worry about it.
Thank you Xiao. 🤓👍
are we meant to add both the obscure strings to the DNS record (Publish Your Public Key in DNS Records section) ?
v=DKIM1; k=rsa; p=obscureString1 obscureString2
i ask cause im getting “key not secure” maybe this is due to propagation delay or maybe its a misunderstanding regarding the fact that there’s 2 obscure strings or maybe its due to that fact that im using ubuntu 16.04 (would that be a problem?)
maybe “Nolan Tenpas” has my solution
after perusing every single comment on this thread i am a wiser man.
the problem is that im not getting a DKIM pass from gmail.
It might be a propagation issue :/ (as suggested by mail-tester.com)
As I have said in this article, when you test the DKIM key, if you see “Key OK” in the command output, your DKIM record is correct.
If you see “Key not secure” in the command output, this is because DNSSEC isn’t enabled on your domain name. There’s absolutely no need to worry about “Key not secure”.
DKIM is the bane of my existance but for some reason its working now so thanks the lordy lord
and thank you dear! the angel of my existence!
I’ve followed your previous tutorials for setting up Postfix and Dovecot which have work perfectly. I’ve done the SPF section in this article which I have working properly. I’ve followed the DKIM instructions, opendkim and postfix are running without errors. When I send an email to any of the test sites I always see DKIM check: none.
Running on Ubunto 18.
At a loss, looked through all logs for any errors. Not seeing any information as to what’s happening. Even looked for ways to debug what’s happening, with no luck. Any ideas on what to even look for?
Add the following lines in
/etc/opendkim.conffile to produce more verbose logs.Save and close the file. Restart OpenDKIM.
Now OpenDKIM will produce more verbose logs in
/var/log/mail.logfile.I’m having the same issue. I’ve got Ubuntu 20.04. I’ve followed everything, checked it multiple times, added the LogWhy, and still no errors indicated but still DKIM=none and the the checker site says not signing. opendkim is running.
At a loss, too.
So, after some exchange with Xiao, I ended up deleting the socket and the directory. Then I recreated them. Turns out I got my lines confuses and improperly labeled the socket. Once I deleted the directory and did it all again, that is check the configs and rebuilt the socket, worked perfectly.
Thanks!
One more thing…I found in my efforts to troubleshoot this that the key.table shouldn’t have refile: at the start as the guide suggests. It has to do with how the domains are listed. When they are all on one line, you don’t use the refile. the signing.table file, since it has the wildcard, does require refile. So it would look like this:
Thank you for the great tutorials you are giving us with super details 🙂
I have a question: The steps above about DKIM. Do I have to repeat it for every domain that I use in postfix admin or just the domain of the mail server?
I mean something like this:
Yes. You can find more details on this page: https://www.linuxbabe.com/mail-server/host-multiple-mail-domains-in-postfixadmin
Great article! I ran into problems when I generated the 2048-bit default._domainkey, since my domain registrar did not allow for such long TXT records. It took me a long time to figure this out. However, after generating a 1024-bit key everything worked.
Hi
I have followed this guide to send and receive email but I have 2 problems
1. When sending an email to gmail -> I get a red icon saying – the sender did not encrypt this message
2. When receiving email, I get the mail when I type the mail command in the linux box. But I am not receiving it in outlook (I am sending and receiving) from outlook.
Problem 1 is more important as I want that users to see a green icon – instead of red.
Thanks,
Have you followed part 2 to enable TLS in Postfix and Dovecot? Add the following two lines in
/etc/postfix/main.cffile to enable TLS when Postfix sends outgoing emails.To enable encryption when Postfix receives emails, add the following two lines.
Restart Postfix
Check your mail log (/var/log/mail.log) to debug outlook issues.
I tested with Mail-tester.com and got a score of 1.2 and a saying that my mail nevers sees a mailbox: https://www.mail-tester.com/test-r31d18hwi&reloaded=1. “SpamAssassin” scores -5.8 and identification -3. The mail still landed in Gmail’s inbox not junk folder for the first time.
my DMARK reports I receive all PASS the DKIM and SPF tests–except for the report from att.net
Same ip address, same domain–always fails the DKIM check. Have you ever encountered that?
google, yahoo, verizon all pass.
—-
Excellent guides by the way 🙂
Hello!
About SPF options:
Never use: ~all – > it softfail and does nothing.
Use: -all -> Block/DROP if someone wants to f..k with UR domain
Actually, it’s better to use DMARC to block forged emails. Use
-allin your SPF policy can cause your own emails to be rejected.I have an error. This is my default._domainkey. What’s wrong Xiao?
Maybe you need to wait some time for your DKIM record to propagate to the Internet.
PERFECT!
I followed all your guides from DMARC and SPF / DKIM and its the best linux tutorials I ever seen… They are extremely easy to follow and I will buy you a beer for it! THANKS!
Two recommendations…
1. For the SPF DNS entries, if you use relay servers which my ISP require, the DNS, record you need to add those servers to the TXT record.
2. In the SPF guide, you write simply to add a new “smtpd_recipient_restrictions =” at the end of the main.conf file. If you already have this in your main, I got warnings “warning: /etc/postfix/main.cf, line 123: overriding earlier entry: smtpd_recipient_restrictions=” in my logs until i merged the entries…
Anyway, thanks a million!
Hi xaoi. first off your guides are some of the best ive ever seen and mostly work without a hitch.
But after i followed this guide i got it to work flawlessly with mail-tester on my main domain but when i duplicate it for the files for key/signing and hosts my first domain remains working but when i try to do it with my second the email doesnt even get sent.
i generated the keys for both domains. updated the dns(also using the bind9 guide you provided) to include the TXT record then used this link https://dkimcore.org/c/keycheck to verify the dkim key is updated to the current one.
Then finally when im done and i run my phpmailer script -> https://pastebin.com/fXuaLwQH in SSH and while my first domain(pihosting.ca) works great my second domain (truebud.ca) doesnt even send email. I tested it with mail-tester and it never got it.
What did i miss?
Always check your mail log (/var/log/mail.log) and Dovecot log (sudo journalctl -eu dovecot) to troubleshoot mail server problems.
Hi,
This is a great article and is extremely help full.
I managed to configure the sever as per the instruction and the DMARC ones as well.
On final testing all seems good on the google original email but I do see an error like this :
Dio you know how to fix this?
My server is Ubuntu 20.04 with postfix/devcoat/spamassing and i alos have install iredmail admin.
If you installed iRedMail, you don’t need to follow this SPF/DKIM tutorial.
The error is probably because you use an email address that hasn’t been created in iRedAdmin.
Hi,
Thank you for the quick reply.
All email accounts are created from within the iRedMail admin web page.
I did this for a learning, and it helped.
A side question do you know I can configure the the mail to act as a relay ONLY for all my all my internal servers, internal networks, I use about 4 different IP ranges.
Currently each ubuntu server is acting as it’s own mail releay, this does work, but I would prefere to have one relay server, rather then 4.
Loving this whole series. This is my first time setting up a personal email server and this series has walked me through it. It is designed for the simple person that has a small knowledge of computers. There are a few things I have had to change for my setup but worked through them. The biggest issue I have faced so far was getting DKIM to pass. Working on it across two days I finally ended up having to regenerate my key and come to find out my original attempt had gotten messed up somehow. Now it works beautifully.
Thanks for this article.
Very nice and easy to follow.
Great job.
Cheers!
I am not able to receive mails from otherwise it working fine. port 25 is blocked on my machine so i have smtp relay setup, and so the dkim and spf records.
So do i need to follow this tutorial. and why am i not retrieving any mails
Possible causes of not receiving email
1. Your MX record is wrong.
2. Your mail server hostname doesn’t have DNS A record.
3. Your firewall doesn’t allow incoming connection to port 25.
4. Postfix isn’t listening on the public IP address.
5. Check the mail log (/var/log/mail.log) and Dovecot log (sudo journalctl -eu dovecot) to find out if there are other errors in your configuration
SMTP relay service will only add DKIM signature to your outgoing emails. They can not verifiy DKIM signatures of incoming emails. So you still need to follow this tutorial to set up SPF and OpenDKIM.
Hi. Very nice tutorial.
When I send email by mail command to my gmail account I recive email from user@host.mydomain.com but if I use sendmail it is user@mydomain.com. Why so?
Hi!
This is one of Internet’s best tutorials! Thanks!
I have done everything, double checked and tested. Everything improves and I get a 9/10 score from mail-tester.com. BUT it marks DKIM as not configured. But the tests made locally seems fine, then meaning that opendkim-testkey seems fine.
My test through mail tester.com says “Your mail is not signed with DKIM”. SPF is just fina and passes with distinction 😉
Any ideas?
Kind regards
Michael