Set Up DNS over HTTPS (DoH) Resolver on Debian with DNSdist

This tutorial will be showing you how to set up your own DNS over HTTPS (DoH) resolver on Debian with DNSdist, so your DNS queries can be encrypted and protected from prying eyes.

What is DNS over HTTPS and Why It’s Important

DNS (Domain Name System) is responsible for translating domain names to IP addresses. It’s designed in 1987 with no security or privacy in mind. By default, DNS queries are not encrypted. They are sent in plain text on the wire and can be exploited by middle entities. For example, the Great Firewall (GFW) of China uses a technique called DNS cache poison to censor the Chinese Internet. (They also use other methods, which are beyond the scope of this article.)

GFW checks every DNS query sent to a DNS server outside of China. Since plain text DNS protocol is based on UDP, which is a connection-less protocol, GFW can spoof both the client IP and server IP. When GFW finds a domain name on its block list, it changes the DNS response. For instance, if a Chinese Internet user wants to visit google.com, GFW returns an IP address located in China instead of Google’s real IP address, to the user’s DNS resolver. Then the DNS resolver returns the bogus IP address to the user’s computer, so the user cannot visit google.com.

HTTPS is the standard way to encrypt plain text HTTP connections. With DNS over HTTPS (DoH), your DNS queries will be encrypted and no third party can see your DNS query.

Why Run Your Own DoH Resolver?

There are already some public DNS resolvers like 1.1.1.1 and 9.9.9.9 that support DNS over HTTPS, so you can use them if you don’t have the skill or time to run your own. Starting with Firefox version 61, you can enable DNS over HTTPS in the browser settings, which is big progress for Internet security and privacy. Firefox uses Cloudflare resolvers (1.1.1.1) by default. However, some folks argue that this allows Cloudflare to gather information on Firefox users. They seem to have more trust in their ISP than Cloudflare. I think if you are paranoid about privacy, you should run your own DoH resolver, so neither Cloudflare nor your ISP can spy on you.

DoH vs DoT

There’s another protocol that also aims to encrypt DNS queries. It’s called DNS over TLS (DoT). What are the advantages of using DNS over HTTPS?

For people living in countries with severe Internet censorship like China, it’s more advantageous to use DoH.

• DoT operates on TCP port 853, which can be easily blocked by a national firewall.
• DoH operates on TCP port 443, which is the standard port for HTTPS websites, which makes DoH super hard to block, because if TCP port 443 is blocked, then nearly all HTTPS websites will also be blocked.

My DoH resolver is running on a VPS (virtual private server) located outside of China and the Great Firewall of China can’t intercept my DNS queries. I can even hide my DoH resolver’s IP address behind Cloudflare CDN (Content Delivery Network). My VPS also runs my VPN server.

Another advantage of DoH is that it allows web applications to access DNS information via existing browser APIs, so no stub resolver is needed.

DoH Support in Major DNS Resolvers

• BIND will support DoH in version 9.17, which is still in development. Debian 10 ships with BIND 9.11.
• Knot resolver supports DoH since version 4.0.0. The current latest version is 5.3.2. It has an official repository for Debian, Debian, CentOS, Fedora.
• Unbound supports DoH since version 1.12.0.
• PowerDNS recursor doesn’t support DoH right now.

Actually, I prefer to run DoH resolver with DNSdist, which added DoH support in version 1.4. The current latest version is 1.6. It has an official repository for Debian, Raspbian, Debian, and CentOS. DNSdist is a DNS load balancer that can forward DNS queries to a backend DNS resolver, so no matter what DNS resolver you are using, you can use DNSdist to run your own DoH server. DNSdist is developed by the PowerDNS team.

Requirements

It’s assumed that you have a DNS resolver running on your Debian server. You can use any DNS resolver (BIND, Knot resolver, Unbound…) I personally use BIND.

• Set Up Your Own BIND9 DNS Resolver on Debian 10 Buster

Once your DNS resolver is up and running, follow the instructions below.

Step 1: Install DNSdist on Debian Server

It’s recommended to install DNSdist from the upstream repository, so you will have the latest stable version. First, you need to create a source list file for DNSdist.

Debian 10

echo "deb [arch=amd64] http://repo.powerdns.com/debian buster-dnsdist-16 main" | sudo tee /etc/apt/sources.list.d/pdns.list

Debian 9

echo "deb [arch=amd64] http://repo.powerdns.com/debian stretch-dnsdist-15 main" | sudo tee /etc/apt/sources.list.d/pdns.list

Next, we create a preference file for DNSdist to pin the package, so we won’t be accidentally installing DNSdist from another repository.

sudo nano /etc/apt/preferences.d/dnsdist

Add the following lines into the file.

Package: dnsdist*
Pin: origin repo.powerdns.com
Pin-Priority: 600

Save and close the file. Then run the following command to import the PowerDNS public key, so the APT package manager can verify the interity of software packages downloaded from this repository.

curl https://repo.powerdns.com/FD380FBB-pub.asc | sudo apt-key add -

Next, update the repository list and install DNSdist.

sudo apt update

sudo apt install dnsdist

By default, DNSdist tries to bind to port 53. Because you have an existing DNS resolver like BIND listening on port 53, the dnsdist.service would fail to start.

Since we are just deploying a DoH resolver and don’t care about DNS load balancing, we can configure DNSdist to listen on another port. Edit the DNSdist configuration file.

sudo nano /etc/dnsdist/dnsdist.conf

There’s no content in this file. For now, simply add the following line in this file, so DNSdist will listen on TCP and UDP port 5353, instead of port 53.

setLocal("127.0.0.1:5353")

Save and close the file. Then restart DNSdist.

sudo systemctl restart dnsdist

Check its status.

systemctl status dnsdist

It should be active (running).

Step 2: Install Let’s Encrypt Client (Certbot) on Debian Server

DNS over HTTPS requires installing a TLS certificate on the server-side. We will obtain and install Let’s Encrypt certificate. The advantage of using Let’s Encrypt certificate is that it’s free, easier to set up, and trusted by client software.

Run the following commands to install Let’s Encrypt client (certbot) from the default Debian repository.

sudo apt install certbot

To check the version number, run

certbot --version

Sample output:

certbot 0.31.0

Step 3: Obtain a Trusted TLS Certificate from Let’s Encrypt

I recommend using the standalone or webroot plugin to obtain TLS certificate for dnsdist.

Standalone Plugin

If there’s no web server running on your Debian server, you can use the standalone plugin to obtain TLS certificate from Let’s Encrypt. Create DNS A record for the subdomain (doh.example.com), then run the following command.

sudo certbot certonly --standalone --preferred-challenges http --agree-tos --email [email protected] -d doh.example.com

Where:

• certonly: Obtain a certificate but don’t install it.
• --standalone: Use the standalone plugin to obtain a certificate
• --preferred-challenges http: Perform http-01 challenge to validate our domain, which will use port 80.
• --agree-tos: Agree to Let’s Encrypt terms of service.
• --email: Email address is used for account registration and recovery.
• -d: Specify your domain name.

Using webroot Plugin

If your Debian server has a web server listening on port 80 and 443, then it’s a good idea to use the webroot plugin to obtain a certificate because the webroot plugin works with pretty much every web server and we don’t need to install the certificate in the web server.

First, you need to create a virtual host for doh.example.com.

Apache

If you are using Apache, then

sudo nano /etc/apache2/sites-available/doh.example.com.conf

And paste the following lines into the file.

<VirtualHost *:80>        
        ServerName doh.example.com

        DocumentRoot /var/www/dnsdist
</VirtualHost>

Save and close the file. Then create the web root directory.

sudo mkdir /var/www/dnsdist

Set www-data (Apache user) as the owner of the web root.

sudo chown www-data:www-data /var/www/dnsdist -R

Enable this virtual host.

sudo a2ensite doh.example.com

Reload Apache for the changes to take effect.

sudo systemctl reload apache2

Once virtual host is created and enabled, run the following command to obtain Let’s Encrypt certificate using webroot plugin.

sudo certbot certonly --webroot --agree-tos --email [email protected] -d doh.example.com -w /var/www/dnsdist

Nginx

If you are using Nginx, then

sudo nano /etc/nginx/conf.d/doh.example.com.conf

Paste the following lines into the file.

server {
      listen 80;
      server_name doh.example.com;

      root /var/www/dnsdist/;

      location ~ /.well-known/acme-challenge {
         allow all;
      }
}

Save and close the file. Then create the web root directory.

sudo mkdir -p /var/www/dnsdist

Set www-data (Nginx user) as the owner of the web root.

sudo chown www-data:www-data /var/www/dnsdist -R

Reload Nginx for the changes to take effect.

sudo systemctl reload nginx

Once virtual host is created and enabled, run the following command to obtain Let’s Encrypt certificate using webroot plugin.

sudo certbot certonly --webroot --agree-tos --email [email protected] -d doh.example.com -w /var/www/dnsdist

Step 4: Enable DoH in DNSdist

Edit the DNSdist configuration file.

sudo nano /etc/dnsdist/dnsdist.conf

Add the following lines into the file.

-- allow query from all IP addresses
addACL('0.0.0.0/0')

-- add a DoH resolver listening on port 443 of all interfaces
addDOHLocal("0.0.0.0:443", "/etc/letsencrypt/live/doh.example.com/fullchain.pem", "/etc/letsencrypt/live/doh.example.com/privkey.pem", { "/" }, { doTCP=true, reusePort=true, tcpFastOpenSize=0 })

-- downstream resolver
newServer({address="127.0.0.1:53",qps=5, name="resolver1"})

Save and close the file. DNSdist runs as the _dnsdist user, so we need to give the _dnsdist user permission to read the TLS certificate with the following commands.

sudo apt install acl

sudo setfacl -R -m u:_dnsdist:rx /etc/letsencrypt/

Then check the syntax of the configuration file.

sudo dnsdist --check-config

If the syntax is ok, restart DNSdist.

sudo systemctl restart dnsdist

Note that if there’s a web server listening on TCP port 443, DNSdist would fail to restart. You can temporarily stop the web server. I will explain how to make the web server and DNSdist use TCP port 443 at the same time at the end of this article.

When Certbot renews the TLS certificate, you need to grant permission again, but you can add the command in the crontab file.

sudo crontab -e

Like this:

@daily certbot renew --quiet; setfacl -R -m u:_dnsdist:rx /etc/letsencrypt/

Step 5: Configure DoH in Firefox Web Browser

Go to Preferences -> General and scroll down to the bottom to configure Network Settings. Enable DNS over HTTPS, and set your own DoH resolver.

We can then fine-tune the DoH configurations by going to about:config tab in Firefox.

network.trr.mode

By default, The network.trr.mode parameter in Firefox is set to 2, which means if DoH query fails, Firefox will pass the DNS query to the host system. I want to always use the DoH resolver, so change the network.trr.mode to 3 so the host resolver won’t be used. This allows us to have an easy way to test if your DoH resolver is working.

network.trr.allow-rfc1918

This is set to false by default, which means if the DNS response includes private IP addresses, then it will be considered a wrong reponse that won’t be used. If you use response policy zone in BIND, you probably have some hostnames pointing to private IP addresses, then set this value to true.

network.trr.bootstrapAddress

Firefox needs to look up the IP address of the DoH resolver in order to send DNS queries. You can put the IP address in this field to eliminate this initial query. I also recommend adding a local DNS entry in the hosts file, so Firefox can find the IP address without looking up the public DNS.

Testing

Now enter a domain name like linuxbabe.com in the Firefox address bar. If the web page loads normally, it’s a good sign your DoH resolver is working. Then go to the terminal console of your DNS server and check the DNS query logs. I use BIND, so I enter the following command to check the DNS query log.

sudo journalctl -eu bind9

As you can see from the BIND log below, Firefox queried the following domains.

• www.linuxbabe.com: my website
• fonts.gstatic.com: This serves Google fonts on my website
• cdn.shareaholic.net: a sharing widget on my website
• newsletter.linuxbabe.com: my self-hosted email marketing platform.
• translate.google.com: the Google translate widget on my website

The above query log tells me that my DNS over HTTPS resolver is working. If I stop the BIND resolver (sudo systemctl stop named), Firefox tells me it can’t find that site. And when I start BIND, the web page loads again.

Step 6: Configure DoH Client on Debian Desktop

I know there are 3 open-source DNS clients on Linux that support encrypted DNS.

• systemd-resolved
• stubby
• dnscrypt-proxy

Currently, dnscrypt-proxy(version 2.x) is the best DoH client implementation on Linux. It also runs on BSD, macOS, Windows and Android. so I will show you how to use it.

Run the following command to install dnscrypt-proxy on your Debian desktop.

sudo apt install dnscrypt-proxy

It will automatically start, as you can see with:

sudo systemctl status dnscrypt-proxy

If it didn’t start, you can run the following command to start it.

sudo systemctl enable dnscrypt-proxy --now

Check the listening address.

sudo ss -lnptu | grep dnscrypt-proxy

On my computer, dnscrypt-proxy listens on 127.0.2.1:53. There’s another systemd service (dnscrypt-proxy-resolvconf.service) installed by dnscrypt-proxy,

systemctl status dnscrypt-proxy-resolvconf

This is a service that sets 127.0.2.1 as the resolver on your computer. You can check the current DNS resolver for your computer with:

cat /etc/resolv.conf

By default, dnscrypt-proxy uses Clouldflare as the upstream DoH resolver. To use your own DoH resolver, edit the configuration file.

sudo nano /etc/dnscrypt-proxy/dnscrypt-proxy.toml

Find the following line

server_names = ['cloudflare']

Change it to your DoH resolver hostname. Note that the location of parameters is important. You should not put the server_names parameter in other places in this file.

server_names = ['doh.example.com']

Then comment out the [sources] section.

#[sources]
#  [sources.'public-resolvers']
#  url = 'https://download.dnscrypt.info/resolvers-list/v2/public-resolvers.md'
#  cache_file = '/var/cache/dnscrypt-proxy/public-resolvers.md'
#  minisign_key = 'RWQf6LRCGA9i53mlYecO4IzT51TGPpvWucNSCh1CBM0QTaLn73Y7GFO3'
#  refresh_delay = 72
#  prefix = ''

We need to add our own DoH resolver like below at the bottom of this file.

[static]  
  [static.'doh.example.com']
  stamp = 'sdns://AgUAAAAAAAAAAAAOZG5zLmdvb2dsZS5jb20NL2V4cGVyaW1lbnRhbA'

The stamp contains your DoH resolver hostname, IP address and query path. To get your own stamp, use the online DNS stamp calculator. Choose DNS-over-HTTPS as the protocol, and enter the IP address, hostname and query path. If you follow this tutorial to set up your own DoH resolver, the path should be set to just /. If you didn’t enable DNSSEC on your resolver, then untick the DNSSEC checkbox.

Once you added your DNS stamp, save and close the file. Then restart dnscrypt-proxy.

sudo systemctl restart dnscrypt-proxy

Check its status. Make sure it’s running.

systemctl status dnscrypt-proxy

Now you can test if dnscrypt-proxy is working.

If you don’t want to publish your DNS over HTTPS resolver to the world, you can delete the DNS A record of your DoH resolver hostname, because dnscrypt-proxy has the IP address in the DNS stamp.

Configure DoH on iOS

iOS supports DNS over HTTPS since iOS 14. However, there’s no a simple config option on iOS to set DoH resolver. You need to generate a .mobileconfig file and install it on iOS. It’s a little complicated to generate a signed iOS .mobileconfig file.

If your DoH resolver won’t be used publicly, you can generate a simple unsigned file. Use a text editor to create a file with .mobileconfig extension. Copy and paste the following text. Replace the IP ddress and server URL.

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
        <key>PayloadContent</key>
        <array>
                <dict>
                        <key>DNSSettings</key>
                        <dict>
                                <key>DNSProtocol</key>
                                <string>HTTPS</string>
                                <key>ServerAddresses</key>
                                <array>
                                        <string>12.34.56.78</string>
                                </array>
                                <key>ServerURL</key>
                                <string>https://doh.example.com/</string>
                        </dict>
                        <key>PayloadDescription</key>
                        <string>Configures device to use Encrypted DNS over HTTPS</string>
                        <key>PayloadDisplayName</key>
                        <string>My own DNS over HTTPS Resolver</string>
                        <key>PayloadIdentifier</key>
                        <string>com.apple.dnsSettings.managed.9d6e5fdf-e404-4f34-ae94-27ed2f636ac4</string>
                        <key>PayloadType</key>
                        <string>com.apple.dnsSettings.managed</string>
                        <key>PayloadUUID</key>
                        <string>35d5c8a0-afa6-4b36-a9fe-099a997b44ad</string>
                        <key>PayloadVersion</key>
                        <integer>1</integer>
                        <key>ProhibitDisablement</key>
                        <false/>
                </dict>
        </array>
        <key>PayloadDescription</key>
        <string>Adds self-hosted DoH resolver to Big Sur and iOS 14 based systems</string>
        <key>PayloadDisplayName</key>
        <string>My own DNS over HTTPS</string>
        <key>PayloadIdentifier</key>
        <string>com.example.doh</string>
        <key>PayloadRemovalDisallowed</key>
        <false/>
        <key>PayloadType</key>
        <string>Configuration</string>
        <key>PayloadUUID</key>
        <string>A4475135-633A-4F15-A79B-BE15093DC97A</string>
        <key>PayloadVersion</key>
        <integer>1</integer>
</dict>
</plist>

Save and close the file. Then you can send the file as an email attachment to your iOS. Open the email on iOS and tap on the attachment to download the config file. After that, go to iOS Settings -> Profile Downloaded. It will prompt you to install the profile.

Tap on the Install button and enter your iOS password to install it. Tap on Install button again.

Once it’s installed, go to iOS Settings -> VPN & Network. Scroll down to the buttom to choose the DNS resolver for your iOS system. Note that this DoH resolver will override the DNS resolver provided by a VPN like ProtonVPN. Your iOS will always use your DoH resolver.

If you ever want to remove the .mobileconfig file from iOS, go to Settings -> General -> Profile.

Make DNSdist and web server use port 443 at the same time

A DNS over HTTPS resolver needs to bind to port 443. If you already have Apache/Nginx listening on port 443, then DNSdist can’t bind to port 443. Normally a port can only be used by one process. However, we can use HAproxy (High Availability Proxy) and SNI (Server Name Indication) to make DNSdist and Apache/Nginx use port 443 at the same time.

DNSdist Configuration

Edit the DNSdist configuration file.

sudo nano /etc/dnsdist/dnsdist.conf

Change the DoH listening address to 127.0.0.1.

addDOHLocal("127.0.0.1:443", "/etc/letsencrypt/live/doh.example.com/fullchain.pem", "/etc/letsencrypt/live/doh.example.com/privkey.pem", { "/" }, { doTCP=true, reusePort=true, tcpFastOpenSize=0 })

Save and close the file. Then restart DNSdist.

sudo systemctl restart dnsdist

Nginx Configuration

If you use Nginx, edit the server block file.

sudo nano /etc/nginx/conf.d/example.com.conf

In the SSL server block, find the following directive.

listen 443 ssl;

Change it to

listen 127.0.0.2:443 ssl;

This time we make it listen on 127.0.0.2:443 because 127.0.0.1:443 is already taken by DNSdist. Save and close the file. The Nginx main configuration file /etc/nginx/nginx.conf and the default server block /etc/nginx/sites-enabled/default might include a default virtual host listening on 443, so you might need to edit this file too.

Then restart Nginx.

sudo systemctl restart nginx

Apache Configuration

If you use Apache web server, edit your virtual host file.

sudo nano /etc/apache2/sites-enabled/example.com.conf

In the SSL virtual host, change

<VirtualHost *:443>

To

<VirtualHost 127.0.0.2:443>

This time we make it listen on 127.0.0.2:443 because 127.0.0.1:443 is already taken by DNSdist. Save and close the file. Then edit the /etc/apache2/ports.conf file.

sudo nano /etc/apache2/ports.conf

Change

Listen 443

To

Listen 127.0.0.2:443

Save and close the file. Restart Apache.

sudo systemctl restart apache2

HAProxy Configuration

Now install HAproxy.

sudo apt install haproxy

Start HAProxy

sudo systemctl start haproxy

Edit configuration file.

sudo nano /etc/haproxy/haproxy.cfg

If you use Nginx, copy and paste the following lines to the end of the file. Replace 12.34.56.78 with the public IP address of your server. Replace doh.example.com with the domain name used by DNSdist and www.example.com with the domain name used by your web server.

frontend https
   bind 12.34.56.78:443
   mode tcp
   tcp-request inspect-delay 5s
   tcp-request content accept if { req_ssl_hello_type 1 }

   use_backend dnsdist if { req_ssl_sni -i doh.example.com }
   use_backend nginx if { req_ssl_sni -i www.example.com }
   use_backend nginx if { req_ssl_sni -i example.com }

   default_backend dnsdist

backend dnsdist
   mode tcp
   option ssl-hello-chk
   server dnsdist 127.0.0.1:443

backend nginx
   mode tcp
   option ssl-hello-chk
   server nginx 127.0.0.2:443 check

If you use Apache, copy and paste the following lines to the end of the file. Replace 12.34.56.78 with the public IP address of your server. Replace doh.example.com with the domain name used by DNSdist and www.example.com with the domain name used by your web server.

frontend https
   bind 12.34.56.78:443
   mode tcp
   tcp-request inspect-delay 5s
   tcp-request content accept if { req_ssl_hello_type 1 }

   use_backend dnsdist if { req_ssl_sni -i doh.example.com }
   use_backend apache if { req_ssl_sni -i www.example.com }
   use_backend apache if { req_ssl_sni -i example.com }

   default_backend dnsdist

backend dnsdist
   mode tcp
   option ssl-hello-chk
   server dnsdist 127.0.0.1:443

backend apache
    mode tcp
    option ssl-hello-chk
    server apache 127.0.0.2:443 check

Save and close the file. Then restart HAproxy.

sudo systemctl restart haproxy

In the configuration above, we utilized the SNI (Server Name Indication) feature in TLS to differentiate VPN traffic and normal HTTPS traffic.

• When doh.example.com is in the TLS Client Hello, HAProxy redirect traffic to the DNSdist backend.
• When www.example.com is in the TLS Client Hello, HAProxy redirect traffic to the apache/nginx backend.
• If the client doesn’t specify the server name in TLS Client Hello, then HAproxy will use the default backend (DNSdist).

You can test this setup with the openssl tool. First, run the following command multiple times.

echo | openssl s_client -connect your-server-IP:443 | grep subject

We didn’t specify server name in the above command, so HAproxy will always pass the request to the default backend (DNSdist), and its certificate will be sent to the client. Next, run the following two commands.

echo | openssl s_client -servername www.example.com -connect your-server-IP:443 | grep subject

echo | openssl s_client -servername doh.example.com -connect your-server-IP:443 | grep subject

Now we specified the server name in the commands, so HAproxy will pass requests according to the SNI rules we defined.

When renewing Let’s Encrypt certificate for your website, it’s recommended that you use the http-01 challenge instead of tls-alpn-01 challenge, because HAproxy is listening on port 443 of the public IP address, so it can interfere with the renewal process.

sudo certbot renew --preferred-challenges http-01

Set Up Your Own VPN Server

To bypass national firewalls and unblock Google, Facebook, Twitter and other websites, I recommend using the HTTPS-based VPN protocol: OpenConnect VPN.

• Set Up OpenConnect VPN Server (ocserv) on Debian 11 with Let’s Encrypt

HTTPS handshake

Want to know how the server and client establish a secure HTTPS connection? Read the article below to learn about HTTPS handshake.

• SSL/TLS Handshake Explained With Wireshark Screenshot

Wrapping Up

I hope this tutorial helped you set up a DNS over HTTPS resolver with Nginx on Debian. As always, if you found this post useful, then subscribe to our free newsletter to get more tips and tricks. Take care 🙂