Run Apache, Nginx & HAProxy on Same Server (Debian, Ubuntu, CentOS)

If you’re a server admin, you probably have a web server of your choice like Apache, or Nginx. Apache is a well-known web server since the 1990s. Nginx was first developed in 2004 and quickly gained traction due to its lightweight memory footprint and fast processing speed for static HTML files.

Both Apache and Nginx support virtual hosting, which means you can host multiple websites or web applications on the same server. However, you will encounter situations in which you have an existing web server running, but a particular web application requires using a different web server. Port 80 or 443 on the public IP address can be used by only one process. If Apache is using the port, then Nginx can’t use (or bind to) it. So what can you do?

You can configure Nginx as a reverse proxy to Apache, so Nginx can redirect HTTP requests to Apache. In my experience, I found that this isn’t always the best way because it has once caused weird problems that I cannot troubleshoot. Instead, I prefer to use HAProxy as a reverse proxy for both Nginx and Apache. HAProxy is a free, open-source high availability load balancer and proxy server for TCP and HTTP-based applications.

Run Apache, Nginx and HAProxy on the Same Server

Here’s how it works.

  • Nginx listens on 127.0.0.1:80 and 127.0.0.1:443
  • Apache listens on 127.0.0.2:80 and 127.0.0.2:443
  • HAProxy listens on port 80 and 443 of the public IP address. It redirects HTTP request on port 80 to port 443. When a request arrives on port 443, it will choose between Nginx and Apache back end by analyzing the SNI (server name indication) header in the HTTPS request.

Run Apache, Nginx & HAProxy on Same Server (Debian, Ubuntu, CentOS)

Actually, Cloudflare (a CDN provider) is also using the SNI header to determine how to route HTTPS requests to origin servers.

Step 1: Stop Nginx and Apache

To stop Nginx on Debian, Ubuntu and CentOS, run

sudo systemctl stop nginx

To stop Apache on Debian/Ubuntu, run

sudo systemctl stop apache2

To stop Apache on CentOS, run

sudo systemctl stop httpd

Step 2: Change the Listen Port in Nginx

We need to make Nginx listen on 127.0.0.1:80. Open your Nginx configuration files in /etc/nginx/conf.d/ or /etc/nginx/sites-enabled/ and find the following line.

listen 80;

Change it to

listen 127.0.0.1:80;

If https is enabled on the Nginx server block, then also find

listen 443 ssl;

And change it to

listen 127.0.0.1:443 ssl;

The Nginx main configuration file /etc/nginx/nginx.conf might include a default virtual host listening on port 80 or 443, so you might need to edit this file too.

Restart Nginx for the changes to take effect.

sudo systemctl restart nginx

Step 3: Change the Listen Port in Apache

We need to make Apache listen on 127.0.0.2:80.

Debian/Ubuntu

On Debian and Ubuntu, edit the /etc/apache2/ports.conf file.

sudo nano /etc/apache2/ports.conf

Change

Listen 80
Listen 443

To

Listen 127.0.0.2:80
Listen 127.0.0.2:443

Save and close the file. Also go to /etc/apache2/sites-enabled/ directory, edit the virtual host files. Change

<VirtualHost *:80>

To

<VirtualHost 127.0.0.2:80>

If there are SSL virtual host, then also change

<VirtualHost *:443>

To

<VirtualHost 127.0.0.2:443>

Restart Apache.

sudo systemctl restart apache2

CentOS

On CentOS, edit the /etc/httpd/conf/httpd.conf file.

sudo nano /etc/httpd/conf/httpd.conf

Find

Listen 80

Change it to

Listen 127.0.0.2:80

Save and close the file. Then go to /etc/httpd/conf.d/ directory, edit the virtual host files. Change

<VirtualHost *:80>

To

<VirtualHost 127.0.0.2:80>

If there are SSL virtual host, then also change

<VirtualHost *:443>

To

<VirtualHost 127.0.0.2:443>

In the /etc/httpd/conf.d/ssl.conf file, there’s

Listen 443 https

Change it to:

Listen 127.0.0.2:443 https

Save and close the file. Restart Apache for the changes to take effect.

sudo systemctl restart httpd

Step 4: Configure HAProxy

Install HAProxy on your distro.

Debian/Ubuntu

sudo apt install haproxy

CentOS

sudo dnf install haproxy

Edit HAProxy configuration file.

sudo nano /etc/haproxy/haproxy.cfg

Add the following code snippet at the end of the file, which will make HAPorxy listen on port 80 of the public IP address and redirect HTTP requests on port 80 to port 443. Replace 12.34.56.78 with your server’s public IP address.

frontend http
    bind 12.34.56.78:80
    mode http
    redirect scheme https code 301

Now we also need to add an HTTPS front end.

frontend https
    bind 12.34.56.78:443
    mode tcp
    tcp-request inspect-delay 5s
    tcp-request content accept if { req_ssl_hello_type 1 }

Then define the Nginx and Apache back ends. The check parameter tells HAProxy to perform health checks on the back-end by sending a TCP packet.

backend nginx
    mode tcp
    option ssl-hello-chk
    server nginx 127.0.0.1:443 check

backend apache
    mode tcp
    option ssl-hello-chk
    server apache 127.0.0.2:443 check

You can define a default back end with:

default_backend nginx

We will use SNI header in the HTTPS request to redirect to the correct back end. For example, if Nginx is serving domain1.com and Apache is serving domain2.com, then you add the following two lines. You can also use sub-domains, as long as they are different.

use_backend nginx if { req_ssl_sni -i domain1.com }
use_backend apache if { req_ssl_sni -i domain2.com }

Note that the default_backend and use_backend directives should be placed above the backend definitions.

frontend http
    bind 12.34.56.78:80
    mode http
    redirect scheme https code 301

frontend https
    bind 12.34.56.78:443
    mode tcp
    tcp-request inspect-delay 5s
    tcp-request content accept if { req_ssl_hello_type 1 }

    default_backend nginx

    use_backend nginx if { req_ssl_sni -i domain1.com }
    use_backend apache if { req_ssl_sni -i domain2.com }

backend nginx
    mode tcp
    option ssl-hello-chk
    server nginx 127.0.0.1:443 check

backend apache
    mode tcp
    option ssl-hello-chk
    server apache 127.0.0.2:443 check

In the configuration above, we utilized the SNI (Server Name Indication) feature in TLS to differentiate HTTPS traffic.

  • When domain1.com is in the TLS Client Hello, HAProxy redirects traffic to the nginx backend.
  • When domain2.com is in the TLS Client Hello, HAProxy redirects traffic to the apache backend.

If the client doesn’t specify the server name in TLS Client Hello, then HAproxy will use the default backend (nginx).

Save and close the file. Then restart HAproxy.

sudo systemctl restart haproxy

Now Apache, Nginx, and HAProxy are able to run on the same server.

How to Forward Client’s IP address to Backend

By default, Apache and Nginx can only see HAProxy’s IP address. To get the client’s real IP address, make sure you added the send-proxy-v2 option in the HAProxy’s back end definition like below.

backend nginx
    mode tcp
    option ssl-hello-chk
    server nginx 127.0.0.1:443 send-proxy-v2 check

backend apache
    mode tcp
    option ssl-hello-chk
    server apache 127.0.0.2:443 send-proxy check

Note: Nginx backend uses the send-proxy-v2 directive; Apache backend uses the send-proxy protocol. Technically they are the same. Apache doesn’t add a version number.

We also need to add some configuration in Nginx and Apache to make it work, otherwise your website will be inaccessible.

Nginx

Add proxy_protocol in the Nginx listen directive like below.

listen 127.0.0.2:443 ssl http2 proxy_protocol;

Then add the following two directives in the Nginx http { } block in /etc/nginx/nginx.conf file.

set_real_ip_from 127.0.0.1;
real_ip_header proxy_protocol;

Save and close the file. Then reload Nginx.

sudo systemctl reload nginx

Note: If your website runs behind Cloudflare CDN, then you should change real_ip_header proxy_protocol; to real_ip_header CF-Connecting-IP; in order to show the client’s real IP address. You can also add this real_ip_header directive to the individual server block file to override the global configuration in /etc/nginx/nginx.conf file.

Finally, restart HAProxy.

sudo systemctl restart haproxy

Apache

If you use Apache on Debian/Ubuntu, you need to enable the remoteip module. (This module is enabled on CentOS by default.)

sudo a2enmod remoteip

Then add the following 3 lines in your Apache virtual host configuration file.

RemoteIPProxyProtocol On
RemoteIPHeader X-Forwarded-For
RemoteIPTrustedProxy 127.0.0.1

Like this.

<VirtualHost 127.0.0.2:443>
    ServerName www.example.com
    RemoteIPProxyProtocol On
    RemoteIPHeader X-Forwarded-For
    RemoteIPTrustedProxy 127.0.0.1

Note: If you have multiple Apache virtual hosts, then you only need to add the above lines in one virtual host. It will be enabled for all virtual hosts. Don’t add the above lines to all virtual hosts, or you might get the RemoteIPProxyProtocol: Unsupported Command Error.

Save and close the file. Then we also need to change the combined log format. Edit the Apache main configuration file.

sudo nano /etc/apache2/apache2.conf

or

sudo nano /etc/httpd/conf/httpd.conf

Find the following line.

LogFormat "%h %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined

Replace it with:

LogFormat "%a %l %u %t \"%r\" %>s %O \"%{Referer}i\" \"%{User-Agent}i\"" combined

Save and close the file. Then restart Apache for the changes to take effect.

sudo systemctl restart apache2

or

sudo systemctl restart httpd

Note that the RemoteIPProxyProtocol On directive is only available in Apache 2.4.31 and newer. To check your Apache version, run

sudo apache2 -v

or

sudo httpd -v

Ubuntu 18.04 ships with Apache 2.4.29. If your Apache version doesn’t meet this requirement, then you should remove the send-proxy-v2 in the HAProxy back end definition. CentOS 8 ships with Apache 2.4.37.

Finally, restart HAProxy.

sudo systemctl restart haproxy

How to Add New Virtual Hosts

If you have Apache and Nginx web servers, then you need only two backends: one for Apache and one for Nginx.

When you need to add another domain (or sub-domain), you don’t need to add a new backend in HAProxy. Instead, you should create a new virtual host for the new domain (or sub-domain) in Apache or Nginx, then use the use_backend directive in HAProxy to redirect traffic to the correct backend.

frontend http
    bind 12.34.56.78:80
    mode http
    redirect scheme https code 301

frontend https
    bind 12.34.56.78:443
    mode tcp
    tcp-request inspect-delay 5s
    tcp-request content accept if { req_ssl_hello_type 1 }

    default_backend nginx

    use_backend nginx if { req_ssl_sni -i domain1.com }
    use_backend nginx if { req_ssl_sni -i sub.domain1.com }
    use_backend apache if { req_ssl_sni -i domain2.com }
    use_backend apache if { req_ssl_sni -i sub.domain2.com }

backend nginx
    mode tcp
    option ssl-hello-chk
    server nginx 127.0.0.1:443 check

backend apache
    mode tcp
    option ssl-hello-chk
    server apache 127.0.0.2:443 check

I recommend using apache and nginx as the backend names like in the above code, so you know which web server is hosting which application.

Obtaining New Let’s Encrypt SSL Certificate

http-01 challenge

First, you need to understand how to use the Certbot apache and nginx authenticator to obtain TLS certificate.

The Certbot apache and nginx authenticator use http-01 challenge, which works on TCP port 80. Because HAProxy is using TCP port 80, so the Certbot http-01 challenge is likely to fail. However, you can make it work with the following method.

  1. Enable SSL host in Apache or Nginx with a self-signed certificate or a certificate for another domain.
  2. Then use the Certbot apache and nginx authenticator as usual.

This method works because the Certbot http-01 challenge can be redirected from TCP port 80 to TCP port 443, and Certbot accepts invalid certificates on port 443.

dns-01 challenge

You can also use the certbot dns-01 challenge, which works by creating a temporary TXT record for your domain to certify that you actually own this domain, so it can bypass TCP port 80 and TCP port 443. The disadvantage of this method is that not all domain registrars are supported.

First, you need to install Certbot DNS plugin. They are several DNS plugins available in the Debian and Ubuntu software repository, which you can find with

apt search python3-certbot-dns

Output:

python3-certbot-dns-cloudflare/bionic,bionic 0.23.0-1 all
Cloudflare DNS plugin for Certbot

python3-certbot-dns-digitalocean/bionic,bionic 0.23.0-1 all
DigitalOcean DNS plugin for Certbot

python3-certbot-dns-dnsimple/bionic,bionic 0.23.0-1 all
DNSimple DNS plugin for Certbot

python3-certbot-dns-google/bionic,bionic 0.23.0-1 all
Google DNS plugin for Certbot

python3-certbot-dns-rfc2136/bionic,bionic 0.23.0-1 all
RFC 2136 DNS plugin for Certbot

python3-certbot-dns-route53/bionic,bionic 0.23.0-1 all
Route53 DNS plugin for Certbot

On CentOS 8, you can find Certbot DNS plugins from the EPEL repository. Note that you need to install Certbot from the default CentOS repository.

sudo dnf install certbot

Find DNS plugins from the EPEL repository.

sudo dnf install epel-release
dnf search certbot-dns

Output:

python3-certbot-dns-ovh.noarch : OVH DNS Authenticator plugin for Certbot
python3-certbot-dns-nsone.noarch : NS1 DNS Authenticator plugin for Certbot
python3-certbot-dns-gehirn.noarch : Gehirn Infrastructure Service DNS Authenticator plugin for Certbot
python3-certbot-dns-google.noarch : Google Cloud DNS Authenticator plugin for Certbot
python3-certbot-dns-linode.noarch : Linode DNS Authenticator plugin for Certbot
python3-certbot-dns-luadns.noarch : LuaDNS Authenticator plugin for Certbot
python3-certbot-dns-rfc2136.noarch : RFC 2136 DNS Authenticator plugin for Certbot
python3-certbot-dns-route53.noarch : Route53 DNS Authenticator plugin for Certbot
python3-certbot-dns-cloudxns.noarch : CloudXNS DNS Authenticator plugin for Certbot
python3-certbot-dns-dnsimple.noarch : DNSimple DNS Authenticator plugin for Certbot
python3-certbot-dns-cloudflare.noarch : Cloudflare DNS Authenticator plugin for Certbot
python3-certbot-dns-cloudflare.noarch : Cloudflare DNS Authenticator plugin for Certbot
python3-certbot-dns-dnsmadeeasy.noarch : DNS Made Easy DNS Authenticator plugin for Certbot
python3-certbot-dns-sakuracloud.noarch : Sakura Cloud DNS Authenticator plugin for Certbot

Install one of these plugins according to which DNS hosting service you are using. I’m using Cloudflare, so I will use Cloudflare as an example. Run the following command to install the python3-certbot-dns-cloudflare package on Debian/Ubuntu.

sudo apt install python3-certbot-dns-cloudflare

Then create a configuration file for Cloudflare.

sudo nano /etc/letsencrypt/cloudflare.ini

We need to add our Cloudflare account email address and API key in this file.

# Cloudflare API credentials used by Certbot
dns_cloudflare_email = [email protected]
dns_cloudflare_api_key = 0123456789abcdef0123456789abcdef01234567

You can find your Cloudflare API key at https://dash.cloudflare.com/profile. Note that the Certbot Cloudflare plugin does not currently support Cloudflare’s “API Tokens”, so ensure you use the “Global API Key” for authentication.

Save and close the file. The API key bypasses two-factor authentication of Cloudflare, so you should only allow root user to read this file.

sudo chmod 600 /etc/letsencrypt/cloudflare.ini

Now run certbot.

sudo certbot --agree-tos -a dns-cloudflare -i nginx --redirect --hsts --staple-ocsp --email [email protected] -d www.your-domain.com,your-domain.com

In the above command, we specified that we will use dns-cloudflare as the authenticator to obtain new TLS certificate and use the nginx plugin to create the HTTPS server block. If you use Apache, then replace nginx with apache.

This command will ask you to enter the path of the .ini file, so enter /etc/letsencrypt/cloudflare.ini and press the Enter key.

certbot dns cloudflare

After the SSL certificate is obtained and installed in the web server configuration file. You need to make it listen on 127.0.0.1:443 or 127.0.0.2:443. For example, if you use Nginx, then find

listen 443 ssl

And change it to

listen 127.0.0.1:443 ssl http2

Also, change port 80 to

listen 127.0.0.1:80;

Save and close the file. Then restart your web server.

Common Errors

1 Wrong Domain Name

If you have multiple Nginx virtual hosts on a server, and when you type one domain name, the web browser takes you to another domain name hosted on the same server, it could be that one of your Nginx virtual host file name doesn’t end with the .conf file name extension, so Nginx didn’t load this virtual host. It also could be that you have set AAAA record for the domain name, but you didn’t configure Nginx to serve the domain name in IPv6.

The above also applies to Apache web server.

2 Network Protocol Violation

If you set the send-proxy-v2 header in HAProxy, but didn’t enable proxy_protocol in Nginx or Apache, then your website will display the following error.

The site  has experienced a network protocol violation that cannot be repaired.

Remember to restart Nginx/Apache after making changes to configuration file.

3 PR_CONNECT_RESET_ERROR

If you enable proxy_protocol in Nginx or Apache, but didn’t set the send-proxy-v2 header in HAProxy, then your website will display this error.

secure connection failed. PR_CONNECT_RESET_ERROR

And you will find the following error message in Nginx or Apache error log.

broken header while reading PROXY protocol

Remember to restart Nginx/Apache after making changes to configuration file.

Also note that if you enable proxy protocol in one Apache/Nginx virtual host file, then it’s also implicitly enabled for all other virtual hosts. And I haven’t found a way to disable proxy_protocol for individual virtual host. Since you are running Apache and Nginx on the same server, a workaround is that you only enable proxy protocol in Nginx and don’t enable it in Apache. If a virtual host don’t need to get the client’s real IP address, configure it to use Apache.

Update: It occurred to me that you can set up a new Apache/Nginx virtual host on another loopback address like 127.0.0.4. In this way, the new virtual host won’t enable proxy protocol.

4 PR_END_OF_FILE_ERROR

If you enabled proxy protocol in Apache/Nginx, but you try to access the virtual host directly (bypassing HAProxy), then you will see the following error.

PR_END_OF_FILE_ERROR

5 SSL_ERROR_SYSCALL

If you use the curl command:

curl -I https://example.com

And you encounter the following error

curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to example.com:443

It’s probably because you didn’t set backend definition for this domain.

6 backend ‘apache’ has no server available!

If you see this error in HAProxy log (/var/log/haproxy.log), then you need to check your HAProxy backend definition.

backend apache
    mode tcp
    option ssl-hello-chk
    server apache 127.0.0.2:443 send-proxy-v2 check

You can remove the check option to fix this error.

backend apache
    mode tcp
    option ssl-hello-chk
    server apache 127.0.0.2:443 send-proxy-v2

Sometimes you need to include the check option, but it can also create problems. That’s a bit weird.

Wrapping Up

I hope this tutorial helped you run Apache, Nginx and HAProxy on the same server. As always, if you found this post useful, then subscribe to our free newsletter to get more tips and tricks. Take care 🙂

Rate this tutorial
[Total: 7 Average: 5]

6 Responses to “Run Apache, Nginx & HAProxy on Same Server (Debian, Ubuntu, CentOS)

  • Thank you. Most useful tutorial.

    I want to send all internet request through one pc in house. Are there any possible way to do

  • Hi,
    Thanks a lot,
    Very useful article

  • Thank you very much, this helped me out a lot!

    Is it possible to apply this to IPv6 the same way? Could you tell the steps necessary to do that?

    • Xiao Guoan (Admin)
      3 years ago

      1. Configure HAProxy to listen on port 80 and 443 of the public IPv6 address.

       frontend http
          bind 12.34.56.78:80
          bind [2607:f8b0:4006:810::200e]:80
          mode http
          redirect scheme https code 301
      
      frontend https
          bind 12.34.56.78:443
          bind [2607:f8b0:4006:810::200e]:443
          mode tcp
          tcp-request inspect-delay 5s
          tcp-request content accept if { req_ssl_hello_type 1 }
      

      2. If your Nginx listens on port 80 or 443 of the public IPv6 address, configure Nginx to listen on port 443 of the IPv6 loopback addresses ::1.
      In you Nginx virutal hosts, change listen [::]:443 to listen [::1]:443. You can go to your Nginx virtual host directory (/etc/nginx/cond.d/ or /etc/nginx/sites-enabled/) and run the following command to replace them.

      sudo sed -i 's/listen \[::\]:443/listen \[::1\]:443/g' *

      Also change listen [::]:80 to listen [::1]:80

       sudo sed -i 's/listen \[::\]:80/listen \[::1\]:80/g' *

      Test Nginx configuration.

      sudo nginx -t

      Then restart Nginx.

      sudo systemctl restart nginx

      ::1 is the only one IPv6 loopback address, so we can’t configure Apache to listen on ::2 (You can’t ping ::2). However, HAProxy can listen to requests on IPv6 interface and forward requests to IPv4 backends, so simply restart HAProxy and you are done.

      sudo systemctl restart haproxy
  • Michael S Costello
    2 months ago

    I’m giving this a try now on Digital Ocean.
    I tried earlier and failed with a redirect loop so I’m going to start fresh on a test droplet to track what I’m doing wrong on a test server. Already reverted everything back.

    The base will be the 22.04 default Haproxy in front of Apache2 and Nginx

    I’ll try and share how to get Nginx hosting an instance of the ghost blogging software with this configuration type as well. That might add a wrinkle because it’s a JavaScript webserver.

Leave a Comment

  • Comments with links are moderated by admin before published.
  • Your email address will not be published.
  • Use <pre> ... </pre> HTML tag to quote the output from your terminal/console.
  • Please use the community (https://community.linuxbabe.com) for questions unrelated to this article.
  • I don't have time to answer every question. Making a donation would incentivize me to spend more time answering questions.

The maximum upload file size: 2 MB. You can upload: image. Links to YouTube, Facebook, Twitter and other services inserted in the comment text will be automatically embedded. Drop file here