Run OpenConnect VPN Server & Apache/Nginx on the Same Box with HAProxy

This tutorial will be showing you how to run OpenConnect VPN server (ocserv) and Apache/Nginx on the same box with HAProxy. OpenConnect (ocserv) is an open-source implementation of the Cisco AnyConnect VPN protocol.

ocserv-apache-nginx-haproxy

Prerequisites

To follow this tutorial, it’s assumed that you have already set up an OpenConnect VPN server with Let’s Encrypt TLS server certificate. If not, please follow one of the following tutorials.

Make OpenConnect VPN server and web server use port 443 at the same time

By default, OpenConnect VPN server listens on port 443. If you already have Apache/Nginx listening on port 443, then ocserv can’t bind to port 443. You can configure ocserv to listen on another port, but it will require end-users to specify the port in client software, which you should avoid if you care about user experience. Also, TLS traffic on TCP port 443 usually enjoys higher priority in QoS (Quality of Service), so you will have better speed.

Normally a port can only be used by one process. However, we can use HAproxy (High Availability Proxy) and SNI (Server Name Indication) to make ocserv and Apache/Nginx use port 443 at the same time.

Ocserv Configuration

First, edit ocserv configuration file.

sudo nano /etc/ocserv/ocserv.conf

Uncomment the following line. This will allow ocserv to obtain the client IP address instead of HAproxy IP address.

listen-proxy-proto = true

Then find the following line.

#listen-host = [IP|HOSTNAME]

Change it to

listen-host = 127.0.0.1

This will make ocserv listen on 127.0.0.1 because later HAproxy will need to listen on the public IP address. Save and close the file. Then restart ocserv.

sudo systemctl restart ocserv

Next, we also need to make the web server listen on localhost only, instead of listening on public IP address.

Nginx Configuration

If you use Nginx, edit the server block file.

sudo nano /etc/nginx/conf.d/example.com.conf

In the SSL server block, find the following directive.

listen 443 ssl;

Change it to

listen 127.0.0.2:443 ssl;

This time we make it listen on 127.0.0.2:443 because 127.0.0.1:443 is already taken by ocserv. Save and close the file. The Nginx main configuration file /etc/nginx/nginx.conf and the default server block /etc/nginx/sites-enabled/default might include a default virtual host listening on 443, so you might need to edit this file too.

Then restart Nginx.

sudo systemctl restart nginx

Apache Configuration

If you use Apache web server, edit your virtual host file.

Debian/Ubuntu

sudo nano /etc/apache2/sites-enabled/example.com.conf

CentOS/RHEL

sudo nano /etc/httpd/conf.d/example.com.conf

In the SSL virtual host, change

<VirtualHost *:443>

To

<VirtualHost 127.0.0.2:443>

This time we make it listen on 127.0.0.2:443 because 127.0.0.1:443 is already taken by ocserv. Save and close the file.

Then edit the /etc/apache2/ports.conf file on Debian/Ubuntu.

sudo nano /etc/apache2/ports.conf

Edit the/etc/httpd/conf.d/ssl.conf file on CentOS/RHEL.

sudo nano /etc/httpd/conf.d/ssl.conf

Change

Listen 443

To

Listen 127.0.0.2:443

Save and close the file. Restart Apache.

sudo systemctl restart apache2

or

sudo systemctl restart httpd

HAProxy Configuration

Now install HAproxy.

sudo apt install haproxy

or

sudo dnf install haproxy

Start HAProxy

sudo systemctl start haproxy

Edit configuration file.

sudo nano /etc/haproxy/haproxy.cfg

If you use Nginx, copy and paste the following lines to the end of the file. Replace 12.34.56.78 with the public IP address of your server. Replace vpn.example.com with the domain name used by ocserv and www.example.com with the domain name used by your web server.

frontend https
   bind 12.34.56.78:443
   mode tcp
   tcp-request inspect-delay 5s
   tcp-request content accept if { req_ssl_hello_type 1 }

   use_backend ocserv if { req_ssl_sni -i vpn.example.com }
   use_backend nginx if { req_ssl_sni -i www.example.com }
   use_backend nginx if { req_ssl_sni -i example.com }

   default_backend ocserv

backend ocserv
   mode tcp
   option ssl-hello-chk
   # pass requests to 127.0.0.1:443. Proxy protocol (v2) header is required by ocserv.
   server ocserv 127.0.0.1:443 send-proxy-v2

backend nginx
   mode tcp
   option ssl-hello-chk
   server nginx 127.0.0.2:443 check

If you use Apache, copy and paste the following lines to the end of the file. Replace 12.34.56.78 with the public IP address of your server. Replace vpn.example.com with the domain name used by ocserv and www.example.com with the domain name used by your web server.

frontend https
   bind 12.34.56.78:443
   mode tcp
   tcp-request inspect-delay 5s
   tcp-request content accept if { req_ssl_hello_type 1 }

   use_backend ocserv if { req_ssl_sni -i vpn.example.com }
   use_backend apache if { req_ssl_sni -i www.example.com }
   use_backend apache if { req_ssl_sni -i example.com }

   default_backend ocserv

backend ocserv
   mode tcp
   option ssl-hello-chk
   # pass requests to 127.0.0.1:443. Proxy protocol (v2) header is required by ocserv.
   server ocserv 127.0.0.1:443 send-proxy-v2

backend apache
    mode tcp
    option ssl-hello-chk
    server apache 127.0.0.2:443 check

Save and close the file. Then restart HAproxy.

sudo systemctl restart haproxy

In the configuration above, we utilized the SNI (Server Name Indication) feature in TLS to differentiate VPN traffic and normal HTTPS traffic.

  • When vpn.example.com is in the TLS Client Hello, HAProxy redirect traffic to the ocserv backend.
  • When www.example.com is in the TLS Client Hello, HAProxy redirect traffic to the apache/nginx backend.
  • If the client doesn’t specify the server name in TLS Client Hello, then HAproxy will use the default backend (ocserv).

You can test this setup with the openssl tool. First, run the following command multiple times.

echo | openssl s_client -connect your-server-IP:443 | grep subject

We didn’t specify server name in the above command, so HAproxy will always pass the request to the default backend (ocserv), and its certificate will be sent to the client. Next, run the following two commands.

echo | openssl s_client -servername www.example.com -connect your-server-IP:443 | grep subject

echo | openssl s_client -servername vpn.example.com -connect your-server-IP:443 | grep subject

Now we specified the server name in the commands, so HAproxy will pass requests according to the SNI rules we defined. Note that the Cisco AnyConnect App doesn’t support TLS SNI, so it’s better to set ocserv as the default backend in HAProxy configuration file.

When renewing Let’s Encrypt certificate for your website, it’s recommended that you use the http-01 challenge instead of tls-alpn-01 challenge, because HAproxy is listening on port 443 of the public IP address, so it can interfere with the renewal process.

sudo certbot renew --preferred-challenges http-01

Fixing HAproxy Error

If your Apache/Nginx website doesn’t show up in your browser and you see the following messages in haproxy log (/var/log/haproxy.log)

Server nginx/nginx is DOWN, reason: Socket error, info: "Connection reset by peer

backend nginx has no server available!

Layer6 invalid response

It might be your backend Nginx web server is using a TLS certificate with OCSP must staple extension. Nginx doesn’t send the OCSP staple information on the first HTTP request. To make it work, be sure to add a resolver in your Nginx virtual host configuration like below.

{
     ....
     ssl_trusted_certificate /etc/letsencrypt/live/www.example.com/chain.pem;
     ssl_stapling on;
     ssl_stapling_verify on;

    resolver 8.8.8.8;
    ....
}

Save and close the file. Then restart Nginx.

sudo systemctl restart nginx

Also, consider removing the health check for the backend server in HAproxy. So change

server nginx 127.0.0.2:443 check

To

server nginx 127.0.0.2:443

Save and close the file. Then restart HAproxy.

sudo systemctl restart haproxy

Wrapping Up

I hope this tutorial helped you run OpenConnect VPN server and Apache/Nginx on the same box. As always, if you found this post useful, then subscribe to our free newsletter to get more tips and tricks. Take care 🙂

Rate this tutorial
[Total: 2 Average: 5]

6 Responses to “Run OpenConnect VPN Server & Apache/Nginx on the Same Box with HAProxy

  • Stephan
    1 month ago

    What is the usecase of OpenConnect AND Webserver on the same box?
    I expected to read in this post, why I would need that.
    Thank you.

    • OpenConnect allows you to run your own VPN server in the cloud. Some folks might also want to build a website using Apache or Nginx. Running a VPN server and web sersver on the same box can save some money, because you don’t have to spin up another cloud server.

  • akash rao
    1 month ago

    nice 🙂

  • Dismissing the issue of ‘money’ and ‘saving some money’; the setup that you wrote is about:

    – Two things jumbled into one box [to same money]; or is it

    – A webserver, that is also connecting to the outside world via the built in VPN; and offering anonymity for the websites that are within the same ‘box’ ?

    Thanx

      • Thank you Xiao for the reply, and of course for the article. I am looking for instructions or how to have a ‘box’ that contains both a Webserver and a VPN services, where the websites IP Address are of the VPN’s IP Address (or anonymized). I don’t have the technical knowledge to know if this is possible or not (that a VPN separate server must be there to accomplish this).

Leave a Comment

  • Comments with links are moderated by admin before published.
  • Your email address will not be published.
  • Use <pre> ... </pre> HTML tag to quote the output from your terminal/console.
  • Please use the community (https://community.linuxbabe.com) for questions unrelated to this article.
  • I don't have time to answer every question. Making a donation would incentivize me to spend more time answering questions.


The maximum upload file size: 2 MB.
You can upload: image.