Proxy Protocol
Introduction
Using Mosquitto as an entirely standalone broker works very well, but sometimes you may wish to place it behind a load balancer / proxy such as HAProxy or NGINX. This provides certain advantages such as carrying out TLS termination in the proxy to reduce load on the broker.
It does have one disadvantage which is that only the proxy IP address is found in the broker logs - the client IP addresses are not seen.
This page describes some different ways you can combine Mosquitto and HAProxy. It is not a complete guide to HAProxy.
Important: Enabling PROXY protocol support requires that the broker itself is not directly accessible on its network port. All communication must go through the proxy. If a client is able to connect to the broker directly, it is trivial to spoof connection information and this is especially important when using client certificates for mutual TLS on HAProxy. In that case, the contents of the PROXY header can directly indicate whether a client is allowed to connect so it must be protected.
It may be desirable to use a firewall to restrict access to the broker port.
Choosing between PROXY v1 and v2
Where it is available you should always prefer PROXY v2. It is more efficient and has more features. In particular, v1 does not support anything other than passing on IPv4 and IPv6 addresses and ports. It does not have support for passing on TLS information when termination is carried out on the proxy, which restricts the possibility of using client certificates.
Not all proxies support PROXY v2, however, so if you are using e.g. nginx you will need to opt for PROXY v1.
Configuration
Pro Mosquitto supports both PROXY v1 and v2 and can be enabled with
the enable_proxy_protocol option.
Example configuration of a port using PROXY v2:
listener 1883
enable_proxy_protocol 2
# Further listener settings
Example configuration of a port using PROXY v1:
listener 1883
enable_proxy_protocol 1
# Further listener settings
Backward compatible
In Mosquitto 2.9 only PROXY v2 is supported and can be enabled using the
enable_proxy_protocol_v2 option.
This option remains valid.
Example configuration of a port:
listener 1883
enable_proxy_protocol_v2
# Further listener settings
HA Proxy Configuration examples
HA Proxy General setup
All examples presented will be of the haproxy.cfg file, typically located at
/etc/haproxy/haproxy.cfg on a native Linux installation.
The first part of the config file contains the global and defaults sections
which are going to be common to all the examples and not repeated.
Global section
This is a fairly standard global section, presented without comment.
global
log /dev/log local0
log /dev/log local1 notice
user haproxy
group haproxy
daemon
# Default SSL material locations
ca-base /etc/ssl/certs
crt-base /etc/ssl/private
# See: https://ssl-config.mozilla.org/#server=haproxy&server-version=2.0.3&config=intermediate
ssl-default-bind-ciphers ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384
ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
ssl-default-bind-options ssl-min-ver TLSv1.2 no-tls-tickets
Defaults section
defaults
mode tcp
timeout connect 5000
timeout client 60000
timeout server 60000
mode tcp- set TCP mode rather than HTTP mode by default. MQTT is not HTTP.timeout connect 5000- the timeout allowed for a client to connect, in milliseconds.timeout client 60000- if a client does not communicate in this interval, the connection will be closed by HAProxy.timeout server 60000- if the broker does not communicate in this interval, the connection will be closed by HAProxy.
The client and server timeout intervals should be chosen based on the intervals you expect to have communication occurring in your clients. If your communication is typically very sparse, the timeout should be chosen based on your keepalive interval, otherwise the quiet clients will be disconnected before they have chance to send a keepalive request.
Proxy configuration examples
Direct pass-through, without PROXY protocol
The most basic approach is to pass connections directly through HAProxy to Mosquitto without being modified. This means that if an unencrypted MQTT connection is made, it will pass through as an unencrypted connection, and if an encrypted MQTT connection is made it will pass through as an encrypted MQTT connection. Likewise, websockets connections are passed through unaffected.
Create a frontend, which is where HAProxy will listen for connections, and a backend, which is where the broker that HAProxy will connect to is defined.
frontend mqtt_frontend
bind *:1883
default_backend mqtt_backend
backend mqtt_backend
server server1 127.0.0.1:1884 check on-marked-down shutdown-sessions
In this case, HAProxy is listening on all interfaces on port 1883 and will attempt to connect to the broker on address 127.0.0.1 on port 1884. In other words, HAProxy and Mosquitto are running on the same instance. This is not required.
The broker configuration could be with an encrypted or unencrypted listener:
Unencrypted:
listener 1884
# Further listener settings
Encrypted:
listener 1884
certfile <path/to/server.crt>
keyfile <path/to/server.key>
# Further listener settings
TLS termination with server certificate only, without PROXY protocol
Place your server certificate and private key in /etc/haproxy/certs/.
frontend mqtts_frontend
bind 0.0.0.0:8883 ssl crt /etc/haproxy/certs/
backend mqtt_backend
server server1 127.0.0.1:1883 check on-marked-down shutdown-sessions
The broker configuration should declare an unencrypted listener:
listener 1883
# Further listener settings
TLS termination with mutual TLS - client and server certificates, without PROXY protocol
In addition to the server certificate, you must also provide the CA certificate that will sign the client certificates, ask for verification of the client certificate and make the certificate required.
frontend mqtts_frontend
bind *:8883 ssl crt /etc/haproxy/certs/ verify required ca-file /etc/haproxy/client-ca.crt
default_backend mqtt_backend
backend mqtt_backend
server server1 127.0.0.1:1883 check on-marked-down shutdown-sessions
The broker configuration should declare an unencrypted listener:
listener 1883
# Further listener settings
Direct pass-through, with PROXY v2 protocol
To enable PROXY protocol v2 on HAProxy, add the send-proxy-v2 option to the backend.
frontend mqtt_frontend
bind *:1883
default_backend mqtt_backend
backend mqtt_backend
server server1 127.0.0.1:1884 check on-marked-down shutdown-sessions send-proxy-v2
The broker configuration should declare an unencrypted listener and enable PROXY protocol v2 support. It is not possible to have a direct pass-through with encrypted connections on the broker.
listener 1883
enable_proxy_protocol 2
# Further listener settings
TLS termination with server certificate only, with PROXY v2 protocol
For TLS connections, use send-proxy-v2-ssl instead of send-proxy-v2. This
ensures that TLS information is added to the PROXY header.
frontend mqtts_frontend
bind *:8883 ssl crt /etc/haproxy/certs/
default_backend mqtt_backend
backend mqtt_backend
server server1 127.0.0.1:1883 check on-marked-down shutdown-sessions send-proxy-v2-ssl
On the broker side, use proxy_protocol_v2_require_tls true to ensure that
only connections that were made using TLS are accepted on the broker. No other
TLS configuration is required.
listener 1883
enable_proxy_protocol 2
proxy_protocol_v2_require_tls true
TLS termination with mutual TLS - client and server certificate, with PROXY v2 protocol
For TLS connections, use send-proxy-v2-ssl instead of send-proxy-v2. This
ensures that TLS information is added to the PROXY header.
frontend mqtts_frontend
bind *:8883 ssl crt /etc/haproxy/certs/ verify required ca-file /etc/haproxy/client-ca.crt
default_backend mqtt_backend
backend mqtt_backend
server server1 127.0.0.1:1883 check on-marked-down shutdown-sessions send-proxy-v2-ssl
The broker configuration uses require_certificate true to indicate that
the broker should check the PROXY protocol header for the valid certificate
result. No other TLS configuration is required.
listener 1883
enable_proxy_protocol 2
proxy_protocol_v2_require_tls true
require_certificate true
TLS termination with mutual TLS - client and server certificate, with username and PROXY v2 protocol
The Mosquitto option use_identity_as_username true can be used with the PROXY
protocol support. This requires that the send-proxy-v2-ssl-cn option is used
on HAProxy.
It is not possible to use use_subject_as_username with the PROXY protocol.
frontend mqtts_frontend
bind *:8883 ssl crt /etc/haproxy/certs/ verify required ca-file /etc/haproxy/client-ca.crt
default_backend mqtt_backend
backend mqtt_backend
server server1 127.0.0.1:1883 check on-marked-down shutdown-sessions send-proxy-v2-ssl-cn
The broker configuration uses require_certificate and
use_identity_as_username. No other TLS configuration is required.
listener 1883
enable_proxy_protocol 2
proxy_protocol_v2_require_tls true
require_certificate true
use_identity_as_username true
NGINX PROXY v1 example
This example configures NGINX to listen on port 8883 and forward TCP connections to a Mosquitto broker configured to use a TLS connection.
NGINX configuration:
stream {
server {
listen 8883;
proxy_pass 127.0.0.1:1884;
proxy_protocol on;
}
}
Mosquitto configuration:
listener 1884
certfile <path/to/server.crt>
keyfile <path/to/server.key>
# Further listener settings