Docker Setup

Prerequisites

Ensure you have the following files ready before configuring Mosquitto:

• Server Certificate (server.crt): The public server certificate.
• Private Key (server.key): The private key associated with the server certificate.

Configuring Pro Mosquitto Broker with a Server Certificate

This document outlines the two available methods for configuring a broker with a server certificate: using a proxy for TLS termination or configuring TLS directly on the broker. In cluster environments, the use of HAProxy, as provided in the default setup, is recommended for managing TLS termination.

Pro Mosquitto TLS termination

Open your mosquitto.conf file to add a certificate to a certain port. Add or modify the following settings to configure the Pro Mosquitto broker to use the server certificate:

# Enable a listener on port 8883 for SSL/TLS connections
listener 8883
protocol mqtt

# SSL/TLS certificate paths
certfile /path/to/server.crt
keyfile /path/to/server.key

• listener 8883: Sets the port for SSL/TLS communication. Port 8883 is the standard MQTT SSL/TLS port.
• certfile: Specifies the path to the server's public certificate file.
• keyfile: Specifies the path to the server's private key file.

HA Proxy TLS termination

In a cluster setup an HA Proxy is used for TLS termination. This is the default config of the frontend haproxy.cfgfile:

frontend mqtt_frontend
    bind *:1883
    mode tcp
    default_backend mqtt_backend
    timeout client 10m

In order to enable TLS termination here add the following behind the bind *:<port> parameter: ssl crt /path/to/certs/server.pem This will do the TLS termination at the frontend stage before routing the traffic further to the set backend (here mqtt_backend ).

Client connections

Use an MQTT client to connect to the broker using the secure port (e.g., 8883) to verify that the server certificate is being used:

mosquitto_sub -h <broker-ip> -p 8883 -t test/topic --cafile /path/to/ca.crt

In this example, a --cafile parameter is added to the test because, depending on the device and the server certificate, your underlying system may not have the required CA in place to validate the server certificate you set.

Certificate access

It is important that the files from the specified path are available to Pro Mosquitto. Ensure that the private key file is securely accessible to the Mosquitto service:

sudo chmod 640 /path/to/server.key
sudo chown mosquitto:mosquitto /path/to/server.key

This ensures that only the Mosquitto service has access to the private key, enhancing security.

To apply the changes, restart Pro Mosquitto. Check the Mosquitto logs for troubleshooting if needed.

Configuring Cedalo Platform with a Server Certificate

You can achieve https:// access either via a reverse proxy of your choice or by adding the server certificates directly to the platform via environment variables.

Environment Variables

To conifgure the server certificates use the following Environment Variables:

• BASE_URL: URL the platform will be reachable at (should have the same IP/hostname as specified in the certificate)
• NEXTAUTH_URL: URL the platform will be reachable at (should have the same IP/hostname as specified in the certificate)
• MOSQUITTO_PLATFORM_HTTPS_KEY_PATH: Path to server's private key file (required)
• MOSQUITTO_PLATFORM_HTTPS_CERT_PATH: Path to server's certificate (required)
• MOSQUITTO_PLATFORM_HTTPS_CA_PATH: path to CA certificate (optional)

Example configuration

            BASE_URL: https://192.168.178.27:3000
            NEXTAUTH_URL: https://192.168.178.27:3000
            MOSQUITTO_PLATFORM_HTTPS_KEY_PATH: /certs/cert.key
            MOSQUITTO_PLATFORM_HTTPS_CERT_PATH: /certs/cert.crt

Docker compose

In docker based deployments make sure, that the linked certificates are reachable. A mount to the local directory allows you to store the used certs in the setup folders:

        volumes:
            ...
            - ./certs:/certs

Kubernetes/OpenShift Setup

TLS Configuration Overview for Kubernetes/OpenShift

The Helm charts for both Kubernetes and OpenShift support dynamic TLS configuration through values.yaml and --set flags, eliminating the need to manually edit and repackage chart files. The approach differs between single-node and high-availability (HA) deployments:

Key Differences:

Deployment Type	TLS Termination Location	Certificate Format	Recommended Approach
Single Node	Mosquitto Broker	Separate cert + key files	Direct Mosquitto TLS
HA Cluster	HAProxy Load Balancer	Combined PEM file (cert+key)	HAProxy TLS Termination

Why Different Approaches?

• Single Node: TLS is configured directly on the Mosquitto broker since there's no load balancer in front.
• HA Cluster: TLS termination at HAProxy provides centralized certificate management, simplifies the setup, and allows the load balancer to handle secure connections before routing to Mosquitto brokers.

---

Single Node TLS Configuration (Server-Only TLS)

For single-node deployments, TLS is configured directly on the Mosquitto broker using server-only TLS. This enables one-way TLS where only the server presents a certificate to clients.

Note: For OpenShift deployments, replace kubectl with oc in all commands below. Chart directory names:

• Kubernetes Single Node: mosquitto-platform-kubernetes-sn
• OpenShift Single Node: mosquitto-platform-openshift-sn

Prerequisites

Ensure you have the following files ready:

• Server Certificate (server.crt): The public server certificate
• Private Key (server.key): The private key associated with the server certificate

Secret Creation

Certificates can be created externally to Helm charts using kubectl or oc commands. This is the recommended approach as it's easier and provides better control over secret management. When using external secrets, set createSecret: false in your Helm installation.

Note: The secret names and paths mentioned here are default values and can be configured via Helm values. See the Configuration Parameters section below for customization options.

For Kubernetes:

# Set namespace
NAMESPACE="your-namespace"

# Create server-only TLS certificate secret (using default secret name)
kubectl create secret generic mosquitto-server-tls \
  --from-file=server.crt=server.crt \
  --from-file=server.key=server.key \
  -n $NAMESPACE

For OpenShift:

# Set namespace
NAMESPACE="your-namespace"

# Create server-only TLS certificate secret (using default secret name)
oc create secret generic mosquitto-server-tls \
  --from-file=server.crt=server.crt \
  --from-file=server.key=server.key \
  -n $NAMESPACE

Note: When creating secrets externally, ensure the secret name matches mosquitto.serverOnlyTls.secretName (default: mosquitto-server-tls) and the secret keys match the certFile and keyFile values (defaults: server.crt and server.key). Set mosquitto.serverOnlyTls.createSecret=false in your Helm installation to use the externally created secret.

First-Time Installation

For Kubernetes Single Node:

# Set namespace
NAMESPACE="your-namespace"

# Install with serverOnlyTLS enabled
helm install mosquitto-platform \
  ./mosquitto-platform-kubernetes-sn \
  -n $NAMESPACE \
  -f values.yaml \
  --set mosquitto.serverOnlyTls.enabled=true \
  --set mosquitto.serverOnlyTls.createSecret=false

For OpenShift Single Node:

# Set namespace
NAMESPACE="your-namespace"

# Install with serverOnlyTLS enabled
helm install mosquitto-platform \
  ./mosquitto-platform-openshift-sn \
  -n $NAMESPACE \
  -f values.yaml \
  --set mosquitto.serverOnlyTls.enabled=true \
  --set mosquitto.serverOnlyTls.createSecret=false

Upgrade Installation

For Kubernetes Single Node:

# Set namespace
NAMESPACE="your-namespace"

# Upgrade with serverOnlyTLS enabled
helm upgrade mosquitto-platform \
  ./mosquitto-platform-kubernetes-sn \
  -n $NAMESPACE \
  -f values.yaml \
  --set mosquitto.serverOnlyTls.enabled=true \
  --set mosquitto.serverOnlyTls.createSecret=false

For OpenShift Single Node:

# Set namespace
NAMESPACE="your-namespace"

# Upgrade with serverOnlyTLS enabled
helm upgrade mosquitto-platform \
  ./mosquitto-platform-openshift-sn \
  -n $NAMESPACE \
  -f values.yaml \
  --set mosquitto.serverOnlyTls.enabled=true \
  --set mosquitto.serverOnlyTls.createSecret=false

Configuration Parameters

Note: All secret names, file names, and mount paths mentioned below are default values and can be customized via Helm values. You can override any of these defaults to match your specific requirements.

• mosquitto.serverOnlyTls.enabled: Enable server-only TLS listener on port 8883. When enabled, Mosquitto will present a server certificate to clients for authentication. Default: false

• mosquitto.serverOnlyTls.secretName: Kubernetes Secret name containing server TLS certificates. This secret must exist if createSecret is set to false. Default: mosquitto-server-tls

• mosquitto.serverOnlyTls.createSecret: If true, Helm will create the secret from certContent and keyContent. If false, the secret must exist externally (created via kubectl or oc). Recommended approach: Set to false and create secrets externally using kubectl create secret or oc create secret commands, as this is easier and provides better control over secret management. Default: true

• mosquitto.serverOnlyTls.certFile: Certificate file name (key) in the Kubernetes Secret. This should match the key name used when creating the secret externally. Default: server.crt

• mosquitto.serverOnlyTls.keyFile: Private key file name (key) in the Kubernetes Secret. This should match the key name used when creating the secret externally. Default: server.key

• mosquitto.serverOnlyTls.tlsMountPath: Directory path where server TLS certificates are mounted in the container. Should be a DIRECTORY path only (e.g., /mosquitto/certs). Do not include the filename. Default: /mosquitto/certs

• mosquitto.serverOnlyTls.certName: Certificate filename when mounted in the container. Should be a FILENAME only (e.g., server.crt), no path. The final path is constructed as: tlsMountPath/certName = /mosquitto/certs/server.crt. Default: server.crt

• mosquitto.serverOnlyTls.keyName: Private key filename when mounted in the container. Should be a FILENAME only (e.g., server.key), no path. The final path is constructed as: tlsMountPath/keyName = /mosquitto/certs/server.key. Default: server.key

• mosquitto.serverOnlyTls.certContent: Base64-encoded server certificate content. Only used when createSecret: true. If createSecret: false, this parameter is ignored and the secret must be created externally with keys matching certFile and keyFile. Default: "" (empty string)

• mosquitto.serverOnlyTls.keyContent: Base64-encoded server private key content. Only used when createSecret: true. If createSecret: false, this parameter is ignored and the secret must be created externally with keys matching certFile and keyFile. Default: "" (empty string)

What Happens Under the Hood

When you enable serverOnlyTLS, the Helm chart automatically:

1. Mounts the Secret as a volume in the Mosquitto StatefulSet at the configured mount path (default: /mosquitto/certs)
2. Updates mosquitto.conf to enable the TLS listener on port 8883:

   listener 8883
   protocol mqtt
   certfile /mosquitto/certs/server.crt
   keyfile /mosquitto/certs/server.key
   Note: The paths shown above are defaults. The actual paths are constructed from tlsMountPath/certName and tlsMountPath/keyName which are configurable.
3. Exposes port 8883 on the Mosquitto service

Verify TLS Configuration

# Check the Mosquitto service ports
kubectl get svc mosquitto -n $NAMESPACE

# Check server certificates are mounted
kubectl exec mosquitto-0 -n $NAMESPACE -- ls -la /mosquitto/certs/
# Should show: server.crt and server.key

# Verify Mosquitto configuration
kubectl exec mosquitto-0 -n $NAMESPACE -- grep -A 5 "listener.*8883" /mosquitto/config/mosquitto.conf

# Test TLS connection
mosquitto_sub -h <mosquitto-external-ip> -p 8883 --cafile ca.crt -u <username> -P <password> -t '#'

Important Path Configuration Notes

IMPORTANT: Follow these conventions to avoid path configuration errors:

• tlsMountPath: Should be a DIRECTORY path only (e.g., /mosquitto/certs)
• certName/keyName: Should be FILENAMES only (e.g., server.crt, server.key)
• Final path is constructed as: tlsMountPath/certName = /mosquitto/certs/server.crt
• ❌ WRONG: Setting tlsMountPath to /mosquitto/certs/server.crt (includes filename)
• ✅ CORRECT: tlsMountPath="/mosquitto/certs" and certName="server.crt"

---

HA Cluster TLS Configuration

For High-Availability (HA) cluster deployments, TLS termination occurs at the HAProxy load balancer, not on individual Mosquitto brokers. This is the recommended approach for production HA deployments.

Why HAProxy TLS for HA?

• ✅ Centralized certificate management: Update certificates in one place
• ✅ Simplified configuration: No need to configure TLS on each broker
• ✅ Better performance: Offload TLS processing to HAProxy
• ✅ Easier certificate rotation: Update one HAProxy Secret vs. multiple broker Secrets

Prerequisites

Ensure you have the following:

• Server Certificate (server.crt): The public server certificate
• Private Key (server.key): The private key associated with the server certificate

Prepare HAProxy PEM Certificate

HAProxy requires certificates in PEM format with the certificate and private key combined in a single file:

# Combine certificate and key into a single PEM file
cat server.crt server.key > haproxy_combined.pem

# Encode to base64 for Kubernetes Secret
cat haproxy_combined.pem | base64 -w 0 > haproxy_combined_base64.txt

Important: The order matters! Certificate first, then the private key.

Configuration Using Helm Values (Recommended)

Install with HAProxy TLS Configuration

# Read base64 content
PEM_CONTENT=$(cat haproxy_combined_base64.txt)

# Install with HAProxy TLS enabled
helm install my-ha-cluster mosquitto-3.1-platform-3.1-kubernetes-cluster.tgz -n ha \
  --set licenseKey=<your-license-key> \
  --set nameserver=10.0.0.10 \
  --set haproxy.tls.enabled=true \
  --set haproxy.tls.pemContent="$PEM_CONTENT" \
  --set haproxy.tls.secretName="haproxy-tls-certs" \
  --set haproxy.tls.pemFile="haproxy_combined.pem" \
  --set haproxy.ports.secureListener=8883

HAProxy TLS Configuration Parameters

• haproxy.tls.enabled: Enable HAProxy TLS termination. Default: false
• haproxy.tls.secretName: Kubernetes Secret name for HAProxy certificates. Default: haproxy-tls-certs
• haproxy.tls.pemFile: PEM filename (combined cert+key). Default: haproxy_combined.pem
• haproxy.tls.pemContent: Base64-encoded PEM content (cert+key combined)
• haproxy.tls.pemPath: Full path to PEM file in container. Default: /etc/haproxy/certs/haproxy_combined.pem
• haproxy.tls.tlsHaproxyPemDirPath: Directory path for HAProxy certificates. Default: /etc/haproxy/certs
• haproxy.ports.secureListener: HAProxy secure MQTT port. Default: 8883
• haproxy.ports.secureListenerTarget: Target port for secure connection. Default: 8883

Alternative: Using Existing Kubernetes Secret

# Create Kubernetes Secret with PEM file
kubectl create secret generic haproxy-tls-certs -n ha \
  --from-file=haproxy_combined.pem=./haproxy_combined.pem

# Install referencing existing secret
helm install my-ha-cluster mosquitto-3.1-platform-3.1-kubernetes-cluster.tgz -n ha \
  --set licenseKey=<your-license-key> \
  --set nameserver=10.0.0.10 \
  --set haproxy.tls.enabled=true \
  --set haproxy.tls.secretName=haproxy-tls-certs

What Happens Under the Hood

When you enable HAProxy TLS, the Helm chart automatically:

1. Creates a Kubernetes Secret (haproxy-tls-certs) with your PEM certificate
2. Mounts the Secret as a volume in the HAProxy Deployment at /etc/haproxy/certs
3. Updates HAProxy configuration to add a secure frontend:

   frontend mqtt_frontend_secure
       bind *:8883 ssl crt /etc/haproxy/certs/haproxy_combined.pem
       mode tcp
       maxconn 10000
       default_backend mqtt_backend
       timeout client 10m
4. Exposes port 8883 on the HAProxy LoadBalancer service
5. Routes secure traffic to Mosquitto brokers (non-TLS communication between HAProxy and brokers)

Verify HAProxy TLS Configuration

# Check HAProxy service ports (should show both 1883 and 8883)
kubectl get svc haproxy -n ha

# Test TLS connection through HAProxy
mosquitto_sub -h <haproxy-external-ip> -p 8883 --cafile ca.crt -u <username> -P <password> -t '#'

Cluster Setup with Server-Only TLS (End-to-End Encryption)

If you require encryption between HAProxy and Mosquitto brokers (end-to-end encryption), you can enable serverOnlyTLS on Mosquitto. HAProxy will automatically perform TCP passthrough, allowing Mosquitto to handle TLS termination directly.

Note: For OpenShift deployments, replace kubectl with oc in all commands below. Chart directory names:

• Kubernetes Cluster: mosquitto-platform-kubernetes-cluster
• OpenShift Cluster: mosquitto-platform-openshift-cluster

Prerequisites, Secret Creation, and Configuration Parameters: See the Single Node TLS Configuration (Server-Only TLS) section above. The same prerequisites, secret creation steps, and configuration parameters apply to cluster deployments.

Key Differences for Cluster Deployments:

• Use cluster chart directories: mosquitto-platform-kubernetes-cluster or mosquitto-platform-openshift-cluster
• HAProxy automatically performs TCP passthrough when serverOnlyTLS is enabled
• HAProxy rollout is required after installation/upgrade to pick up configuration changes

First-Time Installation

For Kubernetes Cluster:

# Set namespace
NAMESPACE="your-namespace"

# Install with serverOnlyTLS enabled
helm install mosquitto-platform \
  ./mosquitto-platform-kubernetes-cluster \
  -n $NAMESPACE \
  -f values.yaml \
  --set mosquitto.serverOnlyTls.enabled=true \
  --set mosquitto.serverOnlyTls.createSecret=false

# Rollout HAProxy to pick up serverOnlyTLS configuration
kubectl rollout restart deployment haproxy -n $NAMESPACE

For OpenShift Cluster:

# Set namespace
NAMESPACE="your-namespace"

# Install with serverOnlyTLS enabled
helm install mosquitto-platform \
  ./mosquitto-platform-openshift-cluster \
  -n $NAMESPACE \
  -f values.yaml \
  --set mosquitto.serverOnlyTls.enabled=true \
  --set mosquitto.serverOnlyTls.createSecret=false

# Rollout HAProxy to pick up serverOnlyTLS configuration
oc rollout restart deployment haproxy -n $NAMESPACE

Upgrade Installation

For Kubernetes Cluster:

# Set namespace
NAMESPACE="your-namespace"

# Upgrade with serverOnlyTLS enabled
helm upgrade mosquitto-platform \
  ./mosquitto-platform-kubernetes-cluster \
  -n $NAMESPACE \
  -f values.yaml \
  --set mosquitto.serverOnlyTls.enabled=true \
  --set mosquitto.serverOnlyTls.createSecret=false

# Rollout HAProxy to pick up serverOnlyTLS configuration changes
kubectl rollout restart deployment haproxy -n $NAMESPACE

For OpenShift Cluster:

# Set namespace
NAMESPACE="your-namespace"

# Upgrade with serverOnlyTLS enabled
helm upgrade mosquitto-platform \
  ./mosquitto-platform-openshift-cluster \
  -n $NAMESPACE \
  -f values.yaml \
  --set mosquitto.serverOnlyTls.enabled=true \
  --set mosquitto.serverOnlyTls.createSecret=false

# Rollout HAProxy to pick up serverOnlyTLS configuration changes
oc rollout restart deployment haproxy -n $NAMESPACE

Verify Configuration

# Check server certificates are mounted
kubectl exec mosquitto-0 -n $NAMESPACE -- ls -la /mosquitto/certs/
# Should show: server.crt and server.key

# Verify Mosquitto configuration
kubectl exec mosquitto-0 -n $NAMESPACE -- grep -A 5 "listener.*8883" /mosquitto/config/mosquitto.conf
# Should show serverOnlyTLS listener configuration with certfile and keyfile

# Test TLS connection through HAProxy (TCP passthrough)
mosquitto_sub -h <haproxy-external-ip> -p 8883 --cafile ca.crt -u <username> -P <password> -t '#'

Note: This approach provides end-to-end encryption where TLS is terminated at Mosquitto brokers. HAProxy performs TCP passthrough and does not inspect or terminate TLS traffic. This is typically not required for most deployments, as communication between HAProxy and Mosquitto occurs within the Kubernetes cluster network. For most use cases, HAProxy TLS termination (described above) is the recommended approach.

---

Updating TLS Configuration

To update certificates after initial deployment:

Server-Only TLS (Single Node and Cluster End-to-End Encryption)

Note: This process applies to both single node deployments and cluster deployments using serverOnlyTLS for end-to-end encryption. Both use the same secret (mosquitto-server-tls) and update process.

For Kubernetes:

# Update the secret with new certificates
kubectl create secret generic mosquitto-server-tls \
  --from-file=server.crt=server.crt \
  --from-file=server.key=server.key \
  -n $NAMESPACE \
  --dry-run=client -o yaml | kubectl apply -f -

# Restart pods to pick up new certificates
kubectl rollout restart statefulset mosquitto -n $NAMESPACE

# For cluster deployments, also restart HAProxy
kubectl rollout restart deployment haproxy -n $NAMESPACE

For OpenShift:

# Update the secret with new certificates
oc create secret generic mosquitto-server-tls \
  --from-file=server.crt=server.crt \
  --from-file=server.key=server.key \
  -n $NAMESPACE \
  --dry-run=client -o yaml | oc apply -f -

# Restart pods to pick up new certificates
oc rollout restart statefulset mosquitto -n $NAMESPACE

# For cluster deployments, also restart HAProxy
oc rollout restart deployment haproxy -n $NAMESPACE

HA Cluster (HAProxy TLS Termination)

For Kubernetes:

# Update HAProxy certificates
kubectl create secret generic haproxy-tls-certs \
  --from-file=haproxy_combined.pem=./haproxy_combined.pem \
  -n $NAMESPACE \
  --dry-run=client -o yaml | kubectl apply -f -

# Restart HAProxy to pick up new certificates
kubectl rollout restart deployment haproxy -n $NAMESPACE

For OpenShift:

# Update HAProxy certificates
oc create secret generic haproxy-tls-certs \
  --from-file=haproxy_combined.pem=./haproxy_combined.pem \
  -n $NAMESPACE \
  --dry-run=client -o yaml | oc apply -f -

# Restart HAProxy to pick up new certificates
oc rollout restart deployment haproxy -n $NAMESPACE

---

Troubleshooting

Single Node Issues

Connection Refused on Port 8883

• Verify mosquitto.serverOnlyTls.enabled=true is set
• Check service exposes port 8883: kubectl get svc mosquitto -n $NAMESPACE
• Verify certificates are mounted: kubectl exec mosquitto-0 -n $NAMESPACE -- ls -la /mosquitto/certs

Certificate Validation Errors

• Ensure certificate matches the hostname/IP you're connecting to
• Use --cafile flag to provide CA certificate if not in system trust store

HA Cluster Issues

Connection Refused on Port 8883 (HAProxy)

• Verify haproxy.tls.enabled=true is set
• Check HAProxy service exposes port 8883: kubectl get svc haproxy -n ha
• Verify PEM file is mounted: kubectl exec <haproxy-pod> -n ha -- ls -la /etc/haproxy/certs

Invalid PEM Format

• Ensure certificate and key are combined in correct order: certificate first, then key
• Verify base64 encoding is correct: echo "$PEM_CONTENT" | base64 -d | openssl x509 -text

HAProxy Not Starting

• Check HAProxy logs: kubectl logs <haproxy-pod> -n ha
• Common issue: Missing or incorrectly formatted PEM file

---

Security Best Practices

1. Use Strong Certificates: Use certificates from a trusted CA or properly configured self-signed certificates
2. Secure Private Keys: Never commit private keys to version control
3. Rotate Certificates Regularly: Update certificates before expiration
4. Use Kubernetes Secrets: Store certificates in Kubernetes Secrets, not in plain text
5. Restrict Secret Access: Use Kubernetes RBAC to limit who can access certificate Secrets
6. Monitor Certificate Expiration: Set up alerts for certificate expiration
7. For HA: Always use HAProxy TLS termination instead of individual broker TLS for easier management