Skip to main content
Version: Mosquitto 3.0

Docker High Availability

The following is a step-by-step tutorial to guide you through the installation process of Pro Mosquitto for an High Availability cluster on Docker. The tutorial assumes you have downloaded the Mosquitto Installer from us and have a valid license to work with. You can get a trial license by setting up an account on our website www.cedalo.com.

Install Docker Environment​

To run Cedalo Docker products, you need an installed version of docker and docker compose. Docker is not system depended and can be installed on Windows, Mac or Linux. Please see more information about docker and usage specifications on their website.

Open the following link and install Docker. If you have not installed a version of docker desktop and only installed the docker engine, make sure to further install Docker Compose.

Figure 1 - Docker Desktop Installer

Make sure your system meets the minimum requirements.

● 64 - bit processor with Second Level Address Translation (SLAT)
● 4GB system RAM
● BIOS-level hardware virtualization support must be enabled in the BIOS settings. The status can be
checked in your Task Manger:
info

If you choose Hyper-V make sure visualization is enabled. If Visualization is not enabled, you will have to turn it on in your BIOS. If you don ́t know how: Here is a very good step-by-step tutorial.

After installing make sure docker is running. On a typical installation the Docker daemon is started by a system utility, not manually by a user. This makes it easier to automatically start Docker when the machine reboots. The command to start Docker depends on your operating system. Check the correct page under Install Docker.

Start the Pro Mosquitto Cluster Setup​

Download the shared setup via our platform. You either have access through our trial or through a valid subscription to an on-premises plan. A trial license gives full access to all features. The downloaded setup has set reasonable default configurations and can be started without further changes. If you already know, which changes you want to make, you can do so now.

The default setup runs an preconfigured cluster on a single device. This is a good and easy start for testing. For a real implementation we recommend to run each Mosquitto node on a different server. This is explained in a later step.

If you have downloaded your setup via the Cedalo Platfrom, the license file, named "license.lic," is already included and can be found at ./license/license.lic in each server folder. If not, please manually add your license file.

To start Pro Mosquitto and the Cedalo MQTT Platform, navigate to the setup folder using the terminal and run the following command:

Depending on which operating system you are, choose the scripts to be used. ".sh" files are for Linux/MacOS and ".bat" files can be used in a Windows environment. The scripts have to be started via shell. The example setup may simply be started using:

Linux/Mac:

./start.sh up -d

Windows:

.\start.bat up -d
info

In-between Docker Desktop can warn you about file-sharing. File-sharing is intended by Cedalo. The Docker container would otherwise not be able to access certain files from the platform configuration that are stored in a regular Windows folder. Make sure to allow this access. Docker Desktop in general is only recommended for testing and developing situations and for real implementations it is suggested to use Docker CE.

Check Status​

During installation, a Docker environment will be created for Mosquitto Pro, consisting of multiple containers:

  • 3 Pro Mosquitto container
  • 3 HA Proxy container
  • 1 Cedalo Platform container

To verify that both containers are running, use the following command:

docker ps

You should see both the Mosquitto broker and the Cedalo MQTT platform listed as running.

For more detailed information, you can check the logs of the containers. To view the logs, run the following command

docker logs <containerid>

Access the Cedalo MQTT Platform​

By default, the test setup is accessible only on localhost, with MQTT port 1883 (via HAProxy) and HTTP port 3000 bound to the local machine. If you need external access to this setup, adjust the port configuration in the docker-compose.yml file accordingly.

To access the UI for your initial broker setup, go to http://localhost:3000/. You will be prompted to create your admin credentials when you first log in.

If you're using the Enterprise version of the Cedalo Platform, you can configure additional users within each project's settings.

Connect a client​

To connect your first client to the Pro Mosquitto broker, create a new client account and assign it a role. For full permissions to publish and subscribe, choose the role "client". For a detailed walkthrough of this process, please refer to our Onboarding Guide.

To connect to your Mosquitto instance, use the address localhost:1883 and the credentials you set up during the client account creation.

If you need to enable anonymous access, add the following line to your mosquitto.conf file under the listener configuration: allow_anonymous true

Configuration​

The configuration of the entire setup is managed through multiple files. Here are the details of each file:

  • docker-compose.yml: This file controls various settings related to the docker environment. It manages configurations such as port mappings and environment variables.

Additionally, there are other configuration files:

  • mosquitto.conf: This file is located in the \mosquitto\config directory. It contains specific configurations for the Mosquitto component.

  • haproxy.cfg: This file is located in the \haproxy directory. It contains the configuration for the incoming ports from clients and maps them to the three Pro Mosquitto broker nodes. It directs the traffic from mqtt clients to the broker leader.

These files play a crucial role in customizing and fine-tuning the settings of the entire system.

Adjusting Docker Settings for Linux Environment​

If you're using Linux for your Docker operations, there's one important setting you need to tweak. By default, Docker uses something called userland-proxy, but for smooth client communication, we need to turn this off.

Follow these steps:

  1. Locate the Configuration File

Open your file explorer and go to the /etc/docker/ directory. Inside, you'll find a file named daemon.json.

  1. Edit the Configuration

Open daemon.json with a text editor (like nano or VSCode) to make changes. If the file doesn't exist, you can create it.

  1. Adjust the Settings

Replace the existing content with the following:

{
  "userland-proxy": false,
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "100m",
    "max-file": "10",
    "labels": "production_status",
    "env": "os,customer"
  }
}

This configuration ensures that the userland-proxy is turned off and sets up some useful logging options.

  1. Save and Apply

Save the file and close the text editor.

  1. Restart Docker

You'll need to restart Docker to apply these changes. You can do this by opening your terminal and running: sudo systemctl restart docker

Reconfiguring the Cluster for separate servers​

The setup is designed to be easily transferred from the existing test configuration to a full implementation across three separate servers.

We recommend running the three servers within a private network, with access to each node available from an external network. For more details, refer to our cluster design documentation here.

Once you have transferred the individual subfolders named server1, server2, and server3, it's crucial to remove the existing cluster to allow for reconfiguration at a later step and update the configurations to match the new environment.

You can find more details for these steps here.

Example Docker Compose file​

This exampless includes the Platform and Mosquitto. You only need one Platform to run on one of the cluster nodes server. Image names in our default setups and this example always pull the latest patch for each minor release. For instance, 3.0 will automatically pull 3.0.x based on the available patch version.

services:
    mosquitto:
        image: registry.cedalo.com/mosquitto/mosquitto:3.0
        volumes:
            - ./mosquitto/config:/mosquitto/config
            - ./mosquitto/data:/mosquitto/data
            - ./license:/mosquitto/license:ro
        hostname: mosquitto1
        networks:
            mosquitto:
              ipv4_address: 172.20.1.1
        environment:
            CEDALO_LICENSE_FILE: /mosquitto/license/license.lic
            MOSQUITTO_DYNSEC_PASSWORD: Wy8TQGDoWJT0ohm3
        restart: unless-stopped
    haproxy:
        image: haproxy:2.7
        ports:
            - 127.0.0.1:1883:1883 # change binding 127.0.0.1:1883 to public IPv4 X.Y.Z:1883
        volumes:
            - ./haproxy:/usr/local/etc/haproxy:ro
        restart: unless-stopped
        networks:
            mosquitto:
              ipv4_address: 172.20.2.1
    platform:
        image: registry.cedalo.com/platform/enterprise:3.0
        # image registry.cedalo.com/platform/business:3.0  // Enterprise images only run with an enterprise license. Choose the right image for your license. 
        platform: linux/amd64
        ports:
            - 127.0.0.1:3000:3000
        networks:
            mosquitto:
                ipv4_address: 172.20.3.1
        environment:
            # CEDALO_HOST_NAME: ${CEDALO_HOST_NAME}
            # CEDALO_IP_ADDRESS: ${CEDALO_IP_ADDRESS}
            API_INTERNAL_HOST: platform
            BASE_URL: http://localhost:3000
            NEXTAUTH_URL: http://localhost:3000
            MOSQUITTO_PLATFORM_LICENSE_FILE: /platform/license/license.lic
            AUTH_SECRET: keHjk98hM2P8y67F
            SETUP_TYPE: cluster
        volumes:
            - ./platform/data:/data
            - ./license:/platform/license/
            # map unix socket for audit trail:
        restart: unless-stopped
networks:
    mosquitto:
      name: mosquitto
      ipam:
        driver: default
        config:
          - subnet: 172.20.0.0/16

Docker Userland Proxy​

If HAProxy is running via docker, make sure to disable the userland proxy in the docker daemon.json file. Make sure to add "userland-proxy": false and restart docker. Verify the configuration using docker info | grep -i "userland"