Google Pub/Sub Bridge
The Google Pub/Sub Plugin serves as a bridge between the Mosquitto broker and Google Pub/Sub service, facilitating the exchange of messages between MQTT and Google Pub/Sub. This interoperability supports various messaging patterns, including 1:1, 1:n, and n:1, as dictated by the designated topic mappings. For instance, a message that arrives at a Mosquitto broker can be relayed to one or multiple topics within the Google Pub/Sub service. Similarly, a Mosquitto broker can fetch messages from one or more Google Pub/Sub topics and distribute them to one or several MQTT topics. For further details, refer to the example configuration.
The plugin cannot configure the Google-Pub/Sub service. This has to be done upfront.
The push feature is currently not supported at all and the pull feature is in the beta state and therefore subject to change!
Plugin activation
To load and enable the Google Pub/Sub plugin in the broker add the following to the mosquitto.conf file:
plugin /usr/lib/cedalo_google_pubsub.so
plugin_opt_key_file pubsub-key.json
plugin_opt_config_file pubsub-config.json
persistence_location /mosquitto/data
Configuration is done via two JSON files. One is a key file which contains the Pub/Sub project and credentials to use. This file is provided by Google. The other file configures the plugin itself (see next section). The filenames can be edited, but the path is resolved relative to the persistence_location property.
Config file format
The configuration contains the settings for each Google Pub/Sub feature, namely publish, pull and push.
The core setting for each feature is the topic mapping defined by the topics property. For publish this defines how messages are forwarded from Mosquitto broker to Google Pub/Sub and for pull/push it defines the other way round, i.e. how messages received by Google Pub/Sub are forwarded to the Mosquitto broker.
The configuration file requires only a descriptive name. This means that each feature settings property, publish, pull and push, is optional! However, if none is specified the plugin does nothing.
Feature properties:
publish: Settings for the publish feature (i.e., topic mapping) (type:object).topics: The topic mappings which define how messages received by Mosquitto broker are forwarded to Google Pub/Sub topics (type:arrayof typeobject).mqtt: MQTT topic to be forwarded to Pub/Sub (type:string)pubsub: Pub/Sub topic to receive forwarded Mosquitto broker messages (type:string)
queue: Settings for internal message queue which is used to try to publish failed messages again (Optional, type:object).limit: Max. total number of queued messages. If limit is reached further received messages are dropped. (Optional, defaults to 100)retryDelaySEC: Delay in seconds to wait before retrying to publish queued messages again. (Optional, defaults to 10 seconds)maxRetries: Max. number of attempts before retry is stopped. Use -1 for unlimited retries. (Optional, defaults to -1)
pull: BETA - Settings for the pull feature. (type:object).pullTopic: The MQTT topic used to start pulling messages (type:string).pullDurationSEC: The duration in seconds for how long messages should be pulled from Google Pub/Sub (type:number).qos: The quality of service flag to use for MQTT publish (type:int; allowed values:0,1,2).topics: The topic mappings which define how messages received from Google Pub/Sub are forwarded to MQTT topics (type:array).
push: Currently not supported. (type:object).
For a more technical overview refer to the format schema at the bottom of this page as well.
Following is an example for a pubsub-config.json:
{
"name": "MQTT - GooglePubSub Topics Mapping",
"publish": {
"topics": [
{
"mqtt": "cedalo/test",
"pubsub": "test_topic"
},
{
"mqtt": "cedalo/test",
"pubsub": "cedalo_test_topic"
}
],
"queue": {
"limit": 10000,
"retryDelaySEC": 5
}
},
"pull": {
"pullTopic": "cedalo/test/pull",
"pullDurationSEC": 10,
"topics": [
{
"mqtt": "cedalo/app",
"pubsub": "cedalo_test_topic"
}
]
}
}
In this example publish defines a 1:n topic mapping, i.e. each message sent to the Mosquitto broker on topic cedalo/test is published to two Google Pub/Sub topics, namely test_topic and cedalo_test_topic. On the other hand pull defines a 1:1 topic mapping and a pull duration of 10 seconds. That means after the broker has started to pull messages each message received on Google Pub/Sub topic cedalo_test_topic within the following 10 seconds will be published by Mosquitto broker on topic cedalo/app. Message pull is started by sending an empty message to the specified pull topic cedalo/test/pull to the broker.
The pull feature is still experimental and subject to change!
JSON Schema
Overview over all possible parameters for the pubsub-config.json:
{
"type": "object",
"description": "Configuration for Google-Pub/Sub client connection.",
"additionalProperties": false,
"properties": {
"name": {
"type": "string",
"description": "Descriptive name for this configuration."
},
"publish": {
"type": "object",
"description": "Topics mapping for publish",
"additionalProperties": false,
"properties": {
"topics": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"additionalProperties": false,
"properties": {
"mqtt": {
"type": "string",
"description": "The mqtt topic to map",
"minLength": 1
},
"pubsub": {
"type": "string",
"description": "The google pubsub topic to publish to.",
"minLength": 1
}
},
"required": ["mqtt", "pubsub"]
}
}
},
"required": ["topics"]
},
"pull": {
"type": "object",
"description": "Topics mapping and settings for pull",
"additionalProperties": false,
"properties": {
"pullDurationSEC": {
"type": "number",
"description": "Specifies in seconds for how long incoming messages should be pulled.",
"minimum": 1,
"default": 30
},
"pullTopic": {
"type": "string",
"description": "The mqtt topic which triggers the pubsub pull.",
"minLength": 1
},
"qos": {
"type": "integer",
"enum": [0, 1, 2],
"description": "The quality of service value to use for publishing received messages to mqtt topic.",
"default": 0
},
"topics": {
"type": "array",
"minItems": 1,
"items": {
"type": "object",
"additionalProperties": false,
"properties": {
"mqtt": {
"type": "string",
"description": "The mqtt topic to publish to.",
"minLength": 1
},
"pubsub": {
"type": "string",
"description": "The google pubsub topic to pull from.",
"minLength": 1
}
},
"required": ["mqtt", "pubsub"]
}
}
},
"required": ["pullTopic", "topics"]
},
"push": {
"type": "object",
"description": "Topics mapping and settings for push",
"additionalProperties": false,
"properties": {
"url": {
"type": "string",
"description": "Specifies the https endpoint for push.",
"minLength": 1
}
},
"required": ["url"]
}
},
"required": ["name"]
}