(Option 3) Upload your configurations

Introduction

This section is useful if you want to make an advanced project configuration not supported by the pre-built binaries.

Advanced configuration means changing the default pins, the MQTT topics, and all the expert parameters that you can find in User_config.h (opens new window) and in all config_XX.h (opens new window) files. If you don't have to change the default parameters except Wi-Fi and MQTT broker settings, you don't need advanced configuration; you can go directly to the Upload Binaries section instead.

Configure & Upload with PlatformIO

• Download the CODE (opens new window) from github.
• Open the OpenMQTTGateway folder

You will find inside the folder a platformio.ini config file. PlatformIO uses this file to define how to build OMG for different kinds of hardware. Not just that, but it also specifies which modules to turn on and off. And there's more: it also lets you configure the settings of those modules.

PlatformIO config files work on the concept of overriding. At first, a very simple base "environment" is specified that specifies common variables shared by all situations:

[env]
framework = arduino
lib_deps =
  ${libraries.pubsubclient}
  ${libraries.arduinojson}
  ${libraries.arduinolog}
build_flags =
  -w ; supress all warnings
;  '-DLOG_LEVEL=LOG_LEVEL_TRACE'  ; Enable trace level logging
monitor_speed = 115200

Later "environments" get more specific, but inherit everything that was defined in this common environment:

[com-esp]
lib_deps =
  ${env.lib_deps}           ; Inherit all the library dependencies from [env]
  ${libraries.wifimanager}  ; Add another library dependency on top of them
build_flags =
  ${env.build_flags}        ; Inherit all the build flags from [env]
  '-DsimpleReceiving=true'  ; Add some of our own build flags
  '-DZmqttDiscovery="HADiscovery"'
  ;'-DCORE_DEBUG_LEVEL=4'

Here, build flags starting with "-D" let us set configuration values you would normally find in User_config.h and config_xx.h files by specifying them here, overriding the default values set in those files. To include special characters, you can triple escape them with a backslash like so:

  '-Dwifi_password="Cato\\\'sYounger\\\$on"' ; Cato'sYounger$on

The different listed configurations in platformio.ini represent some standard environments and boards. For example, the environment

[env:nodemcuv2-pilight]

sets the default settings for ESP8266 (NodeMCU v2) devices using the Plight module.

(Option A) Creating a portable config file

You could make your configuration changes directly by editing the values in User_config.h (for main OMG settings) and config_XX.h (for module-specific settings). You could also make most of those changes by instead writing some -D build flags in platformio.ini. But for maximum portability, a feature of PlatformIO allows you to make your configurations by creating a new file and listing all of your overrides there. This way, you can pull the latest OMG code changes without losing your configurations, or having to re-enter them manually.

To do this, create a file with a name ending in _env.ini, such as production_env.ini, into the root folder next to platformio.ini. PlatformIO will scan for all files ending in _env.ini and use it to override the default values in platformio.ini.

At the top of your *_env.ini file, for example, put the following to enable the ESP8266 (NodeMCU v2) with the Pilight module.

[platformio]
default_envs = nodemcuv2-pilight

This will make this environment the default environment for this PlatformIO project. If another environment isn't specified when building, it will default to this one.

For the rest of your config file, you can override the default configurations or add new configurations to existing environments in platformio.ini, or create a new environment. For example, if want to use both Pilight module and the BME280 module with an ESP8266, we can create a new environment. This is an example my_env.ini file that creates two new environments:

[platformio]
default_envs = nodemcuv2-pilight-bme280-ota

[env:nodemcuv2-pilight-bme280]
extends = env:nodemcuv2-pilight
lib_deps =
  ${env:nodemcuv2-pilight.lib_deps}
  ${libraries.bme280}
build_flags =
  ${env:nodemcuv2-pilight.build_flags}
  '-DGateway_Name="OpenMQTTGateway"'
  '-DZsensorBME280="BME280"'
  '-DBase_Topic="rf/"'
  '-DESPWifiManualSetup=true'
  '-Dwifi_ssid="mynetwork"'
  '-Dwifi_password="Cato\\\'sYounger\\\$on"' ; Cato'sYounger$on
  '-DMQTT_USER="mqttusername"'
  '-DMQTT_PASS="mqttpassword"'
  '-DMQTT_SERVER="mqttserver.local"'
  '-Dgw_password="12345678"'
  '-DLED_RECEIVE=LED_BUILTIN'        ; Comment 1
  '-DLED_RECEIVE_ON=LOW'             ; Comment 2
  '-DRF_RECEIVER_GPIO=13'
  '-DRF_EMITTER_GPIO=15'
  '-DsimpleReceiving=false'
  '-UZmqttDiscovery'                 ; Disable HA discovery
monitor_speed = 115200

[env:nodemcuv2-pilight-bme280-ota]
extends = env:nodemcuv2-pilight-bme280
upload_protocol = espota
upload_port = OpenMQTTGateway.local
upload_flags =
  --auth=otapassword
  --port=8266

Note

Adding manual WiFi and MQTT credentials to an environment also requires to define '-DESPWifiManualSetup=true' for the credentials to be registered correctly.

Note

Manual network configuration (IP, netmask, gateway, DNS) requires to define '-DNetworkAdvancedSetup=true' and related network parameters, e.g.:

'-DNET_IP="192.168.1.99"'
'-DNET_MASK="255.255.255.0"'
'-DNET_GW="192.168.1.1"'
'-DNET_DNS="1.1.1.1"'

The first new environment we create, nodemcuv2-pilight-bme280, inherits the default nodemcuv2-pilight environment in platformio.ini with the extends = env:nodemcuv2-pilight line. In the lib_deps section, it imports all the lib_deps of nodemcuv2-pilight with the ${env:nodemcuv2-pilight.lib_deps} line, and adds BME280 on top of it. (Since the environment we're extending already has this lib_deps attribute, specifying it again would normally replace it completely with our new attribute; instead, to keep its value but simply append to it, we import the original in the beginning of our lib_deps attribute.) In the build_flags section, it again imports all the build_flags of nodemcuv2-pilight and many of its own overrides, e.g. changing Base_Topic found in User_config.h from the default to "rf/" by using the '-DBase_Topic="rf/"' line. It also unsets previously set configurations (i.e. mqttDiscovery) by using '-UZmqttDiscovery'. This environment will work over serial upload.

The second new environment, nodemcuv2-pilight-bme280-ota, inherits everything we specified in the first environment (with the line extends = env:nodemcuv2-pilight-bme280), but modifies it to upload over OTA (Wi-Fi). We also specified this as the default_env in the beginning of the file, so PlatformIO will choose this environment to build and upload if we don't specify otherwise.

The first time we're flashing OMG to the board, we can use the command pio run --target upload --environment nodemcuv2-pilight-bme280 to specify that we want to build and run the nodemcuv2-pilight-bme280 environment (over USB serial). Afterwards, we don't have to specify --environment (e.g. just run pio run --target upload) to run the default nodemcuv2-pilight-bme280-ota environment and update the code over the air.

(Option B) Editing files directly

You can also modify the User_config.h file and your modules' config_XX.h files to your liking, and then edit the platformio.ini file to uncomment the default_envs lines corresponding to your board and chosen modules, like below:

;default_envs = sonoff-basic-rfr3
;default_envs = rfbridge
;default_envs = esp32dev-all
default_envs = esp32dev-rf
;default_envs = esp32dev-ir
;default_envs = esp32dev-ble
;default_envs = ttgo-lora32-v1

If you don't know which env to activate, you can refer to devices.

If you want to add more sensors or gateways to one default_envs you can add the modules directly into your environment definition of your .ini files or uncomment them in User_config.h (opens new window).

Example to add IR to esp32dev-rf add the build_flags below to the env definition:

  '-DZgatewayIR="IR"'

[env:esp32dev-rf]
platform = ${com.esp32_platform}
board = esp32dev
lib_deps =
  ${com-esp.lib_deps}
  ${libraries.rc-switch}
build_flags =
  ${com-esp.build_flags}
  '-DZgatewayRF="RF"'
  '-DZgatewayIR="IR"'
  '-DGateway_Name="OpenMQTTGateway_ESP32_RF_IR"'

Once your configuration is done you can upload the program to your board by clicking on the white arrow at the blue bottom bar of your PIO editor or with the following command: pio run --target upload

PIO will download the necessaries platform and libraries with the correct versions, build the code and upload it.

If you encounter errors the first thing to do is to clean your environment by using the white dust bin in the blue bottom bar of your PIO editor or with the following command: pio run --target clean

With some ESP it could be necessary to push the reset button when the upload begin.

If you want to erase the settings stored in the ESP memory use: pio run --target erase This can be useful especially before the first upload or when you change the board partitions sizing.

Once done the gateway should connect to your network and your broker, you should see it into the broker in the form of the following messages:

home/OpenMQTTGateway/LWT Online
home/OpenMQTTGateway/version

With PIO you can also upload the firmware through Over the Air, so as to do that you can add the upload options flags used below, upload_port is the IP address of your ESP:

[env:esp32-ble]
platform = ${com.esp32_platform}
board = esp32dev
board_build.partitions = min_spiffs.csv
lib_deps =
  ${com-esp.lib_deps}
  ${libraries.ble}
build_flags =
  ${com-esp.build_flags}
  '-DZgatewayBT="BT"'
  '-DGateway_Name="OpenMQTTGateway_ESP32"'
upload_protocol = espota
upload_port = 192.168.1.22
upload_flags =
  --auth=OTAPASSWORD
  --port=8266

API

With the V0.9 we added the support of json for receiving and publishing. Per default Json reception and Json publication is activated, the previous simple reception mode is also activated to avoid regression on commands.

You can deactivate Json or simple mode following theses instructions:

#define jsonPublishing true //define false if you don't want to use Json publishing (one topic for all the parameters)
//example home/OpenMQTTGateway_ESP32_DEVKIT/BTtoMQTT/4XXXXXXXXXX4 {"rssi":-63,"servicedata":"fe0000000000000000000000000000000000000000"}
#define simplePublishing false //define true if you want to use simple publishing (one topic for one parameter)
//example
// home/OpenMQTTGateway_ESP32_DEVKIT/BTtoMQTT/4XXXXXXXXXX4/rssi -63.0
// home/OpenMQTTGateway_ESP32_DEVKIT/BTtoMQTT/4XXXXXXXXXX4/servicedata fe0000000000000000000000000000000000000000
#define simpleReceiving true //define false if you don't want to use old way reception analysis
#define jsonReceiving true //define false if you don't want to use Json  reception analysis

If you are using platformio you can also comment the definitions above and define your parameters into platformio.ini file by setting the following build_flags:

  '-DjsonPublishing=true'
  '-DjsonReceiving=true'
  '-DsimpleReceiving=true'
  '-DsimplePublishing=true'

Note that depending on the environment the default platformio.ini has common option defined see sections:

[com-arduino]
[com-esp]

If you want to use HASS MQTT discovery you need to have #define jsonPublishing true & #define ZmqttDiscovery "HADiscovery" uncommented. Added to that auto discovery box should be selected into your Home Assistant MQTT integration configuration.

With an ESP if you did not set your network and MQTT parameters manually you can now open the web portal configuration.

Note

simpleReceiving on Arduino boards doesn't accept 64 bits MQTT values, you can only send 32bits values from the MQTT broker.

(opens new window)