emonhub emon-pi variant MQTT support & Emoncms MQTT input service

As you may know Glyn's been working on adding nodered and OpenHAB to the next emonpi image alongside emoncms. Both of these make use of MQTT for their messaging. Emoncms and Emonhub on the emonpi have been using MQTT now for some time however the message format is a little harder to use beyond the emoncms nodes module.

I've added a second format option to the EmonHubMqttInterfacer in the emon-pi variant of emonhub. Its now possible to publish individual node/variables to MQTT, i.e:

    topic: basetopic/emontx/power1
    message: 100

Its possible to use this with the emoncms/scripts/phpmqtt_input.php script. The basetopic has been changed from rx/# to nodes/# in phpmqtt_input.php.

There are a couple of new configuration options in emonhub.conf:

[[MQTT]]
    Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 127.0.0.1
        mqtt_port = 1883
        mqtt_user = emonpi
        mqtt_passwd = emonpimqtt2016

    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,

        # emonhub/rx/10/values format - default emoncms nodes module
        node_format_enable = 1
       node_format_basetopic = emonhub/

        # nodes/emontx/power1 format
       nodevar_format_enable = 0
       nodevar_format_basetopic = nodes/

By setting nodevar_format_enable = 1 its possible to send messages with this new format or/and its possible to send messages with the node only format.

By default existing systems will not have these lines in emonhub.conf instead the runtime settings looks like this:

    [[[runtimesettings]]]
        pubchannels = ToRFM12,
        subchannels = ToEmonCMS,
        basetopic = emonhub/

Pulling in the latest emonhub.conf will not change emonhub operation as long as the basetopic was emonhub/ prior to the update - which should be most emonpi systems. If you have changed the basetopic please update your emonhub.conf with the new configuration settings in bold above.

I've also added a section on this to the new emonhub configuration documentation page

Emoncms MQTT Input Script 

To import the new MQTT feeds into Emoncms the Emoncms MQTT input script has been updated to subscribe to base topic 'nodes/#'. Be sure to pull in the latest changes (pushed to Emoncms stable branch) 

The Emoncms MQTT input service needs to be installed following these instructions to run the script as a daemon. Here are some example of how the Emocms MQTT input script can be used:

  •    create an input from emonTx node called power with value 10:

         nodes/emontx/power 10
     

  •      create an input from node 10 called power with value 10 :  

         nodes/10/power 10
         

  •      create input from emontx with key 0 of value 10

         nodes/emontx 10
         

  •      create input from emontx with key 0 of value 10, key 1 of value 11 and key 2 of value 11:

         nodes/emontx 10,11,12 

Reliability: There is a question about the reliability of the MQTT messages perhaps on the phpMQTT client end, it was detected to a degree in the RF testing. But with a higher volume of messages it may be worse. Id be interested to hear other's experience with it. Update: we did initially test this on the old July 2015 emonPi SD card image, Glyn has since tested with the latest version of Mosquitto (1.4.7) on Raspbian Jessie Lite where reliability appears much better: transmission from RFM69Pi / emonPi > Serial > emonHub > Emocms Inputs.

If further testing is positive we will consider using this MQTT structure for the new pre-built SD card image. 

TrystanLea's picture

Re: emonhub emon-pi variant MQTT support & Emoncms MQTT input service

Its possible to upgrade mosquitto on the July 2015 image to version 1.4.7 by following the steps here:

http://mosquitto.org/2013/01/mosquitto-debian-repository/

It does seem to be much more reliable. spoke too soon.. it seems that phpMQTT.php may not support qos=2 and qos=0 still misses messages..

glyn.hudson's picture

Re: emonhub emon-pi variant MQTT support & Emoncms MQTT input service

It seems the PHP MQTT library we have been using from https://github.com/bluerhinos/phpMQTT buggy. It does not like to accept a QOS higher than 0. 

Has anyone got experience with Mosquitto-PHP with https://github.com/mgdm/? It looks promising, I tried to give it a go but having install issues on Raspberry Pi: https://github.com/mgdm/Mosquitto-PHP/issues/19

glyn.hudson's picture

Re: emonhub emon-pi variant MQTT support & Emoncms MQTT input service

Progress. Managed to get  https://github.com/mgdm/Mosquitto-PHP installed. It requires pacakge libmosquitto-dev to be installed. Initial testing indicates improved php mqtt performance over the old library. 

 

Here are the install steps to setup a test using the new MQTT data structure 

rpi-rw

sudo apt-get install libmosquitto-dev​
pecl install Mosquitto-alpha

(​Hit enter to autodetect libmosquitto location)
sudo sh -c 'echo "extension=mosquitto.so" > /etc/php5/cli/conf.d/20-mosquitto.ini'
sudo sh -c 'echo "extension=mosquitto.so" > /etc/php5/apache2/conf.d/20-mosquitto.ini'

Copy new emonhub.conf with the changes Trystan posted above:

cd ~/emonpi/conf

git pull
cp emonhub.conf ~/data/emonhub.conf

turn on new format and turn off the old format MQTT data structure 

        # emonhub/rx/10/values format - default emoncms nodes module
        node_format_enable = 0
       node_format_basetopic = emonhub/

        # nodes/emontx/power1 format
       nodevar_format_enable = 1
       nodevar_format_basetopic = nodes/

sudo service emonhub restart

Check emonHub is posting to MQTT using new structure:

mosquitto_sub -v -u 'emonpi' -P 'emonpimqtt2016' -t 'nodes/#'

You should see:

nodes/emonPi/power1 303
nodes/emonPi/power2 0
nodes/emonPi/power1_plus_power2 303
nodes/emonPi/Vrms 237.55
nodes/emonPi/T1 0
nodes/emonPi/T2 0
nodes/emonPi/T3 0
nodes/emonPi/T4 0
nodes/emonPi/T5 0
nodes/emonPi/T6 0
nodes/emonPi/pulseCount 0
nodes/emonPi/rssi 0

Now switch new dev-mosquitto-php emocms branch and run php mqtt input script 

cd /var/ww/emoncms

git checkout dev-mosquitto-php

cd /var/www/emoncms/scripts 

Run script manually with: 

php phpmqtt_input.php

You should see something like:

pi@emonpi:/var/www/emoncms/scripts$ php phpmqtt_input.php 
Subscribing to: nodes/#
I got code 0 and message Connection Accepted.
Subscribed to a topic
nodes/emonPi/power1 261
set_timevalue
nodes/emonPi/power2 0
set_timevalue
nodes/emonPi/power1_plus_power2 261
set_timevalue
nodes/emonPi/Vrms 238.47

Run as service at startup follow these instructions

hschroyen's picture

Re: emonhub emon-pi variant MQTT support & Emoncms MQTT input service

Looks great.

Any idea if I can be using this with Ubuntu?

see http://openenergymonitor.org/emon/node/11851 in which I more or less understood that MQTT is not working on rc1.2 ubuntu...

Thanks and keep up the great work

Regards

 

glyn.hudson's picture

Re: emonhub emon-pi variant MQTT support & Emoncms MQTT input service

Yes, this should work great on Ubuntu since the Nodes module is not required. The Nodes module was written for the RaspberryPi (emonPi / emonBase), there is no reason why the Nodes module could not be made to work on general linux but currently there are paths hard coded for the emonPi. 

Using the mqtt input scrpt service with the new node variable MQTT topic structure the data goes straight from emonHub > MQTT > emoncms inputs without requiring the Nodes module. 

Follow the MQTT setup guide, currently on the MQTT dev branch. This will be merged into master today hopefully 

https://github.com/emoncms/emoncms/blob/dev-mosquitto-php/docs/Raspberry...

https://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/MQTT.md

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.