Archived Forum

Ah got it I need to add authentication support to Emoncms nodes module.....tomorrows job :-) 

https://github.com/emoncms/nodes/blob/9.0/nodes_mqtt_process.php

Can Emoncms nodes read the authentication credentials from emonhub.conf file? 

Here's a PHP MQTT with authentication example. 

https://www.cloudmqtt.com/docs-php.html

Good to you have started on the MQTT authentication.

The nodes module does have all the MQTT settings hardcoded into it, (as is the redis config I just discovered)  if modules can pick up globals from the main program it shouldn't be hardcoded as there is also the "phpmqtt_input.php" script for the generic inputs module and the "publish to mqtt" process to consider too, there is probably some "event" processes in the pipeline that may require MQTT too.

Ideally all the emoncms MQTT settings should come from the settings.php, currently only the broker IP and an enabled flag have been broken out to the settings.php. The port is hardcoded in multiple places and as yet there are no password or username required for authentication. If using MQTT over ssl then a different port is used so if using outside the LAN then it should be accessible to change too.

The settings.php already has mysql, redis and smtp server details so expanding on the current MQTT settings there would seem the best place.

The nodes module only reads the [nodes] section of emonhub.conf.

Paul

Yes, agree nodes module needs to take MQTT settings from settings.php

I have merged the auth branch into the emonPi branch of emonHub. I have tested the changes I made the emonHub to support authentication and if no username and password is set in emonhub.conf it will default to initiating user/pass as blank which will work on un-authenticated mosquitto servers.

If authentication is required just add the following to emonhub.conf:

 Type = EmonHubMqttInterfacer
    [[[init_settings]]]
        mqtt_host = 127.0.0.1
        mqtt_port = 1883
        mqtt_user = user
        mqtt_passwd = passwrd

I'm now going to take a look at Emoncms nodes module PHP MQTT authentication. 

Yes, we would like to move towards supporting MQTT on emoncms.org. Have you got any thoughts how best this could be implemented. Maybe we could start another thread and collect input from other folk. 

is the nodes module installed by default ? i'm not sure what it is but i have added support for mqtt authentication myselve only doing this described here http://openenergymonitor.org/emon/node/11769 did i forget a place where mqtt is used ? i think i did a grep statement but it could be i overlooked something. Those simple changes i made to my emoncms installation made mqtt authentication already work and i'm still using it like that. Although i have to note that i'm only using it to transmit feed values to mqtt, i'm not reading from mqtt

or is de nodes module some extra addon ?

edit: did not see emonhub, emonhub and emoncms is probably something diffrent ?

emonHub is different to Emoncms. We use emonHub to read data from the RFM69Pi or emonPi via serial and publish to MQTT. Emoncms then subscribes to MQTT and pulls in the data. 

Authentication support has now been added to Emoncms MQTT input script and publish to MQTT process list. See commits today. 

MQTT authentication support has also been added to  Emoncmcs nodes mode. This is currenttly in the 'auth' branch. Would be grateful if you could test.  

https://github.com/emoncms/nodes/tree/auth

The user credentials are set in setting.php e.g 

    $mqtt_enabled = true;          // Activate MQTT by changing to true
    $mqtt_server = "127.0.0.1";
    $mqtt_port = 1883;
    $mqtt_user = "";
    $mqtt_password= "";

Can the MQTT interfaces be fixed so they accept valid JSON as well please?

@glyn.hudson

posting to my password protected mosquito from within emoncms (using feed processlist) works. however there's a bug here https://github.com/emoncms/emoncms/blob/master/scripts/phpmqtt_input.php...

that line should be $log->error ("Cannot connect to MQTT Server"); instead of $log->error "Cannot connect to MQTT Server";

it seems it was a echo statement before but now you're calling a method from a class so ( and ) are needed.

That script does not work on windows so I could not test MQTT input I'll see if i can modify the script for my own needs. It uses hard coded Linux paths for making it a daemon but on windows that path does not exist.

It gave me these errors, all normal since my emoncms installation is located on a windows box. I'll change the script without touching your code for MQTT and see if it works then>

C:\wamp\www\emoncmsnew\scripts>php phpmqtt_input.php

Warning: fopen(/var/lock/phpmqtt_input.lock): failed to open stream: No such file

or directory in C:\wamp\www\emoncmsnew\scripts\phpmqtt_input.php on line 32

Call Stack:
    0.0010     268168   1. {main}() C:\wamp\www\emoncmsnew\scripts\phpmqtt_input
.php:0
    0.0010     269024   2. fopen() C:\wamp\www\emoncmsnew\scripts\phpmqtt_input.
php:32

Warning: flock() expects parameter 1 to be resource, boolean given in C:\wamp\ww
w\emoncmsnew\scripts\phpmqtt_input.php on line 33

Call Stack:
    0.0010     268168   1. {main}() C:\wamp\www\emoncmsnew\scripts\phpmqtt_input
.php:0
    0.0030     269848   2. flock() C:\wamp\www\emoncmsnew\scripts\phpmqtt_input.
php:33

Already running

okay I commented out line 32 and 33 and then it runs on windows.

I then did a C:\Program Files (x86)\mosquitto>mosquitto_pub.exe -t rx/16 -u myuser-P mypassword -m 20.500

and node 16 is created with a value of 20.5 so I guess it works except for the logging bug.

However, I did not test connecting to a non password protected mosquito to see if that still works.

Someone else could probably try that out.

Try it with a valid JSON string.  Does it still work? I could only make it work with a Comma delimited list of values.  I posted a suggested change so it works for valid JSON payload.

I do not think valid json will work. see https://github.com/emoncms/emoncms/blob/stable/scripts/phpmqtt_input.php#L119

It still seems to expect comma separated values (or a single value) as far as I can tell. Have not looked in detail to it though.

Thanks for noticing the logging error, it's been fixed. Thanks so much for testing being an authenticated MQTT guinea pig :-)

None of us here develop on windows or have access to windows machines, therefore dev might be a bit Linux centric. However, I can see value in Emoncms working nice on windows boxes. I would appreciate feedback (ideally pull requests!) as to how we can make the system also work on windows server.   

Actually, core emoncms works "out of the box" on a windows box. No modifications are needed. I basically just installed WAMP and then placed emoncms inside the www directory, edited settings.conf and all is good. It's been running on my Intel Nuc for like a month and a half without a problem. This script was the only thing that needed modification, and all I did was remove lines 32 and 33 about the locking part since that's related to linux for the daemon.

Once those lines were removed, I opened a cmd prompt and ran php.exe with that script as the parameter. The cmd prompt keeps running and MQTT is monitored then.

There probably is a way to install this script as a windows service by using something like this https://nssm.cc/ It can make any program (but most probably non gui ones) a windows service. I have not tested this though. I  use my Intel Nuc as a server and there are a dozen open cmd windows all running different things like node-red, mosquito, home assistant, OpenHAB, etc.

Oh yes, I did make another change to one of the modules. I can't remember then name of it. (events?) It checks to see if an emon node hasn't been updated for a while, and sends an email message listing those. But I think that script did not work anymore on Linux with the latest emoncms version. I modified it to suit my needs and make it work for what I needed it to do (based on my nodes). so it's not "generic".

After more testing, MQTT authentication support has been merged into the main 9.0 branch of the nodes module. 

@glyn - is there any documentation yet about how EmonCMS MQTT input processing works? I.e. what format the MQTT payloads need to be, topic names etc? Cheers, Ben.

It simply accepts a comma separated list.  If you are trying to receive data sent from another system it probably will not work.

Yes you can publish from any device just publish a csv of values to topic emonhub/rx/NODEID/values 

We are working on improving mqtt support and updating docs. 

There is some docs here: https://github.com/emoncms/emoncms/blob/stable/scripts/phpmqtt_input.php

Yes you can publish from any device but only if you are able to publish an invalid JSON string (a list of CSV values).  The current MQTT support does not comply with the MQTT standard (such as it is) as EmonCMS does not accept valid JSON.

I'm trying to publish from another system that generates valid JSON and EmonCMS wil not accept it.

Line 119 is the start of the problem as it assumes CSV not a valid MQTT JSON message.

Cheers