I installed this version yesterday on my emonPi.

Brand new installation on a new SD card from the latest SD image.

I'm trying to create new feeds from my nodes but after I add and save anew node I get a "wait..." in the process list and the feed does not get really added.

If after some times I return to edit the node I find for example "Feedid 16 does not exists or was deleted"

And in Administration -> Logger I read
2016-04-01 08:25:44.629|WARN|PHPFina.php|get_meta() meta file does not exist '/home/pi/data/phpfina/16.meta' 2016-04-01 08:28:22.059|ERROR|feed_model.php|EngineClass() Engine id '1' is not supported.

From /var/log/apache2/error.log:

[Fri Apr 01 08:53:40.090096 2016] [:error] [pid 1518] [client 192.168.1.72:54730] PHP Fatal error:  Uncaught exception 'Exception' with message 'ABORTED: Engine id '1' is not supported.' in /var/www/emoncms/Modules/feed/feed_model.php:71\nStack trace:\n#0 /var/www/emoncms/Modules/feed/feed_model.php(396): Feed->EngineClass('1')\n#1 /var/www/emoncms/Modules/feed/feed_model.php(279): Feed->get_timevalue('1')\n#2 /var/www/emoncms/Modules/feed/feed_model.php(253): Feed->redis_get_user_feeds(1)\n#3 /var/www/emoncms/Modules/feed/feed_controller.php(38): Feed->get_user_feeds('1')\n#4 /var/www/emoncms/core.php(64): feed_controller()\n#5 /var/www/emoncms/index.php(146): controller('feed')\n#6 {main}\n  thrown in /var/www/emoncms/Modules/feed/feed_model.php on line 71, referer: http://emonpi/emoncms/input/view

What should I do?

Please check in the original post the ports and example links for NodeRED and OpenHAB:

NodeRED 13.3 - with custom OEM setup port:18880 user,pass:emonpi,emonpi2016
OpenHab 1.8.2 - with custom OEM setup port:8080 user,pass:pi,emonpi2016

...

Access Via: http://emonpi:1880

(I think it should be "Access Via: http://emonpi:18880")

...

Access via: http://emonpi:8080

e.g. dave.items and dave.sitemaps then navigate to http://emonpi8081/openhab.app?sitemap=dave

(I think it should be "http://emonpi:8080/openhab.app?sitemap=dave")

Regarding OpenHab:

You write "Note: Config files are symlinked to our oem_openhab git repo so we can add functionally with git updates, pulled in automatically with emonPi update. It's recommended to setup your own sitemap and files config so your changes will not be overwritten by out updates: "

Would the file openhab.cfg -> /home/pi/oem_openHab/openhab.cfg be overwritten by an update?

I need to modify it to add  Philips Hue Lighting support, so I would like to know if I need to restore my modified copy after a possible OpenHab update.

_{^{Small item}} for the Expand SD Card section it should be:

$ sudo emonSDexpand

Hi guys, thanks a lot for testing.

@gsiviero thanks for letting me know about the meta file errors. I have been able to re-create and am investigating. I will post a fix as soon as I can. Sorry for the hassle.

Thanks for letting me know about the openHAB port number typo. This has been fixed.  NodeRED port is correct it should be 1880. Does this not work for you? 

If you modify openhab.cfg it won't update with any emonPi updates. If you want to be totally save you could remove the symlink:

sudo rm /etc/openhab/openhab.cfg

sudo cp /home/pi/oem_openHab/openhab.cfg /etc/openhab/openhab.cfg 

@Jon where did you see the emonSDexpand typo? It seems to be correct on the post above. Please correct me if I'm wrong!  

Glyn - in the first post of this thread

@jon thanks, it's been corrected :-) 

Glyn - Everything works really well with the new emonSD-29Mar16 image.  A big “thank you” to the Team - very nice work!

I found one odd item. I went to Administration > Update emonPi > Update Now and nothing happened - no update! The virtual terminal would clear and then just redisplay the first boot emonpi update. I deleted the emonpiupdate.log file, then the virtual terminal went blank/black and now the Update Now button works as expected. I was not able to repeat the issue.

Hy Glyn,

NodeRED port should then be corrected in the first part of the post (but I'm not using it yet):

NodeRED 13.3 - with custom OEM setup port:18880 user,pass:emonpi,emonpi2016

in the ​openHab example link in the openHab section you should also add the colon (sorry if I'm pedant):

e.g. dave.items and dave.sitemaps then navigate to http://emonpi8080/openhab.app?sitemap=dave

(I think it should be "http://emonpi:8080/openhab.app?sitemap=dave")

Another thing: my emonPi display is always lit, I will try to determine why this is happening.

Thanks for your work!

Thanks for the typo corrections. They have been fixed.

@jon mmm, I don't know what is going on here. I'll try and replicate. Is this the 'out of box' experience you had after flashing the new image? It sounds like the first update the should happen automatically on the very first boot of the emonPi was interrupted. This cased the update log display terminal to hang since it was waiting for more data to display. This first update will happen in the factory so most users should not experience this issue. 

@Glyn - I saw the same issue as Jon with the "update" when I used a fresh download of this image at the weekend. When I clicked "update" from the webpage, it goes blank and repopulates without any sign of updating. I did not explore any further, so as far as I'm aware, it is still in the fault condition, if you are not able to reproduce.

I will be back home later.

I did also notice there was no "emonPi updating" type message on the LCD at any time, which I though was odd. But since I have also seen other anomolies with the LCD like garbage at boot, locking up and staying permanently lit, I didn't think too much about it.

I emailed you over the weekend with some other points too.

Paul

Glyn - Yes, this was after the new build on a blank SD. The virtual terminal display acted like it had more to display - I could not scroll upward as the display would jump to the last line. And there was no "emonPi Update done" on the last line.

Hi !

Testing this on a emonBase (Raspberry Pi B Rev2)...

Almost everything seems to work out of the box including NodeRed except:

    /var/lib/phpfina
    /var/lib/phpfiwa
    /var/lib/phptimeseries/

Did not exist on the files system witch prevent the feed creation (with an undefined error message). After creating them and setting the rights manualy, everything was working for me.

Edit: seems like this directories are under /home/pi/data ont the SDCard, no need to create them manually. But this is not working at this point. After creating a feed and trying to display the feeds module logs say :

2016-04-06 18:50:13.656|WARN|PHPFina.php|get_meta() meta file does not exist '/home/pi/data/phpfina/16.meta'
2016-04-06 18:50:28.906|ERROR|feed_model.php|EngineClass() Engine id '1' is not supported.

But the file did exist on the file system...

So in short the first issue submited by @gsiviero is still here for me.

Apart from this i would be glad to have a bit of documentation on how to use the new MQTT based input system (witch apparently completly replace the Node module by now ?). Also a simple tutorial about NodeRED and OpenHAB would be welcome (i do not really understand what they are for).

With OpenHab on the emonPi I successfully:

- read emonPi temperature and power sensors

- read emonTh temperature sensor (waiting for feeds to work to add pulse count)

- control Philips Hue lights

- control Kodi (see status and control play/pause and other functions)

- see network webcam video feeds (with motion installed on different Raspberry Pis)

- see if a given Android phone (or other device) is connected to the local WiFi (LAN)

All of this both from LAN and the Internet (port forwarding needed), and from my android phone.

And that I think is just the beginning... In fact you can add also "rules" (do "this" if "that" happens), and add numerous other "bindings" (control other "IOT" devices that have an OpenHab binding available).

It's a bit long to configure initially, but very interesting.

Sorry, Glyn, but I have to challenge the 4GB SD card minimum requirement - I just tried to write the expanded .img file to a 4 GB SanDisk micro SD card I have laying around, after verifying the md5 sum, and I could not do that.

I tried to write this new image the same I wrote the image on the SD card currently in my EMONPI - on windows 7, using win32 disk imager 0.9.5.

It now says: Not enough space on disk: Size: 7821312 sectors Available: 7744512 Sector size: 512

And since I keep my image files in the same directory, I noticed this: image from 18Dec2015 - 3.67GB, image from 22Dec2015 - 3.67GB, current image 29Mar2016 - 3.72GB

I shall now use a bigger card for my EMONPI, since I need it working asap, but wanted to give a heads-up.

Thanks, and keep up the good work!

Plamen

@Plamen Thanks for testing. It should fit on most 4GB SD cards. I think you have just been unlucky that your card was smaller. I will trim the image file down so it will fit on your card. 

@gsiviero wow, sound like you have been busy. Great to hear. 

@Paul I wil investigate the update terminal freezing issue. I think this is due to an incomplete updated happening in the past. I agree this needs tri be fixed, 

Another update on the "Engine id '1' is not supported." issue.

When I create a new feed I get

2016-04-08 09:25:49.817|WARN|PHPFina.php|get_meta() meta file does not exist '/home/pi/data/phpfina/34.meta'

then the "wait..." in the Inputs view.

If I edit again the feed (process list setup) I get "feedid 34 does not exists or was deleted".

But if I go to /home/pi/data/phpfina/ I see that the file 34.dat keeps growing, so data is added to it.

It looks like the problem is somehow in the web interface that cannot find or visualize the feed, but the feed seems to be working:

-rw-r--r-- 1 www-data www-data 260 Apr  8 11:38 34.dat

-rw-r--r-- 1 www-data www-data 304 Apr  8 11:40 34.dat

-rw-r--r-- 1 www-data www-data 376 Apr  8 11:42 34.dat

I don't know if this finding could be useful.

This issue should now have bee fixed as of four days ago. Are you running latest software? Run emonPi update: 

https://github.com/emoncms/emoncms/commit/7cdbaf13510fc4e9043a724453bb4c...

Please confirm if latest update fixed for you. Recommend a reboot after update

using emonpi v1.6 on raspberry pi 3, with no LCD, 

• I have the same update issue as mentioned above
• I also cannot power off the device using the button, holding for 5 seconds, no matter how many seconds does nothing.

great build on the whole though! any thoughts on the powering on and off?

Pressing the web interface "Update now" button did nothing.

In fact from /var/log/service-runner.log :

STARTING: /home/pi/emonpi/service-runner-update.sh > /home/pi/data/emonpiupdate.log
/home/pi/emonpi/service-runner: riga 20: /home/pi/data/emonpiupdate.log: Permesso negato
COMPLETE: /home/pi/emonpi/service-runner-update.sh > /home/pi/data/emonpiupdate.log

I had to update from the command line:

sudo  /home/pi/emonpi/service-runner-update.sh

Now the error "meta file does not exist" does not appear anymore, but I still have the error

2016-04-08 11:06:19.719|ERROR|feed_model.php|EngineClass() Engine id '1' is not supported.

and still cannot add feeds. I tried also with "Clear browsing data" (Chrome) but nothing changed.

Note: After the command line update now the "Update now" button works correctly.

EDIT:

from /var/log/apache2/error.log

[Fri Apr 08 13:19:29.508538 2016] [:error] [pid 6849] [client 192.168.1.9:54791] PHP Fatal error:  Uncaught exception 'Exception' with message 'ABORTED: Engine id '1' is not supported.' in /var/www/emoncms/Modules/feed/feed_model.php:71\nStack trace:\n#0 /var/www/emoncms/Modules/feed/feed_model.php(411): Feed->EngineClass('1')\n#1 /var/www/emoncms/Modules/feed/feed_model.php(294): Feed->get_timevalue('1')\n#2 /var/www/emoncms/Modules/feed/feed_model.php(268): Feed->redis_get_user_feeds(1)\n#3 /var/www/emoncms/Modules/feed/feed_controller.php(38): Feed->get_user_feeds('1')\n#4 /var/www/emoncms/core.php(64): feed_controller()\n#5 /var/www/emoncms/index.php(146): controller('feed')\n#6 {main}\n  thrown in /var/www/emoncms/Modules/feed/feed_model.php on line 71, referer: http://emonpi.lan/emoncms/
[Fri Apr 08 13:19:30.143287 2016] [:error] [pid 1372] [client 192.168.1.9:54794] PHP Fatal error:  Uncaught exception 'Exception' with message 'ABORTED: Engine id '1' is not supported.' in /var/www/emoncms/Modules/feed/feed_model.php:71\nStack trace:\n#0 /var/www/emoncms/Modules/feed/feed_model.php(411): Feed->EngineClass('1')\n#1 /var/www/emoncms/Modules/feed/feed_model.php(294): Feed->get_timevalue('1')\n#2 /var/www/emoncms/Modules/feed/feed_model.php(268): Feed->redis_get_user_feeds(1)\n#3 /var/www/emoncms/Modules/feed/feed_controller.php(38): Feed->get_user_feeds('1')\n#4 /var/www/emoncms/core.php(64): feed_controller()\n#5 /var/www/emoncms/index.php(146): controller('feed')\n#6 {main}\n  thrown in /var/www/emoncms/Modules/feed/feed_model.php on line 71, referer: http://emonpi.lan/emoncms/input/view

@Iain the shutdown script is part of the LCD script. If the emonpi detects no lcd connected the lcd script (including shutdown) does not run. I will look into creating a sparate shutdown script to run when no lcd is connected. However, almost all the emonPi's we sell include an lcd. Not many users have chosen to use just the emonpi pcb. If you want to use just a shutdown script there is one available in the emonpi LCD folder We are investigating the meta data and feed engine issue.

Hi !

I confirm that:

• the "update now" button was not doing anything. In fact, it was very painful to try to read the corresponding logs inside emoncms admin module, because this log is refreshed to the end of the logs all the time (its scrolled back to the end at each refresh).
• thanks to @gsivierot I was able to update with`sudo  /home/pi/emonpi/service-runner-update.sh`.
• after this update, the "update now" button is now working. The refresh problem is also corrected.
• the feeds still do not work after this update because of `EngineClass() Engine id '1' is not supported`

I use an emonBase... so no need for the emonPiLCD service. But this service is started anyway at the end of the update script, and at boot up. (service emonPiLCD status tells me it's active).

This give me an emoncms admin module suggestion: a module where it's possible to activate/deactivate optional services : emonPiLCD, NodeRED, OpenHab, LightWaveRF. And why not also check the status of necessary services like redis, mysql, mosquitto, emonhub?

@miniwark,

If you have data you don't mind losing, could you try and perform a factory reset and let me know if the issue persists?

sudo su

emonpi/./factoryreset

Glyn - There may be an issue with the EditRealTime.  I tried to delete a spike from July 15, 2015 at 2:54:50 PM CDT and it did not work.  When I finally looked at the zoomed out graph, the EditRealTime ended up changing a value on March 18, 2016 at 10:21:59 PM CDT.  This is the time I was doing the editing. I have two spikes to fix.  How do I delete bad data in my kWh feed?

If this belongs in a new thread please let me know and I will delete this and start a new thread.

@glyn.hudson

I have done the factory reset and reboot:

1) after the reset, emoncms could not connect to the database: manualy restarting the mysql service have solved the problem (the restart script said than mysql was 'not closed cleanly'). By the way what is the default password for mysql root user ? emonpi2016 and emonpiemoncmsmysql2016 are denied.

2) an update for the database was recommended using the admin module (dashboard table)

3) after updating the EmonHub config file, my emonTX v2 node was detected with the correct fields. By the way, what is this rssi field (field with negative value around 40) ? I do not have defined this field in my node firmware and emonhub config.

4) i was able to create a new feed from the input module.

5) the feed now appear correctly in the feeds module and is usable in dashboards :)

Conclusion: the factory reset have solved the problem (but may be updated to avoid the mysql error message problem after reboot). Thanks a lot for your help!

A few final remarks: (but may be specifics to emoncms and not the image)

• after setting the correct local time with raspi-config and inside emoncms, emoncms.log is still at GMT+0
• please update the description with the root mysql password (see above)

@miniwark the mysql 'root' user password is 'emonpimysql2016' and mysql 'emoncms' user password is 'emonpiemoncmsmysql2016'

1. Where did you see the mysql error? I have not seen this error or had issues connecting to DB after factory reset. Here is the MYSQL drop code, can you see how it can be improved? 

https://github.com/openenergymonitor/emonpi/blob/master/factoryreset#L92

2. The emonPi update script should auto update the database. I would recomend running emonpi update after factory reset. 

3. RSSI is Received Signal Strength Indicator it's a signal strength indicator. The less negative the better the signal. If you move the emonTx further away the number should decrease (get more negative). 

4. Great! 

Is the time showing correct on the graphs? Is it just in the log time is incorrect?

Thanks for your help. 

@Jon Just tested realtime edit and it seemed to work fine for me. I'm running Emoncms V9.4. When you click the graph to inform RealTime edit the time location of the data point you wish to edit make sure your click was registered and the 'Edit FEED-NAME @ time:' shows the time string of the data point you wish to edit

Glyn - should I see the data change immediately on the chart?  Or do I need to clear the cache and refresh?  How do I know the data changed?  I've tried Firefox, Safari & Chrome and they all seem to do the same thing...  (and yes I am clicking on the data point I want to change and I can see the current data at that epoch time)

EDIT:  OK it seems to take a little while.  Is the data stored as increments and the fina engine needs to recalculate the entire set of data?

Hmm Seems to show up immediately for me, testing on emonPi 

me too

OK, this is probably an FAQ, but if I use the standard low-write image like this one but then add spinning storage like a WD pi Drive later is there a "big switch" where I can enable all the disabled features like PHPFIWA etc. ?

I modified the factoryreset script to obtain a "DB reset script" to reset only
- mysql emoncms database
- phpfina and phptimeseries data

Now I can add and modify feeds again.

I'm afraid that the previous time there may have been an error when I deleted or modified some of the feeds and the db may have lost the correct feed-ID <-> feed association or something like that.

Can anyone point me to a thread or instructions to create a "pulse-count to power" feed, if this is even possible using the "process list" interface?

Thanks

(1) Sorry if I missed when this happened, but have reporters in emonhub.conf been replaced by new types of interfacers?

I am accustomed to Type = EmonHubEmoncmsReporter to post to my own Ubuntu-based emonCMS server, but I see that the deafult emonhub.conf file in this new release candidate is using EmonHubEmoncmsHTTPInterfacer to get to emoncms.org.

(2) I've been using the emonSD-17Jun2015 image to run a low-write emonHub on a Raspberry Pi. Is this viewed as a successor to that? In particular, the emonSD-17Jun2015 version was not a local emonCMS server (as far as I know), but this clearly is. I like the idea of preserving the longevity of the SD card in the Pi by posting all the data to a server elsewhere.

Thanks! I'm having fun checking out OpenHAB and NodeRED.

On the 4GB card subject.... I have just tried flashing it the to Transcend 4GB card *supplied with* the emonpi I bought and it's still too big. Considering I thought I was supposed to get an 8GB card anyway, but that's by the by...

@Peter I'm afraid there is no 'big-switch' to revert to a non low write image. However I would recommend sticking with the low-write image. I cannot see any advantages of changing. What missing features are you after? The low write optimizations will increase the life span of your disk. 

@gsiviero see here for how to log pluse count to feed https://openenergymonitor.org/emon/opticalpulsesensor

@baldockery 

1.) yes the RC image uses the 'emon-pi- varient of emonHub which supports MQTT. See here for config info https://github.com/openenergymonitor/emonhub/blob/emon-pi/configuration.md. This version of emonhub was also used on the 17Jun2015 image. 

2.) Yes, this is a successor to this 17thJun image. I would recommend using this image if you want Emoncms V9 and latest features e.g openHAB, nodeRED built in. No, incorrect the 17thJune image DOES also run local Emoncms V8 by default. We have optimised the low-write so there is no problem running Emoncms locally. However if you wish you can just post data to Emoncms.org and not log any Inputs to feeds locally. 

@ Peter It's on the list to address the size of the image for the next release to fit on all 4GB sd card sizes. Only the recent SD cards from the shop are 8GB. When was your emonPi ordered? Drop an email to shop support they will happily send you out an 8GB SD card.   

Glyn,

In terms of a non low-write version, maybe I'm just not spending enough time understanding emoncms but most of the functions are missing and I have assumed they are missing because they depend on the phpfiwa functionality - if it's simply a case of more git clones/pulls to add the full set of functions then I'm happy. I have now got a separate Pi 3 and I have a Pi Drive on order so I can "play" with emoncms on Pi (and other stuff) without trashing what was my only Pi. My bad.

As long as I have a live data back up schedule in place *I* don't care about SD card lifespans. I do understand that many others do, especially for less accessible and more critical applications of the emonpi.

The Transcend card was 5MB too small - but to get me out of a hole, since I was the idiot that trashed my original install - I just went back to the earlier release candidate and that works fine. I have now ordered a couple of 64GB cards both for the emonpi and my other Pi3 as above. At today's prices (£12.55 for a Transcend Class10/U1 and £24.99 for a SanDisk Extreme Class10/U3) it's not worth faffing.

@glyn.hudson

Thanks for the MySql root user password.

1) The error message was a generic error "could not connect to the database" message by Emoncms.

At the end of the database reset, just restart the MySQL server. Or maybe use a FLUSH statement. I think the issue may be related to incomplete syncing of the MySQL internal caches.

2) OK.

3) Thanks, than sure is an useful metric.

4) :-)

> Is the time showing correct on the graphs? Is it just in the log time is incorrect?

Yes the time is showing correctly in the graphs: in the visualization extra, and in the 'realtime' graph widget ins the dashboards.

Only the logs display a wrong time.

One minorish thing, but the emonSDexpand is very slow compared to the almost instance NOOBS resize that runs at their install time. I am not sure what the latter does, but the one I started on a 64GB card a while back is still running resizefs

 I believe there could be some confusion with regard to the old low-write branch on github. Low write optimisations have now been merged into Master and the new emonPi (this image) runs the Stable branch which is a stable snapshot of Master. All features are included.

That's very good to know. I now have a copy of this RC installed on a shiny 64GB card, filesystem expanded and emonhub.conf installed - just got to get home from work and install it. Can't see any nodes on this standalone Pi so I can fiddle with feeds yet.

Right, that's almost all good. All the functions are visible, thanks.

I am seeing two problems "out of the box";

1. /var/lib/php5/sessions is not a symlink to a writable directory and so generates errors. I don't see it mentioned in the /var/log set-up in /etc/rc.local. Is it supposed to be taken care of by any other mechanism?

2. The phpmqtt_input.php script is constantly going in a connect loop:

2016-04-15 18:09:15.561|WARN|phpmqtt_input.php|Connecting to MQTT server: Connection Accepted.: code: 0
2016-04-15 18:09:15.562|WARN|phpmqtt_input.php|Subscribed to topic: emon/#
2016-04-15 18:09:16.619|WARN|phpmqtt_input.php|Starting MQTT Input script
2016-04-15 18:09:24.731|WARN|phpmqtt_input.php|Subscribing to: emon/#
2016-04-15 18:09:24.731|WARN|phpmqtt_input.php|Not connected, retrying connection

I am going to hack the php sessions directory in for now. In php.ini I am changing the save_path and then creating /var/log/php5 and /var/log/php5/sessions in /etc/rc.local - I think that's about right. But I recall /etc/rc.local is created from something else?

Edit: Old version used /tmp for sessions. Would try it but my emonpi crashed and wouldn't reboot, so restored old SD card for tonight.

Where are you seeing the . /var/lib/php5/sessions errors? Are you using this RC2 SD card image out of the box? 

I did an apt-get update / upgrade while preparing the card, so it may once again be my fault. It was in /etc/php5/apache2/php.ini I think.

I am going to start from scratch tomorrow, no mods and see.

I have just tested an apt-get upgrade and when updating apache it gave me the option to keep the local version of /etc/php5/apache2/php.ini of upgrade to the package maintainers version. I checked out the difference and the best option will be to keep the local copy. The modifications made to php.ini are critical to emonPi operation.  

Thanks. I've gone back to a raw image, no updates and just grown the file system. All is well in stupid user land. Sorry.

The phpmqtt errors continue, but I've not looked if that needs further config to start working.

What mqtt input error are your experiencing? Does a restart of the service or system reboot fix? 

Sorry, back now. The phpmqtt_input.php errors continued after a clean reinstall but then stopped eventually with no intervention by me, last seen:

2016-04-16 08:32:41.258|WARN|phpmqtt_input.php|Not connected, retrying connection

This time I can say all I think have done is make an image, boot with Ethernet attached and connected to web interface, registered user, added local emoncms and added API keys in emonhub, changed ssh password and grown filesystem. I have tried hard not to fiddle. This time.

I may reboot at some point and see if the connection errors for phpmqtt come back.

All else is well, thanks for your patience!

Earlier in these comments I was under the impression that emonhub low-write didn't run a local Emoncms instance. I discovered why. Apparently when I set up my emonhub in July of 2015 I went with the emonSD-13-03-15.img image that provided me with "emonHub Pre-Release Development Version (rc1.2)". Why, I don't know, since the 17th Jun image would have been available to me at that time.

In any case, on my version the following are installed but not running: apache, mysql, redis-server, mosquitto.

So, my question now is, what is the best upgrade path for me? My emonhub.conf file will clearly need to look very different with the new MQTT-based scheme. I'd like to:

1) Avoid significant down-time where I missing collecting solar and use data.

2) Have a plan to revert to my current stable setup if the upgrade is going poorly.

My current best thought is to use another spare Raspberry Pi, buy another RFM69Pi hat for it and run it in parallel with my current setup until it appears to be working, then take down the current setup. But if there is a better way ...

Thanks much, 

So I'm presuming you don't have any local Emoncms data to backup since your current image is not running local Emoncms. Is all your data stored in Emoncms.org? 

If this is the case then you can just fire up the new image on a fresh SD card and paste in your Emoncms.org API key into the new emonhub.conf. 

Depending on what remote nodes you have set up, you many need to modify emonhub.conf with the correct node decoders to match the RF data being transmitted by your nodes. See here for info on configuring emonhub node decoders: 

https://github.com/openenergymonitor/emonhub/blob/emon-pi/configuration.md

If you want to begin logging to local Emoncms you would then log the required inputs to feeds in local Emoncms

On the subject of "EngineClass() Engine id '1' is not supported." errors, I did a

delete from feeds where engine = 1;

in mysql and rebooted. All was well after that.

I am trying to work this out for myself but given up as I am not a Linux expert. I have a stock EmonPI is their a guide to upgrading from June15 to release candidate?

Paul

Hi Paul

Bit out of date but should still work 

https://openenergymonitor.org/emon/node/11407

Steve

I would recommend starting from fresh with this new image on a new SD card and using the Backup Module to migrate your data:

https://github.com/emoncms/backup

https://www.youtube.com/embed/5U_tOlsWjXM

One minor comment; All user's .bash_history needs to be deleted before building a distribution - it's full of your old commands. Certainly root and emonpi have them.

There may be a problem with the emonPi LCD service.

Every time I reboot the emonPi the display remains lit.

From the log it looks like the service is loaded at startup:

Apr 20 21:18:49 emonpi systemd[1]: Starting LSB: emonPi LCD controller...

Apr 20 21:18:49 emonpi emonPiLCD[418]: Starting system emonPiLCD daemon:.

Apr 20 21:18:49 emonpi systemd[1]: Started LSB: emonPi LCD controller.

If I check it:

sudo service emonPiLCD status
* emonPiLCD.service - LSB: emonPi LCD controller
   Loaded: loaded (/etc/init.d/emonPiLCD)
   Active: active (exited) since Wed 2016-04-20 21:18:49 CEST; 1 day 11h ago
  Process: 418 ExecStart=/etc/init.d/emonPiLCD start (code=exited, status=0/SUCCESS)

Apr 20 21:18:48 emonpi systemd[1]: Starting LSB: emonPi LCD controller...
Apr 20 21:18:48 emonpi emonPiLCD[418]: Starting system emonPiLCD daemon:.
Apr 20 21:18:49 emonpi systemd[1]: Started LSB: emonPi LCD controller.

And when I restart the LCD service with

sudo service emonPiLCD restart

I find that the service is in fact not running:

Apr 22 09:11:26 emonpi systemd[1]: Stopping LSB: emonPi LCD controller...
Apr 22 09:11:26 emonpi emonPiLCD[20444]: Stopping system emonPiLCD daemon:start-stop-daemon: warning: failed to kill 446: No such process
Apr 22 09:11:26 emonpi emonPiLCD[20444]: No process in pidfile '/var/run/emonPiLCD.pid' found running; none killed.
Apr 22 09:11:26 emonpi emonPiLCD[20444]: failed!
Apr 22 09:11:26 emonpi systemd[1]: Starting LSB: emonPi LCD controller...
Apr 22 09:11:26 emonpi emonPiLCD[20451]: Starting system emonPiLCD daemon:.
Apr 22 09:11:26 emonpi systemd[1]: Started LSB: emonPi LCD controller.

Only then:

sudo service emonPiLCD status
* emonPiLCD.service - LSB: emonPi LCD controller
   Loaded: loaded (/etc/init.d/emonPiLCD)
   Active: active (running) since Fri 2016-04-22 09:11:26 CEST; 7s ago
  Process: 20444 ExecStop=/etc/init.d/emonPiLCD stop (code=exited, status=0/SUCCESS)
  Process: 20451 ExecStart=/etc/init.d/emonPiLCD start (code=exited, status=0/SUCCESS)
   CGroup: /system.slice/emonPiLCD.service
           `-20456 python /usr/share/emonPiLCD/emonPiLCD.py

Apr 22 09:11:26 emonpi emonPiLCD[20451]: Starting system emonPiLCD daemon:.
Apr 22 09:11:26 emonpi systemd[1]: Started LSB: emonPi LCD controller.

Anyone else having this problem?

All installed and now up and running, seems ok so far.

A minor comment on guide post:

On MQTT password change

"~/oem_openhab/configuration/openhab.cfg" should read "~/oem_openHab/openhab.cfg"

Is the capitalization intentional?

" /etc/openhab/configurations/openhab.cfg" is a sym link to "~/oem_openHab/openhab.cfg"

@ peter thanks, I have added clearing full bash history the the emonPi factory reset script which gets ran before making an image

@ gsiviero  is the issue that the LCD does not restart when the system is restarted? I have not been able to recreate

@Eno thanks. The config can actually be edited in either location, but I agree, it's clearer for the user to edit the source symlink location. The capitalization of ~/oem_openHab is required.

This may be more a 9.4 thing than a emonpi thing I think, but since it's part of my testing of the image...

I am getting odd maths / sign changes in my solar + grid = home calculation.

I noticed that my home power was going negative occassionally. Looking back through the feed it appears to have starts on Saturday morning. I am not aware of - but maybe did - make changes.

My input for "Solar Power" does:

...
9. Reset to original
10. If > 40, skip
11. Reset to zero
12. + input Grid Power
13. Log to feed Home Power

I see this in the feed view (ticking the other boxes, nice feature):

Date Time,Solar,Grid,Home

2016-04-23 08:24:20, 1484.0, -931.0, 607.0
2016-04-23 08:24:30, 1452.0, -885.0, 521.0
2016-04-23 08:24:40, null, null, null
2016-04-23 08:24:50, null, null, null
2016-04-23 08:25:00, 1504.0, -961.0, 619.0
2016-04-23 08:25:10, 537.0, -24.0, -424.0
2016-04-23 08:25:20, 511.0, -25.0, 487.0
2016-04-23 08:25:30, 510.0, -2.0, 485.0

Similar negatives continue every few minutes.

Only difference I can see is my existing emoncms (also claiming to be 9.4) is using PHPFIWA while on the emonpi they are PHPFINA.

Am I just missing the way the system handles preceding nulls?

Update: actually now noticed that "-424" is 537 + - 961 from the previous sample. Is this the way it's supposed to work with the change in the feed and is this another instance of me not reading some crucial doc about PHPFINA?

Yes looks like solar pv app issue which has just been updated. Please post github issue
https://github.com/emoncms/app

Glyn,

Are you replying on my question? That's the feed, long before the mysolar app reads the data... I am guessing a timing issue. I have only been trying the emonpi for processing data for a couple of weeks or so. Until now I have always used emoncms.org or a Linux server; This could be a performance issue, given it calculates against a delayed data item in another data item's input?

Edit: I did notice a big chunky change being applied to PHPFina for averaging support, so I am updating and rebooting for now...

Edit2: Now I think about this more I think it's simply a consequence of processing the data on the same system as it's collected; When you send data to another emoncms instance it is groups together into a write-per-interval and all the values are updated at the same time. When locally processing the data one of the data points will always lead or lag another. Is there any way I may have missed to lock-step each input cycle and run the processing at a fixed interval?

Hi Peter, The example you give would seem to be correct, if "grid" uses a lower CT channel number than "solar" and is therefore before it in packet arriving into emonhub.

The reason you are seeing a change when moving to an emonPi image is that the emon-pi variant of emonhub passes the individual values one by one rather than passing them as a group (AKA node). 

Previously the group of value would arrive at emoncms (by http) and then get processed, now what you are seeing is each input arriving at emoncms and being processed individually and independently of each other.

You will get better results if you rewrite your processing to the solar input and moving forward only use the inputs and feeds of preceding inputs (according to the original packet order) in any given processlist and hope that the order they arrive at emoncms doesn't change eg a momentary network glitch, any apache, mosquitto or sql server house keeping etc as there is no mechanism to ensure the order they arrive or get processed.

Paul

Thanks. I was considering trying that - move the processing down to the Grid CT, one CT below. I think this worked on day one because I originally had the solar CT as the high sensitivity one (CT4) but subsequently moved both the processing/storage to a Linux instance and also moved the CT to CT1 when the solar output looked like it would exceed the limits of CT4 on the emonTX.

Update: Yup, that was it. All working now. Is this an FAQ or should it be?

Ah sorry when you said 'my solar' I thought you were referring to the 'MySolar' app! Apologies. 

Not sure if this is an RC2 thing, but having tried to install a node-red node using "npm install -g" the node ends up in /usr/loca/lib/... and not /usr/lib/...

Even if this is correct for user installed nodes, it still didn't appear and after removing it and installing it again without the -g I still could not see node-red-node-forecastio after a node-red restart and also a reboot just to be sure. Is there a different supported way to install new nodes and flows using npm?

Edit: Found the non -g install under /home/pi/node_modules/ - when I think it should be /home/pi/data/node-red/node_modules ?

Node red is ready installed on this images via the Jessie sources. This is the recommended way to install NodeRED node, not via NPM.

The nodeRED user directory is in /home/pi/data/node-red  this is where user modules should be installed. See readme on:

https://github.com/openenergymonitor/oem_node-red

Peter - I've only been experimenting with Node-RED for a couple of months so I am not an expert. But this is how I install a new Node-RED node from the Node-RED library http://flows.nodered.org onto the emonPi.

cd /home/pi/data/node-red
rpi-rw
npm install node-red-node-forecastio
​# do not use the global option (e.g., npm install -g node-red-xxxxx) 
rpi-ro
sudo service nodered restart

Once in awhile I will need to prefix the "npm install node-red-xxxxx" with do a "sudo " but I don't know why.  I am guessing it is related to a dependency that is also installed with the new node.

All of your new Node-RED nodes should now be installed at: /home/pi/data/node-red/node_modules

The recommendation for not using the global option (-g) came from the Node-RED Google Group at https://groups.google.com/forum/#!forum/node-red. I cannot find the post at this moment.

Thanks. I also shoved a symlink in /home/pi to protect myself from forgetting to cd into data/node-red.

I'm using the RC2 image after expanding on an 8GB card, on a Pi with the RF hat from the shop, and after doing a Pi update using apt-get. Warning: I'm a novice user and so may be missing something.  I've set up a number of inputs and feeds and they all seem to be working okay or as I would expect anyway.  I have successfully connected to the Pi using SSH and have logged into the web interface of my local Pi.  A couple of times today whilst 'watching' the inputs page and after a few hours attached via the web interface, I have noticed that the inputs have seemingly stopped logging.  When trying to perform any task (such as adding a new feed) I get an error message ('undefined', I think, sorry did not note it).  If I logout, it seems to do that okay and I get the (re)log in screen okay, but if I try to login again a banner appears on a red background in the login box saying 'undefined'.  I don't seem to be able to login.  The SSH connection still seems to be working and I can do an orderly 'sudo halt', restart the Pi and am then able to login into the web interface again using the same credentials. As I say this has happened twice today.  I'm just playing around at the moment, so it's not a major issue for me, but maybe there is a problem - or am I doing something silly? I've tried different browser tabs etc. If I go to the Pi's IP address (192.168.2.2, fwiw), I get the login screen but my credentials do not work until I restart the Pi.

rtX - the discussion on the Tips n Tricks from my own Pi appliance... thread points the finger at the "commit=180" entry in /etc/fstab causing the issues once apt-get upgrade is used, you could try manually editing and rebooting.

Paul

Thanks Paul, Yes this is a known issue and will be fixed in the next sd card release (posted later today) 

https://github.com/openenergymonitor/emonpi/commit/fd145f0797769fd3af082...

Take a peak at the change log for the updated image at the bottom of this page:

https://github.com/openenergymonitor/emonpi/wiki/emonSD-pre-built-SD-car...

New image has been released. Please post on thread on the new forums 

https://community.openenergymonitor.org/t/emonsd-03may16-release/145

Thanks for the help, Paul.  I have been using the June 2015 image just to get things working etc.

I see now a new image is available (3rd May 2016).  Is this an RC or an actual release? Many thanks for all the hard work.

This *should* be an actual release. Please post on the new thread on the new forums:

https://community.openenergymonitor.org/t/emonsd-03may16-release/145