Archived Forum

Hi Eric,

Interesting question and a tough decision :-)

Firstly I would say definitely use oem gateway / emonhub which ever option you take as there are no benefits to not using it and it does provide data buffering so short network outages will not be a problem.

The read-only sd card option is also more robust and quicker to boot if power outages are frequent, it also draws less current if a long power outage is encountered. 

During power outages will the network automatically be down or is the network connection power independent?

How frequent are the power outages? If the power cycles rapidly this is more likely to cause hdd failures and if hdd does fail then no further data can be collected as the pi won't boot. also all previous data including periods that were unable to send to emoncms.org may be lost.

The obvious downfall of the oem gateway / emonhub option is any longer periods of network / power outages will result in some lost data as the data is held in RAM so when the back-up power supply fails the buffered data is lost and also the RAM can only buffers so much data.

The buffer size can be altered so you could experiment with increasing the allocated size and also keep data sizes and sample intervals to the minimum needed.

Overall I think the oem gateway / emonhub option could be the better option unless very prolonged network outages with minimal power interruptions is more likely than a prolonged power outage or random short outages of both/either.

Food for thought...

Paul 

Hi Paul,

Thank's for your reply.  It gives me some good clues.

I guess the network connection is power independent.  Power outages should be quite rare as the building is connected to the grid which is reliable in France.

Did you notice that I'm planning to use a power inverter ? (only for the RPI and the hdd, not for the network)

Eric

Also a possibility is to add a UPS to the equation. I imagine a cheap-ish UPS can keep a Pi running for quite a while. And in the event of a real long outage, software like apcups can shutdown (and powerup) the Pi gracefully. My BackUPS CS 350 cost somewhere around 80 Euro's and is fully monitorable via USB.

I had noticed but was unclear as to why. A "power inverter" suggests you are intending to produce high voltage AC from a DC source, presumably a battery in this instance. That AC will then need to be reduced and rectified to supply the pi & hdd.

A UPS is a better "out of the box" solution but you are still outputing higher voltage AC and then reducing and rectifying to 5v DC.

If you are just wanting a reliable power supply for the pi & hdd, why not just run them direct from a battery (using a DC voltage dropper if required) and charge that battery with an intelligent charger continuously or when suitable ie when power is up, off-peak only or when solar PV available. This would eliminated the need for any "power outage switchover" and smooth any power fluctuations giving an ideal solution for running the hdd.

what are you measuring and with what? emontx may need a supply, also is there any data worth reporting during a power outage? How will the node(s) connect to pi? rf uses more power but direct connection limits the number.

emontx v3 can be powered via ac-ac adapter but will stop functioning when power is down, not really a major problem unless v3 is also measuring something that needs reporting during a power outage. v3 can be powered via 5v or battery but will still need ac adapter for accurate power/voltage reporting. The power inverter wouldn't help here either as you want to measure the buildings supply not the inverter output.

What type of network connection are you using? again wifi will require more power as will a mobile modem device.

Paul

I would like to measure :

   - 16 electrical powers with 4 EmonTX V3, each one powered by an AC/AC adapter. If the main power is off, they have nothing to measure.

   - 4 temperatures using 2 EmonTH. These records can be lost for a moment.

All nodes will send data to the Emonbase using RFM12B. The EMonbase will be connected to internet through a LAN (Ethernet).

Sorry for my english. By "inverter", I mean what you call an UPS, a device that can provide emergy power when the main power fails.  With an UPS, the RPI+hdd solution seems appropriate if power outages are short which is probably the case.

But, if I can't get an UPS, the OEM gateway option seems better. Some additional questions about that solution :

   - Is it possible to forward data simultaneously to two different remote EmonCMS server ?

   - Is there a maximum recommended buffer size ? How long it will  approximately last with 20 record every ten seconds ? What happens when the buffer is full ?

Regards,

Eric

Emonhub (or oem gateway) can send to multiple emoncms instances. I have had a test set up with 2 remote and 2 local instances all being fed simultaneously. 

the default size is 1000 frames of data so 6 nodes (4 emontx's & 2 emontx's) producing 6 frames a minute is 36 a minute or 2160 an hour. you would need to run some tests but the default should be good for around 1/2 an hour or try increasing the value and check RAM usage. the number of frames doesn't consider the size and number of values in a frame but remember though the memory is limited on a Pi and if you are posting to 2 CMS's there are 2 buffers.

this calc is theory so i would recommend testing before relying on it.

Can you use the temp sensor inputs on the v3's each of the 4 v3's could record a temp without extra nodes, less expense, less rf trafffic (collisions), less power used and less frames of data. the additional headroom required for the node id and time stamp would increase the potential buffer size as well.

When the buffers limit is reached it will start replacing the oldest data first so the buffers always retain the latest data. 

I notice your original post says pi+hdd or pi+gateway or both (2 pi's) emoncms and emonhub (oem gateway) will work together on same pi no problem, so you can have local hdd and buffered remote from 1 pi.

Paul

Hi,

Thank's again for your information.  Very instructive !  I will test to increase the buffer size and tell how it affects the RAM.

You said : emoncms and emonhub (oem gateway) will work together on same pi no problem, so you can have local hdd and buffered remote from 1 pi.    According to my understanding (correct me if I am wrong), if the hdd fails, no further data can be forwarded as the pi won't boot. That why I was thinking using two Pi:

    - One Pi with a hdd running a local EmonCMS  (it can work during internet outages but can fail after a power outage)

    - A second Pi  running a rock-solid-gateway that will forward data to a remote server (it shouldn't fail after a power outage because it  uses a read-only file system but it need an internet connection)

Is it possible to have both approach on the same Pi ?

Regards,

Eric

If emonhub is installed to hdd with emoncms, the pi will indeed fail to boot following a hdd failure. that advantage of using emonhub with emoncms is the option to post locally and have a buffer for remote posting to reduce loss of data during short network outages.

The issue with running 2 pi's and hdd from a UPS is that you are massively reducing the length of time the UPS will last, which actually could increase the possibility of failure.

If there is nothing or very little to report during power outages, maybe the read-only solution without a UPS is sufficient. the only data loss potential is if the network goes down for longer than the buffers can cover. Only the data produced at the beginning of the outage up to the earliest retained record is at risk ( eg if buffer can hold 4hrs and network is down 5hrs only the 1st hour is lost)

the ups, hdd, possible 2nd pi etc will only extend the data retention during prolonged network outages but could be prone to failure during prolonged power outages and if you are really unlucky the hdd may fail and no further data will be collected. in this instance the primary objective of the ups is only to reduce the chance of hdd failure as there still won't be data to send. 

Either method has holes, either some data loss during prolonged network outages or a possible hdd failure during prolonged power outages. The first is more likely to happen but the damage is limited to a small amount of data where as the later is unlikely to happen but if it does it's curtains until you can replace the hdd.

I guess if it's important enough you could run 2 pi's, you could have 1 pi&hdd on a ups and the 2nd ro pi direct from the mains, that would ensure after a prolonged power outage the ro pi would still boot up if the hdd has failed, without imposing a load on the ups or just get a bigger ups:-).

Paul

Paul, your last message is a great summary ! Thank's.

Somewhere, it shows that there is lack of reliability in the EmonHub/EmonCMS architecture. On one side, we need a reliable power source and on the other side we need a reliable internet connection ... According to me, the coming developments should be focused on this subject rather than adding new features. When monitoring buildings, there is nothing worse than loosing data ...

Regarding my project, I think I will do what you suggest in your last paragraph.

Bye,

Eric

Hi,

For my local EmonCMS, I'm using the recommended image (2014-02-23-emoncmsraspberrypihdd_stack) found here.

I made some tests without UPS regarding the power outages. I unplugged then re-plugged the power of my USB hub to simulate a power outage. EmonCMS restarted without any problem. No data lost. I made that test several times and everything's seems OK. No hdd failure ! Maybe, it's because I'am using a recent USB hdd ?

On that image, one can post data to one remote EmonCMS server but there is no buffer in case of network outage.

   - How to to activate a buffer on that image ?

   - How to send data to two remote server ?

Bye,

Eric

Hi Eric

The likely hood of hdd failure is very small, power failures can increase the chance but it's still relatively unlikely, almost an acceptable risk. It's the impact IF the unlikely should happen that is usually unacceptable.

i wouldn't use that image because both the raspbian and emoncms softwares have been updated a fair few times since that image was prepared and it has the raspberrypi module installed which if you want to send to 2 remote severs and have a buffer you will want to install oem gateway/emonhub instead.

I would use this guide to install emoncms to hdd on fresh raspbian and where it says

sudo apt-get -y --force-yes install emoncms emoncms-module-event emoncms-module-rfm12pi

(if you do not need or want the event module or rfm2pi module delete them from that last line to suit)

only install emoncms

sudo apt-get -y --force-yes install emoncms

Then install the old oem gateway or the later emonhub to give you the buffering and multiple CMS's etc

Emonhub is still in development stages but it's still as reliable, only the feature set and module structure is being experimented with.

Paul

In your other thread  read-only oem gateway you mention you will not be able to access physically or by ssh. If you are unable to access by ssh is that because of not having a fixed IP ? if so does that mean the emonCMS/hdd install in not accessible either?

The hdd and full emoncms install seems like a big solution just as a back-up to fall back on for missing data after the monitoring period has closed. Maybe if RO emonhub was also able to post each frame to a data stick mounted RW as a text file this would give you a full set of data in csv to fall back on for missing data and save you the cost of the 2nd pi, hdd and ups.

Just a thought :-)

Paul

In your other thread  read-only oem gateway you mention you will not be able to access physically or by ssh. If you are unable to access by ssh is that because of not having a fixed IP ? if so does that mean the emonCMS/hdd install in not accessible either?

>>> The building I'm going to monitor has one fixed Internet IP adress delivered by its ISP. The Raspberry Pi will be connected to a LAN managed by a router. For the moment, I haven't permission to modify the configuration of that router to set static IP adressing and port forwarding. So, I will have no access to the RPI (nor ssh,neither hhtp, ...)

The hdd and full emoncms install seems like a big solution just as a back-up to fall back on for missing data after the monitoring period has closed.

>>> I agree with you. Now, I have no other solution to make the monitoring as reliable as possible. So, here is my plan : I have installed Emonhub+EmonCMS a RPI+HDD. Data will also be sent to a remote EmonCMS server. The HDD failure is clearly a high consequence low probability event. So I will give to the building occupants a second SD card. On that card, I have only installed Emonhub in RO. If the HDD failure happened, I will ask them by phone to put the second SD card in the RPI and then to re-power it.

Paul, I congratulate you for having developed straightforward installation procedure  (EmonHub, switch Raspbian to RO, and EmonCMS on a RPI+HDD). They all worked very well for me. Great job !

Maybe if RO emonhub was also able to post each frame to a data stick mounted RW as a text file this would give you a full set of data in csv to fall back on for missing data and save you the cost of the 2nd pi, hdd and ups.

>>> I really like this idea.  : small, simple, and cheap ! No port forwarding needed, no hd failure risk, no internet outage risk.

EDIT : it may be not necessary to post each frame to a data stick but only the one going out of the buffer when it's full. Then these frames will be reposted later using a separate phpp script.

Regarding the buffer size in Emonhub, I made some tests with the following configuration :
   - 3 EmonTX sending data every 10s
   - 3 EmonTH sending data every minute
   - Emonhub (dev version) installed on a RPI in read-only. Data forwaded to emoncms.org and a local EmonCMS (not on the RPI). No change to the default config (bufferMethod = memory, bufferSize = 1000)

I simulated two network outages (RPI's LAN cable unplugged then re-plugged):
First outage :
- duration 20 min
- data lost : no
- time to get back all data : 10 min
Second outage :
- duration 60 min
- data lost : yes, the one collected during the first 10 minutes
- time to get back all data : 30 min

Note that the RAM used increases after each internet outage. (see attached)

Hi,

I made some tests with another configuration :
   - 3 EmonTX sending data every 10s
   - 3 EmonTH sending data every minute
   - Emonhub (dev version)  and EmonCMS (last apt version) installed on a RPI+HDD.

Emonhub forwards data to a local EmonCMS (on the HDD) and also to EmonCMS.org.  No other changes to the default config (bufferMethod = memory, bufferSize = 1000)

In order to simulate a network outages, I unplugged the RPI's LAN cable and then I re-plugged it 10 minutes later. No data lost on the local EmonCMS server but ... I have no data on EmonCMS.org during the time of the network outage.

Any explanation ? Something to configure in Emonhub ?

Many thank's !

Glad the guides help!

The text file idea came from wanting to be able to see the data being posted by emonhub without all the other log messages or multiple entries for one frame of data, in a consistent layout so you can scan down the columns for rogue values.

Following this discussion it occurred to me that this could work as a simple back-up solution as well as a tool for diagnosing data. I think the overspill idea is good for your need but that would mean building it as an extension to existing dispatchers and would require some functioning logic which makes is less simple and possibly open to error, something that's less desirable for a back-up. a dedicated test/back-up dispatcher could also (in theory) cope with the usb stick being pulled out and the data file copied, queried, deleted or renamed etc and as long as it is plugged back-in before the buffer is filled, pick up where it left off. I did start hashing something together the other night but I'm unlikely to get back to it just yet.

There is a plan to add persistent buffering to emonhub at a later date which will expand the buffer size to the limit of the available storage but that may well need to be hdd only depending on disc activity level etc.

The initial test results sound good. (3tx * 6frame/min) + (3tx *1frame/min) = 21frame/min, a buffersize of 1000/21 = 47.7 mins, so 50 mins (60min outage - 10min data loss) sounds about right.

The logfile is also located in the  RAM on the RO version and that will steadily increase in size up to a max limit of 10mb (I believe)

Regards the loss of data in the second test, no I'm not sure why that would happen, did the data resume posting after the outage? was any clues in the logfile? The logic is that the if emonCMS confirms receipt of data then that data is deleted from buffer and the next batch is created or if no receipt confirmed then the dispatcher will keep trying until there is a confirmed receipt.

Could you try again but this time ssh in and change url from emoncms.org to something else eg xemoncms.org in the conf file, while the system is live, without restarting. that way you can see the logfile while it is unable to post. you should see the same line repeatedly tried and then when you redit the emonhub.conf back to emoncms.org see that same line result is a successful post.

Thanks for feeding back your findings it's most helpful.

Paul

Regards the loss of data in the second test, no I'm not sure why that would happen, did the data resume posting after the outage?

Yes

was any clues in the logfile?

No because logging is working only 10 seconds after Emonhub startup (I'm talking about /var/log/emonhub.log)

On the second configuration (RPI+HDD running EMonhub sending data to EmonCMS.org and a local EmonCMS on the Pi), I have some strange behaviour.

   - When restarting the Emonhub service, I have some bad input in EmonCMS because some frame a cut in two. As an example the frame 6 0 23 25 is seen as 0 23 25. So it logs data to node zero instaed of node 6. I have to restart the rfm12Pi service to make it work as expected.

   - In Emonhub's config file (home/pi/emonhub/conf/emonhub.conf), when I comment out the lines that defines the dispatcher that send data to the local EmonCMS, it has o effect after a reboot. Data are still send to the local EmonCMS.

   - Emonhub logging stops after 10 seconds.

The rfm2pi service shouldn't be installed (see my earlier post Sat, 21/06/2014 - 13:34), emonhub and emoncms cannot successfully access the Pi's serial port at the same time. If you installed rfm2pi using the apt-get method run

sudo apt-get remove emoncms-module-rfm12pi

This could cause erratic serial port data in emonhub and it will bypass emonhub with data from rfm2pi direct to emoncms, which means no data to emoncms.org. This is why posting to local emonCMS still happens when emonHub doesn't have a dispatcher configured for local emoncms.

Not sure why logging stops after 10 secs, can you post the 10 secs so I can see what's there or missing ? what is the logging level set to in emonhub.conf ?

Paul

In fact emonhub may not have anything to log if it cannot access serial port which in turn, maybe the reason the logging stops. This may not be obvious because the data is still arriving at local emoncms  directly from rfm2pi, so it looks like emonhub is still functioning.

Retry the logging after removing the rfm12pi module.

Paul

Hi,

Great !

After running sudo apt-get remove emoncms-module-rfm12pi, everything seems to work fine !

Logging is working more than 10s and data are well buffered during network outages.

I will give further feedbacks later.

Eric