Archived Forum

Hi Alex

1. No, not expected. But there are instances where it is an avoidable or too minimal to warrant the solution. On the emonPi there are 2 parts, emonHub and emonCMS, emonHub will "buffer" data in RAM when it cannot deliver the data until it can but it is not written to disc as is is designed to run on a read-only OS. It will catch up when network is restored if the volatile data is still available ie no restarting, rebooting or power cuts.

2. Yes and No, the data is there in emonCMS to fall back on but unfortunately there is currently no "sync" mechanism, so this is not really a "solution" but an absolute last resort if you really must have that data.

3. You can't really "sync" at all as they are independent, I'm not saying it cannot be done or that it won't be available in the future but right now you would need to "figure it out" for each feed based on the circumstance.

4. Not personally, the edimax seems ok most of the time, I do have a couple og external aerial types and I found they didn't offer any signal improvement or at least if they did it wasn't significant enough to justify the ugly protrusion and/or additional drivers.

Are you sure the network connection was permanently lost in this instance? how did you establish there was no network connection or was it assumed from lack of data?

Generally speaking if there is a network outage, emonhub will buffer OR if there is a powercut the Pi is read-only OS, emoncms normally only buffers 1minuite of data and it is usual to assume there are no power readings during a power outage, at least for the usual short interruptions this doesn't present a big issue.

Getting a power cut after an extended network outage would be unlucky but if a frequent occurrence then perhaps a UPS and/or wired network connection could be on the cards?  

The crux of the issue here will be the wifi connectivity, if you can ensure the network reconnects after being down you should be ok, can you test that scenario? the emonPi should be running a check and reconnect script as far as I recall.

Paul

Thanks Paul, that clarifies a few things for me. 

On further analysis I think the problem may be my wifi configuration in wpa_supplicant.conf. There were a number of free/open networks listed in there which must have been added during initial setup of the emonPi. This is just an educated guess, but I think what happened was that when it lost signal to my access point, it switched to one of the open networks, but that meant it was obviously no longer on my local network. It never came back as the open networks must have a stronger signal than mine.

I've cleaned up the file now and will see how that works - I'll simulate a few wifi outages by powering off my access point a few hours at a time and see what happens.

This is probably a point for a new post, but one other thing I'm trying is to limit the wlan interface rate to reduce packet loss and get a better signal to noise ratio. But it doesn't seem to take any of my settings:

List available and current rates:

root@emonpi:~# iwlist wlan0 bitrate
wlan0     4 available bit-rates :
          1 Mb/s
          2 Mb/s
          5.5 Mb/s
          11 Mb/s
          Current Bit Rate:65 Mb/s

Set rate:

root@emonpi:~# iwconfig wlan0 rate 5.5M fixed

List again (note there is no change in the current rate):

root@emonpi:~# iwlist wlan0 bitrate
wlan0     4 available bit-rates :
          1 Mb/s
          2 Mb/s
          5.5 Mb/s
          11 Mb/s
          Current Bit Rate:65 Mb/s

I have tried all listed options, as well as other unlisted ones. Not sure if this is a limitation of the Edimax dongle/drivers or something else is going on...

You're right that a wired network connection is probably the way to go longer term, however there are a few other things that need to get done in the house first before I can do this! So for the short-to-medium term wifi is unfortunately the only way I can connect...

-Alex

I tried this and got the same result, a quick web search seems to suggest the bitrate "limit" must be set prior to connection for each connection, so ideally you need to add a setting to one of the wifi conf files but I found no guides or examples of that.

Paul

I have experienced a similar problem as the OP today.

The emonPi lost Wi-Fi at approximately 07:00 this morning (I didn't realise until 18:30 when I returned home). I pulled the edimax usb dongle and reinserted it, then Wi-Fi was restored. The local emonCMS has kept on tracking grid & solar, but web emonCMS now has a gap in data for the complete day. As I didn't power cycle the emonPi, I fully expected it to upload to web emonCMS when Wi-Fi was restored.

How can I get the day's data to web emonCMS?

Cheers!

I don't have any idea how to get data from the local emonCMS to emoncms.org.

However, I have a few Raspberry Pi's in remote locations and have fought to keep them connected using various wifi dongles.  The following solution has kept them online for about 6 months straight:

1. Install the script "WiFi_Check" from here:

https://rpi.tnet.com/project/scripts/wifi_check

2.  Install a modification to the script to ping the router to check for wifi connectivity.  If no router response, then restart wifi services.  Be sure to enter your router's address in the appropriate place. 

Step 1:  Open the config file in an editor:

sudo nano /usr/local/bin/WiFi_Check

Step 2:  Replace everything below # Settings with the code below:

##################################################################
# Settings
# Where and what you want to call the Lockfile
lockfile='/var/run/WiFi_Check.pid'
# Which Interface do you want to check/fix
wlan='wlan0'
# Which address do you want to ping to see if you can connect
pingip='192.168.1.1'

##################################################################

echo
echo "Starting WiFi check for $wlan"
date
echo 

# Check to see if there is a lock file

if [ -e $lockfile ]; then
    # A lockfile exists... Lets check to see if it is still valid
    pid=`cat $lockfile`
    if kill -0 &>1 > /dev/null $pid; then
        # Still Valid... lets let it be...
        #echo "Process still running, Lockfile valid"
        exit 1

    else

        # Old Lockfile, Remove it
        #echo "Old lockfile, Removing Lockfile"
        rm $lockfile
    fi
fi

# If we get here, set a lock file using our current PID#

#echo "Setting Lockfile"
echo $$ > $lockfile

# We can perform check
echo "Performing Network check for $wlan"

/bin/ping -c 2 -I $wlan $pingip > /dev/null 2> /dev/null

if [ $? -ge 1 ] ; then
    echo "Network connection down! Attempting reconnection."
    /sbin/ifdown $wlan
    sleep 5
    /sbin/ifup --force $wlan
else
    echo "Network is Okay"   

fi

# Check is complete, Remove Lock file and exit
#echo "process is complete, removing lockfile"
rm $lockfile

exit 0

##################################################################

# End of Script

There is already some sort of wi-fi reconnection script running on the emonPi, I haven't looked closely at it but I think it's cron based.

@Aceshigh84 - I took a closer look at the emon-pi variant of emonhub's code as there definitely shouldn't be any gaps in the data, or so I thought. It turns out there are significant differences in the code in the way emoncms data is passed by http. It appears this version does not resubmit data if undelivered, it seems to just send and forget every 30secs (hardcoded interval not from conf) it deletes the data regardless of whether is is sent or confirmed.

Apparently this version does not provide any buffering beyond 30secs.

@alex_london - I hope you have email notifications on as the info I have given you above is incorrect for this version. I had no idea at the time and cannot explain why the controlled FIFO buffers have been removed, sorry.

Paul

@JD - thanks. I'll try out that script when I get a chance, probably this weekend. Have also considered going down the powerline route and get a small brick to give it an ethernet connection. But if this script works, it'll save some money for sure!

@paul - yes I've been keeping an eye on this thread... the missing data is no big deal, it's not that lives depend on it! Was more of a curiosity for me, trying to understand how it's supposed to work.

I'm getting weak wifi issues too - am going to try moving the wifi dongle away from the emonPi using a USB extension cable.

But the unit does really need to buffer in some way to avoid discontinuity caused by a drop of Internet connection.

Chris.