kWh feeds corrupted after Raspberry Pi reboot - prevention

I've been playing with the low-write Raspberri Pi image of emoncms (emonSD-13-08-14.img.zip) for a few weeks now and found that time sensitive feeds such as Power->kWh are corrupted with huge negative numbers after a reboot of the Pi, be it graceful or otherwise.

What I have discovered is this is due to the system time always resetting to 1970 on reboot as the pi doesn't have a RTC and the feedwriter service manages to start and begin processing data before ntp grabs the current time from the network.  My home installation which is connected by wired Ethernet seems to be immune to this for the most part, but my work setup connects by wifi and everytime I reboot I end up with corrupted kWh feeds.

I noticed a file called fake-hwclock.data in ~/data which contains a date and time close to what the current time actually is.  A bit more digging revealed fake-hwclock is setup as a service which should run when the pi boots and update the date and time with the contents of that file, it also runs as a cron job every hour and updates ~/data/fake-hwclock with the current date and time, it is also executed when you shutdown or reboot.

The framework all seemed to be in place for this to work, but it wasn't it doing its job on bootup.  The problem appears to be in /etc/init.d/fake-hwclock

The line

# Default-Start:     S

needs to be changed to read

# Default-Start:     2 3 4 5

 

Then run the following commands to update the startup rc?.d folders with the new settings.

 $ sudo update-rc.d -f fake-hwclock remove

 $ sudo update-rc.d fake-hwclock defaults

 

I temporarily disabled ntp and did some testing and after every reboot the system would come up with the time only being about 1 min behind time, just the duration of the system rebooting.  Such a small change in time over a couple of samples until ntp picks up the correct time won't introduce a noticeable error to the kWh feeds like 40 odd years will!

I'd found a few threads on here where people had experienced the feeds corrupting, whilst what I offer here doesn't provide a solution for a "now broken" feed with bad data, at least it provides a means to hopefully prevent anything breaking in future.

 

pb66's picture

Re: kWh feeds corrupted after Raspberry Pi reboot - prevention

Hi Bradley

Good to aee another approach to tackling this issue, there is a "Read-only image time issues" thread on this with an alternative fix. The issue is as you say, the npt not getting the time before starting other processes but the actual cause is that fake-hwclock is unreachable (at that point in time) via the symlinked data partition copy.

Your fix above changes the order of things at boot where as "ntp-backup" tackles the problem by putting fake-hwclock back where it was intended to be to correct the start-up timing and directly tackles the read-only issue the relocation was intended to fix by managing the read-only status when the current time needs to be saved to fake-hwclock.

There usually is more than one way to skin the preverbal cat but a small advantage of the "ntp-backup" is that it's unlikely to get overwritten by a future update to ntp or fake-hwclock as the fix is an add-on' rather than edits and relocations.

As it doesn't use the data partition it is also portable for any raspbian image rather than just the emonSD-13-08-14.img.zip and incidentally opens up the future possibility of not having the 3rd data partition until enoncms is enabled which would fix another "quirk" of that SD card image by making "raspi-config expand SD file system" available to the many users not running emoncms locally.

Paul

bradtpt's picture

Re: kWh feeds corrupted after Raspberry Pi reboot - prevention

Hi Paul,

Yes, it was only shortly after posting my "fix" I discovered your thread listed above and subsequently ntp-backup.  I learnt a lot working through the issues to get my fix working but the points you put forward about ntp-backup make it much more appealing and as such I plan on installing it.  I also used the instructions in that thread of yours to re-zero my kWh feeds using manual .json entries through a web browser which was very convenient. A big thanks for your work.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.