Oemgateway in read only

HI,

I'm following that guide to setup an OEMgateway forwarding data to my EmonCMS.org account.

It works fine until I try to setup Raspbian to run in read-only mode for increased stability.

If I put the following lines in /etc/fstab as mentioned in the guide, the OEMgateway doesn't work anymore.

tmpfs           /tmp            tmpfs   nodev,nosuid,size=30M,mode=1777    0    0
tmpfs           /var/log        tmpfs   nodev,nosuid,size=30M,mode=1777    0    0
proc            /proc           proc    defaults          0       0
/dev/mmcblk0p1  /boot           vfat    defaults          0       2
/dev/mmcblk0p2  /               ext4    defaults,ro,noatime,errors=remount-ro  0       1
# /dev/mmcblk0p3  /home           ext4    defaults,noatime

 

Before that step, here is my file system :

pi@raspberrypi ~ $ df -T
Filesystem     Type     1K-blocks    Used Available Use% Mounted on
rootfs            rootfs     2721336 2086676    476708  82% /
/dev/root       ext4       2721336 2086676    476708  82% /
devtmpfs       devtmpfs    215824       0    215824   0% /dev
tmpfs            tmpfs        44820     208     44612   1% /run
tmpfs            tmpfs         5120       0      5120   0% /run/lock
tmpfs            tmpfs        89620       0     89620   0% /run/shm
/dev/mmcblk0p1 vfat         57288   19072     38216  34% /boot

 

Must I replace /dev/mmcblk0p2  with /dev/root in fstab ?

Bye

pb66's picture

Re: Oemgateway in read only

Hi Eric.

No, the the "/dev/mmcblk0p2" in the fstab is the physical location of the root file system, without that it won't boot correctly either.

Have you added RAMPTMP=yes to the end of /etc/default/rc5 and replaced /etc/mtab with a symlink to /proc/self/mounts ?

I have written a guide to this in the emonhub documentation which isn't up and running just yet. But there is a copy in Trystan's documentation for his later, emonhub based version of the gateway you are installing. 

Paul

Eric_AMANN's picture

Re: Oemgateway in read only

I followed each steps of this guide. So I made the modifications that you mentioned. So it 's not the problem.

I just understood that there is typo in the step a) :

a) sudo nano /etc/default/rcS, add line RAMTMP=yes   ---> a) sudo nano /etc/default/rc5, add line RAMTMP=yes

May I edit this guide ?

EDIT : there is a typo in you message, it's /etc/default/rcS and not /etc/default/rc5

EDIT 2 : I will try with "RAMPTMP=yes" instaed of "RAMTMP=yes"

 

pb66's picture

Re: Oemgateway in read only

Apologies for the typos in my post (the guides do have rcS & RAMTMP), seems the old grey matter isn't firing on all cylinders this morning. I've not used the guide you are working from, but I can't see any reason for not working. I regularly use the method in my guide as, just cutting and pasting this chunk of code does everything and eliminates typo errors etc.

sudo cp /etc/default/rcS /etc/default/rcS.orig
sudo sh -c "echo 'RAMTMP=yes' >> /etc/default/rcS"
sudo mv /etc/fstab /etc/fstab.orig
sudo sh -c "echo 'tmpfs           /tmp            tmpfs   nodev,nosuid,size=30M,mode=1777       0    0' >> /etc/fstab"
sudo sh -c "echo 'tmpfs           /var/log        tmpfs   nodev,nosuid,size=30M,mode=1777       0    0' >> /etc/fstab"
sudo sh -c "echo 'proc            /proc           proc    defaults                              0    0' >> /etc/fstab"
sudo sh -c "echo '/dev/mmcblk0p1  /boot           vfat    defaults                              0    2' >> /etc/fstab"
sudo sh -c "echo '/dev/mmcblk0p2  /               ext4    defaults,ro,noatime,errors=remount-ro 0    1' >> /etc/fstab"
sudo sh -c "echo ' ' >> /etc/fstab"
sudo mv /etc/mtab /etc/mtab.orig
sudo ln -s /proc/self/mounts /etc/mtab

it also keeps all 3 original files so reverting to standard is easy. (I think I will post the full instructions on forum as it also has shortcuts to lock and unlock filesystem)

Also just as a "heads up" I don't think the oem_gateway you are using has variable size buffers or bulk upload enabled as discussed in your other thread.

Paul

Eric_AMANN's picture

Re: Oemgateway in read only

Thank's for your chunk of code  and for your support in general!

It doesn't fix the problem. As soon as I setup Raspbian to run in read-only mode, the oemgateway doesn't work anymore after a reboot. When trying to stop the oemgateway service, I gotr the following error :

pi@raspberrypi ~/oem_gateway $ sudo service oemgateway stop
[....] Stopping OpenEnergyMonitor Gateway: oemgatewaystart-stop-daemon: warning: failed to kill 2463: No such process

The only way to make it work, is to come back to the three original files (fstab, mtab and rcS). But then the file systeme is in RW mode ... As you can see below there is no difference between mtab and mtab.origin. Is it normal ?

 

pi@raspberrypi ~ $ diff /etc/mtab /etc/mtab.orig

pi@raspberrypi ~ $ diff /etc/default/rcS.orig /etc/default/rcS
24a25
> RAMTMP=yes

pi@raspberrypi ~ $ diff /etc/fstab /etc/fstab.orig
1,6c1,4
< tmpfs           /tmp            tmpfs   nodev,nosuid,size=30M,mode=1777       0    0
< tmpfs           /var/log        tmpfs   nodev,nosuid,size=30M,mode=1777       0    0
< proc            /proc           proc    defaults                              0    0
< /dev/mmcblk0p1  /boot           vfat    defaults                              0    2
< /dev/mmcblk0p2  /               ext4    defaults,ro,noatime,errors=remount-ro 0    1

---
> proc            /proc           proc    defaults          0       0
> /dev/mmcblk0p1  /boot           vfat    defaults          0       2
> /dev/mmcblk0p2  /               ext4    defaults,noatime  0       1
> # a swapfile is not a swap partition, so no using swapon|off from here on, use  dphys-swapfile swap[on|off]  for that

 

Eric

 

Eric_AMANN's picture

Re: Oemgateway in read only

Anyway, I will try to install the Emonhub on a SD card with a RO file system to get variable size buffers or bulk upload enabled.

Which guide should I follow ? This one ?

Eric

pb66's picture

Re: Oemgateway in read only

Unfortunately I'm not able to access a pi right now and I don't know 100% what to expect as I'm not sure how diff handles symlinks and I don't know the exact content of your mtab.orig. It certainly doesn't look right to me but I cannot be 100% without trying it.

You could use the version that that guide installs, it does have variable buffer size but not bulk upload, I'm not overly familiar with that version so may be of limited help to you.

Personally I would recommend (and appreciate) you installing the dev version of emonhub as I am more familiar with it and actively developing it. It has both variable buffer size and bulk upload. It currently posts to the input module rather than the node module as I'm working on a solution that utilises both rather than only one or other.

One line will install and set-up emonhub

git clone https://github.com/emonhub/dev-emonhub.git && dev-emonhub/install.sh

then the conf file needs to be edited, just add an api key for basic operation, then add node details to change data types.

nano /etc/emonhub/emonhub.conf

 

For example to add the node data for the emonTxV3_continuous_kwhtotals_noeeprom.ino v3 sketch for node 10

[[10]]
  codes = L, h, h, h, h, l, l, l, l

and then use this guide I just posted to forum to switch to read-only.

Paul

 

 

 

Eric_AMANN's picture

Re: Oemgateway in read only

I did a try with that Emonhub installation guide

Firstly, I had to modify the emonhubNode.conf file in order to modify the log file path that wasn't correct (logfile = /var/log/emonhub.log -->  /var/log/emonhub/emonhub.log)

Then, I got the problem described above : after configuring raspbian in RO mode, Emonhub isn't working ... But, I did some progress ! 

 

When restarting the emonhub service, I get the following messages :

Restarting OpenEnergyMonitor emonHub: emonhubstart-stop-daemon: warning: failed to kill 1962: No such process
Could not start EmonHub: [Errno 2] No such file or directory: '/var/log/emonhub/emonhub.log'

If I unlock the file system, I get the same errors.

 

 

The workaround is to execute the following command.

sudo mkdir /var/log/emonhub
sudo chown pi /var/log/emonhub
sudo chmod 750 /var/log/emonhub
rpi-ro
sudo service emonhub restart

 

Then Emonhub works until the next reboot ...

 

 

EDIT : after a reboot :

pi@raspberrypi ~ $ sudo ls /var/log/emonhub
ls: cannot access /var/log/emonhub: No such file or directory

 

pb66's picture

Re: Oemgateway in read only

The original location for the logfile was going to be /var/log/emonhub/emonhub.log but this doesn't work with RO image as the /var/log/emonhub directory is lost every time the pi reboots since /var/log is relocated to RAM and therefore recreated at every boot.

emonhub's default was changed to /var/log/emonhub.log to be RO compatible. If you change the emonhub.conf file back to " logfile = /var/log/emonhub.log " this should no longer be an issue.

Paul

Eric_AMANN's picture

Re: Oemgateway in read only

Ok, it's now working in RO !

Unfortunately, as you said me, this version is limited. So I will install the dev version of emonhub.

Do you think this dev version is enough reliable to be installed on a site that I will not be able to access ? (neither physically speaking, nor by shh)

Eric

 

Eric_AMANN's picture

Re: Oemgateway in read only

I installed the dev version of Emonhub on a SD card and than switch to read-only. YESS !

Paul, I really like your installation-procedure (one line) !

Note that it was not working until I made some modifications :

1. At the bottom of the /etc/inittab file, I commented out the line, by adding a ‘#’ at beginning) and the cmdline.text file

2. Changes  /boot/cmdline.txt file with those lines :

sudo cp /boot/cmdline.txt /boot/cmdline.txt.origin
sudo sh -c "echo 'dwc_otg.lpm_enable=0 console=tty1 root=/dev/mmcblk0p2 rootfstype=ext4 elevator=deadline rootwait' > /boot/cmdline.txt"

 

It seemed to work only after the first modification, so I'm not sure that the second one is required.

Eric

 

pb66's picture

Re: Oemgateway in read only

I have no reason to doubt it's reliability but as with all new software only time will tell. any recent version of emonhub or oem gateway will not have the time served reassurance of the original oem gateway but they are built on the same reliable solution.

I have been using emonhub for a while now without any issues under normal running conditions, you would be wise to "stress test" any software you decide to use before committing. for example the buffers, you should try changing the url to something it cannot find to max out the buffers to test the size, the ability to resume sending and what happens if the buffer size is exceeded as these characteristics may change with larger buffer sizes, especially if you are anywhere close to maxing out the RAM.

The "dev" status is just to denote that emonhub is being tested and things are subject to change before a release version is finalised. In this "dev" version the files are not put where they should be rather they are symlinked so that git pull can be used to update, this method will not be used instead it will be available as a debian package at release time, I hope.

Yes both files need to be edited for the rfm2pi to access the uart with any software or OS on the pi. I did include a little reminder "(note - if  serial UART  access required make those changes before restarting)" at the end of the RO guide.

Paul

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.