Archived Forum

The latest version of rfm12piphp in emoncms/Modules/raspberrypi from github has     RPI_BIN="/usr/share/emoncms/www/Modules/raspberrypi/raspberrypi_run.php" on line 35  which I think may be causing the problem. The update to the Debian packaged version seems to be causing a problem installing via Git.

I've been trying to setup a fresh git install and had to change it back to the old version which had RPI_BIN="var/www/emoncms/Modules/raspberrypi/raspberrypi_run.php" in order to get the interface running.

I tried to reinstall whole raspberry platform image to my HDD. In settings of image from Feb 23rd everything works. Even if after "git pull" update. But if I reboot it then for any reason, communication with Jeelink RFM12 stops.

I checked my rfm12piphp after that update and it seems to be unchanged by update process. And on line 35 is written

RPI_BIN="var/www/emoncms/Modules/raspberrypi/raspberrypi_run.php"

update to v 8.0.9 modified probably something more yet...

There is also a copy at /etc/init.d/rfm12piphp used during start up, you could check the path in that file also.

Paul

I obviously missed that path when backing out the debian changes from the repos, sorry.

It should be as-is for the serving-git-repos approach then patched in the .deb file - I'll try to fix this over the weekend.

IMO the OEM gateway is the future for bridging the rfm12pi to emoncms, but that doesn't mean we should leave the integrated module broken... apologies.

Just to add: you probably did a git pull and didn't restart the rfm12piphp script for some time, so you thought it worked, but that was only because the daemon was still in memory. First time you had to restart your Pi it crapped out trying to reload.

This is one of the structural issues with the rfm12piphp approach that I really dislike, as I lost a chunk of data myself making a bad assumption about this.

I checked that copy in /etc/init.d, but that path is also correct.

Radek

yes, exactly, I didnt restart system for a few days, but yesterday evening we had short dropout of electricity supply due storm, so it restarted itsellf.

To be honest, I would use OEM gateway, but Im little bit confused, how it works, especially its co-existence with rfm12piphp daemon. I need stable system and I dont know, which one of these two solutions is better and why and how to set it up properly. Is there any info resource covering this area?

At this moment I guess, it is best solution for me to downgrade back to that image from Feb 23rd, as it works reliably. Or would you recommend me another scenario?

The Feb 23rd image is probably your best solution at present.

I've just tried fresh installs from git and from apt for the Rpi and couldn't get either to work. The git version had problems loading rfm12pi due to file locations. The apt version has the rfm12pi status as running but no inputs can be created.

@ ngbod   -  when installing the apt version from fresh did did you edit /boot/cmdline.txt and /etc/inittab ?  http://openenergymonitor.org/emon/node/4777#comment-20231

Paul

@ rbreuss  -  You say "I checked that copy in /etc/init.d, but that path is also correct." have you edited or overwritten it at any point? after changes you need to update-rc, probably best to go over these 2 steps again just to be sure

sudo chmod 755 /etc/init.d/rfm12piphp

sudo update-rc.d rfm12piphp defaults

You could revert to the old image or the apt version definitely works ok once serial access is sorted(/boot/cmdline.txt and /etc/inittab edits) 

Paul

Ahhh! something else I've just thought of there are also copies of rfm12piphp & rfm12pi_run.php located in .../Modules/node, I never did find out why but I do recall wondering why there where there and the ..../nodes/rfm12piphp was used rather than the .../raspberrypi/rfm12piphp for some reason so worth checking there as well.

Thanks Paul, my apt install is now working.

I checked and boot/cmdline.txt had been edited but inittab hadn't. I edited it and rebooted and all now seems well.

I remember editing them on the previous git install but there is no mention of these edits in the pkg-emoncms readme which I followed for the apt install.

Paul,

I have very basic knowledge of LINUX administration and configuration. I just copied packaged FEB 23rd image to HDD (and to SD of course) and run "git pull" update in /var/www/emoncms folder. Before update it works, after update not. I didnt edit rfm12piphp file as I have no reason and mainly not knowledge enough to do it. So, Im sorry, for probably stupid question:

Is there ANY way how to update FEB 23rd image to 8.0.9 version of EMONCMS and to keep that system working? Ver 8.0.9 is much better than that original one. Im learning how to take care for my LINUX system, but it goes slowly and each hint from experienced colleague is valuable for me.

Radek

@ngbod - glad it worked, yes the readme does need updating and I sure it will be shortly as these omissions only came to light quite recently.

@Radek - How much data have you collected? if you want a later version than the feb hdd image (which I'm sure you do) the best way forward would probably be to start with a fresh raspbian image and install using the apt method, it is so much easier and definitely works well (aside from the previously mentioned readme ommision).

There are various ways to do it but if you do not have much data apt is the way to go, around 10mins from fresh raspbian image to a fully functioning emoncms install that you can then continue to update regularly using apt-get update.

Paul

Thanks for a new, promising direction for me, Paul! I have no important data, just a few days from testing run. We can forget them.

OK,now I have a fresh installation of Feb 23 Raspi image at my HDD. And now, how to process that update via apt method? I never did it before, so I have zero knowledge at this point. Is there any step by step tutorial available at this forum?

Radek

I would start a fresh by downloading the latest raspbian image and put that image on a SD card and also the hdd.

From there we can just do a couple of things to set up the hdd (that's the trickier bit) and then use apt-get to install emoncms

read http://openenergymonitor.org/emon/node/4384 while your downloading image, although 5 commands is just emoncms, the rfm2pi module isn't that many more :-)

Paul

That's a completely new world for me. I'm going to dive in and we will see. I will let you know, how I'm doing there.

My big thanks for this moment.

Radek

Soooo, I downloaded that raspian image a copied it to SD. There was no any bit of code to be put to HD.

So, I have image on SD, i got in via SSH. Steps descibed in installation cookbook worked and it seems to be installed. But just on SD. How to get from SD to HD installation?

Just as an aside - if anyone would like to submit a pull request to improve any of the documentation on my pkg_* repositories (or indeed the packaging itself) then please feel free!

https://github.com/Dave-McCraw/pkg-emoncms

https://github.com/Dave-McCraw/pkg-emoncms-module-rfm12pi

You can just go to the readme.md file and hit the edit button you know, github will pretty much take care of the rest :-)

@rbreuss, the raspian image will indeed just be on your SD card. I'm not sure of the steps required to get the non-boot partitions onto the HDD, but I presume Trystan or Glyn can point us at some guidance on this (I can't seem to find a build guide for the HDD, perhaps because it's just a little late on a Friday night :)

The easiest way is to put the image on BOTH the sd and the hdd, boot up on the sd and then edit the hdd to prepare it to boot to, so first you need to give the hdd a unique uuid and then mount the hdd

sudo tune2fs -U random -L hdd  /dev/sda2 

sudo mkdir /media/hdd

sudo mount /dev/sda2 /media/hdd

then edit the hdd's fstab

sudo nano /media/hdd/etc/fstab

change " /dev/mmcblk0p2 " to " /dev/sda2 " and Ctrl+x, y, enter,

while we are here just edit the inittab (the bit that's missing above)

sudo nano /media/hdd/etc/inittab

edit the very last line by adding a hash so it reads " # T0:23:respawn:/sbin/getty -L ttyAMA0 115200 vt100 "

edit the cmdline.txt and reboot (the next 2 lines are 1 line wrapped be sure it is entered as 1 line "sudo" to "txt'")

sudo sh -c 'echo "dwc_otg.lpm_enable=0 console=tty1 root=/dev/sda2 rootfstype=ext4 elevator=deadline rootwait" > /boot/cmdline.txt'

sudo shutdown -r now

should now boot up to the hdd so add the repo to sources list, update and install (next 2 lines as one line again)

sudo sh -c "echo 'deb http://emon-repo.s3.amazonaws.com wheezy unstable' >> /etc/apt/sources.list"

sudo apt-get update

sudo apt-get -y --force-yes install emoncms emoncms-module-event emoncms-module-rfm12pi

you will get asked a series of questions about mysql and emoncms passwords

sudo a2ensite emoncms

sudo a2enmod rewrite

sudo /etc/init.d/apache2 restart

sudo service rfm12piphp start

go to http;//IP.ADD.OF.PI/emoncms and register a user and with any luck you're in.

Paul

Dave - I wasn't sure if the readme was to be updated yet as the discussion about automating the process v separate rfm2pi instructions wasn't complete.

I think some hdd image instructions or blank download image maybe a great help, case in hand as an example. I thought I would try this as a complete a to z install and if it is complete it can be used to build a readme maybe. (I only say if because this is from memory I really should make notes as I do an istall from absolute fresh ie single downloaded image)

Paul

Oh, thats really a complex cookbook (In Middle Ages of IT, 30 years ago I was mounting first disk storage into mainframe computers with paper punched tape OS by similar sequences of "secret" commands :-)). THANKS Paul, for your open approach and Im going to try that step by step.

Radek

I confirm, it works! Now I have fresh installation of emoncms 8.0.9. Great tutorial Paul!

Let me to ask you a few additional questions:

1) If I reboot my new system now, all of the services will start automaticaly? Or should I start

sudo service rfm12piphp start  manually?

2) If I will process further updates, which steps of your tutorial I should use?

Radek

Everything should work correctly after a reboot, I would recommend trying it while you are setting up /testing as you don't want to come home one day and find several hour data missing and not know why following a momentary power outage or similar.

In future when you use sudo apt-get update && sudo apt-get upgrade emoncms will get updated along with everything else.

As more modules become available in apt packaging you can install them with apt-get install as well but there maybe other steps also and that will be documented with each module.

The second partition of your sdcard still holds a redundant copy of raspbian, you could copy just the boot partition to a smaller sdcard if you wanted to free up this sdcard or if not just leave it there and if ever you have an issue with your "hdd" version you can whip out the sdcard edit the /boot/cmdline.txt file on windows by changing "/dev/sda2" to "/sda/mmcblk0p2" and pop it back it the pi to boot into the sdcard image and explore / fix your hdd image.

There are tidier methods to do this but this is by far the easiest way without another linux machine.

Paul

Realy good job, Paul! Is works great, everything I tested till now is OK. Nice to meet guy, which knows, whats the game about.

As new modules become available, where I will find them? Is there any central repository?

Only unusal behavior, my new system has now - before at old emoncms installation based on standard Raspi/HDD/SD/Emoncms package I was able to modify any file via WINSCP editor. Now, any modification this way is refused due "permission denied". When I modify file via "sudo nano...", modification is accepted, but any modif via WINSCP is refused. Why? Is there any wrong setup of access permission in my new setup?

I have set up a 2nd Pi for developement/testing as above and I too am experiencing the same problem with WINSCP as the post above (and yes I do have Shell set to sudo su - in the advanced site settings)

I had never used WinSCP before today I have just tried it to recreate your issue and found it really quite good, I found the same situation and the solution is to enable a root user.

In normal use a raspberry pi user can get temporary root access by prefixing command with "sudo" WinSCP isn't able to prefix command to get root access, You can however log in as "root" if you set up the root user password on pi first.

sudo passwd

enter a password and confirm.

You can now open a session in WinSCP logging into pi using the username root and the password you just set up.

(SSH & normal logging in as your usual user (pi) and using sudo will still work exactly as as before.)

To revoke this additional "root" user, whilst logged in as your usual user (pi)

sudo passwd -l root

and all is back as it was :-)

Paul

Your remedy works again to 100%. You are great tutor. Thanks, Paul.

Radek

@ph66;@rbreuss;

thanks for the guide, I am not an expert in computer science, I wanted to ask, when the installation requires a username and password for mysql, I can insert any data or not. When ask me the localhost, I have to enter the IP address of raspberry or leave "localhost".
sorry for my english (google translator), and thank you for the answers.

leonardo

In normal use a raspberry pi user can get temporary root access by prefixing command with "sudo" WinSCP isn't able to prefix command to get root access, You can however log in as "root" if you set up the root user password on pi first.

Thanks for this solution however WinSCP does get root access on my main RaspberryPi (which uses the downloaded Pi + HDD image) by having Shell set to sudo su - in the advanced site settings in WinSCP. Not an issue for me as I'm happy with either solution but there may be security concerns in other environments.

@rbreuss; no problem, glad to help

@rjoem; The first 2 questions are to set the root password for the mySQL database root user, it's most important you set this password correctly, The rest are questions about setting up the emoncms database and email. You can accept all the default values except the second emoncms question (4th overall) this password gives emoncms access to mySQL so It should be set to the MySQL root password you just entered for questions 1 & 2. If you make a mistake with the emoncms settings it's not a problem as you can go through the questions again by using sudo dpkg-reconfigure emoncms --force.

@Bra1n; I'm still a novice in the art of WinSCP but I tried the advanced sudo su setting for shell, It only affects the shell commands issued and doesn't affect the browser/filemanager permissions. logged in as pi I'm unable to save edits and if I run ntpd (a sudo only command to correct the UTC time on pi) it tells me it can only be run as root. But if I then set the shells sudo su setting (phew!) the ntpd runs without a hitch but I still cannot save edits. I think the rpi/hdd file/folder permissions may differ to the apt packages maybe. but as long as it works.....

Yes the security is a concern, I believe the sudo method adopted on the pi is intended to eliminate the necessity for a root user to be set up in a bid to tighten security. As it is so easy to enable & disable It's probably best to only enable it as and when required if you are an infrequent user.

Paul

I had to go back to v7 and then upgraded via git to v809 but as mentioned by the original poster the rf12piphp will not start now. I have copied the old version rfm12pi back to var/www//emoncms/Modules/raspberrypi and also to init.d and the path is correct in the one in the Node directory. but still it will not load.

When I swap to my v7 emoncms directory which is still on the disk the rfm12pi loads fine so something in the new emoncms directory or subdirectories is stopping it loading but after searching all day I have to admit defeat. Does anyone have any ideas what could be preventing this loading.  

@ph66: thanks for the help, now it works.

I have another question, my previous system (rasp + HD) had occasionally blocks as described here ( http://openenergymonitor.org/emon/node/3584 ), it is appropraite even in this version upgrade the firmware RFM12 Pi?

Then, the sd card is of 8GB, while the copied image occupies only a part, I can extend using the menu on the raspberry. The HDD is instead an old IDE 20GB, running the command df-h (see image) does not seem to be indicated, to see its use and occupation I have to give another command? using G-parted, which partition can expand to use the whole HDD.
thanks again

leonardo

Completely confirmed what ngbod says, I have the same problem. Now I have fully functional system (v 8.0.9) thanks to Paul and his great tutorial above, but it is completely different approach and installation process. When I use standard prebuilt package and upgrade EMONCMS to very latest version via git pull, rf12piphp is not started automaticaly during reboot and is not possible to start it manually (at least Im unable to start it).

@ ngbod - reading back some of your earlier posts I can see you're missing the leading / when you edit /etc/init.d/rfm12piphp it should read RPI_BIN="/var/www/emoncms/Modules/raspberrypi/raspberrypi_run.php" try adding the / and then run 

sudo update-rc.d rfm12piphp defaults && sudo service rfm12piphp restart

I hope that sorts it as I'm not sure what else it could be. (these previous changes will be reverted on git very shortly, the pull request is currently queued to be merged)

@ rbreuss - the notes above are possibly relevant to your previous installation but not to the apt installed version, if you want to use the previous copy you will have to manually edit parts as ngbod is doing or wait until the changes are reverted on git and try again.

@  rjoem - expanding the partitions is a whole new topic that may be better addressed on a new thread as this one is getting a little busy :-)

Paul

ps by the way rjoem - it's pb66 not ph66 

Paul, I fully understand, what you say. Im not mixing those two installations.

1) APT way described by you works without problems (there is only one bug regarding My Electricity panel, Trystan already fixed in master, but it remains in my apt installation unfixed yet).

As Im learning various approaches to get more familiar with LINUX, I tried once more that original approach

2) Prebuilt image and upgrade to 8.0.9 via git pull. And this git approach is still unusable, as system stops RFM receiving after upgrade to 8.0.9.

@ pb66 ; ok, sorry Paul for the error. But it needs to upgrade the firmware RFM12Pi?

leonardo

@ rbreuss - yes, my electric has been fixed but may not have reached a release yet, not every edit on git repo results in a new release, but they are very frequent at the moment so you won't need to wait too long. and regards the git version perhaps wait a while and see if ngbod has any success with the edit, again that will be available on git very soon.

@ rjoem - sorry, missed your earlier Q about rfm2pi firmware, The firmware update is relevant to the rfm2pi board only, regardless of what software you are running if you experience rfm lock-ups/drop-outs it would be a good idea to upgrade firmware.

Paul

@ngbod - I think I'm right in saying that the /etc/init.d/rfm12piphp script is the only relevant one (it's not required, or accessed, within the ???/Modules/raspberrypi/ directory, just stored there for convenience in git). However, I'm not sure if this also applies to versions related to the node module.

You can try diffing the rfm12piphp scripts to ensure they really are the same by running

diff /path/to/first/rfm12piphp /path/to/second/rfm12piphp

You should see nothing, but you might see a diff on the config paths between the two files, which would be instructive.

Paul, Dave, I checked my typing and all was ok. I diffed the init.d and rapberrypi versions and they are identical. The one in the node directory is different but I think it should be. I replaced the node version with the init.d version but it still didn't work.

When I replace the v8 Modules directory with the v7 one the rfm12pi runs fine. I tried renaming individual modules and replacing them with the v7 ones but it still wouldn't run. I now realise I maybe should have removed the v8 modules not just rename them to find the offending item.

The v8 modules had pi as owner so I changed them all to root like the v7 modules but still it won't run.

@rjoem perhaps the post I made in this thread http://openenergymonitor.org/emon/node/3889 may help with your HDD question.

I have found this error message when trying to load the rfm12piphp service....

PHP Fatal error:  Class 'EmonLogger' not found in /var/www/emoncms/Modules/user/user_model.php on line 34

and also

PHP Fatal error:  Class 'EmonLogger' not found in /var/www/emoncms/Modules/input/process_model.php on line 27

The line in both files is the same.....

$this->log = new EmonLogger(__FILE__);

I tried commenting the line out and rfm12piphp then loaded ok but no data was transferred.

I've not seen that error message before, but if rfm12piphp is running but not receiving data recheck /boot/cmdline.txt and /etc/inittab again as that is symptom of not enabling serial access for rfm12pi,

Also try restarting rfm12pi with sudo service rfm12piphp restart log and then you can check the log files to see what's happening.

Paul

Thanks Paul, I  checked the cmdline and inittab and they where correct. The rfm12piphp log doesn't help as it's now complaining about something else.....

Fatal error: Call to a member function info() on a non-object in /var/www/emoncms/Modules/input/process_model.php on line 86

I think I'll just go back to v7 for now as the v8 upgrade is just running me round in circles.  I'll wait for a v8 image to hopefully come out.

ngbod - the version of process_model in master uses the EmonLogger class - /var/www/emoncms/Modules/log/EmonLogger.php (even if you don't configure emoncms to do any logging, the PHP files still import the logger so you'd need that file if you're mixing and matching versions).

You can see it here (ironically, it's a wrapper so that log4php doesn't cause any issues for people who don't have it installed!)

Dave,  the file /var/www/emoncms/Modules/log/EmonLogger.php is present but I still get the errors.

I am not mixing and  matching git and apt versions. All the installs have been 100% git derived. There has also been no mixing of the v7 and v8 emoncms directories as the v7 directories have been renamed or removed.

I had looked at the log4php statement in the git settings.php file as it points to a non existent file.

// Log4PHP configuration
    $log4php_configPath = '/etc/emoncms/emoncms_log4j.xml';

The statement is not present in my apt version of settings.php

Just noticed the problematic emonlogger statements are not present in the apt versions of the files either.

Are you running 8.0.9 or the head of the master branch? Now that I think about it, it sounds like the latter because the reason logging is not supported in the APT install is because it was merged after 8.0.9...

Could you try running the actual 8.0.9 (I'm guessing you may have more luck?) - you can get this by git checkout 8.0.9 if you're serving up the git repository.

I spent quite a bit of time ensuring the logging would be transparent on legacy installs but perhaps I missed something, so I will check that just now.

I ran git checkout master on saturday pm and then installed per the step by step raspberry/ linux instructions on git. After a nice clean install I then saved an image so I could go back to fresh after each attempt to get it to work. It had v8.09 at the bottom of each screen.

Guys,

I had fully working system till now, installed from apt by cookbook of Paul, see above in this thread. I just made and upgrade to up-to-date version 8.1.1 by command " sudo apt-get update && sudo apt-get upgrade " recommended by Paul also in above. And result? Communication with RFM12 is again not functional. Does anybody has an idea, what can be wrong this time? Im not LINUX guru, so Im unable to find, where is problem. Is there ANY STABLE version available? I would like to participate in visualisations development, but in case each upgrade makes my system unusable due basic communication malfunction, it is hard.

Radek

Radek, it may be that you need to restart the rfm12piphp service with:

sudo service rfm12piphp restart

Yes, sure, Trystan, I restarted it man times (either with logging off or on). But it is solved already, I found solution in another thread:

https://github.com/emoncms/raspberrypi/pull/42/files

After I added that missing "require" command, everything works again.

Radek

If you use apt to manage your emoncms installation, there's a new version of the raspberrypi module now available which includes the fix from the upstream repo.

sudo apt-get update && sudo apt-get install emoncms-module-rfm12pi

You should now be running 1.0.1-1 (which you can confirm by running dpkg -l  | grep rfm12pi ) - this should play nicely with all versions of the main emoncms package.

I just tried that for laughs as I was running a hand edited rfm12pi version 1.0.1 and after upgrading to 1.0.1-1 got ERROR: file not found : '/var/www/Modules/raspberrypi/raspberrypi_run.php'  which is not surprising as the apt install has it at /usr/share/emoncms/www/Modules/raspberrypi/raspberrypi_run.php . Manually editing /etc/init.d/rfm12piphp to point RPI_BIN to the correct location solved the problem.

I've submitted a pull request 

Good catch. I was thinking about fixing the EmonLogger issue and forgot that the fix to the deployment directory had not previously been a tagged raspberrypi release.

I actually need to write a quilt patch for this so it isn't overwritten by future changes, leaving us back in the same boat - I'll have that up in a couple of hours.

That should be fixed with 1.0.1-2 , cheers.