Archived Forum

As a bit of a git noob, a debian package would be welcomed.  I have refrained from updating to the more recent versions as I'm a bit afraid it will break something!

Unfortunately there's no guarantee it would be backwards compatible - just like installing the next version of Office doesn't guarantee that you can open an old spreadsheet and all the functions still work ;-)

What debian packaging would offer is clear change control and a cleaner way of installing the code, rather than checking out nested git repositories into /var/www.

Alas, you'd still need to use git, unless someone hosts a custom apt-repository (in which case you could use the debian package manager). The advantage for you would be more along the lines of being able to say easily "I'm running version 5.5" rather than having to post up a git commit hash...

I second the idea of clear version labelling if nothing else. We have it for the hardware - we should do it for the software too, including the sketches on Github. And even better if each version carries a meaningful change note that describes what the change was and the reasons for it. It needs to in each and every file, having it in readme.md is not helpful once the files themselves get separated from the readme.

I raised the issue of versioning a few days ago. I think Tryston is like minded and likely to implement it in some form in emoncms v7.

It would be good if the version was linked to a changelog - OR better still to a properly documented Github read.md (as above from Robert).

Paul

At least, it would be easy to tag all release versions in Git (v6, v7, etc) and to work in a dev branch so that any non contributor user would use a release version, labeled as such in the footer of the GUI. Each new version would be the occasion to sum up the changes into a nicely written changelog. Subversions are possible with the same worklow.

However, users may want bleeding-edge functions and be routed to dev versions. Then we'd be back to the current situation. To avoid this, try to merge features regularly as subversions.

Regular merges also minimize bugfixing duplication.

I use Raspbian (debian based) but the ready-to-go SD card uses another distro. I don't know if it is debian based.

I've started work on this. I should point out from the start that I'm no debian packaging expert, but I figure the important thing is to get going, as there may be other contributors at some point who can improve things, but might not have wanted to start from scratch...

While I believe it's possible to do configuration through the installation of a deb (i.e. script upgrade actions from one version to another) this isn't something I'm familiar with, so to start out I'm just going to get a .deb built which copies the emoncms filesystem into place (under /var/www) and doesn't worry about anything else.

The branches are:

https://github.com/Dave-McCraw/emoncms/compare/debian-packaging

https://github.com/Dave-McCraw/raspberrypi/compare/debian-packaging

(Please note that right now this is untested - I'll post another update when I know it's working).

Thanks for this.

Debian user myself, I never packaged. I'd do it the same way : first, copy files and don't bother with update script.

Then we could try to add a script to create the DB at installation. This would be quite useful (practical for user + be sure everyone uses the same credentials).

Then, an update script to update the DB on version up would be neat.

Then we could remove the now useless "update DB" page from emoncms.

At this point we'd have a really integrated mature debian software.

This means each time we'd need to generate a slightly modified "packageable version" (perhaps not too much of a hassle).

This also mean (or maybe I'm wrong) that users of the deb package would'nt have emoncms in a git repo, so couldn't pull, and they would have to wait for the next release.

We'd have a class of users with easy access (no complicated setup). I think that's what the project aims at.

We'd need to be able to produce a new version regularly. To those users, you can't say "git pull to dev versino and bear with potential issues, or wait months until it is fixed".

Thanks for the effort.

This also mean (or maybe I'm wrong) that users of the deb package would'nt have emoncms in a git repo, so couldn't pull, and they would have to wait for the next release.

I think in an ideal world, we'd set up an apt-repo for the project and configure our Pi installation to use it (in addition to the raspian repo). Then it really would just be a question of getting users to run "apt-get upgrade" to level up... otherwise what we'd do is provide the .deb file as a download from emoncms.org (maybe?) and provide instructions for running a GET / dpkg -i to do the same.

The nice thing about the apt repo approach is that it will force us to use (some kind of) versioning for bug fixes and other intermediate releases, rather than just updating the git master. This is because you need to dput a new higher version number (8.0.1 etc) for it to be picked up by the package manager on the Pi.

There are so many approaches... good opportunity to learn some new skills too :)

Yes, good points about the apt-repo. And yes, what I meant is that they'd still have to wait for the next stable release. Or we'd have to backport fixes from dev branch to master and create intermediate releases. This sort of imposes a nice process. Great.

This looks great, nice work, if you let me know how I can help, I'm happy to do so.

What do we need to do to get the apt-repo setup?

Happy to put the package up on the emoncms.org server, and then i guess its updating install / update documentation and so on.

With a bit more testing I think we can use the 'rework' dev branch which I think is almost good to go for clean installs.

Trystan - an apt repo is a file structure that you just need to host somewhere. I was thinking of putting a test repo on my S3 account for a proof of concept, then you can decide how you'd like to proceed for real.

Going forward I recommend running a different branch for each release of emoncms (i.e. you'd have a release-v9 branch next) which people submit pull requests against. This keeps master clean for something like a critical bugfix but it also means there's a clear structure of when things are merged in (there's an obvious history of merging v9, v10, etc. branches into master).

OK - this is now ready for a test run if anyone fancies - based on the current git masters:

From your Pi:

pi@raspberrypi ~/tmp $ wget http://cdn.mccraw.co.uk/emon/emoncms_7.0_armhf.deb

pi@raspberrypi ~/tmp $ wget http://cdn.mccraw.co.uk/emon/emoncms-raspberrypi_1.0_armhf.deb

pi@raspberrypi ~/tmp $ sudo dpkg -i emoncms*.deb

It won't overwrite the settings.php file, any modules you have installed other than raspberrypi, and it won't remove the .git stuff either, but it will overwrite any local changes you've made to PHP files etc. in the main and raspberrypi modules.

It would be great if someone else can validate this as a clean install (just rename /var/www/emoncms first, run the commands and copy in your settings file... should work fine?)

Or: ignore all of the above, and add the following line to your /etc/apt/sources.list file

deb http://emon-repo.s3.amazonaws.com wheezy unstable

now, I recommend you rename your existing /var/www/emoncms directory if significant, then run

sudo apt-get update

sudo apt-get install emoncms emoncms-raspberrypi

Copy in your settings.php and any other modules from your manual backup, and you should still be in business, just debianised under the hood :)

Great. Shouldn't it be wheezy instead of squeeze ?

If stabilized, we may even be able to enter Raspbian's official repos !

It won't matter, but yeah - I only spotted that after it was too late... day job reflex!

Could you package it for i386 as well?  Thinking that instead of running emoncms on my pi, I'd actually run it on another machine and just forward the data from the pi.

I think someone will need to build it on i386 (or maybe there's a flag to build for other architectures? - I'll need to check). The Pi is my only debian-based box.

Good catch though. There will probably be a lot of these things to iron out (current list: fix the distro version... add more architectures!)

It would be nice if we could get happy enough with it to recommend it as the upgrade procedure to v8 for existing installs.

Yes lets try this for v8, Im not sure that there are different requirements for different debian based distributions as there are no compiled binaries in there, especially now that timestore is not needed, I wonder if its possible to supply a single .deb for all?

Yes it is. Absolutely. The architecture is "any" (or "all") as Schism did. Regarding this, the .deb is fine, I think. It is rather in the repo that something needs to be done differently.

Not an expert, just when I tried to fire it into an Ubuntu box it complained about the architecture.

OK, I think the problem was that I had only specified the arm architecture in the repo's conf/distributions file. I've now added i386 and amd64 to the list and re-reprepro'd the emoncms package - I've also updated the debian version to be wheezy.

It should now be possible to pull emoncms using the above instructions on those architectures (I skimmed the list, and wasn't sure that there were any other common ones just now. Unfortunately you can't specify 'all' in the apt configuration).

Note that /etc/apt/sources.list needs a tweak to refer to wheezy now (edited my original post).

Separately I've been considering how we would ideally install emoncms (with a view to complying with Debian's official apt repositories).

There are huge style manuals that appear to regulate everything in great detail but without examples, so I probably haven't yet found the one simple tutorial that will make it all click.

However, provisionally, I think that the PHP should be installed into something like /usr/share/emoncms/www and we'd then configure apache to serve from that directory instead of just /var/www

I haven't investigated this yet, but I'm aware that the same PHP files are used to serve pages and also by rfm12piphp , and I'm not sure how you would have this set up in a nominally ideal environment. Possibly those files need to be in /usr/lib/ to be available to other packages "correctly" (or even packaged as a separate debian package, like emoncms-common)?

I'm looking at the Linux install guide and trying to see what else we'd want the debian package to do. Seems like using dbconfig-common to create the database would be a good step? 

Having apache config in /etc/emoncms/ is probably another thing to consider.

OK - some more progress on this tonight. Again, I'd appreciate any feedback if someone can spare the time to try an install this way.

I really wanted to get away from having nested git repositories under /var/www as this makes it unnecessarily difficult/dangerous to use git (some commands will be fine, some will trash your install). In any case, using git directly to upgrade installations of emonCMS in the web root isn't an ideal solution for all sorts of other reasons, so:

Instead, I've got emoncms (and the raspberrypi module) installing into /usr/share/emoncms/www, and added a specific apache configuration file for sites-available to get that running (required a tweak to .htaccess and the rfm12piphp script).

As part of debian installation, the apache config is copied into sites-available ready for deployment, while the rfm12piphp script is installed into /etc/init.d (and configured). 

The significant downside of this is that it will make merging up a bitch for anyone who wants to issue pull requests back into emonCMS afterwards, and the directory structure obviously isn't backwards compatible, i.e. you can't just git pull this version of emonCMS into an existing install and expect it to work.

However, I think it would be worthwhile if we can get over the obstacle of initial rollout. It will massively simplify upgrades, tighten version control, and a whole load of other benefits.

If you want to help out by giving this a spin (please!) then here's the step-by-step guide:

Add to /etc/apt/sources.list the following line:

deb http://emon-repo.s3.amazonaws.com wheezy unstable

(I recommend you rename your existing /var/www/emoncms directory if present, then) run

sudo apt-get update

sudo apt-get install emoncms

Now, you need to tell apache to enable emoncms:

sudo a2ensite emoncms

sudo /etc/init.d/apache2 reload

It doesn't create the database yet (but that's about all). You do still need to supply the settings.php file.

For the RaspberryPi module, you just add the following:

sudo apt-get install emoncms-module-rfm12pi

And you need to start up the rfm12piphp script:

sudo service rfm12piphp restart log

And that should be it...

(I haven't tried to add support for the python gateway at this time. I'm not sure why there are two different scripts in the same module, and would have thought that it would make more sense to have these packaged separately rather than deploying both via one debian package. Would you ever run both? Is it because they use the same web status form in emonCMS itself?)

Reminder that you can see the changes at github: emoncms and raspberrypi

[thought occurs to me that git will actually be smart enough to allow merging after the directory restructure. Something to try later]

Great job, Dave.

Forget about the python script. It should be dropped from this module. It was my redesign of the php script and it led up to the standalone gateway script. It is not really maintained. The standalone gateway is.

Or we decide to maintain only the python script. But I agree we don't need both in one package. And having two more packages for the scripts with one default choice is overkill.

I think most if not all users use the php script. The python script is now mostly used in the standalone gateway. I may be wrong, of course, as there is not much feedback about this.

Excellent work Dave. are the packages complete enough for a fresh install as they are currently? I do not have an existing emoncms set up but I would be happy to knock up up a clean image of raspbian to give them a road test. 

Having used the python gateway for sometime with a variety of data sources I'm convinced the python gateway is more versatile but I have little experience of the php gateway so far so I'm just going by forum comments. Wouldn't a single "universal" gateway prevent further fracturing or either the emoncms or standalone camps being left behind and in turn help focus development?

Glyn has blogged about using the standalone python version recently and I personally think it's got alot more potential but as the hdd option is probably more popular than the standalone option its not getting fully utilized.

I have just tried to update the standalone image and as I had some problems I suggested a hybrid gateway image on that thread but had no response.

Is there a reason why we need 2 gateways, I mean is there a strong enough argument for using the php version to justify the need for a standalone as well as a emoncms gateway? 

Jerome's done a great job with the gateway and I for one hope we use it in the emoncms package,

Paul  

Great work Schism, Il try and test it later too.

Jérôme - OK, I will remove the python stuff from the RaspberryPi module, but package what remains as something more specific, like emoncms-raspberrypi-php

This will support the separate packaging of the python / standalone gateway option (emoncms-raspberrypi-python ?). With both packages we will support both existing approaches, and then a discussion can take place on which approach should be preferred. 

pb66 - I haven't had to modify the code to any significant degree, so you should get a fully-functioning install from those two packages. I'm holding off on any of the other modules until there is a general consensus around this approach :)

I haven't actually looked at how either the OEM gateway or php script work, since it's not a prerequisite of the debian packaging and my own install is working fine (using the php approach I assume :) but I agree that it would make sense to standardise on one approach, so effort made by contributors is not divided.

I have literally just loaded both packages now and can get the "It works!" message in a local pc's browser, seems to have gone very smoothly, there where no errors and the instruction spot on. I don't have my own settings.php file and haven't created a database so how should I proceed ? can I complete the install following the existing instructions on emoncms or do something specific?

Paul

OK, sweet!

You'll still need to create the database, then a settings.php file unfortunately. At that point I would expect it to work after a fashion (i.e. things like timestore, redis, etc. are all not affected by this debian packaging exercise. It mainly replaces any work you'd need to do with git).

I've forked the event module, packaged it and added it to my apt-repo. This means if you want to add the event module to an installation you can just run

sudo apt-get update && sudo apt-get install emoncms-module-event

(don't forget you need to set up SMTP / other settings from the options page installed inside emoncms for it to actually do anything).

As discussed above, I've removed the python functionality from the raspberrypi module and repackaged it as emoncms-module-rfm12pi (to make sure it's distinct from the OEM gateway). I marked it as Breaks: emoncms-raspberrypi so nobody can end up confused with both names for the same package installed - if you'd already tried experimenting you need to do a sudo apt-get remove emoncms-raspberrypi before you can do sudo apt-get install emoncms-module-rfm12pi

(edited the instructions above accordingly)

I tried to package the packetgen module next, but unfortunately it seems to require changes to be made to one of the files from the raspberrypi module (!!)

IMHO this isn't a great approach. For instance, it would wipe out any branched version of raspberrypi that you installed, or if you later installed an upgrade, packetgen will stop working.

If nobody is familiar with this and able to advise then I will have a dig around (can't we either add packetgen support into raspberrypi permanently? Or install a separate PHP class alongside raspberrypi so they work by composition and not overriding?)

Phew... I think I might have gone a bit blind trying to figure this all out, but I've hopefully made some more solid progress.

If you install emoncms from my apt-repo you'll now be prompted for DB username, password, database and host, and these values are now used to create the database and also to write out a settings.php file automatically.

They default to the usual values (root/raspberry/localhost/emoncms) but can be changed - and if you do change them, that will persist when new emoncms versions are installed - you only change it after initial installation by running sudo dpkg-reconfigure emoncms (or edit the file by hand - but then it will potentially be overwritten next version).

The contents of /var/lib/timestore/adminkey.txt are used to populate the $timestore_adminkey property in settings.php - I can't see why that would cause an issue, so please sing out if I've missed the point (I wasn't sure why we have this value in two config files, there could be installation ordering issues for instance?)

I added mysql as a dependency so it will be installed when emoncms is installed, removing another step from the instructions.

Great news that you've included the DB creation in the installer package,

The other anomaly I found during testing was redis, the original emoncms set-up seems to assume redis is installed and set up. it's not until you get this error "Can't connect to redis database, phpredis is not installed, see readme for redis installation" that you become aware and read the forum threads (or the instructions). so should that also be a dependency or change the default install process to not assume or check and at least recommend or warn maybe.

I will give the newer version a run later and report back.

Will the emoncms package install and operate independently of the emoncms-module-rfm12pi module if the RFM12Pi isn't used?

Paul.

Trying it out here on a blank copy of raspbian, I got the following error, I set the database password to root when mysql prompted for the password.

Attempting to create 'emoncms' mysql database (with the 'root' user)...
ERROR 1045 (28000): Access denied for user 'root'@'localhost' (using password: YES)
Failed (check username and password are correct, using dpkg-reconfigure. Check grants allow table creation, etc...)
dpkg: error processing emoncms (--configure):
subprocess installed post-installation script returned error exit status 1
Processing triggers for menu ...
Errors were encountered while processing:
emoncms
E: Sub-process /usr/bin/dpkg returned an error code (1)

Would it be possible to add redis installation too?

Love being able to type!!:

sudo apt-get install emoncms

@paul- I think v8 makes redis optional, IIRC.

Will the emoncms package install and operate independently of the emoncms-module-rfm12pi module if the RFM12Pi isn't used?

Yes. However, if you tried to install emoncms-module-rfm12pi it would complain if emoncms itself hadn't been installed :)

@trystan, try running sudo dpkg-reconfigure emoncms --force and providing root/root as the DB username and password. That should then set up the DB (you may then need to apt-get install emoncms again to get it to mark as complete).

TBH I'm not that experienced in this side of debian packaging. There's probably a way of forcing the questions to be asked the first time (I assumed they would be, but it sounds like you didn't get a prompt, it just used the defaults?)

OK, I think it's because the debconf/priority isn't high enough in my packaging work. I've used 'low', probably should be 'critical'. Then I think it will ask you when you install regardless of raspian settings, whereas if 'low' is under a default threshold it just uses the defaults instead of asking.

Tried the revised installer and got the same results as Trystan,

Ran sudo dpkg-reconfigure emoncms --force and was prompted for username>password>database>host, each with the default values to overwrite and it "attempted" to build the DB, (although I guess successfully that wasn't confirmed).

After sudo a2ensite emoncms && sudo /etc/init.d/apache2 reload I was able to get the redis error message in browser.

From there I attempted to retrospectively install redis using

sudo apt-get install redis-server php-pear php5-dev -y

sudo pecl install redis

sudo sh -c 'echo "extension=redis.so" > /etc/php5/apache2/conf.d/20-redis.ini'

sudo sh -c 'echo "extension=redis.so" > /etc/php5/cli/conf.d/20-redis.ini'

which last night I thought was the winning combination following a bit of trial and error, but unfortunately this didn't work.today. It does report being successful but with this info 

Build process completed successfully
Installing '/usr/lib/php5/20100525+lfs/redis.so'
install ok: channel://pecl.php.net/redis-2.2.4
configuration option "php_ini" is not set to php.ini location
You should add "extension=redis.so" to php.ini

I having tried it this way without success so I shall now try installing redis before emoncms to see if that is any easier.

i'll shall let you know

Paul

Hey Paul,

Thanks for the continuing trial & error. If it doesn't explode creating the DB then it worked OK. I can make it print a positive confirmation.

I'm off tonight (got to sleep some time :) but will try to look into Redis over the weekend, if someone doesn't get there first.

D.

Tried installing redis first and then emoncms, this is tidier and reaches the log-in screen. currently redis definitely needs to be first, maybe a different procedure would allow redis to be installed after and emoncms reconfigured but while emoncms is expecting redis as default it's better if's there before installation. just for the record I used

sudo apt-get install redis-server php-pear php5-dev apache2 -y
sudo pecl install redis
sudo sh -c 'echo "extension=redis.so" > /etc/php5/apache2/conf.d/20-redis.ini'
sudo sh -c 'echo "extension=redis.so" > /etc/php5/cli/conf.d/20-redis.ini'

after adding the package address to /etc/apt/sources.list I installed emoncms

sudo apt-get update -y
sudo apt-get install emoncms -y --force-yes
mysqlpassword
mysqlpassword
sudo dpkg-reconfigure emoncms --force
root
mysqlpassword
emoncms
localhost
sudo a2ensite emoncms
sudo /etc/init.d/apache2 reload

this resulted in a the following error message in the browser

Can't connect to database, please verify credentials/configuration in settings.php
Error message: Access denied for user 'root@'localhost' (using password: YES)

When i checked the settings.php file mysqlpassword had not been updated by the dpkg-reconfigure so I corrected it and ran sudo /etc/init.d/apache2 reload again and I reached the login screen. 

I'm a little unsure of exactly what's supposed to happen at this point It wouldn't allow a new user to be set up and I have tried every password username/password combo I can find on both sites and the details used during DB set up, to no avail and I haven't been able to find any default user info in the source or using myphpadmin and  either, maybe someone could confirm the current exact initial login situation please.

Paul

From http://openenergymonitor.org/emon/node/2360 try raspi and raspberry

Paul, what happens when you run mysql -uroot -p -e 'select username from emoncms.users' ? That'll tell you the username at least.

When i checked the settings.php file mysqlpassword had not been updated by the dpkg-reconfigure

This is pretty weird. Did it have _DEBCONF_PASSWORD_ instead? (When you run reconfigure it copies the new settings.php into place each time, so it will either leave that placeholder or sub in the configured value).

I just quickly tried the sql query and it returned nothing, just returned to prompt after I entered password. Tried entering a wrong password which return an error so I assume its working but has no results.

As for the settings.php, I thought it was weird at the time and although I can't remember what it before I edited I'm pretty sure it wasn't the placeholder. I'm kicking myself for not remembering (bit of a ahhaaaa! moment while I was doing other things), it wasn't until I was writing the post it occurred to me, sorry!

I will later tonight start over with a fresh image again and follow the routine in my last post, unless you change something in the package and advise differently.

PB

Is it possible to steamline the redis installation with this package:

dh-make-php - Creates Debian source packages for PHP PEAR and PECL extensions

(http://www.dotdeb.org/2008/09/25/how-to-package-php-extensions-by-yourself/)

I think it would remove the need for manually doing the pecl stuff?

Paul - I've now got emoncms v8 debian packaged (see parallel topic), so hopefully we can kill two birds with one stone. If you start clean and can do it without redis, then can add redis afterwards, I guess that would be the ultimate. (Unfortunately the v8 checklist shows redis set up first, although presumably now not a requirement).

Hertog: I'm not sure, but there is a php5-redis debian package already which might do what that PECL stuff is doing. It would be interesting to see if anyone has any success with that, since it would simplify redis just to apt-get install php5-redis (only from Jessie/testing though).

Topic on packaging the new v8.0

Just tried installing from fresh with just

sudo sh -c 'echo "deb http://emon-repo.s3.amazonaws.com wheezy unstable" >> /etc/apt/sources.list'
sudo apt-get update -y
sudo apt-get install emoncms -y --force-yes

and sailed through to the passwords and settings all present and correct (I just entered through all email defaults this lap) 

sudo a2ensite emoncms
sudo /etc/init.d/apache2 reload

emoncms login screen loaded straight up 1st time BUT I still can't get in ! It won't allow me to create a new user and raspi/raspberry doesn't work. mysql -uroot -p -e 'select username from emoncms.users' comes back empty can someone please confirm what is supposed to happen, is there meant to be a default user profile or should it allow one to be created?

If I should create a user are there any rules regards number of characters or must have numerals or uppercase etc

All I get is a pink bar ! (see attached)

The installation process was fantastic, by far the best to date (although I haven't tried to install redis yet) but I have not yet succeeded in getting past the login screen (I have in the past using git) Does the debian packaging change the login process? 

Paul

I'm not sure what's going on here. The debian packaging shouldn't be making any difference to any actual code (the main impact is on things which look at specific file paths, like the gateway scripts).

Just to double check, I logged out and registered a new account with my local emoncms v8 / debian install, and it worked fine.

Compare what you're entering with the rules etc. from https://github.com/emoncms/emoncms/blob/cf2c6584c81ad7873f866abf4781082314caccb4/Modules/user/user_model.php#L148  ?

Alternatively, you could try searching in the mysql standard log for an INSERT INTO users statement (that would tell you if the issue is DB-related).

@Trystan, any ideas?

checked the rules they seem ok and I have tried a variety so I think we're good there.

I was just trying to digest this thread http://openenergymonitor.org/emon/node/2502 similar symptoms but the solution isn't clear, well not to me anyways :-)

Took a look at the apache logs and found several instances of this pair of errors

[Sat Mar 15 23:22:05 2014] [error] [client 192.168.0.201] File does not exist: /usr/share/emoncms/www/user, referer: http://192.168.0.214/emoncms/
[Sat Mar 15 23:22:05 2014] [error] [client 192.168.0.201] script '/var/www/index.php' not found or unable to stat, referer: http://192.168.0.214/emoncms/

neither file exists. In /var/www/ there is only index.html and there is no user file or directory in /usr/share/emoncms/www/.

PB

Ahhah! found um!

/usr/share/emoncms/www/index.php

/usr/share/emoncms/www/Modules/user

are they actually in the wrong place or are they just being looked for in the wrong places ?

PB

a little more info, I crudely tried copying index.php across to /var/www just to see if I could progress at all and in place of the pair of messages we now have

[Sat Mar 15 23:57:39 2014] [error] [client 192.168.0.201] File does not exist: /usr/share/emoncms/www/user, referer: http://192.168.0.214/emoncms/
[Sat Mar 15 23:57:39 2014] [error] [client 192.168.0.201] PHP Warning:  require(process_settings.php): failed to open stream: No such file or directory in /var/www/index.php on line 22, referer: http://192.168.0.214/emoncms/
[Sat Mar 15 23:57:39 2014] [error] [client 192.168.0.201] PHP Fatal error:  require(): Failed opening required 'process_settings.php' (include_path='.:/usr/share/php:/usr/share/pear') in /var/www/index.php on line 22, referer: http://192.168.0.214/emoncms/

which suggests to me process_settings.php's not where it's expected either and doesn't tell me where it is expected to be.

Paul

Don't bother trying to move the files around, that's a red herring.

/var/www/ is just a directory where apache looks by default to serve files. When you run a2ensite emoncms, we add /usr/share/emoncms/www/ to that list, so http://192.168.0.214/emoncms/index.php should serve /usr/share/emoncms/www/index.php

It's working enough that you're able to try and register. Something could be interfering with mod_rewrite I suppose. 

I wonder if it would be interesting to compare our other apache configs.

OK, here's my /etc/apache2/sites-enabled/000-default

<VirtualHost *:80>
        ServerAdmin webmaster@localhost

        DocumentRoot /var/www
        <Directory />
                Options FollowSymLinks
                AllowOverride None
        </Directory>
        <Directory /var/www/>
                Options Indexes FollowSymLinks MultiViews
                AllowOverride None
                Order allow,deny
                allow from all
        </Directory>

        ScriptAlias /cgi-bin/ /usr/lib/cgi-bin/
        <Directory "/usr/lib/cgi-bin">
                AllowOverride None
                Options +ExecCGI -MultiViews +SymLinksIfOwnerMatch
                Order allow,deny
                Allow from all
        </Directory>

        ErrorLog ${APACHE_LOG_DIR}/error.log

        # Possible values include: debug, info, notice, warn, error, crit,
        # alert, emerg.
        LogLevel warn

        # CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

And here's my /etc/apache2/sites-enabled/emoncms (symlink)

Alias /emoncms /usr/share/emoncms/www
<Directory /usr/share/emoncms/www>
  Options FollowSymLinks
  AllowOverride All
  DirectoryIndex index.php
  Order allow,deny
  Allow from all
</Directory>

Both of those files are good,

I've pm'd you my /etc/apache2/apache2.conf for you to take a look (no point in posting a full file that big in the thread)

Paul

I have tried this again from scratch and still unable to log in, the 2 errors logged with each attempted log in say

File does not exist: /usr/share/emoncms/www/user

script '/var/www/index.php' not found or unable to stat, 

has anyone successfully installed from scratch?

any suggestions as to what else to check would be great. I have been following the other v8 upgrade threads and see the debian packaged route works when replacing a previously installed version but have.nt notice any mention of a fresh install.

Having confirmed the apache2.conf, 000-default & emoncms symlinks are as expected i'm looking for another cause and have been a little thrown by this in the Emoncms + RaspberryPI + Hard drive build notes

Emoncms uses a front controller to route requests, modrewrite needs to be configured:

$ sudo a2enmod rewrite
$ sudo nano /etc/apache2/sites-enabled/000-default
Change (line 7 and line 11), "AllowOverride None" to "AllowOverride All". That is the sections and . [Ctrl + X ] then [Y] then [Enter] to Save and exit.

in the working examples above (and mine) line 7 & 11 are both set to "AllowOverride None". which is correct? Neither resolve this issue but it makes me wonder if these values are significant  why doesn't it make any difference when changed? are they being ignored, over written or sourced from some where else? 

I'm clearly out of my depth with this & could do with some assistance unless of course, it's on hold while sorting the v8 &/or git to debian issues first. I'm ok to wait if it is.

Paul

The debian packaged version of emoncms has its own apache config with all necessary settings, so you shouldn't need to worry about configuring anything (although you would need to ensure mod rewrite is enabled using the a2enmod command).

I havent' tried a clean install yet. I really need another pi to use as a dev box :(

Can you get a clean install of the regular emoncms working? If so, it would be interesting to know what the differences are. 

If you capture the output of dpkg -l (lower case L) you can compare what's installed on your current (deb packaged) attempt and the working clean install from git.

We can't proceed until a clean install works, so I'm at the point where I may take my pi out of service temporarily to see if I can find the issue (I'm currently using it to monitor actively, so it's a bit awkward).

OK - if I install a second copy of emoncms to a different path than /emoncms/ then I get the same errors you do. Now we're getting somewhere!

Is your install by any chance not running on /emoncms/ as a path?

Yep - I didn't have the .htaccess file installed on that attempt (missed it with the cp command) and putting it in / taking it out impacts that error message. So, just need to figure out how to get the .htaccess code to work for nonstandard installations.

OK: I previously solved this then evidently forgot all about it. My debian-packaged install had the following just under "RewriteEngine on":

# Set a base of emoncms to allow us to install outside the apache document root
RewriteBase /emoncms

With this in place, I no longer get 

[Tue Mar 18 20:03:05 2014] [error] [client 192.168.1.8] File does not exist: /var/www/usr

Instead, I just get the site.

Any help? (Confused as to why this wouldn't be in place on your system though - see https://github.com/Dave-McCraw/emoncms/blob/debian-packaging/www/.htaccess ).

I think the path is correct as the first error says File does not exist: /usr/share/emoncms/www/user which is because the file is located in the modules folder /usr/share/emoncms/www/modules/user and the second says script '/var/www/index.php' not found or unable to stat, and that's located in the /usr/share/emoncms/www/index.php which I believe to be right.

I have just checked .htaccess and its identical to the one you've linked ie it already has RewriteBase /emoncms and yet it's still giving those errors. And just to clarify I do get to the site but am not able to log in or create a user.

I can try a git derived version to compare, the thought had crossed my mind but beyond seeing if its possible I'm not sure I would know what to look for. I have previously had a fully functional v7 emoncms using the same RPi+HDD+Emoncms image as you but I have since moved away for various minor reasons.

If you want to see the installs side by side to compare you could set up another hdd partition (even if only temporary) and install virgin raspbian on that partition, that way your current monitoring would only be offline long enough to install and then switch back.

I have 3 bootable hdd partitions (4 if you include the sdcard) all with various projects etc and switch between them using these 2 lines to edit the /boot/cmdline.txt just before rebooting. 

sudo sh -c 'echo "dwc_otg.lpm_enable=0 console=tty1 root=/dev/sda1 rootfstype=ext4 elevator=deadline rootwait" > /boot/cmdline.txt'

sudo shutdown -r now

Just changing /dev/sda1 for the target drive to boot to, switching drives takes seconds, I can therefore install an update or try something and switch back within a couple of minutes and then go on to mount, inspect, edit etc from the original image.

if you think it will help I'll free up a second partition and install emoncms from git tomorrow if I get the time.

Paul

I think the path is correct as the first error says File does not exist: /usr/share/emoncms/www/user which is because the file is located in the modules folder /usr/share/emoncms/www/modules/user and the second says script '/var/www/index.php' not found or unable to stat, and that's located in the /usr/share/emoncms/www/index.php which I believe to be right.

I think the first error shows that your install is failing to mod rewrite properly, because we never actually /want/ to serve the user module as a file/directory in its own right.

What mod_rewrite should be doing is taking the path after /emoncms/ (i.e. 'user') and passing that to index.php as a parameter. "# Rewrite URLs of the form 'x' to the form 'index.php?q=x'."

You should be able to get to the right page if you visit /emoncms/index.php?q=user ..?

If so, we just need to figure out what's wrong with your mod_rewrite. You could try the rewrite log directive in your main apache2.conf file - add at the bottom the following lines and restart apache:

RewriteEngine On
RewriteLog "/var/log/apache2/rewrite.log"
RewriteLogLevel 5

If you tail it, you should see it serving rewrites as you hit the site. The request which actually tries to serve the user directory (literal path) isn't being rewritten, and the second error strongly suggests that something is incorrectly hitting index.php in the root of the filesystem.

If you consider the RewriteRule, you'll see it would normally take everything (.*) i.e. all of "/emoncms/user" and send it to "index.php?q=/emoncms/user" (index.php being in /var/www/) but the RewriteBase tells it to make everything relative to /emoncms/ which is why I think it works... it then takes just /user/ and maps that to /emoncms/index.php

I'm not that hot on mod rewrite though, to be honest... maybe this needs something more?

A couple of other thoughts that you could try:

If you stick [R=301] on the end of the RewriteRule thus:

RewriteRule ^(.*)$ index.php?q=$1 [L,QSA,R=301]

It should send your browser a permanent redirect rather than doing it serverside. This means that you'll be able to tell exactly whether or not mod_rewrite is working for your site (for any of the modules) because it will show the real URL in your browser bar.

Could you try adding that and browsing about (including to the user module) and tell me what's in your URL bar when you hit issues?

I've had another quick go at this and

http://192.168.0.214/emoncms/index.php?q=user goes to a blank white page with just the black "log-in" band at the top and the "powered by" footer

adding 

RewriteEngine On
RewriteLog "/var/log/apache2/rewrite.log"
RewriteLogLevel 5

Results in an error when restarting apache so I changed it back, have I missed some formatting or something?

pi@raspberrypi ~ $ sudo nano /etc/apache2/apache2.conf
pi@raspberrypi ~ $ sudo /etc/init.d/apache2 reload
[FAIL] Reloading web server config: apache2 failed!
pi@raspberrypi ~ $ sudo nano /etc/apache2/apache2.conf
pi@raspberrypi ~ $ sudo /etc/init.d/apache2 reload
[....] Reloading web server config: apache2apache2: Could not reliably determine the server's fully qualified domain name, using 127.0.1.1 for ServerName
. ok

adding R=301 to the RewriteRule in /usr/share/emoncms/www/.htaccess made no noticeable difference, restarted apache, I tried 2 browsers and cleared the caches. I tried other changes to RewriteRule and nothing changed, so I deleted it and that made no change either.

I 'm guessing this means the .htaccess is not being used,  presumably it is not being found by apache and therefore not redirecting.I searched and didn't find a duplicate anywhere.

PB

When we crack this, it will be quite satisfying huh?

Are you sure mod_rewrite is working on your system?

For instance, try creating an .htaccess in /var/www/ containing the following:

RewriteEngine on
RewriteRule ^(.*)$ http://openenergymonitor.org/ [R=301,L]

Point your browser at a path like http://pi/foo - does it redirect you to this site? If we can confirm that the basics are working that's a start.

I find it reassuring you say "When we crack this" and not "If we crack this", I also hope its not something really daft, but it usually is :-)

I guess mod_rewrite is not working as I get "This page can’t be displayed - Make sure the web address http://pi is correct."

PB

Sorry, I just put "pi" as a stand-in for whatever your pi IP address is. (Maybe I should have said "edit your emoncms URL so it says /foo instead of /emoncms/")

Doh! That is so obvious now you point it out, sorry!

So I tried it again same result.

Although since my last post I have discovered Module rewrite had "dropped" out at some point. I cannot explain how but I have just rebooted and checked module rewrite starts automatically as expected, but whilst trying the new .htaccess I tried restarting both the modules rewrite and apache2 before trying the "pi" url. It wasn't until I tried it again that I noticed I got a message stating "Module rewrite already enabled" that I realised the previous "restarting module rewrite" signified it wasn't already running. but I can think of no reason why that would of stopped.

Since restarting mod_rewrite I am getting slightly different results in that although I still cannot log in I am now getting an error message in the pink warning box saying "Missing username, password or email parameter" that definitely was not there before today, I can now see the full "ugly" url (R=301 is still in place) and it doesn't record anything in the error.log when login fails.

I have tried several user/password combos and tried to create a new user. all user/password combo's result in "Username or password empty" whether they are or not.

url "http://192.168.0.214/emoncms/index.php?q=user" now works also.

But http://MYPIIP/foo does not. can you confirm exactly what i should put in /var/www. I have tried just

RewriteEngine on
RewriteRule ^(.*)$ http://openenergymonitor.org/ [R=301,L]

and I have also tried copying my existing .htaccess, commenting out the old RewriteRule and adding the oem.org one.

maybe i should start from clean again

PB

no, I think we're good now and you can forget the foo experiment...

The error from the login page suggests that our rewrite is throwing away form parameters (even though it shouldn't be, due to the QSA flag). Have you tried returning it to virgin state, without the R=301?

Yes, I did remove the R=301 whilst trying to back track and workout when/how the changes occurred and removing the R=301 seemed to have no effect when removed.

I'm not confident the install is in a "virgin" state as I'm getting slightly different results (text in pink box and no error.log)  So something must be different even though I have tried to note and undo everything we've tried.

R=301 is what causes you to see the "ugly" URL, as it sends your browser a redirect rather than servicing the request transparently. If you remove R=301 you should stop seeing the "ugly" URLs.

I wonder if you could try the RewriteLog again. It may be that the error you had with Apache restarting was caused by mod_rewrite not being enabled when you last tried this?

http://192.168.0.214/emoncms/ url remains as entered and this is output to RewriteLog

192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c3c058/initial] (3) [perdir /usr/share/emoncms/www/] strip per-dir prefix: /usr/share/emoncms/www/ ->
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c3c058/initial] (3) [perdir /usr/share/emoncms/www/] applying pattern '^(.*)$' to uri ''
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c3c058/initial] (4) [perdir /usr/share/emoncms/www/] RewriteCond: input='/usr/share/emoncms/www/' pattern='!-f' => matched
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c3c058/initial] (4) [perdir /usr/share/emoncms/www/] RewriteCond: input='/usr/share/emoncms/www/' pattern='!-d' => not-matched
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c3c058/initial] (1) [perdir /usr/share/emoncms/www/] pass through /usr/share/emoncms/www/
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/subreq] (3) [perdir /usr/share/emoncms/www/] strip per-dir prefix: /usr/share/emoncms/www/index.php -> index.php
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/subreq] (3) [perdir /usr/share/emoncms/www/] applying pattern '^(.*)$' to uri 'index.php'
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/subreq] (4) [perdir /usr/share/emoncms/www/] RewriteCond: input='/usr/share/emoncms/www/index.php' pattern='!-f' => not-matched
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/subreq] (1) [perdir /usr/share/emoncms/www/] pass through /usr/share/emoncms/www/index.php
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c4d058/initial] (3) [perdir /usr/share/emoncms/www/] strip per-dir prefix: /usr/share/emoncms/www/Lib/jquery-1.9.0.min.js -> Lib/jquery-1.9.0.min.js
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c4d058/initial] (3) [perdir /usr/share/emoncms/www/] applying pattern '^(.*)$' to uri 'Lib/jquery-1.9.0.min.js'
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c4d058/initial] (4) [perdir /usr/share/emoncms/www/] RewriteCond: input='/usr/share/emoncms/www/Lib/jquery-1.9.0.min.js' pattern='!-f' => not-matched
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c4d058/initial] (1) [perdir /usr/share/emoncms/www/] pass through /usr/share/emoncms/www/Lib/jquery-1.9.0.min.js
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (3) [perdir /usr/share/emoncms/www/] strip per-dir prefix: /usr/share/emoncms/www/Lib/bootstrap/css/bootstrap-responsive.css -> Lib/bootst$
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (3) [perdir /usr/share/emoncms/www/] applying pattern '^(.*)$' to uri 'Lib/bootstrap/css/bootstrap-responsive.css'
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (4) [perdir /usr/share/emoncms/www/] RewriteCond: input='/usr/share/emoncms/www/Lib/bootstrap/css/bootstrap-responsive.css' pattern='!-f' $
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (1) [perdir /usr/share/emoncms/www/] pass through /usr/share/emoncms/www/Lib/bootstrap/css/bootstrap-responsive.css
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (3) [perdir /usr/share/emoncms/www/] strip per-dir prefix: /usr/share/emoncms/www/Lib/bootstrap/css/bootstrap.css -> Lib/bootstrap/css/boo$
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (3) [perdir /usr/share/emoncms/www/] applying pattern '^(.*)$' to uri 'Lib/bootstrap/css/bootstrap.css'
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (4) [perdir /usr/share/emoncms/www/] RewriteCond: input='/usr/share/emoncms/www/Lib/bootstrap/css/bootstrap.css' pattern='!-f' => not-matc$
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (1) [perdir /usr/share/emoncms/www/] pass through /usr/share/emoncms/www/Lib/bootstrap/css/bootstrap.css
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (3) [perdir /usr/share/emoncms/www/] strip per-dir prefix: /usr/share/emoncms/www/Lib/bootstrap/js/bootstrap.js -> Lib/bootstrap/js/bootst$
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (3) [perdir /usr/share/emoncms/www/] applying pattern '^(.*)$' to uri 'Lib/bootstrap/js/bootstrap.js'
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (4) [perdir /usr/share/emoncms/www/] RewriteCond: input='/usr/share/emoncms/www/Lib/bootstrap/js/bootstrap.js' pattern='!-f' => not-matched
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (1) [perdir /usr/share/emoncms/www/] pass through /usr/share/emoncms/www/Lib/bootstrap/js/bootstrap.js
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c3c058/initial] (3) [perdir /usr/share/emoncms/www/] strip per-dir prefix: /usr/share/emoncms/www/Modules/user/user.js -> Modules/user/user.js
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c3c058/initial] (3) [perdir /usr/share/emoncms/www/] applying pattern '^(.*)$' to uri 'Modules/user/user.js'
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c3c058/initial] (4) [perdir /usr/share/emoncms/www/] RewriteCond: input='/usr/share/emoncms/www/Modules/user/user.js' pattern='!-f' => not-matched
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c3c058/initial] (1) [perdir /usr/share/emoncms/www/] pass through /usr/share/emoncms/www/Modules/user/user.js
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5334058/initial] (3) [perdir /usr/share/emoncms/www/] strip per-dir prefix: /usr/share/emoncms/www/Theme/emoncms_logo.png -> Theme/emoncms_logo.png
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5334058/initial] (3) [perdir /usr/share/emoncms/www/] applying pattern '^(.*)$' to uri 'Theme/emoncms_logo.png'
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5334058/initial] (4) [perdir /usr/share/emoncms/www/] RewriteCond: input='/usr/share/emoncms/www/Theme/emoncms_logo.png' pattern='!-f' => not-matched
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5334058/initial] (1) [perdir /usr/share/emoncms/www/] pass through /usr/share/emoncms/www/Theme/emoncms_logo.png
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (3) [perdir /usr/share/emoncms/www/] strip per-dir prefix: /usr/share/emoncms/www/Theme/favicon.png -> Theme/favicon.png
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (3) [perdir /usr/share/emoncms/www/] applying pattern '^(.*)$' to uri 'Theme/favicon.png'
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (4) [perdir /usr/share/emoncms/www/] RewriteCond: input='/usr/share/emoncms/www/Theme/favicon.png' pattern='!-f' => not-matched
192.168.0.201 - - [19/Mar/2014:16:36:48 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5c44058/initial] (1) [perdir /usr/share/emoncms/www/] pass through /usr/share/emoncms/www/Theme/favicon.png

clicking the Log-in button in the top right hand corner changed url to http://192.168.0.214/emoncms/index.php?q=user/login and this was RewriteLog.

192.168.0.201 - - [19/Mar/2014:16:39:53 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5334058/initial] (3) [perdir /usr/share/emoncms/www/] strip per-dir prefix: /usr/share/emoncms/www/index.php -> index.php
192.168.0.201 - - [19/Mar/2014:16:39:53 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5334058/initial] (3) [perdir /usr/share/emoncms/www/] applying pattern '^(.*)$' to uri 'index.php'
192.168.0.201 - - [19/Mar/2014:16:39:53 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5334058/initial] (4) [perdir /usr/share/emoncms/www/] RewriteCond: input='/usr/share/emoncms/www/index.php' pattern='!-f' => not-matched
192.168.0.201 - - [19/Mar/2014:16:39:53 +0000] [192.168.0.214/sid#b6c6eec0][rid#b5334058/initial] (1) [perdir /usr/share/emoncms/www/] pass through /usr/share/emoncms/www/index.php

The login button results in a ugly url still (R=301 removed) but thats the only webpage I can currently access to try.

I have started backing up another partition once that is done I can install a second copy from fresh

Okay.. this is strange I have just entered exactly the same details a second time to compare with or without R=301 when entering new user details and it's done it! it has created a new user. all i've changed is deleted the R=301 again, which I have done several times in a row now. I can navigate around no problem.

wooohooo. I'm not sure should do now! i don't want to break it again!

PB

Great!

Leave that partition as it is, start from scratch and if it doesn't work, we can compare the two? :)

I couldn't resist putting the api key in my other pi running the ro gateway 1st and local v8 is now logging data from an emontx 1 node of 5 inputs :-)

I will start a fresh install on another partition shortly,

Completely fresh install from scratch on latest raspbian image and emoncms has installed perfectly first time.

Easily set up a new user when I got to the emoncms login screen and then everything seemed to work.

I am going to try it again a couple of times and also try installing the additional modules.

If it installs ok every time I will create a complete set of feeds and dashboards to monitor for a while, but it certainly seems once your in all is ok - not sure why we had problems yet

When it installs perfectly first time this is quite impressive, very little interaction required.

Installed event module using

sudo apt-get install emoncms-module-event

unfortunately selecting Extras > Event resulted in this message in browser

Fatal error: Call to a member function fetch_array() on a non-object in /usr/share/emoncms/www/Modules/event/event_model.php on line 67

tried restarting apache & mod-rewrite (not getting caught out again) no joy rebooting didn't help emoncms still working but not the events module are there any other install or set up requirements for the events module?

Paul

[edit: confused myself for a sec there]

The line referenced above is a query on the event table. Have you tried running 'update database' inside the admin tab of emoncms?

Superb ! that got it.

I guess I should test the emoncms-module-rfm12pi next,

Are any other module packages ready to test ?

Paul

So I gave the rfm12pi module a whirl even though I didn't have a rfm12pi attached as I assumed that wouldn't effect installing the package, was that right? when I ran

sudo apt-get install emoncms-module-rfm12pi

it seemed to install ok, so I ran 

sudo service rfm12piphp restart log

sort of half expecting it to complain about there being no rfm12pi connected but it returned with

Log is turned on
Restarting RPIWARNING: are you sure raspberrypi_run.php is running ?
pi@raspberrypi ~ $ PHP Fatal error:  Class 'Redis' not found in /usr/share/emoncms/www/Modules/raspberrypi/raspberrypi_run.php on line 28

 

not sure what's occurring here with the error message, the "Restarting RPIWARNING" script seems to output the error message and a new line to the screen after the return prompt and then just wait, you can enter for a new prompt or just type a command and enter. the message and the prompt just seem to be printed to screen in the wrong order the prompt and curser function correctly it just looks a little odd.

The message itself suggests remnants of the redis default installation I guess as I have not attempted any installation of redis.

PB

Good catch, so it looks like redis is not optional for the rfm12piphp module: see the master repo.

Trystan might have plans to fix this as he has made Redis optional on the core package.

I bit the bullet and have a second Pi ordered up for dev (only costs the same as another emonTH and will let me help directly).

Ah missed that, could you try in:

raspberrypi_run.php

removing the lines:

    $redis = new Redis();
    $redis->connect("127.0.0.1");

and replacing with:

    $redis = false;

Thanks Trystan, I changed the redis lines as suggested and got this 

pi@raspberrypi ~ $ sudo service rfm12piphp restart log
Log is turned on
Restarting RPIWARNING: are you sure raspberrypi_run.php is running ?
pi@raspberrypi ~ $ PHP Notice:  Undefined variable: timestore_adminkey in /usr/share/emoncms/www/Modules/raspberrypi/raspberrypi_run.php on line 37
PHP Warning:  fopen(): Unable to find the wrapper "dio.serial" - did you forget to enable it when you configured PHP? in /usr/share/emoncms/www/Modules/raspberrypi/raspberrypi_run.php on line 82
PHP Warning:  fopen(dio.serial:///dev/ttyAMA0): failed to open stream: No such file or directory in /usr/share/emoncms/www/Modules/raspberrypi/raspberrypi_run.php on line 82
PHP Warning:  stream_set_timeout() expects parameter 1 to be resource, boolean given in /usr/share/emoncms/www/Modules/raspberrypi/raspberrypi_run.php on line 83
PHP Warning:  fclose() expects parameter 1 to be resource, boolean given in /usr/share/emoncms/www/Modules/raspberrypi/raspberrypi_run.php on line 326

I tried a "Update & check" in emoncms admin which resulted in

Fatal error: Call to a member function fetch_object() on a non-object in /usr/share/emoncms/www/Modules/admin/update_class.php on line 67

Paul

That looks like it might be down to a missing dependency - have you done the PEAR commands from the install guide? (DIO is one of the three things, along with redis and swift).

The debian package should really depend on DIO, but we need to roll our own debian package (this was discussed up-topic, but I haven't tried to process it since it's uncertain whether this will make the repo master in any case).

I see the problem, will fix it and update the repo

I've made the changes, here's the commit info https://github.com/emoncms/raspberrypi/commit/9eea52de6448318f2f1cd4c68d71cb94db8c0b43

Dave - No I hadn't as I am a little hazy as to precisely what needs to be installed or set-up and I didn't want to just take a stab at it. I had thought the previously discussed missing dependencies related to redis and where no longer an issue.

Trystan - I assume the changes are not relative to DIO, just redis, is that right? I just tried the edits manually and got the same (revised) results minus the error referencing line 37 ie lines 82, 83 & 326 were still an issue. and also 67 when update db used.

I have also just tried another fresh install this time I installed emoncms, event & rfm12pi all at once, it took less than 8 minutes total to update, upgrade, install, initialize & log-in (up to the point of the (original) redis error). 

Paul

Hey Paul,

The raspberry pi module depends on the PHP DIO library, which is only available through PEAR, not as a debian package. If we move forward with debianizing emoncms then I'd anticipate hosting a custom package inside our apt repo which contains DIO, and adding a formal debian dependency, so that when you apt-get install emoncms-module-rfm12pi you automatically get dio into the bargain, and it just works...

Hi

I have tried the emoncms installation follow the debian installer docs on emoncms.org (adding to /etc/apt/sources.list the line deb http://emon-repo.s3.amazonaws.com wheezy unstable) but I get a 'Couldn't find package' error.

Repo seems to be correctly added to sources.list and i run apt-get update as well.

I am trying it in a lubuntu fresh install on a virtual box machine, could this affect?, is repo working? (in a debian wheezy virtual machine doesn't work for me too)

greetings and thanks

It sounds like it might be connectivity between you and S3. The repo is just a collection of static files, assuming Amazon don't go down, it can't really "stop working".

What happens if you hit the repo with a GET request? It should display the bucket contents, including the deb files. If that doesn't work, you'll need to figure out why your box can't see Amazon?

True, if amazon s3 fails everybody would realize about it.

So, I can see the xml contain (deb packages, etc..) when I access http://emon-repo.s3.amazonaws.com/ through a web browser.

Also apt-get update works for every other program and I have tried in a no virtual environment (Linux Mint Ubuntu based) but It keeps without finding the packages (not only emoncms, modules too), quite weird. 

Terminal output

lubuntu@lubuntu-VirtualBox:~$ sudo apt-get update
[sudo] password for lubuntu:
Hit http://archive.ubuntu.com quantal Release.gpg
Hit http://extras.ubuntu.com quantal Release.gpg
Ign http://emon-repo.s3.amazonaws.com wheezy Release.gpg
Hit http://archive.ubuntu.com quantal-updates Release.gpg
Hit http://extras.ubuntu.com quantal Release                        
Hit http://archive.ubuntu.com quantal-backports Release.gpg         
Hit http://emon-repo.s3.amazonaws.com wheezy Release                 
Hit http://archive.ubuntu.com quantal-security Release.gpg           
Hit http://archive.ubuntu.com quantal Release                        
Ign http://emon-repo.s3.amazonaws.com wheezy/unstable i386 Packages/DiffIndex 

...

Reading package lists... Done
lubuntu@lubuntu-VirtualBox:~$ sudo apt-get install emoncms
Reading package lists... Done
Building dependency tree      
Reading state information... Done
E: Unable to locate package emoncms
lubuntu@lubuntu-VirtualBox:~$

Are you sure that you've added the correct entry to your apt.sources file? The emoncms repo is exposing packages as unstable, not release. But then, I see lower down your output that it's aware of unstable... puzzling!

"Ign" rather than "hit" might be a clue? TBH I'm not really sure...

I keep hoping this extra Pi will turn up. Clearly I badly need two environments to support this properly, sorry!

Hello everybody,

I have the same problem as qpro.

I have tried in a virtual machine with Ubuntu and a other with Debian (last version).

Somebody have a solution?

Thanks

Jon

Can you post your apt.sources file in full?

Looking at qpro's post with a fresh eye, the stand out thing is that we've put wheezy in as the distro, but the system is running quantal (ubuntu 12.10), so I suspect this is what causes the problem updating.

Could you update your apt.sources entry so it reads just:

deb http://emon-repo.s3.amazonaws.com unstable

and let me know what happens?

With deb "http://emon-repo.s3.amazonaws.com unstable" in "etc/apt/sources.list" gives an error 

lubuntu@lubuntu-VirtualBox:~$ sudo apt-get update
E: Malformed line 58 in source list /etc/apt/sources.list (dist parse)
E: The list of sources could not be read.

Ok, I can download the package  if i put in a browser directly http://emon-repo.s3.amazonaws.com/pool/unstable/e/emoncms/emoncms_8.0.3-3_armhf.deb (is it main one?)

Maybe the error is related a with repo directory structure?

I either need to upload the package to the repository in a different way, such a way that it can be accessed by your system, or figure out the correct sources syntax... not sure which!

OK - I think that should now work. Can you try using the line

deb http://emon-repo.s3.amazonaws.com quantal unstable

Sorry for the delay but I wanted tried with ubuntu and debian.

I have add the file of source.list for debian and ubuntu.

Just for information, I have install debian 7.4.0 (32bit) and ubuntu 13.10 (64bit).

I have tried to change for "deb http://emon-repo.s3.amazonaws.com unstable" and i have the same problem of qpro.

If I change for "deb http://emon-repo.s3.amazonaws.com quantal unstable", I have the error:

E: Unable to locate package emoncms

I have also tried to edit in the terminal (nano), gedit launch in terminal with sudo and command sudo sh -c 'echo ... >> /etc/apt/sources.list'

Yeah, you're running ubuntu "saucy" whereas I added support for "quantal". I'd need to fix the repo to support an extra distro (too many distros, argh).

However, the debian / wheezy file you've posted looks OK to me. Very odd.

Yes, I have found the readme on your git for pkg-emoncms.

I have check for ubuntu LTS (12.04) and it's doesn't work also. I have tried to this version because I think is "more near" to 12.10. The second reason is I don't found to ISO for 12.10.

Now, I have found the 12.10 version and I download it for check.

I have tried with the 12.10 version of Ubuntu and it's doesn't work :-/

Doesn't work for me too

lubuntu@lubuntu-VirtualBox:~$ sudo apt-get update

Hit http://archive.ubuntu.com quantal Release.gpg
Get:1 http://extras.ubuntu.com quantal Release.gpg [72 B]
Ign http://emon-repo.s3.amazonaws.com quantal Release.gpg                
Get:2 http://archive.ubuntu.com quantal-updates Release.gpg [933 B]
Hit http://extras.ubuntu.com quantal Release                              
Ign http://emon-repo.s3.amazonaws.com quantal Release
Hit http://archive.ubuntu.com quantal-backports Release.gpg
Hit http://archive.ubuntu.com quantal-security Release.gpg           
Ign http://emon-repo.s3.amazonaws.com quantal/unstable i386 Packages/DiffIndex

[...]

W: Failed to fetch http://emon-repo.s3.amazonaws.com/dists/quantal/unstable/binary-i386/Pac... 404  Not Found

E: Some index files failed to download. They have been ignored, or old ones used instead.
lubuntu@lubuntu-VirtualBox:~$
lubuntu@lubuntu-VirtualBox:~$ sudo apt-get install emoncms
Reading package lists... Done
Building dependency tree      
Reading state information... Done
E: Unable to locate package emoncms
lubuntu@lubuntu-VirtualBox:~$

If you could all just use wheezy it would make life so simple :-)

Annoyingly I don't even have a suitable system to test an Ubuntu live CD on, so it's hard to know what's up. Perhaps it would be better to withdraw the claim that it should work for anything other than debian/raspbian until someone comes along who knows this stuff better than I do.

Will try to double check it tonight. I'm not really sure why it wouldn't work, since it's just a way to present static files, and I'm using reprepro to set up up for both distros.

Aha, got it. Apparently when I push a new version (8.0.8) it deletes it from all distributions. I just pushed it up to wheezy, hence why quantal now doesn't work again!

Some kind of script is clearly needed...

Hi Schism, Are you willing to make a package for 8.3.2?

fluppie007, I've contacted Schism and he's busy at the moment but has pointed me towards the procedue for updating the debian package which is up here: https://github.com/Dave-McCraw/emon-repo

I will take a look at it and update on how I get on.

It should be stated somewhere in the documentation that the debian package lags behind the github version.  I see the package is 8.3.2 and git is 8.3.6

Hello,

I use emoncms since a year now and I rencently wanted to update my emoncms version. I tried to use the debian repo method.

My server run on debian wheezy i686 architecture. 

After adding the repo and updating, it tells me that that it couldn't locate the emoncms package.

Is this method still working?