Can't create new feed - [SOLVED]

Whenever I try and create a new feed it throws an error about the feed being undefined and from then onwards the "feeds" tab shows a white screen with loading on it, and nothing has been created in /home/pi/data/phpfina whereas I think the feed exists in mysql. The only way back from this point appears to be a restore of an earlier SD Card image, just restoring with mysqldump gave me 6 undeletable undefined feeds!

I've checked the permissions and they are:

folder /home/pi/data/phpfina has Owner:www-data Group:root drwxr-xr-x

​All the files within /home/pi/data/phpfina are Owner:www-data Group:www-data -rwxrw-rw-

Folder /home/pi/data has Owner:pi Group:pi drwxrwxrwx

​Folder /home/pi/data/mysql has Owner:mysql Group mysql drwx-------

These seem the same as the 22-dec image - does anyone know what's wrong?

I'm running Jessie on a Pi2 having followed the new EMONPI SD card build instructions. I am running the latest EMONHUB / MQTT configuration although I think the fault could have been present before I upgraded EMONHUB

Paul Reed's picture

Re: Can't create new feed - [SOLVED]

What exactly does the error say.

Paul

sheppy's picture

Re: Can't create new feed - [SOLVED]

In a popup box 192.168.1.35 says:

"ERROR: Feed could not be created, undefined.

Afterwards in the Inputs window the process list column has "wait..." in every field that had a set of actions in it,

Clicking on the Feeds Button under "Setup" now has a white screen with "Feeds" top left and "loading..." in the middle.

Selecting my normal dashboard gives white spaces where the gauges should be, and clicking on setup. then on a guage and configure gives me the setup box for the guage with an empty feed drop down list.

Just a thought, could it be trying to create the feed files in the wrong place?

sheppy's picture

Re: Can't create new feed - [SOLVED]

Could the SQL Database I'm importing be corrupt? I just did a test with the 22nd Dec Image and the only change being the SQL user needed to get it working. I could create feeds OK on this, but after doing a restore and creating a new feed, it screwed it up as above. I can how ever read the export file and the only obvious thing is the feeds have a hole in the middle of the numbering range where I deleted some in the past, there are no orphaned phpfina data files.

How can I turn on the logging so I can see what is missing? I've set loglevel=1 in settings.php but it seems to have very minimal logging on restart and nothing afterwards.

Is there a way of testing the imported database for errors?

Another test I did was taking my working image and deleting the SQL database and the feed files, then recreating the database and creating new feeds, before stopping the feedwriter service and overwriting the phpfina files. All worked well until I rebooted and then it had the same empty feeds page and process fields with "wait..." on them

glyn.hudson's picture

Re: Can't create new feed - [SOLVED]

Can you tell me the contents of /var/lib/mysql? This folder should be empty since we have moved the database to /home/pi/data

sheppy's picture

Re: Can't create new feed - [SOLVED]

curiously enough exactly the same as /home/pi/data/mysql but earlier filetimes and only accessible by issuing sudo su first - see below

root@kdf-jessie:/var/lib/mysql# ls --full-time
total 28688
-rw-r--r-- 1 root  root         0 2016-02-02 12:42:52.475573000 +1300 debian-5.5.flag
drwx------ 2 mysql mysql     4096 2016-02-02 13:23:48.471506214 +1300 emoncms
-rw-rw---- 1 mysql mysql 18874368 2016-02-02 13:32:25.215652594 +1300 ibdata1
-rw-rw---- 1 mysql mysql  5242880 2016-02-02 13:32:25.195652821 +1300 ib_logfile0
-rw-rw---- 1 mysql mysql  5242880 2016-02-02 12:43:04.185573000 +1300 ib_logfile1
drwx------ 2 mysql root      4096 2016-02-02 12:43:56.285573000 +1300 mysql
-rw------- 1 root  root         6 2016-02-02 12:43:56.765573000 +1300 mysql_upgrade_info
drwx------ 2 mysql mysql     4096 2016-02-02 12:43:51.355573000 +1300 performance_schema

root@kdf-jessie:/home/pi/data/mysql# ls --full-time
total 28794
-rw-r--r-- 1 root  root         0 2016-02-02 12:42:52.475573000 +1300 debian-5.5.flag
drwx------ 2 mysql mysql     1024 2016-02-02 16:52:04.088431250 +1300 emoncms
-rw-rw---- 1 mysql mysql 18874368 2016-02-09 17:58:13.111509138 +1300 ibdata1
-rw-rw---- 1 mysql mysql  5242880 2016-02-09 17:58:38.279993017 +1300 ib_logfile0
-rw-rw---- 1 mysql mysql  5242880 2016-02-02 12:43:04.185573000 +1300 ib_logfile1
drwx------ 2 mysql root      4096 2016-02-02 12:43:56.285573000 +1300 mysql
-rw------- 1 root  root         6 2016-02-02 12:43:56.765573000 +1300 mysql_upgrade_info
drwx------ 2 mysql mysql     1024 2016-02-02 12:43:51.355573000 +1300 performance_schema

 

curiously enough exactly the same as /home/pi/data/mysql and only accessible by issuing sudo su first

 

sheppy's picture

Re: Can't create new feed - [SOLVED]

I cut and pasted the details into another post and unfortunately its hit the moderation queue. In summary same files, later dates in /home/pi/data/mysql

See attached

ive just checked the instructions on GiT, should I delete the /bar/lib/MySQL files? The datadir parameter points to /home/pi/data/MySQL

glyn.hudson's picture

Re: Can't create new feed - [SOLVED]

Yes delete the whole folder:

sudo rm -rf /var/lib/mysql 

We have now moved the database to /home/pi/data if you followed:

https://github.com/emoncms/emoncms/blob/master/docs/RaspberryPi/read-onl...

sheppy's picture

Re: Can't create new feed - [SOLVED]

Deleting the folder didn't help, adding a symbolic link between the two didn't either.

I followed the guide which copies the files rather than moves them which is why the old ones remained.

I've just loaded up an older image on my test system created from the 22 Dec EMONPI Image, which I'd restored my data to and it doesn't seem to have this problem. The files are present in /var/lib/mysql on this image too so I don't think that's the problem

EDIT: I've just turned on logging, created a new feed, and seen the entries in mysql.log. I then dumped the mysql table and it seems that the mysql bit is working on the dead system. In mysql the table added:

(48,'node:Power:Fridge',1,'Node Power',NULL,NULL,1,0,NULL,5,'') which is a similar format to the other entries, however there is no 48.dat or 48.meta in /home/pi/data/phpfina.

Copying another feed into number 48 returns feeds into working order.after a reboot, but sadly creating another breaks everything again. Each time copying an existing feed to the missing feed id followed by a reboot makes everything work again.

The failure happens after the sql entry for the feed is created but before the blank feed file is created and before the input is tied to the feed.

Turning on logging in emoncms and comparing a working image with a broken one it seems both log the first 2 lines below:

2016-02-10 00:45:18.978|INFO|feedwriter.php|Starting feedwriter script
2016-02-10 00:45:19.719|INFO|feed_model.php|EngineClass() Autoloaded new instance of 'RedisBuffer'.

BUT the faulty one doesn't have the next 4 lines.

2016-02-10 00:45:30.608|INFO|feed_model.php|EngineClass() Autoloaded new instance of 'PHPFina'.
2016-02-10 00:45:30.610|WARN|PHPFina.php|get_meta() meta file does not exist '/home/pi/data/phpfina/49.meta'
2016-02-10 00:45:30.616|INFO|PHPFina.php|create() feedid=49
2016-02-10 00:45:30.625|INFO|feed_model.php|create() feedid=49

The question is why and how to fix it

glyn.hudson's picture

Re: Can't create new feed - [SOLVED]

Apologies, I meant: 

sudo rm -rf /var/lib/var/lib/redis/*

The issue is due to redis loading an old database. There is a note in step 8 to remove all folders in redis folder

https://github.com/openenergymonitor/emonpi/blob/master/docs/SD-card-bui...

sheppy's picture

Re: Can't create new feed - [SOLVED]

Did you mean sudo rm -rf /var/lib/redis/* ?

there was a file called "dump.rdb" in there which I got rid of and rebooted. Unfortunately the problem remains although a quick look at the logs suggest it's subtly different. I'll take a look at it later and add the log entries to my post.

Unfortunately I can't get something repeatable in the time I have available this evening so I'm going to have to spend some time on it in the morning. Please let me know if anything springs to mind. Suffice it to say I still can't create feeds and it appears that redis restarts just after Apache starts on booting.

is there any documentation around that explains exactly how new feeds are created and by which processes? I'll happily hack it if I know where to look

 

sheppy's picture

Re: Can't create new feed - [SOLVED]

OK, things have changed a little, now I can create a feed and it makes all the others disappear.

Creation of the feed:

2016-02-12 03:34:41.322|INFO|feed_model.php|EngineClass() Autoloaded new instance of 'PHPFina'.
2016-02-12 03:34:41.325|WARN|PHPFina.php|get_meta() meta file does not exist '/home/pi/data/phpfina/48.meta'
2016-02-12 03:34:41.329|INFO|PHPFina.php|create() feedid=48
2016-02-12 03:34:41.331|INFO|feed_model.php|create() feedid=48

Fortunately a reboot brought them back, the existing ones in "NoGroup" and the new one in "Node:Power" which is related to the "/emon/Power/AV" topic I added it to.

Unfortunately creating another feed gave me the undefined error again from the start of this thread and broke the feeds page again. Nothing in the log was created that I could see but after a reboot when selecting feeds I got the log attached.

Clicking on any of the inputs gives a message of "Feedid 23 does not exists or was deleted" even though the feeds still exist in /home/pi/data/phpfina.The number "23" is obviously different on the other feeds.

Can you offer any suggestions as to what to look at next?

I also tried wiping out the mysql database and recreating it again, which worked well until I rebooted after I'd created the first 20 feeds, after the reboot with nothing else changed it was all stuffed again.

sheppy's picture

Re: Can't create new feed - [SOLVED]

If anyone else has this problem please add to this thread - it would help to know if I'm the only one especially as the problem seems to extend to the latest February beta image.

Xino's picture

Re: Can't create new feed - [SOLVED]

Yes, having the same issue on a fresh install of emonSD12Feb16 - emonPi.

/var/lib/redis/ is empty

tsavikos's picture

Re: Can't create new feed - [SOLVED]

I made new install emonSD12Feb16 to SD card.

I didn't backup-import any database.

/var/lib/redis -directory is empty.

I succeeded to create some feed, but after switching other page, I cannot create any more feeds.

All I got is error message:

Feed -- Add:

  • ERROR: Feed could not be created, undefined

 

I realize that /home/pi/data/phpfiwa -directory did not exists. I created this manually. - no changes

Also I chmod all files in phpfina, phpfiwa and phptimeseries to 777 - no changes

After reboot, several errors in emoncms.log for missing *meta files. I created dummy 4,5,6.meta files

However I got errors like this:
2016-02-14 08:24:47.294|WARN|PHPFina.php|get_meta() meta file does not exist '/home/pi/data/phpfina/7.meta'
-- and this number changes to next one what is missing.

and feed page is now showing : "loading ..."

What could be missing in setup?

sheppy's picture

Re: Can't create new feed - [SOLVED]

@tsavikos this issue is now being discussed in the 12th February thread and it is something the developers are working on, take a look towards the bottom of http://openenergymonitor.org/emon/node/12164

@Xino thanks for adding to the thread, its good to know I'm not the only one. The issue is present in the latest image so hopefully a fix will appear soon

sheppy's picture

Re: Can't create new feed - [SOLVED]

This has been fixed by updating redis, the fix courtesy of Glyn Hudsen copied from the 12th February Image thread is below:

We have a solution! 

It turns out that php5-redis needed to be updated to work with the latest version of php. The latest version of php5-redis available through Debian Raspbian Jessie and currently installed on the emonSD-12Feb16 image is 2.2.5-1.

You can check php5-redis version in the sources with 

sudo apt-cache show php5-redis

Installing the latest stable version of redis (2.2.7) direct from phpredis github fixed the issue. Here are the steps to update php5-redis: 

git clone --branch 2.2.7 https://github.com/phpredis/phpredis​
cd phpredis
​cat php_redis.h | grep VERSION
phpize
./configure
sudo make
sudo make install

 

gapintheclouds's picture

Re: Can't create new feed - [SOLVED]

Thanks for pointing me towards Glyn's fix. Works a treat :-)!

Bramco's picture

Re: Can't create new feed - [SOLVED]

Just an fyi

I had to use http not https to do the git clone or git gave me an error 400.

alco's picture

Re: Can't create new feed - [SOLVED]

this is because the repo is now 2.2.7 in the master branch. so change the url a bit:

  1. wget https://github.com/phpredis/phpredis/archive/2.2.7.tar.gz
  2. tar -xvf 2.2.7.tar.gz
  3. phpize
  4. ./configure
  5. sudo make
  6. sudo make install

 

 

Bramco's picture

Re: Can't create new feed - [SOLVED]

Thanks Alco, although the http rather than https worked for me.

To make your instructions complete you need to add in the cd to the directory before the phpize.

Simon

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.