I'm running emoncms on my main PC - it does not run 24hrs. My emonBase is a raspberry Pi - it does run 24hrs. The Pi updates the database with calls like ...
curl -s http://blodwyn/emoncms/input/post.json?time=1369038236&node=10&csv=356,1......
When the main PC is off the Pi saves all these calls in a local file, and when the Pi detects that the main PC is on again, it sends all the stuff it couldn't do when the main PC was off and also sends the current data.
So far so good.
The problem is on the emoncms Feeds page while the data is being brought up to date the "Updated" column can show times like "6 hours ago" for realtime. Clearly this is not right, because the SQL database has more up-to-date data than this as the Pi is also sending current data, indeed sometimes when the page refreshes, some lines will say "6 hours ago" but others will say "3 secs". It appears to me that the feeds page is simply taking the last record on file, and not treating the SQL data in any time ordered way.
Re: Updating emoncms data and Feeds Page problems
Good idea to update a emoncms install on your computer from the raspberrypi, Id been thinking of something like this too, I think its could be a good solution to concerns about longevity of the SD cards and also with the data on one's main pc you can access it a little faster than across the web and also get at the data directly for any visualization and data analysis development that one might want to do. So great to hear you've done it!
The feeds list last updated time is updated not from the individual feed table with all the feed data in but from a descriptor table called feeds, this table should always hold the last feed value and last feed updated time. If your sending the data to the input/post.json? api then this should all work as normal and times should display ok.
Is your raspberrypi and computer time the same? After an update does it always show 6 hours ago? are the 6 hour ago feeds realtime data and the 3 secs feeds daily and histogram datatypes?
Re: Updating emoncms data and Feeds Page problems
Thanks for the reply.
I had not realised that the Feeds page was getting it's data from the feeds table. I had assumed it was getting its' data from the individual tables feed_1, feed_2 etc. Then, because I was expecting realtime data, I suppose it depends on the definition of realtime - is it the reading for the current time, or is it the readings for the most recent data received ? Most often these are the same thing, but not always. I had assumed it was the readings for the current time.
Here are some queries showing how the table was updating this morning...
mysql> select * from feeds;
+----+-------+--------+------+---------------------+-------+----------+--------+
| id | name | userid | tag | time | value | datatype | public |
+----+-------+--------+------+---------------------+-------+----------+--------+
| 2 | grid | 1 | NULL | 2013-05-21 23:09:46 | 356 | 1 | 0 |
| 3 | pv | 1 | NULL | 2013-05-21 23:09:46 | 4 | 1 | 0 |
| 4 | volts | 1 | NULL | 2013-05-21 23:09:46 | 24648 | 1 | 0 |
+----+-------+--------+------+---------------------+-------+----------+--------+
3 rows in set (0.00 sec)
mysql> select * from feeds;
+----+-------+--------+------+---------------------+-------+----------+--------+
| id | name | userid | tag | time | value | datatype | public |
+----+-------+--------+------+---------------------+-------+----------+--------+
| 2 | grid | 1 | NULL | 2013-05-22 09:11:13 | 82 | 1 | 0 |
| 3 | pv | 1 | NULL | 2013-05-22 09:11:13 | 191 | 1 | 0 |
| 4 | volts | 1 | NULL | 2013-05-21 23:09:52 | 24614 | 1 | 0 |
+----+-------+--------+------+---------------------+-------+----------+--------+
3 rows in set (0.00 sec)
mysql> select * from feeds;
+----+-------+--------+------+---------------------+-------+----------+--------+
| id | name | userid | tag | time | value | datatype | public |
+----+-------+--------+------+---------------------+-------+----------+--------+
| 2 | grid | 1 | NULL | 2013-05-21 23:10:03 | 354 | 1 | 0 |
| 3 | pv | 1 | NULL | 2013-05-21 23:09:57 | 3 | 1 | 0 |
| 4 | volts | 1 | NULL | 2013-05-21 23:09:57 | 24648 | 1 | 0 |
+----+-------+--------+------+---------------------+-------+----------+--------+
3 rows in set (0.00 sec)
mysql>
You can see how the time shifts about as the Pi rolls in all the historic data as well as loading in the current data readings. Now I understand a bit more. I guess next time I'll look at the code.
Incidentally, if anyone is thinking of doing something similar, it takes about 35 minutes to roll in about 6000 updates.