Archived Forum

Hi Bruno,

There isn't currently an off the shelf solution for uploading historical data, The script discussed in your "Import .csv files" thread could be used to present data of any age to emoncms, the .xls files are easily converted to a csv format the script can read or the script can be altered to suit a raw csv file.

The difficulty is at the emoncms end, feed data of -5years to +48hrs can be accepted by emoncms but it can only write to feeds that have a start point earlier than the data being saved. So if you where to be be able to create a feed with a backdated meta file it might be possible to save data up to 5 years old.

This isn't the case with the low-write version of emoncms, you could manually create a feed from historic data for it to read but part of the "low-write" changes prevent any data being overwritten or backdated beyond the last entry.

As a global rough rule of thumb emoncms modules need to be copied or cloned into the "Modules" folder of emoncms and then just go to admin -> update&check. any module specific instructions are normally explained in the modules readme.md.

Regards using a windows server, I'm sure it is just a capable as a linux server but I think the difficulties you have experienced to date are indicative of things to come if you persist with windows. The majority of users use linux for emoncms it's certainly developed on linux so the support from the community will be linux biased, and getting answers to windows questions maybe difficult.

Paul

Do you know how to create such feed? 

We'll probably set a new server in linux (not a shared server) to be able to have access to a linux command line and install some modules using the readme guidelines. Hopefully that server comes with redis from default :p

Not off-hand, but I wouldn't expect it to be too difficult.

Perhaps you and Marc( mpruessmeier ) could collaborate as I believe he is looking for something very similar in his "Post Processing" thread.I have not long ago added a couple of links to hopefully help him find a solution.

I would be happy to chip in if I can but I'm busy working on something else right now and I do not have the required knowledge (or time to research it) to do it for you.right now.

Paul

Also back on your original thread Frankster ( fversteegen ) is trying to do the exact same thing too, using your script.

Thanks Paul :)

Having the ability to create new feeds which use past data is something that I believe is very important and that I'll need in the future (for example, if need to create a new ratio - kWh/ºC or something - I most likely wanna see the past ratio not only the future ones).

Hope this can be done "easily" :D

In fact if you use a PHPtimeseries you may find it easier as there is no meta file and so you just need to "insert" old data into the system created feed.

Take a look at this python phptimeseries reader as a starting block or there is this (similar ) reader written in PHP or this PHPfina to PHPtimeseries converter that could be combined to create a "CSV to PHPtimeseries converter" maybe.

Paul

Just saw you had posted while I was writing, It is a feature that many people may find useful, I think the reality is there is not (currently) enough demand for in-built past data conversion, but being able to add historic data as a one time upload is a huge step in the right direction.

The need to to historic calculations is exactly what Marc requires so as I pointed out in his thread currently the best (only) way to do that is externally. But once you have cracked it once it will be easy to replicate, possibly even automate or implement into a module perhaps.

Paul