Does anyone have experience of expanding the emoncms built-in graphing capabilities to take data from more than one contributor? We're interested in using the data collected from a number of users dotted round the country in various teaching scenarios.
Chris.
KMi, The Open University
http://projects.kmi.open.ac.uk/greendata/
Re: Graphing data from multiple contributors?
Hello Chris, your project looks really interesting! I've been discussing this with Dominic Zapman of Carbon Coop who we are working with on their smart grid project in Manchester. There's a bit on this here: http://carbon.coop/blog/zapaman/community-smart-grid-prosumer-perspective
The idea is to develop the 'my energy' app in emoncms (or possibly break this out into its own app/module) to aggregate generation and consumption data across a number of households in a group such as the one in Manchester. With aggregated demand, the demand curves should be smoother and matching higher to the variable renewable supply. Looking at generation and supply on an individual household basis could lead to the wrong conclusions, as it might suggest using a battery to distribute or shift peaks (such as kettles, electric cooking, short high power), whereas, it may be more efficient overall, to use the grid to handle the peaks and then storage to handle larger aggregate demand shifts..
The 'my energy' app currently pulls in data from multiple sources, both solar and consumption, from the user's account and then UK grid, wind and co2 intensity data from an account on emoncms.org that's recording that data. It then builds the visualisation on the fly.
I think for larger aggregations of many systems, it might be better to have a background task that polls each individual account for the latest value. Say for solar generation, or house consumption, and then aggregates and records to an aggregator emoncms feed. This would provide a privacy advantage for home consumption, given enough houses in the aggregation, it would be very hard to pick out any particular household's consumption pattern, which would be better than sending each households data to a browser for aggregation on the fly.
Aggregating with a background process, then serving one feed will also greatly improve dashboard load times.
If you're not fixed about the data to aggregate at the start, and page load times are less of a worry, then perhaps building a visualisation front-end that can pull in feeds that are public across a series of accounts might be a good approach. That might be more useful as a research tool.
Re: Graphing data from multiple contributors?
Thanks Trystan - have emailed you.
I don't think we'd be too worried about load times.
Chris.
Re: Graphing data from multiple contributors?
I am taking this further now that I have data from three contributors - and obviously we're expecting many more. I am using CanvasJS for graphing, a package we've used in other projects, and custom PHP scripts to extract the relevant data.
One of the issues is that the database structure does not identify "comparable" feeds - that is there is no flag or indicator, when setting up a feed, that a particular feed represents, say, Solar PV watts. As far as I can see, the feed labels that people enter are purely free text. Without this, its impossible to ask of the system "give me a graph of everyone's power generation for the period 1st Jan 2015 to 1st Jan 2016. To add this would require changes to the feed registration process. Have you got any suggestions for how I should go about this in the "right" way? I'm afraid I'm not a "proper" user of git!
I've already added extra fields to the users table to store "role" (indicates a contributors role inside or outside the University) and "sources" (which renewable energy sources they have). The latter, combined with "location" (in which we store part postcode) has allowed me to create an interactive map using the Google Maps API:
http://projects.kmi.open.ac.uk/greendata/map.php