Hello,
First of all, thanks for putting together OpenEnergyMonitor, great hardware, service and community etc.!
One thing that's struck me as a newbie, is that there seems to be knowledge in the embedded code about each data collected (e.g. type of data, units, and perhaps things like which physical port), but that this all gets discarded by the layers between it and the EmonCMS, and then has to be recreated - which is laborious and error-prone (not to say difficult / intimidating when setting things up for the first time!).
Some possible ways to fix this:
1. Output metadata along with the data every time e.g. have the embedded code output JSON directly.
+ Simple
- Possibly wasteful of batteries and bandwidth for radio devices
2. Output metadata one-off or periodically (e.g. on establishment of communications, or every 10 minutes etc.)
+ Common datatypes could nicknamed to save bandwidth, and embedded code size, or just a hash of the full metadata description string transmitted etc. e.g. code which output.
- Grows the code a bit (but for the basic case e.g. output the metadata once on establishement of comms - hardly any).
I've done a quick proof of concept using the EmonShield 3 phase power code, by getting it to output json over serial, and modifying emonhub to pass that onto EmonCMS - it's a bit rough and could be improved, but it works.
To extend this, I was wondering about using the ATmega's EEPROM to store stuff such as a serial number, node name (e.g. "basement4"), and passing this up too.
Calibration values could also be stored in the EEPROM - so that these aren't hard coded, and could be updated without touching the code, and also set at production time etc.
Such changes wouldn't be mandatory (e.g. for cases where code size / bandwidth / battery etc. efficiency is essential), but relatively simple changes to the emonhub could accomodate it automatically recognizing and handling a "new style" output format - thus removing a lot of daunting (for the beginner) manual setup work.
Any thoughts?
Tim.
Re: Mind the metadata gap
This is very much related to the current development discussions for the link between emonhub and emoncms. emonhub is basically holding the device data on a per node or value basis eg name, unit, scale, datatype etc and this data is going to be made available to both emoncms and a custom sketch uploader.
I would be very interested to hear how your trials go, especially with the use of the eeprom for calibration data. the calibration data could also be held with the planned firmware version and hardware reference (device type) and could also be updated/maintained by emonhub via the serial or rfm network so that calibration edits can be done editing the values in the emonhub.conf via an emoncms page (which is currently the plan for the other settings) and emonhub would pick up the changes and upload them to the device.
Perhaps down the line a emoncms module that allowed to enter a manual meter reading could suggest (or implement?) a new calibration figure based on a data comparison? could be a winner !!
I'm not sure the units, names etc need to be held on the eeprom if they are in emonhub.conf as the sketch and device do not use them directly and they cannot be included in the standard transmissions but potentially even node ids could be set in the eeprom too which moves towards removing the need to edit the sketch at all.
Trystan is working on a "local-only" version where emoncms reads emonhub.conf directly and I am currently looking at an alternative way where emonhub will keep emoncms informed of any changes to the conf so that a remote emoncms can be used with 1 or more emonhubs.
Paul