EmonCMS in a Software as a Service Infrastructure

Hello guys, long time no see, a new family member and a job switch has kept me away from (most of) the fun stuff.

I have been recently given the opportunity to connect what I do for a living (design software,network and hardware systems) and what I do when I want to have fun (play with cool stuff like arduino and emoncms): specifically I have been asked to evaluate costs and benefit of building a complete Software as a Service infrastructure with the specific goal of monitoring fleets of photovoltaic installations.

The current number of installations is about 6K, all of which would be retrofitted with a monitoring system that is able to post sensors data to an url, initially:

  • PV production  kWh counter
  • HOUSE consumption kWh counter
  • PV inverter status
  • temperatures
  • local energy diversion system state
  • possibly wheather data

Since all of this data will probably have to be transmitted through mobile/gprs, sampling frequency will be between 1-15 min depending on gsm SIM data projected costs (no pseudo real time power reading, that is)

The system should be able to:

  • gather sensors data
  • display aggregated data at various levels
  • react to events like inverter failure and trigger notifications
  • integrate power production and consumption data with local weather conditions
  • (in the future) act as the base for more detailed analitycs reasonings


I believe that, with the correct approach, a modicum of new modules, and a lot of forward thinking and coding, emoncms could be one of the basic building blocks for such an infrastructure.

Specifically, I would like to try and use emoncms as the part that:

  • collects data with single photovoltaic install granularity :
    • one user per site (and relative write apikey)
    • nodes preconfigured based on templates (the monitoring hardware will be uniform through the sites, but will undergo different revisions and could log more/less data depending on connectivity availability)
  • displays such data to at least three levels of users (with different granularity):
    • fleet owners 
      • ​ global data visualization (read only)
      •  partners data breakdown and visualization (read only)
      •  sites data breakdown and visualization (read only)
      • new partners/sites management
      • device/node management 
    • partners/installers
      • partners data visualization
      • sites data breakdown (read only)
      • device/node setup/assignment to specific sites
    • site users/end clients
      • data visualization (read only)
  • reacts to events and forwards alerts

I know it would probably mean a lot of work, but in the eventuality that it moves from the thinking and prototipying stage to reality I would like to do it in  such a way that benefits all users of the emoncms platform.

Things that I think would be necessary to handle all the listed functionality:


  • superuser module: a module that allows an admin/partner to
    • create new users and assign them to the appropriate (fleet/partner/site) category
    • associate specific hardware nodes (see below) to sites
    • handle hardware node maintenance: an admin should be able to enable/disable posting of sensor data from a data node depending on pre-set conditions
  • supernode module: a module that allows the definition of specific nodes in terms of inputs and related processes:
    • hardware node: should reflect different versions/configurations of the hrdware doing the actual monitoring
    • collector node: should define the logic with which to pre-aggregate sensors reading at the partner/fleet level
  • custom frontend app module: should take care of correctly displaying data to various levels of users, would probably start from the mysolarpv view and expand on that

At the moment I think emoncms could handle all the requirements except the visual analytics part (displaying data aggregated on variables like site installs, weather condition, pools of sites/partners ) for which emoncms would still be the provider of pre-computed data.

Data collection would be performed through either emonhub or a custom mqtt interface to emoncms, still undecided on that and will depend on the result of stress tests if/when we come to that

I am using this post as a general brainstorming session, mainly for myself ... I have been thinking/tinkering on how to shape it up using emoncms for a week now, and will probably start to play with basic versions of the superuser and supernode modules in the next week.

I have read other posts that discussed strategies for adding hierarchies of users to emoncms, but I honestly don't see how a hierarchical model could work using the actual interface, specifically I see no way of easily aggregating data if the single users have control on feeds and processes, and see no way (other that with an external app) of displaying in a secure way limited amount of information


I also have been thinking a lot about system requirements, both in terms of storage and in terms of connectivity, will probably post my reasonings in another message

All modules developed for this purpose would naturally be shared with  the community, and this would apply to the frontend, if it turns out to be feasible to keep everything in sync with emoncms development.


Please keep in mind that as of now everything is still at the design level, and all of this exists only in my mind (and on a couple of test installations on AWS :) )

Wil keep updating as soon as I have more specifics.






TrystanLea's picture

Re: EmonCMS in a Software as a Service Infrastructure

Hello Mattia, this sounds like a really interesting project and great to hear that your considering using emoncms. I can see these features being useful for quite a few people installing systems in this way. I think Ildefonso Martínez may have been working on something similar. Very interested to hear how this develops, I've also got a cloud focused version of emoncms in the works with the option to have as many storage servers as you need as well as a more streamlined input stage which may be of interest as you get to the numbers of sites your talking about.

Mandela's picture

Re: EmonCMS in a Software as a Service Infrastructure

It is awesome Mattia I would like to encourage you to try as am now struggling to learn how to come up with such wonderful module for emonCMS_user privileges and permission. Keep it ups


Mattia Rossi's picture

Re: EmonCMS in a Software as a Service Infrastructure

Hi, sorry for the late reply,

in these two weeks I have been experimenting more with backend options, the frontend (and the user module) are for the moment still in the planning department (s.g. some ideas in my head :)
Unfortunately the time I am able to dedicate to this project is small, so I am procedding, but at a really slow pace.

I have evaluated/experimented with a couple of options regarding storage, specifically:

  • using the existing storage technology, augmented by MQTT and mqttwarn to pre-aggregate data based on static logic: fairly easy to implement, but unflexible and even more taxing with regards to storage and system requirements
  • adding an aggregation layer using a Module, not so easy to create a flexible enough solution, probably would incur in performance issues pretty soon
  • adding an additional storage engine like InfluxDB (http://influxdb.com/) that in its latest incarnation has the concept of tags and would allow for queries like

SELECT mean(prod_kwh),mean(kw),mean(solar_kwh) FROM /emoncms.feed_*/
WHERE substr(geohash,1,6) = 'dr5ru4'
  AND company ='ACME'
  AND installer = 'installer1'
  AND time > now() - 4h
GROUP BY time(5m), *

Influxdb has support for graphite style data posting (that I have used for my experiments since there was some experimental support in emoncms), but in order to be able to use the tagging feature I would need to write an engine that speaks directly to the InfluxDb API

For the frontend, I am way behind schedule, and my experiments have been focused in toying with the myelectric and mysolarpv app in order to display additional data. 

Next on my todo list is to think/experiment with the user hierarchy/model. Will keep you guys updated



@Trystan-Glynn: Is it too soon to request a 868 Mhz version of the emonpi ? Got me one the day the kickstarter project started :)


[Duplicate post deleted - please read this. Moderator (RW)]

TrystanLea's picture

Re: EmonCMS in a Software as a Service Infrastructure

Hello Mattia, good to hear your progress and thankyou for supporting the kickstarter, I've added a note against your pledge for the 868Mhz module.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.