emoncms time-of-day filter process

Use case: I'm counting pulses of gas at five minute intervals and I want to log to different feeds depending on the time of day, so I can separate gas used for hot water (5-6am) from space heating (other times).

There are a few possibilities and I'd like to implement the one which is most consistent with the rest of emonCMS.

So far my thoughts are:

  • Add a filter process which passes through the value unchanged if the current time is within the desired range, otherwise returns 0. This would support other time-selective uses of the input (such as adding or multiplying inputs only at certain times of day).
  • The hours of the day to be filtered would be input as a bitmask to make it fully flexible (not sure how the UI would work just yet).
  • After the filter I'd place a standard Accumulator process and then Kwh to kwh/d (a daily accumulator by any other name)

The main problem is that you can't chain multiple time periods together on a single input.

Whichever time filter process comes first will sometimes return 0 making the second and subsequent time filters useless. Even were this not the case, you can't chain two accumulators together because the first one will change the $value being propogated down the chain to the new accumulator total, so the next set of processes logging to an alternative accumulator don't have access to the original input anyway.

However I look at it this seems to call for multiple processing chains per input, which might be a bigger undertaking than I really wanted as a first contribution :)

Any thoughts welcome.

pb66's picture

Re: emoncms time-of-day filter process

Hi Schism,

I've been working on multiple daily time period reporting in EmonCMS to support peak and off-peak electric rates, I believe the same processing would be required for your needs.

I have a working solution that could probably be improved vastly as I am currently learning as I go.

I have found you can report different time periods to individual feeds accumulators and daily accumulators on the same input.

There is some more info on this thread http://openenergymonitor.org/emon/node/2499 but I haven't had any feedback as yet.

This workaround relies on another input node to switch time periods, based on your info above you would need 2 transmissions a day, 1 at 5am & another at 6am, each with 3 fields node_id period1 & period2. the only difference between the transmissions would be one having values of period1 = 1 and period2 = 0 and the other reversed to switch back. I've used cron and it works well.

Whilst dev's may not consider this a "proper" solution it does work without any mods to emoncms and has the advantage of adjusting the time period easily eg BST or seasonal etc and multiple time periods can report to the same feeds just by adding more cron lines eg if you needed to have the hot water on for a second period in the afternoon you only need to add cron lines and the existing processing in emoncms would tally the am and pm periods together for both hot water and heating feeds.

Additional time periods with separate reporting can be added easily by creating another processing sub-chain on the same input and adding the corresponding crons and periodX active flag to the node transmission. 

A currentCost field can also be used if you need to report use in monetary terms for each period

If the emoncms processing side is acceptable maybe the cron side could be replaced by a scheduler script which could be edited via packetgen or replaced all together with a packetgen type module posting directly to emoncms

Apologies if I've misinterpreted your requirements or if this is not suitable.

Regards Paul

Schism's picture

Re: emoncms time-of-day filter process

Hi Paul,

Your solution using a synthetic node is really original, I'm impressed!

It would be cleaner to support from within emoncms but the nice thing about yours is that you can add unlimited granularity which would be tricky to support through the UI (i.e. you can have cron enable something for just a few minutes of the day).

I'd never considered using allow positive + negative then re-adding the input to its own processing chain to start over either. That's pretty cunning! (Rather than trying to add multiple processing chains, I could add a "reset" process as a stop-gap which is a little more intuitive for the rest of us!)

Are you on github? If you sign up for an account you can easily upload your source for us to play with :)

Great work.




pb66's picture

Re: emoncms time-of-day filter process

Hi Schism

Thanks for the kind feedback I was wondering if I had suggested something taboo in the dev's world when there where no return comments on the other thread.

I do have a github account but I'm not sure what "source" you refer to as I haven't coded anything (yet) to implement this for Peak & off-peak Elec I did 3 things

1) created 2 copies of Jeromes socket example script offPeak.py & onPeak.py with 2 lines changed


HOST = 'raspberrypi'         changed to     HOST = 'oemgateway'         in both

frame = '15 69 1664'          changed to     frame = '15 15609 1 0'         in onPeak.py

                                                 and      frame = '15 6949 0 1'          in offPeak.py

where 15 is node_id followed by currentrate (unit price x 1000) and then onPeakActive and offPeakActive

2) added 2 cron lines on Pi running oem_gateway to run scripts at 00:15 and 07:15 each day

15 00 * * * python /home/offPeak.py
15 07 * * * python /home/onPeak.py

3) create emoncms sub-processes on the grid monitoring input for each period

Allow Positive

Allow Negative

+ Input                                                               (grid ct in my case & gas pulse in yours)

Allow Positive                                              (filter out solar export, you shouldn't need this)

x Input                   "PeriodActiveFlag"           (multiplies by 1 if period active or 0 if inactive)

Log to Feed             create new "period1use"

x                             0.001            (this undoes the watts to Kw/hr calcs built into power to kWh/d)

Power to kWh/d       create new "period1useToday"

My processes then go on to determine current pence per hr and pence today for that period and then repeats for the second period (starting with allow positive & allow negative)

So as you see there is no source code as such at the moment its more of a workaround

As previously mentioned there are better ways of doing the "synthetic node" side but I haven't yet explored that side as this works for me a gui would be good to set time periods etc. Maybe for your application rather than a timed cron period maybe "HotWaterNowOn" & "HeatingNowOn" scripts triggered by the master control circuit itself would be better so if the hot water period gets moved or extended the emoncms reporting supports it without alteration.

If you think it would be of use to have something on github to help improve/develop, I'm happy to do that with a little guidance.

Excuse the wordy description and terminology I am not a coder, not that you needed me to point that out I'm sure :-)

Regards Paul

Schism's picture

Re: emoncms time-of-day filter process

Hey Paul,

Thanks for the walkthrough. I like that you say you haven't coded anything yet, then give me a line by line of the code you changed :) The main reason I suggested github is that it makes it easy to see exactly the line by line changes, and to grab the files onto one's own Pi if desired. It's not essential but git is so powerful that you'll soon wonder how you got by any other way. That's fine for now though.

Although... I've pushed a branch which supports a new 'reset' process. If you fancy giving it a try you could let me know how it goes...


Assuming your Pi setup is OEM, the following should work:

cd /var/www/emoncms
git remote add mccraw git@github.com:Dave-McCraw/emoncms.git
git fetch mccraw
git checkout -b resetProcess mccraw/resetProcess
sudo service rfm12piphp restart

To demonstrate it works, add a new x10 or something onto the end of a processing chain, then reset it, then log to a temporary new feed. Feed value should not be multiplied...

To go back to stock, run:

cd /var/www/emoncms
git checkout master
sudo service rfm12piphp restart
Schism's picture

Re: emoncms time-of-day filter process

OK - that's the filter by hour of day process working too, if anyone fancies giving it a try.


I would have preferred to pass in an argument of comma separated hours (i.e. 12,13 would cover lunchtime) but emoncms was cropping that to the first number only, so I've stuck with the raw bitmask for now.

You can easily generate arguments using http://www.tfxsoft.com/playground/calc.php

Settings: from binary to decimal

Number to convert: supply 24 digits, one for each hour - just remember that they are in reversed order

111111111111111111111111 (all day)

000000000000000000111111 (first six hours of the day)


Hit convert and copy the "In base 10 this is [number]" and use that number as your emonCMS input.

Keen to hear any feedback.

ukmoose's picture

Re: emoncms time-of-day filter process

Schism, Impressive stuff!  


If I understand your code correctly it's working in UTC timezone?

NOT the local timezone of where people are...

Schism's picture

Re: emoncms time-of-day filter process

Yeah, I was thinking of emoncms.org when I was setting it up and thought consistency might be best (ideally a second input of local timezone could be supplied).

Admittedly, it would be better to use the Pi's local time for everyone else :S


pb66's picture

Re: emoncms time-of-day filter process

Unfortunately I'm not running emoncms on a local server yet, so as much as I'd like to, I can't try it out just yet.
Having said that I have to say my workaround may be more suited to my current needs as if I understand correctly your reporting module works on hourly segments that start on the hour, which wouldn't align to my wonky off-peak/peak periods which change at quarter past the hour.
The reset process I think would definitely be of more use to me although I currently have my additional processes spread over several less processed inputs so rather than reset the input process back to its own input value, I am resetting to a different input to spread the processing and help manage it better, but theres no reason why the same input can't be used over and over again (although the longer the processing list the more of a pain it gets to add new processes to the top of the chain)
None the less I glad you've got a resolution that works for you and I will try the modules when I set up emoncms locally.

Schism's picture

Re: emoncms time-of-day filter process

Hi Paul,

Yes, only hourly on the hour unfortunately. Something much more sophisticated would be needed to do arbitrary times - but it's a good goal.



chaveiro's picture

Re: emoncms time-of-day filter process

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.