Archived Forum

I believe it is under consideration. As you say, there are a number of advantages in performing the integration closer to the source of the data, not the least being that lost (usually internet) packets will have no effect if it is the cumulative energy value that gets transmitted and recorded, and it does of course make it easier to aggregate historical data, which I think is the primary reason that this whole question is being considered. It also has the potential, at the cost of losing detail, of reducing network traffic.

It should be possible to use the EEPROM in the emonTx to store the accumulated energy in the event of a loss of power, but my preference would be to keep that function in the base - even if it means equipping it with a rudimentary voltage monitor that is just good enough to detect a mains failure and trigger a write to EEPROM before the d.c. supply collapses. In other words, the base will behave in much the same way as an energy meter does.  You then calculate power from the rate of change of energy over time, rather than integrating power over time to calculate energy.

You can keep timer2 runing in sleep if it's configured as asynchronous with a slow external xtal.

See for example how to implement an RTC in this appnote:

http://www.atmel.com/Images/doc1259.pdf

Robert,

by "doing the computation in the base", I mean that could be just transforming a pulse into a power value :

power = kwh_per_pulse / (now - last_time)

something like that. The base wouldn't have to store the information any longer. The only difference would be that an instant power (kwh at time t) would be sent rather than a pulse (1 pulse at time t). I don't see an increased need for EEPROM trouble.

It is true that sending power means we don't mind packet loss so much, thanks to interpolation, but in my former message, I was considering the transmission between TX and Base. In both cases anyway (RF or internet), I was suggesting to send a counter (say 0 to 255 and then back to 0) rather than just a pulse, then calculate the difference with latest counter on receiver side, so that if a packet is lost, next packet will account for two pulses. I may be unclear, but I think this is common practice.

Thanks chaviro for the link.

Yet, I'm afraid there's no such crystal on the emonTX, right ?

I was thinking more along the lines of performing the integration (at source - wherever) and sending the accumulated energy onwards for processing. If you only have a small accumulator (say watts stored as a long integer) you'll sooner or later hit a roll-over problem but unless the link goes down for at least half the time it takes to get there (I think - given that it probably needs to be signed!), the roll-over 'carry bit' can be re-inserted at the receiving end and all you lose when the link fails is detail, not the overall energy consumption nor the average power, though of course you lose the peaks and troughs.

As I understand it, your scheme is still sending a pulse but you are tagging it with an identity so that you know when one has gone astray. I think you are saying that each pulse represents a quantum of energy? How do you account for negative energy - generation? Do you number your pulses backwards? - I guess that would work.

I think we are talking of essentially the same mechanism, but I am timing and accumulating the value at the sending end, you are timing and accumulating at the receiving end.

Yes Robert, I think we agree.

My concern is the emonTX not being able to do power computation itself because of the sleep mode, the objective being to save energy when operating on battery. It would just send a pulse and then the power computation can be done on the base or, further, in emoncms.

Besides, my other point was that instead of sending a number of pulses (that would be 1 if a packet is sent each time a pulse is counted, or a greater integer number if a packet is generated every N seconds, which can be after several pulses), I think it is better to send an incremented number (that would happen to be the accumulated energy, times the constant pulse/kwh), so that you may lose packets and keep the total correct, with just the peaks being smashed down. As long as you don't lose all packets in the accumulator's revolution (like losing 260 pulses when using a 8-bit char, but if you have that kind of noisy RF channel, you won't be able to do anything anyway).

I realize I'm just rephrasing your message in a less understandable fashion...

My point in using an incremented counter was not to actually accumulate so as to compute an energy, but merely to have some sort of error correcting code to fight channel noise. In fact, you're right, the incremented counter is homogeneous to energy...

Now, perhaps is there some emoncms input process function that can take this energy and derive the power to generate a nice power chart. If not, it can be created, or there can be something in the Base that does the power computation, and emoncms get the power as an input. The latter is not the logic of OEM, with emonBase transparently forwarding samples.

I didn't think of negative values. My use case is the gas meter. (I don't meter the gas I produce...) But it could work just the same. Yet, I don't see the use case. My starting point is the pulse counter, which always counts up. Sure, if the TX has two pulse counters, for production and consumption, it could compute the subtraction, but this can be done further on just as well, while keeping the detail on both production and consumption.

(In France, consumption and production are really seen as two different things, since generally all production goes to the grid and nothing is consumed directly, except for off-grid sites, obviously. This is because the kwh price when selling PV electricity is higher than the price of the grid kwh, as an incentive for renewable energy production.)

If the emonTx is running on battery, then I think in the emonTx you must have the exact setup that emoncms has - the 'trapezium rule' that calculates energy between two point readings of power. If it is running on mains power, it does not need to sleep and using the interrupt sketch it can accumulate an exact energy value. This can be on a small scale (8- or 16-bit) provided the carry is handled correctly and for integrity along the whole chain that should be done in emonCMS. I think it would work. The emonTx maximum power is about 24 kW, or 6.66 Wh/s. Therefore if the emonTx sleeps for 5 s and the units are Wh (which seems to make sense, but they don't have to be), the count increments by 33 counts, 33 counts, 34 counts in successive transmissions (you must accumulate a float and carry the truncation error to the next transmission). At 50 W load I assume you send the same count every 5 s for 14 times, then the next number on the 15th. That does give a problem with the GLCD, because to indicate 50 W you need a filter with a very long time constant, and that would give a significant delay before the display responded. It almost looks as if the present current/voltage/power readings need to be sent for the GLCD, and the emoncms energy data must be added, increasing the rf packet size

I think it is necessary to have negative counts - the grid c.t. in a  "Type 2" setup is guaranteed to see exported energy at some time or other. And you must surely have a pulse (counter) for each c.t.

If the emonTx is running on battery, then I think in the emonTx you must have the exact setup that emoncms has - the 'trapezium rule' that calculates energy between two point readings of power.

There must be some confusion here. I'm talking about a setup where a pulse counter gets a pulse from a meter, corresponding to a given amount of energy. That can be a gas meter with a reed switch on the least significant wheel, a water meter, even an electricity meter with a flashing led.

The TX doesn't read power, it just gets a pulse every, say, 0.01 m3 of gas, and then the rate can be computed from the time interval between two pulses. Except when using power down mode the TX can't do that, so it has to be either the base or emoncms.

It seems easy to do in the base but I understand the design paradigm of OEM is to do that kind of post-processing in emoncms. I'm wondering if some process function already exists for that. There are some functions for which I don't understand the function but might be suited for that.

I do see what you're saying, and I'm saying that if you do the same thing in the emonTx with electrical energy - and sorry I didn't explain this properly - then it all fits together very well and means that all the energy inputs from emonTx all the way to emoncms are protected from data loss.

This meshes in with Trystan's thoughts about reconfiguring or changing the emoncms.org database so that less data is stored (which is getting expensive The current emoncms feed storage implementation ). One of the proposals he's looking at (Timestore timeseries database) throws away the individual timestamp that is saved with each reading and allocates the reading to the nearest time slot (timeslots happen at preset intervals of your choice) and he's calculated that this will make significant savings.

I can do the following, which would be the easiest and the closest to OEM's rationale :

Send an incremented counter from the emonTX (start at 0, increment each pulse, wrap around at 2^16)

Then in emoncms, use following chain:

pulse diff (to get the nb of pulses received since last update)
scale (by kWh/pulse)
kWh to power

I'll propose that pulse returns a value, not a feed, since in these case we don't need the intermediate result, and anyone how does can always add a log to feed. There are other things I'd like to change in pulse diff. It looks like work in progress and I'll try improve it.

I'm not sure anyone uses pulse diff anyway, or they may get in trouble due to wrapping / counter reset issues.

That sounds good, but does it need to be a new module? It will certainly need to take account of wrapping and reset problems, but as the pulse rate has to fall inside known limits set by your gas supply, CMS can make an educated guess at when there has been an overflow or (usually) a reset and act accordingly.

I think overflow is "the last pulse count was greater than MAXCOUNT - (time since last x max pulse rate) and the present pulse count is less than 0 + (time since last x max pulse rate)".
I think a reset is "the last pulse count was greater than (time since last x max pulse rate) and the present pulse count is less than 0 + (time since last x max pulse rate)."
Clearly it is not easy to be always correct with either.

No, I don't mean to create a new module, just adapt pulse_diff(). After digging a little bit more into it, it appears to be closer to what I need than I expected. Just the reset / wrapping that is not properly handled. The rest is what I was looking for.

I can't just get rid of the feed output as it is used to get the last sample time and value. But it's a shame to log a feed just to remember last sample. I'll investigate. Perhaps is there a way to keep only the last sample, not all the values as a feed.

And I'd like to pass the max rate as an argument, instead of hardcoding it. That would be easy if I didn't have the feed id itself as argument.