Archived Forum

You can reduce it to as little as you want, provided you leave enough time to send the data.

Have you considered using one of Robin's "continuous" sketches instead of the "discrete samples" default? The snag is with those, you'll need to add in code for the temperature measurements. Or you could look at MartinR's PLL energy diverter - strip out the diverter code, add two currents and one more temperature sensor.

If you stay with the discrete samples sketches, you might want to look at how much data you are going to accumulate. If you've got your own server, that's fine, but if you've got an SD card in a RPi, it could be a problem and if you're using emoncms.org, Trystan doesn't like anyone sending more frequently than every 10 s, as it's a bit unsociable to hog all the resources.

Do you have any idea how much data it generates per sample? I'm guessing it's a matter of cutting and pasting between sketches to add in the temperature code to the continuous sampling? With the delay caused by moving data half way around the world (I'm in New Zealand) the lag between turning a light off and it appearing on the app is too long to make sense of things as it is so I think this needs to be moved onto the pi on a memory stick

"Do you have any idea how much data it generates per sample?"

You'll need to look up Trystan's documentation about the databases to get that: https://github.com/openenergymonitor/documentation/tree/master/BuildingB...

If you're storing a lot of data very frequently, you might want to consider running your RPi off a hard disk.

The "continuous" code that I posted here for the emonTx V3.2 supports 4 CTs and one temperature sensor.  I have never used more than one temperature sensor but I know that similar code of mine has been upgraded to operate  in this way. 

Thanks for the link, I have a EmonTX V3.4 do you know if there are any differences from a code point of view?

No problem, I posted a similar sketch for the emonTx V3.4 at

http://openenergymonitor.org/emon/node/10203

The code changes between V3.2 and V3.4 are very few, just for the different RF arrangement.  One of the IO port allocations needs to be changed on account of the different version of Jeelib library that is uses. 

My various sketches are listed on a Summary Page which is always near the top of the  Forum->General section.

I just took a quick look at the summary page - thanks very much for posting, it looks very useful indeed. I'll have time to take a proper look next week. I'm not convinced my phase calculation is correct as I've changed the burden resistors and calibration of both the voltage and current to match the expected values, so I'm expecting the phase offset may not be correct now.

The phase error of the transformers, both current and voltage, is dependent upon their working point, the former more so because current varies over a much greater range than voltage. So if you've changed the burden value, the phase calibration will almost certainly need checking.

Having said that, with 'real' loads (those having power factors close to 1 as normally found in a home), phase calibration makes little difference [cos(0°) isn't vastly different to cos(10°)]. But if you get small, highly reactive loads such as you have with some appliances on 'standby', then the error can be huge. See the reports in Building Blocks for the gory details.

That reflects some of what I'm seeing. I've got a 35w motor in the spa running 24/7 for a start and also 10 micro inverters which cause the solar channel to suggest it's generating 30w in pitch darkness. I'll do some reading and see what I can make of it. I had s current cost ENVIR before and the CT channel over read heaps on low power before under reading as the resistive loads kicked in 

What sort of currents are you measuring, and which CTs are you using? If you don't require the full 100 A maximum current, you can improve the accuracy over a reduced range by changing to a more appropriate CT - though I recognise that this might be hard and/or expensive if you must have a split-core one. We expect the accuracy to start to deteriorate below around 2% of maximum, and below ½% you can expect little more than an indication of current, the ADC error is indeterminate (because it depends on where the quiescent input sits in relation to the ADC steps, and that's essentially random and variable) and can be quite large in percentage terms. A small amount of noise actually helps at those currents, as it can add some dither, but if you're recording generation at night, that's probably mainly digital noise that you're seeing.

CT1 - 60A max 31 ohms burden

CT2 & CT3 13A max 180 ohms burden

CT4 6.5A max 360 ohms burden

all using standard CT, the lower order ones could be changed for a thread through type and the larger one could at a push. That' one is in the back of a crowded meter box so potentially it has the most pickup from other wires. I'm going to trip the solar breaker tonight to see if the noise goes, it's in a different fuse board and is not that close to other cables so the pickup could just be noise from the micro inverters. I've extended the lead in that one with mic cable by 1.1 metres and my meter suggests I haven't got a short between ground and one of the legs. Curiously enough the solar CT buzzed before I plugged it in, I was generating about 1.5kw at the time. I've added a minus 30w to the web monitor for now to remove the nighttime watts.

Unfortunately, while increasing the burden value increases the voltage, there is a price to pay in increased phase error - that's why I suggest a CT with a lower ratio. I wouldn't suggest changing the one on the 60 A circuit, but I think you should see an improvement if you change the others - provided of course they are of reasonable quality.

What fun! Do you have any suggestions for types to try? Its complicated by living in NZ as I'll have to mail order everything. Is it possible to have a different phase value for each CT? Their use is further complicated too as:

CT1 is main incoming power (60A Fuse), CT2 is from 10 soon to be 11 or 12 micro inverters (3Kw Max, 16A Breaker), CT3 is from a Sola Controlla diverting to the HWC (3KW Max, 10A breaker that should trip but doesn't) and CT4 is from a Sola Controlla slave unit diverting to a spa pool (1.5KW element) although I guess at least CT3 & CT4 should have unity power factor even if they don't have a perfect sine wave!

Maybe its easier just to pull one of the CT's and try it in the main CT1 input with an electric fire plugged in and see what your phasecal sketch suggests followed by recalibrating everything.

Do you reckon adding about 2 metres of balanced Mic cable onto one of the CT's would upset the CT calibration enough to need recalibration of it individually or will the value I get from another one plugged into the same input be near enough? 

Curiously I've just checked the solar and the 29 phantom watts of generation overnight drops to 1 watt if I turn off either the breaker on the roof or the breaker in the distribution board, granted its not negative but there must be some odd spikey power draw at work there.

One further question, if I update to continuous sampling code how can I limit the updates to the web version of EmonCMS to 10 seconds whilst having the local pi version updated As often as possible?

" suggestions for types to try"
See what is available locally (to NZ). You need a current output, the actual current doesn't matter as long as it is in the mA region (i.e. not 1 A or 5 A) and you need enough power to drive 1.1 V rms across your burden. That's a figure that is often omitted from the data sheets, so sleuthing or asking is usually necessary.

"Is it possible to have a different phase value for each CT?"
Yes.

"10A breaker that should trip but doesn't"
[Robert puts on Chartered Electrical Engineer's hat] You need to get that sorted out without delay. Meanwhile, I hope your fire insurance is good. Just check that you don't have a fake breaker in there.

"even if they don't have a perfect sine wave"
In that case, assuming it's 'peaky', you need to allow more headroom so that the peaks don't clip. So if it's a 10 A rms true sine, it will have a peak-peak value of 2.828 A, if it's 'peaky', it could have a peak-peak value of a lot more - maybe up to 10 A.

"adding about 2 metres of balanced Mic cable"
... should make no real difference. Remember the CT is a current source, so as long as it doesn't run out of voltage itself, the voltage drop in the cable doesn't come into the sums. But you need to calibrate each particular combination of input and CT, because there are uncertainties in the value of both.

"there must be some odd spikey power draw at work there"
Quite likely it's checking up on the supply to make sure it's still there, etc, and generally keeping itself alive ready for when the sum comes up.

"how can I limit the update..."
I don't know - Paul (pb66) should be able to suggest something, he's the GitHub expert.

I've found my list:

Hardware & hardware-related differences

				V3.2				V3.4
Library			#include <RFu_JeeLib.h>		#include <JeeLib.h>
RF			#define RF69_COMPAT 0		#define RF69_COMPAT 0 ('12B) 
                                                     or #define RF69_COMPAT 1 ('69CW)
DIP SW			not present			const byte DIP_switch1 = 8;
                                                        const byte DIP_switch2 = 9;
Battery voltage		not present			const byte battery_voltage_pin = 7;

Pulse input pin		Dig 2				Dig 3
Pulse interrupt		IRQ 0				IRQ 1		

One further question, if I update to continuous sampling code how can I limit the updates to the web version of EmonCMS to 10 seconds whilst having the local pi version updated As often as possible?

My "continuous" sketches will transmit as often as you want them to.  The transmitted data is for the period that has elapsed since the last transmission.   If you want something to be updated at a slower rate but with the same data, you are likely to lose some data 'packets' unless you do some intermediate processing of the received data.

Maybe I've not understood the question.  Why would having "continuous sampling code" make any difference to how your received data is processed?

This is possibly where I'm not understanding the system correctly. As I understand it currently EmonTX samples current and voltage, turns them into watts and transmits the result before going to speed for 10 seconds.

emonbase site listening on 433MHz and when it receives a transmission it forwards it to EmonCMS locally and remotely.

so if I upgrade to continuous sampling the wait 10 seconds but is removed and almost continuous data is sent to EmonCMS hogging the resources? Not a great problem for me as emonbase isn't doing anything else but won't it upset the shared EmonCMS in the UK?

Ok I'll pull both CT's and recalibrate. Fortunately the most calibration is needed on CT1 and it will have to deal with the lions share of odd loads because most of these appear after the sun has gone down. I'll calibrate this around 500 watts for phase as its in the 150-500 watt range that I'm interested in the finer details. Above that the big stuff kicks in which apart from the heat pumps is resistive loads such as the kettle / spa heater / coffee machine / toaster.

Last time I looked at alternative CT's locally they were prohibitively expensive so I'll see how close the results are to what I need. Solar diversion is handled elsewhere so I don't need the fine level of control from this system. 

Moving onto the breaker it's likely a combination of type c breakers as standard over here plus the former lossy cable both to the heater and to the house causing sufficient volt drop to derate it. I've rewired it in 4mm which increased the power draw and shortly after I added the sola controla so it's not really had a full load on it recently. 

The discreet sketches do sleep for the most part and then do a quick sample and send, where as the continuous sampling sketches do not sleep they sample continuously throughout the periods between sending the data.

The update interval is independent or the sample interval and defines how often the data is reported by the emonTx to th emonbase (usually 10 secs) . possible RF collisions can prevent the update interval being too frequent, but with one device on RF or a direct connection you can get a much shorter update interval. A shorter update interval doesn't improve the accuracy but it does improve the responsiveness and therefore appear more accurate.

Although if everyone started bombarding the emoncms.oeg server once every second or so it would have a melt down so a slower interval is expected. However you can have your cake and eat it if you want to have a short update interval from the emonTx to give faster ressponse on your local emoncms you can set a limit in emonhub to only update emoncms.org every eg 10seconds regardless of how fast the data is produced locally.

In the reporters section of emonhub.conf, add "interval = 10" to the [[[runtimesettings]]] for the emoncms.org reporter and then you can decrease the time between sends in the sketch.

This gets around the issue highlighted by Robin as all the data points are sent but just in batches so as not to over work the server,

Paul

Thanks very much, I've just switched to a continuous sketch and I've changed the settings on the pi and rebooted.

I've just tried the phasecal sketch from the MK2 Solar Diverter project, it allows me to input the phasecal values and the increment and then just sits there - does this sketch work on a V3.4?

value of PHASECAL_MAX?
 PHASECAL_MAX = 1.00
value of PHASECAL_MIN?
 PHASECAL_MIN = 0.25
value of PHASECAL_INCREMENT?
 PHASECAL_INCREMENT = 0.05
In NORMAL mode ...

The initial value of PHASECAL is 0.25

"does this sketch work on a V3.4"
No, not without changing some I/O allocations. Look at lines 16 & 17. (You'll find the I/O for the V3.4 in the Wiki.)

Thanks - changing "byte voltagesensorpin" from 2 to 0 did the trick, and I ended up with a phasecal value of 1.55, however that seems to have made the Solar input overread despite recalibrating both that and the grid CT with an electric fire. I did try the continuous_kwhtotals sketch as it seems to allow different phasecal values per channel but it didn't seem to do anything despite loading the missing timer library.

For now I've changed the update interval to every second and its giving much better responsiveness on the local pi, however I won't know how well its all working until later, when the solar shuts down as as the varying solar output courtesy of passing clouds is distorting things.

The solar CT also doesn't quite close properly and can chatter if its unplugged so I'll visit this again once the 2 I have in the post arrive in case the tiny gap is enough to screw the calibration up between it sitting under my desk and residing on the solar input cable

Hi Sheppy,

What particular micro-inverters do you have? My system behaves similar to yours. I have 30 Enphase M215s and 6 M250s. The inverters draw about 45W at night, but it's a very reactive load. (power factor of about 0.08) It causes my TED5000 to report the micros as "generating" ~45W at night...

Regards,

Bill

Hi Bill,

Renesola Micro Replus 250W x 10

Mine reports around 29-31w of generation at night whereas the monitoring box reports 0w. So far I've just used the + calibration function in EmonCMS with -31 in the box which stops it being reported as real power. It does mean first thing in a morning the graphs are off but its near enough until a better solution appears.

The phantom generation sounds like a micro inverter thing, they probably sniff part of the waveform every cycle to ensure the grid supply exists.

The solar CT also doesn't quite close properly and can chatter if its unplugged so I'll visit this again once the 2 I have in the post arrive in case the tiny gap is enough to screw the calibration up between it sitting under my desk and residing on the solar input cable

A CT that doesn't close completely will definitely not operate correctly. The two surfaces must be touching i.e. no gap, for proper operation.

The phantom generation sounds like a micro inverter thing, they probably sniff part of the waveform every cycle to ensure the grid supply exists.

I've read some posts (can't remember which forum, atm) that indicate some of the string inverters behave similarly.

That all makes sense.

When the 2 CT's I have in the post arrive I will recalibrate and see if I can simply put a cable tie around this CT to ensure its closed when installed, it clips home but has a tiny gap.

sheppy: This is possibly where I'm not understanding the system correctly. As I understand it currently EmonTX samples current and voltage, turns them into watts and transmits the result before going to speed for 10 seconds.

The performance of the emonTx is entirely dependent on the resident software.   The various sketches - both discrete and continuous - will all run on any of these Atmega 368 platforms: emonTx V2 or V3, Arduino Uno, or my own Mk2 PV Router boards.  Only the IO port allocations need to be changed.

Yes it's all starting to make sense now, I've never touched Arduino or Linux i/ GIT n any detail so the learning curve has been a bit steep!

i wonder why the released version of the continuous sampling sketch with KWh totals didn't work - totally blank serial display and no flashing LED. I think that one may work well for me as I can have different phasecal values for each CT which would help with my 3 different burden resistors

i wonder why the released version of the continuous sampling sketch with KWh totals didn't work - totally blank serial display and no flashing LED

Which version did you try? those symptoms can been the result of a v3.2 to v3.4 mismatch as the sketch stalls at initializing the RFM since it's not found with the incorrect pin allocations.

The one at

https://github.com/openenergymonitor/emonTxFirmware/tree/master/emonTxV3...

Duplicate post deleted - BT

I've been using the one on GIT in 

tree/master/emonTxV3/RFM/emonTxV3.4/emonTxV3_4_continuous_kwhtotals

Did you set the #define RF69_COMPAT 0 or 1 line correctly?

the same symptoms are also common to the sketch not finding the expected RFM module too. 

Didn't know about that, as it has something like set to rf69 on the folder, will check in the morning, please let me know if you think of anything else. We're 11 hours ahead of the UK so I can only post late at night or early in the morning

sheppy, it's a long time since I had anything to do with the 'kwhtotals' sketch that you've been using. 

I would recommend using either of the "Mk2" sketches that I posted recently for the emonTx V3.  Although offered as a diverter for surplus power, these sketches can function perfectly well as a continuous monitor of power on four channels.  One of these sketches has been set up for the RFM12B, the other is for the RFM69CW.

I'll give them a go - the problem is I have no way of measuring the house current to check the calibration due to the "unique" way they wire up a 3 phase meter as an import export meter over here, once you add in a liberal dose of non linear power draw by gadgets I have no real idea whether the monitor is telling the truth once it's installed, compared to the calibrated values.

Over here they feed the single phase power through a 3-phase smart meter first forward on phase 1 and then backwards on phase 2. This means the load on the smart meter always says zero and I only get usage on the web a couple of days later. NZ is in many ways a forward looking country but their implementation of import / export meters is a bit backward!

I've just taken a look at your sketch and was thinking of where my open energy monitor journey is likely to end. Do you reckon your sketch can be easily modified to monitor:

Grid import/export on CT1

Solar generation on CT2

Diverted Load 1 power and energy on CT3

Diverted Load 2 power and energy on CT4

Temperature 1 middle of hot water cylinder

Temperature 2 bottom of hot water cylinder

Temperature 3 spa pool

Monitoring of diverted energy could be enhanced by a line of code that reads something like "if diverted energy is greater than SolarPV, diverted energy value = 0" to get around override heating due to winter clouds or guests using all the hot water.

All of this with near realtime updates to emonCMS running on the local pi with different phasecal settings to reflect the different CT burden resistors and cable lengths without actually controlling anything as I have another box doing that.

Can you recommend a specific sketch to use with emonTxV3_4_as_Mk2Router.ino to calibrate the phase for each CT? I've just calibrated the voltage and current with an electric fire and my meter and I've got the following values

CT4 0.017, 360Ohm Burden, max current 6.5A

CT3 0,033 180Ohm Burden, max current 13A

CT2 0,033 180Ohm Burden, max current 13A

CT1 0,148 39Ohm Burden, max current 60A

Unfortunately the phantom overnight solar power of around 42Watts positive I had with the descrete sampling sketch is now around 74Watts which is too far out for accurate solar monitoring at the start end end of the day as we head into winter down here. Also the grid power is under reading a bit so my guess is power factor problems are around especially as the background load right now is an inverter fridge, small electric pump, 2 PC's and various standby items..

I've taken a look at emonTxV3_4_continuous_kwhtotals.ino and found #define RF69_COMPAT 0, changing that to a 1 fixed the problem and it now works, the results I get from this sketch seem more consistent than the periodic sampling one however it doesn't report the voltage so I can't calibrate it properly due to only one meter and a slightly varying mains voltage. It also seems to stop the moment I unplug the serial board so maybe it suffers from excessive power use.

For this sketch GIT says "set to RFM69CW by default" on the folder which is incorrect - is there a way of reporting this?

I hope this all makes sense - its been a long day!

Yes, it may be possible for a single processor to do all that you're looking for, but the code for this is unlikely to exist in exactly the form that you require.  You will need to decide on an existing sketch that you can use as your base build, and then add the extra functionality that you require.  By regularly checking and saving as you go, you will hopefully get there in the end ...

Thanks, as this thread is now long and a little different from the original title I'll start a new one to aid anyone else in future