Archived Forum

How can it cause a permanent negative reading? The purpose is to hold the input at GND (ADC output = 0) when no CT plug is inserted, so that the sketch can detect the presence or absence of a plug - and hopefully of the CT - and skip reading and processing that input if necessary. If the PCB layout is correct, I don't know why the connection has been left off - it may have been decided that this feature was not needed, or it may simply be an oversight that will be corrected at some time in the future. 

Hi.

I thought that the Voltage divider composed of the 2 470k resistors was to create a 1.65 V midpoint voltage.

If we pull the input to zero wouldn't it be equivalent to reading -1.65 V?

Maybe if the reading is zero at the analog input equivalent to an extreme of -1.65 V the software can consider the absence of the CT sensor and that is the purpose of the connection to GND.

Anyway, that connection seems not be be there.

Cheers.

"I thought that the Voltage divider composed of the 2 470k resistors was to create a 1.65 V midpoint voltage."

Yes, that's what it does. Normally.

"If we pull the input to zero wouldn't it be equivalent to reading -1.65 V?"

Stop and look at the circuit diagram. There's no plug in the socket, so the input terminal of the socket is grounded. Hence the ADC input is effectively grounded via the burden resistor - which has no voltage across it because there's no current flowing in it. Where does the -1.65 V come from? Or to ask the question in a different way, what has happened to voltages and currents in the voltage divider?

The way the schematic is drawn, it appears the "error" is actually in the CT2 circuit. The line from the fixed contact (pin 4) to ground is connected directly under the arrowhead, but should be drawn like the CT1 circuit, i.e. just a bit to the right of the arrowhead.

Drawing error only. Nothing more.

Hi.

Bill: I was not referring to that (I don't think that that should even be considered an error, both pin 4 are connected to GND). The possible error I'm referring to is on the tracks schema that the Pin 4 doesn't seem to be connected to the GND.

Robert: The -1.65V I'm referring are related to the midpoint that is considered to calculate the alternate voltages/currents. I know that the circuit will read a real 0 volts at the port, but if the software is not programmed to detect a disconnected CT wouldn't it read -100A (probably not, I cannot compute the calculated current of a line at the -1.65V in all 360º of the wave length)

I'm to confused now...

Cheers.

The arrowhead represents the contact. Thus the schematic shows a connection directly to the contact. It's really a matter of semantics. That's why I emphasized "the way the schematics is drawn" i.e. I was trying to pot out a drawing error.

Having just taken the end panel off an emonPi and measured it, I confirm there is a ground connection, so it's the PCB layout that seems to be incorrect, in that the ground plane isn't shown.

"if the software is not programmed to detect a disconnected CT wouldn't it read -100A"
No. You need to explain your thinking a little better, because I can't see how you think that a socket with no input can read anything.

I don't know how to explain it, maybe I'm wrong.

A reading of GND level wouldn't be considered by the software a permanent reading of negative amperage?

I believe that, to have a zero Amperes reading the analog port reading has to be 1.65V (the midpoint) what intensity will be calculated if the reading at the port is permanent zero or a logic -1.65V?

I don't really know, maybe the result is zero amps also...

Ah, I understand now. You've forgotten the filter in emonLib that removes the 1.65 V bias that the voltage divider adds to the current signal. So, whatever happens to the actual input, provided that the voltage at all times remains within the input range of the ADC, the filter will remove the d.c. component and pass only the a.c component to be measured and reported.

Hi.

I'm not using the emonLib, instead I'm using a firmware I've altered based on the MK2 energy diverter.

I will make the firmware changes to use the 3rd CT before I connect it, so I will see how it will react to that situation.

Cheers.

The Mk 2 has, as far as I remember (without checking), no detection of a missing CT because it's assumed that you need the CT at all times and it is not an option to not have one.  But notice, the Mk2 does NOT report or display current (it only uses the current multiplied by the voltage to calculate the average power), so it's only necessary to remove the offset in the voltage signal. Therefore, the Mk2 has the same filter arrangement on the voltage input, and no filter on the current input(s). If you need the current separately, then you need to apply the filter to the current signal too.

Yea.

I forgot that with MK2 we only have Real Power output, not apparent power (VA).

Cheers.

Hi.

I've made the first changes on the code to accept the additional CT sensor and I'm getting a negative reading of around 780W with absolutely no hardware connected to the analog port.

Let's see how it goes when I connect the CT hardware.

Cheers.

Hi.

NewsFlash:

I've made my external "kit" to connect to the EmonPI and get a 3rd CT sensor working.

I'm using the same circuit above connected to the EmonPI RJ45 port (ADC6) and I'm getting the 3.3V from the ISP1 connector (instead of changing the circuit to use 5V available at the RJ45 port).

For testing and calibration purposes I've put the 3rd CT sensor next to the mains CT so it would measure exactly the same current/power.
At the beginning I noticed a slightly difference in the reading of about 10W and I compensated that adjusting the PowerCal for this CT in the firmware I'm using based on the MK2 thinking that I was getting this difference because of components not being exactly the same, etc etc.

Now I'm noticing that the reading are the same around 500W but are lower at lower powers and higher at higher powers.

This is a sample:

Timestamp P1 P3 Diff

1453666680 1671 1682 11
1453666690 1037 1042 5
1453666695 1301 1309 8
1453666705 1295 1303 8
1453666710 1285 1293 8
1453666720 2523 2544 21
1453666725 1653 1664 11
1453666765 2044 2061 17
1453666770 2533 2549 16
1453666780 1186 1185 -1
1453666795 382 374 -8
1453666800 383 375 -8
1453666810 378 370 -8
1453666815 395 386 -9
1453666820 378 369 -9
1453666825 1162 1161 -1
1453666830 2503 2517 14
1453666840 2414 2427 13
1453666855 1184 1185 1
1453666860 870 866 -4
1453666870 385 377 -8
1453666875 385 376 -9
1453666880 384 375 -9
1453666885 383 374 -9
1453666890 382 374 -8

Any ideas on how to correct this, preferably by software/firmware change?

Cheers 

From your description, that sounds like a straightforward calibration issue. Do not expect two CTs and two burden resistors to give exactly the same answer. The CT has a 3% tolerance, the resistor might be 1% or 5%, depending on what exactly you've used, so if the numbers are within 15% of one another, that is within the band that is acceptable. The whole purpose of calibration is to remove that difference.

Read about component tolerances in Resources > Building Blocks.

Also, be aware that, unless you have changed Robin's code, the Mk2 does not completely remove the d.c offset, so although real power is calculated correctly, current might not be.

Hi.

Thank you for your answer.

I used 1% tolerance resistors.

I'm talking about Ws of power, not VAs or As (the MK2 sketch doesn't even "compute" VAs or Amps).

I would expect a linear result that could be corrected with PowerCal, but this doesn't seem to be the case as it "middle" point where it goes negative and positive.

I will try to change the order of sample collection in the ISR routine so it gets P3 after V instead of P1 after V and see what happens to the results.

Cheers.

Also, remember that phasecal, apart from attempting to correct the difference in phase error between the two transformers, also corrects the timing difference between samples, which is 104 μs between successive samples. It does not make a great difference though with a purely resistive load.

If you have a significant noise contribution, I suggest you subtract a constant from each power to give a zero power reading (or acceptably close to zero) on no load, then adjust powercal at as high a load as you can, to give the same reading on both.

Hi.

I noticed that, when I read the CT3 before CT1 I invert the readings, so it seems that both CTs are reading correctly, but the firmware is not computing the values right.

I've made some changes to my firmware so I can online change the powercal and phasecal for each CT so I can test values without needing to update the firmware on each try.

I've tried changing the phasecal to values like .25, .5, .75, .9, etc. but I don't notice any change in the results.

Is phasecal a value that I should consider trying values with lower differences like .009 or .0009?

Next I will try ti understand what phasecal does in the code to see If I can tell how to use it to correct things.

Cheers.

"Next I will try ti understand what phasecal does in the code to see If I can tell how to use it to correct things."
Reading the Building Blocks article might save you some trouble.

I've read "Explanation of the phase correction algorithm" and I noticed that the values are different from the mk2 system.

But, it works more or less as I expected.

I will read the code to see how it work in the mk2 code so I can apply the theory.

Cheers.

Hi.

I think I understood what the phaseCal value does for each CT, and I think it is very important to be dismissed and set to 1 to all CTs as if it wasn't important.

But there are things that are puzzling me, for instance this text

// Values ouside the 0 to 1 range involve extrapolation, rather than interpolation
// and are not recommended.  By altering the order in which V and I samples are 
// taken, and for how many loops they are stored, it should always be possible to
// arrange for the optimal value of phaseCal to lie within the range 0 to 1.

But the fact is that, from what I understood, a value over 1, like 1.25, 1.5 and 1,75 would be the necessary values to correct the phase error between readings when we have 4 sensors (1 VAC and 3 CTs).

To invert the readings order, it seems to be necessary to collect the values from all CTs and make all the calculations in the Voltage cycle so we can use a phaseCal between 0 and 1 (0.75, 0.5, 0.25 to compensate the delay from the CT readings being made before). But doing this this way would probably exceed the maximum time that the ISR should run on that specific cycle.

Even with not recommended values over 1, I can't correct the readings, I will always get CT1 with higher readings than CT3, with about the same difference.

If I change the reading order from:
Voltage, CT1, CT2, CT3
to
Voltage, CT3, CT2, CT1
the difference will immediately* invert (be negative). This means that both CTs (including my external circuit) work correctly and are accurate enough.

*I've made changes that allow me to change many values in the firmware, including the order that readings are made. At this point the base calculations are still the MK2 code, but the rest of the code may be already unrecognizable.

Cheers.

The description of Phasecal was written before the Mk2 Diverter and the "continuous" sketches appeared. The "discrete" sketches only ever read one CT and the voltage at a time, so if you read CT, Voltage with a modified emonLib, you will get a value for Phasecal between 0 and 1. I do not know how you have changed the Mk2 sketch so I cannot comment. One thing is certain, the same value of Phasecal will not be correct for 3 CTs, but unless you have loads with a very poor power factor, the phase correction does not have a significant effect.

I  think your comment confirms my learnings about this. I'm trying to get similar readings on all CTs so that after I can accurately make a calibration that that will apply to all.

About the poor power factor, this CTs are hooked to the main grid wire, so the  power factor will probably not be the best there.

About my changes to the code, I believe that I haven't made changes to the code that matter to this and the original code should behave the same way.

It would lovely that someone that has access to an EmonTx to test the MK2 sketch with CT4 and CT3* on the same load to see if behaves like mine. But even so, there will be a difference because EmonTX makes 5 reads and mine only makes 4 (but this would be a thing I would be able to simulate), but anyway the the CT4 read is still one cycle delayed from the voltage and the CT3 two cycles.

* I said CT4 and CT3 because the MK2 sketch reads in this order: Voltage, CT4, CT3, CT2 and CT1

At think point I'm having 48 samples per cycle, with emonTX there should be only about 38. When I started I had 64 samples per cycle with only 2 CTs.

I thought about reading in a way that I get all the combinations possible of reading order like this:
V CT1 CT2 CT3 V CT1 CT3 CT2 V CT2 CT1 CT3 V CT2 CT3 CT1 V CT3 CT1 CT2 V CT3 CT2 CT1
But I don't like the idea.

At this point what I don't understand is why the phaseCal correction don't change anything, but at the same time, it seems that if the readings difference invert when I change the reading order, the problem should be related to the phaseCal...

Can there be some idea that I'm missing on all this?

Cheers.

The name Phasecal is wrong - it does not describe all that it does. Phasecal shifts the voltage wave in an attempt to correct for two things: (1) the difference between the phase error of the voltage transformer and the phase error of the current transformer, and (2) the time difference between taking the readings. Both phase errors change as the quantity being measured changes, so Phasecal will never be correct if the voltage and current vary from the values when the errors were measured. If you read the test reports on the two transformers, you can see how the phase error changes. In your sketch, one value of phasecal will never be suitable for all four CTs because the time difference is 104 μs greater for each one.

If you need to have a similar (it can never be the same) Phasecal for each CT, you must read CT1 V CT2 V CT3 V and use pairs for current and voltage readings (for the 3 real powers).

I am working on the "CM" sketch to remove a number of problems, phase calibration is one of them.

About the poor power factor, this CTs are hooked to the main grid wire, so the  power factor will probably not be the best there.

You tweaked my curiosity with that comment, so I decided to plot my whole-house power factor across a day.  Of course looking at power factor without also looking at power is fairly meaningless, but for better or worse here it is. That solid 100% during the middle of the day is the PV export.  You can also see the fridge cycling at night; the compressor running actually improves my power factor because without it, all that's left in the middle of the night is background electronics.

As others have said before, the higher the power flow (in either direction) usually the closer the power factor is to unity.

Good Morning.

I think I've left an idea that is not what I meant:
- With the default MK2 sketch the phaseCal (or PhaseShift as you correctly called it) is 1 to all CTs;
- But I don't need to have the same phaseShift for all CTs, I want to find the best value for each CT that enables me to have similar readings on all CTs when they are measuring the same live wire.

Yesterday night I found something, but I know that the programmer will be pissed off with me again with my next sentence because he believes that his code don't have errors but I suspect that this code shouldn't be there: ">>8" because it is diving the shifted portion by 256 when, I think, it should be at the same greatness size.

      phaseShiftedSampleV_minusDC_long = lastSampleV_minusDC_long
             + (((sampleVminusDC_long - lastSampleV_minusDC_long)*phaseCal_int_CT1)>>8);

 

Even so, if I remove this division from the code, I noticed a slight change but not enough to correct the problem. I will make more tests later.

Cheers. 

No, the ">>8" is correct. It is restoring the scale because phaseCal_int_CT1 is itself left-shifted by 8 bits:

        phaseCal_int_CT[i] = phaseCal_CT[i] * 256; // for integer maths

" With the default MK2 sketch the phaseCal (or PhaseShift as you correctly called it) is 1 to all CTs;"
That arises from what I wrote a little way back - where the dominant load is near unity power factor, phasecal makes little difference.

Humm...

You are right, missed that one, looking again...

shouldn't it be:
 phaseShiftedSampleV_minusDC_long = lastSampleV_minusDC_long
             + (((sampleVminusDC_long - lastSampleV_minusDC_long)*phaseCal_int_CT1>>8));

So that only the variable phaseCal_int_CTx is divided, instead of the shifted value multiplied by the phaseCal_int_CTx?

UPDATE: The representation would be be better, but the result is the same

Cheers.

CidiRome: Yesterday night I found something, but I know that the programmer will be pissed off with me again with my next sentence because he believes that his code don't have errors ...

I don't think this kind of comment is helpful or fair.  If someone has doubts about code that I or anyone else has posted via this forum, it's only reasonable that those doubts are aired here.  But please let's do this in a constructive manner rather than getting personal about it.  Thanks.

I have always been constructive. May not have been always correct, but my objective has been always shed light to doubts I had.

I'm trying to put this code working correctly, let's see if I can do that. Until I can get all CTs making similar readings it is not ok for me. I believe that I can get this code reading CTs with constant differences lower than 5W when reading loads under 1000W (taking in account I use 3 loops in the CT and the division is made by the powercal value).

Since you came to this thread, can you test your code with as many CTs as possible in the same live wire and see if the readings are similar in them all?

Cheers.

Actually, the result is not the same, as phaseCal_int_CT[1] is an double multiplied by 256 (and truncated to int).
With phaseCal_CT[1]=1.12, the upscaled phaseCal_int_CT[1] is 0x11E

Multiplying diff by 0x11E and then >>8 differs from multiplying diff with 0x1

/Per-Ake

Per-Ake, what are you trying to say? The object is to remove the slow floating-point calculation from the time-critical section of code. The real consideration is this (given that phasecal itself is only a rough approximation, knowing the variation in the phase errors that it tries to correct): Is multiplying a floating-point number by 256 and truncating to integer, using that to perform a second multiplication on a varying value, then right-shifting the result by 8 bits, acceptably close to multiplying that same varying value by the original floating-point number?

My reply was to entry "Submitted by CidiRome on Sat, 30/01/2016 - 22:01.", with reply #38914, but obviously all replies goes to the end...

I am trying to say that the code used in the library is correct (and works as intended) and that the suggested rewriting (in the post replied to) does not give the same result, ie the UPDATE-statement is not correct.

The text: "Multiplying diff by 0x11E and then >>8 differs from multiplying diff with 0x1" was intended to show the difference between
1) the correct handling in lib: Multiplying diff by 0x11E and then >>8
2) the suggested (incorrect) handling: multiplying diff with 0x1
/Per-Ake

[ I understand now - thanks. RW.]

Maybe this thread should be reserved for the helpful enquiry that was originally raised about the emonPi and its documentation.  The conversation seems to have since moved off in several different directions so is likely to become lost because the title is no longer relevant.  The purpose and implementation of phaseCal has been well explored in many other threads over the years.

CidiRome:

We, the moderators, have discussed this thread and your contributions in this thread and elsewhere.

This is the second time on these public forums that Robin Emley (calypso_rae) has had cause to object to comments that you have made about him and 'his' software. While we all appreciate that English might not be your native language, I think you have an excellent command of English, and that it should be good enough for you to be able to avoid phrasing questions and making comments in a way that can be understood as a direct criticism of a particular person - someone in this case who has made a huge contribution to the OEM community. 

We also understand that you are trying to learn what is a fairly difficult and complicated subject, but as Robin pointed out last time, the way to do that is to read, read again, investigate, check and try to follow the path of the logic through for yourself. Only when you fail is it time to ask a question. And when that time comes, you must ask for an explanation for why something was done in the way it was, and you must not write it as you did here: "Yesterday night I found something, but I know that the programmer will be pissed off with me again with my next sentence because he believes that his code don't have errors ...".

We are now asking you to note that personal criticism is not acceptable here, and to make your comments and ask your questions in such a way as to avoid any perception of that. If you continue, your account will be closed and you will not be allowed to post.

I apologize again if I insulted anyone, as I said before it was never my intent.

Reading again that sentence, I admit it can have a bad interpretation, but I said it in a fun way.

Thanks PBudmark, your explanation was very constructive, once again I admit I didn't saw that the precision would be lost in the variable if the >>8 was made before. I've made tests without considering the variable limitations, and in this pseudo high level language, that has to be carefully taken in account.

Pass this, any suggestion on how to get the all the CTs reading similar values?

Cheers.

Maybe it's time to go back to basics..... wrap your CTs around a wire with:

• no load
• a large known resistive load

Thank you dBC.

I haven't done that again because, I don't want to take my EmonPI out of service for a while that would be needed to accomplish that. The 2 CT's I'm testing are inside the breaker box wrapped to the mains live wire and later I will take the second (CT3) to another breaker I want to measure.

Do you think that it possible that I'm having a so basic problem when:

• On loads lower than 500W (reading with 3 loops around the CT with calcs made by the FW)
• With sensor order like V, CT1, CT2, CT3 I get CT1 (related to CT3) with higher readings of about 10 to 20 W
• With sensor order like V, CT3, CT2, CT1 I get CT1 (related to CT3) with lower readings of about 10 to 20 W
• On higher loads the problem get worse.

It is a very strange coincidence...

Cheers.

I'm not sure I understood too much of that, and I'm not familiar with your code or the code it was based on, but it seems to me you're trying to calibrate to an unknown, changing signal with non-unity power factor.  That strikes me as challenging.  How do you know which calibration knob to turn when the results don't match?

Nice analogy.

You are right and I should do that calibration.

But, independently of the power factor, is it expected or not, that two CTs should make similar readings when clamped to the same wire?

In my understanding, if they only make the same readings when the reading order is changed in the code, there should be something that I can do to correct the code or improve it.

Anyway, at this point, I'm thinking that this error will happen also with a load with power factor of 1, when I have time I will have to confirm that.

Cheers.

is it expected or not, that two CTs should make similar readings when clamped to the same wire?

Yes, I would expect that, provided they're both properly calibrated.   Way down at the low-end, CTs start to come off the rails, especially in phase error, and each example of a CT does that differently.  So if you were measuring a tiny signal I could imagine they could start to diverge quite significantly, but otherwise I'd expect them to read the same.

But if they haven't been calibrated, then you're kinda' stabbing in the dark.  You may well stumble across some combination that appears to work for one situation, but doesn't for another.

I don't want to take my EmonPI out of service for a while that would be needed to accomplish that. The 2 CT's I'm testing are inside the breaker box wrapped to the mains live wire and later I will take the second (CT3) to another breaker I want to measure.

Do you have a breaker that only has a known, large, resistive load on it?  An electric oven breaker would be a good candidate.  Maybe you could calibrate your new CT to that circuit?

Hi.

The reason I keep stabbing in the dark, is because, from my previous calibration experiences, I ended using the same values for all CTs, slightly different from the original ones in the sketch, but the same for all, since all the CTs are the same model and maker and I get (at least apparent) correct values just by changing the readings order, until I have time and opportunity to make new calibrations ,I have to keep stabbing where I have room to. 

That is the way I normally calibrate the CTs, but I still have to open the breaker box and change the CTs.

I thought of another thing, at this point I have a an even number of readings, one thing I wil try is to make it odd so the readings are irregular along the passing time, probably this means nothing, but...

Cheers.

Hi.

Latest findings.

dBC, you seem to be an expert on this area, please tell me if I'm wrong on this assumptions.

I started looking thoroughly to the ISR code and limitations and found that (at least in my code) there is an ISR cycle in every energy Hz that is taking longer than 104uS and from what I understood this will mess the two values that read next. I suppose that the read value in the next ISR cycle will be the one requested in the present cycle because one cycle has been skipped and the value requested before is lost.

Then I found that this situation seems to be caused by this division
long realPower_for_energyBucket  = sumP_forEnergyBucket / sampleSetsDuringThisCycle;
that delays the code for about 40uS

​A curious thing is that the compiler/linker seems to be smart because if I remove the code where that variable is used (and still leave the division) it seems to also remove the variable creation and (at least) the division because the delay of the division also disappears.

To get to this conclusion I just removed the division part of the above line so that the variable is still used in the code after.

The bad news are that, even with the ISR always under 90uS I still get the reading errors, and I was hoping that this error was coming from this (1 wrong sample for every 48 samples or 240 wrong samples per second)... seems not. :-(

Cheers.

I'm not familiar with your code, or the code it's based on, so I can't really comment.  But I can confirm that gcc is very good at getting rid of anything it's determined can't possibly make a difference.  I had an example a while back where my f/w decides it wants to deliberately restart the box....

    link_led_shadow = original_link_led_shadow;
    while (1);              // Let the wdog bite us

link_led_shadow is a global machine-state variable that my watchdog ISR preserves in non-volatile storage before resetting (to assist in post-mortem analysis).    But gcc was smart enough to look at those two lines and realise there was no point in writing to link_led_shadow because the very next statement would never terminate, so nobody would ever know what was in link_led_shadow.   Of course, making it volatile solved the problem.  This is an example where you need to declare a variable volatile even though the ISR  never writes to it.  The fact that the ISR reads it, is enough to require it being volatile.    The dead-code elimination algorithm is working very well.

found that (at least in my code) there is an ISR cycle in every energy Hz that is taking longer than 104uS and from what I understood this will mess the two values that read next

I think you may be ok, provided it doesn't happen "too often and back-to-back".   Assuming the ISR reads the conversion result  first up, you've got 104 usecs before the ADC has another result for you (I'm ignoring interrupt latency here which obviously I shouldn't for a proper analysis).  But you've actually got 208 usecs before that next result gets clobbered by a third conversion.

So for example, if once in a while your ISR takes 120 usecs (say) but usually only takes 80 usecs, then I don't think you'll lose anything.  You'll return from the long ISR and immediately re-enter to process the next result and then you'll be all caught up.  Effectively you have a 104usec slip buffer that you'd need to consume before anything bad happens.  If you had back-to-back 120 usecs ISRs, then you'd get 16 usecs behind each time.  You could afford to deal with about 6 of them (104/16) before you'd exceed your slip buffer and lose some data  (again, ignoring interrupt latency, which you do at your peril).

The other thing to note is the AVR always promises to execute one main program instruction after the RETI before it looks for more pending interrupts.  You might notice if your ISR consistently takes longer than the interrupt arrival rate, you don't completely hang the main program.  It limps along at one instruction per ISR return.

Hi.

Thank you very much for this explanation.

In fact I was puzzled why the results were so acceptable with a 2% of the values wrong.

This explains it, in fact makes much sense, since the ADC is stored right away in the first few instructions of the ISR.

I was thinking that the ISR call was skipped if the previous one haven't ended, but at the same time my observations of the timings (when I intentionally caused the ISR to take more time) where against that conclusion.

My code has most of the ISR calls under 40uS and about 1/50 between 100uS and 150uS.

I'm presently trying to get rawsamples and try to understand where the reading differences are coming from...

Cheers.

Yeh, Atmel were smart enough to clear down the ADIF flag as they launch your ISR.   So the whole time you're lingering in the ISR,  ADIF is armed and ready to latch the next event.

CidiRome: My code has most of the ISR calls under 40uS and about 1/50 between 100uS and 150uS.

A very interesting observation.  I am hoping to do a similar test in a different way today.  Most of the ISR calls will be short; something different only happens at specific parts of each mains cycle.  The time of highest workload is likely to be when datalog stats are copied for use by the main code.  If necessary, this extra activity could be distributed over multiple ISR visits.  The dataReady flag would only be set after everything had been completed.  With none of the timing in the main code being critical, it wouldn't matter how many ISR visits this workload was distributed over.

dBC:  I think you may be ok, provided it doesn't happen "too often and back-to-back" ... you've actually got 208 usecs before that next result gets clobbered by a third conversion. 

Nicely put.  I believed this to be the case but hadn't quite been able to put it into words.

dBC:The other thing to note is the AVR always promises to execute one main program instruction after the RETI before it looks for more pending interrupts.  You might notice if your ISR consistently takes longer than the interrupt arrival rate, you don't completely hang the main program.  It limps along at one instruction per ISR return.

Very interesting too.  I wonder whether micros() will be updated between back-to-back ISR visits?

Hi.

calypso_rae: If necessary, this extra activity could be distributed over multiple ISR visits.

Despite saying that I have that cycle that goes over the ISR timing, yesterday I've made changes that passed some calculations to the next cycle of the ISR and have already every cycle under 100uS, because, based on what dBC said, I was thinking about going back to the original to have a better (understandable) code.

Cheers.

micros() uses TIMER0 which is free running (regardless of the state of the CPU) at one tick every 4usecs.  But it's only an 8-bit timer, so overflows every 1024 usecs.  The TIMER0 OVF interrupt is used to keep track of how often that's happened.  So micros() fetches it's bottom 8-bits from the free running hardware, and ORs in the top bits from the software overflow count (and then multiples it all by 4 so as to return the more natural unit of usecs rather than 4-usecs).

TIMER0 OVF is a higher priority than the ADC completion interrupt, so if it has overflowed while your ISR is running, and your ISR wants to re-run immediately for a second time, I'd expect the TIMER0 OVF to get in between the two "back to back" invocations of your ISR.

If you call micros() at the beginning and end of your ISR, there's a good slight chance the 8-bit timer will have overflowed in between. In theory you could re-enable global interrupts at the start of your ISR, and let the TIMER0 OVF ISR interrupt your ISR.  But an alternative, assuming you're timing things that take less than 1024 usecs, is to simply add 1024 to the result if you find your second call to micros() returns a number smaller than your first call.

Hi.

I normally don't store the values returned from micros(), instead a subtraction is made. I haven't noticed negative values being returned by the subtraction except on the beginning in cases where I reverse the order of the micros() samples to get the time the whole loop takes and in that case I get a first super high value (unsigned).

Cheers.

Yes, fair comment.  Thinking about it some more, TIMER0 only overflows every 1024 usecs, and your ISR is of the order of 100 usecs, so my "good chance" claim above is a bit strong.  I'll change it to "slight chance".  At any rate, if you ever do see time going backwards inside your ISR, you'll know why (as you say, it'll appear as a very large difference since it's all unsigned).

With regards latency, I dusted off an old DataLogger sketch (attached below) I wrote a while back.  Its ADC ISR takes about 3 usecs normally, and about 7  usecs when a complete oversample is ready to be handed over to the main loop.  I added some code to make it do a bunch of useless stuff every 10th invocation, and to also check at the end of the ISR if a new ADC conversion had rolled in, i.e. if it's starting to slip behind.  

The useless work is just enough to push it over the edge in the case where an oversample is ready to be handed over.  In all the other cases it gets out just in time.  So in summary, when the 10th ISR invocation happens to align with an oversample being ready (every 256th invocation) the slip detection code fires and triggers the scope.

The pink trace is high whenever we're inside the ISR, the blue trace goes briefly high whenever the slip-detection code has detected a slip.  In the first pic you can see a bunch of ISR firings.  Most are so short as to just appear as a blip at that timebase, but every 10th one is longer.    And you can see there is 1.04 msecs between the two long ones.  If you look really closely, you can just see the second firing hard up against the end of the long firing.  It looks like a blemish at the end of the long firing.  

The second pic is zoomed in on that point, and shows exactly what happens when the ISR takes too long.  We're in the ISR doing the useless stuff (pink high) and as that finishes the slip-detect code discovers we've been in there so long that another ADC conversion has completed so indicates a slip (blue high).  So we exit the ISR knowing that we're immediately going to re-enter  as another ADC conversion is waiting to be read.  The last thing the ISR does on the way out is to set pink low and the first thing the new back-to-back ISR does is to set it high again.  And you can see that takes a full 3.58 usecs (or 57 cycles).  So that gives you a feel for how much latency you should allow for (at least for my simple ISR).  

If you look at the prologue and epilogue of the ISR below, you can pretty much account for all of those cycles.   The rjump, push, sbi, cbi and pop all take 2 cycles.  The in, out and eor each take 1, and the reti takes 5 (this was done on a 2560 with the wider PC).  I think that adds up to about 54 cycles, but it takes 5 cycles to get to the vector after the event happens, so we're actually ahead (I may have miscounted some instructions or not lined up the cursors perfectly... one cycle is just 62.5 nsecs). The more complicated your ISR (and I bet yours is way more complicated than mine) the more registers its going to have to preserve and restore, so your latency could be even higher.

      74:    1f c1           rjmp    .+574        ; 0x2b4 <__vector_29>
      76:    00 00           nop

000002b4 <__vector_29>:
     2b4:    1f 92           push    r1
     2b6:    0f 92           push    r0
     2b8:    0f b6           in    r0, 0x3f    ; 63
     2ba:    0f 92           push    r0
     2bc:    11 24           eor    r1, r1
     2be:    2f 93           push    r18
     2c0:    3f 93           push    r19
     2c2:    4f 93           push    r20
     2c4:    8f 93           push    r24
     2c6:    9f 93           push    r25
     2c8:    af 93           push    r26
     2ca:    bf 93           push    r27
     2cc:    2d 9a           sbi    0x05, 5;        PORTB |= 0x20;
...

     3c8:    2d 98           cbi    0x05, 5;        PORTB &= 0xdf;
     3ca:    bf 91           pop    r27
     3cc:    af 91           pop    r26
     3ce:    9f 91           pop    r25
     3d0:    8f 91           pop    r24
     3d2:    4f 91           pop    r20
     3d4:    3f 91           pop    r19
     3d6:    2f 91           pop    r18
     3d8:    0f 90           pop    r0
     3da:    0f be           out    0x3f, r0    ; 63
     3dc:    0f 90           pop    r0
     3de:    1f 90           pop    r1
     3e0:    18 95           reti

[EDIT] Removed a race condition in DataLogger.ino while accessing lagcount, and made the display incremental rather than cumulative.

@dBC,

Nice work - refreshing to have solid, clear data to work with.

Hi.

My assembly coding is a little bit rusty, last time I used was about 10 Years ago to build a simple Washing machine programmer (it is still working while the original mechanical ones didn't last 2 years).

I built it using an ATmega8515 (if I remember correctly).

Before that it was at school with 8086 and some 32bit virtual RISC I can't remember the name and even before that it was Z80 in my very very old spectrum... Reading and recording to tapes Piiiiiiiiii, pip......

Cheers.

Hi.

Today I took the time to tweak my firmware soo I could see what is going on.

This image show the raw samples taken over an entire cycle (in fact they are from 4 different cycles, but within a small interval in time). CT1 and CT3 are hooked to the same live wire (the grid) and CT2 is the solar that curiously seems to have a slight production at night) and no shift calibration has been made, so despite being aligned, CT1 shoud be displaced 104uS from Voltage, CT2 208uS and CT3 312uS.

This thing that is puzzling me more it the Voltage wave that I would expect to be almost a perfect sinusoidal one and is very distorted, is this normal?

Cheers.

Yes, V does look quite distorted.  It varies from region to region based on how good your local supply is but I don't think I've seen one that bad before.  How are you sensing V, and assuming it's via a transformer, is that transformer powering anything else, like your energy monitor for example?

CT1 and CT3 look encouragingly similar, but for the phase shift.  Is the phase shift as predicted in terms of multiples of 104 usecs?  Because you're not measuring a purely resistive load, you can't really comment on any phase shift between V and I, i.e. the phase shift you see between V and I probably exists in the real signals.  Below is a V,I plot from my energy monitor, after sensor+meter phase shift removal, demonstrating how out of phase V and I can be.

Hi.

The V is being taken through an AC-AC transformer that was originally from a DLink Router.
A cable with 1mm wires with about 1.5m comes from the breaker box where it is connected to one of the socket outlets breaker.
To this cable are connected the AC-AC transformer and he AC-DC 5V 2A EmonPi power adapter.
This AC-DC 5V 2A power adapter is coincidentaly one from a DLink Access point that I adapted with an USB socket.

About the CT1 and CT3, I believe that the difference related to the two samples being taken with 208uS apart.

The only thing I forgot to say is that the live wire has 3 loops on CT1 and CT3 and 10 loops on CT2.

Any more thoughts?

Cheers.

Try putting some dummy load on the AC-AC transformer if all it is doing is feeding a high R value potential divider into the ADC pin - the V waveform should change.

About the CT1 and CT3, I believe that the difference related to the two samples being taken with 208uS apart.

So now that you've presumably got the raw numbers in a spreadsheet, you can do your own Power calculation using sigma (CT1*V)  and then again using sigma (CT3*V) and see if you come up with the same discrepancy as your monitor does.  Then you can shift one of them in time (in the spreadsheet) to align with the other, and see how much of the discrepancy in Power disappears.    That'll give you a pretty good feel for how much of your observed discrepancies are down to uncorrected phase errors.

If you look at that lights signal of mine above, you can see how important it is to have the V-I phase information a true reproduction of what's happening on the wire if you want to accurately measure non-linear loads.  When all those CFL caps come on at the same time, the current goes from 0 to ~1.8A in an instant.  That green line is very near vertical and it's all happening along a fairly steep part of the V sine-wave.  If you erroneously shift the green relative to the red you can get quite a different result.

Now that you can see your captured signals, I think you're also in a good position to play with their PHASECAL knobs.  If you can tweak each of their PHASECAL knobs so that the two signals align, you might then get better results.  But keep in mind the usual provisos... CT phase error varies with current etc. so you can't expect any one pair of PHASECAL settings to work perfectly across the entire dynamic range of your meter.  Aligning them for your "average power load" is about the best you can do, and hope that your CTs Phase Vs Current graph is fairly flat.  One rule of thumb I often see quoted is to calibrate your phase errors at expected Imax / 10, but your application might vary.

Hummm.

Would it be a good idea to make 192 sample log of an entire V wave and update it, lets say, every 8 Hz and use that log to calculate the precise power against 192 samples of each CT, one cycle/CT at a time instead of every CT and V each cycle.

I think this way we wouldn't have to deal with phaseShifts. but in the other hand, very irregular loads could be more prone to bad readings...

Is this idea too much crazy?

Cheers

It's an interesting idea.  I think you're saying V doesn't change much so capture it once (refreshed 8 times per second) and then spend your valuable ADC resources focusing more on the CTs?   I don't really have a feel for whether that would give more accurate results, or less.  Others may?

[EDIT] - thinking about it some more, if you stopped looking at V, how would you keep your I readings sync'd to the right place in your stored V array?  The I zero-crossings often happen nowhere near the V zero-crossings and there are often a lot more of them.

But in any case, remember there are two sources of phase shifts you need to deal with.  The one you're addressing is caused by the fact you've only got one ADC and it takes 104usecs to get a reading.  Even if you had dedicated, synchronised ADCs  (I've got 21 24-bit ADCs in my energy monitor, 3 for voltage and 18 for current all running simultaneously) your VTs, CTs and various filters all introduce significant phase shift.  So you're always going to have to deal with correcting phase shifts, even if you eliminate the ones caused by time-sharing  a single ADC.

Did you try emjay's suggestion of loading up your VT a bit?  It's a good idea.

Hi.

I've thought of what you said in the EDIT part and you are right, and I think that would have to be a mater of timing.

To be truthful I'm not sure I understood the source of the second phase shift :-(
If they are all synchronized, are you talking about hardware delays? If yes are they even perceptible at a 104uS scale?

About putting a load on the AC-AC transformer, what value would you suggest? Do you think that the distorted wave can come from that?

Another thing, I looked at the AC-AC input circuit and noticed that there is a 1uF capacitor in series with the input, won't this cause some kind of delay since the capacitor has to charge/discharge every cycle? Is that the hardware delay?

Cheers.

If they are all synchronized, are you talking about hardware delays? If yes are they even perceptible at a 104uS scale?

Yes, your sensing hardware (CTs and VTs) plus any filters you might have all introduce significant phase errors.  Check out "4. Phase error" in this report:  http://openenergymonitor.org/emon/buildingblocks/report-yhdc-sct-013-000-current-transformer

About putting a load on the AC-AC transformer, what value would you suggest?

Something that doesn't exceed the output power of the transformer.... maybe 1/2 its rated output?  It's just an experiment at this stage to see if it improves the shape of your V, so play around with it.

Do you think that the distorted wave can come from that?

It's possible and easy to test, right?

Hi.

I've not tested adding the load to the AC-AC transformer yet, but I will test it as soon as I can (already have a lamp to test that will draw about 500mA).

While I don't have the chance to look at it, I noticed that the the distorted AC wave seems to be related in some way with the CT1 wave. I've multiplied the values by 10 and notice that the most impact on the AC wave happens when the CT1 wave falls almost instantly from a positive value to a negative one.

This raises some questions:
What can cause this reactive (don't know if it is the correct designation) values?
The permanent load on this house are two computers with an APC Smart UPS, the fridge and the freezer (don't know if these ones where on at the moment I took the samples)

In this image the CTX is CT1 multiplied by 10.

Cheers.

I noticed that the the distorted AC wave seems to be related in some way with the CT1 wave.

Do you suspect cross-talk between your V and I channels?  An easy test for that would be to unplug your CT and see if V changes (normal cautions apply about not unplugging a burdenless-CT while it's wrapped around a conductor carrying current).  It looks like there's a bit of blip in your CT2 (PV right?) reading at the same time.  That could be more evidence of cross-talk, or it might be the inverter struggling to track the distorted V wave.

Or do you suspect local V sagging in your house wiring due to your loads?  An easy test for that would be to power off all the loads and see if V changes.

The permanent load on this house are two computers with an APC Smart UPS,

My guess would be them.  Again, easy to test, right?  Unplug it from the wall (let it run off batteries) and take another capture.

Was it daylight hours when you took that measurement?  CT1 is your main street feed right, where power can flow in either direction?   When you've got non-linear loads, and an inverter that only wants to contribute to the 50Hz component, you can get some pretty ugly current signals flowing across that point.  Here's how my house looks to the grid.   There's more stuff going on in the first three odd harmonics than there is at the fundamental.

Hi.

Lets see if I can answer everything, at least what I understood.

About the cross talk, that wasn't my idea (despite being a very good one), I was thinking of something causing some kind of energy spike that would distort the V Wave, I said that because there seems to be something strange on the moment that the CT current goes negative. But, one the other way, the V wave is collected so close to the grid input in the house that I'm inclined for it to be side effect of the inverter, it has is own breaker in the breaker box.
I will do some testing with the PV breaker off, and other options as suggested.

Test with the computers on batteries: I have to improve the way I'm collecting the samples I use to make this graphs before I make more tests (including the PV breaker off), presently it is a multi-step just to collect each sensor.
CT1 (and CT3 when values are present): Grid power (bi directional);
CT2:PV with an APS 500W Inverter and production is registered as positive values

The graph values I posted were at night, this one I'm posting now has been improved:
- There is a production of about 10W (late afternoon and very cloudy)
- The scale is real for Volts (with the same Vcal I use in the firmware);
- The CTs scale in Amps is more or less arbitrary because I don't know the Ical for this firmware;
- There is no phase shift correction, Each CT sample is 104uS apart from the previous one and V is probably ahead because (by the code) the index used for the array of samples is incremented after the V sample has been taken and I didn't took much time trying to correct that;

Cheers.

Hi.

Today I have new findings, I will list them:
- It seems that there is no cross talking between the ADC inputs;
- The CTs read a Permanent value 511 with the their breaker off, or when taken out of the live wire, except one that reads 512 (CT1 reads 512, CT2 and CT3 reads 511; CT1 and CT3 were read out of the wire and CT2 with the breaker off). I attribute the 512 value on CT1 to tolerance values on the voltage divider resistors, do you agree? I think that this 511-512 doesn't make any difference in power readings, am I wrong?
- The strange readings of my last post where CT1 current goes negative around sample 20 and positive around sample 42 seems to be caused by the DC power supply (I bought a one Arduino Nano for tests and one of my projects will be to analyze the voltage output of this power supply with and without load);
- I tested the additional load on the AC-AC adapter to check if I would get a better sinusoidal wave, but the only difference was a drop on the resulting voltage (I intend to try different AC-AC adapters and also make tests with the Arduino Nano on this area);
- I tested a pure resistive load on CT3 and the wave was very similar to the Voltage wave. One more nail in the coffin that probably the grid wave is just bad and not reading errors...
- I still get a huge difference in the readings of CT1 and CT3, I think it is related to the order the samples are taken when the current drops around sample 16 and 40, I will make some changes to alter the reading order during the cycle to see what happens.

Hope this helps some one.

Cheers.

I still get a huge difference in the readings of CT1 and CT3, I think it is related to the order the samples are taken 

You've got all the raw samples in a spreadsheet right?  You might find it easier to do the experiments there, with the advantage that it gives you a stable signal to play with.  You should be able to replicate the maths done in your energy monitor in the spreadsheet, and come up with the exact same result.  You can even "fix" the phase error in your spreadsheet and see how big a difference that makes.

Hi.

I've already made some tests with my new Arduino Nano and found that the AC wave is really distorted here, I will try later on other places to see if there are differences See the picture with a graphic created with samples every 104uS.

About the using of phaseShift to correct the readings, I'm getting nowhere:
- There was a warning not to use phaseshift outside of the range 0 to 1, but from what I understand, in order to get the necessary value the phaseshift would have to be between 1 and 2. Even so, I can't get any difference in the readings no mater what values I put.

Cheers.

Hi.

Today I made some findings:

- I got the phaseShift algorithm to make a difference in the outcome by changing the code (switched the place where lastSampleV_minusDC_long is updated): it seems that when the phaseshift algorithm was applied, sampleVminusDC_long and lastSampleV_minusDC_long have the same value so the algorithm would return the latest value no matter what value phase shift was set to.
- I still get a difference between CTs but the result is already much better.

PS: I took a long time to write and re-write this post in order to avoid offending any one. If still I did manage to do that, I sincerely apologize.

Cheers.

found that the AC wave is really distorted here

Possibly - your series of samples of the V waveform are certainly distorted as before. But part of this may still be from how you sample. Did you use the same random plug-in transformer to step down the voltage from mains to low enough for the ADC input as for the earlier readings posted?

For a definitive (but short term only) test, with sufficient care with checking that there is a true Neutral potential not too far from GND and floating the node supply to reference this neutral, a resistor-only divider can be used. Not to be attempted if you don't thoroughly understand the risks involved.

Hi.

emjay: Probably you are right, I'm using transformers to step down the voltage level and I believe that the capacitor in the Voltage divider may also being affecting the result... To be truthfull I don't like the idea of using only a voltage divider but I understand how it work.

A good thing was to see AC waves taken by the other's EmonPI.

Cheers.

You need to be VERY, VERY careful if you do try to use the resistive divider connected directly to the supply. Remember that if for any reason the neutral becomes disconnected - anywhere, it could be a fault outside your house - everything become live at the full mains voltage. You must still treat the neutral conductor as "live".  I've been an electrical engineer for all my working life, and professionally qualified for around 30 years. I do not recommend you do as emjay suggests unless you fully understand the risks and the consequences should anything go wrong.

Robert:
That's exactly why I'm not even thinking about trying.

But, I have an AC transformer with lower voltage that I will test without the AC voltage divider to se the results.

About the phaseShift correction algorithm, after I made the change, I'm getting a more regular difference of around 10w that I'm trying to correct, but before I had much worse and irregular differences.

Cheers.

Hi.

Today I remembered of a simple test that allowed me to verify a few conclusions that I made before. It is so simple that I'm shocked and annoyed with myself that I didn't remembered it before.

I configured CT3 to read the values from the same analog port of CT1. With this simple test I was able to determine a few things:
- The phaseShift correction I made to the firmware is absolutely crucial to get more accurate readings from the CTs.
-  The 10W difference I referred in my last post may well be hardware related.

Tests with CT1 and CT2 configured to the same Pin (aka same hardware but readings at different timings)

With phaseShift correction
<500W Load = Very similar Readings
@2000W Load = CT3 reads more 15W that CT1 probably result of (it also has a small effect on the amplitude, which may need to be corrected), Is there an effective yet simple algorithm to correct the amplitude?

Without phaseShift correction:
negative to 500W Load = CT1 with a constant 10W reading above CT3
@2000W Load = CT1 with readings above 40W higher than CT3

Cheers.

Hi there.

I think I finally got the phaseShift algorithm to work correctly.

I implemented a way of delaying the calculations so I don't have to extrapolate the value for the next Volts point. So, if the phaseShift is lower than 1 the regular method is used, but if the phaseShift is higher than 1 the I value is stored so the calculation with it is made on the next cycle and in the present cycle the calculation is made with the previous I value. For example, a phaseShift of 1.75 becomes 0.75 and the calcuation is made with the I value from last cycle.

With this improvement I've been able to get a difference of 0 to 1 Watts at any load (up to 4kW) with maximum occasional spikes that go up to 2 W and down to -1W. This measurements where made at my grid connection with the same CT being measured at with 104uS and 312uS apart from the V reading and the described phaseShift correction being applied.

Cheers.

Hi.

Since my last post where I was able to get accurate power reading despite the Phase Shift, that I've been struggling to get two CTs of the same model to read the same values when hooked to the same live wire.

Today I stopped my emonPi to measure the voltage divider, burden and other components and compare the values to my off-EmonPi CT circuit and I wasn't able to measure the capacitors.

There is one thing that is intriguing me, the Voltage divider capacitor is stated to be 10uF on the schematic (C3, C4 and C10), but this capacitors are on board smd that are so small that I can't believe they are 10uF.

The capacitor I'm using on my off-EmonPi CT circuit is a TANTALUM 10UF 10V.

Can someone confirm me that the capacitor is 10uF and that the one I'm using is adequate?

I've been told many times that the differences I'm having may be related to the margin of error of the components specially the CT, but the fact is that at least the CTs perform exactly the same way when I exchange them with each other.

What puzzles me about the readings is that the difference is not linear:
@400W the difference is about 10 W
@2300W the difference is about 25 W
Considering the second difference, the first should be less then 5W.

Any ideas are appreciated.

One thing I will do is confirm that both CTs read zero when not hooked or hooked to a wire that has no current passing.

Cheers.