Archived Forum

You are reading every 6° approx, so 5 samples = a ridiculous phase shift. I honestly don't believe it. I think you need to track down why you have so much phase difference. It shouldn't be that much. Have you made any of the coupling capacitors smaller? Are you sure you had a resistive load ? ! ! ! 

There are two effects: (1) the TIME difference between the two readings (because the software reads sequentially of course), and (2) the difference between the phase shifts of the a.c. voltage sensor and the current sensor. Depending on which is read first and which is larger, that either adds to or subtracts from the time difference. I plotted the phase shifts for the Mascot adapter and the YHDC c.t. at various voltages, currents and burdens and you can check those numbers in the reports. The time taken to read an analogue input is around 180 µs (= 3.24°)

There are ~122 samples taken for every channel (voltage and curent) in one full wave of 20ms (50Hz), means ~164us/sample. How did you do this as one analogRead() takes ~110us? Or did you read one full wave of voltage samples first and then one full wave of current samples? But how did you synchronize this?

Strange. And the 1ms peaks on the current are also quite strange. 

Assuming that you are looking at the arduino link above, not the scope output...

You can reduce the time of analogRead by changing the pre-scaler, without substantially reducing the resolution.

This is the Arduino code I used to make the measurements above, I've using a 32 division on the prescaler, 128 is standard.

 ATMEL reference document... http://www.atmel.com/dyn/resources/prod_documents/DOC2559.PDF

QUOTE: The ADC accuracy also depends on the ADC clock. The recommended maximum ADC clock frequency is limited by the internal DAC in the conversion circuitry. For optimum performance, the ADC clock should not exceed 200 kHz. However, frequencies up to 1 MHz do not reduce the ADC resolution significantly. Operating the ADC with frequencies greater than 1 MHz is not characterized.

// define the input pins
#define voltageSensorPin A2
#define currentSensorPin A3

#define m 250

int sampleV[m];
int sampleI[m];

// defines for setting and clearing register bits
#ifndef cbi
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#endif
#ifndef sbi
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))
#endif


void setup()
{ 
  Serial.begin(115200);

  //set prescale as per forum on http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1208715493/11
  //table of prescale values... http://i187.photobucket.com/albums/x269/jmknapp/adc_prescale.png
  //Prescale 32
  sbi(ADCSRA,ADPS2);  //ON
  cbi(ADCSRA,ADPS1);  //OFF
  sbi(ADCSRA,ADPS0);  //ON
}

void loop() {

  int i=0;
  bool flag1=false;
  bool flag2=false;

  while (i<m) {
    sampleI[i] = analogRead(currentSensorPin);                 //Read in raw current signal
    sampleV[i] = analogRead(voltageSensorPin);                 //Read in raw voltage signal
    delayMicroseconds(100);

    if (sampleV[i]>700 && flag1==false && flag2==false) flag1=true;

    if (sampleV[i]<250 && flag1==true && flag2==false) {
      flag2=true;
//Wait until almost the start of the wave
      delayMicroseconds(6000);
      i--;
    }

    if (flag1==true && flag2==true) i++;
  }


  Serial.println("VOLTAGE=");
  for (int index = 0; index < m; index++)
  {
    Serial.print(sampleV[index]);
    Serial.print('\t');
  }
  Serial.println("");
  Serial.println("CURRENT=");
  for (int index = 0; index < m; index++)
  {
    Serial.print(sampleI[index]);
    Serial.print('\t');
  }
  Serial.println("");

  delay(20000);
}

Thanks for checking the graphs Robert.

I'm using a current circuit the same as this one, MINUS the C2/R3/R4 circuit - for this I'm feeding 2.5v from an OpAmp divider circuit (as per calypso_rae circuit).  I've also not installed the bilateral switches so all that lot is removed and I have C1/R11 and R2 in place.

C1 is 10uF as per below, but just checked the ATMEL sheet where this diagram was copied from and that is using a 47uF - is this the problem?

I'm using a 22ohm for RS (burden), and the SCT-013-000 clamp.

Answering my own posts, I've changed to a 47uF capacitor and that seems to have brought things back in line.

Just for reference, I'm reading current before voltage on the Arduino.

If you get chance, have a look at the google spreadsheet again

https://docs.google.com/spreadsheet/ccc?key=0AvDz3-CEPA_tdGlOb1JQaXJLVW5...

I've also discovered what the small spikes in current are as well - it only happens when I've got an open COM port window to the arduino!  On the scope, you can see the spikes "sliding" along the current wave form.  Without any serial connection, the scope shows the waves as smooth - is this just noise being picked up by the Opamp ?

 My Opamp(s) and Arduino CPU are all on the same breadboard by the way

Additionally, I'm just connecting pin 20 AVCC to VCC just as the emonTX does - looking at most references, if you are using the ADC, this should be connected to a low pass filter - not just to VCC - is this where the issue lies?

Are you sure it was 10µF and not 1µF?  According to me 1µF will give you a 25° or so phase shift, 10µF about 2½°. 

The spikes (if they're asynchronous) are almost certainly digital noise - picked up either by the op-amp or the analogue input itself. You need to clean up your earthing arrangements, and adding some decoupling to AVcc wouldn't hurt.

You can reduce the time of analogRead by changing the pre-scaler, without substantially reducing the resolution.

Thnks for clarifying this, it is always good to know how the measuring data was taken . And, true, you can increase the clock frequency without to much noticable increase in error but the datasheet also specifies an increase in absolute accuracy error from 2 to 4.5 LSBs when increasing the clock from 200kHz to 1MHz.

Answering my own posts, I've changed to a 47uF capacitor and that seems to have brought things back in line.

I would have thought that this change has an effect, but not as big as it seems to be in reality (and my 'stomach feeling' would have predicted a shift in the other direction!?!). Good to know. A Pspice simulation with different values would be nice here.

Regarding the 'noise' from the serial: Breadboards and Opamps can be a tricky combination (although this a a really slow Opamp). Hmmm, and 1ms does not really fit to 115kBaud....

"(and my 'stomach feeling' would have predicted a shift in the other direction!?!)"

See here: http://www.electronics-tutorials.ws/filter/filter_3.html  In the initial results posted, the current curve led the voltage curve and the filter was in the current input. Those results were consistent with a capacitor value of about 1µF, as I wrote above. I think the capacitor was faulty, wrongly labelled or misread.

"Regarding the 'noise' from the serial: Breadboards and Opamps can be a tricky combination"

There is anecdotal evidence that Arduino-based designs with prototype-board front end circuitry are more prone to reading current when none is flowing (i.e. noise pick-up), than emonTx-based installations. And that is without the op-amp.

".... Hmmm, and 1ms does not really fit to 115kBaud...."

And the spikes are probably the effect of ground currents resulting from the message burst and smoothed out by some supply decoupling, rather than individual 1s and 0s. 14 characters transmitted lasts about 1 ms, which sounds about right to me..

Robert,

I think your last but one post came when I was editing. So the effect of lower C is shifting the curve to the left, ok (hope this helps to re-program my 'stomach').

But 115kBaud and 12 bits per character gives ~104us and 14 characters would then give ~1.5ms. This is why I thought it would not really fit. Still, the 'board design' could certainly induce oszillations in the opamp circuit.

The ripples on the sine wave are there even when not outputting any serial data - as long as the arduino COM port window is open, the noise is there !  Could this be one of the clock or DTR lines ??

I even tried moving the serial.begin() instruction to later in the code (just before first print statements) hoping that the serial connection would be closed until this point - no luck I'm afraid.

The 10uF capacitor was fresh out of the bag of recently purchased cap's - I'm sure it was 10uF - will check again to make sure - could be faulty !

Any recommendations on the decoupling to AVcc?  Seems very little of our standard boards include this - including the eMonTX !

I might just swap over the circuit onto strip board so hope that helps.

JBecker, I'm using a 32 prescale on this circuit/code so running at 0.5Mhz, so hopefully the error rate will be okay.  After code optimisation, I will try and go back to the default assuming I get enough CPU cycles to do everything.

I've added a 100nF capacitor between ground and  C1/R11 and that has removed the transmission spikes I was getting - don't think its changed the waveform - any comments please ?

I've updated my spreadsheet at...

https://docs.google.com/spreadsheet/ccc?key=0AvDz3-CEPA_tdGlOb1JQaXJLVW5...

"I've added a 100nF capacitor between ground and  C1/R11 and that has removed the transmission spikes I was getting - don't think its changed the waveform - any comments please ?"

I don't think I would have added it there - but I'm not at all sure where I would have added it - if at all. I don't know what your circuit layout is - from what you mentioned earlier I presume plugboard - so my first thoughts would be to look carefully where the digital currents to the µP flow - both supply current from the power supply and ground current returning to the power supply (not forgetting the ground/common of the serial link) and make sure all these paths are as low resistance as you can manage, and as separate as possible from the equivalent on the analogue side, the two joining as close to the power supply and the main decoupling/smoothing capacitors as possible.

Track down App Note AVR042 on the Atmel website - you might find some inspiration there.

Thanks for the comments Robert, I'll go and check the plugboard layout and read through those notes.

I think the visibility of the spikes is simply down to over-amplification of a small current clamp signal, and hence the ripple is appearing larger than in reality.

Just spotted this thread while browsing around ...

Regarding the large amount of phase advance seen with the standard CT, here are a couple of pics that I took yesterday using a freebie digital scope tool.  The green waveform is simply a potted down version of the mains, with (presumably) no phase-shift.  The red waveform is the output from a standard YHDC 30A CT with a 39R burden.  The load in both cases was a 2kW resistive 'space heater'.

4a shows these waveforms as graphs, and 4b as a Lissajous diagram.  As you can see, the CT is causing quite a significant phase-advance.  If a larger valued burden were to be used, such as the 150R that I initially suggested for my Mk2 design, the phase shift would be considerably greater than this. 

I've never used a Mascot adapter but have obtained AC voltage signals using a variety of transformer-based approaches.  Unfortunately, they all introduce distortion of the waveform including some phase-advance.  Because both types of wound-components shift the phase in the same direction, their effects largely cancel out.  To realign both waveforms for power calculations, PHASECAL can therefore need be less than it would otherwise need to be.

If anyone knows a way of obtaining an image of the mains voltage without distorting the waveform (other than by using a resitive divider), I'd be most pleased to hear about it.  Until a better arrangement is found, I fear that our attempts to divert all surplus PV energy will be unlikely to succeed.

HEALTH WARNING:  If wishing to repeat these tests, please read the comprehensive safety guidance posted by Robert Wall in his report on the Mascot adapter.  Connecting to live mains, and removing earths, can be bad for one's health!

If we assume (for safety) that we would be using a transformer, and accept that this is introducing a phase shift, could we not then artificially delay the current reading using some sort of delay or timing circuit to match ?

Will still need to take into account the delay offset of the 2 analog reads on the Arduino as well though.

If anyone knows a way of obtaining an image of the mains voltage without distorting the waveform (other than by using a resitive divider), I'd be most pleased to hear about it.  Until a better arrangement is found, I fear that our attempts to divert all surplus PV energy will be unlikely to succeed.

Robin,

is the transformer distortion really a factor that will prevent you from making 'good enough' calculations? Then I would still propose to use a 400V transformer. If you have problems getting one, I could supply you with an E30 400V primary 21V secondary 2.1VA for testing. I don't have a price at the moment, but should definitely be less than €8. This is a bare transformer without leads nor enclosure though.

stuart

"could we not then artificially delay the current reading using some sort of delay or timing circuit to match ?"

That's precisely what the interpolation calculation and PhaseCal achieves!

"2 analog reads"

2 ? ? ?

Stuart: If we assume (for safety) that we would be using a transformer, and accept that this is introducing a phase shift, could we not then artificially delay the current reading using some sort of delay or timing circuit to match ?

In short, yes.  If either transformer introduces a small amount of phase-shift, that's not a problem.  Voltage and current measurements are taken alternately approx 2.5 degrees apart.  If one is advanced by no more than 5 degrees relative than to the other, then that difference can be easily accommodated by the use of PHASECAL in the range 0 to 1. 

Will still need to take into account the delay offset of the 2 analog reads on the Arduino as well though.

By measuring V and I in the right order, this offset can be used to our advantage.  Whichever is the more advanced should be measured first. 

For a given set of hardware, if current is the more advanced, then current should be measured first and voltage second.  PHASECAL should then be applied to the voltage samples.  If voltage were found to be the more advanced, then voltage should be measured first, and current second.  In that case, PHASECAL should be applied to the current samples.

Jörg, Thanks for your suggestion (again) of using a 400V transformer.  When I tried using two 240V ones in series, the phase-shift was definitely reduced but the shape of the output waveform was still distorted.  If I can find a better transformer which does not distort the waveform, that would be great.  I may be more likely to find such a device as a 400V model rather than a 240V one.

Attached are the results of comparing my normal 'telecom' AC adapter against a known good voltage reference.  As you can see, it is not the phase advance that is main problem; it is the distortion of the waveform's shape that concerns me.  That's why I need to find a better transformer.

I wonder if an op-amp circuit like this

http://www.tedpavlic.com/teaching/osu/ece209/lab1_intro/lab1_intro_phase...

would be useful to delay the current (or voltage) signal rather than using code to fix ?  Would also free up a couple of CPU cycles and remove some floating point math.

 I might have a play with this tonight and see what happens.

Stuart,

you could do the first tests with a simple RC (e.g. 100k poti + 10n). This will give some attenuation of the amplitude too, but you have to calibrate the amplitude anyway.

JBecker

I know of a way - but it's not practical for general use! You need to use one of the many specialised devices (from all the usual sources) - an ADC with opto-isolated digital output - but they require their own isolated floating supply for the isolated mains side as well as a common supply for the processor non-isolated side.

Aside from cost and complexity, there remains the question of working on live equipment and the safety concerns that go with that. It is not something I could recommend to constructors lacking the required knowledge and experience.

This is essentially what daveb did here: http://www.rotwang.co.uk/projects/images/mains_opto.gif

I think the 'best' general universal answer to the problem is a custom-made transformer that has been specified to have minimal waveform distortion and phase shift, ideally in a plug-top case (and even better if it had interchangeable adapters like my Seagate external drive power supply). Unfortunately, cost is likely to be somewhat more than the 'cheap & cheerful' mass-produced adapter.

Because V and I are measured at different times, some means of re-aligning them needs to be included in the code.  If both of the sensors were perfect, either parameter could be measured first.  Applying PHASECAL = 0.5 to the one that is measured second would then align V and I perfectly. 

Because V and I are measured at different times, some means of re-aligning them needs to be included in the code.

Not neccessarily. If the signal that 'comes first' is delayed by the correct amount, you could get rid without any phase shifting calculation. I think this is what Stuart meant. It would make at least these float calculations obsolete.

If the signal that 'comes first' is delayed by the correct amount, you could get rid without any phase shifting calculation.

Agreed.  If one of both of the signals were to be phase-shifted by the correct amount in hardware, there would be no need for any equivalent adjustment in software.  However, I don't think that float calculations could ever be avoided altogether as some form of filtering of the sample stream(s) will always be needed, if only to remove the DC-offset.

However, I don't think that float calculations could ever be avoided altogether....

Disagree . Float calculations can always be avoided. They do not unconditionally give a better accuracy or higher precision. Float uses 4 bytes per variable, same as long. The number of different values that you can build with a long is exactly the same as with a float. A float can just cover a 'bigger' range of values more easily (but with bigger 'holes' in between values). This makes it easier to combine very small and very big numbers in the same calculation and still get a decent result. But the same can be accomplished with integer calculation with careful choice of values (or 'resizing').

We have discussed this in another thread already, a (float) multiplication (0.966*X) gives the same result as an integer calculation (255*X / 256) or ( (255*X)>>8) or ( (X<<8)-X) and the integer calculations are way faster. (There is a small difference in the result, but this is not a fundamental one.)

But I think this might be a bit off-topic here.

Might be on to something here....

I've got this circuit built...

CT SENSOR >> OP AMP GAIN >> PHASE SHIFT OPAMP >> ANALOGUE PIN

AC ADAPTER >> VOLTAGE DIVIDER + OPAMP OFFSET VOLTAGE >> ANALOGUE PIN

After tweaking the "phase shift" by changing the resistance in the circuit from here..

http://www.tedpavlic.com/teaching/osu/ece209/lab1_intro/lab1_intro_phase...

(I used 7.4k as the variable, 10k for RF, and 0.1uF electrolitic capacitor)

I get the following output...

https://docs.google.com/spreadsheet/ccc?key=0AvDz3-CEPA_tdGlOb1JQaXJLVW5...

which looks good to me!

Additionally, I've seemed to solve the spikes I was getting by putting 100nF caps to ground on the op-amps and seperating analogue and digital +/- connections.

Assuming everyone thinks this is a good idea, the only problem I can see is drift on the components due to temperature changes.

I'm now reading VOLTAGE before CURRENT by the way, and this is the sketch I've used to tweak the resistance until "just right".

// define the input pins
#define voltageSensorPin A2
#define currentSensorPin A3
#define m 250

int sampleV[m];
int sampleI[m];

// defines for setting and clearing register bits
#define cbi(sfr, bit) (_SFR_BYTE(sfr) &= ~_BV(bit))
#define sbi(sfr, bit) (_SFR_BYTE(sfr) |= _BV(bit))

void setup()
{  
  //set prescale as per forum on http://www.arduino.cc/cgi-bin/yabb2/YaBB.pl?num=1208715493/11
  //table of prescale values... http://i187.photobucket.com/albums/x269/jmknapp/adc_prescale.png
  //Prescale 32
  sbi(ADCSRA,ADPS2);  //ON
  cbi(ADCSRA,ADPS1);  //OFF
  sbi(ADCSRA,ADPS0);  //ON
  Serial.begin(115200);
}

void loop() {
  delay(1000);
  int i=0;
  //IMPORTANT - ensure power and voltage readings stay constant throughout measurement!!  
  //Read lots of samples to find the min/max range
  //Do this one at a time to avoid delays in analogRead...
  int lowI=9999, highI=0;
  for(int i=0;i<500;i++) {  
    int r = analogRead(currentSensorPin);
    if (r<lowI) lowI=r;
    if (r>highI) highI=r;
  }

  int lowV=9999, highV=0;
  for(int i=0;i<500;i++) {  
    int r = analogRead(voltageSensorPin);
    if (r<lowV) lowV=r;
    if (r>highV) highV=r;
  }

  Serial.print("Voltage range=");
  Serial.print(lowV);
  Serial.print(" to "); 
  Serial.println(highV);
  Serial.print("Current range=");
  Serial.print(lowI);
  Serial.print(" to "); 
  Serial.println(highI);

  int zeroV=lowV+((highV-lowV)/2);
  int zeroI=lowI+((highI-lowI)/2);

  while(1) {
    sampleV[i] = analogRead(voltageSensorPin);                 //Read in raw voltage signal

    if (sampleV[i]==zeroV) {      
      //Get the next cycle
      delayMicroseconds(20000-5500);
      while (i<m) {
        sampleV[i] = analogRead(voltageSensorPin);                 //Read in raw voltage signal
        sampleI[i] = analogRead(currentSensorPin);                 //Read in raw current signal
        delayMicroseconds(50);
        i++;
      }

      //Find where the current goes over zero cross
      int startI=sampleI[0];
      int indexI=0;
      for (int index = 0; index < m; index++) 
      {
        if ((startI>zeroI && sampleI[index]<zeroI) || (startI<zeroI && sampleI[index]>zeroI)) {
          Serial.print("Current zero cross at index=");
          Serial.print(index);
          indexI=index;
          break;
        }
      }      

      int startV=sampleV[0];
      int indexV=0;
      for (int index = 0; index < m; index++) 
      {
        if ((startV>zeroV && sampleV[index]<zeroV) || (startV<zeroV && sampleV[index]>zeroV)) {
          indexV=index;
          Serial.print("  Voltage zero cross at index=");
          Serial.print(indexV);
          break;
        }
      }      

      Serial.print("  DIFFERENCE   =");
      Serial.println(indexI-indexV);

      if (abs(indexI-indexV) < 10) {
        Serial.println("VOLTAGE=");
        for (int index = 0; index < m; index++) 
        {
          Serial.print(sampleV[index]);          Serial.print('\t');
        }
        Serial.println("");
        Serial.println("CURRENT=");
        for (int index = 0; index < m; index++) 
        {
          Serial.print(sampleI[index]);          Serial.print('\t');
        }
        Serial.println("");

        delay(20000);

      }

      delay(1000);

      i=0;
    }
  }
}

Sorry Jorg, I'm probably confusing long and float :(

I have often wondered how effective the PHASECAL approach is at compensating for phase-shift between voltage and current.   So I've just created a tool, as attached, which tests this out.

Both sets of samples, V and I, are in floating point form and there are no dc-offsets or filters.  It simply synthesizes V & I values for one cycle and combines their instantaneous power contributions.  At the end of the cycle, the accumulated power sum is 25, no doubt because I've chosen to have 50 samples per cycle, and V and I both have half-amplitudes of 1.

Anyway, the numbers are not important.  What is important is the effect that PHASECAL has when used outside the 0 - 1 range, i.e when it is  predicting rather than interpolating.  Here are the results:

Column 1: # samples of phase advance (current);  Column 2: PHASECAL, as applied to voltage;  Column 3: power error ratio

-1      0      1.0000

-0.5   0.5    0.9980

0       1       1.0000

1        2       1.0156

2        3       1.0462

3        4       1.0902

So, with a perfect sinusoid, a matched PHASECAL value of 2 (i.e. predicting just one sample into the future) introduces a magnitude error of 1.5%!  As PHASECAL is increased, so does the error - rapidly. 

The small amount of error (no more than 0.2%) when interpolating in the range 0 to 1 is unavoidable in a digital system when V and I measurements can't be taken at the same instant.  An alternative approach in hardware might not suffer in this way.  Another option would be to perform the interpolation using a better method than straight linear. 

For accurate measurements of power, I feel sure that any phase-shifts within our voltage and current transformers should be within one sample of each other.  Then they can be effectively aligned using PHASECAL within the range 0 to 1.  Moreover, they should not otherwise distort the waveform because that can't be rectified using PHASECAL.

Your "phase shift" circuit is commonly known as an "all-pass" filter. The phase shift is of course frequency-dependent, but as the mains frequency tolerance is ±1%, that should not be an issue.

However, if your load includes switching power supplies, the current input is likely to have a higher harmonic content than the voltage (which ought to be a pure sine wave, but we know it isn't), so my instinct would - without the benefit of a full analysis - have been to apply the filter to the voltage wave so as to more faithfully preserve the shape of the current waveform.

This is what happens to a square wave passed through your filter:

[OK, you won't get a current waveform like that].

"putting 100nF caps to ground on the op-amps"

- do you mean decoupling the supply to the op-amps? - recommended practice and a good idea if so.

"0.1uF electrolitic capacitor"

You really ought to use a non-polarised component there - one of those 100nF plastic film capacitors, which should have a much tighter tolerance on value than an electrolytic anyway, and less liable to change its value with age.

The tool that I've just posted can also show the effect of having a PHASECAL value that is not perfectly matched to the phase-shift that exists between voltage and current. 

It turns out that the 'best' value of PHASECAL is not actually one less than the number of samples of phase-shift.  As phase shift increases, the optimum value of PHASECAL increases at a slower rate.  To keep the power error ratio at 1, we find that:

if current is advanced by 1 sample, the optimal PHASECAL value is approx 1.5, i.e. predicting the voltage by only half a sample;

if current is advanced by 2 samples, the optimal voltage prediction is approx 2, i.e. predicting the voltage by only one sample;

if current is advanced by 3 samples, the optimal voltage prediction is approx 2.5, predicting the voltage by only one and a half samples.

Needless to say, I remain convinced that PHASECAL should be a) matched to the actual phase-shift of the measurement circuitry, and b) kept within the range 0 to 1 :)

Two comments on PHASECAL and integer vs float.

1. Phasecal: If you look at the algorithm carefully and draw pictures, it's clear that the amplitude will be affected. That's why in the explanation in building blocks I wrote "(it also has a small effect on the amplitude, which may need to be corrected)."
As PHASECAL is constant, the amplitude change can easily be calibrated out. What cannot be calibrated out is the effect on harmonics, because a phase shift is equivalent to a time shift at one frequency only.

2. Having the filtered value as a float may be beneficial when very small signals (peak-peak amplitude of just a few counts) are concerned and there is a small amount of noise on the signal that is having the effect of adding dither. In those particular circumstances, it can be shown that accuracy will be improved. However, if those conditions are not met, it's doubtful that there will be a significant difference between using floating point and integer values - except of course in processor load.

Robert Wall on Tue, 21/08/2012 - 14:42:  "2 analog reads"   2 ? ? ?

Typo! Should be delay by 1 analogue read.

Robert Wall: apply the filter to the voltage wave so as to more faithfully preserve the  shape of the current waveform. 

I agree that would have been my preference to, however the CT clamp has already moved the phase forward on the voltage readings, so its difficult to delay the voltage using hardware, but fairly easy to push the current phase forward to match. 

The wave forms are not perfectly in sync, they are delayed by the time it takes the CPU to read the samples, voltage then current.

With regard to the shape of the current curve with switching power supplys - I did try this last night, and see small spikes at the top and bottom of each curve.

Robert, do you think the phase delay/all-pass filter is a route worth taking ?  or am I barking up the wrong tree !

I think it's a question of balance - one of those engineering questions where you need to make a judgement based on all the information available. Surely an analogue solution to an analogue problem is likely to be 'better', but at what cost? If it means several op-amps which you need power supplies for and components around, it adds complication and cost. If you accept that and you think you need the extra accuracy for your application, that's your decision and it's fine by me. But remember that you have two distinct and different effects present, and so you really ought to use both approaches - the time delay by interpolation to compensate for the time delay between readings, and the phase shift to compensate for the differential phase shift between c.t and p.t.

The Atmel application note that the emonTx/Arduino design seems to be based on had a direct voltage input, with only a c.t. contributing phase shift. Now the c.t's phase shift isn't constant, but varies with current, so it's impossible to correct totally with a fixed time shift. However, for the purpose of that design - a meter demonstration - they concluded that after calibration the inaccuracies would be inside the standards set for a tariff meter.
(So just considering that last point, once you approach the accuracy of your tariff meter, how are you to know whether you are right, or the tariff meter is right, or whether both are wrong, without a yet more accurate standard?).

I'd like to do the full analysis, but I'm afraid I just don't have time at the moment.

 Analog approach.

Just an off the wall suggestion following Robert's comments on using both an analog and digital approach. If you used an analog multiplier then voltage and power could be sampled which would remove all the problems associated with spikey currents. Robins immersion controller would need the voltage sample to trigger the triac. The multiplier is expensive £6 but it solves so many problems and the circuit is very simple indeed as the multiplier has an offset facillity built in. Phase compensation could be done via some bilateral switches on the voltage feed which would give at least the same or more resolution as the sample rate of current and voltage. If there is any interest I can post a circuit.

When diverting surplus PV using my Mk2 circuit, very little seems to be of importance.

I currently have two rigs on the go and can switch between them by simply plugging the trigger's control wire into pin 9 of one Arduino or the other.   One rig has all the best ideas that I can muster; the other is just the standard setup with the cut-down code that I posted yesterday.  In today's sunshine, the performance of these two rigs is virtually identical.

For accurate monitoring of energy flows, I've no doubt that all sorts of details need to be taken into account, including phase effects.  But for simply diverting surplus PV, it seems that any setup behaves much the same.  The key reason for this is that measurements are only done at one place, and the net flow of current at that point is zero, hence no need for accurate calibration.

A simple RC filter with variable resistor is enough to compensate phase shifts. It can also help to solve another problem of the original circuit. There is no decent analog filtering implemented yet. This means that when taking digital samples of the original waveform at discrete time intervals of ~400us, you are actually 'undersampling' a waveform with possibly much higher frequency content. There should actually be an integrating analog filter with a similar time constant as the sampling rate to get the 'real' picture.

To give an extreme example of the possible effects of undersampling:

take a 10kHz square-wave signal with 5V amplitude and 50% dutycyle. If you sample this with an adc every ms (1kHz sample rate), you can either get a random (even infinite!) number of 0V readings or 5V readings (or a random mixture of 0's and 1's depending on the exact ratio between original wave frequency and sampling rate). This might never 'average out'! And no digital filter can help to solve this! The only solution is a decent analog filtering.

calypso_rae on Tue, 28/08/2012 - 09:34. When diverting surplus PV using my Mk2 circuit, very little seems to be of importance.

Agreed, the simplicity of your circuit+code is one of its many benefits.

The difficulty starts to creep in if you then try and adapt the circuit to make accurate measurements in an attempt to replace the emonTX along with the solar controller all in one "box".

 Technically there should be an antialias filter at lower than half the sample rate.

pmcalli on Tue, 28/08/2012 - 18:27: Technically there should be an antialias filter at lower than half the sample rate.

Is this best done in software or hardware?

An antialiasing filter can not be done in software! I tried to explain that above with an example about undersampling.

The error that is introduced by undersampling (using a sampling rate that is lower than double the highest frequency in the original waveform) can never be corrected afterwards. Information is lost and cannot be 'reconstructed' again.

JBecker on Tue, 28/08/2012 - 10:52.  This means that when taking digital samples of the original waveform at discrete time intervals of ~400us, you are actually 'undersampling' a waveform with possibly much higher frequency content.

The mains supply may well contain lots of high-order harmonics, but is it necessary for the power at these extreme frequencies to be measured?  With a 50Hz fundamental, and a sampling frequency in excess of 2kHz, a typical monitoring system should already be recording all energy up to the 20th harmonic ;)

Robin,

this is a good question. And there is the other question about the bandwidth of the CT (the problem with higher frequency content is much more relevant for current than for voltage in our case). I did not find anything in the data sheet.

But the bad thing about not using an antialiasing filter is, that higher frequencies than half the sampling frequency will 'fold back' into the lower frequency band and introduce additional errors there. I have no idea how big these errors are in our specific case. 

I just wanted to say that adding an RC for phase correction could have additional benefit (killing two birds with one stone).

 I doubt there is very much harmonic content up there. What there might be is mains-borne signals from baby alarms, network extenders and the like. I have no idea of the signal amplitudes these devices employ, but I suspect they will be in the order of volts (rather than 10s of volts).

Interestingly, there is no mention in Atmel's App. note AVR465 of an anti-aliasing filter, and I suspect they considered it unnecessary given the low amplitudes expected at those frequencies and the limitations of a 10-bit ADC.

 Jörg,

The c.t. is quite good enough to pass through any trouble - pretty much unhindered!

"5. Frequency Response.
 The frequency response was checked at a primary current of 30 A and a burden resistance  of 39 Ohms. The phase error remained below 1 degree and amplitude error remained below 1% up to the 13th harmonic of line frequency (650 Hz). At 1.5 kHz the amplitude error remained below 1% whilst the phase error had increased to 5 degrees. At 2.5 kHz  the amplitude error had risen to 3% whilst the phase error had increased to 10 degrees. "

It might well need rather more than a first order filter to pass significant harmonics for accurate measurements for the phase-controlled fraternity and yet cut off adequately at 2.5 kHz.

Robert,

I have seen quite a lot of bullshit on mains current produced by 'cheap and dirty' electronic ballasts (and this time it was not from chinese stuff but from well known european manufacturers!). I don't want to say that this does effect measuring in a normal household very much, most of my (bad) experience comes from emergency lighting systems in big office buildings.

But the lack of nearly any filtering in the AVR465 application note slightly puzzles me. There is the other fact that they do not use any Cs on the analog inputs. This is against any 'good practice' rules for adc inputs. I know quite a lot of controllers where this would give catastrophic results. OK, the AVR adc is very slow, but we have never done a single circuit without a decent RC on the analog AVR inputs. 

Anti alias Is this best done in software or hardware?

The answer is both. Normal approach is to seriously over sample with an easy to implement anti alias filter and then a very sharp digital filter. Most meter chips use delta sigma converters with a simple anti alias then direct multiplication of the current voltage pairs followed by a digital filter. If you want real accuracy these chips are quite inexpensive (£2) for ADE7755

Last night I swapped over from an OpAmp "phase shifter" to using a RC circuit, works fine.  I'm using this in conjunction with the phase calibration in software, which is now in the range of 0.5-0.85.

 Using voltage reading circuit from the original Mk2 design by Robin (as below) I'm getting lots of noise on the AC waveform, like the top wave form diagram below - although lots more that a single spike.

This is corrected by putting a 100nF cap between the 15k resistor and ground.  I'm assuing that this capacitor is then working as a low pass filter in conjunction with the resistors, and removing the "noise"/high frequency components.

 Noise on ac waveform

You may have a problem with the 100nF as it will be across the ref opamp output and they have a habit of oscillating with a capacitor load. Common solution used to soften spikes and pass emc susceptibility is to put a 1nF across the secondary of the transformer

Anti alias Is this best done in software or hardware? ... The answer is both.

Paul, this is not 100% true (only 50% ). After sampling, the unwanted higher frequency contents are folded everywhere into the lower frequency band and cannot be removed by any digital filter (because they are undistinguishable from the 'real' signal). The digital filter in the ADE is a low pass filter which removes the unwanted higher frequency components which are there due to the high sampling rate (~1MHz), but it is not an anti-aliasing filter.

Using the AVR we are stuck with rather low sampling rates and here we possibly have to think about real anti-aliasing (which has to be done in hardware).

And we already use 'digital filters' in the software (high-pass for DC-removal, low-pass (in Robins code), integration, averaging..).

 Anti alias Is this best done in software or hardware? ... The answer is both.

You should read my post correctly I never called the software post filter an antialias filter it is there to recover the dc component of the power after sampling/multiplying with a correct antialias filter. The real problem here is the sample rate is too low and the a/d type is inappropriate for this type of measurement but that is not to say its not good enough for the purpose intended

My 3kW electric kettle and immersion heater each take 12.5A at 240V rms so look much like a 19 Ohm resistors.  Our 8kW shower looks much like a 7 Ohm resistor.  As the mains waveform gets hammered by non-resistive appliances, I can see how higher harmonics will creep in.  Power at these harmonics will presumably be consumed by my resistive loads and should therefore be charged for by the meter.

What I don't understand is how high frequency signals can be usefully carried over the mains.  Why aren't they just shorted out by all the resistive loads that are out there?

(sorry, a bit off topic ...)

Paul,

I did not want to offend you in any way. I was just reacting to your answer to the original question 'Anti alias Is this best done in software or hardware?' And there is only one correct answer to this: It can not be done in software.

Jörg,

Your problem with the secondary lighting is the standard defines the harmonic currents in absolute terms per load, not in percentage terms. So if you get very many small loads, the total harmonic current can be much larger than would be permitted for fewer large loads, yet still comply. We have to assume that the standard was written in anticipation of a 'normal' mixed load.
 

 Calypso_rae,

Of course the loads on your system affect the transmissions. But you have what is in essence the same arrangement as in any radio receiver, an automatic gain control system that aims to maintain the received carrier at a constant amplitude. Bear in mind also that these are usually fairly narrow-band systems, so benefit from sharply tuned filters - again broadly similar to a radio.

Robert,

I think the regulation (that more harmonic content is allowed for small loads) makes sense, it keeps the cost down for these small appliances and the idea is, that the noise of many of these small loads will 'average out' to a certain extent. This is statistically correct, but we may still find a lot of high frequency content in out current signal, which makes me believe that a decent low-pass anti-aliasing filter would be a good idea.  

I think it all depends on how high-frequency signals are treated by supply meters. 

If the supply meter measures( and charges for) energy that is consumed at the 39th or 139th harmonic, then so should our own systems.  If it doesn't, then neither - I would have thought - should we.

 I think it all depends on how high-frequency signals are treated by supply meters. 

"which you will incorporate and the meter will not"

or possibly vice-versa!  The point is, without an anti-aliasing filter, you are dependant on there being no content at a frequency above half your sample rate. If there is content there, you need to know the amplitude and frequency to calculate what effect it will have. Only then can you know whether it is introducing a significant error or not.