Discrepancies and accuracy of the feeds

With my system up and running from three days now, I am starting to do some statistics. At a first glance, I've noticed a considerable discrepancy between what the emonGLCD stores as historical data and what the emoncms stores as "Power to kWh/d".

Let's take the PV production. My setup is a type 1 solar system, so the emonTx CT2 input is the generation input. Yesterday it was a quite sunny day and I've got the following production results:

  • Inverter read: 14.2 kWh
  • emonGLCD: 12.5 kWh
  • emoncms: 13.6 kWh

I know that my inverter overestimates the production of about 2%. What I didn't expect was such an error between the GLCD and the cms as they're relying on the same data. My scope would be to understand which reading is the more reliable.

I would also ask for an "experienced" opinion about the following. When the sun is over, I see the power2 input counting a power between -1W and +1W, all the time. I believe this is due to some noise and/or to the ADC resolution. I have filtered this in the emonGLCD sketch adding up to the genkhw variable positive values only. From you point of view, am I correct?

More weird is the following: when the inverter starts up in the morning and when it is going to sleep in the afternoon, I have seen from the emoncms graph that power2 is negative, i.e. the inverter is a load to the system. I'm speaking of 7-10 minutes in the morning and 30 minutes in the afternoon at about 20W of consumption. Is it a real consumption or again a false reading of the Arduino?

Any opinion would be really appreciated.

Thanks,

Alberto

Robert Wall's picture

Re: Discrepancies and accuracy of the feeds

"I see the power2 input counting a power between -1W and +1W, all the time. I believe this is due to some noise and/or to the ADC resolution. I have filtered this in the emonGLCD sketch adding up to the genkhw variable positive values only. From you point of view, am I correct?"

The + / - 1 W is almost certainly noise in the system. If you are concerned, I would suggest you ignore values less than 1 W, (or some other small number). Ideally, you need to do this in the emonTx.

"the inverter is a load to the system. I'm speaking of 7-10 minutes in the morning and 30 minutes in the afternoon at about 20W of consumption. Is it a real consumption or again a false reading of the Arduino?"

This is real consumption. The inverter requires some power in order to synchronise itself to the mains and to operate, and until it has started to operate, it cannot use its own power. It is likely to be drawing a very small power all night, but possibly it is too small for you to measure (I would expect a few Watts, around 5 would be a reasonable number. If the inverter specification lists "quiescent consumption" or words like that, this is what we are talking about).

"What I didn't expect was such an error between the GLCD and the cms as they're relying on the same data."

True, but the data is processed differently, and the two systems are not necessarily working over the same time interval even. The data reaches emonCMS via the base and the Internet, every 5 s or so. EmonCMS then draws a straight line between the last data item it received and the current one, and integrates the area under the graph. If a reading goes astray and does not reach emonCMS, it is ignored. EmonGLCD adds up the power every 200 ms and assumes it remains constant during that time. That is my understanding. You can of course look at the source code for both systems and see how the totals are arrived at. 

Have you checked the calibration of both your voltage and current sensors? If not, this could explain the difference between the inverter and the GLCD values.

alpav's picture

Re: Discrepancies and accuracy of the feeds

I have checked the inverter documentation and it actually does report a quiescent consumption < 2W. So the "noise" I see at night on channel 2 might be either noise itself or this quiescent consumption as well (emonTx reads values like -1W mainly). The meter has measured about 1 kWh of consumption in 145 days. The bigger consumption of -20W measured when the inverter wakes up and when it is going to sleep might be due to the fact that the light is enough for the inverter to switch on but the current it's not. It is definitely not relevant but it is not actually considered as a consumption because the CT1 sensor doesn't reveal it.

There is something else I don't understand. The emonTx sends data every 5 seconds. The emonGLCD fast loop takes 200 milliseconds, so the sketch divides by 5 (1 second / 5 = 200 milliseconds) the received power and sums it to the genhwh variable at every loop. It does it until it gets a new value, so I assume that the received value is considered constant until the next one. In ideal conditions it means 5 seconds between two consecutive values. I suppose that this is the same that the emoncms does, as it gets an input every 5 seconds. So why am I facing such a difference between the power to kwh/day values of the emonGLCD and the one of the emoncms?

I'm writing from Italy. Here the solar production is incentivized by the Government with a fixed rate per kWh. There is a specific power meter dedicated to this. As there is a discrepancy between what the inverter shows and what the meter does (guess what? The meter counts less than the inverter!) I'm willing to have a good reading through the emon. I've measured the error between the inverter and the power meter around 1.8% on a 5 months period (inverter +1.8% than power meter) and now this discrepancy between the nodes of the system is confusing me!

I'm using the emonTx_CT123_Voltage sketch with the FP AD 3515 Euro plug AC-AC voltage adaptor. I then changed the voltage calibration coefficient to 212.658. I didn't change anything about the current sensor (leaving the 111.1 value I've found) nor the calibration shift (1.7). I don't know how to change it.

ct1.voltageTX(212.658, 1.7);

ct1.currentTX(1, 111.1);

ct2.voltageTX(212.658, 1.7);

ct2.currentTX(2, 111.1);

Today's data:

  • Inverter: 3.4 kWh
  • GLCD: 2.8 kWh
  • cms: 3.1 kWh
  • Power meter: 3 kWh (the meter doesn't return any other digit)

Thanks,

Alberto

Robert Wall's picture

Re: Discrepancies and accuracy of the feeds

You need to calibrate the emonTx before you can know what is right and what is wrong. All electronic components have manufacturing tolerances, and these will add up. If you are lucky, some will go one way, some the other and they will cancel each other. If you are unlucky, all will add up to give a large error. The voltage error can be up to 2% from resistor tolerances, plus the voltage adapter tolerance, which I don't think we know (I would expect it to be within 5%, but it might be more). The current error should be a maximum of 1% from one resistor + 1% from the c.t itself.

To calibrate the emonTx, you need an accurate calibrated multi-meter and a resistive load - an electrical heater is fine. Check the accuracy of your meter on the ranges that you will be using - it is usually given as x% of reading + y digits, and you will want to know the actual possible error in each measurement.

First, you need to be able to see what the emonTx is measuring. Temporarily, put some print statements into the main loop to display the rms voltage, power factor and rms current for each CT (and optionally real and apparent powers).

Measure your system voltage and adjust the number 212.658 so that the emonTx gives the same voltage as your multi-meter. You might find this has to change by up to 5%, maybe more, to give you the correct result. Put the new number in both places - ct1 & ct2.
Next, you need to calibrate the phase shift for CT1 (that's the 1.7 in ct1.voltageTX). With only your heater connected (at the maximum setting), change this number so that ct1.powerFactor is as close to 1.00 as possible (and then ct1.realPower and ct1.apparentPower should be equal).  This is for the combination of  CT1 and your voltage adapter. Now temporarily put CT2 on the same cable as CT1 and do the same again, this time it is the 1.7 in ct2.voltageTX that you change. If either number changes much, check again the voltage and if necessary adjust each voltage calibration (the change this time should be very small). You might have different numbers in each - the important thing is both ct1.Vrms and ct2.Vrms must read the voltage that you are measuring.
Finally, you need to calibrate the current transformers. You need the biggest current with a heating load that your multi-meter can measure, then with only the heater(s) connected, measure the current and adjust the 111.1 in ct1.currentTX(1, 111.1) so that ct1.Irms reads the same as your multi-meter. Do the same for CT2, then put CT2 back on the inverter cable, and take out the print statements.

"So why am I facing such a difference between the power to kwh/day values of the emonGLCD and the one of the emoncms?"

We must ask Trystan about that. He knows the emonCMS software very well - I believe he wrote it. Are you using your own emonCMS running locally or emoncms.org?

TrystanLea's picture

Re: Discrepancies and accuracy of the feeds

Hello Alpav, Robert

Both emoncms and the emonglcd calculate kwh/d in the same way, time between readings x power / divided by joules in one kwh. The last time I checked they corresponded quite well +- 0.1 kWh I think.

However there can be large errors if for some reason either emoncms or the emonglcd does not recieve an update for a longer than usual time, especially if the power is changing between say a couple of 100W's and 3000W. I have had that happen a few times where the graph draws a straight line over a period of half an hour or so from 100 to 3000W which could lead to as much as a 1.5 kWh discrepancy.

Are you seeing a consistent error? with a few more days of data?

alpav's picture

Re: Discrepancies and accuracy of the feeds

Thanks for your replies, very appreciated. I did expect that both emoncms and emonGLCD calculate kwh/d in the same way, so I have to search for the reason somewhere else. I am monitoring the PV production mainly at the moment, so I don't expect to have "spikes" in the power production. Today I've got the following data:

  • Inverter: 5.9 kWh
  • emonGLCD: 5.2 kWh
  • emoncms: 6.61 kWh

This is strange. As Robert suggests, I did not calibrate the emonTx and this explains the discrepancies with the Inverter reading. I didn't understand from the site that the calibration was required, but it makes more than sense... I will do it as soon as possible, but the only resistive load I can get at the moment is 650W and I'm not sure it is enough.

What I can't explain, however, is the big difference between emonGLCD and emoncms readings. Today the emoncms is logging much more than the inverter itself! As per the Trystan's suggestion, I have to suppose that the emonGLCD and emoncms (the emonBase) get a different number of data from the emonTx. How can I detect this? I was thinking to add a counter and to increment it at every packet received from the emonTx. On both the base and the GLCD. I would expect the two number to be reasonably similar, even because the base and the GLCD are located in the same place. Do you think it worth?

Unfortunately today the emonGLCD is having a weird behavior. It has been running for 6 consecutive days without any apparent problem. Then it stuck the night between Saturday and Sunday. I then restarted it on Sunday morning, optimized the code of the emonGLCD in order to save some Flash (not so much, got close to 31k) and I loaded the new sketch. Tonight it stuck again, at 20.04 - more or less after 25 hours. I think I will contribute to the proper topic for this, but I've noticed that these stuck came after a "basefail" error. Tonight I had to restart the base as well (OKG W5200 running from 7 days) because despite the leds were blinking apparently the normal way, the time on the emonGLCD was not updated. After that reboot, I've noticed that in emoncms today's data were put back of one day (every data shifted back of one day) and it started logging from zero. Now all the data of my database have the wrong date (just because I rebooted the base?).

But I am going off topic...

Cheers,

Alberto

P.S. I am using the emoncms.org

Robert Wall's picture

Re: Discrepancies and accuracy of the feeds

Your 650 W load is usable, but a bigger load will be better. Use the 650 W load if that is all you have (no electric kettle?).

You can certainly try adding a counter. I think there is basic agreement that the nanode base and GLCD problems that have been receiving attention recently are at least partly due to lack of spare memory, so you might have to remove some code temporarily to include the counter.

 

Robert Wall's picture

Re: Discrepancies and accuracy of the feeds

I should have mentioned this earlier: emonTx accuracy test http://openenergymonitor.org/emon/emontx/accuracy

alpav's picture

Re: Discrepancies and accuracy of the feeds

No kettle, but I've just realized that I could use the electric iron: it is 2200W. Just a bit concerned about measuring the current with the multimeter. Thanks for the reference.

Robert Wall's picture

Re: Discrepancies and accuracy of the feeds

Check the meter manual!  Most high current ranges have a time limit - 20 s or something like that. Be quick and allow the meter to cool to ambient (many minutes!) before trying again.

alpav's picture

Re: Discrepancies and accuracy of the feeds

Hi Robert,

I've had the chance to work a little bit on the calibration of the system only today, following your directions.

I've built a piece of cable with separated live and neutral wires. I've clipped the two CT sensors on the live wire and I've started to follow your instructions. My setup was the modified cable, one electric iron (1800 W), one toaster (650 W) and a digital multimeter. I have modified the emonTx standard sketch to print the Vrms, the Irms, the real and apparent powers and the power factor.

First step: searching for the correct calibration factor for the voltage reading. I've found a value that gives me the same Vrms on both the channels. Looking at the multimeter, the calibration constant was giving me the same readings (+/- 0.5 V on average).

When I was satisfied of the result, I've connected the first load, the iron, searching for the constant to put the power factor as much close to 1 as possible. This has been quite tricky as I started incrementing the "original" 1.7 but then I found that the best value was 1.28 (as per http://openenergymonitor.org/emon/emontx/accuracy). A fine tuning moved it to 1.27. I had a power factor of 1 +/-0.0015 on both channels on average. This was confirmed with the second load, the toaster.

I then moved to the calibration for the current. Here I had to find two different values for the two channels as the CT2 seems to return lower values than CT1. I think this is the more accurate calibration I've done as I've found difficulties in measuring the current with the multimeter, but I think that the error is quite low anyway. The two constants I have found are giving the two same readings on the two different channels, even though it seems to give more accurate readings on the lower load (650 W) than on the higher (the electric iron).

As a final step, I went back to check the voltage reading. In the first step I did the calibration with no loads connected to the wire. Now I did check the voltage with the two loads (only one at time) in and I've found that the Vrms read on the multimeter was lower than the one measured! The difference went up to 3 V.

I then re-calibrated all the system (in the end the voltage constant only) measuring the voltage under load.

Big doubt: I don't know what is correct now! If I put the multimeter in a socket of my house I read an higher value than the value I read from the emonTx (more or less 1.5 V of difference). Was it better to leave the first value (calculated with no load) or this is more correct?

Another big doubt: I am not sure (but quite confident) about the Irms readings I get, but for sure the reading is more accurate with the 650 W load than the 1800 W one. If I have the doubt to have an error of 0.2 A on the first load, I guess to have a 0.6 A error on the second one. I didn't find any constant giving an accurate value on both cases, so I decided to prefer the first as I have an average consumption around 500-750W that I suppose will be read with more accuracy. But what will I have when the sun will be up shining again and I'll have up to 12 A flowing in my wires?

Maybe I am focusing on stupid questions: if so, please, tell me. Or maybe I did mistakes. I have a totally different background and I know that my first problem is my lack of knowledge. But I have a lot of willing to learn :)

I'm trying to find a way to have some reasonably accurate readings of consumption and production. My emonGLCD and the emoncms are giving me very different values and I'd like to understand why. But first of all I need to understand what value to rely on, as you told me.

Thanks in advance for your time and patience.

Regards,

Alberto

Robert Wall's picture

Re: Discrepancies and accuracy of the feeds

I think you are doing everything right.  You will read different voltages at different places, this is because the wires in your house, plugs, sockets all have some resistance so the voltage where the supply cable enters will be higher than at the furthest point - unless of course no current is flowing. When there is current flowing, the wires and connections will drop the voltage (and dissipate power and get warm!). That is absolutely normal. A 2.5 mm2 cable will drop 18 mV per Amp per metre (so carrying 10 A, the voltage at the end of 10 m of this cable will be 1.8 V less than at the start).

Why do you think you have an error in the current readings and the lower is more correct? You cannot be sure that either load is exactly what it claims to be, and even if it is, have you calculated what the current should be at the voltage you had when you measured the current. I don't think you can say the high load or the low load gives you the more accurate reading - you simply do not know which is wrong - it could be the load itself, your meter, the emonTx or most likely a combination of all 3.

One test you could do: pass the wire two or three times through your current transformer - that should exactly double or triple the reading on the emonTx, and the 650 W load would read as 1300 W or 1950 W.

The current transformer itself is not perfectly linear, but you will only see that effect at very high currents > 80 A or so. If the difference is within the accuracy specification of your meter, then I think the only way to get a better calibration is to compare kWh recorded by the GLCD against the electricity company's meter, and even then you will not know whether any difference is due to voltage error, current error or both - or the errors could of course be partly cancelling!

alpav's picture

Re: Discrepancies and accuracy of the feeds

Thanks for your prompt reply, I will be sleeping better tonight :)

I will monitor the data recorded by the GLCD with the two electricity company's meters for the next few days and I'll do further considerations. When I feel confident of my calibration I will go on exploring why the data are not matching with the emoncms, that is the point I have started from.

Thanks very much!

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.