Archived Forum

 Hi Paulo,

you can post the code on the forum, or better, on GitHub if you have an account.

looking forward to see your code.

regards

amin

 Hi Paulo,

This sounds interesting. We have an OpenEnergyMonitor git hub: http://github.com/openenergymonitor. Feel free to post to this account. Let me know if I need to make a new repo in the emonTx firmware for you. 

All the best, 

Thanks glyn,

I'm new with github. i'll create an account today to see how it works.

I'm testing and resolving some bugs that i found yesterday, and i'm not finished the 3-phase calculations. Im stuck in a mathematical problem that our language barrier keeps me from explaining it to you.

i need to put a desk in front of my energy panel to test and debug the code...

i think in 2 more days i'll have a stable beta.

Regards,

Paulo

The code is not ready, not Fully commented, and not properly cleaned, but it´s working and it´s beta1.

Hey guys,

The base code (not the one i posted) was updated and was running rock solid form more then 1 month monitoring the 3 phases with a great accuracy, so my home server (a linux freeagent goflex) flash disk crashed and i lost all my emoncms history. I  then decided to continue with the development.

I´m now implementing the RFM code, eeprom config with remote adjustment of calibration values, pool period and RFM  ids.

Trying to implement incremental kw storage i ´m thinking in a remote calibration mode to help with the calibration.

Any ideas? My base ideia is to control all the calibration parameters remotely using the Gmonlcd and nanode. I'm thinking in a calibration mode that you see the data near real time, and imputing an predefined resistive load to it it automatically adjusts itself to the load.

Future TODO´s, will include gemonlcd menus to control the Emontx. that way i think will be possible to deploy and calibrate emontx and gemonlcd without the need of a computer for a full working solution.

Sorry if i´m going in the opposite direction of the "Keep it simple". i´m going towards  the "Keep it easy to maintain"...

This Looks interesting, it would be great if you could put the code in the git repository, this would make it easier for others to look at and post changes as well as a bit more explanation as to how it works. I've had a quick look at the code, it looks pretty involved! 

1st public release of the code is on Github at: https://github.com/pcunha-lab/emonTxFirmware/tree/master/emonTx_Interrupts

This code is stable and running for 1 week. without watchdog.

I´ve just adapted the GLCD code to use the new payload format and i´m working on the nanode firmware to be able to control the config variables by a remote pc using a webserver.

Soon i´ll put the other codes too.

Hi all, 

This new code is running rock solid for 3 months now. 

I am now moving forward to create a configurarion menu for glcd and a new firmware to emonbase to handle the new protocol. and other device types ( acting lile a proxy to a emoncmd or similar soluton)

Looks i'm moving into a not too simple solution, but a more "user friendly". i don't know if that's the objective of the project.

My goal this year is to implement more sensors that will integrate into the rfm network uning the protocol to comunicate via emonbase. That sensors will include an HVAC thermostat, a lamp dimmer switch, a wireless weather station, an aquarium controller  and some other ideas for my home...

will this project be a good place to discuss that projects? since i'll be using all the basic concept of emontx, glcd and base? and they will be the heart of the system?

Hi Pcunha,

i was just starting to write my own three phase monitoring code with nearly the same ideas like you when I found this post. This is just excellent! Very much the same what I wanted to do and ready to use. 

The only thing that puzzled me a bit is the very high prozessor loading in the ADC ISR. I had only very few us per cycle left and quite a lot of  jitter because some calculations seem to take even more time than available. This does not seem to harm accuracy as far as I can say, but is not really nice.

I found a very simple way to shorten the calculation by ~8-9us per interrupt:

- declare New and Previous in CurrentSampleS and VoltageSampleS as int (instead of float).

- calculate the .Filtered value as follows

  xxxSample.Filtered  = 0.996 * ( xxxSample.PreviousFiltered + (xxxSample.New - xxxSample.Previous) );

With this modification, (xxxSample.New-xxxSample.Previous) is calculated using integer math and this seems to make the difference.

Please try and comment.

Big Thanks again for your fantatstic code!!!

BR, Jörg.

PS: is there a way to change the optimization of the compiler from -Os for size to -O2 or -O3 for speed?

PPS: another 5 us per interrupt (and some memory space) can be saved by declaring ADC_V_BUFFER as int.   

I forgot to mention that I had to make one important change to get everything running:

-  set bit 0x40 in ADMUX to select 'AVcc with external cap at AREF pin'

This was missing in Pcunhas code, maybe he is using a different hardware?

In this context I have one question: Is there a decent (eagle?) schematic and board file for the emontx available somewhere?

Wow! I have just spent the afternoon studying your firmware, its really great work, very impressive! I will get back to you with some further thoughts soon

Trystan

There are some small 'bugs' in the code which should be removed first. I have taken Pcunhas code as a starting point, done some modifications and have tested the code with a 1+3 (1 wall transformer + 3CTs) sensor arrangement for real three phase measurement. Accuracy is quite good. Compared it to a professinal mains analyzer tool the power readings and accumulated energy were within +/-3%.This is much better than my 'official' energy meter, which is up to +10% off!

If I am allowed to, I would attach my code (or I can send it as PN if someone is interested). There is for sure room for improvements and the code still has some deficiencies.

Regards, Jörg.

 PS: forgot to mention that my version is more or less fixed for the abovementioned 1+3 arrangement because I removed a lot of the conditional defines from the original code . 

Thats clever use of the free-running mode of the ADC on the Arduino chip.  Got me thinking about other uses now.

 You're not the only ones! I also had to read a lot about ADC interrupts to understand it all (I'm not sure I do now :D)

But definitely inspiring for other projects!

This needs to be documented in the wiki in a decent way

There can be one small problem with using the free running ADC interupt like in Pcunhas code. The code in the ISR has to take less cycles than the ADC conversion (in this case 104us) otherwise the sampling will be jittering (or even delayed).

If the calculations can not be done in a shorter time then using a Timer interrupt (with periode >104us) starting the ADC is the better and more flexible solution. 

In going through the code I pulled out the core part of the interrupt code and the circular buffer used for delaying the voltage waveform so that I could test their workings in isolation, I've put them up here:

Interrupt Sampling

Circular Buffer, both Robert Wall's 3 phase code and Pcunha's code use a circular buffer to delay the waveform.

 Another part which was interesting to learn about was the ability to detect what kind of reset happened:

  if (MCUSR & 1) // Power-on Reset
  {
    // put POR handler here, if required
  }
  else if (MCUSR & 2) // External Reset
  {
    // put external reset handler here, if required
  }
  else if (MCUSR & 4) // Brown-Out Reset
  {
    // put BOR handler here, if required
  }
  else // Watchdog Reset
  {
    // put watchdog reset handler here, if required
  };

ADMUX = (0x02 & 0x07); // Set ADC reference to external VFREF and first defined port

Is this correct? According to the schematic of the emontx on solderpad (isn't  there a better one?) the emontx  uses AVcc as reference with external capacitor on the AVref pin. This was one of the problems I had with the original code. I am using now:

ADMUX = 0x40|(0x02 & 0x07); // Set ADC reference to external VFREF and first defined port

EDIT: Now I'm a bit puzzled.... do you really get correct ADC readings with your (== the original) code?

JBecker: Are you asking me? I havent tested the full sketch in action only studied it, I havent quite got as far as doing an analog read with the IntterruptSampler extract example, only as far as counting the number of samples that are taken in a second.. ? 

Trystan,

sorry, I did not manage to address you correctly...(is there a possibility to quote?)

It would have been strange if you got correct results. Can you confirm that my assumption about the adc reference is correct?

This really puzzled me with the original code. It is very well done but has some real nasty bugs and I cannot believe that it worked as posted. This is why I made some comments in this thread.

Trystan,

if (MCUSR & 1) // Power-on Reset   ...... 

»

this is also directly copied from the application note AVAR465, as is the complete basic code discussed in this thread including interrupts by free running adc. Paulo 'just' added the three-phase capability and the configuration calibration via RF. It is a bit surprising that nobody seems to know the AVR465 code although the hardware clearly follows this app note and the app note is sometimes even given as a reference.

Your quite right JBecker, ADMUX = 0x40 | (pin & 0x07) is correct, this is the same as the default setting set in Arduino-1.0/hardware/arduino/cores/wiring_analog.c, it is AVcc as reference with external capacitor on the AVref pin that should be selected.

Interesting stuff, I had never really tried to understand this low-level stuff, I've just read up about it will write a short blog post for people interested.

EDIT: here's the blog post: http://openenergymonitor.blogspot.co.uk/2012/08/low-level-adc-control-admux.html

 Hi eberybody, 

Im very glad that the code was useful to you!

First of all, how we quote text from others? is there a quote button? I'll  indent it then.

Let-me explain some points here. For some reasom i'm not receiving notifications from the forum, so i just watched the thread after a personal e-mail.

JBecker:

I found a very simple way to shorten the calculation by ~8-9us per interrupt:

- declare New and Previous in CurrentSampleS and VoltageSampleS as int (instead of float).

- calculate the .Filtered value as follows

  xxxSample.Filtered  = 0.996 * ( xxxSample.PreviousFiltered + (xxxSample.New - xxxSample.Previous) );

With this modification, (xxxSample.New-xxxSample.Previous) is calculated using integer math and this seems to make the difference.

PPS: another 5 us per interrupt (and some memory space) can be saved by declaring ADC_V_BUFFER as int.  

First you are right. Im pushing the avr at its edge. there are ways to optimize the code! i did'nt had enouth time.

your ideas are great! I'll implement that and see what it happens. How did you have measured the interrupt times? I've counted the lines in the assemble compiled version. Do you know if this is ok?

In the AVR document that i was inspired, all tha math is in integer. I did'nt had enouth brain processing power to understand that, so i've proceeded with part emontx code part avr doc code.

set bit 0x40 in ADMUX to select 'AVcc with external cap at AREF pin'

That's absolutely correct and i'll add this to the docs. Im using my own hardware, based on EmonTX, with some minor modifications. The analog reference i'm using is from an external source provided by a variable resistor. so i can precisely calibrate the exact middle of the 3.3v source. The 3.3v Source i'm using is from an very precise dc-dc converter.

I have done several tests to came with this hardware. I wanted a configurarion that allows-me to go in a infinite loop, reading the ADC and only get 512 as a result with no load on the CT, without any noise. And the results i achived are 9.999 512 values per 10.000 samples! Thats very good result. I 'll take some pictures of the circuit later.

If I am allowed to, I would attach my code (or I can send it as PN if someone is interested). There is for sure room for improvements and the code still has some deficiencies.

Please send it to me, so i can merge your modifications in the git! I Like condicional defines because, as a global project, everyone can use the same source code. 

There can be one small problem with using the free running ADC interupt like in Pcunhas code. The code in the ISR has to take less cycles than the ADC conversion (in this case 104us) otherwise the sampling will be jittering (or even delayed).

Yes. You shoud be careful with that. The way i found to debug the code and see if it is performing well is to set a volatile variable at the start of the interrupt and unset it at the end, If in the main loop, the variable is still set, i increment a second variable to count the interrupts that did'nt reached the end. It workd in a weird way. I 've also found that you can still debug the code from the serial line, but you need to use the fastest speed your uart can provide. Im using 115200 for debug.

Thystan Lea:

JBecker: Are you asking me? I havent tested the full sketch in action only studied it, I havent quite got as far as doing an analog read with the IntterruptSampler extract example, only as far as counting the number of samples that are taken in a second.. ? 

Have you read my read-me? Im using a 20mhz crystal. The calculations of the number of samples per second are (from the manual):

For 16mhz Crystal: 16000000 / 128 / 13 = 9615 Samples Per Second

128 is the ACD prescaler

13 is the total number of cycles used by the adc circuit do give you an reading

16000000, well, the frequency...

Hi Paulo, very nice to hear from you!

To make it short, I have found three major 'problems' in your code:

- setting of the external reference as explained above (OR 0x40 to ADMUX), solved!

- you put the 'Voltage.PhaseShifted' value into the ADC buffer. This is not so nice as this is already shifted by the phaseshift value for the first CT. Depending on the configuration, this might not be optimal (at least not for the 1+3 configuration). I would store the filtered voltage sample instead. And do the phase shifting when calculating the currents and depending power values. This means some more calculations after taking the current samples which will again need more processing time (see next item)

- runtime of the code. I found that the original code (run @16MHz !) jitters (= ISR is not entered at fixed 104us intervals), because some of the calculation just take too much time. The calculation time depends on the individual values and is not constant. So, sometimes the code jitters more and sometimes less. I made some improvements which help, but even in my code processing time is 'at or over the edge'. I would prefer to do all calculations in integer (as it is in the original AVR465 code. And, I never use float as it 'costs'). This can give big improvements. Float math is easy to do but can have a severe speed (and code size!) penalty. And it does not give more accurate results as some people seem to think!

Debugging, measuring runtime of code:

I measure processing times by setting and resetting an unused digital output before and after the critical code section (or at the beginning and at the end of the ISR). Off course, you will need a scope for that.

That's it for now, hope we can find ways to improve the code so that it becomes even more useful.

Regards, Jörg.

PS: .......count the interrupts that did'nt reached the end.

This assumption is not correct. Every ISR will reach the end and clear the volatile before returning to main.

JBecker,

As i don't have a scope, you will need to help-me with that.

I'll begin studyng integer math to optimize the code. I Think it is possible.

I'll work on this weekend.

Paulo,

I think I will also have some time on saturday and maybe sunday. My Emontx is at the moment 'permanently installed' in my distribution, but I will put it back on my desktop for this. 

I have also made some tests already with conversion to integer calculations, but i had no time to finish it and then just wanted to have the system running.

Hi Jbecker,

Thank you for the explanation on the other topic. 

I didnt know where the 0.996 came from. i was trying to figure it out all the last weekend :)

I had made some minor improvements and corrections on the code that saved about 50 cycles per interrupt, with the same math code. and your suggestion. Its not much but it is something. (counted at avr studio)

The calculations are being done when the adc index is 0, but it shoud be when its (num_cts + num_voltage_Sensors), because we are geting one more voltage sample.

I had worked 2 days and i did'nt solved it. Can i leave that part to you? i think you got the 0.996 math better than me. If yes, i'll work on my idea of auto calibration routine.

As soon as im able to connect to my Github account again, i'll upload the code.

Thanks,

Hi Paulo,

I just found out that my SSD is defect. First I will have to setup my computer again. I have copied the last (?) sketch that I did before the PC was wrecked and will try to append it here.

This sketch hast just the voltage measurement implemented correctly. The current measurement is not ok yet. But I think you can get the idea.

Cheers, Jörg.

PS: it is not allowed to attach .ino files? And .pde is ok?