Archived Forum

Out of curiosity, what if you try to only unplug the Out of curiosity, what if you try to only unplug the RFM12Pi daughter board ? Which revision is it by the way ?

Do you use the "send time to emonGLCD" feature (disabled by default) ?

Revision 1 of the RFM2Pi has serial port issues, especially when using serial port both ways, for instance with the "send time" feature. This bug does not exist on revision 2.

I can't tell about the load. I think the python gateway consumes about nothing, but I don't know about the php gateway.

Thank you for your reply.

Yes i'm using an emonLCD, and receiving from it too (temperature from another room).

And Yes it's a revision 2 i think, here's the line on the order :
1 x RFM12Pi v2 433Mhz + SD Card Bundle
(RFM12Pi V2 433Mhz: RFM12Pi V2 Pre-Assembled Raspberry Pi Expansion..., emoncms Pre-Loaded Raspberry Pi 4GB SD Card: Emoncms Pre-Loaded Raspberry Pi 4GB SD card )

I didn't tried to unplug the RFM12Pi while the power is on on the raspberry, i don't really like the idea of doing this. The original SDCard has already became unusable after an direct unplug without "init 0"... so trying to unplug the daughter card does not seems safe. Plus, the raspberry is in its box, and not really funny to "unbox".

What was the serial port issues exaclty ? Was it the same behavior ?

You all may be aware that there seems to be somekind of bug in the raspberry wlan stack requiring a reboot once per day or so. This is not the problem on this case, i guess ?

Took me some time but I've got a stable Wi-Fi connection with my Pi, now. I think:

http://openenergymonitor.org/emon/node/2941

Revision 2 does not have the serial port bug. It was a bug in the arduino lib that manages the COM port. A different Atmel is used in the rev2, and it does not have this bug. I think the difference is it that has built-in hardware serial communication support, whereas serial communication is software on the chip in rev1. Basically, gibberish was received when sending in the meantime. But you have rev2, anyway.

Thank you for your messages.

Sorry but I'm not talking about any wifi issue... i'm plugged on the switch with a wire...

( And i have other raspberry pi using wifi and don't need a reboot per day (fortunately !). )

My problem is the stability of an RFM12Pi.

Jérôme's point was there was a serial port bug. I'm not familiar with the RPi, but does the RFM12 module not communicate serially? If so, that bug might well be relevant.

I assume you don't have the Jeelib problem that affected the emonGLCD a year or so ago?

The bug I'm referring to only affects RFM12Pi v1, so it has to be something else.

OK, thanks Jérôme.

Hello!

I've similar problems with my Raspberry setup (Pi with 256MB and RFM12Pi V1 module). Sometimes it simply stops logging data, I can only solve this by restarting the Raspberry. I don't use a emonGLCD, so this bug should't appear due to my understanding.

Any ideas for this, is there a newer firmware version that I can install on the RFM12Pi module?

The bug on the V1 appears if the RF is used to send time every now and then to synchronize emonGLCD, whether the GLCD is actually present or not. In practice, this time feature is disabled by default so if you don't have any GLCD, I doubt you'd have enabled it.

The log would give a little more information: is there any data received at all ? Is it corrupted ? Did the RF change freq band for some reason ?

No, I didn't enable the GLCD option.

Could you please tell me how to activate the logfile and to check if the RF band has changed?

Hello!

I think, I've found the logfile. Is it /var/log/rfm2pigateway/rfm2pigateway.log ?

I've attached the last lines of this logfile.

Something went wrong today at 9:30 AM, it looks like there are a lot of measurement values that should be transmitted at this time, this is the latest entry in the file, then transmission to local and emoncms.org stops.

Please, don't use closed formats such as .docx. To share a log, either embed it in your message, or send a plain text file (.txt).

I see several things:

2013-10-24 09:30:36,969 DEBUG URL string: http://localhost/emoncms/input/bulk.json?apikey=APIKEY&data=[[0,9,-210,4,0,22863,-86,15,1,24510,118,0,1,23290]]
2013-10-24 09:30:36,971 INFO Sending to localhost/emoncms
2013-10-24 09:30:37,251 WARNING Send failure

I don't have the explanation, here. Can you try the same URL directly from your browser ? Do you even use the local emoncms instance ?

Then, the last line:

2013-10-24 09:30:44,967 INFO Sending to emoncms.org/

I don't know why it fails.

Perhaps the URL line is a bit long. Can you try to set a shorter sending period ? 5 or 10 seconds instead of 30, for instance.

Anyway, it should timeout at some point, and/or raise an error. I don't get it.

Edit: Long URL shouldn't be an issue in itself.

http://boutell.com/newfaq/misc/urllength.html

http://stackoverflow.com/questions/417142/what-is-the-maximum-length-of-...

Hello
I have the same problem.

I'm using the latest image for SD:

oem_gateway22oct2013.img.zip

But the rfm12pi crashes randomly, not getting more data from my two emontx (id 9 and 10) while the raspberrypi continues to operate, in fact accessing via ssh I can do a reset.

How do I enable logging, to understand what's going on?

See troubleshooting, here: http://emoncms.org/site/docs/raspberrypigateway

I don't use this card, but if I trust the help page,

/root/oem_gateway/oemgateway.py --config-file /boot/oemgateway.conf

will let the gateway run outputing to the terminal.

I think you can specify

/root/oem_gateway/oemgateway.py --config-file /boot/oemgateway.conf --logfile /root/oem_gateway/oemgateway.log

to let it log in this file.

Hello

Now will not start even with the shutdown-r now, I have to turn off the power.

I've tried these instructions, using putty:

ipe-rw

cd oem_gateway

nano oemgateway.log

ctrl+o+enter

ctrl+x+enter

ps-ef | grep python +enter

I came back to this:

root 1122 1 2 09:02? 00:00:09 python / root / oem_gateway / oemgateway.py - config-file / boot / oemgateway.conf
root 1247 1231 0 09:09 pts / 1 00:00:00 grep python

I wrote then:

kill -9 1122

+ enter, and then

python /root/oem_gateway/oemgateway.py --config-file /boot/oemgateway.conf --logfile /root/oem_gateway/oemgateway.log

+ enter.

At this point I left the green cursor and nothing happens.

If I insert:

ctrl + c, and then

python /root/oem_gateway/oemgateway.py --config-file /boot/oemgateway.conf

+ enter

Same thing, blinking cursor, the script does not restart.

There may be a problem on rfm12pi?

Is there any content in /root/oem_gateway/oemgateway.log ?

I removed the power supply and it is restarted.

I did the above procedure, entered:

python /root/oem_gateway/oemgateway.py --config-file /boot/oemgateway.conf --logfile /root/oem_gateway/oemgateway.log

These are the last lines of the log (two hours),  then has received nothing.

2013-10-29 20:26:50,540 INFO Sending to fabiusmontana.dynalias.com/emoncms
2013-10-29 20:26:53,444 DEBUG Send ok
2013-10-29 20:26:53,732 INFO Serial RX: > 0s
2013-10-29 20:26:53,937 INFO Serial RX:  -> 4 b
2013-10-29 20:26:54,146 INFO Serial RX:  9 16 2 0 0 0 0 220 0
2013-10-29 20:26:54,149 DEBUG Node: 9
2013-10-29 20:26:54,151 DEBUG Values: [528, 0, 0, 220]
2013-10-29 20:26:54,153 DEBUG Server fabiusmontana.dynalias.com/emoncms -> send data: [9, 528, 0, 0, 220]
2013-10-29 20:26:54,155 DEBUG Data string: &node=9&json={1:528,2:0,3:0,4:220}
2013-10-29 20:26:54,157 DEBUG URL string: http://fabiusmontana.dynalias.com/emoncms/input/post.json?apikey=****I deleted*********$
2013-10-29 20:26:54,160 INFO Sending to fabiusmontana.dynalias.com/emoncms

Even the values ​​received are strange, at 5 pm (it was almost dark), marked about 1000w PV production.
I rebooted, and it has fallen to zero, the correct value.
The strange thing is that the value went down slowly, so as to seem normal.

OK, it seems stuck while sending to your server.

Can you please try to add a timeout to urllib2.urlopen() ?

In oemgatewaybuffer.py, replace

result = urllib2.urlopen(url_string)

with

result = urllib2.urlopen(url_string, timeout=60)

and see if this still occurs.

Ok, I try.

thanks

Hello Jerome. Last night I made the change you suggested, and so far not 'locked. It remains the problem of misalignment of the data collected.

This morning the inverter was producing 900w, but emoncms marked zero. I rebooted, and you will be updated on the correct value.

It could be a atmega328's buffer that does not empty?

Hi.

Good if it doesn't lock anymore. The timeout has been added in master branch but please don't upgrade now, as latest revision is broken. Just keep your local fix until you upgrade and don't bother.

Regarding your data, it could be that the Pi does not send fast enough (?). This could be seen in the logs. Basically, if you send every 3 seconds, and each "Sending to server" takes more than that, then you end up being late.

We're working on a better sending method to avoid these issues.

Until then, you may want to

- Check this is actually the problem. The logs will help.

- Lower sampling frequency (depending on what you see in the logs, perhaps a period of 10 seconds or more instead of 3 or 5)

> The strange thing is that the value went down slowly, so as to seem normal.

I think this is just a side effect of sampling: one sample up, another down and later, hence a steep that is not perfectly vertical.

I ask forgiveness for my English, I use google to translate.

Ok.

Where do I set the sample rate?

sendtimeinterval in oemgateway.conf?

>>I think this is just a side effect of sampling: one sample up, another down and later, hence a steep that is not perfectly vertical.

This may not be a side effect of sampling.
The first image is taken directly from the inverter logs.
The second image from emoncms (local).
At 14:30 there was a gap of 250w. I restarted the raspberrypi, and the data were realigned.
At 15:15 there was again a difference of 120w.

I'm not a programmer, I'm just a geek, but I think that the couple raspberrypi/rmf12pi, used as a gateway, should only allow the passage to data arriving from emontx, in real time, without manipulation or storage, as happens with the emonbase/nanoderf. Or mark with date and time each incoming packet from emontx when rfm12pi receives them.

Hi.

By sample rate, I mean the frequency at which the emonTX emits. Basically, if emonTX transmits every 3 seconds, this generates, in the gateway, a call to urlopen() every 3 seconds. And if that call lasts more than 3 seconds, then the delay keeps accumulating. Just an hypothesis.

When arriving to the gateway, the data is stored with a timestamp. Then reemitted. There is an issue we are working on that affects latest versions, but I think the gateway version you are using  should work correctly. Maybe older versions were broken as well. Anyway, Trystan is working on an API modification on emoncms side, and this should be sorted out.

Hello Jerome.

I do not think the problem is in emoncms.
Even this morning, with inverter already on and in the production of 500W, the gateway continued to send the last data of last night.
By controlling the input in my account emoncms, these were updated every 3 seconds.
I restarted the raspberry, and inputs were realigned.

OK so my guess is a 3 second period is too much for the gateway to handle.

Would you mind trying to increase the period to 5, ideally 10 seconds and see if it works ? Again, if the bottleneck is here, it should be obvious from the logs.

I'm not saying the problem is in emoncms. I modified the gateway to group http requests and avoid what I think is your problem (see latest commits), but my new implementation does not work with emoncms, which Trystan is trying to solve on emoncms side by modifying the API.

Hello Jerome. The part of the code to be changed in the sketch is this?

emontx_sleep (time in seconds)

Yes, I suppose.

Hello Jerome.

I modified the sketch as you suggested, and now the gateway works well, without any delays.
I brought the sleep of my two emontx to 10 seconds:

emontx_sleep (10);

I also set a threshold for the detection of flashes, because the Italian electricity meter has 2 LEDs, the second shows the reactive power, which I false counts.
Also introduced a threshold for minimum power, which eliminates the problem of false indications at night, because remained the last reading (40W) before turning off the inverter .

// The interrupt routine - runs each time a falling edge of a pulse is detected
void onPulse()
{
lastTime = pulseTime;        //used to measure time between pulses.
pulseTime = micros();

if ((pulseTime - lastTime)>100000000)      //threshold for minimum power
{
emontx.power = 0;
}

else if ((pulseTime - lastTime)>410000)     //minimum threshold between 2 flashes(8.5Kwp = 0.42 second between 2 flashes)
{
pulseCount++;                                                      //pulseCounter             
emontx.power = int((3600000000.0 / (pulseTime - lastTime))/ppwh);  //Calculate power
}

}

OK, thanks for the feedback. This confirms what I suspected: bottleneck in the gateway due to long urlopen() calls.

We're on it already: https://github.com/Jerome-github/oem_gateway/issues/6#issuecomment-27209920