Archived Forum

Please check you have correct values for those parameters (examples between brackets) :

- protocole (http//)

- domain (domain.tld)

- path (/emoncms)

Also check your remote APIKEY.

Hi Jérôme,

  Yes, these are correct, I've checked loads of times and tried more then one domain. I was using a sub domain, but created a new .com domain to test with emoncms folder.

I tried using RPI helper but get the same response for domain that works and one that doesn't  see below

privatedomain.com which doesn't work returns

"success":true,"message":"Field updated","remotesend":false}

emoncms.org which works returns

{"success":true,"message":"Field updated","remotesend":false}

Thanks

Peter

I can send private mail with above tests and also domain, login etc. I'm interested in this as it happened me before and I ended up building new SD card which worked, but didn't shed any light  on issue.

What do you get when entering this URL in your browser ?

http://domain.tld/your_path/time/local.json?apikey=your_remote_apikey

(You don't need to add apikey if you're already connected to emonCMS within your browser.)

You should have something like this:

t11,31,00

(This it the time, obviously.)

This is the test that is used by local emonCMS to check remote emonCMS is working.

Hi Jérôme,

Works fine for emoncms.org but not for any of the others - blank. Note I have used the same database for all other domains so in essence they are the same except for domain name.

Thanks

Peter

P.S. I'll build a completely new install at remote end with new domain and test. Completed no luck.

You mean you have a remote install located at http://domain.tld/emoncms of whatever, you can log into it, but you don't get the "time" page ?

I wonder what goes wrong, here. Do you mind sharing the URL (at least in private, although I don't swear I'll have the time, let alone the skills, to do something) ?

Hi Jérôme,

Sent login details to your contact page in your profile page. It would be great if you could test with one of your installs to see what you get.

Thanks again

Peter

Hello PeterN, sorry about this my fault, looks like I missed adding the time module in the latest release. I will put it back in now.

Ok its back in there now http://github.com/emoncms/emoncms

Great.

Peter, I erased your message with the login parameters.

Hi Trystan,

Thanks, downloaded latest version emoncms from https://github.com/emoncms/emoncms and uploaded to new domain. Worked first time great thanks for fix. Note between the ready to go SD image and great instructions its now possible to get a new system working in minutes  excellent works guys.

Cheers

Peter

P.S. thanks Jérôme for support testing earlier

Hi,

Thanks for help above, one more issue I hope someone can help me with.

One thing I noticed in new releases emoncms & emoncmspi is that the remote send does not come back after a reboot for 3G dongle (3G was like this in previous releases also). I had a crude work around in previous release but it's no longer working. I have attached code in hope that someone will spot my mistake, basically a cron job is scheduled to run php script every minute to kick start remote send for 3G - needed to start TCP communication after reboot.

Process to kick start remotesend: note this works when I use http:// below in raspberry pi browser but not when I run attached php script.

update raspberrypi with wrong parameters

http://localhost/emoncms/raspberrypi/set.json?field={"remotedomain":"xxxxxland.co","remoteprotocol":"http://","remotepath":"/emoncms","remoteapikey":"c98cdf6b42360bf333850c30a52d56ca","remotesend":"1","sendtimeinterval":"0"}');

update raspberrypi with correct parameters

http://localhost/emoncms/raspberrypi/set.json?field={"remotedomain":"xxxxxland.com","remoteprotocol":"http://","remotepath":"/emoncms","remoteapikey":"c98cdf6b42360bf333850c30a52d56ca","remotesend":"1","sendtimeinterval":"0"}');

Thanks

Peter

p.s. one important observation is that if remote send is working on ethernet lan and 3G dongle is connected, when I unplug lan connection the remotesend works fine on 3G. My thoughts are that something happens on reboot up before 3G dongle is connected (typically take about 30 seconds), any ideas could we delay some network activities for 30 seconds or till 3G is up.

Thanks PeterN, good to hear!

Peter, I'm not sure I understand what you mean. Are you saying that after a reboot of the Pi, the data stops being sent remotely ?

I had such an issue when I daemonized the RFM2Pi Gateway : it started before the network was up. I added a retry every few seconds in case the network is down.

Hi Jérôme,

Yes when 3G dongle is used it stops sending remotely after reboot.

My thoughts exactly, I can see the 3G dongle takes about 30-40 seconds to connect to network. Problem is I'm not sure what code to use to rectify or where to start. Either delay some code on reboot (e,g, raspberrypi_run.php - not sure this is used anymore, no cronjob) or like you suggest retry after time period (??). I can send dmesg logs for different each scenario if that would help point to issue.

Thanks

Peter

Yes, raspberry_run.php is still in use. I meant the py script to be an alternative, but it's still a bit young, let's say, and currently, users chose between the two of them. I'm not sure where we're going, here. I believe that by design (language, use of classes) the python script is easier to modify and extend. While the two are in use, it means double work each time we want to add something.

Actually, I was wrong in my earlier message. The issue I had at startup was that the script would start before the database is initialized, and it couldn't get the settings, so it would just stop.

When the network is done, the sample is just lost, that's all, and the script goes on.

I don't see why the script would stop when the network is unreachable.

And I don't understand your workaround.

Did you try with both gateway scripts (php and py) ?

Hope I didn't confuse things.  The default script is running ok after reboot (latest ready to go image), this is case for both lan connection  and 3G. However the remote send is not sending data when 3G connection is used (only). I traced in wireshark and TCP is not sent at all.

For what ever reason the process I  use to get working again is:

In raspberrypi dashboard:

1. input wrong remote domain  and save (no auth)
2. input correct remote domain and save (auth succ)

The above process can be done manually or using a php script and cron job to automate. Not ideal solution. I'm not sure why this is happening and think it may be because 3G takes 40 seconds to connect. Note 3G does come up and I can ping google etc. it just for some reason remote send doesn't start resending automatically.

Question hope you can answer at what stage in boot process does remote send start and could this be delayed.

Thanks

Peter

What happens if you shut down the network (unplug 3G dongle) while remotesend is active ?

Does remote sending work again when you replug the dongle ?

Yes

Test:

1. Start position - raspberry pi 3G dongle connection only and posting to remote domain ok
2. unplugged 3G dongle - stopped posting as expected
3. plugged in 3G dongle - took about 30 - 40 seconds for 3G dongle led to light (solid) and show its connected. Posting to remote domain ok a few seconds later.

FYI Here original post on subject. http://openenergymonitor.org/emon/node/1903

I'm sorry if you already answered that, but what if you

- unplug the 3G dongle (no cable connected either, otherwise it's cheating)

- stop (kill) the script

- launch the script

- wait a bit

Is the script still alive ?

- plug the dongle

Does the script send data ?

Hi Jérôme,

Apologies can you please guide me. Assuming the script is RFM12 to Pi, how can I view this with LXTerminal and start/ stop same.

Thanks

Peter

Hmmm... I'm not sure what your exact setup is. If you're using latest master, you can choose between the php and the python script, and you can have it launched at startup using an init script. In this case, you have instructions in the readme and the command looks like this for the python script (out of memory):

$ sudo /etc/init.d/rfm2pigateway start

$ sudo /etc/init.d/rfm2pigateway stop

$ sudo /etc/init.d/rfm2pigateway status

You can check whether a script is running with ps:

$ ps ax | grep rfm2

Assuming no script is running, you can launch one of them manually and see what happens:

$ python /var/www/emoncms/Modules/raspberrypi/rfm2pigateway.py

or

$ php /var/www/emoncms/Modules/raspberrypi/raspberry_run.php

If you choose the python script, when launched manually, the logging info appears in the terminal. (The default config if using the init script is to log into /var/log/rfm2pigateway/)

A lot to take in. I have downloaded latest raspberrypi and emoncms to make it easier. Seems php is working ok. Not sure command to stop php ? (normally use # in crontab) and I will run test for php first.

root@raspberrypi:~# ps ax | grep rfm2

21322 pts/0    S+     0:00 grep rfm2

root@raspberrypi:~# python /var/www/emoncms/Modules/raspberrypi/rfm2pigateway.py

Traceback (most recent call last):
  File "/var/www/emoncms/Modules/raspberrypi/rfm2pigateway.py", line 23, in <module>
    import serial
ImportError: No module named serial

 root@raspberrypi:/var/www/emoncms/Modules/raspberrypi# php raspberrypi_run.php

Remote upload enabled - details correct
DATA RX: 20 71 9
DATA RX: 11 0 0 0 0 2 0 244 11 85 0

I don't remember being offered option to use python but will look at readme etc.It sounds interesting and if it could be started 60 seconds after boot may be potential solution.

Thanks

Peter

> Not sure command to stop php ? (normally use # in crontab)

crontab line starts it regularly in case of crash. Removing it won't kill the running process. To do so, first remove line in crontab to avoid the process to be relaunched, then kill runnign process :

$ ps ax | grep raspberry

identify the line that corresponds to the process, get its ID number, then

$ kill process_ID (you may need to sudo that)

BTW, if you're using latest version, you could get rid of the crontab line and use the daemon instead. See readme.

> ImportError: No module named serial

Yes, there are two dependencies you need for it so run. See readme.

> root@raspberrypi:/var/www/emoncms/Modules/raspberrypi# php raspberrypi_run.php

> Remote upload enabled - details correct
> DATA RX: 20 71 9
> DATA RX: 11 0 0 0 0 2 0 244 11 85 0

It seems to be working. Does it stop here or does it go on and you only pasted the first three lines ?

In other words, what I suggested is to :

- Disconnect from network to reproduce startup condition before it is up.

- Launch the php script manually.

- ps ax | grep raspberry_run to get its process ID (PID)

- Wait a bit and see if it the process is still up : again ps ax | grep raspberry_run 

- If it is dead, we need to figure out why. If not

- Plug dongle to get network

- See what happens.

If nothing happens and the process is not dead, we need to figure out why.

Hi Jérôme,

This is great stuff thanks a mil. I need more time to digest above information. Below is result of test you suggested and also interesting observation if order is changed.

Cheers

Peter

Test 1:

- Disconnect from network to reproduce startup condition before it is up.

ok - Raspberry Pi powered up, network disconnected, RFM12 Pi stopped

- Launch the php script manually.

ok

- ps ax | grep to get its process ID (PID)

ok

- Wait a bit and see if it the process is still up : again ps ax | grep raspberry_run 

Warning failed to open stream remote domain, data RX RFM12 ok. Dashboard shows RFM12 to pi running (green).

- If it is dead, we need to figure out why. If not

above seem ok as internet is not connected but RFM12 data still being received

- Plug dongle to get network

Dongle connects ok, can browse internet from Raspberry Pi

- See what happens.

Not updating remote domain

If nothing happens and the process is not dead, we need to figure out why.

Observation, if I change order of events slightly, power up network disconnected, then stop RFM12 to Pi, then connect 3G Dongle and wait till connected to internet, then run raspberrypi_run.php. This time it posts data to remote every time.

OK now I think I know. I'm sorry I had you go through all these steps, I could have found earlier if I had taken a little bit of time to dig into the php script, but I was not totally concentrated on this issue. Anyway, it's good for the future if it made you learn stuff...

In the php script, the remote server reachability is tested twice :

- once at startup:

  $sent_to_remote = false;
  if ($remoteprotocol && $remotedomain && $remotepath && $remoteapikey)
  {
    $result = file_get_contents($remoteprotocol.$remotedomain.$remotepath."/time/local.json?apikey=".$remoteapikey);

    if (isset($result[0]) && $result[0]=='t') {echo "Remote upload enabled - details correct \n"; $sent_to_remote = true; }

- once every iteration *if settings were changed*:

        if ($settings->remotedomain !=$remotedomain || $settings->remoteapikey !=$remoteapikey || $settings->remotepath !=$remotepath || $settings->remoteprotocol !=$remoteprotocol)
        {
          $result = file_get_contents($remoteprotocol.$remotedomain.$remotepath."/time/local.json?apikey=".$remoteapikey);
          if ($result[0]=='t') {echo "Remote upload enabled - details correct \n"; $sent_to_remote = true; }
        }

This explains your workaround : you modify the settings, thus trigger a refresh

You could add a timer to check regularly for the availability of the remote server:

          every 30 seconds

          $result = file_get_contents($remoteprotocol.$remotedomain.$remotepath."/time/local.json?apikey=".$remoteapikey);
          if ($result[0]=='t') {echo "Remote upload enabled - details correct \n"; $sent_to_remote = true; }
 

Or perhaps even remove all those tests and rely on the remotesend setting set in the GUI. This is what the python script does and I believe it should be the way to go.

You should probably file an issue in github about this, with a link to this. If the fix is as simple as I think, I could even provide it, who knows...

Can you please test it works if you replace all occurrences of sent_to_remote by $settings->remotesend ?

I believe the python script does not have the same problem. You may want to have a try as well.

Hi Jérôme,

Works, really appreciated your time and effort.  I've tested your fix twice and it worked both times - raspberry pi posting to remote domain after reboot with 3G dongle. I'll raise issue as you suggested. I'd like to test python script also over next few days - keep you posted.

Thanks again

Peter

I can't seem to get emoncms on my Raspberry Pi to  send the data  to emoncms.org. I'm pretty sure I've entered all the data correctly  (http:// selection, emoncms.org, /emoncms, api r/w key from emoncms.org). Hit save, says it's saved but displays "Incorrect server details".

I saw somewhere something about remotesend in the raspberrypi table. If I change the value manually in the DB, I get a green Successful message. But when I change a value in emoncms to a bad value and change it back (click save after each), it goes back to unsuccessful and the remotesend field is again equal to 0, not 1.

Is there something special I'm missing to get the Raspberry Pi to forward data to emoncms.org. The Pi is behind a router firewall that the ISP setup in the house. I'm just curious how to get this to work (for backup purposes really) as the emoncms installation on the Pi works fine for now. Though I'm thinking about reinstalling it so it's running off of a HDD instead of the SD card.

Thanks in advance.

What do you get when entering this URL in your browser ?

http://emoncms.org/your_path/time/local.json?apikey=your_remote_apikey

I believe it should work thanks to Trystan's fix.