Archived Forum

I still have this problem. I have tried a lot of things suggested for other problems on this forum, but no luck. What I've checked:

• redis is running on EmonHub, the ps output is:

1   108  2639     1  20   0  51936 25616 SyS_ep Ssl  ?         53:59 /usr/bin/redis-server /etc/redis/redis.conf

• "extension=redis.so” is in /etc/php5/apache2/conf.d/20-redis.ini
• "extension=redis.so” is in /etc/php5/cli/conf.d/20-redis.ini

• Emoncms version: master:8.2.8, bufferedwrite:0.0.1  on the PI

• Emoncms version: 8.4.0 without LOG4PHP on the shared host

• data does post to EmonHub as in instantaneous value, but does not post to the database, it also updates the time.

• data from the EmonTx (ussing the default node 10) is passed along to the shared host where it can be processed normally

• data posted to EmonHub using the API ...… /emoncms/input/post.json?node=30&json=... also shows up as an instantaneous value, updates the receipt time, but is not passed on to the EmonCMS

• MySQL on the EmonHub seems to work as it takes new feed processing and dashboard definitions

As a newbie I don't know what else to check. Something must be mussed up somewhere. It seems like the input is being processed and then rejected or the database write fails. Any suggestion will be appreciated.

Another clue is the /var/log/error.log contains entries like:

[Sat Feb 28 19:22:16 2015] [error] [client 192.168.2.8] File does not exist: /var/www/currentsetting.htm

The client IP address is the address of the ssh client that I am using.

I don't know if that helps??

I'm a little unsure what to suggest as the information supplied is a little "muddled" so first I will try and help you with understanding the roles and relationship between emonhub and emoncms so you can better describe/understand what is or isn't happening.

emonhub is like a data funnel, it has a in memory buffering feature but no storage or databasing,

emoncms is the "database" and "web server" that the data is funnelled into.

so "redis is running on EmonHub" isn't quite right, but I guess you mean that you have confirmed redis is running, redis is used by emoncms not emonhub. Likewise "MySQL on the EmonHub seems to work as it takes new feed processing and dashboard definitions" I assume you mean emoncms not emonhub as emonhub has no access to mySQL and it's emoncms that has dashboards.

"data posted to EmonHub using the API ...… /emoncms/input/post.json?node=30&json=... also shows up as an instantaneous value, updates the receipt time, but is not passed on to the EmonCMS" it's emoncms that has the api's, when you use this method you bypass emonhub altogether.

"data does post to EmonHub as in instantaneous value, but does not post to the database, it also updates the time." Unless you are looking at the emonhub.logs the instantaneous values must again be in emoncms even though they are not being stored.

These last 2 facts plus the fact your remote emoncms is working suggest emonhub is functioning ok and point towards a emoncms issue. I strongly suspect a permissions issue for the data files.

What are you seeing in the emonhub log if you change the "loglevel" setting to DEBUG in emonhub.conf and check the log entries at /var/log/emonhub/emonhub.log?

Also what is returned by

sudo ls -la ~/data

Paul

I guess I am a bit confused between the physical boxes and the software functions. I am still learning. Thank you for clarifying my misunderstanding.

the emonhub.log has lines like:
2015-03-01 13:13:00,829 WARNING Send failure: wanted 'ok' but got
2015-03-01 13:13:00,932 INFO Sending: http://MY_SHARED_HOST/emoncms/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1425233560.55,10,9,63,703,0,11603,0],[1425233571.63,10,9,62,701,0,11617,0]]&sentat=1425233580
2015-03-01 13:13:01,237 WARNING Couldn't send to server, URLError: [Errno -5] No address associated with hostname
2015-03-01 13:13:01,240 WARNING Send failure: wanted 'ok' but got
2015-03-01 13:13:01,343 INFO Sending: http://MY_SHARED_HOST/emoncms/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1425233560.55,10,9,63,703,0,11603,0],[1425233571.63,10,9,62,701,0,11617,0]]&sentat=1425233581
2015-03-01 13:13:01,655 WARNING Couldn't send to server, URLError: [Errno -5] No address associated with hostname
2015-03-01 13:13:01,658 WARNING Send failure: wanted 'ok' but got
2015-03-01 13:13:01,761 INFO Sending: http://MY_SHARED_HOST/emoncms/input/bulk.json?apikey=E-M-O-N-C-M-S-A-P-I-K-E-Y&data=[[1425233560.55,10,9,63,703,0,11603,0],[1425233571.63,10,9,62,701,0,11617,0]]&sentat=1425233581

I'm not sure what this is saying. MY_SHARED_HOST is updating just fine. http://localhost.emoncms is the one that is not updating.

from /boot/emonhub.conf:

#######################################################################
#######################        Reporters        #######################
#######################################################################
[reporters]
        
# This reporter sends data to emonCMS
[[emonCMS]]
    Type = EmonHubEmoncmsReporter
    [[[init_settings]]]
    [[[runtimesettings]]]        
        url = http://localhost/emoncms
        apikey = MY_API_KEY

[[emonCMS2]]
    Type = EmonHubEmoncmsReporter
    [[[init_settings]]]
    [[[runtimesettings]]]
        url = http://MY_SHARED_HOST/emoncms
        apikey = MY_API_KEY

The ls -la ~/data yields:

/home/pi/data:
total 23
drwxrwxrwx 6 pi       pi        1024 Feb 11 07:28 .
drwxr-xr-x 4 pi       pi        4096 Feb 22 14:14 ..
-rw-r--r-- 1 root     root        20 Mar  1 10:17 fake-hwclock.data
-rw-r--r-- 1 root     root         0 Dec 31  1969 feedrunlock
drwx-w--w- 2 pi       pi       12288 Jul 24  2014 lost+found
drwx------ 5 mysql    mysql     1024 Dec 31  1969 mysql
drwxrwxr-x 2 www-data www-data  1024 Feb 26 11:15 phpfina
drwxrwxr-x 2 www-data www-data  1024 Jul 24  2014 phptimeseries

I see that on the version shipped on an SD card the permissions were different, with those chages made, the ls -la ~/data yields:

total 23
drwxrwxrwx 6 pi       pi     1024 Feb 11 07:28 .
drwxr-xr-x 4 pi       pi     4096 Mar  1 10:30 ..
-rw-r--r-- 1 root     root     20 Mar  1 13:17 fake-hwclock.data
-rw-r--r-- 1 root     root      0 Dec 31  1969 feedrunlock
drwx-w--w- 2 pi       pi    12288 Jul 24  2014 lost+found
drwx------ 5 mysql    mysql  1024 Dec 31  1969 mysql
drwxr-xr-x 2 www-data root   1024 Feb 26 11:15 phpfina
drwxr-xr-x 2 www-data root   1024 Jul 24  2014 phptimeseries

The EmonCMS database on the Raspberry Pi still doesn't update.

I'm not sure what this is saying. MY_SHARED_HOST is updating just fine. http://localhost.emoncms is the one that is not updating.

I'm assuming you have obscured the actual url of MY_SHARED_HOST, are you sure you have obscured them the right way round? the short log you have attached only shows one target so I can't compare or comment, But name aside those "sent" log entries look correct and the response is clearly not "ok" and it gives a error message about the hostname. The 3 messages there are the same data resent so I'm unsure how you can be seeing it "update",

A longer log may reveal more, leaving the urls un-obscurred so they can be checked may reveal something (PM me a log if you are concerned about publicly advertising your urls)

Putting a ' # ' in front of the reporters type will disable that reporter, to aide diagnosis you can focus one or the other only eg    #Type = EmonHubEmoncmsReporter  

Do you have any other name resolving or redirecting etc anywhere on the server/network?

Rather than "localhost" have you tried the ip of that Pi?

Paul

A longer log may reveal more, leaving the urls un-obscurred so they can be checked may reveal something (PM me a log if you are concerned about publicly advertising your urls)

Yes it does. I'll send that to you directly. What I noticed is that yes the messges to localhost/emoncms are accepted all the time, but the ones to my shared host are rejected 20-30 times before being accepted. Because they bulk load all of the datapoints are preserved. I had noticed some lag in updates form time to time and this would explain that.

Do you have any other name resolving or redirecting etc anywhere on the server/network?

This is an interesting and open ended question.

Sometimes DHCP hasn't work from the Raspberry Pi and I have put 8.8.8.8 into the /etc/resolv.conf to solve the problem temporarily. That is currently not true for this Raspberry Pi, but in looking at the log it almost makes sence that DHCP is failing from time to time (most of the time?) and preventing the lookup of my shared host IP address.

I've had problems in accessing my shared host to post data via the emoncms api using the Python httplib library. However the python urllib and urllib2 libraries seem to work. I suspect the problem is that there is a hard coded port number 80 in the former library. I thought I saw a :80 on one of the error messages. Shared hosts using the same IP address won't work with forced port numbers. I haven't played around enough with this yet to be deffinitive, but I don't think this is part of the bigger problem. I am doing this python work on a Mac, and although it has a Unix heart, it's not Unix. The problem could be dealing with the quirkiness of redirection from the shared host.

Rather than "localhost" have you tried the ip of that Pi?

Using either one seems to work just fine... as far as the communication part goes.

I havn't studied the log line by line but at a glance it certainly looks like the local install is fine and the issue is with resolving the shared-host name to an IP.

Try using the current ip address of the shared host instead of the domain name to see if the packets are accepted then, that obviously isn't a permenat solution but it will help rule out "forced port numbers" and other library issues etc. Also try using the direct "sub.domain" url eg mydomainref.theddnsprovider.com if using a DDNS provider or mydomainref.sharedhostingprovider.com.

Your points above and the log do suggest the problem maybe external to the local pi "enroute" to the shared hosting. "The problem could be dealing with the quirkiness of redirection from the shared host" do you mean "by" "to" the shared host? if not where is the shared host redirecting to?

Paul