Archived Forum

Same here.

Paul
Self hosted on Raspberry Pi, v9.0RC 

i managed to recalculate my kwh feed by using and first modifying the power_to_kwh_manual.php script from the usefullscripts repository. you have to remove all references to the log also in the proces\lib folder replace the logging with just an echo so you do see the output.

My kwh feed is now correct yet the my electric app for some reason still shows negative numbers. I already cleared cache and tried in 2 browsers.

Also that script only fixes my kwh feeds, it does not fix my kdh/d feeds

edit:

it seems recalculating has no effect as soon as i post data again it shows the same effect as in my earlier screenshot. And my electric keeps showing negative numbers so the recalculating script which worked in 2014 does not seem to work that good for now. it just goes back to negative but on a later timestamp (after i had recalculated the feed). Also my timezone is set to europe / brussels also in php.ini

edit 2: i've now set my timezone in apache httpd conf as well like so :

SetEnv TZ Europe/Brussels

this was not set in httpd.conf before. i'm not certain if this fixed it or if it got autofixed somehow but now my kwh feed seems to be on track again. The only problem i'm having seems to be getting rid of that dip. If i recalc using the recalc script i for some reason keep having the same dip in my graph data. my powerfeed is PHPFIWA don't know if that matters (i changed the source engine to this) the target engine is still PHPFINA

edit 3: the kwh feed above seems to be fixed now with me. the recalc script only deleted the target feed if the source feed was phpfina i changed that line to phpfiwa and now the kwd feed in above screenshot is fixed.

However i have multiple power & power to kwh feeds in total 5 of each 2 of them seem to be fixed now also when posting new data, the 3 others are not and i'm clueless as to why some feeds are still not working while others are all of a sudden 

edit 4: it seems i was wrong, looking at the raw data i can see a dip each time after i recalculated the values using the script. The only reason it seemed it was showing correct data was because it had caught up with the values it had lost all my kwh feed still show the dip after recalculating. So nothing is fixed with me not even after recalculating

I can't even get the scripts to run!!

If I run power_to_kwh.php (in usefulscripts), I get:

PHP Fatal error:  Call to undefined function readline() in /home/pi/usefulscripts/process/power_to_kwh.php on line 12

If I run power_to_kwh_manual.php (again in usefulscripts) I get:

PHP Fatal error:  require(): Failed opening required 'Modules/log/EmonLogger.php'
 (include_path='.:/usr/share/php:/usr/share/pear') in /home/pi/usefulscripts/process/power_to_kwh_manual.php on line 11

If I add that line, and re-run the script, I get:

PHP Parse error:  syntax error, unexpected '=' in /home/pi/usefulscripts/process/power_to_kwh_manual.php on line 11

Paul

you need to run the other script power_to_kwh_manual.php and edit the php script by removing the require log thingie as well als all $log-> entries (replace them with echo statements) you need to do this for all the files in the process directory including the lib folder. then the  power_to_kwh_manual.php will work, i also changed the check for phpfina to phpfiwa where it would delete the files of the target feed and had set source engine to phpfiwa since well that's the case with me. then this script will work.

the other script i could not get working either i managed to fix the readline thing by adding this function but there are other problems in that script it's easier and faster (to edit) power_to_kwh_manual.php  only thing you have to do is remove the log references at least that was the case with me since that logging thing doesn't seem to exist anymore 

function readline($prompt = null){
    if($prompt){
        echo $prompt;
    }
    $fp = fopen("php://stdin","r");
    $line = rtrim(fgets($fp, 1024));
    return $line;
}

but as far as i can tell the recalculating of the feed with or without deleting the target feed (done by the script) does not seem to solve the problem, it fixes the feed up until you recalculated but new data just have a dip again

oh don't forget to change the variables to reflect your feeds and installation and be prepared to wait a long time, the more data you have the longer recalculation takes. i've only been using emoncms for like 2 weeks and it took about one minute to recalculate each feed and this was on a intel nuc with an i5 processor it will probably take a lot longer on a raspberry pi, possibly hour(s) depending on how much data you have

I've decoded the phpfiwa feed to see if there is something in there which could influence the script and subsequent phpfina feed, and all looks well.

1am 25/10/2015 GMT = 1445734800 UNIX time

1445734710 46
1445734720 46
1445734730 47
1445734740 46
1445734750 47
1445734760 46
1445734770 46
1445734780 47
1445734790 46
1445734800 47
1445734810 47
1445734820 46
1445734830 NAN
1445734840 47
1445734850 46
1445734860 46
1445734870 46
1445734880 47
1445734890 48
1445734900 47

the problem is not the script, it recalculates everything fine, the problem however is that if you start posting data again the power to KWH feed (phpfina) feed gets negative values:

this is from an export to csv of the power to kwh feed it  1445784985 =  time zone: ‎25‎/‎10‎/‎2015‎ ‎15‎:‎56‎:‎25 GMT+1:00 this is the time i last recalculated the power to kwh feed using the script as you can see negative values appear in it right after the recalculations when i start posting data again

1445784970,2.53
1445784975,2.53
1445784980,2.53
1445784985,2.53
1445784990,2.31
1445784995,2.10
1445785000,1.88
1445785005,1.67
1445785010,1.45
1445785015,1.24
1445785020,1.02
1445785025,0.81
1445785030,0.59
1445785035,0.38
1445785040,0.16
1445785045,-0.06
1445785050,-0.27
1445785055,-0.49
1445785060,-0.70
1445785065,-0.92
1445785070,-1.13
1445785075,-1.35
1445785080,-1.56
1445785085,-1.78
1445785090,-2.00
1445785095,-2.21
1445785100,-2.43
1445785105,-2.64
1445785110,-2.86
1445785115,-3.07
1445785120,-3.29

it goes all the way up to this

1445785565,-6.31
1445785570,-6.31
1445785575,-6.31
1445785580,-6.31
1445785585,-6.31
1445785590,-6.31
1445785595,-6.31
1445785600,-6.31
1445785605,-6.31
1445785610,-6.30
1445785615,-6.30
1445785620,-6.30
1445785625,-6.30
1445785630,-6.30
1445785635,-6.30
1445785640,-6.30
1445785645,-6.30
1445785650,-6.30
1445785655,-6.30
1445785660,-6.30
1445785665,-6.30
1445785670,-6.30
1445785675,-6.30
1445785680,-6.30
1445785685,-6.30
1445785690,-6.30
1445785695,-6.30
1445785700,-6.30
1445785705,-6.30
1445785710,-6.30

and then it starts going down again

if i stop sending data and recalculate using the script, the data is correct but as soon as i post new data it goes haywire again, i'll check immediatly to confirm it

edit: just recalculated using the script again at 18:56 GMT + 1 (europe brussels time) this is what i get now in the power feed after recalculation and posting a few data:

1445795465 = GMT: Sun, 25 Oct 2015 17:51:05 GMT my time zone is : ‎25‎/‎10‎/‎2015‎ ‎18‎:‎51‎:‎05 GMT+1:00 (so not sure which 1445795465  related to, i used http://www.epochconverter.com/)

1445795450,2.63
1445795455,2.63
1445795460,2.63
1445795465,2.00
1445795470,1.37
1445795475,0.74
1445795480,0.10
1445795485,-0.53
1445795490,-1.16
1445795495,-1.79
1445795500,-2.42
1445795505,-3.05
1445795510,-3.68
1445795515,-4.31
1445795520,-4.94
1445795525,-5.57
1445795530,-6.20
1445795535,-6.20
1445795540,-6.20
1445795545,-6.20
1445795550,-6.20
1445795555,-6.20
1445795560,-6.20
1445795565,-6.20
1445795570,-6.20
1445795575,-6.20
1445795580,-6.20
1445795585,-6.20
1445795590,-6.20
1445795595,-6.20
1445795600,-6.20

same problem occurs but the problem only seems to happen on a later datetime stamp (after i ran the script) so i think the script itselve works fine and calculates everything fine but as soon as i post new data i'm having the problem again. i tested this by disabling my program that sends data, let the script recalculate and look at raw data graph feed, all looks ok, i thin started the program again to submit data and then i get the huge (negative) dip

i noticed something, emoncms, according to timestamps in the log seems to be thinking it's hour earlier:

my logs are posted at 19:33 but emoncms displays 18:33 in the log entry. could this be the problem that emoncms somehow thinks it's running one hour behind real time ?

look at the last log entry and the date time in my system tray

My log time & system time are identical.

Is your Setup > My Account > Timezone set correctly?

Paul

yes i have it set to europe/brussels as i live in belgium

hmm it seems that's not the problem, the log writes utc time so the hour diffrence is correct (i looked at the sources)

WARNING do this at your own rish or make backups, i noticed this trick only fixed the kwh feed looking at real time data, but if i use this feed in a bargraph feed to display kwh/d or wh/d the bargraph does not seem to correctly display data. This might be a bug with the bargraph and daylight saving time or it might be a problem with the way i "fixed" my kwh feeds

i found a way to fix the kwh feeds:

this is what i did :

1) stop sending data to emoncms

2) use the script to recalculate the kwh feed based on a power feed

3) start sending data again (wait 1 - 2 minutes)

5) use visualisation Editrealtime and select the kwh feed. You'll notice the recalculation went perfect but you get a dip at the end of the graph now (after calculation) like in this screenshot:

6) zoom in enough at the end of the graph so you can see the dip, select the last datapoint at the peak, were data was still valid, copy its value to clipboard

7) select a datapoint from the dip (lowest point) edit it's value so by pasting the value from step 6

8) wait a few minutes and show the editrealtime feed again. You' should notice that the kwh feed is back on track even after posting data but that you still see the dip in graph  like this screenshot:

9) if you see the effect above after zooming in enough on the correct position stop sending data

10) use the script again to recalc the data

11) start sending data again, the feed is now fixed. There might however be a very tiny dip but not a huge one like before and this dip can be ignored

This is a workaround for the problems caused by daylight saving time but i do hope that there'll be a fix before the next daylight saving time. I don't want to do this every 6 months. At the moment it did not take long since i had not much data but the longer i'll be using emoncms the more data i'll have and the more time it would take to recalculate the feed. So a fix for this should really be made

to make the script work: i repalced the all reference to logging with echo statements and remove the require statement. also make sure that the target phpfina feed gets deleted before calculation by default the script only does this if your source feed is phpfina (which was not the case with me) just change that if check with whatever source engine you were using (for example phpfiwa)

this is another bug which happend due to daylight saving :

the kwhd feeds are somehow messed up as well at least i think it is cause the graph is showing oct 25th 2 times instead of oct 25th & oct 26th on the x asis. However if you hoover over the day it's displayed as 26th correctly so only the labels on the x axis are wrong

I don't know how to fix this but i'm going to convert my bargraphs which use kwhd feeds to kwh feeds with the delta option

edit:

using kwh feeds with delta option = 1 gives the problem on the x asis and for some reason the next day (26th is empty) so maybe my kwh feeds are not completly fixed or there are other problems due to daylight saving time:

if i disable this option to automatically adjust the time to the daylightsaving time the xaxis thing is fixed if i enable it again i see two times oct 25, how can this setting have an impact on what you see in the bargraphs ?? this means that the client where your viewing the emoncms website from has an impact on how the graphs are displayed

however the bargraph still does not show any data for oct 26 (today) as in the screenshot above when using my "fixed" kwh feed with delta = 1

the same problem with the option above happens in firefox also (if you restart it after changing the setting in windows) It hapens on a windows 10 as well as on a windows 7. I had a look at the flot javascript library and noticed all graphs are set to timezone browser, so it's using the client's timezone shouldn't this use the timezone from user account settings which is selected there instead of the timezone of computer visiting emoncms ? i noticed there is an option for datejs if timezone in the graph options is not set maybe that should be used to fix the x - axis problem with this setting not sure though it's just a guess

Can you show the input process list that writes to that feed, and the Admin/Server Information tab.

Also it only happen on phpfina? And are you using redis ?

Do you still have the logs close to 25/10/2015 01:59 to 01:00 when the clock reverted? Post a few minutes before and after.

this is my setup:

i have 4 smart plugs and i edited a program that can read the data from the plugs so it can now send the data to emoncms. I'm doing this using the input api like so : http://127.0.0.1:8083/emoncmsnew/input/post.json?node=3&apikey=myapikey&csv=0.38,44.20,4.35,225.65 for example using one node for each plug, each with 4 inputs: amp, watt, volt and kwh. This last input kwh is the elapsed kwh as calculated by the smart plugs, but i don't use this since it has a limit of 1000kwh and these are not the feeds i'm having problems with.

as far as i can remeber it is indeed the phpfina feed that has a problem although my kwh/d feed also had a problem on the day itselve.

also do note i don't see any negative values anymore since i had tried to fix my "calculated" kwh feed (not the kwh input) the calculated kwh feed does not show any errors anymore however the when displayed in a bargraph using delta = 1 they don't show info for today while my kwh/d feed still does (i had usuage today). so it could be the kwhd feeds have a problem as well (not really serieous since if the kwh feed  / bargraphs are completely fixed i can switch to those and remove the kwhd feeds).

my timezone is europe brussels and i had set this in apache, php.ini and on my user page

here are the screenshots of the data you requested. I'm not 100% certain if this is the information you need. if you need more info please let me know what. emoncms runs on a intel nuc (x86 hardware) on windows 7 with timezone in windows set to europe / brussels using wamp

http://www.willemssoft.be/misc/serverinfo.jpg

http://www.willemssoft.be/misc/process1.jpg

http://www.willemssoft.be/misc/process2.jpg

unfortunatly i had disabled logging so i have no logs anymore, the reason i disabled logs was because it was growing too big, on one day it had grown 180 mb so i decided to disable logging. maybe paul might still have logs ?

edit:

when i ran the script to recalculate the power to kwh feeds it was logging warnings about datetimestamps starting before feed creation time might this be an indicator ?

the recalculation of the power to kwh feed worked but as i soon as i posted new data i had negative values again in the feed. I fixed this using a method described above, by editing one of the negative values to the last known correct positive value, this somehow made the phpfina feed (power to kwh) go back on track. i then recalculated again to remove the negative values so i believe my power to kwh feed is now showing correctly but it's the bargraph that's acting up if you use it with (this/) power to kwh feed. (it shows no data on today or depending on a windows setting for the 2nd october 25th) also the initial problem started on the night daylight saving time happend, the shift in dates is because i had recalculated / tried to fix my kwh feed a few times

the double days thing can be fixed by chaning  a setting in windows (since the graphs use the timezone of the browser) i just don't understand why that setting has an impact, it's only a display bug i think for the x axis.

i don't think i'm using redis, (don't even know what that is) just using the input api the post data from an application that can read the data from the smart plugs : https://github.com/joyrider3774/plugctl )

Hi Chaviero.

My input process is:
Log to feed - Grid (Feed 13 - PHPFIWA)
Allow Positive
Log to feed - Power Imported (Feed 14 - PHPFIWA)
Power to kWh (Feed 41 - PHPFINA)
Reset to zero 

Redis is not enabled

Emoncms Timezone is set Europe/London (the same as my Raspberry Pi).
 

I've only experienced the problem on PHPFINA

I've attached my emoncms log (extract)
 

Paul

in this video (which i made for a github issue i created) you can see that the double day on the x axis is fixed if you change a setting dealing with daylight saving time: (skip to the end as the first part of the video shows a problem where all graphs are using the browser's timezone instead of the timezone from the user settings page) https://youtu.be/Go-R1315tpU

Thanks,

Paul can you post a detailed printscreen of your feed 41 graph from 23h to 3h of 25oct to better see the minutes/seconds when the value goes down an then start going up.

When you say you only had problem with phpfina is because you only have this engine saving kwh data, so cant confirm kwh data withother engines, right?

I was able to replicate on paper that strange values from the log, it seems last feed time is offset by -3600s only during the hour before the dst change.

Bad time value comes from feed_model.php get_timevalue method.
It seems related to the way input last time is saved on DB, in text format. The error must be from conversion with strtotime($row['time']).

Redis is caching that time in linux format so i would not expect the issue with redis. Can anyone confirm?

O dear. Il start with the usefulscripts issue.

Id be interested to know which systems show the readline error using the process script:

PHP Fatal error:  Call to undefined function readline() in /home/pi/usefulscripts/process/power_to_kwh.php on line 12

I tested that here on PHP 5.5.9 and it worked fine. Towards the end of the power_to_kwh.php script it forces a reload of the redis feed data for the feed:

   // force a reload of the feeds table
    if ($redis && $redis->exists("user:feeds:$userid")) {
        $redis->del("user:feeds:$userid");
        $redis->del("feed:lastvalue:$target");
    }

This should stop the error where the corrected data just starts negative from the point of last reprocessing.

A more manual step process would be:

1. stop emonhub, emoncms-nodes-service and feedwriter
2. run the processor script.
3. clear redis $ redis-cli "flushall"
4. start emonhub, emoncms-nodes-service and feedwriter

But it would be really great to get the usefulscripts/process/power_to_kwh.php script working consistently for everyone.

Hi trystan, I had that problem as well on my intel nuc running windows 7 64 bit, using wamp. Here you can see the server info from the administration page:http://www.willemssoft.be/misc/serverinfo.jpg I could fix the readline error by adding a function that reads from stdin, which i had found on stackoverflow. However this was not the only problem i had, at a certain point the script tries to read the interval from the feed and displays, you could then choose that existing interval or change it. The problem i had was that it displayed array instead of an interval number. I can give you the line nr as soon as im at work. Thats where i stopped and focussed on the manual script which i could get working. Also for some reason it had problems with require statement of the emonlogger, i did not fully look into this, i just replaced writing to log to echo statements. It could very well be i had placed the scripts in a wrong directory Cheers davy

edit: forgot to mention there is another bug in the script at least i believe so:

if you look at this section : https://github.com/emoncms/usefulscripts/blob/master/process/power_to_kw...

there's a missing $row = $result->fetch_array(); $row is still pointing to the source feed not the target feed with that call. unless the source feed has to be PHPFINA which is not the case with me it's PHPFIWA. If this check needs to check the target feed the fetch array statement is needed. (i'm guessing the target power to kwh feed needs to be phpfina but not the source feed)

the line number where i had gotten array displayed is this one https://github.com/emoncms/usefulscripts/blob/master/process/power_to_kw...

don't know why though

@Chaveiro.

Graph attached.
I do have another 'Power to kWh' PHPFINA feed (21) which records 'solar' power generation from feed 20. This feed did not drop and appears unaffected, however as it's a solar feed, there would be no increments during the affected time - as it was nightime = no power (could this be the reason why it's OK??)

Paul

@Trystan

Attached screenshot of my system, which is a Raspberry Pi v2 running Raspbian Wheezy with php v5.4.44. I've also attached a screenshot of the error.

Paul

EDIT.
running $ sudo apt-get update && sudo apt-get upgrade brings me to php v5.4.45 and the script still errors as above.
 

thanks @joyrider3774, I've fixed the power_to_kwh.php script with the stdin function, the fetch_array call and the interval print output where the source engine is phpfiwa. Let me know if it works ok now.

Still errors after git pull.

Paul

The location from emonlogger has changed to lib/emonlogger.php instead of modules/log/emonlogger.php Im also not certain if it is required to place the script in the root dir of emoncsms www dir after that change.

i'm also not able to test the script until i get home tonight, for some reason i can't reach my intel nuc from work today

Thanks Paul, could you try again now?

@Trystan

Git pulled latest and script ran OK, no errors and looking at the resultant kwh feed before restarting emonhub it looked good, showing a steady daily increase. However, I took the precaution of flushing redis, and as soon as I started emonhub, the kwh feed added 79kwh to the feed!

Tried it again, same result, so ran a third time, started emonhub and stopped it again after 15 seconds. Then using Visualisation > EditRealTime, I edited the +79kwh datapoints back to correct value, and restarted emonhub again.

This time, the resultant graph is OK! - got there in the end!!

Paul

the scripts are running fine with me also except for one windows related thing. The exec(chown) stuff does not work nor is it needed on windows machines, it spits out a warning that it can't find chown. However i think the script executed further on or at least i think so. i'm still in the process of recalculating everything and will test the bargraphs again afterwards to see if they still have problems

the bargraphs still display wrong even after recalculating. this picture is with default windows settings based on a kwh feed (that i just recalculated), as you can see it displays 2x oct 25th on the xaxis and no data for today

changing this windows setting (translated autoadjust time to daylight saving time): fixes the x axis display bug but does not fix the no data display on the current day:

however i believe this might all be related to to using "timezone: browser" on the graph settings (it isn't using timezone from user settings page from emoncms for the graphs in javascript) otherwise the windows setting change whould have no impact.  however i'm not sure if the no data on current day is related to this, but the double days on the x axis probably is (why would changing windows settings otherwise have an impact)

paul could you create a bargraph with same settings and see if this has the same effect with you ? if you don't mind testing

EDIT:

just downloaded https://github.com/mde/timezone-js and the timezone files as per the readme, i added date.js to the bargraph.php and changed timezone:browser to europe/london and europe/brussels for testing so that timezone-js is used in combination with the specified timezone (libflot has support for this) it does not fix the double day display bug nor the no data. so i guess i was wrong that timezone: browser is the culprit, it is not as far as i tested and windows settings still affect graphs (like the setting described above). setting another timezone like nort america as a graph option (instead of browser) only shifted the x axis days a bit (they were not aligned anymore with the graph). so using timezone.js does not seem to fix it sorry for misleading

i just changed the windows setting to fix the x axis thing for now and am still using kwhd graphs (instead of kwh with delta = 1) so i can see the usuage on the current day as well

The duplicated day on the graph is something wrong with the graph javascript.

It's an old bug of flot lib, see : https://code.google.com/p/flot/issues/detail?id=141

A quick test with 'useLocalTime: true' seem to work, but i'd like to do further tests to understand how it works.

adding 'useLocalTime: true'  and not deleting 'timezone:browser' has no impact with me. However removing 'timezone:browser' completely and not adding 'useLocalTime: true' fixes it with me. not adding 'timezone:browser' and only adding 'useLocalTime: true'  fixes it also but i don't see a diffrence with just leaving out timezone:browser.

i checked jquery.flot.time.min.js and if you set timezone:browser it returns new Date(ts) where ts is the timestamp if you do not specify timezone it's returning makeUtcWrapper(new Date(ts)) where ts is the timestamp. i placed 2 alert statements in the if statements to know which one it was using.

Did you omit timezone:browser as well or did you specify timezone:browser along with uselocaltime:true cause that last one did not fix it with me. ps i only tested with bargraphs, i'll have a quick look on realtime graphs to see if the hour offsets or not

leaving out timezone option altogheter makes it use utc time on a daily bargraph it fixes the x axis display however a realtime graph my hour is off by one since i'm utc + 1 it's probably using utc, it can also be seen in the if statements in that jqeury.flot.time.min.js file. there's one other option that i tried yesterday and that's using timezoneJS if you add the library download the timezone files, initialize the library and write the actual timezone as timezone option for the xaxis but that did not fix it either with me

edit: crap i probably have to apply the patch from the google code link first cause as far as i can see a grep statement on all libflot files for uselocaltime does not return any info. I'll try to apply the patch if it still works and see what it does

Yes i omit 'timezone:browser' so it's using makeUtcWrapper, right.

The server time cames always in utc format but my current local time is also utc, so i'm not sure if removing it gives fixes because of that.

Need to dig further. If you look at \flot\examples\axes-time-zones there is some helpfull information.

thats what i tried yesterday but i notice the tz directory contains diffrent files as when i was testing it yesterday so maybe it did not work because of that. It makes use of https://github.com/mde/timezone-js . i'll try again but this time with the tz directory from the examples

edit tried with date.js and tz files from examples directory and setted timezone to europe/brussels it does not fix the double days thing, however the windows setting has no effect anymore on the x axis labels so it's probably a bug in the flot lib somewhere.

maybe timezone-js can be used to lock the x axis time bar to the time zone that is used in the emoncms user settings page but without windows settings having an impact on it

i also noticed flot lib is not being developped anymore last commit was somewhere in 2014 and i could not find a fork that might have continued work (github won't display the fork graph)

Check EmonCMS 9.1 | 2015.10.30. See if its ok.

seems to be fixed with me and does not seem to have any impact on other graphs where realtime data is displayed. As far as i can tell, this fixes it

@tristan i noticed something today. While looking at my kwh feed i still noticed a small drop so today i recalculated the feed again and when i start sending data again the drop occurs again.

You said in an earlier post that this

  // force a reload of the feeds table
    if ($redis && $redis->exists("user:feeds:$userid")) {
        $redis->del("user:feeds:$userid");
        $redis->del("feed:lastvalue:$target");
    }

should stop the error where the corrected data just starts negative from the point of last reprocessing but acutally i'm not using redis so those lines of code are not executed with me.

Could this be why i still see a (very) small drop in de kwh feed data after recalculating ? I also tried fixing it manually again by editing one value of the drop to the last known correct value to counteract the drop. I had done this with all my previous kwh feeds as well so i'm not certain if the script fixed it or my manual messing about with node values actually fixed the drop since the code above is never executed with me. So maybe the script did not fix my drop after recalculating but me messing with the node values in editrealtime visulisation:

it's only a 0.05 drop though so i can safely ignore it but i was still wondering if the script actually fixed it or not and if it's normal that i still see a small dip while this normally should never happen with a power to kwh feed ?

Also i'm using 9.1 from 30.10.2015 but not sure if that matters

It shouldn't happen, and I found the same, but got around it as described here.

Paul

ah yes so also editing the data points, i'm guessing i'm seeing this small 0.05 because i had selected the wrong datapoint to counteract this behaviour i'll try to fix them manually again

If you rebuild your feed again, start emonhub and stop it again a few seconds later, you should only have a few datapoints to edit (that's what I found anyway!)

You may find that without rebuilding it again, you'll have 3 million datapoints to edit!

Paul

@tristan,

i think i figured out why the feeds keep dropping after recalculating. I had a problem with one my kwh feeds to day and had to recalculate again, and i had the drop problem again. Even after editing the nodes as paul suggested i still had problems with the feed, the editing of the values did not work with me.

the problem, i think, is that the value column of the target feed in mysql is never updated. If i add

$mysqli->query("update feeds set value= '".$kwh."' WHERE id='".$target."'"); at the end of the script (after the redis stuff) the drop never happens after i start sending data again. So my guess is the script, at the moment is incomplete because it does not update the value colum of the target feed in mysql. At least with me this seems to stop the drops from happening

edit: this is probably only needed if mysql is used to get the last kwh value for doing the increment calculation to write the new value. which is the case with me since i'm not using MQTT nor redis

Thanks @joyrider3774, I have added this line to the bottom of the script. latest commit info is here: https://github.com/emoncms/usefulscripts/commit/7a395d042517470adaa7f4dd...

 @trystanlea no problem. It might be this only fixes it if one isn't using redis. the power_to_kwh calculation requests the last value using the get_timevalue function and this function first checks if redis is available and if so takes the value from there if redis is not availible it takes the value from mysql since i was not using redis it got the value from mysql feeds table with me. So it fixes it in this circumstance but i'm not certain if it would fix it if someone would be using redis. Unless the deletion of the last value on redis (last part of the script) had already solved it. I could not test that since i'm not using redis