Archived Forum

I've based this on git@github.com:openenergymonitor/emonhub.git

...which is the emonpi variant - and not the emonhub version currently used in v9.0 RC2. (for info)

Paul

I find in general that there's a bit of a dearth of info about which versions developers should be targeting changes at.  Something along the lines of a contributing section on the web site, or just a note in the README for each git tree would be fine (e.g. "use this for x, but don't develop on it unless y, please use the z tree instead") etc.

OpenVPN (to pick a project at random) has a couple of wiki pages like this:

https://community.openvpn.net/openvpn/wiki/Contributing

https://community.openvpn.net/openvpn/wiki/DeveloperDocumentation

Possibly the info is in the forums somewhere, but it's difficult to find, and difficult to know if a particular post is up to date - "You are in a maze of twisty little passages, all alike".

I'll be happy to get these changes ready to merge against whichever tree I should be targeting, so long as I know that there is an appetite to see these features, and I know which tree that is...

I'd certainly like to see this and Paul would know (and has said) which tree to target

The main 'emonhub' version was/is being developed by Paul Burnell, and is which is being used in v9.0 RC - which is the current emoncms release.

However, whilst developing the emonpi, Trystan found it necessary to make changes to emonhub, in effect creating a tweaked version solely for the emonpi. The two are not interchangeable.

so long as I know that there is an appetite to see these features,

I think that there is always an appetite for improvement and further development - see what Chaveiro has achieved in v9, and I would assume that development branch of Paul's git repo would be the appropriate start point, however, let's hopefully get some feedback from Paul Burnell first.

Paul

@Tim - I had previously looked into the python3 jump for emonHub and the info I found suggested development for both 2.7 and 3.4 simultaneously was difficult and it was easier to "convert" 2.7 code to 3.4 than the other way round. so being more familiar with 2.7 and the fact emonHub was already 2.7 I decided this wasn't a bridge I had to cross just yet. I thought I would postpone the change until after the major changes as I had enough to tackle etc etc. I did wonder at the time how do I continue the simultaneous dev for 2.7 and 3.4 after making the switch but again, it wasn't something I had to figure out if sticking with 2.7 for now.

I am keen to adopt 3.4 changes but need to evaluate the full (ongoing) picture to make an informed decision so please feel free to show me you proposed changes and/or advise/steer me in the correct direction.

The "keep alive" sounds good and was something I had wanted to explore for another part of emonHub, How does this effect the server? would emoncms.org see more or less traffic? when it comes to traffic emoncms.org has to be the biggest concern, unless using a pay per byte data service to a private server etc. How does this scale? does it offer a saving across the board or just frequent posting? Correct me if I'm wrong (which isn't unlikely) but would there be a point where this becomes efficient and before that occasional posting better as it is? if so it doesn't mean it cannot be useful, just that some settings or logic applied to manage it.

I'm currently tied up on something that is consuming all my time, which is why I've not been about over the last couple of weeks, please don't think I'm not interested, I would like to pick up the emonHub dev again if the interest is there.

I will post some thoughts and a status of emonhub or maybe add some notes to the repo to help explain the branches, versions and proposed future of emonhub.

Paul 

Hi Paul,

I've not got too much knowledge of Python, but I was chasing a bug which I thought may have been down to a particular library version on python2.7 (it wasn't in the end), the changes just involved using the 2to3 utility from python3.4, along with some manual changes to work around a couple of language differences (which didn't seem too offensive).

In my testing, the resultant code worked under both 2.7 and 3.4.

Re the keepalive support, the upshot should be a saving in bandwidth (should be around a half, but I can check) and CPU on both the client end and the server end due to TCP connection setup and tear down.

With the default Apache config on Debian/Ubuntu etc. the default keep-alive time is 5 seconds, so to be effective the server config would need to be changed to a keep-alive period which is longer than the (typical) emonhub reporting interval.  I used 70 seconds in my testing.

When using Apache+mod_php, and longer keep-alives I'd expect the RAM usage to increase (as there will be more apache processes sitting around, and each has a full copy of PHP in-process), so it would probably be better to switch to PHP-FPM.

Having said that PHP-FPM is probably the best thing to use for running PHP these days with or without keep-alive, this would have the effect of making the number of copies of PHP in RAM just dependent on the number of concurrent PHP code executions (instead of being reliant on the number of HTTP connections - including those which are serving static content which are nothing to do with PHP).

I'm not going to pretend to understand the wizardry of php or the inner workings of a server so I will need to take your word and/or wait to hear from T&G on this, or maybe Chaveiro could comment as I have very little involvement on the emoncms side of things, but certainly if emoncms was "keep alive" ready it could be viable.

I am planning to split the current emoncms http reporter into 2 reporters to help users report to other endpoints without the complication of the emoncms stuff so there will be a basic http reporter and then a emoncmshttp reporter would inherit from that and add the more emoncms specific formatting etc, so it could be possible to add it as a optional setting to the generic http reporter and use as appropriate in the emoncmshttp reporter.

Can you link to some example code?

Regards the python3, I think the better route (for me) at the moment would be to continue in 2.7 and convert later if there is no immediate benefit, your "converted code" won't necessarily be portable or merge from the "converted emon-pi" to another repo anyhow, so may as well run the 2to3 utility once most of the new stuff is in perhaps.

But I would still like to see what you have if it's possible, a "before and after 2to3" for the emon-pi variant may give me an insight into what to aim for (or avoid) whilst moving closer to the conversion and any pointers/advice or assistance you may be able to offer would be very appreciated too.

Paul

Paul, would there be any mileage in creating a new git branch in emonhub which Tim could fork, and lay down his ideas, which you can both work on collaboratively?

Paul