Logfile - change of location

Submitted by Paul Reed on Sat, 02/01/2016 - 21:50

A number of recent changes has taken place to have greater control over emoncms logfiles, including being able to specify different levels of logging from within the emoncms settings.php, and now for emonbase users, the logfile location has been moved from the emoncms installation directory to /var/log/
The reasons for this are;

/var/log is the established location for logfiles, and where people expect them to be
Logrotate will then be able to manage the emoncms log, rotating, and limiting it's size
In low-write mode, /var/log is placed into tmpfs (RAM) and therefore emoncms logging can continue to run without disk-writes

Following a git pull,the implications for emonbase users are; (does not apply to emonPi - as emonPi already uses /var/log for it's logfile)

Existing Installations (not in low-write mode)
Emoncms is unable to create a logfile, only write to one, therefore it's necessary to create the logfile and set it's permissions in it's new location - just like what was done during the initial installation:

touch /var/log/emoncms.log
chmod 666 /var/log/emoncms.log

Emoncms will then commence logging at /var/log/emoncms.log and be managed by Logrotate.

Existing Installations in low-write mode (Raspberry Pi's)
Following a git update, it will be necessary to run an update script, to make a number of required changes:

cd /var/www/emoncms/scripts/logger/ && sudo chmod +x install.sh
sudo ./install.sh

A more detailed explanation of what changes the script makes can be found here.

New Installations
The Raspberry Pi low-write guide has been updated to reflect these changes.

Feedback as always appreciated.

Paul

Re: Logfile - change of location

Submitted by joyrider3774 on Sat, 02/01/2016 - 22:51.

since i'm using windows i have no /var/log so i'm just mentioning the previous value that was in settings.php to make the log save inside emoncms installation folder in case anyone not running linux needs it. It was $log_filename = dirname(__FILE__).'/' . 'emoncms.log';

i have not tried it yet, but i do wonder where the log file would be saved if i just wrote $log_filename = 'emoncms.log'; will check it out once i update.

also i'm guessing logrotate is to make sure the log file size doesn't explode and i'm wondering if logrotate is not used (since i'm on windows) would the log file keep growing ? currently mine is 85 MB but i can't remember if there is / was something inside the php files code that would delete the log file if it had grown over a certain size.

Re: Logfile - change of location

Submitted by Paul Reed on Sat, 02/01/2016 - 23:22.

Good point about the /var/log location in Windows, that needs to be covered within the Windows installation guide.

Regarding the logfile size - yes there is a 250mb limit set in Lib/Emonlogger.php but that is far too high when the logs are placed in tmpfs (low write), as only 30mb is allocated for the entire /var/log tmpfs directory to preserve RAM.

Logrotate really only applies to Raspberry Pi installations, its not enforced in code, and unless you are directing your logfiles to tmpfs, it will use whatever Log Management System is installed by your OS by default.

In Windows, I doubt that you will notice any changes, other than the change of logfile location.

Paul

Re: Logfile - change of location

Submitted by Jon on Sun, 03/01/2016 - 00:49.

Paul - I am not sure if this is related, but I get this error when running an emonPi Update.

Emoncms update started

Sat 2 Jan 18:43:16 CST 2016

git pull /var/www/emoncms
error: insufficient permission for adding an object to repository database .git/objects
fatal: failed to write object
fatal: unpack-objects failed

git pull /var/www/emoncms/Modules/nodes
Already up-to-date.

I tried rebooting the emonPi but got the same error.

Emoncms Version low-write 9.2 | 2015.12.11

emonPi formware 2.0

Re: Logfile - change of location

Submitted by Paul Reed on Sun, 03/01/2016 - 09:54.

Jon, emonpi's should obtain their updates via the 'stable' branch, which has not been merged with development (master) yet. Also, these changes will not affect the emonpi's as they already write their logs to /var/log so these changes are really aligning the 'emonbase' logging to how 'emonpi's' already operate. So no, yours will be a separate issue, probably needing it's own thread. Paul

Re: Logfile - change of location

Submitted by Jon on Sun, 03/01/2016 - 19:39.

Thank you Paul!

moved to new thread - emonPi Update error for emoncms (git pull /var/www/emoncms) http://openenergymonitor.org/emon/node/11926

Jon

Re: Logfile - change of location

Submitted by glyn.hudson on Mon, 04/01/2016 - 15:46.

Hi Paul,

Thanks a lot for packaging this up in a tidy way. I'm planning one more iteration on the new image before launch. I'll merge in these changes so the emonPi image setup will be inline with your RasPi Emoncms install notes.

Re: Logfile - change of location

Submitted by glyn.hudson on Mon, 04/01/2016 - 18:23.

Just tried with the new logfile, after running ./install.sh something seems to have gone wrong, my log file consists of:

Have I missed something?

Re: Logfile - change of location

Submitted by Paul Reed on Mon, 04/01/2016 - 18:47.

That's strange, it must be something unique to the emonpi configuration...

Following a reboot on a new emonbase installation running Jessie, I get;

...and again following a reboot of another pi emonbase running Wheezy:

Re: Logfile - change of location

Submitted by tgmaxx on Sat, 09/01/2016 - 14:41.

I am running a Wheezy configuration and noticed INFO msgs like the following are now appearing in my emoncms.log. Why am I getting these? Is there a possible problem?

2016-01-09 14:18:12.624|INFO|process_processlist.php|power_to_kwhd() feedid=61 last_kwh=0.0016374155555564 kwh_inc=4.5E-7 new_kwh=0.0016378655555564 last_slot=1452315600 current_slot=1452315600
2016-01-09 14:18:12.688|INFO|feed_model.php|insert_data() feedid=67 updatetime=1452349092 feedtime=1452349092 value=1 arg=
2016-01-09 14:19:12.689|INFO|feed_model.php|insert_data() feedid=60 updatetime=1452349152 feedtime=1452349152 value=0.018 arg=

Re: Logfile - change of location

Submitted by Paul Reed on Sat, 09/01/2016 - 14:57.

When did you last update?

I merged a fix a few days ago which sorts the issue.

If you update, you can then set your preferred logging level in settings.php (default is WARN).

Paul

Re: Logfile - change of location

Submitted by tgmaxx on Sat, 09/01/2016 - 15:22.

Hi Paul, I just updated about 12 hrs ago so I have your latest commit. But I still don't understand the meaning of the msgs.

Re: Logfile - change of location

Submitted by Paul Reed on Sat, 09/01/2016 - 15:40.

The messages are just what emoncms is doing under the hood. It's showing you what emoncms is doing with the data inputs, and calculating your daily values.
Im not at home now so can't check, but if you git pull, does it say that you are up to date - nothing to do?

Re: Logfile - change of location

Submitted by tgmaxx on Sat, 09/01/2016 - 15:57.

Paul, I rebooted and I am not getting the messages anymore (with $log_enabled = true, and $log_level = 2). On git pull, I get "Already up-to-date." Everything appears to be OK for now.

Archived Forum