Move emoncms.org data to local install of emoncms

I have downloaded the latest (9.3) emoncms from GitHub, got it running with the required database settings and created a user account with the same username and password that I had on emoncms.org

I now want to migrate the data already on emoncms.org to my own server (Linux). I have downloaded backup.php from "usefulscripts" but I can't get it to work. I'm not entirely clear on the settings in that script - for example it asks for the WRITE API key for emoncms.org - why isn't that the READ key?

Initially the script wouldn't run at all because it couldn't find process_settings.php - I tweaked the 'require' line to be:

require $_SERVER['DOCUMENT_ROOT']."/process_settings.php";

...but now its complaining "Class 'Redis' not found"

Its also not clear what $local_emoncms_location should be set to if I want the data to go into my own emoncms installation - should that be a folder path or a URL?

Can anyone help please? Nowhere does it say where "usefulscripts" should be installed. I don't want to lose the data we've already collected.

Chris.

ChrisValentine's picture

Re: Move emoncms.org data to local install of emoncms

I note that backup.php appears to ignore the value of $redis_enabled in the settings file - as I have it set to false.

Can anyone shed any light on this? I don't have any data on my new server - I specifically want to grab the historical data first.

Chris.

ChrisValentine's picture

Re: Move emoncms.org data to local install of emoncms

OK, looks like Redis is now enabled on our server but I'm getting other errors:

Uncaught exception 'RedisException' with message 'Redis server went away' in /data/web/greendata.kmi.open.ac.uk/www/backup/lib/metadata.php:22 Stack trace: #0 /data/web/greendata.kmi.open.ac.uk/www/backup/lib/metadata.php(22): Redis->hMset('feed:102381', Array) #1 /data/web/greendata.kmi.open.ac.uk/www/backup/backup.php(89): register_emoncms_feed(Object(mysqli), Object(Redis), Object(stdClass)) #2 {main} thrown in /data/web/greendata.kmi.open.ac.uk/www/backup/lib/metadata.php on line 22

Chris

Edit - duplicate posts deleted
Please don't make duplicate posts
Moderator - BT

ChrisValentine's picture

Re: Move emoncms.org data to local install of emoncms

Despite the error messages it does appear that some data has been imported. If I view the Feeds list on our installation, the Updated column just shows n/a and the Value column is all NULL, but the Size column numbers are quite close to what's stated on emoncms.org - does this mean I have the data now or only part of it?

ChrisValentine's picture

Re: Move emoncms.org data to local install of emoncms

...maybe not. If I click on any of the eye icons I get errors like:

Request error {"success":false,"message":"Error reading meta data feedid=102381"}

ChrisValentine's picture

Re: Move emoncms.org data to local install of emoncms

I found that the default Redis folders did not exist on the server (off /var/lib) so I've created them in a different place and changed settings.php to reflect that. If I empty the feeds table then run backup.php again I see:

Warning: Invalid argument supplied for foreach() in /data/web/greendata.kmi.open.ac.uk/www/backup/lib/inputs.php on line 8
11 Emoncms.org feeds found Create feed 102381
Fatal error: Uncaught exception 'RedisException' with message 'Redis server went away' in /data/web/greendata.kmi.open.ac.uk/www/backup/lib/metadata.php:22

 

 

ChrisValentine's picture

Re: Move emoncms.org data to local install of emoncms

In the end I've given up. Lost a load of historic data but I guess I'll just have to put up with it. Frankly disappointed that this procedure isn't properly implemented or supported.

TrystanLea's picture

Re: Move emoncms.org data to local install of emoncms

Hello Chris

Sorry to not have seen your posts sooner. On closer inspection of the backup script I realise that I accidentally uploaded local changes to the script that I was using for local backups to the github repository. I have now reverted them back and tested downloading to emoncms v9.3 running locally and it all ran ok.

The location of usefulscripts on your computer doesnt really matter, I have mine on my desktop. There is also no need to have the same account credentials on the local install to mirror the remote.

The backup script needs to be ran as root:

    sudo php backup.php

The main settings to look out for in backup.php are:

    $remote_server = "http://emoncms.org";

This is the remote server from which to download the data, here emoncms.org by default.

    $remote_apikey = "WRITE-APIKEY";

Yes, write apikey - no writing is actually being done, the writekey is more for elevated permissions here as a full download would ideally require higher level access to an account.

    $link_to_local_emoncms = true;

Link to local needs to be true for the data to appear in your local emoncms account. This will create all the mysql and redis entries for the feeds. With link to local set to false the data will just be downloaded locally as a backup.

    $local_emoncms_location = "/var/www/emoncms/";

If your local emoncms is in a different location such as /var/www/html/emoncms this line will need amending.

    $local_emoncms_userid = 1;

This is the line that specifies the local emoncms account to link the data with. If its the first user registered on the local emoncms the user id will be 1.

    $backup_inputs = true;

Backup_inputs can be used to make a local backup of all inputs and input processing. Useful for moving emoncms installs between raspberrypi's

    $dir = "/var/lib";

$dir specifies the data location which is usually /var/lib on non emonpi linux systems or /home/pi/data on the emonpi/emonbase.

 

I will look into modifying the script to check if redis is enabled in the local emoncms settings.php so that its not a requirement.

 

TrystanLea's picture

Re: Move emoncms.org data to local install of emoncms

Ok, I've modified the backup script to be able to run without redis if redis_enabled is set to false in the local emoncms settings.php. Tested it and it works ok.

ChrisValentine's picture

Re: Move emoncms.org data to local install of emoncms

Does this mean that if I download the latest version I could now grab the historic data without it upsetting the new data?

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.