Backup / Migration tool?

Hi,

 

My emoncms instance is working fine on my RPi + HDD. Goal is to forget it and make it work for months.

For sure, one day, it will crash and I would like to know the best procedure to backup everything regularly in order to restore easily in case of crash. I want to backup all: database but also emoncms application.

Is it enough, If I copy to my NAS all the files from the HDD + make an image of the SD card?

Robert Wall's picture

Re: Backup / Migration tool?

Try the "Search" box. Or the emonCMS documentation.

darrepac's picture

Re: Backup / Migration tool?

Obviously, I already did it but unless I miss it, I did not find a complete solution to backup and restore everything. Some talks about having an other emoncms instance + some scripts, some store only the SQL database, some only the files...

Robert Wall's picture

Re: Backup / Migration tool?

"Obviously, I already did it"  No, not obvious at all.

When I read the instuctions, it seemed quite clear that the method described was intended to provide a complete backup. Have you done that and tested it? Are you saying it does not backup everything?

darrepac's picture

Re: Backup / Migration tool?

Which instructions?

I read tens of forum posts but none seems 100% clear and complete to me. In the manual (http://emoncms.org/site/docs/backup) the instructions need to have another machine running an instance of emoncms which is not trivial to have when you have it running on one RPi

I may have missed such instructions you are talking about, so can you point me to the "right" link?

Robert Wall's picture

Re: Backup / Migration tool?

Are you saying you want to use the RPi to make the backup on the NAS?

The first part of the instructions - yes you were in the correct place - assume you are making the backup on a connected machine. Approach 2 gives the default locations of the databases, it should be adequate to copy those.

However, as much effort goes into setting up the RPi in the first place, you probably need an image of the complete HDD and of your boot SD card in any case, in addition to making regular copies of the data files.

(If you can, it would be a good idea to set up emoncms on another machine anyway, and 'restore' your backup up data to that, thus proving your backup.)

darrepac's picture

Re: Backup / Migration tool?

Well...

Other idea I got was to use emoncms.org as a backup... copying all my input/feeds to emoncms.org

darrepac's picture

Re: Backup / Migration tool?

I did a mysqldump of my database... the resulting files is quite small (47kB). Does it mean that all the data are store in /var/lib/phpfiwa (phpfina, timestore)? or there is an issue with my dump?

hendersoniain's picture

Re: Backup / Migration tool?

Hi,

I'm trying to upgrade my emoncms on my Pi using the following documentation: http://emoncms.org/site/docs/upgrading

When I try to do any of the mysqlbackup commands: mysqldump -u root -p emoncms > emoncms_backup.sql

I always get permission denied: -bash: emoncms_backup.sql: Permission denied

I have not changed any passwords as far as I know. Any ideas?

Thanks.

Simonk695's picture

Re: Backup / Migration tool?

Where are you running the script from?  What directory?  You may not have 'write' permissions on that.  Try...

mysqldump -u root -p emoncms > /home/pi/emoncms_backup.sql

That way you are writing to your home directory.  Better still an attached USB drive?

/mnt/usb etc

hendersoniain's picture

Re: Backup / Migration tool?

Hi,

Thanks. Got it now - I was using the wrong root password for my pi!

 

threldor's picture

Re: Backup / Migration tool?

Hi, 

I have found a few methods to backup/migrate/restore a local emoncms database and have been using them effectively. I am interested in a way to backup/export a local database and import this to emoncms.org as darrepac suggested.
Any help would be great.

cheers

TrystanLea's picture

Re: Backup / Migration tool?

Hello threldor, unfortunately there's no easy way to do that at the moment, you can send bulk historic data to emoncms.org using the input/bulk api:

http://localhost/emoncms/input/bulk.json?data=[[520,16,1137],[530,17,1437,3164],[535,19,1412,3077]]&sentat=543

- The first number of each node is the time offset (see below).
- The second number is the node id, this is the unique identifer for the wireless node.
- All the numbers after the first two are data values. The second node here (node 17) has two data values: 1437 and 3164.

This is documented on the input api helper page in emoncms.

But backing up configuration data is more difficult. The main challenge with migrating data like this is that a local emoncms install will have different unique ID's for nodes/feeds/dashboards etc to the target emoncms server and so there needs to be a mechanism that remaps all these unique ID's.

TrystanLea's picture

Re: Backup / Migration tool?

I realise the emoncms.org backup to a local emoncms tool is out of date, Im working on bringing it up to date now.

TrystanLea's picture

Re: Backup / Migration tool?

I've updated the emoncms.org backup tool to use the new engines, its also possible to download the data to your computer without a local installation of emoncms, documentation and link to backup script is up here: http://emoncms.org/site/docs/backup

borpin123's picture

Re: Backup / Migration tool?

Just looking at this as I need to update an old version.

At the bottom of the page is the warning;

Important Make sure you disable oem_gateway/emonhub or raspberrypi_run and any posting to the http api's (stop apache) before copying the data files so that when you make the copy the data is in a state where its not being written to.

How exactly do you do this?  I am not using the OEM gateway (but others might).  Could this backup page and the upgrade page be updated to tell those less familiar with the inner workings how to do this please.

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.