How to auto back up emoncms data on RPI to USB stick on RPI

Hi

Shutting the door after the horse has bolted.....

I have been running a Pi as a remote emoncms server. I have been collecting data for some months with no trouble at all. I knew I should back up the data and had bought a 32G USB stick to take a copy of the data. Being somewhat linux challenged I had not got round to doing it. Yesterday the server failed. I think the SD card in the RPI is the problem because I created 2 SD cards when I first started and with the second SD card the RPI boots and with the card I was using it will not. Also I don't get any response to the first card when I plug it into my Windows PC but I do with the second card.

I will now set up a new server. This time I would like to add the facility to make a daily copy of the data to the USB drive. I assume that this should be possible with a script. Can somebody with Linux knowledge point me in the right direction.

All the Linux setup on the RPI I have achieved by following the instructions parrott fashion and I am only just beginning to get a glimmer of how it all works.

Regards

Ian

 

o_cee's picture

Re: How to auto back up emoncms data on RPI to USB stick on RPI

http://sourceforge.net/projects/automysqlbackup/

Think that's the one I'm using for our servers. It's in Debian repos.

Ian Eagland's picture

Re: How to auto back up emoncms data on RPI to USB stick on RPI

Thanks.

I will give it a try

Ian

Jérôme's picture

Re: How to auto back up emoncms data on RPI to USB stick on RPI

Hi.

You might as well use automysqlbackup, which I didn't know about, but for what it's worth, here's what i did on my server :

(Comments were in french, I'm adding a translation here)

--------------------------------------------------------------------------------------------------------------------------------

#!/bin/sh

#########################################################################
## Ce script sert a effectuer une sauvegarde des bases de donnees mySQL #
## mySQL backup script #
#########################################################################

db=mysql
rep_base=/backup_dir_wherever_you_want/$db

# 1 - Creer un repertoire de sauvegarde en fonction de la date
# 1 - Create backup dir according to date

repertoire=`date +%y%m%d`
mkdir $rep_base/$repertoire
cd $rep_base/$repertoire

# 2 - Faire les sauvegardes
# 2 - Do the backup

for base in `mysql -u root --password=pwd --execute="SHOW Databases" --skip-column-names | cat`
do
echo "Sauvegarde de $base ..."
# Sauvegarde de la base
#echo "mysqldump --single-transaction --host=localhost --user=root --password=pwd $base | gzip > $base.sql.gz"
mysqldump --single-transaction --host=localhost --user=root --password=pwd $base | gzip > $base.sql.gz
done

# 3 - Effacer les vieilles sauvegardes
# 3 - Delete old backups

# Sur la base d'une sauvegarde quotidienne, et pour conserver un mois d'historique,
# on efface ce qui a plus de 30 jours
# We want one month of daily backups,
# let's erase what is older than 30 days

echo "Suppression des sauvegardes precedentes ..."
#find $rep_base -mtime +30 -print
find $rep_base -mtime +30 -delete
 

--------------------------------------------------------------------------------------------------------------------------------

(You may want to comment the delete line and uncomment the print line at the end, in a first time, to check what is going to be deleted.)

I created a similar script for postgresql and made both executable (command : chmod +x filename)

To call them, I added a file in /etc/cron.daily/

--------------------------------------------------------------------------------------------------------------------------------

#!/bin/sh

#####################################
## Sauvegarde des bases de donnees ##
#####################################

# postgresql
su - postgres -c "path_to_pgsql_backup_file"

mysql
path_to_pgsql_backup_file

--------------------------------------------------------------------------------------------------------------------------------

This appears to work but does not claim to be the most elegant and efficient solution.

 

 

Ian Eagland's picture

Re: How to auto back up emoncms data on RPI to USB stick on RPI

Hi Jérôme

Thanks. I am fairly new to Linux but I think I understand how your script works so I will give it a try.

Regards

Ian

Jérôme's picture

Re: How to auto back up emoncms data on RPI to USB stick on RPI

You're welcome.

<geek mode>Beware, this script might containt cat abuse...</geek mode>

Just like for the delete command, if the mysqldump command fails, you can uncomment the 'echo' line to see what happens.

 

Ian Eagland's picture

Re: How to auto back up emoncms data on RPI to USB stick on RPI

Hi Jérôme

I have started writing the script in sections to test.

First part:-

#!/bin/sh

#########################################################################
## Ce script sert a effectuer une sauvegarde des bases de donnees mySQL #
## mySQL backup script #
#########################################################################

db=mysql
rep_base=/media/KINGSTON/$db

# 1 - Creer un repertoire de sauvegarde en fonction de la date
# 1 - Create backup dir according to date

repertoire=`date +%y%m%d`
mkdir $rep_base/$repertoire
cd $rep_base/$repertoire

Running it I get this error:-

pi@raspberrypi /usr/local/bin $ test.sh
mkdir: cannot create directory `/media/KINGSTON/mysql/130211': No such file or directory
/usr/local/bin/test.sh: 16: cd: can't cd to /media/KINGSTON/mysql/130211
pi@raspberrypi /usr/local/bin $

 

I can't see where my error is, any ideas?

/media/KINGSTON is a usb drive that exists and is mounted. I can get to it with a command line cd.

pi@raspberrypi /usr/local/bin $ cd /media/KINGSTON/

pi@raspberrypi /media/KINGSTON $

 

Regards

 

Ian

 

Jérôme's picture

Re: How to auto back up emoncms data on RPI to USB stick on RPI

Try

     mkdir /media/Kingston/mysql

first.

By default, mkdir won't create all path but only child directory if parent exists. You can pass it '-p' to let it create necessary directories (see man mkdir).

Ian Eagland's picture

Re: How to auto back up emoncms data on RPI to USB stick on RPI

Jérôme

Again many thanks. The script is now working and all looks good. Tomorrow will show if I have added it to cron correctly.

Regards

Ian

 

 

Jérôme's picture

Re: How to auto back up emoncms data on RPI to USB stick on RPI

Glad it helps.

Yet, as I said, if I were you, I'd pay a look at the tool Oscar suggested, which is even packaged for debian.

Well, actually no. I would have paid a look, but now that you got something that seems to work, I'd probably keep it as it is.

Bye.

Note to self : add something in the wiki about http://sourceforge.net/projects/automysqlbackup/

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.