Creating your own backups with Tarsnap
Table of contents
- 1 Managing backups yourself
- 2 Backup your database
- 3 Configuring Tarsnap
- 4 Restore your webshop from Tarsnap
Managing backups yourself
If you choose to create your own backups, we recommend using Tarsnap as an offsite backup provider.
This very robust third party backup service is well known for it’s excellent security and their low prices.
Tarsnap encrypts your data and then stores it on Amazon S3 storage platform. S3 is extremely reliable, as the data is duplicated to at least three different data centers. Nobody but you has access to the data, as it is encrypted using the best open standards (not even the NSA can get to it).
Tarsnap charges $0.25 per GB per month (for storage and bandwidth), but only transfers and stores changed data (incremental updates).
Backup your database
This requires some free space on your Hypernode. For a typical 1GB database, the compressed dump file takes 20MB (the index data is not copied). You can use df -h ~ to verify the available amount of available disk space.
If you want to store your database backup offsite (IE: with Tarsnap), we recommend to create the database backup before copying the file content to an offsite backup location. This way you are ensured your database backup is saved with the file backup on an offsite backup location.
Create a database backup
Add the following lines your crontab. It will not lock your database, so it can be safely run at busy times. This example runs nightly.
MAILTO=yourmailadress@here PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin" 0 2 * * * mkdir -p ~/backup; flock -n ~/.mysqldump chronic n98-magerun db:dump --root-dir=~/public --compression=gz --add-time=suffix --no-interaction --strip @stripped ~/backup 0 5 * * * chronic find /data/web/backup/ -type f -mtime +7 -delete
NB: If you are using Magento 2, replace
/data/web/magento2 and use the
n98-magerun2 utility for Magento 2 instead of the the
Step One – Create a Tarsnap account
Register for an account at Tarsnap and upload some prepaid funds. For an average Magento shop, €100,- will pay for a year of backup service.
You will receive a notification from Tarsnap when you are almost running out of credits.
Step Two – Generate a private key file
You will need to create a private key to encrypt your data. This is necessary to create and decrypt your backups and is needed when creating or restoring backups from Tarsnap.
On your Hypernode, run the following command to generate a key file:
tarsnap-keygen --keyfile ~/tarsnap.key --user <email> --machine <mysite>
You should copy this
tarsnap.key file to a safe location, as your backup is inaccessible without it!
A local USB stick in a safe at home or in the office is a good idea.
Step Three – Configure Tarsnap
Create a Tarsnap configuration file by running the following script:
cat > ~/.tarsnaprc <<EOM cachedir ~/.tarsnapcache keyfile ~/tarsnap.key print-stats humanize-numbers exclude .git exclude *.log exclude *.zip exclude *.lock exclude *.gz exclude .tarsnapcache exclude data/web/*/var/*/* exclude data/web/*/media/catalog/product/cache/* exclude data/web/*/media/js/* exclude data/web/*/media/css/*.css EOM
And second, create a directory where Tarsnap can temporary store and cache files, before sending them to a Tarsnap server:
mkdir -p ~/.tarsnapcache
Step Four – Test the setup
Ensure yourself the backup works as expected. Create a backup and if possible, restore the backup as well to double check the created backup is useful when you need it.
The initial upload could take up to several hours. To create a first backup, run the following command:
# Create a backup archive called "backup-WEEKDAY" tarsnap -c -f backup-`date "+%A"` -v ~
Step 5 – Configure a cronjob
Implement this as a daily cron. To make optimal use of the Tarsnap efficiency algorithm, you should NOT use compression on your local database dump and you should dump always to the same file.
Lets put this in a tiny script first. Save the following script as
cat > ~/bin/makebackup.sh <<EOM #!/bin/sh TODAY=$(date "+%A"); flock -n ~/.mysqldump chronic n98-magerun db:dump --root-dir=~/public --no-interaction --strip @stripped ~/backup/mysql-latest.sql; flock -n ~/.tarsnap.lock tarsnap -d -f backup-$TODAY 2>/dev/null; flock -n ~/.tarsnap.lock chronic tarsnap -c -f backup-$TODAY ~ EOM
NB: If you are using Magento 2, use the
n98-magerun2 utility for Magento 2 instead of the the
Next, make the script executable:
chmod 755 ~/bin/makebackup.sh
If you want to save some space on your Hypernode, you can delete the created backup by adding
rm ~/backup/mysql-latest.sql to your backup script. This implicates there is no database backup present for recovery, so if you need your backup you’ve got to retrieve it from Tarsnap first in case of emergency.
Next, add a cronjob to run the backup script nightly:
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin MAILTO=yourmailadress@here 0 2 * * * flock -n ~/.makebackup.lock ~/bin/makebackup.sh
This will create an archive for every day of the week. It will overwrite old archives. You will only pay for the unique, compressed amount of data. For an average shop, this is 1/3 of the size.
Restore your webshop from Tarsnap
Note: restoring will use the same directory structure.
For example, when you restore from the backup with path “/data/web/public” (in case of Magento 1), the “data/web/public” structure will be created in the current working directory. So do a
cd / first, to overwrite the old files in the right location.
Restoring your content from a Tarsnap backup
- List the available backup archives from Tarsnap:
tarsnap --list-archives --verbose
- List the contents of a specific archive:
tarsnap -t -f <archive>
- Extract a file from Tarsnap:
tarsnap -x -f <archive> <file or files> tarsnap -x -f backup-Monday data/web/site.com/Nieuwsbrief.png
Restoring your database from a Tarsnap backup
- Retrieve the backup from Tarsnap (if you remove the dump file after pushing it to Tarsnap)
cd / tarsnap -x -f backup-Monday data/web/backup/mysql-latest.sql
- Then, restore a database dump, use
n98-magerun(For Magento 2
n98-magerun2), to import the database dump:
n98-magerun db:import --root-dir=~/public ~/backup/mysql-latest.sql