Automated Heroku Backups

by Trevor Turk

Update: This code has been wrapped up into a few gems that are available on GitHub. Start by looking at the one that Eric Davis kindly put together and note that there are forks using fog, which I’ve been using lately for S3 support.

---

Update 2: Heroku developer David Dollar has put together a gem called heroku_backup_task that uses the new pgbackups addon to make this even simpler. Unless you want to store your backups in your own S3 bucket, I’d recommend just using this gem now.

---

It’s easy to enable automatic nightly PostgreSQL database backups from Heroku to Amazon S3. Nick Merwin and Derek Perez have shown us a couple of techniques for this sort of thing already, but I’ve got another one for you.

Start by adding the following to your Rakefile:

http://gist.github.com/366598?file=gistfile1.ru

It’s OK if you already have a cron task defined. Did you know that rake tasks append behavior by default? Weird, I know. Anyway…

Add right_aws to your .gems file:

echo right_aws >> .gems

Commit your changes and push to Heroku:

git add .
git commit -m 'heroku backups'
git push heroku master

Enable the cron:daily addon:

heroku addons:add cron:daily

Provide Heroku with your Amazon S3 keys:

heroku config:add s3_access_key_id=YOUR_ID s3_secret_access_key=YOUR_KEY

Run the cron Rake task manually, for testing purposes:

heroku rake cron

If all goes well, a new private bucket named APP_NAME-heroku-backups will be created and will contain a backup file named APP_NAME-YEAR-MONTH-DAY-HOURMINUTESECOND.dump.

Confirm that the backups are working by downloading the archive and attempting to reload it into a freshly created database:

createdb NEW_DB
pg_restore -d NEW_DB BACKUP_FILE

It may complain about some users and roles from Heroku that are missing, but I think those warnings are safe to ignore. In any case, I’d suggest trying to use this new database with your local app in development, just to make sure it’s working as expected.

From here on out, Heroku will automatically invoke the cron Rake task automatically for you on a daily basis. This will create a new backup file that will be stored on S3. The backups on S3 aren’t being rotated or deleted automatically, which is only a minor annoyance for me. Please do let me know if you take the time to set up some kind of backup rotation, though.

It is highly recommended that you periodically confirm that the backup task is running properly:

heroku logs:cron

You should also periodically verify that the backups are valid like we did when we downloaded the backup file, used pg_restore, and tested the rehydrated database with the app that we have running locally.

Additionally, I’d recommend using Hoptoad in conjunction with Toadhopper so that you can receive a notifications if/when something goes wrong with your backups.

Add toadhopper to your .gems file:

echo toadhopper >> .gems

Uncomment the relevant lines in the heroku:backup Rake task:

# rescue Exception => e
#   require 'toadhopper'
#   Toadhopper(ENV['hoptoad_key']).post!(e)

Commit your changes and push to Heroku:

git add .
git commit -m 'todhopper for Heroku backups'
git push heroku master

Provide Heroku with your Hoptoad API key:

heroku config:add hoptoad_key=YOUR_HOPTOAD_KEY

Run the cron Rake task again, just to make sure it’s not broken:

heroku rake cron

This way, you’ll be notified if something goes wrong with your backup task. I’d still suggest manually verifying the backup files as frequently as possible, though. There’s no such thing as a “set it and forget it” database backup. Viva due diligence!

Finally, please note that I’m not sure if large databases will work with this backup method due to Heroku’s filesystem restrictions. It’s working for me, but your milage may vary.

Advertisement