Firebase Continuous Backup

07 Mar 2014

I have been using Firebase in development recently. It has been great for real time data storage, sync, and transfer between clients. However, as soon as I made the big leap from one developer to two, things started to go downhill.

Now with two developers, we would overwrite each other's data and latest progress or make changes to the data structure and break everything. This quickly exposed an issue we would be sure to hit in the future, data loss. There is strong need for Firebase data to be continuously backed up.

See the solution now on Github.

Quick research revealed that someone else had the same thought and created a script to take care of it: https://github.com/sergiopantoja/firebase-backup-node.

My current stack is a bit different:

• Python - no Node.js
• Heroku - no filesystem.

Firebase Backup S3 Python

The solution: write it in Python, save it to AWS S3.

https://github.com/designed27/firebase-backup-s3-python

Using python, you can automatically (with cron or the Heroku Scheduler) backup all your Firebase data and store it safely on AWS S3.

The python script simply pulls all data directly from Firebase using Python-Firebase over REST - I was hesitant to trust Heroku's websocket support and the relatively new python bindings for Firebase native websocket access as well.

Then it connects to S3 using Boto to push a new key to the bucket set in the environment variable. The data is the JSON dump of the Firebase database.

The result is a new .json file in your S3 bucket whenever the script runs.

I put this to use with the Heroku Scheduler. After setting up the environment variables to hold my AWS keys and Firebase access, the scheduler runs the following code once a day (for now):

$ python backup-firebase.py