30th
Instance reincarnation using duplicity?
For me, the biggest challenge to using Amazon’s EC2 infrastructure (besides the usual trust issues I have with any corporation) has been the ephemeral nature of any data saved to an instance’s hard drive. In a nutshell, if the instance goes away, so does everything in the local filesystem.
I recognize how valuable it is to declare up front that the data is ephemeral, because it forces developers to take appropriate pains to ensure that their information ecosystem is resistant to failure. It’s also a more-than-gentle nudge into either 1) adopting the related S3 service as a live backup mechanism, or 2) creating additional EC2 instances for replication and redundancy.
For today, it looks like my solution will be to use duplicity (spelled rdiff+rsync+gpg) to create a local encrypted backup set on each host. At the same time, I’ll have a dedicated physical backup host (or two) with the keys to log in and grab the duplicity diffs. I looked at storing the backups in S3, but I’d rather have local copies of everything first, and not use S3 storage directly from within EC2 for now.
The plan is to set it up so that, at most, we lose 30 minutes of activity. That may not be enough for the enterprise, but it’s more than enough for me.