chxo internets

Data Migration Pattern

Facebook-er Kent Beck’s Software Design Glossary includes an entry on Succession, “the art of taking a single conceptual change, breaking it into safe steps, and then finding an order for those steps that optimizes safety, feedback, and efficiency.”

As an example, he gives us (for free!) this pattern for migrating from one datastore to another:

1. Convert data fetching and mutating to a DataType, an abstraction that hides where the data is stored.
2. Modify the DataType to begin writing the data to the new store as well as the old store.
3. Bulk migrate existing data.
4. Modify the DataType to read from both stores, checking that the same data is fetched and logging any differences.
5. When the results match closely enough, return data from the new store and eliminate the old store.

Beats shutting everything down for hours while you wait for the data to get copied over the wire.