DR Solutions: Recovering from a True Disaster

Early in 2011 we had our first production disaster recovery event. We had thoroughly tested the Filer’s DR capabilities countless times on our own, and George Crump of Storage Switzerland has analyzed it and reported on his experience. More importantly, many of our customers put it to the test before buying the Filer. But we never really wanted it to be used in a real-world situation. Why? Because it’s there to solve the pain after a disaster. And we wouldn’t wish a disaster on any of our customers. But then it happened and the Filer eased this company’s recovery from a true disaster.

Disasters come in all shapes and sizes, from hurricanes to buildings getting destroyed to failures within the data center. Early last year one of our paying customers in the manufacturing sector had a real disaster. This is always a terrible and stressful event, but thankfully it was IT-related, so no people were injured. The company was using an iSCSI SAN device to store some of their virtual machines and the device suffered a catastrophic disaster. They completely and permanently lost the storage it provided. Their Nasuni Filer, used for primary storage, was one of these virtual machines.

For many of the virtual machines, they had to start recovering from tape, which is a stressful and lengthy process. This involves first hunting for the tapes, then initiating the restore, and finally, praying that there won’t be any tape issues. Even then, the best they can do with tape is recover from their nightly backup. On the other hand, the Nasuni Filer takes a snapshot every hour by default.

To recover the data stored on their Filer, they followed the simple process of downloading the virtual machine again from Nasuni.com, installing the product, and then completing the disaster recovery process. The whole procedure, if you have your encryption keys, takes about 15 minutes for a total recovery (see George’s excellent video on this).

The process was simple and the customer recovered about 400GB of uncompressed data:

Total Capacity Chart
This customer had only recently joined us, so they just had the data they managed to load in the short time before they started having stability issues on their SAN. Yet even if they had terabytes of data, the recovery time and process would have been the same. No knowledge or management of cloud security or API keys was needed.

For a while, until the cache had time to populate, access was a little slower than normal, but all of their files were immediately accessible to their users. Even if this had been one of the worse events – involving the loss of a building or other damage from a natural disaster – the recovery would have been the same. The customer’s data is safely stored offsite in data centers that rival the best of those from enterprise.

A success for the Filer, but not a pleasant day for our customer. Fortunately we were able to make their life a little easier as they dealt with the mess.

If you’re skeptical, we encourage you to take the Filer for a spin and test the DR process yourself. We recommend self-escrowing the keys (the Filer reminds you to do this after install) so you don’t need to call us to de-escrow.

Questions/Comments? Visit our forums or shoot us an email. We are really here to help.