The Nasuni Cloud Storage Blog

All Blog Posts »

Data in the Cloud: Ebbs and Flows

Avoid Vendor Lock-inPart 1: Understanding transfer speeds and avoiding vendor lock-in

In the last few posts we spoke about a couple of the requests we had from our customers to migrate data from one cloud storage provider (CSP) to another and how we helped them accomplish those goals. After the initial concerns about maintaining security throughout the entire process (not an issue with Nasuni), the second question they asked is how fast their data could be moved from one CSP to another.

Underneath the questions regarding cloud transfer speed lies a deep anxiety about vendor lock-in. It is possible for these performance issues to go undetected for years as data trickles in small daily increments. However, if a customer wants to take their business elsewhere they may be shocked to discover that their provider cannot support an all out migration. At Nasuni, we constantly stress test the capabilities of selected backend providers. We know customer data is always growing and we never want their data getting stuck because the pipes fail to scale.

So, what how fast is fast enough when it comes to bulk migration of data in the cloud?

The answer, like many answers related to storage performance, is that it depends. This kind of bulk movement depends on many factors:

  • The performance of the CSP you’re moving data out of (CSP read performance)
  • The performance of the CSP you’re moving data into (CSP write performance)
  • The raw speed of the network between the CSPs and the machines you’re using to move the data (CSPs cant move data to each other directly)
  • The load on the network between the CSPs and the migration machines at the time of the migration
  • The number of machines being used to do the migration
  • The specifications of the machines being used to do the migration (memory, CPU speed, disk speed, etc)
  • Quality and speed of the tools being used to do the migration

“It depends” isn’t a popular answer, so we often used actual performance numbers from prior migrations to frame our answers, but the question begged for some more extensive testing and more data so we set out to get some numbers.

In part 2 of this article we will look at different use cases and benchmark data transfer speeds for the top public cloud storage providers.

Rob Mason

Rob Mason Rob Mason has more than 20 years of operational, management and software development experience, all of it in storage. A meticulous builder and obsessive tester, with an eye for talented engineers, Rob produces rock-solid software, and, through his own example of hard work and ingenuity, inspires his teams to outdo themselves. His determination for thoroughness extends to financial and operational matters, and at Nasuni, he is a powerhouse behind the scenes, managing the company’s operations, in addition to its engineering team. As the VP of Engineering at Archivas from 2004 to acquisition, Rob oversaw all development and quality assurance. After the Hitachi acquisition, he continued in his role, as VP of HCAP Engineering, managing the integration of his team with Hitachi’s and supporting the rollout of HCAP. Before joining Archivas, he was a senior manager at storage giant EMC, where he was responsible for the API, support applications and partner development for EMC’s content-addressed storage product, Centera. In a previous stint at EMC, he was Manager and Principal Design Engineer for the elite Symmetrix Group, where he improved the speed and reliability of EMC’s flagship enterprise storage disk array. Between Centera and Symmetrix, Rob was the co-founder and VP of engineering at I/O Integrity, a storage-based startup developing a high-performance caching appliance. He has a bachelor of science from Rensselaer Polytechnic Institute and a master’s in business administration with honors from Rutgers University. Rob holds upwards of 30 patents.

Subscribe via email

What is Nasuni?

Meet the Bloggers

  • Andres Rodriguez
  • David Shaw
  • Ed DeJesus
  • Fred Pinkett
  • John Gniadek
  • Louis Abate
  • Rob Mason
  • Warren Arnold
  • Wayne St. Amand

Topics

Nasuni Buzz

  • @glbowyer
    Pleasure to connect with you @Nasuni 2 weeks ago
  • @exdej
    if azure was down, our customers couldn't tell! New on the @Nasuni Blog: Was Azure Down? http://t.co/pT7qmT7KPd 2 weeks ago
  • @Louis_Abate
    Check out the latest @nasuni customer story: @tbgpartners. We had a blast hanging out with these guys in Austin! http://t.co/6tAbrC4Wqu 2 weeks ago