In a few of my recent posts I have focused on demystifying some of the concepts related to the new generation of file storage, protection and access solutions, including a piece on how the cloud itself is not some nebulous abstraction, but a collection of the most efficient and secure data centers in the world. The next subject on the list is the difference between object stores and file stores, and how is it that files can be stored within an object store.
Object Storage Vs. File Storage: What’s The Difference?
On the inside, an object and a file are essentially the same thing. Each is simply a chunk of data. Yet there are some very important distinctions between an object store and a file store. An object store has very little structure on top of that basic chunk. You put in a chunk of data with a name attached (and sometimes metadata), and as long as you have the username and password associated with the bucket in which the object is stored, then you have access. All you have to do is give the object store that name and it returns the data. The basic commands are create, delete, get and put. That’s it.
An object store is also a very simple mechanism for storage. First you create a designated, secure volume – otherwise known as a bucket. Then, assuming you have the right security credentials, you take each basic chunk of data and drop it in. This simplicity enables unprecedented scale and redundancy. An object store in the cloud can grow to unlimited scale in part because the basic components are so simple. For the same reason, it is easier to spread data across locations and systems to ensure that you retain access even in the case of multiple hardware node failures.
A file store is more complex. Even though you’re starting with a basic chunk of data, that data has layers of additional information attached. Technically it is still “unstructured” because the data inside has no structure. But files often come with tree structures of directories, metadata, time stamps, access control lists and more. There are also file naming conventions to consider, such as common characters and lengths. Or the fact that file names end with a dot and an extension representing the program with which they’re most often associated. All this additional information is one of the reasons most file stores have limits to their scalability.
To be able to move files into an object store, you need some kind of translation tool. The complete lack of structure in an object store means that the structural metadata associated with a file needs to be attached to that basic chunk – the metadata delineating how each basic chunk of data might relate to another. In effect, what you’re doing is creating a file system on top of an object store, a system with all the scalability and durability of the latter with the human-accessible structure of the former. The tradeoff or expense is that you need that additional glue (usually an appliance) to make it work.
There are varying names for these translation tools or appliances. They have been called gateways and, in a broader sense, cloud-integrated storage solutions. They also vary greatly in terms of capabilities and reach. Some systems stand between a single office location and the cloud-based object store. The AWS Gateway, for example. Others deliver entirely new capabilities to the enterprise, such as global file systems with built-in data protection.
Combining Object Stores and File Stores with Nasuni
At Nasuni, we combined the scalability and durability of cloud-based object stores with the familiarity of a file store. UniFS™, our patented global file system, breaks each file into chunks stored in an object store before it leaves the security of our client’s offices. Then the system encrypts each chunk with customer-controlled keys, ensuring that it remains secure on the wire and in the cloud, before sending it to the object store. The result is Nasuni File Services, a next-generation file system that delivers far more functionality, including global file sharing, secure mobile access to files, unlimited versioning through snapshots to the cloud, and many additional features.