The Reservation Problem: Why File Locking is Such a Challenge

Have you ever made a hotel or restaurant reservation through a travel agent or third-party website and arrived at your destination — only to be told that there’s a problem with your reservation?

Most of us can relate to this frustrating experience. And, surprisingly, this scenario offers a perfect analogy for illustrating the challenges of file locking in distributed systems.

Why File Locking Is Harder Than It Seems

It seems like it should be easy, but file locking is actually quite complex. A large part of the reason is that the locking can occur at multiple places in the stack, and where it occurs is situationally dependent.

For example, let’s say you’re working on an Office document stored on your local computer. The application assumes — correctly — that no one else will try to open that file at the same time. Thus, it handles all the locking internally with ease.

Now, imagine that the same file is placed on a corporate file share accessed by multiple users. Suddenly, single-user access is no longer a safe assumption. In this case, a locking mechanism is required elsewhere, outside of the application, to prevent users A and B from overwriting each other’s work.

Furthermore, the application itself has no idea it’s working with a shared file, so it still thinks it runs the locking roost. Fortunately, in these scenarios, the storage system and the network usually step in to effectively manage locks — as long as users are accessing the file through the same access point.

Global File Systems: The Real Challenge Begins

But what happens when that file share is part of a global file system, and users A and B are geographically separated, accessing it from different access points?

Now you need a global lock, an even higher-level mechanism to arbitrate which user has write permissions to that file. And at this level, latencies suddenly start to matter — a lot! The back-and-forth communications that are part of that arbitration process can be extensive. If you’ve ever tried to resolve a dispute between two passionate parties, you know that to make progress, one must proceed deliberately and slowly to prevent a shouting match.

The arbitration is a lot like that. When that arbitration happens at local LAN speeds, the latency is negligible, and users don’t notice. However, on a global scale when latencies go from sub-millisecond to tens of milliseconds or longer, users notice. This is one of the challenges with a global locking mechanism, and why (historically) it has been so hard to implement effectively.

One File, Many Voices

So now we have this notion of a global file lock mechanism acting like a traffic cop. Meanwhile, the application still thinks the file it’s working on is local and tries to act accordingly. It’s a chaos reminiscent of high school orchestra on the days when we got a new piece of music: some players using email, others using snail mail — yet everyone must stay in sync.

Going back to our earlier analogy: with this context, it makes a little more sense how a problem with the reservation could have occurred! Some people are making reservations with the provider directly (local storage), some people are using a travel agent (network and storage), and some people are using an aggregator website (global lock). It’s highly likely that at some point, a conflict is going to occur. Someone will make a reservation using an aggregator website, and before that process finishes, someone contacting the provider directly will reserve the same room or table.

That’s the crux of the problem; without a unified set of rules, conflicts are inevitable. And because every application behaves differently — some assuming exclusive control, others more flexible — the challenge is compounded.

Implementing Global Locking

We’ve established a need for global locking, and we’ve identified some challenges it brings. Now, how do we implement it?

As it turns out, there are multiple approaches to this problem. Some solutions involve the placement of the lock server directly inside the customer’s environment. This is a double-edged sword. On one hand, the lock server being within the customer’s control can give the customer a sense of security. On the other hand, customer-controlled lock servers are generally not elastic, and the fan-in of lock arbitration traffic to a single location could become a bottleneck at scale. Additionally, the onus of providing maximum resiliency is 100% on the customer.

Another option, which is the approach Nasuni uses, is to make the global locking a SaaS offering. This provides a few advantages to customers:

• Customers don’t need to worry about where to place the lock server, or how to design for varying bandwidth and infrastructure resiliency.
• The SaaS provider (in this case, Nasuni) handles the development and maintenance of the well-architected cloud-based lock server.
• The system is architected for scale elasticity, failover for resilience, and multiple global lock servers to address those previously mentioned issues of latency at global scale.

From Customer Need to Innovation

At Nasuni, we understand that when it comes to global scale, a global file locking mechanism is necessary.

Originally, Nasuni didn’t have a global file locking mechanism — but it didn’t take long for our early customers to tell us they needed one. So we built one, and we built it from the ground up to meet the needs of today’s businesses. And, admittedly, it wasn’t easy.

However, the result is a SaaS-based global locking solution that is elastic, resilient, and built to handle billions of lock requests every year. No more overbooked rooms, or double-booked flights. By bringing clarity, consistency, and control to file access across the world, we can make sure every user has the right key at the right time — no matter where they’re working from.