Integrating With File Storage vs. Object Storage

There are two broad categories of cloud storage for unstructured data:

• File Storage: A personal cloud that users use to synchronize and collaborate on files. Examples include Dropbox and Google Drive.
• Object Storage: A component of an application’s stack that stores unstructured binary data as files or “blobs.” Examples include Amazon S3 and Google Cloud Storage.

When consumers mention “Cloud Storage,” they almost always mean data sync and collaboration tools like Dropbox and Google Drive that let users store files remotely. Both categories of services above store file data and offer APIs to access them, but the primary user of file storage is usually an individual, whereas the primary user of object storage is usually an application.

In this post, we’ll consider the benefits of integrating with each type of storage service.

Why Integrate?

First, apps must decide which set of cloud storage providers to integrate with and why.

File storage platforms offer access to end-user data. Applications that enable end-user workflows, such as productivity tools, find this data valuable to transfer into or out of their application. User-generated content is rarely in a single app, but collectively represents a source of truth. Hence, apps must access each others’ APIs to enable the workflows that users demand.

On the other hand, users rarely use object storage services such as Amazon S3 directly to create, sync, and collaborate on content. However, an application’s user-base may collectively generate a large quantity of content, such as new versions of images, annotated PDFs, or their own files uploaded to the app. An application can integrate its backend with an object storage service to abstract away scaling concerns when storing this vast quantity of data.

This key difference manifests itself when gaining access to the remote storage system in the first place. Apps prompt users to grant access to their data in cloud storage by guiding them through an OAuth 2.0 flow. However, apps access object storage using just a secret key provided by the storage service.

Features and Performance

User-facing cloud storage services offer a myriad of features to differentiate themselves from each other. Most of these features center around boosting end-user productivity. They enable users to view, edit, and collaborate on content flexibly across all their platforms. With some exceptions such as Box, storage services do not prioritize exposing this functionality for applications built on their platform to white-label.

However, object storage services’ primary value proposition centers around performance and cost at scale. These services bill on the quantity of data stored and accessed rather than the number of users accessing them. Their APIs offer higher rate limits and lower latency, especially within an app’s own hosting provider. They offer flexible approaches to store and transfer content, including which data-center stores an app’s data, data durability guarantees, CDNs, custom headers for caching, and more.

Alternate Perspectives

Despite the contrasts described above, we’ve seen applications use cloud storage providers such as Google Drive as a backend for application data uploaded by users. Apps might want their own staff or other users to require direct access to end-user content. 

We’ve also seen apps access their users’ object storage accounts directly. They use integration platforms to abstract object storage providers’ key-based authentication schemes behind a consistent authentication flow to enable easy access to a user’s own S3, Azure, Google, or other S3-compatible services. Tech-savvy customers realize the cost-savings gained by directly using the infrastructure that powers file sharing platforms. Users access object storage directly using open-source file browsers or other tools.