What is Object Storage?
For Part 2 of this series, we are going to focus soley on Object Storage. At the most basic level, Object Storage treats files and folders as “objects”. It is a flat hierarchy, and the objects are generally assigned a unique ‘key’ value in order to locate them. The objects can be grouped by assigning them to ‘buckets’, also known as ‘pools’. This method of storage is a lot simpler, which in turn, makes it vastly more scalable. The metadata stored with a file becomes much smaller using this method, and overhead used to manage all that metadata is reduced along with it. Due to the flat hierarchy, when you need more space you just add more nodes (storage). None of it is dependent on existing objects (since no “tree” to traverse, all flat hierarchy) so there are no real limits of how many nodes can be added to expand this style of storage. The disk space expansion is generally invisible to the end user. They just see that their bucket has a larger amount of space available to it without incurring any downtime.
One of the ways it maintains performance is by its simple nature. This in turn means the list of commands and features you can use are also pretty minimal. You are be able to upload, download, copy, delete objects (files), and control which users can do what (security). That is pretty much the extent of the built in functionality. That does not, however, mean there are no other ways to interact with data stored here.
Most Object Storage platforms also offer an API (Application Programming Interface) to communicate with the system. An API is a set of commands built into a software stack that allows external applications and clients to interact with the software at a programmatic level. When you hear of an applications talking to each other, it was most likely achieved through an API. Basically put, it is a “cheat sheet” for talking to an application that saves some other programmer from having to code a brand-new method of talking to that program (requiring a lot of time and reverse-engineering). With an API they can instead simply use the already-created commands and program their application, script, etc. to use them. While the Object Storage has minimal functionality, the availability of the API means you can program your own higher-level functionality (or use someone else’s application to do the same).
Two common API’s used worldwide are the “Amazon S3” and “OpenStack Swift” API calls, and they are generally worked over a RESTful interface. The RESTful interface is technical-speak, meaning; it talks over the web protocol to communicate these commands and the results. The ability to use the RESTful interface provides a lot of flexibility in which software you can and cannot use. It also opens up where you can access things to just about anywhere with internet access. This flexibility means, if you have a favorite API client that supports either Amazon S3 or OpenStack Swift, and your cloud storage supports it/them as well; you can use that client to talk to your cloud data.
The REST API also allows for you to provide a direct URL link to an object or bucket, providing simple access pretty much anybody can utilize. You can store a few files, or an entire website’s worth of files and use it for a Content Distribution Network to back your existing web solutions. The API helps you plumb it all together.
So is Object Storage useful to people who do not know how to handle an API? Of course!
There are many different Graphical clients that support Amazon S3 API and/or OpenStack Swift API. An example of this is the Cyberduck FTP client, which supports both methods natively. The user just sees an FTP client point-and-click interface, simple to use. The same goes for handing someone an object’s URL. They can open any web browser, punch in the URL, and download the object.
Aside from that, some cloud storage providers will also offer some type of web portal interface to interact with the data you have stored on the cluster. A lot of the time you can perform all the basic functions through this web interface as well as gather helpful information used for API access.
All of this works together to provide a storage solution that is great for plumbing into existing applications and environments, or where you need easy access for many people who may be spread out geographically.
Are there downsides to Object Storage? Just like anything else, with strength also comes weakness. The aforementioned strength in simplicity can also be a weakness. To have functionality with greater complexity, you generally have to lock yourself into a 3rd party application that supports what you want or program the functionality itself. This requires money, time, and expertise.
Another limitation is the type of access. If you are accessing data via Object Storage, it cannot also be accessed via block or file-based storage methods. Generally if you want to change the access method you wind up having to download your data, create a new bucket that uses the method you want, and then copy that data into the new bucket.
Last, network access. Without network access you won’t be able to reach your data. So if you are in an area that gets spotty internet or unreliable, you may find yourself struggling to reach your data. Note, this weakness applies to ALL cloud storage solutions, not just object storage. As you learned earlier though, weaknesses can also be strengths. If your local internet goes down, your data on the cloud is still accessible to anybody else who still with internet. Customers or remote employees are still able to get on and work with the data.
Now you know all about Object Storage: the pros, cons, and how it works. Part 3 of this series will take a good look at Block Storage, and finally, Part 4 will compare and contrast the two.