Data Storage: File, Block, Object?

Everyone ‘knows’ the history of data storage and the explosion in capacities and speed that have been occurring over the years; from the old punch cards to hard drives, CD/DVD, USB, NVMe and all the ‘stuff’ in-between; SATA, SSD, Flash, etc. Even Tape is still around – it’s not dead yet, in fact it’s still growing!

Data is critical to all organizations; being able to share, store and secure data efficiently (and at a reasonable cost) is crucial for businesses to succeed, grow, gain a competitive edge and keep their customers happy.

Customer expectations today are “I want it now” and “find it quick”. Physical hard drives can only spin so fast and can only be ‘packed’ with so much data. In comes the SSD and NVMe drives which, with albeit smaller capacities, made huge improvements in the speed of access.

Less well known is the other ‘stuff’ – how we store and access the data as capacities and speeds increase. The hardware has been changing, allowing higher densities and improved access time, but less well known is the ‘data structure’ changes occurring over time as well. That’s where File, Block, and Object Storage comes into play.

File-based Storage you are probably more familiar with; file-based storage using a tree-like directory structure where data is stored in folders, subfolders and more subfolders (think Windows File Explorer). Like a file room with multiple cabinets, drawers, folders, and subfolders inside. The file room probably only has one door in, so there’s only one path to the data which can impact performance if more than a few people are trying to get in (not very good for streaming videos).

File Storage functions similarly to a local hard drive on your computer, or for file storage beyond the capabilities of your desktop or laptop, utilize Network Attached Storage (NAS) devices where you ‘map’ a network drive to your computer.

Block Storage is exactly that, data chopped up into equal size blocks and stored as separate pieces, each with a unique identifier that allows the SAN (Storage Area Network) to place the blocks wherever it is most efficient. Because the data is stored separately and in blocks, the data is accessible from multiple paths and multiple environments with high performance, no single door into the room.

Block storage is really good for data that is frequently accessed and/or updated. Think database or transactional applications which require high IOPS and low latency like Stock Exchanges or Financial applications.

Object Storage uses a flat structure where files are broken into pieces (objects) and spread out with detailed metadata describing the data; where a video was shot, the date and time it was created, the author, actors, etc. Metadata can be extremely detailed and is used by the Storage Operating System for searches and to find the data quickly. Think about parking your car in a huge valet parking lot – the valet gives you a ticket and parks your car. You don’t care where your car is, just that it’s safe and accessible. When you come back to get your car, the valet checks the ticket (metadata) for the location and can retrieve your car (data) easily, regardless of how large the parking lot is.

The ‘Objects’ in Object Storage are considered complete units and can only be updated and rewritten as complete objects, you have to write the complete object at once, and writing an object can be a slow process. Object Storage is great for large quantities of unstructured, unchanging/static data like music, images, videos, backups, historical, regulatory data & archived files.

These technology improvements have created opportunities for ‘software-defined’, ‘as-a-service’ and ‘Cloud’ solutions from providers like Amazon, Google, Microsoft, Oracle, IBM, Red Hat, and more. They offer solutions across all Storage types (File, Block, Object) running on hardware like Dell EMC Centera/Atmos/ECS for Object, XtremIO for Block, and, Celera/Isilon for File-based (NAS) storage.

Regardless of which solutions or hardware vendor you utilize to share, store and secure data efficiently (and at a reasonable cost) organizations need to ensure that IT is an enabler for future growth. Online Streaming and Content has exploded in recent years – especially in the last eighteen months (think Healthcare, Education & Entertainment) – where will it go from here?