Three recent studies confirm that Object Storage is finally adapted to high-performance use casesLire la version française
Until recently, it was normal to have to arbitrate between cost, scalability and performance when choosing a storage system. And this trade-off was complex. The volume of data collected by enterprises continues to grow: according to IDC, between 2018 and 2023 the average annual growth rate will exceed 25% to reach a global datasphere of approximately 102.6 zettabytes. This non-stop increase in data is placing larger demands on storage systems as companies endeavor to extract the value from their data.
Thanks to the accessibility of computing resources (distributed computing clusters such as Hadoop and GPUs) and the democratization of algorithms (machine learning, deep learning, etc.), companies can finally reap the benefits of Big Data. In this context, Object Storage, which until now was considered as an archiving solution (active archive or long-term archive) must be reconsidered. Analysts such as IDC and GigaOm already recognize that second generation Object Storage solutions such as OpenIO can deliver the high-performance required for Big Data use cases.
Growth in the use of Object Storage in companies that want to take advantage of Big Data
More and more organizations are using Object Storage alongside their Big Data processing platforms. In its December 2019 report, IDC Innovators: Open Source Object Storage for High-Performance Workloads, 2019 [paid access], IDC cites three “innovators” in this category: OpenIO, SoftIron and MinIO. All three develop open source object storage solutions.
“Ceph dominates the market across all industries and customer sizes, with a focus on archiving,” said Amita Potnis, research director, Infrastructure Systems, Platforms, and Technologies Group at IDC. “The market now has more open source options to consider as startups such as MinIO, SoftIron, and OpenIO develop object-oriented storage technologies specifically designed for high-performance environments.”
OpenIO also appears in the latest edition of the IDC MarketScape: “Worldwide Object-Based Storage 2019 Vendor Assessment”, as shown in the following graph.
In an earlier paper, Consider Object Storage for High-Performance Use Cases [available for free consultation], IDC also referred to the explosion in the use of object-based storage in companies that use AI, machine learning, deep learning, and/or connected object fleets (IoT) techniques. “These new uses are at the origin of both the growth of data (in many cases to hundreds of petabytes or even exabytes) and the need for real-time analysis. Cost and performance are immediate concerns for any organization with such projects.”
Object Storage is now viable for use cases requiring high performance (Big Data, Artificial Intelligence, Machine Learning, Deep Learning…), according to IDC. Access the IDC Technology Spotlight study for free!
How to distinguish between Object Storage solutions
In his report Key Criteria for Evaluating Enterprise Object Storage, published in November 2019, GigaOm analyst Enrico Signoretti reviewed 12 competing technologies: Caringo Swarm, Cloudian HyperStore, Dell EMC ECS, Hitachi Vantara HCP, IBM Cloud Object Store, Minio, NetApp StorageGRID, OpenIO, Red Hat Ceph, Scality RING, SwiftStack and Western Digital ActiveScale.
The GigaOm report highlights the advantages and disadvantages of choosing to implement these vendors’ object storage solutions, which all rely on different erasure coding algorithms to ensure data protection.
OpenIO stands out for our efficiency and is ranked in the high-performance segment. These qualities were showcased with the #TbpsChallenge, where OpenIO demonstrated our performance by blasting through the Terabit per second write speed, to set a record of 1,372 Tbps on an infrastructure provided by Criteo Labs.
Performance is becoming a criterion of choice for storage systems
These different studies all agree that a new generation of Object Storage technologies is emerging. OpenIO and MinIO stand out from the crowd as two players engaged in a “war of speed”, as storage specialist Chris Mellor put it in November.
While performance has long been a (difficult to verify) marketing promise, it is now becoming the criterion around which the Object Storage market is reshaping itself. And for good reason! Even though legal archiving of certain types of data is still required, solutions such as public cloud storage, first generation object storage and even tape storage are low cost and more than adequate for this use. However, as soon as data is intensively used, these systems show their limits, mainly in terms of bandwidth and scalability.
Now’s the time to educate the market on performance, which is more complex than it seems since it can be understood in three dimensions, as we wrote in a previous blogpost: “The Performance of Storage Systems: 3 Criteria to Take into Account“.