TLS protocol support by the Rawx service
Server-side on-the-fly data encryption, performed at the gateway using a private key provided by the user, has been available since OpenIO version 18.10
However, encryption is a process that is not widely used because of its cost in terms of CPU and the latency that the encryption/decryption calculations (reduction of the throughput through the gateway) induce when writing and reading data.
To improve OpenIO's security, and to avoid man-in-the-middle attacks which consist of intercepting communications between the internal services of an infrastructure and the gateways present on another network, the rawx service, rewritten in the Go language since the 18.10 release, now supports Transport Layer Security (formerly SSL), to establish secure https connections.
Implementation of IAM (Identity and Access Management)
For the management of authentication tokens, OpenIO uses the Keystone brick from the Openstack project. In addition, and to benefit from a more granular management of the rights of each user (read / write / modification / access to metadata...), OpenIO is now compatible with the S3 IAM for the most widespread needs.
It is now possible to define compartment and user strategies for each bucket, based on groups of users, who are assigned a series of rights / restrictions.
Knowing bucket size
The bucket is the "compartmentalization unit" of a storage space set up by the S3 API. A bucket generally corresponds to the storage space of an application. OpenIO was designed using "containers", which are spaces traditionally assigned to a user. To be compatible with the S3 API, OpenIO is able to create virtual buckets by federating a set of containers according to criteria defined by the platform administrator. It is now possible, with the CLI command openio bucket show <bucket-name>, to know the size of a bucket and the number of objects it contains. You can use this information to monitor the uses and consumption of the different users/projects.
Implementation of Bucket Level Replication
Until now, asynchronous replication of an OpenIO cluster to one or more sites involved replicating the entire namespace (i.e. all the data contained on the cluster). The "Replicator" service now allows you to choose the buckets to be replicated, according to their criticality, in order to optimize the allocation of your resources and the cost of your PRA/PCA.
Improvement of the Real-time monitoring stack and data visualization
To provide statistics on the use of an OpenIO platform, we have set up a system based on several components: collectors (Netdata and Blackbox export) that push the metrics of the servers into an aggregator (Prometheus), which are visualized thanks to Grafana.
Until now, we used Netdata plugins to push the metrics specific to OpenIO. Netdata excels at reporting system statistics, but the plugins were too restrictive for our use. So we decided to group together everything related to monitoring and surveillance of OpenIO bricks by an in-house tool called "OIO export". Thus, the OpenIO monitoring data collection is more accurate, more complete and more easily scalable. We took the opportunity to update Grafana to version 7, and we redesigned our dashboards to be more complete and readable.
Update of the WebUI
Within the WebUI, we added a dashboard of the cluster's key indicators, including those useful for capacity planning (projections based on the growth of the platform). We are counting on users to enrich and develop this interface. Do not hesitate to give us your feedback!
From now on, the WebUI can also launch Meta 2 rebuilds and de-commissions in one click.
Multi-part upload (MPU) is increasingly used by applications to parallel the deposit of large files or to manage connection loss on the user side. As a result, we have optimized MPU management by removing all unnecessary calls in our implementation of the S3 API to significantly speed up file upload times.
Finally, we have implemented a cache for Keystone tokens to significantly reduce the volume of requests during data transfer operations and improve response times.
OpenIO Certified as Compatible with IBM Spectrum Protect Plus
IBM Spectrum Protect Plus is Big Blue's data protection solution. It is a virtual appliance for disaster recovery through near-instantaneous replication, retention and recovery of virtual machines, databases and containers in a hybrid multi-cloud environment. Learn more