In one of our latest projects I was faced with a new kind of demand on the central storage system. Normally customers ask for a fast, reliable, inexpensive storage system but this time, beside these standard demands, the customer needed features like multi-tenancy and a robust quality of service engine. The reason behind this is quite easy to understand. The customer has a central IT and several business units. The central IT is responsible for providing compute and storage ressources. Because of the different requirements from the business units they started to build their own infrastructure, especially in the storage area where the central IT was unable to deliver the services as the BUs needed. Major requirement from the BUs were high capacity, high performance CIFS/NFS shares that were used to store their critical data. Additionally, reliablity was also a key factor but costs should be lowered to a minimum (nothing new :-) ). As the central IT that was bound to high-price SAN-systems and thus was unable to deliver the services in an acceptable way, the BUs started to self-solve their problems by buying cheap NAS systems on their own. So there was a NAS sprawl within the company. The central IT was still responsible for management and esecially backup of this data but was unable to do their job as the NAS systems were bought without their knowledge.
During the last few months this sprawl goes overboard and the central IT was instructed to find a solution. So they asked us for a solution. They talked to big players like Dell, NetApp and EMC but in this special case, none of them really had a product that fits their needs. Sure, all can offer you solutions, more or less fitting in the tight frame of requirements but when it comes to cost they all dropped off.
We as a HP partner first thought about using a 3PAR system as it offers high reliablity, scalability, NAS services, multi-tenancy and even some kind of quality of service but not all together. NAS is still limited in functionality, scalability is cool but way to expensive and in some way the whole solution wasn't what the customer looked for.
A few weeks before I was in contact with some new vendor of software-defined storage systems. The company is called CloudByte and is located in India and the USA. They are pretty new on the market but had some really good ideas worth looking at. So I talked to them and the result was that their product "Elastistor" is exactly what the customer was looking at.
Let's have a deeper look into what CloudByte's ElastiStor is and what it makes different.
First of all ElastiStor is a Linux/Unix based storage solution that uses commodity hardware to run on. In theory you can use whatever hardware you want but there is a HCL on the vendors website that will tell you which hardware should be used because it is already known to work. There are also prebuild models that you can buy if you want a 100% supported and optimized solution.
On the storage side ElastiStor uses ZFS to build a high available and high performance storage solution. ZFS is a pretty "old" filesystem that comes from Solaris and is known to be very scalable and feature rich. There is inbuild functionality for compression, deduplication and Copy-on-write snapshots. Software RAID (RAIDZ) is used to get a higher level of availability for the physical disks. With the option to use SSDs for read and write cache you can scale the system to whatever performance level you need.
But ZFS is only half of the story. In this current level the whole system can act like a NAS but multi tenancy and quality of service is still not offered. That's where CloudByte enters the ring. On top of the OS there is a storage abstraction layer. Within this layer there are 1 or more TSMs (Tenant storage machines) that do the whole job against the storage users. So NAS/SAN functionality is build into the TSMs as well as the QoS service. This way each TSM is used for a special purpose or subunit within a company. This gives you the ability to delegate storage administration to the end user as they only get access to the TSM and not the main unit itself and you can also limit the amount of IOPS/bandwidth each TSM can use on a lower level.
Management itself is done via ElastiCenter, a web based application where you can do all that magic configuration stuff, provide ressources to TSMs, delegate permissions etc. This way you always have the full control over the storage subsystem but can delegate some permissions to tenants.
ElastiStor supports NFS, CIFS, iSCSI and FC but the first three are recommended. FC is only an emulation and shouldn't be used in a high performance environment but if you want to you can do.
The cool thing about this software is that you can build the storage with commodity hardware thus reducing the implementation costs extremely. We saw offers for a 100TB solution in a high performance environment <150k Euro (including all hardware and software licenses), that was less than a half the cheapest competitor could offer. So here the software defined storage approach really saves you a lot of money.
But ElastiStor isn't all about saving money. The QoS feature and the multi tenancy are the real benefit from this solution. No other solution I ever found had implemented this concept in such a decent way as CloudByte did.