DataCore SANsymphony-V 10 PSP2 released

Yesterday DataCore released PSP2 for SSY-V 10 bringing (on the paper) some new cool features. These are:


  • OpenStack Integration in Cinder
  • Deduplication/Compression
  • Veeam Integration
  • Advanced Monitoring
  • Advanced iSCSI stack
  • Improvement of flash usage
  • some general improvements


From all these new features, deduplication and the Veeam integration were on my wish list for a long time. On last Monday there was the partner webinar where the new features got explaind a bit more in detail to the partner community. As expected, deduplication and the Veeam integration were the most interesting features headed by deduplication by far :-)

So looking at the new deduplication feature there are some really important things to know to understand how DataCore implemented this feature and how to compare it's effectiveness against other dedup engines on the market for years like NetApp or DataDomain just to mention some of them.

  • The deduplication engine in SSY-V is considered an EXPERIMENTAL feature so it is OK to use it in production but you should be aware of some general points regarding these features. From the release notes:
    "Experimental features are clearly labeled as such in the management console. These features have been implemented as experimental in order to gain an understanding of their continuing contribution to the software as case studies, analysis, and benchmarking continue to be developed. Features that carry an experimental label have been fully implemented and tested and are safe to use in production; however, the feature functionality is subject to change. Experimental features are supported and comments, suggestions, and issues may be reported to DataCore Technical Support as usual."
  • The dedup engine is not provided by DataCore itself. It uses the Windows Server 2012 R2 dedup engine. That means if you want to use dedup you have your OS SSY-V is running on on Server 2012 (R2 is highly recommended).
  • That is, in my opinion, the most critical part of the implementation. As it relys on the Windows dedup feature, you are also bound to it's limits when it comes to functionality and performance.
    • Windows dedup is only post-processing, no inline deduplication
    • Windows dedup is a single-threaded application. That means, if you only have a single datastore and want to scale the dedup performance beyond one CPU that's impossible. You can only scale by using multiple datastores. A datastore in SSY-V is called a deduplication pool and can only grow up to 1,6TB in size. There is a also a 1:1 relationship between vDisk and deduplication pool so if you want to scale dedup and size of your vdisks, you have to use multiple dedup pools and vdisks. There is no cross-pool deduplication.
    • General limitations exist for the Windows dedup engine. In Server 2012 the largest object to be deduped was limited to 1TB. With Server 2012 R2 the limit is raised to 2TB. This is probably the reason why DataCore only supports dedup in vDisks up to 1.6TB. So SSY-V will always depend on the limits of the Windows dedup engine.
    • Post processing requires a certain amount of memory to hold the dedup tables. Keep that in mind when sizing the RAM of your DCS. This RAM has to available to the OS and can't be used as cache for SSY-V. The amount of RAM needed scales with the number of deduplication pools you have and the schedule you plan to run the dedup process. Simultaneous dedup jobs require more RAM than only a single process running
    • Windows dedup is a background process with low priority. You can set it to high priority during certain time frames but doing this will slow down the access to the data on these disks massively. So high priority optimization is only practicable in times where nearly no access to the data occurs at all.
    • Microsoft specifies the throughput of the dedup engine of ~100GB/hr per volume which is 2,4TB/day. In high priority mode the performance could be higher but the 100GB/hr is already a quite highly estimated value. Tests showed ~50GB/hr. If your data change rate is higher than this, the dedup job will probably never complete.
    • Scheduling and maintaining windows dedup is currently not implemented in the SSY-V GUI. You have to do this via Powershell or Windows MMC.

Generally spoken, Windows dedup is not a bad thing. The engine is quite effective and can handle a good amount of changed data per day but the limits are obvious. So dreaming from a 100TB fully deduped backup store that has a change rate of several 100GB per day is still an illusion. But MS is known to improve things and DataCore surely has some good reasons to stick on the Windows dedup engine as there is no overhead in programming and supporting their own dedup engine.

The Veeam integration itself is also a scripted DataCore implementation, not to be confused with the Veeam storage integration that still only exist for NetApp and HP storage. DataCore uses their own scripting engine that is triggered by Veeam to bring the VMs in a consistent state (including Veeam AAIP), creating VMware snasphots, creating hardware storage snapshots, immediately releasing the VMware snapshot and mounting the hardware snapshot to the Veeam server. There will be, in fact, some difference in the functionality between the Veeam integration and the DataCore implementation but for standard snapshots without any additional functionality needs, this really speeds up the time a VM will be in snapshot mode.

For more specific information you can use the online help

Currently my testlab is down so I can't test dedup nor the Veeam integration but as soon as I have it up again I will post some more details and testresults about these two new technologies.

Leave your comments

Post comment as a guest

Your comments are subjected to administrator's moderation.
  • No comments found
Powered by Komento
joomla templatesfree joomla templatestemplate joomla
2017   globbers joomla template