image1 image1 image1

Critical bug in vSphere Changed Block Tracking

Recently VMware published a new KB article containing information abaout a critical bug within their Changed Block Tracking (CBT) technology. As CBT is widely used by VMware and 3rd party backup tools like Veeam or vRanger, this bug has probably major impact on most of the installation worldwide.

CBT is used to track changes made on block level since the last mark was set by a backup tool. This way, the backup application doesn't have to "scan" the whole vmdk for changed blocks but rather only reads the CBT database to quickly know which blocks have to be backed up.

If there is a bug within this functionalilty, backup taken with CBT can be in worst cases useless.

To make things worse, this bug is "included" in every ESXi version since the CBT functionality was released.

 

Who should be concerned about this bug? Currently only vmdks that were expanded after initial placement and the new size is now bigger than 128GB. The file itself is consistent but incremental backups taken by any CBT enabled backup application will be inconsistent. If you have to restore the vmdk from such a backup your data will be lost.

The reason is that the CBT information, in this special context, is incorrectly recalculated and thus making backups unusable.

The good news: Veeam is working on a patch for v7 and v8 to disable and reenable CBT for affected disks making the feature working again.

The bad news: VMware has currently no patch available to fix that problem. So if you belong the the group of customers that have disks expanded from below 128GB to above 128GB you have to manually disable and reenable CBT for these disks to make CBT work again.

If you are unsure which disks are affected you can disable CBT for all your VMs and let the backup application reenable it with the next backup pass. This will make your next backups take significantly more time than usually because now your backup application has to do a full scan of all blocks to determine which have changed and need to be backed up. But this is an acceptable price to pay to have consistent and recoverable backups.

Veeam customers using SureBackup can use this feature to check if their OS disks are consistent. Data disks that are affected can't be automatically checked by SureBackup as the job only tests that the VM boots up and perhaps makes an application test but if you have data on an expanded disk that is not your boot disk and is not neccessarily needed by your application, SureBackup won't report problems.   

 

Leave your comments

Post comment as a guest

0
Your comments are subjected to administrator's moderation.
  • No comments found
Powered by Komento
joomla templatesfree joomla templatestemplate joomla
2017  v-strange.de   globbers joomla template