Rubrik : Bug on AHV Export




Background

When we had an issue on an AHV VM, the first reflex was to restore it from last night backup. It was very quick and we were all happy ... unless we realized that no networking was connected. We opened a case with Nutanix to troubleshoot the issue and ....

We have found a bug !

After a couple of investigation and forensic, we have discovered the below phenomenon.

Here is a sample of the faulty VM with the original VLAN order :

Here is the exported VM VLAN order :


We can see the difference, the VLAN ID order is inverted. It was not very obvious at first, but when looking at the ARP table on the hypervisor managing that VM, we have seen that MAC address were inverted.
Now, we need to understand what's wrong and how to solve it. Indeed, what's wrong ? The export itself ? or the backups ? If this is the backups it means all our backups are corrupt since a long time and this is serious.

Other option, it can be an issue when the newly exported VM is created. In that case, a hotfix can definitely help.

At this stage we can be sure we have found a bug and not a small one. It could affect other customers with different VLANs created the same way as we did.

We have involved Rubrik support and it has been escalated to Engineering department. After some times reviewing the code they confirmed there is indeed a "sort" instruction on the export module that should be removed. They managed to correct the code, do the Q/A and we have been delivered with a hotfix. The internal issue number is CDM-271956.

The hotfix has been installed today (2nd Nov 2021) and the exact CDM version number is 6.0.1-12877_HP798644

The hotfix can only be applied by Rubrik support and it will be incorporated into any CDM release starting from 6.0.2.

I can confirm the hotfix is indeed solving our issue on restoring AHV VMs with multiple vNIC. The AHV version does not matter, this is not an issue on the Nutanix side.

I hope this is also helping other customers.




Comments

Post a Comment

Thank you for your message, it has been sent to the moderator for review...

What's hot ?

ShredOS : HDD degaussing with style

Image
Background Have you ever had the need to properly erase HDD or SSD before disposing them ? Or selling them ? You do not want to have someone else playing around with your data right ? What we usually do is using a bootable Linux Live CD and use dd style command to write random data on the disk right ? But what if you have 10 disks in your machine ? What if you need a report to proof the world that wiping is a success ? I came across this challenge recently.
Read more

Mac OS X : Display images in-line with terminal

Image
Backgound This time, this is a MacOS tip ! You have a list of files in the terminal window in front of you, some are pictures, some are texts. For text files, this is easy to quickly issue a "more" or "cat" so you immediately figure out what it is. For a picture, this is another story, you need an image viewer. Whatever the speed and simplicity of this editor, you need to load a 3rd party application and switch to it, this is not very handy. What if you could see this image directly from the terminal ? 
Read more