| I've important information on a couple of major Veeam Backup & Replication bugs discovered last week, so please take time to study the below carefully.
The first bug is very annoying, because it will impact everyone who's using the default self-signed certificates, which is unfortunately the majority. As a reminder, we use those certificates to implement secure communication between backup infrastructure components, as well as with managed backup agents. And while we do provide the ability to select or import your own certificate, most don't worry about this and just keep the default certificate that is automatically generated when you install Veeam Backup & Replication. This certificate is set to expire in 1 year from its creation date, and due to some bugs you will see artifacts of its expiration 1 year after your Update 3 installation date. Which means, this will hit many of you in the next few weeks, and may make it a not very merry Christmas, unless you act now!
Assuming you have Update 3a installed, first thing you will see at 11 months after Update 3 installation time will be the "Failed to check certificate expiration date" error message upon opening the backup console. The UI is trying to tell you that the certificate is about to expire, but the logic of this falls through to the universal message for all unhandled exceptions (which is why it does not make much sense). If you ignore this message, everything will continue to work fine for another month, after which the agent management functionality, as well as all granular restores will start failing. Luckily, the issue is super easy to fix by simply generating a new certificate, which takes just a few clicks. So don't wait, and do it at your earliest convenience. Needless to say, we've fixed the related bugs in the Update 4 (and also bumped the self-signed certificate expiration date to 10 years). Here's the official Veeam support KB article for this issue > KB2806
Moving on to the second issue. This one is a data loss bug that is only known to impact environments with IBM storage powered by Spectrum Virtualize software (such as IBM SVC or IBM Storwize) when backup from storage snapshots is used. And while the issue is really hard to run into (because it depends on other storage infrastructure issues), it is very nasty because it may result in freshly created production LUN deleted by Veeam. The root cause of the bug is the fact that Spectrum Virtualize may apparently reuse LUN IDs, instead of leveraging unique identifiers such as GUID or USN. In one particular environment, our cleanup process was unable to delete the storage snapshot LUN we created for quite some time due to an infrastructure issue. This process kept retrying until the customer created a new production LUN, which was assigned the ID that was previously used by the "problematic" snapshot LUN. And as you may have already guessed, that new LUN was then promptly deleted by our cleanup process. So while I said "data loss", normally you won't actually have enough time to create data on that new LUN before it is nuked. Nevertheless, we take this issue very seriously and recommend all customers who use storage snapshot integration with IBM storage to contact Veeam support, so that we could provide you the hot fix as soon as it is released. We'll also contact vendors of all other primary storage we integrate with to confirm they ARE using unique identifiers for their LUNs.
Nutanix AHV users: we've made the Update 1 (build 1.0.457) for our AHV proxy appliance generally available last week. This fixes a number of bugs (see Fixed Issues section), and particularly data corruption due to changed block tracking bug following the source VM disk resize, which obviously makes it a must-have update.
Microsoft Hyper-V users: in addition to the "triple failure" Hyper-V 2019 issue I shared in the previous digest, there're now a couple more to be aware of. The first one is quite big: enabling Hyper-V role in the Server Manager makes some servers go into the endless boot loop with Server 2019, rendering them unusable. I've seen a number of posts on this issue across various communities, and for now the issue looks to be specific to Dell hardware only. The second one is something we found ourselves during Server 2019 support regression testing, and it's quite funny: if you perform an upgrade of virtual hardware for version 5.0 VM on a Hyper-V 2019 host, you will get multiple VMs created by Hyper-V based on the number of checkpoints on the original VM. But in the end it was not funny at all, because those clones consumed all available datastore space, causing all other VMs running off of the same datastore to stop.
Finally, some observations on Amazon re:Invent last week. I did not attend the event, but I followed the announcements carefully, and a couple of them really stood out for me. First, Amazon has officially announced the re:invented (pun intended) Glacier API, which is now unified with S3 API. We already knew through our partner channel this change was coming – and this was actually the reason why we postponed Glacier integration from Update 4. What we did not know however is how well some of the new Glacier API capabilities map into our unique approach of extending backup files into the cloud – unexpected and huge win for us here, really exciting!
And the second notable announcement was Amazon Outpost, which is essentially Amazon's answer to Microsoft Azure Stack. While this announcement is huge for both Amazon and VMware, I believe the biggest winner here is actually Microsoft still, who have already been on this road for a few years now with their Azure Stack. I mean, it's hard for Microsoft to wish for a better endorsement of their hybrid cloud strategy (and proof of their thought leadership) than two of the world's largest cloud companies deciding to follow their steps and playing catch up! And of course, it's great for us customers too, because Azure Stack will no longer be the only solution to choose from – even if being the first to market, it will still have an edge in terms of ecosystem support. Notably, one of the lesser known features of our upcoming Update 4 is direct restore of any backup to Microsoft Azure Stack. |
Komentáře
Okomentovat