What you need to know
- Microsoft services have been impacted by a global outage, reportedly caused by bugs affecting Crowdstrike.
- Crowdstrike has posted that the issue has been identified and isolated, and a fix has been deployed but many services need to restart and are stuck in a boot loop.
- Azure has been effected along with Microsoft 365 apps. Microsoft’s ‘fix’ for this instructs users to restart multiple times, “as many as 15,” to restore service.
It’s been a wild morning for services dependent on Microsoft, with widespread global IT outages some reporters are referring to as a “digital pandemic.” Planes have been grounded, bank account access frozen and even 911 operators affected by the Blue Screen of Death (BSOD).
The mass outage, caused by an issue with security software Crowdstrike, has reportedly been “identified and isolated” with a fix deployed, but for the fix to work, many services must reboot. The old “turn it off and on again” will save us, and if it doesn’t, don’t worry—Microsoft says just do it 15 times. Yes, really. Azure servers are affected, and consequently, Microsoft 365 apps, with the website’s most recent update instructs users to reboot as many times as it takes to get the fix working.
You can read the full update on Microsoft’s website, but this is the part we found particularly amusing.
“We have received reports of successful recovery from some customers attempting multiple Virtual Machine restart operations on affected Virtual Machines. Customers can attempt to do so as follows:
- Using the Azure Portal – attempting ‘Restart’ on affected VMs
- Using the Azure CLI or Azure Shell (https://shell.azure.com)
https://learn.microsoft.com/en-us/cli/azure/vm?view=azure-cli-latest#az-vm-restart
We’ve received feedback from customers that several reboots (as many as 15 have been reported) may be required, but overall feedback is that reboots are an effective troubleshooting step at this stage.”
Other solutions suggested by Microsoft are “forcing an unexpected reboot. See, the driver hooks the function by patching the system call table, so it’s not safe to unload it unless another thread’s about to jump in and do its stuff, and you don’t want to end up in the middle of invalid memory.” Or maybe that entire quote is from The IT Crowd, but the part about rebooting 15 times is actually real. If it works, it works!
Chaos around the world
Have you tried turning it off & on again? pic.twitter.com/7qYT4EncOqJuly 19, 2024
Crowdstrike is a cybersecurity firm based in the US that uses cloud-based AI and machine learning to prevent cyber attacks. It’s used by government agencies, airports, and banks, amongst many others, and the update to its Falcon Sensor software has caused chaos with Windows, Microsoft 365, Amazon, and even Instagram. Microsoft has since suffered a separate outage due to the configuration change in the back end of Azure’s settings, which has been fixed, but you may need to go through many restarts to see an improvement. What’s most worrying is how easy it’s been for this to happen. With so many systems across the world all dependent on the same software, turning it “off and on again” isn’t as simple as it sounds.
Meanwhile, not everyone is unhappy about the outages, and some workers have been celebrating the early Friday finish a lack of working IT equipment has gifted them. Or alternatively, being annoyed that their laptops are working amongst the chaos.
logged onto my work laptop and microsoft apps are still working pic.twitter.com/zIqrO1YmF8July 19, 2024
This is a developing story and we’re keeping tabs on it. Keep an eye on on our coverage for new updates, statements, and more.
