Potential Threats Stemming from CrowdStrike Outage

THE THREAT

On July 19th, 2024, a software update released by CrowdStrike led to widespread outages across the globe. This update caused a critical conflict with Windows OS, leading to system instability and crashes; specifically, the update inadvertently caused errors in the kernel mode driver, a core component of the Windows operating system, resulting in systems crashing to a "Blue Screen of Death" (BSOD). This has resulted in operational disruptions in various sectors including aviation, banking, IT, and other critical infrastructure.

Shortly following the announcement of the issue, eSentire identified suspicious domains that impersonated CrowdStrike support domains. These fraudulent sites aim to deceive users into believing they are accessing legitimate CrowdStrike support resources, potentially leading to further security incidents.

What we’re doing about it

• eSentire is actively monitoring these suspicious domains, and performing threat hunts across all clients for known Indicators of Compromise (IoCs)

What you should do about it

• Organizations can provide user awareness training to help employees identify any suspicious or malicious sites that may be masquerading as a CrowdStrike support or informational page
  • Organizations can also provide employees with an avenue to report any suspicious websites

Additional information

In tech alert to customers as well as in a public statement, CrowdStrike confirmed:

• Symptoms include hosts experiencing a bugcheck\blue screen error related to the Falcon Sensor.
• Windows hosts which have not been impacted do not require any action as the problematic channel file has been reverted.
• Windows hosts which are brought online after 0527 UTC will also not be impacted
• Hosts running Windows 7/2008 R2 are not impacted
• This issue is not impacting Mac- or Linux-based hosts
• Channel file "C-00000291*.sys" with timestamp of 0527 UTC or later is the reverted (good) version.
• Channel file "C-00000291*.sys" with timestamp of 0409 UTC is the problematic version.

Additionally, CrowdStrike provided the following workarounds to resolve the issue.

Workaround Steps for individual hosts:

• Reboot the host to give it an opportunity to download the reverted channel file. If the host crashes again, then:
  • Boot Windows into Safe Mode or the Windows Recovery Environment
    • Note: Putting the host on a wired network (as opposed to WiFi) and using Safe Mode with Networking can help remediation.
• Navigate to the %WINDIR%\System32\drivers\CrowdStrike directory
• Locate the file matching “C-00000291*.sys”, and delete it.
• Boot the host normally.

Note: Bitlocker-encrypted hosts may require a recovery key.

Additional Workaround for individual hosts:

Customers should restart the impacted host multiple times, forcing a race condition where the channel file which is impacting the issue, will be updated.

Workaround Steps for public cloud or similar environment including virtual:

Option 1:

• Detach the operating system disk volume from the impacted virtual server
• Create a snapshot or backup of the disk volume before proceeding further as a precaution against unintended changes
• Attach/mount the volume to a new virtual server
• Navigate to the %WINDIR%\System32\drivers\CrowdStrike directory
• Locate the file matching “C-00000291*.sys”, and delete it.
• Detach the volume from the new virtual server
• Reattach the fixed volume to the impacted virtual server

Option 2:

• Roll back to a snapshot before 0409 UTC.

Appendix

References:

[1] https://www.crowdstrike.com/blog/statement-on-falcon-content-update-for-windows-hosts/
[2] https://x.com/George_Kurtz/status/1814235001745027317
[3] https://x.com/George_Kurtz/status/1814316045185822981