On March 2, the security community became aware of four critical zero-day Microsoft Exchange Server vulnerabilities. These vulnerabilities let adversaries access Exchange Servers and potentially gain long-term access to victims’ environments. While the Microsoft Threat Intelligence Centre, MSTIC, attributes the initial campaign to Hafnium, a group they assess to be state-sponsored and operating out of China, multiple threat intelligence teams are also seeing multiple threat actors now exploiting these zero-day vulnerabilities in the wild.
Attackers are now targeting vulnerable Exchange Servers with DearCry ransomware. Phillip Misner of Microsoft’s security team tweeted that Microsoft observed a new family of human operated ransomware attack customers – detected as Ransom:Win32/DoejoCrypt.A. Human operated ransomware attacks are utilising the Microsoft Exchange vulnerabilities to exploit customers.
Below are some of the top comments form industry leaders on the breach and some remediation tips.
Andrew Brandt, Principal Threat Researchers, Sophos
While some of the attacks looking to take advantage of the ProxyLogon Exchange vulnerabilities took a week or so to emerge, the same cannot be said for crypto-miners. They were hitting vulnerable servers with their payloads within hours of the bugs being reported and security updates released.
QuickCPU, a variant of the xmr-stak Monero crypto-miner is no exception, our analysis of this campaign shows mining value flowing to the attackers’ Monero wallet on March 9, with the attack diminishing rapidly in scale thereafter. This suggests we are looking at yet another rapidly compiled, opportunistic, and possibly experimental attack attempting to make some easy money before widespread patching takes place.
What makes this attack unusual is the fact that the operators installed their crypto-mining payload on an infected Exchange server and then used that as a platform to spread the malicious miners to other infected servers. The attackers implemented a range of standard anti-detection techniques, installing the malicious miner in memory to keep it hidden from security scans, deleting the installation and configuration files after use, and using the traffic encryption of Transport Layer Security to communicate with their Monero wallet. As a result, for most victims the first sign of compromise is likely to be a significant drop in processing power. Servers that remain unpatched could be compromised for quite some time before this becomes clear.
Defenders should take urgent steps to install Microsoft’s patches to prevent exploitation of their Exchange Server. However, patching is not enough on its own, organisations need to determine and address their wider exposure, so they don’t remain vulnerable to later attacks. For instance, admins should scan the Exchange server for web shells and monitor servers for any unusual processes that appear seemingly out of nowhere.
High processor usage by an unfamiliar program could be a sign of crypto-mining activity or ransomware. If this is not possible, closely monitor the server until you migrate the Exchange data to an updated server then disconnect the unpatched server from the internet.
Mat Gangwer, Senior Director, Sophos Managed Threat Response
These vulnerabilities are significant and need to be taken seriously. They allow attackers to remotely execute commands on these servers without the need for credentials, and any threat actor could potentially abuse them. The broad installation of Exchange and its exposure to the internet means that many organisations running an on-premises Exchange server could be at risk.
Attackers are actively exploiting these vulnerabilities with the primary technique being the deployment of web shells. This, if unaddressed could allow the threat actor to remotely execute commands for as long as the web shell is present.
Organisations running an on-premises Exchange server should assume they are impacted, and first and foremost patch their Exchange devices and confirm the updates have been successful. However, simply applying patches will not remove artifacts from your network that pre-date the patch. Organisations need human eyes and intelligence to determine whether they have been impacted and to what extent, and, most importantly to neutralise the attack and remove the adversary from their networks.
Organisations should review the server logs for signs that an attacker may have exploited their Exchange server. Many of the current known indicators of compromise are web shell-based, so there will be file remnants left in the Exchange server. An overview of files and any modifications to them is therefore important. If you have an endpoint detection and response, EDR, product installed, you can also review logs and process command execution.
If you find any anomalous or suspicious activity, you should determine your exposure as this will allow you to decide what to do next. You need to understand how long or impactful this activity may have been. What is the gap between appearance of the web shell or other artifacts in your network and the moment of patching or discovery? This is often a good time to ask for external support if you’re not sure what to do. Third-party forensic and incident response can be vital at this stage, providing experienced threat hunting and human intelligence that can dive deep into your network and find the attackers.
The Sophos Managed Threat Response team is actively hunting and investigating customer environments to see if we can uncover new artefacts or indicators of compromise that can be used to boost detection and defense against this threat. Our 24/7 incident response team is already supporting organisations that believe they may have been attacked and this will also help us to gather more intelligence about this threat and how to protect against it.
Mark Loman, Director, Engineering Technology Office, Sophos
From an encryption-behaviour view, DearCry is what Sophos ransomware experts call a Copy ransomware. It creates encrypted copies of the attacked files and deletes the originals. This causes the encrypted files to be stored on different logical sectors, allowing victims to potentially recover some data, depending on when Windows reuses the freed logical sectors.
More notorious human-operated ransomware like Ryuk, REvil, BitPaymer, Maze and Clop, are In-Place ransomware, where the attack causes the encrypted file to be stored on logically the same sectors, making recovery via undelete tools impossible.
DearCry’s encryption is based on a public-key cryptosystem. The public encryption key is embedded in the ransomware binary, meaning it does not need to contact the attacker’s command-and-control server to encrypt your files. Exchange Servers that are setup to only allow internet access for the Exchange services will still become encrypted. Without the decryption key, which is in possession of the attacker, decryption is not possible.
Interestingly, WannaCry was also a Copy ransomware. DearCry not only shares a similar name, but also has an eerily similar file header. Defenders should take urgent steps to install Microsoft’s patches to prevent exploitation of their Microsoft Exchange patches. If this is not possible, the server should be disconnected from the internet or closely monitored by a threat response team. We expected that attackers would take advantage of the Exchange/ProxyLogon issues, and this could be the first of the subsequent attacks.
John Hultquist, VP of Analysis, Mandiant Threat Intelligence
We are anticipating more exploitation of the exchange vulnerabilities by ransomware actors in the near term. Though many of the still unpatched organisations may have been exploited by cyber espionage actors, criminal ransomware operations may pose a greater risk as they disrupt organisations and even extort victims by releasing stolen emails. Ransomware operators can monetise their access by encrypting emails or threatening to leak them, a tactic they have recently adopted.
This attack vector may be particularly attractive to ransomware operators because it is an especially efficient means of gaining domain admin access. That access enables them to deploy encryption across the enterprise. In cases where organisations are unpatched, these vulnerabilities will provide criminals a faster path to success.
Unfortunately, many of the remaining vulnerable organisations will be small and medium sized businesses, state and local government, and schools, which will struggle to keep up with the deluge of actors leveraging this increasingly available exploit.
Lotem Finkelsteen, Director of Threat Intelligence, Check Point
Since the recently disclosed vulnerabilities on Microsoft Exchange Servers, a full race has started amongst hackers and security professionals. Global experts are using massive preventative efforts to combat hackers who are working day-in and day-out to produce an exploit that can successfully leverage the remote code execution vulnerabilities in Microsoft Exchange.
Check Point Research, CPR, has outlined the disclosed vulnerabilities, the targeted organisations by country and industry, and then recommendations to prevent the attacks, which are yet to come.
Of the targeted organisations, 17% belong to the Government and Military sectors and 14% are in manufacturing. Looking at the attack from a geographical perspective, the most targeted country was Turkey, 19%, followed by the US, 18%, and Italy, 10%.
Background
On March 3, 2021 Microsoft released an emergency patch for its Exchange Server product, the most popular mail server worldwide. All incoming and outgoing emails, calendar invitations and virtually anything accessed within Outlook goes through the Exchange server.
Orange Tsai (Cheng-Da Tsai) from DEVCORE, a security firm based in Taiwan, reported two vulnerabilities in January. Unware of the full magnitude of these findings, Microsoft was prompted to further investigate their Exchange server. The investigation uncovered five more critical vulnerabilities.
The vulnerabilities allow an attacker to read emails from an Exchange server without authentication or accessing an individual’s email account. Further vulnerability chaining enables attackers to completely take over the mail server itself.
Once an attacker takes over the Exchange server, they can open the network to the internet and access it remotely. As many Exchange servers have internet exposer specifically Outlook Web Access feature and are integrated within the broader network, this poses a critical security risk for millions of organisations.
If your organisation’s Microsoft Exchange server is exposed to the Internet, and has not been updated with the latest patches nor protected by a third party software such as Check Point, then you should assume the server is completely compromised. Compromised servers could enable an unauthorised attacker to extract your corporate emails and execute malicious code inside your organisation with high privileges.
Technical explanation
- CVE-2021-26855 is a server-side request forgery, SSRF, vulnerability in Exchange which allows the attacker to send arbitrary HTTP requests and authenticate as the Exchange server.
- CVE-2021-26857 is an insecure deserialisation vulnerability in the Unified Messaging service. Insecure deserialisation is where untrusted user-controllable data is desterilised by a program. Exploiting this vulnerability gives Hafnium the ability to run code as SYSTEM on the Exchange server. This requires administrator permission or another vulnerability to exploit.
- CVE-2021-26858 is a post-authentication arbitrary file write vulnerability in Exchange. If Hafnium could authenticate with the Exchange server then they could use this vulnerability to write a file to any path on the server. They could authenticate by exploiting the CVE-2021-26855 SSRF vulnerability or by compromising a legitimate admin’s credentials.
- CVE-2021-27065 is a post-authentication arbitrary file write vulnerability in Exchange. If HAFNIUM could authenticate with the Exchange server then they could use this vulnerability to write a file to any path on the server. They could authenticate by exploiting the CVE-2021-26855 SSRF vulnerability or by compromising a legitimate admin’s credentials.
In this attack, as in Sunburst, a particularly common platform was used as a front door for covert entry and prolonged stay within the network. The good news is that only highly skilled and well-financed threat actors are capable of using the front door to potentially enter tens of thousands of organisations worldwide. While hacking the exchange server with zero days is quite impressive, the purpose of the attack and what cybercriminals wanted within the network is still unknown. Organisations who are at risk should not only take preventive actions on their Exchange, but also scan their networks for live threats and assess all assets.
Immediately update all Microsoft Exchange Servers to the latest patched versions available by Microsoft. This update is not automatic and you are expected to perform it manually.
Morey Haber, CTO & CISO, BeyondTrust
There is nothing new with zero-day attacks. They happen all the time and have been associated with worms, bots, ransomware, and a plethora of other cybercrimes and malware. Every time one occurs, the news highlights risks and the fallout.
With recent attacks infiltrating the supply chain and ransomware now infecting hypervisors without any end-user interaction, businesses must step up their security to thwart modern attacks but at a risk of becoming numb and complacent to the constant onslaught of exploits. Realistically, many security professionals are burning out from the barrage of attacks and never-ending responses. Unfortunately, we now have another exploit to deal with and the damage is devastating.
The target is Microsoft Exchange Servers. At first, the vulnerability exploit combination compromised approximately 30,000 unique businesses worldwide. On March 8, the count exceeded 260,000 unique organisations worldwide and the impact is still growing.
The vulnerability itself has been patched but businesses have been slow in rolling out the patch and like any critical patch, downtime is required to implement in the form of a reboot. This is not just a simple reboot but for highly regulated organisations unscheduled downtime and potentially the forensics and reporting associated with a breach and clean-up. While patching is a standard practice for most organisations, the clean-up of an infected system is not, and it adds to security professionals’ already heavy workloads.
The attack has been allegedly traced to a Chinese hacking group however like the early days of any mass breach, there are conflicting reports. And, to add to the breach, infected systems can be accessed by the hacker via a web shell that has been added to the Exchange Server allowing them to enter commands via a web browser. The clean-up from this style of massive breach is costly and generally requires the reinstallation of the entire environment.
For any organisation continuing to use on-premise Exchange Servers in lieu of services like Office 365, it is time to stop what you are doing and patch those systems immediately. In addition, Microsoft has released tools to identify infected systems to help the clean-up effort.
The motives of the threat actors are unclear, but one thing is certain. They are wearing our security professionals down even further and we have yet another massive breach to clean-up. The damage and source of the attack will only become more apparent in the next few days. Welcome to 2021. The year of massive cyber-attacks.
Oliver Tavakoli, CTO, Vectra AI
The hack involved the combined exploitation of multiple zero-day vulnerabilities, starting with an OWA SSRF vulnerability and then proceeding to the exploitation of other vulnerability to burrow deeper into the inner workings of the server.
Patching the Exchange Servers will prevent an attack if their Exchange server has not already been compromised. But it will not undo the foothold attackers have on already compromised Exchange Servers. Microsoft has published a technical blog on how to recognise signs that an Exchange Server is already compromised.
Remediation will not be simple, it will effectively require backing up data, re-imaging the Exchange Server, scrubbing the backup of any accounts which should not be present, resetting all passwords and secrets and restoring the remaining backup data.
Complex software that has been around for a long time will almost invariably contain flaws which given sufficient motivation, resources and skill will be discovered and exploited. The key to resilience in these cases is to have the capability to detect downstream activity necessary to capitalise on the foothold gained. The good news is that this activity almost always follows more standard tradecraft which can be detected by Network Detection and Response products.
Ed Hunter, CISO, Infoblox
SolarWinds and now Hafnium show the importance of arming networking and security teams with defence in-depth tools to spot threats sooner. This is crucial in the age of the persistent breach where nation state actors are using cyber as a soft power alternative to diplomacy and military strikes, and where companies are getting hit as collateral damage. That the White House has formed a task force for Hafnium victims underscores the high stakes of preventing further fallout.
A helping hand for US federal network and security teams is on the way in the form of the approximately $14 Billion earmark in the newest stimulus bill for agencies to modernise core technologies. That funding will provide a boost for these organisations who face constant cyberattack from some of the world’s most elite hackers. Meanwhile, security practitioners in the public and private sectors continue to pivot to a zero-trust strategy to better detect these well-cloaked intrusions.
Unit 42, Threat Intelligence Team at Palo Alto Networks
Below is a concise playbook that enterprises can follow to respond to this potential threat in their environments.
#1 Locate all Exchange Servers and determine whether they need to be patched. Exchange Online is not affected. Vulnerable Exchange Server versions include 2013, 2016, and 2019. While Exchange 2010 is not vulnerable to the same attack chain as Exchange 2013, 2016, 2019, Microsoft has released a patch for CVE-2021-26857 for this version of the software.
Microsoft has recently released additional guidance for older, unsupported versions of Exchange. Microsoft is recommending to install updates on all Exchange Servers, prioritising those that are external or internet facing. Even if Exchange Servers are not internet facing, the vulnerabilities can still be exploited if access to the network has been achieved through other methods.
#2 Patch and secure all Exchange Servers. Install the out-of-band security updates for your version of Exchange Server. If you cannot update or patch an Exchange Server immediately, there are some mitigations and workarounds that may reduce the chances of an attacker exploiting an Exchange Server; these mitigations should only be temporary until patching can be completed.
The initial attack requires the ability to make an untrusted connection to Exchange Server port 443. You can protect against this by restricting access to the system from untrusted users. This can be achieved by only allowing access to the system from users who have already authenticated through a VPN, or by using a firewall to limit access to specific hosts or IP ranges.
Using this mitigation will only protect against the initial portion of the attack. Other portions of the chain can still be triggered if an attacker already has access to the network or can convince an administrator to open a malicious file.
#3 Determine whether an Exchange Server has already been compromised.
These vulnerabilities have been in the wild and actively exploited for over a month, with the earliest indications of exploitation leading back to January 3. Any organisation running the vulnerable software must evaluate if their server has been compromised.
Patching the system will not remove any malware already deployed on the system. It would be prudent to assume Exchange Servers that exposed Outlook Web Access or Exchange Web Services to the internet are compromised until proven otherwise.
Check for suspicious process and system behaviour, especially in the context of Internet Information Service, IIS, and Exchange application processes, such as PowerShell, Command shells and other programs executed in the applications’ address space.
Microsoft has released PowerShell and Nmap scripts for checking your Exchange Server for indicators of compromise of these exploits. They have also released another script, available at the same link, that highlights differences in files from the virtual directories of your Exchange Server against those expected for your specific Exchange version. The Cybersecurity and Infrastructure Security Agency, CISA, has also published a list of tactics, techniques and procedures, TTPs.
As documented in the Unit 42 Threat Assessment Courses of Action table, the post-intrusion TTPs used by the initial actors conducting the Exchange attacks included the following:
- Using Procdump to dump the LSASS process memory
- Using 7-Zip to compress stolen data into ZIP files for exfiltration
- Adding and using Exchange PowerShell snap-ins to export mailbox data
- Using the Nishang Invoke-PowerShellTcpOneLine reverse shell
- Downloading PowerCat from GitHub, then using it to open a connection to a remote server
Since the initial attacks, we believe that other actors are trying to capitalise on the Exchange vulnerabilities, but their motivations and objectives may differ vastly, and so could their TTPs.
#4 Engage an Incident Response team if you think you have been compromised. If, at any point, you think your Exchange Server has been compromised, you should still take action to secure it against the vulnerabilities as described above. This will prevent additional adversaries from further compromising the system. Installing the out-of-band security updates for your version of Exchange Server is very important, but this will not remove any malware already installed on systems and will not evict any threat actors present in the network.
The potential impact of this situation is critical due to the ongoing activity described, the vulnerabilities exploited to deliver the attack and the adversaries who could be behind compromises. While exploits of these vulnerabilities may not halt business operations, access to sensitive information and systems is certainly possible, and should be assumed to have occurred. Access to corporate emails could also lead to follow-up phishing attacks.
Lior Div, CEO and Co-founder, Cybereason
SolarWinds had crippling effects on hundreds of businesses and nearly a dozen US government agencies. Yet it is safe to say the Exchange Server breach is 1,000 times more crippling because the Chinese attacked small and medium sized businesses, the lifeblood of the US and global economy. There could be hundreds of thousands of businesses crippled.
As if the devastating effect of Covid-19 isn’t enough for small businesses, municipalities and other organisations that were forced to either close or downsize their staff over the past year, they are now taking another collective punch.
Russia, China, North Korea, and Iran make up the axis of cyber evil and their well-trained and highly skilled teams are cold-blooded, have no conscience and have a singular goal of seeing the US and Europe suffer and scramble. They constantly attack hospitals, research companies and the vaccine supply chain. They do it they are ruthless profiteers.
In the bigger picture, when Microsoft is in trouble, the global economy and our wellbeing is in trouble. They are #21 on the Fortune 500 and their products are used in every corner of the world.
Immediately, the US and other countries at the state and national level need to be threat hunting round the clock in their networks. Sharing intelligence, sharing the locations of where the cyber criminals are located and putting their names on the front pages of every news outlet in the world is a start to putting defenders on higher ground than threat actors.
In addition, an operation centric approach to fight cybercrime is needed so that defenders see every aspect of the threat actors’ malicious operation to better digest disparate pieces of information, pinpoint the malicious behaviour and stop it before it has material impact.
Most companies do not have the luxury of being a massive conglomerate with infinite access to the market like Microsoft. Being a larger player in the business systems space and offering security as an add-on at little-to-no cost by way of licensing bundles demands that security offerings be at least minimally effective especially in defending their own products and services.
The fact that an attack was conducted by a nation state does not mean the attack was indefensible, it simply means the adversary is capable. But a capable adversary is not an excuse for failing to protect customers at such a massive scale, or to downplay that failure by glorifying the assumed prowess of the attackers.
Analysis report by Mandiant Experts
Beginning in January 2021, Mandiant Managed Defence observed multiple instances of abuse of Microsoft Exchange Server within at least one client environment. The observed activity included creation of web shells for persistent access, remote code execution, and reconnaissance for endpoint security solutions.
Our investigation revealed that the files created on the Exchange Servers were owned by the user NT AUTHORITY\SYSTEM, a privileged local account on the Windows operating system. Furthermore, the process that created the web shell was UMWorkerProcess.exe, the process responsible for Exchange Server’s Unified Messaging Service. In subsequent investigations, we observed malicious files created by w3wp.exe, the process responsible for the Exchange Server web front-end.
In response to this activity, we built threat hunting campaigns designed to identify additional Exchange Server abuse. We also utilised this data to build higher-fidelity detections of web server process chains. On March 2, 2021, Microsoft released a blog post that detailed multiple zero-day vulnerabilities used to attack on-premises versions of Microsoft Exchange Server. Microsoft also issued emergency Exchange Server updates for vulnerabilities, CVE-2021-26855, CVE-2021-26857, CVE-2021-26858 and CVE-2021-27065.
The activity reported by Microsoft aligns with our observations. FireEye currently tracks this activity in three clusters, UNC2639, UNC2640, and UNC2643. We anticipate additional clusters as we respond to intrusions. We recommend following Microsoft’s guidance and patching Exchange Server immediately to mitigate this activity.
Based on our telemetry, we have identified an array of affected victims including US-based retailers, local governments, a university, and an engineering firm. Related activity may also include a Southeast Asian government and Central Asian telecom.
Microsoft reported the exploitation occurred together and is linked to a single group of actors tracked as Hafnium, a group that has previously targeted the US-based defence companies, law firms, infectious disease researchers, and think tanks.
From exploit to web shell
Beginning in January 2021, Mandiant Managed Defense observed the creation of web shells on one Microsoft Exchange server file system within a customer’s environment. The web shell, named help.aspx (MD5: 4b3039cf227c611c45d2242d1228a121), contained code to identify the presence of FireEye xAgent, CarbonBlack, or CrowdStrike Falcon endpoint products and write the output of discovery.
The web shell was written to the system by the UMWorkerProcess.exe process, which is associated with Microsoft Exchange Server’s Unified Messaging service. This activity suggested exploitation of CVE-2021-26858.
Approximately twenty days later, the attacker placed another web shell on a separate Microsoft Exchange Server. This second, partially obfuscated web shell, named iisstart.aspx (MD5: 0fd9bffa49c76ee12e51e3b8ae0609ac), was more advanced and contained functions to interact with the file system.
While the use of web shells is common amongst threat actors, the parent processes, timing, and victim(s) of these files clearly indicate activity that commenced with the abuse of Microsoft Exchange.
In March 2021, in a separate environment, we observed a threat actor utilise one or more vulnerabilities to place at least one web shell on the vulnerable Exchange Server. This was likely to establish both persistence and secondary access, as in other environments. In this case, Mandiant observed the process w3wp.exe, the IIS process associated with the Exchange web front-end, spawning cmd.exe to write a file to disk. The file, depicted below, matches signatures for the tried-and-true China Chopper.
We observed that in at least two cases, the threat actors subsequently issued the following command against the Exchange web server: net group Exchange Organisation administrators administrator /del /domain.
This command attempts to delete the administrator user from the Exchange Organisations administrators group, beginning with the Domain Controller in the current domain. If the system is in a single-system domain, it will execute on the local computer.
As per Microsoft’s blog, they have identified additional post-exploitation activities, including:
- Credential theft via dumping of LSASS process memory
- Compression of data for exfiltration via 7-Zip
- Use of Exchange PowerShell Snap-ins to export mailbox data
- Use of additional offensive security tools Covenant, Nishang, and PowerCat for remote access
The activity we have observed, coupled with others in the information security industry, indicate that these threat actors are likely using Exchange Server vulnerabilities to gain a foothold into environments. This activity is followed quickly by additional access and persistent mechanisms. As previously stated, we have multiple ongoing cases and will continue to provide insight as we respond to intrusions.
Investigation tips
We recommend checking the following for potential evidence of compromise:
- Child processes of C:\Windows\System32\inetsrv\w3wp.exeon Exchange Servers, particularly exe.
- Files written to the system by exeor UMWorkerProcess.exe.
- ASPX files owned by the SYSTEMuser
- New, unexpected compiled ASPX files in the Temporary ASP.NET Filesdirectory
- Reconnaissance, vulnerability-testing requests to the following resources from an external IP address:
- /rpc/directory
- /ecp/DDI/DDIService.svc/SetObject
- Non-existent resources
- With suspicious or spoofed HTTP User-Agents
- Unexpected or suspicious Exchange PowerShell SnapIn requests to export mailboxes
In our investigations to date, the web shells placed on Exchange Servers have been named differently in each intrusion, and thus the file name alone is not a high-fidelity indicator of compromise.
If you believe your Exchange Server was compromised, we recommend investigating to determine the scope of the attack and dwell time of the threat actor.
Furthermore, as system and web server logs may have time or size limits enforced, we recommend preserving the following artifacts for forensic analysis:
- At least 14 days of HTTP web logs from the inetpub\Logs\LogFilesdirectories, include logs from all subdirectories
- The contents of the Exchange Web Server, also found within the inetpubfolder
- At least 14 days of Exchange Control Panel, ECP, logs, located in Program Files\Microsoft\Exchange Server\v15\Logging\ECP\Server
- Microsoft Windows event logs
We have found significant hunting and analysis value in these log folders, especially for suspicious CMD parameters in the ECP Server logs. We will continue updating technical details as we observe more related activity.