Skip to Main Content
February 07, 2023

ESXiArgs: What you need to know and how to protect your data

Written by Tyler Hudak, Liz Waddell, Justin Vaicaro, Steven Erwin, Shane Hartman, Olivia Cate and Thomas Millar
Incident Response Incident Response & Forensics Threat Hunting

Threat Overview

Around February 03, 2023, a ransomware campaign called “ESXiArgs” emerged that targeted Internet-facing VMware ESXi servers running versions older than 7.0. Though not confirmed, it has been reported by the French CERT (CERT-FR), BleepingComputer, and other sources that the campaign leverages CVE-2021-21974, which is a three-year-old vulnerability in the OpenSLP component of the ESXi instance. The vulnerability allows for remote code execution (RCE) and is currently being used to deploy ransomware; however, as with any vulnerability allowing RCE, it has the potential for additional attacks. The ransomware in this campaign targets virtual machine volumes within the /vmfs/volumes directory, encrypting their virtual disks and memory files, encrypting their virtual disks and memory files. The encryptor is executed by a bash script placed in /tmp/ as part of the attack. The execution flow of the script is described below, and the TrustedSec Research team will also be publishing an in-depth analysis of the encryptor itself.

If CVE-2021-21974 is the initial attack vector used in this campaign, then the following ESXi platforms are at risk:

  • ESXi 7.x versions earlier than ESXi70U1c-17325551
  • ESXi versions 6.7.x earlier than ESXi670-202102401-SG
  • ESXi versions 6.5.x earlier than ESXi650-202102101-SG

Note: Versions ESXi versions 6.0 and 5.5 have also reportedly been impacted by this campaign; however, TrustedSec has not yet been able to confirm the validity of these claims at this time.

An overview of the execution flow of the encryption deployment script is depicted in the chart below.

Update: CISA has released a recovery script, ESXiArgs-Recover, to assist in the rebuilding of virtual machines in impacted environments. The script works by leveraging unencrypted flat files (which appear to be untouched in this campaign) to rebuild a VM’s .vmdk file. More information can be found on the project’s GitHub page: https://github.com/cisagov/ESXiArgs-Recover.

Ransomware Deployment Script Technical Analysis

TrustedSec examined the ESXiArgs deployment script found on impacted systems - encrypt.sh - and identified the following execution steps:

  • Sets the staging directory to /tmp/ which contains the following attack files:
    • encrypt.sh - The deployment script, which contains the commands executed as part of the post-exploitation phase
    • public.pem - A secondary public RSA key used to encrypt the primary encryption key
    • motd - The ransom note displayed as the “message of the day”
    • index.html - The ransom note (HTML), which replaces the home page of VMware ESXi
  • Sets hard limits on the pipe buffer size (ulimit -Hp) and maximum open file descriptors (ulimit -Hn)
  • Modifies the configuration files for the present virtual machines by corrupting the references to their corresponding .vmdk and .vswp files
  • Kills all VMX processes: kill -9 $(ps | grep vmx | awk '{print $2}')
  • Encrypts virtual machine files within the /vmfs/volumes directory, targeting files with the following extensions: .vmdk, .vmx, .vmxf, .vmsd, .vmsn, .vswp, .vmss, .nvram, .vmem
  • Replaces the VMware ESXi home page with the HTML ransom note
  • Replaces the server’s “message of the day” (motd) banner with a ransom note
Ransomware Note (HTML) Example
  • Deletes all discovered .log files from the server
  • Overwrites /sbin/hostd-probe with /sbin/hostd-probe.bak
    • If the .bak file is present hostd-probe is ESXi’s system monitoring utility.
    • If replaced, the hostd-probe file is timestomped to conceal its modification.
  • If the ESXi build number is detected as 7 or above, lines 1-8 and the last line are deleted from the /var/spool/cron/crontabs/root file. The file is also timestomped to conceal its modification. This deletes the default cron jobs on the host.
    • For build numbers other than 7, the first line from /bin/hostd-probe.sh is deleted. This file is subsequently timestomped to conceal its modification.
  • Deletes the file /store/packages/vmtools.py.
    • Note that the path of this file reflects that of a previously known backdoor. As such, administrators should verify the presence or deletion of the file and, if present, inspect its content.
  • Deletes the last line of the /etc/vmware/rhttpproxy/endpoints.conf file and then timestomps the file to conceal its modification.
    • Deleting this line removes access to the ESXi management API.
  • Overwrites the /etc/rc.local.d/local.sh file with blank content and then timestomps the file to conceal its modification. This file would typically be used to run custom actions upon server boot
  • Deletes the following attacker files from /tmp/: encrypt, nohup.out, index.html, motd, public.pem, archieve.zip (note the spelling)
  • Executes the /bin/auto-backup.sh script.
    • Responsible for backing up the ESXi configuration, which is then restored upon reboot
  • Executes /bin/rm -- "$0" to delete the running attacker script
  • Executes /etc/init.d/SSH start to start the SSH server on the host

Mitigation, Remediation, and Monitoring Recommendations

Because CVE-2021-21974 is observed as a common initial access vector, organizations are strongly urged to apply all vendor-recommended patches to vulnerable ESXi servers as soon as possible. A list of resolutions that address these vulnerabilities can be found on the VMware advisory page (VMSA-2021-0002).

Beyond this, organizations should monitor the directories and files mentioned above for modification. This latter measure will help ensure detection in the event that patches are insufficient for preventing compromise. Furthermore, as is the case with general ransomware preparedness, organizations should maintain secure and frequent backups to aid in quick recovery if compromise does occur.

An organization’s attack surface can be further reduced by performing system hardening actions such as removing unnecessary Internet-facing servers or services, implementing an access control list (ACL) that restricts access to VMware servers to the extent possible, disabling or removing unnecessary VMware components (such as SLP), and enforcing the principle of least privilege for administrative services. Additional details about routine cyber hygiene activities can be found here.

Conclusion

The ESXiArgs ransomware campaign highlights the importance of keeping ESXi servers updated with the latest patches to mitigate risk. However, even with patches applied, organizations should also monitor key directories and files for unauthorized modification and maintain secure and frequent backups. By performing system hardening actions such as reducing the attack surface, implementing an ACL, disabling unnecessary components, and enforcing the principle of least privilege, organizations can further reduce the risk of compromise. Overall, our goal is to arm organizations with the knowledge and measures to proactively prevent and mitigate potential ransomware attacks.

In summary:

  • Patch or upgrade vulnerable ESXi Servers
  • Remove Internet connectivity if not needed
    • At minimum, ensure proper controls are in place to minimize access
  • Disable SLP service

Indicators of Compromise

Filenames

/tmp/encrypt

/tmp/nohup.out

/tmp/index.html

/tmp/motd

/tmp/public.pem

/tmp/archieve.zip

File Hashes

encrypt - 87b010bc90cd7dd776fb42ea5b3f85d3

encrypt - 11b1b2375d9d840912cfd1f0d0d04d93ed0cddb0ae4ddb550a5b62cd044d6b66

encrypt.sh - d0d36f169f1458806053aae482af5010

encrypt.sh - 10c3b6b03a9bf105d264a8e7f30dcab0a6c59a414529b0af0a6bd9f1d2984459

File Access/Modifications

/var/spool/cron/crontabs/root

/bin/hostd-probe.sh

/etc/vmware/rhttpproxy/endpoints.conf

/etc/rc.local.d/local.sh

TOX ID (identified in ransom note)

D6C324719Ad0AA50A54E4F8DED8E8220D8698DD67B218B5429466C40E7F72657C015D86C7E4A

References: