This post will be on how to setup and modify Cuckoo to work with a non-supported hypervisor, Proxmox.
“Proxmox VE is a complete open-source platform for all-inclusive enterprise virtualization that tightly integrates KVM hypervisor and LXC containers, software-defined storage and networking functionality on a single platform, and easily manages high availability clusters and disaster recovery tools with the built-in web management interface.
The enterprise-class features and the 100% software-based focus make Proxmox VE the perfect choice to virtualize your IT infrastructure, optimize existing resources, and increase efficiencies with minimal expense. You can easily virtualize even the most demanding Linux and Windows application workloads, and dynamically scale-out your computing and storage as your needs grow ensuring to stay adaptable for future growth of your data center.”
Proxmox is built on the Debian platform. They provide a bootable ISO image, which can either be burned to a disc or better yet to a bootable USB flash drive. I used Ubuntu and its “Startup Disk Creator” to create the bootable USB drive.
Select the Proxmox ISO by clicking other and using the file dialog to navigate to the downloaded ISO. Next, select the Drive for which to copy the ISO. Lastly, click “Make Startup Disk”. This will take a few minutes but will prompt you when it is complete.
Boot the computer from the USB Drive to install Proxmox. The installation is straightforward and pretty quick. Follow the instructions on their website. The only issue I came across was the partitioning of the drive, as I used a small drive and quickly ran out of space.
The installation of Cuckoo is almost identical to the installation on Debian, as Proxmox is a variant of Debian. See Cuckoo Installation Notes for Debian for the full installation instructions. Here, I will focus on the additional steps needed to get Proxmox and Cuckoo to play nice with each other. Also, some of Cuckoo’s features are not supported as of this writing. I.E. Memory dumps for analysis through Volatility. I believe this can be done, but I have not invested the time as of this writing.
The following commands are to be used in addition to the commands from Cuckoo Installation Notes for Debian.
pip install proxmoxer
pip install requests
Requests is an elegant and simple HTTP library for Python
sudo apt-get install qemu-kvm libvirt-bin Ubuntu-vm-builder bridge-utils
qemu-kvm -> Is already installed as part of Proxmox installation. Package pve-qemu-kvm
ubuntu-vm-builder -> Builds virtual machines from the command line
bridge-utils -> Utilities for configuring Linux Ethernet Bridges, normally already installed.
python-libvirt -> Provides an application written in Python with the ability to call the interface supplied by the libvirt library, manage virtualization capabilities. This functionality is provided by Proxmox. Installing this package will uninstall proxmox-ve and pve-manager. These two packages are the main functionality of Proxmox and the web interface. If you install the python-libvirt package, do the following to reinstall proxmox-ve and pve-manager.
apt-get install proxmox-ve pve-manager
For me, the sources.list was updated and I needed to modify that to get the correct package lists.
libvirt-bin -> Used to interface with different virtualization technologies.
sudo adduser tweety
adduser -> Creates a new user. This example used the name tweety.
Proxmox logs in as the tweety user and does not have sudo by default.
apt-get install sudo
Cuckoo configuration is mostly the same as before, except: edit the cuckoo.conf, setup of the new VLAN and add the new configuration file, proxmox.conf.
Modify the machinery line to run proxmox.
Setup a second network interface that was bridge. Figure 3 demonstrates the network configuration. Modify the file /etc/network/interface to add the new bridge. A good resource for networking is here.
Instructs the system to bring this interface up on boot
iface vmbr1 inet static
Sets the interface to have a static IP address
Assigns the IP address of 192.168.56.1 to this interface
Mask used to divide the network into subnets. 255.255.255.0 allows 255 IP for this subnet. The subnet is 192.168.56.1 – 192.168.56.255
bridge_ports = enp8s0f0
Assigns the bridge to interfaces. This could bridge multiple network cards or none. In the case where the analysis does not want any traffic to leave the system, leave this line out. In this case, the cable is not connected to the spare interface enp8s0f0.
Alternatively, the network configuration can be completed using Proxmox Web UI. Highlight the Node name on the left, expand the System, and highlight the Network. Click on Create, then Linux Bridge.
A new dialog window will be displayed. In this window, add the IP address for the Host, for this example, 192.168.56.1. The subnet will be 255.255.255.0. Check the VLAN aware box and fill out the Bridge ports if tying to interfaces, otherwise just leave blank.
The network settings can be viewed, edited, or deleted at any time from this page.
Adding the Guest VM to a specific network. Highlight the VM on the left pane. Select Hardware and double-click Network Device. Add the created Network to the Bridge field.
The configuration file, proxmox.conf needs to be added to the /home/tweety/.cuckoo/conf/ directory. This configuration file was originally based on virtualbox.conf, but has been modified to work with Proxmox REST API.
Sections of the proxmox.conf file
The sections mirror those of the other machine configs. The first line is the name of the machine in square brackets “[“ and “]”
Name of the hypervisor.
username = tweety password = <the password for the user tweety>
The username and password to the Proxmox GUI. This is used to log into Proxmox API. Currently, the password is in clear-text, so be sure to create a user with only privileges to start stop the sandboxes.
host = pveone
The Host will be the name of the server running the GUI. Usage of the IP address or localhost should also work.
nodename = pveone
Name of the node configured in Proxmox. Located below the Datacenter as shown below.
machines = kali2018, cuckooOffice
The machines are a comma separated list of the different VMs available to use. These names must match the later sections. IE. cuckooOffice muse have a section that starts with [cuckooOffice]
Virtual Machine Sections
Starts the virtual machine section and must match that in the machines tag above. This does not have to match the name of the VM.
label = cuckooOffice
the label must match the name of the VM. The label is used to identify the VMs unique ID. Proxmox uses the ID rather than the name. Both ID and name are shown in Figure 47.
platform = windows
The operating system of the Guest. I.e. windows, linux, or Darwin
ip = 192.168.56.99
IP address of the Guest. The Guest must have this IP address or the Cuckoo server will not be able to transfer the malware sample to the Guest VM.
Cuckoo requires that each VM has at least one snapshot. If this is left blank cuckoo will revert to the last snapshot, otherwise this should be the name of the snapshot to revert too. As an example, if snapshot is left empty cuckoo will revert to the snapshot named Agent. If snapshot is set to InstallWin7 then that snapshot will be used.
interface = vmbr1
The name of the VLAN or bridge created above.
resultserver_ip = 192.168.56.1 resultserver_post = 2042
The resultserver_ip and resultserver_port will match the IP and port configured in the cuckoo.conf and must match the static IP assigned in the VLAN/Bridge configured above.
Cuckoo Code Modifications
Newer versions of Cuckoo have incorporated parts of the configuration files into the Python code itself. This stops the ability to just add a new machine.py file. However, this can be bypassed by adding the following to the /home/tweety/venv/lib/python2.7/site-packages/cuckoo/common/config.py
Since the configuration file was modeled after the Virtualbox file, this section was originally copied from the Virtualbox section. As the items in the configuration file changed, those changes need to be reflected here. These values will be used in the proxmox.py and will be overwritten by values in the proxmox.conf file. Be sure to include the “__star__” line or else Cuckoo will error.
New Machine File
Each individual hypervisor will have a <machine>.py file located under /home/tweety/venv/lib/python2.7/site-packages/cuckoo/machinery/. Each <machine>.py file will be named the same as the configuration file ie. proxmox.conf and proxmox.py. The machine files are inherited from a base class, but leave you to implement the functionality such as start, stop, reverting a snapshot, and memory analysis. For simplicity, the use of the Python module promoxer is used to interact with the Proxmox REST API. This took some trial and error, as the modules did not work as the documentation explained. The following show the proxmoxer commands to retrieve specific hypervisor information.
proxmox.nodes().get(‘pveone/qemu’) => list of vm’s in the the node ‘pveone’
proxmox.nodes().get(‘pveone/qemu/100’) => list of options for the VM 100
proxmox.nodes().post(‘pveone/qemu/100/status/start’) => Start the VM
proxmox.nodes().post(‘pveone/qemu/100/status/stop’) => Stop the VM
proxmos.nodes().post(‘pveone/qemu/100/snapshot’, snapname=’testing’) => snapshot the VM and name it ‘testing’
proxmox.nodes().post(‘pveone/qemu/100/snapshot/testing/rollback’) => revert the snapshot to the ‘testing’ snapshot
The source code for the proxmox.py is located here. The main methods that where modified are restore, start, and stop. Each of these methods used the proxmoxer commands listed above to implement the functionality. The key/value pairs from the proxmox.conf file are accessible through the self.options.proxmox variable. The initialize_check is called first to verify and set up the environment. This section is used to verify the configuration file input and to create the connection to the Proxmox REST API. The API is then used to determine the available VMs. It is possible to have more VMs in Proxmox than registered with Cuckoo.
Subscription Message Removal:
Proxmox is free and fully functional but they offer a subscription. If you use it, I would recommend migrating to their subscription service. After all, this is how they are able to provide the free version for us to use and enjoy. However, for a trial period the Nag message is a minor annoyance. To remove the Nag message, edit the /usr/share/pve-manager/js/pvemanagerlib.js file. Comment out lines 849 – 867. As shown in Figure 52. Then add the orig_cmd(); }.
Now Cuckoo should be set up to run the new machine, proxmox. The actual usage of Proxmox is independent of the machinery. See Blog post Malware analysis is for the (cuckoo) birds.