Through threat hunting, an organization can break away from a reactive approach to identifying incidents and evolve into a proactive operation that actively looks for incidents. The high-level threat hunting pipeline consists of taking a hypothesis built around threats specific to the organization, lab testing and validating the hypothesis, implementing security operation detection, testing live-fire production alerts and detection, and periodic tuning of the deployed detection.
The goal of this blog post is not to dissect the threat hunting process or dive into the various hunting strategies and tactics. Rather, the intent is to show the importance of focusing on a legitimate protocol within a threat hunt engagement that can be easily used for potential data exfiltration, hide in plain sight with other normal traffic, and go unnoticed by a security operations center (SOC) that is untrained to identify potentially suspicious network behavior.
I will walk through an interesting remote desktop protocol (RDP) session that I came across while on a threat hunt engagement and the pivoting process I took to identify the surprising destination device handling the connection. I will identify shortcomings within the network security design that could have proactively blocked the network connection, call out the importance of being aware of daily network traffic patterns and establishing baseline monitoring, and highlight outliers that should easily raise red flags.
Note: Specific low-level information is deliberately excluded to respect the privacy of the client in which this information was identified.
Let the Hunting Begin
The threat hunt process began with a decision to focus on outbound RDP traffic originating from the internal network with suspicious foreign countries as the destination. This was not as simple as it sounds because the client had a geographical presence across the entire globe.
The initial hunt started off with a Security Information and Event Management (SIEM) query that focused solely on the RDP protocol, excluded all internal subnets as the destination, and then filtered out all known client-owned external destination IP addresses. By taking this approach, a suspicious foreign country of interest bubbled to the surface: Hungary.
Further Validation Required
Although the client identified they have a security service provider in that physical location, this insecure type of connectivity should not have been taking place.
An egress firewall access control list (ACL) should be implemented to restrict non-business required port communication, such as allowing only TCP Ports 80, 443, 25, and 53.
Let the Pivoting Commence
Now that I had a finding of interest, it was time to dig in to see how far this journey would take me.
A second SIEM query was built adding Hungary’s geographic location to further trim the SIEM output noise, which verified the RDP session was ALLOWED based on firewall SYSLOG message analysis. The fact that this Hungarian public IP address had RDP exposed to the public Internet and an internal host connected to it was suspicious due to the potential danger associated with this type of connectivity.
Assumptions Being Built
If this was a legitimate RDP connection, I would assume this type of connection would be taking place across a hardened business-to-business (B2B) VPN tunnel to protect this insecure type of connection. Also, if this was not a legitimate RDP connection, this is an unknown blind spot in the SOC monitoring, and a legitimate incident could go unnoticed.
Normal Traffic Baseline Monitoring
It is important to note that daily SOC baseline traffic monitoring is critical to identify network traffic anomalies.
The previous recommended firewall ACL would have prevented the unnoticed RDP session from taking place, but would still require investigation to identify why the connection was initiated in the first place.
Next it was time to focus on the external Hungarian IP address in an effort to derive any useful open source intelligence (OSINT).
WHOIS information gathering identified that the IP address did not belong to the client or their security service provider, which prompted me to dig deeper.
Shodan verified that RDP was, in fact, exposed and identified some additional interesting web application ports that were publicly exposed, which was a piece of information I took note of.
While digging into the SSL certificate information listed in the Port 443 section of the Shodan output, I identified how recently the certificate was issued (the threat hunt engagement was taking place during this identified window) and that it was a certificate issued by Let’s Encrypt.
After identifying from the Shodan output that this Hungarian IP address had some web applications exposed, I then pivoted to identify if any of the landing pages looked interesting. I performed this passive reconnaissance by using Urlscan.io, which is a URL and website scanner used to anonymously identify whether the URL or website of interest is malicious.
While performing this passive scan against port 443, an interesting finding surfaced. A default web service was enabled but not configured. This finding was not critical but began to open increased levels of interest as to what this external host was.
It was now understood that the Hungarian IP address was handling RDP connections and now had a default web application installation externally exposed. The plot thickens.
From the displayed message in the captured web page, I then pivoted to try and identify what type of web application this was. Some simple Google-Fu allowed me to easily narrow down what type of device it was. It appeared to be a Synology NAS device with enabled DiskStation Manager web services.
Quote from link reference:
“With Web Station, you can create a website with web pages on Synology DiskStation.”
I confirmed this finding by using curl to make a manual HTTP GET request to the web application being hosted on the Hungarian IP address.
The hunt then pivoted over the client’s endpoint detection response (EDR) tool in an attempt to identify additional host-level information surrounding the external RDP connections. A query focusing on the Hungarian IP address identified a single internal host and user account with three (3) separate RDP connections. Frequency analysis of the connection details did not provide any significant outliers, but I did identify that one of the sessions lasted for one (1) hour.
Further Verification Required
At this point, it was unclear if this was a compromised internal host and whether there was any type data exfiltration that may have taken place.
From here, I pivoted back to the SIEM to run a targeted query to identify if there was any additional suspicious inbound our outbound activity associated with the Hungarian IP address. Further analysis of this SIEM query output brought about a very odd finding. It turned out that there was actually a VPN session being successfully established from an internal host to the Hungarian Synology NAS appliance. The host associated with the VPN client activity did not correlate to the internal host identified with the RDP session activity, which added increased levels of interest to the evolving list of findings.
With this information, I pivoted over to find out if the Hungarian Synology NAS could also be running VPN services. It appears that a Synology NAS appliance can also provide VPN services.
Quote from Link Reference:
“With Synology’s VPN Server package, your Synology NAS can become a VPN server, allowing DSM users to remotely and securely access resources shared within the Synology NAS’s local area network.”
Assumptions Being Built
Even if this turned out to be a legitimate business connection, there was nothing normal about this type of VPN client connectivity to a Synology NAS appliance located in Hungary.
B2B site-to-site VPN tunnels should be used for this type of external access and all unapproved VPN client connectivity should be denied by default.
Final Assumptions and Recommendations
Research lead to the assumption that the Hungarian RDP connections were being port-forwarded by the Synology NAS to a back-end host (it was unclear what this host was). These are very unusual RDP and VPN client connectivity activities and required further investigation to identify if they were, in fact, legitimate business connectivity. Due to engagement timing, I was unable to perform any further research, but the finding was escalated on the client side to identify the following:
- Perform further analysis on the Hungarian IP address to identify if this was somehow tied to their security service provider located in the same country.
- Reach out to the internal users involved in the RDP and VPN client connections to identify the business purpose of the connections.
- Recommended outbound RDP connectivity should be highly restricted in order to prevent this protocol from being used to exfiltrate data and further lessen the egress threat landscape.
- Recommended detections should be put in place for all unauthorized VPN client usage and any suspicious outbound RDP connections not destined for authorized external physical locations. Even if they are blocked, the reason behind the connectivity attempts is important to understand.
The answers to these questions would drive the next course of action on the client side and identify if an incident should be declared or business processes hardened to prevent these insecure connections from continuing to take place.
As we saw from this hunting example, RDP and client VPN connections can easily go unnoticed when proper network access controls are not in place, egress geo-blocking is not enforced, and SOC network traffic baselining is not understood. These deficiencies allowed for these particular client connections to hide in plain sight.
Hunting starts with a proper hypothesis, leads into sound queries, and then requires using your gut to pivot off of the nuggets of analysis fodder to derive greater context and meaning to your finding. Hunting is a journey—dig deeper, pay attention to detail, and you may be pleasantly surprised at how the outcome develops.
An important takeaway: although the hunt finding was not closed out during the engagement, proper communication protocol was agreed upon and established prior to and during the engagement, allowing the hunt finding triage to continue post-engagement on the client side. At the time of this post, it was unclear if this turned out to be a legitimate connection or evolved into an incident response scenario.