Office 365 has an assortment of capabilities allowing both small to extremely large businesses to move their infrastructure and services to the cloud. In 2015, Microsoft introduced their "Advanced Threat Protection" functionality and has since been bolstered in 2016 and 2017 as a direct way to protect against advanced attacks. If you look at Microsoft's moves in the past year, they are putting a large emphasis on security in certain areas. With the end of life for the Enhanced Mitigation Experience Toolkit (EMET) (which I think was a major mistake), Windows 10 was touted as the most secure operating system to date. Advanced Threat Protection (ATP), Advanced Threat Analytics (ATA), and multiple other products are now being touted as methods for detection and prevention through Microsoft's ecosystem. Being a company that focuses on adversarial simulation, penetration testing, and research, we frequently run into every variation of security products that you can imagine. In this blog, we'll cover Microsoft's "Advanced Threat Protection" functionality for Exchange. During this blog post, the testing infrastructure that was used was the Microsoft E5 plan which is the highest plan you can have and includes the Advanced Threat Protection component. This was confusing at first; if you buy lower programs you are still eligible for ATP but need to contact Microsoft to add the component. By purchasing an E5 license, you have the direct ability for ATP and it is automatically added to your plan, and to Exchange. The main purpose of this blog post is not to point out where Microsoft has gaps. but to demonstrate and educate around its effectiveness or lack thereof when confronted with both basic attacks and advanced attacks. In addition to this post, TrustedSec will intentionally be leaving a number of methods out of the post and hope to have discussions with Microsoft on how to improve. It is also not the intention of myself nor TrustedSec to use this research for self-promotion. On the contrary. Public discussion and disclosure is for the betterment of security in general and has always been our mission to make the industry better. Microsoft has some of the world’s brightest and most talented people including individuals who dedicate their career to the betterment of security. It is the hope that this post puts an emphasis on those individuals in the company and their position to impact change at Microsoft in improving their product around security. It was my intention of releasing all my research, however, since some prior discussions, I've decided to hold back. I will share direct evidence for the areas I make claims to in the article but omit other techniques until they are resolved. Now diving down into Office 365 and ATP. I want to start off by saying that a major show stopper for me and anything Office 365 is the direct inability to go above a 16-character password. This is a hard limit and due to legacy systems that do not support a greater than 16-character password when using Office 365. Complete show stopper for me. My passwords tend to be in the 40-60 character limit. Microsoft strips certain special characters and disallows using greater than 16 characters. My understanding is that the only way around this is by using ADFS (Federated) and handling authentication through your own AD infrastructure into O365. In addition to the 16-character limit, a prior Microsoft acquisition of PhoneFactor allows for two-factor authentication. This is neither enabled by default, nor is it easy to find without digging into the menus. It is heavily recommended that if you are going with Office 365 to enable two-factor authentication immediately. Moving on and away from the 16-character hard limit and no two-factor by default. The intent of this post is to determine if we can deliver malicious content to Office 365 users without triggering any of Microsoft's protections. You can think of Microsoft's protection when it comes to Exchange in two main feature-sets: 1. Anti-Malware / Protection / Common SMTP Methods for Blocking 2. Advanced Threats / Protection The Anti-Malware / Protection components of Exchange 365 is pretty simplistic; it's essentially anti-virus and has traditional rules for SMTP. This category falls in line with traditional spam filtering. Some common rules apply, but you can easily baseline Microsoft's defenses by testing their controls in this area. In order to circumvent the first criteria of anti-malware and common SMTP blocking: 1. Make sure you have an mx record setup for the domain. 2. Reverse DNS can help but is not required. It boosts reputation. 3. The domain needs to be registered for at least 10 days in order to have appropriate reputation. 4. Make sure SPF records are set up properly. 5. Send through web-content providers to get categorized. Some companies block uncategorized or unclassified sites, you can submit your site to each of the web content filtering companies to get categorized prior to sending your phish. A good example here for Barracuda (they all have them) on how to get your phishing site categorized prior to sending the phish. 6. Use mxtoolbox.com to check to make sure everything is set up appropriately. 7. Ensure that if you are using a payload that it doesn't trigger inside of Windows Defender. 8. Make sure you aren't on any spam/blocklist. You can get yourself removed by contacting Microsoft (takes about an hour). This part is fairly trivial and pretty much where the rest of the industry is. Getting mail to be delivered with basic controls continues to be super easy with little to no protection against targeted attacks or mass campaigns. Now moving onto the Advanced Threat Protection (ATP). Microsoft's definition of ATP: "Advanced threat protection (ATP) in Exchange Online Protection (EOP) helps you prevent zero-day malware attacks in your email environment. ATP provides a way for you to create policies in the Exchange admin center (EAC) that help ensure your users access only links in emails or attachments to emails that are identified as not malicious. If you already use EOP to help combat malware in your email messaging environment, adding ATP will provide more effective protection than ever before against attacks propagated by unsafe links and unsafe attachments." ATP can be broken down into two main components. First is the Safe Attachments and the second is Safe Links. When testing the two components, the Safe Attachments and Safe Links settings was set to block and to prohibit any malicious attachments from being delivered to the domains we used: For the Safe Links settings, the same was configured to automatically block, with the option for the user to manually accept: Safe Attachments is a peculiar one, and I'm curious how its implementation works in large organizations and if the mail delay is acceptable. If you have Safe Attachments on, when an email is sent to a customer of ATP with Safe Attachments, it will on average take anywhere from 5 minutes or in my experience 15 minutes to actually receive the email. This is well documented with Microsoft. What this means is that in a corporate environment, if an email is sent, it can take upwards of 15 minutes or more in order to receive an email. My guess is that this is Microsoft's first iteration to compete with sandbox/virtualization SMTP gateways on the market today and in the cloud. I won't spend much time on this area; generally when I'm going after a company, it's through malicious websites versus actual attachments. I will say that during my analysis of this area, large file attachments, double zipped files, obfuscated macro injection, direct data types, large time delays, and more were successful in going through the Safe Attachments protection. During my testing, I didn't notice a difference in being flagged from traditional A/V versus ATP's attachment detection. Moving onto safe links, the way it works is when an email is sent, any direct links sent in the email is replaced by a Microsoft link. When a user clicks the link, it is first requested by the Microsoft link and inspected. There is no interrogation of the website at all prior to going on the website. Microsoft checks to see if the site has previously been added to a blacklist, then allows the user to the site. This is particularly alarming as this moves away completely from what users are taught in education and awareness. While far from foolproof, we teach our users to highlight over the URL in order to determine if the hostname matches internal company resources or the intended destination site. With Microsoft and safe links, this component is completely removed and you can't determine the original link by hovering. Personally, this is a dangerous method and I think Microsoft should rework/rethink this feature completely as it provides little protection, and is a far cry from advanced protection. Here we send an email with a malicious link in it (contains remote code execution capabilities): We can see below that when we hover over the malicious link, it is replaced by a "Microsoft Safe Link" which is supposed to inspect the URL first. I use the word “inspect” loosely. If we look at the definition of Safe Links: From Microsoft. "Safe links is a feature in EOP that helps prevent users from following links in emails that link to web sites recognized as malicious. Here’s how Safe Links identifies links in a message: 1. For messages in HTML, Safe Links identifies any link that uses the href attribute. 2. For messages in plain text, Safe Links uses custom logic to identify any text resembling a URL. Safe links also includes an advanced reporting capability that allows you to determine who has followed a malicious link. This helps EOP customers apply faster remediation for issues that are detected." When researching Safe Links, from what I can tell, it does very little to actually analyze the page itself. It appears that Safe Links only iterates through web pages to determine if they are on a previously banned or harmful website. It does not appear to detect anything abnormal on the page itself including publicly available exploits, or methods for remote code execution. Simply put, it's using a reputation database and prior identified blacklisted sites in order to determine if a site is malicious or not. Below is a video using a stock out of the box attack vector through the Social-Engineer Toolkit (SET) in order to gain code execution through PowerShell Injection: In the second example, we'll use a stock Exploit MS16-051 from Metasploit which gets flagged by traditional Anti-Virus, but does not appear to be flagged by Safe Links: As we can see from both videos, it is extremely trivial to get around Safe Links. Microsoft, if trying to get into this space, should probably get into web analysis dynamically of pages and inspect for certain components prior to allowing a user into the site. I'm guessing this option was probably explored (just a guess) and this would require extensive resources in order to process each web request coming from a link in email. I do believe though that if Microsoft is going to be offering a service touting advanced protection capabilities, it should consider this as a method. There is already a delay between 5 minutes and 15 minutes for mail delivery; at least offer this as an advanced option to do dynamic testing of the site prior to allowing a user to visit the site. This wraps up some of the basic analysis of Advanced Threat Protection (ATP) analysis. I think based on Microsoft's user population and the expansive growth of Exchange and Office 365, Microsoft needs to continue forward in ensuring that it enhances its security capabilities. Based on my research and my personal opinion, Microsoft is not yet at a point where the market is from other private sector companies when it comes to security with Advanced Protection. Microsoft seems committed in getting better and improving, and I hope to have some good discussions with them in the future on how to enhance their product. This brings a larger discussion on if the cloud can ever meet the same or better security requirements. I have yet to see an organization that is able to move to the cloud and maintain the same level of security versus being able to build it internally. There are implications on both; internal requires a large personnel and staff and a lot of upkeep. Cloud is easy. There's a major trade off there though: easy doesn't mean better or enhanced security controls. Internal organizations have the ability to deploy hardware that can meet the demands of their enterprise, such as the ability to inspect every URL (transparent reverse proxies, web plugins, etc.) as it goes out and hopefully provide some protection. In the cloud this is tough. I don't envy Microsoft's position on protecting the cloud. It's not easy and I don't think it's going to get any easier. They do appear to be doing the most in this arena, regardless if it's still a fairly basic level of protection. This article was written by David Kennedy - Founder of TrustedSec