Finally! The long-awaited 2020 ATT&CK evaluation results published. And along with it, almost every participating vendor’s interpretation of the results and how they excelled in the evaluation. As you read the industry’s commentary on the results, keep an eye out for contrived and/or creatively adjusted metrics. Below you will find a data-first approach to understanding our performance.
The benefit of MITRE Engenuity ATT&CK is that testing data is open and publicly accessible. In an effort to be transparent with our results, in this post, we will only talk about the numbers and metrics published by MITRE Engenuity – so that you can validate the information for yourself and separate fact from fiction. No number fudging, no creative invention.
SentinelOne’s MITRE Results
Here is a screenshot of SentinelOne evaluations from MITRE Engenuity; you will see that SentinelOne had:
- 100% Visibility – 174 of 174 steps
- Highest Analytic Coverage – 159 of 174 steps
- Zero Delayed Modifiers
- Zero Config Change Modifiers
Read on to understand how the above metrics are critical for an effective security posture.
The latest ATT&CK results were released Tuesday, April 20, 2021. While the Round 1 ATT&CK Evaluation (the first year of testing) was based on APT3 (Gothic Panda), and the Round 2 ATT&CK Evaluation focused on TTPs associated with APT29 (Cozy Bear), this year’s evaluation focuses on emulating financial threat groups. Testing day 1 simulates the Carbanak adversary group’s attack methodology. Their objective? Breach the HR Manager, quietly move about the network, identify payment data, and exfiltrate it. It involves 4 Windows computers and a Linux server and consists of 96 techniques in 10 steps. See the Carbanak emulation.
Testing day 2 simulates the FIN7 adversary group. Similarly, their objective is to steal financial data. This simulation involves five computers and 78 techniques in 10 steps.
Visibility is the Foundation of Best-In-Class EDR
1. SentinelOne is the ONLY vendor to deliver 100% visibility with ZERO missed detections across all tested operating systems – Windows & Linux
The foundation of a superior EDR solution is its ability to consume pertinent SecOps data at scale across a variety of OSes and cloud workloads while missing nothing in the process. With the increased sophistication and frequency of today’s attacks, depth and breadth of visibility are fundamental capabilities that an EDR solution should deliver. Having no gaps in visibility means no blind spots, significantly reducing the attacker’s ability to operate undetected.
Complete in-depth visibility is table stakes for any worthy EDR solution. No visibility, no breach protection!
As the ATT&CK evaluation data shows, SentinelOne had ZERO misses in this round. We detected 100% of attacks over Windows devices as well as Linux servers.
Detection Quality Separates the Wheat from the Chaff
2. SentinelOne delivered the MOST high-quality analytic detections to provide automated instant insight into adversary actions.
Analytics Detection Coverage (a count of any non-telemetry detection) rather than Detection Counts should be a factor to consider when deciding on the best EDR solution. Having a high number of general, tactic, or technique detections leads to higher quality detections because this ensures fewer attacks are missed. Having access to high-fidelity, high-quality detections gives enterprises more time to investigate events rather than searching through a sea of data that may be predominantly false positives.
In the ATT&CK evaluation, “Techniques” and “Tactics” are the key measures of data precision.
- Technique: The epitome of relevant and actionable data – fully contextualized data points that tell a story, indicating what happened, why it happened, and crucially, how it happened.
- Tactic: The next level down in the hierarchy, representing categories of techniques that tell us the actor’s steps in achieving their ultimate goals (persistence, data egress, evasions, etc.) In short, the ‘what’ and the ‘why.’
These two detection classifications are the core of the MITRE ATT&CK framework and are of the highest value in creating context. According to MITRE Engenuity’s published results, out of all participants in this evaluation, SentinelOne recorded the highest number of analytic detections.
Detection Delays are Deadly
3. SentinelOne experienced zero delayed detections, making EDR real-time.
Time is a critical factor whether you’re detecting an attack or neutralizing it.
A delayed detection, according to MITRE Engenuity, is not immediately available to the analyst; it may come in minutes or hours after the adversary has performed the malicious activity.
A delayed detection during the evaluation often means that an EDR solution required a human analyst to manually confirm suspicious activity due to the inability of the solution to do so on its own. The solution typically needs to send data to the analyst team or third-party services such as sandboxes, which in turn analyzes the data and alerts the customer, if required. However, many critical parts of this process are done manually, resulting in a window of opportunity for the adversary to do real damage.
Adversaries operating at high speed must be countered with machine speed automation that’s not subject to the inherent slowness of humans.
As the ATT&CK evaluation data shows, SentinelOne had zero delayed detections in this evaluation.
Configuration Changes Highlight Fragility & Scaling Problems
4. SentinelOne required zero configuration changes, making EDR effortless.
According to MITRE Engenuity, Config change refers to any detection that was made possible only because the vendor changed the initial configuration.
However, in a real-world scenario, SOC operators do not have time to customize settings, especially during an ongoing attack. Constantly tuning, fine-tuning, and adjusting a product means the battle is lost before it starts. In reality, SOC operators wouldn’t even know what changes to make. Without an alert, they would not know what to look for to drive the configuration change.
Technology-powered solutions should work at an enterprise-scale right out of the box to realize immediate time-to-value. SentinelOne Enterprise-Grade EDR deploys in seconds and works at total capacity instantly, as shown by the MITRE Engenuity evaluation data.
Storyline Automatically Connects the Dots
5. SentinelOne produced one console alert per targeted device.
Ask any SOC Operator about their biggest frustrations, and alert fatigue will be high among them. They constantly struggle to identify the serious threat indicators while wading through false positives. Rather than getting alerted on every piece of telemetry within an incident and fatiguing the already-burdened SOC team, an EDR solution should eliminate the noise before it reaches you by automatically grouping individual data points into combined alerts.
Consolidating hundreds of data points across a 48-hour advanced campaign, SentinelOne correlated and crystallized the attack into one complete story, represented as a single alert per target machine. SentinelOne provides instant insights within seconds rather than having analysts spend hours, days, or weeks correlating logs and linking events manually.
SentinelOne reduces the amount of manual effort needed, helps with alert fatigue, and significantly lowers the skillset barrier of benefiting from EDR.
What the Results Mean for You
As a security leader, it’s important that you look at how you can improve your security posture and reduce risk while reducing the burden on your security team. While evaluating, look for an EDR solution that:
- Provides complete visibility without any blind spots
- Automatically correlates detections instead of relying on humans to interpret and manually stitch the data
- Defeats adversaries in real-time
- Works out-of-the-box as expected without needing continuous tune-ups
- Includes granular remediation capabilities for automated cleanup and recovery
SentinelOne’s exceptional performance in 2020 ATT&CK evaluations once again prove that purpose-built, future-thinking solutions deliver the in-depth visibility, automation, and speed that the modern SOC needs to combat adversaries. As evidenced by the results data, SentinelOne excels at visibility and detection, and even more importantly, in the autonomous mapping and correlating of data into fully indexed and correlated stories through Storyline™ technology. This technology advantage sets us apart from every other vendor on the market.
To learn more about SentinelOne’s performance in the 2020 MITRE Engenuity ATT&CK Evaluation, register for the upcoming webinar on Monday, April 26 at 10 a.m. PDT.
