In recent years, security products utilizing application whitelisting have gained popularity as a cost-effective alternative for fighting malware and advanced persistent threats. The basic idea behind whitelisting is to deny execution permission to any application or process that has not been specifically approved. In this post, we will discuss the limitations of relying on whitelisting to combat both common threats and APTs.
First Generation Whitelisting
Whitelisting mechanisms were first introduced by operating system vendors (e.g., Windows’ AppLocker). They provided limited protection, were cumbersome to maintain and required skilled security administrators to be configured effectively. In addition, creating the initial “gold” image could be quite challenging. Furthermore, typical IT environments are complex and constantly changing due to, among other things, software updates and additional downloads, all of which must be maintained in the whitelisting database. This has led to the emergence of third-party whitelisting solutions offering simplified management. Some examples include Bit9 Parity (later to become Carbon Black) and McAfee Application Protection.
Let’s examine some of the common issues with the whitelisting approach, including why it cannot replace patching or protect against a variety of common exploitation vectors.
Exploitation of Whitelisted Applications and Processes
Modern attacks, and advanced persistent threats in particular, frequently take advantage of unpatched software vulnerabilities, which can allow malicious code to hijack the execution flow of whitelisted software. Once this occurs, an exploit might run additional malicious payloads that perform various activities on the targeted machine, such as data exfiltration and data collection. Modern malware does this by piggybacking valid processes, often to circumvent antivirus scanners that look for anomalies.
Trusted Directories and Certificates
The same evasion techniques used by malware creators against AV products are just as effective against whitelisting engines. In most cases, malware can run without interference if it is able to operate with system-level privileges, whether through a privilege escalation vulnerability or other methods of infection. For example, a common rule used in whitelisting allows binaries in C:Windows to run freely, which means any payload that is dropped in that location (after obtaining system privileges) can run undetected. Another common rule is to allow digitally signed binaries to run at all times, meaning that all an attacker has to do is obtain a valid private key and sign their own software.
After the Fact Custom Rules
Increasingly, attackers exploit vulnerabilities in the Windows operating systems, often by using specially crafted PowerPoint, Word, or Excel files. Since these are loaded and executed by legitimate, whitelisted applications, such attacks can circumvent whitelisting solutions. Sometime after the threat is detected and analyzed, whitelist vendors may respond by creating customized rules that prevent a related dll from being loaded by a specific application. These rules have three major drawbacks:
- They are only effective once the exploits have been fully analyzed and the damage has been done.
- They are not effective against similar exploits that target other Microsoft applications. For example, since OLE is a Windows component used by numerous Office applications, a PowerPoint-targeted rule does not protect other applications.
- They can negatively affect usability, since valid features introduced by packager.dll may be no longer available.
Exploiting OS Functionality
The ability of attacks to bypass whitelisting by exploiting the operating system is well documented by researchers. There has been an abundance of Java and Flash vulnerabilities in recent years, and built-in Windows mechanisms like WMI and PowerShell offer further attack vectors that whitelisting cannot address. Other methods attackers can use to bypass whitelisting include:
- Executing malicious activities as the system (“root”) user
- Running kernel code
- Using one of the numerous built-in Windows IT maintenance mechanisms, such as Task Scheduler, Windows Accessibility tools and others
Conclusion
Whitelisting mechanisms at their core are about establishing trust. They do introduce another layer of protection against malicious binaries on a system, but in most cases, this protection only remains effective against threats that target careless or non-admin users who download unauthorized software. Statistics show that APTs commonly utilize software exploits (whether 0-day or not) in legitimate applications to compromise machines. This implies that in the real world, relying solely on whitelisting against watering hole attacks, spearphishing emails, and many other forms of targeted attacks remains ineffective.
