In the previous two posts, we looked at how to keep yourself informed when Apple make silent updates to macOS’s built-in security tools and how to run diffs on the MRT.app to get an understanding of what’s new. In this final post on macOS security updates, we’ll take a look at how Apple use whitelisting, blacklisting and Yara rules in XProtect and Gatekeeper and how to see what’s new.
Blacklists in XProtect
The XProtect bundle is located at
/Library/Apple/System/Library/CoreServices/XProtect.bundle
on 10.15 and above. For earlier versions, it’s the same path but without the /Library/Apple at the beginning.
Within the bundle, the items we are interested in are in the Resources folder. Be sure to use a local copy either by downloading from Apple’s sucatalog as described in Part 1, or copying the XProtect bundle inside CoreServices to a working directory in your home folder.
Let’s begin with the XProtect.meta.plist. This file appears to have two functions. The first is to block legitimate plug-ins such as Flash Player that fall below a specified minimum version. These older versions have generally been found to contain known security flaws that could potentially undermine the security of the OS. Whether an update is available or not is also recorded for each entry.
The second function of XProtect.meta.plist is to blacklist known malicious extensions. This is done by specifying both the extension’s bundle identifier and the developer ID.
As the images above show, this file is not obfuscated in any way, and a simple diff will show us the difference from one version to another. This file has burgeoned from a mere 6.5K a year ago to 23K today, with 129 developer IDs added to the blacklist since Dec 2018. However, in general, changes are infrequent, and since Safari extensions are now bundled as part of an Application, we suspect that the function of this file may entirely or partially have been superseded by the gk.db file that began to appear in the Resources folder with macOS 10.15 Catalina. Let’s take a look at that next.
If we dump this database, we can see that it consists of a bunch of entries for blocking certain Apple developer “Team IDs”.
Currently, there are 133 entries.
$ sqlite3 gk.db .dump | grep blocked_teams | wc -l
This is actually shorter by about nine entries than earlier versions, but note that some of the bundle identifiers match those found in the earlier XProtect.meta.plist.
First, let’s extract all the Developer Ids from the meta.plist and dump to a text file:
$ grep -A1 'Developer Identifier' XProtect.meta.plist | grep string | sed 's/[<string></string>]//g' | sort -u > metaIDs
Now let’s do the same thing with the gk database:
$ sqlite3 gk.db .dump | grep -i values | cut -d' -f2 | sort -u > gkIds
Inspection of these files shows that many of the same Team IDs exist in both, but there are also additional entries and omissions in each, too, making it difficult to determine exactly how these two files interrelate. One theory is that the entries in gk.db represent Developer IDs that have failed malware scans after attempting Notarization, but at the moment that remains unconfirmed. The gk.db is a much blunter tool than the meta.plist as it appears to blacklist all products signed by Team ID alone, whereas the meta.plist appears to be specifically focused on products that match both the Team ID and bundle identifier for each entry.
Discerning Changes to XProtect’s YARA Rules
The other two files in the XProtect. bundle function as detection rules or signatures for the built-in macOS “AV” engine. Both are lightly obfuscated but easily reversed. Of the two, the most important is the XProtect.yara file, which is an ASCII document containing a list of Yara rules. If you scroll through this document, you’ll see a bunch of signature definitions like this:
If you’re not familiar with how YARA rules work start here, but the basic idea is easy to understand, Each rule specifies some data that may exist in the file to be scanned, usually strings, and some conditions about how the data should be matched. The strings themselves are hex encoded ASCII or data bytes from a sample of the malware that the rule is intended to match.
Reversing individual strings is simple enough just by echoing the string on the command line and piping it through XXD or Rax2:
$ echo '6C61756E636863746C206C6F6164207E2F4C6962726172792F4C61756E63684167656E7473' | xxd -r -p
I use a short AppleScript in BBEdit’s Scripts folder to convert individual strings of interest:
tell application "BBEdit" set n to "" set t to contents of (selection of its front window) set x to (do shell script "echo " & t & "| xxd -r -p") as text set selection to x end tell
You could adapt that to convert the entire file instead of a selection, or use Python and a BBEdit text filter is another option. For the most part, we only want to see the changes since the previous version, and so to do that, we’ll first run a diff on the old and new version, and pipe that into a new document.
$ diff old_XProtect.yara new_XProtect.yara | grep > > changedXProtect.yara
If we grep that file for the “rule ” we get a nice count of how many new rules have been added. We can then check out what strings each rule detects by using our script or text filter to reverse the hex back into ASCII.
The remaining file is the XProtect.plist file, which appears to contain much of the same information as the .yara file, but in a different format. This file also does simple hash matching on files as well as string pattern matching.
Although it’s worth keeping an eye on, I haven’t seen any changes to this file in more than 12 months, and I strongly suspect that it’s been abandoned by Apple in favor of the YARA file. This file remains unchanged from at least Dec 2018 (XProtect version 2101 is the oldest one I still have on file) to Feb 2020 (XProtect version 2112). A BBEdit text filter for decoding the patterns is available here.
Whitelisting through Gatekeeper
That pretty much covers inspecting changes in XProtect, but the gk.db we looked at above – despite the initials, is not the same as the whitelisting functions provided by Gatekeeper, which live at the following path:
/private/var/db/
Herein there are two relevant bundles, gke.bundle and gkopaque.bundle.
The gkopaque.bundle has been around in some form or another since 10.9 Mavericks. Inside its resources folder we’ll find the gkopaque.db. Let’s look at the schema first:
We can examine the .tables to see what kind of data it holds.
There are three tables, but conditions only holds two entries related to Google Chrome and the merged table is empty. In contrast, if we “SELECT” everything from the whitelist table, we’ll find there’s over 70,000 entries. Each entry is two SHA1 hash blobs, that the table says represent “current” and “opaque”. Other researchers have found that this is either partially or entirely a list of legacy apps that Apple have deemed to be “OK”, but which due to changes in code signing format would not pass Gatekeeper on more modern versions of the operating system. If that’s correct, then the whitelisting in gkopaque is probably of little more than historical interest.
The new-to-Catalina gke.bundle contains two files, gke.auth and gk.db and its function is not entirely clear. Yes, that’s the same name as the file we earlier investigated inside the XProtect bundle, but despite the name, the two are not identical and it’s more likely that this bundle is involved in whitelisting rather than blacklisting. The gk.db here holds only two tables:
The timestamp_exceptions table is a data file, while the settings table is fairly sparse, with a single entry:
The Gke.auth file turns out to be an XML file and is straightforward to read.
Each entry includes a SHA1 cdhash that likely matches the same hash used in code signing. The file currently contains over 30,000 lines with some 2,623 individual entries. A cursory attempt to match any of the cdhashes from signed software in the Applications folder on my own machine did not find any matches, but given what we know of the overall structure of the security tools used in macOS, it seems fair to say that this file is used to allow applications that match the rule to pass Gatekeeper without further hindrance. Precisely why these two thousand or so exceptions need to be hardcoded is a mystery we will save for another day. The end of the file contains a UUID and a version number, which makes it easy to diff and check for updates.
Wrapping Up
As we have seen, macOS uses a combination of whitelisting, blacklisting and simple Yara rules to fight malware through XProtect and Gatkeeper. Earlier in this series we looked at the MRT.app, which Apple uses for post-infection clean up. Keeping an eye on changes to these technologies is useful for security researchers to ensure that any threat actors that Apple have detected are known to the rest of the security community. This is vital, for while Apple make a brave attempt to block and detect malicious threats, the nature of their tools means that they can be and regularly are bypassed.
We hope this series has given you some tools and examples to help you investigate those changes for yourself if you’re interested in doing so. And if you enjoyed this series of posts, sign up to any of our social media feeds (below the line) or the weekly blog newsletter (to the left) to find out when we post our next macOS content.
