Introduction

Building detection is a complex task, especially with a constantly increasing amount of data sources. Keeping track of these data sources and their appropriate detection rules or avoiding duplicate detection rules covering the same techniques can give a hard time to detection engineers.

For a SOC, it is crucial to have an good overview and a clear understanding of its actual visibility and detection coverage in order to identify gaps, prioritize the development of new detection rules or onboard new data sources.

In this blog post, we will learn how DeTT&CT can help you build, maintain and score your visibility and detection coverage.

We will first talk about MITRE ATT&CK, which is a knowledge base of adversary TTPs (Tactics, Techniques and Procedures) and its “Navigator”, a matrix that visually describes adversary TTPs. Then, we will cover the structure and functionalities of DeTT&CT. Lastly, we will walk through the different steps to start documenting your own detection coverage.

ATT&CK

MITRE ATT&CK is a knowledge base of adversary TTPs based on real-world observations and used by adversaries against enterprise networks. While ATT&CK does cover some tools and software used by attackers, the focus of the framework is on how adversaries interact with systems to accomplish their objectives.

ATT&CK contains a set of techniques and sub-techniques organized into a set of tactics. Tactics represent the “why” of an ATT&CK technique, the adversary’s tactical objective for a particular action. Such tactical objective can be to gain initial access, achieve persistence, move laterally, exfiltrate data, and so on.

Techniques and sub-techniques represent “how” an adversary achieves a tactical objective. As an example, an adversary may create a new Windows service to repeatedly execute malicious payloads and to persist even after a reboot. There are many ways or techniques to achieve a tactical objective.

These tactics and techniques are represented in a matrix containing, at the time of writing, 14 tactics and 188 techniques.

Nowadays, MITRE ATT&CK is firmly established with security professionals and forms a common vocabulary both for offense and defense. Adversary emulation teams use it to plan engagements and create scenarios based on realistic techniques used by real-world adversaries, detection teams use ATT&CK to assess their detection coverage and find gaps in their defenses, and cyber threat intelligence (CTI) teams track adversaries and threat actor groups by their use of TTPs mapped to the ATT&CK framework.

MITRE ATT&CK™ contains plenty of valuable information on:

• TTPs (Tactics, Techniques and Procedures)
• Groups (threat actors)
• Software (software used by threat actors)
• Data sources (visibility required for detection)
• Mitigations

The relationship between these types of information can be visualised using the following diagram:

To help us visualise this matrix and highlight TTPs, MITRE provides a web interface called ATT&CK Navigator. There is an online instance allowing you to easily and quickly test its functionalities. But, if you intend to use it for more than testing, we highly recommend to have your own instance.

To install a local instance, clone the GitHub repository and follow the procedure as described in the documentation (https://github.com/mitre-attack/attack-navigator#Install-and-Run).

Even though we could use the ATT&CK Navigator to document our detection coverage, it lacks more complex functionalities such as a multi-level scoring, differentiation between visibility and detection and  separation based on platforms and data sources.

This gap is where DeTT&CT comes into play. Let us discover how this tool works and how it can help us build, maintain and score our visibility and detection coverage.

DeTT&CT

Purpose

DeTT&CT stands for Detect Tactics, Techniques & Combat Threats. This framework has been created  at the Cyber Defence Center of Rabobank and is developed and at the time of writing maintained by Marcus Bakker and Ruben Bouman.

The purpose of DeTT&CT is to assist blue teams using MITRE ATT&CK to score and compare data log source quality, visibility coverage and detection coverage. By using this framework, blue teams can quickly detect gaps in the detection or visibility coverage and prioritize the ingest of new log sources.

Functionalities

DeTT&CT delivers a framework than can map the information you have on the entities available in ATT&CK and help you manage your blue teams data, visibility, and detection coverage.

The DeTT&CT framework consists of different components:

• a Python tool (DeTT&CT CLI)
• YAML administration files
• the DeTT&CT Editor (to create and edit the YAML administration files)
• scoring tables for detections, data sources and visibility

DeTT&CT CLI is a python script (dettect.py) that works with six different modes:

• editor: start DeTT&CT editor web interface
• datasource (ds): data source mapping and quality
• visibility (v): visibility coverage mapping based on techniques and data sources
• detection (d): detection coverage mapping based on techniques
• group (g): threat actor group mapping
• generic (ge): includes: statistics on ATT&CK data source and updates on techniques, groups and software

You can either use the command line interface or launch the editor to create and manage the different YAML administration files.

The DeTT&CT Framework uses YAML files to administer data sources, visibility, techniques and groups. The following file types can be identified:

• Data sources administration
• Technique administration (visibility and detection coverage)
• Groups administration

We will talk about these administration files in a bit.

You can find administration file sample in the Github repository.

One of the first step in using DeTT&CT is making an inventory of your data sources by scoring the data quality.

Data sources

Data sources are the raw logs or events generated by systems, e.g., security appliances, network devices, and endpoints. ATT&CK has over 30 different data sources which are further divided into over 90 data components. All those data components are included in this framework. These data sources are administered within the data source administration YAML file. For each data source, among others, the data quality can be scored. Within ATT&CK, these data sources are listed within the techniques themselves (e.g. T1003 in the Detection section).

The data source scoring is based on multiple criteria from the data quality scoring table:

• Data completeness
• Data field completeness
• Timeliness
• Consistency
• Retention

Visibility

Visibility is used within DeTT&CT to indicate if you have sufficient data sources with sufficient quality available to be able to capture evidence for activities associated with ATT&CK techniques. Visibility is necessary to perform incident response, execute hunting investigations and build detections. Within DeTT&CT you can score the visibility coverage per ATT&CK technique. The visibility scores are administered in the technique administration YAML file.

Visibility scores are rated from 0 to 4:

Detection

Only when you have the right data sources with adequate data quality and available to you for data analytics, your visibility can be used to create new detections for ATT&CK techniques. Detections often trigger alerts and are hence followed up on by your blue team. Scoring and administering your detections is also done in the technique administration YAML file.

Detection scores are rated from -1 to 5:

You can assess the score of a detection based on the following table:

Practice 

Let us now walk through the different steps to build detection coverage and perform gap analysis against a threat actor group. First, we need to install DeTT&CT.

Installation

You can easily install DeTT&CT, either using an image from Docker Hub or installing it locally. As for ATT&CK Navigator, we strongly suggest installing DeTT&CT locally if you are documenting your own organization’s detection coverage.

To install it locally, clone the repo from Github and install the required packages. You also need to have Python 3.6 or higher.

Installation

To install DeTT&CT, run the following commands:

git clone https://github.com/rabobank-cdc/DeTTECT.gitcd DeTTECT
pip install -r requirements.txt

Once it is installed, you can either use the command line interface or launch the DeTT&CT Editor.

To launch DeTT&CT Editor, type in the following command:

python3 dettect.py e

• e: start DeTT&CT editor locally

This will automatically launch a web browser the editor interface.

Data source coverage

ATT&CK has over 30 different data sources, which are further divided into over 90 data components. All of the data components are included in this framework.

Using the YAML data source administration file you can administer your data sources and record the following:

• The date when you registered the entry in DeTT&CT
• The data when you connected the data source to your security data lake
• In which product(s) the data resides
• The type of system(s) the data source applies to
• A flag to indicate if the data source can be used in data analytics
• A possible comment
• Data quality

In addition to the pre-defined fields, you can add further information by using key-value pairs.

Let us first list our data sources using the DeTT&CT Editor.

Go to DeTT&CT Editor, select Data Sources and create a new file.

Then add data sources according to the data sources that you already have available.

Click “Add data source” and select one data source. MITRE ATT&CK data sources are documented on their website.

For example, let’s say that you have an EDR installed on your Windows and Linux endpoints. This EDR has the capability to monitor processes so we can add Process Creation data source.
Select the date since you are collecting this data source and the date you registered the data source in your data sources YAML file.

Keeping track of the dates can help you monitor your data source improvement. To generate a graph based on the data source administration file, you can run the command below:

python dettect.py ds -fd sample-data/data-sources-endpoints.yaml -g

The same kind of graph can be generated for visibility and detection improvement.

Enabling the switch “Data source enabled” to yes will set all data quality scores to 1. If you want your configuration to be more accurate, you can modify these values according to the data source quality scoring table.

Enable the “Available for data analytics” switch if you centralized logs in a SIEM for example.

You could also add “Process Creation” to your data sources if you collect Sysmon event ID (EID) 1 or Windows EID 4688 events for example.

Let’s add the following data sources to complete our example:

• Command Execution (Windows EID 4688 of cmd.exe, Powershell logging, bash_history, etc.)
• Windows Registry Key Creation (EDR, Windows EID 4656 or Sysmon EID 12, etc.)
• Network Traffic Flow (Netflow, Zeek logs, etc.)

Once you have added all your data sources, save your data source administration file by clicking on “Save YAML file”.

Now, we are going to convert this YAML file to a JSON file using the DeTT&CT CLI tool and load this JSON file as an ATT&CK layer into ATT&CK Navigator.

python3 dettect.py ds -fd ~/Downloads/data-sources-new.yaml -l

The relevant flags for this command are

• ds: select data source mode
• -fd: path to the data source administration YAML file
• -l: generate a data source layer for the ATT&CK Navigator

Go to the ATT&CK Navigator web page and select Open Existing Layer. Choose “Upload from local” and select the JSON file we just created using the command above.

This layer represents the MITRE ATT&CK mapping based on the data sources that we specified in our data source administration file.

The colours, as explained in the legend, represent the percentage of data sources available for that particular technique.

Let’s look at some techniques from the Privilege Escalation tactic.

As an example, the Logon Script (Windows) technique requires the following data source coverage:

• Windows Registry Key Creation
• Process Creation
• Command Execution

Fortunately for us, we already have all three data sources available.

But, the Network Logon Script requires the following data sources:

• Process Creation
• Command Execution
• Active Directory Object Modification
• File Modification

As we only have 2 data sources available, the ATT&CK layer shows a coverage of 26-50%.

If you would like to improve the coverage for this particular technique, you would now know which data sources you need to integrate next in your detection.

Using the ATT&CK Navigator, you can also compare this data source layer with a threat analysis ATT&CK layer to spot gaps in your detection based on that threat analysis. You can also compare it to another data source layer to emphasize the benefits of integrating an additional data source in your detection.

Visibility coverage

The next step is to have a good understanding of where we have visibility, the level of visibility and where we lack visibility.

To get started, we can generate a technique administration YAML file based on our data source administration file, which will give us rough visibility scores. By default the argument --yaml will only include techniques in the resulting YAML for which the visibility score is greater than 0. To include all ATT&CK techniques that apply to the platform(s) specified in the data source YAML file, add the argument: --yaml-all-techniques.

python3 dettect.py ds -fd ~/Downloads/data-sources-new.yaml --yaml

or

python3 dettect.py ds -fd ~/Downloads/data-sources-new.yaml --yaml --yaml-all-techniques

The relevant flags for this command are

• ds: select data source mode
• -fd: path to the data source administration YAML file
• --yaml: generate a technique administration YAML file with visibility scores based on the number of available data sources
• --yaml-all-techniques: includes all ATT&CK techniques in the generated YAML file that apply to the platform(s) specified in the data source YAML file (you need to provide the --yaml argument for this)

Within the resulting YAML file, you can adjust the visibility score per technique based on expert knowledge or based on the quality of a particular data source.

If you want to easily edit the technique administration YAML file, you can load it using DeTT&CT Editor.

Per technique, you can see and edit the rough visibility score assigned based on the data source administration file. If needed, you can assign different score for different platforms such as Windows, Linux, Network, or Cloud.

The score logbook will keep track of the changes within the score.

To visualize the visibility scores within an ATT&CK Navigator layer, run the following command and load the resulting file in ATT&CK Navigator.

python3 dettect.py v -ft ~/Downloads/techniques-administration-example-all.yaml -l

The relevant parameters and flags for this command are

• v: visibility coverage mapping based on techniques and data sources
• -ft: path the technique administration YAML file
• -l: generate a data source layer for the ATT&CK Navigator

Detection coverage

Now that we listed our data sources and have a good understanding of our visibility, we need to have a good understanding of where we have detection, the level of detection and the lack of detection we have.

Using the same YAML data source administration file we used for our visibility coverage we can administer our level of detection and record the following:

• The type of system(s) the detection applies to (e.g. Windows endpoints, Windows servers, Linux servers, crown jewel x, etc.).
• Where the detection resides (for example, it could be an event ID, the name of a detection rule/use case, SIEM, or a product name)
• A possible comment.
• The date when the detection was implemented or improved.
• A detection score.

In addition to the pre-defined fields, you can add further information by using key-value pairs.

To allow detailed scoring of your detections per type of system, you can select multiple detections per technique in the YAML file. This can be achieved using the “applicable_to” property.

We recommend using the same applicable_to values between your technique and your data source administration file. A score logbook enables you to keep track of changes in the score by having multiple score objects.

To review the details, click on the “Score logbook” button:

Do not forget to save your YAML file is you edit it with DeTT&CT Editor.

To generate a layer file for the ATT&CK Navigator based on the technique administration file, you can run the following command:

python3 dettect.py d -ft ~/Downloads/techniques-administration-example-all.yaml -l

The relevant parameters and flags for this command are

• d: detection coverage mapping based on techniques
• -ft: path the technique administration YAML file
• -l: generate a data source layer for the ATT&CK Navigator

As we gave a score to only one specific technique, then only this technique will appear in our layer.

Gap analysis against threat actor group

Additionally, you could compare your detection layer with your threat analysis layer or with a layer generated for a specific red team exercise to spot any gaps in your detection.

When performing adversary emulation, the red team will define a scope of techniques that mimics a known threat to an organization. They usually represent this scope by generating an ATT&CK matrix layer.

Let’s say this is the generated layer from the adversary emulation:

You can compare a threat actor group layer with either your detection or visibility coverage overlay. Use the following command to generate a layer that highlights the differences:

python3 dettect.py g -g sample-data/groups.yaml -o sample-data/techniques-administration-example-all.yaml -t detection

• g: threat actor group mapping
• -g: specify the ATT&CK Groups to include. Another option is to provide a YAML file with a custom group
• -o: specify what to overlay on the group(s). To overlay Visibility or Detection, provide the technique administration YAML file.
• -t {group,visibility,detection}: specify the type of overlay. You can choose between group, visibility or detection (default = group)

If we compare our detection layer to the red team exercise, we will have the following resulting layer:

Lastly, you can also generate a layer that will compare your visibility and detection coverage. This will give you a decent overview of the techniques where you have visibility or detection.

To generate this layer, type one of the following commands:

python dettect.py d -ft ~/Downloads/techniques-administration-endpoints.yaml -o

or

python dettect.py v -ft sample-data/techniques-administration-endpoints.yaml -o

Both commands will generate the same output as shown in the following picture.

Conclusion

In this blog post, we learned how to build, maintain and score visibility and detection coverage with MITRE ATT&CK and DeTT&CT.  Mapping your visibility and detection coverage to TTPs and visualizing it in the MITRE ATT&CK Navigator will help you better grasp your detection maturity. This also provides the possibility to compare your detection coverage against a threat actor behaviour and spot possible gaps.

Maintaining a clear understanding of your current detection capabilities are crucial for your overall security posture. With this knowledge, detection engineers can prioritize the development of new detection rules, and onboarding of new data sources, red teams can tailor their campaigns to test the defenders’ assumptions about their capabilities, and it helps decision makers to track progress and allocate resources to help improve the security posture.

Setting up a baseline for the DETT&CT framework requires some time and resources at first, but once it has been set up, it can provide you with insight on your current detection capabilities and where to focus on improvements.

References

“DeTT&CT: Mapping your Blue Team to MITRE ATT&CK™ — MB Secure”, https://www.mbsecure.nl/blog/2019/5/dettact-mapping-your-blue-team-to-mitre-attack

“MITRE ATT&CK®”, https://attack.mitre.org/

“ATT&CK 101. This post was originally published May… | by Blake Strom | MITRE ATT&CK® | Medium”, https://medium.com/mitre-attack/att-ck-101-17074d3bc62

“rabobank-cdc/DeTTECT: Detect Tactics, Techniques & Combat Threats”, https://github.com/rabobank-cdc/DeTTECT/

“MITRE DeTTECT – Data Source Visibility and Mapping – YouTube”, https://www.youtube.com/watch?v=EXnutTLKS5o

“ATT&CK® Navigator“, https://mitre-attack.github.io/attack-navigator/