Introduction
We are happy to announce that the awesome people maintaining the Sigma project on GitHub have merged our work to support the ee-outliers backend!
So what you can you do with this? Sigma already contained support for Elasticsearch through the es-dsl and es-qs backends. However, these generate queries then need to be integrated by the analyst in some way into their workflow, which is not always convenient: elastalert is a popular choice for alerting based on these queries, but we missed having the option to actually tag Sigma hits in our Elasticsearch clusters. This tagging is exactly what the ee-outliers backend in Sigma can do for you, by generating use cases that can be loaded into ee-outliers framework!
The reasons why we implemented this:
- Being able to not only alert on new Sigma hits, but to also visualize and dashboard them in Kibana;
- Being able to automatically & continuously tag both new and historical data for Sigma hits;
- Being able to add metadata to the tagged Elasticsearch events such as the relevant MITRE ATT&CK techniques, which are automatically converted along from the Sigma rule onto the ee-outliers use case.
In what follows we look at an example Sigma rule which we convert into an ee-outliers use case in order to tag matching events in our cluster, and to visualize them.
Example
Let’s say we want to tag Sigma events in Elasticsearch matching the following Sigma rule: https://github.com/Neo23x0/sigma/blob/master/rules/windows/builtin/win_susp_mshta_execution.yml
With the latest release of Sigma, we can now use sigmac to convert this rule into an ee-outliers use case:
./tools/sigmac --target ee-outliers --config ./tools/config/winlogbeat.yml ./rules/windows/builtin/win_susp_mshta_execution.yml
The result of this command is the following generated ee-outliers use case, which :
[simplequery_sigma_cc7abbd0-762b-41e3-8a26-57ad50d2eea3]
es_dsl_filter = {"query": {"constant_score": {"filter": {"bool": {"must": [{"wildcard": {"winlog.event_data.Image.keyword": "*\\\\mshta.exe"}}, {"bool": {"should": [{"wildcard": {"winlog.event_data.CommandLine.keyword": "*vbscript*"}}, {"wildcard": {"winlog.event_data.CommandLine.keyword": "*.jpg*"}}, {"wildcard": {"winlog.event_data.CommandLine.keyword": "*.png*"}}, {"wildcard": {"winlog.event_data.CommandLine.keyword": "*.lnk*"}}, {"wildcard": {"winlog.event_data.CommandLine.keyword": "*.xls*"}}, {"wildcard": {"winlog.event_data.CommandLine.keyword": "*.doc*"}}, {"wildcard": {"winlog.event_data.CommandLine.keyword": "*.zip*"}}]}}]}}}}}
es_index = winlogbeat-*
outlier_type = Sigma hit, attack.defense_evasion, attack.t1140
outlier_reason = Sigma hit - MSHTA Suspicious Execution 01
outlier_summary = Sigma hit - MSHTA Suspicious Execution 01
run_model = 1
test_model = 0
These use cases can then be loaded into ee-outliers to tag events matching your Sigma rules. We refer to the documentation of ee-outliers on how to run use cases against your historical Elasticsearch data.
In short, you can simply run ee-outliers by pointing it to the use cases configuration files (like the example above). Example:
# Build the image docker build -t "outliers-dev" . # Run the image docker run \ --network=network_name \ -v "$PWD/defaults:/mappedvolumes/config" \ -i outliers-dev:latest python3 outliers.py interactive \ --config /mappedvolumes/config/outliers.conf \ --use-cases "/my/sigma/usecases/folder/*.conf"
By default, sigmac will write a single configuration file as output containing all the converted Sigma rules as use cases. However, the --use-cases parameter in ee-outliers expects a single use case per file. To bypass this, we simply run sigmac for each Sigma yaml rule individually that we want to convert. We are tracking support for multiple outlier use cases in a single configuration file on Github.
After running ee-outliers, you will see in Elasticsearch that your Sigma hits have been tagged, which can now be visualized and investigated in Kibana! You will also notice that the tags from the Sigma rules are automatically enriched in the event, too, so they can be dashboarded (as seen in one of the tables in the screenshot below).
What’s next?
We love the work being done by the authors of Sigma, and we plan to continue to open-source our contributions to the community; besides maintenance of the ee-outliers backend, we are actively submitting new Sigma rules to the community as well as improvements to existing rules after testing rules in production environments.
About the author
Daan Raman is in charge of NVISO Labs, the research arm of NVISO. Together with the team, he drives initiatives around innovation to ensure we stay on top of our game; innovating the things we do, the technology we use and the way we work form an essential part of this. Daan doesn’t like to write about himself in third-person. You can contact him at draman@nviso.be and find him on Twitter and LinkedIn.