Python Sigma Rule Parsing Library

This library attempts to abstract the handling of Sigma rules in Python. The rules are parsed using a schema defined with pydantic, and can be easily loaded from YAML files into a structured Python object.

from sigma.schema import Rule

# Load a rule into a python object
rule = Rule.from_yaml("test-rule.yml")

# Simple properties are accessible directly
print(rule.title)
print(rule.author)

# Detection conditions are also available unchanged
print(rule.detection.condition)
print(rule.detection.my_condition_name)

# Parsed/unified grammar from the condition is easy!
print(rule.detection.expression)

This project is under active development, and this readme may or may not reflect the most up-to-date documentation. In general, you should refer to the generated documentation (instructions for building below) and the command-line help output for details until the library/tools reach a stable state.

Installation

The library and command line interface can be installed using pip from github with:

# Install directly from github
pip install git+ssh://[email protected]/calebstewart/python-sigma.git

# Checkout the repo, then install
git clone [email protected]:calebstewart/python-sigma.git
cd python-sigma
pip install .

If you would like to participate in development, you should use Python Poetry to manage your virtual environment and dependencies. For more information see the Poetry documentation.

# Setup Python development environment
git clone [email protected]:calebstewart/python-sigma.git
cd python-sigma
poetry install

# Enter the virtual environment to interact with the package
poetry shell

# Type "exit" to leave the poetry virtual environment

Documentation

Documentation can be built using Sphinx from this repository. First, install the package with the documentation dependencies, then run make html from the docs/ directory:

# Install with the docs extras
poetry install -E docs

# Enter the poetry virtual environment
poetry shell

# Build the documentation
cd docs
make html

# Open the documentation in docs/_build/index.html

At this time, documentation is built automatically from docstrings and type-hinting in the project code itself. The plan is to eventually augment this auto-generated documentation, but that is a project for later after the API and CLI interfaces solidify. That being said, extensive examples and documentation have been added where appropriate using module docstrings throughout the project, so the documentation should at least be usable.

Command Line Interface

There is a command line interface exposed by the entrpoint sigma which is installed with this package. The sigma command provides subcommands for inspecting rule and configuration schema, viewing/updating the MITRE ATT&CK database cache, validating serializer or rule configurations, and converting rules using built-in or custom serializers.

This project is still under active development, and the interface could change at any time. You should check the built-in help by running sigma --help at the command line, however for completeness sake, the current help output/list of subcommands is:

$ sigma --help
Usage: sigma [OPTIONS] COMMAND [ARGS]...

  Sigma Rule conversion and validation CLI.

Options:
  --help  Show this message and exit.

Commands:
  convert    Convert Sigma rules to various formats using built-in or...
  list       List built-in transforms and serializers
  mitre      Browse and update the MITRE ATT&CK data cache
  schema     Dump the schema for rules, serializers, and transforms
  transform  Transform a list of rules using a list of transforms in a...
  validate   Validate Sigma rule or serializer schema

But... why?

The official Sigma repository contains the sigmac tool for converting sigma rules from sigma format to a variety of backend detection systems. However, this tool has aged poorly. The code is messy and hard to follow and documentation is limited. It appears the Sigma team is attempting to replace sigmac with pySigma, but the project is pretty new, and I wanted something I could iterate on and have control over in the short term.

Also, the processing of sigma rules simply seems overly complex in both cases. This may be a "grass is greener" problem on my part, but the worst case for me doing this is that I better understand the problems inherent in building a Sigma rule API/converter, and can hopefully give back to the community in some way in the future.

Lastly, I wanted to build this tool with a focus on modern API interfaces and aggressive documentation. I plan to utilize pydantic heavily to make validation of fields and values more straightforward and pythonic as well as provide a simple interface for others to ingest Sigma rules directly. For example, being able to load, inspect and possibly modify sigma rules from Python without using the conversion tool would be a great feature for teams trying to work Sigma into their automation pipeline.

All that being said, I want to be abundantly clear: The sigma project and all the code associated with it have been immensely helpful, and the above is not meant to dig on the team, their code or their contributions to the community. I greatly appreciate and admire all the hard work the SigmaHQ team has put into making the detection of malicious activity better over the years. I only hope that I can either learn something or maybe provide something useful back to the community myself. 😄