Fuzz introspector is a tool to help fuzzer developers to get an understanding of their fuzzer’s performance and identify any potential blockers.

Overview

Fuzz introspector

Fuzz introspector is a tool to help fuzzer developers to get an understanding of their fuzzer’s performance and identify any potential blockers. Fuzz introspector aggregates the fuzzers’ functional data like coverage, hit frequency, entry points, etc to give the developer a birds eye view of their fuzzer. This helps with identifying fuzz bottlenecks and blockers and eventually helps in developing better fuzzers.

High-level goals:

  • Show fuzzing-relevant data about each function in a given project
  • Show reachability of fuzzer(s)
  • Integrate seamlessly with OSS-Fuzz
  • Show visualisations to enable fuzzer debugging
  • Give suggestions for how to improve fuzzing

Testing with OSS-Fuzz

The recommended way of testing this project is by way of OSS-Fuzz. Please see OSS-Fuzz instructions on how to do this.

Testing without OSS-Fuzz integration

You can also build and run the introspector outside the OSS-Fuzz environment.

We use this mainly to develop the LLVM LTO pass as compilation of clang goes faster (recompilation in particular). However, for the full experience we recommend working in the OSS-Fuzz environment as described above.

A complication with testing locally is that the full end-to-end process of both (1) building fuzzers; (2) running them; (3) building with coverage; and (4) building with introspector analysis, is better supported in the OSS-Fuzz environment.

Build locally

Start a python venv

  1. Create a venv: python3 -m venv /path/to/new/virtual/environment
  2. Activate the venv
  3. Install dependencies with pip install -r requirements.txt

Build custom clang

(expect this part to take at least 1 hour)

git clone https://github.com/AdaLogics/fuzz-introspector
cd fuzz-introspector
./build_all.sh

Run local examples

After having built the custom clang above, you can try an example:

cd examples
./build_simple_examples.sh
cd simple-example-4/web
python3 -m http.server 5002

You can also use the build_all_projects.sh and build_all_web_only.sh scripts to control which examples you want to build as well as whether you want to only build the web data.

Output

The output of the introspector is a HTML report that gives data about your fuzzer. This includes:

  • An overview of reachability by all fuzzers in the repository
  • A table with detailed information about each fuzzer in the repository, e.g. number of functions reached, complexity covered and more.
  • A table with overview of all functions in the project. With information such as
    • Number of fuzzers that reaches this function
    • Cyclomatic complexity of this function and all functions reachable by this function
    • Number of functions reached by this function
    • The amount of undiscovered complexity in this function. Undiscovered complexity is the complexity not covered by any fuzzers.
  • A call reachability tree for each fuzzer in the project. The reachability tree shows the potential control-flow of a given fuzzer
  • An overlay of the reachability tree with coverage collected from a fuzzer run.
  • A table giving summary information about which targets are optimal targets to analyse for a fuzzer of the functions that are not being reached by any fuzzer.
  • A list of suggestions for new fuzzers (this is super naive at the moment).

Example output

Here we show a few images from the output report:

Project overview:

project overview

Table with data of all functions in a project. The table is sortable to make enhance the process of understanding the fuzzer-infrastructure of a given project:

Functions table

Reachability tree with coverage overlay

Overlay 1

Reachability tree with coverage overlay, showing where a fuzz-blocker is occurring Overlay 2

Contribute

Code of Conduct

Before contributing, please follow our Code of Conduct.

Connect with the Fuzzing Community

If you want to get involved in the Fuzzing community or have ideas to chat about, we discuss this project in the OSSF Security Tooling Working Group meetings.

More specifically, you can attend Fuzzing Collaboration meeting (monthly on the first Tuesday 10:30am - 11:30am PST Calendar, Zoom Link).

Comments
  • /usr/bin/ld.gold: fatal error: LLVM gold plugin: <unknown>:0: Undefined temporary symbol .Ltmp265928

    /usr/bin/ld.gold: fatal error: LLVM gold plugin: :0: Undefined temporary symbol .Ltmp265928

    Was running ../run_both.sh bitcoin-core 3, but it failed.

    ...
    [Log level 2] : 13:06:58 : Wrapping function sancov.module_ctor_8bit_counters.86728
    [Log level 2] : 13:06:58 : Wrapping function event_listener_getbase
    [Log level 2] : 13:06:58 : Wrapping function event_listener_getfd
    [Log level 2] : 13:06:58 : Wrapping function event_listener_destroy
    [Log level 2] : 13:06:58 : Wrapping function event_listener_disable
    [Log level 2] : 13:06:58 : Wrapping function event_listener_enable
    [Log level 2] : 13:06:58 : Wrapping function evconnlistener_set_error_cb
    [Log level 2] : 13:06:58 : Wrapping function evconnlistener_set_cb
    [Log level 2] : 13:06:58 : Wrapping function evconnlistener_get_base
    [Log level 2] : 13:06:58 : Wrapping function evconnlistener_get_fd
    [Log level 2] : 13:06:58 : Wrapping function evconnlistener_disable
    [Log level 2] : 13:06:58 : Wrapping function evconnlistener_free
    [Log level 2] : 13:06:58 : Wrapping function evconnlistener_new_bind
    [Log level 2] : 13:06:58 : Wrapping function evconnlistener_new
    [Log level 2] : 13:06:58 : Wrapping function listener_read_cb
    [Log level 2] : 13:06:58 : Wrapping function evconnlistener_enable
    [Log level 2] : 13:06:58 : Wrapping function sancov.module_ctor_8bit_counters.86775
    [Log level 2] : 13:06:58 : Wrapping function evthread_posix_get_id
    [Log level 2] : 13:06:58 : Wrapping function evthread_posix_cond_wait
    [Log level 2] : 13:06:58 : Wrapping function pthread_cond_timedwait
    [Log level 2] : 13:06:58 : Wrapping function pthread_cond_wait
    [Log level 2] : 13:06:58 : Wrapping function evthread_posix_cond_signal
    [Log level 2] : 13:06:58 : Wrapping function pthread_cond_broadcast
    [Log level 2] : 13:06:58 : Wrapping function pthread_cond_signal
    [Log level 2] : 13:06:58 : Wrapping function evthread_posix_cond_free
    [Log level 2] : 13:06:58 : Wrapping function pthread_cond_destroy
    [Log level 2] : 13:06:58 : Wrapping function evthread_posix_cond_alloc
    [Log level 2] : 13:06:58 : Wrapping function pthread_cond_init
    [Log level 2] : 13:06:58 : Wrapping function evthread_posix_unlock
    [Log level 2] : 13:06:58 : Wrapping function evthread_posix_lock
    [Log level 2] : 13:06:58 : Wrapping function pthread_mutex_trylock
    [Log level 2] : 13:06:58 : Wrapping function evthread_posix_lock_free
    [Log level 2] : 13:06:58 : Wrapping function evthread_posix_lock_alloc
    [Log level 2] : 13:06:58 : Wrapping function evthread_use_pthreads
    [Log level 2] : 13:06:58 : Wrapping function pthread_mutexattr_init
    [Log level 2] : 13:06:58 : Wrapping function pthread_mutexattr_settype
    [Log level 2] : 13:06:58 : Ended wrapping all functions
    [Log level 1] : 13:06:59 : Finished introspector module
    /usr/bin/ld.gold: fatal error: LLVM gold plugin: <unknown>:0: Undefined temporary symbol .Ltmp265928
    
    clang-14: error: linker command failed with exit code 1 (use -v to see invocation)
    make[2]: *** [Makefile:6708: test/fuzz/fuzz] Error 1
    make[2]: Leaving directory '/src/bitcoin-core/src'
    make[1]: *** [Makefile:17510: all-recursive] Error 1
    make[1]: Leaving directory '/src/bitcoin-core/src'
    make: *** [Makefile:812: all-recursive] Error 1
    ERROR:root:Building fuzzers failed.
    
    bug 
    opened by MarcoFalke 24
  • Possible incorrect coverage interpretation?

    Possible incorrect coverage interpretation?

    Looking into bind9 fuzz report for dns_rdata_fromwire_text_fuzzer, I encounter multiple inconsistent/confusing entries in the calltree:

    for example in calltree idx: 00539, the callsite link shows 352k hits, while the node in call tree is red. It is the same for calltree idx: 00088 with callsite link

    Can it be because the coverage is reporting hits from other fuzz targets? If yes, then #62 can be the solution.

    bug 
    opened by Navidem 15
  • bump oss-fuzz

    bump oss-fuzz

    We should bump OSS-Fuzz as a reasonbly high number of changes has happened since last bump. There was a slight change in the way post-processing unit is called, so a few minor things need change in OSS-Fuzz besides bumping the LLVM number.

    @Navidem @AdamKorcz do you have anything that you would like to complete before bumping on OSS-Fuzz side?

    opened by DavidKorczynski 13
  • Map fuzzer names to output binary names in OSS-Fuzz

    Map fuzzer names to output binary names in OSS-Fuzz

    Current fuzz introspector reports seem to key fuzzers by the filename where the fuzzer is defined (e.g. https://oss-fuzz-introspector.storage.googleapis.com/zstd/inspector-report/20220220/fuzz_report.html#Fuzzer:-sequence_compression_api.c)

    For closer integration with OSS-Fuzz and ClusterFuzz though, we'd like to be able to better map the binary names we see on OSS-Fuzz to these reports. @DavidKorczynski @AdamKorcz WDYT? Would it be possible to include the actual binary names in these reports and key on that instead?

    @Navidem FYI

    enhancement core feature 
    opened by oliverchang 11
  • migrate runner.py features into oss-fuzz/infra/helper.py

    migrate runner.py features into oss-fuzz/infra/helper.py

    Making an issue of https://github.com/ossf/fuzz-introspector/pull/525#issuecomment-1302464937

    oss_fuzz_integration/runner.py has a few features that are convenient for building and running fuzzers by way of oss-fuzz, including:

    • automatically downloading public corpus, which can be used to construct full coverage reports
    • run commands such as: python3 ../runner.py {coverage | introspector} proj_name exec_sec which will build fuzzers of proj_name with the default sanitizer, run the fuzzers for exec_sec seconds and then generate a coverage or introspector report.

    The features are useful when improving fuzzers for a given project as it makes the workflow fast.

    Some of these features would make sense to add to OSS-Fuzz, in particular coverage generation using public corpus, generation of fuzz introspector reports for a given project and also generation just coverage for a given project.

    opened by DavidKorczynski 10
  • Should we always bail if there is a main() in module?

    Should we always bail if there is a main() in module?

    Currently if there is a main() function in the module, introspector pass is skipped.

    Should this be the case all the time? There are at least 4 projects on OSS-Fuzz that fuzz introspector does not generate fuzz_report.html because of this check. Projects include: tmux, tarantool, libssh, libspectre

    @DavidKorczynski WDYT?

    needs discussion 
    opened by Navidem 10
  • Feature: Add function-of-interest reachability lookup

    Feature: Add function-of-interest reachability lookup

    It will be useful to employ the reachability data in a way that the user can lookup a function-of-interest to find out which recommended fuzz target may reach to the FOI.

    enhancement 
    opened by Navidem 10
  • numeric metric for calltree bitmap?

    numeric metric for calltree bitmap?

    It may be useful for devs to compare the improvements they made wrt calltree bitmap. Right now the only way to do this is to eyeball the colouring on the report.

    Would it make sense to add a percentage value here?

    opened by oliverchang 9
  • [OSS-Fuzz] Introspector build failures since using new PM

    [OSS-Fuzz] Introspector build failures since using new PM

    Since merge of https://github.com/google/oss-fuzz/pull/7788, around 59 projects fail to build.

    As I checked some including json, valijson, wabt, wolfssl, and znc, the error message is:

    /usr/bin/ld.gold: fatal error: LLVM gold plugin: <unknown>:0: Undefined temporary symbol .LtmpXXXXX
    
    opened by Navidem 9
  • Exclude std:: functions from fuzz introspector reports

    Exclude std:: functions from fuzz introspector reports

    C++ std::.. calls can be very noisy and likely aren't very useful to fuzzer developers. e.g. for leveldb: https://storage.googleapis.com/oss-fuzz-introspector/leveldb/inspector-report/20220316/calltree_view_0.html

    @Navidem and I discussed this and thought that we should just exclude all of these from the calltree.

    @DavidKorczynski @AdamKorcz WDYT?

    opened by oliverchang 8
  • jvm issues

    jvm issues

    Umbrella issue for minor jvm issues

    • runtime coverage functions is above reachable functions
    • urls is missing some parts (apache-commons-cli is an example), including .java
    opened by DavidKorczynski 7
  • Parse control-flow collected in a way other than LTO

    Parse control-flow collected in a way other than LTO

    To make introspector more versatile it makes sense to accept control-flow collected by other ways and just load it in post-processing. One alternative to LTO is using sancov: https://clang.llvm.org/docs/SanitizerCoverage.html#tracing-control-flow

    opened by Navidem 0
  • Documentation: improve readthedocs

    Documentation: improve readthedocs

    The goal is to drastically improve https://fuzz-introspector.readthedocs.io/en/latest/ to make it a standard page for getting information on fuzz-introspector

    In particular:

    • provide improved installation guides (e.g. for recently added languages)
    • provide a number of tutorials on how to use fuzz introspector
    • provide guides that show why the data in fuzz-introspector is useful
    • provide instructions on how to use Fuzz Introspector from an OSS-Fuzz perspective
    • more developer-friendly docs
    opened by DavidKorczynski 0
  • Add support for diffing two fuzz-introspector runs

    Add support for diffing two fuzz-introspector runs

    The goal of fuzz introspector is by and large to make it easier to improve a fuzzing set up for a given software package. At the moment fuzz introspector only focuses on a single analysis, whereas, in order to determine if an improvement was successful one has to compare two fuzz introspector runs. As such, we should have some features that make it possible to compare fuzz introspector analyses and specifically make it easy to highlight improvements/regressions.

    opened by DavidKorczynski 1
  • tinygltf has calls to asan functions in its report

    tinygltf has calls to asan functions in its report

    I noticed the TinyGltf project has a set of calls to ASAN routines. https://storage.googleapis.com/oss-fuzz-introspector/tinygltf/inspector-report/20221210/fuzz_report.html

    This should not happen as we aim to exclude them from the frontends.

    opened by DavidKorczynski 0
  • JVM implementation frontends code is slow and used up loads of memory and result in stack / memory overflow

    JVM implementation frontends code is slow and used up loads of memory and result in stack / memory overflow

    After the recent update of the JVM frontends code, the execution time and memory usage is increased significantly, which sometimes result in out of memory and stack overflow. Double check of the logic and settings are needed to ensure the code run in acceptable time and resources.

    opened by arthurscchan 1
Releases(v1.0.0)
Owner
Open Source Security Foundation (OpenSSF)
Open Source Security Foundation (OpenSSF)
PoC encrypted diary in Python 3

Encrypted diary Sample program to store confidential data. Provides encryption in the form of AES-256 with bcrypt KDF. Does not provide authentication

1 Dec 25, 2021
Solución al reto BBVA Contigo, Hack BBVA 2021

Solution Solución propuesta para el reto BBVA Contigo del Hackathon BBVA 2021. Equipo Mexdapy. Integrantes: David Pedroza Segoviano Regina Priscila Ba

Gabriel Missael Barco 2 Dec 06, 2021
This repo explains in details about buffer overflow exploit development for windows executable.

Buffer Overflow Exploit Development For Beginner Introduction I am beginner in security community and as my fellow beginner, I spend some of my time a

cris_0xC0 11 Dec 17, 2022
A small Minecraft server to help players detect vulnerability to the Log4Shell exploit 🐚

log4check A small Minecraft server to help players detect vulnerability to the Log4Shell exploit 🐚 Tested to work between Minecraft versions 1.12.2 a

Evan J. Markowitz 4 Dec 23, 2021
Universal Radio Hacker: Investigate Wireless Protocols Like A Boss

The Universal Radio Hacker (URH) is a complete suite for wireless protocol investigation with native support for many common Software Defined Radios.

Dr. Johannes Pohl 9k Jan 03, 2023
Log4j2 CVE-2021-44228 revshell

Log4j2-CVE-2021-44228-revshell Usage For reverse shell: $~ python3 Log4j2-revshell.py -M rev -u http://www.victimLog4j.xyz:8080 -l [AttackerIP] -p [At

FaisalFs 16 Mar 24, 2022
BurpSuite Extension: Log4j2 RCE Scanner

Log4j2 RCE Scanner 作者:[email protected]元亨实验室 声明:由于传播、利用本项目所提供的信息而造成的任何直接或者间接的后果及损失,均由使用者本人负责,项目作者不为此承担任何责

ᴋᴇʏ 87 Dec 29, 2021
A Python tool to automate some dorking stuff to find information disclosures.

WebDork v1.0.3 A open-source tool to find publicly available sensitive information about Companies/Organisations! WebDork A Python tool to automate so

Rahul rc 123 Jan 08, 2023
an impacket-dependent script exploiting CVE-2019-1040

dcpwn an impacket-dependent script exploiting CVE-2019-1040, with code partly borrowed from those security researchers that I'd like to say thanks to.

QAX A-Team 71 Nov 30, 2022
Python3 script for scanning CVE-2021-44228 (Log4shell) vulnerable machines.

Log4j_checker.py (CVE-2021-44228) Description This Python3 script tries to look for servers vulnerable to CVE-2021-44228, also known as Log4Shell, a v

lfama 8 Feb 27, 2022
Details,PoC and patches for CVE-2021-45383 & CVE-2021-45384

CVE-2021-45383 & CVE-2021-45384 There are several network-layer vulnerabilities in the official server of Minecraft: Bedrock Edition (aka Bedrock Serv

20 Apr 07, 2022
Unauthenticated Sqlinjection that leads to dump data base but this one impersonated Admin and drops a interactive shell

Unauthenticated Sqlinjection that leads to dump database but this one impersonated Admin and drops a interactive shell

sam 16 Nov 09, 2022
Hikvision 流媒体管理服务器敏感信息泄漏

Hikvisioninformation Hikvision 流媒体管理服务器敏感信息泄漏 Options optional arguments: -h, --help show this help message and exit -u url, --url url

Henry4E36 13 Nov 09, 2022
A simple Burp Suite extension to extract datas from source code

DataExtractor A simple Burp Suite extension to extract datas from source code. Features in scope parsing file extensions to ignore files exclusion bas

Gwendal Le Coguic 86 Dec 31, 2022
🐝 ℹ️ Honeybee extension for export to IES-VE gem file format

honeybee-ies Honeybee extension for export a HBJSON file to IES-VE GEM file format Installation pip install honeybee-ies QuickStart import pathlib fro

Ladybug Tools 4 Jul 12, 2022
Gefilte Fish GMail filter creator

Gefilte Fish: GMail filter maker Gefilte Fish automates the creation of GMail filters. Use it like this: from gefilte import GefilteFish,

Ned Batchelder 31 Sep 28, 2022
com_media allowed paths that are not intended for image uploads to RCE

CVE-2021-23132 com_media allowed paths that are not intended for image uploads to RCE. CVE-2020-24597 Directory traversal in com_media to RCE Two CVEs

KIEN HOANG 67 Nov 09, 2022
PoC of proxylogon chain SSRF(CVE-2021-26855) to write file by testanull, censored by github

CVE-2021-26855 PoC of proxylogon chain SSRF(CVE-2021-26855) to write file by testanull, censored by github Why does github remove this exploit because

The Hacker's Choice 58 Nov 15, 2022
Security audit Python project dependencies against security advisory databases.

Security audit Python project dependencies against security advisory databases.

52 Dec 17, 2022