Stacs-ci - A set of modules to enable integration of STACS with commonly used CI / CD systems

Related tags

Deep Learningstacs-ci
Overview

Shield Shield Shield Shield Shield Shield




Static Token And Credential Scanner
CI Integrations

What is it?

STACS is a YARA powered static credential scanner which supports source code, binary file formats, analysis of nested archives, composable rule-sets and ignore lists, and SARIF reporting.

This repository contains a set of modules to enable integration of STACS with commonly used CI / CD systems. Currently, supported is:

  • Github Actions

    • Fails the build on unsuppressed findings.
    • Automatically annotates pull requests with findings.
    • Automatically loads suppressions from a stacs.ignore.json in the root of the repository.
  • Generic CI Systems

    • Fails the build on unsuppressed findings.
    • Outputs findings to the console in formatted plain-text.
    • Automatically loads suppressions from a stacs.ignore.json in the scan directory.

Github Actions

This Github action enables running STACS as a Github action. This can be used to identify credentials committed in both source code, or even credentials accidentally compiled into binary artifacts - such as Android APKs, Docker images, RPM packages, ZIP files, and more!

If run as part of a pull request, this action automatically annotates a pull request with findings to allow simplified review integrated with existing code-review processes. As this integration does not use the Github security events framework, no additional subscription to Github is required, even for private repositories!

This action can also be used as part of a release event. Allowing scanning of binaries before publishing to catch credentials which may have been accidentally generated or included as part of the build process.

Additionally, this action can 'fail the build' if any static tokens and credentials are detected.

Appearance

If STACS detects a static credential during a pull request, a review comment will be added to the line containing the static credential:

Github Comment of finding

The STACS Github integration will even check the pull request to see whether there is an existing comment for this finding, preventing multiple comments being added to the same pull request on subsequent commits.

If the credential is found inside of an archive, in a part of a file not modified by the pull request, then a regular comment will be added to the triggering pull request.

Inputs

scan-directory

An optional sub-directory to scan, relative to the repository root. This allows scanning to be limited to a specific directory under the repository root.

Defaults to the repository root.

fail-build

Defines whether this action should 'fail the build' if any static token or credentials are detected. This will take any suppressed / ignore listed entries into account, allowing consumers to ignore known false positives - such as test fixtures.

Defaults to true

Example Usage

The following example scans the currently checked out commit and adds review comments for findings to an associated pull-request (see "Permissions" section below). If the trigger was not a pull-request, findings will instead be printed to the console and STACS CI will exit with a non-zero status (100) if unsupressed findings were present.

uses: stacscan/[email protected]

The following example scans a sub-directory in the repository. In this example the binaries/ sub-directory contains binary objects, compiled for release by another step of a Github actions pipeline.

uses: stacscan/[email protected]
with:
    scan-directory: 'binaries/'

The following example disables 'failing the build' if there are findings which have not been ignored / suppressed.

uses: stacscan/[email protected]
with:
    fail-build: false

Permissions

Please be aware that in order to annotate pull requests with comments, the action must also be granted write permissions to pull-requests. This can be done by adding the following to the respective job in your Github actions pipeline.

permissions:
    contents: read         # Required to read the repository contents (checkout).
    pull-requests: write   # Required to annotate pull requests with comments.

This is only required if running in response to pull-request triggers.

Generic CI

This repository can be integrated with a number of common CI systems using the provided Docker image, or Python module.

The pre-built Docker image greatly simplifies this process and provides a mechanism to quickly execute a STACS scan against a given directory, print the results in an actionable manner, and signal to the CI system that the build should fail on findings.

Appearance

If STACS detects a static credential, a results block will be printed to the console with information required to identify its location:

Terminal output of findings

If the credential is found within an archive, STACS will print a file tree to allow quick identification of exactly where the credential is.

Basic

The simplest form of executing the Generic CI integration can be performed using the following Docker command from the directory to be scanned. Using this default configuration Docker will complete with a non-zero exit code if any unsuppressed findings are found:

docker run -it -v $(pwd):/mnt/stacs/input stacscan/stacs-ci:latest

To prevent a non-zero exit code on unsuppressed findings, such as for initial 'dry run' style operation, the following command can be run:

docker run -it -e FAIL_BUILD=false -v $(pwd):/mnt/stacs/input stacscan/stacs-ci:latest

Jenkins

To be added.

Circle CI

To be added.

Comments
  • [0.1.6] Update to the latest STACS container.

    [0.1.6] Update to the latest STACS container.

    Overview

    Upgrades to the latest STACS container. See the STACS release notes for the new version for a list of changes in this container. Only changes to STACS-CI will be included below.

    πŸ› οΈ New Features

    • N/A

    🍩 Improvements

    • N/A

    πŸ› Bug Fixes

    • N/A
    opened by darkarnium 2
  • [0.1.5] Update to the latest STACS container.

    [0.1.5] Update to the latest STACS container.

    Overview

    PLEASE NOTE: There is a potentially breaking change as part of this update, as the Stripe rule has been relocated under SaaS. This modifies its reference to now be CredentialSaaSStripeAPI. Any previously suppressed findings for this rule will need to be updated to reflect this new identifier.

    πŸ› οΈ New Features

    • Upgrade to the latest STACS rules, which includes new rules for
      • PyPI Token
      • Slack Token
        • User (xoxp-...)
        • Bot (xoxb-...)
      • NPM
        • authToken
        • password
      • PKCS#12 / PFX
      • DER format RSA keys.
        • Detects keys with exponents 3 / 65537, and modulous sizes 64 / 128 / 256 / 512 / 1024.

    🍩 Improvements

    • Minor changes to AWS rule.
    • Simplify matching criteria for a number of rules.

    πŸ› Bug Fixes

    • N/A
    opened by darkarnium 2
  • [0.1.4] Github annotation off-by-one bug fix.

    [0.1.4] Github annotation off-by-one bug fix.

    Overview

    πŸ› οΈ New Features

    • N/A

    🍩 Improvements

    • Pulls in latest STACS rules (r57ce3ce).

    πŸ› Bug Fixes

    • Fix off-by-one edge case which caused Github pull-request annotation to fail.
      • This was triggered when a new file was added in a pull-request containing a credential on the first line.
    opened by darkarnium 1
  • [0.1.3] Update to the latest version of STACS.

    [0.1.3] Update to the latest version of STACS.

    Overview

    πŸ› οΈ New Features

    🍩 Improvements

    • Tweak to STACS-CI deployment pipeline to simplify testing prior to release.

    πŸ› Bug Fixes

    • N/A
    opened by darkarnium 1
  • [0.1.1] Use latest STACS release, absolute URLs for images

    [0.1.1] Use latest STACS release, absolute URLs for images

    Overview

    πŸ› οΈ New Features

    🍩 Improvements

    • N/A

    πŸ› Bug Fixes

    • Ensures that file with the finding appears in the change set.
      • This prevents attempts to add a review comment for a file which is already in the target branch.
      • In these cases a comment will be added instead.
    opened by darkarnium 1
  • [0.1.0] Change container tag format, add Action.

    [0.1.0] Change container tag format, add Action.

    Overview

    Unfortunately, Docker does not support + characters in tag names, so we cannot use semver package numbering to indicate rule version.

    πŸ› οΈ New Features

    • Add Github Action YAML.
    • Update STACS Rules on STACS CI release.

    🍩 Improvements

    • N/A

    πŸ› Bug Fixes

    • N/A
    opened by darkarnium 0
  • [0.1.0] Initial implementation of STACS CI

    [0.1.0] Initial implementation of STACS CI

    Overview

    Initial release of STACS CI Integrations.

    πŸ› οΈ New Features

    • Generic CI integration
    • Github Actions integration

    🍩 Improvements

    • N/A

    πŸ› Bug Fixes

    • N/A
    opened by darkarnium 0
  • Backport CI UI into STACS

    Backport CI UI into STACS

    Overview

    The STACS-CI UI should be moved back into STACS, removing th need for this repository to contain more than a Github action for STACS execution.

    opened by darkarnium 0
  • Documentation of the stacs.ignore.json

    Documentation of the stacs.ignore.json

    Hi!

    Great tool you are building here - happy with it working so well in a CI context and ease of config.

    I have searched a lot through your repos and am struggling to find docs on the ignore file? Some of the things you describe sound awesome but I am yet to find anything on how to do it :)

    Thanks in advance!

    documentation 
    opened by erzz 5
Releases(0.1.6)
  • 0.1.6(Mar 4, 2022)

    Overview

    Upgrades to the latest STACS container. See the STACS release notes for the new version for a list of changes in this container. Only changes to STACS-CI will be included below.

    πŸ› οΈ New Features

    • N/A

    🍩 Improvements

    • N/A

    πŸ› Bug Fixes

    • N/A
    Source code(tar.gz)
    Source code(zip)
  • 0.1.5(Jan 18, 2022)

    Overview

    PLEASE NOTE: There is a potentially breaking change as part of this update, as the Stripe rule has been relocated under SaaS. This modifies its reference to now be CredentialSaaSStripeAPI. Any previously suppressed findings for this rule will need to be updated to reflect this new identifier.

    πŸ› οΈ New Features

    • Upgrade to the latest STACS rules, which includes new rules for
      • PyPI Token
      • Slack Token
        • User (xoxp-...)
        • Bot (xoxb-...)
      • NPM
        • authToken
        • password
      • PKCS#12 / PFX
      • DER format RSA keys.
        • Detects keys with exponents 3 / 65537, and modulous sizes 64 / 128 / 256 / 512 / 1024.

    🍩 Improvements

    • Minor changes to AWS rule.
    • Simplify matching criteria for a number of rules.

    πŸ› Bug Fixes

    • N/A
    Source code(tar.gz)
    Source code(zip)
  • 0.1.4(Jan 11, 2022)

    Overview

    πŸ› οΈ New Features

    • N/A

    🍩 Improvements

    • Pulls in latest STACS rules (r57ce3ce).

    πŸ› Bug Fixes

    • Fix off-by-one edge case which caused Github pull-request annotation to fail.
      • This was triggered when a new file was added in a pull-request containing a credential on the first line.
    Source code(tar.gz)
    Source code(zip)
  • 0.1.3(Jan 10, 2022)

    Overview

    πŸ› οΈ New Features

    🍩 Improvements

    • Tweak to STACS-CI deployment pipeline to simplify testing prior to release.

    πŸ› Bug Fixes

    • N/A
    Source code(tar.gz)
    Source code(zip)
  • 0.1.2(Jan 7, 2022)

  • 0.1.1(Jan 4, 2022)

    Overview

    πŸ› οΈ New Features

    🍩 Improvements

    • N/A

    πŸ› Bug Fixes

    • Ensures that file with the finding appears in the change set.
      • This prevents attempts to add a review comment for a file which is already in the target branch.
      • In these cases a comment will be added instead.
    Source code(tar.gz)
    Source code(zip)
  • 0.1.0(Jan 2, 2022)

    Overview

    Initial release of STACS CI Integrations.

    πŸ› οΈ New Features

    • Github Actions

      • Fails the build on unsuppressed findings.
      • Automatically annotates pull requests with findings.
      • Automatically loads suppressions from a stacs.ignore.json in the root of the repository.
    • Generic CI Systems

      • Fails the build on unsuppressed findings.
      • Outputs findings to the console in formatted plain-text.
      • Automatically loads suppressions from a stacs.ignore.json in the scan directory.

    🍩 Improvements

    • N/A

    πŸ› Bug Fixes

    • N/A
    Source code(tar.gz)
    Source code(zip)
Owner
STACS
Static Token And Credential Scanner
STACS
Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations.

Pyserini Pyserini is a Python toolkit for reproducible information retrieval research with sparse and dense representations. Retrieval using sparse re

Castorini 706 Dec 29, 2022
A Neural Net Training Interface on TensorFlow, with focus on speed + flexibility

Tensorpack is a neural network training interface based on TensorFlow. Features: It's Yet Another TF high-level API, with speed, and flexibility built

Tensorpack 6.2k Jan 09, 2023
SemEval2022 Patronizing and Condescending Language (PCL) Detection

SemEval2022 Patronizing and Condescending Language (PCL) Detection This task is from SemEval 2022. What is Patronizing and Condescending Language (PCL

Daniel Saeedi 0 Aug 05, 2022
CAPRI: Context-Aware Interpretable Point-of-Interest Recommendation Framework

CAPRI: Context-Aware Interpretable Point-of-Interest Recommendation Framework This repository contains a framework for Recommender Systems (RecSys), a

RecSys Lab 8 Jul 03, 2022
Jupyter notebooks showing best practices for using cx_Oracle, the Python DB API for Oracle Database

Python cx_Oracle Notebooks, 2022 The repository contains Jupyter notebooks showing best practices for using cx_Oracle, the Python DB API for Oracle Da

Christopher Jones 13 Dec 15, 2022
The 1st place solution of track2 (Vehicle Re-Identification) in the NVIDIA AI City Challenge at CVPR 2021 Workshop.

AICITY2021_Track2_DMT The 1st place solution of track2 (Vehicle Re-Identification) in the NVIDIA AI City Challenge at CVPR 2021 Workshop. Introduction

Hao Luo 91 Dec 21, 2022
Zalo AI challenge 2021 task hum to song

Zalo AI challenge 2021 task Hum to Song pipeline: ChuαΊ©n bα»‹ dα»― liệu cho quΓ‘ trΓ¬nh train: Sα»­a cΓ‘c file đường dαΊ«n trong config/preprocess.yaml raw_path:

Vo Van Phuc 105 Dec 16, 2022
Pure python implementations of popular ML algorithms.

Minimal ML algorithms This repo includes minimal implementations of popular ML algorithms using pure python and numpy. The purpose of these notebooks

Alexis Gidiotis 3 Jan 10, 2022
Library to enable Bayesian active learning in your research or labeling work.

Bayesian Active Learning (BaaL) BaaL is an active learning library developed at ElementAI. This repository contains techniques and reusable components

ElementAI 687 Dec 25, 2022
MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition

MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition Paper: MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition accepted fo

64 Dec 18, 2022
Codes and scripts for "Explainable Semantic Space by Grounding Languageto Vision with Cross-Modal Contrastive Learning"

Visually Grounded Bert Language Model This repository is the official implementation of Explainable Semantic Space by Grounding Language to Vision wit

17 Dec 17, 2022
Isaac Gym Reinforcement Learning Environments

Isaac Gym Reinforcement Learning Environments

NVIDIA Omniverse 714 Jan 08, 2023
Implementation of 'lightweight' GAN, proposed in ICLR 2021, in Pytorch. High resolution image generations that can be trained within a day or two

512x512 flowers after 12 hours of training, 1 gpu 256x256 flowers after 12 hours of training, 1 gpu Pizza 'Lightweight' GAN Implementation of 'lightwe

Phil Wang 1.5k Jan 02, 2023
Label-Free Model Evaluation with Semi-Structured Dataset Representations

Label-Free Model Evaluation with Semi-Structured Dataset Representations Prerequisites This code uses the following libraries Python 3.7 NumPy PyTorch

8 Oct 06, 2022
Adversarial Color Enhancement: Generating Unrestricted Adversarial Images by Optimizing a Color Filter

ACE Please find the preliminary version published at BMVC 2020 in the folder BMVC_version, and its extended journal version in Journal_version. Datase

28 Dec 25, 2022
Autonomous Perception: 3D Object Detection with Complex-YOLO

Autonomous Perception: 3D Object Detection with Complex-YOLO LiDAR object detect

Thomas Dunlap 2 Feb 18, 2022
PyTorch implementation of "VRT: A Video Restoration Transformer"

VRT: A Video Restoration Transformer Jingyun Liang, Jiezhang Cao, Yuchen Fan, Kai Zhang, Rakesh Ranjan, Yawei Li, Radu Timofte, Luc Van Gool Computer

Jingyun Liang 837 Jan 09, 2023
RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation (CIKM'17)

RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation This is the implementation of RATE: Overcoming Noise and Spar

Yu Zhang 5 Feb 10, 2022
Pytorch implementation of RED-SDS (NeurIPS 2021).

Recurrent Explicit Duration Switching Dynamical Systems (RED-SDS) This repository contains a reference implementation of RED-SDS, a non-linear state s

Abdul Fatir 10 Dec 02, 2022
Edge Restoration Quality Assessment

ERQA - Edge Restoration Quality Assessment ERQA - a full-reference quality metric designed to analyze how good image and video restoration methods (SR

MSU Video Group 27 Dec 17, 2022