MEDS: Enhancing Memory Error Detection for Large-Scale Applications

Related tags

Deep LearningMEDS
Overview

MEDS: Enhancing Memory Error Detection for Large-Scale Applications

Prerequisites

  • cmake and clang

Build MEDS supporting compiler

$ make

Build Using Docker

# build docker image
$ docker build -t meds .

# run docker image
$ docker run --cap-add=SYS_PTRACE -it meds /bin/bash

Testing MEDS

  • MEDS's testing runs original ASAN's testcases as well as MEDS specific testcases.

    • Copied ASAN's testcases in llvm/projects/compiler-rt/test/meds/TestCases/ASan
    • MEDS specific testcases in llvm/projects/compiler-rt/test/meds/TestCases/Meds

  • To run the test,

$ make test

Testing Time: 30.70s
 Expected Passes    : 183
 Expected Failures  : 1
 Unsupported Tests  : 50

Build applications with MEDS heap allocation and ASan stack and global

  • Given a test program test.cc,
$ cat > test.cc

int main(int argc, char **argv) {
  int *a = new int[10];
  a[argc * 10] = 1;
  return 0;
}
  • test.cc can be built using the option, -fsanitize=meds.
$ build/bin/clang++ -fsanitize=meds test.cc -o test
$ ./test

==90589==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x43fff67eb078 at pc 0x0000004f926d bp 0x7fffffffe440 sp 0x7fffffffe438
WRITE of size 4 at 0x43fff67eb078 thread T0
    #0 0x4f926c in main (/home/wookhyun/release/meds-release/a.out+0x4f926c)
    #1 0x7ffff6b5c82f in __libc_start_main /build/glibc-bfm8X4/glibc-2.23/csu/../csu/libc-start.c:291
    #2 0x419cb8 in _start (/home/wookhyun/release/meds-release/a.out+0x419cb8)

Address 0x43fff67eb078 is a wild pointer.
SUMMARY: AddressSanitizer: heap-buffer-overflow (/home/wookhyun/release/meds-release/a.out+0x4f926c) in main
Shadow bytes around the buggy address:
  0x08807ecf55b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x08807ecf55c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x08807ecf55d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x08807ecf55e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x08807ecf55f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x08807ecf5600: fa fa fa fa fa fa fa fa fa fa 00 00 00 00 00[fa]
  0x08807ecf5610: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x08807ecf5620: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x08807ecf5630: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x08807ecf5640: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x08807ecf5650: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==90589==ABORTING

Options

  • -fsanitize=meds: Enable heap protection using MEDS (stack and global are protected using ASAN)

  • -mllvm -meds-stack=1: Enable stack protection using MEDS

  • -mllvm -meds-global=1 -mcmodel=large: Enable global protection using MEDS

    • This also requires --emit-relocs in LDFLAGS

  • Example: to protect heap/stack using MEDS and global using ASAN

$ clang -fsanitize=meds -mllvm -meds-stack=1 test.c -o test
  • Example: to protect heap/global using MEDS and stack using ASAN
$ clang -fsanitize=meds -mllvm -meds-global=1 -mcmodel=large -Wl,-emit-relocs test.c -o test
  • Example: to protect heap/stack/global using MEDS
$ clang -fsanitize=meds -mllvm -meds-stack=1 -mllvm -meds-global=1 -mcmodel=large -Wl,--emit-relocs

Contributors

Owner
Secomp Lab at Purdue University
Secomp Lab at Purdue University
GitHub Repository
An intelligent, flexible grammar of machine learning.

An english representation of machine learning. Modify what you want, let us handle the rest. Overview Nylon is a python library that lets you customiz

Palash Shah 79 Dec 02, 2022
Official implementation for the paper: "Multi-label Classification with Partial Annotations using Class-aware Selective Loss"

Multi-label Classification with Partial Annotations using Class-aware Selective Loss Paper | Pretrained models Official PyTorch Implementation Emanuel

99 Dec 27, 2022
[ICCV 2021] Our work presents a novel neural rendering approach that can efficiently reconstruct geometric and neural radiance fields for view synthesis.

MVSNeRF Project page | Paper This repository contains a pytorch lightning implementation for the ICCV 2021 paper: MVSNeRF: Fast Generalizable Radiance

Anpei Chen 529 Dec 30, 2022
The code for our paper "NSP-BERT: A Prompt-based Zero-Shot Learner Through an Original Pre-training Task —— Next Sentence Prediction"

The code for our paper "NSP-BERT: A Prompt-based Zero-Shot Learner Through an Original Pre-training Task —— Next Sentence Prediction"

Sun Yi 201 Nov 21, 2022
RETRO-pytorch - Implementation of RETRO, Deepmind's Retrieval based Attention net, in Pytorch

RETRO - Pytorch (wip) Implementation of RETRO, Deepmind's Retrieval based Attent

Phil Wang 556 Jan 04, 2023
Simple object detection app with streamlit

object-detection-app Simple object detection app with streamlit. Upload an image and perform object detection. Adjust the confidence threshold to see

Robin Cole 68 Jan 02, 2023
Train/evaluate a Keras model, get metrics streamed to a dashboard in your browser.

Hera Train/evaluate a Keras model, get metrics streamed to a dashboard in your browser. Setting up Step 1. Plant the spy Install the package pip

Keplr 495 Dec 10, 2022
Distributed Asynchronous Hyperparameter Optimization better than HyperOpt.

UltraOpt : Distributed Asynchronous Hyperparameter Optimization better than HyperOpt. UltraOpt is a simple and efficient library to minimize expensive

98 Aug 16, 2022
DAN: Unfolding the Alternating Optimization for Blind Super Resolution

DAN-Basd-on-Openmmlab DAN: Unfolding the Alternating Optimization for Blind Super Resolution We reproduce DAN via mmediting based on open-sourced code

AlexZou 72 Dec 13, 2022
RoboDesk A Multi-Task Reinforcement Learning Benchmark

RoboDesk A Multi-Task Reinforcement Learning Benchmark If you find this open source release useful, please reference in your paper: @misc{kannan2021ro

Google Research 66 Oct 07, 2022
A library for finding knowledge neurons in pretrained transformer models.

knowledge-neurons An open source repository replicating the 2021 paper Knowledge Neurons in Pretrained Transformers by Dai et al., and extending the t

EleutherAI 96 Dec 21, 2022
Definition of a business problem according to Wilson Lower Bound Score and Time Based Average Rating

Wilson Lower Bound Score, Time Based Rating Average In this study I tried to calculate the product rating and sorting reviews more accurately. I have

3 Sep 30, 2021
Ultra-Data-Efficient GAN Training: Drawing A Lottery Ticket First, Then Training It Toughly

Ultra-Data-Efficient GAN Training: Drawing A Lottery Ticket First, Then Training It Toughly Code for this paper Ultra-Data-Efficient GAN Tra

VITA 77 Oct 05, 2022
Facilitating Database Tuning with Hyper-ParameterOptimization: A Comprehensive Experimental Evaluation

A Comprehensive Experimental Evaluation for Database Configuration Tuning This is the source code to the paper "Facilitating Database Tuning with Hype

DAIR Lab 9 Oct 29, 2022
Similarity-based Gray-box Adversarial Attack Against Deep Face Recognition

Similarity-based Gray-box Adversarial Attack Against Deep Face Recognition Introduction Run attack: SGADV.py Objective function: foolbox/attacks/gradi

1 Jul 18, 2022
Links to works on deep learning algorithms for physics problems, TUM-I15 and beyond

Links to works on deep learning algorithms for physics problems, TUM-I15 and beyond

Nils Thuerey 1.3k Jan 08, 2023
graph-theoretic framework for robust pairwise data association

CLIPPER: A Graph-Theoretic Framework for Robust Data Association Data association is a fundamental problem in robotics and autonomy. CLIPPER provides

MIT Aerospace Controls Laboratory 118 Dec 28, 2022
Simple tutorials on Pytorch DDP training

pytorch-distributed-training Distribute Dataparallel (DDP) Training on Pytorch Features Easy to study DDP training You can directly copy this code for

Ren Tianhe 188 Jan 06, 2023
Working demo of the Multi-class and Anomaly classification model using the CLIP feature space

👁️ Hindsight AI: Crime Classification With Clip About For Educational Purposes Only This is a recursive neural net trained to classify specific crime

Miles Tweed 2 Jun 05, 2022
Stratified Transformer for 3D Point Cloud Segmentation (CVPR 2022)

Stratified Transformer for 3D Point Cloud Segmentation Xin Lai*, Jianhui Liu*, Li Jiang, Liwei Wang, Hengshuang Zhao, Shu Liu, Xiaojuan Qi, Jiaya Jia

DV Lab 195 Jan 01, 2023