MEDS: Enhancing Memory Error Detection for Large-Scale Applications

Related tags

Deep LearningMEDS
Overview

MEDS: Enhancing Memory Error Detection for Large-Scale Applications

Prerequisites

  • cmake and clang

Build MEDS supporting compiler

$ make

Build Using Docker

# build docker image
$ docker build -t meds .

# run docker image
$ docker run --cap-add=SYS_PTRACE -it meds /bin/bash

Testing MEDS

  • MEDS's testing runs original ASAN's testcases as well as MEDS specific testcases.

    • Copied ASAN's testcases in llvm/projects/compiler-rt/test/meds/TestCases/ASan
    • MEDS specific testcases in llvm/projects/compiler-rt/test/meds/TestCases/Meds

  • To run the test,

$ make test

Testing Time: 30.70s
 Expected Passes    : 183
 Expected Failures  : 1
 Unsupported Tests  : 50

Build applications with MEDS heap allocation and ASan stack and global

  • Given a test program test.cc,
$ cat > test.cc

int main(int argc, char **argv) {
  int *a = new int[10];
  a[argc * 10] = 1;
  return 0;
}
  • test.cc can be built using the option, -fsanitize=meds.
$ build/bin/clang++ -fsanitize=meds test.cc -o test
$ ./test

==90589==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x43fff67eb078 at pc 0x0000004f926d bp 0x7fffffffe440 sp 0x7fffffffe438
WRITE of size 4 at 0x43fff67eb078 thread T0
    #0 0x4f926c in main (/home/wookhyun/release/meds-release/a.out+0x4f926c)
    #1 0x7ffff6b5c82f in __libc_start_main /build/glibc-bfm8X4/glibc-2.23/csu/../csu/libc-start.c:291
    #2 0x419cb8 in _start (/home/wookhyun/release/meds-release/a.out+0x419cb8)

Address 0x43fff67eb078 is a wild pointer.
SUMMARY: AddressSanitizer: heap-buffer-overflow (/home/wookhyun/release/meds-release/a.out+0x4f926c) in main
Shadow bytes around the buggy address:
  0x08807ecf55b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x08807ecf55c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x08807ecf55d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x08807ecf55e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x08807ecf55f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x08807ecf5600: fa fa fa fa fa fa fa fa fa fa 00 00 00 00 00[fa]
  0x08807ecf5610: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x08807ecf5620: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x08807ecf5630: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x08807ecf5640: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x08807ecf5650: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
Shadow byte legend (one shadow byte represents 8 application bytes):
  Addressable:           00
  Partially addressable: 01 02 03 04 05 06 07
  Heap left redzone:       fa
  Freed heap region:       fd
  Stack left redzone:      f1
  Stack mid redzone:       f2
  Stack right redzone:     f3
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==90589==ABORTING

Options

  • -fsanitize=meds: Enable heap protection using MEDS (stack and global are protected using ASAN)

  • -mllvm -meds-stack=1: Enable stack protection using MEDS

  • -mllvm -meds-global=1 -mcmodel=large: Enable global protection using MEDS

    • This also requires --emit-relocs in LDFLAGS

  • Example: to protect heap/stack using MEDS and global using ASAN

$ clang -fsanitize=meds -mllvm -meds-stack=1 test.c -o test
  • Example: to protect heap/global using MEDS and stack using ASAN
$ clang -fsanitize=meds -mllvm -meds-global=1 -mcmodel=large -Wl,-emit-relocs test.c -o test
  • Example: to protect heap/stack/global using MEDS
$ clang -fsanitize=meds -mllvm -meds-stack=1 -mllvm -meds-global=1 -mcmodel=large -Wl,--emit-relocs

Contributors

Owner
Secomp Lab at Purdue University
Secomp Lab at Purdue University
GitHub Repository
Unofficial pytorch-lightning implement of Mip-NeRF

mipnerf_pl Unofficial pytorch-lightning implement of Mip-NeRF, Here are some results generated by this repository (pre-trained models are provided bel

Jianxin Huang 159 Dec 23, 2022
Code accompanying paper: Meta-Learning to Improve Pre-Training

Meta-Learning to Improve Pre-Training This folder contains code to run experiments in the paper Meta-Learning to Improve Pre-Training, NeurIPS 2021. P

28 Dec 31, 2022
This folder contains the implementation of the multi-relational attribute propagation algorithm.

MrAP This folder contains the implementation of the multi-relational attribute propagation algorithm. It requires the package pytorch-scatter. Please

6 Dec 06, 2022
Official Repository for our ICCV2021 paper: Continual Learning on Noisy Data Streams via Self-Purified Replay

Continual Learning on Noisy Data Streams via Self-Purified Replay This repository contains the official PyTorch implementation for our ICCV2021 paper.

Jinseo Jeong 22 Nov 23, 2022
An example of semantic segmentation using tensorflow in eager execution.

Semantic segmentation using Tensorflow eager execution Requirement Python 2.7+ Tensorflow-gpu OpenCv H5py Scikit-learn Numpy Imgaug Train with eager e

Iñigo Alonso Ruiz 25 Sep 29, 2022
A PyTorch implementation of "DGC-Net: Dense Geometric Correspondence Network"

DGC-Net: Dense Geometric Correspondence Network This is a PyTorch implementation of our work "DGC-Net: Dense Geometric Correspondence Network" TL;DR A

191 Dec 16, 2022
Let's create a tool to convert Thailand budget from PDF to CSV.

thailand-budget-pdf2csv Let's create a tool to convert Thailand Government Budgeting from PDF to CSV! รวมพลัง Dev แปลงงบ จาก PDF สู่ Machine-readable

Kao.Geek 88 Dec 19, 2022
The Python ensemble sampling toolkit for affine-invariant MCMC

emcee The Python ensemble sampling toolkit for affine-invariant MCMC emcee is a stable, well tested Python implementation of the affine-invariant ense

Dan Foreman-Mackey 1.3k Dec 31, 2022
CMT: Convolutional Neural Networks Meet Vision Transformers

CMT: Convolutional Neural Networks Meet Vision Transformers [arxiv] 1. Introduction This repo is the CMT model which impelement with pytorch, no refer

FlyEgle 83 Dec 30, 2022
[CVPR22] Official codebase of Semantic Segmentation by Early Region Proxy.

RegionProxy Figure 2. Performance vs. GFLOPs on ADE20K val split. Semantic Segmentation by Early Region Proxy Yifan Zhang, Bo Pang, Cewu Lu CVPR 2022

Yifan 54 Nov 29, 2022
Code for EMNLP 2021 paper: "Learning Implicit Sentiment in Aspect-based Sentiment Analysis with Supervised Contrastive Pre-Training"

SCAPT-ABSA Code for EMNLP2021 paper: "Learning Implicit Sentiment in Aspect-based Sentiment Analysis with Supervised Contrastive Pre-Training" Overvie

Zhengyan Li 66 Dec 04, 2022
DivNoising is an unsupervised denoising method to generate diverse denoised samples for any noisy input image. This repository contains the code to reproduce the results reported in the paper https://openreview.net/pdf?id=agHLCOBM5jP

DivNoising: Diversity Denoising with Fully Convolutional Variational Autoencoders Mangal Prakash1, Alexander Krull1,2, Florian Jug2 1Authors contribut

JugLab 33 Dec 30, 2022
Simple Text-Generator with OpenAI gpt-2 Pytorch Implementation

GPT2-Pytorch with Text-Generator Better Language Models and Their Implications Our model, called GPT-2 (a successor to GPT), was trained simply to pre

Tae-Hwan Jung 775 Jan 08, 2023
Reproduced Code for Image Forgery Detection papers.

Image Forgery Detection With over 4.5 billion active internet users, the amount of multimedia content being shared every day has surpassed everyone’s

Umar Masud 15 Dec 06, 2022
A curated list of references for MLOps

A curated list of references for MLOps

Larysa Visengeriyeva 9.3k Jan 07, 2023
A curated list of the latest breakthroughs in AI (in 2021) by release date with a clear video explanation, link to a more in-depth article, and code.

2021: A Year Full of Amazing AI papers- A Review 📌 A curated list of the latest breakthroughs in AI by release date with a clear video explanation, l

Louis-François Bouchard 2.9k Dec 31, 2022
COLMAP - Structure-from-Motion and Multi-View Stereo

COLMAP About COLMAP is a general-purpose Structure-from-Motion (SfM) and Multi-View Stereo (MVS) pipeline with a graphical and command-line interface.

4.7k Jan 07, 2023
This repository holds the code for the paper "Deep Conditional Gaussian Mixture Model forConstrained Clustering".

Deep Conditional Gaussian Mixture Model for Constrained Clustering. This repository holds the code for the paper Deep Conditional Gaussian Mixture Mod

17 Oct 30, 2022
Text Generation by Learning from Demonstrations

Text Generation by Learning from Demonstrations The README was last updated on March 7, 2021. The repo is based on fairseq (v0.9.?). Paper arXiv Prere

38 Oct 21, 2022
Self-supervised learning algorithms provide a way to train Deep Neural Networks in an unsupervised way using contrastive losses

Self-supervised learning Self-supervised learning algorithms provide a way to train Deep Neural Networks in an unsupervised way using contrastive loss

Arijit Das 2 Mar 26, 2022