Code for Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding

Last update: Jun 23, 2022

Related tags

Overview

🍐 quince

Code for Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding

🍐 Installation

$ git clone [email protected]:anndvision/quince.git
$ cd quince
$ conda env create -f environment.yml
$ conda activate quince

🍐 Example: Replicating IHDP results

Step 1: Hyperparameter Tuning (optional)

Find the best hyperparameters using the tune function, on a dataset like ihdp for an ensemble model.

$ quince \
    tune \
        --job-dir ~/experiments/quince/tuning/ \
        --max-samples 500 \
        --gpu-per-trial 0.2 \
    ihdp \
    ensemble

Step 2: Train ensembles over a number of trials

Here, we use the train function to fit an ensemble of mixture density networks on 10 realizations of the ihdp with hidden confounding dataset. For the full results change --num-trials 1000

$ quince \
    train \
        --job-dir ~/experiments/quince/ \
        --num-trials 10 \
        --gpu-per-trial 0.2 \
    ihdp \
    ensemble \
        --dim-hidden 200 \
        --num-components 5 \
        --depth 4 \
        --negative-slope 0.3 \
        --dropout-rate 0.5 \
        --spectral-norm 6.0 \
        --learning-rate 5e-4 \
        --batch-size 200 \
        --epochs 500 \
        --ensemble-size 10

Step 3: Evaluate

Plots will be written to the experiment-dir

$ quince \
    evaluate \
        --experiment-dir ~/experiments/quince/ihdp/hc-True_beta-None/ensemble/dh-200_nc-5_dp-4_ns-0.3_dr-0.5_sn-6.0_lr-0.0005_bs-200_ep-500/ \
    compute-intervals \
        --gpu-per-trial 0.2 \
    compute-intervals-kernel \
        --gpu-per-trial 0.2 \
    plot-deferral \
    plot-errorbars \
        --trial 0

🍐 Replicating Other Results

Simulated Data

$ quince \
    train \
        --job-dir ~/experiments/quince/ \
        --num-trials 50 \
        --gpu-per-trial 0.2 \
    synthetic \
        --gamma-star 1.65 \
    ensemble \
        --dim-hidden 200 \
        --num-components 5 \
        --depth 4 \
        --negative-slope 0.0 \
        --dropout-rate 0.1 \
        --spectral-norm 6.0 \
        --learning-rate 1e-3 \
        --batch-size 32 \
        --epochs 500 \
        --ensemble-size 10

$ quince \
    evaluate \
        --experiment-dir ~/experiments/quince/synthetic/ne-1000_gs-1.65_th-4.00_be-0.75_si-1.00_dl-2.00/ensemble/dh-200_nc-5_dp-4_ns-0.0_dr-0.1_sn-6.0_lr-0.001_bs-32_ep-500/ \
    compute-intervals \
        --gpu-per-trial 0.2 \
    compute-intervals-kernel \
        --gpu-per-trial 0.2 \
    plot-ignorance \
    print-summary \
    print-summary-kernel \
    paired-t-test

Repeat the above for --gamma-star 2.72 and --gamma-star 4.48.

HCMNIST

$ quince \
    train \
        --job-dir ~/experiments/quince/ \
        --num-trials 20 \
        --gpu-per-trial 0.5 \
    hcmnist \
        --gamma-star 1.65 \
    ensemble \
        --dim-hidden 200 \
        --num-components 5 \
        --depth 2 \
        --negative-slope 0.0 \
        --dropout-rate 0.15 \
        --spectral-norm 3.0 \
        --learning-rate 5e-4 \
        --batch-size 200 \
        --epochs 500 \
        --ensemble-size 5

$ quince \
    evaluate \
        --experiment-dir ~/experiments/quince/hcmnist/gs-1.65_th-4.00_be-0.75_si-1.00_dl-2.00/ensemble/dh-200_nc-5_dp-2_ns-0.0_dr-0.15_sn-3.0_lr-0.0005_bs-200_ep-500/ \
    compute-intervals \
        --gpu-per-trial 1.0 \
    print-summary

Repeat the above for --gamma-star 2.72 and --gamma-star 4.48.

Code for Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding

Related tags

Overview

🍐 quince

🍐 Installation

🍐 Example: Replicating IHDP results

Step 1: Hyperparameter Tuning (optional)

Step 2: Train ensembles over a number of trials

Step 3: Evaluate

🍐 Replicating Other Results

Simulated Data

HCMNIST

Owner

Andrew Jesson

A JAX implementation of Broaden Your Views for Self-Supervised Video Learning, or BraVe for short.

Package to compute Mauve, a similarity score between neural text and human text. Install with `pip install mauve-text`.

Stream images from a connected camera over MQTT, view using Streamlit, record to file and sqlite

ALFRED - A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

performing moving objects segmentation using image processing techniques with opencv and numpy

A toolkit for document-level event extraction, containing some SOTA model implementations

Tools for manipulating UVs in the Blender viewport.

A comprehensive and up-to-date developer education platform for Urbit.

A collection of inference modules for fastai2

Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models

Adversarial-autoencoders - Tensorflow implementation of Adversarial Autoencoders

A method that utilized Generative Adversarial Network (GAN) to interpret the black-box deep image classifier models by PyTorch.

Complementary Patch for Weakly Supervised Semantic Segmentation, ICCV21 (poster)

A Machine Teaching Framework for Scalable Recognition

DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.

Supervised multi-SNE (S-multi-SNE): Multi-view visualisation and classification

Official PyTorch implementation of DD3D: Is Pseudo-Lidar needed for Monocular 3D Object detection? (ICCV 2021), Dennis Park, Rares Ambrus, Vitor Guizilini, Jie Li, and Adrien Gaidon.

ConvMAE: Masked Convolution Meets Masked Autoencoders

PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation

VACA: Designing Variational Graph Autoencoders for Interventional and Counterfactual Queries

Code for Quantifying Ignorance in Individual-Level Causal-Effect Estimates under Hidden Confounding

Related tags

Overview

🍐 quince

🍐 Installation

🍐 Example: Replicating IHDP results

Step 1: Hyperparameter Tuning (optional)

Step 2: Train ensembles over a number of trials

Step 3: Evaluate

🍐 Replicating Other Results

Simulated Data

HCMNIST

Owner

Andrew Jesson

A JAX implementation of Broaden Your Views for Self-Supervised Video Learning, or BraVe for short.

Package to compute Mauve, a similarity score between neural text and human text. Install with `pip install mauve-text`.

Stream images from a connected camera over MQTT, view using Streamlit, record to file and sqlite

ALFRED - A Benchmark for Interpreting Grounded Instructions for Everyday Tasks

performing moving objects segmentation using image processing techniques with opencv and numpy

A toolkit for document-level event extraction, containing some SOTA model implementations

Tools for manipulating UVs in the Blender viewport.

A comprehensive and up-to-date developer education platform for Urbit.

A collection of inference modules for fastai2

Monocular Depth Estimation - Weighted-average prediction from multiple pre-trained depth estimation models

Adversarial-autoencoders - Tensorflow implementation of Adversarial Autoencoders

A method that utilized Generative Adversarial Network (GAN) to interpret the black-box deep image classifier models by PyTorch.

Complementary Patch for Weakly Supervised Semantic Segmentation, ICCV21 (poster)

A Machine Teaching Framework for Scalable Recognition

DiffQ performs differentiable quantization using pseudo quantization noise. It can automatically tune the number of bits used per weight or group of weights, in order to achieve a given trade-off between model size and accuracy.

Supervised multi-SNE (S-multi-SNE): Multi-view visualisation and classification

Official PyTorch implementation of DD3D: Is Pseudo-Lidar needed for Monocular 3D Object detection? (ICCV 2021), Dennis Park*, Rares Ambrus*, Vitor Guizilini, Jie Li, and Adrien Gaidon.

ConvMAE: Masked Convolution Meets Masked Autoencoders

PocketNet: Extreme Lightweight Face Recognition Network using Neural Architecture Search and Multi-Step Knowledge Distillation

VACA: Designing Variational Graph Autoencoders for Interventional and Counterfactual Queries

Official PyTorch implementation of DD3D: Is Pseudo-Lidar needed for Monocular 3D Object detection? (ICCV 2021), Dennis Park, Rares Ambrus, Vitor Guizilini, Jie Li, and Adrien Gaidon.