PairNorm

Official pytorch source code for PairNorm paper (ICLR 2020)
This code requires pytorch_geometric>=1.3.2

usage

For SGC, we use original PairNorm. Notice norm_scale is data-dependent. One can choose it from {0.1, 1, 10, 50}.

python main.py --data cora --model SGC --nlayer 40 --missing_rate 100 --norm_mode PN --norm_scale 10

For GCN or GAT, we use PairNorm-SI or SCS.

python main.py --data cora --model DeepGCN --nlayer 10 --missing_rate 100 --norm_mode PN-SI --residual 0
python main.py --data cora --model DeepGAT --nlayer 10 --missing_rate 100 --norm_mode PN-SCS --residual 0

update: normalization and PN

we have found that PN works bad with symmetric normalized adjacency matrix, originally the experiments align with the paper used row-normalized adjacency matrix. What's more, we also found a small bug in the old experiments with using PN for GCN and GAT. The current version PN should works good for GCN and GAT also (haven't fully tested). Please start from using PN before testing PN-SI and PN-SCS.

For GCN or GAT, now using PN to start.

python main.py --data cora --model DeepGCN --nlayer 10 --missing_rate 100 --norm_mode PN --residual 0
python main.py --data cora --model DeepGAT --nlayer 10 --missing_rate 100 --norm_mode PN --residual 0

cite

If you use our code, please cite

@inproceedings{
zhao2020pairnorm,
title={PairNorm: Tackling Oversmoothing in {\{}GNN{\}}s},
author={Lingxiao Zhao and Leman Akoglu},
booktitle={International Conference on Learning Representations},
year={2020},
url={https://openreview.net/forum?id=rkecl1rtwB}
}

Source code for PairNorm (ICLR 2020)

Related tags

Overview

PairNorm

usage

update: normalization and PN

cite

Owner

Pytorch implementation of Bert and Pals: Projected Attention Layers for Efficient Adaptation in Multi-Task Learning

Data augmentation for NLP, accepted at EMNLP 2021 Findings

An implementation for the loss function proposed in Decoupled Contrastive Loss paper.

Unimodal Face Classification with Multimodal Training

Pytorch implementation of MixNMatch

Deep Learning Interviews book: Hundreds of fully solved job interview questions from a wide range of key topics in AI.

Polynomial-time Meta-Interpretive Learning

MMGeneration is a powerful toolkit for generative models, based on PyTorch and MMCV.

Python implementation of Project Fluent

A PyTorch Image-Classification With AlexNet And ResNet50.

A mini-course offered to Undergrad chemistry students

Code for the paper "Benchmarking and Analyzing Point Cloud Classification under Corruptions"

code from "Tensor decomposition of higher-order correlations by nonlinear Hebbian plasticity"

[CVPR2021] Invertible Image Signal Processing

Dynamic Token Normalization Improves Vision Transformers

PyTorch implementation of SCAFFOLD (Stochastic Controlled Averaging for Federated Learning, ICML 2020).

Deep Structured Instance Graph for Distilling Object Detectors (ICCV 2021)

An Intelligent Self-driving Truck System For Highway Transportation

Learning Neural Network Subspaces