The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

Last update: Feb 05, 2022

Related tags

Overview

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

This folder contains the code to reproduce the data in "The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks."

To reproduce the 'data' folder:

Activate the environment provided as the 'environment.yml' file.
Run 'shallow_gln.sh', 'shallow_gln_cp.sh', 'shallow_gln_cp_2.sh', 'deep_gln.sh', 'deep_gln_cp.sh', 'relu_net.sh', and 'relu_net_cp.sh'. The convex optimization scripts (end in cp) must be run after their gradient descent counterparts (do not end in cp).

We then created an R package to package the cleaned datasets ('glnsanalysis', available in the folder 'implicit-bias-glns-analysis'). To access the datasets, build the packages. The notebooks in the folder 'vignettes' reproduce the figures.

Owner

Samuel Lippl

Neuroscience, statistics, mathematics

GitHub Repository

Implementation of EMNLP 2017 Paper "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog" using PyTorch and ParlAI

Language Emergence in Multi Agent Dialog Code for the Paper Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog Satwik Kottur, José M.

105 Nov 25, 2022

Reference code for the paper "Cross-Camera Convolutional Color Constancy" (ICCV 2021)

Cross-Camera Convolutional Color Constancy, ICCV 2021 (Oral) Mahmoud Afifi1,2, Jonathan T. Barron2, Chloe LeGendre2, Yun-Ta Tsai2, and Francois Bleibe

76 Jan 07, 2023

This source code is implemented using keras library based on "Automatic ocular artifacts removal in EEG using deep learning"

CSP_Deep_EEG This source code is implemented using keras library based on "Automatic ocular artifacts removal in EEG using deep learning" {https://www

2 Nov 08, 2022

adversarial_multi_armed_bandit_variable_plays

Adversarial Multi-Armed Bandit with Variable Plays This code is for paper: Adversarial Online Learning with Variable Plays in the Evasion-and-Pursuit

1 Oct 28, 2021

Learned Token Pruning for Transformers

LTP: Learned Token Pruning for Transformers Check our paper for more details. Installation We follow the same installation procedure as the original H

52 Dec 29, 2022

CLIP (Contrastive Language–Image Pre-training) trained on Indonesian data

CLIP-Indonesian CLIP (Radford et al., 2021) is a multimodal model that can connect images and text by training a vision encoder and a text encoder joi

17 Mar 10, 2022

Vector Neurons: A General Framework for SO(3)-Equivariant Networks

Vector Neurons: A General Framework for SO(3)-Equivariant Networks Created by Congyue Deng, Or Litany, Yueqi Duan, Adrien Poulenard, Andrea Tagliasacc

332 Dec 29, 2022

PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

Under construction... Attention in Attention Network for Image Super-Resolution (A2N) This repository is an PyTorch implementation of the paper "Atten

71 Dec 30, 2022

Leveraging Social Influence based on Users Activity Centers for Point-of-Interest Recommendation

SUCP Leveraging Social Influence based on Users Activity Centers for Point-of-Interest Recommendation () Direct Friends (i.e., users who follow each o

8 Nov 26, 2022

Code for ICLR2018 paper: Improving GAN Training via Binarized Representation Entropy (BRE) Regularization - Y. Cao · W Ding · Y.C. Lui · R. Huang

code for "Improving GAN Training via Binarized Representation Entropy (BRE) Regularization" (ICLR2018 paper) paper: https://arxiv.org/abs/1805.03644 G

21 Oct 12, 2020

Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Context Terms

LESA Introduction This repository contains the official implementation of Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Cont

20 Dec 31, 2021

Python interface for the DIGIT tactile sensor

DIGIT-INTERFACE Python interface for the DIGIT tactile sensor. For updates and discussions please join the #DIGIT channel at the www.touch-sensing.org

35 Dec 22, 2022

Learning Representations that Support Robust Transfer of Predictors

Transfer Risk Minimization (TRM) Code for Learning Representations that Support Robust Transfer of Predictors Prepare the Datasets Preprocess the Scen

15 Dec 07, 2022

Implementation of ETSformer, state of the art time-series Transformer, in Pytorch

ETSformer - Pytorch Implementation of ETSformer, state of the art time-series Transformer, in Pytorch Install $ pip install etsformer-pytorch Usage im

121 Dec 30, 2022

Transformer - Transformer in PyTorch

Transformer 完成进度 Embeddings and PositionalEncoding with example. MultiHeadAttent

1 Jan 06, 2022

nnFormer: Interleaved Transformer for Volumetric Segmentation Code for paper "nnFormer: Interleaved Transformer for Volumetric Segmentation "

nnFormer: Interleaved Transformer for Volumetric Segmentation Code for paper "nnFormer: Interleaved Transformer for Volumetric Segmentation ". Please

610 Dec 28, 2022

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

Related tags

Overview

The Implicit Bias of Gradient Descent on Generalized Gated Linear Networks

Owner

Samuel Lippl

Implementation of EMNLP 2017 Paper "Natural Language Does Not Emerge 'Naturally' in Multi-Agent Dialog" using PyTorch and ParlAI

Reference code for the paper "Cross-Camera Convolutional Color Constancy" (ICCV 2021)

This source code is implemented using keras library based on "Automatic ocular artifacts removal in EEG using deep learning"

adversarial_multi_armed_bandit_variable_plays

Learned Token Pruning for Transformers

CLIP (Contrastive Language–Image Pre-training) trained on Indonesian data

Vector Neurons: A General Framework for SO(3)-Equivariant Networks

PyTorch code for our paper "Attention in Attention Network for Image Super-Resolution"

Leveraging Social Influence based on Users Activity Centers for Point-of-Interest Recommendation

Code for ICLR2018 paper: Improving GAN Training via Binarized Representation Entropy (BRE) Regularization - Y. Cao · W Ding · Y.C. Lui · R. Huang

Locally Enhanced Self-Attention: Rethinking Self-Attention as Local and Context Terms

Python interface for the DIGIT tactile sensor

Learning Representations that Support Robust Transfer of Predictors

Implementation of ETSformer, state of the art time-series Transformer, in Pytorch

Transformer - Transformer in PyTorch

nnFormer: Interleaved Transformer for Volumetric Segmentation Code for paper "nnFormer: Interleaved Transformer for Volumetric Segmentation "

[NAACL & ACL 2021] SapBERT: Self-alignment pretraining for BERT.

시각 장애인을 위한 스마트 지팡이에 활용될 딥러닝 모델 (DL Model Repo)

PyTorchVideo is a deeplearning library with a focus on video understanding work

Fast and Simple Neural Vocoder, the Multiband RNNMS