Repository of continual learning papers

Overview

Continual learning paper repository

This repository contains an incomplete (but dynamically updated) list of papers exploring continual learning in machine learning and neuroscience, and accompanies the related paper Towards continual task learning in artificial neural networks: current approaches and insights from neuroscience.

arXiv link

The full paper complementing this repository is available at https://arxiv.org/abs/2112.14146

Machine learning

Reviews, surveys, & tutorials

Title Link Relevance
Continual lifelong learning with neural networks: A review Neural Networks •••
Catastrophic forgetting in connectionist networks TICS •••
Neuroscience-Inspired Artificial Intelligence Neuron ••
How to grow a mind: Statistics, structure, and abstraction Science
Deep learning Nature
Universal Intelligence: A Definition of Machine Intelligence arXiv

Classic papers

Title Link Relevance
Catastrophic forgetting in connectionist networks TICS •••
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem Psychology of Learning & Motivation •••
Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions Psychological Review

Architectural approaches to continual learning

Title Link Implementation
Progressive Neural Networks arXiv PyTorch TensorFlow
Neurogenesis deep learning: Extending deep networks to accommodate new classes IEEE
Adaptive structural learning of artificial neural networks ICML TensorFlow

Regularisation

Title Link Implementation
Learning without forgetting arXiv PyTorch
Distilling the knowledge in a neural network arXiv PyTorch TensorFlow
Overcoming catastrophic forgetting in neural networks arXiv PyTorch TensorFlow
Note on the quadratic penalties in elastic weight consolidation PNAS
Measuring catastrophic forgetting in neural networks arXiv
Continual learning through synaptic intelligence ICML TensorFlow
An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks arXiv Theano

Training regime

Title Link Implementation
How transferable are features in deep neural networks? arXiv Caffe
CHILD: A First Step Towards Continual Learning Machine Learning
Curriculum learning ICML
Continual Learning with Deep Generative Replay arXiv PyTorch
Experience Replay for Continual Learning arXiv
Brain-inspired replay for continual learning with artificial neural networks Nature Communications PyTorch
REMIND Your Neural Network to Prevent Catastrophic Forgetting arXiv PyTorch

Neuroscience

Title Link Relevance
Regulation and function of adult neurogenesis: from genes to cognition Physiological Review
When and where do we apply what we learn?: A taxonomy for far transfer Psychological Bulletin
Does the hippocampus map out the future? TICS
Organizing conceptual knowledge in humans with a gridlike code Science
Song replay during sleep and computational rules for sensorimotor vocal learning Science
A theory of the discovery and predication of relational concepts Psychological Review
Preplay of future place cell sequences by hippocampal cellular assemblies Nature ••
Comparing continual task learning in minds and machines PNAS ••
Cascade models of synaptically stored memories Neuron
Selective suppression of hippocampal ripples impairs spatial memory Nature Neuroscience ••
The analogical mind MIT Press
What learning systems do intelligent agents need? Complementary learning systems theory updated TICS
Human replay spontaneously reorganizes experience Cell •••
Compartmentalized dendritic plasticity and input feature storage in neurons Nature
Neuroconstructivism: How the brain constructs cognition Oxford University Press
Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory Psychological Review •••
The Role of Hippocampal Replay in Memory and Planning Current Biology ••
Brain imaging of language plasticity in adopted adults: Can a second language replace the first? Cerebral Cortex •••
Memory formation: Let’s replay Elife
Strengthening individual memories by reactivating them during sleep Science ••
Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience Science
Crossmodal spatial attention Annals of the NY Academy of Sciences
The merging of the senses MIT Press
Development of multisensory integration from the perspective of the individual neuron Nature Reviews Neuroscience ••
The hippocampal indexing theory and episodic memory: updating the index Hippocampus ••
Stably maintained dendritic spines are associated with lifelong memories Nature •••

Citation

BibTeX

@misc{mccaffary2021continual,
      title={Towards continual task learning in artificial neural networks: current approaches and insights from neuroscience}, 
      author={David McCaffary},
      year={2021},
      eprint={2112.14146},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}
Owner
🕳️
Vision-and-Language Navigation in Continuous Environments using Habitat

Vision-and-Language Navigation in Continuous Environments (VLN-CE) Project Website — VLN-CE Challenge — RxR-Habitat Challenge Official implementations

Jacob Krantz 132 Jan 02, 2023
Official implementation of paper Gradient Matching for Domain Generalization

Gradient Matching for Domain Generalisation This is the official PyTorch implementation of Gradient Matching for Domain Generalisation. In our paper,

94 Dec 23, 2022
basic tutorial on pytorch

Quick Tutorial on PyTorch PyTorch Basics Linear Regression Logistic Regression Artificial Neural Networks Convolutional Neural Networks Recurrent Neur

7 Sep 15, 2022
A TensorFlow implementation of FCN-8s

FCN-8s implementation in TensorFlow Contents Overview Examples and demo video Dependencies How to use it Download pre-trained VGG-16 Overview This is

Pierluigi Ferrari 50 Aug 08, 2022
Mini Software that give reminder to drink water as per your weight.

Water Notification Desktop Python The Mini Software built in Python (tkinter) that will remind you to drink water on specific time span based on your

Om Jogani 5 Dec 16, 2022
Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation (CoRL 2021)

Distilling Motion Planner Augmented Policies into Visual Control Policies for Robot Manipulation [Project website] [Paper] This project is a PyTorch i

Cognitive Learning for Vision and Robotics (CLVR) lab @ USC 6 Feb 28, 2022
Pytorch code for paper "Image Compressed Sensing Using Non-local Neural Network" TMM 2021.

NL-CSNet-Pytorch Pytorch code for paper "Image Compressed Sensing Using Non-local Neural Network" TMM 2021. Note: this repo only shows the strategy of

WenxueCui 7 Nov 07, 2022
gACSON software for visualization, processing and analysis of three-dimensional electron microscopy images

gACSON gACSON software is to visualize, segment, and analyze the morphology of neurons in three-dimensional electron microscopy images. If you use any

Andrea Behanova 2 May 31, 2022
This repository is for our paper Exploiting Scene Graphs for Human-Object Interaction Detection accepted by ICCV 2021.

SG2HOI This repository is for our paper Exploiting Scene Graphs for Human-Object Interaction Detection accepted by ICCV 2021. Installation Pytorch 1.7

HT 10 Dec 20, 2022
Cweqgen - The CW Equation Generator

The CW Equation Generator The cweqgen (pronouced like "Queck-Jen") package provi

2 Jan 15, 2022
Experiments for Operating Systems Lab (ETCS-352)

Operating Systems Lab (ETCS-352) Experiments for Operating Systems Lab (ETCS-352) performed by me in 2021 at uni. All codes are written by me except t

Deekshant Wadhwa 0 Sep 06, 2022
Density-aware Single Image De-raining using a Multi-stream Dense Network (CVPR 2018)

DID-MDN Density-aware Single Image De-raining using a Multi-stream Dense Network He Zhang, Vishal M. Patel [Paper Link] (CVPR'18) We present a novel d

He Zhang 224 Dec 12, 2022
Course about deep learning for computer vision and graphics co-developed by YSDA and Skoltech.

Deep Vision and Graphics This repo supplements course "Deep Vision and Graphics" taught at YSDA @fall'21. The course is the successor of "Deep Learnin

Yandex School of Data Analysis 160 Jan 02, 2023
StarGAN v2 - Official PyTorch Implementation (CVPR 2020)

StarGAN v2 - Official PyTorch Implementation StarGAN v2: Diverse Image Synthesis for Multiple Domains Yunjey Choi*, Youngjung Uh*, Jaejun Yoo*, Jung-W

Clova AI Research 3.1k Jan 09, 2023
DPC: Unsupervised Deep Point Correspondence via Cross and Self Construction (3DV 2021)

DPC: Unsupervised Deep Point Correspondence via Cross and Self Construction (3DV 2021) This repo is the implementation of DPC. Tested environment Pyth

Dvir Ginzburg 30 Nov 30, 2022
EMNLP 2021: Single-dataset Experts for Multi-dataset Question-Answering

MADE (Multi-Adapter Dataset Experts) This repository contains the implementation of MADE (Multi-adapter dataset experts), which is described in the pa

Princeton Natural Language Processing 68 Jul 18, 2022
Deploy optimized transformer based models on Nvidia Triton server

🤗 Hugging Face Transformer submillisecond inference 🤯 and deployment on Nvidia Triton server Yes, you can perfom inference with transformer based mo

Lefebvre Sarrut Services 1.2k Jan 05, 2023
Official PyTorch implementation of "AASIST: Audio Anti-Spoofing using Integrated Spectro-Temporal Graph Attention Networks"

AASIST This repository provides the overall framework for training and evaluating audio anti-spoofing systems proposed in 'AASIST: Audio Anti-Spoofing

Clova AI Research 56 Jan 02, 2023
PyTorch implementation of the supervised learning experiments from the paper Model-Agnostic Meta-Learning (MAML)

pytorch-maml This is a PyTorch implementation of the supervised learning experiments from the paper Model-Agnostic Meta-Learning (MAML): https://arxiv

Kate Rakelly 516 Jan 05, 2023
A dead simple python wrapper for darknet that works with OpenCV 4.1, CUDA 10.1

What Dead simple python wrapper for Yolo V3 using AlexyAB's darknet fork. Works with CUDA 10.1 and OpenCV 4.1 or later (I use OpenCV master as of Jun

Pliable Pixels 6 Jan 12, 2022