Repository of continual learning papers

Overview

Continual learning paper repository

This repository contains an incomplete (but dynamically updated) list of papers exploring continual learning in machine learning and neuroscience, and accompanies the related paper Towards continual task learning in artificial neural networks: current approaches and insights from neuroscience.

arXiv link

The full paper complementing this repository is available at https://arxiv.org/abs/2112.14146

Machine learning

Reviews, surveys, & tutorials

Title Link Relevance
Continual lifelong learning with neural networks: A review Neural Networks •••
Catastrophic forgetting in connectionist networks TICS •••
Neuroscience-Inspired Artificial Intelligence Neuron ••
How to grow a mind: Statistics, structure, and abstraction Science
Deep learning Nature
Universal Intelligence: A Definition of Machine Intelligence arXiv

Classic papers

Title Link Relevance
Catastrophic forgetting in connectionist networks TICS •••
Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem Psychology of Learning & Motivation •••
Connectionist models of recognition memory: Constraints imposed by learning and forgetting functions Psychological Review

Architectural approaches to continual learning

Title Link Implementation
Progressive Neural Networks arXiv PyTorch TensorFlow
Neurogenesis deep learning: Extending deep networks to accommodate new classes IEEE
Adaptive structural learning of artificial neural networks ICML TensorFlow

Regularisation

Title Link Implementation
Learning without forgetting arXiv PyTorch
Distilling the knowledge in a neural network arXiv PyTorch TensorFlow
Overcoming catastrophic forgetting in neural networks arXiv PyTorch TensorFlow
Note on the quadratic penalties in elastic weight consolidation PNAS
Measuring catastrophic forgetting in neural networks arXiv
Continual learning through synaptic intelligence ICML TensorFlow
An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks arXiv Theano

Training regime

Title Link Implementation
How transferable are features in deep neural networks? arXiv Caffe
CHILD: A First Step Towards Continual Learning Machine Learning
Curriculum learning ICML
Continual Learning with Deep Generative Replay arXiv PyTorch
Experience Replay for Continual Learning arXiv
Brain-inspired replay for continual learning with artificial neural networks Nature Communications PyTorch
REMIND Your Neural Network to Prevent Catastrophic Forgetting arXiv PyTorch

Neuroscience

Title Link Relevance
Regulation and function of adult neurogenesis: from genes to cognition Physiological Review
When and where do we apply what we learn?: A taxonomy for far transfer Psychological Bulletin
Does the hippocampus map out the future? TICS
Organizing conceptual knowledge in humans with a gridlike code Science
Song replay during sleep and computational rules for sensorimotor vocal learning Science
A theory of the discovery and predication of relational concepts Psychological Review
Preplay of future place cell sequences by hippocampal cellular assemblies Nature ••
Comparing continual task learning in minds and machines PNAS ••
Cascade models of synaptically stored memories Neuron
Selective suppression of hippocampal ripples impairs spatial memory Nature Neuroscience ••
The analogical mind MIT Press
What learning systems do intelligent agents need? Complementary learning systems theory updated TICS
Human replay spontaneously reorganizes experience Cell •••
Compartmentalized dendritic plasticity and input feature storage in neurons Nature
Neuroconstructivism: How the brain constructs cognition Oxford University Press
Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory Psychological Review •••
The Role of Hippocampal Replay in Memory and Planning Current Biology ••
Brain imaging of language plasticity in adopted adults: Can a second language replace the first? Cerebral Cortex •••
Memory formation: Let’s replay Elife
Strengthening individual memories by reactivating them during sleep Science ••
Replay of neuronal firing sequences in rat hippocampus during sleep following spatial experience Science
Crossmodal spatial attention Annals of the NY Academy of Sciences
The merging of the senses MIT Press
Development of multisensory integration from the perspective of the individual neuron Nature Reviews Neuroscience ••
The hippocampal indexing theory and episodic memory: updating the index Hippocampus ••
Stably maintained dendritic spines are associated with lifelong memories Nature •••

Citation

BibTeX

@misc{mccaffary2021continual,
      title={Towards continual task learning in artificial neural networks: current approaches and insights from neuroscience}, 
      author={David McCaffary},
      year={2021},
      eprint={2112.14146},
      archivePrefix={arXiv},
      primaryClass={cs.LG}
}
Owner
🕳️
[CVPR2021] Invertible Image Signal Processing

Invertible Image Signal Processing This repository includes official codes for "Invertible Image Signal Processing (CVPR2021)". Figure: Our framework

Yazhou XING 281 Dec 31, 2022
Yolov3 pytorch implementation

YOLOV3 Pytorch实现 在bubbliiing大佬代码的基础上进行了修改,添加了部分注释。 预训练模型 预训练模型来源于bubbliiing。 链接:https://pan.baidu.com/s/1ncREw6Na9ycZptdxiVMApw 提取码:appk 训练自己的数据集 按照VO

4 Aug 27, 2022
MassiveSumm: a very large-scale, very multilingual, news summarisation dataset

MassiveSumm: a very large-scale, very multilingual, news summarisation dataset This repository contains links to data and code to fetch and reproduce

Daniel Varab 19 Dec 16, 2022
Acoustic mosquito detection code with Bayesian Neural Networks

HumBugDB Acoustic mosquito detection with Bayesian Neural Networks. Extract audio or features from our large-scale dataset on Zenodo. This repository

31 Nov 28, 2022
DLWP: Deep Learning Weather Prediction

DLWP: Deep Learning Weather Prediction DLWP is a Python project containing data-

Kushal Shingote 3 Aug 14, 2022
Semi-supervised learning for object detection

Source code for STAC: A Simple Semi-Supervised Learning Framework for Object Detection STAC is a simple yet effective SSL framework for visual object

Google Research 348 Dec 25, 2022
Code for the IJCAI 2021 paper "Structure Guided Lane Detection"

SGNet Project for the IJCAI 2021 paper "Structure Guided Lane Detection" Abstract Recently, lane detection has made great progress with the rapid deve

Jinming Su 27 Dec 08, 2022
Implementation of Memory-Compressed Attention, from the paper "Generating Wikipedia By Summarizing Long Sequences"

Memory Compressed Attention Implementation of the Self-Attention layer of the proposed Memory-Compressed Attention, in Pytorch. This repository offers

Phil Wang 47 Dec 23, 2022
Official repository of the paper 'Essentials for Class Incremental Learning'

Essentials for Class Incremental Learning Official repository of the paper 'Essentials for Class Incremental Learning' This Pytorch repository contain

33 Nov 27, 2022
From this paper "SESNet: A Semantically Enhanced Siamese Network for Remote Sensing Change Detection"

SESNet for remote sensing image change detection It is the implementation of the paper: "SESNet: A Semantically Enhanced Siamese Network for Remote Se

1 May 24, 2022
This repository implements variational graph auto encoder by Thomas Kipf.

Variational Graph Auto-encoder in Pytorch This repository implements variational graph auto-encoder by Thomas Kipf. For details of the model, refer to

DaehanKim 215 Jan 02, 2023
Shōgun

The SHOGUN machine learning toolbox Unified and efficient Machine Learning since 1999. Latest release: Cite Shogun: Develop branch build status: Donat

Shōgun ML 2.9k Jan 04, 2023
PEPit is a package enabling computer-assisted worst-case analyses of first-order optimization methods.

PEPit: Performance Estimation in Python This open source Python library provides a generic way to use PEP framework in Python. Performance estimation

Baptiste 53 Nov 16, 2022
face property detection pytorch

This is the face property train code of project face-detection-project

i am x 2 Oct 18, 2021
A PyTorch implementation of EfficientNet and EfficientNetV2 (coming soon!)

EfficientNet PyTorch Quickstart Install with pip install efficientnet_pytorch and load a pretrained EfficientNet with: from efficientnet_pytorch impor

Luke Melas-Kyriazi 7.2k Jan 06, 2023
Code to accompany our paper "Continual Learning Through Synaptic Intelligence" ICML 2017

Continual Learning Through Synaptic Intelligence This repository contains code to reproduce the key findings of our path integral approach to prevent

Ganguli Lab 82 Nov 03, 2022
Coded illumination for improved lensless imaging

CodedCam Coded Illumination for Improved Lensless Imaging Paper | Supplementary results | Data and Code are available. Coded illumination for improved

Computational Sensing and Information Processing Lab 1 Nov 29, 2021
Starter Code for VALUE benchmark

StarterCode for VALUE Benchmark This is the starter code for VALUE Benchmark [website], [paper]. This repository currently supports all baseline model

VALUE Benchmark 73 Dec 09, 2022
A collection of IPython notebooks covering various topics.

ipython-notebooks This repo contains various IPython notebooks I've created to experiment with libraries and work through exercises, and explore subje

John Wittenauer 2.6k Jan 01, 2023
OptNet: Differentiable Optimization as a Layer in Neural Networks

OptNet: Differentiable Optimization as a Layer in Neural Networks This repository is by Brandon Amos and J. Zico Kolter and contains the PyTorch sourc

CMU Locus Lab 428 Dec 24, 2022