Ladder network is a deep learning algorithm that combines supervised and unsupervised learning

Related tags

Game Developmentladder
Overview

This repository contains source code for the experiments in a paper titled Semi-Supervised Learning with Ladder Networks by A Rasmus, H Valpola, M Honkala, M Berglund, and T Raiko.

Required libraries

Install Theano, Blocks Stable 0.2, Fuel Stable 0.2

Refer to the Blocks installation instructions for details but use tag v0.2 instead. Something along:

pip install git+git://github.com/mila-udem/[email protected]
pip install git+git://github.com/mila-udem/[email protected]

Fuel comes with Blocks, but you need to download and convert the datasets. Refer to the Fuel documentation. One might need to rename the converted files.

fuel-download mnist
fuel-convert mnist --dtype float32
fuel-download cifar10
fuel-convert cifar10
Alternatively, one can use the environment.yml file that is provided in this repo to create an conda environment.
  1. First install anaconda from https://www.continuum.io/downloads. Then,
  2. conda env create -f environment.yml
  3. source activate ladder
  4. The environment should be good to go!

Models in the paper

The following commands train the models with seed 1. The reported numbers in the paper are averages over several random seeds. These commands use all the training samples for training (--unlabeled-samples 60000) and none are used for validation. This results in a lot of NaNs being printed during the trainining, since the validation statistics are not available. If you want to observe the validation error and costs during the training, use --unlabeled-samples 50000.

MNIST all labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 1000,1,0.01,0.01,0.01,0.01,0.01 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_full
# Bottom
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,0,0,0,0,0,0 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_bottom
# Gamma model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,2 --labeled-samples 60000 --unlabeled-samples 60000 --seed 1 -- mnist_all_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 60000 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_all_baseline
MNIST 100 labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_full
# Bottom-only
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 5000,0,0,0,0,0,0 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_bottom
# Gamma
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,0.5 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 100 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_100_baseline
MNIST 1000 labels
# Full
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --f-local-noise-std 0.2 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_full
# Bottom-only
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,0,0,0,0,0,0 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_bottom
# Gamma model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,10 --labeled-samples 1000 --unlabeled-samples 60000 --seed 1 -- mnist_1000_gamma
# Supervised baseline
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec 0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0 --labeled-samples 1000 --unlabeled-samples 60000 --f-local-noise-std 0.5 --seed 1 -- mnist_1000_baseline
MNIST 50 labels
# Full model
run.py train --encoder-layers 1000-500-250-250-250-10 --decoder-spec gauss --denoising-cost-x 2000,20,0.1,0.1,0.1,0.1,0.1 --labeled-samples 50 --unlabeled-samples 60000 --seed 1 -- mnist_50_full
MNIST convolutional models
# Conv-FC
run.py train --encoder-layers convv:1000:26:1:1-convv:500:1:1:1-convv:250:1:1:1-convv:250:1:1:1-convv:250:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec gauss --denoising-cost-x 1000,10,0.1,0.1,0.1,0.1,0.1,0.1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_conv_fc
# Conv-Small, Gamma
run.py train --encoder-layers convf:32:5:1:1-maxpool:2:2-convv:64:3:1:1-convf:64:3:1:1-maxpool:2:2-convv:128:3:1:1-convv:10:1:1:1-globalmeanpool:6:6-fc:10 --decoder-spec 0-0-0-0-0-0-0-0-0-gauss --denoising-cost-x 0,0,0,0,0,0,0,0,0,1 --labeled-samples 100 --unlabeled-samples 60000 --seed 1  -- mnist_100_conv_gamma
# Conv-Small, supervised baseline. Overfits easily, so keep training short.
run.py train --encoder-layers convf:32:5:1:1-maxpool:2:2-convv:64:3:1:1-convf:64:3:1:1-maxpool:2:2-convv:128:3:1:1-convv:10:1:1:1-globalmeanpool:6:6-fc:10 --decoder-spec 0-0-0-0-0-0-0-0-0-0 --denoising-cost-x 0,0,0,0,0,0,0,0,0,0 --num-epochs 20 --lrate-decay 0.5 --f-local-noise-std 0.45 --labeled-samples 100 --unlabeled-samples 60000 --seed 1 -- mnist_100_conv_baseline
CIFAR models
# Conv-Large, Gamma
./run.py train --encoder-layers convv:96:3:1:1-convf:96:3:1:1-convf:96:3:1:1-maxpool:2:2-convv:192:3:1:1-convf:192:3:1:1-convv:192:3:1:1-maxpool:2:2-convv:192:3:1:1-convv:192:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec 0-0-0-0-0-0-0-0-0-0-0-0-gauss --dataset cifar10 --act leakyrelu --denoising-cost-x 0,0,0,0,0,0,0,0,0,0,0,0,4.0 --num-epochs 70 --lrate-decay 0.86 --seed 1 --whiten-zca 3072 --contrast-norm 55 --top-c False --labeled-samples 4000 --unlabeled-samples 50000 -- cifar_4k_gamma
# Conv-Large, supervised baseline. Overfits easily, so keep training short.
./run.py train --encoder-layers convv:96:3:1:1-convf:96:3:1:1-convf:96:3:1:1-maxpool:2:2-convv:192:3:1:1-convf:192:3:1:1-convv:192:3:1:1-maxpool:2:2-convv:192:3:1:1-convv:192:1:1:1-convv:10:1:1:1-globalmeanpool:0 --decoder-spec 0-0-0-0-0-0-0-0-0-0-0-0-0 --dataset cifar10 --act leakyrelu --denoising-cost-x 0,0,0,0,0,0,0,0,0,0,0,0,0 --num-epochs 20 --lrate-decay 0.5 --seed 1 --whiten-zca 3072 --contrast-norm 55 --top-c False --labeled-samples 4000 --unlabeled-samples 50000 -- cifar_4k_baseline
Evaluating models with testset

After training a model, you can infer the results on a test set by performing the evaluate command. An example use after training a model:

./run.py evaluate results/mnist_all_bottom0
Owner
Curious AI
Deep good. Unsupervised better.
Curious AI
GitHub Repository
A terminal-based number guessing game written in python

A terminal-based number guessing game written in python

Akshay Vs 15 Sep 22, 2022
source codes for my(small indie game developer) games

My repository for most of my finished && unfinished games Table of Contents Getting Started Prerequisites Installation Usage License Contact Prerequis

Gustavs Jākobsons 1 Jan 30, 2022
A simple python script to pregenerate minecraft worlds.

mcloady mcloady is a lightweight python script used to pre-generate Minecraft terrain using MCRcon and carpet mod (optional). Inspired by Pre-Generati

5 Dec 08, 2021
🥕Try and keep da carrot alive or else . . .

Carrot 🥕 Warning 💥 : I am not a botanist. I do not study carrots or plant life. I am a noob programmer :P. So don't belive anything you see in this

1 Jan 03, 2022
Turtle Road Crossing Game in Turtle(python module)

Turtle Road Crossing Game in Turtle(python module) In this project we have built a road crossin game in python with Object-Oriebted Programming. This

Jhenil Parihar 3 Jun 15, 2022
Datamining of 15 Days of (free) Games at the Epic Games Store (EGS).

EGS: 15 Days of Games This repository contains Python code to data-mine the 15 Days of (free) Games at the Epic Games Store (EGS). Requirements Instal

Wok 9 Dec 27, 2022
An all-inclusive Python framework for the Riot Games League of Legends API. We focus on making the data easy and fun to work with, while providing all the tools necessary to create a website or do data analysis.

Cassiopeia A Python adaptation of the Riot Games League of Legends API (https://developer.riotgames.com/). Cassiopeia is the sister library to Orianna

Meraki Analytics 473 Jan 07, 2023
SnailJumper - A game that is developed as an assignment for Computer Intelligence course

Snail jumper Neuroevolution game assignment. Fall 2021 - Computer Intelligence.

Soroush Mehraban 22 Aug 22, 2022
A pure-Python Wordle and Absurdle solver

Pyrdle A pure-Python Wordle and Absurdle solver Find the originals here: Wordle Absurdle Basic solving: Wordle To solve today's Wordle, simply run: ./

3 Feb 09, 2022
This is Minesweeper coded in Python. It has almost all features from the main game

Minesweeper This is Minesweeper coded in Python. It has almost all features from the main game Use right click to open tile, right click on an open ti

3 Jul 12, 2022
Tictactoe py tkinter canvas - Tic Tac Toe written in Python 3 with tkinter mainly using canvas

Python 3 Tic Tac Toe with tkinter This is a tkinter version of my Tic Tac Toe ga

Bojan Adzic 1 Jan 09, 2022
Searches the word list in Wordle based on search pattern.

Wordle Searcher Searches the word list in Wordle based on search pattern. Warning: like all forms of cheating, it trivializes the game, and robs you o

Tyler Martin 1 Jan 29, 2022
A bot that deletes any embeds sent by a tropical webhook containing hex #000000 rancher's boots

tropical-webhook-cleanup how to use download the source code as zip get your discord bot token from https://discord.com/developers/applications put yo

carreb 0 Nov 25, 2022
This is a simple game of rock-paper-scissors developed in Python

This is a simple game of rock-paper-scissors developed in Python. It allows two players to play with one another on different command lines through networking.

NAMAN JAIN 3 Oct 21, 2022
AutoPilot is a game where the player controls a car and tries to get the highest score he can while not dying under falling cement blocks.

AutoPilot AutoPilot is a game where the player controls a car and tries to get the highest score he can while not dying under falling cement blocks. C

Enoc Mena 1 Nov 17, 2021
A Game of Life implementation in Python

Game of Life in Python (Golipy) Golipy is a simulator of John H. Conway's Game of Life, developed in Python based on the Pygame library. This is a toy

Alber 2 Dec 10, 2021
Made to help you create UI using pygame in python, gonna add way more to this project

Pygame visualizer Made to help you create UI using pygame in python, gonna add way more to this project. As of now, this is only hours worth of work.

Ayza 2 Feb 08, 2022
Lucky Balls is gambling game where user try to guess 6 numbers from 1 to 48 that computer has picked.

LUCKY BALLS Lucky Balls is gambling game where user try to guess 6 numbers from 1 to 48 that computer has picked. INSTRUCTIONS User input his bet, tha

rile037 2 Dec 28, 2021
A classic alien shooting game.

Space-Invaders A classic alien shooting game. Description An open source game created by me and friends. How to play Install the latest python version

Phạm Thanh Sơn 1 Feb 08, 2022
PyCharge is an open-source computational electrodynamics Python simulator

PyCharge PyCharge is an open-source computational electrodynamics Python simulator that can calculate the electromagnetic fields and potentials genera

Matthew Filipovich 28 Jan 03, 2023