Keras like implementation of Deep Learning architectures from scratch using numpy.

The project contains implementations for various activation functions, layers, loss functions, model structures and optimizers in files activation.py, layer.py, loss.py, model.py and optimizer.py respectively.

Given below is list of available implementations (which may or may not require any improvements).

Each of the implementations are class-based and follows a keras like structure. A typical model training with Mini-Keras looks like this,

from model import Sequential
from layer import Dense, Conv2D, MaxPool2D, Flatten
from loss import BinaryCrossEntropy
from activation import Sigmoid
from optimizer import GradientDescentOptimizer

model = Sequential()
model.add(Conv2D, ksize=3, stride=1, activation=Sigmoid(), input_size=(8,8,1), filters=1, padding=0)
model.add(MaxPool2D, ksize=2, stride=1, padding=0)
model.add(Conv2D, ksize=2, stride=1, activation=Sigmoid(), filters=1, padding=0)
model.add(Flatten)
model.add(Dense, units=1, activation=Sigmoid())
model.summary()

model.compile(BinaryCrossEntropy())

print("Initial Loss", model.evaluate(X, y)[0])
model.fit(X, y, n_epochs=100, batch_size=300, learning_rate=0.003, optimizer=GradientDescentOptimizer(), verbose=1)
print("Final Loss", model.evaluate(X, y)[0])

As you might have noticed, its very similar to how one will do it in Keras.

The run.py consists of a small code snippet that can be used to test if your new implementation is working properly or not.

All the implementations have a forward propagation and a backward propagation equivalent available as a method in the corresponding class. Below are the details for implementing all the functionalities under different categories.

README.ipynb explains each of the implementations with mathematical proofs for better understanding.