Chinese version link

This project is to utilize facial recognition to create a facial identity system. Our backend is constructed by one-shot models which is more flexible for adding a new face. The system is built on personal computer and Jetson Nano. Jetson Nano is used to recognized the faces and upload the detected information to Firebase. Users who use our application with account and password can log in to control the database and obtain the information.

| - backend - For Personal computer
|
| - csv_file - Contribution for the CelebA dataset
|
| - jetson - Files for Jetson Nano
|
| - model - Model we used for training and detecting

Our facial identity system includes below features:

• One-shot face recognition, add your faces without extra training
• Complete database operation (upload, delete, update)
• Fine-tuned your model at any time
• Used as a monitor
• Visualize the features

$ pip install -r backend/requirements.txt

$ pip install -r jetson/requirements.txt

Our nano would crush when using cuda until we increase its swap memory 🥳

# 4.0G is the swap space
$ sudo fallocate -l 4.0G /swapfile
$ sudo chmod 600 /swapfile
$ sudo mkswap /swapfile
$ sudo swapon /swapfile

# Create swap memory on every reboot
$ sudo bash -c 'echo "/var/swapfile swap swap defaults 0 0" >> /etc/fstab'

This section is to help users to set up facial identity system. The required materials are listed below:

• 1 Camera
• 1 button
• 1 RGB LED
• 4 1K Resistor

a. Create a project in Firebase
b. Select the project created in step 1, and click the Realtime Database on the left sidebar
c. Copy the link, which looks like https://<project-name>-default-rtdb.<place>.firebasedatabase.app/
d. Replace <your firebase url> in line 117 in backend/main.py and line 234 in jetson/main.py with your link

a. Activate the Authentication and get the json file
b. Create Cloud Storage bucket
c. Replace <your json file> in line 110 in backend/main.py and line 224 in jetson/main.py with the location of your json file

User can train the model by:

$ python model/triplet/train.py \
    --csv_path "<csv file>" \
    --img_root "<root for the training file>" \
    --weight "<place to save the weight file>"

See more arguments in model/triplet/train.py

Illustration of Triplet loss

We refer to Tamer Thamoqa's repo for training our model

• Triplet-0.4-regul: With margin 0.4 and trained with regularization
• Triplet-0.2: With margin 0.2

(Left.) PCA. (Right.) TSNE

In order to train the model without extra label, we design a self-supervised learning method based on original triplet loss.

• Anchor image is the original image
• Positive image is the horizontal flip image derived from the original image
• Negative image is made by both vertical and horizontal flip on the original image.

Illustration of Triplet loss

The experiment was examined on triplet-0.4-regul.pt

• second per image (s / img)

It is surprising that cuda consumes lots of time. We guess it is because cuda rely on huge amount of swap memory that slow down its runtime 😢.

In order to train one-shot model, we obtain the face's coordinates beforehand. All files are placed in csv_file.

The coordinates were obtained from facenet-pytorch

@inproceedings{liu2015faceattributes,
  title = {Deep Learning Face Attributes in the Wild},
  author = {Liu, Ziwei and Luo, Ping and Wang, Xiaogang and Tang, Xiaoou},
  booktitle = {Proceedings of International Conference on Computer Vision (ICCV)},
  month = {December},
  year = {2015} 
}

@inproceedings{koch2015siamese,
  title={Siamese neural networks for one-shot image recognition},
  author={Koch, Gregory and Zemel, Richard and Salakhutdinov, Ruslan and others},
  booktitle={ICML deep learning workshop},
  volume={2},
  year={2015},
  organization={Lille}
}

@inproceedings{chen2020simple,
  title={A simple framework for contrastive learning of visual representations},
  author={Chen, Ting and Kornblith, Simon and Norouzi, Mohammad and Hinton, Geoffrey},
  booktitle={International conference on machine learning},
  pages={1597--1607},
  year={2020},
  organization={PMLR}
}

@inproceedings{schroff2015facenet,
  title={Facenet: A unified embedding for face recognition and clustering},
  author={Schroff, Florian and Kalenichenko, Dmitry and Philbin, James},
  booktitle={Proceedings of the IEEE conference on computer vision and pattern recognition},
  pages={815--823},
  year={2015}
}

@misc{wandb,
    title = {Experiment Tracking with Weights and Biases},
    year = {2020},
    note = {Software available from wandb.com},
    url={https://www.wandb.com/},
    author = {Biewald, Lukas},
}