Software for Multimodalty 2D+3D Facial Expression Recognition (FER) UI.

(with RealSense)

• Python >= 3.6
• numpy >= 1.19.5
• Opencv-python >= 4.5
• PySide6 >= 6.2.1
• PyTorch >= 1.10
• TorchVision >= 0.11

1. usb camera
2. Intel RealSense (optional for depth imaging)
3. NV GPU (optional, CPU version also works)

Two ways, try them, depending on your Internet speed.

1. Manually download. Download vgg16 weights from:

   https://drive.google.com/file/d/1f-tKgovJ54l9xR3oIZ6gy77NdPirUddr/view?usp=sharing

   Then, move the weights to "./weights" folder, and rename it with "ui_weights.pth".

or

2. Run script. Open terminal, run:

   > cd weights
   > python download_weights.py

> cd train/dataset
> python main_train.py

This will also allow you to collect your own expression dataset before the network training.

After the training, remember to move the new weights to "weights" folder, and rename it with the "ui_weights.pth".

The current FER network is implemented by vgg16 for simplicity. Current predictions are based on 2D (RGB texture) result.

One can easily change the network as you wish.