Swab AI
This code finds bounding box of a single human mouth. In comparison to other face segmentation methods, it is relatively insusceptible to open mouth conditions, e.g., yawning, surgical robots, etc. The mouth coordinates are found in a more certified way using two independent algorithms. Therefore, the algorithm can be used in more sensitive applications.
Sample Output
Selected images from YawDD and CelebAMask dataset.
Example Code
import swab_ai
import cv2
img = cv2.imread("input.jpg") # img is numpy array
points = swab_ai.Get_BoundingBox(img) # returns bounding box coordinates of img
if points: # bounding box is found
# save output file -> only for debugging purposes
cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2))
print("output.jpg is written.")
Python API
swab_ai.py
It loads required libraries and models. This module contains three functions:
GetFace(input_image, show=False, save=False, live=False): callsFaceInference.pyand returns coordinates of face bounding box and two landmarks of mouth.GetMouth(face_image, show=False, save=False, live=False): return mouth mask.Get_BoundingBox(input_image, show=False, save=False, live=False): callsGetFaceandGetMouthfunctions and checks if the bounding box could be certified by checking for sufficient overlap of two independent algorithms. This function returns the mouth bounding box and also prints the FPS in the output. In case mouth bounding box could not be foundFrame Droppeis printed in the console.
live_test.py
For debugging purposes: The algorithm is run on live webcam video. The result of face detection and segmentation algorithms is visualized in real-time.
Dockerfile
We have used the latest version of pytorch image pytorch:1.9.0-cuda11.1-cudnn8-runtime as of now.
To build the docker image and run the container in one step:
docker build -t swabai .
docker run --runtime=nvidia swabai
Note that
--runtime=nvidiais necessary to enable GPU.
If you see the message output.jpg is written, it means that everything is working correctly. To see the output image use this command to mount local working directory:
docker run -v $PWD:/usr/src/app/ --runtime=nvidia swabai
Model
The first time that the code is run, model is downloaded automatically. Every time that the code restarts, it checks for model updates.
50 Nov 27, 2022
11 Oct 15, 2022
129 Jan 4, 2023
66 Dec 21, 2022
27 Jul 20, 2022
99 Dec 31, 2022
762 Jan 8, 2023
8 Jul 9, 2021
193 Dec 5, 2022
1 Jun 08, 2022
101 Jan 04, 2023
1.7k Dec 27, 2022
25 Aug 30, 2022
1 Feb 03, 2022
143 Jan 05, 2023
3 Feb 17, 2022
2 Apr 20, 2022
58 Dec 02, 2022
1 Aug 23, 2022
4 Nov 14, 2022
57 Dec 26, 2022
23 Dec 28, 2022
131 Dec 31, 2022
9 Dec 23, 2022
62 Dec 21, 2022
105 Dec 30, 2022
1 Dec 26, 2021
2 Jul 16, 2022
1 Jul 14, 2022