Swab AI
This code finds bounding box of a single human mouth. In comparison to other face segmentation methods, it is relatively insusceptible to open mouth conditions, e.g., yawning, surgical robots, etc. The mouth coordinates are found in a more certified way using two independent algorithms. Therefore, the algorithm can be used in more sensitive applications.
Sample Output
Selected images from YawDD and CelebAMask dataset.
Example Code
import swab_ai
import cv2
img = cv2.imread("input.jpg") # img is numpy array
points = swab_ai.Get_BoundingBox(img) # returns bounding box coordinates of img
if points: # bounding box is found
# save output file -> only for debugging purposes
cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2))
print("output.jpg is written.")
Python API
swab_ai.py
It loads required libraries and models. This module contains three functions:
GetFace(input_image, show=False, save=False, live=False): callsFaceInference.pyand returns coordinates of face bounding box and two landmarks of mouth.GetMouth(face_image, show=False, save=False, live=False): return mouth mask.Get_BoundingBox(input_image, show=False, save=False, live=False): callsGetFaceandGetMouthfunctions and checks if the bounding box could be certified by checking for sufficient overlap of two independent algorithms. This function returns the mouth bounding box and also prints the FPS in the output. In case mouth bounding box could not be foundFrame Droppeis printed in the console.
live_test.py
For debugging purposes: The algorithm is run on live webcam video. The result of face detection and segmentation algorithms is visualized in real-time.
Dockerfile
We have used the latest version of pytorch image pytorch:1.9.0-cuda11.1-cudnn8-runtime as of now.
To build the docker image and run the container in one step:
docker build -t swabai .
docker run --runtime=nvidia swabai
Note that
--runtime=nvidiais necessary to enable GPU.
If you see the message output.jpg is written, it means that everything is working correctly. To see the output image use this command to mount local working directory:
docker run -v $PWD:/usr/src/app/ --runtime=nvidia swabai
Model
The first time that the code is run, model is downloaded automatically. Every time that the code restarts, it checks for model updates.
50 Nov 27, 2022
11 Oct 15, 2022
129 Jan 4, 2023
66 Dec 21, 2022
27 Jul 20, 2022
99 Dec 31, 2022
762 Jan 8, 2023
8 Jul 9, 2021
193 Dec 5, 2022
90 Jan 06, 2023
41 Nov 09, 2022
2 Aug 21, 2022
104 Nov 22, 2022
1 Dec 28, 2021
184 Dec 31, 2022
2.5k Jan 04, 2023
236 Dec 26, 2022
80 Dec 27, 2022
31 Sep 04, 2022
0 Apr 02, 2021
375 Dec 30, 2022
46 Dec 25, 2022
8 Sep 03, 2022
35 Aug 23, 2022
2 Dec 12, 2022
757 Jan 04, 2023
108 Dec 29, 2022
20 May 22, 2022
2 Dec 30, 2021