Action Vision-Face-AI: An AI-Based Real-Time Facial Analysis and Recognition System

Abstract

Vision-Face-AI is a facial analysis web system described in this paper, where all the inference methods are implemented in the system of the user rather than uploading the image data to a centralized server. The current facial recognition systems typically require either commercial cloudbased APIs or expensive, in-house back-end systems, whereas our system uses face detection, facial landmark detection (68 points), face recognition through embedding, and multi-label face attribute prediction (age, gender, emotion in seven classes) into a lightweight web application ran on TensorFlow.js faceapi.js and templated TensorFlow models. We also introduce five dimensions of beauty scoring which makes use of facial measures, symmetry and reference numbers in the golden ratio to yield a measured beauty index and rational recommendations that can enhance the features of the user. We ran the system on nine formal test cases all of which passed, and measured the inference time with and without WebGL acceleration. We observe that our analysis takes approximately less than 50 milliseconds on modern, consumerlevel specialized equipment, and recognizes faces at the rate of 96.2% on our data set. Experiencing Vision-Face-AI demonstrates that there is an opportunity to implement privacy-aware facial AI without the cloud.