3D Face Reconstruction

Our System show how a computer is able to detect human faces, track facial key points (eye corner, nose tip, mouth corner etc.) and reconstruct 3D face shapes in real time. This technology has many practical applications in security systems, digital entertainment, human-computer interaction, augmented reality and virtual reality. In this system, we first apply a deep-learning-based face detector to detect human faces in each video frame. We then use our state-of-the-art deep-learning-based facial landmark detector to obtain 2D facial landmarks (key points) of the detected faces. Our cutting-edge facial landmark detection algorithm is robust to a wide spectrum of appearance variations in pose, illumination, expression, occlusion etc. Last, we apply our real-time 3D face reconstruction method to obtain the 3D facial shapes in the form of 3D point cloud. To this end, we fit our Surrey 3D morphable face model to the detected 2D facial landmarks.

Image placeholder Image placeholder