3D Face Reconstruction: A Journey to Accurate, Fast, and Accessible Mobile Solution

Businesses need to stay ahead of the curve in order to remain competitive, in any era and in any industry. In our case, accurate, fast and accessible simple 3D face reconstruction is an innovative technology that will give companies who have it massive advantage over their competitors. Utilizing advanced scanning technologies and algorithms, businesses can leverage 3D face reconstruction to unlock numerous game-changing advantages like customization and personalization, virtual try-on and fitting, improved security measures and more. In today’s competitive landscape, where personalization, efficiency, and customer experiences are paramount, achieving accurate, fast, and simple 3D face reconstructions holds the key to unlocking new opportunities and driving business success.

At tsukat, we recognize the crucial role reliable technology products play in our clients’ operations. To ensure that we provide only the most exceptional and trustworthy solutions, we conduct thorough internal testing on all new technology products before making them available to our clients. This meticulous testing and evaluation process instills confidence in our clients, allowing them to focus on their business operations, knowing they can rely on the quality and performance of the products we offer.

This whitepaper documents our journey of trial and error toward enhancing the 3D face scanning process. Our focus lies on the essential aspects of 3D face reconstruction, namely accuracy, speed, and accessibility, which we briefly discuss throughout this document. What sets our approach apart is the ability to scan a dynamic human face using a mobile device. At the end of the day, we believe that by leveraging advanced scanning techniques, intelligent algorithms, and streamlined workflows, businesses have the potential to revolutionize their operations, products, and services.

Challenge: Obtaining accurate 3D face reconstruction on a mobile device.

3D object reconstruction is a challenging task that involves capturing the spatial structure and appearance of real-world objects in order to obtain an accurate three-dimensional digital representation, which can be later altered, viewed and shared. With ongoing digitalization trends, this problem garnered significant attention due to its wide range of applications across various domains, including design and engineering, architecture, art, healthcare, education, etc.

Among the various applications of 3D object reconstruction, scanning the human face stands out in terms of its complexity and types of use cases. Accurate 3D face reconstruction enables a range of applications, including personalized avatar creation and animations, surgical planning and prosthetics design, as well as virtual apparel and makeup try-on.

Modern phones with depth-sensing capabilities have transformed the 3D scanning field by combining portability, advanced hardware, sophisticated software algorithms and artificial intelligence advances. They made the process more accessible to a broader audience while also consistently raising the quality of the produced results. Still, when it comes to face scanning, there are specific challenges and considerations that need to be accounted for. Scanning procedures rely on matching pictures/point sets between different views. For the matching to be accurate the object has to remain stationary in different perspectives. It is, therefore, typical for the scanned object to be locked in place, and the scanning device to be moved around it. Unlike inanimate objects, however, humans are always in motion consciously or unconsciously. Facial expressions, movements and articulations introduce complexities that render regular approaches impractical. In addition, faces are often occluded by hair or accessories, while variations in lighting conditions introduce noise in the captured data and deteriorate texture quality.

Nevertheless, there are a number of existing solutions for tackling the aforementioned issues to create realistic face scans. The one that stood out was the Bellus3D face scanning solution, a powerful and easy-to-use application for iPhones and iPads with TrueDepth cameras. When Bellus3D exited the face scanning market, it created a void that impacted numerous use cases in which easy-to-create but also accurate facial scans were required. Existing alternatives are designed as general usage scanning apps, and are neither made for making human scans without external help, nor providing the necessary quality of texture and geometry accuracy of the scanned object.

Thus we set out on a mission to develop a custom solution that would address the specific needs of partners who relied on Bellus3D for face scanning functionality.

Our primary goals in this endeavor were:

1. Achieving the highest possible accuracy in scan geometry: We aimed to optimize the scanning process utilizing Apple devices equipped with TrueDepth cameras, while ensuring the resulting scans exhibited exceptional precision in capturing facial features. Additionally, we aimed to maintain high-quality texture representation for a comprehensive and detailed 3D reconstruction.
2. Inclusion of the frontal part of the face, hair, and ears: Our solution aimed to capture not only the frontal region of the face but also the surrounding elements such as hair and ears. This comprehensive scanning approach would enable a more holistic representation of the individual’s facial structure.
3. User-friendly scanning procedure: We strived to design an intuitive and simple scanning procedure for users to perform independently. By holding the device in front of them and rotating their heads, users would be able to conduct the scanning process without requiring external assistance, ensuring ease of use and convenience.
4. On-device processing within a reasonable time frame: To ensure the best user experience, where users can gain real-time or near-real-time results, we focus on performing processing tasks directly on the scanning devices. In cases with limited
   data transmission speeds or low-quality internet, this approach aimed to ensure timely feedback and minimize delays associated with transferring data to external processing resources.
5. Detection of key facial feature positions in 3D, including ears: Beyond the overall face reconstruction, we aimed to develop algorithms capable of accurately detecting and positioning key facial features, including the ears. This feature detection would enable customization and further applications based on specific facial attributes.

By pursuing these goals, we aimed to create a robust and reliable solution that met the requirements of our partners and delivered exceptional results in terms of accuracy, accessibility, and efficiency. Through our research and development efforts, we sought to push the boundaries of existing 3D face scanning technology and empower businesses with advanced capabilities for their respective industries.

Download this White Paper to:

• Gain insights into the innovative technologies and methodologies our engineers have implemented to accomplish our goals;
• Explore how our technology can revolutionize business operations, products, and services;
• Experience the unwavering commitment of our engineers in the intricate development process;
• Discover how our solution can give your company a competitive edge.

Anton Mykytiuk

Head of 3D

Info about author