Behind the Scenes of Blind Face Restoration: An In-depth Look at GFPGAN, CodeFormer, and RestoreFormer

Blind Face Restoration is a field in computer vision that aims at recovering high-quality faces from low-quality faces suffering from unknown degradation. This degradation could be due to factors such as low-resolution, noise, blur, compression artifacts, etc.

Several methods have been proposed to tackle this problem, including the use of Generative Adversarial Networks (GANs), Variation Auto Encoders (VAEs), Transformers in Vision and the latest Diffusion based models.

The algorithms GFPGAN, RestoreFormer and CodeFormer are among the most widely used and accepted in this field.

Indeed, it’s been noted that using a FaceRestoration model in isolation can lead to suboptimal results in real-world scenarios. This is primarily because many images are not perfectly aligned or centrally positioned.

The model’s performance on real-world samples is poor because the actual model has been trained exclusively on aligned faces, where the faces are centered and without any orientation.

To address these real-world complexities, implementations of FaceRestoration algorithms often incorporate additional modules. This multi-module approach enhances the robustness and applicability of the algorithms, leading to improved performance on diverse, real-world images.

The modules that are behind the scenes in a Blind Face Restoration repository along with Face Restoration (eg: GFPGAN) module are:

• Face and landmark Detection (eg : RetinaFace)
• Background upsampling (eg : ESRGAN)
• Face Segmentation module (eg : Parsenet)

When we use all these modules together, they don’t just fix faces but also improve the whole picture, including the background. Plus, they usually make the image larger and clearer.

See the below examples after applying all the above four modules: