As part of a project led by Tencent, several researchers have succeeded in developing a model for restoring images and photographs of faces. It is far from being the first tool of its kind, but the results it offers are much better than the average, as shown by the various comparisons (see illustration). The tool exploits generative adversarial networks coupled with a degradation removal module.

The challenge of restoring facial images and photos with artificial intelligence

Image restoration should not be confused with digital retouching alone. The two processes are quite similar, but the techniques used can be different. If the first one seems rather objective, the other is subjective. Digital retouching is the process of making an image look better, while restoration is the process of reversing known degrading operations applied to images so that they are no longer visible in the images.

Normally, face restoration relies on several factors such as facial geometry. However, the low quality of input images often does not help in applying techniques related to this geometry, which limits restoration applications. To overcome this problem, a research team proposes GFP-GAN which exploits other features and techniques to successfully restore a photograph in which a face appears thanks to artificial intelligence.

Their model was published in a paper written by Xintao Wang, Yu Li, Honglun Zhang Ying Shan, all of whom work for Tencent’s Applied Research Center.

What is the GFP-GAN model made of? How does it work?

In designing the GFP-GAN model, the researchers aimed to achieve a good balance between reality and fidelity of the initially degraded image. As the name suggests, it consists of a generative adversarial network (GAN) that has been combined with Generative Facial Prior (GFP), a tool specifically designed for image restoration.

Traditional restoration models exploit the inversion method: they first invert the degraded image so that it is in a state that the pre-trained GAN can recognize and then perform image-specific optimization techniques to restore it. GFP-GAN, on the other hand, uses a degradation removal module (U-Net) and the pre-trained face GAN to capture facial features. These are interconnected by code mapping and using CS-SFT (Channel-Split Spatial Feature Transform) layers.

GFP-GAN is pre-trained on the FFHQ dataset, which contains about 70,000 high-quality images. All images were resized to 512×512 pixels during training. Thus the model could be trained on synthetic data that approximates low-quality real-world images and generalizes to real-world images during inference output synthesis. As we can by comparing the output image against the input image, several types of image degradation have been removed.

According to the researchers, the GFP-GAN model with CS-SFT layers achieves a good balance between image fidelity and reality in a single pass. The code is freely available on GitHub.

Translated from GFP-GAN, un modèle de restauration d’images exploitant les GAN avec des résultats prometteurs