PFStorer: Personalized Face Restoration and Super-Resolution

Recent developments in face restoration have achieved remarkable results in producing high-quality and lifelike outputs. The stunning results however often fail to be faith-ful with respect to the identity of the person as the models lack necessary context. In this paper, we explore the poten-tial of personalized face restoration with diffusion models. In our approach a restoration model is personalized using a few images of the identity, leading to tailored restoration with respect to the identity while retaining fine-grained de-tails. By using independent trainable blocks for personal-ization, the rich prior of a base restoration model can be ex-ploited to its fullest. To avoid the model relying on parts of identity left in the conditioning low-quality images, a gener-ative regularizer is employed. With a learnable parameter, the model learns to balance between the details generated based on the input image and the degree of personalization. Moreover, we improve the training pipeline of face restoration models to enable an alignment-free approach. We showcase the robust capabilities of our approach in sev-eral real-world scenarios with multiple identities, demon-strating our method’s ability to generate fine-grained de-tails with faithful restoration. In the user study we evalu-ate the perceptual quality and faithfulness of the generated details, with our method being voted best 61% of the time compared to the second best with 25% of the votes.