GAN-Inversion

<!-- # <p align=center>`awesome gan-inversion`</p> --> <!-- [](https://github.com/sindresorhus/awesome) [](https://GitHub.com/Naereen/StrapDown.js/graphs/commit-activity) [](http://makeapullrequest.com) --> <!--  --> <!-- <br/> --> <p align="center"> <h1 align="center">GAN Inversion: A Survey</h1> <p align="center"> TPAMI 2022 <br /> <a href="https://weihaox.github.io/"><strong>Weihao Xia</strong></a> · <a href="https://yulunzhang.com/"><strong>Yulun Zhang</strong></a> · <a href="https://sites.google.com/view/iigroup-thu/about"><strong>Yujiu Yang</strong></a> · <a href="http://www.homepages.ucl.ac.uk/~ucakjxu/"><strong>Jing-Hao Xue</strong></a> · <a href="https://boleizhou.github.io/"><strong>Bolei Zhou</strong></a> · <a href="https://faculty.ucmerced.edu/mhyang/"><strong>Ming-Hsuan Yang</strong></a> </p> <p align="center"> <a href='https://arxiv.org/abs/2101.05278'> <img src='https://img.shields.io/badge/Paper-PDF-green?style=flat&logo=arxiv&logoColor=green' alt='arxiv PDF'> </a> <a href='https://github.com/weihaox/awesome-gan-inversion' style='padding-left: 0.5rem;'> <img src='https://img.shields.io/badge/Project-Page-blue?style=flat&logo=Google%20chrome&logoColor=blue' alt='Project Page'> </a> <a href='https://ieeexplore.ieee.org/document/9792208' style='padding-left: 0.5rem;'> <img src='https://img.shields.io/badge/TPAMI-PDF-red?style=flat&logoColor=red' alt='TPAMI PDF'> </a> </p> </p> <br />

This repo is a collection of resources on GAN inversion, as a supplement for our survey. If you find any work missing or have any suggestions (papers, implementations and other resources), feel free to pull requests. You could manually edit items or use the script to produce them in the markdown format.

<details style="margin-left:3%;"> <summary>citation</summary> <pre><code class="language-bib" style="font-size: 0.9rem;" id="citation">@article{xia2022gan, author = {Xia, Weihao and Zhang, Yulun and Yang, Yujiu and Xue, Jing-Hao and Zhou, Bolei and Yang, Ming-Hsuan}, title = {GAN Inversion: A Survey}, journal = {IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)}, year={2022} } </code></pre> </details> <details><summary>Table of Contents</summary><p>

• Inverted Pretrained Models
  • 2D GANs
  • 3D GANs
• GAN Inversion Methods
  • 3D GAN Inversion
  • 2D GAN Inversion
• Diffusion Inversion
• GAN Latent Space Editing
• Diffusion Latent Space Editing
• Applications
  • Image and Video Generation and Manipulation
  • Image Restoration
  • Image Understanding
  • Face Recognition
  • 3D Reconstruction
  • Other Applications
    • Compressed Sensing
    • Medical Imaging
    • Compression, Fairness, and Security
• Acknowledgement

</p></details><p></p>

Inverted Pretrained Models

2D GANs

Scaling up GANs for Text-to-Image Synthesis.<br> Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park, Eli Shechtman, Sylvain Paris, Taesung Park.<br> CVPR 2023 (Highlight). [PDF] [Project]

StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis.<br> Axel Sauer, Tero Karras, Samuli Laine, Andreas Geiger, Timo Aila.<br> ICML 2023. [Project] [PDF] [Code]

StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets.<br> Axel Sauer, Katja Schwarz, Andreas Geiger.<br> SIGGRAPH 2022. [PDF] [Project] [Code]

Self-Distilled StyleGAN: Towards Generation from Internet Photos.<br> Ron Mokady, Michal Yarom, Omer Tov, Oran Lang, Daniel Cohen-Or, Tali Dekel, Michal Irani, Inbar Mosseri.<br> SIGGRAPH 2022. [PDF] [Project] [Code]

Ensembling Off-the-shelf Models for GAN Training.<br> Nupur Kumari, Richard Zhang, Eli Shechtman, Jun-Yan Zhu<br> CVPR 2022. [PDF] [Project] [Code]

StyleGAN3: Alias-Free Generative Adversarial Networks.<br> Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, Timo Aila.<br> NeurIPS 2021. [PDF] [Project] [Code] [Rosinality]

StyleGAN2-Ada: Training Generative Adversarial Networks with Limited Data.<br> Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, Timo Aila.<br> NeurIPS 2020. [PDF] [Code] [Steam StyleGAN2-ADA]

StyleGAN2: Analyzing and Improving the Image Quality of StyleGAN.<br> Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, Timo Aila.<br> CVPR 2020. [PDF] [PyTorch] [Offical TF] [Unoffical Tensorflow 2.0]

StyleGAN: A Style-Based Generator Architecture for Generative Adversarial Networks.<br> Tero Karras, Samuli Laine, Timo Aila.<br> CVPR 2019. [PDF] [Offical TF]

ProGAN: Progressive Growing of GANs for Improved Quality, Stability, and Variation.<br> Tero Karras, Timo Aila, Samuli Laine, Jaakko Lehtinen.<br> ICLR 2018. [PDF] [Offical TF]

3D-aware GANs

Please check our 3D-aware image synthesis survey, paper list, and project for more details.

EG3D: Efficient Geometry-aware 3D Generative Adversarial Networks.<br> Eric R. Chan, Connor Z. Lin, Matthew A. Chan, Koki Nagano, Boxiao Pan, Shalini De Mello, Orazio Gallo, Leonidas Guibas, Jonathan Tremblay, Sameh Khamis, Tero Karras, Gordon Wetzstein.<br> CVPR 2022. [PDF] [Project] [Code]

StyleSDF: High-Resolution 3D-Consistent Image and Geometry Generation.<br> Roy Or-El, Xuan Luo, Mengyi Shan, Eli Shechtman, Jeong Joon Park, Ira Kemelmacher-Shlizerman.<br> CVPR 2022. [PDF] [Project] [Code]

StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis.<br> Jiatao Gu, Lingjie Liu, Peng Wang, Christian Theobalt.<br> ICLR 2022. [PDF] [Project]

pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesis.<br> Eric R. Chan, Marco Monteiro, Petr Kellnhofer, Jiajun Wu, Gordon Wetzstein.<br> CVPR 2021. [PDF] [Project] [Code]

GAN Inversion Methods

The section primarily encompasses general-purpose 2D or 3D inversion techniques, whereas the methods presented in the following section cater to particular applications.

3D GAN Inversion

TriPlaneNet: An Encoder for EG3D Inversion.<br> Ananta R. Bhattarai, Matthias Nießner, Artem Sevastopolsky.<br> WACV 2024. [PDF] [Project]

In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing.<br> Yiran Xu, Zhixin Shu, Cameron Smith, Jia-Bin Huang, Seoung Wug Oh.<br> CVPR 2024. [PDF] [Project]

Make Encoder Great Again in 3D GAN Inversion through Geometry and Occlusion-Aware Encoding.<br> Ziyang Yuan, Yiming Zhu, Yu Li, Hongyu Liu, Chun Yuan.<br> ICCV 2023. [PDF] [Project] [Code]

LatentSwap3D: Semantic Edits on 3D Image GANs.<br> Enis Simsar, Alessio Tonioni, Evin Pınar Örnek, Federico Tombari.<br> ICCV 2023 Workshops on AI3DCC. [PDF] [Code]

High-fidelity 3D GAN Inversion by Pseudo-multi-view Optimization.<br> *Jiaxin Xie, Hao Ouyang, Jingtan Piao, [Chenyang