This repository contains a collection of resources and papers on Personalization. We also release a survey about personalized content synthesis. You can find it from [arXiv].

If you find any missing work, please report it by creating an Issue in the repository to contribute the community together.

Citation

If you find the information in our paper useful for your research, please consider citing it in your work. Thank you!

@misc{zhang2024survey,
      title={A Survey on Personalized Content Synthesis with Diffusion Models}, 
      author={Xulu Zhang and Xiao-Yong Wei and Wengyu Zhang and Jinlin Wu and Zhaoxiang Zhang and Zhen Lei and Qing Li},
      year={2024},
      eprint={2405.05538},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}

Contents

• Papers
  • Personalized Subject Generation
  • Multi-concept Composition
  • Personalized Style Generation
  • Personalized Face Generation
  • Personalization with Extra Condition
  • Personalized Video Generation
  • Personalized 3D Generation
  • Others

Papers

Personalized Object Generation

An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion
ICLR 2023
[Github] [Paper]
2-Aug-22

DreamBooth: Fine-Tuning Text-to-Image Diffusion Models for Subject-Driven Generation
CVPR 2023
[Github] [Paper]
25-Aug-22

Re-Imagen: Retrieval-Augmented Text-to-Image Generator
ICLR 2023
[Paper]
29-Sep-22

Versatile Diffusion: Text, Images, and Variations All in One Diffusion Model
ICCV 2023
[Github] [Paper]
15-Nov-22

DreamArtist: Towards Controllable One-Shot Text-to-Image Generation via Positive-Negative Prompt-Tuning
arXiv 2022
[Github] [Paper]
21-Nov-22

Is This Loss Informative? Faster Text-to-Image Customization by Tracking Objective Dynamics
NeurIPS 2023
[Github] [Paper]
9-Feb-23

Encoder-Based Domain Tuning for Fast Personalization of Text-to-Image Models
ACM Trans on Graphics
[Github] [Paper]
23-Feb-23

ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation
ICCV 2023
[Github] [Paper]
27-Feb-23

Highly Personalized Text Embedding for Image Manipulation by Stable Diffusion
arXiv
[Github] [Paper]
15-Mar-23

Unified Multi-Modal Latent Diffusion for Joint Subject and Text Conditional Image Generation
arXiv
[Paper]
16-Mar-23

P+: Extended Textual Conditioning in Text-to-Image Generation
arXiv
[Github] [Paper]
16-Mar-23

A Closer Look at Parameter-Efficient Tuning in Diffusion Models
arXiv
[Github] [Paper]
31-Mar-23

Subject-Driven Text-to-Image Generation via Apprenticeship Learning
NIPS 2023
[Paper]
1-Apr-23

Taming Encoder for Zero Fine-Tuning Image Customization with Text-to-Image Diffusion Models
arXiv
[Paper]
5-Apr-23

InstantBooth: Personalized Text-to-Image Generation Without Test-Time Finetuning
arXiv
[Github] [Paper]
6-Apr-23

Controllable Textual Inversion for Personalized Text-to-Image Generation
arXiv
[Github] [Paper]
11-Apr-23

Gradient-Free Textual Inversion
ACM MM 2023
[Github] [Paper]
12-Apr-23

Personalize Segment Anything Model with One Shot
ICLR 2024
[Github] [Paper]
4-May-23

DisenBooth: Identity-Preserving Disentangled Tuning for Subject-Driven Text-to-Image Generation
ICLR 2024
[Github] [Paper]
5-May-23

BLIP-Diffusion: Pre-Trained Subject Representation for Controllable Text-to-Image Generation and Editing
NIPS 2023
[Github] [Paper]
24-May-23

A Neural Space-Time Representation for Text-to-Image Personalization
SIGGRAPH Asia 2023
[Github] [Paper]
24-May-23

Prospect: Prompt Spectrum for Attribute-Aware Personalization of Diffusion Models
ACM Trans on Graphics
[Github] [Paper]
25-May-23

Break-a-Scene: Extracting Multiple Concepts from a Single Image
SIGGRAPH ASIA 2023
[Github] [Paper]
25-May-23

COMCAT: Towards Efficient Compression and Customization of Attention-Based Vision Models
arXiv
[Github] [Paper]
26-May-23

ViCo: Plug-and-Play Visual Condition for Personalized Text-to-Image Generation
arXiv
[Github] [Paper]
1-Jun-23

Controlling Text-to-Image Diffusion by Orthogonal Fine-Tuning
arXiv
[Github] [Paper]
12-Jun-23

Domain-Agnostic Tuning-Encoder for Fast Personalization of Text-to-Image Models
SIGGRAPH 2023
[Github] [Paper]
13-Jul-23

IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models
arXiv
[Github] [Paper]
13-Aug-23

Navigating Text-to-Image Customization: From Lycoris Fine-Tuning to Model Evaluation
ICLR 2024
[Github] [Paper]
26-Sep-23

Kosmos-G: Generating Images in Context with Multimodal Large Language Models
arXiv
[Github] [Paper]
4-Oct-23

Personalized Text-to-Image Model Enhancement Strategies: SOD Preprocessing and CNN Local Feature Integration
arXiv
[Paper]
26-Oct-23

A Data Perspective on Enhanced Identity Preservation for Diffusion Personalization
ICLR 2024
[Github] [Paper]
7-Nov-23

DIFFNAT: Improving Diffusion Image Quality Using Natural Image Statistics
arXiv
[Paper]
16-Nov-23

An Image is Worth Multiple Words: Multi-attribute Inversion for Constrained Text-to-Image Synthesis
arXiv
[Paper]
20-Nov-23

LEGO: Learning to Disentangle and Invert Concepts Beyond Object Appearance in Text-to-Image Diffusion Models
arXiv
[Github] [Paper]
23-Nov-23

Catversion: Concatenating Embeddings for Diffusion-Based Text-to-Image Personalization
arXiv
[Github] [Paper]
24-Nov-23

CLiC: Concept Learning in Context
CVPR 2024
[Github] [Paper]
28-Nov-23

HiFi Tuner: High-Fidelity Subject-Driven Fine-Tuning for Diffusion Models
arXiv
[Paper]
30-Nov-23

InstructBooth: Instruction-Following Personalized Text-to-Image Generation
arXiv
[Paper]
4-Dec-23

Customization Assistant for Text-to-Image Generation
CVPR 2024
[Paper]
5-Dec-23

Decoupled Textual Embeddings for Customized Image Generation
AAAI 2024
[Github] [Paper]
19-Dec-23

Towards Accurate Guided Diffusion Sampling through Symplectic Adjoint Method
arXiv
[Github] [Paper]
19-Dec-23

DreamDistribution: Prompt Distribution Learning for Text-to-Image Diffusion Models
arXiv
[Github] [Paper]
21-Dec-23

DreamTuner: Single Image is Enough for Subject-Driven Generation
arXiv
[Github] [Paper]
21-Dec-23

BootPIG: Bootstrapping Zero-Shot Personalized Image Generation Capabilities in Pretrained Diffusion Models
arXiv
[Paper]
25-Jan-24

Object-Driven One-Shot Fine-Tuning of Text-to-Image Diffusion with Prototypical Embedding
arXiv
[Paper]
28-Jan-24

DisenDreamer: Subject-Driven Text-to-Image Generation with Sample-aware Disentangled Tuning
arXiv
[Paper]
26-Feb-24

Infusion: Preventing Customized Text-to-Image Diffusion from Overfitting
arXiv
[Paper]
22-Apr-24

Customizing Text-to-Image Models with a Single Image Pair
arXiv
[Paper]
2-May-24

Multi-concept Composition

Multi-Concept Customization of Text-to-Image Diffusion
CVPR 2023
[Github] [Paper]
8-Dec-22

CONES: Concept Neurons in Diffusion Models for Customized Generation
ICML 2023
[Github] [Paper]
9-Mar-23

SVDiff: Compact Parameter Space for Diffusion Fine-Tuning
ICCV 2023
[Github] [Paper]
20-Mar-23

Key-Locked Rank One Editing for Text-to-Image Personalization
SIGGRAPH 2023
[Paper]
2-May-23

Mix-of-Show: Decentralized Low-Rank Adaptation for Multi-Concept Customization of Diffusion Models
NIPS 2023
[Github] [Paper]
29-May-23

CONES 2: Customizable Image Synthesis with Multiple Subjects
NIPS 2023
[Github] [Paper]
30-May-23

Generate Anything Anywhere in Any Scene
arXiv
[Github] [Paper]
29-Jun-23

AnyDoor: Zero-Shot Object-Level Image Customization
arXiv
[Github] [Paper]
18-Jul-23

Subject-Diffusion: Open Domain Personalized Text-to-Image Generation Without Test-Time Fine-Tuning
arXiv
[Github] [Paper]
21-Jul-23

CustomNet: Zero-Shot Object Customization with Variable-Viewpoints in Text-to-Image Diffusion Models
arXiv
[Github] [Paper]
30-Oct-23

Compositional Inversion for Stable Diffusion Models
AAAI 2024
[Github] [Paper]
13-Dec-23

Visual Concept-Driven Image Generation with Text-to-Image Diffusion Model
arXiv
[Paper]
18-Feb-24

MIGC: Multi-Instance Generation Controller for Text-to-Image Synthesis
arXiv
[Paper]
27-Feb-24

Multi-Object Editing in Personalized Text-To-Image Diffusion Model Via Segmentation Guidance
arXiv
[Paper]
18-Mar-24

MC2: Multi-concept Guidance for Customized Multi-concept Generation
arXiv
[Paper]
12-Apr-24

MultiBooth: Towards Generating All Your Concepts in an Image from Text
arXiv
[Paper]
22-Apr-24

Personalized Style Generation

StyleDrop: Text-to-Image Synthesis of Any Style
NIPS 2023
[Github] [Paper]
1-Jun-23

StyleAdapter: A Single-Pass LoRA-Free Model for Stylized Image Generation
ICLR 2024
[Paper]
4-Sep-23

StyleBoost: A Study of Personalizing Text-to-Image Generation in Any Style using DreamBooth
ICTC 2023
[Paper]
13-Oct-23

ArtAdapter: Text-to-Image Style Transfer Using Multi-Level Style Encoder and Explicit Adaptation
arXiv
[Github] [Paper]
4-Dec-23

Style Aligned Image Generation via Shared Attention
CVPR 2024
[Github] [Paper]
4-Dec-23

Generative Active Learning for Image Synthesis Personalization
arXiv
[Github] [Paper]
22-Mar-24

Text-to-Image Synthesis for Any Artistic Styles: Advancements in Personalized Artistic Image Generation via Subdivision and Dual Binding
arXiv
[Paper]
8-Apr-24

Personalized Face Generation

Identity Encoder for Personalized Diffusion
CoRR 2023
[Paper]
14-Apr-23

FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention
CoRR 2023
[Github] [Paper]
21-May-23

Enhancing Detail Preservation for Customized Text-to-Image Generation: A Regularization-Free Approach
arXiv
[Paper]
23-May-23

Inserting Anybody in Diffusion Models via Celeb Basis
NIPS 2023
[Github] [Paper]
1-Jun-23

Face0: Instantaneously Conditioning a Text-to-Image Model on a Face
SIGGRAPH 2023
[Paper]
11-Jun-23

DreamIdentity: Improved Editability for Efficient Face-Identity Preserved Image Generation
arXiv
[Github] [Paper]
1-Jul-23

HyperDreamBooth: Hypernetworks for Fast Personalization of Text-to-Image Models
arXiv
[Github] [Paper]
13-Jul-23

Identity-Preserving Aging of Face Images via Latent Diffusion Models
IJCB 2023
[Github] [Paper]
17-Jul-23

Magicapture: High-Resolution Multi-Concept Portrait Customization
arXiv
[Github] [Paper]
13-Sep-23

High-Fidelity Person-Centric Subject-to-Image Synthesis
CVPR 2024
[Paper]
17-Nov-23

When StyleGAN Meets Stable Diffusion: A W+ Adapter for Personalized Image Generation
arXiv
[Github] [Paper]
29-Nov-23

Portrait Diffusion: Training-Free Face Stylization with Chain-of-Painting
arXiv
[Github] [Paper]
3-Dec-23

Retrieving Conditions from Reference Images for Diffusion Models
arXiv
[Paper]
5-Dec-23

FaceStudio: Put Your Face Everywhere in Seconds
arXiv
[Github] [Paper]
5-Dec-23

Personalized Face Inpainting with Diffusion Models by Parallel Visual Attention
WACV 2024
[Github] [Paper]
6-Dec-23

PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding
arXiv
[Github] [Paper]
7-Dec-23

DemoCaricature: Democratising Caricature Generation with a Rough Sketch
CVPR2024
[Github] [Paper]
7-Dec-23

Stellar: Systematic Evaluation of Human-Centric Personalized Text-to-Image Methods
CORR 2023
[Github] [Paper]
11-Dec-23

PortraitBooth: A Versatile Portrait Model for Fast Identity-Preserved Personalization
CVPR 2024
[Github] [Paper]
11-Dec-23

Concept-Centric Personalization with Large-Scale Diffusion Priors
arXiv
[Github] [Paper]
13-Dec-23

Cross Initialization for Personalized Text-to-Image Generation
arXiv
[Github] [Paper]
26-Dec-23

InstantID: Zero-Shot Identity-Preserving Generation in Seconds
arXiv
[Github] [Paper]
15-Jan-24

Face2Diffusion for Fast and Editable Face Personalization
arXiv
[Paper]
8-Mar-24

OMG: Occlusion-friendly Personalized Multi-concept Generation in Diffusion Models
arXiv
[Github] [Paper]
16-Mar-24

Infinite-ID: Identity-preserved Personalization via ID-semantics Decoupling Paradigm
arXiv
[Paper]
18-Mar-24

IDAdapter: Learning Mixed Features for Tuning-Free Personalization of Text-to-Image Models
arXiv
[Paper]
21-Mar-24

MoA: Mixture-of-Attention for Subject-Context Disentanglement in Personalized Image Generation
arXiv
[Paper]
17-Apr-24

ID-Aligner: Enhancing Identity-Preserving Text-to-Image Generation with Reward Feedback Learning
[Paper]
23-Apr-24

InstantFamily: Masked Attention for Zero-shot Multi-ID Image Generation
[Paper]
30-Apr-24

Personalization with Extra Condition

Training-free layout control with cross-attention guidance
WACV 2024
[Github] [Paper]
6-Apr-23

Prompt-Free Diffusion: Taking "Text" Out of Text-to-Image Diffusion Models
CVPR 2024
[Github] [Paper]
25-May-23

Uni-ControlNet: All-in-One Control to Text-to-Image Diffusion Models
NeurIPS 2023
[Github] [Paper]
25-May-23

PhotoSwap: Personalized Subject Swapping in Images
NIPS 2023
[Github] [Paper]
29-May-23

TryonDiffusion: A Tale of Two UNets
CVPR 2023
[Github] [Paper]
14-Jun-23

ViscoNet: Bridging and Harmonizing Visual and Textual Conditioning for ControlNet
arXiv
[Github] [Paper]
5-Dec-23

Context Diffusion: In-Context Aware Image Generation
arXiv
[Github] [Paper]
6-Dec-23

FreeControl: Training-Free Spatial Control of Any Text-to-Image Diffusion Model with Any Condition
CVPR 2024
[Github] [Paper]
12-Dec-23

A Two-Stage Personalized Virtual Try-On Framework with Shape Control and Texture Guidance
CoRR 2023
[Paper]
24-Dec-23

Tuning-Free Image Customization with Image and Text Guidance
arXiv
[Paper]
19-Mar-24

SWAPANYTHING: Enabling Arbitrary Object Swapping in Personalized Visual Editing
arXiv
[Paper]
8-Apr-24

Customizing Text-to-Image Diffusion with Camera Viewpoint Control
arXiv
[Paper]
18-Apr-24

Personalized Video Generation

Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generation
ICCV 2023
[Github] [Paper]
22-Dec-22

Structure and Content-Guided Video Synthesis with Diffusion Models
ICCV 2023
[Paper]
6-Feb-23

Make-A-Protagonist: Generic Video Editing with Visual and Textual Clues
arXiv
[Github] [Paper]
15-May-23

Animate-A-Story: Storytelling with Retrieval-Augmented Video Generation
arXiv
[Github] [Paper]
13-Jul-23

MotionDirector: Motion Customization of Text-to-Video Diffusion Models
arXiv
[Github] [Paper]
12-Oct-23

LAMP: Learn a Motion Pattern for Few-Shot-Based Video Generation
arXiv
[Github] [Paper]
16-Oct-23

VideoDreamer: Customized Multi-Subject Text-to-Video Generation with Disen-Mix Finetuning
arXiv
[Github] [Paper]
2-Nov-23

VideoAssembler: Identity-Consistent Video Generation with Reference Entities Using Diffusion Model
arXiv
[Github] [Paper]
29-Nov-23

VMC: Video Motion Customization using Temporal Attention Adaption for Text-to-Video Diffusion Models
arXiv
[Github] [Paper]
1-Dec-23

VideoBooth: Diffusion-Based Video Generation with Image Prompts
CVPR2024
[Github] [Paper]
1-Dec-23

StyleCrafter: Enhancing Stylized Text-to-Video Generation with Style Adapter
arXiv
[Github] [Paper]
1-Dec-23

SAVE: Protagonist Diversification with Structure Agnostic Video Editing
arXiv
[Github] [Paper]
5-Dec-23

Customizing Motion in Text-to-Video Diffusion Models
arXiv
[Github] [Paper]
7-Dec-23

DreamVideo: Composing Your Dream Videos with Customized Subject and Motion
arXiv
[Github] [Paper]
7-Dec-23

MotionCrafter: One-Shot Motion Customization of Diffusion Models
arXiv
[Github] [Paper]
8-Dec-23

DreaMoving: A Human Video Generation Framework Based on Diffusion Models
arXiv
[Github] [Paper]
8-Dec-23

CustomVideo: Customizing Text-to-Video Generation with Multiple Subjects
arXiv
[Github] [Paper]
18-Jan-24

Magic-Me: Identity-Specific Video Customized Diffusion
arXiv
[Paper]
14-Feb-24

Customize-A-Video: One-Shot Motion Customization of Text-to-Video Diffusion Models
arXiv
[Paper]
22-Feb-24

ID-Animator: Zero-Shot Identity-Preserving Human Video Generation
arXiv
[Paper]
23-Apr-24

Personalized 3D Generation

Magic3D: High-Resolution Text-to-3D Content Creation
CVPR 2023
[Github] [Paper]
18-Nov-22

DreamBooth3D: Subject-Driven Text-to-3D Generation
ICCV 2023
[Github] [Paper]
23-Mar-23

Text-Conditional Contextualized Avatars For Zero-Shot Personalization
arXiv
[Paper]
14-Apr-23

StyleAvatar3D: Leveraging Image-Text Diffusion Models for High-Fidelity 3D Avatar Generation
arXiv
[Paper]
30-May-23

AvatarBooth: High-Quality and Customizable 3D Human Avatar Generation
arXiv
[Github] [Paper]
16-Jun-23

MVDREAM: MULTI-VIEW DIFFUSION FOR 3D GENERATION
ICLR 2024
[Github] [Paper]
31-Aug-23

Chasing Consistency in Text-to-3D Generation from a Single Image
arXiv
[Paper]
7-Sep-23

Animate124: Animating One Image to 4D Dynamic Scene
arXiv
[Github] [Paper]
24-Nov-23

A Unified Approach for Text- and Image-guided 4D Scene Generation
arXiv
[Github] [Paper]
28-Nov-23

TextureDreamer: Image-guided Texture Synthesis through Geometry-aware Diffusion
arXiv
[Paper]
17-Jan-24

TIP-Editor: An Accurate 3D Editor Following Both Text-Prompts And Image-Prompts
arXiv
[Github] [Paper]
26-Jan-24

Others

Anti-DreamBooth: Protecting Users from Personalized Text-to-Image Synthesis
arXiv
[Github] [Paper]
27-Mar-23

Backdooring Textual Inversion for Concept Censorship
arXiv
[Github] [Paper]
21-Aug-23

Personalization as a Shortcut for Few-Shot Backdoor Attack against Text-to-Image Diffusion Models
arXiv
[Paper]
24-Mar-24

ReVersion: Diffusion-Based Relation Inversion from Images
arXiv
[Github] [Paper]
23-Mar-23

Learning Disentangled Identifiers for Action-Customized Text-to-Image Generation
arXiv
[Github] [Paper]
30-Nov-23

Inv-ReVersion: Enhanced Relation Inversion Based on Text-to-Image Diffusion Models
arXiv
[Paper]
15-Apr-24

Continual Diffusion: Continual Customization of Text-to-Image Diffusion with C-LoRA
arXiv
[Github] [Paper]
12-Apr-23

Text-Guided Vector Graphics Customization
SIGGRAPH 2023
[Github] [Paper]
21-Sep-23

Customizing 360-Degree Panoramas Through Text-to-Image Diffusion Models
WACV 2024
[Github] [Paper]
28-Oct-23