A paper list about diffusion models for natural language processing.
Update News:
- Nov. 9, 2022, Add 3 pre-print papers.
- Oct. 18, 2022, Add papers that attempt to apply diffusion models for NLP from scratch.
- Diffusion-LM Improves Controllable Text Generation. Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, Percy Liang, Tatsunori B. Hashimoto. In NeuralPS 2022. [pdf] [code]
- DiffuSeq: Sequence to Sequence Text Generation with Diffusion Models. Shansan Gong, Mukai Li, Jiangtao Feng, Zhiyong Wu, LingPeng Kong. ICLR 2023. [pdf] [code]
- Latent Diffusion Energy-Based Model for Interpretable Text Modeling. Peiyu Yu, Sirui Xie, Xiaojian Ma, Baoxiong Jia, Bo Pang, Ruiqi Gao, Yixin Zhu, Song-Chun Zhu, Ying Nian Wu. [pdf] [code]
- Analog Bits: Generating Discrete Data using Diffusion Models with Self-Conditioning. Ting Chen, Ruixiang Zhang, Geoffrey Hinton. [pdf] [code]
- Structured Denoising Diffusion Models in Discrete State-Spaces. Jacob Austin, Daniel D. Johnson, Jonathan Ho, Daniel Tarlow, Rianne van den Berg. [pdf]
- Composable Text Controls in Latent Space with ODEs. Guangyi Liu, Zeyu Feng, Yuan Gao, Zichao Yang, Xiaodan Liang, Junwei Bao, Xiaodong He, Shuguang Cui, Zhen Li, Zhiting Hu. [pdf]
- DiffusER: Discrete Diffusion via Edit-based Reconstruction. Machel Reid, Vincent J. Hellendoorn, Graham Neubig. ICLR 2023. [pdf]
- Self-conditioned Embedding Diffusion for Text Generation. Robin Strudel, Corentin Tallec, Florent Altché, Yilun Du, Yaroslav Ganin, Arthur Mensch, Will Grathwohl, Nikolay Savinov, Sander Dieleman, Laurent Sifre, Rémi Leblond. [pdf]
- DiffusionBERT: Improving Generative Masked Language Models with Diffusion Models. Zhengfu He, Tianxiang Sun, Qiong Tang, Kuanning Wang, Xuanjing Huang and Xipeng Qiu. ACL 2023 [pdf]
- NUWA-XL: Diffusion over Diffusion for eXtremely Long Video Generation. Shengming Yin, Chenfei Wu, Huan Yang, Jianfeng Wang, Xiaodong Wang, Minheng Ni, Zhengyuan Yang, Linjie Li, Shuguang Liu, Fan Yang, Jianlong Fu, Ming Gong, Lijuan Wang, Zicheng Liu, Houqiang Li and Nan Duan. ACL 2023 [pdf]
- DiffusEmp: A Diffusion Model-Based Framework with Multi-Grained Control for Empathetic Response Generation. Guanqun Bi, Lei Shen, Yanan Cao, Meng Chen, Yuqiang Xie, Zheng Lin and Xiaodong He. ACL 2023 [pdf]
- DiffusionNER: Boundary Diffusion for Named Entity Recognition. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu and Yueting Zhuang. ACL 2023 [pdf]
- What the DAAM: Interpreting Stable Diffusion Using Cross Attention. Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin and Ferhan Ture. ACL 2023 [pdf]
- SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control. Xiaochuang Han, Sachin Kumar and Yulia Tsvetkov. ACL 2023 [pdf]
- DIFFUSION MODELS: A COMPREHENSIVE SURVEY OFMETHODS AND APPLICATIONS. Ling Yang, Zhilong Zhang, Shenda Hong, Wentao Zhang, Bin Cui. [pdf]