Diffusion-Literature-for-Robotics

"Creating noise from data is easy; creating data from noise is generative modeling."

Yang Song in "Score-Based Generative Modeling through Stochastic Differential Equations" Song et al., 2020

This repository offers a brief summary of essential papers and blogs on diffusion models, alongside a categorized collection of robotics diffusion papers and useful code repositories for starting your own diffusion robotics project.

Table of Contents

1. Learning about Diffusion models

2. Diffusion in Robotics

   2.1 Imitation Learning and Policy Learning

   2.2 Video Diffusion in Robotics

   2.3 Online RL

   2.4 Offline RL

   2.5 Inverse RL

   2.6 World Models

   2.7 Task and Motion Planning

   2.8 Tactile Sensing & Pose Estimation

   2.9 Robot Design and Development

3. Code Implementations

4. Diffusion History

Learning about Diffusion models

<a name="Learning-about-Diffusion-models"></a> While there exist many tutorials for Diffusion models, below you can find an overview of some of the best introduction blog posts and video:

• What are Diffusion Models?: an introduction video, which introduces the general idea of diffusion models and some high-level math about how the model works

• Diffusion Models | Paper Explanation | Math Explained another great video tutorial explaining the math and notation of diffusion models in detail with visual aid

• Generative Modeling by Estimating Gradients of the Data Distribution: blog post from the one of the most influential authors in this area, which introduces diffusion models from the score-based perspective

• What are Diffusion Models: a in-depth blog post about the theory of diffusion models with a general summary on how diffusion model improved over time

• Understanding Diffusion Models: an in-depth explanation paper, which explains the diffusion models from both perspectives with detailed derivations

If you don't like reading blog posts and prefer the original papers, below you can find a list with the most important diffusion theory papers:

• Sohl-Dickstein, Jascha, et al. "Deep unsupervised learning using nonequilibrium thermodynamics." International Conference on Machine Learning. PMLR, 2015.

• Ho, Jonathan, et al. "Denoising diffusion probabilistic models." Advances in Neural Information Processing Systems 33 (2020): 6840-6851.

• Song, Yang, et al. "Score-Based Generative Modeling through Stochastic Differential Equations." International Conference on Learning Representations. 2020.

• Ho, Jonathan, and Tim Salimans. "Classifier-Free Diffusion Guidance." NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications. 2021.

• Karras, Tero, et al. "Elucidating the Design Space of Diffusion-Based Generative Models." Advances in Neural Information Processing Systems 35 (2022)

A general list with all published diffusion papers can be found here: Whats the score?

Diffusion in Robotics

<a name="Diffusion-in-Robotics"></a> Since the modern diffusion models have been around for only 3 years, the literature about diffusion models in the context of robotics is still small, but growing rapidly. Below you can find most robotics diffusion papers, which have been published at conferences or uploaded to Arxiv so far:

Imitation Learning and Policy Learning

<a name="Imitation-Learning-and-Policy-Learning"></a>

• Zhou, Hongyi, et al. "Variational Distillation of Diffusion Policies into Mixture of Experts." arXiv preprint arXiv:2406.12538 (2024).

• Jia, Xiaogang, et al. "MaIL: Improving Imitation Learning with Mamba." arXiv preprint arXiv:2406.08234 (2024).

• Hao, Ce, et al. "Language-Guided Manipulation with Diffusion Policies and Constrained Inpainting." arXiv preprint arXiv:2406.09767 (2024).

• Shridhar, Mohit, Yat Long Lo, and Stephen James. "Generative Image as Action Models." arXiv preprint arXiv:2407.07875 (2024).

• Høeg, Sigmund H., and Lars Tingelstad. "TEDi Policy: Temporally Entangled Diffusion for Robotic Control." arXiv preprint arXiv:2406.04806 (2024).

• Vosylius, Vitalis, et al. "Render and Diffuse: Aligning Image and Action Spaces for Diffusion-based Behaviour Cloning." Proceedings of Robotics: Science and Systems (RSS) 2024.

• Prasad, Aaditya, et al. "Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation." Proceedings of Robotics: Science and Systems (RSS) 2024.

• Bharadhwaj, Homanga, et al. "Track2Act: Predicting Point Tracks from Internet Videos enables Diverse Zero-shot Robot Manipulation." ECCV 2024

• Reuss, Moritz, et al. "Multimodal Diffusion Transformer: Learning Versatile Behavior from Multimodal Goals." Proceedings of Robotics: Science and Systems (RSS) 2024.

• Gupta, Gunshi, et al. "Pre-trained Text-to-Image Diffusion Models Are Versatile Representation Learners for Control." First Workshop on Vision-Language Models for Navigation and Manipulation at ICRA 2024 (2024).

• Ke, Tsung-Wei, Nikolaos Gkanatsios, and Katerina Fragkiadaki. "3D Diffuser Actor: Policy Diffusion with 3D Scene Representations." arXiv preprint arXiv:2402.10885 (2024).

• Ze, Yanjie, et al. "3D Diffusion Policy." Proceedings of Robotics: Science and Systems (RSS) 2024.

• Ma, Xiao, et al. "Hierarchical Diffusion Policy for Kinematics-Aware Multi-Task Robotic Manipulation." arXiv preprint arXiv:2403.03890 (2024).

• Yan, Ge, Yueh-Hua Wu, and Xiaolong Wang. "DNAct: Diffusion Guided Multi-Task 3D Policy Learning." arXiv preprint arXiv:2403.04115 (2024).

• Zhang, Xiaoyu, et al. "Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learning." arXiv preprint arXiv:2402.17768 (2024).

• Chen, Kaiqi, et al. "Behavioral Refinement via Interpolant-based Policy Diffusion." arXiv preprint arXiv:2402.16075 (2024).

• Wang, Bingzheng, et al. "DiffAIL: Diffusion Adversarial Imitation Learning." arXiv preprint arXiv:2312.06348 (2023).

• Scheikl, Paul Maria, et al. "Movement Primitive Diffusion: Learning Gentle Robotic Manipulation of Deformable Objects." arXiv preprint arXiv:2312.10008 (2023).

• Octo Model Team et al. Octo: An Open-Source Generalist Robot Policy

• Black, Kevin, et al. "ZERO-SHOT ROBOTIC MANIPULATION WITH PRETRAINED IMAGE-EDITING DIFFUSION MODELS." arXiv preprint arXiv:2310.10639 (2023).

• Reuss, Moritz, and Rudolf Lioutikov. "Multimodal Diffusion Transformer for Learning from Play." 2nd Workshop on Language and Robot Learning: Language as Grounding. 2023.

• Sridhar, Ajay, et al. "NoMaD: Goal Masked Diffusion Policies for Navigation and Exploration." arXiv preprint arXiv:2310.07896 (2023).

• Zhou, Xian, et al. "Unifying Diffusion Models with Action Detection Transformers for Multi-task Robotic Manipulation." Conference on Robot Learning. PMLR, 2023.

• Ze, Yanjie, et al. "Multi-task real robot learning with generalizable neural feature fields." 7th Annual Conference on Robot Learning. 2023.

• Mishra, Utkarsh Aashu, et al. "Generative Skill Chaining: Long-Horizon Skill Planning with Diffusion Models." Conference on Robot Learning. PMLR, 2023.

• Chen, Lili, et al. "PlayFusion: Skill Acquisition via Diffusion from Language-Annotated Play." Conference on Robot Learning. PMLR, 2023.

• Ha, Huy, Pete Florence, and Shuran Song. "Scaling Up and Distilling Down: Language-Guided Robot Skill Acquisition." Conference on Robot Learning. PMLR, 2023.

• Xu, Mengda, et al. "XSkill: Cross Embodiment Skill Discovery." Conference on Robot Learning. PMLR, 2023.

• Li, Xiang, et al. "Crossway Diffusion: Improving Diffusion-based Visuomotor Policy via Self-supervised Learning." arXiv preprint arXiv:2307.01849 (2023).

• Ng, Eley, Ziang Liu, and Monroe Kennedy III. "Diffusion Co-Policy for Synergistic Human-Robot Collaborative Tasks." arXiv preprint arXiv:2305.12171 (2023).

• Chi, Cheng, et al. "Diffusion Policy: Visuomotor Policy Learning via Action Diffusion." Proceedings of Robotics: Science and Systems (RSS) 2023.

• Reuss, Moritz, et al. "Goal-Conditioned Imitation Learning using Score-based Diffusion Policies." Proceedings of Robotics: Science and Systems (RSS) 2023.

• Yoneda, Takuma, et al. "To the Noise and Back: Diffusion for Shared Autonomy." Proceedings of Robotics: Science and Systems (RSS) 2023.

• Jiang, Chiyu, et al. "MotionDiffuser: Controllable Multi-Agent Motion Prediction Using Diffusion." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2023.

• Kapelyukh, Ivan, et al. "DALL-E-Bot: Introducing Web-Scale Diffusion Models to Robotics." IEEE Robotics and Automation Letters (RA-L) 2023.

• Pearce, Tim, et al. "Imitating human behaviour with diffusion models." " International Conference on Learning Representations. 2023.

• Yu, Tianhe, et al. "Scaling robot learning with semantically imagined experience." arXiv preprint arXiv:2302.11550 (2023).

Video Diffusion in Robotics

<a name="Video-Diffusion"></a>

The ability of Diffusion models to generate realistic videos over a long horizon has enabled new applications in the context of robotics.

• Wang, Boyang, et al. "This&That: Language-Gesture Controlled Video Generation for Robot Planning." arXiv:2407.05530 (2024).

• Chen, Boyuan, et al. "Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion." arXiv preprint arXiv:2407.01392 (2024).

• Zhou, Siyuan, et al. "RoboDreamer: Learning Compositional World Models for Robot Imagination." arXiv preprint arXiv:2404.12377 (2024).

• McCarthy, Robert, et al. "Towards Generalist Robot Learning from Internet Video: A Survey." arXiv preprint arXiv:2404.19664 (2024).

• He, Haoran, et al. "Large-Scale Actionless Video Pre-Training via Discrete Diffusion for Efficient Policy Learning." arXiv preprint arXiv:2402.14407 (2024).

• Liang, Zhixuan, et al. "SkillDiffuser: Interpretable Hierarchical Planning via Skill Abstractions in Diffusion-Based Task Execution." arXiv preprint arXiv:2312.11598 (2023).

• Huang, Tao, et al. "Diffusion Reward: Learning Rewards via Conditional Video Diffusion." arXiv preprint arXiv:2312.14134 (2023).

• Du, Yilun, et al. "Video Language Planning." arXiv preprint arXiv:2310.10625 (2023).

• Yang, Mengjiao, et al. "Learning Interactive Real-World Simulators." arXiv preprint arXiv:2310.06114 (2023).

• Ko, Po-Chen, et al. "Learning to Act from Actionless Videos through Dense Correspondences." arXiv preprint arXiv:2310.08576 (2023).

• Ajay, Anurag, et al. "Compositional Foundation Models for Hierarchical Planning." Advances in Neural Information Processing Systems 37 (2023)

• Dai, Yilun, et al. "Learning Universal Policies via Text-Guided Video Generation." Advances in Neural Information Processing Systems 37 (2023)

Online