DI-engine

Conda Conda update PyPI - Python Version

Loc Comments

GitHub Org's stars GitHub commit activity

Updated on 2024.06.27 DI-engine-v0.5.2

Introduction to DI-engine

DI-engine is a generalized decision intelligence engine for PyTorch and JAX.

It provides python-first and asynchronous-native task and middleware abstractions, and modularly integrates several of the most important decision-making concepts: Env, Policy and Model. Based on the above mechanisms, DI-engine supports various deep reinforcement learning algorithms with superior performance, high efficiency, well-organized documentation and unittest:

Most basic DRL algorithms: such as DQN, Rainbow, PPO, TD3, SAC, R2D2, IMPALA
Multi-agent RL algorithms: such as QMIX, WQMIX, MAPPO, HAPPO, ACE
Imitation learning algorithms (BC/IRL/GAIL): such as GAIL, SQIL, Guided Cost Learning, Implicit BC
Offline RL algorithms: BCQ, CQL, TD3BC, Decision Transformer, EDAC, Diffuser, Decision Diffuser, SO2
Model-based RL algorithms: SVG, STEVE, MBPO, DDPPO, DreamerV3
Exploration algorithms: HER, RND, ICM, NGU
LLM + RL Algorithms: PPO-max, DPO, PromptPG
Other algorithms: such as PER, PLR, PCGrad
MCTS + RL algorithms: AlphaZero, MuZero, please refer to LightZero
Generative Model + RL algorithms: Diffusion-QL, QGPO, SRPO, please refer to GenerativeRL

DI-engine aims to standardize different Decision Intelligence environments and applications, supporting both academic research and prototype applications. Various training pipelines and customized decision AI applications are also supported:

<details open> <summary>(Click to Collapse)</summary>

Traditional academic environments
- DI-zoo: various decision intelligence demonstrations and benchmark environments with DI-engine.
Tutorial courses
- PPOxFamily: PPO x Family DRL Tutorial Course
Real world decision AI applications
- DI-star: Decision AI in StarCraftII
- PsyDI: Towards a Multi-Modal and Interactive Chatbot for Psychological Assessments
- DI-drive: Auto-driving platform
- DI-sheep: Decision AI in 3 Tiles Game
- DI-smartcross: Decision AI in Traffic Light Control
- DI-bioseq: Decision AI in Biological Sequence Prediction and Searching
- DI-1024: Deep Reinforcement Learning + 1024 Game
Research paper
- InterFuser: [CoRL 2022] Safety-Enhanced Autonomous Driving Using Interpretable Sensor Fusion Transformer
- ACE: [AAAI 2023] ACE: Cooperative Multi-agent Q-learning with Bidirectional Action-Dependency
- GoBigger: [ICLR 2023] Multi-Agent Decision Intelligence Environment
- DOS: [CVPR 2023] ReasonNet: End-to-End Driving with Temporal and Global Reasoning
- LightZero: [NeurIPS 2023 Spotlight] A lightweight and efficient MCTS/AlphaZero/MuZero algorithm toolkit
- SO2: [AAAI 2024] A Perspective of Q-value Estimation on Offline-to-Online Reinforcement Learning
- LMDrive: [CVPR 2024] LMDrive: Closed-Loop End-to-End Driving with Large Language Models
- SmartRefine: [CVPR 2024] SmartRefine: A Scenario-Adaptive Refinement Framework for Efficient Motion Prediction
- ReZero: Boosting MCTS-based Algorithms by Backward-view and Entire-buffer Reanalyze
- UniZero: Generalized and Efficient Planning with Scalable Latent World Models
Docs and Tutorials
- DI-engine-docs: Tutorials, best practice and the API reference.
- awesome-model-based-RL: A curated list of awesome Model-Based RL resources
- awesome-exploration-RL: A curated list of awesome exploration RL resources
- awesome-decision-transformer: A curated list of Decision Transformer resources
- awesome-RLHF: A curated list of reinforcement learning with human feedback resources
- awesome-multi-modal-reinforcement-learning: A curated list of Multi-Modal Reinforcement Learning resources
- awesome-diffusion-model-in-rl: A curated list of Diffusion Model in RL resources
- awesome-ui-agents: A curated list of of awesome UI agents resources, encompassing Web, App, OS, and beyond
- awesome-AI-based-protein-design: a collection of research papers for AI-based protein design
- awesome-end-to-end-autonomous-driving: A curated list of awesome End-to-End Autonomous Driving resources
- awesome-driving-behavior-prediction: A collection of research papers for Driving Behavior Prediction
</details>

On the low-level end, DI-engine comes with a set of highly re-usable modules, including RL optimization functions, PyTorch utilities and auxiliary tools.

BTW, DI-engine also has some special system optimization and design for efficient and robust large-scale RL training:

<details close> <summary>(Click for Details)</summary>