ReinforcementLearning.jl

<div align="center"> <img src="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/raw/main/docs/src/assets/logo.svg?sanitize=true" width="320px"> <a href="https://wiki.c2.com/?MakeItWorkMakeItRightMakeItFast">"Make It Work Make It Right Make It Fast"</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/actions?query=workflow%3ACI"><img src="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/workflows/CI/badge.svg"></a> <a href="https://juliahub.com/ui/Packages/ReinforcementLearning/6l2TO"><img src="https://juliahub.com/docs/ReinforcementLearning/pkgeval.svg"></a> <a href="https://juliahub.com/ui/Packages/ReinforcementLearning/6l2TO"><img src="https://juliahub.com/docs/ReinforcementLearning/version.svg"></a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/blob/main/LICENSE.md"><img src="http://img.shields.io/badge/license-MIT-brightgreen.svg?style=flat"></a> <a href="https://julialang.org/slack/"><img src="https://img.shields.io/badge/Chat%20on%20Slack-%23reinforcement--learnin-ff69b4"></a> <a href="https://github.com/SciML/ColPrac"><img src="https://img.shields.io/badge/ColPrac-Contributor's%20Guide-blueviolet"></a> </div>

ReinforcementLearning.jl, as the name says, is a package for reinforcement learning research in Julia.

Our design principles are:

Reusability and extensibility: Provide elaborately designed components and interfaces to help users implement new algorithms.
Easy experimentation: Make it easy for new users to run benchmark experiments, compare different algorithms, evaluate and diagnose agents.
Reproducibility: Facilitate reproducibility from traditional tabular methods to modern deep reinforcement learning algorithms.

🏹 Get Started

julia> ] add ReinforcementLearning

julia> using ReinforcementLearning

julia> run(
           RandomPolicy(),
           CartPoleEnv(),
           StopAfterNSteps(1_000),
           TotalRewardPerEpisode()
       )

The above simple example demonstrates four core components in a general reinforcement learning experiment:

Policy. The RandomPolicy is the simplest instance of AbstractPolicy. It generates a random action at each step.
Environment. The CartPoleEnv is a typical AbstractEnv to test reinforcement learning algorithms.
Stop Condition. The StopAfterNSteps(1_000) is to inform that our experiment should stop after 1_000 steps.
Hook. The TotalRewardPerEpisode structure is one of the most common AbstractHooks. It is used to collect the total reward of each episode in an experiment.

Check out the tutorial page to learn how these four components are assembled together to solve many interesting problems. We also write blog occasionally to explain the implementation details of some algorithms. Among them, the most recommended one is An Introduction to ReinforcementLearning.jl, which explains the design idea of this package.

🙋 Why ReinforcementLearning.jl?

🚀 Fast Speed

[TODO:]

🧰 Feature Rich

[TODO:]

-->

🌲 Project Structure

ReinforcementLearning.jl itself is just a wrapper around several other subpackages. The relationship between them is depicted below:

<pre>+-----------------------------------------------------------------------------------+ | | | <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl">ReinforcementLearning.jl</a> | | | | +------------------------------+ | | | <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/tree/main/src/ReinforcementLearningBase">ReinforcementLearningBase.jl</a> | | | +----|-------------------------+ | | | | | | +--------------------------------------+ | | +---->+ <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/tree/main/src/ReinforcementLearningEnvironments">ReinforcementLearningEnvironments.jl</a> | | | | +--------------------------------------+ | | | | | | +------------------------------+ | | +---->+ <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/tree/main/src/ReinforcementLearningCore">ReinforcementLearningCore.jl</a> | | | +----|-------------------------+ | | | | | | +-----------------------------+ | | +---->+ <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/tree/main/src/ReinforcementLearningZoo">ReinforcementLearningZoo.jl</a> | | | +----|------------------------+ | | | | | | +-------------------------------------+ | | +---->+ <a href="https://github.com/JuliaReinforcementLearning/DistributedReinforcementLearning.jl">DistributedReinforcementLearning.jl</a> | | | +-------------------------------------+ | | | +------|----------------------------------------------------------------------------+ | | +-------------------------------------+ +---->+ <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/tree/main/src/ReinforcementLearningExperiments">ReinforcementLearningExperiments.jl</a> | | +-------------------------------------+ | | +----------------------------------------+ +---->+ <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearningAnIntroduction.jl">ReinforcementLearningAnIntroduction.jl</a> | +----------------------------------------+ </pre>

✋ Getting Help

Are you looking for help with ReinforcementLearning.jl? Here are ways to find help:

Read the online documentation! Most likely the answer is already provided in an example or in the API documents. Search using the search bar in the upper left.

Chat with us in Julia Slack in the #reinforcement-learnin channel.
Post a question in the Julia discourse forum in the category "Machine Learning" and use "reinforcement-learning" as a tag.
For issues with unexpected behavior or defects in ReinforcementLearning.jl, then please open an issue on the ReinforcementLearning GitHub page with a minimal working example and steps to reproduce.

🖖 Supporting

ReinforcementLearning.jl is a MIT licensed open source project with its ongoing development made possible by many contributors in their spare time. However, modern reinforcement learning research requires huge computing resource, which is unaffordable for individual contributors. So if you or your organization could provide the computing resource in some degree and would like to cooperate in some way, please contact us!

This package is written in pure Julia. Please consider supporting the JuliaLang org if you find this package useful. ❤

✍️ Citing

If you use ReinforcementLearning.jl in a scientific publication, we would appreciate references to the CITATION.bib.

✨ Contributors

Thanks goes to these wonderful people (emoji key):

<table> <tr> <td align="center"><a href="http://lcn.epfl.ch/~brea/"><img src="https://avatars.githubusercontent.com/u/12857162?v=4?s=100" width="100px;" alt=""/> jbrea</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=jbrea" title="Code">💻</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=jbrea" title="Documentation">📖</a> <a href="#maintenance-jbrea" title="Maintenance">🚧</a></td> <td align="center"><a href="https://tianjun.me/"><img src="https://avatars.githubusercontent.com/u/5612003?v=4?s=100" width="100px;" alt=""/> Jun Tian</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=findmyway" title="Code">💻</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=findmyway" title="Documentation">📖</a> <a href="#maintenance-findmyway" title="Maintenance">🚧</a> <a href="#ideas-findmyway" title="Ideas, Planning, & Feedback">🤔</a></td> <td align="center"><a href="https://github.com/amanbh"><img src="https://avatars.githubusercontent.com/u/911313?v=4?s=100" width="100px;" alt=""/> Aman Bhatia</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=amanbh" title="Documentation">📖</a></td> <td align="center"><a href="https://avt.im/"><img src="https://avatars.githubusercontent.com/u/4722472?v=4?s=100" width="100px;" alt=""/> Alexander Terenin</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=aterenin" title="Code">💻</a></td> <td align="center"><a href="https://github.com/Sid-Bhatia-0"><img src="https://avatars.githubusercontent.com/u/32610387?v=4?s=100" width="100px;" alt=""/> Sid-Bhatia-0</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=Sid-Bhatia-0" title="Code">💻</a></td> <td align="center"><a href="https://github.com/norci"><img src="https://avatars.githubusercontent.com/u/2986988?v=4?s=100" width="100px;" alt=""/> norci</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=norci" title="Code">💻</a> <a href="#maintenance-norci" title="Maintenance">🚧</a></td> <td align="center"><a href="https://github.com/sriram13m"><img src="https://avatars.githubusercontent.com/u/28051516?v=4?s=100" width="100px;" alt=""/> Sriram</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=sriram13m" title="Code">💻</a></td> </tr> <tr> <td align="center"><a href="https://github.com/gpavanb1"><img src="https://avatars.githubusercontent.com/u/50511632?v=4?s=100" width="100px;" alt=""/> Pavan B Govindaraju</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=gpavanb1" title="Code">💻</a></td> <td align="center"><a href="https://github.com/AlexLewandowski"><img src="https://avatars.githubusercontent.com/u/15149466?v=4?s=100" width="100px;" alt=""/> Alex Lewandowski</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=AlexLewandowski" title="Code">💻</a></td> <td align="center"><a href="https://github.com/RajGhugare19"><img src="https://avatars.githubusercontent.com/u/62653460?v=4?s=100" width="100px;" alt=""/> Raj Ghugare</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=RajGhugare19" title="Code">💻</a></td> <td align="center"><a href="https://github.com/rbange"><img src="https://avatars.githubusercontent.com/u/13252574?v=4?s=100" width="100px;" alt=""/> Roman Bange</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=rbange" title="Code">💻</a></td> <td align="center"><a href="https://github.com/felixchalumeau"><img src="https://avatars.githubusercontent.com/u/49362657?v=4?s=100" width="100px;" alt=""/> Felix Chalumeau</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=felixchalumeau" title="Code">💻</a></td> <td align="center"><a href="https://github.com/rishabhvarshney14"><img src="https://avatars.githubusercontent.com/u/53183977?v=4?s=100" width="100px;" alt=""/> Rishabh Varshney</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=rishabhvarshney14" title="Code">💻</a></td> <td align="center"><a href="https://github.com/zsunberg"><img src="https://avatars.githubusercontent.com/u/4240491?v=4?s=100" width="100px;" alt=""/> Zachary Sunberg</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=zsunberg" title="Code">💻</a> <a href="https://github.com/JuliaReinforcementLearning/ReinforcementLearning.jl/commits?author=zsunberg" title="Documentation">📖</a> <a href="#maintenance-zsunberg" title="Maintenance">🚧</a> <a href="#ideas-zsunberg" title="Ideas, Planning, & Feedback">🤔</a></td> </tr> <tr> <td align="center"><a href="https://www.cs.cmu.edu/~jlaurent/"><img