POMDPs

Linux	Mac OS X	Windows

这个软件包提供了一个用于处理马尔可夫决策过程(MDPs)和部分可观察马尔可夫决策过程(POMDPs)的核心接口。 POMDPTools包作为POMDPs.jl接口的"标准库"，提供了常用组件的实现，如策略、信念更新器、分布和模拟器。

我们的目标是为以下方面提供一个通用的编程词汇：

1. 将问题表述为MDPs和POMDPs。
2. 编写求解器软件。
3. 高效运行模拟。

POMDPs.jl与其他生态系统集成：

• 可以通过quickpomdps包或直接通过pyjulia使用Python来定义和求解MDPs和POMDPs。
• POMDPTools提供了与CommonRLInterface的双向集成，因此也与JuliaReinforcementLearning包集成。
• SymbolicMDPs包提供了一个使用PDDL模型的接口。

欲了解详细介绍，请查看我们的Julia Academy课程！如需帮助，请在GitHub讨论区发帖。我们欢迎任何人的贡献！有关贡献的信息，请参阅CONTRIBUTING.md。

安装

可以使用Julia的包管理器安装POMDPs.jl和相关的求解器包。例如，要安装POMDPs.jl和QMDP求解器包，在Julia REPL中输入以下内容：

using Pkg; Pkg.add("POMDPs"); Pkg.add("QMDP")

快速入门

要使用QMDP求解器创建的策略运行经典的Tiger POMDP的简单模拟，你可以使用以下代码（注意，POMDPs.jl不限于具有明确定义分布的离散问题）：

using POMDPs, QuickPOMDPs, POMDPTools, QMDP

m = QuickPOMDP(
    states = ["left", "right"],
    actions = ["left", "right", "listen"],
    observations = ["left", "right"],
    initialstate = Uniform(["left", "right"]),
    discount = 0.95,

    transition = function (s, a)
        if a == "listen"
            return Deterministic(s) # 老虎仍在同一扇门后
        else # 打开一扇门
            return Uniform(["left", "right"]) # 重置
        end
    end,

    observation = function (s, a, sp)
        if a == "listen"
            if sp == "left"
                return SparseCat(["left", "right"], [0.85, 0.15]) # 稀疏分类分布
            else
                return SparseCat(["right", "left"], [0.85, 0.15])
            end
        else
            return Uniform(["left", "right"])
        end
    end,

    reward = function (s, a)
        if a == "listen"
            return -1.0
        elseif s == a # 发现了老虎
            return -100.0
        else # 逃脱了老虎
            return 10.0
        end
    end
)

solver = QMDPSolver()
policy = solve(solver, m)

rsum = 0.0 for (s,b,a,o,r) in stepthrough(m, policy, "s,b,a,o,r", max_steps=10) println("s: $s, b: $([s=>pdf(b,s) for s in states(m)]), a: $a, o: $o") global rsum += r end println("未折扣奖励为 $rsum。")

如需更多示例和可视化示例，请参考文档中的[示例](https://JuliaPOMDP.github.io/POMDPs.jl/latest/examples)和[POMDPs.jl问题展示](https://JuliaPOMDP.github.io/POMDPs.jl/latest/gallery)部分。

## 文档和教程

除了上述提到的[Julia Academy课程](https://juliaacademy.com/p/decision-making-under-uncertainty-with-pomdps-jl)，详细文档和示例可以在[这里](http://juliapomdp.github.io/POMDPs.jl/stable/)找到。

[![文档](https://yellow-cdn.veclightyear.com/0a4dffa0/78b6c568-9ea8-4834-b5cb-781fa9f635f7.svg)](https://JuliaPOMDP.github.io/POMDPs.jl/stable)
[![文档](https://yellow-cdn.veclightyear.com/0a4dffa0/1b286f12-568a-4366-b7b6-d971631e97b0.svg)](https://JuliaPOMDP.github.io/POMDPs.jl/latest)

## 支持的包

许多包使用POMDPs.jl接口，包括MDP和POMDP求解器、支持工具以及POMDPs.jl接口的扩展。POMDPs.jl和JuliaPOMDP项目中的所有包在Linux上都得到完全支持。OSX和Windows支持所有原生求解器*，大多数非原生求解器应该可以工作，但可能需要额外配置。

#### 工具：

POMDPs.jl本身只包含用于问题定义通信的核心接口；这些包包含常用组件的实现：

|  **`包`**   |  **`构建`** | **`覆盖率`** |
|-------------------|----------------------|------------------|
| [POMDPTools](https://github.com/JuliaPOMDP/POMDPs.jl/tree/master/lib/POMDPTools) (托管在此仓库中) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/1b4016bf-0622-435a-9ed5-7cdc885a46a4.svg)](https://github.com/JuliaPOMDP/POMDPs.jl/actions/workflows/CI.yml) | |
| [ParticleFilters](https://github.com/JuliaPOMDP/ParticleFilters.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/a723e780-d6f6-49f9-ad25-78a66a0fd626.svg)](https://github.com/JuliaPOMDP/ParticleFilters.jl) | [![codecov.io](https://yellow-cdn.veclightyear.com/0a4dffa0/d57f2893-b495-4701-9442-206b2807cfa3.svg?)](http://codecov.io/github/JuliaPOMDP/ParticleFilters.jl?) |

#### 已实现的模型：

许多模型已使用POMDPs.jl接口为各种项目实现。此列表包含一些常用模型：
|  **`软件包`**   |  **`构建`** | **`覆盖率`** |
|-------------------|----------------------|------------------|
| [POMDPModels](https://github.com/JuliaPOMDP/POMDPModels.jl) |[![CI](https://yellow-cdn.veclightyear.com/0a4dffa0/6d470453-0a3b-44c6-9cb7-c8be1642e4d7.svg)](https://github.com/JuliaPOMDP/POMDPModels.jl/actions/workflows/CI.yml)| [![codecov](https://yellow-cdn.veclightyear.com/0a4dffa0/51a8d776-7178-4169-9d26-24e5c3490342.svg?token=xPLiTP3IVt)](https://codecov.io/gh/JuliaPOMDP/POMDPModels.jl) |
| [LaserTag](https://github.com/JuliaPOMDP/LaserTag.jl) | [![CI](https://yellow-cdn.veclightyear.com/0a4dffa0/84f838be-01d9-4405-a3fd-152d4597aed4.svg)](https://github.com/JuliaPOMDP/LaserTag.jl/actions/workflows/CI.yml) | [![codecov](https://yellow-cdn.veclightyear.com/0a4dffa0/19ae8653-eb4c-47f9-8734-e8a5447ec6d7.svg?token=Jo3cSe9K15)](https://codecov.io/gh/JuliaPOMDP/LaserTag.jl) |
| [RockSample](https://github.com/JuliaPOMDP/RockSample.jl) | [![CI](https://yellow-cdn.veclightyear.com/0a4dffa0/34f5f569-2daa-4290-afba-d5d756504472.svg)](https://github.com/JuliaPOMDP/RockSample.jl/actions/workflows/CI.yml) | [![codecov](https://yellow-cdn.veclightyear.com/0a4dffa0/db036f36-666f-4b5c-b9bb-bb410f45673c.svg?token=EfDZPMisVB)](https://codecov.io/github/JuliaPOMDP/RockSample.jl) |
| [TagPOMDPProblem](https://github.com/JuliaPOMDP/TagPOMDPProblem.jl) | [![CI](https://yellow-cdn.veclightyear.com/0a4dffa0/1fd8f1c2-afaa-4e89-a030-0a17745a2017.svg)](https://github.com/JuliaPOMDP/TagPOMDPProblem.jl/actions/workflows/CI.yml) | [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/ffaf3069-8af4-4806-a5be-1a3f54155468.svg?token=UNYWMYUBDL)](https://codecov.io/gh/JuliaPOMDP/TagPOMDPProblem.jl) |
| [DroneSurveillance](https://github.com/JuliaPOMDP/DroneSurveillance.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/4449cc29-4505-4959-a960-25fe2112cfe2.svg)](https://github.com/JuliaPOMDP/DroneSurveillance.jl/actions) | [![codecov](https://yellow-cdn.veclightyear.com/0a4dffa0/79e7a1f0-26bc-42b2-90d3-603a09d4d1b2.svg)](https://codecov.io/gh/juliapomdp/DroneSurveillance.jl) |
| [ContinuumWorld](https://github.com/JuliaPOMDP/ContinuumWorld.jl) | [![CI](https://yellow-cdn.veclightyear.com/0a4dffa0/7ee46251-0695-47ac-859e-6a0e31f0463d.svg)](https://github.com/JuliaPOMDP/ContinuumWorld.jl/actions/workflows/CI.yml) | [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/82ef4a2f-f039-4008-8fc5-520562b2d00e.svg?branch=master&service=github)](https://coveralls.io/github/JuliaPOMDP/ContinuumWorld.jl?branch=master) |
| [VDPTag2](https://github.com/zsunberg/VDPTag2.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/d9a10417-2dce-40d9-97b6-4ccdc8548230.svg?branch=master)](https://travis-ci.org/zsunberg/VDPTag2.jl) | |
| [RoombaPOMDPs](https://github.com/sisl/RoombaPOMDPs.jl) (Roomba定位) | [![CI](https://yellow-cdn.veclightyear.com/0a4dffa0/fd1a0c25-6087-416c-9ba3-4e13e0e2a2bb.svg)](https://github.com/sisl/RoombaPOMDPs.jl/actions/workflows/ci.yml)


#### MDP求解器：
|  **`包`**   |  **`构建/覆盖率`** | 在线/<br>离线 | 连续<br>状态 - 动作 |  评级<sup>3</sup> |
|-------------------|----------------------|----------------------|-------------------------|--|
| [离散值迭代](https://github.com/JuliaPOMDP/DiscreteValueIteration.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/32501234-ae29-496f-89a3-90ece66bf5a4.svg)](https://github.com/JuliaPOMDP/DiscreteValueIteration.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/3225dcb0-e463-49d0-938d-c3a270cfe363.svg)](https://codecov.io/gh/JuliaPOMDP/DiscreteValueIteration.jl?branch=master) | 离线 | N-N | ★★★★★ |
| [局部近似值迭代](https://github.com/JuliaPOMDP/LocalApproximationValueIteration.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/2e2a50b3-5251-4cb2-ab62-c180f416c3f5.svg)](https://github.com/JuliaPOMDP/LocalApproximationValueIteration.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/930ccbd6-bd96-4e8a-b91f-a9b17718901b.svg)](https://codecov.io/gh/JuliaPOMDP/LocalApproximationValueIteration.jl?branch=master) | 离线 | Y-N | ★★ |
| [全局近似值迭代](https://github.com/JuliaPOMDP/GlobalApproximationValueIteration.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/ce383c8d-0e7c-40da-a9bb-9799ca633d7d.svg)](https://github.com/JuliaPOMDP/GlobalApproximationValueIteration.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/33c0612c-3f44-4811-a980-5e276e20be6c.svg)](https://codecov.io/gh/JuliaPOMDP/GlobalApproximationValueIteration.jl?branch=master) | 离线 | Y-N | ★★ |
| [MCTS](https://github.com/JuliaPOMDP/MCTS.jl) (蒙特卡罗树搜索) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/627bc18d-751a-49ec-bf01-ad8369db7f60.svg)](https://github.com/JuliaPOMDP/MCTS.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/3a189b70-e13f-4a0f-b923-e51396cb8ddf.svg)](https://codecov.io/gh/JuliaPOMDP/MCTS.jl?branch=master) | 在线 | Y (DPW)-Y (DPW) | ★★★★ |

#### POMDP求解器:
|  **`包`**   |  **`构建/覆盖率`** | 在线/<br>离线 | 连续<br>状态-动作-观测 |  评级<sup>3</sup> |
|-------------------|----------------------|--------------------|---------------------------|--|
| [QMDP](https://github.com/JuliaPOMDP/QMDP.jl) (次优) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/dc085658-b6aa-450c-a3a6-15eb16ffed74.svg)](https://github.com/JuliaPOMDP/QMDP.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/01285f56-8320-470b-ae85-87d9ea4db492.svg)](https://coveralls.io/r/JuliaPOMDP/QMDP.jl)  | 离线 | N-N-N | ★★★★★ |
| [FIB](https://github.com/JuliaPOMDP/FIB.jl) (次优) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/bb504797-96ec-455b-8471-8008b86edc4c.svg)](https://github.com/JuliaPOMDP/FIB.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/330ef336-8bbd-493c-ae92-233482680310.svg)](https://coveralls.io/r/JuliaPOMDP/FIB.jl)  | 离线 | N-N-N | ★★ |
| [信念网格值迭代](https://github.com/JuliaPOMDP/BeliefGridValueIteration.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/03c24213-030d-4dd9-b325-fc68e504e34f.svg)](https://github.com/JuliaPOMDP/BeliefGridValueIteration.jl) <br> [![codecov](https://yellow-cdn.veclightyear.com/0a4dffa0/9cd28c03-20c6-45d1-a1a5-968e62cbd8dd.svg)](https://codecov.io/gh/JuliaPOMDP/BeliefGridValueIteration.jl) | 离线 | N-N-N | ★★ |
| [SARSOP](https://github.com/JuliaPOMDP/SARSOP.jl)* | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/21e37512-a909-4428-b443-dee24cf815a6.svg)](https://github.com/JuliaPOMDP/SARSOP.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/207381aa-9684-4cd5-a8d0-4abed9390fb5.svg)](https://codecov.io/gh/JuliaPOMDP/SARSOP.jl?branch=master) | 离线 | N-N-N | ★★★★ |
| [原生SARSOP](https://github.com/JuliaPOMDP/NativeSARSOP.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/b0e73385-cf90-4289-a95a-cd881d834394.svg)](https://github.com/JuliaPOMDP/NativeSARSOP.jl/actions/workflows/CI.yml) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/c903f10b-8efe-4db1-8cbe-b367884e74b8.svg)](https://codecov.io/gh/JuliaPOMDP/NativeSARSOP.jl) | 离线 | N-N-N | ★★★★ |
| [粒子滤波树](https://github.com/WhiffleFish/ParticleFilterTrees.jl) (稀疏PFT, PFT-DPW) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/bfa939a6-4f53-49d6-90c2-247d8c80fce3.svg)](https://github.com/WhiffleFish/ParticleFilterTrees.jl/actions/workflows/CI.yml) <br> [![codecov](https://yellow-cdn.veclightyear.com/0a4dffa0/411e883a-cf54-40a0-b1cf-ee1d6272626a.svg)](https://codecov.io/gh/WhiffleFish/ParticleFilterTrees.jl) | 在线 | Y-Y<sup>2</sup>-Y | ★★★ |
| [基础POMCP](https://github.com/JuliaPOMDP/BasicPOMCP.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/d5356d85-76a3-4b6d-b463-28ce142e10c9.svg)](https://github.com/JuliaPOMDP/BasicPOMCP.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/4fa18e0e-bec8-4a0e-8ad5-4b6f997f7b20.svg)](https://codecov.io/gh/JuliaPOMDP/BasicPOMCP.jl?branch=master) | 在线 | Y-N-N<sup>1</sup> | ★★★★ |
| [ARDESPOT](https://github.com/JuliaPOMDP/ARDESPOT.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/08855fb6-a53d-46eb-9350-a66d5e689f29.svg)](https://github.com/JuliaPOMDP/ARDESPOT.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/f066895d-ed0b-4d8f-ab2b-b3652822a310.svg)](https://codecov.io/gh/JuliaPOMDP/ARDESPOT.jl?branch=master) | 在线 | Y-N-N<sup>1</sup> | ★★★★ |
| [AdaOPS](https://github.com/JuliaPOMDP/AdaOPS.jl) | [![CI](https://yellow-cdn.veclightyear.com/0a4dffa0/3e5dd14c-0c76-4ece-b5cd-06b76a766690.svg)](https://github.com/JuliaPOMDP/AdaOPS.jl/actions/workflows/CI.yml) <br> [![codecov.io](https://yellow-cdn.veclightyear.com/0a4dffa0/f7021bad-1308-4245-8f11-a4b4ffbca6ce.svg?branch=main)](http://codecov.io/github/JuliaPOMDP/AdaOPS.jl?branch=main) | 在线 | 是-否-是 | ★★★★ |
| [MCVI](https://github.com/JuliaPOMDP/MCVI.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/d5fb7dd8-fe06-4ba4-a16d-84ea104c80ae.svg)](https://github.com/JuliaPOMDP/MCVI.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/c492d51f-e81c-4119-82be-7ca4a09d770c.svg)](https://codecov.io/gh/JuliaPOMDP/MCVI.jl?branch=master) | 离线 | 是-否-是 | ★★ |
| [POMDPSolve](https://github.com/JuliaPOMDP/POMDPSolve.jl)* | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/99b1c006-3ccc-41a4-a7ad-0d00aa419260.svg)](https://github.com/JuliaPOMDP/POMDPSolve.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/df774aca-4db8-4b7c-95d7-e8a3bfe2e9bd.svg)](https://coveralls.io/r/JuliaPOMDP/POMDPSolve.jl) | 离线 | 否-否-否 | ★★★ |
| [增量修剪](https://github.com/JuliaPOMDP/IncrementalPruning.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/c50fa5b4-7477-4701-8b42-2785233fc4a2.svg)](https://github.com/JuliaPOMDP/IncrementalPruning.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/60497dbd-c86e-4fbc-8485-d59152387461.svg)](https://codecov.io/gh/JuliaPOMDP/IncrementalPruning.jl?branch=master)  | 离线 | 否-否-否 | ★★★ |
| [POMCPOW](https://github.com/JuliaPOMDP/POMCPOW.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/0b399d05-11ea-40fa-b84a-909a8496abec.svg)](https://github.com/JuliaPOMDP/POMCPOW.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/490f5bd5-ccb3-46ad-acb8-dd258109bc99.svg)](https://codecov.io/gh/JuliaPOMDP/POMCPOW.jl?branch=master) | 在线 | 是-是<sup>2</sup>-是 | ★★★ |
| [AEMS](https://github.com/JuliaPOMDP/AEMS.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/614db5ec-57f3-494f-8176-874bd434cc41.svg)](https://github.com/JuliaPOMDP/AEMS.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/8bba4c5d-202d-41cb-a070-bb5daf356ddc.svg)](https://coveralls.io/r/JuliaPOMDP/AEMS.jl) | 在线 | 否-否-否 | ★★ |
| [基于点的值迭代](https://github.com/JuliaPOMDP/PointBasedValueIteration.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/29b60005-bda8-406d-9a3d-85acd0509bcc.svg)](https://github.com/JuliaPOMDP/PointBasedValueIteration.jl/actions/workflows/CI.yml) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/26e2df88-8e68-4c68-840a-2eb91ef84bd3.svg)](https://codecov.io/gh/JuliaPOMDP/PointBasedValueIteration.jl?branch=master) | 离线 | 否-否-否 | ★★ |
<sup>1</sup>：可以运行，但无法收敛到最优解

<sup>2</sup>：可以运行，但未证明能收敛到最优解，且在多维动作空间上可能效果不佳。另见 https://github.com/michaelhlim/VOOTreeSearch.jl。

#### 强化学习：

|  **`包名`**   |  **`构建/覆盖率`**  | 连续<br>状态 | 连续<br>动作 | 评级<sup>3</sup> |
|-------------------|----------------------|------------------|------------------|--|
| [TabularTDLearning](https://github.com/JuliaPOMDP/TabularTDLearning.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/88a67af3-862a-40b7-843e-ab7f567c04b7.svg)](https://github.com/JuliaPOMDP/TabularTDLearning.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/3dc414ee-9ab8-4164-b8ea-6d92d8d99a3e.svg)](https://codecov.io/gh/JuliaPOMDP/TabularTDLearning.jl?branch=master) | 否 | 否 | ★★ |
| [DeepQLearning](https://github.com/JuliaPOMDP/DeepQLearning.jl) | [![构建状态](https://yellow-cdn.veclightyear.com/0a4dffa0/ec40a3a7-b1fc-4aa8-9f0d-125d6ac21a12.svg)](https://github.com/JuliaPOMDP/DeepQLearning.jl) <br> [![覆盖率状态](https://yellow-cdn.veclightyear.com/0a4dffa0/e2e0b3dd-3c1b-4fd8-b272-959d1d6bb6c3.svg)](https://codecov.io/gh/JuliaPOMDP/DeepQLearning.jl?branch=master) | 是<sup>1</sup> | 否 | ★★★ |

<sup>1</sup>：对于 POMDP，它将使用观察结果而非状态作为策略的输入。

<sup>3</sup> 主观评级；如果您认为应该更改某个评级，请提出问题
- ★★★★★：可靠地计算每个问题的解决方案。
- ★★★★：适用于大多数问题。可能需要一些配置，或不支持接口的每个边缘。
- ★★★：可能效果不错，但可能需要复杂或重要的配置。
- ★★：最近未使用（状况未知）。可能不完全符合接口，或可能存在包兼容性问题。
- ★：不确定是否可运行

### 性能基准：

|  **`包名`**   | 
|-------------------|
| [DESPOT](https://github.com/JuliaPOMDP/DESPOT.jl/blob/master/test/perflog.md) | 

*_这些包需要非 Julia 依赖项_

## 引用 POMDPs

如果 POMDPs 对您的研究有帮助，您想对其表示感谢，请引用这篇[论文](http://www.jmlr.org/papers/v18/16-300.html)：

@article{egorov2017pomdps, author = {Maxim Egorov and Zachary N. Sunberg and Edward Balaban and Tim A. Wheeler and Jayesh K. Gupta and Mykel J. Kochenderfer}, title = {{POMDP}s.jl: A Framework for Sequential Decision Making under Uncertainty}, journal = {Journal of Machine Learning Research}, year = {2017}, volume = {18}, number = {26}, pages = {1-5}, url = {http://jmlr.org/papers/v18/16-300.html} }