mmdetection3d

<div align="center"> <img src="https://yellow-cdn.veclightyear.com/835a84d5/5f9663a6-b6df-4444-9645-8ae88ab2342c.png" width="600"/> <div> </div> <div align="center"> <b><font size="5">OpenMMLab 网站</font></b> <sup> <a href="https://openmmlab.com"> <i><font size="4">热门</font></i> </a> </sup>      <b><font size="5">OpenMMLab 平台</font></b> <sup> <a href="https://platform.openmmlab.com"> <i><font size="4">立即体验</font></i> </a> </sup> </div> <div> </div>

📘文档 | 🛠️安装 | 👀模型库 | 🆕更新日志 | 🚀进行中的项目 | 🤔报告问题

</div> <div align="center">

English | 简体中文

</div> <div align="center"> <a href="https://openmmlab.medium.com/" style="text-decoration:none;"> <img src="https://yellow-cdn.veclightyear.com/835a84d5/26849115-7031-42aa-959f-dc0a17e4fb0f.png" width="3%" alt="" /></a> <img src="https://yellow-cdn.veclightyear.com/835a84d5/d1dcd504-3686-4f36-859a-fdc5d24a7ce3.png" width="3%" alt="" /> <a href="https://discord.com/channels/1037617289144569886/1046608014234370059" style="text-decoration:none;"> <img src="https://yellow-cdn.veclightyear.com/835a84d5/a984d7d2-f77d-4cea-95a3-cc954aea72be.png" width="3%" alt="" /></a> <img src="https://yellow-cdn.veclightyear.com/835a84d5/d1dcd504-3686-4f36-859a-fdc5d24a7ce3.png" width="3%" alt="" /> <a href="https://twitter.com/OpenMMLab" style="text-decoration:none;"> <img src="https://yellow-cdn.veclightyear.com/835a84d5/64653f70-9072-4b4e-b124-33d7058db2e3.png" width="3%" alt="" /></a> <img src="https://yellow-cdn.veclightyear.com/835a84d5/d1dcd504-3686-4f36-859a-fdc5d24a7ce3.png" width="3%" alt="" /> <a href="https://www.youtube.com/openmmlab" style="text-decoration:none;"> <img src="https://yellow-cdn.veclightyear.com/835a84d5/42cfeb78-0c0a-4a4b-b089-629cac2fd04e.png" width="3%" alt="" /></a> <img src="https://yellow-cdn.veclightyear.com/835a84d5/d1dcd504-3686-4f36-859a-fdc5d24a7ce3.png" width="3%" alt="" /> <a href="https://space.bilibili.com/1293512903" style="text-decoration:none;"> <img src="https://yellow-cdn.veclightyear.com/835a84d5/e96e143a-b1fb-4e88-98c0-1b44e1eef7fc.png" width="3%" alt="" /></a> <img src="https://yellow-cdn.veclightyear.com/835a84d5/d1dcd504-3686-4f36-859a-fdc5d24a7ce3.png" width="3%" alt="" /> <a href="https://www.zhihu.com/people/openmmlab" style="text-decoration:none;"> <img src="https://yellow-cdn.veclightyear.com/835a84d5/5cb72038-69f2-49c2-8491-0e31413571a7.png" width="3%" alt="" /></a> </div>

简介

MMDetection3D 是一个基于 PyTorch 的开源目标检测工具箱，面向下一代通用 3D 检测平台。它是 OpenMMLab 项目的一部分。

主分支适用于 PyTorch 1.8+。

演示图片

开箱即用支持多模态/单模态检测器

直接支持多种多模态/单模态检测器，包括MVXNet、VoteNet、PointPillars等。
开箱即用支持室内/室外3D检测

直接支持流行的室内和室外3D检测数据集，包括ScanNet、SUNRGB-D、Waymo、nuScenes、Lyft和KITTI。对于nuScenes数据集，我们还支持nuImages数据集。
与2D检测自然集成

MMDetection中支持的所有300多个模型、40多篇论文的方法以及模块都可以在此代码库中训练或使用。
高效率

训练速度比其他代码库更快。主要结果如下。详细信息可以在benchmark.md中找到。我们比较了每秒训练的样本数（越高越好）。其他代码库不支持的模型标记为✗。

方法 MMDetection3D OpenPCDet votenet Det3D
VoteNet 358 ✗ 77 ✗
PointPillars-car 141 ✗ ✗ 140
PointPillars-3class 107 44 ✗ ✗
SECOND 40 30 ✗ ✗
Part-A2 17 14 ✗ ✗

方法	MMDetection3D	OpenPCDet	votenet	Det3D
VoteNet	358	✗	77	✗
PointPillars-car	141	✗	✗	140
PointPillars-3class	107	44	✗	✗
SECOND	40	30	✗	✗
Part-A2	17	14	✗	✗

</details>

与MMDetection和MMCV一样，MMDetection3D也可以作为一个库来支持其他基于它的不同项目。

安装

请参考安装指南进行安装。

入门指南

有关详细的用户指南和高级指南，请参考我们的文档：

</details> <details> <summary>高级指南</summary> - [数据集](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/index.html#datasets) - [KITTI 数据集](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/kitti.html) - [NuScenes 数据集](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/nuscenes.html) - [Lyft 数据集](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/lyft.html) - [Waymo 数据集](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/waymo.html) - [SUN RGB-D 数据集](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/sunrgbd.html) - [ScanNet 数据集](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/scannet.html) - [S3DIS 数据集](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/s3dis.html) - [SemanticKITTI 数据集](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/datasets/semantickitti.html) - [支持的任务](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/index.html#supported-tasks) - [基于激光雷达的3D检测](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/supported_tasks/lidar_det3d.html) - [基于视觉的3D检测](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/supported_tasks/vision_det3d.html) - [基于激光雷达的3D语义分割](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/supported_tasks/lidar_sem_seg3d.html) - [自定义](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/index.html#customization) - [自定义数据集](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_dataset.html) - [自定义模型](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_models.html) - [自定义运行时设置](https://mmdetection3d.readthedocs.io/en/latest/advanced_guides/customize_runtime.html)

基准测试和模型库概览

结果和模型可在模型库中获取。

<div align="center"> <b>组件</b> </div> <table align="center"> <tbody> <tr align="center" valign="bottom"> <td> <b>主干网络</b> </td> <td> <b>检测头</b> </td> <td> <b>特性</b> </td> </tr> <tr valign="top"> <td> <ul> <li><a href="configs/pointnet2">PointNet (CVPR'2017)</a></li> <li><a href="configs/pointnet2">PointNet++ (NeurIPS'2017)</a></li> <li><a href="configs/regnet">RegNet (CVPR'2020)</a></li> <li><a href="configs/dgcnn">DGCNN (TOG'2019)</a></li> <li>DLA (CVPR'2018)</li> <li>MinkResNet (CVPR'2019)</li> <li><a href="configs/minkunet">MinkUNet (CVPR'2019)</a></li> <li><a href="configs/cylinder3d">Cylinder3D (CVPR'2021)</a></li> </ul> </td> <td> <ul> <li><a href="configs/free_anchor">FreeAnchor (NeurIPS'2019)</a></li> </ul> </td> <td> <ul> <li><a href="configs/dynamic_voxelization">动态体素化 (CoRL'2019)</a></li> </ul> </td> </tr> </td> </tr> </tbody> </table> <div align="center"> <b>架构</b> </div> <table align="center"> <tbody> <tr align="center" valign="middle"> <td> <b>基于激光雷达的3D目标检测</b> </td> <td> <b>基于相机的3D目标检测</b> </td> <td> <b>多模态3D目标检测</b> </td> <td> <b>3D语义分割</b> </td> </tr> <tr valign="top"> <td> <li><b>室外</b></li> <ul> <li><a href="configs/second">SECOND (Sensor'2018)</a></li> <li><a href="configs/pointpillars">PointPillars (CVPR'2019)</a></li> <li><a href="configs/ssn">SSN (ECCV'2020)</a></li> <li><a href="configs/3dssd">3DSSD (CVPR'2020)</a></li> <li><a href="configs/sassd">SA-SSD (CVPR'2020)</a></li> <li><a href="configs/point_rcnn">PointRCNN (CVPR'2019)</a></li> <li><a href="configs/parta2">Part-A2 (TPAMI'2020)</a></li> <li><a href="configs/centerpoint">CenterPoint (CVPR'2021)</a></li> <li><a href="configs/pv_rcnn">PV-RCNN (CVPR'2020)</a></li> <li><a href="projects/CenterFormer">CenterFormer (ECCV'2022)</a></li> </ul> <li><b>室内</b></li> <ul> <li><a href="configs/votenet">VoteNet (ICCV'2019)</a></li> <li><a href="configs/h3dnet">H3DNet (ECCV'2020)</a></li> <li><a href="configs/groupfree3d">Group-Free-3D (ICCV'2021)</a></li> <li><a href="configs/fcaf3d">FCAF3D (ECCV'2022)</a></li> <li><a href="projects/TR3D">TR3D (ArXiv'2023)</a></li> </ul> </td> <td> <li><b>室外</b></li> <ul> <li><a href="configs/imvoxelnet">ImVoxelNet (WACV'2022)</a></li> <li><a href="configs/smoke">SMOKE (CVPRW'2020)</a></li> <li><a href="configs/fcos3d">FCOS3D (ICCVW'2021)</a></li> <li><a href="configs/pgd">PGD (CoRL'2021)</a></li> <li><a href="configs/monoflex">MonoFlex (CVPR'2021)</a></li> <li><a href="projects/DETR3D">DETR3D (CoRL'2021)</a></li> <li><a href="projects/PETR">PETR (ECCV'2022)</a></li> </ul> <li><b>室内</b></li> <ul> <li><a href="configs/imvoxelnet">ImVoxelNet (WACV'2022)</a></li> </ul> </td> <td> <li><b>室外</b></li> <ul> <li><a href="configs/mvxnet">MVXNet (ICRA'2019)</a></li> <li><a href="projects/BEVFusion">BEVFusion (ICRA'2023)</a></li> </ul> <li><b>室内</b></li> <ul> <li><a href="configs/imvotenet">ImVoteNet (CVPR'2020)</a></li> </ul> </td> <td> <li><b>室外</b></li> <ul> <li><a href="configs/minkunet">MinkUNet (CVPR'2019)</a></li> <li><a href="configs/spvcnn">SPVCNN (ECCV'2020)</a></li> <li><a href="configs/cylinder3d">Cylinder3D (CVPR'2021)</a></li> <li><a href="projects/TPVFormer">TPVFormer (CVPR'2023)</a></li> </ul> <li><b>室内</b></li> <ul> <li><a href="configs/pointnet2">PointNet++ (NeurIPS'2017)</a></li> <li><a href="configs/paconv">PAConv (CVPR'2021)</a></li> <li><a href="configs/dgcnn">DGCNN (TOG'2019)</a></li> </ul> </ul> </td> </tr> </td> </tr> </tbody> </table> | | ResNet | VoVNet | Swin-T | PointNet++ | SECOND | DGCNN | RegNetX | DLA | MinkResNet | Cylinder3D | MinkUNet | | :-----------: | :----: | :----: | :----: | :--------: | :----: | :---: | :-----: | :-: | :--------: | :--------: | :------: | | SECOND | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | PointPillars | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | | FreeAnchor | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | | VoteNet | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | H3DNet | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | 3DSSD | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | Part-A2 | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | MVXNet | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | CenterPoint | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | SSN | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | | ImVoteNet | ✓ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | FCOS3D | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | PointNet++ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | Group-Free-3D | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | ImVoxelNet | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | PAConv | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | DGCNN | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | | SMOKE | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | | PGD | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | MonoFlex | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | | SA-SSD | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | FCAF3D | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | | PV-RCNN | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | Cylinder3D | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | | MinkUNet | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | | SPVCNN | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | | BEVFusion | ✗ | ✗ | ✓ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | CenterFormer | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | TR3D | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | ✗ | | DETR3D | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | PETR | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | | TPVFormer | ✓ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | ✗ | **注意：** [MMDetection](https://github.com/open-mmlab/mmdetection/blob/3.x/docs/en/model_zoo.md) 支持的所有**500多个模型和90多篇论文的方法**，均可在此代码库中进行训练或使用。

常见问题

请参阅常见问题以了解常见问题的解答。

贡献

我们欢迎所有对改进MMDetection3D的贡献。请参阅贡献指南以了解贡献细则。

致谢

MMDetection3D是一个由来自各大院校和公司的研究人员和工程师共同贡献的开源项目。我们感谢所有贡献者以及提供宝贵反馈的用户。我们希望这个工具箱和基准能够为不断增长的研究社区提供一个灵活的工具，用于重新实现现有方法和开发新的3D检测器。

引用

如果您在研究中使用了这个项目，请考虑引用：

@misc{mmdet3d2020,
    title={{MMDetection3D: OpenMMLab} next-generation platform for general {3D} object detection},
    author={MMDetection3D Contributors},
    howpublished = {\url{https://github.com/open-mmlab/mmdetection3d}},
    year={2020}
}