MAP-NEO：一个完全开源的大型语言模型

<div align="center"> <p> <b>MAP-NEO</b>是一个<b>完全开源</b>的大型语言模型，包括预训练数据、数据处理流程（<b>Matrix</b>）、预训练脚本和对齐代码。它在4.5T英文和中文令牌上从头训练，展现出与LLaMA2 7B相当的性能。MAP-Neo模型在推理、数学和编码等具有挑战性的任务中表现出与专有模型相当的性能，优于同等规模的同类模型。出于研究目的，我们旨在实现LLM训练过程的完全透明。为此，我们全面发布了MAP-Neo，包括最终和中间检查点、自训练的分词器、预训练语料库以及高效、稳定的优化预训练代码库。 </p> <p> <img src="https://yellow-cdn.veclightyear.com/835a84d5/2af25921-15c0-4dae-b2ae-94bc60cdaf83.jpg" alt="SLAM-LLM Logo" style="width: 200px; height: auto;"> </p> <p> </p> <div align="center"> <img alt="主页" src="https://yellow-cdn.veclightyear.com/835a84d5/1a86b7a9-4f3d-4aba-90b4-c90e9d83022f.svg" style="width: auto; height: 20px;"/>

<a href="https://map-neo.github.io/"><img src="https://img.shields.io/badge/主页-MAP%20NEO-orange" alt="版本"></a>

<a href="https://huggingface.co/collections/m-a-p/neo-models-66395a5c9662bb58d5d70f04" target="_blank"> <img alt="Hugging Face" src="https://img.shields.io/badge/%F0%9F%A4%97%20Hugging%20Face-MAP%20NEO-ffc107?color=ffc107&logoColor=white" /> </a> <a href="https://arxiv.org/abs/2405.19327" target="_blank"> <img alt="Arxiv" src="https://img.shields.io/badge/Arxiv-MAP%20NEO-7289da?logo=arxiv&logoColor=red&color=red" /> </a> </div> <a href="https://discord.com/invite/qeRsjRdRHf" target="_blank"> <img alt="Discord" src="https://img.shields.io/badge/Discord-MAP%20NEO-7289da?logo=discord&logoColor=white&color=7289da" /> </a> <a href="figure/Wechat_QR.jpg" target="_blank"> <img alt="微信" src="https://img.shields.io/badge/微信-MAP%20NEO-brightgreen?logo=wechat&logoColor=white" /> </a> <a href="https://twitter.com/GeZhang86038849/status/1788874345927889203" target="_blank"> <img alt="Twitter关注" src="https://img.shields.io/badge/Twitter-MAP%20NEO-lightgrey?logo=x&logoColor=white" /> </a> <a href="https://github.com/multimodal-art-projection/MAP-NEO"><img src="https://yellow-cdn.veclightyear.com/835a84d5/576edf41-15a2-46cb-84c7-d0494c78b7b3.svg" alt="mit"></a> </div>

评估结果

MAP-NEO 7B系列

我们在各种基准测试上评估了MAP-Neo 7B系列和其他类似规模的模型，如下所示。

<p align="center"> <img src="https://yellow-cdn.veclightyear.com/835a84d5/56e8341f-59c0-43fa-b531-58038e591ab5.png" alt="表格" width="100%"> </p>

有关不同基准测试性能的更多详细信息，请参阅https://map-neo.github.io/。

模型和数据下载

我们很高兴宣布公开发布MAP-NEO 7B，包括基础模型和一系列中间检查点。此次发布旨在支持学术和商业社区内更广泛、更多样化的研究。请注意，本模型的使用受许可证部分中概述的条款约束。在这些条款下允许商业使用。

Huggingface

模型和数据集	下载
MAP-NEO 7B 基础	🤗 HuggingFace
MAP-NEO 7B 指令	🤗 HuggingFace
MAP-NEO 7B SFT	🤗 HuggingFace
MAP-NEO 7B 中间	🤗 HuggingFace
MAP-NEO 7B 衰减	🤗 HuggingFace
MAP-NEO 2B 基础	🤗 HuggingFace
MAP-NEO 缩放律 980M	🤗 HuggingFace
MAP-NEO 缩放律 460M	🤗 HuggingFace
MAP-NEO 缩放律 250M	🤗 HuggingFace
MAP-NEO 数据 Matrix	🤗 HuggingFace

许可证

此代码仓库根据MIT许可证授权。

引用

@article{zhang2024mapneo,
    title   = {MAP-Neo: Highly Capable and Transparent Bilingual Large Language Model Series},
    author  = {Ge Zhang and Scott Qu and Jiaheng Liu and Chenchen Zhang and Chenghua Lin and Chou Leuang Yu and Danny Pan and Esther Cheng and Jie Liu and Qunshu Lin and Raven Yuan and Tuney Zheng and Wei Pang and Xinrun Du and Yiming Liang and Yinghao Ma and Yizhi Li and Ziyang Ma and Bill Lin and Emmanouil Benetos and Huan Yang and Junting Zhou and Kaijing Ma and Minghao Liu and Morry Niu and Noah Wang and Quehry Que and Ruibo Liu and Sine Liu and Shawn Guo and Soren Gao and Wangchunshu Zhou and Xinyue Zhang and Yizhi Zhou and Yubo Wang and Yuelin Bai and Yuhan Zhang and Yuxiang Zhang and Zenith Wang and Zhenzhu Yang and Zijian Zhao and Jiajun Zhang and Wanli Ouyang and Wenhao Huang and Wenhu Chen},
    year    = {2024},
    journal = {arXiv preprint arXiv: 2405.19327}
}

联系方式

如需进一步交流，请扫描以下微信和Discord二维码：

<table> <tr> <td>微信 </td> <td>Discord </td> </tr> <tr> <td><img src="https://yellow-cdn.veclightyear.com/835a84d5/f35b1140-dfab-4d36-a549-8a4d45556380.jpg" alt="微信二维码" width="200"/></td> <td><img src="https://yellow-cdn.veclightyear.com/835a84d5/d6baddca-3515-4fd6-aec6-02bab71eb7ef.jpg" alt="Discord二维码" width="200"/></td> </tr> </table>