&nbspEnglish&nbsp |   <a href="README_CN.md">中文</a> <div align="center"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://raw.githubusercontent.com/01-ai/Yi/main/assets/img/Yi_logo_icon_dark.svg" width="200px"> <source media="(prefers-color-scheme: light)" srcset="https://raw.githubusercontent.com/01-ai/Yi/main/assets/img/Yi_logo_icon_light.svg" width="200px"> <img alt="specify theme context for images" src="https://raw.githubusercontent.com/01-ai/Yi/main/assets/img/Yi_logo_icon_light.svg" width="200px"> </picture> <a href="https://github.com/01-ai/Yi/actions/workflows/build_docker_image.yml"> <img src="https://github.com/01-ai/Yi/actions/workflows/build_docker_image.yml/badge.svg"> </a> <a href="mailto:oss@01.ai"> <img src="https://img.shields.io/badge/✉️-yi@01.ai-FFE01B"> </a> </div> <div id="top"></div> <div align="center"> <h3 align="center">Building the Next Generation of Open-Source and Bilingual LLMs</h3> </div> 🤗 <a href="https://huggingface.co/01-ai" target="_blank">Hugging Face</a> • 🤖 <a href="https://www.modelscope.cn/organization/01ai/" target="_blank">ModelScope</a> • 🟣 <a href="https://wisemodel.cn/organization/01.AI" target="_blank">wisemodel</a> 👩‍🚀 Ask questions or discuss ideas on <a href="https://github.com/01-ai/Yi/discussions" target="_blank"> GitHub </a> 👋 Join us on <a href="https://discord.gg/hYUwWddeAu" target="_blank"> 👾 Discord </a> or <a href="https://github.com/01-ai/Yi/issues/43#issuecomment-1827285245" target="_blank"> 💬 WeChat </a> 📝 Check out <a href="https://arxiv.org/abs/2403.04652"> Yi Tech Report </a> 📚 Grow at <a href="#learning-hub"> Yi Learning Hub </a> 💪 Learn at <a href="https://01-ai.github.io/"> Yi Tech Blog </a>  <hr> <details open> <summary>📕 Table of Contents</summary>

What is Yi?
- Introduction
- Models
- News
How to use Yi?
- Quick start
  - Choose your path
  - pip
  - docker
  - llama.cpp
  - conda-lock
  - Web demo
- Fine-tuning
- Quantization
- Deployment
- FAQ
- Learning hub
Why Yi?
Who can use Yi?
Misc.

</details> <hr>

What is Yi?

Introduction

🤖 The Yi series models are the next generation of open-source large language models trained from scratch by 01.AI.
🙌 Targeted as a bilingual language model and trained on 3T multilingual corpus, the Yi series models become one of the strongest LLM worldwide, showing promise in language understanding, commonsense reasoning, reading comprehension, and more. For example,
- Yi-34B-Chat model landed in second place (following GPT-4 Turbo), outperforming other LLMs (such as GPT-4, Mixtral, Claude) on the AlpacaEval Leaderboard (based on data available up to January 2024).
- Yi-34B model ranked first among all existing open-source models (such as Falcon-180B, Llama-70B, Claude) in both English and Chinese on various benchmarks, including Hugging Face Open LLM Leaderboard (pre-trained) and C-Eval (based on data available up to November 2023).
- 🙏 (Credits to Llama) Thanks to the Transformer and Llama open-source communities, as they reduce the efforts required to build from scratch and enable the utilization of the same tools within the AI ecosystem.
<details style="display: inline;"><summary> If you're interested in Yi's adoption of Llama architecture and license usage policy, see Yi's relation with Llama. ⬇️</summary> <ul>

💡 TL;DR

The Yi series models adopt the same model architecture as Llama but are NOT derivatives of Llama.

Both Yi and Llama are based on the Transformer structure, which has been the standard architecture for large language models since 2018.
Grounded in the Transformer architecture, Llama has become a new cornerstone for the majority of state-of-the-art open-source models due to its excellent stability, reliable convergence, and robust compatibility. This positions Llama as the recognized foundational framework for models including Yi.
Thanks to the Transformer and Llama architectures, other models can leverage their power, reducing the effort required to build from scratch and enabling the utilization of the same tools within their ecosystems.
However, the Yi series models are NOT derivatives of Llama, as they do not use Llama's weights.
- As Llama's structure is employed by the majority of open-source models, the key factors of determining model performance are training datasets, training pipelines, and training infrastructure.
- Developing in a unique and proprietary way, Yi has independently created its own high-quality training datasets, efficient training pipelines, and robust training infrastructure entirely from the ground up. This effort has led to excellent performance with Yi series models ranking just behind GPT4 and surpassing Llama on the Alpaca Leaderboard in Dec 2023.

</ul> </details> [ <a href="#top">Back to top ⬆️ </a> ]

News

<details> <summary>🔥 2024-07-29: The <a href="https://github.com/Haijian06/Yi/tree/main/Cookbook">Yi Cookbook 1.0 </a> is released, featuring tutorials and examples in both Chinese and English.</summary> </details> <details> <summary>🎯 2024-05-13: The <a href="https://github.com/01-ai/Yi-1.5">Yi-1.5 series models </a> are open-sourced, further improving coding, math, reasoning, and instruction-following abilities.</summary> </details> <details> <summary>🎯 2024-03-16: The <code>Yi-9B-200K</code> is open-sourced and available to the public.</summary> </details> <details> <summary>🎯 2024-03-08: <a href="https://arxiv.org/abs/2403.04652">Yi Tech Report</a> is published! </summary> </details> <details open> <summary>🔔 2024-03-07: The long text capability of the Yi-34B-200K has been enhanced. </summary> In the "Needle-in-a-Haystack" test, the Yi-34B-200K's performance is improved by 10.5%, rising from 89.3% to an impressive 99.8%. We continue to pre-train the model on 5B tokens long-context data mixture and demonstrate a near-all-green performance. </details> <details open> <summary>🎯 2024-03-06: The <code>Yi-9B</code> is open-sourced and available to the public.</summary> <code>Yi-9B</code> stands out as the top performer among a range of similar-sized open-source models (including Mistral-7B, SOLAR-10.7B, Gemma-7B, DeepSeek-Coder-7B-Base-v1.5 and more), particularly excelling in code, math, common-sense reasoning, and reading comprehension. </details> <details open> <summary>🎯 2024-01-23: The Yi-VL models, <code><a href="https://huggingface.co/01-ai/Yi-VL-34B">Yi-VL-34B</a></code> and <code><a href="https://huggingface.co/01-ai/Yi-VL-6B">Yi-VL-6B</a></code>, are open-sourced and available to the public.</summary> <code><a href="https://huggingface.co/01-ai/Yi-VL-34B">Yi-VL-34B</a></code> has ranked first among all existing open-source models in the latest benchmarks, including <a href="https://arxiv.org/abs/2311.16502">MMMU</a> and <a href="https://arxiv.org/abs/2401.11944">CMMMU</a> (based on data available up to January 2024).</li> </details> <details> <summary>🎯 2023-11-23: <a href="#chat-models">Chat models</a> are open-sourced and available to the public.</summary> This release contains two chat models based on previously released base models, two 8-bit models quantized by GPTQ, and two 4-bit models quantized by AWQ.

Yi-34B-Chat
Yi-34B-Chat-4bits
Yi-34B-Chat-8bits
Yi-6B-Chat
Yi-6B-Chat-4bits
Yi-6B-Chat-8bits

You can try some of them interactively at:

</details> <details> <summary>🔔 2023-11-23: The Yi Series Models Community License Agreement is updated to <a href="https://github.com/01-ai/Yi/blob/main/MODEL_LICENSE_AGREEMENT.txt">v2.1</a>.</summary> </details> <details> <summary>🔥 2023-11-08: Invited test of Yi-34B chat model.</summary> Application form:

</details> <details> <summary>🎯 2023-11-05: <a href="#base-models">The base models, </a><code>Yi-6B-200K</code> and <code>Yi-34B-200K</code>, are open-sourced and available to the public.</summary> This release contains two base models with the same parameter sizes as the previous release, except that the context window is extended to 200K. </details> <details> <summary>🎯 2023-11-02: <a href="#base-models">The base models, </a><code>Yi-6B</code> and <code>Yi-34B</code>, are open-sourced and available to the public.</summary> The first public release contains two bilingual (English/Chinese) base models with the parameter sizes of 6B and 34B. Both of them are trained with 4K sequence length and can be extended to 32K during inference time. </details> [ <a href="#top">Back to top ⬆️ </a> ]

Models

Yi models come in multiple sizes and cater to different use cases. You can also fine-tune Yi models to meet your specific requirements.

If you want to deploy Yi models, make sure you meet the software and hardware requirements.

Chat models

Model	Download
Yi-34B-Chat	• 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel
Yi-34B-Chat-4bits	• 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel
Yi-34B-Chat-8bits	• 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel
Yi-6B-Chat	• 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel
Yi-6B-Chat-4bits	• 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel
Yi-6B-Chat-8bits	• 🤗 Hugging Face • 🤖 ModelScope • 🟣 wisemodel

- 4-bit series models are quantized by AWQ. - 8-bit series models are quantized by GPTQ - All quantized models have a low barrier to use since they can be deployed on consumer-grade GPUs (e.g., 3090, 4090).