Awesome-instruction-tuning 学习资料汇总 - 指令微调数据集、模型和论文集锦

Awesome-instruction-tuning

Awesome-instruction-tuning 是 GitHub 上一个精心策划的开源指令微调资源列表,包含了大量指令微调相关的数据集、模型、论文和代码库。本文将为您详细介绍该项目的核心内容,帮助您快速了解和学习指令微调技术。

Instruction tuning datasets

该项目列出了两类主要的指令微调数据集:

这类数据集是在传统NLP任务的基础上修改而成的,主要包括:

这类数据集是由大语言模型直接生成的,主要包括:

Multilingual tools

该项目还提供了一个基于Helsinki-NLP的开源翻译工具,可以将英语数据集免费翻译成100多种语言。虽然翻译质量可能不如高质量人工翻译,但可以作为一个低成本的替代方案。

使用方法:

python translator.py model_name source_data_path

例如:

python translator.py Helsinki-NLP/opus-mt-en-zh alpaca_data.json

该项目还整理了指令微调领域的重要论文,按时间顺序包括:

Finetuned language models are zero-shot learners (2021.9)
Multitask Prompted Training Enables Zero-Shot Task Generalization (2021.10)
Training language models to follow instructions with human feedback (2022.3)
Super-NaturalInstructions: Generalization via Declarative Instructions on 1600+ NLP Tasks (2022.4)
Unsupervised Cross-Task Generalization via Retrieval Augmentation (2022.4)
Instruction Induction: From Few Examples to Natural Language Task Descriptions (2022.5)
Scaling Instruction-Finetuned Language Models (2022.10)
Guess the Instruction! Flipped Learning Makes Language Models Stronger Zero-Shot Learners (2022.10)
Unnatural Instructions: Tuning Language Models with (Almost) No Human Labor (2022.12)
Improving Cross-task Generalization of Unified Table-to-text Models with Compositional Task Configurations (2022.12)
Self-Instruct: Aligning Language Model with Self Generated Instructions (2022.12)
MultiInstruct: Improving Multi-Modal Zero-Shot Learning via Instruction Tuning (2022.12)
The Flan Collection: Designing Data and Methods for Effective Instruction Tuning (2023.1)
In-Context Instruction Learning (2023.2)