InstructLab 🐶 (`ilab`)

Release License

📖 Contents

Welcome to the InstructLab CLI
🎺 What's New
❓ What is ilab
📋 Requirements
✅ Getting started
💻 Creating new knowledge or skills and training the model
📣 Chat with the new model (not optional this time)
🚀 Upgrade InstructLab to latest version
🎁 Submit your new knowledge or skills
📬 Contributing

Welcome to the InstructLab CLI

InstructLab 🐶 uses a novel synthetic data-based alignment tuning method for Large Language Models (LLMs.) The "lab" in InstructLab 🐶 stands for Large-Scale Alignment for ChatBots [1].

[1] Shivchander Sudalairaj*, Abhishek Bhandwaldar*, Aldo Pareja*, Kai Xu, David D. Cox, Akash Srivastava*. "LAB: Large-Scale Alignment for ChatBots", arXiv preprint arXiv: 2403.01081, 2024. (* denotes equal contributions)

🎺 What's new

InstructLab release 0.17.0 on June 14, 2024 contains updates to the ilab CLI design. The ilab commands now fall into groups for an easier workflow and understanding of the commands. For more information, see the InstructLab CLI reference To view all the available flags for each command group, use the --help tag after the command. The original commands are still in effect, but will be deprecated in release 0.19.0 on July 11, 2024.

❓ What is `ilab`

ilab is a Command-Line Interface (CLI) tool that allows you to perform the following actions:

Download a pre-trained Large Language Model (LLM).
Chat with the LLM.

To add new knowledge and skills to the pre-trained LLM, add information to the companion taxonomy repository.

After you have added knowledge and skills to the taxonomy, you can perform the following actions:

Use ilab to generate new synthetic training data based on the changes in your local taxonomy repository.
Re-train the LLM with the new training data.
Chat with the re-trained LLM to see the results.

graph TD;
  download-->chat
  chat[Chat with the LLM]-->add
  add[Add new knowledge\nor skill to taxonomy]-->generate[generate new\nsynthetic training data]
  generate-->train
  train[Re-train]-->|Chat with\nthe re-trained LLM\nto see the results|chat

For an overview of the full workflow, see the workflow diagram.

[!IMPORTANT] We have optimized InstructLab so that community members with commodity hardware can perform these steps. However, running InstructLab on a laptop will provide a low-fidelity approximation of synthetic data generation (using the ilab data generate command) and model instruction tuning (using the ilab model train command, which uses QLoRA). To achieve higher quality, use more sophisticated hardware and configure InstructLab to use a larger teacher model such as Mixtral.

📋 Requirements

🍎 Apple M1/M2/M3 Mac or 🐧 Linux system (tested on Fedora). We anticipate support for more operating systems in the future.
C++ compiler
Python 3.10 or Python 3.11
Approximately 60GB disk space (entire process)

NOTE: Python 3.12 is currently not supported, because some dependencies don't work on Python 3.12, yet.

NOTE: When installing the ilab CLI on macOS, you may have to run the xcode-select --install command, installing the required packages previously listed.

✅ Getting started

🧰 Installing `ilab`

When installing on Fedora Linux, install C++, Python 3.10 or 3.11, and other necessary tools by running the following command:
```
sudo dnf install gcc gcc-c++ make git python3.11 python3.11-devel
```
If you are running on macOS, this installation is not necessary and you can begin your process with the following step.
Create a new directory called instructlab to store the files the ilab CLI needs when running and cd into the directory by running the following command:
```
mkdir instructlab
cd instructlab
```
NOTE: The following steps in this document use Python venv for virtual environments. However, if you use another tool such as pyenv or Conda Miniforge for managing Python environments on your machine continue to use that tool instead. Otherwise, you may have issues with packages that are installed but not found in venv.
There are a few ways you can locally install the ilab CLI. Select your preferred installation method from the following instructions. You can then install ilab and activate your venv environment.

NOTE: ⏳ pip install may take some time, depending on your internet connection. In case installation fails with error unsupported instruction `vpdpbusd', append -C cmake.args="-DLLAMA_NATIVE=off" to pip install command.

See the GPU acceleration documentation for how to to enable hardware acceleration for interaction and training on AMD ROCm, Apple Metal Performance Shaders (MPS), and Nvidia CUDA.

Install using PyTorch without CUDA bindings and no GPU acceleration
```
python3 -m venv --upgrade-deps venv
source venv/bin/activate
pip cache remove llama_cpp_python
pip install 'instructlab[cpu]' \
   --extra-index-url=https://download.pytorch.org/whl/cpu \
   -C cmake.args="-DLLAMA_NATIVE=off"
```
NOTE: Additional Build Argument for Intel Macs

If you have an Mac with an Intel CPU, you must add a prefix of CMAKE_ARGS="-DLLAMA_METAL=off" to the pip install command to ensure that the build is done without Apple M-series GPU support.

(venv) $ CMAKE_ARGS="-DLLAMA_METAL=off" pip install ...

Install with AMD ROCm
```
python3 -m venv --upgrade-deps venv
source venv/bin/activate
pip cache remove llama_cpp_python
pip install 'instructlab[rocm]' \
   --extra-index-url https://download.pytorch.org/whl/rocm6.0 \
   -C cmake.args="-DLLAMA_HIPBLAS=on" \
   -C cmake.args="-DAMDGPU_TARGETS=all" \
   -C cmake.args="-DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang" \
   -C cmake.args="-DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++" \
   -C cmake.args="-DCMAKE_PREFIX_PATH=/opt/rocm" \
   -C cmake.args="-DLLAMA_NATIVE=off"
```
On Fedora 40+, use -DCMAKE_C_COMPILER=clang-17 and -DCMAKE_CXX_COMPILER=clang++-17.

Install with Apple Metal on M1/M2/M3 Macs

NOTE: Make sure your system Python build is Mach-O 64-bit executable arm64 by using file -b $(command -v python), or if your system is setup with pyenv by using the file -b $(pyenv which python) command.
```
python3 -m venv --upgrade-deps venv
source venv/bin/activate
pip cache remove llama_cpp_python
pip install 'instructlab[mps]'
```
Install with Nvidia CUDA
```
python3 -m venv --upgrade-deps venv
source venv/bin/activate
pip cache remove llama_cpp_python
pip install 'instructlab[cuda]' \
   -C cmake.args="-DLLAMA_CUDA=on" \
   -C cmake.args="-DLLAMA_NATIVE=off"
```

From your venv environment, verify ilab is installed correctly, by running the ilab command.

ilab

Example output of the ilab command

(venv) $ ilab
Usage: ilab [OPTIONS] COMMAND [ARGS]...

CLI for interacting with InstructLab.

If this is your first time running InstructLab, it's best to start with `ilab config init` to create the environment.

Options:
--config PATH  Path to a configuration file.  [default: config.yaml]
--version      Show the version and exit.
--help         Show this message and exit.

Command:
   config      Command group for Interacting with the Config of InstructLab
   data        Command group for Interacting with the Data of generated by...
   model       Command group for Interacting with the Models in InstructLab
   sysinfo     Print system information
   taxonomy    Command group for Interacting with the Taxonomy in InstructLab

Aliases:
   chat: model chat
   convert: model convert
   diff: taxonomy diff
   download: model download
   generate: data generate
   init: config init
   serve: model serve
   test: model test
   train: model train

IMPORTANT Every ilab command needs to be run from within your Python virtual environment. You can enter the Python environment by running the source venv/bin/activate command.

Optional: You can enable tab completion for the ilab command.

Bash (version 4.4 or newer)

Enable tab completion in bash with the following command:
```
eval "$(_ILAB_COMPLETE=bash_source ilab)"
```
To have this enabled automatically every time you open a new shell, you can save the completion script and source it from ~/.bashrc:
```
_ILAB_COMPLETE=bash_source ilab > ~/.ilab-complete.bash
echo ". ~/.ilab-complete.bash" >> ~/.bashrc
```
Zsh

Enable tab completion in zsh with the following command:
```
eval "$(_ILAB_COMPLETE=zsh_source ilab)"
```
To have this enabled automatically every time you open a new shell, you can save the completion script and source it from ~/.zshrc:
```
_ILAB_COMPLETE=zsh_source ilab > ~/.ilab-complete.zsh
echo ". ~/.ilab-complete.zsh" >> ~/.zshrc
```
Fish

Enable tab completion in fish with the following command:
```
_ILAB_COMPLETE=fish_source ilab | source
```
To have this enabled automatically every time you open a new shell, you can save the completion script and source it from ~/.bashrc:
```
_ILAB_COMPLETE=fish_source ilab > ~/.config/fish/completions/ilab.fish
```

🏗️ Initialize `ilab`

Initialize ilab by running the following command:

ilab config init

Example output

Welcome to InstructLab CLI. This guide will help you set up your environment.
Please provide the following values to initiate the environment [press Enter for defaults]:
Path to taxonomy repo [taxonomy]: <ENTER>

When prompted by the interface, press Enter to add a new default config.yaml file.

When prompted, clone the https://github.com/instructlab/taxonomy.git repository into the current directory by typing y.

Optional: If you want to point to an existing local clone of the taxonomy repository, you can pass the path interactively or alternatively with the --taxonomy-path flag.

Example output after initializing ilab

(venv) $ ilab config init
Welcome to InstructLab CLI. This guide will help you set up your environment.
Please provide the following values to initiate the environment [press Enter for defaults]:
Path to taxonomy repo [taxonomy]: <ENTER>
`taxonomy` seems to not exists or is empty. Should I clone https://github.com/instructlab/taxonomy.git for you? [y/N]: y
Cloning https://github.com/instructlab/taxonomy.git...
Generating `config.yaml` in the current directory...
Initialization completed successfully, you're ready to start using `ilab`. Enjoy!

ilab will use the default configuration file unless otherwise specified. You can override this behavior with the --config parameter for any ilab command.

📥 Download the model

Run the ilab model download command.
```
ilab model download
```
ilab model download downloads a compact pre-trained version of the model (~4.4G) from HuggingFace:
```
(venv) $ ilab model download
Downloading model from Hugging Face: instructlab/merlinite-7b-lab-GGUF@main to /home/user/.cache/instructlab/models...
...
INFO 2024-08-01 15:05:48,464 huggingface_hub.file_download:1893: Download complete. Moving file to /home/user/.cache/instructlab/models/merlinite-7b-lab-Q4_K_M.gguf
```
NOTE ⏳ This command can take few minutes or immediately depending on your internet connection or model is cached. If you have issues connecting to Hugging Face, refer to the Hugging Face discussion forum for more details.

Downloading a specific model from a Hugging Face repository
Specify repository, model, and a Hugging Face token if necessary. More information about Hugging Face tokens can be found here
```
HF_TOKEN=<YOUR HUGGINGFACE TOKEN GOES HERE> ilab model download
```