orch

orch是一个用于构建基于语言模型的应用程序和代理的Rust编程语言库。它主要为magic-cli项目而构建，但也可以在其他场景中使用。

[!注意]

如果该项目获得关注，它可以被编译为其他语言（如Python）的插件或独立的WebAssembly模块。

安装

cargo add orch
cargo add orch_response

或者，将orch作为依赖项添加到你的Cargo.toml文件中：

[dependencies]
orch = "*" # 替换为最新版本
orch_response = "*" # 替换为最新版本

基本用法

简单文本生成

use orch::execution::*;
use orch::lm::*;

#[tokio::main]
async fn main() {
  let lm = OllamaBuilder::new().try_build().unwrap();
  let executor = TextExecutorBuilder::new().with_lm(&lm).try_build().unwrap();
  let response = executor.execute("2+2等于多少？").await.expect("执行失败");
  println!("{}", response.content);
}

流式文本生成

use orch::execution::*;
use orch::lm::*;
use tokio_stream::StreamExt;

#[tokio::main]
async fn main() {
  let lm = OllamaBuilder::new().try_build().unwrap();
  let executor = TextExecutorBuilder::new().with_lm(&lm).try_build().unwrap();
  let mut response = executor.execute_stream("2+2等于多少？").await.expect("执行失败");
  while let Some(chunk) = response.stream.next().await {
    match chunk {
      Ok(chunk) => print!("{chunk}"),
      Err(e) => {
        println!("错误：{e}");
        break;
      }
    }
  }
  println!();
}

结构化数据生成

use orch::execution::*;
use orch::lm::*;
use orch::response::*;

#[derive(Variants, serde::Deserialize)]
pub enum ResponseVariants {
    Answer(AnswerResponseVariant),
    Fail(FailResponseVariant),
}

#[derive(Variant, serde::Deserialize)]
#[variant(
    variant = "Answer",
    scenario = "你知道该国家的首都",
    description = "国家的首都"
)]
pub struct AnswerResponseVariant {
    #[schema(
        description = "收到的国家的首都",
        example = "伦敦"
    )]
    pub capital: String,
}

#[derive(Variant, serde::Deserialize)]
#[variant(
    variant = "Fail",
    scenario = "你不知道该国家的首都",
    description = "不知道首都的原因"
)]
pub struct FailResponseVariant {
    #[schema(
        description = "不知道首都的原因",
        example = "'foobar'国家不存在"
    )]
    pub reason: String,
}

#[tokio::main]
async fn main() {
    let lm = OllamaBuilder::new().try_build().unwrap();
    let executor = StructuredExecutorBuilder::new()
    .with_lm(&lm)
    .with_preamble("你是一位地理专家，帮助用户了解世界各国的首都。")
		.with_options(Box::new(variants!(ResponseVariants)))
    .try_build()
    .unwrap();
    let response = executor
        .execute("傻瓜国的首都是什么？")
        .await
        .expect("执行失败");

    println!("回应：");
    match response.content {
        ResponseVariants::Answer(answer) => {
            println!("首都：{}", answer.capital);
        }
        ResponseVariants::Fail(fail) => {
            println!("模型无法生成回应：{}", fail.reason);
        }
    }
}

嵌入生成

use orch::execution::*;
use orch::lm::*;

#[tokio::main]
async fn main() {
  let lm = OllamaBuilder::new().try_build().unwrap();
  let executor = TextExecutorBuilder::new()
    .with_lm(&lm)
    .try_build()
    .unwrap();
  let embedding = executor
    .generate_embedding("用于生成嵌入的短语")
    .await
    .expect("执行失败");

  println!("嵌入：");
  println!("{:?}", embedding);
}

更多示例

查看examples目录以获取使用示例。