Anserini <img src="docs/anserini-logo.png" width="300" />

Anserini is a toolkit for reproducible information retrieval research. By building on Lucene, we aim to bridge the gap between academic information retrieval research and the practice of building real-world search applications. Among other goals, our effort aims to be the opposite of this.* Anserini grew out of a reproducibility study of various open-source retrieval engines in 2016 (Lin et al., ECIR 2016). See Yang et al. (SIGIR 2017) and Yang et al. (JDIQ 2018) for overviews.

❗ Anserini was upgraded from JDK 11 to JDK 21 at commit 272565 (2024/04/03), which corresponds to the release of v0.35.0.

💥 Try It!

Anserini is packaged in a self-contained fatjar, which also provides the simplest way to get started. Assuming you've already got Java installed, fetch the fatjar:

wget https://repo1.maven.org/maven2/io/anserini/anserini/0.36.1/anserini-0.36.1-fatjar.jar

The follow commands will generate a SPLADE++ ED run with the dev queries (encoded using ONNX) on the MS MARCO passage corpus:

java -cp anserini-0.36.1-fatjar.jar io.anserini.search.SearchCollection \
  -index msmarco-v1-passage.splade-pp-ed \
  -topics msmarco-v1-passage.dev \
  -encoder SpladePlusPlusEnsembleDistil \
  -output run.msmarco-v1-passage-dev.splade-pp-ed-onnx.txt \
  -impact -pretokenized

To evaluate:

java -cp anserini-0.36.1-fatjar.jar trec_eval -c -M 10 -m recip_rank msmarco-passage.dev-subset run.msmarco-v1-passage-dev.splade-pp-ed-onnx.txt

See detailed instructions for the current fatjar release of Anserini (v0.36.1) to reproduce regression experiments on the MS MARCO V2.1 corpora for TREC 2024 RAG, on MS MARCO V1 Passage, and on BEIR, all directly from the fatjar!

<details> <summary>Older instructions</summary>

</details>

🎬 Installation

Most Anserini features are exposed in the Pyserini Python interface. If you're more comfortable with Python, start there, although Anserini forms an important building block of Pyserini, so it remains worthwhile to learn about Anserini.

You'll need Java 21 and Maven 3.9+ to build Anserini. Clone our repo with the --recurse-submodules option to make sure the eval/ submodule also gets cloned (alternatively, use git submodule update --init). Then, build using Maven:

mvn clean package

The tools/ directory, which contains evaluation tools and other scripts, is actually this repo, integrated as a Git submodule (so that it can be shared across related projects). Build as follows (you might get warnings, but okay to ignore):

cd tools/eval && tar xvfz trec_eval.9.0.4.tar.gz && cd trec_eval.9.0.4 && make && cd ../../..
cd tools/eval/ndeval && make && cd ../../..

With that, you should be ready to go. The onboarding path for Anserini starts here!

<details> <summary>Windows tips</summary>

If you are using Windows, please use WSL2 to build Anserini. Please refer to the WSL2 Installation document to install WSL2 if you haven't already.

Note that on Windows without WSL2, tests may fail due to encoding issues, see #1466. A simple workaround is to skip tests by adding -Dmaven.test.skip=true to the above mvn command. See #1121 for additional discussions on debugging Windows build errors.

</details>

⚗️ End-to-End Regression Experiments

Anserini is designed to support end-to-end experiments on various standard IR test collections out of the box. Each of these end-to-end regressions starts from the raw corpus, builds the necessary index, performs retrieval runs, and generates evaluation results. See individual pages for details.

<details> <summary>MS MARCO V1 Passage Regressions</summary>

MS MARCO V1 Passage Regressions

	dev	DL19	DL20
Unsupervised Sparse
Lucene BoW baselines	🔑	🔑	🔑
Quantized BM25	🔑	🔑	🔑
WordPiece baselines (pre-tokenized)	🔑	🔑	🔑
WordPiece baselines (Huggingface)	🔑	🔑	🔑
WordPiece + Lucene BoW baselines	🔑	🔑	🔑
doc2query	🔑
doc2query-T5	🔑	🔑	🔑
Learned Sparse (uniCOIL family)
uniCOIL noexp	🫙	🫙	🫙
uniCOIL with doc2query-T5	🫙	🫙	🫙
uniCOIL with TILDE	🫙
Learned Sparse (other)
DeepImpact	🫙
SPLADEv2	🫙
SPLADE++ CoCondenser-EnsembleDistil	🫙🅾️	🫙🅾️	🫙🅾️
SPLADE++ CoCondenser-SelfDistil	🫙🅾️	🫙🅾️	🫙🅾️
Learned Dense (HNSW indexes)
cosDPR-distil	full:🫙🅾️ int8:🫙🅾️	full:🫙🅾️ int8:🫙🅾️	full:🫙🅾️ int8:🫙🅾️
BGE-base-en-v1.5	full:🫙🅾️ int8:🫙🅾️	full:🫙🅾️ int8:🫙🅾️	full:🫙🅾️