<div align="center" style="text-align: center; width: 100%"> <img src="https://archivebox.io/icon.png" height="90px"/> <h1>ArchiveBox<br/>_{Open-source self-hosted web archiving.}</h1> <br/>

▶️ <a href="https://github.com/ArchiveBox/ArchiveBox/wiki/Quickstart">Quickstart</a> | <a href="https://demo.archivebox.io">Demo</a> | <a href="https://github.com/ArchiveBox/ArchiveBox">GitHub</a> | <a href="https://github.com/ArchiveBox/ArchiveBox/wiki">Documentation</a> | <a href="#background--motivation">Info & Motivation</a> | <a href="https://github.com/ArchiveBox/ArchiveBox/wiki/Web-Archiving-Community">Community</a>

<br/> <!--<a href="http://webchat.freenode.net?channels=ArchiveBox&uio=d4"><img src="https://img.shields.io/badge/Community_chat-IRC-%2328A745.svg"/></a>-->

<a href="https://github.com/ArchiveBox/ArchiveBox/blob/dev/LICENSE"><img src="https://img.shields.io/badge/Open_source-MIT-green.svg?logo=git&logoColor=green"/></a> <a href="https://github.com/ArchiveBox/ArchiveBox/commits/dev"><img src="https://img.shields.io/github/last-commit/ArchiveBox/ArchiveBox.svg?logo=Sublime+Text&logoColor=green&label=Active"/></a>   <a href="https://github.com/ArchiveBox/ArchiveBox"><img src="https://img.shields.io/github/stars/ArchiveBox/ArchiveBox.svg?logo=github&label=Stars&logoColor=blue"/></a>   <a href="https://hub.docker.com/r/archivebox/archivebox"><img src="https://img.shields.io/docker/pulls/archivebox/archivebox.svg?label=Docker+Pulls"/></a> <a href="https://pypi.org/project/archivebox/"><img src="https://img.shields.io/pypi/dm/archivebox?label=PyPI%20Installs&color=%235f7dae"/></a> <a href="https://chromewebstore.google.com/detail/archivebox-exporter/habonpimjphpdnmcfkaockjnffodikoj"><img src="https://img.shields.io/chrome-web-store/users/habonpimjphpdnmcfkaockjnffodikoj?label=Chrome%20Store&color=%231973e8"/></a>

<!--<pre lang="bash" align="left"><code style="white-space: pre-line; text-align: left" align="left"> curl -fsSL 'https://get.archivebox.io' | sh # (or see pip/brew/Docker instructions below) </code></pre>--> </div> <hr/> <br/>

ArchiveBox is a powerful, self-hosted internet archiving solution to collect, save, and view websites offline.

Without active preservation effort, everything on the internet eventually dissapears or degrades. Archive.org does a great job as a centralized service, but saved URLs have to be public, and they can't save every type of content.

ArchiveBox is an open source tool that lets organizations & individuals archive both public & private web content while retaining control over their data. It can be used to save copies of bookmarks, preserve evidence for legal cases, backup photos from FB/Insta/Flickr or media from YT/Soundcloud/etc., save research papers, and more... <br/>

➡️ Get ArchiveBox with pip install archivebox on Linux, macOS, and Windows (WSL2), or via Docker ⭐️.

Once installed, it can be used as a CLI tool, self-hosted Web App, Python library, or one-off command.

<br/> <hr/> <br/>

📥 You can feed ArchiveBox URLs one at a time, or schedule regular imports from your bookmarks or history, social media feeds or RSS, link-saving services like Pocket/Pinboard, our Browser Extension, and more.
_{See <a href="#input-formats">Input Formats</a> for a full list of supported input formats...}

<br/> <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/90f1ce3c-75bb-401d-88ed-6297694b76ae" alt="snapshot detail page" align="right" width="190px" style="float: right"/>

It saves snapshots of the URLs you feed it in several redundant formats.
It also detects any content featured inside pages & extracts it out into a folder:

• 🌐 HTML/Any websites ➡️ original HTML+CSS+JS, singlefile HTML, screenshot PNG, PDF, WARC, title, article text, favicon, headers, ...
• 🎥 Social Media/News ➡️ post content TXT, comments, title, author, images, ...
• 🎬 YouTube/SoundCloud/etc. ➡️ MP3/MP4s, subtitles, metadata, thumbnail, ...
• 💾 Github/Gitlab/etc. links ➡️ clone of GIT source code, README, images, ...
• ✨ and more, see Output Formats below...

You can run ArchiveBox as a Docker web app to manage these snapshots, or continue accessing the same collection using the pip-installed CLI, Python API, and SQLite3 APIs. All the ways of using it are equivalent, and provide matching features like adding tags, scheduling regular crawls, viewing logs, and more...

<br/> <hr/>

🛠️ ArchiveBox uses standard tools like Chrome, wget, & yt-dlp, and stores data in ordinary files & folders.
(no complex proprietary formats, all data is readable without needing to run ArchiveBox)

The goal is to sleep soundly knowing the part of the internet you care about will be automatically preserved in durable, easily accessible formats for decades after it goes down.

<hr/> <br/>

📦  Install ArchiveBox using your preferred method: docker / pip / apt / etc. (see full Quickstart below).

<details> &nbsp; <summary><i>Expand for quick copy-pastable install commands...</i> &nbsp; ⤵️</summary> <br/> <pre lang="bash"><code style="white-space: pre-line"># Option A: Get ArchiveBox with Docker Compose (recommended): mkdir -p ~/archivebox/data && cd ~/archivebox curl -fsSL 'https://docker-compose.archivebox.io' > docker-compose.yml # edit options in this file as-needed docker compose run archivebox init --setup # docker compose run archivebox add 'https://example.com' # docker compose run archivebox help # docker compose up <br/> <br/> # Option B: Or use it as a plain Docker container: mkdir -p ~/archivebox/data && cd ~/archivebox/data docker run -it -v $PWD:/data archivebox/archivebox init --setup # docker run -it -v $PWD:/data archivebox/archivebox add 'https://example.com' # docker run -it -v $PWD:/data archivebox/archivebox help # docker run -it -v $PWD:/data -p 8000:8000 archivebox/archivebox <br/> <br/> # Option C: Or install it with your preferred pkg manager (see Quickstart below for apt, brew, and more) pip install archivebox mkdir -p ~/archivebox/data && cd ~/archivebox/data archivebox init --setup # archivebox add 'https://example.com' # archivebox help # archivebox server 0.0.0.0:8000 <br/> <br/> # Option D: Or use the optional auto setup script to install it curl -fsSL 'https://get.archivebox.io' | sh </code></pre> <br/> _{Open <a href="http://localhost:8000"><code>http://localhost:8000</code></a> to see your server's Web UI ➡️} </details> <br/> <div align="center" style="text-align: center"> <br/><br/> <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/5a7d95f2-6977-4de6-9f08-42851a1fe1d2" height="70px" alt="bookshelf graphic"> &nbsp; <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/b2765a33-0d1e-4019-a1db-920c7e00e20e" height="75px" alt="logo" align="top"/> &nbsp; <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/5a7d95f2-6977-4de6-9f08-42851a1fe1d2" height="70px" alt="bookshelf graphic"> <br/><br/> <small><a href="https://demo.archivebox.io">Demo</a> | <a href="#screenshots">Screenshots</a> | <a href="https://github.com/ArchiveBox/ArchiveBox/wiki/Usage">Usage</a></small> <br/> _{. . . . . . . . . . . . . . . . . . . . . . . . . . . .} <br/><br/> <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/8d67382c-e0ce-4286-89f7-7915f09b930c" width="22%" alt="cli init screenshot" align="top"> <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/dad2bc51-e7e5-484e-bb26-f956ed692d16" width="22%" alt="cli init screenshot" align="top"> <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/e8e0b6f8-8fdf-4b7f-8124-c10d8699bdb2" width="22%" alt="server snapshot admin screenshot" align="top"> <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/ace0954a-ddac-4520-9d18-1c77b1ec50b2" width="28.6%" alt="server snapshot details page screenshot" align="top"/> <br/><br/> </div>

Key Features

• Free & open source, own your own data & maintain your privacy by self-hosting
• Powerful CLI with modular dependencies and support for Google Drive/NFS/SMB/S3/B2/etc.
• Comprehensive documentation, active development, and rich community
• Extracts a wide variety of content out-of-the-box: media (yt-dlp), articles (readability), code (git), etc.
• Supports scheduled/realtime importing from many types of sources
• Uses standard, durable, long-term formats like HTML, JSON, PDF, PNG, MP4, TXT, and WARC
• Usable as a oneshot CLI, self-hosted web UI, Python API (BETA), REST API (ALPHA), or desktop app
• Saves all pages to archive.org as well by default for redundancy (can be disabled for local-only mode)
• Advanced users: support for archiving content requiring login/paywall/cookies (see wiki security caveats!)
• Planned: support for running JS during archiving to adblock, autoscroll, modal-hide, thread-expand

<br/>

🤝 Professional Integration

ArchiveBox is free for everyone to self-host, but we also provide support, security review, and custom integrations to help NGOs, governments, and other organizations run ArchiveBox professionally:

• Journalists: crawling during research, preserving cited pages, fact-checking & review
• Lawyers: collecting & preserving evidence, detecting changes, tagging & review
• Researchers: analyzing social media trends, getting LLM training data, crawling pipelines
• Individuals: saving bookmarks, preserving portfolio content, legacy / memoirs archival
• Governments: snapshoting public service sites, recordkeeping compliance

Contact us if your org wants help using ArchiveBox professionally. (we are also seeking grant funding)
We offer: setup & support, CAPTCHA/ratelimit unblocking, SSO, audit logging/chain-of-custody, and more
ArchiveBox is a 🏛️ 501(c)(3) nonprofit FSP and all our work supports open-source development.

<br/> <div align="center" style="text-align: center"> <br/> <img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/0db52ea7-4a2c-441d-b47f-5553a5d8fe96" width="49%" alt="grass"/><img src="https://github.com/ArchiveBox/ArchiveBox/assets/511499/0db52ea7-4a2c-441d-b47f-5553a5d8fe96" width="49%" alt="grass"/> </div>

<a name="install"></a>

Quickstart

🖥  Supported OSs: Linux/BSD, macOS, Windows (Docker)   👾  CPUs: amd64 (x86_64), arm64, arm7 ^(raspi>=3)<br/>

<br/>

✳️  Easy Setup

<details> <summary><b><img src="https://user-images.githubusercontent.com/511499/117447182-29758200-af0b-11eb-97bd-58723fee62ab.png" alt="Docker" height="28px" align="top"/> <code>docker-compose</code></b> (macOS/Linux/Windows) &nbsp; <b>👈&nbsp; recommended</b> &nbsp; <i>(click to expand)</i></summary> <br/> <i>👍 Docker Compose is recommended for the easiest install/update UX + best security + all <a href="#dependencies">extras</a> out-of-the-box.</i> <br/><br/> <ol> <li>Install <a href="https://docs.docker.com/get-docker/">Docker</a> on your system (if not already installed).</li> <li>Download the <a href="https://raw.githubusercontent.com/ArchiveBox/ArchiveBox/dev/docker-compose.yml" download><code>docker-compose.yml</code></a> file into a new empty directory (can be anywhere). <pre lang="bash"><code style="white-space: pre-line">mkdir -p ~/archivebox/data && cd ~/archivebox # Read and edit docker-compose.yml options as-needed after downloading curl -fsSL 'https://docker-compose.archivebox.io' > docker-compose.yml </code></pre></li> <li>Run the initial setup to create an admin user (or set ADMIN_USER/PASS in docker-compose.yml) <pre lang="bash"><code style="white-space: pre-line">docker compose run archivebox