ArchiveBox – ArchiveBox
🗃 开源自托管网页归档。接收 URL/浏览器历史/书签/Pocket/Pinboard 等，保存 HTML、JS、PDF、媒体文件等...
关键指标一览
主题标签
README 详细介绍
ArchiveBox
Open-source self-hosted web archiving.
▶️ Quickstart | Demo | GitHub | Documentation | Info & Motivation | Community
ArchiveBox is a self-hosted app that lets you preserve content from websites in a variety of formats.
We aim to make your data immediately useful, and kept in formats that other programs can read directly. As output, we save standard HTML, PNG, PDF, TXT, JSON, WARC, SQLite, all guaranteed to be readable for decades to come. ArchiveBox also has a CLI, REST API, and webhooks so you can set up integrations with other services.
Without active preservation effort, everything on the internet eventually disappears or degrades.
ArchiveBox is an open source tool that lets organizations & individuals archive both public & private web content while retaining control over their data. It can be used to save copies of bookmarks, preserve evidence for legal cases, backup photos from FB/Insta/Flickr or media from YT/Soundcloud/etc., save research papers, and more...
> ➡️ Get ArchiveBox with uv tool install --python 3.13 --upgrade 'git+https://github.com/ArchiveBox/ArchiveBox.git@dev' on Linux/macOS, or via Docker ⭐️ on any OS.
Once installed, you can interact with it through the: Browser Extension, CLI, self-hosted web interface, Python API, or filesystem.
📥 You can feed ArchiveBox URLs one at a time, or schedule regular imports from your bookmarks or history, social media feeds or RSS, link-saving services like Pocket/Pinboard, our Browser Extension, and more.
See Input Formats for a full list of supported input formats...
It saves snapshots of the URLs you feed it in several redundant formats.
It also detects any content featured inside pages & extracts it out into a folder:
- 🌐 HTML/Any websites ➡️
original HTML+CSS+JS,singlefile HTML,screenshot PNG,PDF,WARC,title,article text,favicon,headers, ... - 🎥 Social Media/News ➡️
post content TXT,comments,title,author,images, ... - 🎬 YouTube/SoundCloud/etc. ➡️
MP3/MP4s,subtitles,metadata,thumbnail, ... - 💾 Github/Gitlab/etc. links ➡️
clone of GIT source code,README,images, ... - ✨ and more, see Output Formats below...
You can run ArchiveBox as a Docker web app to manage these snapshots, or continue accessing the same collection using the uv-installed CLI, Python API, and SQLite3 APIs.
All the ways of using it are equivalent, and provide matching features like adding tags, scheduling regular crawls, viewing logs, and more...
🛠️ ArchiveBox uses standard tools like Chrome, <code class="ra0-md-code">wget</code>, & <code class="ra0-md-code">yt-dlp</code>, and stores data in ordinary files & folders.
(no complex proprietary formats, all data is readable without needing to run ArchiveBox)
The goal is to sleep soundly knowing the part of the internet you care about will be automatically preserved in durable, easily accessible formats for decades after it goes down.
📦 Install ArchiveBox using your preferred method: docker / uv / brew / apt / etc. (see full Quickstart below).
Expand for quick copy-pastable install commands... ⤵️
# Option A: Get ArchiveBox with Docker Compose (recommended):
mkdir -p ~/archivebox/data && cd ~/archivebox
curl -fsSL 'https://docker-compose.archivebox.io' > docker-compose.yml # edit options in this file as-needed
docker compose pull
docker compose run archivebox init
docker compose run archivebox install
docker compose run archivebox manage createsuperuser
docker compose run archivebox add 'https://example.com'
docker compose run archivebox help
docker compose up
Option B: Or use it as a plain Docker container:
mkdir -p ~/archivebox/data && cd ~/archivebox/data
docker run -it -v $PWD:/data archivebox/archivebox:dev init
docker run -it -v $PWD:/data archivebox/archivebox:dev install
docker run -it -v $PWD:/data archivebox/archivebox:dev add 'https://example.com'
docker run -it -v $PWD:/data archivebox/archivebox:dev help
docker run -it -v $PWD:/data -p 8000:8000 archivebox/archivebox:dev
Option C: Or install it with uv (see Quickstart below for apt, brew, and more)
uv tool install --python 3.13 --upgrade 'git+https://github.com/ArchiveBox/ArchiveBox.git@dev'
mkdir -p ~/archivebox/data && cd ~/archivebox/data
archivebox init
archivebox install
archivebox add 'https://example.com'
archivebox help
archivebox server 0.0.0.0:8000
Option D: Or use the optional auto setup script to install it
curl -fsSL 'https://get.archivebox.io' | bash
Open http://web.archivebox.localhost:8000 for the public UI and http://admin.archivebox.localhost:8000 for the admin UI ➡️
Set BASE_URL to change the public base domain. The default auto mode uses web. and admin. subdomains on *.localhost, but one host for ordinary DNS names. BIND_ADDR only controls the local listen address.
Key Features
- Free & open source, own your own data & maintain your privacy by self-hosting
- Powerful CLI with modular dependencies and support for Google Drive/NFS/SMB/S3/B2/etc.
- Comprehensive documentation, active development, and rich community
- Extracts a wide variety of content out-of-the-box: media (yt-dlp), articles (readability), code (git), etc.
- Supports scheduled/realtime importing from many types of sources
- Uses standard, durable, long-term formats like HTML, JSON, PDF, PNG, MP4, TXT, and WARC
- Powerful CLI, self-hosted web UI, Python API (BETA), REST API (ALPHA), or desktop app
- Saves all pages to archive.org as well by default for redundancy (can be disabled for local-only mode)
- Advanced users: support for archiving content requiring login/paywall/cookies (see wiki security caveats!)
- Planned: support for running JS during archiving to adblock, autoscroll, modal-hide, thread-expand
🤝 Professional Integration
ArchiveBox is free for everyone to self-host, but we also provide support, security review, and custom integrations to help NGOs, governments, and other organizations run ArchiveBox professionally:
- Journalists:
crawling during research, preserving cited pages, fact-checking & review
- Lawyers:
collecting & preserving evidence, detecting changes, tagging & review
- Researchers:
analyzing social media trends, getting LLM training data, crawling pipelines
- Individuals:
saving bookmarks, preserving portfolio content, legacy / memoirs archival
- Governments:
snapshotting public service sites, recordkeeping compliance
> Contact us if your org wants help using ArchiveBox professionally.
> We offer: setup & support, CAPTCHA/ratelimit unblocking, SSO, audit logging/chain-of-custody, and more
> ArchiveBox is a 🏛️ 501(c)(3) nonprofit FSP and all our work supports open-source development.
Quickstart
🖥 Supported OSs: Linux/BSD, macOS, Windows (Docker) 👾 CPUs: amd64 (x86_64), arm64, arm7 (raspi>=3)
✳️ Easy Setup
docker-compose (macOS/Linux/Windows) 👈 recommended (click to expand)
👍 Docker Compose is recommended for the easiest install/update UX + best security + all extras out-of-the-box.docker-compose.yml file into a new empty directory (can be anywhere).mkdir -p ~/archivebox/data && cd ~/archivebox
Read and edit docker-compose.yml options as-needed after downloading
curl -fsSL 'https://docker-compose.archivebox.io' > docker-compose.yml
docker compose pull
docker compose run archivebox init
docker compose run archivebox install
docker compose run archivebox manage createsuperuser
docker compose up
completely optional, CLI can always be used without running a server
docker compose run [-T] archivebox [subcommand] [--help]
docker compose run archivebox add 'https://example.com'
docker compose run archivebox help
For more info, see Install: Docker Compose in the Wiki. ➡️
See below for more usage examples using the CLI, Web UI, or filesystem/SQL/Python to manage your archive.
docker run (macOS/Linux/Windows)mkdir -p ~/archivebox/data && cd ~/archivebox/data
docker run -v $PWD:/data -it archivebox/archivebox:dev init
docker run -v $PWD:/data -it archivebox/archivebox:dev installdocker run -v $PWD:/data -p 8000:8000 archivebox/archivebox:dev
completely optional, CLI can always be used without running a server
docker run -v $PWD:/data -it [subcommand] [--help]
docker run -v $PWD:/data -it archivebox/archivebox:dev help
For more info, see Install: Docker Compose in the Wiki. ➡️
See below for more usage examples using the CLI, Web UI, or filesystem/SQL/Python to manage your archive.
bash auto-setup script (macOS/Linux)curl -fsSL 'https://get.archivebox.io' | bash
For more info, see Install: Bare Metal in the Wiki. ➡️
See below for more usage examples using the CLI, Web UI, or filesystem/SQL/Python to manage your archive.
See setup.sh for the source code of the auto-install script.
See "Against curl | sh as an install method" blog post for my thoughts on the shortcomings of this install method.
🛠 Package Manager Setup
uv (macOS/Linux/BSD)
uv.uv tool install --python 3.13 --upgrade 'git+https://github.com/ArchiveBox/ArchiveBox.git@dev'
archivebox version
install any missing extras shown using apt/brew/pkg/etc. see Wiki for instructions
python@3.13 node curl wget git ripgrep ...
See the Install: Bare Metal Wiki for full install instructions for each OS...
mkdir -p ~/archivebox/data && cd ~/archivebox/data # for example
archivebox init # initialize a new collection
archivebox install # install all the runtime dependencies (e.g. chrome, single-file, yt-dlp, etc.)
archivebox manage createsuperuser
archivebox server 0.0.0.0:8000
completely optional, CLI can always be used without running a server
archivebox [subcommand] [--help]
archivebox help
See below for more usage examples using the CLI, Web UI, or filesystem/SQL/Python to manage your archive.
See the uv tool documentation for more details about this installation method.
apt (Ubuntu/Debian/etc.)archivebox.echo 'deb [trusted=yes] https://archivebox.github.io/debian-archivebox dev main' | sudo tee /etc/apt/sources.list.d/archivebox.list
sudo apt update
sudo apt install archivebox
(cd /tmp && archivebox version) # make sure the package is installedmkdir -p ~/archivebox/data
cd ~/archivebox/data
archivebox init
archivebox install
archivebox add 'https://example.com'archivebox manage createsuperuser
archivebox server 0.0.0.0:8000
completely optional, CLI can always be used without running a server
archivebox [subcommand] [--help]
archivebox help
See below for more usage examples using the CLI, Web UI, or filesystem/SQL/Python to manage your archive.
The apt package is a thin dev-channel wrapper around the normal Python install flow. See the debian-archivebox repo for details, and run sudo archivebox install only if you want it to install missing system packages via apt.
brew (macOS and Linux)brew.brew tap archivebox/archivebox
brew trust archivebox/archivebox
brew install archivebox
archivebox version # make sure all dependencies are installed
See the Install: Bare Metal Wiki for more granular instructions for macOS... ➡️mkdir -p ~/archivebox/data && cd ~/archivebox/data
archivebox init
archivebox installarchivebox manage createsuperuser
archivebox server 0.0.0.0:8000
completely optional, CLI can always be used without running a server
archivebox [subcommand] [--help]
archivebox help
See below for more usage examples using the CLI, Web UI, or filesystem/SQL/Python to manage your archive.
See the homebrew-archivebox repo for more details about this distribution.
pacman / 
pkg / 
nix (Arch/FreeBSD/NixOS/more)
> Warning: These are contributed by external volunteers and may lag behind the official uv and Docker channels.
- Arch:
yay -S archivebox(contributed by@imlonghao, maintained by@jasongodev) - FreeBSD:
curl -fsSL 'https://get.archivebox.io' | bash(usespkg+uvunder-the-hood) - Nix:
nix-env --install archivebox(contributed by@siraben) - Guix:
guix install archivebox(contributed by@rakino) - More: contribute another distribution...!
See below for usage examples using the CLI, Web UI, or filesystem/SQL/Python to manage your archive.
🎗 Other Options
docker + 
electron Desktop App (macOS/Linux/Windows)ArchiveBox.app.zipArchiveBox.deb (alpha: build manually)ArchiveBox.exe (beta: build manually)
✨ Alpha (contributors wanted!): for more info, see the: Electron ArchiveBox repo.
TrueNAS / UNRAID / YunoHost / Cloudron / etc. (self-hosting solutions)
> Warning: These are contributed by external volunteers and may lag behind the official pip channel.
TrueNAS: Official ArchiveBox TrueChart / Custom App Guide(TrueCharts is discontinued, wait for Electric Eel)- UnRaid
- Proxmox
- Yunohost
- Cloudron
- Saltbox
- Portainer
- AppImage
- Runtipi
- Umbrel (need contributors...)
- More: contribute another distribution...!
See below for usage examples using the CLI, Web UI, or filesystem/SQL/Python to manage your archive.
Paid hosting solutions (cloud VPS)
(get hosting, support, and feature customization directly from us)
(generalist consultancy that has ArchiveBox experience)
Other providers of paid ArchiveBox hosting (not officially endorsed):
(USD $29-250/mo, pricing)
(from USD $2.6/mo)
(USD $5-50+/mo, 🎗 referral link, instructions)
(USD $2.5-50+/mo, 🎗 referral link, instructions)
(USD $10-50+/mo, instructions)
(USD $0-5+/mo)
(USD $60-200+/mo)
(USD $60-200+/mo)
Referral links marked 🎗 provide $5-10 of free credit for new users and help pay for our demo server hosting costs.
For more discussion on managed and paid hosting options see here: Issue #531.
➡️ Next Steps
- Import URLs from some of the supported Input Formats or view the supported Output Formats...
- (Optional) Create a persona and import browser cookies to archive logged-in sites:
archivebox persona create --import=chrome personal - Tweak your UI or archiving behavior Configuration, read about some of the Caveats, or Troubleshoot
- Read about the Dependencies used for archiving, the Upgrading Process, or the Archive Layout on disk...
- Or check out our full Documentation or Community Wiki...
Usage
⚡️ CLI Usage
ArchiveBox commands can be run in a terminal directly on your host, or via Docker/Docker Compose.
(depending on how you chose to install it above)
mkdir -p ~/archivebox/data # create a new data dir anywhere
cd ~/archivebox/data # IMPORTANT: cd into the directory
# archivebox [subcommand] [--help]
archivebox version
archivebox help
# equivalent: docker compose run archivebox [subcommand] [--help]
docker compose run archivebox help
# equivalent: docker run -it -v $PWD:/data archivebox/archivebox:dev [subcommand] [--help]
docker run -it -v $PWD:/data archivebox/archivebox:dev help
# optional: import your browser cookies into a persona for logged-in archiving
archivebox persona create --import=chrome personal
# supported: chrome/chromium/brave/edge (Chromium-based only)
# use --profile to target a specific profile (e.g. Default, Profile 1)
# re-running import merges/dedupes cookies.txt (by domain/path/name) but replaces chrome_user_dataArchiveBox Subcommands
archiveboxhelp/versionto see the list of available subcommands / currently installed version infoarchiveboxsetup/init/config/status/shell/manageto administer your collectionarchiveboxadd/scheduleto pull in fresh URLs from bookmarks/history/RSS/etc.archiveboxlist/update/removeto manage existing Snapshots in your collection
CLI Usage Examples: non-Docker
make sure you have uv-installed ArchiveBox and it's available in your $PATH first
archivebox [subcommand] [--help]
mkdir -p ~/archivebox/data && cd ~/archivebox/data
archivebox init
archivebox install
archivebox add 'https://example.com'
archivebox version # get archivebox version info + check dependencies
archivebox help # get list of archivebox subcommands that can be run
For more info, see our Usage: CLI Usage wiki. ➡️
CLI Usage Examples: Docker Compose
make sure you have docker-compose.yml from the Quickstart instructions first
docker compose run archivebox [subcommand] [--help]
docker compose run archivebox init
docker compose run archivebox install
docker compose run archivebox version
docker compose run archivebox help
docker compose run archivebox add 'https://example.com'
to start webserver: docker compose up
For more info, see our Usage: Docker Compose CLI wiki. ➡️
CLI Usage Examples: Docker
make sure you create and cd into in a new empty directory first
docker run -it -v $PWD:/data archivebox/archivebox:dev [subcommand] [--help]
docker run -v $PWD:/data -it archivebox/archivebox:dev init
docker run -v $PWD:/data -it archivebox/archivebox:dev install
docker run -v $PWD:/data -it archivebox/archivebox:dev version
docker run -v $PWD:/data -it archivebox/archivebox:dev help
docker run -v $PWD:/data -it archivebox/archivebox:dev add 'https://example.com'
to start webserver: docker run -v $PWD:/data -it -p 8000:8000 archivebox/archivebox:dev
For more info, see our Usage: Docker CLI wiki. ➡️
🗄 SQL/Python/Filesystem Usage
archivebox shell # explore the Python library API in a REPL
sqlite3 ./index.sqlite3 # run SQL queries directly on your index
find ./archive/users -path '/snapshots////index.html' # inspect snapshot data directly
For more info, see our Python Shell, SQL API, and Disk Layout wikis. ➡️
🖥 Web UI & API Usage
Start the server on bare metal (pip/apt/brew/etc):
archivebox manage createsuperuser # create a new admin user via CLI
archivebox server 0.0.0.0:8000 # start the server
Or with Docker Compose:
nano docker-compose.yml # setup initial ADMIN_USERNAME & ADMIN_PASSWORD
docker compose up # start the server
Or with a Docker container:
docker run -v $PWD:/data -it archivebox/archivebox:dev manage createsuperuser
docker run -v $PWD:/data -it -p 8000:8000 archivebox/archivebox:dev
Open http://web.archivebox.localhost:8000 for the public UI and http://admin.archivebox.localhost:8000 for the admin UI ➡️
Set BASE_URL to change the public base domain. The default auto mode uses web. and admin. subdomains on *.localhost, but one host for ordinary DNS names. BIND_ADDR only controls the local listen address.
For more info, see our Usage: Web UI wiki. ➡️
Optional: Change permissions to allow non-logged-in users
archivebox config --set PUBLIC_ADD_VIEW=True # allow guests to submit URLs
archivebox config --set PERMISSIONS=public # make newly added snapshots public
archivebox config --set PUBLIC_INDEX=True # allow guests to see list of all snapshots
or
docker compose run archivebox config --set PERMISSIONS=public
restart the server to apply any config changes
> [!TIP]
> Whether in Docker or not, ArchiveBox commands work the same way, and can be used to access the same data on-disk.
> For example, you could run the Web UI in Docker Compose, and run one-off commands with uv-installed ArchiveBox.
Expand to show comparison...
archivebox add --depth=1 'https://example.com' # add a URL with uv-installed archivebox on the host
docker compose run archivebox add --depth=1 'https://example.com' # or w/ Docker Compose
docker run -it -v $PWD:/data archivebox/archivebox:dev add --depth=1 'https://example.com' # or w/ Docker, all equivalent
For more info, see our Docker wiki. ➡️
. . . . . . . . . . . . . . . . . . . . . . . . . . . .
DEMO:
https://demo.archivebox.ioUsage | Configuration | Caveats
Overview
Input Formats: How to pass URLs into ArchiveBox for saving
From the official ArchiveBox Browser Extension
Provides realtime archiving of browsing history or selected pages from Chrome/Chromium/Firefox browsers.
From manual imports of URLs from RSS, JSON, CSV, TXT, SQL, HTML, Markdown, etc. files
ArchiveBox supports injesting URLs in any text-based format.
From manually exported browser history or browser bookmarks (in Netscape format)
Instructions: Chrome, Firefox, Safari, IE, Opera, and more...
From URLs visited through a MITM Proxy with <code class="ra0-md-code">archivebox-proxy</code>
Provides realtime archiving of all traffic from any device going through the proxy.
From bookmarking services or social media (e.g. Twitter bookmarks, Reddit saved posts, etc.)
Instructions: Pocket, Pinboard, Instapaper, Shaarli, Delicious, Reddit Saved, Wallabag, Unmark.it, OneTab, Firefox Sync, and more...
# archivebox add --help
archivebox add 'https://example.com/some/page'
archivebox add --depth=1 --plugins=parse_rss_urls "file://$HOME/Downloads/some_feed.xml"
archivebox add --depth=1 'https://news.ycombinator.com#2020-12-12'
echo 'http://example.com' | archivebox add
echo 'any text with <a href="https://example.com">urls</a> in it' | archivebox add
# if using Docker, add -i when piping stdin:
# echo 'https://example.com' | docker run -v $PWD:/data -i archivebox/archivebox:dev add
# if using Docker Compose, add -T when piping stdin / stdout:
# echo 'https://example.com' | docker compose run -T archivebox addSee the Usage: CLI page for documentation and examples.
It also includes a built-in scheduled import feature with archivebox schedule, handled by the same orchestrator that powers archivebox server, so you can pull in URLs from RSS feeds and websites regularly without a separate cron container.
Output Formats: What ArchiveBox saves for each URL
For each web page added, ArchiveBox creates a Snapshot folder and preserves its content as ordinary files inside the folder (e.g. HTML, PDF, PNG, JSON, etc.).
It uses all available methods out-of-the-box, but you can disable extractors and fine-tune the configuration as-needed.
Expand to see the full list of ways it saves each page...
data/archive/{Snapshot.id}/
- Index:
index.html&index.jsonHTML and JSON index files containing metadata and details - Title, Favicon, Headers Response headers, site favicon, and parsed site title
- SingleFile:
singlefile.htmlHTML snapshot rendered with headless Chrome using SingleFile - Wget Clone:
example.com/page-name.htmlwget clone of the site withwarc/TIMESTAMP.gz - Chrome Headless
- PDF:
output.pdfPrinted PDF of site using headless chrome - Screenshot:
screenshot.png1440x900 screenshot of site using headless chrome - DOM Dump:
output.htmlDOM Dump of the HTML after rendering using headless chrome
- PDF:
- Article Text:
article.html/jsonArticle text extraction using Readability & Mercury - Archive.org Permalink:
archive.org.txtA link to the saved site on archive.org - Audio & Video:
media/all audio/video files + playlists, including subtitles & metadata w/yt-dlp - Source Code:
git/clone of any repository found on GitHub, Bitbucket, or GitLab links - More coming soon! See the Roadmap...
