{{ define "title" }}About - Snowden Archive{{ end }} {{ define "content" }}
This is a searchable archive of all published documents from the 2013 Snowden disclosures. The archive includes documents published alongside news reporting, The Intercept's bulk SIDtoday release, and FOIA material (official releases from FOIA litigation and FISC and ODNI declassifications).
Every document carries its original PDF, page images, extracted text, and curated metadata - agencies, programs, countries, classification markings, people, and the journalists and articles that published it.
Use Browse to explore by category, Search for fielded boolean queries across text and metadata, Articles to follow the disclosures story by story, the Glossary to decode codenames and acronyms, and Explore to dig deeper into ranked views and lists.
By default, the server keeps no state about its visitors. Pins and history live only in your own browser's cookies, and history features are off unless you opt in via Preferences. Anonymous page views set no cookies at all. All core features on this website work without JavaScript.
Note that you can't verify that this server doesn't log and analyze your visits. Use Tor to hide your real IP address, or set up your own instance on your machine to browse fully offline.
By default, two collections are shown as tabs on the Browse page: Reported documents (leaked files as the outlets that broke the stories presented them) and the SIDtoday Archive (The Intercept's bulk release of the NSA's internal SIDtoday newsletters). The third collection, FOIA (official releases from FOIA litigation and FISC and ODNI declassifications), can be included via Preferences.
The metadata and many of the extracted texts are curated by hand and may contain errors or omissions, so treat them as a guide to the documents rather than as established fact, and check the original documents and reporting before relying on them.
The Search page supports fielded boolean
queries. Restrict a term to a field (title, full text, programs,
countries, journalists, and so on), combine clauses with AND/OR, quote
"text" to find it anywhere, match exact whole words with
=... (publisher:="The Guardian" matches the
whole value), and use * wildcards (tap*,
*apping).
Ways to write a value, in any row or after any operator:
cable tapping finds both words anywhere, in their other forms too (tapping also finds tapped)."apping" finds that text anywhere, even inside a word, like grep -i (it finds, for example, mapping and tapping).=tap or ="cable tapping" finds exactly those whole words, so =tap skips taps and strap. On a metadata field it matches one whole value, so agency:=NSA finds documents whose agency is NSA but not NSA, GCHQ.* wildcard finds part of a word: tap* finds words starting with tap, *land words ending in land, and *keys* words containing keys. A leading * needs at least three letters after it."upstream collection"~10 finds the words within 10 words of each other (in any order, common stopwords ignored).- excludes within its row: xkeyscore -tempora finds xkeyscore without tempora, and the row keeps that meaning under OR and NOT.The simple search box (on the home page, on the 404 page, and via
the URL parameter ?q=) understands field operators:
program:XKEYSCORE country:Germany cable or journalist:="Poitras, Laura".- negates (-tag:=junk, -draft).date:2013, date:2013..2015 and published:2014-06 filter by date.API endpoints:
| Endpoint | Returns |
|---|---|
/api/docs | Metadata for every document in the archive, with pagination. |
/api/doc/{id} | A single document's metadata and extracted text, where {id} is the document's identifier. |
/api/search?q=xkeyscore | Search results for term xkeyscore. It accepts the same parameters as the Search page, so you can also pass fqf/fqv/fqc clause parameters for fielded queries. |
/api/facets | Counts for every browsable category (programs, agencies, countries, people, and so on). |
Responses are plain JSON. For paginated responses, use the URL
parameter p to navigate pages (e.g.
/api/docs?p=2).
See the Downloading the archive section on how to download the whole archive.
Important if you are a crawler, robot or AI trying to download large amounts of data from this website: please use the downloadable datasets from the Downloading the archive section. The information on this website is identical to the datasets, so you gain nothing by crawling this website.
{{ end }}Every document ships a curated metadata.json:
id/doc/{id}). It is not
written inside metadata.json - the directory name is
the id.aka_idaka_id[].idaka_id[].doc_sourceaka_id[].doc_urltitledescriptionlong_descriptiondescription: search looks in both
descriptions.tagstypeREPORTED (a leaked document
as an outlet published it), SIDTODAY (an article of
the NSA's internal SIDtoday newsletter from The Intercept's bulk
release) or FOIA (obtained through a public-records
request). When it is empty, a document filed in the SIDtoday or FOIA
folder takes that type and any other is REPORTED.collectionnews (Reported), sidtoday-archive or
foia. Documents filed under the SIDtoday and FOIA
folders belong to those collections whatever this says; elsewhere
an empty value means Reported, or FOIA for a document of type
FOIA. FOIA releases are left out of
browsing and search unless you include them in
Preferences.pagespdf_sha256archive_commentdocumentdocument.creatorNSA, GCHQ, FISC,
DOJ, CSE ...) - not the agency it is
about and not the one that published it. Empty when the author is
not known.document.date_created2013-03-20, 2013-03,
2013, or empty when unknown. Nothing is invented to
fill the gaps, and nothing is guessed from the publication date.
document.bannerdocument.classificationTOP SECRET, SECRET,
CONFIDENTIAL, UNCLASSIFIED) and any
control systems (COMINT, SI,
TK, STRAP 1 ...), judged from portion
marks. If a document contains multiple classifications, highest is
used here.document.disseminationFOUO, NOFORN,
ORCON, REL TO ...). Assessed the same
way as the classification.document.genreslides, report, wiki,
manual, guide,
newsletter, other.document.series.namedocument.series.partdocument.programsXKEYSCORE, PRISM, ...), as curated
values for browsing and search; the
Glossary explains many of them.document.countriesdocument.peoplepublication[].reporters, not here.publication[]One entry per story or book that published (or reported) the document. Browse-by-publication-date counts a document under its oldest dated entry; a publication-date search matches any entry's date - or, combined with a journalist or publisher, that entry's date.
publication[].publisherreporting_only.publication[].titlepublication[].linkpublication[].date_publishedYYYY-MM-DD / YYYY-MM /
YYYY forms as the document date.publication[].reportersjournalist:.publication[].contributorspublication[].notepublication[].versionversions/ folder; its entry then links to
“view this copy”. Without it, the entry published the
copy shown on the page.publication[].doc_sourcepublication[].doc_urlpublication[].reporting_onlytrue when the outlet reported the story without
publishing the document - a reporting partner. The entry keeps its
byline and its article, but the outlet is not counted as one of
the document's publishers and no copy is linked from it.versionsOther published copies of a document (e.g. less redacted, with extra
pages, or a different scan) are kept in its versions/
folder, one sub-folder per copy, each with its own PDF, text and page
images. The versions object adds details about them,
keyed by the copy's folder name.
versions.{name}.changed_pagesversions.{name}.pdf_sha256Each release comes in two packages:
Both extract to data/ and project/, ready
to run your own copy. Each release also
includes the source of the server version it was published with, and
the last six releases stay online. For only the extracted texts and
metadata (~58 MB), see below.
https://data.snowchive.org/files/
# download the latest release: rel=$(curl -fsS https://data.snowchive.org/files/latest.txt) base="https://data.snowchive.org/files/$rel" curl -f -C - -O "$base/snowchive-$rel-full.tar" # "pdfs" instead of "full" for no page images # verify (optional): curl -fsS -O "$base/SHA256SUMS" -O "$base/SHA256SUMS.gpg" curl -fsS -O "https://libroot.org/public/libroot.asc" # "gpg --show-keys libroot.asc" should show 3B054BC2E7B2CC825FF23E94108F97B8FC5B2EBC gpg --import libroot.asc gpg --verify SHA256SUMS.gpg SHA256SUMS sha256sum --check --ignore-missing SHA256SUMS # extract to ./snowchive-data/: mkdir -p snowchive-data && tar -xf "snowchive-$rel-full.tar" -C snowchive-data
Each release folder on
https://data.snowchive.org/files/
has a .torrent for both packages.
Repository for the extracted texts and metadata: https://git.disroot.org/libroot/snowchive-meta
git clone https://git.disroot.org/libroot/snowchive-meta cd snowchive-meta/data # grep only the extracted texts: grep "fiber optic" -ri --include=*.md # grep only the metadata: grep "fiber optic" -ri --include=*.json
Yes. The server software behind this archive is free software with no database and no external dependencies, designed to run offline or, for example, as a Tor hidden service.
First, get a release package and extract it: full for everything, or pdfs for a site that works the same but shows no page images.
Then get the server (Go required). Each release folder has the
source of the server version it was published with,
snowchive-server-<commit>.tar.gz, covered by the same
SHA256SUMS - the version known to work with that data:
tar -xzf snowchive-server-<commit>.tar.gz cd snowchive-server-<commit> go run . -root=/path/to/snowchive-data/ -listen=127.0.0.1:8081
Or clone the latest version from the snowchive-server repository:
git clone https://git.disroot.org/libroot/snowchive-server cd snowchive-server go run . -root=/path/to/snowchive-data/ -listen=127.0.0.1:8081
Recommended minimal setup for production instances:
go build -ldflags "-X main.buildCommit=$(git rev-parse --short HEAD)" -o snowchive . ./snowchive -root=/path/to/snowchive-data/ -listen=127.0.0.1:8081 -base=https://archive.example.com
System requirements: at least 300 MB of RAM (1 GB or more recommended for production).
The maintainers of this project do not endorse any public Snowchive instance other than the official snowchive.org instance (unless you are a reputable organization that wants to host a public instance).
See the snowchive-server README for the full documentation.
If the archive is missing a document, or some metadata is wrong, please submit an issue on the project repository: https://git.disroot.org/libroot/snowchive-meta/issues
You can also report issues and errors or give feedback directly to the project maintainers via email: https://libroot.org/about
Please submit server-related issues to the Snowchive server repository: https://git.disroot.org/libroot/snowchive-server/issues
We welcome anyone who wants to contribute to the project, for example by fixing errors in the metadata or in the extracted texts. Please see the Contact section above.
Running this project doesn't cost much, but if you wish, you can send a donation to:
85gTBgYuLB4Mvo1ZCGo62126hBQQ678UeESTuSi3L9SECHXr6S7mXE28yAKvURgdJneGKqbPDcBydFfTkK7H5bvh2WXE4oTbc1q5f5mc4pytajrn2xrp3lsftx64fw6hsdxxvs3haWe want to thank