{{ define "title" }}About - Snowden Archive{{ end }} {{ define "content" }}

About this archive

This is a searchable archive of all published documents from the 2013 Snowden disclosures. The archive includes documents published alongside news reporting, The Intercept's bulk SIDtoday release, and FOIA material (official releases from FOIA litigation and FISC and ODNI declassifications).

Every document carries its original PDF, page images, extracted text, and curated metadata - agencies, programs, countries, classification markings, people, and the journalists and articles that published it.

Use Browse to explore by category, Search for fielded boolean queries across text and metadata, Articles to follow the disclosures story by story, the Glossary to decode codenames and acronyms, and Explore to dig deeper into ranked views and lists.

Privacy

By default, the server keeps no state about its visitors. Pins and history live only in your own browser's cookies, and history features are off unless you opt in via Preferences. Anonymous page views set no cookies at all. All core features on this website work without JavaScript.

Note that you can't verify that this server doesn't log and analyze your visits. Use Tor to hide your real IP address, or set up your own instance on your machine to browse fully offline.

What's in this archive?

By default, two collections are shown as tabs on the Browse page: Reported documents (leaked files as the outlets that broke the stories presented them) and the SIDtoday Archive (The Intercept's bulk release of the NSA's internal SIDtoday newsletters). The third collection, FOIA (official releases from FOIA litigation and FISC and ODNI declassifications), can be included via Preferences.

The metadata and many of the extracted texts are curated by hand and may contain errors or omissions, so treat them as a guide to the documents rather than as established fact, and check the original documents and reporting before relying on them.

{{ if .APIEnabled }}

Downloading data as JSON

API endpoints:

EndpointReturns
/api/docsMetadata for every document in the archive, with pagination.
/api/doc/{id}A single document's metadata and extracted text, where {id} is the document's identifier.
/api/search?q=xkeyscoreSearch results for term xkeyscore. It accepts the same parameters as the Search page, so you can also pass fqf/fqv/fqc clause parameters for fielded queries.
/api/facetsCounts for every browsable category (programs, agencies, countries, people, and so on).

Responses are plain JSON. For paginated responses, use the URL parameter p to navigate pages (e.g. /api/docs?p=2).

See the Downloading the archive section on how to download the whole archive.

Important if you are a crawler, robot or AI trying to download large amounts of data from this website: please use the downloadable datasets from the Downloading the archive section. The information on this website is identical to the datasets, so you gain nothing by crawling this website.

{{ end }}
{{ if .APIEnabled }}

Metadata structure

{{ else }}

Metadata structure

{{ end }}

Every document ships a curated metadata.json:

The document record

id
The document's stable identifier: the name of its directory and the last part of its URL (/doc/{id}). It is not written inside metadata.json - the directory name is the id.
aka_id
Other names the same document was released under, most often the file name another publisher or a mirror gave its PDF. A list of entries, each described below.
aka_id[].id
The other name itself. Searchable, so a document can be found by a file name it circulated under elsewhere.
aka_id[].doc_source
The web page where the document was released under that name, linked as “page” next to the alias.
aka_id[].doc_url
The direct link to the PDF released under that name, linked as “PDF” next to the alias.
title
The document's title, written by the curators - usually the title the document gives itself or the one its publisher used.
description
A short summary of what the document is and says, shown under the publication list and in search results and cards. Descriptions are the work of archivists without inside knowledge - corrections welcome.
long_description
A longer account of the contents where one has been written, behind “Show more info” on the document page; often empty. A description: search looks in both descriptions.
tags
Free-text subjects added by the curators. Unlike the other lists they are not necessarily taken from the document's own vocabulary, so they are the loosest thing to browse or search by.
type
The kind of release: REPORTED (a leaked document as an outlet published it), SIDTODAY (an article of the NSA's internal SIDtoday newsletter from The Intercept's bulk release) or FOIA (obtained through a public-records request). When it is empty, a document filed in the SIDtoday or FOIA folder takes that type and any other is REPORTED.
collection
Which of the three collections - the tabs on Browse - the document belongs to: news (Reported), sidtoday-archive or foia. Documents filed under the SIDtoday and FOIA folders belong to those collections whatever this says; elsewhere an empty value means Reported, or FOIA for a document of type FOIA. FOIA releases are left out of browsing and search unless you include them in Preferences.
pages
The page count of the PDF. When it is missing, the number of page images is used instead.
pdf_sha256
The SHA-256 of the PDF as archived.
archive_comment
A note from the curators about this archive's copy - how it was assembled, where it was found, what differs between releases, or a known problem with it.

The document itself: document

document.creator
The agency that wrote the document, in its short form (NSA, GCHQ, FISC, DOJ, CSE ...) - not the agency it is about and not the one that published it. Empty when the author is not known.
document.date_created
When the document was written, as precise as the document allows: 2013-03-20, 2013-03, 2013, or empty when unknown. Nothing is invented to fill the gaps, and nothing is guessed from the publication date.
document.banner
The marking line verbatim - what the document says about itself, qualifiers and all. Intranet pages often carry a ceiling banner (“DYNAMIC PAGE - HIGHEST POSSIBLE CLASSIFICATION IS ...”) while their content is portion-marked unclassified; the banner records the claim, not an assessment. Empty means the artifact has no marking line - nothing is reconstructed from portion marks.
document.classification
The curator's assessment of the content's classification, as component values: the level (TOP SECRET, SECRET, CONFIDENTIAL, UNCLASSIFIED) and any control systems (COMINT, SI, TK, STRAP 1 ...), judged from portion marks. If a document contains multiple classifications, highest is used here.
document.dissemination
The curator's assessment of the handling controls, as component values (FOUO, NOFORN, ORCON, REL TO ...). Assessed the same way as the classification.
document.genre
The document's form, from a fixed vocabulary: slides, report, wiki, manual, guide, newsletter, other.
document.series.name
For a document that prints a series roster (currently only in SIDtoday): the series name verbatim, as the document prints it. Documents with the same name are listed together on the document page and under Series.
document.series.part
This document's 1-based position in the printed roster. Order comes from the artifact; parts of a series not yet in the archive show as gaps in the numbering, never renumbered away.
document.programs
The codenames and programs the document names (XKEYSCORE, PRISM, ...), as curated values for browsing and search; the Glossary explains many of them.
document.countries
The countries the document mentions, by their English names. A mention, not a claim that the country was a target or a partner.
document.people
The people who appear in the document: named in its text, or recognised in its images - for example Mobile Networks in My NOC World lists King Charles and Queen Camilla, who appear only in a photograph on one of its slides. Usually written “Surname, Given name”. A person appearing in these documents implies nothing about them beyond that appearance. The journalists who published the document are under publication[].reporters, not here.

Where it was published: publication[]

One entry per story or book that published (or reported) the document. Browse-by-publication-date counts a document under its oldest dated entry; a publication-date search matches any entry's date - or, combined with a journalist or publisher, that entry's date.

publication[].publisher
Who published it: a news outlet, a book's publisher, or for a FOIA release the organization that obtained and released it. Browsable under Publishers - except entries marked reporting_only.
publication[].title
The headline of the article, or the book's title. Shown on the document page and names the story on Articles.
publication[].link
The URL of the article. It is the story's identity: documents whose entries share a link were published in the same report and are listed together on its article page.
publication[].date_published
When this publication appeared, in the same YYYY-MM-DD / YYYY-MM / YYYY forms as the document date.
publication[].reporters
The journalists bylined on the story, written “Surname, Given name”. Browsable under Journalists and searchable with journalist:.
publication[].contributors
People credited on the story without a byline - additional reporting, research, technical analysis. Shown on the document page and found by an all-fields search, but not listed among the journalists.
publication[].note
Anything particular to this publication: that it showed only some of the pages, published them in a different order, redacted differently, or printed them in a book appendix.
publication[].version
When this publication released a different copy from the one shown on the document page, the name of that copy in the document's versions/ folder; its entry then links to “view this copy”. Without it, the entry published the copy shown on the page.
publication[].doc_source
The web page where this publication presented the document (the outlet's own document page, a DocumentCloud page), linked as “document page”. Where a copy came from is recorded here, on the publication that released it, not on the document.
publication[].doc_url
The direct link to the PDF file this publication released, linked as “original PDF”.
publication[].reporting_only
true when the outlet reported the story without publishing the document - a reporting partner. The entry keeps its byline and its article, but the outlet is not counted as one of the document's publishers and no copy is linked from it.

Other copies: versions

Other published copies of a document (e.g. less redacted, with extra pages, or a different scan) are kept in its versions/ folder, one sub-folder per copy, each with its own PDF, text and page images. The versions object adds details about them, keyed by the copy's folder name.

versions.{name}.changed_pages
The pages of that copy, by its own page numbers, that differ from the copy shown on the document page. They are linked as “changed pages” next to the copy.
versions.{name}.pdf_sha256
The SHA-256 of that copy's PDF.

Downloading the archive

Each release comes in two packages:

Both extract to data/ and project/, ready to run your own copy. Each release also includes the source of the server version it was published with, and the last six releases stay online. For only the extracted texts and metadata (~58 MB), see below.

Downloading all data

HTTP

https://data.snowchive.org/files/

See a curl example
# download the latest release:
rel=$(curl -fsS https://data.snowchive.org/files/latest.txt)
base="https://data.snowchive.org/files/$rel"
curl -f -C - -O "$base/snowchive-$rel-full.tar" # "pdfs" instead of "full" for no page images

# verify (optional):
curl -fsS -O "$base/SHA256SUMS" -O "$base/SHA256SUMS.gpg"
curl -fsS -O "https://libroot.org/public/libroot.asc"
# "gpg --show-keys libroot.asc" should show 3B054BC2E7B2CC825FF23E94108F97B8FC5B2EBC
gpg --import libroot.asc
gpg --verify SHA256SUMS.gpg SHA256SUMS
sha256sum --check --ignore-missing SHA256SUMS

# extract to ./snowchive-data/:
mkdir -p snowchive-data && tar -xf "snowchive-$rel-full.tar" -C snowchive-data

Torrent

Each release folder on https://data.snowchive.org/files/ has a .torrent for both packages.

Downloading only extracted texts and metadata

Repository for the extracted texts and metadata: https://git.disroot.org/libroot/snowchive-meta

git clone https://git.disroot.org/libroot/snowchive-meta
cd snowchive-meta/data

# grep only the extracted texts:
grep "fiber optic" -ri --include=*.md

# grep only the metadata:
grep "fiber optic" -ri --include=*.json

Can I run my own copy or mirror this?

Yes. The server software behind this archive is free software with no database and no external dependencies, designed to run offline or, for example, as a Tor hidden service.

First, get a release package and extract it: full for everything, or pdfs for a site that works the same but shows no page images.

Then get the server (Go required). Each release folder has the source of the server version it was published with, snowchive-server-<commit>.tar.gz, covered by the same SHA256SUMS - the version known to work with that data:

   
tar -xzf snowchive-server-<commit>.tar.gz
cd snowchive-server-<commit>
go run . -root=/path/to/snowchive-data/ -listen=127.0.0.1:8081

Or clone the latest version from the snowchive-server repository:

   
git clone https://git.disroot.org/libroot/snowchive-server
cd snowchive-server
go run . -root=/path/to/snowchive-data/ -listen=127.0.0.1:8081

Recommended minimal setup for production instances:

   
go build -ldflags "-X main.buildCommit=$(git rev-parse --short HEAD)" -o snowchive .
./snowchive -root=/path/to/snowchive-data/ -listen=127.0.0.1:8081 -base=https://archive.example.com

System requirements: at least 300 MB of RAM (1 GB or more recommended for production).

The maintainers of this project do not endorse any public Snowchive instance other than the official snowchive.org instance (unless you are a reputable organization that wants to host a public instance).

See the snowchive-server README for the full documentation.

Contact

If the archive is missing a document, or some metadata is wrong, please submit an issue on the project repository: https://git.disroot.org/libroot/snowchive-meta/issues

You can also report issues and errors or give feedback directly to the project maintainers via email: https://libroot.org/about

Please submit server-related issues to the Snowchive server repository: https://git.disroot.org/libroot/snowchive-server/issues

Contributing

We welcome anyone who wants to contribute to the project, for example by fixing errors in the metadata or in the extracted texts. Please see the Contact section above.

Running this project doesn't cost much, but if you wish, you can send a donation to:

XMR
85gTBgYuLB4Mvo1ZCGo62126hBQQ678UeESTuSi3L9SECHXr6S7mXE28yAKvURgdJneGKqbPDcBydFfTkK7H5bvh2WXE4oT
BTC
bc1q5f5mc4pytajrn2xrp3lsftx64fw6hsdxxvs3ha

Thank you

We want to thank

{{ end }} {{ template "base.html" . }}