first commit
This commit is contained in:
commit
0d90363116
6510 changed files with 679146 additions and 0 deletions
118
data/unofficial-xks-user-guide/metadata.json
Normal file
118
data/unofficial-xks-user-guide/metadata.json
Normal file
|
|
@ -0,0 +1,118 @@
|
|||
{
|
||||
"title": "The Unofficial XKEYSCORE User Guide",
|
||||
"type": "REPORTED",
|
||||
"description": "A TOP SECRET XKeyscore user guide written by a Booz Allen Hamilton consultant, providing step-by-step instructions for 32 classic search types — including email address queries, password extraction from mail server IPs, phone number extraction, HTTP activity analysis, document metadata fingerprinting and workflow automation — with worked examples targeting counterterrorism subjects in Afghanistan, Pakistan, Iran and Sweden, viewed through an Agility instance bearing the ECI RAGTIME classification banner confirming the tool was accredited for FISA-authorised collection.",
|
||||
"long_description": "This is a TOP SECRET//COMINT//REL TO USA, FVEY document authored by a consultant from Booz Allen Hamilton (unit E92-ADET), published by The Intercept on 1 July 2015 as part of the XKeyscore document release. The document is a practical analyst training guide covering XKeyscore's query interface, search types and workflow system, written in an accessible tutorial style with annotated screenshots.\n\nThe document's most operationally significant content is the Logins and Passwords section, which describes how analysts can extract username and password pairs from bulk internet collection by querying on a foreign mail server's IP address — with the explicit note that \"Logins and passwords are valuable tools to enable Tailored Access Operations (TAO).\" The guide describes using TUNINGFORK (including FoggyBottom) as a source of passwords, alongside content of emails and text messages and previous XKeyscore queries. The worked example queries on an Afghanistan network mail server IP to retrieve all login credentials passing through it. The query decision flowchart explicitly lists \"Logins and Passwords for the server for TAO\" as a primary use case alongside email addresses, phone numbers and file attachments.\n\nThe Email Addresses query section demonstrates searching on a target's email username and domain, but also shows how querying an IP address returns all email addresses seen on that IP — effectively mapping all users of a foreign mail server. A constraint noted is that the IP must be hosted outside Five Eyes countries and the domain must be foreign owned. The HTTP Activity section shows analysts querying for all visitors to a specific website — the worked example uses al-hisbah.com with Sweden as the country filter — or querying all web activity from a specific IP block to discover unknown sites visited by a target. The Document Metadata section demonstrates tracking document authors across intercepted file attachments using Author and LastAuthor fields from Microsoft Office and PDF metadata, enabling identification of target-associated documents even without knowing the sender's email address.\n\nThe Document Metadata section contains a significant classification disclosure. The Agility viewer interface displays the classification banner TOP SECRET//COMINT-ECI RAGTIME///NOFORN//20291123. RAGTIME is an Exceptionally Controlled Information compartment associated with FISA-authorised domestic collection programmes. The banner indicates the Agility tool itself is accredited to handle RAGTIME-compartmented data — meaning analysts using this tool were working within a FISA collection environment where all viewed content carried the RAGTIME classification regardless of individual document sensitivity. The two email subjects visible in the Agility viewer — \"raporti UN la kurdistan\" and \"Impotence meds at best prices\" — illustrate the nature of bulk collection, with a substantive human rights document sitting alongside pharmaceutical spam in the same intercept stream. The PDF properties identify the document as \"Microsoft Word - HR Report April or 2007 AR\" created with Acrobat Distiller 8.0.0 — this is the UNAMI (United Nations Assistance Mission for Iraq) Human Rights Report for 1 April to 30 June 2007, Arabic edition, whose contents include a section on freedom of expression in the Kurdistan Region. The title parses as Human Rights Report April-June 2007 Arabic, and the email subject \"raporti UN la kurdistan\" corresponds to the report's Kurdistan content.\n\nThe Workflow section describes standing periodic queries that run automatically on a schedule — bridging the gap between selector discovery and tasking delivery to UTT/Cadence dictionaries, and enabling sustained monitoring without manual query repetition. Workflows are submitted to the XKeyscore team for review but are not reviewed for USSID-18 compliance, which is left entirely to the analyst. The site selection step explicitly notes that NOFORN selectors require deselecting second and third party sites. The workflow list visible in the My Workflows screenshot shows active and inactive queries targeting French MFA documents, Iraqi telecommunications, Pakistani IP blocks, Swedish phone numbers and a phone-number query named HERNANDEZION.\n\nThe tips and tricks section reveals operational details about XKeyscore's query syntax — underscore wildcard behaviour, IP range regex construction, and Boolean operators — alongside a practical limitation: geolocation-based country filtering captures only 50-60% of actual traffic from a given country due to IP geolocation inaccuracy.\n\nThe document's existence as an unofficial guide written by a private contractor employee (Booz Allen Hamilton, the same firm that employed Edward Snowden) is itself significant — confirming that commercial contractors had top secret access to ECI RAGTIME-accredited analytical tools processing FISA-authorised collection.",
|
||||
"tags": [],
|
||||
"document": {
|
||||
"creator": "NSA",
|
||||
"date_created": "",
|
||||
"classification": [
|
||||
"TOP SECRET",
|
||||
"COMINT"
|
||||
],
|
||||
"dissemination": [
|
||||
"REL TO USA, AUS, CAN, GBR, NZL"
|
||||
],
|
||||
"banner": "TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL//20320108",
|
||||
"genre": "guide",
|
||||
"programs": [
|
||||
"XKEYSCORE",
|
||||
"PINWALE",
|
||||
"RAGTIME",
|
||||
"AGILITY",
|
||||
"METAFRAME",
|
||||
"TIMBERLINE",
|
||||
"YAKIMA",
|
||||
"REDMINE",
|
||||
"TUNINGFORK",
|
||||
"FOGGYBOTTOM",
|
||||
"FOXTRAIL",
|
||||
"CADENCE",
|
||||
"UTT",
|
||||
"NKB",
|
||||
"AIRGAP",
|
||||
"DOPHONE"
|
||||
],
|
||||
"countries": [
|
||||
"Afghanistan",
|
||||
"Iran",
|
||||
"Pakistan",
|
||||
"Sweden",
|
||||
"France",
|
||||
"Iraq",
|
||||
"Russia"
|
||||
],
|
||||
"people": [
|
||||
"bin Laden, Osama"
|
||||
]
|
||||
},
|
||||
"publication": [
|
||||
{
|
||||
"publisher": "The Intercept",
|
||||
"date_published": "2015-07-01",
|
||||
"title": "XKEYSCORE: NSA’s Google for the World’s Private Communications",
|
||||
"link": "https://theintercept.com/2015/07/01/nsas-google-worlds-private-communications/",
|
||||
"reporters": [
|
||||
"Marquis-Boire, Morgan",
|
||||
"Greenwald, Glenn",
|
||||
"Lee, Micah"
|
||||
],
|
||||
"contributors": [],
|
||||
"doc_source": "https://theintercept.com/document/2015/07/01/unofficial-xks-user-guide/",
|
||||
"doc_url": "https://s3.documentcloud.org/documents/2116191/unofficial-xks-user-guide.pdf"
|
||||
},
|
||||
{
|
||||
"publisher": "Metropolitan Books",
|
||||
"date_published": "2014-05-13",
|
||||
"title": "No Place to Hide",
|
||||
"link": "https://www.glenngreenwald.net/#BookDocuments",
|
||||
"reporters": [
|
||||
"Greenwald, Glenn"
|
||||
],
|
||||
"contributors": [],
|
||||
"version": "no-place-to-hide",
|
||||
"note": "The pages reprinted in the document appendix of No Place to Hide (appendix page 76)."
|
||||
},
|
||||
{
|
||||
"publisher": "SVT Nyheter",
|
||||
"date_published": "2013-12-11",
|
||||
"title": "FRA has access to controversial surveillance system",
|
||||
"link": "https://www.svt.se/nyheter/granskning/ug/fra-has-access-to-controversial-surveillance-system",
|
||||
"reporters": [
|
||||
"Bergman, Sven",
|
||||
"Dyfvermark, Joachim",
|
||||
"Gallagher, Ryan",
|
||||
"Greenwald, Glenn",
|
||||
"Laurin, Fredrik",
|
||||
"Struwe, Filip"
|
||||
],
|
||||
"contributors": [],
|
||||
"doc_source": "https://www.svt.se/nyheter/granskning/ug/read-the-snowden-documents-from-the-nsa",
|
||||
"doc_url": "https://s3.documentcloud.org/documents/894389/xkeyscore-slide-with-swedish-example.pdf",
|
||||
"version": "xkeyscore-slide-with-swedish-example",
|
||||
"note": "SVT published the \"HTTP_in_Sweden\" query screenshot from the page 13 (with host redacted)."
|
||||
}
|
||||
],
|
||||
"aka_id": [
|
||||
{
|
||||
"id": "EFF_-_20131211-svt-xkeyscore_slide_with_swedish_example"
|
||||
},
|
||||
{
|
||||
"id": "nsa-se-fra-xkeyscore-slide"
|
||||
}
|
||||
],
|
||||
"pdf_sha256": "a415f2f8e87868a167c19e9295483b05f1e153af906a6a491f151395d2d12aa4",
|
||||
"pages": 27,
|
||||
"versions": {
|
||||
"xkeyscore-slide-with-swedish-example": {
|
||||
"pdf_sha256": "87dfa6bf88fb7ec4129fce7b88cb17eb0f47a5d2499cecefa5b9306e59986a25",
|
||||
"changed_pages": [
|
||||
1
|
||||
]
|
||||
},
|
||||
"no-place-to-hide": {
|
||||
"pdf_sha256": "d2019ca1d818deeec78d33539f3807f80774aa4d4813e7a8fa5801de967231c1"
|
||||
}
|
||||
}
|
||||
}
|
||||
Loading…
Add table
Add a link
Reference in a new issue