first commit
This commit is contained in:
commit
0d90363116
6510 changed files with 679146 additions and 0 deletions
79
data/writing-xks-fingerprints/metadata.json
Normal file
79
data/writing-xks-fingerprints/metadata.json
Normal file
|
|
@ -0,0 +1,79 @@
|
|||
{
|
||||
"title": "Writing XKS Fingerprints",
|
||||
"type": "REPORTED",
|
||||
"description": "This internal NSA training manual serves as a technical guide for analysts using the X-KEYSCORE (XKS) system. It provides a comprehensive breakdown of how to write, name, and test \"fingerprints\"—the specific code signatures used to identify, tag, and categorize various types of internet traffic across the agency's global collection points. The manual transitions from basic naming conventions to advanced Boolean logic and context-sensitive scanning, illustrating how the NSA automates the discovery of high-value targets, such as those using specific encryption tools or those involved in weapons procurement.",
|
||||
"long_description": "This document defines a fingerprint as a unique search signature that allows the X-KEYSCORE system to recognize specific patterns in data packets as they are intercepted. The manual emphasizes a strict, directory-like naming convention, such as encryption/archive/rar, which enables analysts to organize their signatures and perform broad searches using wildcards. By categorizing fingerprints into folders like botnet, vulnerability, or topic/wmd, the agency ensures that its vast library of search terms remains manageable and searchable for the entire analyst community.\n\nThis document describes the use of Boolean logic to refine these searches, moving beyond simple keyword matching. Analysts are taught to use operators like AND, OR, and AND NOT to create precise filters. For instance, a fingerprint for a Pakistani agency like the National Development Complex (NDC) might require the presence of the term \"National Development Complex\" AND the word \"quotation,\" while explicitly excluding \"spam\" terms like \"viagra\" to reduce false positives. The guide also introduces Variables, which act as master lists of terms (e.g., a list of all known WMD-related chemicals) that can be updated once and automatically applied to every fingerprint that references that variable.\n\nThis document highlights the power of Context-Sensitive Scanning, which allows the NSA to specify exactly where within a data packet a term should be found. Rather than a \"blind\" search of the entire internet, an analyst can use over 70 unique contexts to target specific behaviors. Examples include web_search() to capture what a target types into a search engine, user_agent() to identify specific hardware like an iPhone, or email_body() to ignore metadata and focus only on the text of a message. This precision is intended to improve system performance and accuracy, ensuring that a mention of a \"banned item\" in a blog post is not confused with the same term appearing in a formal document or an encrypted header.\n\nThis document concludes with significant legal and compliance considerations, specifically regarding USSID 18 and HRA (Human Rights Act) protocols. It warns analysts that certain fingerprints—such as one that simply identifies all iPhone users—are not compliant if queried on their own, as they may capture data on \"U.S. persons\" or non-targets. Instead, these fingerprints must be used as one component of a \"more complex query\" that includes a validated foreign intelligence target. The manual also showcases the XKS Central GUI, a web-based interface that allows analysts to compile, test against live session data, and save their signatures for enterprise-wide deployment.",
|
||||
"tags": [
|
||||
"backdoors",
|
||||
"bots",
|
||||
"botnets",
|
||||
"antivirus",
|
||||
"anonymizers",
|
||||
"Tailored Access Operations",
|
||||
"deep packet inspection",
|
||||
"data tagging",
|
||||
"iPhone",
|
||||
"User-Agent",
|
||||
"web search",
|
||||
"email body",
|
||||
"chat body",
|
||||
"document body",
|
||||
"HTTP",
|
||||
"cookie tracking",
|
||||
"IP addresses",
|
||||
"MAC address",
|
||||
"geolocation",
|
||||
"pattern matching"
|
||||
],
|
||||
"document": {
|
||||
"creator": "NSA",
|
||||
"date_created": "2010-11-01",
|
||||
"classification": [
|
||||
"TOP SECRET",
|
||||
"COMINT"
|
||||
],
|
||||
"dissemination": [
|
||||
"REL TO USA, AUS, CAN, GBR, NZL"
|
||||
],
|
||||
"banner": "TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL",
|
||||
"genre": "slides",
|
||||
"programs": [
|
||||
"XKEYSCORE",
|
||||
"QUANTUMBOT",
|
||||
"NKB"
|
||||
],
|
||||
"countries": [
|
||||
"Afghanistan",
|
||||
"Iran",
|
||||
"Pakistan",
|
||||
"Germany",
|
||||
"Liberia",
|
||||
"Sudan",
|
||||
"Bulgaria",
|
||||
"Serbia",
|
||||
"Tanzania",
|
||||
"Somalia",
|
||||
"Russia"
|
||||
],
|
||||
"people": []
|
||||
},
|
||||
"publication": [
|
||||
{
|
||||
"publisher": "The Intercept",
|
||||
"date_published": "2015-07-01",
|
||||
"title": "XKEYSCORE: NSA’s Google for the World’s Private Communications",
|
||||
"link": "https://theintercept.com/2015/07/01/nsas-google-worlds-private-communications/",
|
||||
"reporters": [
|
||||
"Marquis-Boire, Morgan",
|
||||
"Greenwald, Glenn",
|
||||
"Lee, Micah"
|
||||
],
|
||||
"contributors": [],
|
||||
"doc_source": "https://theintercept.com/document/2015/07/01/writing-xks-fingerprints/",
|
||||
"doc_url": "https://s3.documentcloud.org/documents/2116277/writing-xks-fingerprints.pdf"
|
||||
}
|
||||
],
|
||||
"aka_id": [],
|
||||
"pdf_sha256": "94897dd17554c012474f91f6fe82c7aad94a0466d61c72ad8075694e79dc544b",
|
||||
"pages": 67
|
||||
}
|
||||
1342
data/writing-xks-fingerprints/writing-xks-fingerprints.md
Normal file
1342
data/writing-xks-fingerprints/writing-xks-fingerprints.md
Normal file
File diff suppressed because it is too large
Load diff
Loading…
Add table
Add a link
Reference in a new issue