{ "title": "Introduction to Context Sensitive Scanning with X-KEYSCORE Fingerprints", "type": "REPORTED", "description": "This May 2010 NSA training document introduces XKEYSCORE's context-sensitive scanning capabilities using \"fingerprints\"—customizable detection rules that allow analysts to surgically target specific types of Internet traffic without needing prior knowledge of email addresses or other strong selectors, using over 70 unique contexts to identify patterns in communications and behaviors.", "long_description": "The document serves as a comprehensive introduction to XKEYSCORE fingerprints, addressing the fundamental challenge analysts face when they don't have identifying information like email addresses to track targets. Context-sensitive scanning allows analysts to create sophisticated queries that precisely target specific traffic types by combining contextual filters—such as geographic location, communication content type, encryption methods, device types, and behavioral patterns—rather than relying on broad keyword searches that generate excessive false positives. The presentation demonstrates that XKEYSCORE already contains nearly 10,000 AppIDs and fingerprints covering various intelligence needs, from detecting Mujahideen Secrets encryption to identifying specific botnet traffic or Iranian nuclear procurement network communications.\n\nThe training walks through multiple example scenarios with increasing complexity: finding web searches for \"Jihad\" originating from Kabul, detecting Mujahideen Secrets encryption used from iPhones, identifying documents mentioning Iranian shipping companies, and monitoring emails discussing weapons procurement involving specific government positions and countries. The document explains that fingerprints can utilize simple keyword matching, Boolean logic, regular expressions, binary patterns, positional logic, and even custom C++ code for sophisticated pattern recognition when keywords alone are insufficient. Particularly notable is the discussion of \"hidden\" Mujahideen Secrets messages where adversaries removed identifying headers, requiring algorithmic detection rather than keyword matching.\n\nThe presentation details over 70 available contexts including HTTP-specific contexts (URL paths, cookies, user agents, referrer fields, web search terms), protocol contexts (IP addresses, ports, country codes, MAC addresses), and communication contexts (email bodies, chat bodies, document bodies, calendar entries, HTTP POST data). Advanced capabilities include metadata-extracting fingerprints that automatically parse and store structured information from file upload sites, capturing filenames, URLs, deletion links, and uploader usernames. The document introduces a new GUI allowing analysts to directly write, test, and submit fingerprints through a web interface with compilation verification. Throughout, the training emphasizes USSID 18 compliance concerns, noting that some fingerprints (like detecting iPhone usage alone) would violate regulations but become permissible when combined with foreign targeting criteria like Iranian proxy servers.", "tags": [ "training", "pattern matching", "keyword search", "regular expression", "metadata extraction", "Mujahideen Secrets", "counterterrorism", "botnets", "file upload", "HTTP traffic", "encryption detection", "cryptanalysis", "mobile tracking", "behavioral analysis", "signature development", "analyst workflow", "automatic target recognition", "automation", "web search", "VoIP interception", "chat monitoring", "document scanning", "malware tracking", "proxy detection", "geolocation", "port scanning", "protocol analysis", "surveillance tradecraft" ], "document": { "creator": "NSA", "date_created": "2010-05", "classification": [ "TOP SECRET", "COMINT" ], "dissemination": [ "REL TO USA, AUS, CAN, GBR, NZL" ], "banner": "TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL", "genre": "slides", "programs": [ "XKEYSCORE", "PILBEAM", "QUANTUMBOT", "NKB", "BYZANTINERAPTOR" ], "countries": [ "Iran", "Pakistan", "Somalia", "Liberia", "Sudan", "South Africa", "Serbia", "Bulgaria", "Myanmar", "Iraq", "Tanzania", "Germany", "Russia" ], "people": [] }, "publication": [ { "publisher": "The Intercept", "date_published": "2015-07-01", "title": "XKEYSCORE: NSA’s Google for the World’s Private Communications", "link": "https://theintercept.com/2015/07/01/nsas-google-worlds-private-communications/", "reporters": [ "Marquis-Boire, Morgan", "Greenwald, Glenn", "Lee, Micah" ], "contributors": [], "doc_source": "https://theintercept.com/document/2015/07/01/intro-context-sensitive-scanning-xks-fingerprints/", "doc_url": "https://s3.documentcloud.org/documents/2116103/intro-to-context-sensitive-scanning-with-xks.pdf" } ], "pdf_sha256": "9341b99f65b95c70af27c08d2294e489bb6e7fb3f67737e959ff0cf5ceda915c", "pages": 61 }