snowchive-meta/data/xks-tech-extractor-2010/metadata.json
2026-11-19 04:39:11 +02:00

64 lines
6.1 KiB
JSON
Raw Blame History

This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

{
"title": "XKEYSCORE Tech Strings in Documents (aka Tech Extractor) 2010",
"type": "REPORTED",
"description": "An April 2010 TOP SECRET XKeyscore training presentation on the Tech Extractor — an expanded version of the December 2009 briefing — teaching analysts to find intelligence by tagging keywords inside email, chat and document bodies, with new detail on tasking rules (including that terms hit as substrings so \"ricin\" matches \"pricing\"), its non-retrospective operation, its limitations versus the 65+ context fingerprint language, and confirmation that the capability was operated jointly with GCHQ, whose Cheltenham analysts submitted tasking via an Excel template and wiki.",
"long_description": "This is a TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL XKeyscore training presentation dated April 2010 on the Tech Extractor (Tech Strings in Documents). It is an expanded revision of the December 2009 Tech Extractor briefing, retaining the core explanation while adding tasking rules, operational limitations, a comparison with the fingerprint language, and significant GCHQ-specific content.\n\nThe core material is unchanged from the earlier version: the Tech Extractor is XKeyscore's first context-aware scanning capability, addressing the weakness of traditional soft selection (content-based keyword selection) which returned excessive junk because legacy engines designed for formatted TELEX traffic could not distinguish communication content from incidental web content. It tags keywords appearing in three contexts — email bodies, chat bodies, and document bodies (Microsoft Office, PDF, PostScript, RTF) — with metadata, which analysts then use in USSID-18 compliant queries (with the caveat that tags alone are not compliant and may need a foreign IP, MAC address or country code).\n\nThis version adds several operational details. Tasking rules are now specified: all terms must be classified REL FVEY, terms are case-insensitive by default (but can be forced case-sensitive), and terms hit as substrings by default — the example warning that \"ricin\" will hit inside \"pricing\" unless forced to match as a whole word via surrounding spaces or a regular expression. The capability is explicitly non-retrospective: after a list is tasked, XKeyscore scans only data collected from that point forward, not previously stored data. The term lists (Tech Dictionaries with category Tech Names) are still supplied by analysts to the XKeyscore team, but a GUI for analyst self-entry is noted as almost complete.\n\nThe limitations section is new and candid: even restricted to document, email and chat bodies, that is still a large volume of traffic; generic or short terms still produce many false hits; and the term scans all data even when results are later narrowed by country or IP. The comparison with fingerprints is the key analytical addition — the Tech Extractor treats email, chat and document bodies as a single combined context, whereas the XKeyscore fingerprint language offers over 65 contexts that can be combined into powerful, specific signatures. Analysts are advised to switch to the full fingerprint language when generic terms return too many poor results. The Tech Extractor's distinct advantage is preserved, however: it shows exactly which term hit in the metadata results, whereas a fingerprint only indicates that its full (potentially complex) equation was satisfied without revealing which specific terms matched.\n\nThe worked example results table shows GSM-related tech dictionary hits — HLR, ICCID, IMEI, IMSI, MSISDN — extracted from document and email bodies including files with revealing paths (\"25 APR 10 Daily Break Down.xls,\" \"Evo_Complaints_sheet_25-Apr-2010.xls,\" a file referencing \"SE EWSD\" and \"New Franchisees Status\"), suggesting collection against telecommunications operator business documents.\n\nThe most significant new content is the confirmation of GCHQ's operational role. The presentation directs analysts to a GCHQ wiki page (wiki.gchq/index.php/Tech_dictionaries) for more information, instructs them to submit tasking using an Excel spreadsheet template \"developed by GCHQ CP\" (Cheltenham/a GCHQ unit), and to email the term lists to a gchq.gov.uk address. This confirms that the Tech Extractor tasking process was administered by GCHQ, not NSA — demonstrating that this XKeyscore capability was operated as a genuinely joint NSA-GCHQ function with GCHQ handling the dictionary submission workflow. The document is significant both as documentation of the evolution of context-aware scanning toward the mature fingerprint language and as concrete evidence of the depth of NSA-GCHQ integration in XKeyscore's day-to-day operation.",
"tags": [
"context-aware scanning",
"soft selection",
"content-based selection",
"fingerprints",
"email body",
"chat body",
"document body",
"Microsoft Office",
"TELEX",
"GSM",
"MSISDN",
"MSISDN",
"EWSD",
"telecommunications",
"Nokia",
"metadata tagging"
],
"document": {
"creator": "NSA",
"date_created": "2010-04",
"classification": [
"TOP SECRET",
"COMINT"
],
"dissemination": [
"REL TO USA, AUS, CAN, GBR, NZL"
],
"banner": "TOP SECRET//COMINT//REL TO USA, AUS, CAN, GBR, NZL",
"genre": "slides",
"programs": [
"XKEYSCORE",
"UIS"
],
"countries": [
"Pakistan"
],
"people": []
},
"publication": [
{
"publisher": "The Intercept",
"date_published": "2015-07-01",
"title": "XKEYSCORE: NSA’s Google for the World’s Private Communications",
"link": "https://theintercept.com/2015/07/01/nsas-google-worlds-private-communications/",
"reporters": [
"Marquis-Boire, Morgan",
"Greenwald, Glenn",
"Lee, Micah"
],
"contributors": [],
"doc_source": "https://theintercept.com/document/2015/07/01/xks-tech-extractor-2010/",
"doc_url": "https://s3.documentcloud.org/documents/2116499/xks-tech-extractor-2010.pdf"
}
],
"aka_id": [],
"pdf_sha256": "3b1a6fd56a0b7e796f4aa6df7695b2ef1a63e7d2bf7755727c54b3f635b8b691",
"pages": 23
}