first commit

This commit is contained in:
libroot 2026-11-19 04:39:11 +02:00
commit 0d90363116
Signed by: libroot
GPG key ID: 2AC68C029EBE9ACE
6510 changed files with 679146 additions and 0 deletions

View file

@ -0,0 +1,76 @@
{
"title": "How Is Human Language Technology (HLT) Progressing?",
"type": "SIDTODAY",
"description": "This document provides an operational update on the deployment of \"HLT Labs\"—a specialized suite of voice analytics designed to automate the discovery of intelligence \"nuggets\" within massive volumes of audio collection. The document highlights successful field applications in Afghanistan, Georgia, Texas, and Latin America, noting that these tools are becoming \"force multipliers\" that allow linguists to bypass manual listening in favor of automated keyword searching and speaker recognition.",
"long_description": "By late 2011, the NSA’s HLT capabilities had moved from research prototypes into critical operational environments. In Afghanistan, the deployment marked a significant technical milestone by utilizing cloud computing architectures for the first time, specifically integrating the DISTILLERY and CLOUDBASE systems on the GHOSTMACHINE platform. These tools supported the full suite of Pashto speech-to-text (STT) analytics, enabling international teams and special operations task forces (such as CJSOTF) to identify target speakers on previously unknown selectors.\n\nThe document candidly addresses the technical hurdles of \"low-resource\" languages like Pashto and Farsi. Because these languages have limited training data, the STT models are less accurate than more \"mature\" languages like Spanish. This is further complicated at sites like NSA Georgia, where noisy VHF radio collections degrade the performance of the algorithms. Despite these limitations, the memorandum notes that speaker recognition remains effective, allowing analysts to geolocate target voices even on unknown frequencies.\n\nIn contrast, the Spanish STT models are described as the most advanced in the NSA arsenal. The memorandum cites high keyword-search accuracy and immediate results in the field: an analyst in Texas identified a drug trafficking lead the morning after training, while a Special Collection Service (SCS) site in Latin America used the tool to rapidly extract intelligence on a Cuban official. The update concludes by noting that these \"HLT Labs\" successes are being integrated into the broader SPIRITFIRE (voice analytics) and TransX (translation/transcription) programs for a full enterprise-wide rollout in 2012.",
"tags": [
"media mining",
"speech-to-text",
"keyword search",
"speaker identification",
"Special Collection Service",
"VHF",
"cloud computing",
"drug trafficking",
"Cuban official",
"voice analytics",
"linguists"
],
"document": {
"creator": "NSA",
"date_created": "2011-09-06",
"classification": [
"SECRET",
"SI"
],
"dissemination": [
"REL"
],
"genre": "newsletter",
"programs": [
"HLT",
"SPIRITFIRE",
"TRANSX",
"DISTILLERY",
"CLOUDBASE",
"GHOSTMACHINE"
],
"countries": [
"Afghanistan",
"France",
"Spain",
"Cuba"
],
"people": []
},
"publication": [
{
"publisher": "The Intercept",
"date_published": "2015-05-05",
"title": "How the NSA Converts Spoken Words Into Searchable Text",
"link": "https://theintercept.com/2015/05/05/nsa-speech-recognition-snowden-searchable-text/",
"reporters": [
"Froomkin, Dan"
],
"contributors": [
"Fishman, Andrew"
],
"doc_source": "https://theintercept.com/document/2015/05/05/human-language-hlt-progressing/",
"doc_url": "https://s3.documentcloud.org/documents/2072020/sidtoday-is-hlt-progressing-final.pdf"
},
{
"publisher": "The Intercept",
"date_published": "2018-01-19",
"title": "Finding Your Voice: Forget About Siri and Alexa — When It Comes to Voice Identification, the “NSA Reigns Supreme”",
"link": "https://theintercept.com/2018/01/19/voice-recognition-technology-nsa/",
"reporters": [
"Kofman, Ava"
],
"contributors": [],
"doc_source": "https://theintercept.com/document/2018/01/19/how-is-human-language-technology-progressing/",
"doc_url": "https://s3.documentcloud.org/documents/4352001/2011-09-06-How-Is-Human-Language-Technology-HLT.pdf"
}
],
"pdf_sha256": "00e9b1a43f1ef11dca4a6eea47f4453898a37f03e05f3695bce7b14a7c969bbd",
"pages": 2
}

View file

@ -0,0 +1,35 @@
(S//SI//REL) How Is Human Language Technology (HLT) Progressing?
FROM: (U//FOUO) [REDACTED]
Language Analysis Modernization Lead (S2)
Run Date: 09/06/2011
(S//SI//REL) Editor's intro: At the SID town hall meeting of February 2011, [REDACTED] (pictured) briefed on Human Language Technology, i.e., tools that sort through SIGINT voice collection and automatically find the most promising nuggets, thereby saving linguists countless hours. What's happened with HLT since that time?
(S//SI//REL) In 2011 we deployed HLT Labs to Afghanistan, NSA Georgia, Latin American SCS sites, and NSA Texas.
(U) Afghanistan-area targets
(S//SI//REL) Afghan Regional Operating Cryptologic Center (AROCC) analysts started using HLT Labs to track their targets in April, and when the analytics were successfully used to find new information, the mission was expanded to include international teams.\* The Afghanistan deployment boasts some technological firsts associated with cloud computing\*\* and includes the full suite of analytics with Pashto speech-to-text (STT). Recently French analysts in the ARC were able to find target speakers on new selectors using speaker recognition.
(S//SI//REL) Our deployment to NSA Georgia enables us to partner with analysts to assess the performance of our newest STT models: Pashto and Farsi. These languages have limited training data which creates challenges for STT, and we have been focused on finding applications that are beneficial even for these low-resource languages. NSA-Georgia traffic includes noisy VHF collections which seriously degrade analytic performance; however, analysts can still find target speaker cuts on unknown frequencies.
(U) Spanish-speaking targets
(S//SI//REL) Spanish is the most mature of our speech-to-text analytics, and has higher keyword-search accuracy than other deployed STT models. We've had great success searching for Spanish keywords at NSA Texas and Latin America SCS sites.
(S//SI//REL) For example, in early August a new NSA Texas user applied keyword search the morning after his training to find a previously unreported cut from a drug trafficking target. Likewise, the OIC of one of the Latin American SCS sites recently reported he was able to find foreign intelligence regarding a Cuban official in a fraction of the usual time. His comment: This same example could be used over and over by many that have to go over countless voice cuts to finally dig that gold nugget that will turn into a report.
(U) Development work continues
(U//FOUO) The R6 research team is working to add new applications, improve keyword search capability, enhance analytics, add new languages, and refine the user interface. Recently the Summer Camp for Applied Language Exploration (SCALE) -- a joint NSA Johns Hopkins University exercise -- investigated new ways to use the results of HLT analytics from existing targets to find new targets. Research is also working closely with the SPIRITFIRE (voice analytics)
and TransX (translation, transcription and transliteration) efforts to ensure HLT Labs capabilities are included in the corporate solution for enterprise deployment in 2012.
(U//FOUO) More information about HLT Labs is available here.
(U//FOUO) See a related SIDtoday article about HLT here.
(U) Notes:
\* (S//REL) The international teams were from the Analysis and Research Cell (ARC), Task Force 310, and Combined Joint Special Operations Task Force (CJSOTF).
\*\* (S//SI//REL) Specifically, the Afghan deployment is the first use of DISTILLERY and CLOUDBASE on a GHOSTMACHINE platform.